ChatGPT Health Is a Strategic Liability

TL;DR

ChatGPT Health links to medical records for more personal answers, but it brings two risks, one to privacy and one to accuracy.

  • Plain explanations cut follow-up work
  • Tech firms often fall outside HIPAA
  • One confident error could delay care

Hundreds of millions of people treat ChatGPT as an informal health adviser, asking about symptoms, blood results, rashes and medications. An Australian survey found the habit widespread and likely to grow. OpenAI has built a product around it, and the product’s usefulness comes with two risks, one to privacy and one to accuracy.

ChatGPT Health is a separate space that can connect to medical records and wellness apps. Once people upload their records, it can answer with more context, and the answers seem smarter because they draw on each person’s own numbers.

Personal health data is scattered across patient portals, apps and PDFs, so context gets lost. One place that explains results in plain language cuts confusion and follow-up work. That saves time and helps people stick with treatment, even setting aside whether the tool makes them healthier.

People also use AI to prepare for appointments, working out what to ask a doctor or how a new medication fits their existing conditions. That changes where the conversation starts, and better-prepared patients can shorten consultations and cut repeated explanations.

The first risk starts where privacy law ends. In the United States, health data protection depends on who holds the data rather than how sensitive it is. Consumer technology companies often fall outside HIPAA, the main US health privacy law. That holds even for uploaded records that look identical to a medical chart, as an overview of the law explains. People expect one level of protection, and the law gives another.

Terms of service add to the problem. Consumer platforms can change how they handle data, and users have little say. As a tool becomes part of a daily routine, leaving it gets harder. Research on digital health privacy found that weak or patchy protections erode trust and make people less willing to share data, even when the tools are useful.

Accuracy is the second risk. ChatGPT writes the answer that seems most likely, without checking that it is true. A paper on why AI models make things up explains that plausible wrong answers persist because the usual ways of training and testing models reward them. In health, one confident error could push someone to delay care or make an unsafe choice.

Lead smarter with AI in just 5 minutes a week

Join 50K+ leaders getting our AI For Leaders newsletter every Wednesday: deep analysis, frameworks, and strategies.

No spam ever • Unsubscribe anytime

ChatGPT Health Is a Strategic Liability | Chief AI Leader