ChatGPT Diagnoses Better than Doctors: How It Works in Practice.

Unforeseen Consequences of Medical Self-Diagnosis

According to Vox: A German artist who loved painting outdoors ended up in the hospital due to an insect bite with several symptoms that doctors could not decipher. After a month of unsuccessful treatment, he started entering his medical information into ChatGPT, which eventually suggested a diagnosis: tularemia, also known as rabbit fever. The chatbot turned out to be correct; this case was later documented in a peer-reviewed medical study.

At the same time, another study described a man who went to a hospital in the United States showing signs of psychosis, suspecting that his neighbor was poisoning him. As it turned out, the patient had asked ChatGPT about alternatives to sodium chloride (table salt). The chatbot advised him to use sodium bromide, which is used for swimming pool cleaning. He consumed this toxic substance for three months, and when he stopped, he needed a three-week course of psychiatric care to stabilize.

We all know how to search in Google for information about mysterious illnesses. You enter symptoms, sometimes find helpful advice, and sometimes feel anxious, convincing yourself that you have a rare undiagnosed form of cancer. Thanks to generative artificial intelligence, the process of searching for information has taken on a new twist. Meet Dr. ChatGPT.

AI chatbots are an appealing stand-in for a human physician, especially given the ongoing doctor shortage as well as the broader barriers to accessing health care in the United States.

ChatGPT is not a doctor the same way Google is not a doctor. Searching for medical information on both platforms can equally lead to incorrect conclusions. However, unlike Google, which merely directs users to information, ChatGPT and other large language models (LLMs) offer interaction capabilities. They are designed for convenience, engagement, and constant availability. This makes AI chatbots appealing alternatives to human doctors, especially considering the shortage of healthcare workers and other barriers to access in the U.S.

As the rabbit fever case shows, these tools can process varied data, and having learned from numerous medical articles, they sometimes arrive at conclusions that doctors might have missed. However, they can also provide genuinely dangerous medical advice.

There is a difference between asking a chatbot for medical advice and discussing your health. If done correctly, communicating with ChatGPT may help improve conversations with your doctor and consequently enhance medical care. Just don't let AI convince you to eat pool cleaner.

Right and Wrong Ways to Communicate with Dr. ChatGPT

Many people are already using ChatGPT for medical consultations. According to a KFF 2024 survey, approximately one in six adults in the U.S. claims to use AI chatbots for medical advice every month. However, most of them are not confident in the accuracy of the information provided by the bots, which makes sense considering the common tendency of LLMs to form 'hallucinations' and the possibility of providing harmful medical information. The main challenge for the average user is the ability to distinguish fact from fiction.

“Honestly, I think people should be very cautious using this for medical purposes, especially if they lack the expertise to distinguish what is true and what is not,” said Dr. Roxanne Danesh, a Professor and AI researcher at Stanford Medical School. “When it works, it looks quite good, but when it doesn't—it can be catastrophic.”

Chatbots can also be 'yes men' or aim to please, which can lead to incorrect advice if they sense that it's what you want to hear.

If you've got a list of things to ask your doctor about, ChatGPT could help you craft questions.

However, if you remember to check symptoms, tools like ChatGPT can be very useful for gaining additional information about your health based on what your doctor has already said or for better understanding their medical records. If you have a list of questions for your doctor, ChatGPT can assist in formulating them. If you have test results and need to discuss next steps with your doctor, you can go over it with the chatbot, but you shouldn’t expect it to provide specific medical advice.

And in fact, when it comes to communication, there is evidence that ChatGPT may even be better at it. One 2023 study compared the responses of doctors to health questions with the answers generated by AI to the same questions. Medical professionals evaluated all the responses and found that those generated by the chatbot had better quality and more empathy. However, this does not replace a real encounter with a doctor. It’s worth noting that, on average, patients get only 18 minutes during their primary care physician appointment. If you visit just once a year, that’s not a lot of time for conversation with your doctor.

It's important to remember that, unlike your doctor, ChatGPT does not comply with HIPAA regulations. Chatbots typically have limited privacy safeguards in place. This means that any medical information you provide could be stored in AI memory and used to train large language models in the future.

The Future of Bot Services in Medical Care

Even if you aren’t using AI to solve medical mysteries, your doctor likely already is. According to an Elsevier report for 2025, about half of doctors said they are working with AI tools, and a few more say it saves them time. One in five doctors claims to have used AI for a second opinion on complex cases. This doesn't necessarily mean your doctor is asking ChatGPT about your symptoms.

Doctors have been using AI-based tools to aid in patient diagnostics and note-taking long before ChatGPT emerged. Among these tools are clinical decision support systems, specifically designed for physicians, which already outperform regular chatbots—though chatbots can indeed complement existing tools. A 2023 study showed that doctors who worked with ChatGPT diagnosed test cases only slightly better than those who worked alone. Interestingly, ChatGPT itself performed the best.

This research became a topic of media discussion, likely due to the implication that AI chatbots might be better than doctors at diagnosing. One of the co-authors, Dr. Adam Rodman, suggests that this might not be the case if doctors were more open to listening to ChatGPT than considering that chatbots are wrong when the results do not align with their own conclusions. Of course, AI can be wrong, but it can also spot connections that humans might miss. Once again, recall the case of rabbit fever.

“Patients need to talk to their doctors about their LLM use, and honestly, doctors should talk to their patients about their LLM use.”

“The average physician can clearly see when something is off,” said Rodman, an internist at Beth Israel Deaconess Medical Center and a lecturer at Harvard Medical School. “I don’t know if the average patient has the same capacity.”

In the near future, don’t expect to see Dr. ChatGPT in your local clinic. Rather, you will see AI in the role of an assistant saving the doctor time on note-taking, and possibly later helping analyze data. Your physician may use AI to compile messages for patients more quickly. With the development of AI technologies, the wider adoption of such tools in diagnostics and obtaining second opinions is possible. However, this doesn’t mean you should rush to ChatGPT with urgent medical questions. If you decide to do so, be sure to inform your doctor about your experience.

“Patients should talk to their doctors about LLM usage, and honestly, doctors should communicate with their patients about their LLM usage,” emphasized Rodman. “If we are just open with each other, it will lead to more productive conversations.”