Is ‘Doctor AI’ Reliable? We Tested ChatGPT Against a General Practitioner: “If You Follow This, You’ll Make Things Worse”
The increasing accessibility of artificial intelligence chatbots like ChatGPT raises questions about their potential role in healthcare. A recent test compared the diagnostic capabilities of ChatGPT with those of a general practitioner, revealing significant discrepancies and potential risks for patients who might rely solely on AI for medical advice. This comparison highlights the importance of critical evaluation when considering AI tools for health-related concerns.
Researchers put ChatGPT to the test by presenting it with patient cases and comparing its responses to those of a practicing physician. The results indicated that while ChatGPT can provide information, it sometimes offers inaccurate or even harmful advice. “If you follow this, you’ll make things worse,” one physician stated, emphasizing the potential dangers of relying on AI without professional medical oversight.
The study found that ChatGPT sometimes struggled to differentiate between minor ailments and more serious conditions. In some instances, the chatbot failed to recognize red flags that a human doctor would immediately identify. This raises concerns about the potential for misdiagnosis and delayed treatment if individuals self-diagnose and self-treat based on AI-generated recommendations.
ChatGPT, developed by OpenAI, interacts in a conversational way, attempting to answer follow-up questions and acknowledge errors. OpenAI details the model’s capabilities, noting its ability to challenge incorrect premises and reject inappropriate requests. However, the recent testing suggests that its medical knowledge and judgment are not yet on par with a trained healthcare professional.
The availability of ChatGPT on platforms like Windows, with features like voice interaction, further expands its accessibility. The Microsoft Store listing for ChatGPT highlights its convenience, but also underscores the need for caution when using it for health-related inquiries. The findings from this test serve as a reminder that AI should be viewed as a tool to *supplement*—not replace—the expertise of qualified medical professionals.
Experts emphasize that while AI has the potential to revolutionize healthcare, it is crucial to approach its use with a healthy dose of skepticism and to always consult with a doctor for any health concerns. The study’s results suggest that relying solely on AI for medical advice could have detrimental consequences for patient health.
For those seeking reliable health information, resources like the Centers for Disease Control and Prevention (CDC) and the National Institutes of Health (NIH) remain the gold standard. ChatGPT’s website describes it as an AI chatbot for everyday use, but the recent testing suggests its limitations in the complex field of medical diagnosis and treatment.
