The doctor will not see you now—but the chatbot will, and its advice might just be dangerous. A new study has exposed the alarming risks of relying on AI for medical triage.

In the rush to integrate Artificial Intelligence into every facet of life, healthcare has become the new frontier. However, a damning global study published in the Annals of Internal Medicine suggests that we are moving too fast. The research found that popular AI chatbots—including ChatGPT, Claude, and Bard—frequently provide inaccurate, unsafe, and "hallucinatory" advice when faced with medical queries.

The findings are a wake-up call for a public that increasingly turns to "Dr. Google" and now "Dr. AI" for diagnosis. The study evaluated how these models handled 32 realistic patient scenarios, ranging from minor ailments to life-threatening emergencies. The results were disturbing: the chatbots offered incorrect triage guidance in about 35% of cases, often failing to recognize when a patient needed immediate emergency care.

The Hallucination Hazard

The most insidious issue identified is the AI's tendency to "hallucinate"—to confidently invent non-existent medical facts. In one cited example, a chatbot suggested a patient replace salt with sodium bromide, a toxin, leading to severe poisoning. These models are language predictors, not medical professionals; they prioritize plausible-sounding sentences over factual accuracy.

"These AI tools often struggle to judge how urgent a situation is," said Dr. Natansh D. Modi, the study's lead author. "That's particularly concerning when underestimating a serious condition might delay critical care." The AI lacks clinical judgment, the intuitive ability to read between the lines of a patient's complaint that a human doctor develops over years of practice.

A False Sense of Security

The danger is amplified by the authoritative tone of the chatbots. Unlike a search engine that lists sources, a chatbot provides a singular, conversational answer that mimics a consultation. This creates a false sense of security for the user.

• The Disinformation Machine: The study found that customized Large Language Models (LLMs) generated health disinformation in 88% of tested responses. Some models failed 100% of the time on specific health questions.
• The Safeguard Failure: While some models like Claude 3.5 Sonnet showed better safeguards, the overall landscape is a "Wild West" of unregulated medical advice.
• The Vulnerability: The bots are easily misled. When researchers introduced a single fake medical term into a scenario, the bots often elaborated on it, inventing symptoms and treatments for a disease that does not exist.

The verdict from the medical community is clear: AI is a powerful tool for administration, but it is nowhere near ready to replace the family doctor. Until robust, health-specific auditing is implemented, taking medical advice from a chatbot is a game of Russian roulette with your health.