Doctors Much Better at Diagnosis than Symptom Checkers

Online symptom checkers – websites and apps that use computer algorithms to help patients self-diagnose – have become very popular. Many people jump on the internet to figure out what ails them by using WebMD, the Mayo Clinic’s Symptom Checker, AskMD,, and the like.

A study published this month in JAMA Internal Medicine suggests this is no substitute for talking to your doctor. It found that physicians were far more accurate than online symptom checkers in their diagnoses.

Modern Healthcare reports that researchers at Harvard Medical School and Brigham and Women’s Hospital tested whether trained physicians or online symptom checkers were more likely to diagnose a clinical case accurately. The results weren’t even close: Physicians listed the correct diagnosis first 72.1% of the time, whereas symptom checkers listed the correct diagnosis first only 34% of the time.

Researchers used 45 standardized patient cases to test the accuracy of 234 doctors against 23 symptom checkers. The cases were paragraph-long scenarios that provided symptoms and medical history, but no physical exam or test findings.

The case scenarios ranged from very serious diagnoses to benign situations. The study used 15 high, 15 medium, and 15 low-acuity condition scenarios. The case scenarios included 26 common and 19 uncommon conditions.

Researchers first used these 45 cases to test the diagnostic accuracy of 23 symptom checkers; results were published in a separate 2015 study. They then disseminated the same cases to physicians using the Human Diagnosis Project (Human Dx), an online platform where doctors share their own clinical cases and submit answers to others. Each case was solved by at least 20 physicians, and their diagnoses were reviewed independently by two physicians.

Doctors listed the correct diagnosis in their top three 84.3% of the time, while symptom checkers only did so 51.2% of the time.

Dr. Ateev Mehrotra, associate professor of health care policy and medicine at Harvard Medical School, authored the study. “I wasn’t surprised that the physicians would perform better,” Mehrotra said. He noted, however, that while physicians fare better than computers now, “I wouldn’t be surprised in the coming decade if we develop computer programs that do make diagnoses as accurate as physicians.”

Mehrotra said it would be useful to study how computers could improve physicians’ diagnostic accuracy. Though physicians outperformed the symptom checkers in all scenarios, they provided the incorrect diagnosis in about 15% of cases, the researchers found.

Interestingly, there were differences between doctors and symptom checkers in which types of cases they were most likely to diagnose correctly. Physicians were more likely to list the right diagnosis first for high-acuity cases than for low-acuity cases, and they listed the correct diagnosis first more often for uncommon conditions. Symptom checkers were better at diagnosing low-acuity and more common conditions than at diagnosing high-acuity and uncommon conditions.


Click here to read the Modern Healthcare article on the recent study comparing the diagnostic accuracy of doctors and online symptom checkers. (Access to the article requires free registration on the Modern Healthcare website.)


Leave a Reply