Microsoft's AI beat generalist doctors at diagnosing tough cases, in a lab test
A Microsoft-built AI system worked through medical cases step by step and got the right diagnosis far more often than individual doctors — but the test had real limits.
What happened: Microsoft researchers published a study in Science describing an AI tool, called MAI-DxO, that diagnosed difficult medical cases far more accurately than individual generalist doctors. On 304 hard cases from the New England Journal of Medicine, it got the right answer 83.2% of the time versus 36.1% for doctors using reference materials. On 325 real emergency-department admission records, it scored 80.3% versus 46.8%. The system isn't used in real clinics yet.
Why it matters: Most AI medical studies dump all the symptoms and test results on the AI at once, which doesn't match how doctors actually work. This study made both the AI and the doctors build a diagnosis step by step: deciding what to ask, what tests to order, and when they had enough information. That's a tougher, more realistic test, and Microsoft argues it better predicts how these tools would perform if deployed as working aids for clinicians facing a shortage of medical expertise.
How it works, plainly: MAI-DxO coordinates several AI 'agents' that act like a panel of doctors debating a case, running on OpenAI's GPT-5 in the main results (it also worked with Google, Anthropic, Meta and DeepSeek models). It asked far more questions before ordering a first test than human doctors did (25.5 versus 5.6 on average). When it was wrong, its guesses tended to land close to the real answer; doctors' misses were often further off. Cost can be tuned: without limits it cost more than doctors' work-ups, but capped versions still beat doctors on accuracy for less money.
The rollout: The comparison doctors were generalists working alone, not the specialists or teams who'd normally handle cases this complex, and the test was text-only with no physical exams or imaging. All the study's authors are Microsoft employees with equity in the company, and several hold a related patent application. Next, Microsoft will test MAI-DxO alongside real patients and doctors at Beth Israel Deaconess Medical Center in Boston, comparing its suggestions to human decisions in emergency and urgent care settings.