Microsoft’s AI Doctor Panel Beats Physicians 83% to 36%
In short
- Microsoft published its MAI-DxO diagnostic AI study in the journal Science on October 22, 2026.
- On 304 of medicine’s hardest NEJM cases: 83% correct for the AI vs 36% for physicians with textbooks and internet (22% without).
- It got there at half to one third of the test costs – using a “panel” of AI agents that argue with each other.
A panel of AI “doctors” beat human physicians more than 2 to 1 on the hardest cases in medicine. And this time it’s not a company blog claim – it’s a peer-reviewed paper in Science. The method behind it is something every business can copy.
What happened
Microsoft AI built a test called SDBench. It turns real cases into step-by-step puzzles: you start with little information and decide what to ask and which tests to order. Every test has a price. The cases: 304 complex ones from The New England Journal of Medicine, plus everyday emergency-room cases from the MIMIC-IV-ED dataset.
The results. On the hard NEJM cases, MAI-DxO got 83% right. Physicians got 36% with textbooks and internet, 22% without. On normal ER cases: 80% vs 47%. And the AI reached its answers at half or even a third of the cost, with fewer tests.
The interesting part is how. MAI-DxO isn’t one model giving one answer. It acts like a panel of virtual physicians: each one proposes diagnoses, picks the next question or test, and challenges the others. They debate over several rounds and weigh each test’s value against its cost. Swap in a newer model (Microsoft tested GPT-5.5) and the gains held.
Important: this is research, not a product. You can’t use MAI-DxO for your health, and nothing here is medical advice. The real test comes next – a prospective study with Beth Israel Deaconess Medical Center in Boston, measured against physicians on real patients.
How to use it
You don’t run a hospital. But the idea – an AI panel instead of one AI answer – works for any big business decision:
- Give roles. One agent proposes, one plays the sceptic, one checks the cost. Make them argue before you get a final answer.
- Make it ask before it answers. Like the doctor panel, the AI should request missing info step by step.
- Put a price on every step. Tell the panel what each extra check costs – it will stop wasting your money.
- Where: supplier selection, contract review, pricing, diagnosing why sales dropped. Build it in Make or n8n with 3 AI calls in a row.
Do the math: an example, use your real prices. A panel makes about 3 AI calls instead of 1. If one call costs you €0.05, a panel costs €0.15. For 1,000 decisions a month that’s €150 instead of €50 – €100 more, €1,200 a year. One avoided bad supplier deal pays for years of that.
What it means
For healthcare: AI as a second opinion for doctors is getting closer, but benchmarks aren’t hospitals. Microsoft itself says real clinical tests come first.
For business: the lesson is that orchestration beats a single prompt. Microsoft is also open-sourcing part of SDBench, so others can check and challenge the results. Right move.
Ivan’s Take
Impressive result – and peer-reviewed, which matters. But I’m not here to tell you about medicine; I’m here to tell you what to steal. Stop asking one AI one question for big decisions. Build a small panel that argues, asks for data and counts the cost – that’s the cheapest upgrade to your decision-making this year.
Source: Microsoft AI
If it doesn’t save you time or money, you won’t see it here
- Facts, not hypeEvery price and number is checked against the official source
- Always the real priceYou see what it costs before you click
- I say when it’s weakEven when it’s OpenAI or Google
