Human-AI Teams Make Better Medical Diagnoses

Hybrid collectives consisting of humans and artificial intelligence make significantly more accurate medical diagnoses than either medical professionals or AI systems alone. New research analyzing over 40,000 diagnoses reveals that combining human expertise with AI models creates a powerful diagnostic partnership that outperforms traditional approaches.

The study, published in Proceedings of the National Academy of Sciences, examined how physicians and five leading AI language models diagnosed more than 2,100 clinical cases. When working together, these human-AI teams achieved diagnostic accuracy that surpassed both individual doctors and AI-only systems.

Complementary Strengths and Weaknesses

The key to success lies in error complementarity—humans and AI make systematically different mistakes. When AI models failed to identify the correct diagnosis, human physicians often provided the right answer, and vice versa.

“Our results show that cooperation between humans and AI models has great potential to improve patient safety,” explains lead author Nikolas Zöller, a postdoctoral researcher at the Max Planck Institute for Human Development.

The research team found that AI collectives outperformed 85% of individual human diagnosticians. However, in numerous cases where AI failed completely, humans knew the correct diagnosis, often ranking it first in their differential diagnosis lists.

Dramatic Performance Improvements

Adding just one AI model to a group of human diagnosticians—or adding one human to AI systems—substantially improved results across multiple metrics:

• Top-5 accuracy increased when combining the best AI models with physician groups
• Even the worst-performing AI model improved human diagnostic teams
• Multiple AI models working together generally outperformed individual systems
• Hybrid teams showed the most reliable outcomes across all medical specialties tested

Real-World Clinical Potential

The researchers used clinical vignettes from the Human Diagnosis Project, which provides realistic case descriptions similar to what physicians encounter in practice. Each case included patient symptoms, medical records, and test results, creating authentic diagnostic challenges.

“It’s not about replacing humans with machines. Rather, we should view artificial intelligence as a complementary tool that unfolds its full potential in collective decision-making,” notes co-author Stefan Herzog, Senior Research Scientist at the Max Planck Institute for Human Development.

The study employed sophisticated text-processing techniques to standardize diagnoses using SNOMED CT medical terminology, allowing precise comparison between human and AI responses. This methodology enabled researchers to analyze diagnosis accuracy across different ranking positions and medical specialties.

Error Patterns Reveal Opportunities

When AI systems missed correct diagnoses entirely—occurring in 34% to 54% of cases depending on the model—individual humans provided the right answer 30% to 38% of the time. Conversely, when humans failed completely, AI models compensated in 31% to 51% of cases.

The research revealed that humans and AI disagree on their top diagnosis choice in substantial numbers of cases, but this disagreement proves beneficial rather than problematic. The error diversity ensures correct diagnoses accumulate more frequently than incorrect ones in collective decision-making.

Broader Applications and Limitations

Study coordinator Vito Trianni sees applications beyond medicine: “The approach can also be transferred to other critical areas—such as the legal system, disaster response, or climate policy—anywhere that complex, high-risk decisions are needed.”

However, researchers acknowledge important limitations. The study analyzed text-based case vignettes rather than actual patients in clinical settings. Whether results translate directly to real medical practice requires further investigation.

The research also focused solely on diagnosis, not treatment decisions. A correct diagnosis doesn’t automatically guarantee optimal patient care, and the study didn’t examine how AI-based support systems would be accepted by medical staff and patients.

Future Implications

The findings highlight particular promise for regions with limited healthcare access, where hybrid human-AI systems could contribute to more equitable medical care. The approach might help bridge gaps in medical expertise while maintaining essential human oversight.

As diagnostic errors cause an estimated 795,000 deaths and permanent disabilities in the United States annually, these results suggest significant potential for improving patient safety through thoughtful human-AI collaboration rather than wholesale replacement of human judgment.