Performance of Large Language Models Across Diagnostic Tasks in Oral and Maxillofacial Pathology: A Systematic Review
In brief
Across 16 studies, language models show inconsistent accuracy in oral pathology
A review of 16 studies found that large language models did not consistently match specialists on exact diagnoses. They often included the correct answer among several possibilities, while classification results showed high specificity but variable sensitivity; evidence on report generation came from just one study. The findings suggest possible support for selected tasks, not autonomous diagnosis.
- Journal
- Pathobiology : journal of immunopathology, molecular and cellular biology (Q1)
- Published
- 16 October 2026
- Study design
- Systematic review of cohort studies
- Evidence level
- Level 2, Moderate (CEBM 2a)
- Authors
- Sohaib Shujaat, Marryam Riaz, Hisham Albrikeet, Hongyang Ma
- PMID
- 42886619
- DOI
- 10.1159/pat/aehag008
Why clinicians should know about it
- Picked for Pathology and Forensic Medicine (top studies of the week, 18 October 2026): Systematic review of LLM diagnostic tasks in oral pathology
Abstract
Introduction Large language models (LLMs) are increasingly evaluated for diagnostic applications; however, studies in oral and maxillofacial pathology (OMFP) assess materially different diagnostic tasks and outcomes. This systematic review synthesised evidence according to the specific task evaluated. Methods PubMed, Web of Science, and Cochrane Central were searched without language or date restrictions, supplemented by grey-literature screening and citation tracking. Studies evaluating defined OMFP diagnostic tasks using task-appropriate outcomes were included. Screening, data extraction, and quality assessment were independently performed by two reviewers. Owing to clinical and methodological heterogeneity, evidence was synthesised narratively by diagnostic task. Results Sixteen studies published between 2024 and 2025 were included. Seven evaluated single-diagnosis identification, six differential-diagnosis generation or ranking, five binary or multiclass classification/detection, and one structured diagnostic report generation; counts were non-mutually exclusive. Exact-diagnosis performance varied considerably and did not demonstrate consistent specialist equivalence. Differential-diagnosis studies generally reported higher Top-k inclusion than Top-1 accuracy, but these outcomes did not establish exact diagnostic correctness. Classification studies frequently reported high specificity but variable sensitivity. The single report-generation study reported item-level rather than whole-case diagnostic outcomes. Conclusion LLMs may support selected OMFP tasks, particularly differential-diagnosis generation. However, exact-diagnosis performance remains inconsistent, classification sensitivity is task-dependent, and evidence for structured report generation is limited. Current evidence does not support a single overall estimate of diagnostic performance or autonomous diagnostic use.
Abstract as published, via PubMed.
For healthcare professionals. The summary is generated by AI from the published abstract, and the evidence level is assigned automatically from the study design on the Oxford CEBM hierarchy. Neither is medical advice. Read the full paper before changing practice.