Effectiveness of Artificial Intelligence Systems in Healthcare: A Systematic Review of Clinical and Organizational Outcomes
In brief
40 of 54 real-world AI healthcare trials reported positive effects
In this review of 54 randomized trials, 40 reported a positive direction of effect, though that does not mean every intervention improved patient outcomes. Diagnostic AI showed consistent accuracy gains, while conversational tools had positive signals for self-management and adherence; decision-support results depended on workflow integration. Larger pragmatic trials with standardized outcomes are still needed to show where AI works reliably.
- Journal
- Healthcare (Basel, Switzerland) (Q2)
- Published
- 20 October 2026
- Study design
- Systematic review / meta-analysis of RCTs
- Evidence level
- Level 1, High (CEBM 1a)
- Authors
- Andrea Ceccarelli, Tommaso Montagnani, Tristan Darius Marin, Vincenzo Cassetta, Marco Senni, Marco Montalti, et al.
- PMID
- 42940577
- DOI
- 10.3390/healthcare14203532
Why clinicians should know about it
- Picked for Health Informatics (top studies of the week, 1 November 2026): Real-world AI trials assess clinical and organizational effects
Abstract
BACKGROUND: The integration of artificial intelligence (AI)-based systems into healthcare is accelerating rapidly, yet evidence of their real-world clinical and organizational effectiveness remains fragmented. METHODS: A PRISMA 2020-compliant systematic review of randomized controlled trials (RCTs) evaluating AI-based interventions in real-world healthcare settings was conducted. MEDLINE (PubMed), CINAHL, Cochrane CENTRAL, and ClinicalTrials.gov were searched up to January 2026. AI interventions were classified post hoc into three functional categories: perceptive AI, predictive/decision support AI, and conversational/adaptive AI. The direction of effect was classified as positive, neutral, mixed, or negative based on the primary endpoint, effect magnitude, and authors' interpretation. Risk of bias was assessed using the Cochrane RoB-2 tool. RESULTS: A total of 54 RCTs were included. Overall, 40 studies (74.1%) reported a positive direction of effect, nine a neutral result, four a mixed result, and one a negative result. Perceptive AI (n = 17) showed consistent benefits in diagnostic accuracy. Predictive/decision support AI (n = 15) showed heterogeneous results, with effectiveness dependent on workflow integration. Conversational/adaptive AI (n = 22) showed the most consistent positive signal, particularly for chronic disease self-management, behavioral change, and therapeutic adherence. CONCLUSIONS: Evidence from randomized trials suggests that AI-based interventions may provide meaningful benefits in real-world healthcare settings, with effectiveness varying by functional category, implementation context, and outcome domain. Future research should prioritize pragmatic multicenter trials with standardized reporting of clinical, organizational, and implementation outcomes.
Abstract as published, via PubMed.
For healthcare professionals. The summary is generated by AI from the published abstract, and the evidence level is assigned automatically from the study design on the Oxford CEBM hierarchy. Neither is medical advice. Read the full paper before changing practice.