A local outpatient practice-level prediction model for short-term psychiatric emergency presentation
In brief
HIE data model predicted psychiatric emergency visits with 0.85 AUC
In one Montana psychiatry practice, a model using health information exchange data distinguished snapshots preceding a psychiatric or substance-related emergency visit within 30 days, with an area under the curve of 0.85. It outperformed general risk scores, but only 68 patients had visits and estimates were imprecise; prospective testing must show whether using the model improves care.
- Journal
- JAMIA open (Q1)
- Published
- 2 November 2026
- Study design
- Prospective / inception cohort
- Evidence level
- Level 2, Moderate (CEBM 2b)
- Authors
- John L Havlik, Ben Tyrrell, Jeanette Polaschek, Nathan Bell, Eric R Arzubi
- PMID
- 42974260
- DOI
- 10.1093/jamiaopen/ooag215
Why clinicians should know about it
- Picked for Health Informatics (paper of the day, 5 November 2026): Tests HIE-based risk prediction in practice with temporal validation
Abstract
OBJECTIVES: To determine whether statewide health information exchange (HIE) data can support practice-level prediction of short-term psychiatric emergency presentation, and to compare a locally trained model with general-purpose clinical risk scores. MATERIALS AND METHODS: Retrospective cohort of patients at 1 Montana psychiatry practice with records in the Big Sky Care Connect statewide HIE. Structured HIE data were linked to zip code-level Social Vulnerability Index and Area Deprivation Index measures. Each observation was a patient snapshot. Logistic regression, random forest, and LightGBM models were tuned with 5 × 5 nested cross-validation and evaluated on a temporally held-out test set. The primary outcome was psychiatric or substance-related emergency department (ED) presentation within 30 days. Discrimination was compared with a parsimonious clinical baseline and the LACE and Elixhauser indices. RESULTS: The test set contained 16 469 snapshots, of which 343 (2.1%) preceded a qualifying ED presentation (102 visits among 68 patients). A random forest model achieved area under the receiver-operator characteristic curve 0.85 (95% CI, 0.75-0.91). At a threshold targeting one-third recall, precision was 0.29 (0.12-0.44) and number needed to evaluate 3.4 (2.3-8.1). Discrimination exceeded the clinical baseline (Δ area under the curve 0.10), LACE (0.19), and Elixhauser (0.27); all P values less than .001. DISCUSSION: Structured HIE data, without clinical notes, supported a practice-specific model that outperformed general-purpose scores such as LACE and Elixhauser. Wide CIs reflect a small event count and constrain point-estimate claims. CONCLUSION: health information exchange infrastructure may allow participating practices to build locally calibrated psychiatric risk models. Prospective study of clinical utility is needed.
Abstract as published, via PubMed.
For healthcare professionals. The summary is generated by AI from the published abstract, and the evidence level is assigned automatically from the study design on the Oxford CEBM hierarchy. Neither is medical advice. Read the full paper before changing practice.