Deep learning multimodal fusion model based on ultrasound images and clinical-ultrasound texts for differentiating benign and malignant renal tumors
In brief
Combined ultrasound AI reached 0.92 AUC distinguishing benign from malignant kidney tumors
In an independent prospective test of 82 patients, a model combining ultrasound images with clinical-ultrasound text achieved an area under the curve of 0.92 and outperformed radiologists at different experience levels. The results suggest the tool could support tumor assessment, but testing at other hospitals and in larger patient groups is needed before its role in care is clear.
- Journal
- The British journal of radiology (Q1)
- Published
- 23 October 2026
- Study design
- Randomized controlled trial
- Evidence level
- Level 1, High (CEBM 1b)
- Authors
- Hongli Cao, Yilei Shi, Shumeng Yu, Xu Cao, Chen Li, Lichao Mou, et al.
- PMID
- 42922188
- DOI
- 10.1093/bjr/tqag238
Why clinicians should know about it
- Picked for Radiology, Radiation Oncology, Nuclear Medicine, Medical Physics and Imaging (top studies of the week, 1 November 2026): Prospective test evaluates ultrasound-based renal tumor diagnosis
Abstract
OBJECTIVES: To develop a deep learning multimodal fusion model based on ultrasound (US) images and clinical-US texts to differentiate benign and malignant renal tumors. METHODS: The retrospective dataset from January 2012 to February 2023 in a tertiary hospital was used for model development and randomly allocated to training and validation sets. An independent prospective test set was obtained from March 2023 to October 2024. We developed Model-G using grayscale US (GUS) images, Model-GC using GUS and color Doppler flow imaging images, Model-T using clinical-US texts, and Model-UT using US images and clinical-US texts. ResNet50 was used for image feature extraction, BERT for text encoding, and ViT for multimodal classification. RESULTS: In total, 1008 patients (740 in training set, 186 validation set, and 82 prospective test set) with a single renal tumor were included. In the validation set, Model-UT achieved the highest diagnostic performance in comparison to Model-G, Model-GC, and Model-T with an area under the receiver operating characteristic curve (AUC) of 0.89 (95% CI, 0.86-0.93) (all p-values < 0.05). In the prospective test set, Model-UT yielded an AUC of 0.92 (95% CI, 0.86-0.97), outperforming high-level, mid-level, and low-level radiologists (all p-values < 0.05). CONCLUSIONS: The multimodal fusion model presents promising potential in differentiating benign and malignant renal tumors. ADVANCES IN KNOWLEDGE: Deep learning model integrating US images with clinical-US texts improved benign-versus-malignant renal tumor differentiation. Decision-curve, controlled ablation, and SHAP analyses support its potential use as a clinical decision-support tool.
Abstract as published, via PubMed.
For healthcare professionals. The summary is generated by AI from the published abstract, and the evidence level is assigned automatically from the study design on the Oxford CEBM hierarchy. Neither is medical advice. Read the full paper before changing practice.