From ACR O-RADS 2022 to Explainable Deep Learning: Comparative Performance of Expert Radiologists, Convolutional Neural Networks, Vision Transformers, and Fusion Models for Ultrasound-Based Risk Stratification of Ovarian Masses.

Abstract

The 2022 update of the Ovarian-Adnexal Reporting and Data System (O-RADS) improves risk stratification of adnexal lesions; however, radiologist interpretation remains subject to inter-observer variability and conservative diagnostic thresholds. Concurrently, deep learning (DL) models demonstrated promise in ovarian mass characterization. This study evaluates radiologist performance applying O-RADS version 2022 (v2022), compares it to convolutional neural network (CNN) and vision transformer (ViT) models, and investigates diagnostic gains from hybrid human-artificial intelligence (AI) frameworks with emphasis on explainable DL approaches that could enhance clinical applicability. In this retrospective study, a total of 512 ultrasound images from 227 patients (110 with at least 1 malignant lesion) were analyzed. Sixteen DL models, including DenseNets, EfficientNets, ResNets, VGGs, Xception, and ViTs were trained and validated. For each model, a hybrid framework integrating radiologist-assigned O-RADS scores with DL-predicted malignancy probabilities was constructed. Radiologist-only O-RADS assessment achieved an area under the curve (AUC) of 0.683 and an accuracy of 68.0%. CNN models yielded AUCs of 0.620-0.908 and accuracies of 59.2-86.4%, while ViT16-384 reached the best performance, with an AUC of 0.941 and an accuracy of 87.4%. Hybrid human-AI frameworks significantly enhanced most CNNs (9 out of 12 CNNs, p < .05) and ViTs (3 out of 4 ViTs, p < .05). DL models outperform radiologist-only O-RADS v2022 assessment. The integration of expert radiologist scores with AI yields the highest accuracy, supporting hybrid human-AI paradigms as a promising approach to standardize ultrasound interpretation, reduce false-positive diagnoses, and improve identification of high-risk ovarian lesions.