To develop an artificial intelligence (AI) system that can automatically segment and diagnose benign and malignant thyroid nodules.

The ultrasound images of 872 patients with thyroid nodules confirmed by puncture biopsy at Shanghai Tenth People's Hospital from October 2017 to October 2018 were collected, and the results were processed, monitored, and finally fed back by AI methods. Then, an AI system was established, and the system was verified and tested internally. According to a ratio of 6:2:2, all the collected ultrasound images were divided into training set, validation set, and internal test set for preliminary verification test. The ultrasound images of 209 patients with thyroid nodules (a total of 209 nodules) in other hospitals were re-verified, and the sensitivity, specificity, accuracy, positive predictive value, and negative predictive value of a junior physician group, a senior physician group, and the AI system in the diagnosis of benign and malignant thyroid nodules were calculated using the pathological results of puncture biopsy or surgery as the diagnostic criteria. The receiver operation characteristic curves of the three in the diagnosis of benign and malignant thyroid nodules were plotted, and the area under the curve (AUC) was calculated. The Delong test was used to compare the diagnostic performance of the AI system with junior physicians and senior physicians.

The automatic nodule segmentation rates of the AI system were 98.8%, 98.9%, and 98.1% in the validation set, internal test set, and external test set, respectively. In the external test set, there were no significant differences in the diagnostic sensitivity, specificity, or accuracy between the AI system and the junior or senior physician group (P>0.017 for all). The AUC of the AI system in the diagnosis of benign and malignant thyroid nodules was better than that of junior physicians [0.885 (95%CI: 0.842-0.929) vs 0.823 (95%CI: 0.771-0.875), P=0.022], but similar to that of senior physicians [0.932 (95%CI: 0.897-0.966)] (P=0.096).

We have developed an AI system that can automatically segment and diagnose benign and malignant thyroid nodules, which has high diagnostic efficacy in the external test set, and it is expected to assist junior physicians to more accurately identify benign and malignant thyroid nodules.