To compare the diagnostic performance of computer-aided diagnosis (CAD) system versus 111 radiologists with different experiences (senior and junior radiologists) in identifying benign and malignant thyroid nodules.

A total of 50 thyroid nodules and 111 radiologists were enrolled in this study. All the diagnostic results for the 50 nodules were estimated by radiologists and CAD system simultaneously. The diagnostic sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and accuracy of senior and junior radiologists with maximum accuracy and CAD for differentiation between benign and malignant thyroid nodules were compared. The McNemar test was used to compare the accuracy between the two groups. Receiver operating characteristic curve (ROC) analysis was performed, and the Z test was used to compare the area under the ROC curve (AUC).

The sensitivity, specificity, PPV, NPV, and overall accuracy of CAD in differentiation between benign and malignant thyroid nodules were 76.9%, 87.5%, 86.9%, 77.8%, and 82.0%, respectively; the corresponding percentages were 86.9%, 77.8%, 76.9%, 87.5%, and 82.0% for the senior radiologist with maximum accuracy, and 82.6%, 70.4%, 70.4%, 82.6%, and 76% for the junior radiologist with maximum accuracy. In ROC curve analysis, the AUC values were 0.82, 0.82, and 0.76 for CAD, senior and junior radiologists with maximum accuracy, separately, and the CAD system achieved a diagnostic accuracy that was comparable to that of the senior radiologist but higher than that of the junior radiologist (P<0.05). Both CAD system and senior radiologist had a larger AUC in the differential diagnosis of thyroid nodules than the junior radiologist; however, the difference between CAD system and the senior radiologist was not statistically significant (0.82 vs 0.76, P=0.5). For radiologists, the nodules in Hashimoto's thyroiditis and small hypoechoic nodules with colloid inside the lesion tended to be misdiagnosed. For CAD system, the distribution of nodules and the presence of macrocalcification inside the lesion with wide acoustic shadow may also influence the analysis of the CAD system.

In thyroid nodule diagnosis, CAD system can achieve a diagnostic accuracy comparable to that of the senior radiologist with maximum accuracy, but higher than that of the junior radiologist with maximum accuracy. The distribution of thyroid nodules and the presence of macrocalcification inside the lesion may influence the performance the CAD system, while the nodules in Hashimoto's thyroiditis and small hypoechoic nodules with colloid inside the lesion are harder to distinguish for radiologists.