探究基于大语言模型DeepSeek-R1构建的智能质量控制系统在甲状腺超声报告书写质量控制中的应用价值。

随机抽取2024年1月至12月四川省肿瘤医院超声医学科的甲状腺超声检查报告120份作为研究对象，分别由DeepSeek-R1和2名超声主治医师进行甲状腺超声检查报告的独立评估，将3名副高级职称医师组成的专家组的联合评审结果作为金标准。绘制DeepSeek-R1与人工质量控制的受试者操作特征（ROC）曲线评估其效能，计算曲线下面积（AUC）、敏感度、特异度、阳性预测值、阴性预测值，采用DeLong检验比较AUC的差异；采用Kappa检验评价不同质量控制者之间的结果一致性程度。

在120份报告中，以专家组评审为金标准，DeepSeek-R1模型的ROC曲线AUC为0.959[95%可信区间（CI）：0.902~1.000]，其敏感度为92.86%，特异度为98.91%，均优于2名超声主治医师[AUC：0.828（95%CI：0.720~0.936），0.835（95%CI：0.731~0.940）；敏感度：67.86%，71.43%；特异度：97.83%，95.65%）]。经DeLong检验，DeepSeek-R1与2位超声主治医师的AUC差异具有统计学意义（P=0.020、0.025）。Kappa检验显示，DeepSeek-R1与2位超声主治医师的判定结果具有良好的一致性（κ值分别为0.64和0.63）。

基于DeepSeek-R1模型的智能质量控制系统在甲状腺超声检查报告规范性审核中表现出优于人工的效能，能有效降低主观误差，具有良好的临床应用价值，但仍需关注模型局限性等问题。

To evaluate the utility of an intelligent quality control system based on the large language model DeepSeek-R1 in quality control of thyroid ultrasound report writing.

A total of 120 thyroid ultrasound examination reports from the Department of Ultrasound, Sichuan Cancer Hospital, collected between January and December 2024, were randomly selected as study subjects. The reports were independently evaluated by DeepSeek-R1 and two attending ultrasound physicians. The joint assessment results of an expert panel consisting of three associate chief physicians were regarded as the reference standard. Receiver operating characteristic (ROC) curves were generated to assess the performance of DeepSeek-R1 and human quality control, with the area under the curve (AUC), sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) calculated. Differences in AUC were compared using the DeLong test. Agreement among different quality controllers was evaluated with the Kappa statistic.

Using the expert panel's assessment as the reference standard, the AUC of the DeepSeek-R1 model was 0.959 [95% confidence interval (CI): 0.902–1.000], with a sensitivity of 92.86% and specificity of 98.91%, both superior to those of the two human quality controllers [AUC: 0.828 (95%CI: 0.720-0.936), 0.835(95%CI: 0.731-0.940); sensitivity: 67.86%, 71.43%; specificity: 97.83%, 95.65%]. The DeLong test showed that the differences in AUC between DeepSeek-R1 and the two physicians were statistically significant (P=0.020, 0.025). The Kappa test demonstrated good consistency between DeepSeek-R1 and the two physicians, with κ values of 0.64 and 0.63, respectively.

The intelligent quality control system based on the DeepSeek-R1 model demonstrated superior performance to human reviewers in the normative evaluation of thyroid ultrasound reports, effectively reducing subjective errors and showing promising clinical applicability. However, the limitations of the model requires careful consideration.