Clinical value of online artificial intelligent quality control system in assessing obstetric ultrasound images

doi:10.3877/cma.j.issn.1672-6448.2022.07.010

Abstract

Abstract:

Objective

To assess the clinical value of online artificial intelligent quality control system in assessing obstetric ultrasound images.

Methods

A total of 374 191 images of 15 640 obstetric ultrasound cases from 998 doctors in 137 hospitals in Shenzhen and Chongqing were selected from January 1 to June 30, 2021, and the quality of the images was evaluated using online artificial intelligent quality control system. Based on each image quality assessment result, the proportion of different standard levels of all planes was calculated. The appeal results were also recorded to observe the accuracy of the system. To survey the efficiency of the system, paired sample t test was used to compare the time spent by intelligent quality control and manual quality control.

Results

The overall standard rate, substandard rate, and nonstandard rate were 81.18%, 12.06%, and 6.76%, respectively. A total of 285 appealed images (0.076%) were reviewed by authoritative experts, who confirmed the initial diagnosis in 126 images (44.21%), and did not support the initial diagnosis in 159 images (55.79%); the accuracy of the system reached 99.96% (374 032/374 191). The average time spent by intelligent quality control for 100 images was (32.7±5.1) s, significantly shorter than that spent by manual quality control by two ultrasound physicians [(705.3±37.2) s and (724.6±40.4) s, t=62.667 and 56.396, respectively, P＜0.001].

Conclusion

The intelligent quality control system of obstetric ultrasound images allows the quality control to be performed objectively, accurately, and efficiently, which is of great significance to guide the improvement of image quality.

Key words: Artificial intelligence, Obstetrics, Ultrasound, Quality control

Ying Tan, Huaxuan Wen, Guiyan Peng, Dandan Luo, Xin Wen, Yao Jiang, Wenlan Huang, Shengli Li. Clinical value of online artificial intelligent quality control system in assessing obstetric ultrasound images[J]. Chinese Journal of Medical Ultrasound (Electronic Edition), 2022, 19(07): 649-655.

/ / Recommend

Add to citation manager EndNote|Ris|BibTeX

URL: https://chaosheng.cma-cmc.com.cn/EN/10.3877/cma.j.issn.1672-6448.2022.07.010

https://chaosheng.cma-cmc.com.cn/EN/Y2022/V19/I07/649

Figures/Tables 4

References 16

1	李胜利, 邓学东. 产前超声检查指南(2012) [J/CD]. 中华医学超声杂志(电子版), 2012, 9(7): 574-580
2	Salomon LJ, Alfirevic Z, Berghella V, et al. Practice guidelines for performance of the routine mid-trimester fetal ultrasound scan [J]. Ultrasound Obstet Gynecol, 2011, 37(1): 116-126.
3	Jaudi S, Granger B, Herpin CN, et al. Online audit and feedback improve fetal second-trimester four-chamber view images: a randomised controlled trial [J]. Prenat Diagn, 2013, 33(10): 959-964.
4	Ursem N, Peters I, Kraan-van der Est M, et al. An audit of second-trimester fetal anomaly scans based on a novel image-scoring method in the southwest region of the netherlands [J]. J Ultrasound Medicine, 2017, 36(6): 1171-1179.
5	Wu L, Cheng J, Li S, et al. FUIQA: Fetal ultrasound image quality assessment with deep convolutional networks [J]. IEEE Trans Cybern, 2017, 47(5): 1336-1349.
6	Lin Z, Li S, Dong N, et al. Multi-task learning for quality assessment of fetal head ultrasound images [J]. Med Image Anal, 2019, 58: 101548.
7	Dong J, Liu S, Liao Y, et al. A generic quality control framework for fetal ultrasound cardiac four-chamber planes [J]. IEEE J Biomed Health Inform, 2020, 24(4): 931-942.
8	Ciritsis A, Rossi C, Eberhard M, et al. Automatic classification of ultrasound breast lesions using a deep convolutional neural network mimicking human decision-making [J]. Eur Radiol, 2019, 29(10): 5458-5468.
9	Anas E, Mousavi P, Abolmaesumi P. A deep learning approach for real time prostate segmentation in freehand ultrasound guided biopsy [J]. Med Image Anal, 2018, 48: 107-116.
10	Yu Z, Tan E, Ni D, et al. A deep convolutional neural network-based frame work for automatic fetal facial standard plane recognition [J]. IEEE J Biomed Health Inform, 2018, 22(3): 874-885.
11	Kim B, Kim K, Park Y, et al. Machine-learning-based automatic identification of fetal abdominal circumference from ultrasound images [J]. Physiol Meas, 2018, 39(10): 105007.
12	Abdi A, Luong C, Tsang, T, et al. Automatic quality assessment of echocardiograms using convolutional neural networks: feasibility on the apical four-chamber view [J]. IEEE Trans Med Imaging, 2017, 36(6): 1221-1230.
13	Chaoui R, Benoit B, Mitkowska-Wozniak H, et al. Assessment of intracranial translucency (IT) in the detection of spina bifida at the 11-13-week scan [J]. Ultrasound Obstet Gynecol, 2009, 34(3): 249-252.
14	罗丹丹, 黄怡, 李胜利, 等. 胎儿脊髓圆锥位置新测距法及其在脊髓拴系中的应用 [J]. 中华超声影像学杂志, 2018, 27(3): 252-258.
15	Rodriguez M, Prats P, Munoz A, et al. Sonographic evaluation of the fetal conus medullaris [J]. Prenat Diagn, 2014, 34(11): 1111-1114.
16	Napolitano R, Donadono V, Ohuma E, et al. Scientific basis for standardization of fetal head measurements by ultrasound: a reproducibility study [J]. Ultrasound Obstet Gynecol, 2016, 48(1): 80-85.

切面名称	常见显示不足的原因
胎儿颈项透明层切面	颈后皮肤透明层显示不清；鼻尖及鼻前皮肤显示不清；菱脑显示不清；出现侧脑室；出现锁骨；出现肋骨；间脑显示不清；鼻骨显示不清；上颌骨显示不清；下颌骨呈长条形；下颌骨显示不清
胎儿头臀长测量切面	生殖器显示不清；鼻尖及鼻前皮肤显示不清；菱脑显示不清；出现侧脑室；出现锁骨；出现肋骨；间脑显示不清；鼻骨显示不清；上颌骨显示不清；下颌骨呈长条形；下颌骨显示不清
颈胸段脊柱矢状切面	脊髓显示不清；椎体骨化中心显示不清
腰骶尾段脊柱矢状切面	脊髓和脊髓圆锥显示不清；脊柱末端显示不清；椎体骨化中心显示不清
三血管切面	右肺动脉显示不清；上腔静脉显示不清
中段脊柱矢状切面	脊髓显示不清；椎体骨化中心显示不清
尺桡骨冠状切面	尺骨显示不清；桡骨显示不清；尺骨近端骨骺显示不清；桡骨近端骨骺显示不清；尺骨远端骨骺显示不清；桡骨远端骨骺显示不清
鼻唇冠状切面	鼻子显示不清；下唇显示不清；上唇显示不清
丘脑水平横切面	出现小脑；一侧丘脑显示不清；透明隔腔显示不清；大脑外侧裂显示不清
全程脊柱矢状切面	脊髓显示不清；椎体骨化中心显示不清
宫颈内口矢状切面	宫颈内口显示不清；宫颈显示不清
膈肌冠状切面	膈肌显示不清；胃泡显示不清；心脏显示不清
右心室流出道切面	右心室显示欠清；主肺动脉显示不清；主动脉弓或升主动脉或左心室显示不清
颜面部正中矢状切面	下颌骨呈长条形；出现眼眶；下颌骨显示不清；鼻子显示不清；上唇显示不清；下唇显示不清
心底短轴切面	右肺动脉显示不清；右心房显示欠清；右心室显示欠清
双肾冠状切面	一侧肾显示不清；一侧肾盂显示不清；双侧肾盂显示不清
左侧膈肌矢状切面	膈肌显示不清；胃泡显示不清；心脏显示不清
上腹部水平横切面	出现肺；出现肾；脐静脉和门静脉显示不清；胃泡显示不清
双眼球水平横切面	一侧晶状体显示不清；双侧晶状体显示不清；一侧眼球显示不清
胫腓骨冠状切面	胫骨显示不清；腓骨显示不清；胫骨近端骨骺显示不清；腓骨近端骨骺显示不清；胫骨远端骨骺显示不清；腓骨远端骨骺显示不清
右侧膈肌矢状切面	膈肌显示不清；出现心脏
侧脑室水平横切面	出现小脑；一侧丘脑显示不清；透明隔腔显示不清；大脑外侧裂显示不清
小脑水平横切面	小脑实质显示不清；丘脑显示不清；透明隔腔显示不清；大脑外侧裂显示不清
三血管气管切面	气管显示不清；上腔静脉显示不清
左肾矢状切面	胃泡显示不清；肾盂显示不清
右肾矢状切面	肾盂显示不清
左右肺动脉分叉切面	左肺动脉显示不清；右肺动脉显示不清
双肾横切面	一侧肾盂显示不清；双侧肾盂显示不清；一侧肾显示不清
膀胱水平横切面彩色多普勒	双侧脐动脉显示不清；一侧脐动脉显示不清；膀胱显示不清
左心室流出道切面	左心室流出道及主动脉显示欠清；左心室显示欠清
肱骨长轴切面	肱骨显示不清；肱骨近端骨骺显示不清；肱骨远端骨骺显示不清；肱骨没有平行放置
四腔心切面	左心房显示不清；右心房显示不清；左心室显示不清；右心室显示不清，室间隔显示不清
脐带腹壁插入处横切面	脐带腹壁插入口显示不清；腹壁显示不清
股骨长轴切面	股骨显示不清；股骨近端骨骺显示不清；股骨远端骨骺显示不清；股骨没有平行放置
胆囊水平横切面	胆囊显示欠清
胎盘脐带插入口切面	脐带插入口显示不清

切面名称	常见显示不足的原因
胎儿颈项透明层切面	颈后皮肤透明层显示不清；鼻尖及鼻前皮肤显示不清；菱脑显示不清；出现侧脑室；出现锁骨；出现肋骨；间脑显示不清；鼻骨显示不清；上颌骨显示不清；下颌骨呈长条形；下颌骨显示不清
胎儿头臀长测量切面	生殖器显示不清；鼻尖及鼻前皮肤显示不清；菱脑显示不清；出现侧脑室；出现锁骨；出现肋骨；间脑显示不清；鼻骨显示不清；上颌骨显示不清；下颌骨呈长条形；下颌骨显示不清
颈胸段脊柱矢状切面	脊髓显示不清；椎体骨化中心显示不清
腰骶尾段脊柱矢状切面	脊髓和脊髓圆锥显示不清；脊柱末端显示不清；椎体骨化中心显示不清
三血管切面	右肺动脉显示不清；上腔静脉显示不清
中段脊柱矢状切面	脊髓显示不清；椎体骨化中心显示不清
尺桡骨冠状切面	尺骨显示不清；桡骨显示不清；尺骨近端骨骺显示不清；桡骨近端骨骺显示不清；尺骨远端骨骺显示不清；桡骨远端骨骺显示不清
鼻唇冠状切面	鼻子显示不清；下唇显示不清；上唇显示不清
丘脑水平横切面	出现小脑；一侧丘脑显示不清；透明隔腔显示不清；大脑外侧裂显示不清
全程脊柱矢状切面	脊髓显示不清；椎体骨化中心显示不清
宫颈内口矢状切面	宫颈内口显示不清；宫颈显示不清
膈肌冠状切面	膈肌显示不清；胃泡显示不清；心脏显示不清
右心室流出道切面	右心室显示欠清；主肺动脉显示不清；主动脉弓或升主动脉或左心室显示不清
颜面部正中矢状切面	下颌骨呈长条形；出现眼眶；下颌骨显示不清；鼻子显示不清；上唇显示不清；下唇显示不清
心底短轴切面	右肺动脉显示不清；右心房显示欠清；右心室显示欠清
双肾冠状切面	一侧肾显示不清；一侧肾盂显示不清；双侧肾盂显示不清
左侧膈肌矢状切面	膈肌显示不清；胃泡显示不清；心脏显示不清
上腹部水平横切面	出现肺；出现肾；脐静脉和门静脉显示不清；胃泡显示不清
双眼球水平横切面	一侧晶状体显示不清；双侧晶状体显示不清；一侧眼球显示不清
胫腓骨冠状切面	胫骨显示不清；腓骨显示不清；胫骨近端骨骺显示不清；腓骨近端骨骺显示不清；胫骨远端骨骺显示不清；腓骨远端骨骺显示不清
右侧膈肌矢状切面	膈肌显示不清；出现心脏
侧脑室水平横切面	出现小脑；一侧丘脑显示不清；透明隔腔显示不清；大脑外侧裂显示不清
小脑水平横切面	小脑实质显示不清；丘脑显示不清；透明隔腔显示不清；大脑外侧裂显示不清
三血管气管切面	气管显示不清；上腔静脉显示不清
左肾矢状切面	胃泡显示不清；肾盂显示不清
右肾矢状切面	肾盂显示不清
左右肺动脉分叉切面	左肺动脉显示不清；右肺动脉显示不清
双肾横切面	一侧肾盂显示不清；双侧肾盂显示不清；一侧肾显示不清
膀胱水平横切面彩色多普勒	双侧脐动脉显示不清；一侧脐动脉显示不清；膀胱显示不清
左心室流出道切面	左心室流出道及主动脉显示欠清；左心室显示欠清
肱骨长轴切面	肱骨显示不清；肱骨近端骨骺显示不清；肱骨远端骨骺显示不清；肱骨没有平行放置
四腔心切面	左心房显示不清；右心房显示不清；左心室显示不清；右心室显示不清，室间隔显示不清
脐带腹壁插入处横切面	脐带腹壁插入口显示不清；腹壁显示不清
股骨长轴切面	股骨显示不清；股骨近端骨骺显示不清；股骨远端骨骺显示不清；股骨没有平行放置
胆囊水平横切面	胆囊显示欠清
胎盘脐带插入口切面	脐带插入口显示不清

[1]	Qi Lyu, Pinjing Hui, Yafang Ding, Yanhong Yan. Carotid contrast-enhanced ultrasound assessment of plaque vulnerability and its correlation with ischemic stroke [J]. Chinese Journal of Medical Ultrasound (Electronic Edition), 2023, 20(10): 1040-1045.
[2]	Lei Ding, Wen Luo, Xiao Yang, Lina Pang, Peidi Zhang, Haijing Liu, Jiani Yuan, Jin Liu. Application of high-frame-rate contrast-enhanced ultrasound in evaluation of imaging features of C-TIRADS 4-5 thyroid nodules [J]. Chinese Journal of Medical Ultrasound (Electronic Edition), 2023, 20(09): 887-894.
[3]	Qian Zhang, Jiahui Chen, Xuemeng Gao, Aoxue Zhao, Ying Huang. Value of radiomics features based on high-frame-rate contrast-enhanced ultrasound in differential diagnosis of benign and malignant thyroid nodules [J]. Chinese Journal of Medical Ultrasound (Electronic Edition), 2023, 20(09): 895-903.
[4]	Lianhua Zhu, Xiang Fei, Peng Han, Bo Jiang, Nan Li, Yukun Luo. Value of high frame rate contrast-enhanced ultrasound in differential diagnosis of gallbladder polyp lesions [J]. Chinese Journal of Medical Ultrasound (Electronic Edition), 2023, 20(09): 904-910.
[5]	Miao Li, Lianhua Zhu, Peng Han, Bo Jiang, Xiang Fei. Evaluation of relationsip between vascular morphology and risk factors for hepatocellular carcinoma by high-frame-rate contrast-enhanced ultrasound [J]. Chinese Journal of Medical Ultrasound (Electronic Edition), 2023, 20(09): 911-915.
[6]	Weiping Zhang, Jingling Wang, Zhixing Liu, Li Chen, Fangqun Chen. Characteristics of time-intensity curve of high-frame-rate contrast-enhanced ultrasound in clear cell renal cell carcinoma [J]. Chinese Journal of Medical Ultrasound (Electronic Edition), 2023, 20(09): 916-922.
[7]	Jianmin Ding, Zhengyi Qin, Xiang Zhang, Yan Zhou, Hongyu Zhou, Yandong Wang, Xiang Jing. Contrast-enhanced ultrasound versus gadoxetic acid-enhanced magnetic resonance imaging for LI-RADS classification of focal liver lesions ≤3 cm in patients at high risk of hepatocellular carcinoma: a prospective study [J]. Chinese Journal of Medical Ultrasound (Electronic Edition), 2023, 20(09): 930-938.
[8]	Dan Han, Ting Wang, Huan Xiao, Lirong Zhu, Jingyu Chen, Yi Tang. Diagnostic value of contrast enhanced ultrasound versus contrast enhanced computed tomography in benign and malignant liver lesions in children [J]. Chinese Journal of Medical Ultrasound (Electronic Edition), 2023, 20(09): 939-944.
[9]	Meifang Zhang, Ying Tan, Qiaozhen Zhu, Xin Wen, Ying Yuan, Yue Qin, Hongbo Guo, Lingxiu Hou, Wenlan Huang, Guiyan Peng, Shengli Li. Artificial intelligence-based quality control of mid-sagittal plane ultrasound images for first trimester fetal crown-rump length [J]. Chinese Journal of Medical Ultrasound (Electronic Edition), 2023, 20(09): 945-950.
[10]	Shun Chen, Ensheng Xue, Qin Ye. Value of PDCA cycle in continuous improvement of ultrasonic critical value management system [J]. Chinese Journal of Medical Ultrasound (Electronic Edition), 2023, 20(09): 974-978.
[11]	Tingting Liu, Yanbing Lin, Shan Wang, Murong Chen, Zijian Tang, Dongling Dai, Bei Xia. Evaluation of metabolic dysfunction-associated fatty liver disease in children by ultrasound-guided attenuation parameter [J]. Chinese Journal of Medical Ultrasound (Electronic Edition), 2023, 20(08): 787-794.
[12]	Yanbo Ma, Yang Hua, Guimei Liu, Xiufeng Meng, Liping Cui. Risk factors for carotid atherosclerotic lesions in middle adults [J]. Chinese Journal of Medical Ultrasound (Electronic Edition), 2023, 20(08): 822-826.
[13]	Hongwei Yao, Pengyu Wei, Jiale Gao, Zhongtao Zhang. Continuous improvement in the standardization of laparoscopic D3 dissection in patients undergoing laparoscopic colectomy for right colon cancer [J]. Chinese Journal of Operative Procedures of General Surgery(Electronic Edition), 2024, 18(01): 1-4.
[14]	bing Feng, Qiuguo Zou, Zhenbo Liang, Yanming Lu, Yi Zeng, Shumiao Wu. Clinical study of ultrasonic signs and molecular biological indicators of senile non-specific invasive breast cancer [J]. Chinese Journal of Operative Procedures of General Surgery(Electronic Edition), 2024, 18(01): 48-51.
[15]	Wenyi Zhao, Bingzi Zou, Guanhui Cai, Yongzhi Liu, Hong Wen. Application of ultrasound strain elastography combined with MRI-DWI guided targeted puncture in diagnosis of prostate lesions [J]. Chinese Journal of Clinicians(Electronic Edition), 2023, 17(9): 988-994.

Please choose a citation manager

Content to export