浏览全部资源
扫码关注微信
网络出版日期： 2024-10-16 ，
移动端阅览
刘英莉,茶寅秋,黄一山等.嵌入切片序列关联模式的前交叉韧带损伤分类算法[J].中国图象图形学报,
Liu Yingli,Cha Yinqiu,Huang Yishan,et al.Classification algorithm for anterior cruciate ligament injury embedded with slice sequence association mode[J].Journal of Image and Graphics,
目的
2
膝关节前交叉韧带（anterior cruciate ligament，ACL）损伤的及时诊断是降低骨关节炎等风险的方式。作为ACL损伤判别的常用影像学方法，三维磁共振成像（magnetic resonance imaging，MRI）相较于二维自然图像细节更繁杂，医生诊断所需时间长，智能辅助医疗是有益的。目前应用三维卷积神经网络（3D convolutional neural networks，3D CNNs）的ACL损伤分类算法存在计算成本大且数据使用不充分的问题，而基于二维卷积神经网络（2D convolutional neural networks，2D CNNs）的方法则忽略了第三维度的相关性和ACL的形态多样性。针对上述问题，提出一种嵌入切片序列关联模式的网络（slice sequence association mode network，SSAMNet）。
方法
2
SSAMNet通过设计切片序列信息融合（slice sequence information fusion，SFS）模块从MRI数据的邻近、全量切片中并行学习序列性质，建立切片特征映射中的关联模式，整合切片信息。借助多级尺度特征自适应关注（multi-level scale feature adaptive attention，MSFAA）模块，对不同关联尺度权重再分配，以适应ACL区域形状和位置表征多变的特性。
结果
2
在MRNet数据集的实验中，模型的AUC（area under curve）值达到98.4%，相较于其它的ACL损伤分类算法性能最优，准确性及特异性指标也分别达到了91.4%和97%的最优值。在kneeMRI数据集上进行五折交叉验证实验后，SSAMNet的AUC平均值最高，ROC（receiver operating characteristic curve）曲线始终保持着平稳的趋势。可视化结果也表明提出的方法可以有效关注病变区域。
结论
2
针对ACL损伤分类任务，所提模型性能优异且具有鲁棒性，存在临床应用价值。代码开源地址：
https：//github.com/wabk/SSAMNet
https://github.com/wabk/SSAMNet
。
Objective
2
The prompt diagnosis of anterior cruciate ligament （ACL）injuries in the knee has been shown to reduce the risk of osteoarthritis， knee injuries， and more. As a common imaging method for identifying ACL injuries， magnetic resonance imaging（MRI） has been characterized by low acquisition time cost and non-invasiveness. MRI can be regarded as a three-dimensional data. Compared with two-dimensional images， it has more details. Doctors need to combine multiple images within a dataset to make a comprehensive judgment and draw a conclusion， The diagnosis takes a long time， so intelligent assisted medical treatment is necessary. Currently， deep learning-based methods are mainly used for ACL injury classification， which can be divided into algorithms basedon 3D convolutional neural networks（3D CNNs） and 2D convolutional neural networks（2D CNNs）. ACL injury classification algorithms using 3D CNNs often suffer from high computational costs and insufficient data usage， while algorithms based on 2D CNNs ignore the third-dimensional correlation （correlation of slice dimensions） and the morphological diversity of ACL. To solve the problems， an ACL injury classification algorithm embedded with slice sequence association mode is proposed（SSAMNet）.
Method
2
SSAMNet uses the classic AlexNet model based on 2D CNNs as the backbone network to acquire discriminative features for ACL injury classification. By designing the slice sequence information fusion （SFS） module， sequence properties are learned in parallel from adjacent and full slices of MRI data. Firstly， the initial slice features extracted through the backbone network are divided into slice groups according to the channel dimension， and a shift operation is performed between the groups to achieve the purpose of merging adjacent slice features. Then， key information is extracted with the help of the initial slice features， the slice sequence relationship is modeled by SSAMNet， global slice information is merged and processed， and finally， the association pattern in the slice feature mapping is established. Secondly， the multi-level scale feature adaptive attention （MSFAA） module is used to splice the multiple levels features processed by the backbone network into multi-scale feature groups. The horizontal and vertical direction features are processed through mean expansion to obtain the directional scale weight coefficients， the goal of redistributing weights at different correlation scales is accomplished to accommodate the variable nature of ACL region shape and location representations. There exists a common problem of class imbalance in the knee MRI datasets selected for this task， where the number of positive samples representing ACL tears exceeds the number of negative samples representing intact ACLs. This problem is not only reflected in the selected datasets but also exists in the real world， where the number of individuals with torn ACLs surpasses the number of those with intact ACLs. Therefore， we choose to minimize the weighted binary cross-entropy loss to reduce the impact of class imbalance on algorithm performance. During the training process， the learning rate is initialized to 1e-5， and it is updated to 0.95 times the original value every 10 epochs of training. All experiments are performed on PyTorch using an NVIDIA RTX 3090 GPU. To ensure the fairness of comparative experiments， we report the mean and standard of three experimental results.
Result
2
We use accuracy， sensitivity， specificity， and area under the curve （AUC） as evaluation metrics for this algorithm， calculated using data such as true positives， true negatives， false positives， and false negatives. The experiments are conducted on the MRNet dataset and the kneeMRI dataset， and their data distributions are different. The MRNet dataset is currently the largest public dataset on the knee joint， including 1370 sets of MRI data， and the kneeMRI dataset contains 917 sets of data. Because the kneeMRI dataset is not divided into training sets and test sets， we use stratified random sampling to separate kneeMRI， and we use the five-fold cross-validation method to complete the external validation experiment. The final experimental results on the MRNet dataset show that the AUC value of SSAMNet reaches 98.4%， which has the best performance compared with other comparison networks based on 2D CNNs and 3D CNNs， with a specificity value of 97% and an accuracy value of 91.4%. Moreover， the ablation experimental results on this dataset also prove that SFS and MSFAA have a gain value for the ACL injury discrimination task， especially the significant performance improvement brought by adding the SFS module （AUC increased by more than 2.5%）， which also illustrates the effectiveness of the embedded slice sequence correlation mode in improving model performance. The joint use of SFS and MSFAA also increases AUC by more than 4%. Then， a five-fold cross-validation experiment is conducted on the kneeMRI dataset. The AUC value of the proposed model reaches 88.7%， and the accuracy value exceeds other models by 3%. The ROC curve tends to be stable on the five-fold datasets of different divisions， proving that the model is stable and generalizable. Finally， the visualization results also show that SSAMNet can effectively focus on the ACL area and the indicators that affect the identification of ACL injuries， proving that the model's conclusion of identifying tear injuries is consistent with the actual basis， further proving that the model can effectively identify ACL injuries.
Conclusion
2
The proposed SSAMNet has excellent performance in ACL injury discrimination， and its robustness is confirmed in external validation experiments with different data distributions， further revealing the potential clinical application value of the proposed model. However， the current dataset also has the issue of single-label data. Considering the comprehensive data required for clinical usage， we will utilize patients' past medical records and other information as prior knowledge to construct an auxiliary diagnostic framework for knee joint diseases in the subsequent steps. The code of this paper will be open source soon：
https：//github.com/wabk/SSAMNet
https://github.com/wabk/SSAMNet
.
磁共振成像（MRI）3D图像分类切片特征聚合自适应尺度注意力2D CNNs
MRI3D image classificationslice feature aggregationadaptive scale attention2D CNNs
Belton N， Welaratne I， Dahlan A， Hearne R T， Hagos M T， Lawlor A and Curran K M. 2021. Optimising Knee Injury Detection with Spatial Attention and Validating Localisation Ability// Proceedings of Medical Image Understanding and Analysis. Oxford， UK： Springer International Publishing： 71-86 ［DOI： 10.1007/978-3-030-80432-9_6http://dx.doi.org/10.1007/978-3-030-80432-9_6］
Bien N， Rajpurkar P， Ball R L， Irvin J， Park A， Jones E， Bereket M， Patel B N， Yeom K W and Shpanskaya K. 2018. Deep-learning-assisted diagnosis for knee magnetic resonance imaging： development and retrospective validation of MRNet. Plos Medicine， 15（11）： e1002699 ［DOI： 10.1371/journal.pmed.1002699http://dx.doi.org/10.1371/journal.pmed.1002699］
Bradley A P. 1997. The use of the area under the ROC curve in the evaluation of machine learning algorithms. Pattern recognition， 30（7）： 1145-1159 ［DOI： 10.1016/S0031-3203（96）00142-2http://dx.doi.org/10.1016/S0031-3203（96）00142-2］
Crawford R， Walley G， Bridgman S and Maffulli N. 2007. Magnetic resonance imaging versus arthroscopy in the diagnosis of knee pathology， concentrating on meniscal lesions and ACL tears： a systematic review. British Medical Bulletin， 84（1）： 5-23 ［DOI： 10.1093/bmb/ldm022http://dx.doi.org/10.1093/bmb/ldm022］
Dunnhofer M， Martinel N and Micheloni C. 2022. Deep convolutional feature details for better knee disorder diagnoses in magnetic resonance images. Computerized Medical Imaging and Graphics， 102： 102142 ［DOI： 10.1016/j.compmedimag.2022.1021http://dx.doi.org/10.1016/j.compmedimag.2022.1021］
He K， Zhang X， Ren S and Sun J. 2016. Deep residual learning for image recognition// Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition （CVPR）. Las Vegas， USA： IEEE： 770-778 ［DOI： 10.1109/cvpr.2016.90http://dx.doi.org/10.1109/cvpr.2016.90］
Hou Q， Zhou D and Feng J. 2021. Coordinate attention for efficient mobile network design// Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. Nashville， USA： IEEE： 13713-13722 ［DOI： 10.1109/cvpr46437.2021.01350http://dx.doi.org/10.1109/cvpr46437.2021.01350］
Hu J， Shen L and Sun G. 2018. Squeeze-and-excitation networks// Proceedings of 2018 IEEE Conference on Computer Vision and Pattern Recognition （CVPR）. Salt Lake City， USA： IEEE： 7132-7141 ［DOI： 10.1109/CVPR.2018.00745http://dx.doi.org/10.1109/CVPR.2018.00745］
Huang G， Liu Z， Van Der Maaten L and Weinberger K Q. 2017. Densely connected convolutional networks// Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition （CVPR）. Honolulu， USA： IEEE： 4700-4708 ［DOI： 10.1109/cvpr.2017.243http://dx.doi.org/10.1109/cvpr.2017.243］
Huang S H and Wang J F.2022.Brain MRI image classification incorporating smooth group sparseness. Journal of Image and Graphics， 27（03）：0885-0897
黄帅辉，王金凤.2022. 融入平滑组稀疏化的脑部 MRI 图像分类.中国图象图形学报，27（03）：0885-0897［DOI：10.11834/ jig. 210367http://dx.doi.org/10.11834/jig.210367］
Irmakci I， Anwar S M， Torigian D A and Bagci U. 2019. Deep Learning for Musculoskeletal Image Analysis// Proceedings of 2019 53rd Asilomar Conference on Signals， Systems， and Computers（ACSSC）. Pacific Grove， USA： IEEE： 1481-1485 ［DOI： 10.1109/IEEECONF44664.2019.9048671http://dx.doi.org/10.1109/IEEECONF44664.2019.9048671］
Jeon Y S， Yoshino K， Hagiwara S， Watanabe A， Quek S T， Yoshioka H and Feng M. 2021. Interpretable and lightweight 3-D deep learning model for automated ACL diagnosis. IEEE Journal of Biomedical and Health Informatics， 25（7）： 2388-2397 ［DOI： 10.1109/JBHI.2021.3081355http://dx.doi.org/10.1109/JBHI.2021.3081355］
Kingma D P and Ba J. 2014. Adam： A method for stochastic optimization［EB/OL］.［2024-02-11］. http://arxiv.org/pdf/1412.6980.pdfhttp://arxiv.org/pdf/1412.6980.pdf
Krizhevsky A， Sutskever I and Hinton G E. 2017. ImageNet classification with deep convolutional neural networks. Communications of the ACM， 60（6）： 84-90 ［DOI： 10.1145/3065386http://dx.doi.org/10.1145/3065386］
Li F， Zhai P， Yang C， Feng G， Yang J and Yuan Y. 2023. Automated diagnosis of anterior cruciate ligament via a weighted multi-view network. Frontiers in Bioengineering and Biotechnology， 11： 1268543 ［DOI： 10.3389/fbioe.2023.1268543http://dx.doi.org/10.3389/fbioe.2023.1268543］
Liu Z， Wang L， Wu W， Qian C and Lu T. 2021. Tam： Temporal adaptive module for video recognition// Proceedings of 2021 IEEE/CVF International Conference on Computer Vision （ICCV）. Montréal， Canada： IEEE： 13708-13718 ［DOI： 10.1109/iccv48922.2021.01345http://dx.doi.org/10.1109/iccv48922.2021.01345］
Mackenzie R， Palmer C， Lomas D and Dixon A. 1996. Magnetic resonance imaging of the knee： diagnostic performance statistics. Clinical radiology， 51（4）： 251-257 ［DOI： 10.1016/S0009-9260（96）80341-2http://dx.doi.org/10.1016/S0009-9260（96）80341-2］
Oei E H， Nikken J J， Verstijnen A C， Ginai A Z and Myriam Hunink M. 2003. MR imaging of the menisci and cruciate ligaments： a systematic review. Radiology， 226（3）： 837-848 ［DOI： 10.1148/radiol.2263011892http://dx.doi.org/10.1148/radiol.2263011892］
Shea K G and Carey J L. 2015. Management of Anterior Cruciate Ligament Injuries： Evidence-Based Guideline. Journal of the American Academy of Orthopaedic Surgeons， 23（5）： e1-e5 ［DOI： 10.5435/jaaos-d-15-00094http://dx.doi.org/10.5435/jaaos-d-15-00094］
Simonyan K and Zisserman A. 2014. Very deep convolutional networks for large-scale image recognition［EB/OL］.［2024-02-16］. http://arxiv.org/pdf/1409.1556.pdfhttp://arxiv.org/pdf/1409.1556.pdf
Štajduhar I， Mamula M， Miletić D and Uenal G. 2017. Semi-automated detection of anterior cruciate ligament injury from MRI. Computer methods and programs in biomedicine， 140： 151-164 ［DOI： 10.1016/j.cmpb.2016.12.006http://dx.doi.org/10.1016/j.cmpb.2016.12.006］
Szegedy C， Wei L， Yangqing J， Sermanet P， Reed S， Anguelov D， Erhan D， Vanhoucke V and Rabinovich A. 2015. Going deeper with convolutions// Proceedings of 2015 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. Boston， USA： IEEE： 1-9 ［DOI： 10.1109/CVPR.2015.7298594http://dx.doi.org/10.1109/CVPR.2015.7298594］
Tsai C-H， Kiryati N， Konen E， Eshed I and Mayer A. 2020. Knee injury detection using MRI with efficiently-layered network （ELNet）// Proceedings of the Third Conference on Medical Imaging with Deep Learning. Montréal， Canada： PMLR： 784-794 ［DOI： 10.48550/arXiv.2005.02706http://dx.doi.org/10.48550/arXiv.2005.02706］
Yu B and Garrett W E. 2007. Mechanisms of non-contact ACL injuries. British Journal of Sports Medicine， 41（suppl 1）： i47-i51 ［DOI： 10.1136/bjsm.2007.037192http://dx.doi.org/10.1136/bjsm.2007.037192］
Zhang L， Li M， Zhou Y， Lu G and Zhou Q. 2020. Deep learning approach for anterior cruciate ligament lesion detection： evaluation of diagnostic performance using arthroscopy as the reference standard. Journal of Magnetic Resonance Imaging， 52（6）： 1745-1752 ［DOI： 10.1002/jmri.27266http://dx.doi.org/10.1002/jmri.27266］
Zhao X M， Liao Y H， Zhang S Q， Fang J X， He X X， Wang G Y and Lu H S. 2024. Method of classifying benign and malignant
breast tumors by DCE-MRI incorporating local and global features. Journal of Image and Graphics， 29（01）：0256-0267
赵小明， 廖越辉， 张石清， 方江雄， 何遐遐， 汪国余， 卢洪胜. 2024. 融合局部与全局特征的DCE-MRI乳腺肿瘤良恶分类. 中国图象图形学报，29（01）：0256-0267［DOI：10. 11834/jig. 230092http://dx.doi.org/10.11834/jig.230092］
Zhou B， Khosla A， Lapedriza A， Oliva A and Torralba A. 2016. Learning deep features for discriminative localization// Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition （CVPR）. Las Vegas， USA： IEEE： 2921-2929 ［DOI： 10.1109/cvpr.2016.319http://dx.doi.org/10.1109/cvpr.2016.319］
相关文章
相关作者
相关机构
京公网安备11010802024621