浏览全部资源
扫码关注微信
纸质出版日期： 2022-03-16 ，
录用日期： 2021-11-02
移动端阅览
陈舞, 孙军梅, 李秀梅. 融合多尺度残差和注意力机制的特发性肺纤维化进展预测[J]. 中国图象图形学报, 2022,27(3):812-826.
Wu Chen, Junmei Sun, Xiumei Li. Multi-scale residual and attention mechanism fusion based prediction for the progression of idiopathic pulmonary fibrosis[J]. Journal of Image and Graphics, 2022,27(3):812-826.
目的
2
特发性肺纤维化（idiopathic pulmonary fibrosis，IPF）是一种致死率极高的肺间质疾病，呈渐进式发展且诊断手段有限，给病情的预后带来极大的困难。目前已有的肺功能衰退严重程度预测方法仍存在准确率较低的问题。
方法
2
提出了一种融合多尺度残差和注意力机制的特发性肺纤维化进展预测模型，包括计算机断层扫描（computed tomography，CT）特征提取网络和多模态特征预测网络，以此预测IPF患者不同周数的用力肺活量（forced vital capacity，FVC）。CT特征提取网络以InceptionV1为骨干网络，添加残差模块和改进的CBAM通道注意力模块（improved channel attention of convolutional block attention module，CBAM-ICA）来扩大网络的感受野，关注肺部区域的有效特征，添加与卷积层并行的空洞卷积模块，补充丢失的细节信息，并对改进的多尺度CT特征融合模块进行两次串联堆叠，提升网络获取CT特征的能力，最终获得更加有效的多尺度CT特征信息。多模态预测网络利用CT特征和临床特征进行融合形成的多模态特征对IPF患者不同周数的FVC值进行预测，提高了网络的预测性能。
结果
2
在OSIC（Open Source Imaging Consortium）提供的肺纤维化测试数据集上的实验结果表明，本文提出的特发性肺纤维化进展预测模型得到了-6.810 7的拉普拉斯对数似然得分，较其他方法具有更好的性能。
结论
2
提出的融合多尺度残差和注意力机制的特发性肺纤维化进展预测模型对预测IPF患者不同周数的FVC值具有一定的有效性，可以帮助医生更好地了解IPF患者肺功能衰退的严重程度，对IPF患者的预后具有指导意义。
Objective
2
Idiopathic pulmonary fibrosis (IPF)
which is characterized by progressive dyspnea and decreased pulmonary function
is a chronic lung disease with occult onset and unknown etiology
histological or imaging manifestations of common interstitial pneumonia. Considering the progressive development of IPF and limited diagnostic methods
it may eventually lead to complete pulmonary dysfunction
causing great difficulties to the prognosis of IPF. Currently
the existing methods for predicting the severity of pulmonary function decline still have the problem of low accuracy. Forced vital capacity (FVC) can be used as an important index to predict the decline of pulmonary function.
Method
2
By integrating multi-scale residual and attention mechanism
a prediction model for the progression of IPF was proposed
and this mechanism includes a computed tomography(CT) feature extraction network and a multimodal feature prediction network. The CT feature extraction network is responsible for CT feature extraction of pre-processed lung CT images
while the multimodal feature prediction network is responsible for extracting the multimodal features. The multimodal features fused the CT features and clinical features as input to predict the FVC of IPF patients in different weeks. The pulmonary fibrosis data set provided by the Open Source Imaging Consortium (OSIC) was used as benchmark data. First
the data were pre-processed. Unnecessary data were removed
and the lung CT image size was unified to 512×512 pixels. Feature engineering was performed
the clinical text data were normalized
and more effective data features for model training were generated. Secondly
a CT feature extraction network based on InceptionV1 as the backbone network was constructed. The residual module and the improved channel attention of convolutional block attention module (CBAM-ICA) were added to the multi-scale CT feature fusion module to expand the receptive field of the network to focus the attention to the effective features of the lung area and suppress unimportant information. A parallel dilated convolution module was added to the convolution layers to increase the receptive field
supplement the lost details and improve the network performance. The three improved multi-scale CT feature fusion modules were stacked in series twice to improve the ability of the network to obtain CT features. Finally
a multimodal prediction network composed of attention module and multi-layer perceptron was constructed. The multi-modal prediction network fused the multi-scale CT features extracted by the CT feature extraction network with some clinical features (e.g.
age
gender and smoking) to form the first multimodal feature that predicts the linear change rate of FVC in IPF patients. Then
the FVC linear change rate was fused with other clinical features as a new feature to form the second multimodal feature that predicts the FVC values of IPF patients with different expected weeks to improve the prediction performance of the network.
Result
2
By using the prediction model for the progression of IPF on the pulmonary fibrosis test data set provided by the OSIC
a Laplace log likelihood score -6.810 7 was obtained. Experimental results show that the proposed method was better than the existing methods in predicting pulmonary function decline. The ablation experiments were conducted to show that the Laplace log likelihood scores of CT feature extraction network in the prediction model for the progression of IPF were improved by adding residual module
dilated convolution module and CBAM-ICA respectively
and the prediction score of the model was the best when the above three modules were added simultaneously. Comparison experiments were conducted to further verify the effectiveness of the model. First
the results of the quantile selection experiment show that taking[0.2
0.5
0.8] as the parameter of quantile loss function can improve the prediction results. Second
the attention module comparison experiment shows that the performance of CBAM-ICA is better than those of other attention modules. Third
the comparison results among different attention mechanism locations show that adding attention mechanism to the multi-scale CT feature fusion module A and C simultaneously can provide the best performance. Fourth
the comparison experiments on CT feature extraction network show that the Laplace log likelihood score of using InceptionV1 as the CT feature extraction network's backbone network is better than those of using most other networks. Finally
the prediction performance comparisons on different modal data show that multimodal data can effectively improve the accuracy of model prediction compared with monomodal data. These experiments further verify the accuracy and effectiveness of the proposed prediction model for the progression of IPF.
Conclusion
2
By integrating multi-scale residual and attention mechanism
the proposed prediction model for the progression of IPF is effective in predicting FVC values of IPF patients with different weeks. The model can help doctors better understand the severity of pulmonary function decline in IPF patients
and it can provide guidance for the prognosis of patients with.
特发性肺纤维化(IPF)预测模型多尺度残差用力肺活量(FVC)注意力机制
idiopathic pulmonary fibrosis(IPF)prediction modelmulti-scale residualforced vital capacity(FVC)attention mechanism
Glotov A and Lyakhov P. 2021. Pulmonary fibrosis progression prognosis using machine learning//2021 Ural Symposium on Biomedical Engineering, Radioelectronics and Information Technology. Yekaterinburg, Russia: IEEE: 327-329[DOI: 10.1109/USBEREIT51232.2021.9455070http://dx.doi.org/10.1109/USBEREIT51232.2021.9455070]
He K M, Zhang X Y, Ren S Q and Sun J. 2016. Deep residual learning for image recognition//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, USA: 770-778[DOI: 10.1109/CVPR.2016.90http://dx.doi.org/10.1109/CVPR.2016.90]
Hu J, Shen L and Sun G. 2018. Squeeze-and-excitation networks//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, USA: IEEE: 7132-7141[DOI: 10.1109/CVPR.2018.00745http://dx.doi.org/10.1109/CVPR.2018.00745]
Huang G, Liu Z, Van Der Maaten L and Weinberger K Q. 2017. Densely connected convolutional networks//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, USA: IEEE: 2261-2269[DOI: 10.1109/CVPR.2017.243http://dx.doi.org/10.1109/CVPR.2017.243]
Ioffe S and Szegedy C. 2015. Batch normalization: accelerating deep network training by reducing internal covariate shift[EB/OL]. [2021-07-30].https://arxiv.org/pdf/1502.03167.pdfhttps://arxiv.org/pdf/1502.03167.pdf
Kaggle. 2020. OSIC pulmonary fibrosis progression[EB/OL]. [2021-07-10].https://www.kaggle.com/c/osic-pulmonary-fibrosis-progressionhttps://www.kaggle.com/c/osic-pulmonary-fibrosis-progression
Maher T M, Bendstrup E, Dron L, Langley J, Smith G, Khalid J M, Patel H and Kreuter M. 2021. Global incidence and prevalence of idiopathic pulmonary fibrosis. Respiratory Research, 22(1): #197[DOI:10.1186/s12931-021-01791-z]
Mandal S, Balas V E, Shaw R N and Ghosh A. 2020. Prediction analysis of idiopathic pulmonary fibrosis progression from OSIC dataset//Proceedings of 2020 IEEE International Conference on Computing, Power and Communication Technologies. Greater Noida, India: IEEE: 861-865[DOI: 10.1109/GUCON48875.2020.9231239http://dx.doi.org/10.1109/GUCON48875.2020.9231239]
Nazi Z A, Mashrur F R, Islam M A and Saha S. 2021. Fibro-CoSANet: pulmonary fibrosis prognosis prediction using a convolutional self attention network[EB/OL]. [2021-07-30].https://arxiv.org/pdf/2104.05889.pdfhttps://arxiv.org/pdf/2104.05889.pdf
Raghu G, Remy-Jardin M, Myers J L, Richeldi L, Ryerson C J, Lederer D J, Behr J, Cottin V, Danoff S K, Morell F, Flaherty K R, Wells A, Martinez F J, Azuma A, Bice T J, Bouros D, Brown K K, Collard H R, Duggal A, Galvin L, Inoue Y, Jenkins R G, Johkoh T, Kazerooni E A, Kitaichi M, Knight S L, Mansour G, Nicholson A G, Pipavath S N J, Buendía-Roldán I, Selman M, Travis W D, Walsh S L F and Wilson K C. 2018. Diagnosis of idiopathic pulmonary fibrosis. An official ATS/ERS/JRS/ALAT clinical practice guideline. American Journal of Respiratory and Critical Care Medicine, 198(5): e44-e68[DOI:10.1164/rccm.201807-1255ST]
Roy A G, Navab N and Wachinger C. 2018. Concurrent spatial and channel 'squeeze and excitation' in fully convolutional networks//Proceedings of the 21st International Conference on Medical Image Computing and Computer-Assisted Intervention. Granada, Spain: Springer: 421-429[DOI: 10.1007/978-3-030-00928-1_48http://dx.doi.org/10.1007/978-3-030-00928-1_48]
Shen Q X and Peng H. 2021. Clinical evaluation on idiopathic pulmonary fibrosis. Journal of Central South University (Medical Science), 46(3): 309-315
沈沁雪, 彭红. 2021. 特发性肺纤维化病情的临床评估. 中南大学学报(医学版), 46(3): 309-315[DOI:10.11817/j.issn.1672-7347.2021.190402]
Song H F, Sun D J, Ban C J, Liu Y, Zhu M, Ye Q, Yan W, Ren Y H and Dai H P. 2019. Independent clinical factors relevant to prognosis of patients with idiopathic pulmonary fibrosis. Medical Science Monitor, 25: 4193-4201[DOI:10.12659/MSM.914725]
Szegedy C, Ioffe S, Vanhoucke V and Alemi A A. 2017. Inception-v4, inception-ResNet and the impact of residual connections on learning//Proceedings of the 31st AAAI Conference on Artificial Intelligence. San Francisco, USA: AAAI: 4278-4284[DOI: 10.5555/3298023.3298188http://dx.doi.org/10.5555/3298023.3298188]
Szegedy C, Liu W, Jia Y Q, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V and Rabinovich A. 2015. Going deeper with convolutions//Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition. Boston, USA: IEEE: 1-9[DOI: 10.1109/CVPR.2015.7298594http://dx.doi.org/10.1109/CVPR.2015.7298594]
Szegedy C, Vanhoucke V, Ioffe S, Shlens J and Wojna Z. 2016. Rethinking the inception architecture for computer vision//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, USA: IEEE: 2818-2826[DOI: 10.1109/CVPR.2016.308http://dx.doi.org/10.1109/CVPR.2016.308]
Tan M X and Le Q V. 2019. EfficientNet: rethinking model scaling for convolutional neural networks[EB/OL]. [2021-07-30].https://arxiv.org/pdf/1905.11946.pdfhttps://arxiv.org/pdf/1905.11946.pdf
Wang Q L, Wu B G, Zhu P F, Li P H, Zuo W M and Hu Q H. 2020. ECA-Net: efficient channel attention for deep convolutional neural networks//Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle, USA: IEEE: 11531-11539[DOI: 10.1109/CVPR42600.2020.01155http://dx.doi.org/10.1109/CVPR42600.2020.01155]
Wong A, Lu J, Dorfman A, McInnis P, Famouri M, Manary D, Lee J R H and Lynch M. 2021. Fibrosis-Net: a tailored deep convolutional neural network design for prediction of pulmonary fibrosis progression from chest CT images[EB/OL]. [2021-07-30].https://arxiv.org/pdf/2103.04008.pdfhttps://arxiv.org/pdf/2103.04008.pdf
Woo S, Park J, Lee J Y and Kweon I S. 2018. CBAM: convolutional block attention module//Proceedings of the 15th European Conference on Computer Vision. Munich, Germany: Springer: 3-19[DOI: 10.1007/978-3-030-01234-2_1http://dx.doi.org/10.1007/978-3-030-01234-2_1]
Yu F and Koltun V. 2016. Multi-scale context aggregation by dilated convolutions[EB/OL]. [2021-07-30].https://arxiv.org/pdf/1511.07122.pdfhttps://arxiv.org/pdf/1511.07122.pdf
相关作者
相关机构
京公网安备11010802024621