浏览全部资源
扫码关注微信
纸质出版日期： 2022-03-16 ，
录用日期： 2021-11-05
移动端阅览
夏峰, 邵海见, 邓星. 融合跨阶段深度学习的脑肿瘤MRI图像分割[J]. 中国图象图形学报, 2022,27(3):873-884.
Feng Xia, Haijian Shao, Xing Deng. Cross-stage deep-learning-based MRI fused images of human brain tumor segmentation[J]. Journal of Image and Graphics, 2022,27(3):873-884.
目的
2
磁共振成像(magnetic resonance imaging，MRI)作为一种非侵入性的软组织对比成像方式，可以提供有关脑肿瘤的形状、大小和位置等有价值的信息，是用于脑肿瘤患者检查的主要方法，在脑肿瘤分割任务中发挥着重要作用。由于脑肿瘤本身复杂多变的形态、模糊的边界、低对比度以及样本梯度复杂等问题，导致高精度脑肿瘤MRI图像分割非常具有挑战性，目前主要依靠专业医师手动分割，费时且可重复性差。对此，本文提出一种基于U-Net的改进模型，即CSPU-Net(cross stage partial U-Net)脑肿瘤分割网络，以实现高精度的脑肿瘤MRI图像分割。
方法
2
CSPU-Net在U-Net结构的上下采样中分别加入两种跨阶段局部网络结构(cross stage partial module，${\rm{CSP}}$)提取图像特征，结合${\rm{GDL}}$(general Dice loss)和${\rm{WCE}}$(weighted cross entropy)两种损失函数解决训练样本类别不平衡问题。
结果
2
在BraTS (brain tumor segmentation) 2018和BraTS 2019两个数据集上进行实验，在BraTS 2018数据集中的整体肿瘤分割精度、核心肿瘤分割精度和增强肿瘤分割精度分别为87.9%、80.6%和77.3%，相比于传统U-Net的改进模型(ResU-Net)分别提升了0.80%、1.60%和2.20%。在BraTS 2019数据集中的整体肿瘤分割精度、核心肿瘤分割精度和增强肿瘤分割精度分别为87.8%、77.9%和70.7%，相比于ResU-Net模型提升了0.70%、1.30%和1.40%。
结论
2
本文提出的跨阶段局部网络结构，通过增加梯度路径、减少信息损失，可以有效提高脑肿瘤分割精度，实验结果证明了该模块对脑肿瘤分割任务的有效性。
Objective
2
Human brain tumors are a group of mutant cells in the brain or skull. These benign or malignant brain tumors can be classified based on their growth characteristics and influence on the human body. Gliomas are one of the most frequent forms of malignant brain tumors
accounting for approximately 40% to 50% of all brain tumors. Glioma is classified as high-grade glioma (HGG) or low-grade glioma (LGG) depending on the degree of invasion. Low-grade glioma (LGG) is a well-differentiated glioma with a prompt prognosis. High-grade glioma (HGG) is a poorly differentiated glioma with a in qualified prognosis. Gliomas with varying degrees of differentiation are appeared following the varied degrees of peritumoral edema
edema types
and necrosis. the boundary of gliomas and normal tissues is often blurred. It is difficult to identify the scope of lesions and surgical area
which has a significant impact on surgical quality and patient prognosis. As a non-invasive and clear soft tissue contrast imaging tool
magnetic resonance imaging (MRI) can provide vital information on the shape
size
and location of brain tumors. High-precision brain tumor MRI image segmentation is challenged due to the complicated and variable morphology
fuzzy borders
low contrast
and complicated sample gradients of brain tumors. Manual segmentation is time-consuming and inconsistent. The International Association for Medical Image Computing and Computer-Aided Intervention (MICCAI)'s Brain Tumor Segmentation (BraTS) is a global medical image segmentation challenge concentrating on the evaluation of automatic segmentation methods for human brain tumors. There are four types of automatic brain tumor segmentation algorithms as mentioned below: supervised learning
semi-supervised learning
unsupervised learning
and hybrid learning. Supervised-learning-based algorithm is currently the effective method. Various depth neural network models for computer vision problems
such as Visual Geometry Group Network (VGGNet)
GoogLeNet
ResNet
and DenseNet
have been presented in recent years. The above deep neural network model proposes a novel approach to the problem of MRI brain image segmentation
and it significantly advances the development of deep learning-based brain tumor diagnosis methods. As a result
deep learning method is to develop the task of automatic segmentation of brain tumor MRI images.
Method
2
Our research integrates low resolution information and high resolution information via the U-Net structure of the all convolutional neural network. An improved cross stage partial U-Net(CSPU-Net) brain tumor segmentation network derived from the U-Net network achieves high-precision brain tumor MRI image segmentation. The basic goal of the cross stage partial (CSP) module is to segment the grass-roots feature mapping into two sections as following: 1)Dividing the gradient flow to extend distinct network paths
and then 2) fusing the two portions based on horizontal hierarchy. The conveyed gradient information can have huge correlation discrepancies by alternating series and transition procedures. To extract image features
CSPU-Net adds two types of cross stage partial network structures to the up and down sampling of the U-Net network. The number of gradient routes is increased using the splitting and merging technique. The drawbacks of utilizing explicit feature map replication for connection are mitigated
enhancing the model's feature learning capabilities. To overcome the imbalance issue of sample class
two loss functions
general dice loss and weighted cross-entropy are combined. The cross stage partial structure is final compared to ResU-Net
which adds a residual block
to in identify the effectiveness of cross stage partial structure as ResU-Net in the brain tumor segmentation test.
Result
2
Our experimental results of the CSPU-Net model in the context of the BraTS 2018 and BraTS 2019 datasets has its priority. The BraTS 2018 dataset yielded 87.9% accuracy in whole tumor segmentation
80.6% accuracy in core tumor segmentation
and 77.3% accuracy in enhanced tumor segmentation
respectively. This method enhances the segmentation accuracy of brain tumor MRI images by 0.80%
1.60%
and 2.20% each. In the BraTS 2019 dataset
the whole tumor segmentation accuracy is 87.8%
the core tumor segmentation accuracy is 77.9%
and the improved tumor segmentation accuracy is 70.7%
respectively. This method enhances the segmentation accuracy of brain tumor MRI images by 0.70%
1.30%
and 1.40%
respectively in comparison of the traditional improved ResU-Net.
Conclusion
2
This research provides a cross-stage deep learning-based 2D segmentation network for human brain tumor MRI images. Using cross stage partial network structure in U-Net up and down sampling
the model enhances the accuracy of brain tumor segmentation via gradient path expansion and information loss deduction. The demonstrated results illustrate that our model has its potentials on BraTS datasets on 2D segmentation models development and demonstrates the module's efficiency in the brain tumor segmentation task.
脑肿瘤分割深度学习U-Net跨阶段局部网络结构残差模块
brain tumor segmentationdeep learningU-Netcross stage partial network structureresidual module
Alom M Z, Yakopcic C, Hasan M, Taha T M and Asari V K. 2019. Recurrent residual U-Net for medical image segmentation. Journal of Medical Imaging, 6(1): #014006[DOI:10.1117/1.JMI.6.1.014006]
Amian M and Soltaninejad M. 2020. Multi-resolution 3D CNN for MRI brain tumor segmentation and survival prediction//Proceedings of the 5th International MICCAI Brainlesion Workshop. Shenzhen, China: Springer: 221-230[DOI: 10.1007/978-3-030-46640-4_21http://dx.doi.org/10.1007/978-3-030-46640-4_21]
Baid U, Shah N A and Talbar S. 2020. Brain tumor segmentation with cascaded deep convolutional neural network//Proceedings of the 5th International MICCAI Brainlesion Workshop. Shenzhen, China: Springer: 90-98[DOI: 10.1007/978-3-030-46643-5_9http://dx.doi.org/10.1007/978-3-030-46643-5_9]
Bhalerao M and Thakur S. 2020. Brain tumor segmentation based on 3D residual U-Net//Proceedings of the 5th International MICCAI Brainlesion Workshop. Shenzhen, China: Springer: 218-225[DOI: 10.1007/978-3-030-46643-5_21http://dx.doi.org/10.1007/978-3-030-46643-5_21]
Bochkovskiy A, Wang C Y and Liao H Y M. 2020. YOLOv4: optimal speed and accuracy of object detection[EB/OL]. [2021-05-03].https://arxiv.org/pdf/2004.10934.pdfhttps://arxiv.org/pdf/2004.10934.pdf
Chen W, Liu B Q, Peng S T, Sun J W and Qiao X. 2019. S3D-UNet: separable 3DU-Net for brain tumor segmentation//Proceedings of the 4th International MICCAI Brainlesion Workshop. Granada, Spain: Springer: 358-368[DOI: 10.1007/978-3-030-11726-9_32http://dx.doi.org/10.1007/978-3-030-11726-9_32]
Di Ieva A, Russo C, Liu S D, Jian A N, Bai M Y, Qian Y and Magnussen J S. 2021. Application of deep learning for automatic segmentation of brain tumors on magnetic resonance imaging: a heuristic approach in the clinical scenario. Neuroradiology, 63(8): 1253-1262[DOI: 10.1007/s00234-021-02649-3]
Ding Y, Gong L P, Zhang M F, Li C and Qin Z G. 2020. A multi-path adaptive fusion network for multimodal brain tumor segmentation. Neurocomputing, 412: 19-30[DOI: 10.1016/j.neucom.2020.06.078]
Dong H, Yang G, Liu F D, Mo Y H and Guo Y K. 2017. Automatic brain tumor detection and segmentation using U-Net based fully convolutional networks//Proceedings of the 21st Annual Conference on Medical Image Understanding and Analysis. Edinburgh, UK: Springer: 506-517[DOI: 10.1007/978-3-319-60964-5_44http://dx.doi.org/10.1007/978-3-319-60964-5_44]
Feng B W. 2020. Research on multi-scale feature segmentation of Brain Tumors MRI based on parallel CNN
冯博文. 2020. 脑肿瘤MRI的并行CNN多尺度特征分割技术研究. 内蒙古: 内蒙古科技大学[DOI:10.27724/d.cnki.gnmgk.2020.000421http://dx.doi.org/10.27724/d.cnki.gnmgk.2020.000421]
Gondal A H and Khan M N A. 2013. A review of fully automated techniques for brain tumor detection from MR images. International Journal of Modern Education and Computer Science, 5(2): 55-61[DOI: 10.5815/ijmecs.2013.02.08]
Gordillo N, Montseny E and Sobrevilla P. 2013. State of the art survey on MRI brain tumor segmentation. Magnetic Resonance Imaging, 31(8): 1426-1438[DOI: 10.1016/j.mri.2013.05.002]
Guo X T, Yang C S, Ma T, Zhou P Z, Lu S F, Ji N, Li D L, Wang T and Lyu H Y. 2020. Brain tumor segmentation based on attention mechanism and multi-model fusion//Proceedings of the 5th International MICCAI Brainlesion Workshop. Shenzhen, China: Springer: 50-60[DOI: 10.1007/978-3-030-46643-5_5http://dx.doi.org/10.1007/978-3-030-46643-5_5]
Haris M, Gupta R K, Singh A, Husain N, Husain M, Pandey C M, Srivastava C, Behari S and Rathore R K S. 2008. Differentiation of infective from neoplastic brain lesions by dynamic contrast-enhanced MRI. Neuroradiology, 50(6): 531-540[DOI: 10.1007/s00234-008-0378-6]
Havaei M, Davy A, Warde-Farley D, Biard A, Courville A, Bengio Y, Pal C, Jodoin P M and Larochelle H. 2017. Brain tumor segmentation with deep neural networks. Medical Image Analysis, 35: 18-31[DOI: 10.1016/j.media.2016.05.004]
He K M, Zhang X Y, Ren S Q and Sun J. 2016. Deep residual learning for image recognition//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas, USA: IEEE: 770-778[DOI: 10.1109/CVPR.2016.90http://dx.doi.org/10.1109/CVPR.2016.90]
Hu K, Gan Q H, Zhang Y, Deng S H, Xiao F, Huang W, Cao C H and Gao X P. 2019. Brain tumor segmentation using multi-cascaded convolutional neural networks and conditional random field. IEEE Access, 7: 92615-92629[DOI:10.1109/ACCESS.2019.2927433]
Huang G, Liu Z, Van Der Maaten Land Weinberger K Q. 2017. Densely connected convolutional networks//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, USA: IEEE: 2261-2269[DOI: 10.1109/CVPR.2017.243http://dx.doi.org/10.1109/CVPR.2017.243]
Isensee F, Kickingereder P, Wick W, Bendszus M and Maier-Hein K H. 2019. No New-Net//Proceedings of the 4th International MICCAI Brainlesion Workshop. Granada, Spain: Springer: 234-244[DOI: 10.1007/978-3-030-11726-9_21http://dx.doi.org/10.1007/978-3-030-11726-9_21]
Işın A, Direkoǧlu C and Şah M. 2016. Review of MRI-based brain tumor image segmentation using deep learning methods. Procedia Computer Science, 102: 317-324[DOI: 10.1016/j.procs.2016.09.407]
Jiang Z Y, Ding C X, Liu M F and Tao D C. 2020. Two-stage cascaded U-Net: 1st place solution to brats challenge 2019 segmentation task//Proceedings of the 5th International MICCAI Brainlesion Workshop. Shenzhen, China: Springer: 231-241[DOI: 10.1007/978-3-030-46640-4_22http://dx.doi.org/10.1007/978-3-030-46640-4_22]
Kong X M, Sun G X, Wu Q, Liu J and Lin F M. 2018. Hybrid pyramid U-Net model for brain tumor segmentation//Proceedings of the 10th International Conference on Intelligent Information Processing. Nanning, China: Springer: 346-355[DOI: 10.1007/978-3-030-00828-4_35http://dx.doi.org/10.1007/978-3-030-00828-4_35]
Li X M, Chen H, Qi X J, Dou Q, Fu C W and Heng P A. 2018. H-DenseUNet: hybrid densely connected UNet for liver and tumor segmentation from CT volumes. IEEE Transactions on Medical Imaging, 37(12): 2663-2674[DOI: 10.1109/TMI.2018.2845918]
Litjens G, Kooi T, Bejnordi B E, Setio A A A, Ciompi F, Ghafoorian M, Van Der Laak J A W M, Van Ginneken B and Sánchez C I. 2017. A survey on deep learning in medical image analysis. Medical Image Analysis, 42: 60-88[DOI: 10.1016/j.media.2017.07.005]
Ma F. 2013. 3DVolume Enhanced Scanning Techniques of Magnetic Resonance Imaging Used in Intracranial Tumors. Yinchuan: Ningxia Medical University
马凤. 2013. 磁共振三维容积增强扫描技术在颅内肿瘤中的应用. 银川: 宁夏医科大学[DOI: 10.7666/d.D421508]
Ma J and Yang X P. 2019. Automatic brain tumor segmentation by exploring the multi-modality complementary information and cascaded 3D lightweight CNNs//Proceedings of the 4th International MICCAI Brainlesion Workshop. Granada, Spain: Springer: 25-36[DOI: 10.1007/978-3-030-11726-9_3http://dx.doi.org/10.1007/978-3-030-11726-9_3]
Marcinkiewicz M, Nalepa J, Lorenzo P R, Dudzik W and Mrukwa G. 2019. Segmenting brain tumors from MRI using cascaded multi-modal U-Nets//Proceedings of the 4th International MICCAI Brainlesion Workshop. Granada, Spain: Springer: 13-24[DOI: 10.1007/978-3-030-11726-9_2http://dx.doi.org/10.1007/978-3-030-11726-9_2]
Moeskops P, Viergever M A, Mendrik A M, de Vries L S, Benders M J N L and Išgum I. 2016. Automatic segmentation of MR brain images with a convolutional neural network. IEEE Transactions on Medical Imaging, 35(5): 1252-1261[DOI: 10.1109/TMI.2016.2548501]
Myronenko A. 2019. 3D MRI brain tumor segmentation using autoencoder regularization//4th International MICCAI Brainlesion Workshop. Granada, Spain: Springer: 311-320[DOI: 10.1007/978-3-030-11726-9_28http://dx.doi.org/10.1007/978-3-030-11726-9_28]
Pereira S, Pinto A, Alves V and Silva C A. 2016. Brain tumor segmentation using convolutional neural networks in MRI images. IEEE Transactions on Medical Imaging, 35(5): 1240-1251[DOI: 10.1109/TMI.2016.2538465]
Ren X X, Xing Z C, Xia X, Lo D, Wang X Y and Grundy J. 2019. Neural network-based detection of self-admitted technical debt: from performance to explainability. ACM transactions on software engineering and methodology, 28(3): #15[DOI: 10.1145/3324916]
Ronneberger O, Fischer P and Brox T. 2015. U-Net: convolutional networks for biomedical image segmentation//Proceedings of the 18th International Conference on Medical Image Computing and Computer-Assisted Intervention. Munich, Germany: Springer: 234-241[DOI: 10.1007/978-3-319-24574-4_28http://dx.doi.org/10.1007/978-3-319-24574-4_28]
Shaikh M, Anand G, Acharya G, Amrutkar A, Alex V and Krishnamurthi G. 2018. Brain tumor segmentation using dense fully convolutional neural network//Proceedings of the 3rd International MICCAI Brainlesion Workshop. Quebec City, Canada: Springer: 309-319[DOI: 10.1007/978-3-319-75238-9_27http://dx.doi.org/10.1007/978-3-319-75238-9_27]
Shelhamer E, Long J and Darrell T. 2017. Fully convolutional networks for semantic segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(4): 640-651[DOI: 10.1109/TPAMI.2016.2572683]
Shen H C, Wang R X, Zhang J G and McKenna S. 2017. Multi-task fully convolutional network for brain tumour segmentation//Proceedings of the 21st Annual Conference on Medical Image Understanding and Analysis. Edinburgh, UK: Springer: 239-248[DOI: 10.1007/978-3-319-60964-5_21http://dx.doi.org/10.1007/978-3-319-60964-5_21]
Simonyan K and Zisserman A. 2015. Very deep convolutional networks for large-scale image recognition[EB/OL]. [2021-05-03].https://arxiv.org/pdf/1409.1556.pdfhttps://arxiv.org/pdf/1409.1556.pdf
Sudre C H, Li W Q, Vercauteren T, Ourselin S and Cardoso M J. 2017. Generalised Dice overlap as a deep learning loss function for highly unbalanced segmentations//Proceedings of the 3rd International Workshop on Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support. Québec City, Canada: Springer: 240-248[DOI: 10.1007/978-3-319-67558-9_28http://dx.doi.org/10.1007/978-3-319-67558-9_28]
Sun H. 2020. Research on Classification and Segmentation of MRI of Brain Tumor based on Deep Learning. Hangzhou: Zhejiang University
孙浩. 2020. 基于深度学习的脑肿瘤MRI分类与分割技术研究. 杭州: 浙江大学[DOI: 10.27461/d.cnki.gzjdx.2020.000444]
Sun J W. 2020. Application of Deep Learning Methods in Brain Tumors. Ji'nan: Shandong University
孙佳伟. 2020. 深度学习在脑肿瘤中的应用. 济南: 山东大学[DOI: 10.27272/d.cnki.gshdu.2020.004996]
Sun L, Zhang S T, Chen H and Luo L. 2019. Brain tumor segmentation and survival prediction using multimodal MRI scans with deep learning. Frontiers in Neuroscience, 13: #810[DOI: 10.3389/fnins.2019.00810]
Szegedy C, Liu W, Jia Y Q, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V and Rabinovich A. 2015. Going deeper with convolutions//Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Boston, USA: IEEE: 1-9[DOI: 10.1109/CVPR.2015.7298594http://dx.doi.org/10.1109/CVPR.2015.7298594]
Tian W Q. 2020. The Study of Medical Image Segmentation based on Convolution Neural Network. Baoding: Hebei University
田伟倩. 2020. 基于卷积神经网络的医学图像分割算法研究. 保定: 河北大学[DOI: 10.27103/d.cnki.ghebu.2020.001097]
Wang C Y, Liao H Y M, Wu Y H, Chen P Y, Hsieh J W and Yeh I H. 2020. CSPNet: a new backbone that can enhance learning capability of CNN//Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. Seattle, USA: IEEE: 1571-1580[DOI: 10.1109/CVPRW50498.2020.00203http://dx.doi.org/10.1109/CVPRW50498.2020.00203]
Wang G T, Li W Q, Ourselin S and Vercauteren T. 2019a. Automatic brain tumor segmentation using convolutional neural networks with test-time augmentation//Proceedings of the 4th International MICCAI Brainlesion Workshop. Granada, Spain: Springer: 61-72[DOI: 10.1007/978-3-030-11726-9_6http://dx.doi.org/10.1007/978-3-030-11726-9_6]
Wang W, Yu K C, Hugonot J, Fua P and Salzmann M. 2019b. Recurrent U-Net for resource-constrained segmentation//Proceedings of 2019 IEEE/CVF International Conference on Computer Vision (ICCV). Seoul, Korea (South): IEEE: 2142-2151[DOI: 10.1109/ICCV.2019.00223http://dx.doi.org/10.1109/ICCV.2019.00223]
Wong K P. 2005. Medical image segmentation: methods and applications in functional imaging//Handbook of Biomedical Image Analysis. Boston, USA: Springer: 111-182[DOI: 10.1007/0-306-48606-7_3http://dx.doi.org/10.1007/0-306-48606-7_3]
Zhao L Y and Jia K B. 2016. Multiscale CNNs for brain tumor segmentation and diagnosis. Computational and Mathematical Methods in Medicine, 2016: #8356294[DOI: 10.1155/2016/8356294]
Zhou Z X, He Z S, ShiM F, Du J L and Chen D D. 2020. 3D dense connectivity network with atrous convolutional feature pyramid for brain tumor segmentation in magnetic resonance imaging of human heads. Computers in Biology and Medicine, 121: #103766[DOI: 10.1016/j.compbiomed.2020.103766]
相关作者
相关机构
京公网安备11010802024621