浏览全部资源
扫码关注微信
纸质出版日期： 2024-09-16 ，
移动端阅览
焦润海， 褚佳杰， 刘嘉骥， 余济民. 2024. SC-Net：用于重叠染色体分割的上下文信息跳跃连接网络. 中国图象图形学报， 29(09):2806-2824
Jiao Runhai， Chu Jiajie， Liu Jiaji， Yu Jimin. 2024. SC-Net： a contextual information skip connection network for overlapping chromosome segmentation. Journal of Image and Graphics， 29(09):2806-2824
目的
2
染色体核型分析从细胞分裂中期图像中分离和分类染色体，是遗传疾病诊断广泛采用的方法，其中形态多样的重叠染色体簇的分割，依赖于准确的边界等细节特征。为此，本文融合目标的上下文信息，构建了一种两阶段的重叠染色体分割模型SC-Net（skip connection network）。
方法
2
首先，在语义分割基线模型U-Net++中增加混合池化模块捕获重叠染色体的局部上下文信息，在解码器网络中并联上下文融合模块和上下文先验辅助分支，增强通道和空间上的全局上下文信息。其次，利用已标注样本的类别先验信息生成真实亲和矩阵，加入训练过程以有效区分重叠染色体图像中易混淆的空间信息。最后，通过染色体实例重建算法对重叠与非重叠区域的元素迭代进行配对，拼接形成单条染色体。
结果
2
在公开的ChromSeg（chromosome segmentation）数据集上进行实验，结果表明SC-Net分割出的重叠染色体区域交并比值为83.5%，与对比算法中的较优算法相比性能提升2.7%。
结论
2
本文构建的重叠染色体分割模型通过融合上下文信息，能更有效地解决形态多样的重叠染色体簇的分割问题，相比对比方法可以得到更精细和准确的结果。
Objective
2
Chromosome karyotype analysis separates and categorizes chromosomes in midcell division images， and it is widely used for the diagnosis of genetic diseases， in which overlapping chromosome segmentation is one of the key steps. Based on image analysis of overlapping chromosomes， the morphologically diverse chromosome clusters depend on detailed features， such as accurate boundaries during segmentation， in addition to obtaining the basic contour， texture， and semantic information. For this reason， in this paper， a two-stage overlapping chromosome segmentation model SC-Net was constructed through fusion of the contextual information of the target to improve the segmentation performance of the network.
Method
2
First， the model SC-UNet++ added the hybrid pooling module （HPM） to the baseline model U-Net++ for semantic segmentation to capture the local context information of overlapping chromosomes and complemented the detailed features of chromosomes， such as color， thickness， and stripes， based on the superposition operation of empty space pyramid pooling and stripe pooling. The context fusion module （CFM） was connected in parallel to a decoder network， i.e.， the channel correlation of input features was extracted using the efficient channel attention module， and the features obtained via the multiplication of the output with the input were subsequently fed to the HPM and the spatial attention module （SAM）， which explored the correlation of the region around the pixel to obtain the local context and extract the global context through global pooling operation， respectively. In addition， context prior auxiliary branch （CPAB） was introduced after CFM to improve the global context information on channel and space. Second， the category a priori information of labeled training samples， which serves as an additional source of supervisory information during training and effectively distinguishes confusing spatial features in overlapping chromosome images， was used to generate the true affinity matrix. Finally， the elements of overlapping and non-overlapping regions were iteratively paired by the chromosome instance reconstruction algorithm to splice and form a single chromosome. In this paper， experimental analysis were based on ChromSeg dataset， and the hardware resources used included a desktop server with 32 GB RAM， a 3.3 GHz Intel Xeon CPU， and an NVIDIA RTX 3070 GPU. The model was used based on the semantic segmentation toolkit MMSegmentation version 0.30.0 and implemented under the Ubuntu 18.04 operating system， with PyTorch 1.10.0 serving as a deep learning framework. The following sections describe relevant hyperparameter settings and initialization methods for network training and the loss function selection strategy.
Result
2
SC-Net fully extracted and utilized the contextual and category prior information of overlapping chromosome images and showed good performance in segmentation scenarios with various numbers of overlapping chromosomes. The effect of each improvement on the algorithm performance was investigated through ablation experiments， where various combinations of CFM， HPM， CPAB， and segmentation loss were designed on the baseline model U-Net++. The results proved the better performance of SC-UNet++ compared with models in all evaluation metrics. This condition confirms the effectiveness of the method proposed in this paper， i.e.， SC-UNet++ attained better performance in the segmentation of overlapping chromosomes. Through comparative experiments， the SC-Net proposed in this paper caused improvements on the ChromSeg dataset， which outperformed several models in terms of all metrics， and the model achieved an overlapping chromosome region intersection and merger ratio of 83.5%. The overall accuracy obtained after the reconstruction of chromosome instances was 92.3%， which is higher than the best and the same two-stage ChromSeg segmentation methods by 2.7% and 1.8%. The SC-Net outperformed these models mainly due to its capability to extract contextual information and category relevance of the target， and it enables the model to further gain insights into overlapping regions.
Conclusion
2
The overlapping chromosome segmentation model constructed in this paper can effectively solve the segmentation problem of morphologically diverse overlapping chromosome clusters by fusing contextual information and obtaining finer and more accurate results compared with the existing methods.
重叠染色体分割混合池化模块 （HPM）上下文融合模块 （CFM）上下文先验辅助分支 （CPAB）真实亲和矩阵
overlapping chromosome segmentationhybrid pooling module （HPM）contextual fusion module （CFM）contextual prior aided branch （CPAB）real affinity matrix
Altinsoy E， Yilmaz C， Wen J， Wu L Q， Yang J and Zhu Y M. 2019. Raw G-band chromosome image segmentation using U-Net based neural network//Proceedings of the 18th International Conference on Artificial Intelligence and Soft Computing. Zakopane， Poland： Springer： 117-126 ［DOI： 10.1007/978-3-030-20915-5_11http://dx.doi.org/10.1007/978-3-030-20915-5_11］
Cao X， Lan F Z， Liu C M， Lam T W and Luo R B. 2020. ChromSeg： two-stage framework for overlapping chromosome segmentation and reconstruction//Proceedings of 2020 IEEE International Conference on Bioinformatics and Biomedicine （BIBM）. Seoul， Korea （South）： IEEE： 2335-2342 ［DOI： 10.1109/BIBM49941.2020.9313458http://dx.doi.org/10.1109/BIBM49941.2020.9313458］
Chen L C， Papandreou G， Kokkinos I， Murphy K and Yuille A L. 2018. DeepLab： semantic image segmentation with deep convolutional nets， atrous convolution， and fully connected CRFs. IEEE Transactions on Pattern Analysis and Machine Intelligence， 40（4）： 834-848 ［DOI： 10.1109/TPAMI.2017.2699184http://dx.doi.org/10.1109/TPAMI.2017.2699184］
Chen P J， Cai J Z and Yang L. 2020. Chromosome segmentation via data simulation and shape learning//Proceedings of the 42nd Annual International Conference of the IEEE Engineering in Medicine and Biology Society （EMBC）. Montreal， Canada： IEEE： 1637-1640 ［DOI： 10.1109/EMBC44109.2020.9176020http://dx.doi.org/10.1109/EMBC44109.2020.9176020］
Deng J， Dong W， Socher R， Li L J， Li K and Li F F. 2009. ImageNet： a large-scale hierarchical image database//Proceedings of 2009 IEEE Conference on Computer Vision and Pattern Recognition. Miami， USA： IEEE： 248-255 ［DOI： 10.1109/CVPR.2009.5206848http://dx.doi.org/10.1109/CVPR.2009.5206848］
Fan J H， Qian Z and Qiao S Q. 2009. Research on chromosome image segmentation. Journal of Qiqihar Medical College， 30（23）： 2927-2928
范继红， 钱真， 乔树清. 2009. 染色体图像分割的研究. 齐齐哈尔医学院学报， 30（23）： 2927-2928 ［DOI：10.3969/j.issn.1002-1256.2009.23.042http://dx.doi.org/10.3969/j.issn.1002-1256.2009.23.042］
Firdaus-Nawi M， Noraini O， Sabri M Y， Siti-Zahrah A， Zamri-Saad M， and Latifah H. 2011. DeepLabv3+ _encoder-decoder with Atrous separable convolution for semantic image segmentation. Pertanika Journal of Tropical Agricultural Science， 34（1）： 137-143
Gorelick L， Galun M， Sharon E， Basri R and Brandt A. 2006. Shape representation and classification using the Poisson equation. IEEE Transactions on Pattern Analysis and Machine Intelligence， 28（12）： 1991-2005 ［DOI： 10.1109/TPAMI.2006.253http://dx.doi.org/10.1109/TPAMI.2006.253］
Grau V， Mewes A U J， Alcaniz M， Kikinis R and Warfield S K. 2004. Improved watershed transform for medical image segmentation using prior information. IEEE Transactions on Medical Imaging， 23（4）： 447-458 ［DOI： 10.1109/TMI.2004.824224http://dx.doi.org/10.1109/TMI.2004.824224］
Grisan E， Poletti E and Ruggeri A. 2009. Automatic segmentation and disentangling of chromosomes in Q-band prometaphase images. IEEE Transactions on Information Technology in Biomedicine， 13（4）： 575-581 ［DOI： 10.1109/TITB.2009.2014464http://dx.doi.org/10.1109/TITB.2009.2014464］
Guangzhou Electronic Technology Co.， Ltd.， Chinese Academy of Sciences. 2022. Chromosome image instance segmentation method and device based on improved Mask RCNN. China， 201911021995.9 （中国科学院广州电子技术有限公司. 2022. 基于改进Mask RCNN的染色体图像实例分割方法及装置. 中国， 201911021995.9
Guo J S， Zhou H Y， Wang L S and Yu Y Z. 2022. UNet-2022： exploring dynamics in non-isomorphic architecture ［EB/OL］. ［2023-08-10］. http://arxiv.org/pdf/2210.15566.pdfhttp://arxiv.org/pdf/2210.15566.pdf
Hosang J， Benenson R and Schiele B. 2017. Learning non-maximum suppression//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu， USA： IEEE： 6469-6477 ［DOI： 10.1109/CVPR.2017.685http://dx.doi.org/10.1109/CVPR.2017.685］
Hou Q B， Zhang L， Cheng M M and Feng J S. 2020. Strip pooling： rethinking spatial pooling for scene parsing//Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle， USA： IEEE： 4002-4011 ［DOI： 10.1109/CVPR42600.2020.00406http://dx.doi.org/10.1109/CVPR42600.2020.00406］
Hu R L， Karnowski J， Fadely R and Pommier J P. 2017. Image segmentation to distinguish between overlapping human chromosomes. ［EB/OL］. ［2023-08-10］. http://arxiv.org/pdf/1712.07639.pdfhttp://arxiv.org/pdf/1712.07639.pdf
Jing X， Teng Q Z， Tang T and Wang S. 2008. Efficient method for segmentation of chromosome images. Computer Engineering and Applications， 44（35）： 182-184， 187
敬雪， 滕奇志， 唐棠， 王姝. 2008. 有效的染色体分割新方法. 计算机工程与应用， 44（35）： 182-184， 187 ［DOI： 10.3778/j.issn.1002-8331.2008.35.054http://dx.doi.org/10.3778/j.issn.1002-8331.2008.35.054］
Kluvanec D， Phillips T B， McCaffrey K J W and Al Moubayed N. 2022. Using orientation to distinguish overlapping chromosomes//Proceedings of the 31st International Conference on Artificial Neural Networks. Bristol， UK： Springer： 391-403 ［DOI： 10.1007/978-3-031-15919-0_33http://dx.doi.org/10.1007/978-3-031-15919-0_33］
Lin C C， Zhao G S， Yin A H， Ding B C， Guo L and Chen H B. 2020. AS-PANet： a chromosome instance segmentation method based on improved path aggregation network architecture. Journal of Image and Graphics， 25（10）： 2271-2280
林成创， 赵淦森， 尹爱华， 丁笔超， 郭莉， 陈汉彪. 2020. AS-PANet： 改进路径增强网络的重叠染色体实例分割. 中国图象图形学报， 25（10）： 2271-2280 ［DOI： 10.11834/jig.200236http://dx.doi.org/10.11834/jig.200236］
Lin C C， Zhao G S， Yin A H， Yang Z R， Guo L， Chen H B， Zhao L， Li S Y， Luo H Y and Ma Z H. 2021. A novel chromosome cluster types identification method using ResNeXt WSL model. Medical Image Analysis， 69： #101943 ［DOI： 10.1016/j.media.2020.101943http://dx.doi.org/10.1016/j.media.2020.101943］
Lin T Y， Dollr P， Girshick R， He K M， Hariharan B and Belongie S. 2017. Feature pyramid networks for object detection//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu， USA： IEEE： 936-944 ［DOI： 10.1109/CVPR.2017.106http://dx.doi.org/10.1109/CVPR.2017.106］
Liu S， Qi L， Qin H F， Shi J P and Jia J Y. 2018. Path aggregation network for instance segmentation//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City， USA： IEEE： 8759-8768 ［DOI： 10.1109/CVPR.2018.00913http://dx.doi.org/10.1109/CVPR.2018.00913］
Long J， Shelhamer E and Darrell T. 2015. Fully convolutional networks for semantic segmentation//Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition. Boston， USA： IEEE： 3431-3440 ［DOI： 10.1109/CVPR.2015.7298965http://dx.doi.org/10.1109/CVPR.2015.7298965］
Luo C L and Zhao Y. 2023. Review of deep learning methods for karyotype analysis. Journal of Image and Graphics， 28（11）： 3363-3385
罗纯龙， 赵屹. 2023. 染色体核型分析深度学习方法综述. 中国图象图形学报， 28（11）： 3363-3385 ［DOI： 10.11834/jig.221094http://dx.doi.org/10.11834/jig.221094］
Luo K K， Wang T and Ye F F. 2021. U-Net segmentation model of brain tumor MR image based on attention mechanism and multi-view fusion. Journal of Image and Graphics， 26（9）： 2208-2218
罗恺锴， 王婷， 叶芳芳. 2021. 引入注意力机制和多视角融合的脑肿瘤MR图像U-Net分割模型. 中国图象图形学报， 26（9）： 2208-2218 ［DOI： 10.11834/jig.200584http://dx.doi.org/10.11834/jig.200584］
Madian N， Jayanthi K B and Suresh S. 2015. Contour based segmentation of chromosomes in G-band metaphase images//Proceedings of 2015 IEEE Global Conference on Signal and Information Processing （GlobalSIP）. Orlando， USA： IEEE： 943-947 ［DOI： 10.1109/GlobalSIP.2015.7418336http://dx.doi.org/10.1109/GlobalSIP.2015.7418336］
Milletari F， Navab N and Ahmadi S A. 2016. V-Net： fully convolutional neural networks for volumetric medical image segmentation//Proceedings of the 4th International Conference on 3D Vision （3DV）. Stanford， USA： IEEE： 565-571 ［DOI： 10.1109/3DV.2016.79http://dx.doi.org/10.1109/3DV.2016.79］
Minaee S， Fotouhi M and Khalaj B H. 2014. A geometric approach to fully automatic chromosome segmentation//Proceedings of 2014 IEEE Signal Processing in Medicine and Biology Symposium （SPMB）. Philadelphia， USA： IEEE： 1-6 ［DOI： 10.1109/SPMB.2014.7163174http://dx.doi.org/10.1109/SPMB.2014.7163174］
Robbins H and Monro S. 1951. A stochastic approximation method. The Annals of Mathematical Statistics， 22（3）： 400-407 ［DOI： 10.1214/aoms/1177729586http://dx.doi.org/10.1214/aoms/1177729586］
Ronneberger O， Fischer P and Brox T. 2015. U-Net： convolutional networks for biomedical image segmentation//Proceedings of the 18th International Conference on Medical Image Computing and Computer-Assisted Intervention （MICCAI）. Munich， Germany： Springer： 234-241 ［DOI： 10.1007/978-3-319-24574-4_28http://dx.doi.org/10.1007/978-3-319-24574-4_28］
Saleh H M， Saad N H and Isa N A M. 2019. Overlapping chromosome segmentation using U-Net： convolutional networks with test time augmentation. Procedia Computer Science， 159： 524-533 ［DOI： 10.1016/j.procs.2019.09.207http://dx.doi.org/10.1016/j.procs.2019.09.207］
Sharma M， Saha O， Sriraman A， Hebbalaguppe R， Vig L and Karande S. 2017. Crowdsourcing for chromosome segmentation and deep classification//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops. Honolulu， USA： IEEE： 786-793 ［DOI： 10.1109/CVPRW.2017.109http://dx.doi.org/10.1109/CVPRW.2017.109］
Sun X F， Li J M， Ma J L， Xu H Q， Chen B， Zhang Y F and Feng T. 2021. Segmentation of overlapping chromosome images using U-net with improved dilated convolutions. Journal of Intelligent and Fuzzy Systems， 40（3）： 5653-5668 ［DOI： 10.3233/JIFS-201466http://dx.doi.org/10.3233/JIFS-201466］
Wang G J. 2022. Research on End-to-End Chromosome Segmentation Method based on Adaptive and Regression Correction Network. Xuzhou： China University of Mining and Technology
王广杰. 2022. 基于自适应与回归修正网络的端到端染色体分割方法研究. 徐州： 中国矿业大学 ［DOI： 10.27623/d.cnki.gzkyu.2022.000244http://dx.doi.org/10.27623/d.cnki.gzkyu.2022.000244］
Wang G T， Li W Q， Ourselin S and Vercauteren T. 2019. Automatic brain tumor segmentation using convolutional neural networks with test-time augmentation//4th International Workshop on Brainlesion： Glioma， Multiple Sclerosis， Stroke and Traumatic Brain Injuries. Granada， Spain： Springer： 61-72 ［DOI： 10.1007/978-3-030-11726-9_6http://dx.doi.org/10.1007/978-3-030-11726-9_6］
Wang P L， Hu W J， Zhang J P， Wen Y F， Xu C M and Qian D H. 2021. Enhanced rotated mask R-CNN for chromosome segmentation//Proceedings of the 43rd Annual International Conference of the IEEE Engineering in Medicine and Biology Society （EMBC）. Mexico： IEEE： 2769-2772 ［DOI： 10.1109/EMBC46164.2021.9630695http://dx.doi.org/10.1109/EMBC46164.2021.9630695］
Woo S， Park J， Lee J Y and Kweon I S. 2018. CBAM： convolutional block attention module//Proceedings of the 15th European Conference on Computer Vision （ECCV）. Munich， Germany： Springer： 3-19 ［DOI： 10.1007/978-3-030-01234-2_1http://dx.doi.org/10.1007/978-3-030-01234-2_1］
Xie S N， Girshick R， Dollr P， Tu Z W and He K M. 2017. Aggregated residual transformations for deep neural networks//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu， USA： IEEE： 5987-5995 ［DOI： 10.1109/CVPR.2017.634http://dx.doi.org/10.1109/CVPR.2017.634］
Xu M D， Zhang Z， Wei F Y， Hu H and Bai X. 2023. Side adapter network for open-vocabulary semantic segmentation//Proceedings of 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Vancouver， Canada： IEEE： 2945-2954 ［DOI： 10.1109/CVPR52729.2023.00288http://dx.doi.org/10.1109/CVPR52729.2023.00288］
Ye Y Q. 2021. Joint Segmentation of Multiple Types of Effusions in Retinal OCT Images. Suzhou： Soochow University
叶妍青. 2021. 视网膜OCT图像中多类积液的联合分割. 苏州： 苏州大学 ［DOI： 10.27351/d.cnki.gszhu.2021.000822http://dx.doi.org/10.27351/d.cnki.gszhu.2021.000822］
Yilmaz I C， Yang J， Altinsoy E and Zhou L. 2018. An improved segmentation for raw G-band chromosome images//Proceedings of the 5th International Conference on Systems and Informatics （ICSAI）. Nanjing， China： IEEE： 944-950 ［DOI： 10.1109/ICSAI.2018.8599328http://dx.doi.org/10.1109/ICSAI.2018.8599328］
Vincent L. 1993. Morphological grayscale reconstruction in image analysis： applications and efficient algorithms. IEEE Transactions on Image Processing， 2（2）： 176-201 ［DOI： 10.1109/83.217222http://dx.doi.org/10.1109/83.217222］
Yu C Q， Wang J B， Gao C X， Yu G， Shen C H and Sang N. 2020. Context prior for scene segmentation//Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle， USA： IEEE： 12413-12422 ［DOI： 10.1109/CVPR42600.2020.01243http://dx.doi.org/10.1109/CVPR42600.2020.01243］
Zhang L， Zhang X L， Lu L and Liu H. 2022. Image segmentation of chromosomes based on residual U-Net. Journal of Chongqing University of Posts and Telecommunications （Natural Science Edition）， 34（2）： 208-215
张林， 张雪利， 路霖， 刘辉. 2022. 基于残差U-Net网络的染色体图像分割方法. 电大学学报（自然科学版）， 34（2）： 208-215 ［DOI： 10.3979/j.issn.1673-825X.202010160309http://dx.doi.org/10.3979/j.issn.1673-825X.202010160309］
Zhang X F， Zhang S， Zhang D H and Liu R. 2023. Group attention-based medical image segmentation model. Journal of Image and Graphics， 28（10）： 3231-3242
张学峰， 张胜， 张冬晖， 刘瑞. 2023. 引入分组注意力的医学图像分割模型. 中国图象图形学报， 28（10）： 3231-3242 ［DOI： 10.11834/jig.220748http://dx.doi.org/10.11834/jig.220748］
Zhao H S， Shi J P， Qi X J， Wang X G and Jia J Y. 2017. Pyramid scene parsing network//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu， USA： IEEE： 6230-6239 ［DOI： 10.1109/CVPR.2017.660http://dx.doi.org/10.1109/CVPR.2017.660］
Zhou Z W， Siddiquee M M R， Tajbakhsh N and Liang J M. 2020. UNet++： redesigning skip connections to exploit multiscale features in image segmentation. IEEE Transactions on Medical Imaging， 39（6）： 1856-1867 ［DOI： 10.1109/TMI.2019.2959609http://dx.doi.org/10.1109/TMI.2019.2959609］
Zhu Q. 2022. Research on Medical Image Segmentation Method based on Improved U-Net. Guangzhou： Guangzhou University
朱旗. 2022. 基于改进U-Net的医学图像分割方法研究. 广州： 广州大学 ［DOI：10.27040/d.cnki.ggzdu.2022.001247http://dx.doi.org/10.27040/d.cnki.ggzdu.2022.001247］
相关文章
相关作者
相关机构
京公网安备11010802024621