浏览全部资源
扫码关注微信
收稿日期：2023-04-27，
修回日期：2023-08-30，
纸质出版日期：2024-04-16
移动端阅览
目的
2
随着全球人口老年化趋势日益加剧，阿尔茨海默症（Alzheimer's disease， AD）的及时诊断与病理区域的可视化及其准确定位具有重要的临床意义。目前的研究中，基于块级和区域级的检测，由于采用非线性交互很难解释影响模型决策的病理区域。针对此问题，提出了一种AD病理区域定位及诊断的联合学习框架。
方法
2
利用反事实推理的思想，基于前景背景注意力掩码构建注意力引导的循环生成对抗网络（attention-guided cycle generative adversarial network， ACGAN）可视化AD患者的病理区域，并使用生成的病理区域知识指导增强诊断模型。具体来说，通过在ACGAN模型的生成器中设计注意力掩码来引导生成方案，使模型更好地聚焦于疾病的病理区域，有效地捕捉突出的全局特征。并通过ACGAN模型中病理区域生成器实现结构磁共振图像（structural magnetic resonance imaging， sMRI）在源域和目标域之间的转换清晰地划分出细微的病理区域。利用生成的病理区域知识作为指导，并结合三维坐标注意力与全局局部注意力，获取三维图像之间的依赖关系及三维空间的位置信息，优化诊断模型。
结果
2
为了验证方法的有效性，在公开的ADNI（Alzheimer’s disease neuroimaging initiative）数据集上对模型进行评估，与传统的卷积神经网络（convolutional neural network，CNN）模型及几种较为先进的AD分类诊断模型相比，本文使用病理区域知识指导增强诊断模型显示出优越的诊断性能，相比于性能较好的方法，ACC（accuracy）、F1-score、AUC（area under curve）分别提高了3.60%、5.02%、1.94%。并对生成的病理区域图像进行定性及定量评估，本文方法得到的病理区域图像归一化互相关分数和峰值信噪比均优于对比方法。
结论
2
与现有方法相比，本文模型可以学习sMRI图像在源域和目标域之间的转换，能够准确地捕获全局特征及病理区域。并将学习到的病理区域知识用于AD诊断模型的改进，使分类诊断模型取得了卓越性能。
Objective
2
Alzheimer’s disease （AD） is a neurodegenerative disease commonly occurring in middle-aged and elderly populations and is accompanied by cognitive impairment and memory loss. With the increasing trend of global population aging， timely diagnosis of AD and visualization of pathological regions and its accurate localization are of considerable clinical importance. In current research， one conventional approach is to extract patch-level features based on voxel morphology and prior knowledge for detecting structural changes and identifying AD-related voxel structures. Another approach is to learn AD-related pathological regions by focusing the network on specific brain regions of interest （e.g.， cortical and hippocampal regions） based on regional features. However， these approaches ignore other pathological locations in the brain and fail to obtain accurate global structural information for the diagnosis of AD. A joint learning model for localization and diagnosis of AD pathological regions is proposed using the idea of counterfactual reasoning to obtain a convincing model architecture and increase interpretability of the output， highlighting the information of pathological regions.An attention-guided cycle generative adversarial network（ACGAN） is constructed based on foreground-background attention mask.
Method
2
In the vast majority of image classification methods， the network model aims to find which part of the input is
X
and which part influences the decision of the classifier to determine the final result as
Y
. From another viewpoint， in a hypothetical scenario where the input
X
is
C
， would the result be
Z
instead of
Y
？ This idea is defined as counterfactual reasoning. The AD classification model was first trained as a classifier in the hypothetical scenario to construct its output， and the pathological features of AD were then obtained. The hypothetical scenario was constructed using a generative adversarial network to learn the mapping of images from the source domain to the target field. However， achieving good results by directly generating image to image transformation is difficult due to the complexity of whole brain structural magnetic resonance imaging（sMRI） images and the considerable amount of information in 3D space. Drawing inspiration from two models， namely CycleGAN and AttentionGAN， the image can be mapped from the source to the target domain by changing the region in the original image that affects the category judgment and using the foreground-background attention to guide the model to focus on the dynamic change region， which reduces the complexity of the model and facilitates easy model fitting. Therefore， this paper proposes an attention-guided recurrent generative adversarial network to construct a counterfactual mapping model for AD， thereby outputting the corresponding pathological regions. If a counterfactual map conditional on the target label （i.e.， hypothetical scenario） is generated， then this counterfactual map is added to the input image to diagnose the transformed image as the target type. For example， when the counterfactual map is added to the sMRI image of a subject with AD， modifying the corresponding region changes the input sMRI image and facilitates the diagnosis by the classifier as a normal subject. The pathological regions represented by the counterfactual map were used as privileged information （i.e.， the location information of the counterfactual map influenced the category determination） to further optimize the diagnostic model. Therefore， the diagnostic model focused on learning and discovering disease-related discriminative regions to combine the pathological region generation and AD diagnostic models.
Result
2
The proposed model was evaluated against traditional convolutional neural network（CNN） models and several highly advanced AD diagnostic models on a publicly available ADNI dataset using quantitative evaluation metrics， including accuracy（ACC）， F1-score， and area under curve（AUC）. Experimental results showed that the model improved ACC， F1-score， and AUC by 3.60%， 5.02%， and 1.94%， respectively， compared with the best performing method. The generated pathological region images were also qualitatively and quantitatively evaluated， and the normalized correlation scores and peak signal-to-noise ratios of the pathological region images obtained by the method were better than those of the compared methods. More importantly， the proposed AD diagnostic model visualized the global features and finegrained discriminative regions of the pathological regions compared with the benchmark model， and the average accuracy after three iterations was improved by +4.90%， +11.03%， and +11.08% compared with the benchmark method.
Conclusion
2
Compared with existing methods， this ACGAN model can learn the transformation of sMRI images between source and target fields and accurately capture global features and pathological regions. The learned knowledge of the pathological region is used for the improvement of AD diagnosis models. Therefore， the classification diagnosis model achieves excellent performance.
Alzheimer’s Association . 2019 . 2019 Alzheimer’s disease facts and figures . Alzheimer’s and Dementia ， 15 （ 3 ）： 321 - 387 ［ DOI： 10.1016/j.jalz.2019.01.010 http://dx.doi.org/10.1016/j.jalz.2019.01.010 ］
Avants B B ， Tustison N and Song G . 2009. Advanced normalization tools （ANTS） . The Insight Journal ［J/OL］. ［ 2023-04-12 ］. https://scicomp.ethz.ch/public/manual/ants/2.x/ants2.pdf https://scicomp.ethz.ch/public/manual/ants/2.x/ants2.pdf
Ben Ahmed O ， Mizotin M ， Benois-Pineau J ， Allard M ， Catheline G and Ben Amar C . 2015 . Alzheimer’s disease diagnosis on structural MR images using circular harmonic functions descriptors on hippocampus and posterior cingulate cortex . Computerized Medical Imaging and Graphics ， 44 ： 13 - 25 ［ DOI： 10.1016/j.compmedimag.2015.04.007 http://dx.doi.org/10.1016/j.compmedimag.2015.04.007 ］
Cui R X and Liu M H . 2019 . RNN-based longitudinal analysis for diagnosis of Alzheimer’s disease . Computerized Medical Imaging and Graphics ， 73 ： 1 - 10 ［ DOI： 10.1016/j.compmedimag.2019.01.005 http://dx.doi.org/10.1016/j.compmedimag.2019.01.005 ］
Dong Y P ， Su H and Zhu J . 2022 . Interpretability analysis of deep neural networks with adversarial examples . Acta Automatica Sinica ， 48 （ 1 ）： 75 - 86
董胤蓬 ， 苏航 ， 朱军 . 2022 . 面向对抗样本的深度神经网络可解释性分析 . 自动化学报 ， 48 （ 1 ）： 75 - 86 ［ DOI： 10.16383/j.aas.c200317 http://dx.doi.org/10.16383/j.aas.c200317 ］
Ebrahimi A ， Luo S H and Chiong R . 2020 . Introducing transfer learning to 3D ResNet-18 for Alzheimer’s disease detection on MRI images // Proceedings of the 35th International Conference on Image and Vision Computing New Zealand （IVCNZ） . Wellington， New Zealand ： IEEE： 1 - 6 ［ DOI： 10.1109/IVCNZ51579.2020.9290616 http://dx.doi.org/10.1109/IVCNZ51579.2020.9290616 ］
Fan Y ， Batmanghelich N ， Clark C M and Davatzikos C . 2008 . Spatial patterns of brain atrophy in MCI patients， identified via high-dimensional pattern classification， predict subsequent cognitive decline . NeuroImage ， 39 （ 4 ）： 1731 - 1743 ［ DOI： 10.1016/j.neuroimage.2007.10.031 http://dx.doi.org/10.1016/j.neuroimage.2007.10.031 ］
Gunning D ， Stefik M ， Choi J ， Miller T ， Stumpf S and Yang G Z . 2019 . XAI——Explainable artificial intelligence . Science Robotics ， 4 （ 37 ）： #eaay 7120 ［ DOI： 10.1126/scirobotics.aay7120 http://dx.doi.org/10.1126/scirobotics.aay7120 ］
Han K F ， He M ， Yang F and Zhang Y . 2022a . Multi-task multi-level feature adversarial network for joint Alzheimer’s disease diagnosis and atrophy localization using sMRI . Physics in Medicine and Biology ， 67 （ 8 ）： # 085002 ［ DOI： 10.1088/1361-6560/ac5ed5 http://dx.doi.org/10.1088/1361-6560/ac5ed5 ］
Han X M ， Fei X Y ， Wang J ， Zhou T ， Ying S H ， Shi J and Shen D G . 2022b . Doubly supervised transfer classifier for computer-aided diagnosis with imbalanced modalities . IEEE Transactions on Medical Imaging ， 41 （ 8 ）： 2009 - 2020 ［ DOI： 10.1109/TMI.2022.3152157 http://dx.doi.org/10.1109/TMI.2022.3152157 ］
He K M ， Zhang X Y ， Ren S Q and Sun J . 2016 . Deep residual learning for image recognition // Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition （CVPR） . Las Vegas， USA ： IEEE： 770 - 778 ［ DOI： 10.1109/CVPR.2016.90 http://dx.doi.org/10.1109/CVPR.2016.90 ］
Hou Q B ， Zhou D Q and Feng J S . 2021 . Coordinate attention for efficient mobile network design // Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Nashville， USA ： IEEE： 13708 - 13717 ［ DOI： 10.1109/CVPR46437.2021.01350 http://dx.doi.org/10.1109/CVPR46437.2021.01350 ］
Isensee F ， Schell M ， Pflueger I ， Brugnara G ， Bonekamp D ， Neuberger U ， Wick A ， Schlemmer H P ， Heiland S ， Wick W ， Bendszus M ， Maier-Hein K H and Kickingereder P . 2019 . Automated brain extraction of multisequence MRI using artificial neural networks . Human Brain Mapping ， 40 （ 17 ）： 4952 - 4964 ［ DOI： 10.1002/hbm.24750 http://dx.doi.org/10.1002/hbm.24750 ］
Jr Jack C R ， Bernstein M A ， Fox N C ， Thompson P ， Alexander G ， Harvey D ， Borowski B ， Britson P J ， Whitwell J L ， Ward C ， Dale A M ， Felmlee J P ， Gunter J L ， Hill D L G ， Killiany R ， Schuff N ， Fox‐bosetti S ， Lin C ， Studholme C ， Decarli C S ， Gunnar Krueger N ， Ward H A ， Metzger G J ， Scott K T ， Mallozzi R ， Blezek D ， Levy J ， Debbins J P ， Fleisher A S ， Albert M ， Green R ， Bartzokis G ， Glover G ， Mugler J and Weiner M W . 2008 . The Alzheimer’s disease neuroimaging initiative （ADNI）： MRI methods . Journal of Magnetic Resonance Imaging ， 27 （ 4 ）： 685 - 691 ［ DOI： 10.1002/jmri.21049 http://dx.doi.org/10.1002/jmri.21049 ］
Jin D ， Zhou B ， Han Y ， Ren J J ， Han T ， Liu B ， Lu J ， Song C Y ， Wang P ， Wang D W ， Xu J ， Yang Z Y ， Yao H X ， Yu C S ， Zhao K ， Wintermark M ， Zuo N M ， Zhang X Q ， Zhou Y Y ， Zhang X ， Jiang T Z ， Wang Q and Liu Y . 2020 . Generalizable， reproducible， and neuroscientifically interpretable imaging biomarkers for Alzheimer’s disease . Advanced Science ， 7 （ 14 ）： # 2000675 ［ DOI： 10.1002/advs.202000675 http://dx.doi.org/10.1002/advs.202000675 ］
Jung E ， Luna M and Park S H . 2020 . Conditional generative adversarial network for predicting 3D medical images affected by Alzheimer’s diseases // Proceedings of the 3rd International Workshop on Predictive Intelligence in Medicine . Lima， Peru ： Springer： 79 - 90 ［ DOI： 10.1007/978-3-030-59354-4_8 http://dx.doi.org/10.1007/978-3-030-59354-4_8 ］
Li Q F ， Xing X D ， Sun Y ， Xiao B ， Wei H ， Huo Q ， Zhang M Q ， Zhou XS ， Zhan Y Q ， Xue Z and Shi F . 2019 . Novel iterative attention focusing strategy for joint pathology localization and prediction of MCI progression // Proceedings of Medical Image Computing and Computer Assisted Intervention（MICCAI 2019） . Shenzhen， China ： Springer： 307 - 315 ［ DOI： 10.1007/978-3-030-32251-9_34 http://dx.doi.org/10.1007/978-3-030-32251-9_34 ］
Lian C F ， Liu M X ， Pan Y S and Shen D G . 2022 . Attention-guided hybrid network for dementia diagnosis with structural MR images . IEEE Transactions on Cybernetics ， 52 （ 4 ）： 1992 - 2003 ［ DOI： 10.1109/TCYB.2020.3005859 http://dx.doi.org/10.1109/TCYB.2020.3005859 ］
Linsley D ， Shiebler D ， Eberhardt S and Serre T . 2019 . Learning what and where to attend // Proceedings of the 7th International Conference on Learning Representations . New Orleans， USA ： OpenReview.net
Liu K H ， Ye Z H ， Guo H Y ， Cao D P ， Chen L and Wang F Y . 2021a . FISS GAN： a generative adversarial network for foggy image semantic segmentation . IEEE/CAA Journal of Automatica Sinica ， 8 （ 8 ）： 1428 - 1439 ［ DOI： 10.1109/JAS.2021.1004057 http://dx.doi.org/10.1109/JAS.2021.1004057 ］
Liu M J ， Tang J L ， Yu W X and Jiang N . 2021b . Attention-based 3D ResNet for detection of Alzheimer’s disease process // Proceedings of Neural Information Processing . Bali， Indonesia ： Springer： 342 - 353 ［ DOI： 10.1007/978-3-030-92185-9_28 http://dx.doi.org/10.1007/978-3-030-92185-9_28 ］
Liu S ， Yadav C ， Fernandez-Granda C and Razavian N . 2020 . On the design of convolutional neural networks for automatic detection of Alzheimer’s disease // Proceedings of the Machine Learning for Health NeurIPS Workshop PMLR . Vancouver， Canada ： PMLR： 184 - 201 .
Liu Y ， Fan Y Y ， Guo Z ， Lyu G Y and Liu S Y . 2022 . Solo face image-based panoramic texture map generation . Journal of Image and Graphics ， 27 （ 2 ）： 602 - 613
刘洋 ， 樊养余 ， 郭哲 ， 吕国云 ， 刘诗雅 . 2022 . 单幅人脸图像的全景纹理图生成方法 . 中国图象图形学报 ， 27 （ 2 ）： 602 - 613 ［ DOI： 10.11834/jig.210332 http://dx.doi.org/10.11834/jig.210332 ］
Lu D N ， Tao C Y ， Chen J Y ， Li F ， Guo F and Carin L . 2020 . Reconsidering generative objectives for counterfactual reasoning // Proceedings of the 34th International Conference on Neural Information Processing Systems . Vancouver， Canada ： Curran Associates， Inc.： 1808
Mulyadi A W ， Jung W ， Oh K ， Yoon J S and Suk H I . 2022 . XADLiME： eXplainable Alzheimer’s disease likelihood map estimation via clinically-guided prototype learning ［EB/OL］. ［ 2023-04-12 ］. https://arxiv.org/pdf/2207.13223.pdf https://arxiv.org/pdf/2207.13223.pdf
Oh K ， Yoon J S and Suk H I . 2020 . Born identity network： multi-way counterfactual map generation to explain a classifier’s decision ［EB/OL］. ［ 2023-04-12 ］. https://arxiv.org/pdf/2011.10381.pdf https://arxiv.org/pdf/2011.10381.pdf
Pfohl S R ， Duan T ， Ding D Y and Shah N H . 2019 . Counterfactual reasoning for fair clinical risk prediction // Proceedings of the 4th Machine Learning for Healthcare Conference . Ann Arbor， USA ： PMLR： 325 - 358
Qayyum A ， Ang C K ， Sridevi S ， Khan M K A A ， Hong L W ， Mazher M and Chung T D . 2020 . Hybrid 3D-ResNet deep learning model for automatic segmentation of thoracic organs at risk in CT images // Proceedings of 2020 International Conference on Industrial Engineering， Applications and Manufacturing （ICIEAM） . Sochi， Russia ： IEEE： 1 - 5 ［ DOI： 10.1109/ICIEAM48468.2020.9111950 http://dx.doi.org/10.1109/ICIEAM48468.2020.9111950 ］
Qin Z W ， Liu Z ， Guo Q H and Zhu P . 2022 . 3D convolutional neural networks with hybrid attention mechanism for early diagnosis of Alzheimer’s disease . Biomedical Signal Processing and Control ， 77 ： # 103828 ［ DOI： 10.1016/j.bspc.2022.103828 http://dx.doi.org/10.1016/j.bspc.2022.103828 ］
Selvaraju R R ， Cogswell M ， Das A ， Vedantam R ， Parikh D and Batra D . 2017 . Grad-CAM： visual explanations from deep networks via gradient-based localization // Proceedings of 2017 IEEE International Conference on Computer Vision （ICCV） . Venice， Italy ： IEEE： 618 - 626 ［ DOI： 10.1109/ICCV.2017.74 http://dx.doi.org/10.1109/ICCV.2017.74 ］
Shi J ， Zheng X ， Li Y ， Zhang Q and Ying S H . 2018 . Multimodal neuroimaging feature learning with multimodal stacked deep polynomial networks for diagnosis of Alzheimer’s disease . IEEE Journal of Biomedical and Health Informatics ， 22 （ 1 ）： 173 - 183 ［ DOI： 10.1109/JBHI.2017.2655720 http://dx.doi.org/10.1109/JBHI.2017.2655720 ］
Siddique N ， Paheding S ， Elkin C P and Devabhaktuni V . 2021 . U-Net and its variants for medical image segmentation： a review of theory and applications . IEEE Access ， 9 ： 82031 - 82057 ［ DOI： 10.1109/ACCESS.2021.3086020 http://dx.doi.org/10.1109/ACCESS.2021.3086020 ］
Singh A ， Sengupta S and Lakshminarayanan V . 2020 . Explainable deep learning models in medical image analysis . Journal of Imaging ， 6 （ 6 ）： # 52 ［ DOI： 10.3390/jimaging6060052 http://dx.doi.org/10.3390/jimaging6060052 ］
Springenberg J T ， Dosovitskiy A ， Brox T and Riedmiller M . 2015 . Striving for simplicity： the all convolutional net // Proceedings of the 3rd International Conference on Learning Representations . San Diego， USA ： ICLR
Sundararajan M ， Taly A and Yan Q Q . 2017 . Axiomatic attribution for deep networks // Proceedings of the 34th International Conference on Machine Learning . Sydney， Australia ： JMLR.org： 3319 - 3328
Tang H ， Liu H ， Xu D ， Torr P H S and Sebe N . 2023 . AttentionGAN： unpaired image-to-image translation using attention-guided generative adversarial networks . IEEE Transactions on Neural Networks and Learning Systems ， 34 （ 4 ）： 1972 - 1987 ［ DOI： 10.1109/TNNLS.2021.3105725 http://dx.doi.org/10.1109/TNNLS.2021.3105725 ］
Wang H F ， Shen Y Y ， Wang S Q ， Xiao T F ， Deng L M ， Wang X Y and Zhao X Y . 2019 . Ensemble of 3D densely connected convolutional network for diagnosis of mild cognitive impairment and Alzheimer’s disease . Neurocomputing ， 333 ： 145 - 156 ［ DOI： 10.1016/j.neucom.2018.12.018 http://dx.doi.org/10.1016/j.neucom.2018.12.018 ］
Woolrich M W ， Jbabdi S ， Patenaude B ， Chappell M ， Makni S ， Behrens T ， Beckmann C ， Jenkinson M and Smith S M . 2009 . Bayesian analysis of neuroimaging data in FSL . NeuroImage ， 45 （ 1S1 ）： S173 - S186 ［ DOI： 10.1016/j.neuroimage.2008.10.055 http://dx.doi.org/10.1016/j.neuroimage.2008.10.055 ］
Wu J P ， Zhao K ， Li Z Z ， Wang D ， Ding Y H ， Wei Y B ， Zhang H and Liu Y . 2022 . A systematic analysis of diagnostic performance for Alzheimer’s disease using structural MRI . Psychoradiology ， 2 （ 1 ）： 1 - 9 ［ DOI： 10.1093/psyrad/kkac001 http://dx.doi.org/10.1093/psyrad/kkac001 ］
Wu L W ， Sun R ， Kan J S and Gao J . 2020 . Double dual generative adversarial networks for cross-age sketch-to-photo translation . Journal of Image and Graphics ， 25 （ 4 ）： 732 - 744
吴柳玮 ， 孙锐 ， 阚俊松 ， 高隽 . 2020 . 双重对偶生成对抗网络的跨年龄素描—照片转换 . 中国图象图形学报 ， 25 （ 4 ）： 732 - 744 ［ DOI： 10.11834/jig.190329 http://dx.doi.org/10.11834/jig.190329 ］
Yadav S S and Sutar S R . 2021 . Alzheimer’s disease diagnosis using structural MRI and machine learning techniques // Proceedings of Machine Vision and Augmented Intelligence —— Theory and Applications . Singapore， Singapore ： Springer： 645 - 665 ［ DOI： 10.1007/978-981-16-5078-9_53 http://dx.doi.org/10.1007/978-981-16-5078-9_53 ］
Zhang D Q ， Wang Y P ， Zhou L P ， Yuan H and Shen D G . 2011 . Multimodal classification of Alzheimer’s disease and mild cognitive impairment . NeuroImage ， 55 （ 3 ）： 856 - 867 ［ DOI： 10.1016/j.neuroimage.2011.01.008 http://dx.doi.org/10.1016/j.neuroimage.2011.01.008 ］
Zhu J Y ， Park T ， Isola P and Efros A A . 2017 . Unpaired image-to-image translation using cycle-consistent adversarial networks // Proceedings of 2017 IEEE International Conference on Computer Vision （ICCV） . Venice， Italy ： IEEE： 2242 - 2251 ［ DOI： 10.1109/ICCV.2017.244 http://dx.doi.org/10.1109/ICCV.2017.244 ］
相关作者
相关机构
京公网安备11010802024621