浏览全部资源
扫码关注微信
Published： 16 March 2024 ，
移动端阅览
陆森良， 冯宝， 徐坤财， 陈业航， 陈相猛. 2024. 自适应迁移鲁棒特征的个性化联邦医学图像分类. 中国图象图形学报， 29(03):0798-0810
Lu Senliang， Feng Bao， Xu Kuncai， Chen Yehang， Chen Xiangmeng. 2024. Personalized federated medical image classification with adaptive transfer robust features. Journal of Image and Graphics， 29(03):0798-0810
目的
2
针对联邦学习中多中心医学数据的异质性特征导致全局模型性能不佳的问题，提出一种基于特征迁移的自适应个性化联邦学习算法（adaptive personalized federated learning via feature transfer， APFFT）。
方法
2
首先，为降低全局模型中异质性特征信息影响，提出鲁棒特征选择网络（robust feature selection network， RFS-Net）构建个性化本地模型。RFS-Net通过学习两个迁移权重分别确定全局模型向本地模型迁移时的有效特征以及特征迁移的目的地，并构建基于迁移权重的迁移损失函数以加强本地模型对全局模型中有效特征的注意力，从而构建个性化本地模型。然后，为过滤各本地模型中异质性特征信息，利用自适应聚合网络（adaptive aggregation network， AA-Net）聚合全局模型。AA-Net基于全局模型交叉熵变化更新迁移权重并构建聚合损失，使各本地模型向全局模型迁移鲁棒特征，提高全局模型的特征表达能力。
结果
2
在3种医学图像分类任务上与4种现有方法进行比较实验，在肺结核肺腺癌分类任务中，各中心曲线下面积（area under the curve，AUC）分别为0.791 5， 0.798 1， 0.760 0， 0.705 7和0.806 9；在乳腺癌组织学图像分类任务中，各中心准确率分别为0.984 9、0.980 8、0.983 5、0.982 6和0.983 4；在肺结节良恶性分类任务中，各中心AUC分别为 0.809 7， 0.849 8， 0.784 8和0.792 3。
结论
2
所提出的联邦学习方法，降低了多中心的异质性特征影响，实现基于鲁棒特征的个性化本地模型自适应构建和全局模型自适应聚合，模型性能有较大提升。
Objective
2
Patient data cannot be shared among medical institutions due to medical data confidentiality regulations， considerably limiting data scale. Federated learning ensures that all clients can train local models and aggregate global models in a decentralized manner without sharing data. However， the heterogeneity of medical data substantially affects the aggregation and deployment of global models in federated learning. In most federated learning methods， the aggregation of global model parameters is achieved by multiplying the fixed weight with the local model parameters and then summing them. The local model personalization method requires a large number of manual experiments to select the appropriate model layer for personalization construction. Although these methods can realize the aggregation of global models or the construction of personalized local models， they cannot automatically aggregate global model parameters and construct personalized local models. Moreover， they lack pertinence to heterogeneity characteristics. Therefore， an adaptive personalized federated learning algorithm via feature transfer （APFFT） is proposed. This algorithm can automatically identify and select robust features for personalized local model construction and global model aggregation. It can also suppress and filter heterogeneous feature information.
Method
2
To construct a personalized local model， a robust feature selection network （RFS-Net） was proposed in this study. RFS-Net can automatically identify and select features by calculating transfer weights and the amount of feature transfer on the basis of model representation. When transferring features from a global model to a local model， RFS-Net constructs transfer loss functions on the basis of transfer weights and the amount of feature transfer to constrain the local model and strengthen its attention toward effective transfer features. In the aggregation of the global model， the adaptive aggregation network （AA-Net） was proposed to transfer features from the local model to the global model. AA-Net updated the transfer weight and constructed the aggregation loss on the basis of the cross-entropy change of the global model for filtering the heterogeneity feature information of each local model. In this study， PyTorch was used to build and train the models， while ResNet18 was used for the convolutional neural network （CNN） structure. RFS-Net and AA-Net were composed of fully connected， pooling， softmax， and ReLU6 layers. The parameters of RFS-Net， AA-Net， and the CNN were updated via stochastic gradient descent with a momentum of 0.9. Experiments were conducted on three medical image datasets： the nonpublic dataset of pulmonary adenocarcinoma and tuberculosis classification， the public dataset Camelyon17， and the public dataset LIDC. The dataset of pulmonary adenocarcinoma tuberculosis classification came from 5 hospitals， with 1 009 cases. Among which， Center 1 （training set
n
= 260， test setn = 242）， Center 2 （training set
n
= 34， test set
n
= 54）， Center 3 （training set
n
= 39， test set
n
= 40）， Center 4 （training set
n
= 145， test set
n
= 108）， and Center 5 （training set
n
= 36， test set
n
= 51） were used in the experiment. The learning rate and decay rate of RFS-Net and AA-Net were both 0.000 1， while the learning rate and decay rate of the CNN were 0.001 and 0.000 5， respectively. Focal loss was used to calculate cross-entropy. In addition， gender， age， and nodule size in clinical information are of considerable reference value in the diagnosis of tuberculosis and lung adenocarcinoma. Therefore， we provided statistics for this information， and the results showed that in Center 2， the overall age and nodule size were small， while in Center 4， the overall nodule size was large， exhibiting a certain gap with the global average level. Camelyon17 was composed of 450 000 histological images from 5 hospitals. In the experiment， the learning rate and decay rate of the CNN， RFS-Net， and AA-Net were all 0.000 1. Standard cross-entropy was used to constrain CNN training. LIDC data came from 7 research institutions and 8 medical image companies， with 1 018 cases. Lesions with Grades 1 to 2 malignancies were classified as benign， while those with Grades 4 to 5 malignancies were classified as malignant. Finally， 1 746 lesions were included in the dataset to simulate the federated learning application scenario. The lesions were then randomly divided into 4 centers in accordance with the cases. Center 1 （training set
n
= 254， test set
n
= 169）， Center 2 （training set
n
= 263， test set
n
= 190）， Center 3 （training set
n
= 305， test set
n
= 124）， and Center 4 （training set
n
= 247， test set
n
= 194） were used in the experiment. The learning rate and decay rate of RFS-Net and AA-Net were both 0.000 1. The learning rate and decay rate of the CNN were 0.001 and 0.000 1， respectively. The cross-entropy loss was calculated using standard cross-entropy.
Result
2
Three types of medical image classification tasks were compared with four existing methods. The evaluation indexes included receiver operating characteristic （ROC） and accuracy. The experimental results showed that in the tuberculosis lung adenocarcinoma classification task， the center test sets of the end-to-end area under the ROC curve （AUC） were 0.791 5， 0.798 1， 0.76， 0.705 7， and 0.806 9. In the breast cancer histological image classification task， the center test sets of end-to-end accuracy were 0.984 9， 0.980 8， 0.983 5， 0.982 6， and 0.983 4. In the pulmonary nodule benign and malignancy classification task， the center test sets of the end-to-end AUC were 0.809 7， 0.849 8， 0.784 8， and 0.792 3.
Conclusion
2
The federated learning method proposed in this study can reduce the influence of heterogeneous characteristics and realize the adaptive construction of personalized local models and the adaptive aggregation of global models. The results show that our model is superior to several existing federated learning methods， and model performance is considerably improved.
特征迁移联邦学习异质性特征鲁棒特征选择网络自适应聚合网络医学图像分类
feature transferfederated learningheterogeneity featuresrobust feature selection networkadaptive aggregation networkmedical image classification
Bandi P， Geessink O， Manson Q， van Dijk M， Balkenhol M， Hermsen M， Ehteshami Bejnordi B， Lee B， Paeng K， Zhong A X， Li Q Z， Zanjani F G， Zinger S， Fukuta K， Komura D， Ovtcharov V， Cheng S H， Zeng S Q， Thagaard J， Dahl A B， Lin H J， Chen H， Jacobsson L， Hedlund M， Cet alin M， Halici E， Jackson H， Chen R， Both F， Franke J， Kusters-Vandevelde H， Vreuls W， Bult P， Van Ginneken B， Van Der Laak J and Litjens G. 2019. From detection of individual metastases to classification of lymph node status at the patient level： the CAMELYON17 challenge. IEEE Transactions on Medical Imaging， 38（2）： 550-560 ［DOI： 10.1109/TMI.2018.2867350http://dx.doi.org/10.1109/TMI.2018.2867350］
Chen H Y， Gao J Y， Zhao D， Wang H Z， Song H and Su Q H. 2021. Review of the research progress in deep learning and biomedical image analysis till 2020. Journal of Image and Graphics， 26（3）： 475-486
陈弘扬， 高敬阳， 赵地， 汪红志， 宋红， 苏庆华. 2021. 深度学习与生物医学图像分析2020年综述. 中国图象图形学报， 26（3）： 475-486 ［DOI： 10.11834/jig.200351http://dx.doi.org/10.11834/jig.200351］
Chen Y Q， Lu W， Qin X， Wang J D and Xie X. 2023. MetaFed： federated learning among federations with cyclic knowledge distillation for personalized healthcare ［EB/OL］. ［2023-03-12］. https://arxiv.org/pdf/2206.08516.pdfhttps://arxiv.org/pdf/2206.08516.pdf
Deng Y Y， Kamani M M and Mahdavi M. 2020. Adaptive personalized federated learning［EB/OL］. ［2023-03-12］. https://arxiv.org/pdf/2003.13461v3.pdfhttps://arxiv.org/pdf/2003.13461v3.pdf
He K M， Zhang X Y， Ren S Q and Sun J. 2016. Deep residual learning for image recognition//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas， USA： IEEE： 770-778 ［DOI： 10.1109/CVPR.2016.90http://dx.doi.org/10.1109/CVPR.2016.90］
Huang Y T， Chu L Y， Zhou Z R， Wang L J， Liu J C， Pei J and Zhang Y. 2021. Personalized cross-Silo federated learning on Non-IID Data. Proceedings of the AAAI Conference on Artificial Intelligence， 35（9）： 7865-7873 ［DOI： 10.1609/aaai.v35i9.16960http://dx.doi.org/10.1609/aaai.v35i9.16960］
Jang Y， Lee H， Hwang S J and Shin J. 2019. Learning what and where to transfer ［EB/OL］. ［2023-03-12］. https://arxiv.org/pdf/1905.05901.pdfhttps://arxiv.org/pdf/1905.05901.pdf
Jiang M R， Wang Z R and Dou Q. 2022. HarmoFL： harmonizing local and global drifts in federated learning on heterogeneous medical images. Proceedings of the AAAI Conference on Artificial Intelligence， 36（1）： 1087-1095 ［DOI： 10.1609/aaai.v36i1.19993http://dx.doi.org/10.1609/aaai.v36i1.19993］
Kairouz P， McMahan H B， Avent B， Bellet A， Bennis M， Bhagoji A N， Bonawitz K， Charles Z， Cormode G， Cummings R， D’Oliveira R G L， Eichner H， Rouayheb S E， Evans D， Gardner J， Garrett Z， Gascón A， Ghazi B， Gibbons P B， Gruteser M， Harchaoui Z， He C Y， He L， Huo Z Y， Hutchinson B， Hsu J， Jaggi M， Javidi T， Joshi G， Khodak M， Konecný J， Korolova A， Koushanfar F， Koyejo S， Lepoint T， Liu Y， Mittal P， Mohri M， Nock R， Özgür A， Pagh R， Qi H， Ramage D， Raskar R， Raykova M， Song D， Song W K， Stich S U， Sun Z T， Suresh A T， Tramèr F， Vepakomma P， Wang J Y， Xiong L， Xu Z， Yang Q， Yu F X， Yu H and Zhao S. 2021. Advances and open problems in federated learning. Foundations and Trends® in MachineLearning， 14（1-2）： 1-210 ［DOI： 10.1561/2200000083http://dx.doi.org/10.1561/2200000083］
Li F F， Deng J and Li K. 2010. ImageNet： constructing a large-scale image database. Journal of Vision， 9（8）： 1037-1037 ［DOI： 10.1167/9.8.1037http://dx.doi.org/10.1167/9.8.1037］
Li Q B， He B S and Song D. 2021a. Model-contrastive federated learning//Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Nashville， USA： IEEE： 10708-10717 ［DOI： 10.1109/CVPR46437.2021.01057http://dx.doi.org/10.1109/CVPR46437.2021.01057］
Li T， Sahu A K， Zaheer M， Sanjabi M， Talwalkar A and Smith V. 2020. Federated optimization in heterogeneous networks ［EB/OL］. ［2023-03-12］. https://arxiv.org/pdf/1812.06127.pdfhttps://arxiv.org/pdf/1812.06127.pdf
Li W Q， Milletarì F， Xu D G， Rieke N， Hancox J， Zhu W T， Baust M， Cheng Y， Ourselin S， Cardoso M J and Feng A. 2019. Privacy-preserving federated brain tumour segmentation//10th International Workshop on Machine Learning in Medical Imaging. Shenzhen， China： Springer： 133-141 ［DOI： 10.1007/978-3-030-32692-0_16http://dx.doi.org/10.1007/978-3-030-32692-0_16］
Li X X， Jiang M R， Zhang X F， Kamp M and Dou Q. 2021b. FedBN： federated learning on non-iid features via local batch normalization ［EB/OL］. ［2023-03-12］. https://arxiv.org/pdf/2102.07623.pdfhttps://arxiv.org/pdf/2102.07623.pdf
Li Y and Song P H. 2022. Review of transfer learning in medical image classification. Journal of Image and Graphics， 27（3）： 672-686
黎英， 宋佩华. 2022. 迁移学习在医学图像分类中的研究进展. 中国图象图形学报， 27（3）： 672-686 ［DOI： 10.11834/jig.210814http://dx.doi.org/10.11834/jig.210814］
Lin T Y， Goyal P， Girshick R， He K M and Dollr P. 2020. Focal loss for dense object detection. IEEE Transactions on Pattern Analysis and Machine Intelligence， 42（2）： 318-327 ［DOI： 10.1109/TPAMI.2018.2858826http://dx.doi.org/10.1109/TPAMI.2018.2858826］
McMahan H B， Moore E， Ramage D， Hampson S and Arcas B A. 2023. Communication-efficient learning of deep networks from decentralized data ［EB/OL］. ［2023-03-12］. https://arxiv.org/pdf/1602.05629.pdfhttps://arxiv.org/pdf/1602.05629.pdf
McNitt-Gray M F， Armato S G III， Meyer C R， Reeves A P， Mclennan G， Pais R C， Freymann J， Brown M S， Engelmann R M， Bland P H， Laderach G E， Piker C， Guo J F， Towfic Z， Qing D P Y， Yankelevitz D F， Aberle D R， Van Beek E J R， Macmahon H， Kazerooni E A， Croft B Y and Clarke L P. 2007. The lung image database consortium （LIDC） data collection process for nodule detection and annotation. Academic Radiology， 14（12）： 1464-1474 ［DOI： 10.1016/j.acra.2007.07.021http://dx.doi.org/10.1016/j.acra.2007.07.021］
Razavian A S， Azizpour H， Sullivan J and Carlsson S. 2014. CNN features off-the-shelf： an astounding baseline for recognition//Proceedings of 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops. Columbus， USA： IEEE： 512-519 ［DOI： 10.1109/CVPRW.2014.131http://dx.doi.org/10.1109/CVPRW.2014.131］
Romero A， Ballas N， Kahou S E， Chassang A， Gatta C and Bengio Y. 2015. FitNets： hints for thin deep nets ［EB/OL］. ［2023-03-12］. https://arxiv.org/pdf/1412.6550.pdfhttps://arxiv.org/pdf/1412.6550.pdf
Schneider J and Vlachos M. 2020. Personalization of deep learning ［EB/OL］. ［2023-03-12］. https://arxiv.org/pdf/1909.02803.pdfhttps://arxiv.org/pdf/1909.02803.pdf
Wang K K， Mathews R， Kiddon C， Eichner H， Beaufays F and Ramage D. 2019. Federated evaluation of on-device personalization ［EB/OL］. ［2023-03-12］. https://arxiv.org/pdf/1910.10252.pdfhttps://arxiv.org/pdf/1910.10252.pdf
相关文章
相关作者
相关机构
京公网安备11010802024621