浏览全部资源
扫码关注微信
Published： 16 July 2024 ，
移动端阅览
石程， 刘莹， 赵明华， 苗启广， 潘治文. 2024. 面向高光谱图像分类网络的对比半监督对抗训练方法. 中国图象图形学报， 29(07):1861-1874
Shi Cheng， Liu Ying， Zhao Minghua， Miao Qiguang， Chi-Man Pun. 2024. Contrastive semi-supervised adversarial training method for hyperspectral image classification networks. Journal of Image and Graphics， 29(07):1861-1874
目的
2
深度神经网络在高光谱图像分类任务中表现出明显的优越性，但是对抗样本的出现使其鲁棒性受到严重威胁，对抗训练方法为深度神经网络提供了一种有效的保护策略，但是在有限标记样本下提高目标网络的鲁棒性和泛化能力仍然需要进一步研究。为此，本文提出了一种面向高光谱图像分类网络的对比半监督对抗训练方法。
方法
2
首先，根据少量标记样本预训练目标模型，并同时利用少量标记样本和大量无标记样本构建训练样本集合；然后，通过最大化训练样本集合中干净样本和对抗样本在目标模型上的特征差异生成高迁移性对抗样本；最后，为了减少对抗训练过程对样本标签的依赖以及提高目标模型对困难对抗样本的学习和泛化能力，充分利用目标模型和预训练模型的输出层及中间层特征，构建对比对抗损失函数对目标模型进行优化，提高目标模型的对抗鲁棒性。对抗样本生成和目标网络优化过程交替进行，并且不需要样本标签的参与。
结果
2
在PaviaU和Indian Pines两组高光谱图像数据集上与主流的5种对抗训练方法进行了比较，本文方法在防御已知攻击和多种未知攻击上均表现出明显的优越性。面对6种未知攻击，相比于监督对抗训练方法AT（adversarial training）和TRADES（trade-off between robustness and accuracy），本文方法分类精度在两个数据集上平均提高了13.3%和16%，相比于半监督对抗训练方法SRT（semi-supervised robust training）、RST（robust self-training）和MART（misclassification aware adversarial risk training），本文方法分类精度再两个数据集上平均提高了5.6%和4.4%。实验结果表明了提出模型的有效性。
结论
2
本文方法能够在少量标记样本下提高高光谱图像分类网络的防御性能。
Objective
2
Deep neural networks have demonstrated significant superiority in hyperspectral image classification tasks. However， the emergence of adversarial examples poses a serious threat to their robustness. Research on adversarial training methods provides an effective defense strategy for protecting deep neural networks. However， existing adversarial training methods often require a large number of labeled examples to enhance the robustness of deep neural networks， which increases the difficulty of labeling hyperspectral image examples. In addition， a critical limitation of current adversarial training approaches is that they usually do not capture intermediate layer features in the target network and pay less attention to challenging adversarial samples. This oversight can lead to the reduced generalization ability of the defense model. To further enhance the adversarial robustness of hyperspectral image classification networks with limited labeled examples， this paper proposes a contrastive semi-supervised adversarial training method.
Method
2
First， the target model is pre-trained using a small number of labeled examples. Second， for a large number of unlabeled examples， the corresponding adversarial examples are generated by maximizing the feature difference between clean unlabeled examples and adversarial examples on the target model. Adversarial samples generated using intermediate layer features of the network exhibit higher transferability compared with those generated only using output layer features. In contrast， feature-based adversarial sample generation methods do not rely on example labels. Therefore， we generate adversarial examples based on the intermediate layer features of the network. Third， the generated adversarial examples are used to enhance the robustness of the target model. The defense capabilities of the target model for the challenging adversarial samples are enhanced by defining the robust upper bound and robust lower bound of the target network based on the pre-trained target model， and a contrastive adversarial loss is designed on both intermediate feature layer and output layer to optimize the model based on the defined robust upper bound and robust lower bound. The defined contrastive loss function consists of three terms： classification loss， output contrastive loss， and feature contrastive loss. The classification loss is designed to maintain the classification accuracy of the target model for clean examples. The output contrastive loss encourages the output layer of the adversarial examples to move closer to the pre-defined output layer robust upper bound and away from the pre-defined output layer robust lower bound. The feature contrastive loss pushes the intermediate layer feature of the adversarial example closer to the pre-defined intermediate robust upper bound and away from the pre-defined intermediate robust lower bound. The proposed output contrastive adversarial loss and feature contrastive loss help improve the classification accuracy and generalization ability of the target network against challenging adversarial examples. The training process of adversarial example generation and target network optimization is performed iteratively， and example labels are not required in the training process. By incorporating a limited number of labeled examples in model training， both the output layer and intermediate feature layer are used to enhance the defense ability of the target model against known and unknown attack methods.
Result
2
We compared the proposed method with five mainstream adversarial training methods， two supervised adversarial training methods and three semi-supervised adversarial training methods， on the PaviaU and Indian Pines hyperspectral image datasets. Compared with the mainstream adversarial training methods， the proposed method demonstrates significant superiority in defending against both known and various unknown attacks. Faced with six unknown attacks， compared with the supervised adversarial training methods AT and TRADES， our method showed an average improvement in classification accuracy of 13.3% and 16%， respectively. Compared with the semi-supervised adversarial training methods SRT， RST， and MART， our method achieved an average improvement in classification accuracy of 5.6% and 4.4%， respectively. Compared with the target model without defense method， for example on the Inception_V3， the defense performance of the proposed method in the face of different attacks improved by 34.63%–92.78%.
Conclusion
2
The proposed contrastive semi-supervised adversarial training method can improve the defense performance of hyperspectral image classification networks with limited labeled examples. By maximizing the feature distance between clean examples and adversarial examples on the target model， we can generate highly transferable adversarial examples. To address the limitation of defense generalization ability imposed by the number of labeled examples， we define the concept of robust upper bound and robust lower bound based on the pre-trained target model and design an optimization model according to a contrastive semi-supervised loss function. By extensively leveraging the feature information provided by a few labeled examples and incorporating a large number of unlabeled examples， we can further enhance the generalization ability of the target model. The defense performance of the proposed method is superior to that of the supervised adversarial training methods.
对抗防御高光谱图像分类半监督学习深度神经网络对抗攻击
adversarial defensehyperspectral image classificationsemi-supervised learningdeep neural networkadversarial attack
Carlini N and Wagner D. 2017. Towards evaluating the robustness of neural networks//2017 IEEE Symposium on Security and Privacy （SP）. San Jose， USA： IEEE： 39-57 ［DOI： 10.1109/SP.2017.49http://dx.doi.org/10.1109/SP.2017.49］
Carmon Y， Raghunathan A， Schmidt L， Liang P and Duchi J C. 2019. Unlabeled data improves adversarial robustness//Proceedings of the 33rd International Conference on Neural Information Processing Systems. Vancouver， Canada： Curran Associates Inc.： 11192-11203
Croce F and Hein M. 2020. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks//Proceedings of the 37th International Conference on Machine Learning. Virtual， Online： JMLR.org： 2206-2216
Dong Y P， Liao F Z， Pang T Y， Su H， Zhu J， Hu X L and Li J G. 2018. Boosting adversarial attacks with momentum//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City， USA： IEEE： 9185-9193 ［DOI： 10.1109/CVPR.2018.00957http://dx.doi.org/10.1109/CVPR.2018.00957］
Dong Y P， Pang T Y， Su H and Zhu J. 2019. Evading defenses to transferable adversarial examples by translation-invariant attacks//Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach， USA： IEEE： 4307-4316 ［DOI： 10.1109/CVPR.2019.00444http://dx.doi.org/10.1109/CVPR.2019.00444］
Eykholt K， Evtimov I， Fernandes E， Li B， Rahmati A， Xiao C W， Prakash A， Kohno T and Song D. 2018. Robust physical-world attacks on deep learning visual classification//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City， USA： IEEE： 1625-1634 ［DOI： 10.1109/CVPR.2018.00175http://dx.doi.org/10.1109/CVPR.2018.00175］
Goodfellow I J， Shlens J and Szegedy C. 2015. Explaining and harnessing adversarial examples ［EB/OL］. ［2023-07-16］. http://arxiv.org/pdf/1412.6572.pdfhttp://arxiv.org/pdf/1412.6572.pdf
He K M， Zhang X Y， Ren S Q and Sun J. 2016. Deep residual learning for image recognition//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas， USA： IEEE： 770-778 ［DOI： 10.1109/CVPR.2016.90http://dx.doi.org/10.1109/CVPR.2016.90］
Hendrycks D， Lee K and Mazeika M. 2019. Using pre-training can improve model robustness and uncertainty//Proceedings of the 36th International Conference on Machine Learning. Long Beach， USA： ICML： 2712-2721
Huang Q， Katsman I， Gu Z Q， He H， Belongie S and Lim S N. 2019. Enhancing adversarial example transferability with an intermediate level attack//Proceedings of 2019 IEEE/CVF International Conference on Computer Vision. Seoul， Korea （South）： IEEE： 4732-4741 ［DOI： 10.1109/ICCV.2019.00483http://dx.doi.org/10.1109/ICCV.2019.00483］
Jin G Q， Shen S W， Zhang D M， Dai F and Zhang Y D. 2019. APE-GAN： adversarial perturbation elimination with GAN//Proceedings of 2019 IEEE International Conference on Acoustics， Speech and Signal Processing. Brighton， UK： IEEE： 3842-3846 ［DOI：10.1109/ICASSP.2019.8683044http://dx.doi.org/10.1109/ICASSP.2019.8683044］
Kang X D， Duan P H， Xiang X L， Li S T and Benediktsson J A. 2018. Detection and correction of mislabeled training samples for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing， 56（10）： 5673-5686 ［DOI： 10.1109/TGRS.2018.2823866http://dx.doi.org/10.1109/TGRS.2018.2823866］
Kurakin A， Goodfellow I and Bengio S. 2017. Adversarial examples in the physical world ［EB/OL］. ［2023-07-16］. http://arxiv.org/pdf/1607.02533.pdfhttp://arxiv.org/pdf/1607.02533.pdf
Lamb A， Verma V， Kawaguchi K， Matyasko A， Khosla S， Kannala J and Bengio Y. 2022. Interpolated adversarial training： achieving robust neural networks without sacrificing too much accuracy. Neural Networks， 154： 218-233 ［DOI： 10.1016/j.neunet.2022.07.012http://dx.doi.org/10.1016/j.neunet.2022.07.012］
Li X and Li F X. 2017. Adversarial examples detection in deep networks with convolutional filter statistics//Proceedings of 2017 IEEE International Conference on Computer Vision. Venice， Italy： IEEE： 5775-5783 ［DOI： 10.1109/ICCV.2017.615http://dx.doi.org/10.1109/ICCV.2017.615］
Li Y M， Wu B Y， Feng Y， Fan Y B， Jiang Y， Li Z F and Xia S T. 2022. Semi-supervised robust training with generalized perturbed neighborhood. Pattern Recognition， 124： #108472 ［DOI： 10.1016/j.patcog.2021.108472http://dx.doi.org/10.1016/j.patcog.2021.108472］
Lin J D， Song C B， He K， Wang L W and Hopcroft J E. 2020. Nesterov accelerated gradient and scale invariance for adversarial attacks ［EB/OL］. ［2023-07-16］. http://arxiv.org/pdf/1908.06281.pdfhttp://arxiv.org/pdf/1908.06281.pdf
Machado G R， Silva E and Goldschmidt R R. 2021. Adversarial machine learning in image classification： a survey toward the defender’s perspective. ACM Computing Surveys， 55（1）： #8 ［DOI： 10.1145/3485133http://dx.doi.org/10.1145/3485133］
Madry A， Makelov A， Schmidt L， Tsipras D and Vladu A. 2019. Towards deep learning models resistant to adversarial attacks ［EB/OL］. ［2023-07-16］. http://arxiv.org/pdf/1706.06083.pdfhttp://arxiv.org/pdf/1706.06083.pdf
Miyato T， Maeda S I， Koyama M and Ishii S. 2019. Virtual adversarial training： a regularization method for supervised and semi-supervised learning. IEEE Transactions on Pattern Analysis and Machine Intelligence， 41（8）： 1979-1993 ［DOI： 10.1109/TPAMI.2018.2858821http://dx.doi.org/10.1109/TPAMI.2018.2858821］
Moosavi-Dezfooli S M， Fawzi A and Frossard P. 2016. DeepFool： a simple and accurate method to fool deep neural networks//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas， USA： IEEE： 2574-2582 ［DOI： 10.1109/CVPR.2016.282http://dx.doi.org/10.1109/CVPR.2016.282］
Naseer M， Khan S H， Rahman S and Porikli F. 2019. Task-generalizable adversarial attack based on perceptual metric ［EB/OL］. ［2023-07-16］. http://arxiv.org/pdf/1811.09020.pdfhttp://arxiv.org/pdf/1811.09020.pdf
Papernot N， Mcdaniel P， Wu X， Jha S and Swami A. 2016. Distillation as a defense to adversarial perturbations against deep neural networks//2016 IEEE Symposium on Security and Privacy （SP）. San Jose， USA： IEEE： 582-597 ［DOI： 10.1109/sp.2016.41http://dx.doi.org/10.1109/sp.2016.41］
Ren Y C， Zhu H G， Sui X Y and Liu C. 2023. Crafting transferable adversarial examples via contaminating the salient feature variance. Information Sciences， 644： #119273 ［DOI： 10.1016/j.ins.2023.119273http://dx.doi.org/10.1016/j.ins.2023.119273］
Shi C， Dang Y N， Fang L， Lyu Z Y and Zhao M H. 2022. Hyperspectral image classification with adversarial attack. IEEE Geoscience and Remote Sensing Letters， 19： #5510305 ［DOI： 10.1109/lgrs.2021.3122170http://dx.doi.org/10.1109/lgrs.2021.3122170］
Simonyan K and Zisserman A. 2015. Very deep convolutional networks for large-scale image recognition ［EB/OL］. ［2023-07-16］. http://arxiv.org/pdf/1409.1556.pdfhttp://arxiv.org/pdf/1409.1556.pdf
Szegedy C， Vanhoucke V， Ioffe S， Shlens J and Wojna Z. 2016. Rethinking the inception architecture for computer vision//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas， USA： IEEE： 2818-2826 ［DOI： 10.1109/CVPR.2016.308http://dx.doi.org/10.1109/CVPR.2016.308］
Szegedy C， Zaremba W， Sutskever I， Bruna J， Erhan D， Goodfellow I and Fergus R. 2014. Intriguing properties of neural networks ［EB/OL］. ［2023-07-16］. http://arxiv.org/pdf/1312.6199.pdfhttp://arxiv.org/pdf/1312.6199.pdf
Tan K， Wang X and Du P J. 2019. Research progress of the remote sensing classification combining deep learning and semi-supervised learning. Journal of Image and Graphics， 24（11）： 1823-1841
谭琨， 王雪， 杜培军. 2019. 结合深度学习和半监督学习的遥感影像分类进展. 中国图象图形学报， 24（11）： 1823-1841 ［DOI： 10.11834/jig.190348http://dx.doi.org/10.11834/jig.190348］
Tramèr F， Kurakin A， Papernot N， Goodfellow I， Boneh D and McDaniel P. 2020. Ensemble adversarial training： attacks and defenses ［EB/OL］. ［2023-07-16］. http://arxiv.org/pdf/1705.07204.pdfhttp://arxiv.org/pdf/1705.07204.pdf
Uesato J， Alayrac J B， Huang P S， Stanforth R， Fawzi A and Kohli P. 2019. Are labels required for improving adversarial robustness？//Proceedings of the 33rd International Conference on Neural Information Processing Systems. Vancouver， Canada： Curran Associates Inc.： 12214-12223
Wang X S and He K. 2021a. Enhancing the transferability of adversarial attacks through variance tuning//Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Nashville， USA： IEEE： 1924-1933 ［DOI： 10.1109/CVPR46437.2021.00196http://dx.doi.org/10.1109/CVPR46437.2021.00196］
Wang Y S， Zou D F， Yi J F， Bailey J， Ma X J and Gu Q Q. 2020. Improving adversarial robustness requires revisiting misclassified examples//Proceedings of the 8th International Conference on Learning Representations. Addis Ababa， Ethiopia： 1-14 ［Online］
Wang Z B， Guo H C， Zhang Z F， Liu W X， Qin Z and Ren K. 2021b. Feature importance-aware transferable adversarial attacks//Proceedings of 2021 IEEE/CVF International Conference on Computer Vision. Montreal， Canada： IEEE： 7619-7628 ［DOI： 10.1109/ICCV48922.2021.00754http://dx.doi.org/10.1109/ICCV48922.2021.00754］
Wu L， Zhu Z X， Tai C and E W N. 2018. Understanding and enhancing the transferability of adversarial examples ［EB/OL］. ［2023-07-16］. http://arxiv.org/pdf/1802.09707.pdfhttp://arxiv.org/pdf/1802.09707.pdf
Xie C H， Wang J Y， Zhang Z S， Ren Z and Yuille A. 2018. Mitigating adversarial effects through randomization ［EB/OL］. ［2018-02-28］. http://arxiv.org/pdf/1711.01991.pdfhttp://arxiv.org/pdf/1711.01991.pdf
Xie C H， Zhang Z S， Zhou Y Y， Bai S， Wang J Y， Ren Z and Yuille A L. 2019. Improving transferability of adversarial examples with input Diversity//Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach， USA： IEEE： 2725-2734 ［DOI： 10.1109/CVPR.2019.00284http://dx.doi.org/10.1109/CVPR.2019.00284］
Yang S S， Yang Y， Zhou L N， Zhan R and Man Y F. 2022. Intermediate-layer transferable adversarial attack with DNN attention. IEEE Access， 10： 95451-95461 ［DOI： 10.1109/access.2022.3204696http://dx.doi.org/10.1109/access.2022.3204696］
Yuan L， Li X M， Pan Z X， Sun J M and Xiao L. 2022. Review of adversarial examples for object detection. Journal of Image and Graphics， 27（10）： 2873-2896
袁珑， 李秀梅， 潘振雄， 孙军梅， 肖蕾. 2022. 面向目标检测的对抗样本综述. 中国图象图形学报， 27（10）： 2873-2896 ［DOI： 10.11834/jig.210209http://dx.doi.org/10.11834/jig.210209］
Zhang H Y， Yu Y D， Jiao J T， Xing E P， El Ghaoui L and Jordan M I. 2019. Theoretically principled trade-off between robustness and accuracy//Proceedings of the 36th International Conference on Machine Learning. Long Beach， USA： ICML： 7472-7482
Zhang J P， Wu W B， Huang J T， Huang Y Z， Wang W X， Su Y X and Lyu M R. 2022a. Improving adversarial transferability via neuron attribution-based attacks//Proceedings of 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New Orleans， USA： IEEE： 14973-14982 ［DOI： 10.1109/CVPR52688.2022.01457http://dx.doi.org/10.1109/CVPR52688.2022.01457］
Zhang X R， Chen S T， Zhu P， Tang X， Feng J and Jiao L C. 2022b. Spatial pooling graph convolutional network for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing， 60： #5521315 ［DOI： 10.1109/tgrs.2022.3140353http://dx.doi.org/10.1109/tgrs.2022.3140353］
Zhou D W， Liu T L， Han B， Wang N N， Peng C and Gao X. 2021. Towards defending against adversarial examples via attack-invariant features. InInternational Conference on Machine Learning. ICML： 12835-12845 ［DOI： 10.48550/arXiv.2106.0503http://dx.doi.org/10.48550/arXiv.2106.0503］
Zhu H G， Zheng H R， Zhu Y and Sui X Y. 2023. Boosting the transferability of adversarial attacks with adaptive points selecting in temporal neighborhood. Information Sciences， 641： #119081 ［DOI： 10.1016/j.ins.2023.119081http://dx.doi.org/10.1016/j.ins.2023.119081］
相关文章
相关作者
相关机构
京公网安备11010802024621