浏览全部资源
扫码关注微信
纸质出版日期： 2024-06-16 ，
移动端阅览
郑向涛， 肖欣林， 陈秀妹， 卢宛萱， 刘小煜， 卢孝强. 2024. 跨域遥感场景解译研究进展. 中国图象图形学报， 29(06):1730-1746
Zheng Xiangtao， Xiao Xinlin， Chen Xiumei， Lu Wanxuan， Liu Xiaoyu， Lu Xiaoqiang. 2024. Advancements in cross-domain remote sensing scene interpretation. Journal of Image and Graphics， 29(06):1730-1746
遥感对地观测中普遍存在多平台、多传感器和多角度的多源数据，为遥感场景解译提供协同互补信息。然而，现有的场景解译方法需要根据不同遥感场景数据训练模型，或者对测试数据标准化以适应现有模型，训练成本高、响应周期长，已无法适应多源数据协同解译的新阶段。跨域遥感场景解译将已训练的老模型迁移到新的应用场景，通过模型复用以适应不同场景变化，利用已有领域的知识来解决未知领域问题。本文以跨域遥感场景解译为主线，综合分析国内外文献，结合场景识别和目标识别两个典型任务，论述国内外研究现状、前沿热点和未来趋势，梳理总结跨域遥感场景解译的常用数据集和统一的实验设置。本文实验数据集及检测结果的公开链接为：
https://github.com/XiangtaoZheng/CDRSSI
https://github.com/XiangtaoZheng/CDRSSI
。
In remote sensing of Earth observation， multi-source data can be captured by multiple platforms， multiple sensors， and multiple perspectives. These data provide complementary information for interpreting remote sensing scenes. Although these data offer richer information， they also increase the demand for model depth and complexity. Deep learning plays a pivotal role in unlocking the potential of remote sensing data by delving deep into the semantic layers of scenes and extracting intricate features from images. Recent advancements in artificial intelligence have greatly enhanced this process. However， deep learning networks have limitations when applied to remote sensing images. 1） The huge number of parameters and the difficulty in training， as well as the over-reliance on labeled training data， can affect these images. Remote sensing images are characterized by “data miscellaneous marking difficulty”， which makes manual labeling insufficient for meeting the training needs of deep learning. 2） Variations in remote sensing platforms， sensors， shooting angles， resolution， time， location， and weather can all impact remote sensing images. Thus， the interpreted images and training samples cannot have the same distribution. This inconsistency results in weak generalization ability in existing models， especially when dealing with data from different distributions. To address this issue， cross-domain remote sensing scene interpretation aims to train a model on labeled remote sensing scene data （source domain） and apply it to new， unlabeled scene data （target domain） in an appropriate way. This approach reduces the dependence on target domain data and relaxes the assumption of the same distribution in existing deep learning tasks. The shallow layers of convolutional neural networks can be used as general-purpose feature extractors， but deeper layers are more task-specific and may introduce bias when applied to other tasks. Therefore， the migration model must be modified to accomplish the task of interpreting the target domain. Cross-domain interpretation tasks aim to establish a model that can adapt to various scene changes by utilizing migration learning， domain adaptation and other techniques for reducing model prediction inaccuracy caused by changes in the data domain. This approach improves the robustness and generalization ability of the model. Interpreting cross-domain remote sensing scenes typically requires using data from multiple remote sensing sources， including radar， aerial and satellite imagery. These images may have varying views， resolutions， wavelength bands， lighting conditions and noise levels. They may also originate from different locations or sensors. As the Global Earth Observation Systems continues to advance， remote sensing images now include cross-platform， cross-sensor， cross-resolution， and cross-region， which results in enormous distributional variances. Therefore， the study of cross-domain remote sensing scene interpretation is essential for the commercial use of remote sensing data and has theoretical and practical importance. This report categorizes scene decoding tasks into four main types based on the labeled set of data： methods based on closed-set domain adaptation， partial-domain adaptation， open-set domain adaptation and generalized domain adaptation. Approaches based on closed-set domain adaptation focus on tasks where the label set of the target domain is the same as that of the source domain. Partial domain adaptation focuses on tasks where the label set of the target domain is a subset of the source domain. Open-set domain adaptation aims to research tasks where the label set of the source domain is a subset of the label set of the target domain， and it does not apply restrictions in the approach of generalized domain adaptation. This study provides an in-depth investigation of two typical tasks in cross-domain remote sensing interpretation： scene recognition and target knowledge. The first part of the study utilizes domestic and international literature to provide a comprehensive assessment of the current research status of the four types of methods. Within the target recognition task， cross-domain tasks are further subdivided into cross-domain for visible light data and cross-domain from visible light to Synthetic Aperture Radar images. After a quantitative analysis of the sample distribution characteristics of different datasets， a unified experimental setup for cross-domain tasks is proposed. In the scene classification task， the dataset is explored by classifying it according to the label set categorization， and specific examples are given to provide the corresponding experimental setup for the readers’ reference. The fourth part of the study discusses the research trends in cross-domain remote sensing interpretation， which highlights four challenging research directions： few-shot learning， source domain data selection， multi-source domain interpretation， and cross-modal interpretation. These areas will be important directions for the future development of remote sensing scene interpretation， which offers potential choices for readers’ subsequent research directions.
跨域遥感场景解译分布外泛化模型泛化多样性数据集迁移学习自适应算法
cross-domain remote sensing scene interpretationout-of-distribution generalizationmodel generalizationdiverse datasetmigration learningadaptive algorithm
Al Rahhal M M， Bazi Y， Al-Hwiti H， Alhichri H and Alajlan N. 2021. Adversarial learning for knowledge adaptation from multiple remote sensing sources. IEEE Geoscience and Remote Sensing Letters， 18（8）： 1451-1455 ［DOI： 10.1109/LGRS.2020.3003566http://dx.doi.org/10.1109/LGRS.2020.3003566］
Chen S Q， Zhan R H， Wang W and Zhang J. 2022. Domain adaptation for semi-supervised ship detection in SAR images. IEEE Geoscience and Remote Sensing Letters， 19： #4507405 ［DOI： 10.1109/lgrs.2022.3171789http://dx.doi.org/10.1109/lgrs.2022.3171789］
Chen Y， Wang H R， Li W， Sakaridis C， Dai D X and Van Gool L. 2021. Scale-aware domain adaptive faster R-CNN. International Journal of Computer Vision， 129（7）： 2223-2243 ［DOI： 10.1007/s11263-021-01447-xhttp://dx.doi.org/10.1007/s11263-021-01447-x］
Cheng G， Han J W， Zhou P C and Guo L M. 2014. Multi-class geospatial object detection and geographic image classification based on collection of part detectors. ISPRS Journal of Photogrammetry and Remote Sensing， 98： 119-132 ［DOI： 10.1016/J.ISPRSJPRS.2014.10.002http://dx.doi.org/10.1016/J.ISPRSJPRS.2014.10.002］
Cheng G， Han J W and Lu X Q. 2017. Remote sensing image scene classification： Benchmark and State of the Art. Proceedings of the IEEE， 105（10）： 1865-1883
Du Y D， Feng L， Tao P， Gong X and Wang J. 2023. Meta-transfer learning in cross-domain image classification with few-shot learning. Journal of Image and Graphics， 28（9）： 2899-2912
杜彦东， 冯林， 陶鹏， 龚勋， 王俊. 2023. 元迁移学习在少样本跨域图像分类中的研究. 中国图象图形学报， 28（9）： 2899-2912 ［DOI： 10.11834/jig.220664http://dx.doi.org/10.11834/jig.220664］
Fu B， Cao Z J， Long M S and Wang J M. 2020. Learning to detect open classes for universal domain adaptation//Proceedings of the 16th European Conference on Computer Vision. Glasgow， UK： Springer： 567-583 ［DOI： 10.1007/978-3-030-58555-6_34http://dx.doi.org/10.1007/978-3-030-58555-6_34］
Gong T F， Zheng X T and Lu X Q. 2021. Cross-domain scene classification by integrating multiple incomplete sources. IEEE Transactions on Geoscience and Remote Sensing， 59（12）： 10035-10046 ［DOI： 10.1109/TGRS.2020.3034344http://dx.doi.org/10.1109/TGRS.2020.3034344］
Guo Q， Pu S L， Zhang S F and Li B. 2022. Cross-domain object detection based foggy image enhancement. Journal of Image and Graphics， 27（5）： 1481-1492
郭强， 浦世亮， 张世峰， 李波. 2022. 适合跨域目标检测的雾霾图像增强. 中国图象图形学报， 27（5）： 1481-1492 ［DOI： 10.11834/jig.210994http://dx.doi.org/10.11834/jig.210994］
He J J， Wu L， Tao C F and Lv F M. 2023. Source-free domain adaptation with unrestricted source hypothesis. Pattern Recognition， 149： #110246. ［DOI： 10.1016/j.patcog.2023.110246http://dx.doi.org/10.1016/j.patcog.2023.110246］
Huang W， Shi Y L， Xiong Z T， Wang Q and Zhu X X. 2023. Semi-supervised bidirectional alignment for remote sensing cross-domain scene classification. ISPRS Journal of Photogrammetry and Remote Sensing， 195： 192-203 ［DOI： 10.1016/j.isprsjprs.2022.11.013http://dx.doi.org/10.1016/j.isprsjprs.2022.11.013］
Isobe T， Jia X， Chen S J， He J Z， Shi Y J， Liu J Z， Lu H C and Wang S J. 2021. Multi-target domain adaptation with collaborative consistency learning//Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Nashville， USA： IEEE ［DOI： 10.1109/cvpr46437.2021.00809http://dx.doi.org/10.1109/cvpr46437.2021.00809］
Li H F， Jiang H， Gu X， Peng J， Li W B， Hong L and Tao C. 2020a. CLRS： continual learning benchmark for remote sensing image scene classification. Sensors， 20（4）： #1226 ［DOI： 10.3390/s20041226http://dx.doi.org/10.3390/s20041226］
Li J W， Qu C W and Shao J Q. 2017. Ship detection in SAR images based on an improved faster R-CNN//Proceedings of 2017 SAR in Big Data Era： Models， Methods and Applications. Beijing， China： IEEE： 1-6 ［DOI： 10.1109/BIGSARDATA.2017.8124934http://dx.doi.org/10.1109/BIGSARDATA.2017.8124934］
Li K， Wan G， Cheng G， Meng L Q and Han J W. 2020b. Object detection in optical remote sensing images： a survey and a new benchmark. ISPRS Journal of Photogrammetry and Remote Sensing， 159： 296-307 ［DOI： 10.1016/J.ISPRSJPRS.2019.11.023http://dx.doi.org/10.1016/J.ISPRSJPRS.2019.11.023］
Liu B， Li L G， Xiao Q， Ni W and Yang Z. 2022. Remote sensing fine-grained ship data augmentation pipeline with local-aware progressive image-to-image translation. IEEE Transactions on Geoscience and Remote Sensing， 60： #5631716 ［DOI： 10.1109/TGRS.2022.3211517http://dx.doi.org/10.1109/TGRS.2022.3211517］
Liu Z K， Yuan L， Weng L B and Yang Y P. 2017. A high resolution optical satellite image dataset for ship recognition and some new baselines//Proceedings of the 6th International Conference on Pattern Recognition Applications and Methods. Porto， Portugal： SciTePress： 324-331 ［DOI： 10.5220/0006120603240331http://dx.doi.org/10.5220/0006120603240331］
Lu X Q， Gong T F and Zheng X T. 2020. Multisource compensation network for remote sensing cross-domain scene classification. IEEE Transactions on Geoscience and Remote Sensing， 58（4）： 2504-2515 ［DOI： 10.1109/TGRS.2019.2951779http://dx.doi.org/10.1109/TGRS.2019.2951779］
Ma X H， Gao J Y and Xu C S. 2021. Active universal domain adaptation//Proceedings of 2021 IEEE/CVF International Conference on Computer Vision. Montreal， Canada： IEEE ［DOI： 10.1109/iccv48922.2021.00884http://dx.doi.org/10.1109/iccv48922.2021.00884］
Ngo B H， Chae Y J， Park J H， Kim J H and Cho S I. 2023. Easy-to-hard structure for remote sensing scene classification in multitarget domain adaptation. IEEE Transactions on Geoscience and Remote Sensing， 61： #4700215 ［DOI： 10.1109/TGRS.2023.3235886http://dx.doi.org/10.1109/TGRS.2023.3235886］
Ngo B H， Kim J H， Park S J and Cho S I. 2022. Collaboration between multiple experts for knowledge adaptation on multiple remote sensing sources. IEEE Transactions on Geoscience and Remote Sensing， 60： #4707815 ［DOI： 10.1109/TGRS.2022.3190476http://dx.doi.org/10.1109/TGRS.2022.3190476］
Nguyen-Meidine L T， Belal A， Kiran M， Dolz J， Blais-Morin L A and Granger E. 2021. Unsupervised multi-target domain adaptation through knowledge distillation//Proceedings of 2021 IEEE Winter Conference on Applications of Computer Vision. Waikoloa，USA： IEEE： 1339-1347 ［DOI： 10.1109/wacv48630.2021.00138http://dx.doi.org/10.1109/wacv48630.2021.00138］
Niu B， Pan Z X， Chen K Y， Hu Y X and Lei B. 2023. Open set domain adaptation via instance affinity metric and fine-grained alignment for remote sensing scene classification. IEEE Geoscience and Remote Sensing Letters， 20： #6005805 ［DOI： 10.1109/LGRS.2023.3276968http://dx.doi.org/10.1109/LGRS.2023.3276968］
Niu B， Pan Z X， Wu J X， Hu Y X and Lei B. 2022. Multi-representation dynamic adaptation network for remote sensing scene classification. IEEE Transactions on Geoscience and Remote Sensing， 60： #5633119 ［DOI： 10.1109/TGRS.2022.3217180http://dx.doi.org/10.1109/TGRS.2022.3217180］
Saha S， Zhao S and Zhu X X. 2022. Multitarget domain adaptation for remote sensing classification using graph neural network. IEEE Geoscience and Remote Sensing Letters， 19： #6506505 ［DOI： 10.1109/LGRS.2022.3149950http://dx.doi.org/10.1109/LGRS.2022.3149950］
Shi， R. Zhao， B. Pan， Z. Zou and Z. Shi. 2023. Unsupervised multimodal remote sensing image registration via domain adaptation. IEEE Transactions on Geoscience and Remote Sensing， 61， 1-11 ［DOI］： 10.1109/TGRS.2023.3333889.
Sun X， Wang Z R， Sun Y R， Diao W H， Zhang Y， Fu K. 2019. AIR-SARShip-1.0： High resolution SAR ship detection dataset. Journal of Radars， 8（6）： 852-862
孙显，王智睿，孙元睿，刁文辉，张跃，付琨.2019.AIR-SARShip-1.0：高分辨率SAR舰船检测数据集.雷达学报， 8（6）： 852-862 ［DOI： 10.12000/JR19097http://dx.doi.org/10.12000/JR19097］
Tasar O， Happy S L， Tarabalka Y and Alliez P. 2020. SEMI2I： semantically consistent image-to-image translation for domain adaptation of remote sensing data//2020 IEEE International Geoscience and Remote Sensing Symposium. Waikoloa， USA： IEEE： 1837-1840 ［DOI： 10.1109/IGARSS39084.2020.9323711http://dx.doi.org/10.1109/IGARSS39084.2020.9323711］
Wang K， Zhang G and Leung H. 2019a. SAR target recognition based on cross-domain and cross-task transfer learning. IEEE Access， 7： 153391-153399 ［DOI： 10.1109/ACCESS.2019.2948618http://dx.doi.org/10.1109/ACCESS.2019.2948618］
Wang Q， Liu S T， Chanussot J and Li X L. 2019b. Scene classification with recurrent attention of VHR remote sensing images. IEEE Transactions on Geoscience and Remote Sensing， 57（2）： 1155-1167 ［DOI： 10.1109/TGRS.2018.2864987http://dx.doi.org/10.1109/TGRS.2018.2864987］
Wang S Y， Hou D Y and Xing H Q. 2023. A self-supervised-driven open-set unsupervised domain adaptation method for optical remote sensing image scene classification and retrieval. IEEE Transactions on Geoscience and Remote Sensing， 61： #5605515 ［DOI： 10.1109/TGRS.2023.3260873http://dx.doi.org/10.1109/TGRS.2023.3260873］
Wang Y， Xu C F， Liu C W and Li Z K. 2022. Context information refinement for few-shot object detection in remote sensing images. Remote Sensing， 14（14）： #3255 ［DOI： 10.3390/rs14143255http://dx.doi.org/10.3390/rs14143255］
Wang Y Y， Wang C， Zhang H， Dong Y B and Wei S S. 2019c. A SAR dataset of ship detection for deep learning under complex backgrounds. Remote Sensing， 11（7）： #765 ［DOI： 10.3390/rs11070765http://dx.doi.org/10.3390/rs11070765］
Wei S J， Zeng X F， Qu Q Z， Wang M， Su H and Shi J. 2020. HRSID： a high-resolution SAR images dataset for ship detection and instance segmentation. IEEE Access， 8： 120234-120254 ［DOI： 10.1109/ACCESS.2020.3005861http://dx.doi.org/10.1109/ACCESS.2020.3005861］
Xia G S， Bai X， Ding J， Zhu Z， Belongie S， Luo J B， Datcu M， Pelillo M and Zhang L P. 2018. DOTA： a large-scale dataset for object detection in aerial images//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City， USA： IEEE： 3974-3983 ［DOI： 10.1109/CVPR.2018.00418http://dx.doi.org/10.1109/CVPR.2018.00418］
Xiao Z F， Long Y， Li D R， Wei C S， Tang G F and Liu J Y. 2017. High-resolution remote sensing image retrieval based on CNNs from a dimensional perspective. Remote Sensing， 9（7）： #725 ［DOI： 10.3390/rs9070725http://dx.doi.org/10.3390/rs9070725］
Xu C J， Zheng X T and Lu X Q. 2022a. Multi-level alignment network for cross-domain ship detection. Remote Sensing， 14（10）： #2389 ［DOI： 10.3390/rs14102389http://dx.doi.org/10.3390/rs14102389］
Xu Q S， Shi Y L and Zhu X X. 2022b. Universal domain adaptation without source data for remote sensing image scene classification//Proceedings of 2022 IEEE International Geoscience and Remote Sensing Symposium. Kuala Lumpur， Malaysia： IEEE： 5341-5344 ［DOI： 10.1109/IGARSS46834.2022.9884889http://dx.doi.org/10.1109/IGARSS46834.2022.9884889］
Xu Q S， Shi Y L， Yuan X and Zhu X X. 2023. Universal domain adaptation for remote sensing image scene classification. IEEE Transactions on Geoscience and Remote Sensing， 61： #4700515 ［DOI： 10.1109/TGRS.2023.3235988http://dx.doi.org/10.1109/TGRS.2023.3235988］
Yang Y and Newsam S. 2010. Bag-of-visual-words and spatial extensions for land-use classification//Proceedings of the 18th SIGSPATIAL International Conference on Advances in Geographic Information Systems. San Jose， USA： Association for Computing Machinery： 270-279 ［DOI： 10.1145/1869790.1869829http://dx.doi.org/10.1145/1869790.1869829］
You K C， Long M S， Cao Z J， Wang J M and Jordan M I. 2019. Universal domain adaptation//Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach： IEEE： #283 ［DOI： 10.1109/cvpr.2019.00283http://dx.doi.org/10.1109/cvpr.2019.00283］
Zhang Y L， Yuan Y， Feng Y C and Lu X Q. 2019. Hierarchical and robust convolutional neural network for very high-resolution remote sensing object detection. IEEE Transactions on Geoscience and Remote Sensing， 57（8）： 5535-5548 ［DOI： 10.1109/TGRS.2019.2900302http://dx.doi.org/10.1109/TGRS.2019.2900302］
Zhao X Y， Hu J L， Mou L C， Xiong Z T and Zhu X X. 2023. Cross-city Landuse classification of remote sensing images via deep transfer learning. International Journal of Applied Earth Observation and Geoinformation， 122： #103358 ［DOI： 10.1016/j.jag.2023.103358http://dx.doi.org/10.1016/j.jag.2023.103358］
Zheng J P， Wu W Z， Fu H H， Li W J， Dong R M， Zhang L X and Yuan S. 2020. Unsupervised mixed multi-target domain adaptation for remote sensing images classification//Proceedings of 2020 IEEE International Geoscience and Remote Sensing Symposium. Waikoloa， USA： IEEE： 1381-1384 ［DOI： 10.1109/IGARSS39084.2020.9323602http://dx.doi.org/10.1109/IGARSS39084.2020.9323602］
Zheng J P， Wu W Z， Yuan S， Zhao Y， Li W J， Zhang L X， Dong R M and Fu H H. 2022a. A two-stage adaptation network （TSAN） for remote sensing scene classification in single-source-mixed-multiple-target domain adaptation （S²M²T DA） scenarios. IEEE Transactions on Geoscience and Remote Sensing， 60： #5609213 ［DOI： 10.1109/TGRS.2021.3105302http://dx.doi.org/10.1109/TGRS.2021.3105302］
Zheng J P， Zhao Y， Wu W Z， Chen M X， Li W J and Fu H H. 2023a. Partial domain adaptation for scene classification from remote sensing imagery. IEEE Transactions on Geoscience and Remote Sensing， 61： #5601317 ［DOI： 10.1109/TGRS.2022.3229039http://dx.doi.org/10.1109/TGRS.2022.3229039］
Zheng X T， Cui H W， Xu C J and Lu X Q. 2023b. Dual teacher： a semisupervised cotraining framework for cross-domain ship detection. IEEE Transactions on Geoscience and Remote Sensing， 61： #5613312 ［DOI： 10.1109/TGRS.2023.3287863http://dx.doi.org/10.1109/TGRS.2023.3287863］
Zheng Z D， Zhong Y F， Su Y and Ma A L. 2022b. Domain adaptation via a task-specific classifier framework for remote sensing cross-scene classification. IEEE Transactions on Geoscience and Remote Sensing， 60： #5620513 ［DOI： 10.1109/TGRS.2022.3151689http://dx.doi.org/10.1109/TGRS.2022.3151689］
Zhou W X， Newsam S， Li C M and Shao Z F. 2018. PatternNet： a benchmark dataset for performance evaluation of remote sensing image retrieval. ISPRS Journal of Photogrammetry and Remote Sensing， 145： 197-209 ［DOI： 10.1016/j.isprsjprs.2018.01.004http://dx.doi.org/10.1016/j.isprsjprs.2018.01.004］
Zhu S H， Du B， Zhang L P and Li X. 2022. Attention-based multiscale residual adaptation network for cross-scene classification. IEEE Transactions on Geoscience and Remote Sensing， 60： #5400715 ［DOI： 10.1109/TGRS.2021.3056624http://dx.doi.org/10.1109/TGRS.2021.3056624］
Zhu S H， Wu C， Du B and Zhang L P. 2023. Adversarial divergence training for universal cross-scene classification. IEEE Transactions on Geoscience and Remote Sensing， 61： #5609712 ［DOI： 10.1109/TGRS.2023.3274781http://dx.doi.org/10.1109/TGRS.2023.3274781］
Zou Q， Ni L H， Zhang T and Wang Q. 2015. Deep learning based feature selection for remote sensing scene classification. IEEE Geoscience and Remote Sensing Letters， 12（11）： 2321-2325 ［DOI： 10.1109/LGRS.2015.2475299http://dx.doi.org/10.1109/LGRS.2015.2475299］
相关作者
相关机构
京公网安备11010802024621