浏览全部资源
扫码关注微信
网络出版日期： 2024-08-15 ，
移动端阅览
王仁芳,杨梓健,邱虹等.FIFLNet：一种融合特征交互和融合的轻量级变化检测网络[J].中国图象图形学报,
Wang Renfang,Yang Zijian,Qiu Hong,et al.FIFLNet： A Lightweight Change Detection Network Integrating Feature Interaction and Fusion[J].Journal of Image and Graphics,
目的
2
利用深度学习开展变化检测是遥感智能解译热点研究方向之一。针对基于Transformer变化检测模型结构复杂、参数过多、训练耗时的问题，本文设计了一种融合特征交互和融合的轻量级变化检测网络。
方法
2
提出了一种融合特征交互和融合的轻量级变化检测网络（feature interaction and fusion lightweight network， FIFLNet）。解码器中采用EfficientNet作为特征提取网络，其能利用模型的放缩（model scaling）能力来扩大模型的感受野。然后通过设计通道、像素交互模块（spatial and channel interact block）和浅层跳跃连接（low-level skip-connection）来实现浅层双时相的细节特征交互和上采样阶段的传递，以此增加模型对局部特征的判别精度。此外，利用特征融合分组卷积模块（feature fusion and groups convolution block，FFGCB）对双时相数据进行降维融合，来降低模型计算量。最后，设计了融合上采样模块（fusion upsampling block，FUB）对局部特征与全局特征进行融合还原，同时利用局部特征的细节、纹理来补偿全局特征细节的缺失。
结果
2
本文方法在两个遥感影像数据集（LEVIR-CD和SYSU-CD）上与13种SOTA（state-of-the-art）方法进行比较。客观上，本文方法对比现有变化检测方法在各项定量评价指标上均具有明显优势。在LEVIR-CD和SYSU-CD数据集中，本文方法F1分别取得91.51%和82.19%，相较于对比方法的最优值分别提升了0.43%和1.58%，并且模型的每秒浮点运算量和参数量分别为1.66G和0.56M，低于所有对比方法。主观上，本文方法相对于对比方法的检测区域准确、漏检率低，具有丰富的细节。
结论
2
本文提出的轻量级变化检测网络FIFLNet以较少的参数量和每秒浮点运算量获得了优越的性能，改善了小目标漏检、边界误检的情况，能够获得高质量的变化检测结果。
Objective
2
Change detection in remote sensing imagery is a process that leverages remote sensing technology to compare and analyze images from the same geographical area but captured at different time intervals. The primary objective of this process is to identify changes on the Earth’s surface. The main challenge in this process is the extraction of effective change features from a large volume of image data and subsequently mapping them onto pixel-level change labels for high-precision detection. Methods for detecting changes in remote sensing imagery can be broadly divided into traditional and deep learning-based methods. Traditional methods primarily rely on image processing and pattern recognition techniques. However， these methods often require manual selection of suitable features and thresholds， which can introduce a degree of subjectivity and limitations. On the other hand， deep learning methods have the ability to automatically learn abstract and high-level change features from remote sensing images， thereby enabling end-to-end change detection. This significantly enhances the accuracy and efficiency of change detection. Among these， change detection models based on Convolutional Neural Networks （CNN） and Transformer architectures have shown remarkable performance. Models that utilize these mechanisms have seen significant advancements in recent years， thanks to the extensive research conducted by scholars worldwide. However， for the currently effective models based on the Transformer architecture， as the detection accuracy of the model improves， the complexity of the model also increases. Therefore， designing a change detection method with a small parameter size， low computational cost， and high detection accuracy is a pressing issue that urgently needs to be addressed in this field.
Method
2
This paper proposes a lightweight change detection method for remote sensing images based on feature interaction and fusion. The main idea of this method is to use EfficientNet B7 as a lightweight backbone network to extract both deep and low-level features from bi-temporal remote sensing images. To enhance the spatio-temporal feature fusion， we introduce channel swap and pixel swap modules to enable the interaction and combination of bi-temporal features. To preserve more edge and texture details and reduce artifact generation， we employ low-level skip-connections to transfer the original image information to the up-sampling phase. To effectively fuse the deep and low-level features obtained in the down-sampling stage， we design a feature fusion group convolution module that reduces the computational overhead and the number of parameters. Finally， we use the feature fusion group convolution module and the up-sampling module to fuse and recover the deep and low-level features， and generate the pixel-level change detection map.
Result
2
In this paper， we conducted experiments on two datasets for remote sensing image change detection LEVIR-CD and SYSU-CD. We split each dataset into 7：1：2 for training， validation and testing， and segmented each image into 256×256 pixels. This facilitated the processing and increased the generalization ability of the model. We used the binary cross entropy （BCELoss） as the loss function， and evaluated the performance of our proposed method using three metrics： F1 score （F1）， Intersection Over Union （IoU）， and Overall Accuracy （OA）. Our method achieved F1 scores of 91.51% and 82.19%， IoU 84.35% and 69.76%， and OA of 99.14% and 91.99% on the LEVIR-CD and SYSU-CD datasets， respectively. Compared with previous classical methods， such as DSIFN， DTCDSCN， STANet， SNUNet， BiT， Changeformer， DDPM-CD， USSFC-Net， ELGC-Net， LRNet， etc.， our model obtained the best change detection results， especially preserving more details on the change boundary. To illustrate the effect of low-level skip-connections and channel and spatial exchange modules， we also performed ablation experiments on the same dataset. The results showed that the channel and spatial exchange module significantly optimized the utilization and representation of spatio-temporal features in the network， while the low-level skip-connection compensated for the loss of detailed features in the downsampling process and further enhanced the feature learning capability of the network.
Conclusion
2
our network used channel and spatial exchange modules to increase the utilization and understanding of spatio-temporal features， and low-level skip-connections to focus the model on local detailed features. Finally， we used a binary cross-entropy loss function at the output layer to achieve optimal change detection performance. Experiments show that the method proposed in this paper can improve the ability of recognizing changing regions while ensuring a light network， and can improve the detection performance of change detection in various environments and terrains.
遥感影像变化检测局部特征特征交互轻量级网络
Remote sensingChange detectionLocal featureFeature interactLight network
Bandara W G C， Nithin N G， Patel V M. 2022. DDPM-CD： Remote sensing change detection using denoising diffusio-n probabilistic models.［EB/OL］.［2023-08-12］. https://arxiv.org/pdf/2206.11892.pdfhttps://arxiv.org/pdf/2206.11892.pdf
Bandara W G C， Patel V M. 2022. A Transformer-based siam-ese network for change detection//Proceedings of 2022 IE-EE International Geoscience and Remote Sensing Symposium， Kuala Lumpur， Malaysia：207-210［DOI：10.1109/IGARSS46834.2022.9883686http://dx.doi.org/10.1109/IGARSS46834.2022.9883686］
Chen H， Shi Z. 2020. A spatial-temporal attention-based meth-ods and a new dataset for remote sensing image change detection. Remote Sensing. 12（10）：1662［DOI：10.3390/rs12101662http://dx.doi.org/10.3390/rs12101662］
Chen H， Qi Z P， Shi Z W. 2022. Remote sensing image cha-nge detection with transformers. IEEE Transactions on Ge-oscience and Remote Sensing， 60：1-14［DOI：10.1109/T-GRS.2021. 3095166http://dx.doi.org/10.1109/T-GRS.2021.3095166］
Chen H， Shi Z W. 2020. A spatial-temporal attention-based m-ethod and a new dataset for remote sensing image change detection. Remote Sensing， 12（10）：1662［DOI：10.3390/rs12101662http://dx.doi.org/10.3390/rs12101662］
Cardille J A， Perez E， Crowley M A， Wulder M A， White J C and Hermosilla T. 2022. Multi-sensor change detection for withi-year capture and labelling of forest disturbance. Remote Sensing of Environment，.268：112741［DOI：10.1016/j.rse. 2021.112741http://dx.doi.org/10.1016/j.rse.2021.112741］
Deng J， Dong W， Socher R， Li L J， Li K and Li F F. 2009. Imagenet： A large-scale hierarchical image database//Pro-ceedings of 2009 IEEE Conference on Computer Vis-ion and Pattern Recognition， Miami， FL， USA： IEEE：248-255［DOI：10.1109/CVPR.2009.5206848http://dx.doi.org/10.1109/CVPR.2009.5206848］
Du P J and Liu S C. 2012. Change detection from multi-tem-poral remote sensing images by integrating multiple featu-res. Journal of Remote Sensing， 16（4）：663-677
杜培军， 柳思聪.2012.融合多特征的遥感影像变化检测.遥感学报，16（4）：663-677［DOI：10.11834/jrs.20121168http://dx.doi.org/10.11834/jrs.20121168］
Daudt C R， Saux L B and Boulch A. 2018. Fully convolutio-nal siamese networks for change detection//Proceedings of 2018 IEEE International Conference on Image Processing， Athens， Greece： IEEE：4063-4067［DOI：10.1109/ICIP.2018.8451652http://dx.doi.org/10.1109/ICIP.2018.8451652］
Fang S， Li K， Shao J and Li Z. 2021. SNUNet-CD： A dens-ely connected siamese network for change detection of V-HR images. IEEE Geoscience and Remote Sensing Letters，19：1-5［DOI：10.1109/LGRS.2021.3056416http://dx.doi.org/10.1109/LGRS.2021.3056416］
Fang S， Li K and Li Z. 2023. Changer： Features interaction is what you beed for change detection. IEEE Transactions on Geoscience and Remote Sensing， 61：1-11［DOI：10.1109/TGRS.2023.3277496http://dx.doi.org/10.1109/TGRS.2023.3277496］
He K， Zhang X， Ren S and Sun J. 2015. Delving deeps into rectifiers： Surpassing human-level performance on imagenet classification.//Proceedings of 2015 IEEE International C-onference on Computer Vision（ICCV）， Santiago， Chile：IEEE：1026–1034［DOI：10.1109/ICCV.2015.123http://dx.doi.org/10.1109/ICCV.2015.123］
Ilya L and Frank H. 2017. Decoupled weight decay regulariza-tion.［EB/OL］.［2023-10-10］. https://arxiv.org/pdf/1711.05101.pdfhttps://arxiv.org/pdf/1711.05101.pdf
Liu Y， Chao P， Zhan Z Q， Zhan X M and Yang X. 2019. B-uilding change detection for remote sensing images using a dual-task constrained deep siamese convolutional networ-k model. IEEE Geoscience and Remote Sensing Letters，18：811-815［DOI：10.1109/LSRS.2020.2988032http://dx.doi.org/10.1109/LSRS.2020.2988032］
Long J， Shelhamer E and Darrell T. 2015. Fully convolutional networks for semantic segmentation//Proceedings of 2015 IEEE Conference on Computer Vision and Pattern R-ecognition （CVPR）， Boston， MA， USA： IEEE：3431-3440［DOI：10.1109/CVPR.2015.7298965http://dx.doi.org/10.1109/CVPR.2015.7298965］
Li L， Wang C， Zhang H and Zhang B. 2015. Residual UNet for urban building change detection with sentinel-1SAR d-ata//Proceedings of 2019 IEEE International Geoscien-ce and Remote Sensing Symposium. Yokohama， Japan， IE-EE：1498-1501［DOI：10.1109/IGARSS.2019.8898146http://dx.doi.org/10.1109/IGARSS.2019.8898146］
Li， C， Zhang， B， Hong， D， Zhou， J， Vivone， G， Li， S and Jocelyn， C. 2024. Casformer： cascaded transformers for fusion-aware computational hyperspectral imaging. Information Fusion. 108. ［DOI：10.1016/j.inffus.2024.102408http://dx.doi.org/10.1016/j.inffus.2024.102408］
Lecun Y， Bottou L， Bengio Y and Haffner P. 1998. Gradient-based learning applied to document recognition. Proceedings of the IEEE，86（11）：2278-2324［DOI：10.1109/5.726791http://dx.doi.org/10.1109/5.726791］
Lei T， Geng X Z， Ning H L， Lv Z Y， Gong M G， Jin Y C and Nandi A K. 2023. Ultra-lightweight Spatial–Spectral Feature Cooperation Network for Change Detection in Re-mote Sensing Images. IEEE Transactions on Geoscience and Remote Sensing， 61：1-14［DOI：10.1109/TGRS.2023.3261273http://dx.doi.org/10.1109/TGRS.2023.3261273］
Noman M， Fiaz M， Cholakkal H， Khan S， and Khan F S. 2024. ELGC-Net： Efficient Local–Global Context Aggregat-ion for Remote Sensing Change Detection. IEEE Transac-tions on Geoscience and Remote Sensing， 62：1-11［DOI：10.1109/TGRS.2024.3362914http://dx.doi.org/10.1109/TGRS.2024.3362914］
Paszke A， Gross S， Massa F and et al.2019. Pytorch： An im-perative style， high-performance deep learning library//Pro-ceedings of the 33rd Conference on Neural Information P-rocessing Systems， Curran Associates Inc， Red Hook， NY， USA： NeurIPS：8026–8037［DOI：10.5555/3454287.3455008http://dx.doi.org/10.5555/3454287.3455008］
Silva C， Marino A ， Lopez-Sanchez J M and Cameron I. 2020. Agricultural fields monitoring with multi-temporal polar-imetric SAR （MTPOLSAR） change detection//Proceedings of 2020 IEEE International Geoscience and Remote Sensi-ng Symposium：IEEE：4554-4557［DOI：10.1109/IGARSS39084.2020.9324626http://dx.doi.org/10.1109/IGARSS39084.2020.9324626］
Shi q， Liu M， Li S， Liu X， Wang F and Zhan L. 2022. A d-eeply supervised attention metric-based network and an o-pen aerial image dataset for Remote Sensing Change Det-ection. IEEE Transactions on Geoscience and Remote Sen-sing， 60：1-16［DOI：10.1109/TGRS.2021.3085870http://dx.doi.org/10.1109/TGRS.2021.3085870］
Sui H G， Feng W Q， Li W Z， Sun K M， Xu C.2018. Revie-w of Change Detection Methods for Multi-temporal Remo-te Sensing Imagery. Geomatics and Information Science of Wuhan University. 43（12）：1885-1898.
眭海刚， 冯文卿， 李文卓， 孙开敏， 徐川. 2018. 多时相遥感影像变化检测方法综述. 武汉大学学报 （ 信息科学版）.43（12）：1885-1898［DOI：10.13203/j.whugis20180251http://dx.doi.org/10.13203/j.whugis20180251］
Tan M X and Le Q V. 2019. Efficientnet： Rethinking model scaling for convolutional neural networks//Proceeding of t-he 36th International Conference on Machine Learning. L-ong Beach：PMLR：6105–6114［DOI：10.48550/arXiv.1905.11946http://dx.doi.org/10.48550/arXiv.1905.11946］
Ulyanov D， Vedaldi A and Lempitsky V. 2016. Instance norm-alization： The missing ingredient for fast stylization.［EB/OL］［2023-10-11］. https://arxiv.org/pdf/1607.08022.pdfhttps://arxiv.org/pdf/1607.08022.pdf
Vaswani S， Shazeer N， Parmar N， et al. 2017. Attention is all you need//Proceedings of the 31st International Conferen-ce on Neural Information Processing Systems，Curran Asso-ciates Inc.， Red Hook， NY， USA：NIPS：6000–6010［DOI：10.5555/3295222.3295349http://dx.doi.org/10.5555/3295222.3295349］
Wang R F， Wang L， Li C， Huo C L and Chen J W. 2023. IIQ-CNN-based cross-domain change detection of SAR images. Journal of Image and Graphics，28（07）：2208-2220
王蓉芳，王良，李畅，霍春雷，陈佳伟. 2023. 整型推理量化CNN的SAR图像跨域变化检测. 中国图象图形学报，28（07）：2208-2220［DOI：10.11834/jig.211159http://dx.doi.org/10.11834/jig.211159］
Zhang C， Yue P， Tapete D， Jiang L C Huang L and Liu G C. 2020. A deeply supervised image fusion network for change detection in high resolution bi-temporal remote sensing images. ISPRS Journal of Photogrammetry and Remote Sensing，166：183-200［DOI：10.1016/j.isprsjprs.2020.06.003http://dx.doi.org/10.1016/j.isprsjprs.2020.06.003］
Zhong H， Wu C， and Xiao Z. 2024. LRNet： Change detection of high-resolution remote sensing imagery via strategy of localization-then-refinement.［EB/OL］.［2024-04-05］. https://arxiv.org/pdf/2404.04884.pdfhttps://arxiv.org/pdf/2404.04884.pdf
Zhang K， Fu X， Lv X and Yuan J. 2021. Unsupervised multi-temporal building change detection framework based on c-osegmentation using time-series SAR. Remote Sensing，13（3）：471［DOI：10.3390/rs13030471http://dx.doi.org/10.3390/rs13030471］
Zhou Z W， Siddiquee M M R， Tajbakhsh N and Liang J M. 2018. U-Net++： a nested u-net architecture for medical i-mage segmenta⁃ tion//Proceedings of 2018 Deep Learning in Medical Image Analy⁃ sis and Multimodal Learning for Clinical Decision Support. Granada， Spain： Springer：3-11［DOI：10.1007/978-3-030-00889-5_1http://dx.doi.org/10.1007/978-3-030-00889-5_1］
Yang S， Li M， Peng Z G， and Feng C. 2009. A Novel Multi-band Remote Sensing Image Change Detection Algorithm. Journal of Image and Graphics，14（04）：2208-2220
杨胜，李敏，彭振国，冯春. 2009. 一种新的多波段遥感影像变化检测方法. 中国图象图形学报，14（04）：2208-2220［DOI：10.11834/jig.211159http://dx.doi.org/10.11834/jig.211159］
Yan C Q， Zhang Y G， Zhang Q L， Yang Y M， Jiang X Y， Yang Y Q and Wang B Y. 2022. Privacy-preserving online AutoML for Domain-Specific face detection//Proceeding of 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）， New Orleans， LA， USA：IEEE：4124-4134［DOI：10.1109/CVPR52688.2022.00410http://dx.doi.org/10.1109/CVPR52688.2022.00410］
相关作者
相关机构
京公网安备11010802024621