浏览全部资源
扫码关注微信
网络出版日期： 2024-12-23 ，
移动端阅览
肖振久,李士博,曲海成等.上下文信息和多尺度特征序列引导的遥感图像检测[J].中国图象图形学报,
Xiao Zhenjiu,Li Shibo,Qu Haicheng,et al.Remote sensing image detection guided by context information and multi-scale feature sequences[J].Journal of Image and Graphics,
目的
2
针对遥感图像（remote sensing image，RSI）检测中目标尺寸小且密集、尺度变化大，尤其在复杂背景信息下容易出现漏检和误检问题，提出一种上下文信息和多尺度特征序列引导的遥感图像检测方法，以提升遥感图像的检测精度。
方法
2
首先，设计自适应大感受野机制（adaptive large receptive field，ALRF）用于特征提取。该机制通过级联不同扩张率的深度卷积进行分层特征提取，并利用通道和空间注意力对提取的特征进行通道加权和空间融合，使模型能够自适应地调整感受野大小，从而实现遥感图像上下文信息的有效利用。其次，为解决颈部网络特征融合过程中小目标语义信息丢失问题，设计多尺度特征序列融合架构（multi-scale feature fusion，MFF）。该架构通过构建多尺度特征序列，并结合浅层语义特征信息，实现复杂背景下多尺度全局信息的有效融合，从而减轻深层网络中特征模糊性对小目标局部细节捕捉的影响。最后，因传统交并比（IoU，intersection over union）对小目标位置偏差过于敏感，引入归一化Wasserstein距离（normalized Wasserstein distance，NWD）。NWD将边界框建模为二维高斯分布，计算这些分布间的Wasserstein距离来衡量边界框的相似性，从而降低小目标位置偏差敏感性。
结果
2
在NWPU VHR-10（Northwestern Polytechnical University very high resolution 10 dataset）和DIOR（dataset for object detection in aerial images）数据集上与10种方法进行综合比较，结果表明，提出的方法优于对比方法，其中，相较于基准模型YOLOv8n，平均精度（average precision，AP）分别达到93.15%和80.89%，提升5.48%和2.97%，同时参数量下降6.96%。
结论
2
提出一种上下文信息和多尺度特征序列引导的遥感图像检测方法，该方法提升目标的定位能力，改善复杂背景下遥感图像检测中的漏检和误检问题。
Objective
2
Remote sensing images （RSI） have been widely used in environmental monitoring， resource management， and emergency response due to their wide coverage， rich information content， and high temporal and spatial resolution. However， in the actual object detection process， the objects in remote sensing images are usually small in size， densely distributed， and have large-scale variations， which can easily lead to missed detection and false detection problems， especially in complex background. To address these challenges， we propose a remote sensing image detection method guided by contextual information and multi-scale feature sequences.
Method
2
Firstly， in the feature extraction stage， we designed an adaptive large receptive field （ALRF）. In remote sensing image processing， the design of a reasonable receptive field （RF） is crucial. A large receptive field can not only effectively capture the global structure of large-sized targets， but also use contextual information to capture the details of small targets， thereby improving detection performance. Large kernel convolution is a common method to expand the receptive field， but its computational cost is high. As the size of the convolution kernel increases， the number of parameters will also increase rapidly， affecting the inference speed and efficiency of the model. To solve this problem， ALRF adopts multi-layer cascade dilated convolution to expand the receptive field by gradually increasing the size and dilation rate of the convolution kernel， while maintaining a low number of parameters， achieving a receptive field and context perception capability comparable to that of a large kernel convolution. For the hierarchical features extracted by cascade dilated convolution， ALRF performs channel weighing through channel attention and then spatial fusion through spatial attention， thereby adaptively adjusting the receptive field size according to the features of different targets， effectively capturing the contextual information in remote sensing images and realizing the extraction of multi-scale features. Secondly， in the feature fusion stage， we proposed multi-scale feature fusion （MFF） architecture. MFF is mainly composed of a focal scale fusion engine （FSFE） and a fine-scale feature encoder （FFE） and fuses the multi-scale feature sequences extracted from the backbone network through a bidirectional feature pyramid network （BiFPN） structure. FSFE constructs a multi-scale feature sequence using Gaussian convolution kernels with increasing standard deviations. FFE， on the other hand， introduces detailed information from the shallow feature layer of P2 and uses hybrid coding to enhance semantic expression. MFF achieves effective fusion of multi-scale global information under complex background information through the collaborative work of FSFE and FFE， thereby reducing the impact of feature ambiguity in deep networks on capturing small target details. Finally， we introduce the normalized Wasserstein distance （NWD） to replace the traditional intersection over union （IoU）. NWD measures the similarity of bounding boxes by modeling them as two-dimensional Gaussian distributions and calculating the Wasserstein distance between these distributions. As a more robust similarity metric， NWD can more accurately evaluate the alignment between the bounding box of a tiny object and the ground truth box， thereby reducing sensitivity to positional deviations.
Result
2
Our method is experimentally verified on two datasets， NWPU VHR-10 （Northwestern Polytechnical University very high resolution 10 dataset） and DIOR （dataset for object detection in aerial images）， and compared with ten detection methods from the two-stage， one-stage， and DETR series. The experimental results show that the method has significant advantages in small target detection and complex backgrounds. On the NWPU VHR-10 dataset， compared with the baseline model YOLOv8n， the average precision （AP） of the method is improved by 5.48% to 93.15%， while the number of parameters is reduced by 6.96%. In the DIOR dataset， the AP of the method is 80.89%， which is 2.97% higher than that of YOLOv8n. The effectiveness of ALRF， MFF， and NWD is verified through ablation experiments. ALRF achieves efficient use of contextual information through cascaded dilated convolutions； MFF enhances the ability to capture small target details by constructing multi-scale feature sequences and introducing shallow features； NWD improves the balance of positive and negative sample distribution by modeling the bounding box as a two-dimensional Gaussian distribution. The use of ALRF， MFF and NWD together can achieve the best effect on the model， with precision， recall and AP increased by 2.76%， 4.61% and 5.48% respectively compared with the baseline model. In addition， the PR （precision-recall） curve of this method is closer to the upper right and more stable， indicating that it is stable in the detection of different categories of targets. At the same time， this method is effective in reducing false detections and missed detections， and shows stronger generalization and robustness compared with other comparative methods.
Conclusion
2
A remote sensing image detection method guided by context information and multi-scale feature sequences is proposed to solve the problems of dense small targets， large target size variations and complex background in remote sensing image detection. This method first uses the adaptive large receptive field （ALRF） to effectively utilize context information. Then the multi-scale feature fusion （MFF） is used to reduce the semantic information loss of small targets. Finally， the normalized Wasserstein distance （NWD） is introduced to replace the traditional IoU to optimize the distribution of positive and negative samples. Experimental results show that on the NWPU VHR-10 and DIOR datasets， the average precision （AP） of this method reaches 93.15% and 80.89% respectively， and the model parameters are only 2.94 M， which is significantly better than the comparison method. In future work， it will be extended to target detection under multi-view imaging conditions， and a lightweight model suitable for real-time detection applications will be developed.
遥感图像目标检测感受野特征融合归一化Wasserstein距离
Remote sensing imagestarget detectionreceptive fieldfeature fusionnormalized Wasserstein distance
Carion N， Massa F， Synnaeve G， Usunier N， Kirillov A and Zagoruyko S. 2020. End-to-end object detection with transformers//Proceedings of the 16th European conference on computer vision. Cham， Switzerland： Springer： 213-229 ［DOI：10.1007/978-3-030-58452-8_13http://dx.doi.org/10.1007/978-3-030-58452-8_13］
Chen H B， Jiang S， He G， Zhang B and Yu H. 2021. TEANS： a target enhancement and attenuated nonmaximum suppression object detector for remote sensing images. IEEE Geoscience and Remote Sensing Letters， 18（4）： 632-636 ［DOI： 10.1109/LGRS.2020.2983070http://dx.doi.org/10.1109/LGRS.2020.2983070］
Cheng G and Han J. 2016. A survey on object detection in optical remote sensing images. ISPRS Journal of Photogrammetry and Remote Sensing， 117： 11-28 ［DOI： 10.1016/j.isprsjprs.2016.03.014http://dx.doi.org/10.1016/j.isprsjprs.2016.03.014］
Feng C， Zhong Y， Gao Y， Scott M R， and Huang W. 2021. Tood： task-aligned one-stage object detection//Proceedings of 2021 IEEE/CVF International Conference on Computer Vision. Montreal， Canada： IEEE： 3490-3499 ［DOI： 10.1109/ICCV48922.2021.00349http://dx.doi.org/10.1109/ICCV48922.2021.00349］
Girshick R. 2015. Fast r-cnn//Proceedings of the 14th IEEE International Conference on Computer Vision. Santiago， Chile： IEEE： 1440-1448 ［DOI： 10.1109/ICCV.2015.169http://dx.doi.org/10.1109/ICCV.2015.169］
Huang Z， Wei Y， Wang X， Liu W， Huang T S and Shi H. 2021. Alignseg： feature-aligned segmentation networks. IEEE Transactions on Pattern Analysis and Machine Intelligence， 44（1）： 550-557 ［DOI： 10.1109/TPAMI.2021.3062772http://dx.doi.org/10.1109/TPAMI.2021.3062772］
Jocher G， Chaurasia A and Qiu J. 2023. YOLOv8 by Ultralytics ［EB/OL］. ［2023-01-01］. https://github.com/ultralytics/ultralyticshttps://github.com/ultralytics/ultralytics
Li K， Wan G， Cheng G， Meng L and Han J. 2020. Object detection in optical remote sensing images： A survey and a new benchmark. ISPRS Journal of Photogrammetry and Remote Sensing， 159： 296-307 ［DOI： 10.1016/j.isprsjprs.2019.11.023http://dx.doi.org/10.1016/j.isprsjprs.2019.11.023］
Li X， Wang W， Hu X， and Yang J. 2019. Selective kernel networks//Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach， USA： IEEE： 510-519 ［DOI： 10.1109/CVPR.2019.00060http://dx.doi.org/10.1109/CVPR.2019.00060］
Li Y， Hou Q， Zheng Z， Cheng M M， Yang J， and Li X. 2023. Large selective kernel network for remote sensing object detection//Proceedings of 2023 IEEE/CVF International Conference on Computer Vision. Paris， France： IEEE： 16794-16805 ［DOI： 10.1109/ICCV51070.2023.01540http://dx.doi.org/10.1109/ICCV51070.2023.01540］
Li Z H， An J T， Jia H Y and Fang Y. 2023. Lightweight object detection model in remote sensing image by combining rotation box and attention mechanism. Journal of Image and Graphics， 28（09）： 2706-2718
李朝辉， 安金堂， 贾红雨， 方艳. 2023. 结合旋转框和注意力机制的轻量遥感图像检测模型. 中国图象图形学报， 28（09）： 2706-2718 ［DOI： 10.11834/jig.220839http://dx.doi.org/10.11834/jig.220839］
Lin T Y， Dollár P， Girshick R， He K， Hariharan B， and Belongie S. 2017. Feature pyramid networks for object detection//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu， USA： IEEE： 2117-2125 ［DOI： 10.1109/CVPR.2017.106http://dx.doi.org/10.1109/CVPR.2017.106］
Liu C， Zhang S， Hu M and Song Q. 2024. Object detection in remote sensing images based on adaptive multi-scale feature fusion method. Remote Sensing， 16（5）： 907 ［DOI： 10.3390/rs16050907http://dx.doi.org/10.3390/rs16050907］
Liu S， Li F， Zhang H， Yang X， Qi X， Su H， Zhu J and Zhang L. 2022. Dab-detr： dynamic anchor boxes are better queries for detr ［EB/OL］. ［2022-01-28］. https://arxiv.org/pdf/2201.12329v4.pdfhttps://arxiv.org/pdf/2201.12329v4.pdf
Liu S， Qi L， Qin H， Shi J， and Jia J. 2018. Path aggregation network for instance segmentation//Proceedings of 2018 IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake， USA： IEEE： 8759-8768 ［DOI： 10.1109/CVPR.2018.00913http://dx.doi.org/10.1109/CVPR.2018.00913］
Liu W， Anguelov D， Erhan D， Szegedy C， Reed S， Fu C Y， and Berg A C. 2016. SSD： Single shot multibox detector//Proceedings of the 14th European Conference on Computer Vision. Amsterdam， the Netherlands： Springer： 21-37 ［DOI： 10.1007/978-3-319-46448-0_2http://dx.doi.org/10.1007/978-3-319-46448-0_2］
Lyu C， Zhang W， Huang H， Zhou Y， Wang Y， Liu Y， Zhang S and Chen K. 2022. Rtmdet： an empirical study of designing real-time object detectors ［EB/OL］. ［2022-12-14］. https://arxiv.org/pdf/2212.07784.pdfhttps://arxiv.org/pdf/2212.07784.pdf
Ma D， Liu B， Huang Q and Zhang Q. 2023. MwdpNet： towards improving the recognition accuracy of tiny targets in high-resolution remote sensing image. Scientific Reports， 13（1）： 13890 ［DOI： 10.1038/s41598-023-41021-8http://dx.doi.org/10.1038/s41598-023-41021-8］
Redmon J， Divvala S， Girshick R， and Farhadi A. 2016. You only look once： unified， real-time object detection//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas， USA： IEEE： 779-788 ［DOI： 10.1109/CVPR.2016.91http://dx.doi.org/10.1109/CVPR.2016.91］
Ren S， He K， Girshick R and Sun J. 2016. Faster R-CNN： towards real-time object detection with region proposal networks. IEEE Transactions on Pattern Analysis and Machine Intelligence， 39（6）： 1137-1149 ［DOI： 10.1109/TPAMI.2016.2577031http://dx.doi.org/10.1109/TPAMI.2016.2577031］
Shamsolmoali P， Zareapoor M， Chanussot J， Zhou H， and Yang J. 2021. Rotation equivariant feature image pyramid network for object detection in optical remote sensing imagery. IEEE Transactions on Geoscience and Remote Sensing， 60： 1-14 ［DOI： 10.1109/TGRS.2021.3112481http://dx.doi.org/10.1109/TGRS.2021.3112481］
She H D and Zhao L J. 2024. Rotating target detection network that combines key points and guide vectors. Journal of Image and Graphics， 29（02）： 0533-0544
佘浩东， 赵良瑾. 2024. 结合关键点与引导向量的旋转目标检测网络. 中国图象图形学报， 29（02）： 0533-0544 ［DOI：10.11834/jig.230207http://dx.doi.org/10.11834/jig.230207］
Tan M， Pang R， and Le Q V. 2020. Efficientdet： scalable and efficient object detection//Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle， USA： IEEE： 10781-10790 ［DOI： 10.1109/CVPR42600.2020.01079http://dx.doi.org/10.1109/CVPR42600.2020.01079］
Wang A， Chen H， Liu L， Chen K， Lin Z， Han J and Ding G. 2024. Yolov10： real-time end-to-end object detection ［EB/OL］. ［2024-05-23］. https://arxiv.org/pdf/2405.14458.pdfhttps://arxiv.org/pdf/2405.14458.pdf
Xu C， Wang J， Yang W， Yu H， Yu L， and Xia G S. 2022. Detecting tiny objects in aerial images： a normalized Wasserstein distance and a new benchmark. ISPRS Journal of Photogrammetry and Remote Sensing， 190： 79-93 ［DOI： 10.1016/j.isprsjprs.2022.06.002http://dx.doi.org/10.1016/j.isprsjprs.2022.06.002］
Zhang H， Chang H， Ma B， Wang N and Cheng X. 2020. Dynamic R-CNN： towards high quality object detection via dynamic training//Proceedings of the 16th European Conference on Computer Vision. Glasgow， UK： Springer： 260-275 ［DOI： 10.1007/978-3-030-58555-6_16http://dx.doi.org/10.1007/978-3-030-58555-6_16］
Zhang H， Wen S， Wei Z and Chen Z. 2024. High-resolution feature generator for small-ship detection in optical remote sensing images. IEEE Transactions on Geoscience and Remote Sensing， 62： 3377999 ［DOI： 10.1109/TGRS.2024.3377999http://dx.doi.org/10.1109/TGRS.2024.3377999］
Zhang S， Wang X， Wang J， Pang J， Lyu C， Zhang W， Luo P， and Chen K. 2023. Dense distinct query for end-to-end object detection//Proceedings of 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Vancouver， Canada： IEEE： 7329-7338 ［DOI： 10.1109/CVPR52729.2023.00708http://dx.doi.org/10.1109/CVPR52729.2023.00708］
Zhang Z， Wang C， Zhang H， Qi D， Liu Q， Wang Y and Ding W. 2024. SREDet： semantic-driven rotational feature enhancement for oriented object detection in remote sensing images. Remote Sensing， 16（13）： 2317 ［DOI： 10.3390/rs16132317http://dx.doi.org/10.3390/rs16132317］
相关作者
相关机构
京公网安备11010802024621