浏览全部资源
扫码关注微信
收稿日期：2025-01-09，
修回日期：2025-04-05，
录用日期：2025-04-09，
网络出版日期：2025-04-09，
移动端阅览
目的
2
近年来，随着深度学习、4K相机和无人机技术的迅猛发展，远距离小目标检测取得了显著进展，其应用范围广泛涵盖刑侦、城市规划和灾害救援等领域。然而，无人机远距拍摄导致目标尺寸较小，细节信息易在成像过程中丢失。尽管扩散模型在超分辨率领域取得了一定进展，但无人机图像具有高分辨率和物体稀疏分布的特点，对整幅图像进行超分辨率处理会造成大量资源浪费。此外，局部空间特征和全局判别性特征对小目标的定位和分类至关重要。现有模型通过顺序堆叠卷积神经网络和Transformer结构，分别对图像的局部空间特征和全局语义特征进行编码，以实现更精确的目标检测。但Transformer的序列编码结构忽略了图像的局部空间信息，这种局部空间信息的缺失在卷积与自注意力层的混合堆叠过程中被进一步放大，导致特征表示存在固有的分离性。
方法
2
针对上述问题，提出融合前景细化和多维归纳偏置自注意力的无人机小目标检测。前景细化模块通过类别无关的多层协同显著性映射方法筛选前景图像块，结合条件扩散模型生成前景区域的细化图像，恢复小目标的关键细节，从而在减少背景区域计算负担的同时提高了检测精度。多维归纳偏置自注意力网络包括多维自注意力模块、混合增强前馈模块、尺度耦合模块和邻域特征交互模块。多维自注意力模块将自注意力分解到水平和垂直两个维度，强化了对空间位置信息的捕捉，弥补图像转为一维序列后空间结构信息的损失，同时引入并行的归纳偏置感知路径，实现局部与全局特征的协同表征，避免了特征分离。融合后的特征进入混合增强前馈模块，通过动态卷积核响应不同目标区域，进一步提升对局部信息的敏感度。此外，尺度耦合模块通过多重卷积与自注意力交互，能够最大限度保留局部与全局特征。最后，邻域特征交互模块通过逐层聚合邻域特征汇总不同层的信息，确保预测特征图中包含充分的无人机小目标信息。
结果
2
为了验证所提方法的有效性，在3个数据集上与当前先进的检测方法进行了广泛的对比实验，实验结果在精确率、召回率、平均检测精度和IoU（intersection of union）阈值为0.5的平均准确率均有显著提高，IoU阈值为0.5的平均准确率在3个数据集上分别达到53.2%、38.7%和93.9%。相较于基线方法分别提高了9.7%、9.5%和1.2%。
结论
2
实验结果表明，所提方法拥有在无人机场景下处理复杂背景和小目标检测的强大能力。代码开源在
https：//github.com/CUMT-GMSC/MIBSN
https://github.com/CUMT-GMSC/MIBSN
。
Objective
2
Drones equipped with 4K cameras have emerged as a transformative technology across diverse domains， from traffic surveillance and criminal investigation to military reconnaissance and disaster response. The ability to capture aerial images over vast areas enables unprecedented visual analysis capabilities. In drone-based applications， object detection serves as a fundamental technology for scene perception， facilitating precise object recognition and tracking to enhance operational efficiency and safety in complex scenes. Despite the success of deep learning-based object detection models on natural scenes， drone-based detection presents unique challenges. One of the most significant challenges is the motion blur introduced by the drone's high-speed flight， which degrades fine details of small objects in the captured images. High-speed motion often leads to blurred images that are difficult for traditional models to process， especially when it comes to detecting small or distant objects that require high spatial resolution. Though recent advances in diffusion models have shown promise in addressing super-resolution tasks， providing finer details in blurry images， their use in drone images remains limited due to the sparse distribution of objects in the scenes. Full-image super-resolution is computationally expensive and inefficient when the objects of interest are few and scattered across large areas. Another major challenge stems from the high-altitude perspective of drone images， where ground objects appear smaller and are easily overwhelmed by complex background. This inherent scale reduction makes small objects particularly difficult to detect and classify， as their features become less distinctive against varying ground patterns and environmental conditions. The accurate detection of such small objects demands both fine-grained local spatial information and global contextual features. Local features are essential for capturing distinctive small object characteristics， while global features provide crucial contextual cues for distinguishing objects from similar background patterns. Although hybrid architectures that combine CNN （convolutional neural networks） and Transformers attempt to leverage their complementary advantages in local feature extraction and global dependency modeling， these approaches still face inherent limitations. The Transformer's sequence-based processing， even with positional encoding， fundamentally constrains local spatial perception due to the conversion of 2D spatial structures into 1D sequences. Furthermore， the independent stacking of convolution and self-attention layers， rather than facilitating feature integration， tends to intensifies the disconnect between local and global representations. This architectural limitation results in feature separation rather than meaningful fusion， particularly impacting the model's ability to detect and classify small objects in complex aerial scenes.
Method
2
This paper proposes a novel small object detection model based on foreground refinement and a multidimensional inductive bias self-attention to address the aforementioned challenges. To overcome the computational inefficiency of processing high-resolution drone images， we introduce a foreground refinement module （FRM） that employs a class-agnostic multi-layer aggregated activation map （CAM-AM） to identify regions of interest， enabling targeted refinement that both preserves computational resources and enhances critical small object details. At the core of our method， we propose multidimensional inductive bias self-attention network （MIBSN）， which consists of multidimensional self-attention， hybrid enhanced feed-forward， scale coupling module， and progressive interaction module. The multidimensional self-attention module decomposes attention operations across horizontal and vertical dimensions， implementing adaptive window-based attention to significantly enhance spatial position awareness. Within the multidimensional self-attention， we innovatively incorporate an inductive bias perception path that operates in parallel with the attention mechanism， specifically designed to capture fine-grained local features. The parallel computation generates both global contextual information and local spatial details， which are jointly processed by a hybrid enhanced feed-forward module utilizing dynamic convolution kernels. This design ensures the Transformer inherits strong inductive bias properties while maintaining comprehensive feature representation capabilities in complex backgrounds. To effectively handle the multi-scale nature of objects in drone images， the scale coupling module employs adaptive kernel configurations while dynamically interact with multidimensional self-attention， maximizing the preservation of both local and global features across different scales. Finally， the progressive interaction module progressively aggregates neighborhood features in the network decoder， enabling sufficient restoration of small object details in the final prediction maps. By integrating FRM and MIBSN， our method provides a comprehensive solution for small object detection. The FRM enhances fine-grained details through targeted refinement of regions of interest， while MIBSN ensures robust feature extraction through its multi-component architecture. Specifically， the combination of local spatial information and global context in MIBSN， coupled with the enhanced detail preservation from FRM， enables accurate detection and classification of small objects even in complex drone images.
Result
2
To validate the effectiveness of our proposed method， we conduct extensive experiments on three challenging datasets： VisDrone2019（vision meets drones 2019）， UAVDT（unmanned aerial vehicle detection and tracking）， and NWPU VHR-10（northwestern polytechnical university very high resolution 10 dataset）. Our method demonstrates remarkable performance on VisDrone2019， a large-scale drone-captured dataset， achieving 53.2% average precision at the IoU threshold of 0.5 （AP@50） and 32.8% mean average precision across all IoU thresholds from 0.5 to 0.95（mAP）， substantially surpassing existing state-of-the-art object detection methods. Notably， compared to the current leading Transformer-based detector DINO （detection Transformer with improved denoising anchor boxes）， our approach achieves a significant improvement of 7.9%. On UAVDT benchmark， our approach achieves 38.7% AP@50 and 23.4% mAP， this substantial gain can be attributed to our innovative multi-dimensional inductive bias self-attention network， which effectively enhances both local feature learning and global dependency modeling. To further demonstrate the generalization ability of our method in handling small objects from aerial perspectives， we evaluate our approach on the NWPU VHR-10 remote sensing dataset. Despite the different imaging characteristics between drone and satellite imagery， both scenarios share common challenges in small object detection， including scale variation and complex backgrounds. Our method achieves a state-of-the-art mAP of 64.5% and AP@50 of 93.9% on this dataset， validating its robust feature extraction capabilities across different aerial imaging domains. To provide deeper insights into the effectiveness of our method， we conduct comprehensive ablation studies on both FRM and MIBSN， systematically analyzing the contribution of each proposed component to the overall performance improvement.
Conclusion
2
In this paper， we propose a novel small object detection method for drone images that integrates a foreground refinement mod
ule for targeted refinement processing and a multidimensional inductive bias self-attention network for enhanced feature learning. Our approach effectively addresses the challenges of small object detection. Comprehensive evaluations on VisDrone2019， UAVDT and NWPU VHR-10 datasets demonstrate superior performance of our method， significantly advancing the detection accuracy of small objects in drone images. The code is available at
https：//github.com/CUMT-GMSC/MIBSN
https://github.com/CUMT-GMSC/MIBSN
．
Bai Y ， Zhang Y ， Ding M and Ghanem B . 2018 . SOD-MTGAN： Small object detection via multi-task generative adversarial network // Proceedings of the European Conference on Computer Vision . Munich， Germany ： Springer： 206 – 221 ［ DOI： 10.1007/978-3-030-01261-8_13 http://dx.doi.org/10.1007/978-3-030-01261-8_13 ］
Carion N ， Massa F ， Synnaeve G ， Usunier N ， Kirillov A and Zagoruyko S . 2020 . End-to-end object detection with transformers // Proceedings of the 16th European Conference on Computer Vision . Glasgow ： Springer： 213 – 229 ［ DOI： 10.1007/978-3-030-58452-8_13 http://dx.doi.org/10.1007/978-3-030-58452-8_13 ］
Cheng G ， Zhou P and Han J . 2016 . Learning rotation-invariant convolutional neural networks for object detection in VHR optical remote sensing images . IEEE Transactions on Geoscience and Remote Sensing ， 54 （ 12 ）： 7405 – 7415 ［ DOI： 10.1109/TGRS.2016.2601622 http://dx.doi.org/10.1109/TGRS.2016.2601622 ］
Croitoru F A ， Hondru V ， Ionescu R T and Shah M . 2023 . Diffusion models in vision： A survey . IEEE Transactions on Pattern Analysis and Machine Intelligence ， 45 （ 9 ）： 10850 – 10869 ［ DOI： 10.1109/TPAMI.2023.3261988 http://dx.doi.org/10.1109/TPAMI.2023.3261988 ］
Chattopadhay A ， Sarkar A ， Howlader P and Balasubramanian VN . 2018 . Grad-cam++： Generalized gradient-based visual explanations for deep convolutional networks // Proceedings of 2018 IEEE winter conference on applications of computer vision . Lake Tahoe， USA ： IEEE： 839 – 847 ［ DOI： 10.1109/WACV.2018.00097 http://dx.doi.org/10.1109/WACV.2018.00097 ］
Du B ， Huang Y ， Chen J and Huang D . 2023 . Adaptive sparse convolutional networks with global context enhanSCMent for faster object detection on drone images // Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . Vancouver， Canada ： IEEE： 13435 – 13444 ［ DOI： 10.1109/CVPR52729.2023.01291 http://dx.doi.org/10.1109/CVPR52729.2023.01291 ］
Dong X ， Bao J ， Chen D ， Zhang W ， Yu N ， Yuan L ， Chen D and Guo B . 2022 . CSWin transformer： A general vision transformer backbone with cross-shaped windows // Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . Louisiana， USA ： IEEE： 12124 – 12134 ［ DOI： 10.1109/CVPR52688.2022.01181 http://dx.doi.org/10.1109/CVPR52688.2022.01181 ］
Du D ， Zhu P ， Wen L ， Bian X ， Lin H ， Hu Q ， Peng T ， Zheng J ， Wang X ， Zhang Y and others . 2019 . VisDrone-DET2019： The vision meets drone object detection in image challenge results // Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision Workshops . Seoul， Korea ： IEEE： 0 – 0 ［ DOI： 10.1109/ICCVW.2019.00030 http://dx.doi.org/10.1109/ICCVW.2019.00030 ］
Du D ， Qi Y ， Yu H ， Yang Y ， Duan K ， Li G ， Zhang W ， Huang Q ， Tian Q . 2018 . The Unmanned Aerial Vehicle Benchmark： Object Detection and Tracking // Proceedings of the 15th European Conference on Computer Vision . Munich， Germany ： Springer： 516 – 531 ［ DOI： 10.1007/978-3-030-01249-6_23 http://dx.doi.org/10.1007/978-3-030-01249-6_23 ］
Fei B ， Lyu Z Y ， Pan L ， Zhang J Z ， Yang W D ， Luo T Y ， Zhang B and Dai B . 2023 . Generative diffusion prior for unified image restoration and enhancement // Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . Vancouver， Canada ： IEEE： 9935 - 9946 ［ 10.1109/CVPR52729.2023.00958 http://dx.doi.org/10.1109/CVPR52729.2023.00958 ］
Glenn J ， Ayush C and Jing Q ， Ultralytics YOLOv8 ［EB/OL］. ［2023］. https://github.com/ultralytics/ultralytics https://github.com/ultralytics/ultralytics
Ho J ， Jain A and Abbeel P . 2020 . Denoising diffusion probabilistic models . Advances in Neural Information Processing Systems ， 33 ： 6840 – 6851 ［ DOI： 10.5555/3495724.3496298 http://dx.doi.org/10.5555/3495724.3496298 ］
Huang Y X ， Liu H I ， Shuai H H and Cheng W H . 2024 . DQ-DETR： DETR with dynamic query for tiny object detection // Proceedings of the 18th European Conference on Computer Vision . Milan， Italy ： Springer： 290 – 305 ［ DOI： 10.1007/978-3-031-73116-7_17 http://dx.doi.org/10.1007/978-3-031-73116-7_17 ］
Lin T Y ， Dollár P ， Girshick R ， He K ， Hariharan B and Belongie S . 2017 . Feature pyramid networks for object detection // Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition . Honolulu， USA ： IEEE： 2117 – 2125 ［ DOI： 10.1109/CVPR.2017.106 http://dx.doi.org/10.1109/CVPR.2017.106 ］
Lin T Y ， Maire M ， Belongie S ， Hays J ， Perona P ， Ramanan D ， Dollár P and Zitnick C L . 2014 . Microsoft COCO： Common objects in context // Proceedings of the 13th European Conference on Computer Vision . Zurich ： Springer： 740 – 755 ［ DOI： 10.1007/978-3-319-10602-1_48 http://dx.doi.org/10.1007/978-3-319-10602-1_48 ］
Liang S ， Wu H ， Zhen L ， Hua Q ， Garg S ， Kaddoum G ， Hassan M M and Yu K . 2022 . Edge YOLO： Real-time intelligent object detection system based on edge-cloud cooperation in autonomous vehicles . IEEE Transactions on Intelligent Transportation Systems ， 23 （ 12 ）： 25345 – 25360 ［ DOI： 10.1109/TITS.2022.3158253 http://dx.doi.org/10.1109/TITS.2022.3158253 ］
Lei S ， Shi Z and Zou Z . 2020 . Coupled adversarial training for remote sensing image super-resolution . IEEE Transactions on Geoscience and Remote Sensing ， 58 （ 5 ）： 3633 – 3643 ［ DOI： 10.1109/TGRS.2019.2959020 http://dx.doi.org/10.1109/TGRS.2019.2959020 ］
Liu H I ， Tseng Y W ， Chang K C ， Wang P J ， Shuai H H and Cheng W H . 2024 . A DeNoising FPN with transformer R-CNN for tiny object detection . IEEE Transactions on Geoscience and Remote Sensing ， 62 ： 1 - 5 ［ DOI： 10.1109/TGRS.2024.3396489 http://dx.doi.org/10.1109/TGRS.2024.3396489 ］
Li Y S ， Liang H J ， Zheng H F and Yu R . 2024 . CR-CAM： Generating explanations for deep neural networks by contrasting and ranking features . Pattern Recognition ， 149 ： 110251 ［ DOI： 10.1016/j.patcog.2024.110251 http://dx.doi.org/10.1016/j.patcog.2024.110251 ］
Liang J Y ， Cao J Z ， Sun G L ， Zhang K ， Van G L and Timofte R . 2021 . SwinIR： Image restoration using swin transformer // Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision Workshops . Montreal， Canada ： 1833 - 1844 ［ 10.1109/ICCVW54120.2021.00210 http://dx.doi.org/10.1109/ICCVW54120.2021.00210 ］
Selvaraju R R ， Cogswell M ， Das A ， Vedantam R ， Parikh D and Batra D . 2017 . Grad-CAM： Visual explanations from deep networks via gradient-based localization // Proceedings of the 2017 IEEE International Conference on Computer Vision . Venice， Italy ： IEEE： 618 – 626 ［ DOI： 10.1109/ICCV.2017.74 http://dx.doi.org/10.1109/ICCV.2017.74 ］
Saharia C ， Ho J ， Chan W ， Salimans T ， Fleet D J and Norouzi M . 2022 . Image super-resolution via iterative refinement . IEEE Transactions on Pattern Analysis and Machine Intelligence ， 45 （ 4 ）： 4713 – 4726 ［ DOI： 10.1109/TPAMI.2022.3204461 http://dx.doi.org/10.1109/TPAMI.2022.3204461 ］
Tian H D ， Zhang J P and Wang Y H . 2023 . Sparse constraint and spatial-temporal regularized correlation filter for UAV tracking . Journal of image and graphics ， 28 （ 2 ）： 458 - 470
田昊东 ， 张津浦 ， 王岳环 . 2023 . 稀疏约束的时空正则相关滤波无人机视觉跟踪 . 中国图象图形学报 ， 28 （ 2 ）： 458 - 470 . ［ DOI： 10.11834/jig.210611 http://dx.doi.org/10.11834/jig.210611 ］
Wang S K ， Wang X D ， Qu L ， Yao F Q ， Liu Y Y ， Li C H ， Wang Y Z and Zhong G Q . 2024 . Semantic segmentation benchmark dataset for coastal ecosystem monitoring based on unmanned aerial vehicle . Journal of Image and Graphics ， 29 （ 08 ）： 2162 - 2174
王胜科 ， 王贤栋 ， 曲亮 ， 姚凤芹 ， 刘莹莹 ， 李聪慧 ， 王玉瑱 ， 仲国强 . 2024 . 面向无人机海岸带生态系统监测的语义分割基准数据集 . 中国图象图形学报 ， 29 （ 08 ）： 2162 - 2174 ［ DOI： 10.11834/jig.230670 http://dx.doi.org/10.11834/jig.230670 ］
Wang W ， Xie E ， Li X ， Fan D P ， Song K ， Liang D ， Lu T ， Luo P and Shao L . 2021 . Pyramid vision transformer： A versatile backbone for dense prediction without convolutions // Proceedings of the IEEE/CVF International Conference on Computer Vision . Montreal， Canada ： IEEE： 568 – 578 ［ DOI： 10.1109/ICCV48922.2021.00061 http://dx.doi.org/10.1109/ICCV48922.2021.00061 ］
Wang C Y ， Yeh I H and Liao H Y M . 2024 . YOLOv 9 ： Learning what you want to learn using programmable gradient information //Proceedings of the 18th European Conference on Computer Vision. Milan， Italy ： Springer： 740 – 755 ［ DOI： 10.1007/978-3-031-72751-1_1 http://dx.doi.org/10.1007/978-3-031-72751-1_1 ］
Xiao Y ， Yuan Q ， Jiang K ， He J ， Jin X and Zhang L . 2024 . EDiffSR： An efficient diffusion probabilistic model for remote sensing image super-resolution . IEEE Transactions on Geoscience and Remote Sensing ， 62 ： 1 – 14 ［ DOI： 10.1109/TGRS.2023.3341437 http://dx.doi.org/10.1109/TGRS.2023.3341437 ］
Yu J R ， Yao S Y ， Chen X Q ， Lin Y L and Wang F Y . 2024 . Cross-domain collaborative technology among vehicles infrastructure and drones for connected and autonomous driving deployment . Journal of Image and Graphics ， 29 （ 11 ）： 3293 - 3304
于静茹 ， 姚升悦 ， 陈喜群 ， 林懿伦 ， 王飞跃 . 2024 . 面向网联自动驾驶部署的车—路—无人机跨域协同技术 . 中国图象图形学报 ， 29 （ 11 ）： 3293 - 3304 ［ DOI： 10.11834/jig.230786 http://dx.doi.org/10.11834/jig.230786 ］
Yang J ， Fu K ， Wu Y ， Diao W ， Dai W and Sun X . 2022 . Mutual-feed learning for super-resolution and object detection in degraded aerial imagery . IEEE Transactions on Geoscience and Remote Sensing ， 60 ： 1 – 16 ［ DOI： 10.1109/TGRS.2022.3198083 http://dx.doi.org/10.1109/TGRS.2022.3198083 ］
Zhu X ， Su W ， Lu L ， Li B ， Wang X and Dai J . 2020 . Deformable DETR： Deformable transformers for end-to-end object detection ［EB/OL］. ［ 2022-11-05 ］ https://arxiv.org/abs/2010.04159 https://arxiv.org/abs/2010.04159
Zhao Y ， Lv W ， Xu S ， Wei J ， Wang G ， Dang Q ， Liu Y and Chen J . 2024 . DETRs beat YOLOs on real-time object detection // Proceedings of the 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Seattle， USA ： IEEE： 16965 – 16974 ［ DOI： 10.1109/CVPR52733.2024.01605 http://dx.doi.org/10.1109/CVPR52733.2024.01605 ］
Zhu X ， Lyu S ， Wang X and Zhao Q . 2021 . TPH-YOLOv 5 ： Improved YOLOv5 based on transformer prediction head for object detection on drone-captured scenarios //Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision Workshops. Montreal， Canada ： IEEE： 2778 – 2788 ［ DOI： 10.1109/ICCVW54120.2021.00312 http://dx.doi.org/10.1109/ICCVW54120.2021.00312 ］
Zhang Z ， Lu X ， Cao G ， Yang Y and Jiao L . 2021 . ViT-YOLO： Transformer-based YOLO for object detection // Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision Workshops . Montreal， Canada ： IEEE： 2799 – 2808 ［ DOI： 10.1109/ICCVW54120.2021.00314 http://dx.doi.org/10.1109/ICCVW54120.2021.00314 ］
Zhang H ， Li F ， Liu S ， Zhang L ， Su H ， Zhu J ， Ni L M and Shum H Y . 2023 . DINO： DETR with improved denoising anchor boxes for end-to-end object detection ［EB/OL］. ［ 2022-07-11 ］ https://arxiv.org/abs/2203.03605 https://arxiv.org/abs/2203.03605
相关作者
相关机构
京公网安备11010802024621