浏览全部资源
扫码关注微信
收稿日期：2024-09-30，
修回日期：2025-02-15，
录用日期：2025-03-03，
网络出版日期：2025-03-04，
移动端阅览
目的
2
地点识别是机器人利用实时扫描到的点云数据进行定位和自主导航的核心。现有的针对大规模点云的地点识别方法往往忽略了真实驾驶中存在的旋转问题。当查询场景发生旋转时，这些方法识别性能会显著下降，这严重阻碍了它们在复杂现实场景中的应用。因此，本文提出一种有效的面向三维点云的具有旋转感知地点识别网络（Efficient Rotation-Aware Network for Point Cloud based Place Recognition，ERA-Net）。
方法
2
首先，利用自注意机制与邻域注意力机制，在捕捉点与点之间的全局依赖关系的同时，捕捉每个点与其邻域点之间的局部依赖关系，充分提取点间的语义特征。同时，利用点与其
k
邻近点的坐标信息，计算距离、角度以及角度差等低维几何特征，并设计基于特征距离的注意力池化模块，通过在高维空间分析特征之间的相关性，提取具有较强区分性且具有旋转特性的几何特征。最后，将提取的语义特征以及几何特征进行
有效融合，通过NetVLAD模块，产生更具判别性的全局描述符。
结果
2
将提出的ERA-Net在公共数据集Oxford Robotcar上进行验证并与最先进的方法（state-of-the-art methods，SOTA）进行比较。在Oxford数据集中，ERA-Net的Average Recall@1%指标可以达到96.48%，在University Sector（U.S.）、Residential Area（R.A.）以及Business District（B.D.）数据集上的识别效果均优于其它方法。特别的，当查询场景进行旋转时，ERA-Net的识别效果优于已有方法。
结论
2
实验结果表明，ERA-Net能够充分考虑点间的上下文信息，以及特征间的相关性，提取具有较强独特性的场景特征，在面对旋转问题时能够展现出较好的鲁棒性，具有较强的泛化能力。
Objective
2
Point cloud based place recognition involves identifying and matching a specific location within a pre-built map by comparing global descriptors generated from the scene. This process is crucial for robot localization and autonomous navigation， where the real-time point cloud is continuously scanned. For instance， autonomous vehicles and robots rely on robust place recognition to safely and efficiently navigate， ensuring they can recognize and re-identify locations within a scene. The primary goal of place recognition is to enable a system to accurately determine whether it has previously encountered a location， which is essential for tasks like loop closure in Simultaneous Localization and Mapping （SLAM） and re-localization in dynamic environments. Place recognition methods can be categorized into two approaches according to the data type： Image-based methods and LiDAR-based methods. Image -based methods have achieved promising results using 2D images in the place recognition task. However， the performance of image-based methods is easily degraded in real-world environments owing to illumination changes and viewpoint variation. As point clouds can be captured easily by 3D sensors and provide high precision geometry information of objects， it can avoid misleading caused by lighting change and viewpoint variation to some extent. While many methods have been proposed for 3D place recognition， these methods typically perform well in the prescribed travel routes. However， a significant challenge often overlooked is the effect of data rotation. In real-world scenarios， vehicles do not travel on fixed routes. When a vehicle deviates from a prescribed route in the same place， the acquired point clouds are equivalent to be rotated relative to the submaps. Existing location recognition methods for large-scale point clouds often ignore the rotation problem in real driving. When the query scene is rotated， the recognition performance of these methods degrades significantly， which seriously hinders their application in complex real-world scenarios.
Method
2
In this paper， we propose an Efficient Rotation-Aware Network for Point Cloud based Place Recognition （ERA-Net） for point cloud based place recognition that leverages semantic and geometric features for better discriminative ability. ERA-Net employs an attention mechanism that thoroughly considers the global dependencies among points as well as the local correlations between a point and its neighboring points to extract semantic features. Additionally， it utilizes the coordinate information of a point and its
k
nearest neighbors to derive low-dimensional geometric features that exhibit rotational invariance， such as distance， angle， and angular difference. By calculating the associations of these features in a high-dimensional space， it extracts geometric characteristics with strong uniqueness and rotational invariance. First， global dependencies between points are captured using a self-attention mechanism， while local dependencies between each point and its
k
-nearest neighboring points are modeled through neighboring-attention mechanisms. These mechanisms ensure the comprehensive extraction of attention features between points. The self-attention mechanism computes self-coefficients by considering the geometric properties of individual points， whereas the neighboring-attention mechanism focuses on local coefficients by considering the neighborhood structure. At the same time， se
veral low-dimensional geometric features are calculated by using the coordinates of points. The sampled points， their local center of mass， and the
k
-nearest neighbors form a stable local triangular structure. Low-dimensional geometric features such as distances， angles， and angle differences are calculated. These geometric features remain invariant to point cloud rotations. Next， a novel feature-distance-based attention pooling module is designed to extract geometric features with high discrimination and rotational invariance by analyzing the correlations between features in high-dimensional space. Feature distance measures the similarity between different points in high-dimensional feature space. Points may be geometrically distant from each other in terms of their spatial positions. However， they can share similar features semantically. The feature-distance-based pooling method captures the relationships between these geometrically distant but semantically similar points. It can also effectively distinguish points with similar spatial coordinates but large semantic differences. Finally， the extracted semantic and geometric features are fused through the NetVLAD module， generating more discriminative global descriptors for the point cloud.
Result
2
The proposed ERA-Net is validated on the public dataset Oxford Robotcar and compared with the state-of-the-art methods. In the Oxford dataset， the Average Recall@1% of ERA-Net reaches 96.48% and the Average Recall@1 reaches 90.47%. It outperforms other SOTA（state-of-the-art methods） on the U.S. （university area）， R.A. （business area）， and B.D. （residential area） datasets. We select three representative queries of the Oxford dataset to verify the retrieval capability of ERA-Net. Even though the scene contains structurally unstable trees， slender poles， or a large number of planar points， ERA-Net can retrieve the correct place. In particular， ERA-Net outperforms the existing methods in terms of recognition when the query scene is rotated. When the query scene is rotated， the recognition performance of most existing methods decreases significantly. The results of Average Recall@1% and Average Recall@1 of ERA-Net and other SOTA methods under different rotation angles are tested respectively. The tested rotation angles include 30， 60， 90， 120， 150， and 180 degrees， with these angles representing the maximum rotation range. The query point cloud is randomly rotated from 0 degrees to the maximum rotation angle. The performance of ERA-Net is better than other methods when the rotation angle of the query scene is increasing. The experimental results show that ERA-Net can still extract the features with strong uniqueness and rotation invariance from the scene， and correctly retrieve the corresponding scene even when facing the rotation of point cloud data.
Conclusion
2
In this paper， we propose a rotation aware place recognition network ERA-Net that leverages the semantic and geometric features. The experimental results show that the recognition performance of ERA-Net is better than the state-of-the-art methods. ERA-Net performs significantly better than other methods on the three in-house datasets. For the AR@1%， ERA-Net reaches 91.04%， 87.37%， and 87.12% on U.S.， R.A. and B.D. datasets， respectively. At the same time， ERA-Net also achieves better results in the rotation test. The extraction of low-dimensional geometric features can better improve the recognition effect of the ERA-Net. The attention pooling module based on feature distance can capture the relationship between distant but semantically similar points， and effectively distinguish the points with similar coordinates but large semantic differences. ERA-Net is able to extract scene features with strong uniqueness by fully considering the contextual information between the points and the correlation between the features. It has good robustness and strong generalization ability when facing the rotation problem.
Ali-Bey A ， Chaib-Draa B ， Giguere P . 2023 . Mixvpr： Feature mixing for visual place recognition // Proceedings of the IEEE/CVF winter conference on applications of computer vision . Waikoloa， HI， USA ： IEEE： 2998 - 3007 .［ DOI： 10.1109/wacv56688.2023.00301 http://dx.doi.org/10.1109/wacv56688.2023.00301 ］
Arandjelovic R ， Gronat P. ， Torii A ， Pajdla T ， and Sivic J . 2016 . NetVLAD： CNN architecture for weakly supervised place recognition // Proceedings of the IEEE conference on computer vision and pattern recognition ， Las Vegas， NV， USA ： IEEE： . 5297 – 5307 . ［ DOI： 10.1109/cvpr.2016.572 http://dx.doi.org/10.1109/cvpr.2016.572 ］
Chen C ， Fragonara L Z ， Tsourdos A . 2021 . GAPointNet： Graph attention based point neural network for exploiting local feature of point cloud . Neurocomputing ， 438 ： 122 - 132 .［ DOI： 10.1016/j.neucom.2021.01.095 http://dx.doi.org/10.1016/j.neucom.2021.01.095 ］
Chen Y ， Zhuang Y ， Huai J ， Li Q ， Wang B ， El-Bendary N ， Yilmaz A . 2024 . Semantics-enhanced discriminative descriptor learning for LiDAR-based place recognition . ISPRS Journal of Photogrammetry and Remote Sensing ， 210 ： 97 - 109 .［ DOI： 10.1016/j.isprsjprs.2024.03.002 http://dx.doi.org/10.1016/j.isprsjprs.2024.03.002 ］
Dubé R ， Dugas D ， Stumm E ， Nieto J ， Siegwart R ， Cadena C . 2017 . Segmatch： Segment based place recognition in 3D point clouds // IEEE International Conference on Robotics and Automation （ICRA） . Singapore ： IEEE： 5266 - 5272 . ［ DOI： 10.1109/icra.2017.7989618 http://dx.doi.org/10.1109/icra.2017.7989618 ］
Fan S ， Dong Q ， Zhu F ， Lv Y ， Ye P ， Wang FY . 2021 . SCF-Net： Learning spatial contextual features for large-scale point cloud segmentation // Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . Nashville， TN， USA ： IEEE： 14504 - 14513 . ［ DOI： 10.1109/cvpr46437.2021.01427 http://dx.doi.org/10.1109/cvpr46437.2021.01427 ］
Fan Z ， Liu H ， He J ， Sun Q ， Du X . 2020 . SRNet： A 3D scene recognition network using static graph and dense semantic fusion ， Computer Graphics Forum . 39 （ 7 ）： 301 - 311 . ［ DOI： 10.1111/cgf.14146 http://dx.doi.org/10.1111/cgf.14146 ］
Fan Z ， Song Z ， Zhang W ， Liu H ， He J ， Du X . 2022 . Rpr-net： A point cloud-based rotation-aware large scale place recognition network // Proceedings of the European Conference on Computer Vision ， Tel Aviv， Israel ： IEEE： 709 - 725 . ［ DOI： 10.1007/978-3-031-25056-9_45 http://dx.doi.org/10.1007/978-3-031-25056-9_45 ］
Hao W ， Zhang W ， Jin H . 2024 . SAGE-Net： Employing Spatial Attention and Geometric Encoding for Point Cloud based Place Recognition . IEEE Robotics and Automation Letters ， 9 （ 6 ）： 4958 - 4965 .［ DOI： 10.1109/LRA.2024.3387112 http://dx.doi.org/10.1109/LRA.2024.3387112 ］
Hao W ， Zhang W ， Liang W ， Xiao Z ， Jin H . 2022 . Scene recognition for 3D point clouds： a review ， Optics and Precision Engineering ， 30 （ 16 ）： 1988 - 2005 .
郝雯 ， 张雯静 ， 梁玮 ， 肖照林 ， 金海燕 ， 面向三维点云的场景识别方法综述 ， 光学精密工程 ， 2022 ， 30 （ 16 ）： 1988 - 2005 .［ DOI： CNKI：SUN：GXJM.0.2022-16-008 http://dx.doi.org/CNKI：SUN：GXJM.0.2022-16-008 ］
He L ， Wang X ， Zhang H . 2016 . M 2 DP： A novel 3D point cloud descriptor and its application in loop closure detection // IEEE/RSJ International Conference on Intelligent Robots and Systems （IROS） . Daejeon， Korea （South） ： IEEE： 231 - 237 . ［ DOI ： 10.1109/iros.2016.7759060 http://dx.doi.org/10.1109/iros.2016.7759060 ］
Hou Z ， Yan Y ， Xu C ， Kong H . 2022 . Hitpr： Hierarchical transformer for place recognition in point cloud // 2022 International Conference on Robotics and Automation （ICRA） . Philadelphia， PA， USA ： IEEE： 2612 - 2618 . ［ DOI： 10.1109/icra46639.2022.9811737 http://dx.doi.org/10.1109/icra46639.2022.9811737 ］
Hui L ， Cheng M ， Xie J ， Yang J ， Cheng M . 2022 . Efficient 3D point cloud feature learning for large-scale place recognition . IEEE Transactions on Image Processing ， 31 ： 1258 - 1270 . ［ DOI： 10.1109/tip.2021.3136714 http://dx.doi.org/10.1109/tip.2021.3136714 ］
Hui L ， Yang H ， Cheng M ， Xie J ， Yang J . 2021 . Pyramid point cloud transformer for large-scale place recognition // Proceedings of the IEEE/CVF International Conference on Computer Vision . Montreal， QC， Canada ： IEEE： 6098 - 6107 . ［ DOI： 10.1109/iccv48922.2021.00604 http://dx.doi.org/10.1109/iccv48922.2021.00604 ］
Izquierdo S ， Civera J . 2024 . Optimal transport aggregation for visual place recognition // Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . Seattle， WA， USA ： IEEE： 17658 - 17668 . ［ DOI： 10.1109/cvpr52733.2024.01672 http://dx.doi.org/10.1109/cvpr52733.2024.01672 ］
Jaegle A ， Gimeno F ， Brock A . 2021 .
Vinyals O ， Zisserman A ， Carreira J . Perceiver： General perception with iterative attention // International conference on machine learning . PMLR，： 4651 - 4664 . ［ DOI： 10.48550/arXiv.2103.03206 http://dx.doi.org/10.48550/arXiv.2103.03206 ］
Komorowski J . 2021 . Minkloc3 d： Point cloud based large-scale place recognition// Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision . Waikoloa， HI， USA ： IEEE： 1790 - 1799 . ［ DOI： 10.1109/wacv48630.2021.00183 http://dx.doi.org/10.1109/wacv48630.2021.00183 ］
Kong X ， Yang X ， Zhai G ， Zhao X ， Zeng X ， Wang M ， Liu Y ， Li W ， Wen F . 2020 . Semantic graph based place recognition for 3D point clouds // IEEE/RSJ International Conference on Intelligent Robots and Systems （IROS） . Las Vegas， NV， USA ： IEEE： 8216 - 8223 . ［ DOI： 10.1109/iros45743.2020.9341060 http://dx.doi.org/10.1109/iros45743.2020.9341060 ］
Li L ， Kong X ， Zhao X ， Huang T ， Li W ， Wen F ， Zhang H ， Liu Y . 2022 . RINet： Efficient 3D lidar-based place recognition using rotation invariant neural network . IEEE Robotics and Automation Letters ， 7 （ 2 ）： 4321 - 4328 . ［ DOI： 10.1109/lra.2022.3150499 http://dx.doi.org/10.1109/lra.2022.3150499 ］
Li Q ， Zhuang Y ， Huai J ， Chen Y ， Yilmaz A . 2024 . An efficient point cloud place recognition approach based on transformer in dynamic environment . ISPRS Journal of Photogrammetry and Remote Sensing ， 207 ： 14 - 26 . ［ DOI： 10.1016/j.isprsjprs.2023.11.013 http://dx.doi.org/10.1016/j.isprsjprs.2023.11.013 ］
Liu C ， Chen G ， Song R . 2024 . LPS-Net： Lightweight Parameter-shared Network for Point Cloud-based Place Recognition // 2024 IEEE International Conference on Robotics and Automation （ICRA） . Yokohama， Japan ： IEEE ， 448 - 454 .［ DOI： 10.1109/ICRA57147.2024.10610758 http://dx.doi.org/10.1109/ICRA57147.2024.10610758 ］
Liu Y ， Yu L ， Yu H ， Xiao Z ， Yang W . 2024 . Visual place recognition with fusion event cameras ， Journal of Image and Graphics ， 29 （ 04 ）： 1018 - 1029 .
刘熠晨 ， 余磊 ， 余淮 ， 杨文 . 融合事件相机的视觉场景识别 . 中国图象图形学报 ， 2024 ， 29 （ 04 ）： 1018 - 1029 .［ DOI： CNKI：SUN：GXJM.0.2022-16 -008 http://dx.doi.org/CNKI：SUN：GXJM.0.2022-16-008 ］
Liu Z ， Zhou S ， Suo C ， Yin P ， Chen W ， Wang H ， Li H ， Liu YH . 2019 . Lpd-net： 3d point cloud learning for large-scale place recognition and environment analysis // Proceedings of the IEEE/CVF international conference on computer vision . Seoul， Korea （South） ： IEEE： 2831 - 2840 . ［ DOI： 10.1109/iccv.2019.00292 http://dx.doi.org/10.1109/iccv.2019.00292 ］
Maddern W ， Pascoe G ， Linegar C ， Newman P . 2017 . 1 year， 1000 km： The oxford robotcar dataset . The International Journal of Robotics Research ， 36 （ 1 ）： 3 - 15 . ［ DOI： 10.1177/0278364916679498 http://dx.doi.org/10.1177/0278364916679498 ］
Qi C R ， Su H ， Mo K ， Guibas LJ . 2017 . Pointnet： Deep learning on point sets for 3d classification and segmentation // Proceedings of the IEEE conference on computer vision and pattern recognition . Hawaii ， USA， IEEE： 652 - 660 . ［ DOI： 10.1109/cvpr.2017.16 http://dx.doi.org/10.1109/cvpr.2017.16 ］
Ramezani M ， Wang L ， Knights J ， Li Z ， Pounds P ， Moghadam P . 2024 . Pose-Graph attentional graph neural network for Lidar place recognition . IEEE Robotics and Automation Letters ， 9 （ 2 ）： 1182 - 1189 . ［ DOI： 10.1109/lra.2023.3341766 http://dx.doi.org/10.1109/lra.2023.3341766 ］
Rizzini D L . 2017 . Place recognition of 3D landmarks based on geometric relations // IEEE/RSJ International Conference on Intelligent Robots and Systems （IROS） . Vancouver， BC， Canada ： IEEE： 648 - 654 . ［ DOI： 10.1109/iros.2017.8202220 http://dx.doi.org/10.1109/iros.2017.8202220 ］
Röhling T ， Mack J ， Schulz D . 2015 . A fast histogram-based similarity measure for detecting loop closures in 3D LiDAR data // IEEE/RSJ International Conference on Intelligent Robots and Systems （IROS） . Hamburg，Germany ： IEEE： 736 - 741 . ［ DOI： 10.1109/iros.2015.7353454 http://dx.doi.org/10.1109/iros.2015.7353454 ］
Shu D W ， Kwon J . 2024 . Hierarchical bidirected graph convolutions for large-scale 3-D point cloud place recognition . IEEE Transactions on Neural Networks and Learning Systems . 35 （ 7 ）： 9651 - 9662 . ［ DOI： 10.1109/tnnls.2023.3236313 http://dx.doi.org/10.1109/tnnls.2023.3236313 ］
Sun Q ， Liu H ， He J ， Fan Z ， Du X . 2020 . Dagc： Employing dual attention and graph convolution for point cloud based place recognition // Proceedings of the 2020 International Conference on Multimedia Retrieval . New York， USA ： ACM： 224 - 232 . ［ DOI： 10.1145/3372278.3390693 http://dx.doi.org/10.1145/3372278.3390693 ］
Uy M A ， Lee G H . 2018 . Pointnetvlad： Deep point cloud based retrieval for large-scale place recognition // Proceedings of the IEEE conference on computer vision and pattern recognition . Salt Lake City， UT， USA ： IEEE： 4470 - 4479 . ［ DOI： 10.1109/cvpr.2018.00470 http://dx.doi.org/10.1109/cvpr.2018.00470 ］
Wen C ， Huang H ， Liu Y S ， Fang Y . 2023 . Pyramid learnable tokens for 3D LiDAR place recognition // 2023 IEEE International Conference on Robotics and Automation （ICRA） . London， United Kingdom ： IEEE： 4143 - 4149 . ［ DOI： 10.1109/icra48891.2023. 10161523 http://dx.doi.org/10.1109/icra48891.2023.10161523 ］
Weng S ， Zhang R ， Li G ， 2022 . ERINet ： Effective rotation invariant network for point cloud based place recognition， 2022 IEEE International Conference on Visual Communications and Image Processing， Suzhou， China ：IEEE： 1- 5 . ［ DOI： 10.1109/vcip56404.2022.10008830 http://dx.doi.org/10.1109/vcip56404.2022.10008830 ］
Wiesmann L ， Marcuzzi R ， Stachniss C ， Behley J . 2022 . Retriever： Point cloud retrieval in compressed 3D maps // 2022 international conference on robotics and automation （ICRA） . Philadelphia， PA， USA ： IEEE： 10925 - 10932 . ［ DOI： 10.1109/icra46639.2022.9811785 http://dx.doi.org/10.1109/icra46639.2022.9811785 ］
Xia Y ， Xu Y ， Li S ， Wang R ， Du J ， Cremers D ， Stilla U . 2021 . SOE-Net： A self-attention and orientation encoding network for point cloud based place recognition // Proceedings of the IEEE/CVF Conference on computer vision and pattern recognition . Nashville， TN， USA ： IEEE： 11348 - 11357 . ［ DOI： 10.1109/cvpr46437.2021.01119 http://dx.doi.org/10.1109/cvpr46437.2021.01119 ］
Zhao S ， Yin P ， Yi G ， Scherer S . 2022 . Spherevlad++： Attention-based and signal-enhanced viewpoint invariant descriptor . IEEE Robotics and Automation Letters ， 8 （ 1 ）： 256 - 263 . ［ DOI： 10.1109/lra.2022.3223555 http://dx.doi.org/10.1109/lra.2022.3223555 ］
Zhang W ， Xiao C . 2019 . PCAN： 3D attention map learning using contextual information for point cloud based retrieval // Proceedings of the IEEE/CVF conference on computer vision and pattern recognition . Long Beach， CA， USA ： IEEE： 12436 - 12445 . ［ DOI： 10.1109/cvpr.2019.01272 http://dx.doi.org/10.1109/cvpr.2019.01272 ］
Zhang Y ， Shi P ， Li J . 2024 . Lidar-based place recognition for autonomous driving： A survey . ACM Computing Surveys ， 1 - 26 . ［ DOI： 10.1145/3707446 http://dx.doi.org/10.1145/3707446 ］
Zhang Z ， Hua B S ， Yeung S K . 2022 . Riconv++： Effective rotation invariant convolutions for 3d point clouds deep learning . International Journal of Computer Vision ， 130 （ 5 ）： 1228 - 1243 . ［ DOI： 10.1007/s11263-022-01601-z http://dx.doi.org/10.1007/s11263-022-01601-z ］
相关作者
相关机构
京公网安备11010802024621