浏览全部资源
扫码关注微信
Published： 16 December 2024 ，
移动端阅览
杨军， 孙鸿炜. 2024. 边缘特征增强与层次注意力融合的低重叠点云配准. 中国图象图形学报， 29(12):3739-3755
Yang Jun， Sun Hongwei. 2024. Edge feature enhancement and hierarchical attention fusion for low-overlap point cloud registration. Journal of Image and Graphics， 29(12):3739-3755
目的
2
针对目前基于深度学习的低重叠度点云配准方法在学习全局点云场景后进行特征匹配时，忽略局部特征间作用的问题，提出了一种结合边缘特征增强的层次注意力点云配准方法。
方法
2
首先，利用边缘自适应核点卷积（edge adaptive kernel point convolution， EAKPConv）模块提取源点云、目标点云特征，增强边缘特征识别能力。然后，利用局部空间差异注意模块（local spatial contrast attention module， LSCAM）聚合局部空间差异捕捉点云的几何细节，利用序列相似度关联模块（sequential similarity association module， SSAM）计算量化两点云间的相似分数，并利用相似分数引导局部匹配。最后，通过LSCAM模块与SSAM模块结合的层次化注意力融合模块（hierarchical attention fusion module， HAFM）整合局部、全局特征，实现全局匹配。
结果
2
在室内场景点云配准数据集3DMatch和三维模型数据集ModelNet-40上进行了对比实验，本算法在3DMatch和3DLoMatch上的配准召回率分别达到93.2%和67.3%；在ModelNet-40和ModelLoNet-40上取得了最低的旋转误差（分别为1.417和3.141）以及平移误差（分别为0.013 91和0.072）。此外，本文算法在推理效率上比REGTR算法减少了10 ms左右。
结论
2
本文算法通过自底向上的层次化处理方式显著提升了有限重叠场景点云的配准精度，同时降低了推理时间。
Objective
2
Low overlap point cloud registration presents a significant obstacle in the realm of computer vision， specifically in the context of deep-learning-based approaches. After acquiring knowledge from global point cloud scenes for feature matching， these deep-learning-based methods often fail to consider the interactions among local features， thus greatly impeding the efficiency of registration in settings where local feature interactions are vital for establishing precise alignment. The intricate interplay among local characteristics， which is crucial for accurately identifying and aligning partially intersecting point clouds， is also inadequately represented. This lack of consideration not only affects the reliability of point cloud registration in situations with limited overlap but also restricts the use of deep learning methods in varied and intricate settings. Therefore， techniques that include the comprehension of local feature interactions into the deep learning framework are crucial for point cloud registration， especially in situations with limited overlap.
Method
2
The present study introduces a novel technique for aligning point clouds with low overlap. This technique uses the edge adaptive KPConv （EAKPConv） module to enhance the identification of edge characteristics. The integration of local and global features is effectively accomplished by the combination of the hierarchical attention fusion module （HAFM） and the local spatial comparison attention module （LSCAM）. LSCAM exploits the capacity of the attention mechanism to consolidate information， thus enabling the model to prioritize those connections with target nodes and to position itself near the clustered center of mass. In this way， the model can flexibly capture complex details of the point cloud. The SSAM system utilizes a hierarchical architecture， in which each tier of local matching modules applies its own similarity metric to quantify the similarities among point clouds. The local features are subsequently modified and transmitted to the subsequent layer of attention modules to establish a hierarchical structure. This structure also allows the model to collect and merge the inputs from local matches at different scales and levels of complexity， thereby forming global feature correspondences. In this model， the multilayer perceptron （MLP） is used to accurately find the ideal correspondences and successfully complete the alignment procedure.
Result
2
This work provides empirical evidence supporting the improved efficacy of the proposed algorithm as demonstrated by its consistent performance across multiple datasets. Notably， this algorithm achieved impressive registration recall rates of 93.2% and 67.3% on the 3DMatch and 3DLoMatch datasets， respectively. In the experimental evaluation conducted on the ModelNet-40 and ModelLoNet-40 datasets， this algorithm achieved minimal rotational errors of 1.417 degrees and 3.141 degrees， respectively， and recorded translational errors of 0.013 91 and 0.072. These outcomes highlight the effectiveness of this algorithm in point cloud registration and demonstrate its capability to accurately align point clouds with low rotational and translational discrepancies. These results also point to a significant enhancement in the accuracy of the proposed algorithm compared with the REGTR approach. Specifically， in contrast to REGTR， the proposed algorithm achieved significantly reduced inference times of 27.205 ms and 30.991 ms on the 3DMatch and ModelNet-40 datasets， respectively. The findings of this study emphasize the performance of the proposed algorithm in effectively addressing the challenging issue of disregarding features in point cloud registration tasks with minimal overlap.
Conclusion
2
This article presents a novel point cloud matching technique that combines edge improvement with hierarchical attention. This technique integrates polynomial kernel functions into the EAKPConv framework to improve the identification of edge features in point clouds and uses HAFM to extract specific local information. The module improves feature matching by using the similarities in edge features. This approach successfully achieves a harmonious combination of local and global feature matching， hence enhancing the comprehension of point cloud data. Implementing a hierarchical analysis technique greatly increases the registration accuracy by accurately matching local-global information. Furthermore， increasing the cross-entropy loss function enhances the accuracy of local matching and reduces misalignments. This study assesses the performance of the proposed algorithm on the ModelNet-40， ModelelloNet-40， 3DMatch， and 3DLoMatch datasets， and results indicate that this algorithm substantially enhances registration accuracy， particularly in difficult situations with limited data overlap. This algorithm also exhibits superior registration efficiency compared with standard approaches.
三维点云配准低重叠度点云边缘特征层次注意力局部相似匹配
3D point cloud registrationlow-overlap point cloudedge featureshierarchical attentionlocal similar matching
Aoki Y， Goforth H， Srivatsan R A and Lucey S. 2019. PointNetLK： robust and efficient point cloud registration using PointNet//Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. Long Beach， USA： IEEE： 7156-7165 ［DOI： 10.1109/CVPR.2019.00733http://dx.doi.org/10.1109/CVPR.2019.00733］
Bai X Y， Luo Z X， Zhou L， Fu H B， Quan L and Tai C L. 2020. D3Feat： joint learning of dense detection and description of 3D local features//Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. Seattle， USA： IEEE： 6358-6366 ［DOI： 10.1109/CVPR42600.2020.00639http://dx.doi.org/10.1109/CVPR42600.2020.00639］
Bauer D， Patten T and Vincze M. 2021. ReAgent： point cloud registration using imitation and reinforcement learning//Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. Nashville， USA： IEEE： 14581-14589 ［DOI： 10.1109/cvpr46437.2021.01435http://dx.doi.org/10.1109/cvpr46437.2021.01435］
Berg A， Oskarsson M and O’Connor M. 2022. Points to patches： enabling the use of self-attention for 3D shape recognition//Proceedings of the 26th International Conference on Pattern Recognition （ICPR）. Montreal， Canada： IEEE： 528-534 ［DOI： 10.1109/ICPR56361.2022.9956172http://dx.doi.org/10.1109/ICPR56361.2022.9956172］
Besl P J and McKay N D. 1992. A method for registration of 3-D shapes. IEEE Transactions on Pattern Analysis and Machine Intelligence， 14（2）： 239-256 ［DOI： 10.1109/34.121791http://dx.doi.org/10.1109/34.121791］
Cao A Q， Puy G， Boulch A and Marlet R. 2021. PCAM： product of cross-attention matrices for rigid registration of point clouds//Proceedings of 2021 IEEE/CVF International Conference on Computer Vision （ICCV）. Montreal， Canada： IEEE： 13209-13218 ［DOI： 10.1109/ICCV48922.2021.01298http://dx.doi.org/10.1109/ICCV48922.2021.01298］
Chen S Y， Xu H， Li R， Liu G H， Fu C W and Liu S C. 2023. SIRA-PCR： sim-to-real adaptation for 3D point cloud registration//Proceedings of 2023 IEEE/CVF International Conference on Computer Vision （ICCV）. Paris， France： IEEE： 14348-14359 ［DOI： 10.1109/ICCV51070.2023.01324http://dx.doi.org/10.1109/ICCV51070.2023.01324］
Chen Z L， Chen H H， Gong L N， Yan X F， Wang J， Guo Y W， Qin J and Wei M Q. 2022. UTOPIC： uncertainty-aware overlap prediction network for partial point cloud registration. Computer Graphics Forum， 41（7）： 87-98 ［DOI： doi.org/10.1111/cgf.14659http://dx.doi.org/doi.org/10.1111/cgf.14659］
Choy C， Park J and Koltun V. 2019. Fully convolutional geometric features//Proceedings of 2019 IEEE/CVF International Conference on Computer Vision （ICCV）. Seoul， Korea （South）： IEEE： 8957-8965 ［DOI： 10.1109/ICCV.2019.00905http://dx.doi.org/10.1109/ICCV.2019.00905］
Huang S Y， Gojcic Z， Usvyatsov M， Wieser A and Schindler K. 2021. PREDATOR： registration of 3D point clouds with low overlap//Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. Nashville， USA： IEEE： 4265-4274 ［DOI： 10.1109/CVPR46437.2021.00425http://dx.doi.org/10.1109/CVPR46437.2021.00425］
Huang X S， Mei G F and Zhang J. 2020. Feature-metric registration： a fast semi-supervised approach for robust point cloud registration without correspondences//Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. Seattle， USA： IEEE： 11363-11371 ［DOI： 10.1109/CVPR42600.2020.01138http://dx.doi.org/10.1109/CVPR42600.2020.01138］
Huang Z L， Wang X G， Wei Y C， Huang L C， Shi H， Liu W Y and Huang T S. 2023. CCNet： criss-cross attention for semantic segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence， 45（6）： 6896-6908 ［DOI： 10.1109/TPAMI.2020.3007032http://dx.doi.org/10.1109/TPAMI.2020.3007032］
Pais G D， Ramalingam S， Govindu V M， Nascimento J C， Chellappa R and Miraldo P. 2020. 3DRegNet： a deep neural network for 3D point registration//Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. Seattle， USA： IEEE： 7191-7201 ［DOI： 10.1109/cvpr42600.2020.00722http://dx.doi.org/10.1109/cvpr42600.2020.00722］
Qi C R， Su H， Mo K C and Guibas L J. 2017. PointNet： deep learning on point sets for 3D classification and segmentation//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition （CVPR）. Honolulu， USA： IEEE： 77-85 ［DOI： 10.1109/CVPR.2017.16http://dx.doi.org/10.1109/CVPR.2017.16］
Qin H X， Liu Z T and Tan B Y. 2022. Review on deep learning rigid point cloud registration. Journal of Image and Graphics， 27（2）： 329-348
秦红星， 刘镇涛， 谭博元. 2022. 深度学习刚性点云配准前沿进展. 中国图象图形学报， 27（2）： 329-348 ［DOI： 10.11834/jig.210556http://dx.doi.org/10.11834/jig.210556］
Ross G T and Soland R M. 1975. A branch and bound algorithm for the generalized assignment problem. Mathematical Programming， 8（1）： 91-103 ［DOI： 10.1007/BF01580430http://dx.doi.org/10.1007/BF01580430］
Rusu R B， Blodow N and Beetz M. 2009. Fast point feature histograms （FPFH） for 3D registration//Proceedings of 2009 IEEE International Conference on Robotics and Automation. Kobe， Japan： IEEE： 3212-3217 ［DOI： 10.1109/ROBOT.2009.5152473http://dx.doi.org/10.1109/ROBOT.2009.5152473］
Rusu R B， Marton Z C， Blodow N and Beetz M. 2008. Persistent point feature histograms for 3D point clouds//Proceedings of the 10th International Conference on Intelligent Autonomous Systems （IAS-10）. Baden-Baden， Germany： IOS： 119-128 ［DOI： 10.3233/978-1-58603-887-8-119http://dx.doi.org/10.3233/978-1-58603-887-8-119］
Sarode V， Li X Q， Goforth H， Aoki Y， Srivatsan R A， Lucey S and Choset H. 2019. PCRNet： point cloud registration network using PointNet encoding ［EB/OL］. ［2024-03-05］. https://arxiv.org/pdf/1908.07906.pdfhttps://arxiv.org/pdf/1908.07906.pdf
Thomas H， Qi C R， Deschaud J E， Marcotegui B， Goulette F and Guibas L. 2019. KPConv： flexible and deformable convolution for point clouds//Proceedings of 2019 IEEE/CVF International Conference on Computer Vision （ICCV）. Seoul， Korea （South）： IEEE： 6410-6419 ［DOI： 10.1109/ICCV.2019.00651http://dx.doi.org/10.1109/ICCV.2019.00651］
Vaswani A， Shazeer N， Parmar N， Uszkoreit J， Jones L， Gomez A N， Kaiser Ł and Polosukhin I. 2017. Attention is all you need//Proceedings of the 31st International Conference on Neural Information Processing Systems. Long Beach， USA： Curran Associates Inc.： 6000-6010
Wang Y and Solomon J. 2019a. PRNet： self-supervised learning for partial-to-partial registration//Proceedings of the 33rd International Conference on Neural Information Processing Systems. Vancouver， Canada： Curran Associates Inc.： 8814-8826
Wang Y and Solomon J. 2019b. Deep closest point： learning representations for point cloud registration//Proceedings of 2019 IEEE/CVF International Conference on Computer Vision （ICCV）. Seoul， Korea （South）： IEEE： 3522-3531 ［DOI： 10.1109/ICCV.2019.00362http://dx.doi.org/10.1109/ICCV.2019.00362］
Wang Y， Sun Y B， Liu Z W， Sarma S E， Bronstein M M and Solomon J M. 2019. Dynamic graph CNN for learning on point clouds. ACM Transactions on Graphics， 38（5）： #146 ［DOI： 10.1145/3326362http://dx.doi.org/10.1145/3326362］
Wu Y， Yuan Y Z， Xiang B H， Sheng J L， Lei J Y， Hu C Y， Gong M G， Ma W P and Miao Q G. 2023. Overview of the computational intelligence method in 3D point cloud registration. Journal of Image and Graphics， 28（9）： 2763-2787
武越， 苑咏哲， 向本华， 绳金龙， 雷佳熠， 胡聪颖， 公茂果， 马文萍， 苗启广. 2023. 三维点云配准中的计算智能方法综述. 中国图象图形学报， 28（9）： 2763-2787 ［DOI： 10.11834/jig.220727http://dx.doi.org/10.11834/jig.220727］
Wu Z R， Song S R， Khosla A， Yu F， Zhang L G， Tang X O and Xiao J X. 2015. 3D ShapeNets： a deep representation for volumetric shapes//Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition （CVPR）. Boston， USA： IEEE： 1912-1920 ［DOI： 10.1109/CVPR.2015.7298801http://dx.doi.org/10.1109/CVPR.2015.7298801］
Xie Z Y， Chen J Z and Peng B. 2020. Point clouds learning with attention-based graph convolution networks. Neurocomputing， 402： 245-255 ［DOI： 10.1016/j.neucom.2020.03.086http://dx.doi.org/10.1016/j.neucom.2020.03.086］
Xu H， Liu S X， Wang G F， Liu G H and Zeng B. 2021. OMNet： learning overlapping mask for partial-to-partial point cloud registration//Proceedings of 2021 IEEE/CVF International Conference on Computer Vision （ICCV）. Montreal， Canada： IEEE： 3112-3121 ［DOI： 10.1109/ICCV48922.2021.00312http://dx.doi.org/10.1109/ICCV48922.2021.00312］
Yan Y J， An J Y， Zhao J and Shen F R. 2023. Hybrid optimization with unconstrained variables on partial point cloud registration. Pattern Recognition， 136： #109267 ［DOI： 10.1016/j.patcog.2022.109267http://dx.doi.org/10.1016/j.patcog.2022.109267］
Yang J and Zhang C. 2023. Semantic segmentation of 3D point cloud by fusing dual attention mechanism and dynamic graph convolution neural network （杨军， 张琛. 2023. 融合双注意力机制和动态图卷积神经网络的三维点云语义分割） ［EB/OL］. ［2023-12-12］. https://bhxb.buaa.edu.cn/bhzk/cn/article/doi/10.13700/j.bh.1001-5965.2022.0775https://bhxb.buaa.edu.cn/bhzk/cn/article/doi/10.13700/j.bh.1001-5965.2022.0775
Yang J， Zhang Y and Huang L. 2018. Research on 3D model registration by improved ICP algorithm. Journal of Frontiers of Computer Science and Technology， 12（1）： 153-162
杨军， 张瑶， 黄亮. 2018. 改进的ICP算法在三维模型配准中的研究. 计算机科学与探索， 12（1）： 153-162 ［DOI： 10.3778/j.issn.1673-9418.1610033http://dx.doi.org/10.3778/j.issn.1673-9418.1610033］
Yang J L， Li H D， Campbell D and Jia Y D. 2016. Go-ICP： a globally optimal solution to 3D ICP point-set registration. IEEE Transactions on Pattern Analysis and Machine Intelligence， 38（11）： 2241-2254 ［DOI： 10.1109/TPAMI.2015.2513405http://dx.doi.org/10.1109/TPAMI.2015.2513405］
Yang J Q， Zhang X Y， Fan S C， Ren C L and Zhang Y N. 2024. Mutual voting for ranking 3D correspondences. IEEE Transactions on Pattern Analysis and Machine Intelligence， 46（6）： 4041-4057 ［DOI： 10.1109/TPAMI.2023.3268297http://dx.doi.org/10.1109/TPAMI.2023.3268297］
Yew Z J and Lee G H. 2020. RPM-Net： robust point matching using learned features//Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. Seattle， USA： IEEE： 11821-11830 ［DOI： 10.1109/CVPR42600.2020.01184http://dx.doi.org/10.1109/CVPR42600.2020.01184］
Yew Z J and Lee G H. 2022. REGTR： end-to-end point cloud correspondences with transformers//Proceedings of 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. New Orleans， USA： IEEE： 6667-6676 ［DOI： 10.1109/CVPR52688.2022.00656http://dx.doi.org/10.1109/CVPR52688.2022.00656］
Zeng A， Song S R， Nießner M， Fisher M， Xiao J X and Funkhouser T. 2017. 3DMatch： learning local geometric descriptors from RGB-D reconstructions//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition （CVPR）. Honolulu， USA： IEEE： 199-208 ［DOI： 10.1109/CVPR.2017.29http://dx.doi.org/10.1109/CVPR.2017.29］
Zhou Q Y， Park J and Koltun V. 2016. Fast global registration//Proceedings of the 14th European Conference on Computer Vision. Amsterdam， the Netherlands： Springer： 766-782 ［DOI： 10.1007/978-3-319-46475-6_47http://dx.doi.org/10.1007/978-3-319-46475-6_47］
相关作者
相关机构
京公网安备11010802024621