浏览全部资源
扫码关注微信
Published： 16 July 2024 ，
移动端阅览
沈澍， 张文昊， 丁浩， 张浩， 沙超， 王森， 陈书军. 2024. 三维步态识别研究进展. 中国图象图形学报， 29(07):1921-1933
Shen Shu， Zhang Wenhao， Ding Hao， Zhang Hao， Sha Chao， Wang Sen， Chen Shujun. 2024. Research progress of three-dimensional gait recognition. Journal of Image and Graphics， 29(07):1921-1933
步态识别在身份识别领域具有重要的研究意义。随着技术的发展，步态识别的研究热点正从二维（2D）转向三维（3D）。与图像固有的2D信息相比，用视觉技术还原的3D信息能更有效地预测人员的身份。在2D视觉领域中，由于受到物体遮挡、视角变化等因素的影响，传统的步态识别方法在实际应用中难以取得理想的识别性能。基于人体3D重建和人体3D姿态估计等3D人体技术，近年来的研究在3D步态识别领域取得了一系列进展。本文介绍了3D步态识别方法，探讨了基于3D步态的身份识别领域的研究现状、优势与不足；总结了主要的3D步态数据集；讨论了3D识别方法与2D识别方法的对比；提出了3D身份识别领域未来潜在的研究方向，包括3D数据集的采集和整理、2D 和 3D 数据的多模态融合等。
Gait recognition is a new biometric identification technology that uses human walking posture and gait to determine a person’s identity. Face recognition， which is considered traditional biometric recognition technology， is widely used， but it has the following defects： 1） recognition distance is limited； 2） it is vulnerable to occlusion and light and other factors； and 3） the results are at risk of being attacked by using face photos， video playback， and three-dimensional masks. In contrast， gait has the following advantages： 1） it can be identified from a long distance； 2） it is less affected by occlusion and illumination； and 3） it is not easy to disguise and deceive. Therefore， gait recognition is playing an increasingly important role in public safety， biometric authentication， video surveillance， and other fields. Gait recognition is mainly divided into two categories according to the dimension of input data： 1） two-dimensional （2D） gait recognition based on 2D data and 2） three-dimensional （3D） gait recognition based on 3D data. At present， the research review in the field of gait recognition focuses on 2D gait recognition， usually from the perspective of traditional machine learning or deep learning. Gait recognition is moving from 2D to 3D. Compared with the inherent 2D information of the image， the 3D information restored by visual technology can more effectively predict human identity. In the field of 2D vision， traditional gait recognition methods have difficulty achieving better recognition performance because of the influence of object occlusion and view changes. On the basis of 3D human technology such as 3D human reconstruction and 3D human pose estimation， a series of progress has been made in the field of 3D gait recognition in recent years. To fully understand the existing research in the field of 3D gait recognition， this paper reviews and summarizes the research in this field. This paper discusses the research status， advantages， and disadvantages of identity recognition based on 3D gait； summarizes 3D gait recognition methods and 3D gait datasets； and provides potential research directions in the field of 3D identity recognition. This paper summarizes the different input data of existing 3D gait recognition methods and the recognition effect （recognition accuracy and speed） of these methods. These methods include multi-view-based， depth image-based， 3D skeleton-based， 3D point cloud-based， and 3D reconstruction-based recognition methods. This paper divides the 3D gait dataset into the indoor dataset and the outdoor dataset according to the acquisition environment. The 3D gait data include depth images， 3D skeleton， 3D human body grid， and 3D point cloud. In addition， this paper collates and compares the experimental results of different 3D gait recognition methods on various 3D gait datasets. Finally， this paper provides potential research directions for the field of 3D identity recognition. 1） Performance improvement and model optimization. Different from 2D gait， the performance of 3D gait is closely related to the 3D model. The 3D deep learning model needs to be optimized to improve the recognition performance of 3D gait in real scenes. For example， the training skills of vision Transformer （ViT） to improve performance are applied to 3D models such as 3D convolutional neural networks to improve the generalization and robustness of the model. The 3D model with the ViT concept is expected to learn more discriminative features from 3D gait data. 2） Collection and collation of 3D datasets. Compared with the 2D gait data set， the number of 3D gait datasets is small and the data types are not rich enough， which limits the development of 3D gait and requires further data collection by researchers. When the collected 3D gait dataset is sorted out， the training set and the test set can be divided in advance. For the test set， the registration set and the verification set are expected to be divided again， and the baseline algorithm that is easy to reproduce is used for evaluation. Rank-1 accuracy and mean average precision can be used as evaluation metrics. 3） Multi-modal fusion of 2D and 3D data. Compared with 2D data， 3D data contains more information， so the effective use of 3D data can improve the recognition performance in real scenes. In the field of gait recognition， current research mainly focuses on 2D data （human 2D skeleton， gait contour map， etc.） but has gradually shifted to 3D data （human 3D skeleton， human 3D mesh， depth image， etc.） in recent years. Future researchers can explore multidimensional gait recognition networks based on multimodal fusion to dynamically fuse 2D and 3D gait data. This fusion network combines the advantages of high 2D recognition efficiency and high 3D recognition accuracy and is expected to improve the performance of gait recognition in complex outdoor scenes. 4） Promotion and application of 3D vision technology. This paper mainly discusses the application of 3D vision technology in the emerging field of biometrics， particularly in gait recognition. Traditional biometrics， such as face recognition and fingerprint recognition， are also gradually transitioning from 2D to 3D. It is anticipated that this paper will aid researchers in understanding the latest advancements in 3D gait recognition and inspire the development of novel and advanced algorithms and models.
计算机视觉生物特征识别步态识别三维人体身份识别三维建模
computer visionbiometric recognitiongait recognitionthree-dimensional human bodyidentification recognitionthree-dimensional modeling
An W Z， Yu S Q， Makihara Y， Wu X H， Xu C， Yu Y， Liao R J and Yagi Y. 2020. Performance evaluation of model-based gait on multi-view very large population database with pose sequences. IEEE Transactions on Biometrics， Behavior， and Identity Science， 2（4）： 421-430 ［DOI： 10.1109/TBIOM.2020.3008862http://dx.doi.org/10.1109/TBIOM.2020.3008862］
Ben X Y， Xu S and Wang K J. 2012. Review on pedestrian gait feature expression and recognition. Pattern Recognition and Artificial Intelligence， 25（1）： 71-81
贲晛烨， 徐森， 王科俊. 2012. 行人步态的特征表达及识别综述. 模式识别与人工智能， 25（1）： 71-81 ［DOI： 10.16451/j.cnki.issn1003-6059.2012.01.010http://dx.doi.org/10.16451/j.cnki.issn1003-6059.2012.01.010］
Caesar H， Bankiti V， Lang A H， Vora S， Liong V E， Xu Q， Krishnan A， Pan Y， Baldan G and Beijbom O. 2020. nuScenes： a multimodal dataset for autonomous driving//Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle， USA： IEEE： 11618-11628 ［DOI： 10.1109/CVPR42600.2020.01164http://dx.doi.org/10.1109/CVPR42600.2020.01164］
Chao H Q， Wang K， He Y W， Zhang J P and Feng J F. 2022. GaitSet： cross-view gait recognition through utilizing gait as a deep set. IEEE Transactions on Pattern Analysis and Machine Intelligence， 44（7）： 3467-3478 ［DOI： 10.1109/TPAMI.2021.3057879http://dx.doi.org/10.1109/TPAMI.2021.3057879］
Chen C H and Ramanan D. 2017. 3D human pose estimation = 2D pose estimation + matching//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition （CVPR）. Honolulu， USA： IEEE： 7035-7043 ［DOI： 10.1109/CVPR.2017.610http://dx.doi.org/10.1109/CVPR.2017.610］
Deng M Q， Wang C and Chen Q F. 2016. Human gait recognition based on deterministic learning through multiple views fusion. Pattern Recognition Letters， 78： 56-63 ［DOI： 10.1016/j.patrec.2016.04.004http://dx.doi.org/10.1016/j.patrec.2016.04.004］
Deng M Q， Wang C， Cheng F J and Zeng W. 2017. Fusion of spatial-temporal and kinematic features for gait recognition with deterministic learning. Pattern Recognition， 67： 186-200 ［DOI： 10.1016/j.patcog.2017.02.014http://dx.doi.org/10.1016/j.patcog.2017.02.014］
dos Santos C F G， de Souza Oliveira D， Passos L A， Pires R G， Santos D F S， Valem L P， Moreira T P， Santana M C S， Roder M， Papa J P and Colombo D. 2023. Gait recognition based on deep learning： a survey. ACM Computing Surveys， 55（2）： #34 ［DOI： 10.1145/3490235http://dx.doi.org/10.1145/3490235］
Fan C， Liang J H， Shen C F， Hou S H， Huang Y Z and Yu S Q. 2023. OpenGait： revisiting gait recognition toward better practicality//Proceedings of 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. Vancouver， Canada： IEEE： 9707-9716 ［DOI： 10.1109/CVPR52729.2023.00936http://dx.doi.org/10.1109/CVPR52729.2023.00936］
Fan C， Peng Y J， Cao C S， Liu X， Hou S H， Chi J N， Huang Y Z， Li Q and He Z Q. 2020. GaitPart： temporal part-based model for gait recognition//Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle， USA： IEEE： 14213-14221 ［DOI： 10.1109/CVPR42600.2020.01423http://dx.doi.org/10.1109/CVPR42600.2020.01423］
Han J and Bhanu B. 2006. Individual recognition using gait energy image. IEEE Transactions on Pattern Analysis and Machine Intelligence， 28（2）： 316-322 ［DOI： 10.1109/TPAMI.2006.38http://dx.doi.org/10.1109/TPAMI.2006.38］
He Y W and Zhang J P. 2018. Deep learning for gait recognition： a survey. Pattern Recognition and Artificial Intelligence， 31（5）： 442-452
何逸炜， 张军平. 2018. 步态识别的深度学习： 综述. 模式识别与人工智能， 31（5）： 442-452 ［DOI： 10.16451/j.cnki.issn1003-6059.201805006http://dx.doi.org/10.16451/j.cnki.issn1003-6059.201805006］
Hofmann M， Bachmann S and Rigoll G. 2012. 2.5D gait biometrics using the depth gradient histogram energy image//Proceedings of the 5th IEEE International Conference on Biometrics： Theory， Applications and Systems （BTAS）. Arlington， USA： IEEE： 399-403 ［DOI： 10.1109/BTAS.2012.6374606http://dx.doi.org/10.1109/BTAS.2012.6374606］
Hofmann M， Geiger J， Bachmann S， Schuller B and Rigoll G. 2014. The TUM gait from audio， image and depth （GAID） database： multimodal recognition of subjects and traits. Journal of Visual Communication and Image Representation， 25（1）： 195-206 ［DOI： 10.1016/j.jvcir.2013.02.006http://dx.doi.org/10.1016/j.jvcir.2013.02.006］
Hou S H， Cao C S， Liu X and Huang Y Z. 2020. Gait lateral network： learning discriminative and compact representations for gait recognition//Proceedings of the 16th European Conference on Computer Vision. Glasgow， UK： Springer： 382-398 ［DOI： 10.1007/978-3-030-58545-7_22http://dx.doi.org/10.1007/978-3-030-58545-7_22］
Huang X H， Zhu D W， Wang H， Wang X G， Yang B， He B T， Liu W Y and Feng B. 2021. Context-sensitive temporal feature learning for gait recognition//Proceedings of 2021 IEEE/CVF International Conference on Computer Vision. Montreal， Canada： IEEE： 12889-12898 ［DOI： 10.1109/ICCV48922.2021.01267http://dx.doi.org/10.1109/ICCV48922.2021.01267］
Kim W， Kim Y and Lee K Y. 2020. Human gait recognition based on integrated gait features using Kinect depth cameras//Proceedings of the 44th IEEE Annual Computers， Software， and Applications Conference （COMPSAC）. Madrid， Spain： IEEE： 328-333 ［DOI： 10.1109/COMPSAC48688.2020.0-225http://dx.doi.org/10.1109/COMPSAC48688.2020.0-225］
Li X， Makihara Y， Xu C and Yagi Y. 2022. Multi-view large population gait database with human meshes and its performance evaluation. IEEE Transactions on Biometrics， Behavior， and Identity Science， 4（2）： 234-248 ［DOI： 10.1109/TBIOM.2022.3174559http://dx.doi.org/10.1109/TBIOM.2022.3174559］
Liao R J， Yu S Q， An W Z and Huang Y Z. 2020. A model-based gait recognition method with body pose and human prior knowledge. Pattern Recognition， 98： #107069 ［DOI： 10.1016/j.patcog.2019.107069http://dx.doi.org/10.1016/j.patcog.2019.107069］
Lin B B， Zhang S L and Yu X. 2021. Gait recognition via effective global-local feature representation and local temporal aggregation//Proceedings of 2021 IEEE/CVF International Conference on Computer Vision. Montreal， Canada： IEEE： 14628-14636 ［DOI： 10.1109/ICCV48922.2021.01438http://dx.doi.org/10.1109/ICCV48922.2021.01438］
Loper M， Mahmood N， Romero J， Pons-Moll G and Black M J. 2015. SMPL： a skinned multi-person linear model. ACM Transactions on Graphics， 34（6）： #3248 ［DOI： 10.1145/2816795.2818013http://dx.doi.org/10.1145/2816795.2818013］
López-Fernndez D， Madrid-Cuevas F J， Carmona-Poyato A， Muñoz-Salinas R and Medina-Carnicer R. 2016. A new approach for multi-view gait recognition on unconstrained paths. Journal of Visual Communication and Image Representation， 38： 396-406 ［DOI： 10.1016/j.jvcir.2016.03.020http://dx.doi.org/10.1016/j.jvcir.2016.03.020］
Meng C Y， He X B， Tan Z and Luan L. 2023. Gait recognition based on 3D human body reconstruction and multi-granular feature fusion. The Journal of Supercomputing， 79（11）： 12106-12125 ［DOI： 10.1007/s11227-023-05143-0http://dx.doi.org/10.1007/s11227-023-05143-0］
Nunes J F， Moreira P M and Tavares J M R S. 2019. Benchmark RGB-D gait datasets： a systematic review//Proceedings of 2019 VII ECCOMAS Thematic Conference on Computational Vision and Medical Image Processing. Porto， Portugal： Springer： 366-372 ［DOI： 10.1007/978-3-030-32040-9_38http://dx.doi.org/10.1007/978-3-030-32040-9_38］
Owaidah E M， Aloufi K S and Alkhatib J H. 2019. Gait recognition for Saudi Costume using Kinect skeletal tracking//Proceedings of the 2nd International Conference on Computer Applications and Information Security （ICCAIS）. Riyadh， Saudi Arabia： IEEE： 1-5 ［DOI： 10.1109/CAIS.2019.8769552http://dx.doi.org/10.1109/CAIS.2019.8769552］
Ryu J and Kamata S I. 2011. Front view gait recognition using spherical space model with human point clouds//Proceedings of the 18th IEEE International Conference on Image Processing. Brussels， Belgium： IEEE： 3209-3212 ［DOI： 10.1109/ICIP.2011.6116351http://dx.doi.org/10.1109/ICIP.2011.6116351］
Sepas-Moghaddam A and Etemad A. 2023. Deep gait recognition： a survey. IEEE Transactions on Pattern Analysis and Machine Intelligence， 45（1）： 264-284 ［DOI： 10.1109/TPAMI.2022.3151865http://dx.doi.org/10.1109/TPAMI.2022.3151865］
Shen C F， Fan C， Wu W， Wang R， Huang G Q and Yu S Q. 2023a. LidarGait： benchmarking 3D gait recognition with point clouds//Proceedings of 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. Vancouver， Canada： IEEE： 1054-1063 ［DOI： 10.1109/CVPR52729.2023.00108http://dx.doi.org/10.1109/CVPR52729.2023.00108］
Shen S， Sun S S， Li W J， Wang R C， Sun P， Wang S and Geng X Y. 2023b. A classifier based on multiple feature extraction blocks for gait authentication using smartphone sensors. Computers and Electrical Engineering， 108： #108663 ［DOI： 10.1016/j.compeleceng.2023.108663http://dx.doi.org/10.1016/j.compeleceng.2023.108663］
Shiraga K， Makihara Y， Muramatsu D， Echigo T and Yagi Y. 2016. Geinet： view-invariant gait recognition using a convolutional neural network//Proceedings of 2016 International Conference on Biometrics （ICB）. Halmstad， Sweden： IEEE： 1-8 ［DOI： 10.1109/ICB.2016.7550060http://dx.doi.org/10.1109/ICB.2016.7550060］
Sivapalan S， Chen D， Denman S， Sridharan S and Fookes C. 2011. Gait energy volumes and frontal gait recognition using depth images//Proceedings of 2011 International Joint Conference on Biometrics （IJCB）. Washington， USA： IEEE： 1-6 ［DOI： 10.1109/IJCB.2011.6117504http://dx.doi.org/10.1109/IJCB.2011.6117504］
Song C F， Huang Y Z， Wang W N and Wang L. 2023. CASIA-E： a large comprehensive dataset for gait recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence， 45（3）： 2801-2815 ［DOI： 10.1109/TPAMI.2022.3183288http://dx.doi.org/10.1109/TPAMI.2022.3183288］
Sun Z N， He R， Wang L， Kan M N， Feng J J， Zheng F， Zheng W S， Zuo W M， Kang W X， Deng W H， Zhang J， Han H， Shan S G， Wang Y L， Ru Y W， Zhu Y H， Liu Y F and He Y. 2021. Overview of biometrics research. Journal of Image and Graphics， 26（6）： 1254-1329
孙哲南， 赫然， 王亮， 阚美娜， 冯建江， 郑方， 郑伟诗， 左旺孟， 康文雄， 邓伟洪， 张杰， 韩琥， 山世光， 王云龙， 茹一伟， 朱宇豪， 刘云帆， 何勇. 2021. 生物特征识别学科发展报告. 中国图象图形学报， 26（6）： 1254-1329 ［DOI： 10.11834/jig.210078http://dx.doi.org/10.11834/jig.210078］
Takemura N， Makihara Y， Muramatsu D， Echigo T and Yagi Y. 2018. Multi-view large population gait dataset and its performance evaluation for cross-view gait recognition. IPSJ Transactions on Computer Vision and Applications， 10（1）： #4 ［DOI： 10.1186/s41074-018-0039-6http://dx.doi.org/10.1186/s41074-018-0039-6］
Teepe T， Khan A， Gilg J， Herzog F， Hörmann S and Rigoll G. 2021. Gaitgraph： graph convolutional network for skeleton-based gait recognition//Proceedings of 2021 IEEE International Conference on Image Processing （ICIP）. Anchorage， USA： IEEE： 2314-2318 ［DOI： 10.1109/ICIP42928.2021.9506717http://dx.doi.org/10.1109/ICIP42928.2021.9506717］
Uy M A， Pham Q H， Hua B S， Nguyen T and Yeung S K. 2019. Revisiting point cloud classification： a new benchmark dataset and classification model on real-world data//Proceedings of 2019 IEEE/CVF International Conference on Computer Vision. Seoul， Korea （South）： IEEE： 1588-1597 ［DOI： 10.1109/ICCV.2019.00167http://dx.doi.org/10.1109/ICCV.2019.00167］
Wang K J， Ding X N， Xing X L and Liu M C. 2019. A survey of multi-view gait recognition. Acta Automatica Sinica， 45（5）： 841-852
王科俊， 丁欣楠， 邢向磊， 刘美辰. 2019. 多视角步态识别综述. 自动化学报， 45（5）： 841-852 ［DOI： 10.16383/j.aas.2018.c170559http://dx.doi.org/10.16383/j.aas.2018.c170559］
Wang K J and Hou B B. 2007. A survey of gait recognition. Journal of Image and Graphics， 12（7）： 1152-1160
王科俊， 侯本博. 2007. 步态识别综述. 中国图象图形学报， 12（7）： 1152-1160 ［DOI： 10.3969/j.issn.1006-8961.2007.07.002http://dx.doi.org/10.3969/j.issn.1006-8961.2007.07.002］
Wang Y F， Sun J D， Li J and Zhao D. 2016. Gait recognition based on 3D skeleton joints captured by Kinect//Proceedings of 2016 IEEE International Conference on Image Processing （ICIP）. Phoenix， USA： IEEE： 3151-3155 ［DOI： 10.1109/ICIP.2016.7532940http://dx.doi.org/10.1109/ICIP.2016.7532940］
Wang Z Y， Sun H Y and Sun X P. 2023. Survey on large scale 3D point cloud processing using deep learning. Computer Systems and Applications， 32（2）： 1-12
王振燕， 孙红岩， 孙晓鹏. 2023. 基于深度学习的大规模三维点云处理综述. 计算机系统应用， 32（2）： 1-12 ［DOI： 10.15888/j.cnki.csa.008743http://dx.doi.org/10.15888/j.cnki.csa.008743］
Wu Z F， Huang Y Z， Wang L， Wang X G and Tan T N. 2017. A comprehensive study on cross-view gait based human identification with deep CNNs. IEEE Transactions on Pattern Analysis and Machine Intelligence， 39（2）： 209-226 ［DOI： 10.1109/TPAMI.2016.2545669http://dx.doi.org/10.1109/TPAMI.2016.2545669］
Xu W Z， Huang T H， Ben X Y， Zeng Y and Zhang J P. 2023. Cross-view gait recognition： a review. Journal of Image and Graphics， 28（5）： 1265-1286
许文正， 黄天欢， 贲晛烨， 曾翌， 张军平. 2023. 跨视角步态识别综述. 中国图象图形学报， 28（5）： 1265-1286 ［DOI： 10.11834/jig.220458http://dx.doi.org/10.11834/jig.220458］
Yu S Q， Tan D L and Tan T N. 2006. A framework for evaluating the effect of view angle， clothing and carrying condition on gait recognition//Proceedings of the 18th International Conference on Pattern Recognition. Hong Kong， China： IEEE： 441-444 ［DOI： 10.1109/ICPR.2006.67http://dx.doi.org/10.1109/ICPR.2006.67］
Yu S Q， Wang Q and Huang Y Z. 2013. A large RGB-D gait dataset and the baseline algorithm//Proceedings of the 8th Chinese Conference on Biometric Recognition. Jinan， China： Springer： 417-424 ［DOI： 10.1007/978-3-319-02961-0_52http://dx.doi.org/10.1007/978-3-319-02961-0_52］
Zeng Y， Wu L F and Xie D L. 2021. Gait analysis based on azure Kinect 3D human skeleton//Proceedings of 2021 International Conference on Computer Information Science and Artificial Intelligence （CISAI）. Kunming， China： IEEE： 1059-1062 ［DOI： 10.1109/CISAI54367.2021.00212http://dx.doi.org/10.1109/CISAI54367.2021.00212］
Zhang Z Y. 2012. Microsoft Kinect sensor and its effect. IEEE Multimedia， 19（2）： 4-10 ［DOI： 10.1109/MMUL.2012.24http://dx.doi.org/10.1109/MMUL.2012.24］
Zhao G Y， Liu G Y， Li H and Pietikainen M. 2006. 3D gait recognition using multiple cameras//Proceedings of the 7th International Conference on Automatic Face and Gesture Recognition （FGR06）. Southampton， UK： IEEE： 529-534 ［DOI： 10.1109/FGR.2006.2http://dx.doi.org/10.1109/FGR.2006.2］
Zheng J K， Liu X C， Liu W， He L X， Yan C G and Mei T. 2022. Gait recognition in the wild with dense 3D representations and a benchmark//Proceedings of 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New Orleans， USA： IEEE， 20196-20205 ［DOI： 10.1109/CVPR52688.2022.01959http://dx.doi.org/10.1109/CVPR52688.2022.01959］
Zhu Z， Guo X D， Yang T， Huang J J， Deng J K， Huang G， Du D L， Lu J W and Zhou J. 2021. Gait recognition in the wild： a benchmark//Proceedings of 2021 IEEE/CVF International Conference on Computer Vision. Montreal， Canada： IEEE， 14769-14779 ［DOI： 10.1109/ICCV48922.2021.01452http://dx.doi.org/10.1109/ICCV48922.2021.01452］
相关文章
相关作者
相关机构
京公网安备11010802024621