浏览全部资源
扫码关注微信
Published： 16 May 2024 ，
移动端阅览
汪琦， 雪心远， 闵卫东， 汪晟， 盖迪， 韩清. 2024. 跨域联合学习与共享子空间度量的车辆重识别. 中国图象图形学报， 29(05):1364-1380
Wang Qi， Xue Xinyuan， Min Weidong， Wang Sheng， Gai Di， Han Qing. 2024. Cross-domain joint learning and shared subspace metric for vehicle re-identification. Journal of Image and Graphics， 29(05):1364-1380
目的
2
现有的跨域重识别任务普遍存在源域与目标域之间的域偏差大和聚类质量差的问题，同时跨域模型过度关注在目标域上的泛化能力将导致对源域知识的永久性遗忘。为了克服以上挑战，提出了一个基于跨域联合学习与共享子空间度量的车辆重识别方法。
方法
2
在跨域联合学习中设计了一种交叉置信软聚类来建立源域与目标域之间的域间相关性，并利用软聚类结果产生的监督信息来保留旧知识与泛化新知识。提出了一种显著性感知注意力机制来获取车辆的显著性特征，将原始特征与显著性特征映射到一个共享子空间中并通过它们各自全局与局部之间的杰卡德距离来获取共享度量因子，根据共享度量因子来平滑全局与局部的伪标签，进而促使模型能够学习到更具鉴别力的特征。
结果
2
在3个公共车辆重识别数据集VeRi-776（vehicle re-identification-776 dataset）、VehicleID（large-scale vehicle re-identification dataset）和VeRi-Wild（vehicle re-identification dataset in the wild）上与较新方法进行实验对比，以首位命中率（rank-1 accuracy，Rank-1）和平均精度均值（mean average precision，mAP）作为性能评价指标，本文方法在VeRi-776→VeRi-Wild，VeRi-Wild→VeRi-776，VeRi-776→VehicleID，VehicleID→VeRi-776的跨域任务中，分别在目标域中取得了42.40%，41.70%，56.40%，61.90%的Rank-1准确率以及22.50%，23.10%，41.50%，49.10%的mAP准确率。在积累源域的旧知识表现中分别取得了84.60%，84.00%，77.10%，67.00%的Rank-1准确率以及55.80%，44.80%，46.50%，30.70%的mAP准确率。
结论
2
相较于无监督域自适应和无监督混合域方法，本文方法能够在积累跨域知识的同时有效缓解域偏差大的问题，进而提升车辆重识别的性能。
Objective
2
Vehicle re-identification （Re-ID） is a technology that uses computer vision technology to determine whether a specific target vehicle exists in an image or video sequence， which is considered a subproblem of image retrieval. Vehicle Re-ID technology can be used to monitor specific abandoned vehicles and prevent driving escape and is widely applied in the fields of intelligent surveillance and transportation. The previous methods mainly focused on supervised training in a single domain. If the effective Re-ID model in the single domain is transferred to an unlabeled new domain for testing， retrieval accuracy will significantly decrease. Some researchers have gradually proposed many cross-domain-based Re-ID methods to alleviate the manual annotation cost of massive surveillance data. This study aims to transfer the trained supervised Re-ID model from the labeled source domain to the unlabeled target domain for clustering. The entire transfer process uses unsupervised iteration and update of model parameters， thereby achieving the goal of reducing manual annotation costs. However， the existing cross-domain Re-ID tasks generally have two main challenges： on the one hand， the existing cross-domain Re-ID methods focus too much on the performance of the target domain， often neglecting the old knowledge previously learned in the source domain， which will cause catastrophic forgetting of the old knowledge. On the other hand， the large deviation between the source and target domains will directly affect the generalization ability of the Re-ID model mainly because of the significant differences in data distribution and domain attributes in different domains. Hence， a vehicle Re-ID method based on cross-domain joint learning and a shared subspace metric is proposed to overcome the above challenges.
Method
2
First， a cross-confidence soft cluster is designed in cross-domain joint learning to establish the inter-domain correlation between the source and target domains. The cross-confidence soft cluster aims to introduce prior knowledge of the source domain data into the target domain by calculating the confidence level of the cross mean. The cluster also aims to jointly perform soft clustering， thereby effectively integrating prior knowledge of the source domain with new knowledge of the target domain. The training data are re-labeled with pseudo labels based on the cross-mean confidence of each type of source domain data. Moreover， the supervised information generated by the soft clustering results is ultimately retained to preserve old knowledge and generalize new knowledge. Then， a salient-aware attention mechanism is proposed to obtain the salient features of vehicles. The salient-aware attention mechanism module is embedded into the reference network to improve the Re-ID model’s ability to identify significant regions of vehicles in the channel and spatial dimensions. Then， the expression of vehicle significant region features is improved by calculating the channel and spatial weight factors. For the channel weight factor， a convolution operation with a convolution kernel of 1 is used to compress the channel dimensions of the feature matrix， and the importance of each channel in the feature matrix is calculated in a self-aware manner. In addition， global average pooling is applied to the feature matrix to prevent the loss of some channel spatial information when compressing channel dimensions. Moreover， further refined channel style attention is jointly inferred by considering channel self-attention and channel-by-channel spatial information. The original and salient features are mapped into a shared subspace， and the shared metric factors are obtained through the Jaccard distance of their respective global and local regions. Finally， a shared metric factor is used to smooth global and local pseudo-labels based on the results of cross-confidence soft clustering to further alleviate the label noise caused by domain bias. This approach enables the training model to learn further discriminating features. The proposed method in this study is trained in the Python 3.7 and Python 1.6.0 frameworks， with an operating system of Ubuntu 18.04 and CUDA 11.2. The hardware configuration is an Intel （R） Xeon （R） Silver 4210 CPU @ 2.20 GHz model CPU， a Tesla V100 graphics card with 32 GB of graphics memory， and a running memory of 64 GB. The whole training uses ResNet-50 as the baseline model， and the size of the input image is uniformly cropped to 224 × 124 pixels. The total number of training iteration epochs is 50， and the batch size is set to 64. The pre-training model on ImageNet is used as the initialization model in this study， and the initial learning rate is set to 0.000 35. Moreover， stochastic gradient descent （SGD） is used to iterate and optimize the model weight.
Result
2
Experimental comparisons are conducted on three public vehicle Re-ID datasets， the vehicle Re-ID-776 dataset （VeRi-776）， the large-scale vehicle Re-ID dataset （VehicleID）， and the vehicle Re-ID dataset in the wild （VeRi-Wild）， with the latest existing methods. This study uses rank-1 accuracy （Rank-1） and mean average precision （mAP） as evaluation indicators. The proposed method achieved a Rank-1 accuracy of 42.40%， 41.70%， 56.40%， and 61.90% in the target domain in the cross-domain tasks of VeRi-776→VeRi-Wild， VeRi-Wild→VeRi-776， VeRi-776→VehicleID， and VehicleID→VeRi-776， respectively. The accuracy of mAP is 22.50%， 23.10%， 41.50%， and 49.10%， respectively. The method also achieved a Rank-1 accuracy of 84.60%， 84.00%， 77.10%， and 67.00%， respectively， in accumulating old knowledge representation in the source domain. The mAP accuracy is 55.80%， 44.80%， 46.50%， and 30.70%， respectively. In addition， a series of experiments is conducted to further demonstrate the robustness of the proposed method in cross-domain tasks， including ablation comparison of different modules， comparison of different training methods， comparison of outliers and visualization of attention maps， comparison of rank lists， and comparison of t-distributed stochastic neighbor embedding （t-SNE） visualization.
Conclusion
2
In this study， compared with unsupervised domain adaptive and unsupervised hybrid domain methods， the proposed method can effectively alleviate the problem of large domain deviation while accumulating cross-domain knowledge， thereby improving the performance of vehicle Re-ID tasks.
车辆重识别跨域联合学习（CJL）交叉置信软聚类共享子空间度量（SSM）显著性感知注意力机制伪标签平滑
vehicle re-identificationcross-domain joint learning（CJL）cross-confidence soft clusteringshared subspace metric（SSM）salient-aware attention mechanismpseudo label smoothing
Cho Y， Kim W J， Hong S and Yoon S E. 2022. Part-based pseudo label refinement for unsupervised person re-identification//Proceedings of 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New Orleans， USA： IEEE： 7298-7308 ［DOI： 10.1109/cvpr52688.2022.00716http://dx.doi.org/10.1109/cvpr52688.2022.00716］
Deng W J， Zheng L， Ye Q X， Kang G L， Yang Y and Jiao J B. 2018. Image-image domain adaptation with preserved self-similarity and domain-dissimilarity for person re-identification//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City， USA： IEEE： 994-1003 ［DOI： 10.1109/cvpr.2018.00110http://dx.doi.org/10.1109/cvpr.2018.00110］
Ge Y X， Chen D P and Li H S. 2020a. Mutual mean-teaching： pseudo label refinery for unsupervised domain adaptation on person re-identification ［EB/OL］. ［2023-06-19］. https://arxiv.org/pdf/2001.01526.pdfhttps://arxiv.org/pdf/2001.01526.pdf
Ge Y X， Zhu F， Chen D P， Zhao R and Li H S. 2020b. Self-paced contrastive learning with hybrid memory for domain adaptive object Re-ID//Proceedings of the 34th International Conference on Neural Information Processing Systems. Vancouver， Canada： Curran Associates Inc.： 11309-11321
Ge W H， Du J L， Wu A C， Xian Y Q， Yan K， Huang F Y and Zheng W S. 2022. Lifelong person re-identification by pseudo task knowledge preservation//Proceedings of the 36th AAAI Conference on Artificial Intelligence. Palo Alto， USA： AAAI： 688-696 ［DOI： 10.1609/aaai.v36i1.19949http://dx.doi.org/10.1609/aaai.v36i1.19949］
Han X M， Yu X H， Li G R， Zhao J， Pan G， Ye Q X， Jiao J B and Han Z J. 2023. Rethinking sampling strategies for unsupervised person re-identification. IEEE Transactions on Image Processing， 32： 29-42 ［DOI： 10.1109/tip.2022.3224325http://dx.doi.org/10.1109/tip.2022.3224325］
He Z J， Zhao H B， Wang J R and Feng W Q. 2023. Multi-level progressive learning for unsupervised vehicle re-identification. IEEE Transactions on Vehicular Technology， 72（4）： 4357-4371 ［DOI： 10.1109/tvt.2022.3228127http://dx.doi.org/10.1109/tvt.2022.3228127］
Hu Z， Zhu C and He G. 2021. Hard-sample guided hybrid contrast learning for unsupervised person re-identification//Proceedings of the 7th IEEE International Conference on Network Intelligence and Digital Content. Beijing， China： IEEE： 91-95 ［DOI： 10.1109/ic-nidc54101.2021.9660560http://dx.doi.org/10.1109/ic-nidc54101.2021.9660560］
Li M K， Li C G and Guo J. 2022. Cluster-guided asymmetric contrastive learning for unsupervised person re-identification. IEEE Transactions on Image Processing， 31： 3606-3617 ［DOI： 10.1109/tip.2022.3173163http://dx.doi.org/10.1109/tip.2022.3173163］
Liu H Y， Tian Y H， Wang Y W， Pang L and Huang T J. 2016a. Deep relative distance learning： tell the difference between similar vehicles//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas， USA： IEEE： 2167-2175 ［DOI： 10.1109/cvpr.2016.238http://dx.doi.org/10.1109/cvpr.2016.238］
Liu X C， Liu W， Mei T and Ma H D. 2016b. A deep learning-based approach to progressive vehicle re-identification for urban surveillance//Proceedings of the 14th European Conference on Computer Vision. Amsterdam， the Netherlands： Springer： 869-884 ［DOI： 10.1007/978-3-319-46475-6_53http://dx.doi.org/10.1007/978-3-319-46475-6_53］
Lou Y H， Bai Y， Liu J， Wang S Q and Duan L Y. 2019. VERI-Wild： a large dataset and a new method for vehicle re-identification in the wild//Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach， USA： IEEE： 3230-3238 ［DOI： 10.1109/cvpr.2019.00335http://dx.doi.org/10.1109/cvpr.2019.00335］
Peng J J， Wang H B， Zhao T T and Fu X P. 2019. Cross domain knowledge transfer for unsupervised vehicle re-identification//Proceedings of 2019 IEEE International Conference on Multimedia and Expo Workshops. Shanghai， China： IEEE： 453-458 ［DOI： 10.1109/icmew.2019.00084http://dx.doi.org/10.1109/icmew.2019.00084］
Pu N， Chen W， Liu Y， Bakker E M and Lew M S. 2021. Lifelong person re-identification via adaptive knowledge accumulation//Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Nashville， USA： IEEE： 7897-7906 ［DOI： 10.1109/cvpr46437.2021.00781http://dx.doi.org/10.1109/cvpr46437.2021.00781］
Pu N， Liu Y， Chen W， Bakker E M and Lew M S. 2022. Meta reconciliation normalization for lifelong person re-identification//Proceedings of the 30th ACM International Conference on Multimedia. Lisboa， Portugal： ACM： 541-549 ［DOI： 10.1145/3503161.3548234http://dx.doi.org/10.1145/3503161.3548234］
Qian W， He Z Q， Chen C and Peng S L. 2022. Navigating diverse salient features for vehicle re-identification. IEEE Transactions on Intelligent Transportation Systems， 23（12）： 24578-24587 ［DOI： 10.1109/TITS.2022.3190959http://dx.doi.org/10.1109/TITS.2022.3190959］
Rami H， Ospici M and Lathuilière S. 2022. Online unsupervised domain adaptation for person re-identification//Proceedings of 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. New Orleans， USA： IEEE： 3829-3838 ［DOI： 10.1109/cvprw56347.2022.00428http://dx.doi.org/10.1109/cvprw56347.2022.00428］
Song L C， Wang C， Zhang L F， Du B， Zhang Q， Huang C and Wang X G. 2020. Unsupervised domain adaptive re-identification： theory and practice. Pattern Recognition， 102： #107173 ［DOI： 10.1016/j.patcog.2019.107173http://dx.doi.org/10.1016/j.patcog.2019.107173］
Sun Z C and Mu Y D. 2022. Patch-based knowledge distillation for lifelong person re-identification//Proceedings of the 30th ACM International Conference on Multimedia. Lisboa， Portugal： ACM： 696-707 ［DOI： 10.1145/3503161.3548179http://dx.doi.org/10.1145/3503161.3548179］
Tang L S， Wang Y and Chau L P. 2022. Weakly-supervised part-attention and mentored networks for vehicle re-identification. IEEE Transactions on Circuits and Systems for Video Technology， 32（12）： 8887-8898 ［DOI： 10.1109/TCSVT.2022.3197844http://dx.doi.org/10.1109/TCSVT.2022.3197844］
Wang H B， Peng J J， Jiang G Q and Fu X P. 2021. Learning multiple semantic knowledge for cross-domain unsupervised vehicle re-identification//Proceedings of 2021 IEEE International Conference on Multimedia and Expo. Shenzhen， China： IEEE： 1-6 ［DOI： 10.1109/icme51207.2021.9428440http://dx.doi.org/10.1109/icme51207.2021.9428440］
Wang Q， Zhong Y L， Min W D， Zhao H Y， Gai D and Han Q. 2023. Dual similarity pre-training and domain difference encouragement learning for vehicle re-identification in the wild. Pattern Recognition， 139： #109513 ［DOI： 10.1016/j.patcog.2023.109513http://dx.doi.org/10.1016/j.patcog.2023.109513］
Wang Y， Peng J J， Wang H B and Wang M. 2022. Progressive learning with multi-scale attention network for cross-domain vehicle re-identification. Science China Information Sciences， 65（6）： #160103 ［DOI： 10.1007/s11432-021-3383-yhttp://dx.doi.org/10.1007/s11432-021-3383-y］
Wang Z X， Xu Z M， Xue Y Y， Lang C Y， Li Z and Wei L L. 2023. Global and spatial multi-scale contexts fusion for vehicle re-identification. Journal of Image and Graphics， 28（2）： 471-482
王振学， 许喆铭， 雪洋洋， 郎丛妍， 李尊， 魏莉莉. 2023. 融合全局与空间多尺度上下文信息的车辆重识别. 中国图象图形学报， 28（2）： 471-482 ［DOI： 10.11834/jig.210849http://dx.doi.org/10.11834/jig.210849］
Wei R， Gu J Y， He S T and Jiang W. 2023. Transformer-based domain-specific representation for unsupervised domain adaptive vehicle re-identification. IEEE Transactions on Intelligent Transportation Systems， 24（3）： 2935-2946 ［DOI： 10.1109/tits.2022.3225025http://dx.doi.org/10.1109/tits.2022.3225025］
Wu G L and Gong S G. 2021. Generalising without forgetting for lifelong person re-identification//Proceedings of the 35th AAAI Conference on Artificial Intelligence. Palo Alto， USA： AAAI： 2889-2897 ［DOI： 10.1609/aaai.v35i4.16395http://dx.doi.org/10.1609/aaai.v35i4.16395］
Wu Y H and Sang N. 2023. Consistency constraints and label optimization-relevant domain-unsupervised adaptive pedestrians’re-identification. Journal of Image and Graphics， 28（5）： 1372-1383
吴禹航， 桑农. 2023. 基于一致性约束和标签优化的无监督域适应行人重识别. 中国图象图形学报， 28（5）： 1372-1383 ［DOI： 10.11834/jig.220618http://dx.doi.org/10.11834/jig.220618］
Yang F X， Zhong Z， Luo Z M， Cai Y Z， Lin Y J， Li S Z and Sebe N. 2021. Joint noise-tolerant learning and meta camera shift adaptation for unsupervised person re-identification//Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Nashville， USA： IEEE： 4853-4862 ［DOI： 10.1109/cvpr46437.2021.00482http://dx.doi.org/10.1109/cvpr46437.2021.00482］
Yu J and Oh H. 2021. Unsupervised person re-identification via multi-label prediction and classification based on graph-structural insight ［EB/OL］. ［2023-06-19］. https://arxiv.org/pdf/2106.08798.pdfhttps://arxiv.org/pdf/2106.08798.pdf
Zhai Y P， Ye Q X， Lu S J， Jia M X， Ji R R and Tian Y H. 2020. Multiple expert brainstorming for domain adaptive person re-identification//Proceedings of the 16th European Conference on Computer Vision. Glasgow， UK： Springer： 594-611 ［DOI： 10.1007/978-3-030-58571-6_35http://dx.doi.org/10.1007/978-3-030-58571-6_35］
Zheng A H， Sun X， Li C L and Tang J. 2022. Viewpoint-aware progressive clustering for unsupervised vehicle re-identification. IEEE Transactions on Intelligent Transportation Systems， 23（8）： 11422-11435 ［DOI： 10.1109/tits.2021.3103961http://dx.doi.org/10.1109/tits.2021.3103961］
Zhong Z， Zheng L， Cao D L and Li S Z. 2017. Re-ranking person re-identification with k-reciprocal encoding//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu， USA： IEEE： 3652-3661 ［DOI： 10.1109/cvpr.2017.389http://dx.doi.org/10.1109/cvpr.2017.389］
Zhong Z， Zheng L， Luo Z M， Li S Z and Yang Y. 2019. Invariance matters： exemplar memory for domain adaptive person re-identification//Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach， USA： IEEE： 598-607 ［DOI： 10.1109/cvpr.2019.00069http://dx.doi.org/10.1109/cvpr.2019.00069］
Zhu J Y， Park T， Isola P and Efros A A. 2017. Unpaired image-to-image translation using cycle-consistent adversarial networks//Proceedings of 2017 IEEE International Conference on Computer Vision. Venice， Italy： IEEE： 2242-2251 ［DOI： 10.1109/iccv.2017.244http://dx.doi.org/10.1109/iccv.2017.244］
相关作者
相关机构
京公网安备11010802024621