浏览全部资源
扫码关注微信
纸质出版日期： 2025-02-16 ，
移动端阅览
沈傲， 沈燕进， 蒋俊锋， 陈正鸣， 黄瑞， 何坤金， 陈杰. 2025. X线片椎骨语义边缘引导的2D/3D配准方法. 中国图象图形学报， 30(02):0601-0614
Shen Ao， Shen Yanjin， Jiang Junfeng， Chen Zhengming， Huang Rui， He Kunjin， Chen Jie. 2025. Semantic edge-guided 2D/3D registration for vertebrae in radiographs. Journal of Image and Graphics， 30(02):0601-0614
目的
2
基于影像引导的脊柱手术机器人系统中，2D/3D配准指的是将术前计算机断层扫描影像与术中X线片配准，用于实现手术机器人对于人体组织的精准空间定位。常见做法是先用标志点进行粗配准，再用灰度法修正位姿。标志点配准问题在于标志点识别精度不高且识别效率较低，灰度法的捕获范围小且对初始位姿敏感。由于脊柱关节边缘重叠且术中X线片图像质量较低，故利用物理边缘作为特征进行2D/3D配准精度不高。因此，提出一种基于语义边缘提取的2D/3D配准方法。
方法
2
首先，提取X线片中成像清晰的椎弓根边缘和椎体两侧边缘作为语义特征进行2D/3D配准；同时，面向边缘提取任务，研究一种间距约束的高效“U”形变形器网络，该深度学习网络提高分割效率的同时，保持了边缘分割的准确性，并加入了椎骨间距约束损失的先验信息，进一步提升了多椎骨语义边缘提取的精度。
结果
2
模拟数据与真实数据上评估结果表明，本文方法在配准精度与效率方面均优于现有方法；位姿修正后，本文方法平移误差小于1 mm，旋转误差小于0.1°，配准耗时在5 s左右，能较好满足实际临床需求。
结论
2
本文提出的基于椎骨语义边缘的2D/3D粗配准方法有效缩小了后续精配准过程的搜索空间，从而提高了配准精度。在边缘提取方面，将哈达玛乘积代替卷积操作的方式以及加入椎骨间距约束损失，提高了语义边缘的提取效率和精度。因此，本文方法能够较好满足2D/3D配准的精度与实时性需求。
Objective
2
In image-guided spine surgery-based robotic systems， 2D/3D registration refers to aligning preoperative 3D computed tomography （CT） images with intraoperative 2D X-rays to achieve precise spatial localization of the surgical robot for human tissues. The prevalent approach involves the use of landmark points for initial coarse registration and subsequently applying the intensity-based method to rectify the position. However， the landmark-based registration method typically uses a heatmap regression method， which can be GPU intensive. Simultaneously， the random field of view during intraoperative X-rays and the overlapping of human tissues on X-rays can result in the loss of tissue information. This loss can cause incorrect predictions or predictions with a considerable deviation of landmark points. The intensity-based method is considered the most accurate and efficient approach because it utilizes the entire image information. However， intensity-based methods usually have problems such as a small capture range and sensitivity to the initial pose. The accuracy of 2D/3D registration when physical edges are used as features is low because of the overlap of spinal joint edges and the low quality of intraoperative radiographs. Here， a 2D/3D registration method based on semantic edge extraction is proposed.
Method
2
The semantic edge-based 2D/3D registration method uses noverlapping pedicle edges and edges on both sides of the vertebra in the X-ray as semantic features for 2D/3D registration. The real-time detection Transformer model is first used to predict the bounding boxes of the vertebrae to be registered in intraoperative X-rays. The CT images of the vertebrae to be registered are then extracted from the known vertebrae masks. The semantic edges of the vertebrae from the intraoperative X-rays and the digitally reconstructed radiological images from the CT image projection are extracted by spacing constrained and efficient UNet Transformers （SCE-UNETR）. Finally， the pose is updated iteratively by minimizing the reprojection error until convergence. The SCE-UNETR network uses a U-shaped structure for encoding and decoding； it comprises an encoder module， skip connection layers， and a decoder. The SCE-UNETR encoder consists of a vision Transformer （ViT） network， where the image features are connected to the decoder by skipping connection layers. The ViT network contains a total of 12 groups of Transformer blocks， each consisting of two normalization layers， a multihead attention layer， and a multilayer perceptron. In this study， the skip connection layer and the decoder of SCE-UNETR are illuminated by a multiaxis group Hadamard product attention module instead of a convolutional operation， which changes the quadratic complexity into linear complexity and reduces the quantity of network parameters. In addition， the network incorporates the loss of the vertebral spacing constraint as a priori information， which further improves the accuracy of multivertebra semantic edge extraction.
Result
2
The method proposed in this study was quantitatively and qualitatively evaluated from different perspectives， including semantic edge extraction， coarse registration， pose refinement， method comparison， registration accuracy， and registration time. For the semantic edge extraction task， the SCE-UNETR was compared with SwinUNETR， TransUNet， and UNETR. SCE-UNETR has approximately half the number of network parameters of the other networks. However， in terms of the metrics of segmentation overlap， average Dice， average intersection over union， and average 95% Hausdorff distance， the SCE-UNETR is better than those of the other three networks. In addition， all networks show some improvement in semantic edge extraction accuracy after the spacing constraint loss term is combined with Dice loss. For the 2D/3D registration task， the experiments demonstrate that the proposed method outperforms the state-of-the-art methods on both simulated and real data. Following pose refinement， the proposed method achieves a translation error of less than 1 mm and a rotation error of less than 0.1°. The registration process itself takes approximately 5 s， which is well suited to the clinical environment.
Conclusion
2
The proposed semantic edge-based 2D/3D registration method effectively reduces the search space of the subsequent pose refinement process， thus improving the registration accuracy. In terms of edge extraction， the network is illuminated， and the vertebrae spacing constraint loss is added， thereby improving the efficiency and accuracy of semantic edge extraction. Therefore， the proposed method can better meet the accuracy and real-time requirements of 2D/3D registration.
Canny J F . 1983 . Finding edges and lines in images ［EB/OL］. ［ 2024-01-02 ］. http://hdl.handle.net/1721.1/6939 http://hdl.handle.net/1721.1/6939
Chen Z Q ， Wang Z W ， Fang L W ， Jian F Z ， Wu Y H ， Li S and He H G . 2018 . Real-time 2D/3D registration of vertebra via machine learning and geometric transformation . Acta Automatica Sinica ， 44 （ 7 ）： 1183 - 1194
陈智强 ， 王作伟 ， 方龙伟 ， 菅凤增 ， 吴毅红 ， 李硕 ， 何晖光 . 2018 . 基于机器学习和几何变换的实时2D/3D脊椎配准 . 自动化学报 ， 44 （ 7 ）： 1183 - 1194 ［ DOI： 10.16383/j.aas.2017.c160711 http://dx.doi.org/10.16383/j.aas.2017.c160711 ］
Dosovitskiy A ， Beyer L ， Kolesnikov A ， Weissenborn D ， Zhai X H ， Unterthiner T ， Dehghani M ， Minderer M ， Heigold G ， Gelly S ， Uszkoreit J and Houlsby N . 2020 . An image is worth 16 × 16 words： transformers for image recognition at scale ［EB/OL］. ［ 2024-01-02 ］. https://arxiv.org/pdf/2010.11929.pdf https://arxiv.org/pdf/2010.11929.pdf
Du X G ， Dang J W ， Wang Y P . 2015 . Generation algorithm of digital reconstruction radiographs based on CUDA . Computer Science ， 42 （ 2 ）： 301 - 305
杜晓刚 ， 党建武 ， 王阳萍 . 2015 . 基于CUDA的数字重建影像生成算法 . 计算机科学 ， 42 （ 2 ）： 301 - 305 ［ DOI： 10.11896/j.issn.1002-137X.2015.2.064 http://dx.doi.org/10.11896/j.issn.1002-137X.2015.2.064 ］
Galibarov P E ， Prendergast P J and Lennon A B . 2010 . A method to reconstruct patient-specific proximal femur surface models from planar pre-operative radiographs . Medical Engineering and Physics ， 32 （ 10 ）： 1180 - 1188 ［ DOI： 10.1016/j.medengphy.2010.08.009 http://dx.doi.org/10.1016/j.medengphy.2010.08.009 ］
Gao C ， Liu X T ， Gu W H ， Killeen B ， Armand M ， Taylor R and Unberath M . 2020 . Generalizing spatial transformers to projective geometry with applications to 2D/3D registration // Proceedings of the 23rd International Conference on Medical Image Computing and Computer Assisted Intervention . Lima， Peru ： Springer： 329 - 339 ［ DOI： 10.1007/978-3-030-59716-0_32 http://dx.doi.org/10.1007/978-3-030-59716-0_32 ］
Gollmer S T ， Lachner R and Buzug T M . 2007 . Registration algorithm for statistical bone shape reconstruction from radiographs-an accuracy study // Proceedings of the 29th Annual International Conference of the IEEE Engineering in Medicine and Biology Society . Lyon， France ： IEEE： 6375 - 6378 ［ DOI： 10.1109/IEMBS.2007.4353814 http://dx.doi.org/10.1109/IEMBS.2007.4353814 ］
Grupp R B ， Hegeman R A ， Murphy R J ， Alexander C P ， Otake Y ， McArthur B A ， Armand M ， and Taylor R H . 2020a . Pose estimation of periacetabular osteotomy fragments with intraoperative X-ray navigation . IEEE Transactions on Biomedical Engineering ， 67 （ 2 ）： 441 - 452 ［ DOI： 10.1109/TBME.2019.2915165 http://dx.doi.org/10.1109/TBME.2019.2915165 ］
Grupp R B ， Unberath M ， Gao C ， Hegeman R A ， Murphy R J ， Alexander C P ， Otake Y ， McArthur B A ， Armand M and Taylor R H . 2020b . Automatic annotation of hip anatomy in fluoroscopy for robust and efficient 2D/3D registration . International Journal of Computer Assisted Radiology and Surgery ， 15 （ 5 ）： 759 - 769 ［ DOI： 10.1007/s11548-020-02162-7 http://dx.doi.org/10.1007/s11548-020-02162-7 ］
Hatamizadeh A ， Nath V ， Tang Y C ， Yang D ， Roth H R and Xu D G . 2022 . Swin UNETR： swin transformers for semantic segmentation of brain tumors in MRI images // International MICCAI Brainlesion Workshop . Springer ： 272 - 284 ［ DOI： 10.1007/978-3-031-08999-2_22 http://dx.doi.org/10.1007/978-3-031-08999-2_22 ］
Hatamizadeh A ， Tang Y C ， Nath V ， Yang D ， Myronenko A ， Landman B ， Roth H R and Xu D G . 2021 . Unetr： Transformers for 3D medical image segmentation ［EB/OL］. ［ 2024-01-02 ］. https://arxiv.org/pdf/2103.10504.pdf https://arxiv.org/pdf/2103.10504.pdf
Howard A G ， Zhu M L ， Chen B ， Kalenichenko D ， Wang W J ， Weyand T ， Andreetto M and Adam H . 2017 . Mobilenets： efficient convolutional neural networks for mobile vision applications ［EB/OL］. ［ 2024-01-02 ］. https://arxiv.org/pdf/1704.04861.pdf https://arxiv.org/pdf/1704.04861.pdf
Huang Y F ， Li S and Zhao J . 2006 . The registration of 2D/3D images based on external markers . Journal of Shanghai Jiaotong University ， 40 （ 7 ）： 1108 - 1111 .
黄永锋 ， 李杉 ， 赵俊 . 2006 . 基于外标记点的2D/3D图像配准方法 . 上海交通大学学报 ， 40 （ 7 ）： 1108 - 1111 ［ DOI： 10.16183/j.cnki.jsjtu.2006.07.009 http://dx.doi.org/10.16183/j.cnki.jsjtu.2006.07.009 ］
Jaganathan S ， Kukla M ， Wang J ， Shetty K and Maier A . 2023 . Self-supervised 2D/3D registration for X-ray to CT image fusion // Proceedings of 2023 IEEE/CVF Winter Conference on Applications of Computer Vision . Waikoloa， USA ： IEEE： 2787 - 2797 ［ DOI： 10.1109/WACV56688.2023.00281 http://dx.doi.org/10.1109/WACV56688.2023.00281 ］
Jaganathan S ， Wang J ， Borsdorf A ， Shetty K and Maier A . 2021 . Deep iterative 2D/3D registration // Proceedings of the 24th International Conference on Medical Image Computing and Computer Assisted Intervention . Strasbourg， France ： Springer： 383 - 392 ［ DOI： 10.1007/978-3-030-87202-1_37 http://dx.doi.org/10.1007/978-3-030-87202-1_37 ］
Jang J S ， Kim J I ， Ku B and Lee J H . 2023 . Reliability analysis of vertebral landmark labelling on lumbar spine X-ray images . Diagnostics ， 13 （ 8 ）： # 1411 ［ DOI： 10.3390/diagnostics13081411 http://dx.doi.org/10.3390/diagnostics13081411 ］
Kadoury S ， Cheriet F and Labelle H . 2009 . Personalized X-ray 3-D reconstruction of the scoliotic spine from hybrid statistical and image-based models . IEEE Transactions on Medical Imaging ， 28 （ 9 ）： 1422 - 1435 ［ DOI： 10.1109/TMI.2009.2016756 http://dx.doi.org/10.1109/TMI.2009.2016756 ］
Liu K ， Lyu X Q ， Gu Y ， Yu H F ， Ren G Y and Zhang M . 2016 . The 2D/3D medical image registration algorithm based on rapid digital image reconstruction . Journal of Image and Graphics ， 21 （ 1 ）： 69 - 77
刘坤 ， 吕晓琪 ， 谷宇 ， 于荷峰 ， 任国印 ， 张明 . 2016 . 快速数字影像重建的2维/3维医学图像配准 . 中国图象图形学报 ， 21 （ 1 ）： 69 - 77 ［ DOI： 10.11834/jig.20160109 http://dx.doi.org/10.11834/jig.20160109 ］
Lu Y H ， Zheng K ， Li W J ， Wang Y R ， Harrison A P ， Lin C H ， Wang S ， Xiao J ， Lu L ， Kuo C F and Miao S . 2020 . Contour transformer network for one-shot segmentation of anatomical structures . IEEE Transactions on Medical Imaging ， 40 （ 10 ）： 2672 - 2684 ［ DOI： 10.1109/TMI.2020.3043375 http://dx.doi.org/10.1109/TMI.2020.3043375 ］
Mai Y F ， Sun Q C ， Jia P F and Chen X J . 2021 . 2D/3D medical image registration algorithm based on DRR and the similarity measure . Beijing Biomedical Engineering ， 40 （ 3 ）： 263 - 272
麦永锋 ， 孙启昌 ， 贾鹏飞 ， 陈晓军 . 2021 . 基于DRR及相似性测度的2D-3D医学图像配准算法 . 北京生物医学工程 ， 40 （ 3 ）： 263 - 272 ［ DOI： 10.3969/j.issn.1002-3208.2021.03.008 http://dx.doi.org/10.3969/j.issn.1002-3208.2021.03.008 ］
Miao S ， Wang Z J and Liao R . 2016 . A CNN regression approach for real-time 2D/3D registration . IEEE Transactions on Medical Imaging ， 35 （ 5 ）： 1352 - 1363 ［ DOI： 10.1109/TMI.2016.2521800 http://dx.doi.org/10.1109/TMI.2016.2521800 ］
Naik R R ， Anitha H ， Bhat S N ， Ampar N and Kundangar R . 2022 . Realistic C-arm to pCT registration for vertebral localization in spine surgery： a hybrid 3D-2D registration framework for intraoperative vertebral pose estimation . Medical and Biological Engineering and Computing ， 60 （ 8 ）： 2271 - 2289 ［ DOI： 10.1007/s11517-022-02600-5 http://dx.doi.org/10.1007/s11517-022-02600-5 ］
Netter F H . 2014 . Atlas of Human Anatomy. 6th ed . Elsevier Health Sciences Press .
Otake Y ， Armand M ， Armiger R S ， Kutzer M D ， Basafa E ， Kazanzides P and Taylor R H . 2011 . Intraoperative image-based multiview 2D/3D registration for image-guided orthopaedic surgery： incorporation of fiducial-based C-arm tracking and GPU-acceleration . IEEE Transactions on Medical Imaging ， 31 （ 4 ）： 948 - 962 ［ DOI： 10.1109/TMI.2011.2176555 http://dx.doi.org/10.1109/TMI.2011.2176555 ］
Ronneberger O ， Fischer P and Brox T . 2015 . U-net： convolutional networks for biomedical image segmentation // Proceedings of the 18th International Conference on Medical Image Computing and Computer-Assisted Intervention . Munich， Germany ： Springer： 234 - 241 ［ DOI： 10.1007/978-3-319-24574-4_28 http://dx.doi.org/10.1007/978-3-319-24574-4_28 ］
Ruan J C ， Xie M Y ， Gao J S ， Liu T and Fu Y Z . 2023 . EGE-UNet： an efficient group enhanced UNet for skin lesion segmentation // Proceedings of the 26th International Conference on Medical Image Computing and Computer Assisted Intervention . Vancouver， Canada ： Springer： 481 - 490 ［ DOI： 10.1007/978-3-031-43901-8_46 http://dx.doi.org/10.1007/978-3-031-43901-8_46 ］
Taha A A and Hanbury A . 2015 . Metrics for evaluating 3D medical image segmentation： analysis， selection， and tool . BMC Medical Imaging ， 15 （ 1 ）： # 29 ［ DOI： 10.1186/s12880-015-0068-x http://dx.doi.org/10.1186/s12880-015-0068-x ］
Unberath M ， Gao C ， Hu Y C ， Judish M ， Taylor R H ， Armand M and Grupp R . 2021 . The impact of machine learning on 2D/3D registration for image-guided interventions： a systematic review and perspective . Frontiers in Robotics and AI ， 8 ： # 716007 ［ DOI： 10.3389/frobt.2021.716007 http://dx.doi.org/10.3389/frobt.2021.716007 ］
Van De Kraats E B ， Penney G P ， Tomazevic D ， Van Walsum T and Niessen W J . 2005 . Standardized evaluation methodology for 2-D-3-D registration . IEEE Transactions on Medical Imaging ， 24 （ 9 ）： 1177 - 1189 ［ DOI： 10.1109/TMI.2005.853240 http://dx.doi.org/10.1109/TMI.2005.853240 ］
Wang Q Z ， Huang W ， Zhang Y ， Li X Y ， Ye X J and Hu K . 2023 . Automatic segmentation of nasopharyngeal carcinoma in CT images using dual attention and edge detection // Proceedings of 2023 IEEE International Conference on Acoustics， Speech and Signal Processing （ICASSP） . Rhodes Island， Greece ： IEEE： 1 - 5 ［ DOI： 10.1109/ICASSP49357.2023.10095233 http://dx.doi.org/10.1109/ICASSP49357.2023.10095233 ］
Wang Y Y ， Dong F ， Shang L N ， Zhang C and Xia S R . 2023 . Research on the IVUS border detection method based on improved transunet . Chinese Journal of Biomedical Engineering ， 42 （ 1 ）： 41 - 50
王媛媛 ， 董芳 ， 尚丽娜 ， 张翠 ， 夏顺仁 . 2023 . 基于改进TransUnet网络的血管内超声图像边界提取方法研究 . 中国生物医学工程学报 ， 42 （ 1 ）： 41 - 50 ［ DOI： 10.3969/j.issn.0258-8021.2023.01.005 http://dx.doi.org/10.3969/j.issn.0258-8021.2023.01.005 ］
Wang Z ， Bovik A C ， Sheikh H R and Simoncelli E P . 2004 . Image quality assessment： from error visibility to structural similarity . IEEE Transactions on Image Processing ， 13 （ 4 ）： 600 - 612 ［ DOI： 10.1109/TIP.2003.819861 http://dx.doi.org/10.1109/TIP.2003.819861 ］
Willmott C J and Matsuura K . 2005 . Advantages of the mean absolute error （MAE） over the root mean square error （RMSE） in assessing average model performance . Climate Research ， 30 （ 1 ）： 79 - 82 ［ DOI： 10.3354/cr030079 http://dx.doi.org/10.3354/cr030079 ］
Xu M M . 2018 . Effect of minimally invasive surgery and traditional open surgery on spinal trauma . Medical Innovation of China ， 15 （ 34 ）： 122 - 125
徐敏铭 . 2018 . 脊柱创伤患者采用传统开放手术和微创手术治疗的对比研究 . 中国医学创新 ， 15 （ 34 ）： 122 - 125 ［ DOI： 10.3969/j.issn.1674-4985.2018.34.033 http://dx.doi.org/10.3969/j.issn.1674-4985.2018.34.033 ］
Yang H ， Fan B B and Guo L L . 2020 . Anchor-free object detection with mask attention . EURASIP Journal on Image and Video Processing ， 2020 ，（ 1 ）： 1 - 17 ［ DOI： 10.1186/s13640-020-00517-3 http://dx.doi.org/10.1186/s13640-020-00517-3 ］
Zeng Z H ， Fan C D ， Xiao L Y and Qu X L . 2022 . DEA-UNet： a dense-edge-attention UNet architecture for medical image segmentation . Journal of Electronic Imaging ， 31 （ 4 ）： # 043032 ［ DOI： 10.1117/1.Jei.31.4.043032 http://dx.doi.org/10.1117/1.Jei.31.4.043032 ］
Zhang B C ， Faghihroohi S ， Azampour M F ， Liu S T ， Ghotbi R ， Schunkert H and Navab N . 2023 . A patient-specific self-supervised model for automatic X-ray/CT registration // Proceedings of the 26th International Conference on Medical Image Computing and Computer Assisted Intervention . Vancouver， Canada ： Springer： 515 - 524 ［ DOI： 10.1007/978-3-031-43996-4_49 http://dx.doi.org/10.1007/978-3-031-43996-4_49 ］
Zhang Z J ， Fu H Z ， Dai H ， Shen J B ， Pang Y W and Shao L . 2019 . ET-net： a generic edge-attention guidance network for medical image segmentation // Proceedings of the 22nd International Conference on Medical Image Computing and Computer Assisted Intervention . Shenzhen， China ： Springer： 442 - 450 ［ DOI： 10.1007/978-3-030-32239-7_49 http://dx.doi.org/10.1007/978-3-030-32239-7_49 ］
Zhao Y A ， Lyu W Y ， Xu S L ， Wei J M ， Wang G Z ， Dang Q Q ， Liu Y and Chen J . 2024 . Detrs beat YOLOs on real-time object detection // Proceedings of 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Seattle， USA ： IEEE： 16965 - 16974 ［ DOI： 10.1109/CVPR52733.2024.01605 http://dx.doi.org/10.1109/CVPR52733.2024.01605 ］
Zhou J K ， Wang P C ， Wang F ， Liu Q ， Li H and Jin R . 2021 . Elsa： Enhanced local self-attention for vision transformer ［EB/OL］. ［ 2024-01-02 ］. https://arxiv.org/pdf/2112.12786.pdf https://arxiv.org/pdf/2112.12786.pdf
相关作者
相关机构
京公网安备11010802024621