浏览全部资源
扫码关注微信
纸质出版日期： 2025-01-16 ，
移动端阅览
刘春晓, 胡鹏靖, 厉世昌, 王成骅, 凌云. 面向非均质图像去雾的解耦合三阶段增强网络[J]. 中国图象图形学报, 2025,30(1):83-94.
LIU CHUNXIAO, HU PENGJING, LI SHICHANG, WANG CHENGHUA, LING YUN. Decoupled triple-stage enhancement network for non-homogeneous image dehazing. [J]. Journal of image and graphics, 2025, 30(1): 83-94.
目的
2
在雾霾环境下拍摄的图像通常具有结构对比度较低、细节信息模糊和颜色饱和度失真等特点。虽然目前的去雾算法已经能较好地处理均质雾霾图像，但是对于非均质雾霾图像的去雾能力仍较差。为此，提出了一种面向非均质雾霾图像去雾的解耦合三阶段增强网络。
方法
2
通过颜色空间变换将输入图像解耦为亮度、饱和度和色度3个通道之后，该算法首先通过对比度增强模块增强亮度图的对比度，使去雾结果具有更清晰的结构和细节信息；然后，通过饱和度增强模块增强图像的饱和度，使去雾结果具有更鲜艳的颜色；最后，使用颜色矫正增强模块对总体颜色进行微调，使去雾结果更符合人眼视觉感知。特别地，在饱和度增强模块中设计了一个雾霾密度编码矩阵，通过计算亮度图在对比度增强前后的梯度差异，估计出雾霾图像的雾霾密度信息，为饱和度增强模块提供指导，以保证饱和度恢复的准确性。
结果
2
在3个数据集上与14种方法进行了对比实验，本文方法在NHD（non-homogeneous dataset）数据集上得到了最优结果，相比于性能第2的模型，平均峰值信噪比提升了8.5 dB，平均结构相似性提升了0.12；在Real-World数据集中，本文方法的感知雾密度预测值为0.47，雾密度估计值为0.21，均处于前列；在SOTS（synthetic object testing set）数据集中，本文方法的平均峰值信噪比为16.52 dB，平均结构相似性为0.80，在人眼感知效果方面不输于已有方法。
结论
2
本文所提方法对于非均质雾霾图像具有优秀的处理能力，可以有效地去除图像的雾霾并还原出雾霾图像的真实细节信息和颜色。
Objective
2
The absorption or scattering effect of microscopic particles in the atmosphere， such as aerosols， soot， and haze， will reduce image contrast， blur image details， and cause color distortion. These problems can decrease the accuracy of subsequent advanced computer vision tasks， such as object detection and image segmentation. Therefore， image dehazing has attracted increasing attention， and various image dehazing methods have been proposed. The ultimate goal of image dehazing is to recover a haze-free image from the input hazy image. At present， existing image dehazing algorithms can be divided into two categories： traditional dehazing algorithms based on image prior and image dehazing algorithms based on deep learning. The image priori-based dehazing algorithm uses the prior information and empirical rules of the image itself to estimate the transmittance map and atmospheric light value， and it utilizes the atmospheric scattering model to realize the image dehazing process. This approach can improve the contrast of the image to a certain extent but easily leads to excessive enhancement or color distortions in the dehazed results. Driven by a large amount of image data， the image dehazing algorithm based on deep learning can flexibly learn the mapping from hazy image to haze-free images by directly constructing an efficient convolutional neural network and obtain dehazed effects with better generalization performance and human visual perception. However， because of domain differences， the image dehazing algorithm trained on the synthesized homogeneous haze dataset usually has difficulty achieving satisfactory results on heterogeneous hazy images in the real world.
Method
2
Haze will reduce the contrast of the image and make it look blurry. Thus， we train the network （i.e.， the contrast enhancement module） with the brightness map of the hazy image and the brightness map corresponding to the clear image as the training image pairs， which effectively enhances the contrast of the brightness map and obtains the brightness enhancement map with a clear image structure and details. Furthermore， we calculate the gradient differences of the brightness maps before and after the contrast enhancement process and estimate the haze density information in the hazy images to guide saturation enhancement of the hazy images. Therefore， we propose an end-to-end decoupled triple-stage enhancement network for the heterogeneous haze dehazing task， which decouples the input hazy image with color space conversion into three channels， i.e.， brightness， saturation， hues. Our algorithm first enhances the contrast of the brightness map through the contrast enhancement module so that the dehazed result holds clear structure and detail information. Then， it enhances the saturation channel of the image through the saturation enhancement module so that the dehazed result takes on a more vivid color. Finally， the color correction and enhancement module is used to fine-tune the overall color of the image so that the final dehazed result will be more in line with human visual perception. In particular， we design a haze density coding matrix in the saturation enhancement module and estimate the haze density information of the hazy image by calculating the gradient differences of the brightness maps before and after the contrast enhancement process. This step will provide guidance for the saturation enhancement module to ensure the accuracy of saturation recovery. The U-Net network structure exhibits superior performance in image enhancement tasks. Thus， we choose U-Net as the backbone network of our contrast and saturation enhancement modules and obtain multi-scale information of images through the encoder and decoder structure for better dehazing results. For the color correction and enhancement module， we only need to fine-tune the previously enhanced image results， which is why we only use a simple network with convolutional layers and skip connections to prevent the loss of image information with upsampling and downsampling operations.
Result
2
Compared with the second best-performing model in performance， the average peak signal-to-noise ratio is increased by 8.5 dB and the average structural similarity is increased by 0.12. Our perceived fog density prediction value is 0.47 and the estimated haze density is 0.21 in the real-world dataset， both of which rank first. In the SOTS dataset， our average peak signal-to-noise ratio is 16.52 dB and the average structural similarity is 0.80， which are comparable to the existing algorithms in terms of human visual perception.
Conclusion
2
Through a series of subjective and objective experimental comparisons， the experimental results show that our algorithm has excellent processing ability for non-homogeneous hazy images and can effectively restore the real details and colors of hazy images.
深度学习非均质图像去雾饱和度增强对比度增强三阶段增强，雾霾密度编码矩阵
deep learningnon-homogeneous image dehazingsaturation enhancementcontrast enhancementtriple-stage enhancementhaze density coding matrix
Bai H R， Pan J S， Xiang X G and Tang J H. 2022. Self-guided image dehazing using progressive feature fusion. IEEE Transactions on Image Processing， 31： 1217-1229 ［DOI： 10.1109/TIP.20223140609http://dx.doi.org/10.1109/TIP.20223140609］
Berman D， Treibitz T and Avidan S. 2016. Non-local image dehazing//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas， USA： IEEE： 1674-1682 ［DOI： 10.1109/CVPR.2016.185http://dx.doi.org/10.1109/CVPR.2016.185］
Chen Z Y， Wang Y C， Yang Y and Liu D. 2021. PSD： principled synthetic-to-real dehazing guided by physical priors//Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Nashville， USA： IEEE： 7176-7185 ［DOI： 10.1109/CVPR46437.2021.00710http://dx.doi.org/10.1109/CVPR46437.2021.00710］
Choi L K， You J and Bovik A C. 2015. Referenceless prediction of perceptual fog density and perceptual image defogging. IEEE Transactions on Image Processing， 24（11）： 3888-3901 ［DOI： 10.1109/TIP.2015.2456502http://dx.doi.org/10.1109/TIP.2015.2456502］
Das S D and Dutta S. 2020. Fast deep multi-patch hierarchical network for nonhomogeneous image dehazing//Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. Seattle， USA： IEEE： 1994-2001 ［DOI： 10.1109/CVPRW50498.2020.00249http://dx.doi.org/10.1109/CVPRW50498.2020.00249］
Fu M H， Liu H， Yu Y K， Chen J and Wang K Y. 2021. DW-GAN： a discrete wavelet transform GAN for nonhomogeneous dehazing//Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. Nashville， USA： IEEE： 203-212 ［DOI： 10.1109/CVPRW53098.2021.00029http://dx.doi.org/10.1109/CVPRW53098.2021.00029］
Guo C L， Yan Q X， Anwar S， Cong R M， Ren W Q and Li C Y. 2022. Image dehazing transformer with transmission-aware 3D position embedding//Proceedings of 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New Orleans， USA： IEEE： 5812-5820 ［DOI： 10.1109/CVPR52688.2022.00572http://dx.doi.org/10.1109/CVPR52688.2022.00572］
He K M， Sun J and Tang X O. 2011. Single image haze removal using dark channel prior. IEEE Transactions on Pattern Analysis and Machine Intelligence， 33（12）： 2341-2353 ［DOI： 10.1109/TPAMI.2010.168http://dx.doi.org/10.1109/TPAMI.2010.168］
Jo E and Sim J Y. 2021. Multi-scale selective residual learning for non-homogeneous dehazing//Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. Nashville， USA： IEEE： 507-515 ［DOI： 10.1109/CVPRW53098.2021.00062http://dx.doi.org/10.1109/CVPRW53098.2021.00062］
Li B Y， Peng X L， Wang Z Y， Xu J Z and Feng D. 2017. An all-in-one network for dehazing and beyond ［EB/OL］. ［2024-01-31］. http://doi.org/10.48550/arXiv.1707.06543.pdfhttp://doi.org/10.48550/arXiv.1707.06543.pdf
Li B Y， Ren W Q， Fu D P， Tao D C， Feng D， Zeng W J and Wang Z Y. 2019. Benchmarking single-image dehazing and beyond. IEEE Transactions on Image Processing， 28（1）： 492-505 ［DOI： 10.1109/TIP.2018.2867951http://dx.doi.org/10.1109/TIP.2018.2867951］
Li Z G， Zheng C B， Shu H Y and Wu S Q. 2022. Dual-scale single image dehazing via neural augmentation. IEEE Transactions on Image Processing， 31： 6213-6223 ［DOI： 10.1109/TIP.2022.3207571http://dx.doi.org/10.1109/TIP.2022.3207571］
Ling P Y， Chen H A， Tan X， Jin Y and Chen E H. 2023. Single image dehazing using saturation line prior. IEEE Transactions on Image Processing， 32： 3238-3253 ［DOI： 10.1109/TIP.2023.3279980http://dx.doi.org/10.1109/TIP.2023.3279980］
Liu C X， Ye S S， Zhang L D， Bao H Y， Wang X and Wu F D. 2022. Non-homogeneous haze data synthesis based real-world image dehazing with enhancement-and-restoration fused CNNs. Computers and Graphics， 106： 45-57 ［DOI： 10.1016/j.cag.2022.05.008http://dx.doi.org/10.1016/j.cag.2022.05.008］
Liu W J， Bai W S， Qu H C and Zhao Q G. 2019. Image dehazing based on GF-MSRCR and dark channel prior. Journal of Image and Graphics， 24（11）： 1893-1905
刘万军， 白宛司， 曲海成， 赵庆国. 2019. 融合GF-MSRCR和暗通道先验的图像去雾. 中国图象图形学报， 24（11）： 1893-1905 ［DOI： 10.11834/jig.190089http://dx.doi.org/10.11834/jig.190089］
Liu X H， Ma Y R， Shi Z H and Chen J. 2019. GridDehazeNet： attention-based multi-scale network for image dehazing//Proceedings of 2019 IEEE/CVF International Conference on Computer Vision. Seoul， Korea （South）： IEEE： 7313-7322 ［DOI： 10.1109/ICCV.2019.00741http://dx.doi.org/10.1109/ICCV.2019.00741］
Liu Y， Zhu L， Pei S D， Fu H Z， Qin J， Zhang Q， Wan L and Feng W. 2021. From synthetic to real： image dehazing collaborating with unlabeled real data//Proceedings of the 29th ACM International Conference on Multimedia. Chengdu， China： ACM： 50-58 ［DOI： 10.1145/3474085.3475331http://dx.doi.org/10.1145/3474085.3475331］
Qin X， Wang Z L， Bai Y C， Xie X D and Jia H Z. 2020. FFA-Net： feature fusion attention network for single image dehazing//Proceedings of the 34th AAAI Conference on Artificial Intelligence. New York， USA： AAAI： 11908-11915 ［DOI： 10.1609/aaai.v34i07.6865http://dx.doi.org/10.1609/aaai.v34i07.6865］
Qu Y Y， Chen Y Z， Huang J Y and Xie Y. 2019. Enhanced pix2pix dehazing network//Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach， USA： IEEE： 8152-8160 ［DOI： 10.1109/CVPR.2019.00835http://dx.doi.org/10.1109/CVPR.2019.00835］
Shen Y Y， Shao Y Q， Liu C X， Zhou H J， Zhao J W. 2017. Integrating sky detection with texture smoothing for image defogging. Journal of Image and Graphics， 22（7）： 897-905
沈逸云， 邵雅琪， 刘春晓， 周华健， 赵锦威. 2017. 结合天空检测与纹理平滑的图像去雾. 中国图象图形学报， 22（7）： 897-905 ［DOI： 10.11834/jig.170030http://dx.doi.org/10.11834/jig.170030］
Song Y D， He Z Q， Qian H and Du X. 2023. Vision transformers for single image dehazing. IEEE Transactions on Image Processing， 32： 1927-1941 ［DOI： 10.1109/TIP.2023.3256763http://dx.doi.org/10.1109/TIP.2023.3256763］
Ullah H， Muhammad K， Irfan M， Anwar S， Sajjad M， Imran A S and De Albuquerque V H C. 2021. Light-DehazeNet： a novel light weight CNN architecture for single image dehazing. IEEE Transactions on Image Processing， 30： 8968-8982 ［DOI： 10.1109/TIP.2021.3116790http://dx.doi.org/10.1109/TIP.2021.3116790］
Wang Z， Bovik A C， Sheikh H R and Simoncelli E P. 2004. Image quality assessment： From error visibility to structural similarity. IEEE Transactions on Image Processing， 13（4）： 600-612 ［DOI： 10.1109/TIP.2003.819861http://dx.doi.org/10.1109/TIP.2003.819861］
Yang Y， Wang C Y， Liu R S， Zhang L， Guo X J and Tao D C. 2022. Self-augmented unpaired image dehazing via density and depth decomposition//Proceedings of 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New Orleans， USA： IEEE： 2027-2036 ［DOI： 10.1109/CVPR52688.2022.00208http://dx.doi.org/10.1109/CVPR52688.2022.00208］
Yu Y K， Liu H， Fu M H， Chen J， Wang X Y and Wang K Y. 2021. A two-branch neural network for non-homogeneous dehazing via ensemble learning//Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. Nashville， USA： IEEE： 193-202 ［DOI： 10.1109/CVPRW53098.2021.00028http://dx.doi.org/10.1109/CVPRW53098.2021.00028］
Yuan F N， Li Z Q， Shi J T， Xia X and Li Y. 2021. Image defogging algorithm using a two-phase feature extraction strategy. Journal of Image and Graphics， 26（3）： 568-580
袁非牛， 李志强， 史劲亭， 夏雪， 李雅. 2021. 两阶段特征提取策略的图像去雾. 中国图象图形学报， 26（3）： 568-580 ［DOI： 10.11834/jig.200057http://dx.doi.org/10.11834/jig.200057］
Zhang J H， Min X K， Zhu Y C， Zhai G T， Zhou J T， Yang X K and Zhang W J. 2022. HazDesNet： an end-to-end network for haze density prediction. IEEE Transactions on Intelligent Transportation Systems， 23（4）： 3087-3102 ［DOI： 10.1109/TITS.2020.3030673http://dx.doi.org/10.1109/TITS.2020.3030673］
Zhao J W， Shen Y Y， Liu C X and Ouyang Y. 2016. Dark channel prior-based image dehazing with atmospheric light validation and halo elimination. Journal of Image and Graphics， 21（9）： 1221-1228
赵锦威， 沈逸云， 刘春晓， 欧阳毅. 2016. 暗通道先验图像去雾的大气光校验和光晕消除. 中国图象图形学报， 21（9）： 1221-1228 ［DOI： 10.11834/jig.20160911http://dx.doi.org/10.11834/jig.20160911］
Zhao X. 2021. Single image dehazing using bounded channel difference prior//Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. Nashville， USA： IEEE： 727-735 ［DOI： 10.1109/CVPRW53098.2021.00082http://dx.doi.org/10.1109/CVPRW53098.2021.00082］
Zhu H Y， Cheng Y， Peng X， Zhou J T， Kang Z， Lu S J， Fang Z W， Li L Y and Lim J H. 2021. Single-image dehazing via compositional adversarial network. IEEE Transactions on Cybernetics， 51（2）： 829-838 ［DOI： 10.1109/TCYB.2019.2955092http://dx.doi.org/10.1109/TCYB.2019.2955092］
Zhu Q S， Mai J M and Shao L. 2015. A fast single image haze removal algorithm using color attenuation prior. IEEE Transactions on Image Processing， 24（11）： 3522-3533 ［DOI： 10.1109/TIP.2015.2446191http://dx.doi.org/10.1109/TIP.2015.2446191］
相关作者
相关机构
京公网安备11010802024621