浏览全部资源
扫码关注微信
收稿日期：2024-10-04，
修回日期：2025-02-22，
录用日期：2025-02-25，
网络出版日期：2025-02-26，
移动端阅览
目的
2
使用低光图片训练神经渲染模型进行新视角合成无法得到正常光照条件下的图片，目标检测、语义分割等在处理低光照片时会产生明显的性能退化，面临着严峻的挑战，并且现有的方法在渲染速度和图像高频细节上存在问题。针对现有问题，本文提出一种对三维高斯泼溅模型进行低光增强的方法。
方法
2
首先利用一个轻量化的光照预测网络将三维高斯泼溅模型中三维高斯分布的颜色属性分解为物体本征颜色和光照两个部分，利用本征颜色渲染得到正常光照场景图片，同时使用多种损失函数从结构和颜色上改善图像质量；为了提高图片中高频细节的清晰度，采用固定几何的优化方案。
结果
2
实验在低光场景的新视角合成数据集LOM上与主流方法进行了比较，与现有最佳方法相比，在峰值信噪比指标上平均提升了0.12dB，在结构相似性指标（Structural Similarity Index， SSIM）上平均提升了1.3%，在学习感知图像块相似度指标（Learned Perceptual Image Patch Similarity， LPIPS）上平均提高了5.5%，训练时间仅有以往方法的1/5，渲染速度则达到以往方法的1000倍以上。
结论
2
本文所提出的方法能够更快地进行训练和渲染，同时也具有更高的图像质量，图像的高频细节和结构更加清晰，并通过全面的对比实验验证了方法的有效性与先进性。
Objective
2
Low-light image enhancement for neural rendering is a critical challenge， especially in the domain of novel view synthesis （NVS）， which aims to generate realistic images from new viewpoints based on a set of pre-captured images. When trained under low-light conditions， neural rendering models fail to produce high-quality results that resemble images captured under normal lighting. These low-light images introduce problems， particularly in downstream tasks such as object detection and semantic segmentation， due to their degraded quality. Traditional methods for addressing this issue often struggle to balance rendering speed with the preservation of high-frequency details in the generated images. This research aims to address these limitations by proposing a new method for enhancing low-light images using a 3D Gaussian point cloud splatting model. The goal is to improve the quality of rendered images while significantly reducing both the training and rendering times， making the approach suitable for practical applications in real-world scenarios where low-light conditions are common.
Method
2
The proposed method introduces a low-light enhancement framework based on the 3D Gaussian point cloud splatting model. The model is enhanced by decomposing the color attributes of the 3D Gaussian distributions into intrinsic object color and lighting components. A lightweight lighting prediction network is used to estimate the lighting conditions， which allows for the rendering of images that simulate normal lighting conditions， even when the training data consists solely of low-light images. To further improve image quality， a multi-stage gradient weighting strategy is employed. This helps in addressing noise issues commonly found in low-light images， where image noise is amplified due to insufficient lighting. The multi-stage gradient weighting technique ensures that the noise does not obscure important image details， leading to clearer results. Moreover， to retain high-frequency details in the rendered images， the method uses a fixed geometry optimization scheme. This is crucial because low-light conditions can cause distortions in the image structure， and fixed geometry ensures that the overall image clarity and structure are preserved. The core of the methodology revolves around the combination of color decomposition and geometry optimization， allowing the system to maintain high accuracy in rendering while improving efficiency.
Result
2
Experiments were conducted using the LOM dataset， which consists of low-light scene images and their corresponding normal-light images， and the results were compared to several mainstream methods in the field of neural rendering and low-light enhancement. The evaluation metrics used in the comparison include Peak Signal-to-Noise Ratio （PSNR）， Structural Similarity Index （SSIM）， and Learned Perceptual Image Patch Similarity （LPIPS）. The proposed method outperformed the current state-of-the-art approaches across all metrics. In terms of PSNR， the proposed method achieved a 0.12 dB improvement over the best existing method， indicating a higher signal-to-noise ratio and thus clearer images. SSIM saw a 1.3% improvement， showing that the structural similarity between the generated images and the ground truth was significantly better. LPIPS， which measures perceptual similarity， improved by 5.5%， demonstrating that the generated images were closer to human perception of the original scenes. In addition to improvements in image quality， the method also delivered significant gains in computational efficiency. The proposed approach reduced training and rendering times by over tenfold compared to traditional methods. This substantial speed-up is due to the explicit point cloud rendering technique used in the 3D Gaussian point cloud splatting model， which eliminates the need for complex and time-consuming calculations typically required in other neural rendering methods.
Conclusion
2
The proposed low-light enhancement method based on the 3D Gaussian point cloud splatting model offers a significant advancement over existing techniques in both image quality and computational efficiency. By decomposing the color attributes of the 3D Gaussian distributions into intrinsic object color and lighting components， the method enables the rendering of high-quality images under normal lighting conditions， even when the training data consists solely of low-light images. The multi-stage gradient weighting strategy effectively addresses the noise amplification issue common in low-light image processing， while the fixed geometry optimization ensures the preservation of high-frequency details and overall image structure. These innovations contribute to the method's superior performance in generating high-quality images for novel view synthesis under challenging low-light conditions. In terms of performance， the proposed method achieves notable improvements in PSNR， SSIM， and LPIPS compared to the state-of-the-art methods. The method's ability to produce clearer images with better structural and perceptual quality， combined with its significantly faster training and rendering times， makes it a promising solution for real-time applications in fields such as autonomous driving， virtual reality， and 3D scene reconstruction， where both image quality and computational efficiency are of paramount importance. In conclusion， this work demonstrates the effectiveness and superiority of the proposed method in enhancing low-light images for neural rendering， making it a valuable contribution to the fields of computer vision and image synthesis.
Barron J T ， Mildenhall B ， Verbin D ， Srinivasan P P and Hedman P . 2023 . Zip-nerf： Anti-aliased grid-based neural radiance fields // Proceedings of the IEEE/CVF International Conference on Computer Vision . Paris， France ： IEEE： 19697 - 19705 ［ DOI： 10.1109/ICCV51070.2023.01804 http://dx.doi.org/10.1109/ICCV51070.2023.01804 ］
Brooks T ， Mildenhall B ， Xue T F ， Chen J W ， Sharlet D and Barron J T . 2019 . Unprocessing images for learned raw denoising // Proceedings of the IEEE/CVF conference on computer vision and pattern recognition . Long Beach， USA ： IEEE： 11036 - 11045 ［ DOI： 10.1109/CVPR.2019.01129 http://dx.doi.org/10.1109/CVPR.2019.01129 ］
Cao Y K ， Cao Y P ， Han K ， Shan Y and Wong K K . 2023 . Dreamavatar： Text-and-shape guided 3d human avatar generation via diffusion models ［EB/OL］. ［ 2023-11-30 ］. https://arxiv.org/pdf/2304.00916 https://arxiv.org/pdf/2304.00916
Chen A P ， Xu Z X ， Geiger A ， Yu J Y and Su H . 2022 . Tensorf： Tensorial radiance fields // European Conference on Computer Vision . Tel Aviv， Israel ： Springer： 333 - 350 ［ DOI： 10.1007/978-3-031-19824-3_20 http://dx.doi.org/10.1007/978-3-031-19824-3_20 ］
Cui Z T ， Gu L ， Sun X ， Ma X Z ， Qiao Y and Harada T . 2024 . Aleth-nerf： Illumination adaptive nerf with concealing field assumption // Proceedings of the AAAI Conference on Artificial Intelligence . Vancouver， Canada ： AAAI： 1435 - 1444 ［ DOI： 10.1609/aaai.v38i2.27908 http://dx.doi.org/10.1609/aaai.v38i2.27908 ］
Cui Z T ， Li K C ， Gu L ， Su S H ， Gao P ， Jiang Z K ， Qiao Y and Harada T . 2022 . You only need 90k parameters to adapt light： a light weight transformer for image enhancement and exposure correction ［EB/OL］.［ 2022-10-08 ］. https://arxiv.org/pdf/2205.14871 https://arxiv.org/pdf/2205.14871
Cui Z T ， Qi G J ， Gu L ， You S D ， Zhang Z H and Harada T . 2021 . Multitask aet with orthogonal tangent regularity for dark object detection // Proceedings of the IEEE/CVF international conference on computer vision . Montreal， Canada ： IEEE： 2553 - 2562 ［ DOI： 10.1109/ICCV48922.2021.00255 http://dx.doi.org/10.1109/ICCV48922.2021.00255 .］
Guo C L ， Li C Y ， Guo J C ， Loy C C ， Hou J H ， Kwong S and Cong R M . 2020 . Zero-reference deep curve estimation for low-light image enhancement .// Proceedings of the IEEE/CVF conference on computer vision and pattern recognition . Seattle， USA ： IEEE： 1780 - 1789 ［ DOI： 10.1109/CVPR42600.2020.00185 http://dx.doi.org/10.1109/CVPR42600.2020.00185 ］
Huang Ying ， Peng Hui ， Li Changsheng ， Gao Shengmei ， Chen Feng . 2024 . LLFlowGAN： a low-light image enhancement method for constraining invertible flow in a generative adversarial manner . Journal of Image and Graphics ， 29 （ 01 ）： 0065 - 0079
黄颖 ， 彭慧 ， 李昌盛 ， 高胜美 ， 陈奉 . 2024 . LLFlowGAN：以生成对抗方式约束可逆流的低照度图像增强 . 中国图象图形学报 ， 29 （ 01 ）： 0065 - 0079 ［ DOI： 10.11834/jig.230063 http://dx.doi.org/10.11834/jig.230063 ］
Kerbl B ， Kopanas G ， Leimkühler T and Drettakis G . 2023 . 3D Gaussian Splatting for Real-Time Radiance Field Rendering . ACM Transactions on Graphics ， 42 （ 4 ）： 139 ： 1 - 139 ： 14 ［ DOI： 10.1145/3528223. 3530127 http://dx.doi.org/10.1145/3528223.3530127 ］
Land E H . 1986 . An alternative technique for the computation of the designator in the retinex theory of color vision . Proceedings of the National Academy of Sciences ， 83 （ 10 ）： 3078 - 3080 ［ DOI： 10.1073/pnas.83.10.3078 http://dx.doi.org/10.1073/pnas.83.10.3078 ］
Li J H ， Zhang J W ， Bai X ， Zheng J ， Ning X ， Zhou J and Gu L . 2024 . Dngaussian： Optimizing sparse-view 3d gaussian radiance fields with global-local depth normalization // Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . Seattle， USA ： IEEE： 20775 - 20785 ［ DOI： 10.1109/CVPR52733.2024.01963 http://dx.doi.org/10.1109/CVPR52733.2024.01963 ］
Li K J ， Tang Y S ， Prisacariu V A and Torr P H . 2022 . Bnv-fusion： Dense 3d reconstruction using bi-level neural volume fusion // Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . New Orleans， USA ： IEEE： 6166 - 6175 ［ DOI： 10.1109/CVPR52688.2022.00607 http://dx.doi.org/10.1109/CVPR52688.2022.00607 ］
Liu L J ， Habermann M ， Rudnev V ， Sarkar K ， Gu J T and Theobalt C . 2021 . Neural actor： Neural free-view synthesis of human actors with pose control . ACM transactions on graphics （TOG） ， 40 （ 6 ）： 1 - 16 ［ DOI： 10.1145/3478513.3480528 http://dx.doi.org/10.1145/3478513.3480528 ］
Liu S ， Zhang X M ， Zhang Z T ， Zhang R ， Zhu J Y and Russell B . 2021 . Editing conditional radiance fields // Proceedings of the IEEE/CVF international conference on computer vision . Montreal， Canada ： IEEE： 5773 - 5783 ［ DOI： 10.1109/iccv48922.2021.00572 http://dx.doi.org/10.1109/iccv48922.2021.00572 ］
Ma L ， Ma T Y ， Liu R S ， Fan X and Luo Z X . 2022 . Toward fast ， flexible， and robust low-light image enhancement// Proceedings of the IEEE/CVF conference on computer vision and pattern recognition . New Orleans， USA ： IEEE： 5637 - 5646 ［ DOI： 10.1109/CVPR52688.2022.00555 http://dx.doi.org/10.1109/CVPR52688.2022.00555 .］
Mildenhall B ， Pratul P . Srinivasan P P ， Tancik M ， Barron J T，Ramamoorthi R and Ng R. 2021 . NeRF： representing scenes as neural radiance fields for view synthesis. Communications of the ACM， 65 （ 1 ）： 99 – 106 ［ DOI： 10.1145/3503250 http://dx.doi.org/10.1145/3503250 ］
Omprakash P ， Maravi Y P and Sharma S . 2013 . A comparative study of histogram equalization based image enhancement techniques for brightness preservation and contrast enhancement ［EB/OL］.［ 2013-11-16 ］. https://arxiv.org/pdf/1311.4033 https://arxiv.org/pdf/1311.4033
Peng S D ， Dong J T ， Wang Q Q ， Zhang S Z ， Shuai Q ， Zhou X W and Bao H J . 2021 . Animatable neural radiance fields for modeling dynamic human bodies // Proceedings of the IEEE/CVF International Conference on Computer Vision . Montreal Canada ： ICCV： 14314 - 14323 ［ DOI： 10.1109/ICCV48922.2021.01405 http://dx.doi.org/10.1109/ICCV48922.2021.01405 ］
Rosu R A ， Saito S ， Wang Z Y ， Wu C L ， Behnke S and Nam G . 2022 . Neural strands： Learning hair geometry and appearance from multi-view images // European Conference on Computer Vision . Tel Aviv， Israel ： Springer： 73 - 89 ［ DOI： 10.1007/978-3-031-19827-4_5 http://dx.doi.org/10.1007/978-3-031-19827-4_5 ］
Wang H Y ， Xu X G ， Xu K and Lau R W . 2023 . Lighting up nerf via unsupervised decomposition and enhancement // Proceedings of the IEEE/CVF International Conference on Computer Vision . Paris， France ： IEEE： 12632 - 12641 ［ DOI： 10.1109/ICCV51070.2023.01161 http://dx.doi.org/10.1109/ICCV51070.2023.01161 ］
Wei C ， Wang W J ， Yang W H and Liu J Y . 2018 . Deep retinex decomposition for low-light enhancement ［EB/OL］.［ 2018-08-14 ］ https://arxiv.org/pdf/1808.04560 https://arxiv.org/pdf/1808.04560
Xu Q G ， Xu Z X ， Philip J ， Bi S ， Shu Z X ， Sunkavalli K and Neumann U . 2022 . Point-nerf： Point-based neural radiance fields // Proceedings of the IEEE/CVF conference on computer vision and pattern recognition . New Orleans， USA ： IEEE： 5438 - 5448 ［ DOI： 10.1109/CVPR52688.2022.00536 http://dx.doi.org/10.1109/CVPR52688.2022.00536 .］
Yan Xiaoyang ， Wang Huake ， Hou Xingsong ， Dun Yujie . 2023 . Dual-branch low-light image enhancement network via YCbCr space divide-and-conquer . Journal of Image and Graphics ， 28 （ 11 ）： 3415 - 3427
闫晓阳 ， 王华珂 ， 侯兴松 ， 顿玉洁 . 2023 . YCbCr空间分治的双分支低照度图像增强网络 . 中国图象图形学报 ， 28 （ 11 ）： 3415 - 3427 ［ DOI： 10.11834/jig.221028 http://dx.doi.org/10.11834/jig.221028 ］
Yang Z P ， Chai Y N ， Anguelov D ， Zhou Y ， Sun P ， Erhan D ， Rafferty S and Kretzschmar H . 2020 . Surfelgan： Synthesizing realistic sensor data for autonomous driving // Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . Seattle， USA ： IEEE： 11118 - 11127 ［ DOI： 10.1109/CVPR42600.2020.01113 http://dx.doi.org/10.1109/CVPR42600.2020.01113 ］
Ye S ， Dong Z H ， Wen Y H and Liu Y J . 2024 . Gaussian in the Dark： Real-Time View Synthesis From Inconsistent Dark Images Using Gaussian Splatting . Computer Graphics Forum ， 43 （ 7 ）：e 15213 ［ DOI： 10.1111/cgf.15213 http://dx.doi.org/10.1111/cgf.15213 ］
You J J ， Zhang Y Y ， Zhou T C ， Zhao Y C and Yao L . 2024 . LO-Gaussian： Gaussian Splatting for Low-light and Overexposure Scenes throught Simulated Filter . Pacific Graphics Conference Papers and Posters . ［ DOI： 10.2312/pg.20241301 http://dx.doi.org/10.2312/pg.20241301 ］
Zhang F ， Li Y ， You S D and Fu Y . 2021 . Learning temporal consistency for low light video enhancement from single images // Proceedings of the IEEE/CVF conference on computer vision and pattern recognition . Nashville， USA ： CVPR： 4967 - 4976 ［ DOI： 10.1109/CVPR46437.2021.00493 http://dx.doi.org/10.1109/CVPR46437.2021.00493 ］
Zhang Hang ， Yan Jia . 2024 . Low-light image enhancement guided by semantic segmentation and HSV color space . Journal of Image and Graphics ， 29 （ 04 ）： 0966 - 0977 DOI： 10.11834/jig.230182 http://dx.doi.org/10.11834/jig.230182 .
张航 ， 颜佳 . 2024 . 语义分割和HSV色彩空间引导的低光照图像增强 . 中国图象图形学报 ， 29 （ 04 ）： 0966 - 0977 ［ DOI： 10.11834/jig.230182 http://dx.doi.org/10.11834/jig.230182 ］
Zhao Minghua ， Wen Yichun ， Du Shuangli ， Hu Jing ， Shi Cheng ， Li Peng . 2024 . Low-light image enhancement algorithm based on illumination and scene texture attention map . Journal of Image and Graphics ， 29 （ 04 ）： 0862 - 0874 DOI： 10.11834/jig.230271 http://dx.doi.org/10.11834/jig.230271 .
赵明华 ， 汶怡春 ， 都双丽 ， 胡静 ， 石程 ， 李鹏 . 2024 . 基于照度与场景纹理注意力图的低光图像增强 . 中国图象图形学报 ， 29 （ 04 ）： 0862 - 0874 ［ DOI： 10.11834/jig.230271 http://dx.doi.org/10.11834/jig.230271 ］
Zhuang Y Y ， Zhu H ， Sun X S and Cao X . 2022 . Mofanerf： Morphable facial neural radiance field // European conference on computer vision . Tel Aviv， Israel ： Springer： 268 - 285 ［ DOI： 10.1007/978-3-031-20062-5_16 http://dx.doi.org/10.1007/978-3-031-20062-5_16 ］
Zou Y ， Li X Y ， Jiang Z Y and Liu J Y . 2024 . Enhancing Neural Radiance Fields with Adaptive Multi-Exposure Fusion： A Bilevel Optimization Approach for Novel View Synthesis // Proceedings of the AAAI Conference on Artificial Intelligence . Vancouver， Canada ： AAAI： 7882 - 7890 ［ DOI： 10.1609/aaai.v38i7.28624 http://dx.doi.org/10.1609/aaai.v38i7.28624 ］
相关作者
相关机构
京公网安备11010802024621