Abstract
Low-light face videos suffer from severe noise and detail loss, limiting their use in surveillance and photography applications. To address these challenges, this paper proposes DL-Diff, a novel low-light face video enhancement framework that formulates this task as a conditional video-to-video (V2V) generation problem based on pre-trained Latent Diffusion Models (LDMs). DL-Diff extends pre-trained text-to-video models through three components: a pseudo-3D UNet backbone, a restoration component for spatial detail recovery, and a temporal component for inter-frame consistency. A multi-stage training strategy enables efficient domain adaptation from images to videos. Experiments on DID and SDSD datasets demonstrate that DL-Diff achieves superior performance in both perceptual quality (FID: 41.29, LPIPS: 0.17) and temporal consistency (AB(Var): 25.40, MABD: 0.08), significantly outperforming existing methods. The framework produces high-quality videos with realistic visual effects and no flickering artifacts, particularly excelling in extremely dark scenarios. This work demonstrates the potential of leveraging pre-trained diffusion models for video enhancement tasks.
Similar content being viewed by others
Data availability
The datasets generated and/or analyzed during the current study are publicly available. The Diagnostic Imaging Data Set (DID) can be accessed at https://digital.nhs.uk/data-and-information/data-collections-and-data-sets/data-sets/diagnostic-imaging-data-set. Additionally, the SDSD Dataset is available at https://doi.org/10.48550/arXiv.2404.00834.
References
He, C., Li, K., Xu, G. et al Degradation-resistant unfolding network for heterogeneous image fusion. In 2023 IEEE/CVF International Conference on Computer Vision (ICCV). 12577–12587. https://doi.org/10.1109/ICCV51070.2023.01159 (2023).
Xia, B., Zhang, Y., Wang, S. et al Diffir: Efficient diffusion model for image restoration. In 2023 IEEE/CVF International Conference on Computer Vision (ICCV). 13049–13059. https://doi.org/10.1109/ICCV51070.2023.01204 (2023).
Liu, Z., Luo, P., Wang, X. et al Deep learning face attributes in the wild. In 2015 IEEE International Conference on Computer Vision (ICCV). 3730–3738. https://doi.org/10.1109/ICCV.2015.425 (2015).
Moser, B. B. et al. Diffusion models, image super-resolution, and everything: A survey. IEEE Trans. Neural Netw. Learn. Syst. 36(7), 11793–11813. https://doi.org/10.1109/TNNLS.2024.3476671 (2025).
Saharia, C. et al. Image super-resolution via iterative refinement. IEEE Trans. Pattern Anal. Mach. Intell. 45(4), 4713–4726. https://doi.org/10.1109/TPAMI.2022.3204461 (2023).
Yu, F., Gu, J., Li, Z. et al Scaling up to excellence: Practicing model scaling for photo-realistic image restoration in the wild. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 25669–25680. https://doi.org/10.1109/CVPR52733.2024.02425 (2024).
Duan, J. et al. A survey of embodied AI: From simulators to research tasks. IEEE Trans. Emerg. Top. Comput. Intell. 6(2), 230–244. https://doi.org/10.1109/TETCI.2022.3141105 (2022).
Liu, J., Fan, X., Huang, Z. et al Target-aware dual adversarial learning and a multi-scenario multi-modality benchmark to fuse infrared and visible for object detection. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 5792–5801. https://doi.org/10.1109/CVPR52688.2022.00571 (2022).
Agustsson, E. & Timofte, R. Ntire 2017 challenge on single image super-resolution: Dataset and study. In 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). 1122–1131. https://doi.org/10.1109/CVPRW.2017.150 (2017).
He, C. et al. Hqg-net: Unpaired medical image enhancement with high-quality guidance. IEEE Trans. Neural Netw. Learn. Syst. 35(12), 18404–18418. https://doi.org/10.1109/TNNLS.2023.3315307 (2024).
Shen, Z., Wang, W., Lu, X. et al Human-aware motion deblurring. In 2019 IEEE/CVF International Conference on Computer Vision (ICCV). 5571–5580. https://doi.org/10.1109/ICCV.2019.00567 (2019).
Yuan, X., Baek, J., Xu, K. et al Inflation with diffusion: Efficient temporal adaptation for text-to-video super-resolution. In 2024 IEEE/CVF Winter Conference on Applications of Computer Vision Workshops (WACVW). 489–496. https://doi.org/10.1109/WACVW60836.2024.00059 (2024).
Chen, Y. et al. Dual degradation image inpainting method via adaptive feature fusion and u-net network. Appl. Soft Comput. 174, 113010. https://doi.org/10.1016/j.asoc.2024.113010 (2025).
Xu, G. et al. Dm-fusion: Deep model-driven network for heterogeneous image fusion. IEEE Trans. Neural Netw. Learn. Syst. 35(7), 10071–10085. https://doi.org/10.1109/TNNLS.2023.3238511 (2024).
Yi, X., Xu, H., Zhang, H. et al Diff-retinex: Rethinking low-light image enhancement with a generative diffusion model. In 2023 IEEE/CVF International Conference on Computer Vision (ICCV). 12268–12277. https://doi.org/10.1109/ICCV51070.2023.01130 (2023).
Yue, J. et al. Dif-fusion: Toward high color fidelity in infrared and visible image fusion with diffusion models. IEEE Trans. Image Process. 32, 5705–5720. https://doi.org/10.1109/TIP.2023.3322046 (2023).
Zhao, Z., Bai, H., Zhu, Y. et al DDFM: Denoising diffusion model for multi-modality image fusion. In 2023 IEEE/CVF International Conference on Computer Vision (ICCV). 8048–8059. https://doi.org/10.1109/ICCV51070.2023.00742 (2023).
Chen, Z., Long, F., Qiu, Z. et al Learning spatial adaptation and temporal coherence in diffusion models for video super-resolution. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 9232–9241. https://doi.org/10.1109/CVPR52733.2024.00882 (2024).
Liu, J., Anirudh, R., Thiagarajan, J.J. et al Dolce: A model-based probabilistic diffusion framework for limited-angle CT reconstruction. In 2023 IEEE/CVF International Conference on Computer Vision (ICCV). 10464–10474. https://doi.org/10.1109/ICCV51070.2023.00963 (2023).
Zhang, R., Isola, P., Efros, A.A. et al The unreasonable effectiveness of deep features as a perceptual metric. In 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 586–595. https://doi.org/10.1109/CVPR.2018.00068 (2018).
Li, Y. et al. Zero-shot medical image translation via frequency-guided diffusion models. IEEE Trans. Med. Imaging 43(3), 980–993. https://doi.org/10.1109/TMI.2023.3325703 (2024).
Peng, Y. et al. Top-level design and simulated performance of the first portable CT-MR scanner. IEEE Access 10, 102325–102333. https://doi.org/10.1109/ACCESS.2022.3208278 (2022).
Zhao, K. et al. MRI super-resolution with partial diffusion models. IEEE Trans. Med. Imaging 44(3), 1194–1205. https://doi.org/10.1109/TMI.2024.3483109 (2025).
Wang, Y., Yang, W., Chen, X. et al Sinsr: Diffusion-based image super-resolution in a single step. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 25796–25805. https://doi.org/10.1109/CVPR52733.2024.02437 (2024).
Ancuti C.O., Ancuti, C. & Timofte, R. Nh-haze: An image dehazing benchmark with non-homogeneous hazy and haze-free images. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). 1798–1805. https://doi.org/10.1109/CVPRW50498.2020.00230 (2020).
Blattmann, A., Rombach, R., Ling, H. et al Align your latents: High-resolution video synthesis with latent diffusion models. In 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 22563–22575. https://doi.org/10.1109/CVPR52729.2023.02161 (2023).
Chen, Y., Chen, L., Xia, R. et al Cat: Image super-resolution algorithm via channel attention and transformer. Array 100628. https://doi.org/10.1016/j.array.2024.100628 (2025).
Guan, S. et al. Fully dense unet for 2-D sparse photoacoustic tomography artifact removal. IEEE J. Biomed. Health Inform. 24(2), 568–576 (2019).
Fu, H., Zheng, W., Wang, X. et al Dancing in the dark: A benchmark towards general low-light video enhancement. In 2023 IEEE/CVF International Conference on Computer Vision (ICCV). 12831–12840. https://doi.org/10.1109/ICCV51070.2023.01183 (2023).
Wang, R., Xu, X., Fu, C.W. et al Seeing dynamic scene in the dark: A high-quality video dataset with mechatronic alignment. In 2021 IEEE/CVF International Conference on Computer Vision (ICCV). 9680–9689. https://doi.org/10.1109/ICCV48922.2021.00956(2021).
Dosovitskiy, A. & Brox, T. Generating images with perceptual similarity metrics based on deep networks. Adv. Neural Inf. Process. Syst. 29 (2016).
Buzuti, L. F. & Thomaz, C. E. Fréchet autoencoder distance: A new approach for evaluation of generative adversarial networks. Comput. Vis. Image Underst. 235, 103768 (2023).
Erfurt, J., Helmrich, C.R., Bosse, S. et al A study of the perceptually weighted peak signal-to-noise ratio (WPSNR) for image compression. In 2019 IEEE International Conference on Image Processing (ICIP). 2339–2343 (IEEE, 2019)
Bakurov, I. et al. Structural similarity index (SSIM) revisited: A data-driven approach. Expert Syst. Appl. 189, 116087 (2022).
Xu, X., Zhou, K., Hu, T. et al Low-light video enhancement via spatial-temporal consistent decomposition. arXiv preprint arXiv:2405.15660 (2024).
Liu, L., An, J., Liu, J. et al Low-light video enhancement with synthetic event guidance. In Proceedings of the AAAI Conference on Artificial Intelligence. 1692–1700. https://doi.org/10.1609/aaai.v37i2.25257 (2023).
Hou, J., Zhu, Z., Hou, J. et al Global structure-aware diffusion process for low-light image enhancement. In Advances in Neural Information Processing Systems (NeurIPS). 79734–79747 (2023)
Zhu, Y., Zhao, W., Li, A. et al Flowie: Efficient image enhancement via rectified flow. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 13–22. https://doi.org/10.1109/CVPR52733.2024.00010 (2024).
Author information
Authors and Affiliations
Contributions
Xiaofeng Ding: Writing—original draft; Writing—review & editing; Conceptualization; Resources; Formal Analysis. Kaitong He: Writing—review & editing; Methodology;Supervision; Methodology; Software. Huo Sun: Writing—original draft;Writing—review & editing;Conceptualization;Resources;Data curation; Juying Yang: Writing—review & editing;Methodology; Supervision;Formal analysis. All authors have read and agreed to the published version of the manuscript.
Corresponding author
Ethics declarations
Competing interests
The authors declare no competing interests.
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/.
About this article
Cite this article
Ding, X., He, K., Sun, H. et al. Temporally consistent low-light face video enhancement via video-to-video conditional diffusion. Sci Rep (2026). https://doi.org/10.1038/s41598-026-44219-8
Received:
Accepted:
Published:
DOI: https://doi.org/10.1038/s41598-026-44219-8
