Abstract
Deep learning has demonstrated remarkable success in augmenting fluorescence imaging under photon-limited conditions. However, existing restoration networks are typically devised for training with augmented patches far smaller than the full-view raw data, an overlooked aspect that compromises fidelity and noise-resistance due to the loss of global statistics. To address this limitation, we propose a large-patch network (LargePNet), which synergizes the large effective receptive field provided by shallow ultra-large-kernel convolutions and the nonlinear representation capabilities of deep networks through scale separation. It effectively and efficiently leverages large-view global information for restoration. Directly trained with large-view images, LargePNet shows contrasting advantages over state-of-the-art small-patch networks, with 0.5-2 dB higher peak signal-to-noise ratio across eight representative restoration tasks, involving implementations for single-image, video, and volumetric fluorescence data. For full-view processing, LargePNet generally holds around 4-fold and 20-fold higher computational efficiency compared to advanced convolution-based and Transformer-based networks, respectively. The assistance of LargePNet helps achieve 30-hour-long fluorescence imaging to monitor cytoskeleton dynamics, and hour-long tri-color super-resolution imaging to investigate organelle interaction, showcasing its advancement in live-cell imaging.
Similar content being viewed by others
Data availability
The open-source data used in this study are all publicly available, as listed in Supplementary Table 15. The self-established dataset of STED denoising/deblurring, virtual sampling of SMLM, volumetric background removal datasets, and the source training data for the open-source BioSR and BioTISR datasets are available: https://zenodo.org/records/15694668.
Code availability
The source Python code of the LargePNet series, including LargePNet (for single-image restoration), LargeP-GAN (for generative single-image restoration), LargeP-TISR (for time-lapse video restoration), and 3D-LargePNet (for volumetric data restoration) are all publicly available at the GitHub repository56: https://github.com/YiweiHou/LargePNet-for-fluorescence-image-restoration. Trained LargePNet models that can reproduce the results in the paper are available at: https://figshare.com/s/05f576c96b08add7eee0.
References
Ledig, C. et al. Photo-realistic single image super-resolution using a generative adversarial network. In Proc. IEEE Conference on Computer Vision and Pattern Recognition 105–114 (IEEE, 2017).
Liang, J. et al. SwinIR: image restoration using Swin Transformer. In Proc. IEEE/CVF International Conference on Computer Vision Workshops 1833–1844 (IEEE, 2021).
Saharia, C. et al. Image super-resolution via iterative refinement. IEEE Trans. Pattern Anal. Mach. Intell. 45, 4713–4726 (2023).
Zhang, K., Zuo, W., Chen, Y., Meng, D. & Zhang, L. Beyond a Gaussian denoiser: residual learning of deep CNN for image denoising. IEEE Trans. Image Process. 26, 3142–3155 (2016).
Zhang, Y. et al. Image super-resolution using very deep residual channel attention networks. In Proc. European Conference on Computer Vision 294–310 (Springer, 2018).
Chen, J. et al. Three-dimensional residual channel attention networks denoise and sharpen fluorescence microscopy image volumes. Nat. Methods 18, 678–687 (2021).
Chen, R. et al. Single-frame deep-learning super-resolution microscopy for intracellular dynamics imaging. Nat. Commun. 14, 2854 (2023).
Ebrahimi, V. et al. Deep learning enables fast, gentle STED microscopy. Commun. Biol. 6, 674 (2023).
Jin, L. et al. Deep learning enables structured illumination microscopy with low light levels and enhanced speed. Nat. Commun. 11, 1934 (2020).
Li, X. et al. Real-time denoising enables high-sensitivity fluorescence time-lapse imaging beyond the shot-noise limit. Nat. Biotechnol. 41, 282–292 (2022).
Li, Y. et al. Incorporating the image formation process into deep learning improves network performance. Nat. Methods 19, 1427–1437 (2022).
Lu, C. et al. Diffusion-based deep learning method for augmenting ultrastructural imaging and volume electron microscopy. Nat. Commun. 15, 4677 (2024).
Ma, C., Tan, W., He, R. & Yan, B. Pretraining a foundation model for generalizable fluorescence microscopy-based image restoration. Nat. Methods 21, 1558–1567 (2024).
Ounkomol, C., Seshamani, S., Maleckar, M. M., Collman, F. & Johnson, G. R. Label-free prediction of three-dimensional fluorescence images from transmitted-light microscopy. Nat. Methods 15, 917–920 (2018).
Ouyang, W., Aristov, A., Lelek, M., Hao, X. & Zimmer, C. Deep learning massively accelerates super-resolution localization microscopy. Nat. Biotechnol. 36, 460–468 (2018).
Qiao, C. et al. Evaluation and development of deep neural networks for image super-resolution in optical microscopy. Nat. Methods 18, 194–202 (2021).
Qiao, C. et al. Rationalized deep learning super-resolution microscopy for sustained live imaging of rapid subcellular processes. Nat. Biotechnol. 41, 367–377 (2022).
Qiao, C. et al. A neural network for long-term super-resolution imaging of live cells with reliable confidence quantification. Nat. Biotechnol. 44, 110–119 (2026).
Qu, L. et al. Self-inspired learning for denoising live-cell super-resolution microscopy. Nat. Methods 21, 1895–1908 (2024).
Wang, H. et al. Deep learning enables cross-modality super-resolution in fluorescence microscopy. Nat. Methods 16, 103–110 (2018).
Wang, Z., Xie, Y. & Ji, S. Global voxel transformer networks for augmented microscopy. Nat. Mach. Intell. 3, 161–171 (2021).
Weigert, M. et al. Content-aware image restoration: pushing the limits of fluorescence microscopy. Nat. Methods 15, 1090–1097 (2018).
Zhang, G. et al. Bio-friendly long-term subcellular dynamic recording by self-supervised image enhancement microscopy. Nat. Methods 20, 1957–1970 (2023).
Chen, X. et al. Self-supervised denoising for multimodal structured illumination microscopy enables long-term super-resolution live-cell imaging. PhotoniX 5, 4 (2024).
Bilodeau, A. et al. Development of AI-assisted microscopy frameworks through realistic simulation with pySTED. Nat. Mach. Intell. 6, 1197–1215 (2024).
Fu, Y., Hou, Y., Liang, Q. et al. Triangle-beam interference structured illumination microscopy. Nat. Photon. 19, 1122–1131 (2025).
Guo, M. et al. Deep learning-based aberration compensation improves contrast and resolution in fluorescence microscopy. Nat. Commun. 16, 313 (2025).
Hou, Y. et al. Multi-resolution analysis enables fidelity-ensured deconvolution for fluorescence microscopy. eLight 4, 14 (2024).
Huang, X. et al. Fast, long-term, super-resolution imaging with Hessian structured illumination microscopy. Nat. Biotechnol. 36, 451–459 (2018).
He, K., Zhang, X., Ren, S. & Sun, J. Deep residual learning for image recognition. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2016).
Lecun, Y. & Bottou, L. Gradient-based learning applied to document recognition. Proc. IEEE 86, 2278–2324 (1998).
Ronneberger, O., Fischer, P. & Brox, T. U-Net: convolutional networks for biomedical image segmentation. In Proc. International Conference on Medical Image Computing and Computer-Assisted Intervention (Springer, 2015).
Raghu, M., Unterthiner, T., Kornblith, S., Zhang, C. & Dosovitskiy, A. Do vision transformers see like convolutional neural networks? In Proc. Conference and Workshop on Neural Information Processing Systems (MIT Press, 2021).
Ding, X. et al. Scaling up your kernels to 31x31: revisiting large kernel design in CNNs. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2022).
Ding, X. et al. UniRepLKNet: a universal perception large-kernel ConvNet for audio, video, point cloud, time-series and image recognition. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2023).
Dosovitskiy, A., Beyer, L., Kolesnikov, A. & Weissenborn, D. An image is worth 16x16 words: transformers for image recognition at scale. In Proc. International Conference on Learning Representations (OpenReview.net, 2021).
Liu, Z. et al. Swin Transformer: hierarchical vision transformer using shifted windows. In Proc. IEEE/CVF International Conference on Computer Vision (IEEE, 2021).
Vaswani, A., Shazeer, N., Parmar, N. & Uszkoreit, J. Attention is all you need. In Proc. Advances in Neural Information Processing Systems (MIT Press, 2017).
Levin, A., Nadler, B., Durand, F. & Freeman, W. T. Patch Complexity, Finite Pixel Correlations and Optimal Denoising. In Computer Vision—ECCV 2012 (eds Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y. & Schmid, C.) 73–86 (Springer Berlin Heidelberg, 2012).
Chu, X., Chen, L., Chen, C. & Lu, X. Improving image restoration by revisiting global information aggregation. In Proc. ECCV (Springer, 2022).
Yu, Z. et al. Scale calibrated training: improving generalization of deep networks via scale-specific normalization. In Proc. CVPR (IEEE, 2020).
Chen, L., Lu, X., Zhang, J., Chu, X. & Chen, C. HINet: half instance normalization network for image restoration. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2021).
Ulyanov, D., Vedaldi, A. & Lempitsky, V. Instance normalization: the missing ingredient for fast stylization. Preprint at arXiv https://doi.org/10.48550/arXiv.1607.08022 (2016).
Luo, W., Li, Y., Urtasun, R. & Zemel, R. Understanding the effective receptive field in deep convolutional neural networks. In Proc. Advances in Neural Information Processing Systems (MIT Press, 2017).
Liu, B., Zhu, Y., Song, K. & Elgammal, A. Towards faster and stabilized GAN training for high-fidelity few-shot image synthesis. In Proc. International Conference on Learning Representations (OpenReview.net, 2021).
Simonyan, K. & Zisserman, A. Very deep convolutional networks for large-scale image recognition. In Proc. International Conference on Learning Representations (OpenReview.net, 2014).
Hell, S. W. & Wichmann, J. Breaking the diffraction resolution limit by stimulated emission: stimulated-emission-depletion fluorescence microscopy. Opt. Lett. 19, 780–782 (1994).
Betzig, E. et al. Imaging intracellular fluorescent proteins at nanometer resolution. Science 313, 1642–1645 (2006).
Rust, M. J., Bates, M. & Zhuang, X. Sub-diffraction-limit imaging by stochastic optical reconstruction microscopy (STORM). Nat. Methods 3, 793–796 (2006).
Goodfellow, I. et al. Generative adversarial nets. In Proc. Conference on Neural Information Processing Systems (MIT Press, 2014).
Wang, X. et al. ESRGAN: Enhanced Super-resolution Generative Adversarial Networks. In Proc. European Conference on Computer Vision (Springer, 2018).
Wang, L., Guo, Y., Liu, L., Lin, Z. & An, W. Deep video super-resolution using HR optical flow estimation. IEEE Trans. Image Process. PP, 1–1 (2020).
Loos, V., Pardasani, R. & Awasthi, N. Demystifying the effect of receptive field size in U-Net models for medical image segmentation. J. Med. Imaging 11, 054004 (2024).
Wang, F. et al. Phase imaging with an untrained neural network. Light Sci. Appl. 9, 77 (2020).
Schnitzbauer, J., Strauss, M. T., Schlichthaerle, T., Schueder, F. & Jungmann, R. Super-resolution microscopy with DNA-PAINT. Nat. Protoc. 12, 1198–1228 (2017).
Hou, Y. et al. Pushing the limits of fluorescence imaging with a restoration neural network aggregating large-view statistics. Zenodo, https://doi.org/10.5281/zenodo.18820335 (2026).
Arganda-Carreras, I. et al. Trainable Weka Segmentation: a machine learning tool for microscopy pixel classification. Bioinformatics 33, 2424–2426 (2017).
Descloux, A., Grußmayer, K. S. & Radenovic, A. Parameter-free image resolution estimation based on decorrelation analysis. Nat. Methods 16, 918–924 (2019).
Acknowledgements
This work was supported by the National Natural Science Foundation of China (62025501 to P.X.; 62335008, 62405010 to M.L.), National Key R&D Program of China (2022YFC3401100 to P.X.), and Major Basic Research Project of the Natural Science Foundation of Shandong Province (ZR2024ZD27 to P.X.). The authors thank the National Center for Protein Sciences at Peking University in Beijing, China, for assistance with STED super-resolution imaging.
Author information
Authors and Affiliations
Contributions
Y. Hou and P. Xi conceived the idea of LargePNet for fluorescence image restoration tasks. P. Xi and M. Li supervised this project. Y. Hou developed all the codes and performed all the training and validation experiments. S. Gao captured the training dataset for STED and SMLM and performed all the live-cell imaging experiments. W. Ren provided part of the STED mitochondria imaging data. Y. Fu provided insightful discussions. Y. Hou, S. Gao, M. Li, and P. Xi wrote the paper after discussions with all the authors.
Corresponding authors
Ethics declarations
Competing interests
The authors declare no competing interests.
Peer review
Peer review information
Nature Communications thanks Ian Dobbie and the other, anonymous, reviewer(s) for their contribution to the peer review of this work. A peer review file is available.
Additional information
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary information
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/.
About this article
Cite this article
Hou, Y., Gao, S., Ren, W. et al. Pushing the limits of fluorescence imaging with a restoration neural network aggregating large-view statistics. Nat Commun (2026). https://doi.org/10.1038/s41467-026-71278-2
Received:
Accepted:
Published:
DOI: https://doi.org/10.1038/s41467-026-71278-2
