Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Article
  • Published:

All-optical image transportation through a multimode fibre using a miniaturized diffractive neural network on the distal facet

Nature Photonics volume 19, pages 486–493 (2025)Cite this article

Subjects

Abstract

The direct optical transportation of images through multimode fibres (MMFs) is highly sought after in compact photonic systems for MMF-based optical information processing. However, MMFs are highly scattering media, thus degrading information transmitted through them. Existing approaches utilize artificial neural networks or spatial light modulators to reconstruct images scrambled after propagation through the fibre. Despite these advances, achieving direct optical image transportation through MMFs using integrated optical elements with micrometre-scale footprints remains challenging. Here we develop a miniaturized diffractive neural network (DN2s) integrated on the distal facet of a MMF for the direct all-optical image transportation through the fibre. The DN2s has a footprint of 150 μm by 150 μm and is fabricated on the facet of a 0.35-m-long MMF using three-dimensional two-photon nanolithography. The fibre-integrated DN2s enables single-shot optical transportation of images with flat phases in real time for a constant configuration of the MMF. The system achieves a minimum image reconstruction feature size of approximately 4.90 μm over a field of view 65 μm by 65 μm when imaging handwritten digits. Transfer learning is also demonstrated by the direct optical transportation of HeLa cell images projected by spatial light modulators, which were not part of the training dataset. The concept and implementation pave the way to the integration of miniaturized DN2s with MMFs for compact photonic systems with unprecedented functionalities.

Access through your institution
Buy or subscribe

This is a preview of subscription content, access via your institution

Access options

Access through your institution

Buy this article

  • Purchase on SpringerLink
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Schematic illustration of the fibre-integrated miniaturized DN2s for all-optical image transportation through a MMF.
Fig. 2: Development process of the fibre-integrated miniaturized DN2s based on the complex TM of the MMF.
Fig. 3: Performance of the fibre-integrated miniaturized DN2s.
Fig. 4: Fabrication of the fibre-integrated miniaturized DN2s.
Fig. 5: Experimental characterization of the fibre-integrated miniaturized DN2s in the straight state.
Fig. 6: Experimental characterization of the fibre-integrated miniaturized DN2s in a bent state (bending angle of 90°).

Similar content being viewed by others

Data availability

All data needed to evaluate the conclusions in this article are available via Code Ocean at https://codeocean.com/capsule/9464191/tree/v1. Additional data related to this paper may be requested from the corresponding authors.

Code availability

The custom code and algorithm used to calculate the DN2s for speckled image reconstruction within this article are available via Code Ocean at https://codeocean.com/capsule/9464191/tree/v1.

References

  1. Mitra, P. P. & Stark, J. B. Nonlinear limits to the information capacity of optical fiber communications. Nature 411, 1027–1030 (2001).

    Article  ADS  Google Scholar 

  2. Čižmár, T. & Dholakia, K. Exploiting multimode waveguides for pure fibre-based imaging. Nat. Commun. 3, 1027 (2012).

    Article  ADS  Google Scholar 

  3. Constable, A., Kim, J., Mervis, J., Zarinetchi, F. & Prentiss, M. Demonstration of a fiber-optical light-force trap. Opt. Lett. 18, 1867–1869 (1993).

    Article  ADS  Google Scholar 

  4. Fu, L. & Gu, M. Fiber‐optic nonlinear optical microscopy and endoscopy. J. Microsc. 226, 195–206 (2007).

    Article  ADS  MathSciNet  Google Scholar 

  5. Pahlevaninezhad, H. et al. Nano-optic endoscope for high-resolution optical coherence tomography in vivo. Nat. Photonics 12, 540–547 (2018).

    Article  ADS  Google Scholar 

  6. Leedumrongwatthanakun, S. et al. Programmable linear quantum networks with a multimode fibre. Nat. Photonics 14, 139–142 (2020).

    Article  ADS  Google Scholar 

  7. Hecht, J. Understanding Fiber Optics (Pearson, 2005).

  8. Ren, H. et al. An achromatic metafiber for focusing and imaging across the entire telecommunication range. Nat. Commun. 13, 4183 (2022).

    Article  ADS  Google Scholar 

  9. Gigan, S. et al. Roadmap on wavefront shaping and deep imaging in complex media. J. Phys. Photon. 4, 042501 (2022).

    Article  Google Scholar 

  10. Wen, Z. et al. Single multimode fibre for in vivo light-field-encoded endoscopic imaging. Nat. Photonics 17, 679–687 (2023).

    Article  ADS  Google Scholar 

  11. Richardson, D. J., Fini, J. M. & Nelson, L. E. Space-division multiplexing in optical fibres. Nat. Photonics 7, 354–362 (2013).

    Article  ADS  Google Scholar 

  12. Cristiani, I. et al. Roadmap on multimode photonics. J. Opt. 24, 083001 (2022).

    Article  ADS  Google Scholar 

  13. Goel, S. et al. Inverse design of high-dimensional quantum optical circuits in a complex medium. Nat. Phys. 20, 232–239 (2024).

    Article  Google Scholar 

  14. Teğin, U., Yıldırım, M., Oğuz, İ., Moser, C. & Psaltis, D. Scalable optical learning operator. Nat. Comput. Sci. 1, 542–549 (2021).

    Article  Google Scholar 

  15. Dietrich, P. I. et al. In situ 3D nanoprinting of free-form coupling elements for hybrid photonic integration. Nat. Photonics 12, 241–247 (2018).

    Article  ADS  Google Scholar 

  16. Cao, H., Mosk, A. P. & Rotter, S. Shaping the propagation of light in complex media. Nat. Phys. 18, 994–1007 (2022).

    Article  ADS  Google Scholar 

  17. Carpenter, J., Eggleton, B. J. & Schröder, J. 110×110 optical mode transfer matrix inversion. Opt. Express 22, 96–101 (2014).

    Article  ADS  Google Scholar 

  18. Mahalati, R. N., Gu, R. Y. & Kahn, J. M. Resolution limits for imaging through multi-mode fiber. Opt. Express 21, 1656–1668 (2013).

    Article  ADS  Google Scholar 

  19. Papadopoulos, I. N., Farahi, S., Moser, C. & Psaltis, D. Focusing and scanning light through a multimode optical fiber using digital phase conjugation. Opt. Express 20, 10583–10590 (2012).

    Article  ADS  Google Scholar 

  20. Turtaev, S. et al. High-fidelity multimode fibre-based endoscopy for deep brain in vivo imaging. Light Sci. Appl. 7, 92 (2018).

    Article  ADS  Google Scholar 

  21. Popoff, S. M. et al. Measuring the transmission matrix in optics: an approach to the study and control of light propagation in disordered media. Phys. Rev. Lett. 104, 100601 (2010).

    Article  ADS  Google Scholar 

  22. Choi, Y. et al. Scanner-free and wide-field endoscopic imaging by using a single multimode optical fiber. Phys. Rev. Lett. 109, 203901 (2012).

    Article  ADS  Google Scholar 

  23. Li, S. et al. Compressively sampling the optical transmission matrix of a multimode fibre. Light Sci. Appl. 10, 88 (2021).

    Article  ADS  Google Scholar 

  24. Li, S., Horsley, S. A., Tyc, T., Čižmár, T. & Phillips, D. B. Memory effect assisted imaging through multimode optical fibres. Nat. Commun. 12, 3751 (2021).

    Article  ADS  Google Scholar 

  25. Būtaitė, U. G. et al. How to build the “optical inverse” of a multimode fibre. Intell Comput. 2022, 9816026 (2022).

    Article  Google Scholar 

  26. Kupianskyi, H., Horsley, S. A. & Phillips, D. B. All-optically untangling light propagation through multimode fibers. Optica 11, 101–112 (2024).

    Article  ADS  Google Scholar 

  27. Lim, J. & Psaltis, D. MaxwellNet: physics-driven deep neural network training based on Maxwell’s equations. APL Photonics 7, 011301 (2022).

    Article  ADS  Google Scholar 

  28. Caramazza, P., Moran, O., Murray-Smith, R. & Faccio, D. Transmission of natural scene images through a multimode fibre. Nat. Commun. 10, 2029 (2019).

    Article  ADS  Google Scholar 

  29. Rahmani, B. et al. Multimode optical fiber transmission with a deep learning network. Light Sci. Appl. 7, 69 (2018).

    Article  ADS  Google Scholar 

  30. Borhani, N., Kakkava, E., Moser, C. & Psaltis, D. Learning to see through multimode fibers. Optica 5, 960–966 (2018).

    Article  ADS  Google Scholar 

  31. Liu, Z. et al. All-fiber high-speed image detection enabled by deep learning. Nat. Commun. 13, 1433 (2022).

    Article  ADS  Google Scholar 

  32. Bertolotti, J. & Katz, O. Imaging in complex media. Nat. Phys. 18, 1008–1017 (2022).

    Article  Google Scholar 

  33. Lu, K. et al. Empowering high-dimensional optical fiber communications with integrated photonic processors. Nat. Commun. 15, 3515 (2024).

    Article  ADS  Google Scholar 

  34. Kues, M. et al. Quantum optical microcombs. Nat. Photon 13, 170–179 (2019).

    Article  ADS  Google Scholar 

  35. Zhang, Q., Yu, H., Barbiero, M., Wang, B. & Gu, M. Artificial neural networks enabled by nanophotonics. Light Sci. Appl. 8, 42 (2019).

    Article  ADS  Google Scholar 

  36. Psaltis, D. et al. Holography in artificial neural networks. Nature 343, 325–330 (1990).

    Article  ADS  Google Scholar 

  37. Lin, X. et al. All-optical machine learning using diffractive deep neural networks. Science 361, 1004–1008 (2018).

    Article  ADS  MathSciNet  Google Scholar 

  38. Goi, E., Schoenhardt, S. & Gu, M. Direct retrieval of Zernike-based pupil functions using integrated diffractive deep neural networks. Nat. Commun. 13, 7531 (2022).

    Article  ADS  Google Scholar 

  39. Kulce, O., Mengu, D., Rivenson, Y. & Ozcan, A. All-optical synthesis of an arbitrary linear transformation using diffractive surfaces. Light Sci. Appl. 10, 196 (2021).

    Article  ADS  Google Scholar 

  40. Li, Y., Luo, Y., Mengu, D., Bai, B. & Ozcan, A. Quantitative phase imaging (QPI) through random diffusers using a diffractive optical network. Light Adv. Manuf. 4, 206–221 (2023).

    Google Scholar 

  41. Zhang, Y. et al. Memory-less scattering imaging with ultrafast convolutional optical neural networks. Sci. Adv. 10, eadn2205 (2024).

    Article  Google Scholar 

  42. Rahman, M. S. S. et al. Learning diffractive optical communication around arbitrary opaque occlusions. Nat. Commun. 14, 6830 (2023).

    Article  ADS  Google Scholar 

  43. Yuan, G. H., Rogers, E. T. & Zheludev, N. I. Achromatic super-oscillatory lenses with sub-wavelength focusing. Light Sci. Appl. 6, e17036 (2017).

    Article  ADS  Google Scholar 

  44. Kuchmizhak, A., Gurbatov, S., Nepomniaschii, A., Vitrik, O. & Kulchin, Y. High-quality fiber microaxicons fabricated by a modified chemical etching method for laser focusing and generation of Bessel-like beams. Appl. Opt. 53, 937–943 (2014).

    Article  ADS  Google Scholar 

  45. Plöschner, M., Tyc, T. & Čižmár, T. Seeing through chaos in multimode fibres. Nat. Photonics 9, 529–535 (2015).

    Article  ADS  Google Scholar 

  46. Ebner, F. H. et al. Developments in neuroendoscopy: trial of a miniature rigid endoscope with a multidirectional steerable tip camera in the anatomical lab. Neurosurg. Rev. 35, 45–51 (2012).

    Article  Google Scholar 

  47. Lecun, Y., Bottou, L., Bengio, Y. & Haffner, P. Gradient-based learning applied to document recognition. Proc. IEEE 86, 2278–2324 (1998).

    Article  Google Scholar 

  48. Vasquez-Lopez, S. A. et al. Subcellular spatial resolution achieved for deep-brain imaging in vivo using a minimally invasive multimode fiber. Light Sci. Appl. 7, 110 (2018).

    Article  ADS  Google Scholar 

  49. Porte, X. et al. Direct (3+1)D laser writing of graded-index optical elements. Optica 8, 1281–1287 (2021).

    Article  ADS  Google Scholar 

  50. Hu, X., Zhao, J., Antonio-Lopez, J. E., Correa, R. A. & Schülzgen, A. Unsupervised full-color cellular image reconstruction through disordered optical fiber. Light Sci. Appl. 12, 125 (2023).

    Article  ADS  Google Scholar 

Download references

Acknowledgements

This work was supported by the funding from the National Key Research and Development Program of China (grant no. 2021YFB2802000, Q.Z.), the National Key Research and Development Program of China (grant no. 2022YFB2804301, H.L.), the Science and Technology Commission of Shanghai Municipality (grant no. 21DZ1100500, M.G.), the Shanghai Municipal Science and Technology Major Project, the Shanghai Frontiers Science Center Program (grant no. 2021-2025 No. 20, M.G.), the National Natural Science Foundation of China (grant no. 61975123, Q.Z.; 62305219, S.L.; 62205208, H.Y.), the Shanghai Natural Science Foundation (grant no. 23ZR1443200, S.L.), the China Postdoctoral Science Foundation (2022M712138, S.L.; 2021M702192, H.Y.) and the Shanghai Super Postdoctoral Incentive Scheme (5B22904002, S.L.; 5B22904006, H.Y.).

Author information

Authors and Affiliations

  1. School of Artificial Intelligence Science and Technology, University of Shanghai for Science and Technology, Shanghai, China

    Haoyi Yu, Zihao Huang, Simone Lamon, Jian Lin, Qi Wang, Haitao Luan, Min Gu & Qiming Zhang

  2. Institute of Photonic Chips, University of Shanghai for Science and Technology, Shanghai, China

    Haoyi Yu, Zihao Huang, Simone Lamon, Jian Lin, Qi Wang, Haitao Luan, Min Gu & Qiming Zhang

  3. Institute for Biomedical Materials and Devices, Faculty of Science, University of Technology Sydney, Sydney, New South Wales, Australia

    Baokai Wang

  4. State Key Laboratory of Digital Medical Engineering, School of Biological Science and Medical Engineering, Southeast University, Nanjing, China

    Haibo Ding

  5. Zhangjiang Laboratory, Shanghai, China

    Min Gu

Authors
  1. Haoyi Yu
    View author publications

    Search author on:PubMed Google Scholar

  2. Zihao Huang
    View author publications

    Search author on:PubMed Google Scholar

  3. Simone Lamon
    View author publications

    Search author on:PubMed Google Scholar

  4. Baokai Wang
    View author publications

    Search author on:PubMed Google Scholar

  5. Haibo Ding
    View author publications

    Search author on:PubMed Google Scholar

  6. Jian Lin
    View author publications

    Search author on:PubMed Google Scholar

  7. Qi Wang
    View author publications

    Search author on:PubMed Google Scholar

  8. Haitao Luan
    View author publications

    Search author on:PubMed Google Scholar

  9. Min Gu
    View author publications

    Search author on:PubMed Google Scholar

  10. Qiming Zhang
    View author publications

    Search author on:PubMed Google Scholar

Contributions

H.Y., Q.Z. and M.G. conceived the concept. M.G. and Q.Z. supervised the project. Z.H. and Q.W. contributed to the implementation of the programming of the diffractive neural networks. B.W. contributed to offering professional advice in the fabrication experiments while he was employed by the University of Shanghai for Science and Technology. H.D., from South East China University, S.L. and J.L. contributed in offering generous suggestions in the characterization and fabrication systems. H.Y. and Q.W. carried out the numerical and experimental characterization of the device. All authors participated in discussions and contributed to the writing of the paper.

Corresponding authors

Correspondence to Min Gu or Qiming Zhang.

Ethics declarations

Competing interests

The authors declare no competing interests.

Peer review

Peer review information

Nature Photonics thanks Juergen Czarske, Qing Yang and the other, anonymous, reviewer(s) for their contribution to the peer review of this work.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Supplementary Information

Supplementary Notes 1–11, Figs. 1–24 and Table 1.

Supplementary Video 1

Theoretical modelling of the scanning spots reconstruction results.

Supplementary Video 2

Experimental results of the scanning spots reconstruction at the distal facet.

Supplementary Video 3

DN2s and U-net reconstruction results of natural scenes scrambled by the multimode fibre.

Supplementary Video 4

Experimental video showing the fibre integration of DN2s using the two-photon nanolithographic technique.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yu, H., Huang, Z., Lamon, S. et al. All-optical image transportation through a multimode fibre using a miniaturized diffractive neural network on the distal facet. Nat. Photon. 19, 486–493 (2025). https://doi.org/10.1038/s41566-025-01621-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue date:

  • DOI: https://doi.org/10.1038/s41566-025-01621-4

This article is cited by

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing