Abstract
Multiplexed fluorescence imaging provides valuable biological insights from the cellular to the tissue level but remains limited in live-mammal studies by the lack of a fluorescent palette capable of overcoming photon scattering and autofluorescence noise for real-time, multiplexed in vivo imaging. Here we present a fluorophore palette engineered from erbium(III)-phthalocyanine complexes, termed the lanthanide rainbow (Lanbow), which offers tunable near-infrared absorption and a unified 1,530 nm emission with brightness surpassing that of existing molecular dyes. Lanbow uses excitation-encoded and efficient single-band detection in the 1,500–1,900 nm shortwave infrared subregion, where tissue scattering and autofluorescence are minimized, enabling up to nine-colour imaging in deep tissues. We also demonstrate fluorescence-guided surgery featuring multiparametric anatomical identification and functional assessment, with deep-learning networks automating real-time analysis for intraoperative guidance. This study establishes a transformative platform for real-time, highly multiplexed imaging in live mammals.
This is a preview of subscription content, access via your institution
Access options
Access Nature and 54 other Nature Portfolio journals
Get Nature+, our best-value online-access subscription
$32.99 / 30 days
cancel any time
Subscribe to this journal
Receive 12 print issues and online access
$259.00 per year
only $21.58 per issue
Buy this article
- Purchase on SpringerLink
- Instant access to the full article PDF.
USD 39.95
Prices may be subject to local taxes which are calculated during checkout
Similar content being viewed by others
Data availability
All training and testing data involved in the experiments can be downloaded from the Zenodo links at https://doi.org/10.5281/zenodo.13622692 or from the Huggingface links at https://huggingface.co/datasets/Orange066/Unmixing_TrainValTestData. The experimental data supporting the findings of this study are available within the article and Supplementary Information. Source data are provided with this paper.
Code availability
The PyTorch code of our EndmemberNet, together with trained models, as well as some example images for inference are publicly available at https://github.com/Orange066/EndmemberNet; you can also download it from the Zenodo links at https://doi.org/10.5281/zenodo.13622929 or from the Huggingface links at https://huggingface.co/Orange066/Unmixing_Model. Furthermore, we also provide an offline demo for EndmemberNet at http://fdudml.cn:6789. This newly built interactive software platform facilitates users to freely and easily use our trained model. We shared all models on Zenodo at https://doi.org/10.5281/zenodo.13622929 and Huggingface at https://huggingface.co/Orange066/Unmixing_Model. Finally, we have shared our software on Zenodo at https://doi.org/10.5281/zenodo.13622929 and Huggingface at https://huggingface.co/Orange066/Unmixing_Model. The software provides tools for real-time fluorescence image acquisition and unmixing, offering an efficient solution for researchers in related fields. In addition, it includes examples of unmixing as demonstrated in this paper. We used the Pycharm software for code development.
References
Livet, J. et al. Transgenic strategies for combinatorial expression of fluorescent proteins in the nervous system. Nature 450, 56–62 (2007).
Grimm, J. B. et al. A general method to fine-tune fluorophores for live-cell and in vivo imaging. Nat. Methods 14, 987–994 (2017).
Benaissa, H. et al. Engineering of a fluorescent chemogenetic reporter with tunable color for advanced live-cell imaging. Nat. Commun. 12, 6989 (2021).
Bruns, O. T. et al. Next-generation in vivo optical imaging with short-wave infrared quantum dots. Nat. Biomed. Eng 1, 0056 (2017).
Hellweg, L. et al. A general method for the development of multicolor biosensors with large dynamic ranges. Nat. Chem. Biol. 19, 1147–1157 (2023).
Sharma, S., Boyer, J. & Teyton, L. A practitioner’s view of spectral flow cytometry. Nat. Methods 21, 740–743 (2024).
Kwok, S. J. J. et al. High-dimensional multi-pass flow cytometry via spectrally encoded cellular barcoding. Nat. Biomed. Eng 8, 310–324 (2024).
Valm, A. M. et al. Applying systems-level spectral imaging and analysis to reveal the organelle interactome. Nature 546, 162–167 (2017).
Berry, S. et al. Analysis of multispectral imaging with the AstroPath platform informs efficacy of PD-1 blockade. Science 372, eaba2609 (2021).
Manifold, B., Men, S., Hu, R. & Fu, D. A versatile deep learning architecture for classification and label-free prediction of hyperspectral images. Nat. Mach. Intell. 3, 306–315 (2021).
Kim, J. et al. Unsupervised discovery of tissue architecture in multiplexed imaging. Nat. Methods 19, 1653–1661 (2022).
van Ineveld, R. L. et al. Revealing the spatio-phenotypic patterning of cells in healthy and tumor tissues with mLSR-3D and STAPL-3D. Nat. Biotechnol. 39, 1239–1245 (2021).
Seo, J. et al. PICASSO allows ultra-multiplexed fluorescence imaging of spatially overlapping proteins without reference spectra measurements. Nat. Commun. 13, 2475 (2022).
Valm, A. M., Oldenbourg, R. & Borisy, G. G. Multiplexed spectral Imaging of 120 different fluorescent labels. PLoS ONE 11, e0158495 (2016).
Luthman, A. S., Dumitru, S., Quiros-Gonzalez, I., Joseph, J. & Bohndiek, S. E. Fluorescence hyperspectral imaging (fHSI) using a spectrally resolved detector array. J. Biophoton. 10, 840–853 (2017).
Hedde, P. N., Cinco, R., Malacrida, L., Kamaid, A. & Gratton, E. Phasor-based hyperspectral snapshot microscopy allows fast imaging of live, three-dimensional tissues for biomedical applications. Commun. Biol. 4, 721 (2021).
Scipioni, L., Rossetta, A., Tedeschi, G. & Gratton, E. Phasor S-FLIM: a new paradigm for fast and robust spectral fluorescence lifetime imaging. Nat. Methods 18, 542–550 (2021).
Chen, Y., Wang, S. & Zhang, F. Near-infrared luminescence high-contrast in vivo biomedical imaging. Nat. Rev. Bioeng. 1, 60–78 (2023).
Yang, Y. et al. Fluorescence-amplified nanocrystals in the second near-infrared window for in vivo real-time dynamic multiplexed imaging. Nat. Nanotechnol. 18, 1195–1204 (2023).
Chen, Z.-H. et al. An extended NIR-II superior imaging window from 1500 to 1900 nm for high-resolution in vivo multiplexed imaging based on lanthanide nanocrystals. Angew. Chem. Int. Ed. 62, e202311883 (2023).
Wang, F. et al. Light-sheet microscopy in the near-infrared II window. Nat. Methods 16, 545–552 (2019).
Carr, J. A. et al. Absorption by water increases fluorescence image contrast of biological tissue in the shortwave infrared. Proc. Natl Acad. Sci. USA 115, 9080–9085 (2018).
Meador, W. E. et al. Silicon-RosIndolizine fluorophores with shortwave infrared absorption and emission profiles enable in vivo fluorescence imaging. Nat. Chem. 16, 970–978 (2024).
Yang, Y. et al. Counterion-paired bright heptamethine fluorophores with NIR-II excitation and emission enable multiplexed biomedical imaging. Angew. Chem. Int. Ed. 61, e202117436 (2022).
Yakovliev, A. et al. Hyperspectral multiplexed biological imaging of nanoprobes emitting in the short-wave infrared region. Nanoscale Res. Lett. 14, 243 (2019).
Roxbury, D. et al. Hyperspectral microscopy of near-infrared fluorescence enables 17-chirality carbon nanotube imaging. Sci. Rep. 5, 14167 (2015).
Bünzli, J.-C. G. & Piguet, C. Taking advantage of luminescent lanthanide ions. Chem. Soc. Rev. 34, 1048–1077 (2005).
Mei, M., Wu, B., Wang, S. & Zhang, F. Lanthanide-dye hybrid luminophores for advanced NIR-II bioimaging. Curr. Opin. Chem. Biol. 80, 102469 (2024).
Mack, J. & Kobayashi, N. Low symmetry phthalocyanines and their analogues. Chem. Rev. 111, 281–321 (2011).
Cutrale, F. et al. Hyperspectral phasor analysis enables multiplexed 5D in vivo imaging. Nat. Methods 14, 149–152 (2017).
Pian, Q., Yao, R., Sinsuebphon, N. & Intes, X. Compressive hyperspectral time-resolved wide-field fluorescence lifetime imaging. Nat. Photon. 11, 411–414 (2017).
Fereidouni, F., Bader, A. N. & Gerritsen, H. C. Spectral phasor analysis allows rapid and reliable unmixing of fluorescence microscopy spectral images. Opt. Express 20, 12729–12741 (2012).
Koman, V. B. et al. A wavelength-induced frequency filtering method for fluorescent nanosensors in vivo. Nat. Nanotechnol. 17, 643–652 (2022).
Lauwerends, L. J. et al. Real-time fluorescence imaging in intraoperative decision making for cancer surgery. Lancet Oncol. 22, E186–E195 (2021).
Koch, M., Symvoulidis, P. & Ntziachristos, V. Tackling standardization in fluorescence molecular imaging. Nat. Photon. 12, 505–515 (2018).
Lu, G. et al. Tumour-specific fluorescence-guided surgery for pancreatic cancer using panitumumab-IRDye800CW: a phase 1 single-centre, open-label, single-arm, dose-escalation study. Lancet Gastroenterol. Hepatol. 5, 753–764 (2020).
Zilberstein, B. et al. in Handbook of Robotic Surgery (eds Zequi, S. C. & Ren, H.) 715–723 (Academic, 2025).
Achterberg, F. B. et al. Real-time surgical margin assessment using ICG-fluorescence during laparoscopic and robot-assisted resections of colorectal liver metastases. Ann. Transl. Med. 8, 1448 (2020).
Mc Larney, B. E. et al. A pan-cancer dye for solid-tumour screening, resection and wound monitoring via short-wave and near-infrared fluorescence imaging. Nat. Biomed. Eng. 8, 1092–1108 (2024).
Keller, D. S., Ishizawa, T., Cohen, R. & Chand, M. Indocyanine green fluorescence imaging in colorectal surgery: overview, applications, and future directions. Lancet Gastroenterol. Hepatol. 2, 757–766 (2017).
Faber, R. A. et al. Indocyanine green near-infrared fluorescence bowel perfusion assessment to prevent anastomotic leakage in minimally invasive colorectal surgery (AVOID): a multicentre, randomised, controlled, phase 3 trial. Lancet Gastroenterol. Hepatol. 9, 924–934 (2024).
Ailiani, A. C. et al. Quantitative analysis of peristaltic and segmental motion in vivo in the rat small intestine using dynamic MRI. Magn. Reson. Med. 62, 116–126 (2009).
Vallmitjana, A., Torrado, B. & Gratton, E. Phasor-based image segmentation: machine learning clustering techniques. Biomed. Opt. Express 12, 3410–3422 (2021).
Zhang, Y. et al. Automatic segmentation of intravital fluorescence microscopy images by K-means clustering of FLIM phasors. Opt. Lett. 44, 3928–3931 (2019).
Ronneberger, O., Fischer, P. & Brox, T. U-Net: convolutional networks for biomedical image segmentation. In Medical Image Computing and Computer-Assisted Intervention—MICCAI 2015 (eds Navab, N. et al.) 234–241 (Springer, 2015).
Chen, L.-C. et al. Encoder-decoder with atrous separable convolution for semantic image segmentation. In Proc. European Conference on Computer Vision (eds Ferrari, V. et al.) 801–818 (Springer, 2018).
Wang, T. et al. A hybrid erbium(III)–bacteriochlorin near-infrared probe for multiplexed biomedical imaging. Nat. Mater. 20, 1571–1578 (2021).
Acknowledgements
This work was supported by the National Natural Science Foundation of China (NSFC, grant numbers 22104017, 22088101, 22004018, 22274030, 22522401 and 22474026), New Cornerstone Science Foundation through the XPLORER PRIZE, and the Research Program of Science and Technology Commission of Shanghai Municipality (grant numbers 21142201000, 22JC1400400, 23QA1407100, 25TS1404900 and 20490710600) and Innovation Program of Shanghai Municipal Education Commission (2023ZKZD08).
Author information
Authors and Affiliations
Contributions
S.W. and F.Z. conceived and designed the experiments. L.Z. and B.W. synthesized the fluorophores and conducted the characterization of EP dyes. S.W. and L.Z. built the multispectral imaging system. L.Z., M.M. and Z.H. conducted the in situ colorectal tumour imaging in mice. R.C., L.Z., S.W. and W.T. developed the algorithms and conducted the model training. F.Z., S.W., B.Y., L.Z., R.C. and W.T. analysed the results, figures and Supplementary Information. L.Z., R.C., S.W. and F.Z. wrote the paper. All the authors contributed to discussing and editing the paper.
Corresponding authors
Ethics declarations
Competing interests
The authors declare no competing interests.
Peer review
Peer review information
Nature Photonics thanks Hak Soo Choi and Jun Qian for their contribution to the peer review of this work.
Additional information
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Extended data
Extended Data Fig. 1 Tissue phantom study of multispectral imaging in 850-1000 nm.
a, Spectral phasor transformation for a representative multispectral stack (left panel). Pixel intensities across excitation wavelengths are transformed into spectra (middle panel) and represented as points on a four-quadrant polar plot (right panel). Plot coordinates denote the real (G) and imaginary (S) components of the Fourier transform. Phasor point positions indicate spectral profiles: counterclockwise movement from (-1,0) represents spectral red shift, while distance from the center reflects spectral bandwidth. b, Capillary imaging setup for the tissue phantom study in the 850-1000 nm range. Component 6, 7 and 9: EP737, EP765, and EP772. c. Phasor plots of datasets across varying depths. d, Normalized excitation spectral signatures. Solid lines represent mean values averaged across depths; shaded areas indicate standard deviation (s.d.). e, Normalized RMSE of excitation spectral signatures at depths of 1-3 mm, compared to the reference depth of 0 mm. Each point represents one fluorophore. 1500-1700 nm, n = 9; 850-1000 nm, n = 3. Data are presented as mean values ± s.d. f, Unmixed capillary signals and cross-sectional profiles (bottom) at representative phantom depths of 0 and 3 mm. g, h, Contrast (g) and bleed-through (h) values for each unmixed channel at varying phantom depths. Each point represents one fluorophore. n = 3 for 850-1000 nm. Data are presented as mean values ± s.e.m.
Extended Data Fig. 2 Influence of excitation channel numbers on spectral unmixing performance.
a, Top panel: schematic showing the reduction in the number of excitation channels, each peak representing one excitation wavelength. Bottom panel: Normalized excitation spectral signatures of nine fluorophores (n = 9). Solid lines represent mean values averaged across different depths; shaded areas indicate standard deviation (s.d.). Despite a reduction in spectral resolution with fewer channels, the unique characteristics of each EP fluorophore remain intact due to their narrowband nature. Comp., component. The series numbers 1–9 represent the fluorophores EP673, EP679, EP699, EP720, EP725, EP737, EP765, EB766 and EP772. b, Unmixed capillary signals and cross-sectional profiles at representative phantom depths of 0 and 3 mm. c, Normalized RMSE of excitation spectral signatures at depths of 1-3 mm (n = 9). Data are presented as mean values ± s.d. d,e, Contrast (d) and bleed-through (e) values for each unmixed channel. (n = 9). Data are presented as mean values ± s.e.m.
Extended Data Fig. 3 Biodistribution study of EP fluorophores with different formulations and delivery routes.
a, c, e, Schematics illustrating in vivo biodistribution tracking of: EP772/F127 micelles (a) in a mouse model bearing an in situ CT26 tumor via intravenous injection; EP772/BSA complex (c) via intravenous injection in two normal mice. Images were obtained from two mice to minimize the impact of multiple laparotomies; EP772/F127 micelles (e) within the normal mouse intestinal system via intragastric administration. b, Representative in vivo fluorescence images of EP772/F127 micelles at different time points. Vascular signals diminish by 72 h post-injection, while tumor signals remain detectable. Scale bar, 5 mm. d, Representative in vivo fluorescence images showing EP772/BSA accumulation in mesenteric lymph nodes at different time points post-injection. Fluorescence in mesenteric lymph nodes is detectable at 1 h post-injection and persists up to 72 h. Scale bar, 5 mm. f, Representative in vivo fluorescence images of the mouse intestinal system at different time points post-administration. Fluorescent signals are first detected in the small intestine at 0.5 h post-administration. By 2.5 h, signals in the small intestine diminish, and fluorescence becomes predominantly localized to the cecum at later time points. Scale bar, 10 mm.
Extended Data Fig. 4 Live-mammal multispectral imaging with single fluorophore labelling.
a, Schematic illustrating the administration of fluorophore formulations and corresponding imaging timepoints. Each experiment used one EP fluorophore. Multiple excitation was performed at 671, 690, 730, 760, and 785 nm, with emissions collected in the 1500–1700 nm ranges, respectively. b–d, In vivo fluorescence images (b), phasor plots (c), and excitation spectra (d) of different EP fluorophores. The compact phasor cluster patterns for all fluorophores demonstrates that their excitation spectra remained unaffected by biodistribution. Scale bar: 5 mm.
Extended Data Fig. 5 Ex-vivo histological validation of tumor and lymph node resection.
a,b, Bright-field (a) and fluorescence (b) images of a representative colorectal cancer surgical view. Scale bar, 5 mm. c, H&E staining of resected tumor regions, confirming tumorous areas characterized by high nuclear density. Scale Bar, 200 μm. d, H&E staining of a resected mesenteric lymph nodes (MLNs), showing preserved architecture and no evidence of tumor infiltration. Scale bar, 1000 μm.
Extended Data Fig. 6 Qualitative results of the predicted bounding boxes and segmentation by EndmemberNet using manual processing as ground truth.
Each row displays, from left to right: the input fluorescence image, detection and segmentation outputs from EndmemberNet, and the corresponding manually annotated results. Results were obtained from ten individual mice. Scale Bar, 5 mm.
Extended Data Fig. 7 Comparison of unmixing results between EndmemberNet and manual processing in a test set.
Gray images (left panel) represent the maximum intensity projection of a multispectral datacube. Unmixing results of 9 examples (another example is presented in Fig. 4) in the test set using EndmemberNet are shown in the middle panel. Manual unmixing results as ground truth for the same 9 examples are shown in the right panel. Scale bar: 5 mm.
Extended Data Fig. 8 Performance evaluation of spectral signature prediction and unmixing residuals by EndmemberNet using manual processing as ground truth.
Each row displays, from left to right: Spectral signatures extracted by EndmemberNet compared to those obtained through manual selection; Relative residual maps for 9 examples (another example is presented in Fig. 4) in the test datasets, highlighting the comparison between EndmemberNet results and manual ground truth.
Supplementary information
Supplementary Information (download PDF )
Supplementary Figs. 1–41, Tables 1–5, Methods, Notes 1–6 and references.
Supplementary Video 1 (download MP4 )
Dynamic intraoperative visualization of mouse colorectal tumour surgery. This video showcases the dynamic multiplexing capability of our method, demonstrating real-time intraoperative imaging of a colorectal cancer mouse model. The footage highlights synchronized motion across different channels, including colon-attached tumour metastases, mesenteric lymph nodes and overlying blood vessels. Tumour metastasis and associated vessels exhibit dynamic motion in response to intestinal motility, revealing the high spatial and temporal resolution achieved with our multispectral imaging system. This illustrates the ability of our method to track multiple fluorophore-labelled structures in a complex biological environment with minimal crosstalk.
Supplementary Video 2 (download MP4 )
Real-time online workflow of Lanbow-enabled multispectral imaging with EndmemberNet. This video demonstrates the real-time application of EndmemberNet, our AI-driven pipeline for endmember extraction and spectral unmixing during colorectal tumour surgery. The video shows the automated, real-time display of unmixed channels, processed within a 1 s cycle, providing immediate feedback during surgery. The integration of EndmemberNet allows for seamless identification and segmentation of tumour regions, vessels and other critical structures, significantly enhancing surgical precision and supporting decision-making in real time. The high-speed processing enables rapid adaptation to the evolving surgical scene.
Supplementary Data 1
Crystallographic data of EP737; CCDC reference 2404247.
Source Data for Supplementary Figures (download XLSX )
Statistical source data for Supplementary figures.
Source data
Source Data Fig. 2 (download XLSX )
Statistical source data.
Source Data Fig. 4 (download XLSX )
Statistical source data.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Zhang, L., Cheng, R., He, Z. et al. Excitation-encoded single-emission shortwave infrared lanthanide fluorophore palette for real-time in vivo multispectral imaging. Nat. Photon. 19, 1209–1218 (2025). https://doi.org/10.1038/s41566-025-01736-8
Received:
Accepted:
Published:
Version of record:
Issue date:
DOI: https://doi.org/10.1038/s41566-025-01736-8
