Introduction

The fine regulation of hemodynamics and neural activities in the brain, termed neurovascular coupling, is closely linked to neurophysiological issues, including Parkinson’s disease, Alzheimer’s disease, and amyotrophic lateral sclerosis, et al.1,2,3. In pursuit of elucidating the complex interplay of neurovascular coupling, a variety of imaging tools with specialized insight into specific aspects of brain functions have been extensively applied in animal models4,5,6. However, as each imaging technique has inherent strengths and limitations, to facilitate a comprehensive understanding of neurovascular coupling, integration of optimally complementary modalities is a preferable way7,8,9,10.

In terms of observing neural activities, fluorescence imaging combined with endogenous and exogeneous calcium indicators is irreplaceable. Among all the fluorescence imaging modalities, multiphoton microscopy possesses ultrahigh spatial resolution, large penetration depth, and limited field-of-view (FOV), making it an impeccable tool for the investigation of brain functions in local cerebral cortex regions11,12. Compared with laser-scanning-based multiphoton microscopy, widefield Ca2+ fluorescence microscopy is a better option to study cortex-wide neural activities due to the features of ultrahigh temporal resolution, large FOV, and microscale spatial resolution13,14,15,16,17. Apart from the observation of neural activities via fluorescent emission, imaging techniques are able to employ intrinsic contrasts to visualize hemodynamics. Functional magnetic resonance imaging (fMRI), one of the most widely used modalities to investigate macroscale brain functions, utilizes the blood oxygenation level dependent (BOLD) signal to reveal changes in deoxyhemoglobin. Functional ultrasound imaging has already been successfully used to reflect blood flux via the Doppler effect6,18. Compared with fMRI and functional ultrasound imaging featuring deep penetration and large FOV, label-free optical imaging modalities offer higher spatiotemporal resolutions, richer intrinsic contrasts, and inbuilt compatibility with widefield Ca2+ fluorescence microscopy. Comparing with fMRI and functional ultrasound imaging featuring deep penetration and large FOV, label-free optical imaging modalities such as optical intrinsic signal imaging19,20, optical coherence tomography angiography8,21, optical resolution photoacoustic microscopy (OR-PAM)22,23, fluorescence localization microscopy10,24, and laser speckle contrast imaging (LSCI)25,26 offer higher spatiotemporal resolutions, richer intrinsic contrasts, and inbuilt compatibility with widefield Ca2+ fluorescence microscopy.

Through comprehensive consideration of fundamental principles and imaging performances, widefield Ca2+ fluorescence microscopy, OR-PAM, and LSCI can provide fully complementary information with the same cortex-wide FOV and microscale spatial resolution. However, the major challenge preventing the integration of these three imaging techniques is the mismatched temporal resolution between laser-scanning-based OR-PAM and the other two widefield modalities27. To promote the imaging speed of OR-PAM (Supplementary Table 1), there are three advanced scanning mechanisms, including micro-electro-mechanical system (MEMS) scanning23,28, polygon scanning29, and rotary scanning30. Both MEMS and polygon scanning employ a post-objective confocal configuration, in which the scanners are positioned after the objective to simultaneously deflect both optical and acoustic focuses. Although the post-objective confocal configuration provides an optimal signal-to-noise ratio, it is incompatible with widefield imaging modalities. In comparison, the rotary scanning mechanism utilizes the pre-objective confocal configuration and is fully compatible with widefield microscopies. However, the imaging-speed promotion based on the rotary scanning mechanism for OR-PAM is mainly restricted due to the innate nonuniform sampling pattern and unwanted pulse excitation in the non-vascular area31. Therefore, the key to address these challenges is to optimize the imaging scheme that precisely controls each pulse for highly efficient sampling and vascular excitation.

In this work, we propose a uniform rotary scanning mechanism to eliminate the inherent oversampling issue via ultrafast laser modulation, and further combine it with adaptive vascular excitation and sparse-sampling strategies to achieve ultrafast OR-PAM. Moreover, we develop an infinity-corrected rotary scan engine to make OR-PAM compatible with widefield imaging modalities, and successfully achieve a multimodal microscope termed multiScope. The multimodal imaging platform integrates widefield Ca2+ fluorescence microscopy, OR-PAM, and LSCI with a cortex-wide FOV (Ø 8.6 mm), a single-vessel resolution (10.7 ± 3.1 and 7.1 ± 0.8 μm), and an ultrafast imaging speed (The maximum speed is up to 4 Hz) for simultaneous observation of both neural activities and hemodynamics via endogenous expression biomarkers (GCaMP6f and hemoglobin). We demonstrate the feasibility of the multiScope in behaving mice to study cortex-wide, high spatiotemporal resolution neurovascular coupling using anesthesia and electric shock-induced epilepsy.

Results

Development of ultrafast photoacoustic microscopy

To enable coordinate temporal resolution of OR-PAM with the other two widefield imaging modalities in the multiScope, we developed a pre-objective, infinity-corrected rotary scanning-based OR-PAM with a high spatiotemporal resolution and a large FOV. In rotary scanning-based OR-PAM, the ultrasonic sheet field of the cylindrically focused ultrasound transducer overlaps with the laser scanning spot to achieve an optimal detection of photoacoustic emissions (Fig. 1a). Simultaneous rotation of the laser scanning trace and ultrasound sheet field along the center can cover the entire imaging domain and provide a three-dimensional (3D) depth-coded visualization. The imaging-speed promotion of the rotary scanning-based OR-PAM faces two major challenges: (1) The polar-coordinate scanning mechanism always suffers from inherent nonuniform sampling that lowers the efficiency of pulse usage and easily induces overheat photodamage in the central region (Fig. 1a insert); (2) For laser-scanning-based imaging modalities, there remains a trade-off between the imaging speed and the laser safety for bio-tissues.

Fig. 1: Ultrafast photoacoustic microscopy of the multiScope.
figure 1

a The schematic of the rotary scanning-based OR-PAM that suffers from inherent nonuniform sampling and photodamage in the central region. b Pixel-by-pixel ultrafast laser modulation used in the multiScope to achieve uniform sampling of OR-PAM in the imaging plane. c Spatial sampling density distribution map of the rotary scanning. The x-axis and y-axis represent the normalized position coordinates of the image, and the z-axis is the normalized oversampling factor. In addition, the oversampling factor-encoded MAP images are shown in the x-y plane. d Comparison of cerebral cortex imaging with conventional rotary scanning-based OR-PAM and the multiScope to simulate the thermal damage in the center of the FOV. The laser energy was set to 450 nJ to simulate the damage in the central position for long-term imaging. e The vessel mask derived from the pre-scan image after dilation processing with different sizes to avoid the influence of motion artifacts and vessel deformation during long-term imaging. The laser energy is 180 nJ (Supplementary Movie 1). f The pipeline of sparse sampling and upsampling reconstruction used in the multiScope.

Full size image

To address these challenges, we proposed a compound strategy consisting of ultrafast laser modulation, adaptive photoacoustic excitation, and deep-learning-based sparse sampling reconstruction (Supplementary Movie 1). According to the transformation between Cartesian and polar coordinates, we derived and delivered a modulation sequence to the laser source, disabled the pulse output at the centrally oversampled pixels (Fig. 1b, Supplementary Fig. 1, and Supplementary Note 1.1). Figure 1c shows a comparison of an estimated oversampling factor with and without the ultrafast laser modulation, where the normalized oversampling factor (Normalized log10(sampling points per pixel + 1)) decreases from 1 to 0, and the estimated average power density in the center area decreases by 34% after the ultrafast laser modulation. Figure 1d shows a comparison of photoacoustic imaging with and without laser modulation. Compared to the pulse energy used in the in vivo model-based neurovascular imaging experiments (Supplementary Table 2)29, we set a higher pulse energy (~450 nJ) to simulate thermal damage in the center during long-term imaging. The results show that the laser modulation can eliminate the thermal damage induced by the inherent oversampling of the rotary scanning mechanism during long-term imaging.

To further reduce the average power during imaging, we proposed an adaptive photoacoustic excitation scheme (Supplementary Fig. 1d and Supplementary Note 1.2). We have developed a robust feature-based aligned model32 (Supplementary Note 2.1 and Supplementary Fig. 2) and a U-Net segmentation model33 (Supplementary Note 2.2 and Supplementary Fig. 3) to register the raw data captured from the round-trip rotary scanning and derive the distribution of blood vessels as a target pattern. To avoid the effects of motion artifacts and vessel deformation, an image dilation operation with a proper size was applied in the segmented vessel pattern to minimize modulation failure caused by motion artifacts during in vivo imaging (Fig. 1e). Based on the target pattern, we generated a specific laser modulation sequence for the adaptive photoacoustic excitation scheme that disenables the laser pulses in non-vascular areas. We further validate the effect of the adaptive photoacoustic excitation scheme in a continuous imaging session (Supplementary Movie 1). During high-speed, long-term imaging (>30 min), sustained heat accumulation can still cause potential tissue damage even using a low pulse energy (180 nJ). After immediate detection of tissue damage, we used adaptive photoacoustic excitation to reduce the average power. The damage did not worsen even when imaging was continued for a longer time (34 min).

Moreover, we employed a sparse sampling strategy and a deep-learning algorithm with a transformer-based model to further improve the temporal resolution32 (Fig. 1f, Supplementary Note 2.3, and Supplementary Fig. 4). Compared to the sparsely sampled raw data, the recovered image presents smoother vessel boundaries, fewer sparse sampling artifacts, and a higher signal-to-noise ratio (Supplementary Fig. 5). Our results suggest that the proposed compound strategy can significantly reduce the number of sampling points, avoid heat damage, and maintain the structural and functional information of cerebral vasculatures (Supplementary Note 3 and Supplementary Fig. 6).

Design and performance evaluation of the multiScope for simultaneous Ca2+ and hemodynamic observation

Owing to the successful development of the ultrafast OR-PAM, it is feasible to achieve simultaneous observation of neural and hemodynamic activities across the whole mouse cortex via the integration of widefield Ca2+ fluorescence microscopy, OR-PAM, and LSCI (Fig. 2a). To become compatible with widefield imaging modalities, we developed a pre-objective, infinity-corrected scan engine composed of a galvanometer/MEMS scanner, a scan lens, a tube lens, and an objective (Fig. 2b), rather than a single objective or a telecentric scan lens in conventional OR-PAMs23,29,30. Meanwhile, a stepper motor (SM) with gears was employed to rotate UST, resulting in a rotary scanning of the ultrasound sheet field. These two independent scan engines minimized the water damping force during acoustic scanning and make OR-PAM compatible with widefield imaging modalities (Fig. 2c).

Fig. 2: The schematic and performance of the multiScope.
figure 2

a In the mouse, the functional cortical areas are widely distributed. The multiScope with a FOV of about Ø8.6 mm can encompass the entire cortical neurovascular activity. b The schematic diagram of the multiScope. AL1–AL4, aspheric lens. DM1–DM4, dichroic mirror. F1–F4, filter. DL1–DL2, doublet lens. FC1–FC2, fiber collimator. CL concave lens, Galvo galvanometer scanner, SL scan lens, TL1–TL3 tube lens, CG cover glass, UST ultrasound transducer, P-SMF polarization-maintaining single-mode fiber, Obj1–Obj2 objective lens. c Illustration of the imaging interface for multimodal imaging of the cortex through a cranial window. The mice were head-fixed atop a circular treadmill (150 mm in diameter) with white and dark bars 10 mm apart. d The multiScope features a large FOV over 50 mm2. e The image of a 1951 USAF resolution test target captured by the multiScope. f The lateral resolutions of multiScope-OR-PAM across the FOV are estimated by the FWHMs of 1-μm-diameter Fe3O4 micro-beads. The resolution is 7.1 ± 0.8 μm (mean ± Std, n = 97). g The lateral resolutions of multiScope-Fluo across the FOV are estimated by the FWHMs of 1-μm-diameter fluorescence micro-beads. The resolution is 10.7 ± 3.1 μm (Mean ± Std, n = 121). h–j Representative normalized images of the cortex from widefield Ca2+ fluorescence microscopy, OR-PAM, and LSCI, respectively. Scale bar: 1 mm. Source data of Fig. 2f, g is provided as a Source Data file.

Full size image

In the multiScope-Fluo module, a Kohler illumination light path delivered the excitation beam (470 nm) onto the imaging plane, and the fluorescence signals are collected via the same objective (Fig. 2b). To carry out multiScope-LSCI, a single-mode fiber, a collimator, and a concave lens were used to deliver the laser beam (785 nm) to the imaging plane through a transparent window on the side of the opto-acoustic combiner (Fig. 2c). After passing through the imaging interface and objective, the fluorescent and speckle signals were split into two independent pathways using a dichroic mirror. Each pathway consisted of a bandpass filter, a tube lens, and a scientific complementary metal-oxide semiconductor (sCMOS) camera. Moreover, an additional channel of reflectance data was collected by the sCMOS camera under green illumination (522 nm) for hemodynamic correction of the fluorescence signals19,20.

The optical path and footprint for the multiScope (60 cm × 80 cm × 110 cm, Supplementary Fig. 7) are nearly identical to a conventional upright optical microscope. The simulated FOV and spot size at the image plane were 9 mm and 5.2 μm (Supplementary Fig. 8). We measured and found that the multiScope features a large FOV with a diameter of over 8.6 mm to encompass the whole cortex of mice using a 2× objective (Fig. 2d). The resolution of the multiScope-OR-PAM is better than 8.8 μm, with the 114 lines pairs per millimeter clearly resolved (Fig. 2e). We further measured the full-width at half-maximums (FWHMs) of the imaged photoacoustic and fluorescence 1-μm-diameter micro-beads across the FOV. The derived average spatial resolutions of OR-PAM and multiScope-Fluo are 7.1 ± 0.8 and 10.7 ± 3.1 μm, respectively (Fig. 2f, g). Supplementary Fig. 9 shows the details of the point spread functions estimated by imaging micro-beads, indicating better resolutions at the center (multiScope-OR-PAM: 5.8 μm, multiScope-Fluo: 6.3 μm) and deteriorated resolutions at the edges with aberrations (multiScope-OR-PAM: 9.2 μm, multiScope-Fluo: 15 μm). Supplementary Fig. 10 shows the temporal resolution of the multiScope by monitoring the flowing ink in a micro-channel in real time. The result supports that multiScope-OR-PAM can achieve an imaging rate up to 4 Hz. In contrast, sCMOS cameras capture the widefield images at a 16.6 Hz frame rate. To perform 4-Hz imaging, the relatively large scanning step leads to deteriorated spatial resolution (17.5 μm) due to insufficient spatial sampling (Supplementary Fig. 10). The acquired images were then recovered to a pixel size of 2000 × 2000 by upsampling reconstruction, making the resolution close to the mean spatial resolution (Supplementary Fig. 5)29.

To further validate in vivo imaging capabilities of the multiScope, we performed experiments on transgenic mice with expressing GCaMP6f in cortex (Thy1-GCaMP6f). Figures 2h–j are the representative cerebral cortical images of an awake mouse with cranial window implementation, showing the whole-cortex neural activity (Ca2+), blood vessel (HbT), and cerebral blood flow velocity (\({\mbox{CBF}}v\)), respectively. Supplementary Movie 2 shows movies of neural activities and hemodynamics in an awake mouse at rest, running, and grooming.

Anesthesia-induced neural and hemodynamic responses

To validate the capability of the multiScope to link hemodynamics with neural activities, we first performed a long-term (60 min) imaging of GCaMP6f mice under anesthesia. In the first 5 min, we conducted the multiScope imaging and derived the signals of Ca2+, HbT, and \({\mbox{CBF}}v\) as baselines. Then, we turned on the isoflurane to induce anesthesia, which lasted for about 5 min. Mice were determined to be under deep anesthesia based on the respiratory rate and response to stimulus. For the last 10 min, we turned off the isoflurane and allowed mice to recover. This process was carried out three times to confirm the experimental phenomenon.

After anesthesia ventilation, we observed a rapid decline in the fluorescence signal and an increase in HbT and \({\mbox{CBF}}v\) (Fig. 3a–c). Due to the strong absorption of hemoglobin in the band of visible light, there is significant hemodynamic crosstalk20 in the fluorescence images (Supplementary Fig. 11a). To remove the influence of HbT variations on fluorescence signals, we performed hemodynamic calibration based on the 522 nm reflectance data (Supplementary Fig. 11b, c). Figure 3d–f shows the decreased neural activity, vasodilation, increased HbT, and accelerated blood flow induced by isoflurane anesthesia (Supplementary Movie 3). Post the hemodynamic calibration, motion correction, and multimodal image registration, we mapped the imaging data to a known regional division of the mouse brain (Fig. 3g)34. We quantified individual neural and hemodynamic response amplitudes as the average signals within the selected 26 Atlas region of interests (ROIs). Data were filtered using low- (0.005–0.02 Hz) and high- (0.04–0.1 Hz) bandpass filters9,19 as shown in Fig. 3h.

Fig. 3: Simultaneous multimodal imaging during anesthesia and recovery.
figure 3

a Averaged fluorescence signal (after hemodynamic calibration). b Averaged HbT signal during isoflurane-induced anesthesia. c Averaged \({\mbox{CBF}}v\) signal. The blue bars indicate the time for anesthesia. The solid lines in (a–c) are the mean data, and the shaded curves represent the range of signal fluctuations (from the minimum to the maximum). d–f Multimodal images show the changes in neural activity (\(\Delta {{{\rm{Fluo}}}}/{{{\rm{Fluo}}}}\)), total hemoglobin (\(\Delta {{{\rm{HbT}}}}/{{{\rm{HbT}}}}\)), and cerebral blood flow velocity (\({{{\rm{CBF}}}}v\)) distributed in the cortex. The fluorescence images before and after image-to-image hemodynamic calibration are shown in Supplementary Fig. 11. Scale bar: 1 mm. g Atlas regions of interest (ROIs) within the cortex used in this study. A total of 26 ROIs are distributed across both hemispheres. h Two bands (Low: 0.005–0.02 Hz; High: 0.04–0.1 Hz) of multimodal temporal traces of neural activity (\(\Delta {{{\rm{Fluo}}}}/{{{\rm{Fluo}}}}\)), total hemoglobin (\(\Delta {{{\rm{HbT}}}}/{{{\rm{HbT}}}}\)), and cerebral blood flow velocity (\(\Delta {{{\rm{CBF}}}}v/{{{\rm{CBF}}}}v\)). i–k Connectivity matrices in awake (i), anesthetized (j), and recovery (k) process across modalities of low (upper triangular part) and high (lower triangular part) components in multimodal time traces derived from 26 ROIs. Source data of Fig. 3a–c, h are provided as a Source Data file.

Full size image

To quantify the relationship between different modalities, we further computed connectivity matrixes within and between Ca2+, HbT, and \({\mbox{CBF}}v\) data (Fig. 3i–k). The upper triangle of the matrix represents the correlation of low-frequency signals (0.005–0.02 Hz), while the lower triangle represents the correlation of high-frequency signals (0.04–0.1 Hz). When the mouse switched from awake to anesthesia, the correlation between high-frequency Ca2+, HbT, and \({\mbox{CBF}}v\) traces declined obviously, while that of the low-frequency traces did not change significantly (Fig. 3i, j). As the mouse recovered, the correlation between high-frequency multimodal signals became stronger again (Fig. 3k). Since the calibration method cannot completely remove the hemodynamic crosstalk, the low-frequency signals contain the overall hemodynamic fluctuation and present a stronger consistency in the multimodal signals19 (Fig. 3h). Moreover, the correlation of signals within the same modalities also changed, especially in multiScope-OR-PAM. These results suggest that the use of anesthesia alters neural activities, hemodynamics, as well as the intrinsic neurovascular coupling1,2. The distinct changes in the correlation between signals of different frequency bands may result from various dominant factors. The high-frequency components reflect the rapid fluctuations of neural activities and the hemodynamics it regulates via neurovascular coupling, while low-frequency signals represent relatively slow physiological changes, which are also classified as non-neuronal origin factors35. Based on the results, it is clear that the effects of anesthesia on neurovascular coupling are complex since anesthetics can affect neurons, vascular endothelial cells, hemodynamics, and have long-term effects36,37,38. Many anesthetics, including isoflurane, can induce vasodilation. Previous studies have supported that this is related to the changes in Ca2+ levels in vascular mural cells39. In addition, compared to the central sagittal sinus and its branch veins, smooth muscle cells and pericytes outside the arterioles and capillaries can induce vasodilation and vasoconstriction under the control of neurovascular coupling, which may also be affected by anesthesia40,41. Our results validate the technical capabilities of the multiScope based on previously reported physiological responses to anesthesia. More rigorously designed experiments can facilitate the understanding of the neurovascular coupling of wakefulness and anesthesia42,43,44.

Epilepsy-evoked neural and hemodynamic responses

The monitoring of anesthesia-induced response mainly focused on cortex-wide, slow changes in neural and hemodynamic responses. We further demonstrated the high spatiotemporal resolving capability of the multiScope by imaging epileptic mice. We performed three sets of electric shocks to anesthetized mice (ten times in each group, 0.5 mA, 10 s apart) to induce mild epilepsy, and conducted imaging for 5 min post the electric shocks. Figure 4a shows a co-registered neural and hemodynamic imaging result. For each imaging modality, to minimize the distortion caused by brain pulsation, we performed an improved affine registration between each frame45. Then, we carried out multimodal registration via vascular branch points to register Ca2+, HbT, and \({\mbox{CBF}}v\) images based on local weighted mean transformation (Supplementary Fig. 12a–c). To reduce the number of pixels, we adopted a two-dimensional superpixel function using a simple linear iterative clustering (k-means) algorithm to group the pixels into regions (Supplementary Fig. 12d–g)46. Compared with the method using a Gaussian filter and node averaging in our previous studies45, this method can maintain the cerebral vascular structure and reduce the processing complexity. The compressed data were finally converted into a set of time series signals, where the horizontal axis represented the time stamp of each frame, and the vertical axis represented the index of each superpixel. Fig. 4b–d show the Ca2+, HbT, and \({\mbox{CBF}}v\) time series, respectively. We observed corresponding neural and hemodynamic responses (#1-#7) across all three imaging modalities (Supplementary Movie 4). Compared to the fluorescence imaging and LSCI, the sensitivity to the HbT fluctuations (\(\Delta {{{\rm{HbT}}}}/{{{\rm{HbT}}}}\)) appears noisier, mainly due to the lower sensitivity of acoustic detection in OR-PAM. More sensitive ultrasound transducers and methods can further promote the application of the multiScope in quantitative functional imaging of weak responses (e.g., imaging during mouse spatial exploration behavior and responses to visual stimulation).

Fig. 4: Stimulus-evoked responses of Ca2+, HbT, and CBFv signals.
figure 4

a Co-registered multimodal image captured by the multiScope. The fluorescence signal is visualized in gray, and the hemodynamics (Normalized [HbT] × Normalized [\({\mbox{CBF}}\)v]) is visualized in red. Scale bar: 500  μm. b–d Time series of fluctuation in neural activity (\(\triangle {{{\rm{Fluo}}}}/{{{\rm{Fluo}}}}\)), total hemoglobin (\(\triangle {{{\rm{HbT}}}}/{{{\rm{HbT}}}}\)), and cerebral blood flow velocity (\(\triangle {{{\rm{CBF}}}}v/{{{\rm{CBF}}}}v\)) illustrate that HbT and \({\mbox{CBF}}\)v changes following the rises in calcium signals. e–g Enlarged multimodal images of the ROI in a. Scale bar: 200 μm. h Averaged time traces of fluorescence, HbT, and \({\mbox{CBF}}\)v signal changes of ROIs 1 to 3. The solid lines are the mean data, and the shaded curves represent the range of signal fluctuations (from the minimum to the maximum). i The cross-correlation coefficient map between the average fluorescence signal change and the HbT time traces of each pixel in ROI 1. j The cross-correlation coefficient map between the average fluorescence signal change and the \({{{\rm{CBF}}}}v\) time traces of each pixel in ROI 1. k The cross-correlation coefficient map between the average \({{{\rm{CBF}}}}v\) signal change and the HbT time traces of each pixel in ROI 1. Scale bar: 200 μm. Source data of Fig. 4h are provided as a Source Data file.

Full size image

In addition to the whole-cortex imaging, the multiScope is able to provide detailed information, benefiting from the high spatial and temporal resolution. Three ROIs indicated in Fig. 4a were selected to further analyze the relationship between neural activities and hemodynamics (Fig. 4e–g). We quantified individual neural responses based on the average amplitude within the selected region (Fig. 4h, blue traces). For HbT and \({\mbox{CBF}}v\), we calculated average signal within specific blood vessels (ROI 1 to 3, v1, dotted line in Fig. 4e–g) as hemodynamic traces (Fig. 4h, green and red traces for HbT and \({\mbox{CBF}}v\), respectively). We observed a sharp rise in calcium signal due to neural firing, followed by increased HbT and \({\mbox{CBF}}v\) (Supplementary Movie 4)19,47. Then, we derived the time of peak for each individual response across modalities and calculated the time delays between different modalities. The mean delay between hemodynamic changes and neural responses was around 1 s (Supplementary Fig. 13), which is consistent with previous studies47. In addition, some blood vessels showed an increased \({\mbox{CBF}}v\) but a decreased HbT following the rise in the calcium signal (ROI 1, v2, Supplementary Fig. 14). The opposite changes in v1 and v2 vessels may indicate that they are distributed in different anatomical structures. For example, the surface artery and dural arteriole have distinct mechanisms to regulate dilation. Dural arterioles probably have slow flow, while arterioles on the brain surface that dilate robustly and have high flow. In awake mice, the dural vessel diameters are dynamically regulated not only by pathological states but also by the behaviors of mice48. Such different variations in adjacent small vessels may be neglected in low-resolution functional imaging, which highlights the superiority of the multiScope.

To compare the fluctuation trends of different vessels, we performed correlation analyses for small vessels on a single-pixel level. Figure 4i, j presents the correlation of HbT and \({\mbox{CBF}}v\) traces with the average calcium signal in ROI 1, respectively. The maps highlight the correlation between hemodynamic parameters and neural activities, where the HbT trace is strongly correlated with the Ca2+ trace (ROI 1-v2, HbT). It is worth noting that the maps show a predominantly negative correlation between vascular pixels and neural activity due to the delay between hemodynamic changes and neural responses (Supplementary Fig. 14), as there is a strong positive correlation between the time-consistent CBFv and HbT signals (Fig. 4k). The HbT change of v1 vessel is positively correlated with its \({\mbox{CBF}}v\), while the change in v2 vessel is inversely correlated. The correlation maps highlight two different populations of vascular pixels. There are positive correlations within the same group and negative correlations between groups. This correlation may be related to the propagation of neuroregulatory signals along the connected vasculatures that regulate vasomotion of blood vessels, and further results in HbT changes49,50. Supplementary Fig. 15 shows a similar correlation analysis of ROIs 2 and 3. The results highlight stronger correlations within hemodynamics (HbT and \({\mbox{CBF}}v\), Supplementary Fig. 15c, f) compared to the correlation between neural activities and hemodynamics (Supplementary Fig. 15a–e). The HbT signals in the center and edge of the blood vessel present different correlations with the Ca2+ signal (Supplementary Fig. 15a, v3), indicating the mutation of HbT at the edge of the blood vessel due to neuro-regulated vasoconstriction. Besides, the results show the high sensitivity and resolution of photoacoustic imaging to HbT, enabling the multiScope to capture functional change in tiny blood vessels in contrast to multimodal optical imaging using only widefield modalities. This is further emphasized by the results of the correlation of HbT traces (Supplementary Fig. 16) and \({\mbox{CBF}}v\) traces (Supplementary Fig. 17), respectively. Compared to the \({\mbox{CBF}}v\) signals captured by LSCI, OR-PAM deriving HbT signals present a stronger single-vessel consistency.

Discussion

In summary, we present a multimodal microscope integrating widefield Ca2+ fluorescence microscopy, OR-PAM, and LSCI. The multiScope enables simultaneous imaging of neural activities and hemodynamics in behaving mice with a single-vessel spatial resolution, a whole-cortex FOV, and an ultrafast imaging speed. To facilitate the integration of different modalities, we developed an ultrafast, infinite-corrected, rotary scan engine for OR-PAM. Through engineering a compound imaging scheme including ultrafast laser modulation, adaptive photoacoustic excitation, and sparse-sampling, we have addressed the issue of inherent nonuniform sampling in rotary scanning-based OR-PAM, eliminated photodamage and photobleaching, and significantly promoted the temporal resolution of OR-PAM. Among them, the adaptive photoacoustic excitation scheme in this work reduces the average energy of light by blocking the laser output pulses in the non-vascular region. By using a completely inertia-free scanner, such as acousto-optic deflectors51,52, this scheme can further improve the temporal resolution of OR-PAM.

By integrating widefield Ca2+ fluorescence microscopy, OR-PAM, and LSCI, the multiScope allows us to simultaneously measure neural activities, vasomotion-induced HbT change, and blood flow velocity with an endogenous contrast agent (note that the term “endogenous” in this work refers to GCaMP expressed in a transgenic mouse model and hemoglobin in vasculature). All the imaging modalities of the multiScope provide fully complementary neurovascular coupling information with their respective strengths. Compared to other studies of multimodal recording/imaging for neural activities and hemodynamics, the value of the multiScope lies in providing simultaneous measurement of neural activities and hemodynamics at the same mesoscopic scale with a large FOV (whole-cortex), a high spatial resolution (10.7 ± 3.1 and 7.1 ± 0.8 μm), and a fast-imaging speed (up to 4 Hz). The laser pulse repetition rate (PRR) and high-speed scanning mechanisms are the major bottlenecks for high-speed OR-PAM imaging. In existing high-speed OR-PAMs, reducing the sampling pixels through sparse-sampling and image restoration are the most preferred options. The proposed imaging scheme, including uniform rotary scanning mechanism, adaptive vascular excitation, and sparse-sampling can not only bridge the imaging speed gap between laser-scanning OR-PAM and widefield modalities, but also reduce the average laser power, high PRR laser requirement, potential damage in tissue27,53,54, and photon bleaching of fluorescence indicators in non-vessel areas31. To the best of our knowledge, the multiScope-OR-PAM is one of the fastest photoacoustic microscopy systems available for high-resolution, cortex-wide imaging (Supplementary Table 1).

We first demonstrated the superiority of the multiScope by investigating the neurovascular response to the anesthesia challenge. Although a large variety of imaging tools have been developed to investigate anesthesia using different animal models, simultaneous neural and hemodynamic imaging of awake and anesthetized mice with endogenous contrast agents remains valuable55,56. During the awake-anesthetized-awake process, we observed that the correlation of high-frequency multimodal signals firstly decreased and then enhanced. The results were consistent with the dis-coupling and restoration of the neurovascular coupling relationship under anesthesia55. We further derived different changes in the correlation between different frequency bands of signals in the whole cortex during anesthesia, which might distinguish neurovascular coupling-dominated components and non-neural contributions to hemodynamics48,57. It has been known that motion and physiological cycles (e.g., anesthesia) can affect BOLD-fMRI signals, which are typically used to infer neural activities based on neurovascular coupling. A better understanding of neurovascular coupling and non-neural effects on hemodynamic signals could help to interpret human brain activities using fMRI58.

In addition to anesthesia, we employed the multiScope to investigate electric shock-induced epilepsy. With the multiScope, we could rapidly capture single-vessel-level neural and hemodynamic responses throughout the mouse cortex. We observed consistent increased HbT and \({\mbox{CBF}}v\) following sharp rises in calcium signals due to neural firing. We derived two vascular populations negatively correlated with each other in terms of the change in HbT, possibly due to the vasomotion of blood vessels. This finding suggests that different branches of the cerebrovascular system have different, even opposite, dynamic regulations.

Overall, the multiScope can simultaneously image neural activities and hemodynamics with Ca2+ indicator and hemoglobin to generate rich datasets for uncovering the neurovascular coupling in healthy and diseased brains. To cover the mouse cortex meanwhile maintain a single-vessel resolution, we use a low magnification objective in this study. Higher NA objectives allow the multiScope to perform capillary-scale imaging (Supplementary Fig. 18 and Supplementary Table 3). The multiScope adopts an infinite-corrected configuration and is compatible with commercial objectives (with a working distance greater than 15 mm). However, to improve the spatial resolution without scarifying FOV, an objective with a larger diameter is required. Existing studies have validated the feasibility of large FOV, high-resolution widefield, and laser scanning microscopes with customized objectives59,60,61. The multiScope with a capillary resolution and an FOV nearly covering the cortex can contribute to a firmer understanding of the cellular and capillary origins of the neurovascular coupling. In addition, using mice with transgenic indicators such as jGCaMP7 and jGCaMP862,63 will further increase the sensitivity of the multiScope. Different labels of neurons and neurogliocytes can contribute to distinguishing direct and indirect neurovascular regulations64. Furthermore, by combining advanced analysis methods of functional network organization, the multiScope can bridge the gap between rodents and primates, neural activities and hemodynamics, and provide a more comprehensive understanding of brain activities. Moreover, there are three CBFv imaging techniques in the multiScope, including decorrelation estimation based on photoacoustic signals, fluorescence localization imaging using an exogenous contrast agent, and LSCI (Supplementary Note 4, Supplementary Table 4, and Supplementary Movie 5). The multiScope-OR-PAM enables absolute estimation of CBFv by analyzing decorrelation between A-lines without labeling (Supplementary Figs. 19, 20). The current decorrelation estimation method is still limited by imaging speed. Further improvement of the CBFv estimation based on repeated B-scans can accelerate the imaging speed to seconds54. Fluorescence localization imaging can perform high-resolution CBFv measurements beyond the diffraction limit (Supplementary Fig. 20). The requirement of exogenous injected contrast agents10,65, high-performance cameras24, and big data storages makes it more suitable for applications with specific needs for ultrahigh-resolution imaging (Supplementary Figs. 21, 22). In contrast, LSCI allows rapid CBFv estimation in the whole cortical FOV with advantages of label-free, cost-effective data storage, and fast reconstruction speed, which has been widely used in brain studies (Supplementary Fig. 20)25,47,66. In the multiScope, flow imaging with LSCI and OR-PAM offers the complementary advantages of label-free, large FOV, high spatiotemporal resolution, and absolute measurement, while fluorescence localization imaging can provide ultrahigh-resolution CBFv and direction estimation. All of these three methods can provide consistent blood flow measurements to meet the different requirements of applicable scenarios.

Methods

Multimodal microscope design

The multiScope is a combination of a widefield Ca2+ fluorescence microscopy module, OR-PAM module, and a LSCI module. In the Ca2+ imaging module, epifluorescence excitation is performed using a light-emitting diode (LED, M470L5, Thorlabs) mated to the multiScope with an epi-illuminator (AL1-AL4, AL2018-A, LBF254-040-A ×2, LBF254-100-A, Thorlabs). A cage cube is used for mounting a fluorescence filter set consisting of an excitation filter (F1, MF469-35, Thorlabs), a dichroic beam splitter (DM1, MD498, Thorlabs), and a bandpass filter (F2, MF525-39, Thorlabs). To combine the optical paths of the Ca2+ imaging module and OR-PAM module, we insert a dichroic beam splitter (DM2, T525lpxr, Chroma) before the objective (Obj1, Thorlabs TL2X-SAP, 0.1 NA). Together with a tube lens (TL1, AC508-150-A, Thorlabs), the objective images the mouse cortex to the intermediate imaging plane, which is captured by a sCMOS camera (sCMOS1, Andor Neo).

The multiScope-OR-PAM module adopts a fiber laser (IPG Photonics/Spectral Physics) to excite photoacoustic signals. The 532-nm beam passing through a half-wave plate (HWP, AHWP05M-580, Thorlabs) is coupled into a polarization-maintaining single-mode fiber (P-SMF, HB450-SC, Fibercore) via an objective (Obj2, UPLFLN 20×, Olympus) to generate a 559-nm laser beam. The 559-nm beam is then collimated and filtered by a fiber collimator (FC1, F810APC-543, Thorlabs) and a bandpass filter (F3, FBH560-10, Thorlabs). The filtered beam is expanded by a 2×Keplerian beam expander (DL1 and DL2, AC254-030-A and AC254-060-A, Thorlabs). An infinity-corrected rotary scan engine composed of a two-dimensional galvanometer (GVS002, Thorlabs)/electrostatic MEMS (Mirrorcle) scanner, a scan lens (SL, CLS-SL, Thorlabs), and a tube lens (TL2, AC508-200-A, Thorlabs) is employed to achieve rotary scanning. The objective (Obj1) with a long working distance of 56 mm is used to focus the laser beam on the mouse cortex through the imaging interface. The induced photoacoustic wave is reflected by a tilted cover glass into a customized cylindrically focused ultrasonic transducer (UST, Olympus, 15 MHz center frequency, 50% bandwidth). To achieve acoustic rotary scanning, we use a stepper motor with gears to drive the UST rotating around the central axis.

In the multiScope-LSCI module, a near-infrared (NIR) laser diode (LP785-SF20, Thorlabs) delivers the 785-nm beam onto the imaging plane by a single-mode fiber (780HP, Thorlabs). A large beam collimator (FC2, F810APC-780, Thorlabs) and a concave lens (CL, f = −30 mm, OLB-I1-30N, Oeabt) are used to enlarge the laser spot. A dichroic mirror (DM4, DM10-605LP, LBTEK) is used to combine the 785-nm laser beam and 522-nm beam from LED2 (LED-D1-522, Oeabt) to calibrate global hemodynamic crosstalk (Supplementary Fig. 23).

To suppress the direct reflected light from brain surface, the expanded beam (785-nm and 522-nm beam) is obliquely incident onto the imaging plane through the transparent imaging interface (Fig. 1c). The speckle images are collected by another widefield imaging path consist of an objective (Obj1), a tube lens (TL3, AC508-150-A, Thorlabs), and a sCMOS camera (sCMOS2, C13440-20U, Hamamatsu). A dichroic mirror (DM3, DM20-650LP, LBTEK) and a NIR bandpass filter (F4, #87-759, Edmund) are used to split the visible and NIR beams. The multiScope is assembled using the optical cage system and microscope dovetails. The imaging FOV of the widefield Ca2+ fluorescence microscopy and LSCI modules are respectively aligned to that of the OR-PAM module through a set of coarse and fine adjustments (Supplementary Fig. 7). Coarse adjustment is performed by moving the optical axis of the light path to achieve a rough mechanical alignment. Fine adjustment involves a slight movement of the optical axes during the imaging experiment to ensure the image alignment for the three modules. The image registration algorithm will further achieve the alignment of multimodal images. All the lenses we used in this study are commercially available (Supplementary Table 5).

System control and experiment setup

Supplementary Fig. 24 shows the electronics block diagram of the multiScope. We used a multifunctional analog output device (PCI 6733 and BNC 2110, National Instruments) to (1) deliver a laser modulation sequence; (2) control the two-dimensional galvanometer/MEMS scanner; (3) enable/disenable the light source for widefield Ca2+ fluorescence microscopy and LSCI; (4) trigger the sCMOS cameras. A stepper motor controller/driver module (TMCM-6110, Trinamic) was used to control the rotary scanning motor. The signal from the ultrasonic transducer was amplified with a customized 64 dB amplifier and filtered with a 5–30 MHz filter, then delivered to a high-speed digitizer (ATS9350, Alazar Tech) for high-speed data acquisition. We used a multi-channel digitizer (USB3100N, ART Technology) to record the time stamp of multimodal imaging. We head-fixed the mice so that they could run freely on a cylindrical treadmill with a diameter of 15 cm (Supplementary Fig. 7). To perform simultaneous monitoring of the behaving mice, we installed a camera and an infrared light LED on the side front of the treadmill. The speed of movement can be extracted by identifying the stripes on the surface of the treadmill with an interval of 1 cm (Supplementary Fig. 7c).

Animal subjects and preparation

For multimodal imaging experiments, we used C57BL/6J-Tg (Thy1-GCaMP6f GP5.17Dkim/J, the Jackson Laboratory strain #025393) mice. For the test of ultrafast laser modulation, adaptive photoacoustic excitation, and sparse-sampling imaging in OR-PAM, we used C57BL/6J mice (Shanghai Model Organisms). All the mice were housed on a 12-h light/dark cycle with food and water available ad libitum (Animals Center of SUSTech). All surgeries and experiments were performed in accordance with the regulations set by the Southern University of Science and Technology (SUSTech-JY202202010-202302A1 and SUSTech-JY202501025). The transcranial brain window implant was performed using the technique adapted from our previous study67. Mice were first injected 2.5 mg/kg of dexamethasone and 2.5 mg/kg of enrofloxacin for preventing inflammation. Then, the mice were anesthetized in an induction chamber containing 5% isoflurane in air and followed by 1.5% isoflurane in air to maintain anesthesia using an animal anesthesia machine (RWD Life Science). The mice were then fixed to a standard mouse stereotaxic apparatus (Zhongshi Science and Technology). The scalp was excised and sterilized, and the fascia was removed using hydrogen peroxide. A craniotomy over the cortex was performed manually using a high-speed cranial drill (RWD Life Science). The headplate and membrane were sterilized by soaking in 70% ethanol and rinsed with sterile saline. The membrane was placed on the skull, and the surrounding area was dried using cotton-tipped applicators. Super glue (Loctite 401) was applied around the edge of the membrane to make it adhere to the skull. Then the headplate was placed on the membrane. Both super glue and dental cement (Shanghai New Century Dental Material Co., Ltd.) were then applied around the headplate. Postoperative analgesia and anti-inflammatory drugs were administered for 72 h post-surgery. Mice were subsequently allowed to recover for 7 days before the imaging experiments.

Data acquisition

All datasets were collected using a customized acquisition software based on LabVIEW. For long-term imaging (Fig. 3), the sampling point of the multiScope-OR-PAM was set to 2000 × 2000 for better raw image quality. For 4-Hz multiScope-OR-PAM imaging (Fig. 4), the sampling point was set to 500 × 500. The acquired images were recovered to 2000 × 2000 after upsampling reconstruction. The raw data recorded by rotary scanning were first aligned and upsampling reconstructed by deep-learning models. The reconstructed data were then converted to Cartesian coordinates, to form 2D maximal intensity projection images by Hilbert transform and maximum projection in the depth direction, or to form volumetric images by calculating the ultrasonic time of flight. The 2D maximal intensity projection images were further processed by a U-Net model-based vascular enhancement and segmentation algorithm. For Ca2+ fluorescence images, the frame rate was set at 16.6 Hz. We used a pipeline including motion correction, hemodynamic calibration with OR-PAM data, global regression, and filtering. In the multiScope-LSCI, the frame rate was also set at 16.6 Hz, and we used a spatiotemporal contrast reconstruction (spatiotemporal window: 1 × 1 pixel, 10 s for long-term imaging and 2 × 2 pixels, 1 s for high-speed imaging) method to derive a cerebral blood flow velocity map.

For ultrahigh-resolution imaging (Supplementary Fig. 18), we used a 10× Nikon Plan Fluor objective with a 0.3 high NA and a working distance of 16 mm. The FOV and spatial resolution were measured as 1.7 mm in diameter and better than 4 μm, respectively. The specific parameters of data acquisition for in vivo imaging experiments are provided in Supplementary Table 2.

In vivo imaging procedures

For validation of behaving mouse imaging, we used Thy1-GCaMP6f mice with cranial windows described above. Mice were anesthetized briefly in a chamber with 5% isoflurane. Subsequently, mice were fixed on a homemade head bar and coupled to the imaging interface. Multimodal imaging was lasting for a total of 5 min and a stack of the multiScope images was acquired.

For the isoflurane-induced anesthesia experiments, mice were anesthetized first to facilitate fixation to the imaging interface. We waited until the mice were fully awake to start imaging. At the beginning of data acquisition, mice freely breathed. After 5 min, a mixture of air and 4% isoflurane was delivered to rapidly induce anesthesia. After that, a mixture of air and 1.5% isoflurane was adjusted to maintain anesthesia for 5 min. Finally, the isoflurane channel was shut down, and pure air was provided only. The data acquisition continued for ~10 min until the mice were fully awake. This process was repeated three times.

For the electric shock-induced epilepsy experiments, mice were anesthetized first to facilitate fixation to the imaging interface. Mice freely breathed a mixture of air and 1.5% isoflurane to maintain light anesthesia. Hind-paw electric shocks were delivered at 0.5 mA, 0.1 Hz, with a duration of 0.35 ms, and lasted for 10 s. This procedure was then repeated three times. We performed a continuous imaging with the multiScope after electric shocks. Each frame captured by the multiScope was time-stamped at millisecond temporal resolution (sampling rate: 1000 Hz).

For fluorescence localization imaging, 1-μm-diameter fluorescence micro-beads (YMFlo, Yuan Biotech) were diluted in saline buffer and administered intravenously through the tail vein. Mice were anesthetized with a mixture of air and 1.5 % isoflurane during imaging.

Algorithm models used in the multiScope

We used three algorithm models (see Supplementary Note 2 for more details), including (1) an aligned model to eliminate the displacements that occurred across adjacent rows in the raw OR-PAM images; (2) an upsampling model to recover high-resolution (HR) images from low-resolution (LR) ones32; (3) segmentation model to segment blood vessels in OR-PAM images. For the aligned model, we used the built-in MATLAB (MathWorks) functions to complete feature point alignment and image rectification. For the upsampling model, we evaluated the performance on two datasets: a digital downsampling dataset and a real downsampling dataset. The digital downsampling dataset consisted of 100 HR images with a resolution of 2000 × 2000, and the paired LR images were obtained by applying adjacent downsampling to the HR images. The real downsampling dataset consisted of 20 paired real LR images with 500 × 500 pixels and the corresponding HR images with a resolution of 2000 × 2000. The data were then randomly divided into ~80% for training and 20% for testing. For the segmentation model, there were ten pairs of corresponding original and labeled images. Considering that the training data required tedious manual labeling, it was difficult to obtain the training set in large quantities. We further performed data augmentation. The ten pairs of images were randomly cropped into sub-patches with 250 × 250 pixels. For each pair of sub-patches, we performed random geometric transformations such as flips, rotations, and translations to improve the invariance and robustness properties of the trained network. Finally, 64 patches were randomly selected and spliced to create 200 pairs of original and labeled images (https://github.com/MFOIL-Lab/multiScope). The data was then randomly divided into approximately 90% for training and 10% for testing. The upsampling model and segmentation model used in this study were implemented using Python v3.7. The workstation setup included an Intel Core i7-7700K CPU 3.60 GHz, 32 GB system RAM, and a NVIDIA GPU (GeForce GTX 1080Ti), running a Microsoft Windows 10 Professional operating system.

Data analysis

Data analysis was performed using custom-written codes implemented in MATLAB. For Fig. 3, the multimodal image stacks were motion corrected (affine), image registered (fitgeotrans, two-dimensional transformation based on local weighted mean, the blood vessel branches were manually selected as control points, Supplementary Fig. 12), superpixel clustered, and finally derived to time series. In order to calibrate the global crosstalk of fluorescent signals due to the hemodynamic changes induced by anesthesia, we used the interleaved recording 522 nm (green) diffuse-reflectance signals to correct the fluorescence signals (Supplementary Fig. 11)20. In addition, time series of superpixels located in the same ROI within the imaging FOV were averaged to further reduce computational complexity34. A neurovascular coupling analysis method based on a functional connection matrix was adopted, referring to previous studies9,19. Data were then filtered using two frequency-band filters (0.005–0.02 Hz, 0.01–0.1 Hz). The functional connectivity matrices were finally derived by calculating the Pearson correlation coefficient of the filtered time series.

For Fig. 4, multimodal imaging data were used to study the relationship between neural activity and hemodynamics with a pipeline inspired by refs. 49,56. Ca2+ signals of non-vascular pixels in each ROI were derived according to the vessel binarization mask, and then averaged as the neural activity in the corresponding ROI (e.g., ROI 1-Ca2+). The hemodynamic signals of different blood vessels were manually divided based on the ROI binary vessel masks, and the pixels in the cross position of blood vessels were discarded. The signals within a single vessel were averaged as hemodynamic activity in the specific vessel (e.g., ROI, v1-CBFv). Then, we derived the functional connectivity matrices by calculating the Pearson coefficient between the averaged neural/ hemodynamic activity and the pixel-to-pixel time series. Finally, the correlation coefficients between the neural/hemodynamic activity and time series were transformed into correlation maps.

Statics and reproducibility

This work did not use statistical methods to determine sample sizes. Data were collected as a series of preliminary experiment cases (two mice for Fig. 1d, one mouse for Fig. 1e, one mouse for Fig. 2h–j, one mouse for Fig. 3, and one mouse for Fig. 4) to demonstrate the capabilities of the imaging system. Thus, experiments were not randomized and were not blinded.

Reporting summary

Further information on research design is available in the Nature Portfolio Reporting Summary linked to this article.