Abstract
Functional imaging of biological dynamics generally begins with acquiring time-series images, followed by quantifying spatially averaged intensity traces for the regions of interest (ROIs). The conventional pipeline discards a substantial portion of the acquired data when quantifying intensity traces, indicative of inefficient data acquisition. Here we propose a conceptually novel acquisition pipeline that assigns each ROI to a single pixel in the detector, enabling optimally compressed acquisition of the intensity traces. As a proof-of-principle, we implemented a detection module composed of a pair of spatial light modulators and a microlens array, which segments the original image into multiple subimages by introducing distinct angular shifts to each ROI. Each subimage exclusively encodes the signal for the corresponding ROI, facilitating the compressed readout of its intensity trace using a single pixel. This spatial compression allowed for maximizing the temporal information without compromising the spatial information on ROIs. Harnessing our novel approach, we demonstrate the recording of circuit-scale neuronal voltage dynamics at over 5 kHz sampling rate, revealing the individual action potential waveforms within subcellular structures, as well as their submillisecond-scale temporal delays.
Similar content being viewed by others
Introduction
Temporal dynamics of neuronal membrane potentials serve as the fundamental signal in neural computation1. In particular, action potentials encode rich information in their millisecond-scale analogue waveforms, such as the composition of ion channels, electrical excitability, and neural connectivity, which are indispensable for understanding neuronal physiology and pathology2,3,4,5. Patch clamp recording, which measures the electrical properties of individual neurons with an electrode, has long been the gold-standard tool for studying neuronal waveforms due to its unparalleled temporal resolution and signal-to-noise ratio (SNR). However, its experimental throughput is low, typically limited to one neuron at a time, and targeting the fine processes of neurons is often infeasible due to the physical size of the electrode6. These limitations hinder a deeper understanding of neuronal dynamics at a circuit level.
With recent advances in fluorescent voltage indicators, voltage imaging has gained interest since it provides a non-contact optical readout on voltage dynamics with subcellular-scale spatial resolution7,8,9,10,11,12,13,14,15. This unique capability, in principle, allows for kilohertz-scale recording of voltage dynamics in neural processes, as well as cell bodies, in a large neural population9,10,16,17,18,19,20,21. However, simultaneously achieving the subcellular-level spatial and submillisecond-scale temporal resolutions over the wide field-of-view remains challenging due to the technical limitations of optical detectors22. For example, increasing a camera’s acquisition speed is achieved by compromising spatial information, either by decreasing the field-of-view (subarray readout) or reducing the spatial sampling rate (pixel binning). Consequently, achieving kilohertz-scale acquisition speed inevitably accompanies spatial crosstalk among adjacent objects or sacrifices field-of-view, which poses a fundamental challenge in capturing complex dynamics from spatially entangled neural circuits.
For measuring functional neural dynamics, time-series images are generally acquired first and the intensity traces from the selected regions of interest (ROIs) are subsequently analyzed by spatial averaging (Fig. 1a)20,23,24,25,26,27. In this conventional pipeline, the data size of the acquired time-series images is proportional to the number of pixels in the image multiplied by the number of frames. For instance, an acquisition of 20 neurons with 512 × 512 pixels and a 1 kHz frame rate at a 16-bit depth exceeds 30 GB in a minute. In contrast, the data size of the resulting intensity traces for the 20 neurons is merely ~2.4 MB, corresponding to less than 0.01% of the acquired data. In this regard, the conventional pipeline is deemed inefficient, because a large portion of the acquired data, obtained at the cost of compromised temporal resolution, is discarded during the analysis28,29.
a A conventional functional imaging pipeline. Time-series images are acquired first (image block size: M × N × T), and then intensity traces for the selected n ROIs are obtained, resulting in ‘n × T’ data. b The proposed DeMOSAIC imaging pipeline. The ROIs are selected from a digital snapshot image prior to the acquisition. The subimages corresponding to individual ROIs are optically segmented from the original image, and are assigned to each pixel in a detector. Thus, the dimension of the acquired data closely matches with that of the resulting data. c A conceptual illustration of the DeMOSAIC acquisition. The original image is optically relayed to the ROI-based blazed grating pattern, which provides distinct angular modulations for each ROI. Subsequently, the second optical relay including a microlens array (MLA) projects each ROI-based subimage to each pixel in a detector.
To address the limitation of the conventional pipeline, we developed a conceptually novel detection scheme named DeMOSAIC (Diffractive Multisite Optical Segmentation Assisted Image Compression), in which only a single pixel is assigned for each ROI so that the data dimension of the acquisition matches that of the resulting data (Fig. 1b). In the DeMOSAIC pipeline, the ROIs are preselected from a snapshot image prior to time-series acquisition, and the field-of-view is optically divided into a number of subimages configured to contain individual ROIs. Consequently, the number of pixels closely matches the number of ROIs, providing optimal efficiency in data size. We first demonstrate the implementation of the DeMOSAIC system providing optical segmentation by introducing a patterned blazed grating and a microlens array along the detection path (Fig. 1c). Second, we show a proof-of-principle that DeMOSAIC effectively realizes compressive recording at >100 kHz without introducing spatial crosstalk. Lastly, we demonstrate subcellular-scale voltage imaging of neural circuit dynamics at a sampling rate of >5 kHz.
Results
Implementation of the DeMOSAIC system
We implemented the DeMOSAIC system as an add-on to the detector port of an inverted epifluorescence microscope (Fig. 2a, Supplementary Fig. 1 and Supplementary Table 1). The sCMOS camera mounted in the original detection path provided a snapshot image for the user-defined selection of ROIs. Following ROI selection, we redirected the detection beam path to the DeMOSAIC system using a motorized flip mirror, which relayed the original image to the spatial light modulators (SLM) for optical segmentation. In the meanwhile, we generated the grating pattern for the SLMs by assigning one of eight 3-level blazed grating patterns to each ROI, providing directional first-order diffraction at an angle of ~1° (Fig. 2a and Supplementary Fig. 2).
a Schematic optical layout of the DeMOSAIC system. A motorcycle-shaped light pattern was generated by a digital micromirror device (DMD) coupled with an LED and projected onto the mirror at the sample plane (I0). The widefield image is captured by an sCMOS camera for ROI selection (I1) and is subsequently relayed to the SLMs using a motorized flip mirror (FM). Using a pair of SLMs at the conjugate image plane (I2), original image is optically segmented by introducing a distinct angular shift. The diffracted beams are refocused by a microlens array (MLA) to form 3-by-3 subimages (I3), each of which encodes the individual ROI. The subimage plane is optically demagnified and relayed to a detector (I4) in the compressed detection module. OL, objective lens. TL, tube lens. BS, beam splitter. LED, light emitting diode. M: mirror. PBS, polarizing beam splitter; HWP, half-waveplate. b A reflectance widefield image of the motorcycle-shaped pattern. Scalebar, 50 μm. c A grating pattern generated based on the selected ROIs. The 8 types of pattern units are used. d Image of the Fourier plane at MLA plane (î2), with and without pattern displayed on SLM as indicated. Scalebar, 1 mm. e The subimage plane (I3) with and without pattern displayed on SLM. Scalebar, 1 mm.
To impose phase modulation to unpolarized fluorescence emission, we divided the emission signal into s- and p-polarized beams by a polarizing beam splitter (PBS) and introduced a pair of polarization-sensitive reflective SLMs for each polarization (Fig. 2a and Supplementary Fig. 1). In the beam path for the s-polarized beam, we placed a half-wave plate at 45° to rotate the polarization by 90°. To account for the reflection geometry at the PBS, we configured the input grating pattern for the s-polarized beam to be inverted with respect to that for the p-polarized beam. We coregistered the image planes of the SLMs and the original image plane using the space transformation matrices obtained by the point-based multimodal registration algorithm (Supplementary Fig. 3). Consequently, we achieved distinct angular shifts for up to 8 individual ROIs, while the remaining non-selected background remained in the zeroth order.
After angular modulation of the ROIs, we introduced another image relay system comprising a microlens array (MLA) to segregate individual ROIs into subimages. The MLA was fabricated by laser-cutting of nine plano-convex lenses into 2.8-by-2.8 mm2 square-shaped lenslets and assembling them into a 3-by-3 square grid using optical adhesive (Supplementary Fig. 4). This image relay system provided the subimage plane (I3), composed of the central background subimage and the surrounding eight ROI-based subimages. Each subimage exclusively encodes the signal of the selected ROI, thus the intensity trace for each ROI can be compressively recorded by a single pixel.
Having implemented the DeMOSAIC system, we evaluated its feasibility for optical segmentation using a synthetic sample image. Using a digital micromirror device (DMD) coupled to a red light-emitting diode (LED), we projected a motorcycle-shaped light pattern onto a mirror surface positioned at the sample plane. We captured a widefield image with an sCMOS camera (Fig. 2b) and generated the ROI-based grating pattern to segment the motorcycle into eight distinct parts (Fig. 2c). Next, the flip mirror redirected the detection path to the DeMOSAIC beam path. Without the grating pattern, all the signal was in the zeroth order (Fig. 2d, e). After displaying the ROI-based grating pattern on the SLMs, the zeroth-order signal was segmented and redistributed towards the selected first-orders with high precision (Fig. 2d, e, Supplementary Fig. 5, and Supplementary Note 1). Other than the user-defined selection of ROIs, the process of DeMOSAIC acquisition is automated and is completed within several seconds.
Demonstration of DeMOSAIC acquisition at 125 kHz
Given that DeMOSAIC acquisition provides optical segmentation enabling compressed readout, we designed an experiment to demonstrate its high-speed capability (Fig. 3a). First, we created a set of graffiti letters (D, E, M, O, S, A, i, C) and loaded them onto the DMD memory in the sequence of ‘i AM CODES’ and displayed each letter sequentially at the maximum refresh rate of 9.5 kHz (Fig. 3b). The sequential light patterns were projected onto a mirror surface, and the reflected signal was recorded via DeMOSAIC acquisition. This synthetic sample poses a challenge for conventional imaging approaches since it requires for both high spatial resolution to distinguish the intertwined graffiti letters and high temporal resolutions exceeding >19 kHz to avoid temporal aliasing.
a The experimental procedures of the DeMOSAIC acquisition equipped with a PMT array detector. Each letter in the graffiti image was selected, segmented, and assigned to each PMT channel. Scalebar, 200 μm. b Dynamic graffiti image. Using the binary pattern display mode of the DMD, each letter is displayed sequentially at a refresh rate of 9.5 kHz (105 μs for each letter with 105 or 210 μs intervals). c, d Linear unmixing of inter-channel crosstalk. Inter-channel crosstalk by −1st order signals is evident in the raw diffraction efficiency matrix and subimages (left). Linear unmixing algorithm relocates the 0th and −1st order signals to the desired 1st order (right). The dashed boxes indicate the −1st, 0th, and +1st orders for the letter ‘S’, as indicated. Scalebar, 200 μm. e The 125-kHz readout of the dynamic graffiti pattern. The data was acquired by a 9-channel PMT array at 125 kHz and post-processed by applying the linear unmixing. The sequence is decoded as ‘i AM CODES’.
To capture the dynamic signal with a refresh rate of 9.5 kHz, we employed a PMT array connected to a 125-kHz digitizer as a detection module (Supplementary Figs. 1 and 6). After optical segmentation, however, we observed inter-channel crosstalk, which primarily arose from the −1st order diffractions (Fig. 3c, d). The measured diffraction efficiencies for the 8 directions were 47.4 ± 11.9% for the +1st order, 13.6 ± 3.5% for the 0th order, 5.3 ± 0.3% for the −1st order, and the residual 32.8 ± 9.4%% for higher orders. Diffraction towards diagonal directions exhibited lower diffraction efficiency conceivably due to the pixelation of the grating. To resolve this issue, we employed a linear unmixing algorithm, which computationally redistributed the zeroth and −1st order signals to their corresponding +1st order channels (Supplementary Fig. 6 and Supplementary Note 2). After applying the linear unmixing, the signal improved to 63.2 ± 8.8%, while diffraction crosstalk decreased from 5.3% to 0.3%. Consequently, we faithfully decoded the sequence of letters ‘i AM CODES’ by the DeMOSAIC acquisition with a temporal resolution of 8 µs (Fig. 3e).
DeMOSAIC acquisition on neuronal calcium dynamics
We proceeded to apply the DeMOSAIC acquisition for imaging calcium dynamics of live neurons. To accomplish this, we adopted an EMCCD camera due to its high quantum efficiency, minimal read noise, and notably, the capability of analogue pixel binning. This capability, which is not supported by the alternative sCMOS camera, enabled us to customize pixel dimensions for optimized readout while preserving low read noise (Supplementary Fig. 7).
To capture functional dynamics over a large number of neurons, we optimized the acquisition pipeline (Fig. 4). First, we introduced pixel binning along the vertical axis using the ‘asymmetric binning mode’. In our demonstration, we used a subarray readout of 120 × 120 pixels and introduced a vertical pixel binning of 40 pixels. The overall pixel dimension was reduced to 120 × 3 pixels, with each subimage containing 40 × 1 pixels. This barcode-like subimage allowed to accommodate multiple ROIs in a subimage, particularly if the ROIs were sparsely distributed along the horizontal axis in a subimage. This configuration facilitated the simultaneous recording of over 20 neurons with DeMOSAIC acquisition. Second, we observed that the residual zeroth-order signal often saturated the sensor and interfered with the nearby subimages. To address this issue, we introduced patterned excitation, which selectively targets the excitation light to the signal-producing ROIs (Supplementary Fig. 8). The patterned excitation also offered advantages in recovering the zeroth-order signal through the linear-unmixing algorithm and minimizing photodamage to the neurons. Moreover, it enabled robust delineation of the optical segmentation boundary using the watershed algorithm (Supplementary Fig. 8).
a A widefield fluorescence image of cultured neurons loaded with a calcium dye, Cal-630 AM. Scalebar, 50 μm. Representative image of five field-of-views. b A fluorescence image with patterned excitation on the selected ROIs (n = 21 neurons). c Segmented subimages taken by the EMCCD camera (120 × 120 pixels, 16 µm pixel size). Scalebar, 100 μm. d Segmented subimages after vertical pixel binning by 40 pixels. The pixel dimension is compressed to 120 × 3 pixels, resulting in the barcode-like image. The image is displayed after applying the linear unmixing algorithm. e Spontaneous neuronal calcium activity recorded by the DeMOSAIC pipeline (n = 21 neurons).
DeMOSAIC acquisition on neuronal voltage dynamics
Harnessing the optimized DeMOSAIC acquisition pipeline, we observed subcellular-scale voltage dynamics in an intact neural circuit. We stained cultured neurons with a voltage-sensitive dye, BeRST1, providing excellent response kinetics and linearity to membrane potentials14,15. In a widefield fluorescence image, we selected 23 ROIs from three neurons, encompassing their neuronal processes and cell bodies. While recording their functional dynamics at a frame rate of 5.5 kHz, we periodically applied electric field stimulations (pulse width: 1 ms, pulse interval: 200 ms), sufficient to elicit action potentials in most neurons. Following linear unmixing, we extracted the voltage dynamics for each ROI and represented them as ‘dF/F0’ (Fig. 5a).
a Electrical stimulation evoked voltage dynamics recorded from the 23 ROIs at a frame rate of 5.5 kHz. The shaded areas indicate the electric field stimulations. b Stimulus-triggered averaging analysis of the voltage dynamics acquired in (a). The electrical stimulation artifact was measured in the presence of TTX and subtracted from the action potential waveforms. n = 75 spikes for each ROI. c The resulting waveforms for the 23 ROIs. d Quantification of kinetics parameters from the action potential waveform. The action potential waveform and its time derivative are shown. tFWHM, full-width-half-maxima of the action potential. trise, the 10–90% rise time. tfall, the 90–10% fall time. e, Spontaneously spiking neural dynamics acquired at 5.5 kHz. The denoised traces by applying the DeepCAD-RT are overlayed with the raw data. f A representative single-trial recording of action potential waveform. The data is from ROI 9 in (e). The denoised data (magenta) is overlayed with the raw data (light pink). The residual is obtained by subtracting the denoised trace from the raw trace. g Submillisecond time delays of action potentials. The arrows indicate the peaks of action potentials. The ROIs 1 and 4 indicate the soma and the connected neural process, respectively.
To analyze the analogue waveform of the measured action potentials, we temporally aligned the action potentials using the trigger input to the electric field stimulator and averaged the 75 action potentials, referred to as stimulus-triggered averaging. This process increased the SNR by a factor of ~8. Subsequently, to eliminate the direct electric field stimulation artifact, we treated the same neurons with a sodium channel blocker, tetrodotoxin (TTX), suppressing action potential generation, and repeated the same measurement and analysis. The obtained waveform for each ROI, corresponding only to the stimulation artifact, was subtracted from the averaged action potential waveform of the ROI, as reported previously (Fig. 5b)30. Individual subcellular compartments showed distinct kinetic parameters of action potentials, including rise/fall kinetics and spike width (Fig. 5c, d).
We further applied the DeMOSAIC acquisition to quantitatively investigate a spontaneously active functional neural circuit. In-depth understanding of neural circuit function requires a faithful quantitative analysis at the level of individual action potentials, referred to as a single-trial analysis. However, waveform analyses on individual spikes at submillisecond-scale temporal resolution (5.5 kHz) suffered from low SNR, caused primarily by shot noise. To address this issue, we employed a deep learning-based statistically non-biased denoising technique, DeepCAD-RT, to the DeMOSAIC data31,32,33,34,35,36 (Fig. 5e). To our surprise, the denoising algorithm demonstrated highly effective in suppressing the shot noise, resulting in SNR improvement of over 2-fold without introducing significant waveform distortion (Fig. 5f). Since single-trial data faithfully recapitulated the action potential waveforms, we were able to reliably quantify the peaks and widths of individual spikes, as well as their submillisecond-scale delays (Fig. 5g).
Additionally, we demonstrated an all-optical neurophysiology study by integrating patterned optogenetics into the DeMOSAIC system (Supplementary Fig. 9). In cultured neurons transfected with a soma-targeted optogenetic actuator, STChRger2, patterned optogenetic stimuli were sequentially delivered to each target neuron using a programmed DMD input while the DeMOSAIC system acquired voltage dynamics at 4 kHz. The optogenetic stimuli reliably elicited action potentials in targeted neurons and generated post-synaptic potentials in functionally connected non-targeted neurons, providing single-neuron scale information on effective functional connectivity.
In vivo applicability
To test the in vivo feasibility of DeMOSAIC, we first prepared a mouse cranial window model with blood cells sparsely labeled with a fluorescent membrane stain, DiD (Fig. 6a, Supplementary Methods). Under anesthesia, the mouse was secured on a microscope using a custom-built head fixative. We selected 7 ROIs along the pial blood vessels and performed DeMOSAIC acquisition at 4 kHz, mimicking the line-scan mode in laser-scanning microscopy (Fig. 6b-d). To measure the velocity of DiD-labeled blood cells, we extracted the space-time (x-t) kymograph for each ROI and quantified the slope, corresponding to the flow velocity (dx/dt).
a Schematic illustration of the mouse model used for in vivo vessel imaging. The mouse was anesthetized prior to imaging. b Widefield fluorescent image of pial blood vessels and the selected ROIs along the vessels (representative image from four field-of-views acquired in one mouse). c Images captured at the image plane of EMCCD camera. d Traces extracted from seven ROIs over 57,000 frames, recorded at a sampling rate of 4 kHz. e Kymographs depicting the trajectories of labeled cells through each ROI. The numbers indicate the flow rates determined by the slope (v = dx/dt). f Schematic illustration of the mouse model for voltage imaging. The mouse was anesthetized prior to imaging. g Fluorescent images of the cortical GAD2+ neurons under widefield and targeted illuminations. h Comparison of the signal-to-background ratio from the neurons in (g) except neuron 4 since the membrane boundary was not visible. Error bars represent standard deviation. i Images at the plane of the EMCCD camera with and without optical segmentation or vertical pixel binning. j, k In vivo voltage dynamics of GAD2+ neurons recorded at 4 kHz.
For in vivo voltage imaging, we prepared the mouse cranial window model expressing soma-targeted Voltron2 in GAD2+ inhibitory neurons, using the same anesthesia and mounting procedures as in the blood vessel imaging (Fig. 6f). In a widefield fluorescence image, tissue scattering impeded observation of deep-lying cortical neurons, resulting in signal-to-background ratio (SBR) of less than 1. To resolve this issue, we introduced the morphology-based ROI-specific illumination technique referred to as targeted illumination (Fig. 6g, h). Compared to conventional widefield illumination, targeted illumination improved SBR by approximately an order-of-magnitude for layer 2–3 cortical interneurons located at ~100 μm depth. Additionally, we expanded each segmentation ROI by 15% to capture diffuse fluorescence emission. Consequently, DeMOSAIC reliably acquired spontaneous voltage dynamics of multiple neurons in a living mouse brain at 4 kHz, enabling the detection of millisecond-scale latency in action potentials between two neurons with a temporal resolution of 0.25 ms (Fig. 6j, k).
Discussion
We have reported a novel optical segmentation-based detection scheme, DeMOSAIC, which assigns each user-defined ROI to a minimal number of pixels in the detector. By minimizing the number of used pixels in the detector, the DeMOSAIC system provides optimally compressed data acquisition, advantageous especially for high-speed functional imaging on spatially entangled structures. We demonstrate its unique detection capabilities in a synthetic graffiti sample at >100 kHz and in neuronal circuit dynamics at >5 kHz, which is challenging for the conventional detection scheme (Supplementary Table 3). We expect the DeMOSAIC acquisition will open new opportunities for investigating circuit-scale neural dynamics at unprecedented spatiotemporal resolutions.
In conventional ROI-based functional imaging, spatial resolution is required for defining the margins of the ROIs. Since the sensors of conventional cameras are composed of an array of square pixels, a large number of pixels is required to represent the margins and inner areas of the ROIs at high spatial precision. Conceptually, our DeMOSAIC system resolves this inefficiency in spatial representation by optically transforming the square pixel into the shape of the ROI. Employing adaptive optics, the pixel-to-ROI transformation is flexibly configured to match the spatial distribution of the ROIs. This spatial compression constitutes the essence of the DeMOSAIC acquisition for maximizing the temporal information.
The current DeMOSAIC system has room for further technical improvement. First, we used a 3-by-3 MLA for relaying the zeroth and first-order diffractions therefore the higher-order diffractions were discarded, and the collection efficiency was limited to ~70%. Expanding the MLA to a 5-by-5 or larger array would enable the collection of additional higher-order signals (Supplementary Fig. 4). In addition, the electron multiplication gain of the emCCD camera introduces a shot noise penalty by a factor of ~1.41. Replacing the detector with a low-noise alternative such as a SPAD array could mitigate this noise penalty. Combined, these improvements could yield a 2-fold increase in signal-to-noise ratio. Second, the field-of-view of the current DeMOSAIC system is limited by the size of SLM active window (Supplementary Fig. 8). Reducing the relay magnification on the SLM increases the field-of-view but also compromises the resolution for optical segmentation. Introducing the SLM with a higher pixel resolution can be a solution to this problem. The smaller pixel size also accompanies increase in the diffraction angle, thereby improving the maximum collection NA (Supplementary Fig. 2). Third, the current DeMOSAIC system, designed for the near-infrared window, lacks compatibility with visible fluorophores due to the wavelength dependency of the diffraction angle (Supplementary Fig. 1). The introduction of differently sized MLAs mounted in a stepper wheel could facilitate the use of diverse fluorophores across a broader spectral range. Fourth, post hoc computational source separation algorithms27,37,38, which rely on dense spatial information, may not be applicable to the data acquired by DeMOSAIC. Developing a new algorithm tailored for the spatially compressed data requires further investigation. Lastly, the ROIs are preselected for the DeMOSAIC acquisition, so that the sample needs to be stationary during the acquisition, which could be challenging for in vivo imaging. In our in vivo study with head-restrained condition, the motion artifact was nearly negligible, in the order of a micrometer, which was readily mitigated as the neurons stayed stably within the assigned ROIs. To further address this limitation, a closed-loop approach could be introduced in which structural images acquired in real time are used to automatically update ROI positions. These updated ROIs can then be dynamically fed to the SLM or DMD, to continuously track target neurons even under motion artifact. Alternatively, a single ROI may be divided into four subimages to function as a quadrant cell photodetector for tracking the center-of-mass while recording its intensity trace7.
Our DeMOSAIC system has a modular design so that it can be flexibly integrated into a camera port of various widefield microscopes (Supplementary Fig. 8 and Supplementary Table 2). The DeMOSAIC system may be combined with structured illumination microscopy to attain higher resolution in defining the ROIs39,40. For imaging thick biological specimens such as organoids, brain tissues, and whole organisms, optical sectioning techniques such as light-sheet illumination, temporal focusing two-photon excitation, or spinning disk confocal microscopy can be applied39,40,41,42. Alternatively, targeted photostimulation combined with optogenetic actuators will allow precise control of neural activities while recording the circuit-scale neural dynamics12,41,43,44,45,46,47. Moreover, the DeMOSAIC system is also compatible with various detectors for versatile applications. Faster detectors such as a silicon photomultiplier array with a GHz digitizer can be adopted for allowing fluorescence lifetime imaging35,48 or fluorescence correlation spectroscopy36,49,50. We anticipate that our DeMOSAIC system will be broadly adopted to study complex dynamic phenomena.
Methods
A detailed description of all methods is provided in the Supplementary Material.
Reporting summary
Further information on research design is available in the Nature Portfolio Reporting Summary linked to this article.
Data availability
Source data are provided with this paper. Additional data are available from the corresponding authors upon request due to the structure of the DeMOSAIC output. Requests will be fulfilled within 2 weeks. Source data are provided with this paper.
Code availability
All the source codes written in Python and MATLAB are available on a GitHub repository: https://github.com/Neurophotonic/DeMOSAIC.
References
Debanne, D., Bialowas, A. & Rama, S. What are the mechanisms for analogue and digital signalling in the brain?. Nat. Rev. Neurosci. 14, 63–69 (2013).
Stam, C. J. & Straaten, E. C. W. van. The organization of physiological brain networks. Clin. Neurophysiol. 123, 1067–1087 (2012).
Fornito, A., Zalesky, A. & Breakspear, M. The connectomics of brain disorders. Nat. Rev. Neurosci. 16, 159–172 (2015).
Bassett, D. S. & Sporns, O. Network neuroscience. Nat. Neurosci. 20, 353–364 (2017).
Sasaki, T., Matsuki, N. & Ikegaya, Y. Action-potential modulation during axonal conduction. Science 331, 599–601 (2011).
Jayant, K. et al. Targeted intracellular voltage recordings from dendritic spines using quantum-dot-coated nanopipettes. Nat. Nanotechnol. 12, 335–342 (2017).
Tian, H. et al. Video-based pooled screening yields improved far-red genetically encoded voltage indicators. Nat. Methods 20, 1082–1094 (2023).
Abdelfattah, A. S. et al. Bright and photostable chemigenetic indicators for extended in vivo voltage imaging. Science 365, 699–704 (2019).
Piatkevich, K. D. et al. Population imaging of neural activity in awake behaving mice. Nature 574, 413–417 (2019).
Villette, V. et al. Ultrafast two-photon imaging of a high-gain voltage indicator in awake behaving mice. Cell 179, 1590–1608 (2019).
Adam, Y. et al. Voltage imaging and optogenetics reveal behaviour-dependent changes in hippocampal dynamics. Nature 569, 413–417 (2019).
Hochbaum, D. R. et al. All-optical electrophysiology in mammalian neurons using engineered microbial rhodopsins. Nat. Methods 11, 825–833 (2014).
Platisa, J. et al. High-speed low-light in vivo two-photon voltage imaging of large neuronal populations. Nat. Methods 20, 1095–1103 (2023).
Liu, Z. et al. Sustained deep-tissue voltage recording using a fast indicator evolved for two-photon microscopy. Cell 185, 3408–3425 (2022).
Kannan, M. et al. Dual-polarity voltage imaging of the concurrent dynamics of multiple neuron types. Science 378, eabm8797 (2022).
Xiao, S. et al. Large-scale voltage imaging in behaving mice using targeted illumination. iScience 24, 103263 (2021).
Hoppa, M. B., Gouzer, G., Armbruster, M. & Ryan, T. A. Control and plasticity of the presynaptic action potential waveform at small CNS nerve terminals. Neuron 84, 778–789 (2014).
Cohen, C. C. H. et al. Saltatory conduction along myelinated axons involves a periaxonal nanocircuit. Cell 180, 1–12 (2020).
Wu, J. et al. Kilohertz two-photon fluorescence microscopy imaging of neural activity in vivo. Nat. Methods 17, 287–290 (2020).
Weber, T. D., Moya, M. V., Kılıç, K., Mertz, J. & Economo, M. N. High-speed multiplane confocal microscopy for voltage imaging in densely labeled neuronal populations. Nat. Neurosci. 26, 1642–1650 (2023).
Xiao, S., Giblin, J. T., Boas, D. A. & Mertz, J. High-throughput deep tissue two-photon microscopy at kilohertz frame rates. Optica 10, 763 (2023).
Wu, J., Ji, N. & Tsia, K. K. Speed scaling in multiphoton fluorescence microscopy. Nat. Photonics 15, 800–812 (2021).
Reynolds, S. et al. ABLE: An Activity-Based Level Set Segmentation Algorithm for Two-Photon Calcium Imaging Data. eNeuro 4, ENEURO.0012-17.2017 (2017).
Brondi, M. et al. High-accuracy detection of neuronal ensemble activity in two-photon functional microscopy using smart line scanning. Cell Rep. 30, 2567–2580.e6 (2020).
Zhang, Y. et al. Rapid detection of neurons in widefield calcium imaging datasets after training with synthetic data. Nat Methods https://doi.org/10.1038/s41592-023-01838-7 (2023).
Giovannucci, A. et al. CaImAn an open source tool for scalable calcium imaging data analysis. Elife 8, e38173 (2019).
Cai, C. et al. VolPy: Automated and scalable analysis pipelines for voltage imaging datasets. Plos Comput Biol. 17, e1008806 (2021).
Grinvald, A., Ross, W. N. & Farber, I. Simultaneous optical measurements of electrical activity from multiple sites on processes of cultured neurons. Proc. Natl Acad. Sci. 78, 3245–3249 (1981).
Wang, T. et al. Image sensing with multilayer nonlinear optical neural networks. Nat. Photonics 17, 408–415 (2023).
Mena, G. E. et al. Electrical stimulus artifact cancellation and neural spike detection on large multi-electrode arrays. PLOS Computational Biol. 13, e1005842 (2017).
Li, X. et al. Real-time denoising enables high-sensitivity fluorescence time-lapse imaging beyond the shot-noise limit. Nat. Biotechnol. 41, 282–292 (2023).
Li, X. et al. Reinforcing neuron extraction and spike inference in calcium imaging using deep self-supervised denoising. Nat. Methods 18, 1395–1400 (2021).
Lecoq, J. et al. Removing independent noise in systems neuroscience data using DeepInterpolation. Nat. Methods 18, 1401–1408 (2021).
Eom, M. et al. Statistically unbiased prediction enables accurate denoising of voltage imaging data. Nat. Methods 20, 1581–1592 (2023).
Bowman, A. J., Huang, C., Schnitzer, M. J. & Kasevich, M. A. Wide-field fluorescence lifetime imaging of neuron spiking and subthreshold activity in vivo. Science 380, 1270–1275 (2023).
Sankaran, J. et al. Simultaneous spatiotemporal super-resolution and multi-parametric fluorescence microscopy. Nat. Commun. 12, 1748 (2021).
Zhou, P. et al. Efficient and accurate extraction of in vivo calcium signals from microendoscopic video data. Elife 7, e28728 (2018).
Xie, M. E. et al. High-fidelity estimates of spikes and subthreshold waveforms from 1-photon voltage imaging in vivo. Cell Rep. 35, 108954 (2021).
Gustafsson, M. G. L. Nonlinear structured-illumination microscopy: Wide-field fluorescence imaging with theoretically unlimited resolution. Proc. Natl Acad. Sci. 102, 13081–13086 (2005).
Dan, D. et al. DMD-based LED-illumination Super-resolution and optical sectioning microscopy. Sci. Rep. 3, 1116 (2013).
Emiliani, V., Cohen, A. E., Deisseroth, K. & Häusser, M. All-Optical Interrogation of Neural Circuits. J. Neurosci. 35, 13917–13926 (2015).
Oron, D. et al. Scanningless depth-resolved microscopy. Opt. Express 13, 1468–1476 (2005).
Bounds, H. A. et al. All-optical recreation of naturalistic neural activity with a multifunctional transgenic reporter mouse. Cell Rep. 42, 112909 (2023).
Adesnik, H. & Abdeladim, L. Probing neural codes with two-photon holographic optogenetics. Nat. Neurosci. 24, 1356–1366 (2021).
Papagiakoumou, E. et al. Scanless two-photon excitation of channelrhodopsin-2. Nat. Methods 7, 848–854 (2010).
Papagiakoumou, E., Ronzitti, E. & Emiliani, V. Scanless two-photon excitation with temporal focusing. Nat. Methods 17, 571–581 (2020).
Shemesh, O. A. et al. Temporally precise single-cell-resolution optogenetics. Nat. Neurosci. 20, 1796–1806 (2017).
Brinks, D., Klein, A. J. & Cohen, A. E. Two-photon lifetime imaging of voltage indicating proteins as a probe of absolute membrane voltage. Biophys. J. 109, 914–921 (2015).
Singh, A. P. et al. The performance of 2D array detectors for light sheet based fluorescence correlation spectroscopy. Opt. Express 21, 8652–8668 (2013).
Walker, A. S. et al. Optical spike detection and connectivity analysis with a far-red voltage-sensitive fluorophore reveals changes to network connectivity in development and disease. Front. Neurosci. 15, 643859 (2021).
Acknowledgements
This work was supported by the Basic Science Research Program through the National Research Foundation of Korea (NRF), funded by the Ministry of Education, Science, and Technology (RS-2024-00334680, RS-2024-00436783, RS-2024-00466703, 2020R1A5A1018081, all granted to M.C.). We thank Dr. Seok-Hyun Yoon (Harvard Medical School) for supporting initiation of this work.
Author information
Authors and Affiliations
Contributions
S.K. initiated this study. M.C. supervised the research. S.K. implemented the overall DeMOSAIC system. J.Y. and S.K. implemented the optogenetic stimulation and targeted illumination modules. S.K. conducted system calibration and in vitro experiments. J.Y. conducted in vitro optogenetics and in vivo experiments. J.W. and Q.D. contributed to designing the optical system. G.X. and Q.Z. contributed to sample preparation. S.K., G.K., I.K., and J.Y. analyzed the data. Y.L., J.W., and Q.D. contributed to applying the denoising algorithm. H.T., L-Z.F., and A-E.C. contributed to the initial conceptualization, preparation of neurons, and acquisition of preliminary data. S.K., J.Y. and M.C. co-wrote the paper. All authors reviewed and edited the paper.
Corresponding authors
Ethics declarations
Competing interests
M.C. and S.K. are inventors of the patent-pending technology regarding the DeMOSAIC system (US patent application, 18/242,632). No other authors declare competing interests.
Peer review
Peer review information
Nature Communications thanks Martin Thunemann and the other, anonymous, reviewer(s) for their contribution to the peer review of this work. A peer review file is available.
Additional information
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Source data
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/.
About this article
Cite this article
Kim, S., Yoon, J., Ko, G. et al. Optical segmentation-based compressed readout of neuronal voltage dynamics. Nat Commun 16, 7194 (2025). https://doi.org/10.1038/s41467-025-62663-4
Received:
Accepted:
Published:
Version of record:
DOI: https://doi.org/10.1038/s41467-025-62663-4
