Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Article
  • Published:

A subdural CMOS optical device for bidirectional neural interfacing

Nature Electronics volume 7, pages 829–841 (2024)Cite this article

Subjects

Abstract

Optical neurotechnologies use light to interface with neurons and can monitor and manipulate neural activity with high spatial-temporal precision over large cortical areas. There has been considerable progress in miniaturizing microscopes for head-mounted configurations, but existing devices are bulky and their application in humans will require a more non-invasive, fully implantable form factor. Here we report an ultrathin, miniaturized subdural complementary metal–oxide–semiconductor (CMOS) optical device for bidirectional optical stimulation and recording. We use a custom CMOS application-specific integrated circuit that is capable of both fluorescence imaging and optogenetic stimulation, creating a probe with a total thickness of less than 200 µm, which is thin enough to lie entirely within the subdural space of the primate brain. We show that the device can be used for imaging and optical stimulation in a mouse model and can be used to decode reach movement speed in a non-human primate.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on SpringerLink
  • Instant access to the full article PDF.

USD 39.95

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Comparison of displacement factor of miniature microscopes.
Fig. 2: SCOPe device packaging and data acquisition system.
Fig. 3: Co-integrated optical components of SCOPe.
Fig. 4: In vivo imaging of calcium dynamics with electrical stimulus in the mouse model.
Fig. 5: All-optical in vivo imaging of calcium dynamics with optical stimulus in the mouse model.
Fig. 6: In vivo imaging of calcium dynamics in an NHP.

Similar content being viewed by others

Data availability

All imaging data are available at https://github.com/klshepard/scope. All other relevant data are available from the corresponding authors upon reasonable request.

Code availability

Scripts used for image processing, and the decoder model are available at https://github.com/klshepard/scope.

References

  1. Maynard, E. M., Nordhausen, C. T. & Normann, R. A. The Utah Intracortical Electrode Array: a recording structure for potential brain–computer interfaces. Electroencephalogr. Clin. Neurophysiol. 102, 228–239 (1997).

    Article  Google Scholar 

  2. Emiliani, V., Cohen, A. E., Deisseroth, K. & Häusser, M. All-optical interrogation of neural circuits. J. Neurosci. 35, 13917–13926 (2015).

    Article  Google Scholar 

  3. Chen, T.-W. et al. Ultrasensitive fluorescent proteins for imaging neuronal activity. Nature 499, 295–300 (2013).

    Article  Google Scholar 

  4. Jin, L. et al. Single action potentials and subthreshold electrical events imaged in neurons with a fluorescent protein voltage probe. Neuron 75, 779–785 (2012).

    Article  Google Scholar 

  5. Boyden, E. S., Zhang, F., Bamberg, E., Nagel, G. & Deisseroth, K. Millisecond-timescale, genetically targeted optical control of neural activity. Nat. Neurosci. 8, 1263–1268 (2005).

    Article  Google Scholar 

  6. Kleinfeld, D. et al. Can one concurrently record electrical spikes from every neuron in a mammalian brain? Neuron 103, 1005–1015 (2019).

    Article  Google Scholar 

  7. Moreaux, L. C. et al. Integrated neurophotonics: toward dense volumetric interrogation of brain circuit activity—at depth and in real time. Neuron 108, 66–92 (2020).

    Article  Google Scholar 

  8. Ghosh, K. K. et al. Miniaturized integration of a fluorescence microscope. Nat. Methods 8, 871–878 (2011).

    Article  Google Scholar 

  9. Skocek, O. et al. High-speed volumetric imaging of neuronal activity in freely moving rodents. Nat. Methods 15, 429–432 (2018).

    Article  Google Scholar 

  10. Liberti, W. A., Perkins, L. N., Leman, D. P. & Gardner, T. J. An open source, wireless capable miniature microscope system. J. Neural Eng. 14, 045001 (2017).

    Article  Google Scholar 

  11. Rynes, M. L. et al. Miniaturized head-mounted microscope for whole-cortex mesoscale imaging in freely behaving mice. Nat. Methods 18, 417–425 (2021).

    Article  Google Scholar 

  12. Zong, W. et al. Miniature two-photon microscopy for enlarged field-of-view, multi-plane and long-term brain imaging. Nat. Methods 18, 46–49 (2021).

    Article  Google Scholar 

  13. Xue, Y., Davison Ian, G., Boas David, A. & Tian, L. Single-shot 3D wide-field fluorescence imaging with a computational miniature mesoscope. Sci. Adv. 6, eabb7508 (2020).

    Article  Google Scholar 

  14. Yanny, K. et al. Miniscope3D: optimized single-shot miniature 3D fluorescence microscopy. Light. Sci. Appl. 9, 171 (2020).

    Article  Google Scholar 

  15. Adams Jesse, K. et al. Single-frame 3D fluorescence microscopy with ultraminiature lensless FlatScope. Sci. Adv. 3, e1701548 (2017).

    Article  Google Scholar 

  16. Adams, J. K. et al. In vivo lensless microscopy via a phase mask generating diffraction patterns with high-contrast contours. Nat. Biomed. Eng. https://doi.org/10.1038/s41551-022-00851-z (2022).

    Article  Google Scholar 

  17. Choi, J. et al. A 512-pixel, 51-kHz-frame-rate, dual-shank, lens-less, filter-less single-photon avalanche diode CMOS neural imaging probe. IEEE J. Solid-State Circuits 54, 2957–2968 (2019).

    Article  Google Scholar 

  18. Moazeni, S. et al. A mechanically flexible, implantable neural interface for computational imaging and optogenetic stimulation over 5.4×5.4mm2 FOV. IEEE Trans. Biomed. Circuits Syst. https://doi.org/10.1109/TBCAS.2021.3138334 (2021).

  19. Bruschini, C., Homulle, H., Antolovic, I. M., Burri, S. & Charbon, E. Single-photon avalanche diode imagers in biophotonics: review and outlook. Light. Sci. Appl. 8, 87 (2019).

    Article  Google Scholar 

  20. Kawahito, S. Column-parallel ADCs for CMOS image sensors and their FoM-based evaluations. IEICE Trans. Electron. E101-C, 444–456 (2018).

    Article  Google Scholar 

  21. Yokota, T. et al. A conformable imager for biometric authentication and vital sign measurement. Nat. Electron. 3, 113–121 (2020).

    Article  Google Scholar 

  22. Kim, T. H. et al. Long-term optical access to an estimated one million neurons in the live mouse cortex. Cell Rep. 17, 3385–3394 (2016).

    Article  Google Scholar 

  23. Hayakawa, C. K. et al. Perturbation Monte Carlo methods to solve inverse photon migration problems in heterogeneous tissues. Opt. Lett. 26, 1335–1337 (2001).

    Article  Google Scholar 

  24. Johansson, J. Spectroscopic method for determination of the absorption coefficient in brain tissue. J. Biomed. Opt. 15, 057005 (2010).

    Article  Google Scholar 

  25. Wang, L & Wu, H.-I. in Biomedical Optics. 343–346 (John Wiley & Sons, 2009).

  26. Marshel James, H. et al. Cortical layer-specific critical dynamics triggering perception. Science 365, eaaw5202 (2019).

    Article  Google Scholar 

  27. Richard, C., Renaudin, A., Aimez, V. & Charette, P. G. An integrated hybrid interference and absorption filter for fluorescence detection in lab-on-a-chip devices. Lab Chip 9, 1371–1376 (2009).

    Article  Google Scholar 

  28. Boominathan, V., Robinson, J. T., Waller, L. & Veeraraghavan, A. Recent advances in lensless imaging. Optica 9, 1–16 (2022).

    Article  Google Scholar 

  29. Taal, A. J., Lee, C., Choi, J., Hellenkamp, B. & Shepard, K. L. Toward implantable devices for angle-sensitive, lens-less, multifluorescent, single-photon lifetime imaging in the brain using Fabry–Perot and absorptive color filters. Light. Sci. Appl. 11, 24 (2022).

    Article  Google Scholar 

  30. Antipa, N. et al. DiffuserCam: lensless single-exposure 3D imaging. Optica 5, 1–9 (2018).

    Article  Google Scholar 

  31. Boominathan, V., Adams, J. K., Robinson, J. T. & Veeraraghavan, A. PhlatCam: designed phase-mask based thin lensless camera. IEEE Trans. Pattern Anal. Mach. Intell. 42, 1618–1629 (2020).

    Article  Google Scholar 

  32. Uguz, I. & Shepard, K. L. Spatially controlled, bipolar, cortical stimulation with high-capacitance, mechanically flexible subdural surface microelectrode arrays. Sci. Adv. 8, eabq6354 (2022).

    Article  Google Scholar 

  33. Arenkiel, B. R. et al. In vivo light-induced activation of neural circuitry in transgenic mice expressing channelrhodopsin-2. Neuron 54, 205–218 (2007).

    Article  Google Scholar 

  34. Racicot, I. et al. High resolution, wide field optical imaging of macaque visual cortex with a curved detector. J. Neural Eng. 19, 066022 (2022).

    Article  Google Scholar 

  35. Ebina, T. et al. Two-photon imaging of neuronal activity in motor cortex of marmosets during upper-limb movement tasks. Nat. Commun. 9, 1879 (2018).

    Article  Google Scholar 

  36. Mathis, A. et al. DeepLabCut: markerless pose estimation of user-defined body parts with deep learning. Nat. Neurosci. 21, 1281–1289 (2018).

    Article  Google Scholar 

  37. Cisek, P., Crammond, D. J. & Kalaska, J. F. Neural activity in primary motor and dorsal premotor cortex in reaching tasks with the contralateral versus ipsilateral arm. J. Neurophysiol. 89, 922–942 (2003).

    Article  Google Scholar 

  38. Tchoe, Y. et al. Human brain mapping with multithousand-channel PtNRGrids resolves spatiotemporal dynamics. Sci. Transl. Med. 14, eabj1441 (2022).

    Article  Google Scholar 

  39. Chiang, C.-H. et al. Development of a neural interface for high-definition, long-term recording in rodents and nonhuman primates. Sci. Transl. Med. 12, eaay4682 (2020).

    Article  Google Scholar 

  40. Hong, L., Li, H., Yang, H. & Sengupta, K. Fully integrated fluorescence biosensors on-chip employing multi-functional nanoplasmonic optical structures in CMOS. IEEE J. Solid-State Circuits 52, 2388–2406 (2017).

    Article  Google Scholar 

  41. Shemesh, O. A. et al. Precision calcium imaging of dense neural populations via a cell-body-targeted calcium indicator. Neuron 107, 470–486.e411 (2020).

    Article  Google Scholar 

  42. Yu, N. & Capasso, F. Flat optics with designer metasurfaces. Nat. Mater. 13, 139–150 (2014).

    Article  Google Scholar 

  43. Lecoq, J. et al. Removing independent noise in systems neuroscience data using DeepInterpolation. Nat. Methods 18, 1401–1408 (2021).

    Article  Google Scholar 

  44. Simon, C.-J., Sahel, J.-A., Duebel, J., Herlitze, S. & Dalkara, D. Opsins for vision restoration. Biochem. Biophys. Res. Commun. 527, 325–330 (2020).

    Article  Google Scholar 

  45. Qin, H., Lu, J., Jin, W., Chen, X. & Fu, L. Multichannel fiber photometry for mapping axonal terminal activity in a restricted brain region in freely moving mice. Neurophotonics 6, 035011 (2019).

    Article  Google Scholar 

  46. Andersen, R. A., Musallam, S. & Pesaran, B. Selecting the signals for a brain–machine interface. Curr. Opin. Neurobiol. 14, 720–726 (2004).

    Article  Google Scholar 

  47. Silversmith, D. B. et al. Plug-and-play control of a brain–computer interface through neural map stabilization. Nat. Biotechnol. 39, 326–335 (2021).

    Article  Google Scholar 

  48. Yazdan-Shahmorad, A. et al. A large-scale interface for optogenetic stimulation and recording in nonhuman primates. Neuron 89, 927–939 (2016).

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported in part by DARPA under contract N66001-17-C-4012 (K.L.S. and V.A.P.), by the National Science Foundation under grant 1706207 (K.L.S.) and by the NIH under grant NS-103518 (B.P.). We gratefully acknowledge TSMC for chip fabrication and their support in the use of experimental SPAD devices.

Author information

Authors and Affiliations

  1. Department of Electrical Engineering, Columbia University, New York, NY, USA

    Eric H. Pollmann, Heyu Yin, Ilke Uguz, Sajjad Moazeni, Yatin Gilhotra, Victoria Andino-Pavlovsky, Adam Banees, Abhinav Parihar & Kenneth L. Shepard

  2. Center for Neural Science, New York University, New York, NY, USA

    Agrita Dubey, Katie E. Wingel, John S. Choi & Bijan Pesaran

  3. Department of Neurosurgery, University of Pennsylvania, Philadelphia, PA, USA

    Agrita Dubey, Katie E. Wingel & Bijan Pesaran

  4. Department of Electrical Engineering, University of Washington, Seattle, WA, USA

    Sajjad Moazeni

  5. Department of Electrical Engineering, Rice University, Houston, TX, USA

    Vivek Boominathan, Jacob T. Robinson & Ashok Veeraraghavan

  6. The John B. Pierce Laboratory, New Haven, CT, USA

    Vincent A. Pieribone

  7. Department of Cellular and Molecular Physiology, Yale University, New Haven, CT, USA

    Vincent A. Pieribone

  8. Department of Neuroscience, Yale University, New Haven, CT, USA

    Vincent A. Pieribone

  9. Department of Neuroscience, University of Pennsylvania, Philadelphia, PA, USA

    Bijan Pesaran

  10. Department of Bioengineering, University of Pennsylvania, Philadelphia, PA, USA

    Bijan Pesaran

  11. Department of Biomedical Engineering, Columbia University, New York, NY, USA

    Kenneth L. Shepard

  12. Department of Neurological Surgery, Columbia University, New York, NY, USA

    Kenneth L. Shepard

Authors
  1. Eric H. Pollmann
    View author publications

    Search author on:PubMed Google Scholar

  2. Heyu Yin
    View author publications

    Search author on:PubMed Google Scholar

  3. Ilke Uguz
    View author publications

    Search author on:PubMed Google Scholar

  4. Agrita Dubey
    View author publications

    Search author on:PubMed Google Scholar

  5. Katie E. Wingel
    View author publications

    Search author on:PubMed Google Scholar

  6. John S. Choi
    View author publications

    Search author on:PubMed Google Scholar

  7. Sajjad Moazeni
    View author publications

    Search author on:PubMed Google Scholar

  8. Yatin Gilhotra
    View author publications

    Search author on:PubMed Google Scholar

  9. Victoria Andino-Pavlovsky
    View author publications

    Search author on:PubMed Google Scholar

  10. Adam Banees
    View author publications

    Search author on:PubMed Google Scholar

  11. Abhinav Parihar
    View author publications

    Search author on:PubMed Google Scholar

  12. Vivek Boominathan
    View author publications

    Search author on:PubMed Google Scholar

  13. Jacob T. Robinson
    View author publications

    Search author on:PubMed Google Scholar

  14. Ashok Veeraraghavan
    View author publications

    Search author on:PubMed Google Scholar

  15. Vincent A. Pieribone
    View author publications

    Search author on:PubMed Google Scholar

  16. Bijan Pesaran
    View author publications

    Search author on:PubMed Google Scholar

  17. Kenneth L. Shepard
    View author publications

    Search author on:PubMed Google Scholar

Contributions

E.H.P. and K.L.S. conceptualized the study. E.H.P., S.M., Y.G. and A.B. designed the circuits and performed the die thinning. E.H.P., H.Y. and S.M. designed and performed the optical packaging. V.B., J.T.R. and A.V. wrote the computational imaging algorithm. E.H.P., H.Y., I.U. and V.A.-P. performed the in vivo mouse experiments. E.H.P., H.Y., A.P., A.D. and K.E.W. performed the in vivo NHP experiments and data processing. B.P. and J.S.C. performed the surgical procedures in NHP. V.A.P., K.L.S. and B.P. acquired the funding. E.H.P., B.P. and K.L.S. wrote and edited the paper. K.L.S. provided overall supervision and guidance.

Corresponding authors

Correspondence to Bijan Pesaran or Kenneth L. Shepard.

Ethics declarations

Competing interests

The authors declare no competing interests.

Peer review

Peer review information

Nature Electronics thanks Ying Fang, and the other, anonymous, reviewer(s) for their contribution to the peer review of this work.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Supplementary Information (download PDF )

Supplementary Sections 1–14, Figs. 1–32, Tables 1–6, Videos 1–3 and Refs. 1–76.

Supplementary Video 1 (download MP4 )

In vivo mouse experiment—electrical stimulation and optical recording.

Supplementary Video 2 (download MP4 )

In vivo mouse experiment—optical stimulus and optical recording.

Supplementary Video 3 (download MP4 )

In vivo NHP experiment—behavioural stimulus and optical recording.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Pollmann, E.H., Yin, H., Uguz, I. et al. A subdural CMOS optical device for bidirectional neural interfacing. Nat Electron 7, 829–841 (2024). https://doi.org/10.1038/s41928-024-01209-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • Version of record:

  • Issue date:

  • DOI: https://doi.org/10.1038/s41928-024-01209-w

This article is cited by

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing