Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Article
  • Published:

Self-supervised video processing with self-calibration on an analogue computing platform based on a selector-less memristor array

Abstract

Memristor-based platforms could be used to create compact and energy-efficient artificial intelligence (AI) edge-computing systems due to their parallel computation ability in the analogue domain. However, systems based on memristor arrays face challenges implementing real-time AI algorithms with fully on-device learning due to reliability issues, such as low yield, poor uniformity and endurance problems. Here we report an analogue computing platform based on a selector-less analogue memristor array. We use interfacial-type titanium oxide memristors with a gradual oxygen distribution that exhibit high reliability, high linearity, forming-free attribute and self-rectification. Our platform—which consists of a selector-less (one-memristor) 1 K (32 × 32) crossbar array, peripheral circuitry and digital controller—can run AI algorithms in the analogue domain by self-calibration without compensation operations or pretraining. We illustrate the capabilities of the system with real-time video foreground and background separation, achieving an average peak signal-to-noise ratio of 30.49 dB and a structural similarity index measure of 0.81; these values are similar to those of simulations for the ideal case.

This is a preview of subscription content, access via your institution

Access options

Buy this article

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: The developed hardware platform for real-time video processing and electrical characteristics of the analogue memristor array.
Fig. 2: Architecture and experimental characteristics of the analogue computing unit.
Fig. 3: The implemented video processing and experimental data using the developed unit.
Fig. 4: Large-size video processing results under different levels of device variation conditions.

Similar content being viewed by others

Data availability

The data that support the plots within this article and other findings of this study are available from the corresponding author upon reasonable request.

Code availability

The code used for the simulation is available from the corresponding author upon reasonable request.

References

  1. Verbraeken, J. et al. A survey on distributed machine learning. ACM Comput. Surv. 53, 30 (2020).

    MATH  Google Scholar 

  2. Masanet, E., Shehabi, A., Lei, N., Smith, S. & Koomey, J. Recalibrating global data center energy-use estimates. Science 367, 984–986 (2020).

    Article  Google Scholar 

  3. Yu, W. et al. A survey on the edge computing for the Internet of Things. IEEE Access 6, 6900–6919 (2018).

    Article  MATH  Google Scholar 

  4. Li, E., Zeng, L. K., Zhou, Z. & Chen, X. Edge AI: on-demand accelerating deep neural network inference via edge computing. IEEE Trans. Wirel. Commun. 19, 447–457 (2020).

    Article  MATH  Google Scholar 

  5. Take it to the edge. Nat. Electron. 2, 1 (2019).

  6. Satyanarayanan, M., Bahl, P., Cáceres, R. & Davies, N. The case for VM-based cloudlets in mobile computing. IEEE Pervasive Comput. 8, 14–23 (2009).

    Article  Google Scholar 

  7. Cao, K. Y., Liu, Y. F., Meng, G. J. & Sun, Q. M. An overview on edge computing research. IEEE Access 8, 85714–85728 (2020).

    Article  MATH  Google Scholar 

  8. Murshed, M. G. S. et al. Machine learning at the network edge: a survey. ACM Comput. Surv. 54, 170 (2021).

    MATH  Google Scholar 

  9. Yao, P., Gao, B. & Wu, H. Transforming edge hardware with in situ learning features. Nat. Rev. Electr. Eng. 1, 141–142 (2024).

    Article  MATH  Google Scholar 

  10. Zhang, W. Q. et al. Neuro-inspired computing chips. Nat. Electron. 3, 371–382 (2020).

    Article  MATH  Google Scholar 

  11. Song, M. K. et al. Recent advances and future prospects for memristive materials, devices, and systems. ACS Nano 17, 11994–12039 (2023).

    Article  MATH  Google Scholar 

  12. Wang, Z. R. et al. Reinforcement learning with analogue memristor arrays. Nat. Electron. 2, 115–124 (2019).

    Article  MATH  Google Scholar 

  13. Liu, Q. et al. Proc. 2020 IEEE International Solid-State Circuits Conference (ISSCC) (IEEE, 2020).

  14. Hung, J. M. et al. A four-megabit compute-in-memory macro with eight-bit precision based on CMOS and resistive random-access memory for AI edge devices. Nat. Electron. 4, 921–930 (2021).

    Article  MATH  Google Scholar 

  15. Kim, H., Mahmoodi, M. R., Nili, H. & Strukov, D. B. 4K-memristor analog-grade passive crossbar circuit. Nat. Commun. 12, 5198 (2021).

    Article  Google Scholar 

  16. Choi, S. et al. SiGe epitaxial memory for neuromorphic computing with reproducible high performance based on engineered dislocations. Nat. Mater. 17, 335–340 (2018).

    Article  MATH  Google Scholar 

  17. Wu, W. et al. Proc. 2018 IEEE Symposium on VLSI Technology (IEEE, 2018).

  18. Li, C. et al. Efficient and self-adaptive in-situ learning in multilayer memristor neural networks. Nat. Commun. 9, 2385 (2018).

    Article  MATH  Google Scholar 

  19. Park, S. O. et al. Linear conductance update improvement of CMOS-compatible second-order memristors for fast and energy-efficient training of a neural network using a memristor crossbar array. Nanoscale Horiz. 8, 1366–1376 (2023).

    Article  MATH  Google Scholar 

  20. Rao, M. Y. et al. Thousands of conductance levels in memristors integrated on CMOS. Nature 615, 823–829 (2023).

    Article  MATH  Google Scholar 

  21. Zhang, W. B. et al. Edge learning using a fully integrated neuro-inspired memristor chip. Science 381, 1205–1211 (2023).

    Article  MATH  Google Scholar 

  22. Yao, P. et al. Fully hardware-implemented memristor convolutional neural network. Nature 577, 641–646 (2020).

    Article  MATH  Google Scholar 

  23. Wan, W. E. et al. A compute-in-memory chip based on resistive random-access memory. Nature 608, 504–512 (2022).

    Article  MATH  Google Scholar 

  24. Alibart, F., Gao, L. G., Hoskins, B. D. & Strukov, D. B. High precision tuning of state for memristive devices by adaptable variation-tolerant algorithm. Nanotechnology 23, 075201 (2012).

    Article  Google Scholar 

  25. Milo, V. et al. Accurate program/verify schemes of resistive switching memory (RRAM) for in-memory neural network circuits. IEEE Trans. Electron Devices 68, 3832–3837 (2021).

    Article  MATH  Google Scholar 

  26. Yao, P. et al. Face classification using electronic synapses. Nat. Commun. 8, 15199 (2017).

    Article  Google Scholar 

  27. Yan, B. N., Yang, J. H., Wu, Q., Chen, Y. R. & Li, H. Proc. 2017 IEEE/ACM International Conference on Computer-Aided Design (ICCAD) (IEEE, 2017).

  28. Liu, X. X. et al. Harmonica: a framework of heterogeneous computing systems with memristor-based neuromorphic computing accelerators. IEEE Trans. Circuits Syst. I Regul. Pap. 63, 617–628 (2016).

    Article  MATH  Google Scholar 

  29. Xia, Q. F. & Yang, J. J. Memristive crossbar arrays for brain-inspired computing. Nat. Mater. 18, 309–323 (2019).

    Article  MATH  Google Scholar 

  30. Ielmini, D. & Pedretti, G. Device and circuit architectures for in-memory computing. Adv. Intell. Syst. 2, 2000040 (2020).

    Article  MATH  Google Scholar 

  31. Park, S. O., Jeong, H., Park, J., Bae, J. & Choi, S. Experimental demonstration of highly reliable dynamic memristor for artificial neuron and neuromorphic computing. Nat. Commun. 13, 2888 (2022).

    Article  MATH  Google Scholar 

  32. Kim, K. M. et al. Low-power, self-rectifying, and forming-free memristor with an asymmetric programing voltage for a high-sensity crossbar application. Nano Lett. 16, 6724–6732 (2016).

    Article  MATH  Google Scholar 

  33. Li, C. et al. Three-dimensional crossbar arrays of self-rectifying Si/SiO2/Si memristors. Nat. Commun. 8, 15666 (2017).

    Article  MATH  Google Scholar 

  34. Candès, E. J., Li, X., Ma, Y. & Wright, J. Robust principal component analysis? J. ACM 58, 11 (2011).

    Article  MathSciNet  MATH  Google Scholar 

  35. Han, S., Cho, E.-S., Park, I., Shin, K. & Yoon, Y.-G. Efficient neural network approximation of robust PCA for automated analysis of calcium imaging data. In Proc. Medical Image Computing and Computer-Assisted Intervention (MICCAI) 2021: 24th International Conference Part VII 24 (eds de Bruijne, M. et al.) 595–604 (Springer, 2021).

  36. Chen, P. et al. Proc. 2023 IEEE International Solid-State Circuits Conference (ISSCC) (IEEE, 2023).

  37. Yang, Y., Nagarajaiah, S. & Ni, Y. Q. Data compression of very large‐scale structural seismic and typhoon responses by low‐rank representation with matrix reshape. Struct. Control Health Monit. 22, 1119–1131 (2015).

    Article  MATH  Google Scholar 

  38. Ohno, T. et al. Short-term plasticity and long-term potentiation mimicked in single inorganic synapses. Nat. Mater. 10, 591–595 (2011).

    Article  MATH  Google Scholar 

  39. Wang, Z. et al. Memristors with diffusive dynamics as synaptic emulators for neuromorphic computing. Nat. Mater. 16, 101–108 (2017).

    Article  MATH  Google Scholar 

  40. Wan, J. et al. Efficient implementation of synaptic learning rules for neuromorphic computing based on plasma-treated ZnO nanowire memristors. J. Phys. D 53, 055303 (2019).

    Article  MATH  Google Scholar 

  41. Grossi, A. et al. Impact of intercell and intracell variability on forming and switching parameters in RRAM arrays. IEEE Trans. Electron Devices 62, 2502–2509 (2015).

    Article  MATH  Google Scholar 

  42. Abbas, Y. et al. Compliance-free, digital SET and analog RESET synaptic characteristics of sub-tantalum oxide based neuromorphic device. Sci. Rep. 8, 1228 (2018).

    Article  MATH  Google Scholar 

  43. Siegel, S. et al. Trade‐off between data retention and switching speed in resistive switching ReRAM devices. Adv. Electron. Mater. 7, 2000815 (2021).

    Article  MATH  Google Scholar 

  44. Ambrogio, S. et al. Equivalent-accuracy accelerated neural-network training using analogue memory. Nature 558, 60–67 (2018).

    Article  Google Scholar 

  45. Li, C. et al. Analogue signal and image processing with large memristor crossbars. Nat. Electron. 1, 52–59 (2018).

    Article  MATH  Google Scholar 

  46. Cai, F. X. et al. A fully integrated reprogrammable memristor-CMOS system for efficient multiply-accumulate operations. Nat. Electron. 2, 290–299 (2019).

    Article  MATH  Google Scholar 

  47. Wu, H., Judd, P., Zhang, X., Isaev, M. & Micikevicius, P. Integer quantization for deep learning inference: principles and empirical evaluation. Preprint at https://arxiv.org/abs/2004.09602 (2020).

  48. Cho, J., Han, S., Cho, E.-S., Shin, K. & Yoon, Y.-G. Proc. 2023 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) (IEEE, 2023).

Download references

Acknowledgements

This work was supported by the National Research Foundation (NRF) funded by the Korean government (MSIT) (grant nos. RS-2024-00401234 to H.J., S.-O.P., J.B., T.J., Y.C., S.S., T.P. and S.C.; 2022M3I7A2078273 to H.J., S.-O.P., J.B., T.J., Y.C., S.S., H.-J.J., S.P., T.P. and S.C.; 2022M3F3A2A01072851 to H.J., S.-O.P., J.B., T.J., Y.C., S.S., H.-J.J., S.P., T.P., J.O., J.P., D.J., I.K. and S.C.; RS-2023-00209473 to S.H. and Y.-G.Y.; and 2020R1C1C1007464 to H.J., S.-O.P., T.R.K., J.B., T.J., Y.C., S.S., H.-J.J., S.P., T.P. and S.C.) and Institute of Information & Communications Technology Planning & Evaluation (IITP) grant funded by the Korea government (MSIT) (grant no. RS-2023-00216370 to K.K. and K.-H.K.).

Author information

Authors and Affiliations

Authors

Contributions

H.J., S.H., S.-O.P., Y.-G.Y. and S.C. conceived this work. H.J., S.H., Y.-G.Y. and S.C. designed the experiments and overall simulation. H.J., S.-O.P., H.-J.J. and T.J. designed and fabricated the memristor array. H.J., S.-O.P., J.B., S.S. and T.P. performed material analysis. H.J., J.B. and Y.C. conducted electrical measurement of the device. H.J., T.R.K., S.P., J.O., J.P., D.J. and I.K. designed the analogue computing unit. S.H. designed the video processing. H.J. and S.H. designed the real-time platform. H.J., S.H., K.K. and K.-H.K. conducted and improved the video processing implementation. H.J., S.H., S.-O.P., Y.-G.Y. and S.C. prepared the manuscript. All authors contributed to the discussion and analysis of the results regarding the manuscript. Y.-G.Y. and S.C. supervised the study.

Corresponding authors

Correspondence to Young-Gyu Yoon or Shinhyun Choi.

Ethics declarations

Competing interests

The authors declare no competing interests.

Peer review

Peer review information

Nature Electronics thanks Muhammad Khan, Kyusang Lee and Guangdong Zhou for their contribution to the peer review of this work.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Supplementary Information

Supplementary Figs. 1–31, Discussion and Tables 1 and 2.

Supplementary Video 1

Real-time video foreground and background separation using the developed platform with 1 K highly reliable selector-less memristor array.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Jeong, H., Han, S., Park, SO. et al. Self-supervised video processing with self-calibration on an analogue computing platform based on a selector-less memristor array. Nat Electron 8, 168–178 (2025). https://doi.org/10.1038/s41928-024-01318-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue date:

  • DOI: https://doi.org/10.1038/s41928-024-01318-6

This article is cited by

Search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing