Introduction

In the dynamically evolving realm of artificial intelligence, the scaling law of deep learning has entrenched itself as a foundational concept1,2,3,4. This scalability has fueled remarkable performance advancements across diverse domains, including computer vision, natural language processing, and multimodal learning paradigms5,6,7,8,9. Empirically, the model’s accuracy in tackling complex tasks exhibits a positive correlation with its architectural scale. However, larger models suffer from high computational costs10,11, which lead to low model efficiency and pose significant challenges to environmental sustainability. These concerns have prompted the exploration of alternative computing paradigms beyond the von Neumann framework12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33, with physical reservoir computing (PRC) emerging as a transformative paradigm. Unlike conventional digital systems rooted in sequential logic, PRC fundamentally redefines computation by embedding information processing within the analog dynamics of physical substrates, marking a paradigmatic shift from abstract instruction-based operations to physics-driven computation34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63. The essence of PRC lies in its ability to transform input signals through the inherent complex dynamics to produce computationally relevant outputs, leveraging the physical system’s intrinsic computational capabilities17,46,64,65,66.

On the other hand, optogenetics, an experimental method at the intersection of genetics and optics, has dramatically transformed the field of neuroscience67,68,69,70. Optogenetics enables researchers to control and monitor the activity of neurons using light. The optogenetics approach involves the cells that express light-sensitive proteins that can be activated or inhibited by light of specific wavelengths. This allows for the precise control of cellular signaling and has been employed to probe complex biological questions regarding a variety of behaviors and physiological processes, including movement, learning, memory, and sensory processing67,68,69,70. While traditional optogenetics depends on genetic modification to express the light sensitive proteins, the essence is fundamentally about light manipulation, and the scope of this field has expanded to include non-genetic methods71,72,73,74,75,76,77,78. Therefore, the optoelectrical response of the WS2 nanotubes and their interface allow light to manipulate projections inside the WS2 nanotube cluster, translating the principles of optogenetics into the PRC paradigm.

In this context, our research introduces a convergence of very-large-scale (VLS) computational physical reservoir with the feasible control afforded by light. The 3D reservoir, composed of 0D van der Waals interfaces on the order of 108 or 1010 mm-3, not only emulates the synaptic quantity and density of fruit flies’ brains but also integrates photosensitive mimetic optogenetic connections. A key strength of the VLS PRC system is its ability to handle a diverse range of tasks. It effectively supports both monomodal and multimodal endeavors including speech-to-image, colorectal tissue image generation, and blood cell image generation, while maintaining high energy efficiency with 7.0 μW per input. The physical reservoir is designed to respond to light stimuli, effectively translating the principles of optogenetics into the PRC paradigm. By integrating the scale of PRC, the capability for monomodal/multimodal tasks, the exceptional energy efficiency and the mimetic optogenetics, this interdisciplinary research provides a perspective on how optoelectronics, neuroscience, material science, and computational science merge to shape a future pathway for the next-generation neuromorphic computing hardware platform.

Results and discussion

Scaling law and principle of optogenetics

As shown in Fig. 1a, scaled artificial neural network models have set benchmarks in tasks of computer vision, natural language processing, robotics, autonomous driving, scientific discovery, etc. By scaling up the number of parameters, deep learning models advance across various domains and enable more intelligent systems79. The parameter volume of deep learning models has increased significantly in the past few decades, from the representative LeNet (60,000 parameters)80 to GPT-4 (around 1 trillion parameters)81. The increased number of parameters enables enhanced capability to process tasks of increasing complexity, and even the emergence of unpredictable intelligence82,83,84,85, as shown in Supplementary Fig. 1. A similar pattern of the scaling law is found in the neural systems of biological organisms. Figure 1b shows the number of neurons and synapses of Caenorhabditis elegans, fruit flies, mice, orangutans, and humans. Extensive networks of neurons and synapses support more complex functions, such as abstract reasoning, problem-solving, language-using, social interaction, self-awareness, and deep emotional experiences86,87,88,89. The research on biological neural systems has led to the development of optogenetics. As shown in Fig. 1c, by implanting optical sensitive ion channel proteins, the operation of the neural system can be manipulated with an incident light beam. The exposure to light selectively modulates the active or suppressed status of ion channels, thereby altering the membrane potential of neuronal cell membranes. This precise manipulation of neural cells has the capacity to shape and direct the behavior of living organisms, transforming the level of control researchers can exert over neural networks.

Fig. 1: Overview of scaling law, optogenetics, and characterization of WS2 nanotubes.
Fig. 1: Overview of scaling law, optogenetics, and characterization of WS2 nanotubes.
Full size image

a Scaling law of deep learning models with expanding parameter volume. b The quantities of neurons and synapses in various species, from Caenorhabditis elegans to humans. c Schematic diagram of optogenetics. The right panel shows three light-sensitive proteins, with only the central one irradiated. The small red dots within it represent the ion flow process. d SEM images of WS2 nanotubes, showing the hierarchical aggregation of WS2 nanotubes. Scale bars: 100 μm (left) and 25 μm (right). e Micro X-ray CT images of WS2 nanotubes. Scale bars: 100 μm (left) and 100 μm (right). f The images of the WS2 cluster on a substrate with fabricated electrodes. Scale bars: 500 μm (center), 100 μm (right).

WS2 nanotube cluster as the scaled physical reservoir

Drawing inspiration from the scaled biological neural networks and optogenetics, we propose a computational paradigm using a self-organized, submillimeter-long, photoreactive tungsten disulfide (WS2) nanotube cluster as a dynamic physical reservoir. The bottom-up synthesis progress and material characterization of the WS2 nanotube cluster are described in Method and Supplementary Figs. 2–590. The length of the nanotube is measured in the transmission electron microscope (TEM) with a maximum length of 0.5 mm and an average diameter below 50 nm. As shown by the scanning electron microscope (SEM) image in Fig. 1d and Supplementary Fig. 2, the synthesized ultra-long nanotubes aggregate into bundles, each of which is approximately hundreds of micrometers in size. These bundles coalesce into a larger cluster, resulting in a bundle-cluster hierarchical structure. Such a bi-level structure is verified by the micro X-ray computed tomography (CT) with a resolution of 5 μm. Figure 1e shows the shape and the cross-section of the cluster by the micro X-ray CT, showing the sub-structure with interspace and filled bundle blocks. The porosity of the cluster Φ is 32%. The sub-structure aligns with the hierarchical characteristics of large deep learning models and the architecture of brain networks which are adept at capturing distinct levels of features91,92,93. Figure 1f displays the WS₂ nanotube cluster on a substrate with fabricated electrodes, while the complete setup is illustrated in Supplementary Fig. 6.

The self-organized WS2 nanotube cluster provides intricate pathways through the numerous van der Waals interfaces. Here we evaluate the quantity of the van der Waals contacts by the WS2 nanotubes. The WS2 bundles are modeled using the random packing of 1D rods that form the numerous 0D interfaces, developed by Philipse94,95 and refined in this study considering the hollow morphology and flexibility of the WS2 nanotubes. The model is elaborated in Supplementary Fig. 7 and Supplementary Note 1. The random packing volume fraction φ of 14% is derived using:

$$\varphi=\frac{m{D}^{2}}{\rho \left({D}^{2}-{d}^{2}\right)\cdot{{{\rm{\cdot }}}}V(1-\varPhi )},$$
(1)

where m and V are the mass and volume of the WS2 nanotube cluster, ρ is the density of WS2 (7.5 g·cm-3), D and d are the outer and inner diameter of the hollow WS2 nanotube. According to the random contact model, we have

$$\varphi \cdot{{{\rm{\cdot }}}}\frac{{L}^{{\prime} }}{D}= < c > ,$$
(2)

where L’ is the equivalent length of the nanotube with rigidness, D is the diameter of the nanotube, and <c> is the average contact per rod. Note that L’ is shorter than the actual length of the nanotube considering the long flexible nanotube as the combination of shorter rigid segments with an equivalent length of L’. For the thin segment, the average contact number is 5.494,95. From Eq. (1), the approximate value of L’ is obtained, which is 2.8 μm. We can further derive the contact density as 1.0×1010 mm-3 and the total contact quantity as 1.4×108 for the WS2 cluster in Fig. 1e with a volume of 1.4×10-2 mm3 extracted from the micro X-ray CT. Comparably, the brain of a fruit fly has a synapse density of 0.7×1010 mm-3, a volume of 1.4×10-2 mm3, and a total synapse number of ~108. The abundant connectivity in the WS2 nanotube cluster resembles the fruit fly’s synapses, providing the rich complexity needed for the scaled PRC. The electrodes and peripheral circuits are deliberately positioned outside the WS₂ nanotube clusters to preserve the intrinsic density of the WS2 nanotubes’ cluster without disruption. This design ensures that the high-density van der Waals interfaces remain undisturbed while enabling electrical access to the reservoir. The electrodes, connecting gaps, and peripheral circuits in the system are constructed on the web surface and do not interfere with the number or functionality of the mimetic synapses. Importantly, the computational capability of our PRC system relies on the collective behavior of these interfaces rather than precise control of individual connections. This approach mirrors the statistical nature of biological neural networks, where global network properties dominate over single-synaptic precision. The inherent randomness and complexity within the cluster structure are the key features that enable it to mimic biological synaptic networks. This bioinspired design allows the system to maintain structural integrity while exhibiting robust computational performance.

Before utilizing the ultralong WS2 nanotube cluster with massive van der Waals 0D interfaces for PRC, we investigate the electrical and photoelectrical characteristics of the single 1D WS2 nanotube and single 0D van der Waals interfacial connection, respectively. In essence, the electrical path within the cluster, despite its complexity, is composed of the nanotubes themselves and the van der Waals contacts. The four-terminal devices of single and crossbred WS2 nanotubes were fabricated, as shown in Fig. 2a and b. The schematic and equivalent circuit diagrams of electrical measurement are shown in Fig. 2c. The conductivity of a single WS2 nanotube averages at 192 S·m-1, as shown in Fig. 2d90. The I-V curve presents high nonlinearity of the 0D van der Waals interface, as shown in Fig. 2e and f. As temperature decreases, the current exhibits a concomitant reduction, a phenomenon indicative of an energy barrier that charge carriers must surmount to traverse the 0D van der Waals interface as shown in Fig. 2g. An equivalent barrier height on the order of several hundred meV is derived from an Arrhenius analysis of how the current varies with temperature, as detailed in Supplementary Fig. 8. Therefore, the 0D van der Waals barriers provide the nonlinearity in the electrical dynamics of the self-aligned WS2 nanotube cluster, which is essential for the nonlinear projection in reservoir computing.

Fig. 2: Characterization of the WS2 nanotube devices and 3D cluster.
Fig. 2: Characterization of the WS2 nanotube devices and 3D cluster.
Full size image

SEM images of (a) 1D WS2 nanotube (scale bar: 5 μm) and (b) 0D van der Waals interface (scale bar: 3 μm) devices. c, Schematic of the four-probe measurement and equivalent circuits. d I–V curves of the 1D WS2 nanotube device. Inset: schematic of the four-probe measurement. The results are identical to Ref. 90. e I–V curves of the 0D van der Waals interface device. Inset: schematic of the four-probe measurement. f The nonlinearity factor as a function of voltage. Inset: schematic of the four-probe measurement. g I-V curves of a single van der Waals interface device, with temperatures ranging from 150 K to 400 K in increments of 50 K. Inset: schematic of the four-probe measurement. h I-V curves of a single WS2 nanotube device, with laser powers of 0 mW, 0.1 mW, 1 mW, and 2 mW. Inset: schematic of the photocurrent measurement. i I-V curves of a single 0D van der Waals interface device. Inset: schematic of the photocurrent measurement. j The absorption and photocurrent spectrum of the WS2 devices. The photocurrent spectrum indicates a bandgap of about 1.8 eV. k The electrochemical impedance spectroscopy of the 3D nanotube cluster. l, Response of the WS2 nanotube cluster corresponding to different pulse streams.

We further study the optoelectrical response of the single 1D nanotube and the single 0D van der Waals interface device. Upon light illumination, the WS2 nanotube presents an intrinsic photovoltaic effect in accordance with the report by Zhang et al.96, as shown in Fig. 2h and Supplementary Fig. 9. This photovoltaic effect is programmable based on the history of the applied current and voltage, accompanied by interlayer sliding inside the nanotubes97. The programmable photovoltaic response is also observed at the single van der Waals interface device due to the 0D sliding ferroelectric effect98, as shown in Fig. 2i and Supplementary Fig. 10. The absorption spectrum of WS2 nanotubes covers most of the visible wavelength range, corresponding to a bandgap of 1.83 eV, as shown in the upper panel of Fig. 2j. This is in accordance with the photovoltaic current spectra of the 1D nanotube device and 0D van der Waals interface device, as shown in the middle and lower upper panels of Fig. 2j. The optoelectrical response of the WS2 nanotube and their 0D interface allow the intervention of light as a method for manipulating the projections inside the WS2 nanotube cluster, which imitates the optogenetic response of the neural ion channels67,68,69,70,99,100,101,102,103,104.

We further study the dynamics of the 3D nanotube cluster as the physical reservoir. The electrochemical impedance spectroscopy of the 3D nanotube cluster is shown in Fig. 2k and in Supplementary Fig. 11. The cluster shows a capacitive impedance, potentially resulting from the combination of interlayer capacitance105 and nanobattery effect106. The capacitance includes the intrinsic capacitance determined by the complex network structure of the cluster, which is formed by the geometric coupling of the nanotubes and does not change with the voltage bias, and the parasitic capacitance that is sensitive to the electrical dynamics of nanotubes, whose origins is from the nanobattery effect. The capacitive character of the 3D nanotube cluster enables temporal information storage. By applying the voltage pulse stimulation, the responding current reduces with a relaxation time of 0.06 ms, as shown in Supplementary Fig. 12. The cluster was then subjected to different input pulse streams, as shown in Fig. 2l. The read states collected with the input of a 4-bit temporal binary series can be well separated at distinguishable levels. The classification accuracy for distinct pulse signals remains unaffected by readout voltage fluctuations due to the high-dimensional projection property in the reservoir space, which ensures different temporal patterns are mapped to non-overlapping regions. Moreover, the system exhibits intrinsic tolerance to minor perturbations through its nonlinear dynamics and short-term memory properties, while the trained readout layer effectively adapts to such fluctuations by extracting the fundamental statistical features of reservoir responses. The WS₂ nanotube cluster exhibits synaptic behaviors, transitioning from short-term plasticity (STP) to long-term plasticity (LTP) under repeated voltage pulses (0.5 V, 100 ms, 200 ms intervals). As shown in Supplementary Fig. 13a-c, with increasing pulse counts (N = 2, 5, 10), the device shifts from transient STP to persistent LTP, mimicking biological synaptic potentiation. Additionally, the paired-pulse facilitation (PPF) ratio, defined as (A₂ - A₁) / A₁ × 100%, was measured. Supplementary Fig. 13d shows the PPF ratio decreasing with longer pulse intervals, consistent with biological synapses. The ability to store and process temporal information lays the foundation for its application as a computing reservoir for various tasks, as presented below.

Execution of monomodal tasks

We demonstrate the paradigm of VLS PRC based on the 3D WS2 nanotube cluster by implementing various monomodal tasks.

For the handwritten digit recognition task, the digit images from 0 to 9 were binarized and transformed into spatiotemporal sequences, represented by low (0 V) and high voltage (5 V) electrical pulses with eight input channels, as depicted in Fig. 3a. The handwritten digit recognition task was performed by feeding voltage pulses into the 3D WS₂ nanotube network, where the elicited responses were recorded as inputs for the next-level artificial neural network. This network was subsequently fine-tuned through standard backpropagation training to achieve digit classification. As shown in Fig. 3b, classification accuracies were tracked over 200 training epochs, with input data augmented by adding Gaussian noise (σ = 0.05, 0.1,…, 0.4) to evaluate system robustness. In Fig. 3c the confusion matrix was generated for the data with a noise level of σ = 0.05. Despite a gradual decrease in accuracy on the test set with increasing noise levels, the VLS PRC model retained its recognition ability, showcasing the robust image recognition capabilities of the 3D VLS PRC platform. Supplementary Fig. 14 and Supplementary Table 1 compare the performance of the computation using different sizes of the reservoir. It can be seen that the accuracy of the PRC was improved with increasing cluster size. This may be attributed to the fact that projections within a smaller reservoir occur in a lower-dimensional space, which limits the performance of the PRC.

Fig. 3: Monomodal tasks using the VLS PRC.
Fig. 3: Monomodal tasks using the VLS PRC.
Full size image

a Schematic of handwritten digit recognition. b Classification accuracies of the handwritten digit recognition task after 200 training epochs, with noise levels of σ = 0.05 to 0.4. c Confusion matrix corresponding to the noise level of σ = 0.05. Color intensity is positively correlated with classification accuracy. d Schematic of speech recognition. Speech digits from 0 to 9 are preprocessed using the Lyon passive cochlear model. e Accuracies of speech recognition. f Confusion matrix with noise of σ = 0.05. g Schematic of biological signal recognition with the four types of ECG waveforms (normal sinus rhythm, coronary artery disease, myocardial infarction, and complete atrioventricular block). h Confusion matrix of the ECG classification task. i MSE curve for time-series forecasting task. j Predicted values (red line) compared with ground truth (black line).

To further assess the capability of the 3D VLS PRC platform in temporal information classification, we conducted a benchmark test focusing on speech recognition. As shown in Fig. 3d, the audio waveforms of isolated spoken digits (zero to nine in English), sourced from the NIST TI-46 database, were resampled at a rate of 12.5 kHz. The waveforms were preprocessed into 8 frequency channels using the Lyon cochlear model107. The Lyon passive cochlear model is elaborated in Supplementary Fig. 15 and Supplementary Note 2. The preprocessed data were then fed into the reservoir. As shown in Fig. 3e and f, we used 200 samples to form the test set and the accuracies increase with the training epochs for all noise levels and reach a satisfying recognition ability for different noise levels. This demonstrates the 3D VLS PRC’s capability to conduct speech classification task even in chaotic auditory environments.

For the biological signal recognition, electrocardiogram (ECG) signals of normal sinus and arrhythmic rhythm—coronary artery disease, myocardial infarction, and complete atrioventricular block108 were used as input signals for the reservoir in Fig. 3g. The signal was amplified and fed into the reservoir via a signal generator. The output was collected and processed with the tunable artificial neural network. The 3D VLS PRC exhibited excellent performance in the recognition task and can do the accurate classification of the normal and the arrhythmic electrocardiogram signals, as shown in Fig. 3h.

Besides the classification tasks, this work explored the prediction task of nonlinear time-series using the 3D VLS PRC. The research builds upon the nonlinear equation of autoregressive moving average with exogenous inputs established in prior studies109, aiming to evaluate the capability of the 3D VLS PRC to process dynamic input time signals and its predictive performance. The dynamic equation is given by

$$y(k)=0.4y(k-1)+0.4(k-1)y(k-2)+0.6{u}^{3}(k)+0.1,$$
(3)

The primary objective of the task is to predict the value y(k), which is dependent on the current input u(k) as well as the recent outputs y(k-1) and y(k-2). The results depend on the system’s response to the current input and its short-term memory capabilities in relation to historical outputs. The input signal u(k), which is represented by the electrical voltage, was subsequently fed into the VLS PRC, while dynamic response was collected and transferred to the readout layer. By training the weights of the readout layer, the mean squared error decreased with increasing number of epochs, as shown in Fig. 3i. The ground truth time-series values and the prediction through the 3D VLS PRC exhibited a strong correlation, as illustrated in Fig. 3j.

The self-organized, submillimeter-long WS₂ nanotube cluster functions as a 3D physical reservoir for very-large-scale computation. The abundance of van der Waals interfaces (1010 mm-3) provides a high-dimensional state space for capturing input patterns. The density of the 3D WS2 nanotube cluster serves as a critical design parameter for optimizing reservoir computing performance, providing essential guidance for future neuromorphic hardware development. Even as the reservoir’s scale and complexity expand, internal information mapping occurs simultaneously. Thus, differently from neural networks implanted in very-large-scale logic circuits, here the power consumption and computation time are not sensitive to the scale. We analyzed the energy consumption of the 3D VLS PRC. The input voltage (V) pulses to the 3D WS2 nanotube cluster reservoir are set to 5 V, and the average operating current (I) is approximately 1.4 μA, attributed to the large impedance of the 3D WS2 nanotube cluster. The power consumption of each input node is P = V × I = 5 V × 1.4 μA = 7.0 μW. The power can be further lowered by decreasing the amplitude of the input voltage, for example, the input voltage of 2.5 V corresponds to an operating current of 0.22 μA and a power of 0.6 μW. For the readout layer, a digital implementation using analog-to-digital converters and an FPGA consumes approximately 81 mW in total. Additional details are provided in the Supplementary Note 3 and Supplementary Table 2. Therefore, the repeatable performance of the entire reservoir computing system has been demonstrated, with additional specific details provided in Supplementary Table 3.

Mimetic optogenetics

Optogenetics offers the second sight that uses light to modulate the organism’s activity or behavior. Here, the mimetic optogenetic behavior is demonstrated using the light-sensitive 3D VLS PRC system. The WS2 nanotube reservoir is exposed to a light beam with an intensity of 3×10-4 W·cm-2. As shown in Fig. 4a, upon illumination, the accuracy of handwritten digit recognition notably decreases, with a reduction of 32% at a noise level of σ = 0.05. The confusion matrix from the illuminated reservoir is presented in Fig. 4b. The increased confusion is caused by the photoconductive and photovoltaic effect in the 1D WS2 nanotube and the 0D van der Waal interfaces, which changes the projection inside the reservoir and disables the trained PRC model. However, this disability can be mitigated by a second-time training of the fully connected layers in the PRC model, as shown by the confusion matrix in Fig. 4c and the learning curves in Supplementary Fig. 16. Similar mimetic optogenetic change is also observed in the speech recognition task (Figs. 4d-f, Supplementary Fig. 17), the electrocardiogram signal recognition task (Figs. 4g-i) and the time-series prediction task (Figs. 4j-l). The light exposure impacts the feature vectors extracted from the WS2 nanotube-network reservoir, confuses the previously learned model, and lowers the recognition or prediction accuracies. The damaged function can be repaired by readjusting the weights in the fully connected layers through the second-time training. In prior research, Cai et al. established that electrical stimulation of neural network reservoirs significantly enhances computational accuracy through augmented neuronal interconnectivity within the reservoir architecture34. Our experimental findings reveal analogous precision enhancement in photo-stimulated reservoir computing systems, potentially stemming from the optimized 3D connectivity established within the nanotube matrix. This structural evolution in the WS₂ nanocluster configuration appears to facilitate strengthened inter-nanotube signal transduction pathways, mirroring the neural network optimization observed in bioelectrical systems. Therefore, the optoelectrical response of the WS2 nanotubes and their interface allow light to manipulate projections inside the WS2 nanotube cluster, translating the principles of optogenetics into the PRC paradigm.

Fig. 4: Optogenetic effects on monomodal tasks.
Fig. 4: Optogenetic effects on monomodal tasks.
Full size image

a Accuracies and losses of handwritten digit classification under dark and illuminated conditions. b, c Confusion matrices (b) under illumination and (c) after second-time training with noise of σ = 0.05. d Accuracies and losses of speech recognition. e, Confusion matrix under illumination with noise of σ = 0.05. f Confusion matrix after second-time training with noise of σ = 0.05. g Accuracies of biological signal recognition under three conditions: darkness, illumination, and after second-time training. h, i Confusion matrices (h) under illumination and (i) after second-time training. j MSE function of the time-series prediction. k, l Predicted values from the VLS PRC compared with ground truth (k) under illumination and (l) after second-time training.

Execution of multimodal tasks

The development of multimodal models is regarded as an essential step towards general artificial intelligence. Current deep learning models still struggle with multimodal tasks that humans can easily accomplish. In this work, we utilize the 3D VLS PRC to demonstrate a speech-to-image multimodal generative task which exceeds previous reservoir computing systems, as shown in Supplementary Table 4. Gaussian noises are introduced to an MNIST handwritten digit image dataset over timesteps and a diffusion model is trained to denoise for the reverse process. The 3D VLS PRC device, demonstrated in this work, projects the speech input signals into higher dimensional vectors as the latent conditions to guide the generation of images via the cross-attention mechanism, as illustrated in Fig. 5a. The device performance is reflected in the algorithm through the quality of these projections, as higher nonlinearity and richer dynamics enable separation of features in the latent space, directly influencing the accuracy and robustness of the multimodal output. The loss function curves for generating digital images are shown in Fig. 5b. The generated digit images, guided by the latent vectors from the reservoir, closely resembled their original counterparts, despite a small portion of irregularities highlighted by the blue box, as shown in Fig. 5c. Detailed information about the generative denoising process is depicted in Supplementary Fig. 18. It can be seen that the latent vectors, from the projection of the reservoir, are qualified to condition the generation of the handwriting digits.

Fig. 5: Multimodal tasks using the VLS PRC.
Fig. 5: Multimodal tasks using the VLS PRC.
Full size image

a Schematic of a speech-to-image task using the 3D VLS PRC. b Loss curves of the speech-to-image task. c–f Left: digital images generated (c) in darkness, (d) under weak illumination, (e) under intense illumination, and (f) after second-time training. The blue and red boxes label the irregular images and errors. Right: visualization of the latent vectors with reduced dimensions. g, h Blood cell images generated (g) in darkness, (h) under intense illumination. The blue and red boxes label the irregular images and errors. i, j Colorectal tissue images (i) in darkness, (j) under intense illumination. The blue and red boxes label the irregular images and errors.

We further conduct the mimetic optogenetic test on the multimodal model. In the biological system, the disorder in neural and synaptic activity causes cognitive dissonance and intellectual dysfunction. In this work, we used a 532 nm laser with adjustable intensity to modulate reservoir projections, mimicking neural and synaptic activity dynamics in biological intelligence systems. As the 3D VLS reservoir was exposed to weak illumination (1.6×10-4 W·cm-2), there was a noticeable degeneration of the output image, as depicted by the blue and red boxes in Fig. 5d and Supplementary Fig. 19. These annotations, generated by an ANN-based detection algorithm110, classify errors as either critical (red boxes) or partial defects (blue boxes). These results are consistent with our manual observations. Interestingly, we note that the generated images for “7” look more like the shape of “5”. Despite the general lack of explainability of neuromorphic computing, the latent space from the reservoir provides interpretability to a certain extent that the projected vectors of “7” merge with that of “5”, as indicated in the right panel of Fig. 5d. Furthermore, increasing the light intensity (9.3×10-4 W·cm-2) resulted in a more significant degeneration, in which case, the model even fails to generate the shape of a digit or generates a digit image with a wrong label (for example, image “8” is generated with an input speech of “0”), as shown in Fig. 5e, Supplementary Fig. 20, Supplementary Movie 1 and Supplementary Movie 2. The optogenetic perturbation, by modulating the reservoir states through light illumination, further demonstrates the device’s functional role. Using a second-time training with adjustments of weights, the multimodal diffusion model regains the ability of speech-to-image generation, as shown in Fig. 5f and Supplementary Fig. 21. Interestingly, the second-time training enhances the accuracies of the 3D VLS PRC under high-illumination conditions. The enhanced performance can be attributed to the changes in functional connectivity, which enables the dynamic reshaping of the reservoir34. We further extended the 3D VLS PRC multimodal generative paradigm to biomedical domains. The 3D WS₂ nanotube reservoir successfully performed colorectal tissue image and blood cell image generation tasks in Fig.5g-j and Supplementary Figs. 22-25, demonstrating its capability to process complex medical imaging data. The system maintained robust performance despite the increased complexity of the biomedical features, while still exhibiting the characteristic optogenetic modulation under light illumination. This expansion to clinically significant imaging modalities underscores the system’s potential for medical applications in healthcare settings.

This study introduces a computational framework that leverages a self-organized 3D physical reservoir and verifies the feasibility and potential of this concept. The reservoir utilizes the abundant 0D van der Waals interfaces within a WS2 nanotube cluster as mimetic synaptic connections, resembling the neural system of a fruit fly (with a synaptic quantity of ~108 and synaptic density of ~1010 mm-3). Such a high-density synaptic connection enables the system to handle complex sequential tasks and endows it with a high degree of parallel computing capability. The 3D VLS PRC exhibits outstanding performance in both monomodal tasks, such as handwritten digit recognition, speech recognition, biological signal recognition, time-series prediction, and multimodal tasks like speech-to-image generation, highlighting its potential for complex applications. By exploiting the photoelectrical response of the 1D semiconducting nanotubes and their 0D interfaces, the study demonstrates a mimetic optogenetic behavior, enabling the manipulation of the VLS PRC dynamics and projections through light stimulation. This work investigates the principles of the scaling law, the capability to perform multimodal tasks, and the mimetic optogenetic mechanisms in the VLS physical reservoir, providing an innovative perspective on how computer science, nanotechnology, optoelectronics, and neuroscience merge to shape the next-generation artificial intelligence paradigm. Looking ahead, the development of all-hardware neuromorphic systems represents a critical frontier for future research. Future efforts will focus on bridging the gap between software-based training and hardware implementations, leveraging the unique properties of WS₂ nanotube cluster and their optoelectronic responses to create more robust and efficient neuromorphic computing platforms.

Methods

Materials

WS2 nanotubes were synthesized through two steps: firstly, the ultralong W5O14 nano whiskers were obtained by the high-temperature annealing of HxWO3 in a sealed quartz ampoule environment. These nanowhiskers were then used as precursors and subjected to a sulfidation process at 845 °C for 6 h in a flowing mixture of H2S and H2 gases, ultimately yielding almost pure ultralong (0.1 − 0.5 mm) WS2 nanotubes. The synthesis process is highly controlled and scalable, with a yield approaching 100%. The nanotubes exhibit exceptional stability, retaining their structural and functional properties for over 25 years, making them ideal for long-term applications in neuromorphic computing. The final WS2 nanotube crystals consist of a complex, bottom-up randomly stacked network of WS2 nanotubes, providing the material foundation for this research. The nanotube sample was then transferred for X-ray diffraction, SEM, TEM, micro X-ray CT, Raman spectroscopy, and absorption spectroscopy characterization.

Devices

WS2 nanotubes were dispersed on a silicon substrate coated with 500 nm of silicon dioxide. The dispersion density of WS2 on the silicon wafer was examined using an optical microscope, which allowed for the selection of suitable single nanotubes and their crossbars. Subsequently, a thin film of PMMA photoresist was deposited via spin-coating, followed by thermal treatment to remove residual solvent. Electron beam lithography was then employed to pattern the electrode geometry. After the development of the pattern, Cr (20 nm)/Au (120 nm) metal layers were deposited as the electrode via electron beam evaporation. The lift-off process was done by dissolving the PMMA in acetone. Subsequently, electrical and optoelectrical measurement of the device were conducted with a semiconductor characterization system with a probe station. The incident lights were introduced by lasers with wavelengths of 488, 532, 633 nm and a supercontinuum laser with a range between 655 nm to 1,000 nm.

Physical reservoir computing framework

The WS2 nanotube cluster was integrated into a reservoir computing framework. A microcontroller or DG1032Z signal generator was used to transmit digital or analog signals (handwritten digits, spoken digits, bio-signals, time-series signals) to the WS2 nanotube cluster in the form of electrical voltage via gold wire or tungsten probes. The reservoir layer received the temporal electrical stimuli and mapped them into a high-dimensional computational space. The output voltage signals were collected by an oscilloscope and fed into a layer of fully connected artificial neural networks, which delivered the final results for classification and prediction.

Dataset

The handwriting images were from the MNIST dataset, which originates from the National Institute of Standards and Technology. The 28×28 pixel grayscale images were downscaled to 8×8 binary images that can be represented by the binary high and low voltages. All images are labeled with the corresponding digit they represent, ranging from 0 to 9. The speech recognition data were from the TI-46 dataset, which consists of read utterances including the numbers 0 to 9. The sound was collected using an Electro-Voice RE-16 Dynamic Cardioid microphone at a 12.5 kHz sample rate with 12-bit quantization. The waveforms of the speeches were fed into 8 frequency channels using the Lyon cochlear model, and then into the reservoir. Biological signals were sampled from the built-in electrocardiogram waveforms of the signal generator (one normal rhythm and three arrhythmias) for the task of arrhythmia detection. The input for time-series prediction was derived from the random discontinuous waveforms built into the signal generator.

Monomodal task training

For classification tasks such as handwritten digit recognition, speech recognition, and biological signal recognition, the feature vector from the reservoir serves as the input to the readout layer. Using the states of the reservoir at the final timestep as input x, and the output\(\,\hat{y}\) was obtained through the softmax layer.

$${\hat{y}}_{j}=\frac{{e}^{{z}_{j}}}{{\sum }_{k=1}^{j}{e}^{{z}_{k}}},{z}_{j}={w}_{{ij}}\cdot{{{\rm{\cdot }}}}{x}_{i}+b,$$
(4)

where \({w}_{{ij}}\) refers to the synaptic weight between the reservoir and the softmax layer, b is the bias.

The cross-entropy loss function was used to measure the difference between the read-out prediction \({\hat{y}}_{j}\) and the desired target \({y}_{j}\)

$${L}_{1}=-\frac{1}{n}\mathop{\sum }_{m=1}^{n}{\sum }_{k=1}^{j}{y}_{k}^{(m)}\log ({\hat{y}}_{k}^{(m)}),$$
(5)

where n is the total number of samples.

Linear regression algorithm was used to train the readout function for nonlinear time-series prediction task. Assume the state of the reservoir is x, which is fed to the readout network as the input. The cost function is defined as

$${{J}}({{w}})\,=\frac{1}{2{{n}}}{\sum }_{{{i}}=1}^{{{n}}}{({{{w}}}^{{{T}}}{{{x}}}^{({{i}})}-{{{y}}}^{({{i}})})}^{2},$$
(6)

where n is the total number of samples, \({y}^{(i)}\) is the desired value of the input \({x}^{(i)}\), w refers to the weight matrix.

Gradient descent was used to find the parameters that correspond to the minimum value of the cost function J.

Multimodal task training

In the time-series signal forecasting task, the input signal feature vector generated by the VLS PRC is utilized through the cross-attention mechanism as a conditional vector to guide the process from Gaussian noises to the target images. The equation for adding Gaussian noise\(\,\epsilon\) to an original image x0 is given by

$${x}_{t}=\sqrt{{\prod }_{s=0}^{t}\left(1-{\beta }_{s}\right)}{x}_{0}+\sqrt{1-{\prod }_{s=0}^{t}\left(1-{\beta }_{s}\right)}\epsilon,$$
(7)

where xt refers to the generated noisy image, βs refers to the noise variance added at time step s.

The mean squared error (MSE) loss function was used to measure the difference between the predicted noise and the actual noise. The MSE loss is defined as

$${L}_{2}={\frac{1}{n}{\sum }_{i=1}^{n}({\epsilon }_{i}-{\hat{\epsilon }}_{i})}^{2}$$
(8)

where n refers to the total number of samples, \({\epsilon }_{i}\) represents the actual noise, and \({\hat{\epsilon }}_{i}\) represents the predicted noise.