Introduction

The era of data explosion necessities a sharp increase in computation efficiency. However, the von Neumann bottleneck slows down breakthroughs in computing power, thereby hindering the progress of artificial intelligence (AI)1,2,3. Bionic visual system, known for parallel computation with ultra-low power consumption, has been developed as a promising alternative for AI data processing4,5,6. Notably, the weight storage and update capabilities of optoelectronic neuromorphic units are crucial for constructing efficient neural networks. Up to date, numerous optoelectronic synaptic devices have been explored for neuromorphic computing, including phase-change memories (PCMs)7, memristors8,9, and field-effect transistors10,11,12,13. Among these, organic field-effect transistors (OFETs) have been extensively studied due to their flexibility, solution processability, and high optical gain14,15. However, affected by the charge screening effects of OFETs, there are still some challenges for advanced neuromorphic visual systems, including high operation voltage, obvious programming nonlinearity, limited storage memories, and uncontrollable conductance update state, which are difficult to fully resolve through innovating OFETs16,17,18. Furthermore, the operation of OFETs is dependent on electronic charge transport, which leads to difficulties in their bio-integration with living organisms which primarily function via ionic signals16. Therefore, there is an urgent need to develop the novel iontronic optoelectronic synapses2.

Organic electrochemical transistors have emerged as a competitive platform across diverse scenes such as brain-machine interface driven by their advantages in low operation voltage, linear weight programming, and possibility of bio-integration19,20,21,22. Recently, researchers have devoted to developing organic photoelectrochemical transistors (OPECTs) for biosensing by introducing photosensitive materials16,23,24,25 or coupling them with other optoelectronic devices26,27,28, while the potential of OPECTs for neuromorphic visual system is often overlooked29,30,31. Until now, only a few studies have successfully proposed OPECTs-based optoelectronic synapses with multi-task learning applications16,24,32,33, based on photo-induced electrochemical doping mechanism. Nevertheless, the pivotal merits of OPECTs in terms of ultra-low nonlinear optical programming, wide dynamic range (Gmax/Gmin), and multi-level storage memories have not yet been simultaneously realized in a unified platform34,35. Critically, existing OPECTs are confined to ultraviolet (UV) and visible (vis) light responses, while near-infrared (NIR) response capabilities remain largely unexplored, especially the NIR response above 1000 nm. This spectral constraint significantly hinders their practical application in scenarios where NIR light is essential, such as deep-tissue biosensing for wearable health monitors, nighttime imaging, and neuromorphic vision systems36. Furthermore, the absence of wafer-scale fabrication methods precludes functional integration of these capabilities into scalable arrays. The aforementioned challenges arise from two primary factors: (1) Conventional organic mixed ionic-electronic conductors generally lack effective NIR photosensitivity; (2) It’s challenging to fabricate OPECTs array in wafer-scale without sacrificing their photosensitivity. Overall, developing NIR-responsive OPECT arrays is imperative to enable high-throughput sensing, parallel signal processing, and advanced bio-interfacing systems beyond current limitations.

Herein, we developed the NIR (1000 nm) OPECTs arrays based on bulk heterojunction (BHJ) and photo-modulated ion doping designs. Under high gate bias and NIR illumination, ion injection into BHJ film caused by photo-induced electrochemical doping can stabilize photogenerated charges, resulting in non-volatile conductance update behavior. Wafer-scale OPECTs array (1111 units/cm2) with decent uniformity was fabricated by photolithography method, demonstrating the feasibility of further integrating OPECTs into neuromorphic visual systems. Remarkable merits as neuromorphic visual units were presented, including non-volatile multi-level storage memories (34 states, 60 s) and linear weight programming with ultra-low nonlinearity (NL = −0.015) over a wide range (Gmax/Gmin = 47.3). The aforementioned properties rival or even surpass those of state-of-the-art organic optoelectronic synapses, beneficial of the performance for visual computing (Supplementary Tables 1, 2). In addition, systematic characterization was carried out to study the effect of photo-induced electrochemical doping on the non-volatile memories of OPECTs. Notably, the OPECTs array enabled NIR image recognition, memorization, and visualization through integrating back-end light-emitting diode (LED) arrays. By combining front-end convolutional processing kernels with back-end neural networks, we firstly demonstrated an ideal convolutional neural network (CNN) system (“Ideal”) for handwritten digit recognition with a total accuracy of 98.03%. We further integrated an OPECTs-based CNN system for accurate noisy (σ) handwritten digit recognition, reflecting the superiority of the OPECTs array-based neuromorphic visual system. This work provides a notable approach to building an emerging neuromorphic visual system for versatile AI applications.

Results

OPECTs with multi-level linear conductance update capability

Figure 1a demonstrates the schematic diagram of a human vision system used for image recognition37. The human retina can extract image information via photoreceptor cells and encode it into electrical signals. The pre-processed information is then transmitted to the brain for further decoding and identification to draw conclusions38. Inspired by this excellent information processing capability of retina, many efforts have been paid to exploiting advanced optoelectronic synapses based on OFETs (Fig. 1b). However, OFETs usually cannot meet the requirements of neuromorphic visual computing including low operation voltage, ultra-low programming nonlinearity, and multi-level storage memories (Supplementary Fig. 1). In this work, we exploited OPECTs using BHJ film based on poly[2,5-(2octyldodecyl)-3,6-diketopyrrolopyrrole-alt-5,5-(2,5-di(thien-2-yl)thieno[3,2-b] thiophene)] (DPPDTT) and 2,2′((2Z,2′Z)-((5,5′-(4,4-bis(2-ethylhexyl)-4H-cyclopenta[1,2-b:5,4b′]dithiophene-2,6-diyl)bis(4-((2-ethylhexyl)oxy) thiophene-5,2diyl)) bis(methanylylidene))bis(5,6-difluoro-3-oxo-2,3-dihydro1H-indene-2,1-diylidene)) dimalononitrile (COTIC-4F) acting as sensing materials, as shown in Supplementary Fig. 2. In this design, DPPDTT was utilized to take advantage of its solution processability and relatively high electronic conductivity, while COTIC-4F can contribute to high NIR responsivity. The DPPDTT/COTIC-4F BHJ film is not only coated onto the gate electrode but also acts as the channel, which is fully covered by the sequentially deposited ion-gel layer. In this configuration, special ion migration and injection processes induced by illumination cause specific synaptic behaviors. Specifically, under illumination, photogenerated charges (electrons and holes) are created and separated within the DPPDTT/COTIC-4F BHJ film. The subsequent behavior critically depends on the applied gate voltage. At low negative gate biases, ion penetration from the ion-gel into the channel is minimal, and the photo capacitive effect dominates39. The device exhibits a transient photo response but lacks memory. This is because the photogenerated charges primarily modulate the electrical double layer (EDL) capacitance at the semiconductor/ion-gel interface (Supplementary Fig. 3). At high negative gate biases, illumination triggers a distinct photoelectrochemical response. The combined photovoltaic potential and applied gate bias surpass the ion injection threshold, facilitating ion transport from the ion-gel into the organic semiconductor matrix. This process involves two complementary mechanisms: (1) cations progressively infiltrate the BHJ film on top of the gate electrode, where they stabilize photogenerated electrons through charge compensation, while (2) anions permeate the channel, enhancing its conductivity16. After illumination, the egress of anions from the channel is significantly inhibited, which may be due to combined steric constraints imposed by the surrounding crystallites19, semiconductor’s bulky side chains, and solvation effects (Fig. 1c)24. This ionic confinement mechanism prevents charge recombination, thereby maintaining elevated channel conductivity and producing a non-volatile photo response.

Fig. 1: Bio-inspired OPECTs based on photo-induced electrochemical doping mechanism for neuromorphic visual computing.
figure 1

a Schematic of bio-inspired neuromorphic visual computing system for image recognition. b Optoelectronic synapses based on OFETs and OPECTs. c Schematic illustration of photo-induced electrochemical doping mechanism at high negative gate biases. The size of the arrow represents the driving force of ion movement. At high negative gate biases, the dedoping force of ions after light off is much lower than the doping force caused during light on, resulting in non-volatile behavior. d Schematic of controllable multi-level linear weight programming based on OPECTs.

Full size image

Figure 1d presents the corresponding potential changes from the gate to channel. During illumination, the redistribution of cations at the gate region causes an increase in the gate potential. In addition, the potential of the channel will increase with the gate potential to maintain the VGS constant. Furthermore, the increase of the gate capacitance after illumination (Supplementary Fig. 4) allows a larger potential drop at the channel/electrolyte interface (Fig. 1d, |VEC*| > |VEC| ). i.e., the effective potential applied to the channel/ion-gel is enlarged24. At high negative gate biases, the process of ion injection is non-volatile. Increasing the pulse number will bring the channel to a more doped state gradually, enabling multi-level conductance states. When programming the device conductance state in the linear range of the transistor’s transfer curve, the conductance change upon illumination will be linear as light pulses gradually increase the effective gate bias value to the channel19. Overall, our OPECTs possess the capability to linearly modulate multi-level non-volatile conductance states, which opens a path for the construction of visual neural networks40.

The device adopts a planar three-electrode configuration, with the ion-gel located above the photosensitive layer. Upon stimulation, the device undergoes an ion diffusion process, modulating the channel conductance in a manner analogous to the regulation of chemical synapses (Fig. 2a)41. The light transmittance of the ion-gel layer was reflected in Supplementary Fig. 5a, no obvious absorption peak presented in the range of 200–1100 nm, which ensured that the light signal received by the photosensitive layer would not be weakened by the ion-gel layer. The DPPDTT/COTIC-4F film possessed the absorption properties of two components, endowing OPECTs with NIR response (Supplementary Fig. 5b)42. The influence of blending ratio of DPPDTT and COTIC-4F on the photo response was systematically investigated as shown in Supplementary Fig. 6, revealing the optimal blending ratio is 1:2. This ratio exhibited the highest photo response under 1000 nm light. Figure 2b depicts the optical writing and electrical erasing processes of OPECTs. Temporal IDS increased after applying a photonic pulse, and then remained almost unchanged until a negative electrical pulse was applied, which demonstrated the non-volatile optical programming capability of devices14. By applying 120 consecutive NIR light pulses, the conductance state of a device increased quasi-linearly from 1.54 μS to 361.15 μS, realizing a 234-fold improvement (Fig. 2c).

Fig. 2: The synaptic functions of OPECTs.
figure 2

a Schematic of neurons, synapses, and OPECTs based on DPPDTT/COTIC-4F. b IDS of transistor induced by a photonic pulse (1000 nm, 3 mW/cm2, 0.5 s) and an electric pulse (0 V, 0.5 s) at VGS = −1.2 V. c Conductance update by continuously applying 120 optical pulses (1000 nm, 3 mW/cm2, 0.5 s). d Retention time of three conductance states. e Retention characteristics of an OPECT at 32 conductance states, with G32/G1 ≈ 84.7 (1000 nm, 2 mW/cm2). f Conductance changes induced by optical pulses with different wavelengths (400–1064 nm, 2 mW/cm2). g LTP/D curves activated by a series of optical pulses (1000 nm, 2 mW/cm2) and electric pulses (0 V) at different VGS. h LTP/D curves at VGS = −1.2 V. The insert is the one-to-one correspondence conductance states, showing controllable multi-level weight programming capability of OPECTs.

Full size image

To evaluate the multi-level conductance programming capability of the device, 728 light pulses at 1000 nm were applied to the device in Supplementary Fig. 7. The conductance state was programmed to 140 μS and then slowly decreased to 110 μS with a retention time more than 1000 s. Similarly, by prolonging the light exposure time, the conductance of devices could be programmed linearly with long retention time. Figure 2d demonstrates the retention time of OPECTs at 3 conductance states (G3/G1 ≈ 81.9) induced by 1000 nm light over 1000 s. The multi-level non-volatile conductance programming capability of the device was further demonstrated in Fig. 2e. 32 conductance states (G32/G1 ≈ 84.7) maintained good discrimination within 60 s (Supplementary Fig. 8)43. The effect of electric and light pulses on the photo response characteristics44 of OPECTs at different baseline VGS was reflected in Supplementary Fig. 9. When |VGS| was above -1.1 V, the photo response of the device changed from volatile to non-volatile memory, demonstrating the existence of an ion injection threshold19. More detailed explanations for this transition were provided in Supplementary Note 1. When VGS was set at −1.2 V, the influence of light wavelength on photoconductance was investigated in Fig. 2f, demonstrating the broadband (400–1064 nm) quasi-linear conductance programming ability of OPECTs. Notably, there were the largest photo response under 1000 nm light, which origined from the higher blending ratio of COTIC-4F. The corresponding NL and Gmax/Gmin were depicted in Supplementary Fig. 10a, presenting the largest Gmax/Gmin and relatively good NL under 1000 nm light. In addition, the NIR light pulse duration dependent plasticity of ΔIDS was demonstrated in Supplementary Fig. 10b. It could be found that ΔIDS increased linearly with the increase of pulse duration. We have further investigated the synaptic weight changes under simultaneous light and voltage stimulation. As shown in Supplementary Fig. 11. The synaptic current triggered by the light and electric pulses simultaneously (I1) was approximately equal to the sum of that triggered by one pulse (I2 + I3), which suggested the light pulse functioned analogously to gate pulse, revealing the potential for optoelectronic co-modulation in OPECTs-based neuromorphic devices.

In view of the photo response and linear weight programming characteristics of the device at 1000 nm light, the long-term light potentiation and electrical depression (LTP/D) curves at different VGS were illustrated in Fig. 2g and Supplementary Fig. 1245. Each conductance state of LTP/D curves exhibited good stability, reflecting the decent weight programming function of devices (Fig. 2h). This precise weight update characteristic has not been well reflected in other types of organic optoelectronic synaptic devices, which hinders the advancement of efficient visual computing systems. To further prove the advantages of OPECTs in non-volatile weight programming, we fabricated OFETs based on the same photosensitive material system. Under similar light pulse conditions, the device exhibited volatile, nonlinear, and unstable weight programming behaviors (Supplementary Fig. 13).

To measure the long-term stability of designed OPECTs, we investigated the long-term conductance retention behavior under varying temperature and humidity. As shown in Supplementary Fig. 14, under a certain humidity, the conductance of the device could remain stable beyond 1000 s. With the increase of humidity, the conductance of the device increased. For temperature stability, when the temperature was lower than 80 °C, there was no obvious conductance change within 1000 s. When the temperature increased to 100 °C, the conductance increased significantly and remained relatively stable, which reflects that OPECTs have a certain ability to resist temperature interference. To further demonstrate the operation stability of our OPECTs, we placed the device in a dual 85 testing chamber (85 °C and 85% RH conditions). After 120 h, we measured the long-term stability of the device beyond 1000 s at three conductance states similar to Fig. 2d. There was no obvious degradation in different states during 1000 s, which reflects that our OPECTs have good temperature and humidity operation stability. Moreover, the cycling endurance was evaluated by measuring the LTP/D curves during 8 cycles in Supplementary Fig. 15. There was no obvious degradation of LTP/D curves under 8 cycles, and the LTP/D curves had good consistency during 800 s of operation.

The uniformity and photoconductance update performance of OPECTs array

Device miniaturization is a committed step towards further vision system integration and high-resolution image processing46,47. We successfully reduced the device channel size to tens of microns using photolithography strategies, obtaining the wafer-scale OPECTs array with a density of 1111 devices/cm2 (Fig. 3a). The detailed preparation processes of an OPECTs array were shown in Supplementary Fig. 16. To achieve patterning of the heterojunction film, firstly, a precise protective layer was formed on the semiconductor layer using positive photoresist AZ5214E. Then, after plasma etching (PE), an orthogonal solvent, ethanol, was used to remove the residual resist to obtain the patterned films48. The precisely patterned ion-gel layer effectively eliminates crosstalk in our OPECT array, with inter-device leakage current as low as ~10−11 A (Supplementary Fig. 17), demonstrating superior electrical isolation. The thickness of the patterned semiconductor layer was shown in Supplementary Fig. 18. The thickness of nearly 70 nm provides sufficient doping depth while ensuring device photo response performance. The downscaled OPECTs still maintained decent transistor performance and photo response characteristics. As shown in Supplementary Fig. 19, 80 devices were selected randomly to investigate the uniformity of OPECTs array. The distribution of transfer curves in dark and light conditions was concentrated, with uniform Ion/Ioff ratio (Fig. 3b) and threshold voltage (VTh) (Fig. 3c)49. Notably, light enhanced the Ion/Ioff ratio of the device while shifting the VTh of the device toward a positive voltage. The average values of VTh in dark and light conditions were −1.50 V and −1.37 V, with the standard deviation of 0.01 V and 0.02 V, which were only 0.5% and 1% of the operating voltage range (2 V), respectively. The aforementioned results further prove the possibility of device integration into large-scale neural networks. For submicron-scale devices, further process optimization, including the selection of suitable photoresists, high-precision crosslinker, and more sophisticated lithography equipment will be necessary. Progress in OPECT miniaturization underscores the feasibility of high-density organic device integration via complementary metal oxide semiconductor (CMOS) compatible processes46.

Fig. 3: The photoconductance programming performance of OPECTs arrays.
figure 3

a Compatible photolithography processes for an OPECTs array with a density of 1111 units/cm2. Scale bar, 500 μm. The effect of NIR light on b Ion/Ioff  ratio and c VTh of OPECTs. d Conductance updates linearly by continuously applying 600 optical pulses (1000 nm, 2 mW/cm2, 0.1 s) under VGS = -1.1 V. e IDS varies linearly with the pulse number. f The influence of light wavelength on LTP/D curves under VGS = −1.2 V, with minimal NLP = −0.015. g Three reproducible LTP/D curves with 100 optical pulses (1000 nm, 2 mW/cm2, 0.5 s) and 100 electric pulses (0 V, 0.5 s). h Retention characteristics of OPECTs at 34 conductance states, with G34/G1 ≈ 65.8. i Comparison of state number (with a retention time ≥ 50 s), range (Gn/G1), |NLP | , and Gmax/Gmin with previously published works.

Full size image

The analog programmability of OPECTs array was depicted in Fig. 3d, presenting remarkable conductance update performance with NL = 0.112 over a wide switching window from 0.0337 μS to 80.181 μS (Gmax/Gmin > 2379) by continuously applying 600 optical pulses under VGS = -1.1 V. Subsequently, the effect of pulse number and frequency on IDS was studied in Fig. 3e and Supplementary Fig. 20. IDS varied linearly with pulse number, and no saturation trend was presented. Furthermore, under constant pulse duration and number, varying the pulse frequency resulted in no significant change in ΔIDS, demonstrating the characteristic of anti-pulse frequency-dependent plasticity50. This performance originates from the capability of impressive non-volatile conductance programming, which is conducive to the structural optimization of neural networks for neuromorphic visual computing. Similarly, the effect of VGS and light wavelength on the weight update performance was investigated in Fig. 3f and Supplementary Fig. 21, and the related NL, Gmax/Gmin were depicted in Supplementary Fig. 22. Notably, when VGS was set as −1.2 V under 1000 nm NIR light, the highest Gmax/Gmin of 47.3 and lowest NL of −0.015/2.01 were achieved. The detailed one-to-one conductance state update process was demonstrated in Supplementary Fig. 23, demonstrating the characteristic of controllable and uniform weight programming. The above phenomenon emphasizes the intrinsic advantage of OPECTs-based optoelectronic synapses in building optical neural networks over traditional OFETs.

The repeatable analog state programming feature of devices is of great significance in information processing51. Figure 3g depicts the stability of conductance update in three cycles under constant 100 consecutive optical and electrical pulses. After repeated simulation, there was no obvious degradation presented, which ensures the update stability of neural networks. Moreover, 34 distinguishable conductance states with retention more than 60 s were realized in OPECTs array owing to decent photo-induced ion doping (Fig. 3h). We note that the aforementioned photo-induced analog weight programming performance of OPECTs array, to our knowledge, is superior to that of state-of-the-art organic optoelectronic synapses (Fig. 3i), which facilitates accurate and efficient neuromorphic computing.

The working mechanism of non-volatile optoelectronic synaptic behaviors

To understand the formation of non-volatile optoelectronic synaptic behaviors, a three-electrode configuration as shown in Fig. 4a was utilized to study the electrochemical properties of BHJ film under light irradiation23,24. The electrochemical impedance spectrum (EIS) and cyclic voltammetry (CV) tests were depicted in Fig. 4b and Supplementary Fig. 4. At open circuit potential (OCP) conditions, there is a substantial increase in capacitance (C) after illumination. The increase in C of BHJ film at the gate area under illumination causes an additional potential drop at the channel/ion-gel interface, manifested as |VEC*| > |VEC| in Fig. 1d. The BHJ film generated charge carriers when exposed to light, resulting in rapid redistribution of ions and an increase in electrochemical potential, which was reflected by the increase of OCP and electrochemical reaction peak currents. Interestingly, after removing light, the OCP weakened briefly and then remained stable more than 300 s (Fig. 4c), which facilitates the formation of non-volatile persistent photoconductance through continuous ion doping16.

Fig. 4: Mechanism of photo-induced electrochemical doping on OPECTs.
figure 4

a Schematic diagram of the three-electrode setup for investigating the photo-induced electrochemical doping properties of DPPDTT/COTIC-4F-based OPECTs. b EIS analysis of DPPDTT/COTIC-4F coated ITO electrode in the dark and light conditions at OCP states. The insets are a magnified view of the impedance profile and fit circuit. RL is the solution resistance, C is the film capacitance, Rct is the charge transfer resistance, and W is the Warburg impedance. c OCP changes of the polymeric electrode induced by light illumination. d Comparison of ΔIDS responses with positional lighting at channel (I1) and gate (I2). The ratio of ΔIDS activated by irradiating gate (ΔI2) and ΔIDS activated by irradiating channel (ΔI1) is > 7.9. e The effect of the BHJ film/ion-gel structure on device current and retention time. The insert is the schematic of the test setup. f Schematic diagram of the photo-induced electrochemical doping mechanism at high negative gate biases.

Full size image

Since both the channel and gate areas were coated by the same BHJ film, to investigate the driving terminal for photo response of devices, we illuminated the gate and channel regions of the same device selectively and recorded the corresponding photocurrents (Supplementary Fig. 24)24. As shown in Fig. 4d, ΔIDS of illuminating the gate (ΔI2) was 7.9-fold greater than that of illuminating the channel (ΔI1). This phenomenon indicates the BHJ film at the gate area plays a major role in the photo response of devices, which is attributed to the fact that OPECTs exhibit high sensitivity to gate potential variations due to their intrinsic amplification characteristics.

The significant difference between OPECTs and OFETs is the difference in dielectric layer. We designed a two-terminal device configuration shown in Fig. 4e and Supplementary Fig. 25 to investigate the influence of the ion-gel layer on the photocurrent. Under a constant voltage of −1.2 V, we measured the NIR response behavior of two devices by applying a photo pulse (1000 nm, 5 mW/cm2, 10 s). As shown in Fig. 4e, the device with ion-gel demonstrated a higher response current than that of the device without ion-gel (ΔI1 > ΔI2). In addition, after illumination, ΔI2 rapidly decayed to about 0 μA, while ΔI1 firstly decayed and then remained stable even after 60 s, which was induced by anion doping from ion-gel to the BHJ film. The aforementioned results demonstrate the BHJ/ion-gel structure contributes to the device’s non-volatile photocurrent performance through photo-induced electrochemical doping process.

To sum up, as shown in Fig. 4f, the generation of photogenerated charges in the BHJ film activated by light will cause anion doping into the channel under a high negative gate bias, accompanied by cation doping into the BHJ film on top of the gate electrode. While the ion de-doping after light off will be weakened due to the blocking of the surrounding crystallites, bulky semiconductor side chains, and the solvation effect of ion-gel, leading to non-volatile behavior in the conductance update process24. As a medium for effective gate voltage regulation, illumination can easily change the doping state of the channel, resulting in remarkable non-volatile photo response behaviors, which makes OPECTs an ideal platform for neuromorphic vision system.

Demonstration of image memorization and visualization

Take advantage of the persistent memory effect for NIR optical information, our OPECTs array can realize the recognition and memorization of NIR patterns as an artificial bio-inspired vision system52. NIR light (1000 nm, 5 mW/cm2, 20 s) was illuminated through the “note” mask on a region of 7 × 7 in an OPECTs array (Fig. 5a). IDS of the exposed devices was significantly increased to realize the mapping of the “note” pattern. Due to the decent uniformity of the devices, ΔIDS in each exposed pixel was uniform, providing a strong contrast with the unexposed area. After light off, the “note” pattern could still be clearly displayed after 600 s, demonstrating impressive NIR light memory capability (Fig. 5b), which is superior to the memory level of most organic optoelectronic synapses53,54. By utilizing the wavelength selectivity of devices, the OPECTs array can filter out optical noise of a specific wavelength to achieve pattern recognition. As depicted in Fig. 5c, a 3 × 3 array was utilized for NIR image processing. A letter “H”, corresponding to irradiated NIR pattern (1000 nm, 2 mW/cm2, 5 pulses), blurred by introducing UV noises (400 nm, 2 mW/cm2, 5 pulses) (Fig. 5d), were applied to the array and then recorded the current of each pixel. The mapping images of IDS were presented in Fig. 5d, e. IDS triggered by UV noises is below 2 μA while IDS triggered by NIR signals exceeded 2 μA easily. According to the difference of IDS in each pixel, the NIR pattern “H” could be selectively recognized.

Fig. 5: Image memorization and visualization.
figure 5

a Schematic display of musical note image sensing and memorization based on our OPECTs array (1000 nm, 5 mW/cm2, 20 s). b IDS mapping varies with different decay times after light off. c Schematic diagram of NIR pattern visualization by an OPECTs array containing 9 devices. d NIR images of “H” and corresponding current mapping with noise pulses (400 nm, 2 mW/cm2) and signal pulses (1000 nm, 2 mW/cm2). e IDS of nine pixels triggered by irradiation of image “H”. f Image of NIR pattern visualization system. g The mapping of LED array for patterns of “S, H, T, J, U” after irradiating the corresponding position of the OPECTs array.

Full size image

Although the NIR pattern can be easily identified by the magnitude of IDS reflected on each pixel, weak noise current still exists. To further enhance the information preprocessing capability of OPECTs array, we integrated LED matrix one-to-one at the back end of the OPECTs array through the series-connected amplifier circuits55. The aforementioned visualization system was shown in Fig. 5f. When the output voltage applied to a LED exceeded 2 V, the LED lighted up. Utilizing the threshold voltage switching characteristic of LED array to filter out noise, our system could achieve high-contrast visualization of NIR pattern information. We successfully realized the visualization of the initial letters “S, H, T, J, U” of Tongji university in Shanghai (Fig. 5g). Moreover, the pattern brightness of LED array remained unchanged after 60 s, facilitating the storage and reuse of image information.

In addition, OPECTs have potential in the field of light-controlled inverters. We demonstrated a NIR OPECTs inverter array by similar photolithography strategy, achieving a gain of 12.5 at VDD = 1.5 V. Light illumination can cause a shift in the VTh of devices, which induces changes in the switching voltage of inverters. As shown in Supplementary Fig. 26, the switching voltage of an inverter was reduced by applying NIR light, which provides a promising avenue for the development of light-controlled logic circuits56.

Image processing and classification by OPECTs-based convolutional neural networks

In view of the exciting multi-level linear conductance programming capabilities of OPECTs, our OPECTs array is expected to construct reconfigurable visual neural networks for high-level information processing and classification57,58. Using optoelectronic signals as a means of device weight programming to construct reconfigurable convolution kernels, the preprocessing of the original image such as feature extraction and contrast enhancement can be achieved. Figure 6a and Supplementary Fig. 27 demonstrate that five 3 × 3 convolution kernels are utilized for outline extraction of a captured picture including the sounding houses and a tree. The weight setting of kernels was detailed in Supplementary Note 2. By mapping the convolution kernel weights to the pixel value at each position and performing vector-matrix multiplication, specific pixel information can be selectively enhanced or suppressed. A simple kernel could roughly extract the outline of the image, unfortunately, many details were ignored (Supplementary Fig. 27a)59,60. To further increase the extracted details, four additional kernels were designed to extract the upper, bottom, left, and right edge details, respectively. Afterwards, the four images were accumulated and strengthened to generate a combined outline image containing more accurate details while removing unnecessary noises.

Fig. 6: CNN simulation for image processing and classification.
figure 6

a Reconfigurable kernels for image processing. b Schematic of MNIST digits recognition processes based on CNN system, including feature extraction and classification. c Accuracy of different systems. d The variation of accuracy and difference between the OPECTs-based CNN and “Original” systems when σ increased from 0 to 0.6. e Confusion matrix of OPECTs-based CNN system (σ = 0).

Full size image

Given the impressive image preprocessing and linear weight programming functions of OPECTs array, we constructed a complete CNN system conceptually for the classification of handwritten digits in the Mixed National Institute of Standards and Technology (MNIST) database, involving the front-end 3 × 3 convolutional kernels for image feature extraction and a back-end fully-connected neural network for image classification (Fig. 6b). To verify the advantages of our OPECTs-based CNN system in information classification, we added gaussian noise (σ = 0.3) to the original database. The “Original” system only utilized the fully-connected neural network based on OPECTs for recognition without front-end kernels for information preprocessing, while the “Ideal” and OPECTs-based CNN system first enhanced the image contrast and reduced the noises through kernels before back-end classification. In our designed CNN system, the grayscale value of each pixel in the original digit image was encoded by mapping it to the convolution kernels. In each computing epoch, after training and testing the encoded data through the back-end network, the weight gradient was updated based on the test results through the back-propagation algorithm. The recognition accuracy of different systems was depicted in Fig. 6c. The total accuracy of CNN system based on ideal neural networks (“Ideal”) reached 98.03%. Notably, thanks to linear weight programming capability of OPECTs, after 40 training epochs, the accuracy of OPECTs-based CNN system was higher than the “Original” system, reaching 93.29% (σ = 0) and 89.37% (σ = 0.3), while the accuracy of the “Original” system was down to 91.99% (σ = 0) and 83.01% (σ = 0.3). After kernels for preprocessing, the contrast of the images is significantly enhanced, especially for images with high σ (Supplementary Fig. 28). Notably, the accuracy difference between the OPECTs-based CNN and “Original” systems was further enhanced to 10.69% when σ increased from 0 to 0.6 (Fig. 6d), proving the impressive preprocessing ability of CNN kernels. Meanwhile, the confusion matrix with σ = 0 and 0.3 was shown in Fig. 6e and Supplementary Fig. 29. The recognition results of most digits by OPECTs-based CNN system were also improved compared with that by the “Original” system. The accuracy of 93.29% for handwritten digit recognition is comparable to or better than that of the previously reported works (Supplementary Fig. 30 and Supplementary Table 3). Overall, the aforementioned results emphasized the remarkable performance of OPECTs array-based CNN kernels and neural networks, demonstrating the feasibility of OPECTs-based CNN system for neuromorphic visual computing.

To further showcase the potential of our OPECTs-based neuromorphic vision system for processing images in complex environments, we conducted an object tracking simulation using the open-source night object tracking (NOT-156) dataset, as shown in Supplementary Fig. 31 and Supplementary Table 4. Leveraging the non-volatile weight programming capability across the vis to NIR light spectrum, our OPECTs system successfully performed target contour detection in dim conditions, which demonstrates the viability of OPECTs-based neuromorphic vision system for robust target detection in challenging environments.

Discussion

In conclusion, we developed the NIR (1000 nm) OPECTs array based on organic BHJ film, charactered by linear weight programming (NL = −0.015), large modulation range (47.3), multi-level storage memories, and long retention time. Based on NIR photo-induced electrochemical doping mechanism, various non-volatile synaptic behaviors were simulated, exhibiting pulse number and wavelength-dependent plasticity. By applying compatible photolithographic methods, OPECTs arrays were successfully fabricated without sacrificing the photosensitive properties. Thanks to good array uniformity, we utilized OPECTs arrays for NIR pattern imaging and memorization. Moreover, both in-sensor noise filtering and NIR image visualization functions were accomplished by combining an OPECTs array with an LED array. As a proof of concept, an OPECTs-based CNN system was built for the recognition of noisy handwritten digits with high accuracy, which highlights the potential of our OPECTs array for future neuromorphic visual computing. This work partly addresses the current bottlenecks of organic optoelectronic synaptic devices, including nonlinear weight programming, limited storage memories, and difficulty in large-scale fabrication, advancing the development of neuromorphic visual system for AI data processing.

Methods

Materials availability

The DPPDTT and COTIC-4F semiconductors were purchased from Hangzhou Order Science & Technology Co., Ltd. and used without any further purification. The chrome bars and gold particles were obtained from ZhongNuo Advanced Material (Beijing) Technology Co., Ltd. The poly(ethyleneglycol) diacrylate (PEGDA, Mn = 575) was obtained from Sigma-Aldrich. 2-hydroxy-2-methylpropiophenone (HOMPP, purity ≥ 98%) and 1-ethyl-3-methylimidazolium bis(trifluoromethylsulfonyl)imide ([EMIM]+[TFSI]−, purity ≥ 99%) were purchased from Adamas-beta®.

Fabrication of OPECT arrays

The bottom S/D/G electrodes were patterned on Si/SiO2 substrate by photolithography with the assistance of AZ5214E, and then Cr/Au (8/35 nm) layers were evaporated onto the formed template. DPPDTT and COTIC-4F hybrid precursor (w./w., 1:2, 15 mg/mL in chloroform in total) was spin-coated on the substrate (800 rpm/min) with patterned electrodes and treated at 100 °C on a hotplate in inert gas atmosphere. The prepared substrate was afterwards spin-coated with AZ5214E (3000 rpm/min) and patterned with 365 nm light to form an accurate protective layer in channel and gate regions. After plasma etching for 30 min to remove the redundant semiconductor outside the capping layer, the device was immersed in ethanol for 20 s, which could dissolve the protective AZ5214E and be orthogonal to semiconductor layer. Finally, ionic precursor composed of PEGDA, HOMPP, and [EMIM]+[TFSI]− with a weight ratio of 8:4:88 was patterned to form the patterned ion-gel layer.

Calculation of V Th

The value of VTh was extracted by the following equation:

$${{I}^{\frac{1}{2}}}_{{DS}}={\left(\frac{{{\rm{\mu }}}{C}^{*}{dW}}{2L}\right)}^{\frac{1}{2}}({V}_{{GS}}-{V}_{{Th}})$$
(1)

Where IDS stands for the saturated drain-source current; W, L, and d are the width, length, and thickness of the channel, respectively, C* is the capacitance per unit volume of the channel, μ is the saturated mobility. In IDS – VGS curves, the intersection of the tangent of the curve with the VGS axis when IDS1/2 equals zero is the value of VTh.

Algorithm simulation process

  1. (1)

    For front-end image preprocessing with kernels: Firstly, the original images composed of 28 × 28 pixels based on MNIST dataset were mapped with the 3 × 3 gaussian kernels, which applies a smoothing filter that reduces noise and blurs fine details by weighting nearby pixels according to a gaussian (normal) distribution. Then, vector-matrix multiplication on the pixel value of the original images and the weight of the gaussian kernels was performed to obtain the output pixel data, and the data was converted into.mat format for subsequent classification tasks through MATLAB. The weight configuration of a gaussian kernel is as follows:

    $${K}_{{gaussian}}=\left[\begin{array}{ccc}\frac{1}{16} & \frac{2}{16} & \frac{1}{16}\\ \frac{2}{16} & \frac{4}{16} & \frac{2}{16}\\ \frac{1}{16} & \frac{2}{16} & \frac{1}{16}\end{array}\right]$$
    (2)
  2. (2)

    For back-end image classification with a single-layer ANN algorithm: The processed image data was transferred to the input layer of a single-hidden-layer feedforward neural network. The value of NL was calculated using Python by fitting the LTP/LTD curves in Fig. 3f with the following weight update formula:

$${G}_{n+1}={G}_{n}+{\Delta G}_{P}={G}_{n}+{{{{\rm{\alpha }}}}_{P}e}^{{-{NL}}_{P}\frac{{G}_{n}-{G}_{\min }}{{G}_{\max }-{G}_{\min }}}({G}^{+}{or}\,{G}^{-}\uparrow )$$
(3)
$${G}_{n+1}={G}_{n}+{\Delta G}_{D}={G}_{n}-{{{\rm{\alpha }}}}_{D}{e}^{{-{NL}}_{D}\frac{{G}_{\max }-{G}_{n}}{{G}_{\max }-{G}_{\min }}}({G}^{+}{or}\,{G}^{-}\downarrow )$$
(4)

Here, Gn and Gn+1 represent the synaptic conductance after applying the nth and n + 1th pulses. The parameters α and NL reflect the value of conductance change and nonlinearity. Unfortunately, the limited conductance update states in a device are insufficient for the simulation of the weight update process in MATLAB. Therefore, we used the conductance differences of six OPECTs to model synaptic weights to address the aforementioned issues, obtaining the weight states of 50 × 50 × 50 = 125,000. In this way, W1,1 could be calculated by the following equation:

$${W}_{1,1}=\frac{{50}^{2}\times ({{G}_{2}}^{+}-{{G}_{2}}^{-})+{50}^{1}\times ({{G}_{1}}^{+}-{{G}_{1}}^{-})+{50}^{0}\times ({{G}_{0}}^{+}-{{G}_{0}}^{-})}{({50}^{2}+{50}^{1}+{50}^{0})\times ({G}_{\max }-{G}_{\min })}$$
(5)

The single-hidden-layer feedforward neural network is trained by the error backpropagation algorithm, and then the gradient descent method is used to modulate the synaptic weight.

The “Ideal” neural network was simulated by Python, and the “Original” and OPECTs-based CNN neural network were simulated by Python and MATLAB.

Characterization

The UV-vis-NIR transmission and absorption spectra was tested by a spectrometer (Agilent Technologies Co., Ltd., Cary 60). AFM thickness image was obtained using Dimension Icon (Bruker Inc). The EIS and CV measurements were implemented by an electrochemical workstation (CHI600E, CH Instruments Ins., USA). All electrical measurements were tested by semiconductor characterization systems (PDA Fs-Pro and Keithley 4200A SCS) in the atmosphere. Various light sources under different wavelengths and intensities were provided by a dual grating monochromator (Omno 330150, Beijing NBeT) and Xenon lamps. NL of LTP/D curves was calculated by Python.