Introduction

The global market of machine vision was validated at USD 16.89 billion in 2022, which is expanding with an annual growth rate of 12.3% (2023–2030)1. Machine visions are profoundly prevailing in autonomous vehicles, medical diagnosis, video processing, iris encryption, cyborgs, etc. However, traditional visual systems are bulky, energy-intensive, and lengthy, especially for cognitive tasks owing to discrete computation hierarchy2. The sensory, memory, and computing units are separated from each other. The location and format (analog/digital) of image data need to be changed frequently, rising the penalty of energy consumption and time delay (Fig. 1a). To solve this issue, mimicking human visual systems is a promising strategy3,4. In human visions, photoreceptors and neurons in the human retina detect and pre-process images that are later sent to the visual cortex for cognitive signal processing. For current cutting-edge machine visions, intelligent photodetectors can simultaneously sense and pre-process light stimuli like a human retina5,6. Received images are then directly transferred into artificial neural networks (ANNs) for complex visual processing. ANNs fundamentally imitate the fundamental principles of human brains relying on the activities of synapses and neurons, which can realize huge parallel computing and high-energy efficiency (Fig. 1b)7,8. However, human eyes only provide very limited photodetection abilities regarding the light wavelength (380 nm to 700 nm), intensity (comfortable intensity for human eyes: 200 lux to 750 lux), and vector (not sensitive to linear and circular polarity)9,10. This significantly restrains the application of cutting-edge humanoid machine vision. Notably, interspecies-chimera machine vision seamlessly integrates the cognitive function of humans and the special visual function of other species (Fig. 1c). This can leverage functions beyond traditional artificial intelligence (AI). Machines solve problems like humans while possessing super-human capabilities enabled by other appealing functions11.

Fig. 1: Interspecies-chimera machine vision.
figure 1

a The architecture of traditional machine vision systems. The system is bulky with separated sensing, analog-to-digital converter (ADC), memory, and computing units, resulting in a high energy budget and slow computing speed. b The architecture of humanoid machine vision systems. The system shows limited sensing ability, including the narrow bandwidth, small margin of signal density, and lack of polarimetric sensitivity. c The architecture of interspecies-chimera machine vision systems. This can complete cognitive tasks like the intelligence species of humans and other functions. d Schematic of an interspecies-chimera visual system. It integrates the ommateum from the honeybee and the cognitive visual cortex from human beings. e Schematics of an application scenario of autonomous vehicles with the interspecies-chimera machine vision. Real-time navigation and anti-glare recognition can be achieved simultaneously.

Full size image

Humanoid machine vision has been reported to reduce background noise12, scotopic/photopic adaptation13, broadband sensing14,15, convolution processing16,17, etc. In the human vision system, the image information is received and pre-processed by the retina and transferred into the visual cortex region for cognitive processing (Fig. 1d)18. Human vision, although, is capable of cognitive tasks. It is not sensitive to the polarization of light which includes critical information19. It is rudimentary to integrate polarimetric functions into humanoid machine vision. Honeybees are known as excellent navigators with the help of ommateum measuring polarized lights. The skylight pattern is determined by the position of the sun, described by the Rayleigh sky model (Supplementary Fig. 1)20. For instance, real-time navigation can be achieved via monitoring the sun-position-related celestial polarization cues21. Indeed, the polarimetric capability makes honeybees a master for navigating from the honeycomb to a flower patch. But honeybees are not intelligent species like humans. Therefore, a rational design philosophy is to integrate the polarimetry from honeybees and the intelligence from humans, creating an interspecies-chimera machine vision (Fig. 1d). Particularly, the reflected light from shiny surfaces, including cars’ windows, buildings, water on the road, and the road itself, forms glare spots. This compromises the processing accuracy in traditional machine visions due to glare-induced distortion22. The formation of glare spots is because the reflected light is partially polarized light parallel to the surface. The polarimetric function can suppress the light intensity with a certain polarization (glare spots) and keep high fidelity in authentic images. One of the promising applications of the design is autonomous vehicles (Fig. 1e). This interspecies-chimera machine vision, on the one hand, can detect the polarization pattern in the sky for real-time navigation. On the other hand, the system can realize anti-glare pattern recognition.

Van der Waals (vdW) heterostructures are promising for optical sensing, memory, and computing. Two-dimensional (2D) materials have excellent optoelectronic properties, atomically thin thickness, high-carrier mobility, and tuneable electrical transports23. In particular, 2D materials with in-plane anisotropic structures can realize polarimetric functions to lights24. Although other materials are also sensitive to polarized lights, such as metamaterials25, anisotropic halide perovskites26, anisotropic and polymer crystals27. 2D polarimetric devices utilizing atomic-level anisotropy are compatible with complementary metal-oxide semiconductor (CMOS) fabrication techniques. They have the potential to realize high-density integration and excellent scalability28. Furthermore, 2D materials exhibit good memory, and computing capabilities, which are easier to fuse with polarimetry functionalities29. Multi-terminal memtransistors based on vdW heterostructures can achieve programmable optoelectronics and complex functions of hetero-synaptic plasticity that can be used for brain-inspired neuromorphic computing30,31.

Among the 2D materials library, ReS2 and GeSe2 with excellent optoelectronic properties have been employed for memory and neuromorphic vision systems. Mo-irradiated ReS2 (Mo-ReS2) showed promising resistive switching under electronic stimuli32. Photoconductivity was also investigated in ReS2-based devices, indicating the potential for machine vision33. Interestingly, the polarization sensitivity of ReS2 was also investigated for neuromorphic vision with 3D visual polarization imaging and visual adaptation abilities34,35. Previous work has also confirmed the good performance of GeSe2 due to excellent photosensitivity and charging conducting features36,37. Resistive switching behaviors that are crucial to neuromorphic computing systems were observed in GeSe238. The tuneable electrical properties and photosensitivity make the two materials desirable for machine visions. Optical-controlled 2D devices for machine vision have been summarized (Supplementary Table 1). A multifunctional device with polarization sensitivity, long-term nonvolatility, and positive/negative reconfigurable photoconductance simultaneously has few investigations and is very promising. This can seamlessly integrate polarization sensitivity, light sensing, memory, and neuromorphic computing, which will leverage AI applications.

In this work, a vdW heterostructure device employing 2D ReS2 and GeSe2 with anisotropic structures was developed. The optically-controlled polarimetry memtransistor (OCPM) can realize optically-programed non-volatile states and has the sensitivity to polarized lights. Both positive photoconductivity (PPC) and negative photoconductivity (NPC) under the light of 808 nm and 405 nm were obtained respectively. This can achieve continuous transition and a similar conductance range in both potentiation and depression operations, demonstrating better computing accuracy (Supplementary Fig. 2). The behaviors mimicked the antagonistic shunting and memory of bipolar cells, demonstrating the sensing function of the retina and the computation function of the visual cortex. The conductance of the device can be gradually increased and decreased by light stimuli, corresponding to long-term potentiation and depression. This can be used to construct artificial neural networks (ANNs) for neuromorphic computing. The polarization sensitivity was used for real-time navigation inspired by honeybees. Experimental values were consistent with actual values, indicating practicable accuracy. Besides, the polarimetry was integrated with cognitive neuromorphic seamlessly to realize anti-glare pattern recognition. The OCPM array with anti-glare abilities decreased the energy budget to 11% compared to the neuromorphic vision without polarimetry. This interspecies-chimera machine vision provided a concept to design AI systems and will benefit autonomous vehicles, medical diagnoses, holography techniques, intelligent robotics, etc.

Results

Materials characterizations of OCPM device

The stacked ReS2/GeSe2 layers were employed to construct the OCMP (Fig. 2a). The source-drain current (Ids) flowed along the b-axis direction of both ReS2 and GeSe2 layers39,40. The ReS2 has a distorted 1T structure deriving from a hexagonal structure (Supplementary Fig. 3). Meanwhile, GeSe2 is the two-dimensional crystal with the monoclinic crystal structure (Supplementary Fig. 4). ReS2 and GeSe2 have in-plane anisotropic structures and direct bandgaps, indicating excellent bi-axial optical and electrical properties41,42. Polarization-sensitive absorption spectra were measured to further investigate the bi-axial optical properties. In the experimental setup (Supplementary Fig. 5), a linear polarizer was utilized to polarize the irradiated light. For the initial state (rotation angle = 0°), the polarization direction of irradiated light was parallel with the b-axis direction of measure materials. To acquire the polarization-sensitive absorption spectra mapping, the sample was rotated clockwise by an angle of 10° for each step. Polarization-sensitive absorption spectra mapping of the ReS2 layer was recorded (Fig. 2b). The results presented an angle-dependent absorption feature, indicating linear dichroic effects43. Notably, exciton absorption peaks at around 800 nm were observed when the rotation angles were 0° and 180° compared to the 90° and 270° counterparts44. Typical polarization-sensitive absorption spectra of ReS2 under the rotation angle of 0° and 90° were extracted (Supplementary Fig. 6). A higher absorption coefficient was observed when the polarized light was parallel with the b-axis of ReS2 ([010] direction of the ReS2 crystal) compared to the perpendicular counterpart. Besides, the blueshift feature of absorption spectra was found when the rotation angle increased from 0° to 90° (Supplementary Fig. 6). This implied the anisotropic electronic properties. The polarization-sensitive absorption spectra mapping of the GeSe2 layer was also measured (Fig. 2c). An angle-dependent absorption feature was presented with linear dichroic effects. Typical polarization-sensitive absorption spectra of GeSe2 under the rotation angle of 0° and 90° were extracted (Supplementary Fig. 7). The material showed higher absorption ability when the polarized light was parallel with the b-axis of the GeSe2. The bi-axial optoelectronic properties of ReS2 and GeSe2 are the fundamentals of polarimetric capabilities.

Fig. 2: Material characterizations of OCPM device.
figure 2

a Schematical diagram of the OCPM. b Polarization-sensitive absorption spectra mapping of the ReS2 layer. EAP means exciton absorption peak. The rotation angle is with respect to the b-axis in-plane clockwise. c Polarization-sensitive absorption spectra mapping of the GeSe2 layer. The rotation angle is with respect to the b-axis in-plane clockwise. d Typical angle-resolved polarized Raman spectra mapping of the ReS2/GeSe2 bi-layer. e The angle-dependent Raman intensity of the A1g mode for ReS2. f The angle-dependent Raman intensity of the A1g mode for GeSe2. g The cross-section of the ReS2/GeSe2 bilayer and the corresponding element distribution. h The fast Fourier transform (FFT) pattern of ReS2. ZA means zone axis. i The fast Fourier transform (FFT) pattern of GeSe2.

Full size image

To confirm the anisotropic characteristic of ReS2 and GeSe2, the Raman spectra of the materials were measured. The typical Raman spectrum of the ReS2/GeSe2 double layer was shown (Supplementary Fig. 8). The observed peaks all belong to the used materials as labeled. The peak located at 520 cm−1 was attributed to the substrate of Si. The typical peaks of ReS2 were located at 149 cm−1 and 209 cm−1 corresponding to the in-plane E2g mode and A1g modes respectively45. The A1g mode of GeSe2 was found at 194 cm−1 46. The angle-resolved polarized Raman spectra of the ReS2/GeSe2 bi-layer were measured for investigating anisotropic properties (Fig. 2d). The ReS2/GeSe2 vdW heterostructure with aligned b-axis direction was placed on a SiO2/Si substrate (Supplementary Fig. 9). The b-axis direction of stacked GeSe2/ReS2 layers was parallel with the polarized light at the beginning (rotation angle = 0°) of the measurement. Then the sample was rotated by the step of 10° (GeSe2/ReS2 layers rotate together). The corresponding Raman spectra were collected at each step. The angle-resolved polarized Raman spectra mapping is depicted (Fig. 2d). Raman signal amplitudes changed periodically with the increased rotation angles, meaning an anisotropic structure in the in-plane dimension47. The classical Placzek model gave an insight understanding to the anisotropic structure. The Raman signal intensity is described by the following equation48:

$${{\bf{S}}}\propto {\left|{{{\bf{e}}}}_{{{\bf{i}}}}\cdot {{\bf{R}}}\cdot {{{\bf{e}}}}_{{{\bf{s}}}}\right|}^{2}$$
(1)

where S is the Raman signal intensity, ei is the unit polarization vectors described by Eq. (2):

$${{{\bf{e}}}}_{{{\bf{i}}}}=\left[0\,\cos \theta \sin \theta \right]$$
(2)

R represents the Raman tensor described as a 3 × 3 matrix49:

$${{\bf{R}}}=\left[\begin{array}{c}a \quad 0 \quad 0\\ 0 \quad b \quad 0\\ 0 \quad 0 \quad c\end{array}\right]$$
(3)

es means the unit polarization vectors of scattered Raman signals under the parallel or cross-polarization modes. The parallel mode was used in this work, which is described as Eq. (4):

$${{{\bf{e}}}}_{{{\bf{s}}}{{\Vert }}}={\left|0 \, {\mathrm{cos}}\theta \, {\mathrm{sin}}\theta \right|}^{T}$$
(4)

where θ gives the rotation angle. The angle-dependent Raman signal intensity for Ag mode under the parallel working mode is described by Eq. (5):

$${{{\bf{S}}}}_{{{{\bf{A}}}}_{{{\bf{g}}}}}^{{\Vert }}\propto {\left|b{\cos \theta }^{2}+c{\sin \theta }^{2}\right|}^{2}$$
(5)

The angle-dependent Raman intensity of the A1g mode was studied. The experimental and fitting plots of ReS2 (Fig. 2e) and GeSe2 (Fig. 2f) are presented. Notably, the experimental data points are fitted well with theoretical values. A signature periodic variation feature of 180° was observed for both ReS2 and GeSe2 (Fig. 2e, f). This clearly demonstrating a bi-axial anisotropic structure50. Besides, the angle deviation of the two materials’ orientation (b-axis direction) was merely 6°, which indicates very good alignment.

The thickness of ReS2 and GeSe2 layers were identified by the Atomic Force Microscopy (AFM) (Supplementary Figs. 10 and 11). The thickness of ReS2 and GeSe2 for constructing the OCPM were approximately 42 nm and 34 nm, respectively. Meanwhile, the high-angle annular dark-field (HAADF) image was measured to analyze the cross-section of the stacked ReS2/GeSe2 structure (Fig. 2g). The laminate structure was observed in both ReS2 and GeSe2 layers. The Re, S, Ge, and Se were distributed uniformly in the corresponding regions in the energy disperse X-ray spectroscopy elemental mapping (Fig. 2g). No other impurity elements were found (Fig. 2g). The HAADF atomic scale image of ReS2 indicated a high crystallinity of the exfoliated ReS2 (Supplementary Fig. 12). The interlayer spacing was approximately 0.64 nm along the [001] direction. The atomic scale HAADF image of GeSe2 also demonstrated a crystalline feature with the interlayer spacing of 0.59 nm (Supplementary Fig. 13). Meanwhile, the fast Fourier transform (FFT) patterns were obtained to get more insight into crystalline structures. In the FFT pattern of ReS2 crystalline, clear spots were observed in the reciprocal space (Fig. 2h), indicating the single crystalline structure. The FFT spots were consistent with the distorted 1 T structure51. The angle between \((00\bar{2})\) and \((\bar{1}0\bar{2})\) reflections in ReS2 was 53.73° which was 11.08° deviating from the angle between \((00\bar{2})\) and \((10\bar{2})\) reflections (42.65°). The angle between crystal planes reflections of \((00\bar{2})\) and \((\bar{1}00)\) in ReS2 was 99.98° which was 19.95° bigger than that of \((00\bar{2})\) and \((100)\) (80.03°)52. This confirmed a strong in-plane anisotropy53. Meanwhile, the corresponding FFT pattern along the [100] zone axis of GeSe2 was detected (Fig. 2i). The monoclinic β-GeSe2 was identified by the FFT pattern37. The angle between crystal planes reflections of \((00\bar{2})\) and \((040)\) was 89.62°, which was smaller than that of \((00\bar{2})\) and \((0\bar{4}0)\) (90.38°), demonstrating an anisotropic structure42. Both optical spectra and atomic structure anisotropies of ReS2 and GeSe2 were characterized, which are the fundamentals of bi-axial optoelectronics in the OCPM.

Optoelectronic properties and working mechanism of OCPM

Optoelectronic properties and working mechanism of the OCPM have been systematically investigated. The light-wavelength-dependent transfer curves (Supplementary Fig. 14) were measured to identify the operation gate voltage. The light illuminations with the wavelengths of 405 nm and 808 nm were applied to the OCPM individually. The light power was low to avoid potential photo-induced defects54. The device was only exposed to the power density lower than 100 mW/cm2 referring to light sensing application based on GeSe2 and ReS234,35,37,55. The gate voltage effectively modulated the photoconductivity under illumination with different wavelengths. Both positive photocurrents and negative photocurrents were observed with respect to the dark current under a certain gate voltage slot (approximately −14.1 V to 1.4 V) as marked by a gray area (Supplementary Fig. 14). This enabled fully optical-controlled information processing with artificial neural networks (ANN)56. The Ids-Vds photocurrent output trace has been measured (Supplementary Fig. 15). It showed both positive and negative photo responses under different light illuminations. The source-drain currents increased with the increase of gate voltage, demonstrating the n-type characteristic in the channel57. The I-t plot under an unpolarized pulsed light stimulation (808 nm wavelength) indicates non-volatile positive photoconductivity (PPC) properties (Fig. 3a). The initial conductance was different before the shining of light (Fig. 3a vs 3b). The Gds increased quickly as the light shining (808 nm) on the device. Particularly, the current remained at a larger value than the original dark state value after turning off the light illumination. This is a signature character of non-volatile functions58. In comparison, after the device was programmed to the high conductivity by 808 nm, the conductance decreased when the light with the wavelength of 405 nm was shined on the device, and the lowered conductance was remained after the illumination, which meant the non-volatile negative photoconductivity (NPC) (Fig. 3b). This phenomenon indicated a non-volatile negative photoconductivity (NPC). The retention time of the device programmed by pulsed light was measured (Supplementary Fig. 16). The programmed currents remained for approximately 1000 s without any signs of degeneration. The co-existence of non-volatile PPC and NPC shows promising prospects for cognitive information processing based on ANNs59. The output currents can be modulated gradually, corresponding to potentiation and depression behaviors (Fig. 3c). The programmable conductance can represent synaptic weights in ANNs, implementing the matrix-vector multiplication (MVM) for deep learning algorithms. The architecture used lights to program synaptic weights. The photonic device increases processing speed owing to high bandwidth, lowered parasitic crosstalk, and achieves ultralow power consumption comparing with electrical counterpart60. Minimizing the asymmetric nonlinearity (ANL) of weight updating (potentiation/depression) is crucial for computing accuracy. A small ANL of 0.19 was obtained in the OCPM (Supplementary Fig. 17)61. The feature was comparable with the electronic-controlled artificial synapses62. Meanwhile, further analysis on potentiation/depression programming was implemented63,64,65. The typical non-volatile conductance changes were programmed by light pulses (Supplementary Fig. 18a). The ΔG versus G0 over 30 cycles were plotted. ΔG is the increase/decrease of conductance under light illumination, and the G0 is the initial state before the programming operations63,64. The potentiation and depression are depicted (Supplementary Fig. 18b, c). The inset shows the Normal Probability Plot. The mean value μ of the potentiation and depression were 7.16266 and -7.86515 respectively, demonstrating a near-symmetric feature. The standard deviation (σ) was 5.46145 and 6.67841 for potentiation and depression respectively. The device was operated for over 30 cycles without obvious variation of conductance (Supplementary Fig. 19a). Furthermore, the optoelectronic performances of fresh and three-month-old parts were compared (Supplementary Fig. 19b). A negligible difference was observed, indicating the excellent stability of the device. In particular, the OCPM also endowed polarization-sensitive potentiation/depression abilities (Fig. 3c). This can realize image preposing with polarimetric functions and in-sensor computing simultaneously, which goes beyond traditional humanoid machine vision.

Fig. 3: Optoelectronic properties and working mechanism of ReS2/GeSe2 heterostructures OCPM.
figure 3

a I-t curve demonstrating non-volatile PPC (Vgs = −10 V). The illumination intensity of 808 nm wavelength was fixed as 19 mW/cm2. This power was used for all the rest of PPC operations without special instructions. b I-t curve demonstrating non-volatile NPC (Vgs = −10 V). The intensity illumination with 405 nm wavelength was fixed as 19 mW/cm2. This power was used for all the rest of NPC operations without special instructions. c Fully optical-controlled potentiation/depression with polarimetric functions (Light pulse width 10 ms, Vgs = −10V, Vds = 0.1 V). d Photocurrent changes (ΔIds) under different light PNs. e Photocurrent changes (ΔIds) under different light PWs. f Photocurrent changes (ΔIds) under different light PP. g Density functional theory calculated band structure of the ReS2/GeSe2 heterojunction. h–l The schematic diagram of the physical mechanism underlying the PPC and NPC performances.

Full size image

Moreover, conductance modulations under multiple optical pulses with different pulse numbers (PN), pulses widths (PW), and pulse power (PP) were characterized (Fig. 3d–f). Identical neurological responses to different stimuli are essential in biological brains associating to learning and memory66. More stimuli, higher frequent stimuli, and stronger stimuli trigger more intensive or more suppressive neurological responses and feedback. Bidirectional synaptic excitation and inhibition are the foundation for information processing in brains. Pyramidal cells in the primary visual cortex need to equalize excitation/inhibition ratios for visualization tasks67. Conductance changes (ΔGds) under different light PNs were tested (Fig. 3d). The absolute ΔGds increased monotonically with the increase of PNs (PN from 1 to 50). The ΔGds of PPC increased from 0.41 μS to 1.44 μS (Fig. 3d). The absolute ΔGds of NPC increased from 0.17 μS to 1.73 μS, which indicates spike-number-dependent plasticity (SNDP) mimicking biological systems (Fig. 3d). Meanwhile, the PPC induced by 808 nm light was more obvious with longer pulse widths (PW varied from 10 ms to 10 s) (Fig. 3e). ΔGds was 0.1 μS under the shining of a 10 ms pulse (Fig. 3e). In comparison, ΔGds enlarged to 1.24 μS with a pulse width of 10 s (Fig. 3e). The 405 nm light also induced SRDP phenomena regarding the NPC behavior. The absolute values of ΔGds increased from 0.02 μS (PW of 10 ms) to 0.92 μS (PW of 10 s) (Fig. 3e). The limit of response speed was evaluated by shining the 808 nm light with the pulse width of 0.1 ms (Supplementary Fig. 20). 0.1 ms was the fastest optical pulse that is limited by our current equipment. The non-volatile optical response can be potentially faster. The device can be potentially faster. Besides, the mimicking of biological spike-amplitude-dependent plasticity (SADP) was studied (Fig. 3f). Typical PP-dependent increase of ΔGds was measured. The ΔGds of PPC changed from 0.14 μS (1 mW/cm2) to 0.67 μS (61.9 mW/cm2) (Fig. 3f). Meanwhile, the absolute ΔGds of NPC changed from 0.14 μS (0.6 mW/cm2) to 0.28 μS (499 mW/cm2) (Fig. 3f). The light power dependent photocurrent (Iph = Ilight − Idark)68 for both PPC (808 nm) and NPC (405 nm) has been measured (Supplementary Fig. 21). Both NPC and PPC showed increased Abs(Iph) under higher light density. The linear dynamic range (LDR) was calculated as 35.78 dB and 37.89 dB for PPC and NPC respectively. The influence of light intensity on the responsivity and detectivity was characterized (Supplementary Fig. 22). For the PPC under the illumination of 808 nm, the responsivity decreased from 22.24 A/W to 1.77 A/W when the light density decreased from 0.78 mW/cm2 to 48.62 mW/cm2 (Supplementary Fig. 22a). The highest and lowest detectivity of 2.91 ×1010 Jones and 2.31 ×109 Jones were measure under the highest and lowest illumination respectively (Supplementary Fig. 22b). The sensitivity was tending to the saturation status under higher incident intensity. Similar features were observed under the illumination of 405 nm. The highest responsivity of 35.90 A/W was observed when the light density was 0.5 mW/cm2 (Supplementary Fig. 22c). The lowest responsivity (9.16 A/W) was obtained under the high-power illumination (39.21 mW/cm2). Meanwhile, the detectivity decreased from 8.64 × 109 Jones to 2.20 × 109 Jones when the power decreased (Supplementary Fig. 22d). Besides, the 405 nm light can achieve positive photoconductivity by modulating the gate voltage (Supplementary Fig. 23). The tuneable PPC and NPC responses in the OCPM provided more operational freedoms for sensing, memory, and bio-inspired neuromorphic computing. The endurance of the OCPM was tested by programming the device with electronic stimuli (Supplementary Fig. 24). The high resistance state (HRS) and low resistance state (LRS) were achieved by applying a positive gate voltage pulse and a negative gate voltage pulse, respectively. No degradation was observed after 10,000 switching cycles, demonstrating excellent reliability. The retention time of the programmed states was longer than 3000 s (Supplementary Fig. 25).

The working mechanism of the OCPM has been investigated. Two reference devices with a single layer of ReS2 or GeSe2 were fabricated. The two devices only showed positive photo responses. Neither of the devices showed non-volatile and negative photo-responsive features (Supplementary Fig. 26). This confirmed that the vdW heterostructure ReS2/GeSe2 layer is essential for non-volatile memory and positive/negative photoresponsivity. The band structure of the vdW heterostructure dominates the carrier generation, separation, and transportation. This is the foundation of 2D optoelectronic performances. Fermi level, EC, and EV were measured to get more insights into the working mechanism. The Fermi levels of ReS2 and GeSe2 were measured by the Kelvin probe force microscopy (KPFM)69. The deduction of the Fermi level for ReS2 and GeSe2 was −5.135 eV and −5.031 eV, respectively (Supplementary Fig. 27e)70. Besides, ultraviolet photoelectron spectroscopy (UPS) was utilized to measure the position of the valence band maximum (VBM), ionization energy, and work function of the ReS2 and GeSe2 flakes. The bandgap was deduced from the Tauc plot of the UV-visible optical spectroscopy (Supplementary Fig. 28a, d). The bandgap of ReS2 and GeSe2 were around 1.34 eV and 2.66 eV, respectively6. The Fermi levels of ReS2 and GeSe2 were 5.214 and 5.029 eV, which was consistent with the KPFM results (Supplementary Fig. 28b, e). The UPS binding energy profiles of ReS2 (Supplementary Fig. 28c) flake and GeSe2 (Supplementary Fig. 28f) can be used to calculate the energy difference between the valance band and the Fermi level71,72. The VBMs of ReS2 and GeSe2 were −6.024 eV and −6.030 eV, respectively. The CBM of ReS2 (−4.7 eV)73,74 is lower than that of GeSe2 (−3.3 eV)75. The density functional theory (DFT) was employed to calculate the energy band structure of the ReS2/GeSe2 heterojunction (Fig. 3g)76. The corresponding density of states (DOS) curves are presented (Supplementary Fig. 29). Based on the orbital character, the valence and conduction bands were exhibited in different colors ranging from red (orbitals contribution from GeSe2) to blue (orbitals contribution from ReS2)77,78. The values of EC and EV were determined by the interlayer interaction strength. The GeSe2 possessed a larger band gap compared to the ReS2 counterpart (Fig. 3g). The conduction band offset at the interface was relatively big. While the valance band offset was small (Supplementary Fig. 30). The DFT simulation was consistent with the experimental results of KPFM and UPS. The band structure of ReS2/GeSe2 was identified by both experiments and the theoretical simulation, which was close to the reported results72,75.

The Ids-Vgs curves under different sweeping rates were measured to study the carrier’s trapping and de-trapping processes (Supplementary Fig. 31). An obvious hysteresis loop of Ids was obtained when the forward/backward sweeping gate voltages were applied. The mechanism was gate-voltage-controlled carrier’s redistribution in the device (Supplementary Fig. 32)79. The working mechanisms of PPC and NPC are schematically depicted (Fig. 3h to Fig. 3l). To achieve optical-controlled conductance modulation, a gate voltage of −10 V was applied. For the PPC behavior, the light (wavelength of 808 nm) generated excitons in the ReS2 layer due to the strong absorption of ReS2. At the initial state, the vacancy states on the GeSe2 surface were occupied by electrons (Fig. 3h), which depleted the majority carriers of electrons in the ReS2 channel. The device showed relatively low conductivity. The 808 nm light excited the electrons to the conduction band, generating electron-hole pairs in ReS2 (Fig. 3i). The holes migrated to the GeSe2 under the negative gate voltage of −10 V and neutralized with the trapped electrons (Fig. 3i). The electronic strength depleting the channels’ carriers decreased due to less trapped electrons, resulting in higher conductivity (Fig. 3i). The high conductivity remained when the light was removed. Positive charged vacancies caused non-volatile PPC. When further applying 808 nm light, more electrons in the vacancy’s states were neutralized, corresponding to higher conductivity (Fig. 3j). This explains why the conductance was modulated gradually, which was essential for ANNs implementing matrix-vector multiplication (MVM)80. In comparison, the 405 nm light generated electron-hole pairs in the GeSe2 layer. The excessive electrons inside the GeSe2 were pushed to the surface of GeSe2 and were trapped by vacancy defects (Fig. 3k). The majority carriers of electrons were depleted by the trapped electrons, leading to decreased conductivity. More 450 nm light pulses induce more trapped electrons, leading to lower conductivity (Fig. 3l). Although the 450 nm light may also generate electron-hole pairs in ReS2. The screening effect from the trapped electron dominated the conductivity modulation. The similar theory has been reported in previous work81,82,83. The ReS2/GeSe2 heterostructure has bi-axial optoelectronic properties due to in-plane anisotropic structures. This was the fundamentals for the PPC and NPC with polarimetric functions.

Moreover, the influence of the twist angles (the twist angle between the b-axis of ReS2 and the b-axis of GeSe2) on optoelectronic responses has been investigated. The twist angle in the twistronic optoelectronic devices was adjusted from 0° to 90° by in-situ rotating the ReS2 layer (Supplementary Fig. 33). A typical optical microscopy image of the twistronic optoelectronic device with 60° was provided (Supplementary Fig. 34). The polarized light with 405 nm wavelength was used to modulate the optoelectronic properties. The linear polarization direction was fixed and was parallel with the b-axis of GeSe2. The Ids-Vgs curves with different twist angles between ReS2 and GeSe2 were measured (Supplementary Fig. 35). The suppression of current amplitude was obtained for all angles, corresponding to NPC responses. A negligible difference was observed when changing the angle between ReS2 and GeSe2. This means that the alignment angle of anisotropic ReS2 and GeSe2 barely affected NPC responses. Trap states screening the gate electric field is the main reason for suppressed photocurrents (Fig. 3h–l). This is consistent with the previous report84.

Biomimetic real-time navigation by OCPM arrays

OCPM machine vision systems are designed for multiple functions, including image sensing, polarimetric measurement, and optical-controlled synaptic weights. The sensory functions realize front-end image sensing, which is the input for high-level back-end cognitive tasks. On the other hand, the fully optical-controlled conductance represents the weight updating in ANNs. OCPM arrays can deploy deep learning algorithms for cognitive tasks. Moreover, the ability of polarimetric measurement can be used for navigation and anti-glare pattern recognition, which goes beyond traditional humanoid machine vision and revolutionizes interspecies-chimera machine vision.

Many animals (for example, honeybees) rely on the sun compass for spatial orientation. They can effectively identify directions even in poor weather (e.g. invisible sun, cloudy) by measuring the celestial polarization patterns, which cannot be done by human eyes85. Honeybees are masters at real-time navigating to attractive flower patches with the help of polarized-light patterns. A scout honeybee uses waggle dance (waggles back and forth as moving forward in a straight line) on the honeycomb to transmit the location information to its nestmates (Fig. 4a)86. The wangle phase gives the distance information from their hive to the follower. The angle between the waggle segment’s axis and the upwards of the honeycomb (a reference direction defined by honeybees living in the hive) represents the angle between the sun and the fly direction. The reason that a honeybee can navigate is that they can measure the polarized light in the sky. Even on cloudy days, they can steer correctly on the way to the target owing to inferring the sun’s position by measuring the pattern of polarized light (Fig. 4b). The major components of polarization-vision pathways for identifying navigation maps are demonstrated (Supplementary Fig. 36a–c). The dorsal rim area (DRA) is sensitivity to polarized light. This can be realized by orthogonal positioned OCMP arrays for polarization measurements. Notably, polarization-sensitive synaptic plasticity can be mimicked by OCMP with both excitation and inhibition capabilities. The OCMP realizes fetterless, polarizer-free, and miniaturization for polarization navigation, avoiding bulky configurations with separated filters and polarizers. This demonstrates that OCMP is an excellent candidate building block for real-time navigation systems.

Fig. 4: Biomimetic real-time navigation using OCPM arrays.
figure 4

a Schematic diagram of honeybee’s waggle dance in the honeycomb, which tells watching bees the flower patch’s location (direction and distance). b Schematic diagram of a honeybee flying toward the indicated flower patch using the sun as the reference. c Schematic diagram of the experimental setup for measuring celestial polarization patterns. d Polarization-dependent Ids-Vgs mapping for the OCPM at HRS. (Vds = 1 V). e Polarization-dependent Ids-Vgs mapping for the OCPM at LRS. (Vds = 1 V). f The perceptive architecture of OCPM array for navigation. g The polar coordinate plot of angle-dependent Ids at LRS. The black and blue arrows represent the PD and SM, respectively. The blue dashed line is the fitting result. (9.14−9.51 AM, Longitude of 113.91°, Latitude of 35.34°, March 12, 2023). h The real-time orientations of PD and SM at zenith. Experimentally measured results locate in the theoretical slot.

Full size image

In the sky, the polarized sunlight is symmetrically distributed about the solar meridian (SM). The polarization direction (PD) is perpendicular to the direction of SM. Therefore, direction identification can be achieved by measuring the direction of SM or PD. As a proof-of-concept, the navigation accuracy was experimentally evaluated on a typical cloudy day. The OCPM was tested outdoors with natural skylights. No packaging processing was implemented. The device exhibited excellent resilience in ambient environments. The test setup is shown in Fig. 4c. The angle-dependent Ids was measured (Fig. 4d, e). The interval angle for each test was 15° with respect to the reference direction of north. The b-axis of ReS2 pointing the north and south was defined as 0° (Supplementary Fig. 37). The detailed methodology can be found in Supplementary Note 1. The results showed angle-dependent current amplitudes. The device at HRS and LRS both showed polarization sensitivity. The reconfigurable electronic feature makes it more compatible with other ancillary circuits. Because the operation currents have different margins in HRS (0–0.2 nA) and LRS (20–94 nA) working models, which can have better compatibility with a wider range of peripheral circuits (Supplementary Fig. 38). The perceptive architecture of OCPM array coupling with algebraic analysis realizes navigation applications (Fig. 4f, Supplementary Note 1). Typical polar coordinate plots of angle-dependent Ids at LRS (Fig. 4g) and HRS (Supplementary Fig. 39) were measured. The experimentally measured SM angles were 119.3° (PD angle of 29.3°) and 119.6° (PD angle of 29.6°) under HRS and LRS working models respectively. The two figures (Fig. 4g and Supplementary Fig. 39) were very close with a relative deviation (RD) of merely 0.12% (RD = |SMHRS − SMLR|/ (SMHRS + SMLRS)), demonstrating the consistency of the two working models. The experimental results were compared with the theoretical values (Fig. 4h). The north was employed as the reference direction (north was 0°). The evolution of solar azimuth angle over a day was calculated (Supplementary Figs. 40). The theoretical α was 118.8°–126.7° over the measuring time slot (9.14-9.51 AM), corresponding to the θ of 28.8°–36.7° (PD direction relative to the north direction). The red and blue arrows represent the measured results with the device at LRS and HRS respectively. The experimental values with OCPM were located right in the theoretical slot, demonstrating a high navigation accuracy and practical applications. Notably, the presented system simplifies the system’s complexity due to the filter-free and polarizer-free features87.

Anti-glare pattern recognition by OCPM arrays

Human-bee chimera machine vision integrates polarimetry and cognitive recognition functions, which overmatch traditional neuromorphic vision systems. Reflected light from a surface (usually non-metallic) is partially polarized according to Fresnel’s law88. The reflected light is usually very strong, inducing glare spots that will distort visualized images and decrease cognitive accuracy in machine vision. Specifically, automatic vehicles and drones require real-time sensing and processing traffic information. Therefore, three major challenges for image processing in automatic machines are sensing and processing accuracy, computing speed, and energy consumption. However, reflective materials used in buildings, advertising boards with smooth surfaces, and a body of water produce lots of glare spots. This will exacerbate the processing accuracy and increase the risk. Note, the formation of glare spots is because the reflected light is polarized light parallel to the surface. The polarimetric capabilities of OCPM can filter the polarized light and reduce the influence of glare spots. Besides, fully optic-controlled synapse can be used to build ANNs for cognitive pattern recognition16. An anti-glare machine vision can be developed by integrating polarimetric and synaptic functions.

The comparison between human vision systems and OCPM-based honeybee-chimera machine vision is illustrated (Fig. 5a). A real image with glare spots is sensed by photoreceptors in human eyes. The image is projected on the retina. Human eyes cannot reduce the intensity of glare spots on images. The projected image reserves the glare spot that distorts the picture of the letter “F” (on the top of Fig. 5a). The image will then be transformed into the visual cortex via optic afferent nerves for cognitive processing. The remained spot glare will influence the processing accuracy due to the distortion and partial-missing of the image. In comparison, the impact of spot glare can be obviously suppressed with the OCPM array thanks to the polarimetric ability (the letter “F” shown in the middle of Fig. 5a). Importantly, the OCPM array can perform as ANNs implementing deep learning for cognitive tasks, mimicking the function of the visual cortex.

Fig. 5: Anti-glare pattern recognition.
figure 5

a Schematic diagram of human eyes versus OCPM arrays processing patterns with glare spots. The OCPM array can effectively suppress the glare-spot-induced image distortion. Neuromorphic computing can be implemented by OCPM arrays. b Examples of ideal patterns, no anti-glare processing, and the patterns with anti-glare processing with OCPM. c Comparison of pattern recognition accuracy with and without OCPM-based anti-glare processing. The inset shows the energy budgets. The anti-glare function significantly improved the interspecies-chimera computing efficiency.

Full size image

As a proof of concept, the excellence of anti-glare pattern recognition with OCPM has been presented. A dataset with three letters of “O”, “F”, and “U” with 6 × 7 pixels was adopted for the training and testing. The scale of each pixel was 0-1. Ideal patterns (no background noise and glare-spot-like noise), no anti-glare patterns, and anti-glare patterns are shown (Fig. 5b). The background noise was randomly generated in a margin of 0–0.5. It was assumed that the Brewster angle was reached for the reflected light, indicating the reflected light was linearly polarized89. The pixel in the glare spot was considered as “1”, and the glare spot size was 2 × 2. The suppression degree of glare spots with OCPM was determined by the sensitivity of linearly polarized lights. The dichroic ratio of ~6.4 under pulsed operation was employed based on experimental results. The noise due to glare spots decreased to the 1/6.4 of original values with the anti-glare processing based on the dichroic ratio. The glare spots were effectively decreased (Fig. 5b). The patterns were fed into an artificial network for training and recognition. More detailed simulation details are presented in (Supplementary Note 2, Figs. 41–44). The pattern recognition accuracies with and without OCPM-based anti-glare processing were calculated (Fig. 5c). The anti-glare processing can improve the accuracy compared to the no anti-glare counterparts (Fig. 5c). The recognition accuracy plots for 9 running cycles were calculated to implement statistics analysis (Supplementary Fig. 45). The same conclusion can be obtained that the anti-glare interspecies-chimera machine vision showed better performances despite minor differences among different running cycles. The statistical comparison of training efficiency with and without anti-glare abilities was demonstrated (Supplementary Fig. 46). The average values were summarized in (Supplementary Table 2). To get the recognition accuracy of 70%, 57 and 99 training epochs were required for with and without anti-glare functions respectively (Supplementary Table 2). 2410 epochs were needed to get 99% accuracy for the humanoid neuromorphic machine vision without anti-glare functions (Supplementary Table 2). Remarkably, the anti-glare function can significantly reduce the training epochs to merely 245 using interspecies-chimera machine vision, almost one magnitude of difference (Supplementary Table 2). The ratio of improvement (RoI) was as high as nearly 90% (Supplementary Table 2) which was much higher than previous reported results with image pre-processing abilities12. Besides, the training processing is the most energy-intensive step which is determined by neural network size, dataset size, training epochs, etc. As we fixed other variables (network size, dataset size, etc.) during the training process. The training epochs qualitatively reflect the significantly improved energy efficiency (Inset of Fig. 5c). The training energy consumption was reduced to 11.4% aided by the anti-glare abilities (Inset of Fig. 5c). The abovementioned anti-glare pattern recognition assumed that the OCPM was perpendicular to the polarized light reflection. The randomness of polarization directions may influence the effectiveness of the anti-glare recognition. In the system-level design, a polarization measurement component is required to make sure that OCPM is perpendicular to the polarized reflected light. Notably, our OCMP can identify the polarization of the reflected light, which can potentially make the whole system simplified. This work focuses on device-level development. It is a proof of concept that our device can improve computing efficiency. The relative angle (the angle between the polarized light reflection and device orientation) impacts the recognition accuracy (Supplementary Fig. 47). The recognition accuracy improved by different margins under different angles due to the anti-glare function. Therefore, even without the polarization calibration components, the device can still surpass traditional technology. The OCPM equipped interspecies-chimera machine vision with the anti-glare function, which can effectively improve computing speed with significant lower energy consumption. This technique has a bright future for energy-efficient interspecies-chimera computing systems.

We have reported an interspecies-chimera machine vision by integrating the polarimetric function of honeybees and the intelligence of human beings. The device based on a van der Waals heterostructure (ReS2/GeSe2) provides polarization sensitivity, nonvolatility, and positive/negative photoconductance simultaneously. The polarimetric measurement can identify celestial polarizations for real-time navigation. Meanwhile, the anti-glare recognition with polarimetry improved one magnitude of energy efficiency compared to the traditional neuromorphic machine vision counterpart. This technique will boost the applications in autonomous vehicles, medical diagnoses, intelligent robotics, and navigation without satellite-based GPS systems etc.

Methods

Device fabrications: Multilayer GeSe2 was exfoliated from bulk crystal using Nitto tape and directly transferred onto a highly p-doped silicon substrate that was covered by silicon dioxide (90 nm). ReS2 was mechanically exfoliated and transferred onto the GeSe2 flake, which was assisted by an aligned transfer system equipped with an optical microscope (Shanghai Onway Technology Co., Ltd.). After preparing the heterostructures, electrodes with 5 nm Cr/45 nm Au were prepared by maskless lithography (Tuotuo Technology (Suzhou) Co., Ltd.), thermal evaporation, and lift-off techniques. The GeSe2 holder can be rotated to change the alignment direction. Meanwhile, the ReS2 layer can be picked up or dropped down at desired positions with the help of polyvinyl alcohol (PVA) stamp. This can fabricate different alignment angles of ReS2/GeSe2.

Material characterizations: The thickness and morphology were observed by optical microscope (Nikon LV100POL), atomic force microscope (Bruker ContourGT). The optical properties were measured by a micro UV-Vis spectrophotometer (Jasco MSV-5000). The anisotropic Raman spectra were obtained using a LabRAM-HR Evolution Raman spectrometer with an excitation wavelength of 532 nm. The HAADF images and FFT patterns were measured by Spectra Ultra aberration corrected Scanning Transmission Electron Microscope (Thermo Scientific Spectra 300 S/TEM). Ultraviolet photoelectron spectroscopy (UPS) measurements were performed with a monochromated photon energy of 21.21 eV (HeI) through a toroidal mirror monochromator (SPECS GmbH). The chamber of base pressure was lower than 5 × 10−10 mbar. Kelvin probe force microscopy (KPFM) measurements were carried out in Amplitude Modulation mode (AM-KPFM) using a conductive tip.

Optoelectronic measurement: The optoelectrical measurements were carried out on a probe station (Linkam, HFS350EV-PB4) equipped with a semiconductor characterization system (Keysight, B1500A). A 405-nm laser (TEM-F-405, Cnilaser), 670-nm laser (MDL-III-670-SM, Cnilaser), and 808-nm laser (TEM-F-808, Cnilaser) were used as light sources. The time response photocurrents were tested by switching the light on/off with an electronic shutter. Linear polarization-resolved photodetection was carried out with the help of a polarizer and a half-wave plate. All the measurements were carried out at room temperature in an ambient environment.

Density Functional Theory calculations: First-principles calculations were performed using the projector-augmented wave (PAW) method as implemented in the Vienna ab initio simulation package (VASP). The Perdew-Burke-Ernzerhofer (PBE) formula within the generalized gradient approximation (GGA) was used to describe the exchange correlation. The DFT-D2 method of Grimme was used to describe the vdW heterostructure interaction. The vacuum region of 20 Å was set along the x direction to avoid virtual interaction between adjacent images for 2D vdW heterostructure. The cutoff energy was 450 eV. All the structures were relaxed until the forces on all unconstrained atoms were smaller than 0.01 eV/Å, and the total energy convergence criterion was 10−4 eV.

Simulation of anti-glare pattern recognition: The code was written with MATLAB. Three-layered ANN was utilized with an input layer (42 neurons), a hidden layer (20 neurons), and an output layer (3 neurons). The training images have 6 × 7 pixels. The training and recognition processes were based on the backpropagation (BP) algorithm. The activation function was Sigmoid. A more detailed description can be found in the supplementary document (Supplementary Note 2).