Artificial organic afferent nerves enable closed-loop tactile feedback for intelligent robot

Abstract

The emulation of tactile sensory nerves to achieve advanced sensory functions in robotics with artificial intelligence is of great interest. However, such devices remain bulky and lack reliable competence to functionalize further synaptic devices with proprioceptive feedback. Here, we report an artificial organic afferent nerve with low operating bias (−0.6 V) achieved by integrating a pressure-activated organic electrochemical synaptic transistor and artificial mechanoreceptors. The dendritic integration function for neurorobotics is achieved to perceive directional movement of object, further reducing the control complexity by exploiting the distributed and parallel networks. An intelligent robot assembled with artificial afferent nerve, coupled with a closed-loop feedback program is demonstrated to rapidly implement slip recognition and prevention actions upon occurrence of object slippage. The spatiotemporal features of tactile patterns are well differentiated with a high recognition accuracy after processing spike-encoded signals with deep learning model. This work represents a breakthrough in mimicking synaptic behaviors, which is essential for next-generation intelligent neurorobotics and low-power biomimetic electronics.

Introduction

The future development of prosthetics, bionic electronics, and intelligent robotic systems looks to inspiration from human tactile systems^1,2,3. The human tactile system generates the sensation of pressure and vibration from spatiotemporal mechanical deformations in the skin^4,5. The sensing of stimuli and their response are governed by the peripheral nervous system through the coordination of afferent and efferent nerves. Tactile perception in humans occurs via specialized mechanoreceptors on the skin that generate action potentials that travel along nerve fibers to the spinal cord and to the brain. Touch sensations are processed in the primary somatosensory cortex of the human brain, which generates the perception of pressure as well as extracts features such as its location and intensity. This information is used to determine how to respond to pressure, including protective measures (Fig. 1a). The response signal to the appropriate muscles is carried by motor neurons which causes the muscle to contract, generating the movement.

Fig. 1: Nerve-inspired artificial organic afferent nerve in comparison with a biological one.

a The working process of biological tactile sensory system that is stimulated by slippage of object from fingers. Mechanoreceptors in skin transduce touch stimuli to electrical signal and afferent nerves convey touch information to human CNS and efferent nerves carry motor information away from CNS to the muscles of body in order to initiate slip control. b The artificial organic afferent nerve, consisting of touch sensors and OEST, serves as artificial mechanoreceptors and neuromorphic device, respectively, to execute the collection, transmission and processing of touch information in intelligent robots.

To achieve tactile perception, information processing, and motion responses, afferent nerves play a key role in carrying touch information from cutaneous mechanoreceptors to the central nervous system (CNS), while efferent nerves carry motor information away from CNS to the muscles of body to initiate an action (Fig. 1a). Cutaneous mechanoreceptors determine the conversion from mechanical stimuli to electrical signals, which are then transmitted to CNS via neurons. Among the various types of slow adaptive (SA) mechanoreceptors, Merkel cells are capable of producing sustained responses to static pressure stimulation and possess high sensitivity to light touch, while Ruffini corpuscles associated with fast adaptive (FA) mechanoreceptors perceive directional tactile stimuli and monitor slippage of objects along the surface of skin, enabling grip readjustment^5,6,7. Neurons communicate with one another at junctions known as synapses, where chemical fluxes regulate the strength of synaptic connections between two neurons to enable signal processing. The action potential generated by mechanical stimuli triggers the release of neurotransmitters from presynaptic vesicles into the synaptic cleft, followed by binding to receptors of the postsynaptic neuron, which will cause the opening of ion channels for permeant ions (Na⁺ and K⁺) and produce an excitatory postsynaptic potential. Distributed and parallel networks of mechanoreceptors, neurons, and synapses are required to efficiently sense, transmit, and process complex tactile stimuli, facilitating locomotion and manipulation.

Efforts to mimic the human tactile system have included the development of simple two-terminal capacitive and piezoresistive touch sensors to emulate the signal transduction mechanism of SA mechanoreceptors^1,8,9. However, such artificial mechanoreceptors exhibit volatile behaviors for input stimuli, generating a temporal state enhancement that rapidly decays to its initial state, which hinders subsequent signal transmission and processing of tactile information in a neuromorphic mode. Inspired by biological neural systems, artificial synaptic devices, which offer analogous functionalities as biological synapses, have attracted widespread attention in recent years^10,11,12. For instance, classical neuromorphic computing systems at software level have been used to mimic biological tactile nervous systems based on the von Neumann model, allowing to discriminate Braille-characters and control actuators in legged robots for intelligent neuro-robotic systems and bionic limb prostheses^13,14. However, the conventional von Neumann architecture exhibits inevitable limitations in emulating brain-like computing process due to the fundamental difference from the massive parallel signal processing that occurs in human brain, and realizing power-efficient cognitive computing systems due to the physically discrete core computing processor and memory unit. To overcome these bottlenecks, brain-inspired neuromorphic chips using microelectronic devices have emerged as hardware implementation of artificial intelligence to imitate biological nervous system^15,16. However, most prior reports show synaptic activation by simulated action potential stimuli and do not integrate synaptic activation into a parallel and event-driven artificial sensory system. To construct a bioinspired artificial tactile system, integrating artificial mechanoreceptors together with artificial synapses and neurons provides a strategic opportunity to emulate tactile sensing and information processing as humans do^{17,18,19,20,21,22}. Although artificial afferent nerves constructed in this way have been demonstrated to be capable of actuating muscles by stimulation when connected to the motor nerve of an insect, the system unconsciously reacts to tactile stimuli²². Closed-loop operation for sensing, processing, and actuation is indispensable for creating an artificial tactile system with a conscious response. Such closed-loop tactile feedback would enable intelligent decision-making in complicated tasks, such as allowing a gripper to re-adjust force to prevent slipping or for evasive actions to be taken above a pressure threshold. In addition, most synaptic integrations are only characterized by a transient or short-term response to tactile stimuli. The response must be long enough to prevent signal loss and improve signal conversion efficiency during transmission. In addition to integration, there remain challenges including high writing voltage, nonlinearity, and lack of biocompatibility, thus calling for the development of new materials and architectures for artificial mechano-reflex arcs.

Organic semiconductors are promising candidates for building sensing and neuromorphic devices, due to their biocompatibility, mechanical flexibility, low-temperature processability, excellent tunability, and low voltage operation^23,24,25,26. Akin to the generation and modulation of excitatory postsynaptic current (EPSC) by presynaptic action potential in neural synapses, the three-terminal organic transistors operate in a similar way in which the charge carriers in channel are governed by the input gate potential, which have been widely explored to emulate synaptic behaviors and construct artificial nerves^20,27,28. Recently, organic field effect transistor (OFET) based artificial visual and tactile sensory systems have been exploited by encoding the information of external stimuli into output electrical signals, due to the coupling effect between the potential and semiconducting transport properties^29,30,31. Despite the success of OFETs in emulating neuromorphic functions, a large write bias (~10 V) is usually required for operation, coupled with a limited long state-retention, which are essential for hardware-based artificial neural networks^32,33. In view of these drawbacks, organic synaptic transistors with low operating bias and long-term state retention are highly desirable. Organic electrochemical transistors, which are mixed ionic/electronic devices with large transconductance (g_m) and low operating bias (<1 V), exploit the electrochemical doping of ions throughout volumetric semiconductors to facilitate efficient transduction of ionic flux to electrical current^34,35,36. Interestingly, the release and uptake of neurotransmitters triggered by action potentials and the generation of EPSC, are closely analogous to the operation of an organic electrochemical transistor, where the conductivity modulation in semiconductors is achieved by voltage-controlled injection and extraction of mobile ions (Fig. 1b)^37,38. The coupled ionic/electronic effect endows organic electrochemical synaptic transistors (OESTs) with low power consumption, short-term or long-term state-retention, and multi-level distinct conductance states for neuromorphic computing, thus providing a potential route to achieve bioinspired mechano-electric transduction and signal processing in artificial tactile neural systems. However, most recently reported OESTs operate in depletion mode, which consumes power even at no driving bias. In addition, work on the use of OESTs for constructing artificial afferent nerves to process tactile information is still lacking. Although OESTs for sensorimotor integration and learning in robotics has been demonstrated³⁹, intelligent artificial tactile systems with smart sensing, efficient information processing, and adaptive control for neurorobotics are still in their early stages.

In this work, we propose a flexible artificial organic afferent nerve (AOAN) realized by touch-triggered OEST, which demonstrates the hardware-based cognitive recognition capability for intelligent robotics. The AOAN can not only perceive external tactile information but also modulate its response to static and dynamic touch stimuli, thus enabling in-memory computation of sensory information, analogous to the biological tactile system. The AOAN shows a pressure intensity-dependent transient response and an enhancement of tactile sensitivity to light touch by training process at a low operating bias of –0.6 V. Furthermore, to illustrate the potential application in neurorobotics, a dendritic integration function in AOAN is achieved to perceive directional tactile simulations and recognize the slippage of object. A detectable motion frequency of 40 Hz is attained, which fully covers the sensitive range of Merkel disc mechanoreceptors (5–15 Hz). Finally, an intelligent neuro-robot with our AOAN system, coupled with a closed-loop feedback program to implement slip recognition and prevention actions upon occurrence of slippage incidents, is demonstrated by learning the real-time spatial information of directional touch. The AOANs open promising avenues for the next generation of power-efficient, and large-scale hardwired neuromorphic systems endowed with interactive sensation/memory/computation capabilities.

Results

Organic electrochemical synaptic transistors

In this work, an asymmetric regio-regular conjugated polymer, namely, poly[(5-fluoro-2,1,3-benzothiadiazole-4,7-diyl)(4,4-dihexadecyl-4H-cyclopenta[2,1-b:3,4-b’] dithiophene-2,6-diyl)(6-fluoro-2,1,3-benzothiadiazole-4,7-diyl)(4,4-dihexadecyl-4H-cyclopenta[2,1-b:3,4-b’]dithiophene-2,6-diyl)]; PFT-100, is utilized in our organic transistors as the channel layer. Our experiment revealed that PFT-100 based transistor provides a relatively lower operating bias than the commonly used accumulation mode poly(3-hexythio-phene-2,5-diyl) (P3HT) device, resulting from the well-ordered aggregation in PFT-100 polymer with fluorinated benzothiadiazole (Supplementary Fig. 1). A solid electrolyte (SE) consisting of poly(vinylidene fluoride-co-hexafluoropropylene) (PVDF-co-HFP) ionic conductive polymer matrix and ionic liquid (IL) acts as the ion source for doping PFT-100 conjugated semiconductor. The usage of SE gating provides a stable operation environment thanks to its robust features and easy scalability for integrated applications. A high ion-dipole electrostatic interaction occurs due to the incorporation of fluorinated copolymer matrix and IL, which results in the formation of an ionic elastomer that allows facile ion migration with high ionic conductivity^40,41,42,43. Owing to ionic and electronic interactions and coupled transport properties, the conductivity of PFT-100 is tunable by the gate-controlled injection of anions from SE into bulk semiconductor for volumetric electrochemical doping. As shown by the UV–Vis–IR absorption results, the neutral state of PFT-100 exhibits absorption peaks at 431 nm and 771 nm, corresponding to its π–π* transitions (Supplementary Fig. 2). For V_G bias beyond –0.8 V, an oxidation state emerges with an absorption band beyond 1100 nm, indicating the accumulation of charges in bulk PFT-100 polymer through the doping effect of anions.

PFT-100-based OEST was fabricated layer by layer in the device architecture shown in Supplementary Fig. 3, where a parylene-C insulating layer separated the source/drain electrodes and SE allowed low leakage current (I_G) of <100 nA. According to the transfer and output characteristics, OEST operates as a p-type enhancement mode transistor at low voltage (V_D = –0.3 V) and with a high on-off ratio (~3 × 10⁴). The ionic accessibility of semiconductor strongly impacts the threshold voltage (V_th) and transconductance (g_m), which in turn is determined by the hydrophobic-hydrophilic properties of the ions and their blend ratio in the SE^44,45. Here, ILs with different anions, bis(trifluoromethylsulfonyl)amide [TFSI]^- and trifluoromethanesulfonate [OTF]^- were mixed into PVDF-co-HFP matrix at different mass ratios. The anion ratios in PVDF-co-HFP were controlled from 33% to 80%, which can be denoted as, for example, 80% TFSI. The ionic conductivity of SE strongly depends on the ion type and ion blend ratio, as summarized in Supplementary Fig. 4. The cyclic voltammetry (CV) curves indicate a greater charging capacity and lower redox potential for [TFSI] anion (–0.5 V), than that for [OTF] anion (–0.9 V). As shown in Fig. 2a, b, a low V_th (~ –0.42 V) and high g_m (~1.86 mS) were demonstrated by blending 80% TFSI in SE for OEST. Hydrophilic [OTF]^- anion or decrease the ion concentration will increase V_th and degrade g_m.

Fig. 2: Pressure-triggered neuromorphic behaviors of AOAN.

a Transfer characteristics of OEST with different ion types and blend ratios. b Comparison of V_th and g_m values of OEST devices. c Cycling stability of OEST with 80% TFSI over 1000 cycles of V_G pulsing from 0 V to −0.6 V. V_D fixed at −0.3 V, V_G pulse width = 0.1 s, pulse interval = 5 s. d Resistance change of touch sensor and effective gate bias (V_eff.G) on OEST in response to an increasing pressure. V_G was fixed at −0.6 V. e Pressure modulating transfer curves of AOAN. V_D was fixed at −0.3 V. f OESTs alone with different ion types and blend ratio response to a 5 s pre-synaptic potential pulse from 0 to −0.6 V. g Integrated AOAN with different ion types and blend ratios response to a 5 s pressure pulse from 0 to 40 kPa. h The integrated AOAN with different ion types and ion blend ratio response to a 0.1 s pressure pulse from 0 to 40 kPa. i The integrated AOAN with ion blend ratio of 80% TFSI response to a 0.1 s pressure pulse from 6 to 30 kPa, while maintaining V_G at –0.6 V and V_D at −0.3 V. The results were normalized. j, k Analogy of biological afferent nerves and our AOAN integrated device. The release and uptake of neurotransmitters triggered by action potentials from mechanoreceptors, are closely analogous to the operation of OEST, where the conductivity of semiconductor is modulated by touch-controlled injection and extraction of mobile ions.

Before mimicking synaptic behaviors, the memory behavior of OEST is first evaluated. The I_D in the on-state and rise time (τ) from off-to-on state (transient response) was measured by applying a 0.1 Hz V_G pulse from 0 to –0.6 V (Supplementary Fig. 5). OEST with 80% TFSI exhibits a relatively fast response (τ ~ 234 ms) and higher channel conductance change (ΔG, ~0.62 mS) upon applying negative V_G, but fast recovery (~12 ms) to its initial off-state after switching to V_G = 0 V. In contrast, OEST with 33% TFSI exhibits a relatively low conductance change (ΔG, ~0.23 mS) and insensitivity to pulsed V_G, especially during the first second after biasing V_G. However, OEST with 33% TFSI possesses a longer relaxation time (~260 ms), resulting from the ion-trapping in conjugated semiconductor and slower ion diffusion process from PFT-100 to SE. Thus, the anions of SE could be used to optimize the ionic doping and dedoping process in conjugated semiconductor and thus the memory effect in OEST (Supplementary Table 1), which are highly analogous to the release and uptake of neurotransmitters in synapse. OEST with 80% TFSI was also consecutively pulsed 1000 times between 0 V and –0.6 V, exhibiting superior operational stability with no significant drop of conductance after 1000 cycles (Fig. 2c and Supplementary Fig. 6). The topographical AFM images of PFT-100 film indicate an intact surface morphology before and after doping with TFSI anions, proving its good cycling stability. To mimic synaptic behaviors using our OESTs, we applied a V_G pulse at frequency of 5 Hz, serving as action potential from pre-synapse (V_Pre). The post-synaptic current (I_D) exhibits a signal facilitation with successive triggering of V_Pre, demonstrating its synaptic behavior (Supplementary Fig. 7a). Moreover, the post-synaptic current increases with increasing V_Pre, resembling a larger influx of ions in biological synapses when a higher nerve impulse occurs (Supplementary Fig. 7b). The amplitudes of the second current pulse (A₂) in paired-pulse facilitation (PPF) and the tenth current pulse (A₁₀) in post-tetanic potentiation (PTP) were compared to that of the first one (A₁), where a stronger facilitation effect occurred with increasing pulse frequency of V_Pre or with decreasing interval time (Δt), as illustrated in Supplementary Fig. 8. As a result, the next current pulse was facilitated when V_Pre pulses had a short Δt. This is due to the anions in OEST semiconductor that are triggered by the first pulse not able to diffuse efficiently back to SE, and accumulated with additional anions injected by successive pulses, thus synergistically modulating the conductivity of semiconductor.

Integration of artificial organic afferent nerve

In view of the bionic synaptic behavior of OEST, a bioinspired AOAN was constructed to imitate the tactile sensory and memory behaviors in biological system. This is achieved by integrating a touch sensor to the gate of OEST (see the circuit diagram in inset of Fig. 2d). This AOAN resembles the biological tactile sensory nerve, where the touch sensor and OEST act as artificial mechanoreceptor in epidermis for tactile perception and synapse for signal transmission respectively. Supplementary Fig. 9 shows the detailed device structure and pressure-sensitive performance of the piezoresistive touch sensor, which can stably detect a pressure frequency of 100 Hz. We convert pressure stimuli to resistance changes in touch sensor (R_TS) and show the dependence of effective gate bias (V_eff.G) and applied pressure (P) in Fig. 2d. A significant resistance drop was observed under increasing P, especially in low-pressure region (0–10 kPa). The equivalent circuit as depicted, proves an increasing V_eff.G on OEST for a fixed V_G but increasing P. Figure 2e shows the transfer characteristics of AOAN at different pressure level, indicating typical pressure-dependent transistor behaviors operating at low voltage (V_G = –0.6 V, V_D = –0.3 V). The AOAN exhibited a high sensitivity of 106.8 kPa^–1 in a linear pressure range of 0–30 kPa and a weak response to high pressure level of 50 kPa, which fully covers the low-pressure sensing range of gentle touch (<10 kPa) and medium-pressure scope of object manipulation (<50 kPa)^46,47. A higher pressure induces a higher gate-modulation behavior, resulting from the higher V_eff.G applied to OEST. In this way, the application of external pressure allows the triggering and control of the ionic penetration in conjugated semiconductor and modulates its conductivity. The OEST operates at low voltage and behaves analogously to biological synapses since it can convert pressure-dependent action potential to postsynaptic current signals. The integration of separated devices is beneficial for suppressing interference between the sensing and processing components and endows AOAN with flexibility, making it conducive for easy integration with robots.

In AOAN, the application of a constant V_G bias leads to a potential drop in the touch sensor, SE, and semiconductor channel. When no pressure is applied, R_TS is very large (R_TS  ≫ R_SE, where R_SE is the bulk resistance of SE) and the potential drop on touch sensor is almost equal to V_G, i.e., nearly zero V_eff.G, thus the OEST remains in its OFF state. Upon applying pressure, R_TS decreases (R_TS < R_SE) and V_eff.G increases, thus OEST switches to its activated state. Such an abrupt increase in V_eff.G is similar to a V_G pulse on OEST, which would trigger ions fluxes from SE to semiconductor channel. Under pressure, an increasing amount of anions accumulate in the semiconductor, which further leads to a variation in channel conductance. V_eff.G therefore determines the number of anions penetrating semiconductor for electrochemical doping, until the mixed ionic-electronic system reaches equilibrium⁴⁸. Upon removing the pressure (R_TS ≫ R_SE), V_eff.G across OEST will drop abruptly. However, the doped anions remain in semiconductor channel before gradually drifting back to their equilibrium position, since there is no driving bias to push anions back. The local drift-diffusion process of ions is much slower under no bias, especially the diffusion to SE with high ion blend ratio (80% TFSI). Therefore, a longer memory capability of tactile information in semiconductor was obtained after touch sensor integration, compared with OEST alone (Fig. 2f, g).

We investigated the correlation between synaptic weight and relaxation time (τ) after presynaptic triggering, which can be well fitted with an exponential decay function⁴⁹:

$$I=({I}_{0}-{I}_{\infty })\times \exp (-t/\tau )+{I}_{\infty }$$

(1)

where I₀ is the triggered I_D at the end of the presynaptic spike, I_∞ is the final value of the decay current, and τ denotes the relaxation time constant. The charge relaxation time varied markedly with ion type and blend ratio in SE (Supplementary Fig. 10 and Table 2). The value of τ is fitted to be ~306.5 s for 80% TFSI, and 57.1 s for 33% TFSI, indicating a good non-volatile charge retention characteristic, longer than that of recently reported ion-gel-gated synaptic transistors (τ of 75 ms) and ion-electron conductor-based artificial receptor (τ of 160 ms)^18,50. Charge retention time as long as 4 h triggered by pressure spike was demonstrated in AOAN system (Supplementary Fig. 11), even with 0.3% synaptic weight retaining.

The main merit of our design is that it supports an event-driven approach that utilizes external touch stimuli to trigger dynamic conductance changes in synaptic transistors, while other devices are achieved by programmed voltage pulses^27,38,51,52.

The synaptic weight is defined as the absolute value of channel current variation (|ΔI_D|) between the initial current (I_D,0) before touch and the spike current (I_D,T) after touch. It depends on the input pressure and ion types in SE. To investigate the synaptic features of AOAN, synaptic weight and charge retention property were studied by applying a single transient pressure. The synaptic weight increases either by increasing ion blend ratio (Fig. 2h), applied pressure level (Fig. 2i), or reducing pressure frequency (Supplementary Fig. 12), demonstrating pressure-dependent and ion-concentration-dependent spiking responses. Moreover, the high pressure and ion blend or low frequency not only give rise to an enhanced synaptic weight but also a longer relaxation time. An increase in the input pressure from 6 to 30 kPa enhances the retention of synaptic weight from 8.1% to 19.6% after pressure release time of 20 s. This behavior is attributed to a higher V_eff.G generated by higher pressure, where more anions can penetrate and be trapped in semiconductor channel. This result is analogous to biological synaptic behaviors, in which a higher Ca²⁺ concentration activated by action potentials in presynaptic cell, enables the release of more neurotransmitters, thus producing a larger excitatory postsynaptic potential (Supplementary Fig. 13). In addition, the injection and extraction of mobile ions in AOAN which are controlled by touch stimulation, are quite similar to the release and uptake of neurotransmitters triggered by action potentials from mechanoreceptors in the skin (Fig. 2j, k).

Next, the reversibility of OEST and AOAN was evaluated by programming the devices with discrete pulsed voltages or pressures, respectively, as shown in Supplementary Fig. 14. Discrete levels between low and high channel conductivity are attained with negligible fluctuations, indicating good reversibility. Grounding the OEST gate can quickly reset the device to its initial state (Supplementary Fig. 15), due to the internal closed circuit which accelerates the discharging process, allowing the continuous execution of different tasks. In this way, our AOAN can simplify circuit design in more complicated active arrays, which avoids the use of memristor-based memory devices.

The synaptic behaviors of artificial organic afferent nerve

Short-term plasticity (STP), a critical parameter to evaluate changes in synaptic efficacy over time during information processing, was emulated by the AOAN. PPF and PTP, two typical performance indexes in STP are pivotal for decoding temporal information in tactile signals, which are defined as below.

$${{{\rm{P}}}}{{{\rm{PF\; index}}}}=({{{{\rm{A}}}}}_{2}/{{{{\rm{A}}}}}_{1}\times 100)\%$$

(2)

$${{{\rm{PTP\; index}}}}=({{{{\rm{A}}}}}_{{{{\rm{N}}}}}/{{{{\rm{A}}}}}_{1}\times 100)\%$$

(3)

where A₁, A₂ and A_N represent the I_D amplitude of the 1st, 2nd and Nth spike, respectively.

As shown in Fig. 3a, a train of pressure pulses is applied on AOAN to evaluate PPF and PTP. For instance, Fig. 3b displays the triggered synaptic weight after applying 10 successive pressure spikes while maintaining P of 40 kPa, t_p of 100 ms, and Δt of 5 s. A reference memory level was defined as the retained I_D after being activated by last pressure spike for 300 s to enable comparison.

Fig. 3: Mimicking synaptic behaviors by AOAN.

a Patterns of pulsed pressure with different parameters. P, pressure amplitude; Δt, interval time between pulsed pressure; t_p, duration time of pulsed pressure; T_p: period time for one pulse. b Post-synaptic current modulation induced by 10 consecutive pressure pulses. (P = 40 kPa, Δt = 5 s, t_p = 100 ms, V_G = −0.6 V, V_D = −0.3 V) c Comparison of post-synaptic weight by the modulation of t_p, Δt and P. d, e Paired-pulse facilitation (PPF) index as a function of Δt and P for AOAN with different ion blend ratio. The black dashed lines are the fitting curves. f Post-tetanic potentiation (PTP) index as a function of pulse number for AOAN with different ion blend ratio. g, h Memory level as a function of pulse number N and applied P, which shows the transition from STP to LTP triggered by pulsed pressure number and pressure level. The error bars in the plots indicate standard deviations (n = 3 independent devices). i Enhancement of tactile sensitivity to subtle touch by training times.

As depicted in Fig. 3c, the peak I_D triggered by the second pressure spike (A₂) is much larger than that triggered by the first one (A₁), indicating a facilitation of PPF ratio (A₂/A₁). Moreover, this facilitation was stronger when increasing t_p of pressure spike (from 100 ms to 1 s) or decreasing their Δt (from 5 to 1 s) or applying higher P level (from 10 to 30 kPa) (Fig. 3d, e). The ion blend ratio in SE strikingly regulates the PPF index. For instance, PPF index was calculated to be ~255% for AOAN with 80% TFSI, 226% for AOAN with 50% TFSI and 104% for AOAN with 33% TFSI, when fixing P = 10 kPa, t_p = 100 ms and Δt = 5 s.

The PTP index is defined as the ratio of postsynaptic weight generated by N spikes (A_N) to that generated by a single spike (A₁), to study synaptic facilitation when repeated stimuli are applied. The relationships between PTP index with presynaptic spikes number (N) and ion blend ratio are shown in Fig. 3f. The synaptic weight continuously increases as the number of pressure spikes increases. For instance, PTP index for 80% TFSI device reaches 2500%, whereas that of 33% TFSI shows less facilitation of 500% after 10 spikes. Supplementary Fig. 16 exhibits that synaptic weight varies from an evident increment tendency to a subdued state, resulting from the gradual saturation of penetrated anions in semiconductor. This behavior resembles the saturation of released neurotransmitters under multiple mechanical stimuli.

Both PPF and PTP behaviors can be fitted by the double-phase exponential decay function:

$$1+{C}_{1}\exp (-\varDelta t/{\tau }_{1})+{C}_{2}\exp (-\varDelta t/{\tau }_{2})$$

(4)

where C₁ and C₂ are the initial facilitation magnitudes of each phase, and τ₁ and τ₂ are the respective decay time constants, which are useful to mimic the relaxation time of rapid and slower decay phase in a synapse^35,53,54. The fitting of τ₁ and τ₂ generally follow the trend that an elevated τ is associated with an increase in the ion blend ratio, and the facilitation magnitude decreases as the interpulse interval increases (Supplementary Table 3). The fitting results indicate that the facilitation process occurs when Δt is shorter than the decay time, due to the ion trapping in semiconductor after first pulse triggering and accumulation with additional anions by subsequent triggering, thus elevating the conductance in semiconductor successively. The results are consistent with the decay of synaptic facilitation in a biological synapse.

In contrast to STP, long-term plasticity (LTP) plays key role in memory or learning by facilitating synaptic weight more persistently, which can last as long as minutes to days. In biological sensory system, the long-term memory formation in brain can be achieved by repetitive rehearsal or training^35,55,56,57. The evolution process from STP to LTP was emulated in our AOAN by repeatedly applying pressure spikes, and the memory level at 300 s after the last spike, was investigated. The changes in synaptic weight after various numbers of pressure spikes for AOAN with 80% TFSI are shown in Supplementary Fig. 17. The memory level highly depends on the number of pressure spikes and applied pressure level, and a monotonic increase of memory level was found as shown in Fig. 3g, h. Furthermore, AOAN with higher ion blend ratio contributes to an accordingly higher memory level (~126 μA), whereas, device with 33% TFSI shows inconspicuous memory effect (~0.38 μA) after 300 s, due to the small population of residual ions. This phenomenon is analogous to the enhancement of transmission efficiency between pre- and post-neurons, where large amounts of neurotransmitters can be released when repeatedly stimulating the presynaptic neuron. Considering the high synaptic weight and long-state retention behavior exhibited, anion concentration of 80% TFSI is selected for device used in subsequent applications.

Enhancement of tactile sensitivity by training process

By utilizing the neural facilitation behavior of OEST, we demonstrated enhanced sensitivity to gentle touch beyond the capabilities of the touch sensor alone. Figure 3i shows a looping bounce of ping-pong ball on the touch sensor. The generated touch force during each contact between ping-pong ball and sensor was around ~200 mN (Supplementary Fig. 18), which translates to a pressure of around 2 kPa to which the touch sensor is insensitive (<1.1 μA), resulting in the information lost due to the existence of noise. The bouncing process is also fast, with a short duration time of ~100 ms. Despite the subtle touch stimulation and short duration of the ball bounce, the OEST is able to respond quickly with enhanced sensitivity. As indicated by the enlarged image of the signal trace, each bounce results in enhanced response, confirming the ability of AOAN to detect subtle touch. Twenty trials of these looping bounce of the ping-pong ball experiment were carried out and similar enhanced responses were obtained (Supplementary Fig. 19), reflecting the good reproducibility of the attained results. However, as the number of bouncing cycles increases, the current potentiation of the device finally converges due to the complete doping of anions in polymer semiconductor (Supplementary Fig. 20). The video to demonstrate the enhancement of tactile sensitivity to subtle pressure by AOAN can be found in Movie S1.

Robotic integration and motion detection

Tactile perception begins with the complex integration of various spatiotemporal sensory stimuli, which allows us to react appropriately during tactual interaction such as object grabbing and tactile feedback. The AOAN allows for reduced complexity by spatiotemporally integrating several touch sensors, offering a forthright method to mimic the integration of external tactile stimuli from various mechanoreceptors of pre-neurons to dendrites of post-neurons via synapses (Fig. 4a).

Fig. 4: Dendritic integration function in AOAN for robotic integration.

a Schematic illustration of spatiotemporal neural networks distributed on human fingertip. Four pre-synapses were selected for integration with OEST to emulate the spatiotemporal tactile sensory. b postsynaptic current triggered by different input types including single input, synchronous input and asynchronous input. c Neuro-inspired AOAN on robot finger, which couples four artificial mechanoreceptors with neuromorphic device, to transduce signals from mechanoreceptors to brain haptic center. The robot finger with AOAN can hold objects and achieve a motion detection. d postsynaptic current triggered by sliding objects on robot fingertip from top to bottom with different sliding speeds. Top and bottom results were collected by touch sensor and AOAN, respectively. The post-synaptic weight highly depends on the sliding speed (ν). The error bars in the plots indicate standard deviations (n = 3 independent devices).

Here, four artificial mechanoreceptors were integrated into a single OEST via parallel connections such that each unit could trigger the OEST by pressure. A constant mechanical stimulus was loaded on each mechanoreceptor to generate the pre-neuron signal, and multiple patterns were achieved by varying the spatiotemporal input. As shown in Fig. 4b, in the first case, an isolated pressure was used to trigger each mechanoreceptor individually, and all mechanoreceptors could excite the shared OEST with nearly identical post-synaptic strength. In the second case, mechanoreceptors were triggered synchronously, and an enhanced post-synaptic response was observed over the first case, indicating a concurrent integration of spatially isolated tactile stimuli. Here, the synchronous triggering of more units is similar to the activation of more mechanoreceptors in the skin, i.e., a larger spatially activated area. This phenomenon resembles the strengthening of tactile perception and awareness generated by touching two or more fingers over a single finger^19,58. In the third case, mechanoreceptors were triggered asynchronously, with each unit excited with a time interval (Δt = 5 s) between them. A much stronger post-synaptic response is observed, compared to other cases. For instance, the post-synaptic current of four mechanoreceptors triggered successively reaches ~200 μA, compared to ~90 μA for four mechanoreceptors triggered synchronously at identical pressure. The reason for this difference is the PTP of our AOAN, in which the synaptic weight is augmented with repetitive spikes.

Feature extraction in post-neurons enables identification of input pressure information coming from pre-neurons and is widely used in training and learning process in biological sensory systems^21,59. A single output post-synaptic signal for multiple input stimuli allows the sensory data to be more informative with encoded pressure information, greatly minimizing the raw data and simplifying information processing. As a proof-of-concept, we integrated our AOAN onto a robot finger to implement tactile information processing and motion recognition, in which four mechanoreceptors were arranged in line (Fig. 4c). During the experiment, a ball is brought onto the mechanoreceptor arrays and moved from top to down, regarded as a completed touch-slide action. Different sliding speeds (ν) were implemented on AOAN for feature extraction and identification of the coded sliding information (Supplementary Fig. 21). Here the action frequency of n Hz denotes an interval time of (Δt = 1/n) s to slide over two neighboring mechanoreceptors. The output data carry larger volumes of information by spike coding, including rate coding and temporal coding, which can be used to distinguish the inputs from multiple mechanoreceptors with a single processor²¹. Evident differences are observed in the post-synaptic responses to different input patterns, which allows information regarding the slide to be extracted via the spiking rate and timing in the post-synaptic signal (Fig. 4d). The post-synaptic current highly depends on the action frequency and spike duration, since the effective gate bias on OEST remains same level with respect to the sliding speeds (Supplementary Fig. 22). However, there is a trade-off between the synaptic weight and PTP index as a function of the action frequency as revealed in Supplementary Fig. 23. An inverse relationship between them is achieved during the synaptic operation. As frequency increases, the synaptic weight decreases due to a shorter dwell duration on sensors, but a high PPF index is obtained for signal potentiation. The video to demonstrate the sliding of objects on robot finger with AOAN can be found in Movie S2. Our system can read and recognize an action frequency as high as 40 Hz (ν = 160 mm/s) with clear spikes (Supplementary Fig. 24), and importantly, it can memorize the touch-event even after completing the sliding action, whereas a touch sensor alone cannot. Furthermore, artificial tactile receptors with high spatial density (~123 units cm^–2) were also assembled on a fingertip to demonstrate spatially resolved pressure mapping as well as human tactile system (Supplementary Fig. 25). These capabilities are well suitable for slip detection and tactile feedback of robotic grippers during object manipulation.

Intelligent robot e-skin for slip sensation and real-time feedback

Intelligent robots which can integrate smart sensing capabilities including touch recognition, object manipulation, situation awareness, and real-time feedback is of great importance^60,61,62. However, the lack of real-time tactile feedback in previous works has caused issues such as unstable grasping manipulation or slippage of objects, compelling the systems to rely on visual feedback to ensure that the robotic gripper behaves appropriately¹⁹. Human-like tactile perception with real-time feedback is urgently required for promoting robotic intelligence.

To execute a real-time dynamic tactile feedback system, we integrated AOAN with four sensors on robot gripper, as shown in Fig. 5a. The grasping force can be well controlled by adjusting the open state of two fingers of gripper and the consistent performance of repeated trails demonstrates the excellent reproducibility and reliability of this strategy (Supplementary Fig. 26). When the gripper is holding a metal ball which is heavy and slippery, the real-time touch information is monitored and used for slip sensation and closed-loop tactile feedback. To demonstrate these capabilities of our intelligent robot gripper, a closed-loop control with tactile feedback was demonstrated by learning the real-time spatial information of touch. The block diagram for this algorithm to operate this closed-loop control is shown in Supplementary Fig. 27. Once a grip command was issued, the gripper was gradually closed and approached the ball, until a certain force is detected from the system. The initial gripping point was at the top of gripper. A threshold close state of gripper is set to avoid the continuous closing movement of the gripper which would generate excessive force. The gripper with ball is then lifted to test the gripping state and to determine the occurrence of slip motion. Figure 5b indicates the sliding of the ball due to insufficient gripping force, with the sliding path passing over the four sensors in sequence. The ball’s states (holding, slipping) and positions (p1, p2, p3, p4, p5 as indicated in Fig. 5c) can be well recognized from the output plots by feature extraction.

Fig. 5: Real-time dynamic slip sensation and prevention of intelligent robot based on AOAN and closed-loop control program.

a Diagram of the AOAN on robot gripper and the slip motion of a metal ball. b The slip recognition process and the implementation of slip prevention action upon slippage. c Camera image of the slip motion process of ball that passes through four sensors in sequence from top to bottom. d Real-time signal recording for a series of object manipulations according to the algorithm (gripping, slipping, slip prevention and shaking). i: the gripper holding the ball securely with no slippage happens; ii: the slippage of ball when the closed-loop algorithm is not activated; iii: the gripper implements slip recognition and prevention actions when the closed-loop algorithm is activated; iv: after implementing slip prevention program, the secure grip at even shakes the gripper. e Illustration of CNN in which a LSTM layer is followed by a fully connected layer. Bottom shows the internal architecture of standard LSTM cell. f Evolution of the recognition accuracy with training epochs for different AOAN systems and the ideal numeric, respectively. g Probability of three possible results after inputting instances of AOAN with OEST and without OEST into the trained neural network.

Once the close state of gripper reaches to the preprogramed threshold value, gripper stops to close the fingers and then a stable grip is achieved. Only one sensor is in contact with the ball and the generated gripping force can trigger AOAN with no spike emerging as shown in Fig. 5d-i. The slippage of objects often happens in complex manipulation tasks, unstable control or sudden shaking of the gripper. Slip detection can be achieved by spike-feature extraction from the recorded plot and the closed-loop algorithm can further control the system to implement a slip-prevention action once slip occurs. However, before activating the closed-loop algorithm, the gripper with AOAN can only sense slippage but is unable to perform slip prevention (Fig. 5d-ii). During the slippage, the ball passed through four sensors in sequence from top to bottom, and four signal spikes form the output signal of AOAN. Upon the ball fully dropping from gripper, the output signal would be reduced gradually. The slippage of ball was judged by the emergent spike signature and signal attenuation, and the slippage speed could be decoded by analyzing the relative timing between spikes as discussed above (Supplementary Fig. 28a). After activating the closed-loop feedback program, the detection of slippage would automatically trigger a control command to the gripper, thus achieving a secure grip by adjusting its open state without human intervention. As shown in Fig. 5d-iii, a spike appeared, which was judged as an occurrence of slippage, and then activated the algorithm to adjust the open state of gripper. The output signal kept stably after successfully implementing the closed-loop feedback program. Importantly, while the gripper shook rapidly, the intelligent gripper could prevent sudden slip of the ball, due to the dynamic adjustment upon slip was detected (Fig. 5d-iv). As a contrast, the shaking activity or weak grip force always cause immediate dropping of ball without AOAN (Supplementary Fig. 28b).

The memory behavior of OESTs is more reliable for information processing and learning than transient signals. To validate the indispensable role of OESTs in AOAN, a recurrent neural network (RNN) based long and short-term memory (LSTM) model was adopted to process the data collected from AOAN with OEST and without OEST, respectively. LSTM model is capable to recognize time-based information in data by extracting critical features embedded in a time-series, which is necessary to classify signals generated when performing a series of grasping activities in bionic robotics^63,64,65. The LSTM neural network works according to a flowchart as shown in Fig. 5e (see Supplementary Note). Compared with touch sensor alone without OEST, the perception and recognition accuracy of gripper activities can be significantly improved using our AOAN system. The three activities (i, ii, iii) can be classified with an overall accuracy of 98.7 ± 0.3% after 10 training epochs, and 99.0 ± 0.3% after 50 training epochs, which are close to the accuracy value for the ideal numeric simulation (99.4%) (Fig. 5f and Supplementary Fig. 29). However, for AOAN system without OEST, lower accuracies of 75.2 ± 0.5% after 10 epochs and 97.1 ± 0.3% after 50 epochs were achieved. The improved accuracy and reduced training epochs indicate the higher learning efficiency using our AOAN system and RNN model. To further reflect the advantages of our AOAN system, we calculated the probability of each activities yielding possible outcomes by inputting parts of the tactile information from the two databases into the 10-epochs trained neural network. As shown in Fig. 5g, the probability of AOAN is significantly improved for predicting the correct result, compared with the misjudgments that often occur when using touch sensors without OEST. The accuracy of this classifier can be visualized utilizing a confusion matrix (Supplementary Fig. 30), in which samples belonging to the three activities were correctly classified by AOAN system. The video to demonstrate the slip sensation and prevention of intelligent robot gripper can be found in Movie S3. These results provide evidence that our AOAN enables robot gripper with real-time slippage recognition and prevention in response to slippage of object, providing a promising route for the development of intelligent robots.

Discussion

In this work, we utilized an ion-permeable conjugated polymer to construct organic electrochemical synaptic transistors that exhibited low operating voltage (–0.6 V), high on-off ratio (>10⁴) and excellent stability (>1000 cycles). The effects of ion electrochemical doping and trapping in bulk ion-permeable conjugated semiconductor were demonstrated by altering the ionic hydrophobic-hydrophilic properties and their blend ratios in solid electrolytes. By integrating artificial mechanoreceptors and synaptic transistors, an AOAN was constructed, that can capture essential morphological and functional resemblances to biological tactile sensory system. The enhancement of tactile sensitivity to light touch and recognition/sensation of spatiotemporal tactile information, as well as the low retention loss during signal transmission were demonstrated by our AOAN. The capacity to perceive static and dynamic touch stimuli, enables in-memory computation of sensory information. The dendritic integration function in AOAN was achieved to perceive directional tactile simulations and recognize the slippage of object. An intelligent robot with our AOAN, coupled with a closed-loop feedback program was demonstrated by learning the real-time spatial information of directional touch, in order to implement slip recognition and prevention actions upon slippage. The temporal and spatial features of tactile patterns are well differentiated with a high recognition accuracy after processing spike-encoded signals with deep learning model. This work represents a promising strategy for developing next-generation neurorobotics with artificial intelligence, biomimetic electronics, and smart prostheses.

Methods

Materials and chemicals

PFT-100 (Mw > 50k, PDI < 3) was purchased from 1-Material Inc (Canada). EMIMTFSI, EMIMOTF, PVDF-HFP, (3-glycidoxypropyl)trimethoxysilane (GOPS), ethylene glycol (EG), acetone, and chloroform were all purchased from Sigma-Aldrich and used without further purification. Poly(3,4-ethylenedioxythiophene):poly(styrenesulfonate) (PEDOT:PSS) (Clevios PH1000) was purchased from Heraeus.

Preparation of precursor solutions

To coat PFT-100 films, 5 mg/ml PFT-100 solution in chloroform was heated to 60 °C to form a homogeneous solution in glovebox. Polymer electrolyte precursor solution was prepared by dissolving PVDF-co-HFP in acetone at a concentration of 110 mg/ml, followed by vigorous magnetic stirring at 70 °C for 3 h, then adding ionic liquid ([EMIM][TFSI], [EMIM][OTF]) into the above solution with weight concentration ranging from 6.0% to 33% and followed by vigorous vibration for 30 min. For the PEDOT:PSS film preparation, 1 mL PEDOT:PSS solution was first blended with 1 mL deionized water; followed by mixing of 10 µL EG to improve PEDOT:PSS conductivity and 5 µL GOPS as a crosslinker, and vigorous stirring for 15 min at room temperature.

Fabrication of organic synaptic transistors and touch sensors

Transistors were fabricated following the Parylene C peel-off procedure as described in our previous work. In brief, the source and drain electrodes (Cr/Au,10/50 nm) were photolithographically patterned on the cleaned p-Si⁺⁺/SiO₂ (200 nm) substrate by thermal evaporation. A 2 μm thick Parylene C layer, together with an adhesion promoter of 3-(trimethoxysilyl)propyl methacrylate (A-174 Silane) was deposited, to insulate the contacts from electrolyte by chemical vapor deposition (Lavida-110, Femto Science Inc). An anti-adhesion layer of dilute cleaner (Micro-90, 10 wt.% in deionized water) was spin-coated to facilitate the peel-off procedure. After that, a second Parylene-C layer (~2.5 μm) was deposited, acting as a sacrificial layer. Substrates were subsequently coated with a 12 μm thick AZ P4620 photoresist, soft-baked at 110 °C for 180 s, exposed using SUSS MJB4 mask aligner, and developed by AZ developer. The patterned channel area and contact pads were etched via reactive ion etching (Oxford Plasmalab 80) at 50 sccm O₂, 10 sccm CHF₃ and 160 W operating condition. PFT-100 solutions (30 μl) were spin-coated on top of the patterned area to form the active material films in glovebox. After that, the second sacrificed Parylene-C layer was peeled off and cleaned with DI water before polymer electrolyte coating. Then, the polymer electrolyte was spin-coated on top of the channel at 2000 rpm for 30 s. PEDOT:PSS/Au on plastic substrate was utilized as non-polarizable gate electrode, which covered the full channel area for device characterization.

Touch sensors were printed by Voltera V-one PCB printer (Voltera Inc.). Flexible conductive ink (Flex2 silver ink, Voltera Inc.) was printed on polyimide substrates and cured at 160 °C for 30 min, as two-terminal electrodes of touch sensors. PEDOT:PSS precursor solution (2 ml) was spray coated on another polyimide substrate through shadow mask, and then annealed at 140 °C for 20 min, serving as patterned sensing layer. Double-side polyimide tape (50 μm thick) was chosen as the insulating layer to form a gap between sensing layer and electrodes under non-touch condition.

Electrical and mechanical characterization

The output and transfer characteristics and other electrical properties were carried out by Keysight Precision Source/Measure Unit (B2912A series) and probe station (Karl Suss PM5). The scan rate was 5 mVs⁻¹ during the measurements of transfer and output characteristics. TA ElectroForce 3310 was used for the force measurement, which can carry out a dynamic pressure measurement with frequency as high as 100 Hz. The cyclic voltammetry measurements of the films were recorded using an electrochemical analyzer (Metrohm Autolab, PGSTAT302N). The working electrode was PFT-100 film with various geometries cast on patterned Au electrode. The absorption spectra were recorded using a UV–Vis–NIR spectrophotometer (SHIMADZU, UV-3600) over the wavelength range from 300 to 1500 nm. The thickness of the polymer films was characterized with Alpha-Step profilemeter (KLA-Tencor). The metal electrodes were deposited by thermal evaporator system (Solar-Passage, Daedong high Technologies).

Robotic integration and motion detection

To test the sliding tactile perception capability of AOAN on robotic finger, a metal ball (Ø1.5 cm, 20 g) was fixed on a linear motor (E1100, LinMot), which was used to control the sliding on the sensors array from top to down with different sliding speeds. The action frequency was defined as how fast the ball passed through each touch sensor. The higher the action frequency is, the fast the ball moves. A contact pressure of about 1 N was modulated by a commercial force sensor that was mounted between the metal ball and the platform of linear motor. The output signal was collected by the Keysight Precision Source/Measure Unit (B2912A series) in room temperature and humidity. For the slip motion detection of robot gripper, the printed sensors array was mounted on one side of the robot grippers and the grasping force was controlled by the open state of two fingers of gripper. The sliding of the metal ball from grippers was triggered by the command of slightly opening the gripper’s fingers after it’s firmly grasped. The closed-loop control with tactile feedback was performed using Matlab R2022 software with self-designed codes. For the deep learning of AOAN, we collected a custom dataset comprising three classes of object manipulation, gripping securely, slipping, slipping, and regripping for AOAN system and touch sensor alone, respectively, under identical conditions. The data set comprised 900 samples, and each sample was composed of the raw signal data from corresponding devices. The deep learning architecture was built with the PyCharm deep learning framework and Python scripts.