Introduction

Soft flex-sensors that transduce bending deformations into measurable electrical signals represent a promising technology for real-time body motion tracking, which finds important applications such as biomechanical assessment for rehabilitation, and posture reconstruction for augmented and virtual reality (AR/VR)1,2,3,4,5,6,7,8. Conventional rigid techniques, including multi-camera systems9,10 and inertia measurement units11,12, both offer mature hardware performance, but are limited by costly setups, complex calibration, and wearing discomfort. In contrast, emerging flexible and/or soft flex-sensors based on resistive13,14,15, capacitive16,17, piezoelectric18,19,20, triboelectric21,22, and optical23 mechanisms represent a promising wearable strategy to record joint flexion states. While resistive and capacitive flex-sensors adopt simple structures, they suffer from continuous power consumption and susceptibility to electromagnetic interference. Piezoelectric and triboelectric flex-sensors provide self-powered electrical signals, yet they respond selectively to dynamic stimuli. A distributed fiber-optic sensor has been devised to decouple complex deformation modes for reconstruction of multi-joint finger bending24. However, its utility might be compromised by complicated integration of multiple optical components.

Recently, the mechanically gated ion migration in skin mechanoreceptors has inspired the design of piezoionic flex-sensors to convert bending-induced stress/strain gradient into ionic voltage/current, accompanied by desirable characteristics of mechanical softness, self-powering capability, and compatibility with ionic biological processes25. Nafion-based designs have demonstrated feasibility as flexible matrix in piezoionic flex-sensors, but are constrained by low stretchability resulting in a limited sensing range (below 120 bending degrees)26,27,28. Additionally, other stretchable ionic polymers such as PAM29, PVA30, PEO31, TPU32,33,34, and PVDF-HFP35 offer improved deformability but exhibit low bending sensitivity (below 0.5 mV/degree) due to the absence of polymer-ion interaction modulation. More recently, Jia et al. regulated the polymer-ion interaction by selecting chloride salts to match the ion flux polarity with the polycation/polyanion matrix36. Although the flexoelectric coefficient was boosted, the flex-sensing performance (bending range of 75o and sensitivity of 0.04 mV/degree) still struggled to detect subtle or vigorous joint movements. Besides, previously reported piezoionic flex-sensors mostly were targeting at one dimensional or single-axis bending, which made the monitoring of multi-degree-of-freedom (multi-Dof) joint motion unachievable through a compact single device.

In this work, we propose a soft, piezoionic, and multi-Dof (SPIM) flex-sensor that offers high resolution, real-time body motion tracking. The SPIM flex-sensor features a combination of compositional and structural design. Specifically, the deliberate pairing of organic salt with polyzwitterionic hydrogel leverages ion-dipole interactions and size-induced steric hindrance effects, facilitating fast ion channels formation and amplifying charge mobility imbalance, respectively. Such dual factors synergistically contribute to a record-high piezoionic flex-sensitivity of 3.2 mV/degree, along with a broad detection range of up to 180 degrees. Finite element simulations combining poroelastic mechanics and Poisson-Nernst-Planck model corroborate the mechanistic origins of this enhancement. By adopting an architecture of square prism-shaped fiber with two pairs of symmetric nanomesh electrodes, the SPIM flex-sensor is able to decouple multi-Dof joint bending to distinguish complex body movements (e.g., flexion/extension, abduction/adduction, and clockwise/anticlockwise circumduction). We showcased the efficacy of the self-adhesive SPIM flex-sensor as an on-skin solution for high-fidelity joint kinematics tracking in metaverse applications, including digital-twin of free-pose hand motions and complex yet intuitive VR control through single-joint dexterity. Altogether, the SPIM flex-sensor marks a significant step forward in soft electronics, merging ultrahigh sensitivity, wide-angle responsiveness, and multidimensional motion resolution within a compact, wearable format suitable for next-generation human-machine interfaces.

Results

The design concept of SPIM flex-sensor

Mechanical pressure on the skin stimulates cutaneous mechanoreceptors, initiating ion migration through opened channels that transmit transmembrane ionic fluxes to the human brain for sensory interpretation (Fig. 1a). Inspired by this biological mechano-sensing process, piezoionic flex-sensors comprising a hydrogel matrix infused with mobile ions have been developed. These sensors generate a net ionic transport (i.e., ionic voltage/current) due to differences in cation/anion displacement rates under a stress/strain gradient caused by bending deformation (Supplementary Fig. 1). Previous efforts were mostly devoted to constructing piezoionic flex-sensors using different combinations of polymer and ion species, while few focused on the rational modulation of polymer-ion interactions to boost the flex-sensitivity. The piezoionic signal generation can be described as a cross-scale process: a strain gradient field across the hydrogel thickness first generates a hydrostatic stress gradient to drive an imbalanced ionic flux due to the polymer-ion modulation and ion size difference, and then the ionic flux transforms into electronic output signal at the hydrogel-electrode interface via the electrical double layer (EDL) charging (Supplementary Note 1, Supplementary Figs. 2 and 3).

Fig. 1: The design of SPIM flex-sensor and the mechanism analysis of ultrahigh flex-sensitivity.
Fig. 1: The design of SPIM flex-sensor and the mechanism analysis of ultrahigh flex-sensitivity.
Full size image

a The mechano-sensing mechanism of human skin based the mechanically gated ion transport. b Three distinct hydrogel-ion combinations of (i) inorganic salt LiCl in nonionic PAM hydrogel, (ii) inorganic salt LiCl in zwitterionic PSA hydrogel, and (iii) organic salt Li(TFSI) in zwitterionic PSA hydrogel. c Geometric configuration, and binding energy between Li+ and H2O/SBMA. d The mean square displacement (MSD) of Li+ ion in PAM and PSA hydrogels. Insets are MD simulation snapshots of the LiCl-PAM (right) and LiCl-PSA (left) systems. e The electrical potential distribution simulations under bending stimuli for LiCl-PAM, LiCl-PSA, and Li(TFSI)-PSA. VAB denotes the potential difference between the bottom and top layer at the middle position. f Experimental results of conductivity, Li+ transference number, and flex-sensitivity for the three different hydrogel-ion systems. The error bars represent the standard deviation, which is used to reflect the degree of dispersion of data for different groups (PAM-LiCl, PSA-LiCl, PSA-Li(TFSI)) in terms of conductivity, Li⁺ transference number, and flex-sensitivity. g Comparison of bending sensitivity and sensing range between SPIM sensor and other piezoionic flex-sensors.

In principle, piezoionic output is dependent on two key factors: the number of migrated charge carriers during an ionic flux event (associated with ionic conductivity) and the degree of imbalance in cationic/anionic mobility (related to ion transference number)25,37. With this in mind, we propose a synergistic strategy to achieve a record-high flex-sensitivity by regulating the imbalanced ionic flux behavior. Figure 1b illustrates three typical combinations including inorganic salt in a nonionic hydrogel, inorganic salt in a zwitterionic hydrogel, and organic salt in a zwitterionic hydrogel (chemical structures and hydrated ionic radii are listed in Supplementary Fig. 4). For the first conventional design (Fig. 1b(i)), taking lithium chloride (LiCl) in polyacrylamide (PAM) as an example, it shows a sluggish ion transport due to uncoordinated ionic transfer within the porous hydrogel network. For LiCl in a zwitterionic copolymer poly[2-(methacryloyloxy)ethyl] dimethyl-(3-sulfopropyl) (PSBMA)-co-PAM (denoted as PSA, Fig. 1b(ii)), the zwitterionic side chains contain positively charged quaternary ammonium group and negatively charged sulfobetaine sulfonate group on the same repeating unit. The zwitterionic groups enable the adsorption of counterions in the electrolyte by zwitterionic dipole-ion interaction, thereby establishing separate cationic/anionic migration channels38,39, as supported by FTIR characterization in Supplementary Fig. 5. These migration channels facilitate the effective separation of Li+ and Cl- by readily overcoming electrostatic attraction between the charged groups and counterions during deformation-induced ion flux.

Meanwhile, the solvation structure of the hydrated Li+ cations strongly affects the ion diffusion kinetics40,41,42. Density functional theory (DFT) calculations reveal that the binding energies of Li+-SBMA (−4.3 eV) is more negative than that of Li+-H2O (−1.6 eV), suggesting the preferential association with the sulfobetaine sulfonate group (Fig. 1c, detailed procedure in Supplementary Note 2). The electrostatic potential (ESP) results show that Li+-SBMA exhibits a lower ESP value with uniform surface charge distribution compared with Li+-H2O (Supplementary Fig. 6), indicating a kinetically favorable migration of Li+-SBMA. Radial distribution functions (RDFs) of the Li+-PAM and Li+-PSA systems further evidenced the strong coordinating ability of PSA to Li+ through the O atom of SBMA, which partially replaces the H2O molecules in the first Li+ solvation shell (Supplementary Fig. 7). Desolvation activation energies extrapolated from the Arrhenius equation using temperature-variable EIS curves (Supplementary Fig. 8), revealed values of 73.1 kJ/mol for PAM-LiCl and 56.2 kJ/mol for PSA-LiCl, validating the enhanced desolvation of hydrated Li+ and the resulting acceleration of cation transport kinetics. Molecular dynamics (MD) simulation was employed to further understand the ion transport behaviors inside the porous networks of PAM and PSA hydrogels upon applicaiton of an unidirectional flow pressure (detailed simulation procedure in Supplementary Note 3). According to the mean square displacement (MSD) shown in Fig. 1d, the calculated diffusion coefficients of Li+ in PAM and PSA were 4.3×10−6 and 5.5×10−6 cm2 s−1, and those of Cl- in PAM and PSA were 4.8×10−6 and 7.2×10−6 cm2 s−1 (Supplementary Fig. 9). The diffusion coefficients of both the cations and anions increased in the polyzwitterionic PSA hydrogel, indicating the boosted ion mobility due to the migration channel formation. Figure 1f compares the experimental results of conductivity, transference number and flex-sensitivity of the three hydrogel-ion combinations. These ion migration channels and the enhanced desolvation of Li+ were evidenced by the increased ionic conductivity from 16.9 to 30.2 mS/cm (EIS measurements for conductivity in Supplementary Fig. 10) and Li+ transference number (tLi+) from 0.32 to 0.4 (steady-state current method for transference number in Supplementary Fig. 11), respectively. The dual factors contributed to significantly improved flex-sensitivity from 0.17 to 1.9 mV/degree (testing details in Methods).

We further replace LiCl with Li[bis(trifluoromethylsulfonyl)imide] (Li(TFSI)) to magnify the size difference between the cation and anion (Fig. 1b(iii)). The bulky (TFSI)- anion with a large size (0.79 nm) tends to become immobilized within the crosslinked polymer networks due to steric hindrance. PSA-Li(TFSI) exhibited a remarkably improved tLi+ of 0.85, suggesting an amplified difference in cationic/anionic mobility. This significant increase of tLi+ compared with that of PSA-LiCl was due to a coupled ion-polymer transport dynamics of suppressed anion mobility and difference in ion association structure (details in Supplementary Note 4, Supplementary Fig. 12). The synergistic hydrogel-ion tuning in PSA-Li(TFSI), i.e., the cationic migration channel boosted by desolvation effect and the anionic migration channel impeded by steric hindrance, empowers a record-high piezoionic flex-sensitivity of 3.2 mV/degree.

To better understand the mechano-ionic-electrostatic coupling process in piezoionic sensing, we performed finite element simulations using COMSOL Multiphysics by combining poroelastic mechanics and Poisson-Nernst-Planck model. The simulations reproduced the physical process in a hydrogel to reveal the coupling relationship between deformation geometry and multiphysical responses including the distributions of pore pressure, ionic concentration, and electric potential (details in Supplementary Note 5, Supplementary Figs. 13 and 14). The simulation results in Fig. 1e revealed varying electrical potential difference between the bottom and top surfaces (VAB) of −2, −10.1 and 19.4 mV, respectively. The polarity reversal and amplitude increase agreed well with the proposed modulation in ion migration mobility and the resulting cationic/anionic transport imbalance (associated with ionic conductivity and ion transference number) across the three different systems. These simulation approaches constitute a cross-scale framework ranging from molecular scale to device scale, which illustrates how the polymer-ion interactions synergistically regulate ion migration to achieve an ultrahigh piezoionic sensitivity (detailed exposition in Supplementary Note 6): The molecular-level DFT calculation exhibited the energetically favorable binding of zwitterionic side chain to the Li+ ion over water molecules; These energetic parameters supported the mesoscopic-level MSD simulation of the ion transport dynamics, verifying the promoted Li+ diffusion coefficient; Device-level simulation employed FEM to transform the ion transport imbalance to the piezoionic output potential, explaining the varied sensing performance between different polymer-ion combinations. To analyze the correlation between the external mechanical excitation and the electrical signal generation, piezoionic equivalent circuit model coupling a mechanically driven current source and electrochemical impedance elements was established (Supplementary Note 7, Supplementary Figs. 15 and 16).

Unlike previous piezoionic sensors that primarily relied on enlarging ion-size disparity or introducing structural asymmetry, our polymer-ion cooperative modulation simultaneously united enhanced ionic conductivity through zwitterionic coordination and amplified ionic mobility imbalance owing to the adoption of bulky organic anions. Compared with previous piezoionic flex-sensors based on various polymer-ion combinations27,28,30,31,32,34,36 (Fig. 1g and Supplementary Table 1), the SPIM flex-sensor ranks among the highest in both flex-sensitivity and sensing range, owing to the synergistic hydrogel-ion interaction and robust hydrogel-electrode system discussed in the following section. Also, a comprehensive performance comparison between our piezoionic flex-sensor and reported flex-sensors based on other sensing mechanisms were conducted, including resistive, capacitive, piezoelectric, triboelectric, and fiber-optic types (Supplementary Table 2). Our SPIM flex-sensor shows a marked advantage in terms of the sensing range and cyclic stability (Supplementary Fig. 17).

Mechanical and electrical robustness of the hydrogel-electrode interface

The working range, signal reproducibility, and device robustness of mechanoelectrical sensors are highly dependent on the mechanical/electrical resilience and interfacial toughness43,44,45,46. Due to the ultra-softness of PSBMA hydrogel (Young’s modulus of 4 kPa), PAM was copolymerized to enhance its mechanical strength. When the molar ratio of monomers (AM to SBMA) was increased to 1:3, the resulting copolymerized zwitterionic hydrogel exhibited improved mechanical strength (Young’s modulus of 14 kPa), offering elastic resilience and free-standing capability (Fig. 2a). PSA-Li(TFSI) with a monomer ratio of 1:3 and a salt concentration of 1 mol/L was selected for further investigation. To characterize mechanical resilience, stretch-release events with stepwise tensile strains from 50% to 450% were conducted on the PSA-Li(TFSI) hydrogel. The recorded stress-strain curves indicated residual strains of 0.4% and 5.3% at tensile strains of 100% and 400%, suggesting negligible hysteresis loops during the stretch-release process (Fig. 2b). Cyclic test at tensile strain of 100% showed a low residual strain of 0.7% after 100 cycles, which further evidenced the superior elastic resilience (Fig. 2c). This rapid elastic recovery with low hysteresis may be related to the reformation of the entropy-driven zwitterionic-zwitterionic self-associations with fast crosslink dynamics47. The PSA-Li(TFSI) hydrogel was chosen as the piezoionic matrix as it possessed optimal balance in mechanical compliance, adhesive strength, and ionic conductivity (Figure Supplementary Figs. 18 and 19) compared with control hydrogels of PAM-LiCl, PAM-Li(TFSI), PSA-LiCl, resulting a highest piezoionic performance (Supplementary Figs. 20 and 21). The concentration of Li(TFSI) also influences the mechanical and electrical properties of the PSA-Li(TFSI) hydrogel. A systematic investigation was conducted to determine an optimal concentration of 1 M, which exhibited highest mechanical strength and electrical conductivity, leading to a highest piezoionic flex-sensitivity (detailed analysis in Supplementary Note 8, Supplementary Figs. 22 to 24).

Fig. 2: Mechanical/electrical resilience and interfacial toughness.
Fig. 2: Mechanical/electrical resilience and interfacial toughness.
Full size image

a Tensile stress-strain curves of the PSA hydrogel with different monomer molar ratios. b Cyclic loading-unloading stress-strain curves of PSA-Li(TFSI) hydrogel at vary strains from 50% to 450%. Inset is the bar chart of residual strains. The error bars represent the standard deviation, which is used to reflect the data dispersion degree of the residual strains in the inset bar chart. c SEM image of eTPU scaffold. Scale bar, 15 μm. d Cyclic loading-unloading stress-strain curves of the PSA-Li(TFSI) hydrogel at 100% strain for 100 cycles. Inset is the bar chart of residual strains. The error bars represent the standard deviation, which is used to reflect the data dispersion degree of the residual strains in the inset bar chart. e Resistance variation of the flexible electrodes upon cyclic bending deformation. Inset are the photos of the tested eTPU-AgNP nanomesh electrode at flat and bending states. f SEM image of a single TPU fiber, showing uniformly deposited AgNPs. Scale bar, 500 nm. g Adhesive stress curves of PSA-Li(TFSI) hydrogel attached to different substrates. h Schematic diagram of the adhesion mechanism of PSA-Li(TFSI) to different substrates.

Maintaining electromechanical stability in flexible electrodes is imperative for ensuring consistent electrical outputs. Three kinds of flexible electrodes were prepared as alternatives, as shown in Supplementary Fig. 25, including PDMS-AgNP film, TPU-AgNP film, and eTPU-AgNP nanomesh (preparation details in Methods). The eTPU-AgNP nanomesh refers to electrospun thermoplastic polyurethane (eTPU) nanofiber membrane coated with silver nanoparticles (AgNPs). The eTPU-AgNP nanomesh electrode was fabricated by combining the techniques of electrospinning and vacuum evaporation deposition to achieve a uniform nanofibrous network with conductive coating (Supplementary Fig. 26). We systematically investigated the influencing parameters in manufacturing the eTPU-AgNP nanomesh electrode with the purpose of promoting the electromechanical and sensing performance (Supplementary Figs. 27 to 29), and the optimized fabrication parameters were determined to be applied voltage of 12 kV and solution volume of 1 mL in electrospinning, and Ag usage amount of 0.474 g in AgNP deposition. The eTPU-AgNP nanomesh outperforms the TPU-AgNP film and PDMS-AgNP film as a flexible electrode in flex-sensor, as it exhibits not only a low initial resistance of 0.65 Ω cm⁻¹, but also a more stable resistance under bending deformation than those of the TPU-AgNP film and PDMS-AgNP film (Supplementary Fig. 30). Cyclic bending tests further confirmed the electromechanical robustness of the eTPU-AgNP nanomesh electrode (Fig. 2d), the eTPU-AgNP nanomesh electrode presented an extremely stable resistance (from 1.2 Ω/cm to 1.3 Ω/cm) after 10,000 bending cycles, while the PDMS-AgNP and TPU-AgNP electrodes both displayed a sharp rise in resistance leading to electric failure after several thousands of bending cycles. We attributed the superb electrical resilience to the porous microstructure of the nanofibrous eTPU scaffold and the anchoring of the percolated AgNPs network on the fiber surface (Figs. 2e and f), which could accommodate the bending strain via fiber deflection and reorientation, thereby preserving the percolation network on the fiber surface48. Besides, the eTPU-AgNP nanomesh exhibited a strongest adhesive strength to the hydrogel to facilitate a tight and gap-free bonding, which accounted for a low and stable interfacial impedance of the hydrogel-electrode interface (confocal microscopy images of electrode-hydrogel interfaces in Supplementary Fig. 31 and interfacial impedance spectra in Supplementary Fig. 32). These combined merits of the eTPU-AgNP nanomesh electrode collectively resulted in a more sensitive and stable piezoionic signal than those of TPU-AgNP film and PDMS-AgNP film (Supplementary Figs. 33 and 34). Comparative analysis between different electrodes in terms of equivalent circuit parameters was conducted to show the advantages of eTPU-AgNP nanomesh electrode in interfacial polarization, and piezoionic output (Supplementary Note 9, Supplementary Fig. 35).

An adhesive hydrogel is advantageous for achieving strong interfacial toughness with the attached electrodes for robust device operation. The PSA-Li(TFSI) gel could adhere to various substrates including glass, polyethylene terephthalate (PET), polypropylene (PP), porcine skin, and the eTPU-AgNP nanomesh electrode. The adhesive strength was measured by probe tack test (Supplementary Fig. 36), exhibiting strong adhesion across all substrates, with the highest adhesion strength of 27 kPa observed for the nanomesh electrode (Fig. 2g). This desirable interfacial robustness stems from the multiple interactions enabled by the polar and charged zwitterionic groups, such as hydrogen bonding, ion-polar interactions, electrostatic attraction (Fig. 2h). Compared with the PAM and PSA hydrogels, the PSA-Li(TFSI) exhibited markedly enhanced adhesion to different substrates owing to the electrostatic interaction offered by zwitterionic groups and the energy dissipation effect from the ion-dipole interaction (Supplementary Fig. 37).

Sensing properties of the SPIM flex-sensor

The sensing properties of the PSA-Li(TFSI) based flex-sensor were systematically investigated by sandwiching the hydrogel between two eTPU-AgNP nanomesh electrodes and recording the output open-circuit voltage (Voc) under varying bending angles (Supplementary Fig. 38 and Experimental section). The Voc increased with bending angle (θ), and the sensitivity (S), determined by linear fitting, was 3.2 mV/degree below 60 degrees and 8.6 mV/degree in the range of 60 to180 degrees (Fig. 3a). Figure 3b records the change of output potential in response to a quasi-transient bending input of 6 degrees, with the response and recovery times measured to be 300 and 250 ms, respectively. Moreover, when bent upward or downward at a same bending degree (60 degrees as the case in Fig. 3c), the Li(TFSI) flex-sensor generated equal-magnitude voltages of opposite polarity, which not only verified the piezoionic sensing mechanism of unbalanced ion migration, but also provided the capacity to distinguish two opposite bending directions. Cyclic bending of the sandwiched PSA-Li(TFSI) flex-sensor at angles ranging from 6 to 180 degrees generated consistent potential responses, indicating superb signal reproducibility (Fig. 3d). In frequency response analysis, the sensor effectively captured bending stimuli across a wide frequency range from 0.05 to 4 Hz (Fig. 3e), covering the typical motion frequencies of human joints in daily life. A cyclic bending test at 40 degrees showed no obvious degradation of output potential throughout 10,000 cycles (Fig. 3f), indicating reliable sensing durability. The combinatory sensing parameters of ultrahigh sensitivity, wide sensing range and extended durability stem from the integrated effects of synergistic hydrogel-ion interactions, mechanical/electrical resilience and robust hydrogel-electrode interface.

Fig. 3: Sensing properties of PSA-Li(TFSI).
Fig. 3: Sensing properties of PSA-Li(TFSI).
Full size image

a Potential signal versus increasing bending angle, showing two linear segments. The error bars represent the standard deviation, which reflects the dispersion degree of the data. b Potential signal variation under transient bending and recovery conditions. c Potential signal during consecutive upward and downward bending deformation of 60 degree. d Potential signal under repeated bending of increasing angle from 6 to 180 degrees. Inset shows the potential signal at 6 degrees. e Potential signal at different bending frequencies. f Potential signal during 10,000 bending cycles at 40 degrees. Insets show the consistent signal across the whole cycling test.

SPIM flex-sensor enables digital-twin of free-pose hand motion

The human hand possesses complicated motion agility essential for manipulation tasks for daily functions49,50. The excellent flex-sensing performance of the SPIM device motivated the design of a flex-sensor composed of a square prism-shaped PSA-Li(TFSI) fiber (side length of 6 mm, height of 40 mm) and two pairs of eTPU-AgNP nanomesh electrodes (Fig. 4a). The wrist joint was selected as the target considering its multi-Dof motions (Fig. 4b), including flexion/extension, abduction/adduction, and clockwise/counterclockwise circumduction. The adhesive SPIM flex-sensor was attached to the carpal bone position to record multi-directional joint bending. Upon wrist flexion at increasing bending angles from 10 to 90 degrees, the SPIM flex-sensor generated potential signals with corresponding increasing magnitudes from 16 to 450 mV accordingly (Fig. 4c). During consecutive wrist extensions at a fixed bending angle of 40 degrees, performed at varying frequencies from 0.2 to 1 Hz, the SPIM flex-sensor produced potential signals of consistent amplitude matching the actuation frequencies (Fig. 4d).

Fig. 4: Digital-twin of free-pose hand motion.
Fig. 4: Digital-twin of free-pose hand motion.
Full size image

a Schematic diagram of SPIM sensor attached to wrist joint with multi-Dof motions. b Schematic illustration of multi-Dof wrist bending, including abduction, adduction, flexion, extension, clockwise and anticlockwise circumduction. c Potential signals generated by the SPIM sensor upon wrist flexion at increasing bending angles from 10° to 90°. Inset shows the potential signal at bending angle of 10°. d Potential signals generated by the SPIM sensor upon wrist extension at 40° across varying frequencies from 0.2 to 1 Hz. e Flow chart of the SPIM sensor based wearable platform to creating real-time digital-twin of hand motion. f A list of the wrist motions for digital-twin creation in VR space through the SPIM sensor platform.

The accurate differentiation of bending rate and amplitude allowed the integration of the SPIM flex-sensor into a wearable platform for a real-time digital-twin reconstruction of hand motions (Fig. 4e). The two pairs of electrodes, aligned along the X- and Y-axes directions, generated output potential signals upon wrist bending, with the signal polarity (positive or negative) and magnitude relating to the bending direction and amplitude. For axial bending or bending around arbitrary direction, the bending amplitude (or bending curvature) and direction can be deduced from the piezoionic potential output of the two pairs of electrodes in X- and Y-axis (Supplementary Note 10). Through data collection and processing via customized microcontroller modules (see Methods), the potential signals could be translated into free-pose hand motion animations in digital Unity interface. Figure 4f delineates the transformation of distinct signal patterns of X- and Y-axis electrodes to digital-twin representation of hand movements in VR space (Supplementary Movie 1), including flexion and reversion, abduction and reversion, clockwise circumduction, etc. For hand motions of flexion/extension or abduction/adduction, only one pair of the X- or Y-axis electrodes produced potential signals. In contrast, for hand motions of wrist circumduction, the SPIM flex-sensor gave rise to triangular wave potential patterns in X- and Y-axis electrodes, with a 1/4 cycle phase difference for anticlockwise motion or a 3/4 cycle phase difference for clockwise motion. To showcase the versatility of the SPIM-flex sensor in multi-joint use, we extended the deployment from wrist joint to elbow and shoulder joints, which also involved multi-Dof motions and even larger angular displacement. The deployment of SPIM flex-sensor, the typical output signal of multi-Dof motions, and the corresponding motion capture in Unity space were shown in Supplementary Fig. 39. This extensible approach allows the monitoring of different joints without interference, enabling synchronized and more complex motion capture applications, such as the “arm raising and hand waving” movements (Supplementary Fig. 40). The accurate reconstruction of joint kinematics is essential for biomechanical evaluation in rehabilitation and immersive experiences in VR/AR scenarios.

Single-joint VR control

The high-fidelity capturing of dexterous joint motion using a single SPIM flex-sensor is uniquely suitable for sophisticated yet intuitive VR control. The skin-integrated SPIM flex-sensor generates distinctive signal patterns from the combinations of X- axis (flexion/extension direction) and Y-axis (abduction/adduction direction) electrodes, which can be further translated into control commands for driving a virtual car in VR space. In this system, the amplitude and direction of joint bending are intuitively converted to the speed and direction of the virtual car (Fig. 5a). With X-axis signals assigned to forward/backward motion and Y-axis signals to left/right steering in the VR space, an operator wearing the self-adhesive SPIM flex-sensor on the wrist was able to issue diverse driving commands through static wrist postures and dynamic wrist motions. The mathematical model of the multi-Dof bending in VR control was investigated to obtain a quantitative mapping from mechanical input to electrical output, which lays the foundation for the technical implementation in transforming physical deformations to virtual movements (Supplementary Note 11, Supplementary Figs. 41 to 45). Figure 5b outlines eight representative driving commands, accompanied by the corresponding joint postures/motions with specific time-evolving bending angles in X- and Y-axis directions. Specifically, flexion or extension postures led to constant-speed driving forward or backward respectively; The bending angle increase or decrease in flexion angle resulted in car accelerating or decelerating and braking; When abduction or adduction was coupled with flexion, the virtual automobile turned left or right; Coupling mixed abduction-flexion or adduction-flexion motions with subsequent pure flexion posture translated into lane-changing to left or right. As a result, the SPIM flex-sensor enabled the steering of a virtual automobile in a VR space of traffic roads through a series of driving commands, as shown in Fig. 5c (Supplementary Movie 2). The ability to perform complicated VR control through a single joint qualified the skin-integrated SPIM flex-sensor as an efficient, intuitive, and versatile controlling solution for metaverse applications.

Fig. 5: Single-joint VR control through a single compact sensor device.
Fig. 5: Single-joint VR control through a single compact sensor device.
Full size image

a Flow chart of single-joint VR control through a skin-integrated SPIM flex-sensor, including the distinct output potential signal patterns, the MCU system and the single-joint VR control. b A list of driving commands for a virtual automobile in VR space, including photos of the joint postures/motions, the specific bending angle variation in X- and Y-axis directions, and the snapshots of the controlled automobile in VR space. c The SPIM flex-sensor enabled the steering of a virtual automobile in a VR space of traffic roads by easy and intuitive hand motion interactions.

Discussion

Accurate human motion tracking relies on preciserecording of the pertinent joint kinematics. We devised a soft, piezoionic, and skin-integrated flex-sensor capable of identifying joint bending states with exceptional sensitivity and multi-degree-of-freedom (multi-Dof) resolution. The synergistic hydrogel-ion interaction of PSA-Li(TFSI) promote cationic migration via boosted channels by desolvation effect and anionic migration channel impeded by steric hindrance, collectively enabling ultrahigh piezoionic sensitivity. A rational geometry of square prism-shaped hydrogel fiber with symmetric electrode pairs empowered the SPIM flex-sensor to detect and differentiate multi-Dof bending movements. Furthermore, the superior mechanical/electrical resilience and robust interface of the hydrogel-electrode system bestowed durable flex-sensing performance. The combinatory sensing parameters of the SPIM flex-sensor exhibits one of the highest-performing sets of sensing parameters among reported piezoionic flex-sensors. To prove the versatility of the SPIM flex-sensor in high-fidelity motion tracking, we showcased metaverse applications including digital-twin of free-pose hand motion and complex yet intuitive VR control through single-joint dexterity. The proposed SPIM flex-sensor holds great potential for future real-time and precise biomechanical analysis and next generation immersive entertainment scenarios.

Methods

Materials

The raw materials used in the experiments were: sulfobetaine methacrylate (SBMA), acrylamide (AM), lithium chloride (LiCl), lithium trifluoromethylsulfonimide (Li(TFSI)), potassium persulphate (KPS), N, N’-methylene bisacrylamide (MBAA), N, N-dimethylformamide (DMF), thermoplastic polyurethane (TPU), Polydimethylsiloxane (PDMS), silver particles (Ag), copper-nickel conductive adhesive tapes, and silver paint.

eTPU-AgNP nanomesh

The detailed fabrication procedure of the eTPU-AgNP nanomesh electrode is as follows: (i) Thermoplastic polyurethane (TPU) was dissolved in the solvent of N, N-dimethylformamide (DMF), with the polymer concentration and viscosity adjusted to achieve stable jet formation. (ii) The prepared solution was loaded into a syringe and ejected through a metallic needle under a high-voltage electric field. Electrospinning parameters were controlled, including applied voltage (10-16 kV), solution feeding volume (0.2-2 mL), flow rate (0.5-1.0 mL h⁻¹), and tip-to-collector distance (15-20 cm). (iii) Electrospun nanofibers were collected on a rotating drum collector, where the rotation speed (100-300 rpm) was adjusted to control fiber alignment and network density. (iv) AgNPs were deposited onto the fibrous mats by vacuum evaporation deposition. The deposition thickness was precisely controlled to achieve an optimal balance between high electrical conductivity and mechanical flexibility.

TPU-AgNP film

A solution with a mass fraction of 20% was prepared by dissolving 5 g of TPU in 20 g of DMF and stirred at 70 °C for 5 h. After that, the solution was drop-coated on a glass template and dried in a vacuum oven at 70 °C for 9 h to obtain a TPU film. Finally, metallic silver particulate material was deposited on the TPU film.

PDMS-AgNP film

Droplets of PDMS precursor were applied to the glass template and cured in a vacuum oven at 80 °C for 30 minutes to obtain the PDMS film. Finally, metallic silver particulate material was deposited on the PDMS film.

Preparation of the hydrogels

For control experiments, three different gels were experimentally prepared, including PAM and P(SBMA-AM) hydrogels without salt and P(SBMA-AM) ionic hydrogels containing LiCl or Li(TFSI), respectively. In this work, the ionic hydrogels were prepared by a one-pot method as follows: 0.5 g of PAM, 1.5 g of SBMA (or none in the case of pure PAM) were homogeneously mixed with 10 ml of deionized water, then 2.87 g of Li(TFSI), or 0.42 g of LiCl were added and stirred for half an hour, followed by the addition of 4 mg of MBAA and 20 mg of KPS and stirred for one hour to obtain a transparent solution, which was finally injected into a homemade glass mold and removed after 9 hours of heat curing to obtain a hydrogel.

Fabrication of the ionic flexible sensor

Ionic sensor with sandwich structure: the electrode material was cut into a rectangle of 7 cm (length) × 1 cm (width), copper-nickel conductive tape was applied as a lead at one of its ends and fixed using silver paste at the joint. The ionic gel was cut into long strips of 4 cm (length) × 1 cm (width) × 2 mm (thickness) size. Finally, the electrodes were affixed to the top and bottom sides of the ionic hydrogel, assembled into an ionic sensor with a sandwich structure and encapsulated using silicone rubber.

Strip-structured ionic sensor

the electrode material was cut into a rectangle of 7 cm (length) × 0.5 cm (width), which was cut to obtain the topology-optimized structure of the electrodes, and Cu-Ni conductive adhesive tape was applied to one end as a lead, and the joint was fixed using silver paste. The ionic gel was cut into long strips of 4 cm (length) × 6 mm (width) × 6 mm (thickness) size. Finally, the electrodes were attached to the four side walls of the ionic hydrogel to assemble the ionic sensor in a strip structure and encapsulated using silicone rubber.

Characterization

Morphology characterization was conducted using a field emission scanning electron microscope (FESEM, SU8010, Hitachi, Japan). The transmittance spectrum was measured by an ultraviolet-visible spectrophotometer (LAMBDA1050, PerkinElmer Instruments, USA). The attenuated-total-reflection (ATR) FTIR spectrum was collected using an attenuated-total-reflection Fourier transform infrared spectrometer (PerkinElmer, Spotlight 400, USA). The mechanical tests were conducted on a universal material testing machine (CMT6103, Metus Industrial Systems China). The resistance of the electrodes was characterized by measuring the voltammetric (VA) curves using an electrochemical workstation (Autolab, Germany), and the slopes are resistance values.

EIS test

Two clean Pt sheets were used as electrodes, and 1.4 cm (radius) × 2 mm (thickness) gel electrolyte was filled between the two stainless steel sheets to ensure good contact, then the sample was subjected to an AC impedance test, with the frequency range set from 0.01 Hz to 100,000 Hz. The test results show a Nyquist impedance curve with an intercept on the x-axis, which is the solution resistance R. The ionic conductivity σ (mS/cm) of the electrolyte is determined by the following equation.

$$\sigma=\frac{1}{R}\cdot \frac{L}{S}$$
(1)

Where L is the gel electrolyte thickness (m); S is the gel electrolyte effective area (m2).

Ion mobility number test

Lithium ion transference number (tLi+) of Li | |PAM-LiCl | |Li, Li | |PSA-LiCl | |Li, and Li | |PSA-Li(TFSI)||Li symmetric cells were tested by the steady current method (i-t) at a constant polarization potential of 10 mV. According to the following equation.

$${t}_{{{Li}}^{+}}=\frac{{I}_{S}\left(\Delta V-{I}_{0}{R}_{0}\right)}{{I}_{0}\left(\Delta V-{I}_{s}{R}_{s}\right)}$$
(2)

where I0 is the initial current, Is is the steady-state current, ΔV is the applied polarization voltage, R0 is the initial resistance, and Rs is the steady-state resistance.

Adhesion test

The adhesion strength was determined by a manometer (M5-20, Mark-10, America) by first applying a compressive pre-stress of 20 N between the electrode material and the gel, and then the manometer was separated at a travelling rate of 50 mm/min after 20 s. The tensile adhesion strength was determined by the following equation:

$$P=-\frac{F}{S}$$
(3)

Where F is the peak force that can be read by the dynamometer during the movement (the tension is negative), indicating the instant of separation of the gel film from the electrode film. S is the area of the electrode film in contact with the gel film, the size of the area is a 1.2 cm (diameter) circle.

Bending sensing performance test

The sensor was placed on a PET membrane and the PET membrane with the sensor was fixed between a motorized translation table (CMT6103, MTS Systems, China) using a fixture with an initial free length of the membrane of 5 cm. A computer-controlled stepper motor was used to control the displacement and speed of the membrane. When the device is bent, the change in open-circuit voltage at the upper and lower ends of the electrodes of the device can be tested by using an electrochemical workstation (Autolab, Germany).

The thin film sensor has a thickness of h and a length of L. The film bends when the displacement stage is moved by x. We assume that the center angle of the curved arc is θ, and the radius is R. According to the geometric relationship, we can list the equation.

$$\theta \times R=L$$
(4)
$$2R\times \sin \frac{\theta }{2}=L-x$$
(5)

Knowing L and x, we can obtain θ and R from the equations. Finally, the bending strain ε can be calculated.

$$\varepsilon=\frac{h}{2R}$$
(6)

Finally, the bending strain and angle were calculated using the software MATLAB based.

Data processing process from the sensor to the user interface:

  1. 1.

    Sensor Input:

    Flex sensors are fixed on the wrist and are capable of monitoring the bending of hand joints along the x and y axes. The sensors convert physical movements into electrical signals.

  2. 2.

    Sensor Output:

    The output signal is typically analog. Oscilloscope graphs display the signal waveforms over time across different channels, representing various hand movements.

  3. 3.

    Analog-to-Digital Converter (ADC Converter):

    The ADC converter transforms the continuously varying analog signals from the sensors into discrete digital signals that computers can handle, facilitating processing by microcontrollers (Arduino).

  4. 4.

    Arduino Signal Processing and Data Collection:

  1. (1)

    Signal Processing

    Inside the microcontroller system, the signal first undergoes filtering through a moving average filter, followed by real-time angle data calculation using an angular voltage linear regression model based on the voltage signals.

  2. (2)

    Data Collection:

Arduino is responsible for collecting the real-time angle data and communicating it to the computer via a serial port connection.

  1. 5.

    Serial Port Connection:

    Data is transferred from the Arduino to the PC using a serial port (USB).

  2. 6.

    C# Script:

    On the PC, a C# script is used for further data processing and analysis. The script controls virtual objects for visualization based on the data received.

  3. 7.

    Unity Interface:

    Unity, a powerful game engine, is used here to develop the user interface (UI), visualize the processed data, and facilitate interaction with virtual objects.

  4. 8.

    Virtual Objects:

In Unity, users can see direct feedback of hand movements, such as a virtual hand on the screen mimicking the user’s actual hand movements, or controlling the movement of a car through wrist motion.