Introduction

Cutaneous receptors found in skin tissue, such as the dermis and epidermis, provide an immediate and sensitive perception of external stimuli. These receptors can be classified into mechanoreceptors that detect pressure and shear stress and thermoreceptors that sense temperature1. Advances in research fields, including sensors2,3,4,5,6,7,8,9,10,11,12,13, prosthetics14,15,16,17, and robotics17,18,19 have been associated with the development of artificial tactile sensory systems that mimic the perceptive characteristics of cutaneous receptors or human skin (i.e., electronic skin, e-skin)20. In particular, the significance of electronic skin (e-skin) has been growing owing to its applications in personalized healthcare and biomedical fields2,3,4,5,6,7,8,14. Wireless, skin-mountable e-skin provides medical staff with sensing data in real-time for disease prevention and treatment, including pressure injury prevention2,3,5,8, sleep monitoring4, compression therapy for venous leg ulcers6, nerve injury prevention7, and optimizing prosthetic interfaces14. Unfortunately, the development of wireless sensors that can simultaneously measure pressure, shear stress, and temperature at the skin interfaces has been limited due to the challenges in integrating battery-free wireless technologies (e.g., Near Field Communication; NFC) with multi-axial sensors while satisfying a thin and small form factor without cumbersomeness and ensuring robustness of the wireless sensor11,21,22,23,24,25.

Augmented reality (AR) platform overlays virtual information (e.g., sensing data) onto the user’s field of vision in the real world using smart devices such as AR glasses, tablets, and smartphones, thereby offering an immersive and interactive environment to enhance the user’s intuitive understanding of the sensing data26. Although demand for AR technology in the healthcare market has been growing due to its applications in surgery, rehabilitation, diagnosis, education, and therapy27,28, previously reported wearable sensors integrated with AR platforms mainly focused on Human-Machine Interface (HMI) for interactive experiences, such as light dimming29, flower planting30, and estimation of body joint angles31 by measuring gestures, pressure, and strain at human skin interfaces. Despite the importance of integrating wearable sensors and AR platforms for healthcare applications, only a limited number of such systems have been developed28,32. The majority of wearable devices currently used in conjunction with AR systems are rigid and cumbersome, making it difficult to apply them to the skin interfaces of patients for healthcare applications26,31, including wired sensors in surgical training systems for educational purposes28,32. For comfortable wearability, the development of wearable sensors integrated with an AR platform would meet challenging requirements, including (1) wireless power transfer to enable both a thin device design without bulky batteries and long-term use31,33, (2) flexible and stretchable substrates that are mechanically compliant with the skin interfaces31, and (3) a miniaturized electronics form factor for user’s comfort2,26,31.

In this paper, we introduce a novel system that integrates a wireless, battery-free platform with AR platform for healthcare applications28,32,34,35,36,37,38. Specifically, we have developed a wireless, battery-free multi-axial sensor for AR-assisted monitoring of pressure, shear stress, and temperature at the skin interfaces. The multi-axial sensor includes four tri-layered cantilever strain gauges bound by a soft elastomeric cover. The combination of electrical signals obtained from the four cantilever strain gauges under applied pressure and shear stress provides information on the magnitude and direction of the resultant forces (i.e., pressure and shear stress). The overall system is divided into two main parts as follows: (1) an NFC-based wireless sensing platform for continuous measurement of pressure, shear stress, and temperature in real-time, and (2) an AR system that visualizes the sensing data using smart devices for an immersive and interactive environment. For multi-purpose biomedical applications, we applied the overall system to the subjects sitting in a wheelchair and lying in bed, using machine learning to demonstrate its efficacy in (1) preventing pressure injuries6,7,14,39, (2) monitoring posture to prevent disc herniation40,41,42,43,44,45,46,47,48, and (3) intuitive monitoring of forces applied to the skin interfaces with the AR system28,32. Moreover, the validation trials with the subjects demonstrated the robustness and compliance of the wireless sensor without any discomfort during multi-purpose monitoring, thanks to its thin and small form factor enabled by battery-free, wireless technology.

Results

Research overview and device design

Figure 1a presents wireless, battery-free multi-axial sensors attached to the skin interfaces of the subject for AR-assisted, intuitive monitoring of resultant forces (i.e., pressure and shear stress) applied to the body. The changes in electrical outputs (i.e., sensing data) of the wireless sensor under applied pressure and shear stress are wirelessly transmitted to the laptop according to the ISO 15963 protocol of Near Field Communication (NFC) at a resonant frequency of 13.56 MHz. The laptop converts the digital signals into XYZ-axis forces through machine learning for real-time monitoring of resultant forces applied to the skin interfaces. Then, the data obtained from multiple wireless sensors can be input into the machine learning algorithm to accurately predict the subject’s posture (i.e., posture monitoring). For an immersive and interactive AR environment in biomedical applications, the laptop transmits the sensing data obtained from an NFC-based wireless platform to the smart devices (e.g., AR glasses and smartphones), leading to an overlay of sensing data onto the user’s field of vision. Table S1 highlights the significant advantages of our sensing platform integrated with an AR system, including multi-modal sensing capabilities, wireless operation without bulky batteries, and a thin and small form factor, compared to existing sensing platforms integrated with AR/VR systems for healthcare applications. For application on skin interfaces, achieving a small and thin form factor of the wireless sensor while ensuring mechanical durability during clinical trials is both essential and challenging. As listed in Table S1, previous flexible sensors integrated with AR/VR systems have not demonstrated sufficient mechanical durability to withstand human body weight. On the other hand, our multi-axial sensor not only demonstrated the mechanical robustness and durability during healthcare applications but also exhibited excellent capabilities (i.e., form factor, linearity, and sensitivity) compared to previous multi-axial sensors capable of measuring pressure and shear stress, as summarized in Table S2. In particular, the integration of the multi-axial sensor with an AR system is rarely seen, despite its potential importance and utility in personalized healthcare and biomedical fields2,3,4,5,6,7,8,14.

Fig. 1: Schematic illustrations and photographs of a battery-free, wireless multi-axial sensor.
figure 1

a Research concept of an augmented reality (AR)-assisted, wireless, battery-free multi-axial sensors for an immersive AR environment in biomedical applications. b Exploded view of the wireless sensor. Insets indicate an assembled multi-axial sensor (highlighted in red) and a photograph of electronic components on fPCB (highlighted in blue). Photographs of the wireless sensor (c) before PDMS encapsulation and (d) after PDMS encapsulation, attached to lower back using transparent film dressing (1628 W, TegadermTM, USA).

Full size image

Figure 1b illustrates an exploded view of the wireless sensor that includes the multi-axial sensor (highlighted in red; 8 × 8 × 1.9 mm3) and flexible PCB (fPCB, polyimide (PI)/Cu/PI, 35/44/35 μm thick) with electronic components (highlighted in blue) including NFC SoC (RF430FRL152H, Texas Instruments, USA), a multiplexer (MUX; TMUX1108, Texas Instruments, USA), and an amplifier (AMP; INA333AIDRGT, Texas Instruments, USA). The multi-axial sensor consists of the following components: a bottom rigid substrate (silicon wafer, 250 μm thick), an NTC thermistor (NTCG064EF104FTBX, TDK Corporation, Japan) under fPCB (Supplementary Fig. 1), four tri-layered cantilever strain gauges (PI/Gold (Au)/PI, 73 μm thick) on glass substrates (glass wafer, 250 μm thick), a soft cover (polydimethylsiloxane (PDMS), 1350 μm thick), and a rigid cover (Poly(methyl methacrylate) (PMMA), 200 μm thick). Figure 1c presents the wireless sensor before a PDMS encapsulation process, which involves using upper and lower PDMS layers fabricated through a molding process. After PDMS encapsulation of the wireless sensor, the wireless sensor can be protected from sweat or liquid while maintaining its flexible and elastic properties under mechanical stimuli such as stretching, twisting, and bending owing to serpentine structure of fPCB and elastic property of PDMS layers as shown in Supplementary Fig. 2. Figure 1d shows a skin-mountable, wireless sensor attached to the lower back using an irritation-free, breathable, transparent film dressing (1628 W, TegadermTM, USA) for biomedical applications.

Design features of the multi-axial sensor

Figure 2a, b present a tri-layered cantilever strain gauge (PI/Au/PI, 3/0.03/70 μm thick) on a glass substrate with a cavity and Au patterns with 5 μm pitch on a PI film, fabricated by a lithography process, respectively (Supplementary Fig. 3). Figure 2c shows a cross-sectional view of the multi-axial sensor consisting of (1) four cantilever strain gauges on a glass substrate with Au patterns below the neutral axis (N.A.), (2) a soft cover (PDMS) bonded with four cantilevers, (3) a NTC thermistor in the bottom center, and (4) a rigid cover (top; PMMA) and a rigid substrate (bottom; silicon wafer) sandwiching the multi-axial sensor. The components are firmly bonded to complete the multi-axial sensor using epoxy resin (1365868-6, Loctite, USA) and silver epoxy (8331-14 G, MG chemicals, Canada) after MPTMS treatment for the elastomeric components in Supplementary Fig. 449. The rigid components of the multi-axial sensor at the top and bottom provide bending insensitivity in Supplementary Fig. 52,3,5,8. Four cantilever strain gauges bound with the soft cover deform simultaneously under external stimuli (i.e., pressure of 10 kPa and shear stress of 10 kPa) applied to the soft cover, as shown by finite element analysis (FEA) results in Fig. 2d. The deformation of four cantilever strain gauges generates four electrical outputs, thereby distinguishing the direction and magnitude of resultant forces11,22. Figure 2e presents the small form factor of each cantilever strain gauge on a glass substrate (3 × 2 × 0.3 mm3) and the multi-axial sensor for accurate measurement2,6.

Fig. 2: Design features of a multi-axial sensor.
figure 2

a Schematic illustration of a tri-layered cantilever strain gauge (PI/Au/PI) on glass with a cavity, and b microscopic image of Au trace of the strain gauge. c Cross-sectional schematic illustration of the multi-axial sensor. d FEA results of the multi-axial sensor under a pressure of 10 kPa and a shear stress of 10 kPa. e Photograph of top view of the multi-axial sensor consisting of four cantilevers without a soft cover. f Schematic illustration and (g) photograph of three-axis testing machine, including XYZ-axis actuator and XYZ-axis load cell to apply and measure pressure and shear stress.

Full size image

Figure 2f, g indicates a schematic illustration and photograph of a customized three-axis testing machine that includes XYZ-axis actuator and XYZ-axis load cell for precise loading and measuring of pressure and shear stress. The experimental setup for precise measurement obtains seven data simultaneously using software (LabVIEW, National Instruments, USA) on a laptop, as shown in the equivalent circuit diagram in Supplementary Fig. 6: (1) four sensing data from four Wheatstone bridges and four amplifiers (HX711, Avia Semiconductor) for each cantilever strain gauge, linked to a development board (Arduino Mega2560, BCMI, Italy), and (2) three XYZ-axis force data from XYZ-axis load cell of three-axis testing machine.

Performance characteristics of the multi-axial sensor

External stimuli applied to the top of the multi-axial sensor deform four cantilever strain gauges, resulting in fractional changes in resistance (ΔR/R0) where ΔR = R0−R. Figure 3a presents FEA results of the strain distribution of four cantilever strain gauges of the multi-axial sensor under a pressure of 30 kPa. Inset, highlighted in red, indicates a compressive strain of 0.23% on the bottom side of the cantilever, opposite to the top side of the cantilever that shows a tensile strain of 0.21%. The strain distribution resulted from the structural characteristics of a cantilever under a vertical load, wherein the section below the neutral axis (N.A.) exhibits compressive strain, while the section above the N.A. simultaneously exhibits tensile strain, in accordance with the principle of solid mechanics. As Au patterns are placed below the N.A. as shown in Fig. 2a, c, resistance (R) of the cantilever decreases under pressure, while changes in resistance (ΔR = R0−R) increase with the initial constant resistance (R0). As the length (L) of the cantilever shown in Fig. 3a increases from 0.6 mm to 1.2 mm, the sensitivity of the multi-axial sensor for pressure increases from 4.6 × 10−5 kPa−1 to 1.35 × 10−4 kPa−1 in Fig. 3b. A length of 1.2 mm was chosen, given that the maximum pressure measured at the interfaces between skin and memory foam does not exceed 50 kPa2,3,5,8. For convenience, Supplementary Fig. 7a shows the schematic illustration of the multi-axial sensor with each cantilever strain gauge numbered (e.g., cantilever n, n = 1–4) for an intuitive understanding of the direction of external stimuli applied to the multi-axial sensor. Figure 3c shows the fractional changes in resistance of the multi-axial sensor under pressure loading and unloading of 43 kPa.

Fig. 3: Mechanism and performance of a multi-axial sensor.
figure 3

a FEA results of the multi-axial sensor under a pressure of 30 kPa. b Fractional change in resistance of cantilever strain gauge with different lengths (L) of cantilever. c Fractional changes in resistance of four cantilevers under pressure loading and unloading. d FEA results of the multi-axial sensor under a shear stress of 10 kPa. Fractional changes in resistance of four cantilevers under shear stress loading and unloading (e) in the direction from cantilever 3 to cantilever 2 and (f) in the direction from cantilever 2 to cantilever 3. g FEA results of the multi-axial sensor under a shear stress of 14 kPa. Fractional changes in resistance of four cantilevers under shear stress loading and unloading (h) in the direction from the midpoint between cantilever 1 and cantilever 3 to the midpoint between cantilever 2 and cantilever 4 and (i) in the direction from the midpoint between cantilever 1 and cantilever 2 to the midpoint between cantilever 3 and cantilever 4. j FEA results of the multi-axial sensor under a pressure of 30 kPa and then shear stress of 10 kPa. Responses of the multi-axial sensor under (k) combined loads (e.g., pressure of 15 kPa and shear stress of 15 kPa) and (l) shear stress of 7 kPa.

Full size image

Figure 3d shows the deformation of two cantilever strain gauges under a pure shear stress of 10 kPa in the direction from left to right. The FEA results present the compressive strain distribution of 0.12% in the right cantilever strain gauge and the tensile strain distribution of 0.12% in the left cantilever strain gauge, while the other two cantilever strain gauges remained undeformed. For clarity, while the FEA results present the strain on the top side of the cantilever, the bottom side of the cantilever, where Au patterns are located, exhibits the opposite strain distribution. Likewise, the experimental results in Fig. 3e, f showed a similar trend to the FEA results. Figure 3e presents the fractional changes in resistance under shear stress loading and unloading of 12 kPa in the direction from cantilever 3 to cantilever 2, indicating a decrease in resistance of cantilever 2 (ΔR > 0) and an increase in resistance of cantilever 3 (ΔR < 0) without changes in resistance of cantilever 1 and 4. The sensitivity of the multi-axial sensor for shear stress was 2.42 × 10−4 kPa−1.

In Fig. 3g, FEA results present the strain distribution of the multi-axial sensor when a pure shear stress of 14 kPa is applied in the direction from left to right. The compressive strain distribution of 0.12% occurred in the two right cantilever strain gauges, while the tensile strain distribution of 0.12% occurred in the two left cantilever strain gauges. Likewise, while the FEA results show the strain on the top side of the cantilever, the bottom side with Au patterns presents the opposite strain distribution. Figure 3h, i and Supplementary Fig. 7c, d indicates the experimental results for a pure shear stress applied to the multi-axial sensor, demonstrating a similar trend to the FEA results. Figure 3h presents the fractional changes in resistance of four cantilever strain gauges under shear stress loading and unloading of 14 kPa in the direction from the midpoint between cantilever 1 and cantilever 3 to the midpoint between cantilever 2 and cantilever 4, showing decreases in resistance of cantilever 2 and 4 (ΔR > 0) and increases in resistance of cantilever 3 and 1 (ΔR < 0).

Moreover, a combined pressure of 30 kPa and shear stress of 10 kPa were sequentially applied to the multi-axial sensor to assess the effect of the resultant forces as shown in the FEA results (Fig. 3j). The FEA results present the compressive strain distribution of 0.23% in four cantilever strain gauges under pressure of 30 kPa. Sequentially, the compressive strain distribution of 0.35% occurred in the right cantilever strain gauge under shear stress loading of 10 kPa in the direction from left to right, while the compressive strain distribution in the left cantilever strain gauge decreased from 0.23% to 0.11%, a reduction of 0.12%, due to the directionality of the shear stress. The experimental results for a combined pressure of 15 kPa and shear stress of 15 kPa applied to the multi-axial sensor also demonstrated a clear trend in recognizing the directionality of the resultant forces in Fig. 3k. The fractional changes in resistance of four cantilevers (1–4) increased (ΔR > 0) from 0% to 0.2% under a pressure of 15 kPa. Subsequently, the fractional change in resistance of cantilever 2 increased (ΔR > 0) from 0.2% to 0.56%, an increase of 0.36%, while the fractional change in resistance of cantilever 3 decreased (ΔR < 0) from 0.2% to −0.18%, a reduction of 0.38%, under shear stress of 15 kPa in the direction from cantilever 3 to cantilever 2. Under shear stress of the same magnitude in the opposite direction, the fractional change in resistance of cantilever 3 increased (ΔR > 0) from 0.18% to 0.56%, an increase of 0.38%, whereas the fractional change in resistance of cantilever 2 decreased (ΔR < 0) from 0.2% to −0.17%, a reduction of 0.37%. In addition, the drift of the multi-axial sensor was determined to be 0.09% over the full scale range (ΔR/R0 of 0.6) using the equation (Output 1–Output 2)/0.6 as shown in Supplementary Fig. 8. Supplementary Fig. 9 presents the response of the multi-axial sensor under more than 1000 cyclic pressure loadings of 20 kPa and shear loadings of 10 kPa.

The responses of four cantilever strain gauges (i.e., ΔRn = R0−Rn, n = 1–4) can determine the pressure applied to the multi-axial sensor using Eq. (1). Shear stress applied to the multi-axial sensor in X and Y-axis (See Supplementary Fig. 7a) can be expressed using Eqs. (2) and (3).

$${F}_{Z}=(\Delta {R}_{1}+\Delta {R}_{2}+\Delta {R}_{3}+\Delta {R}_{4})/{C}_{1}$$
(1)
$${F}_{X}=(\Delta {R}_{3}-\Delta {R}_{2})/{C}_{2}$$
(2)
$${F}_{Y}=(\Delta {R}_{1}-\Delta {R}_{4})/{C}_{2}$$
(3)

Where C1 and C2 are constants that vary according to the sensitivity of the multi-axial sensor. For a sensitivity of 1.35 × 10−4 kPa−1 for pressure and 2.42 × 10−4 kPa−1 for shear stress, C1 and C2 are 1.75 and 5, respectively. The linear superposition of the responses of four cantilever strain gauges can determine the combined pressure (i.e., FZ) and shear stress (i.e., FX and FY) using Eqs. (1)–(3) as shown in Supplementary Fig. 10a–c, which was determined based on the responses of four cantilever strain gauges in Fig. 3k. In addition, the root mean square error (RMSE) between the reference value (i.e., XYZ-axis load cell) and the determined value from Eqs. (1)–(3) are 5.7%, 7.2%, and 2.9% for X, Y, and Z-axis, respectively, as shown in Supplementary Figs. 10a–c. Supplementary Fig. 11 shows the temperature-compensated response of the cantilever strain gauge (blue), the response of the cantilever strain gauge before compensation (yellow), and the temperature measured by the NTC thermistor across a temperature range of 28–41 °C (green). The response of the cantilever strain gauge can be compensated as follows: (ΔR/R0)a = (ΔR/R0)b + βΔT, where (ΔR/R)a is the compensated response and (ΔR/R)b is the response before compensation, and β is the calibration factor of 0.15. The compensated response demonstrates a consistent response of the cantilever strain gauge under varying temperatures.

Performance characteristics of a battery-free, wireless multi-axial sensor

Figure 4a presents a block diagram that includes three main parts of the wireless sensing platform: (1) a wireless, battery-free multiaxial sensor with NFC SoC for measurement of resultant forces and temperature; (2) a primary antenna and a reader for wireless power transfer and wireless data communication with the wireless sensor; and (3) a laptop for real-time monitoring of post-processed data. In particular, the wireless, battery-free multiaxial sensor consists of two main circuits for the multi-axial sensor (Wheatstone bridge) and NTC thermistor (voltage divider), respectively. The Wheatstone bridge converts changes in resistance of four cantilever strain gauges into an amplified voltage output using MUX and AMP, as shown in Supplementary Fig. 12. The two main circuits of NFC SoC obtain analog signals from four cantilever strain gauges and an NTC thermistor and, then convert these analog signals to digital signals using an analog-to-digital converter (ADC) of NFC SoC for wireless data transmission at a resonant frequency of 13.56 MHz, following the guidance of ISO 15693 protocols (See Supplementary Notes1–3). The primary antenna (ID ANT310/310, Feig Electronic, Germany) transmits data from the NFC SoC to a laptop through the antenna reader (ID LRM2500, Feig Electronics, Germany), as shown in the photograph of the experimental setup in Supplementary Fig. 13. Excellent characteristics of the multi-axial sensor, such as high linearity (R2 of 0.995), negligible hysteresis, and minimal drift (<0.1%), are achieved due to its elastic, simple structure (i.e., bonded four cantilevers). These characteristics enable the determination of force vectors using Eqs. (1)–(3), derived from predefined physical relationships between cantilever strain gauge outputs and applied force vectors as shown in Supplementary Fig. 10. However, we adopted a deep neural network (DNN) as a static regression model that learns the mapping between multi-channel raw outputs and target force values. The DNN does not require handcrafted modeling of complex interactions, but rather eliminates the need for empirical tuning for each sensor unit, making the system more scalable and robust in practical deployments22,50,51. This approach improves reproducibility and consistency across devices and users, while maintaining low computational complexity suitable for real-time, embedded operation. In this context, DNN serves as a practical, lightweight, and scalable tool for calibration of the multi-axial sensor in advanced wearable sensing systems.

Fig. 4: Performance characteristics of a battery-free, wireless multi-axial sensor.
figure 4

a Block diagram of a wireless, battery-free sensing system including three parts: (i) wireless sensor with four cantilevers and NFC SoC for measurement of pressure, shear stress and temperature; (ii) primary antenna and reader for wireless power transfer and wireless data communication; (iii) laptop for monitoring post-processed, visualized data (GUI) in real-time. b Structure of deep neural network (DNN) for calibration of the multi-axial sensor. c The change in ADC values measured from the wireless multi-axial sensor under pressure loading and unloading. The change in ADC values measured from the wireless multi-axial sensor under shear stress loading and unloading (d) in the direction from cantilever 4 to cantilever 1 and (e) in the direction from cantilever 3 to cantilever 2. Calibration results of the multi-axial sensor for test data that include reference data (i.e., XYZ-axis load cell data) and predicted forces of the multi-axial sensor with respect to (f) X-axis, (g) Y-axis, and (h) Z-axis.

Full size image

Figure 4b indicates a structure of DNN that includes (1) an input layer for four cantilever strain gauges (4 nodes), (2) six hidden layers consisting of a batch normalization layer, dense layer, and LeakyReLU layer, and (3) an output layer for XYZ-axis forces (3 nodes). In more detail, four voltage outputs from four cantilever strain gauges were input to the four-node input layer, while the three force measurements from the XYZ-axis load cell were input to the three-node output layer. In between, six hidden layers were configured, each consisting of a batch normalization layer, a dense layer, and a LeakyReLU layer, for the construction of the feedforward deep neural network (DNN). For model training and evaluation, the dataset of 1800 calibration data points was split into training and test sets at a 4:1 ratio. The neural network was trained for 500 epochs using the Adam optimizer (learning rate = 0.01) with a batch size of 256, and mean squared error (MSE) was used as the loss function. The calibration process was conducted three times. Error margins (i.e., standard deviations between target and predicted values) were calculated as 2.9% for X-axis, 3.4% for Y-axis, and 3.8% for Z-axis.

Figure 4c–e show the change in ADC values measured from the wireless sensor under pressure and shear stress applied to the multi-axial sensor. Figure 4c presents the change in ADC values of the wireless sensor under pressure loading and unloading of 45 kPa, exhibiting linearity (R2 of 0.995) and negligible hysteresis. Figure 4d, e show the change in ADC values of the wireless sensor under shear stress loading and unloading of 14 kPa in the direction from cantilever 4 to cantilever 1 and in the direction from cantilever 3 to cantilever 2, respectively. Supplementary Fig. 14 shows the time-dependent performance characteristics of the wireless sensor under pressure and shear stress loading and unloading, corresponding to Fig. 4c–e. Data collected during wireless characterization exhibit more fluctuation compared to data acquired via wired connection due to unstable wireless communication, caused by the metal frame (e.g., three-axis testing machine) around the multi-axial sensor and the primary antenna. To improve data communication quality, we have used wooden frames around the wireless sensing platform prior to its application in clinical settings2,3,5,8. Figure 4f–h present the calibration results of the multi-axial sensor using the proposed DNN for X, Y, and Z-axis. The yellow and green graphs indicate the reference value and the predicted value obtained from DNN, respectively. RMSE values between the reference value and the predicted value are 3.44%, 3.45%, and 4.31% for X, Y, and Z-axis, respectively.

Posture monitoring of a subject sitting in a wheelchair using continuously monitored data from the wireless device

Figure 5a presents a schematic illustration of the wireless sensing platform integrated with a wheelchair, including a primary antenna mounted on the back, connected to an NFC reader, and a laptop for real-time monitoring. Figure 5b shows a photograph of a subject (male, 34-year-old; height, 180 cm; mass, 72 kg) sitting in a wheelchair for both (1) sitting posture monitoring and (2) continuous monitoring of pressure, shear stress, and temperature at critical sites such as bony prominences. Figure 5c indicates the photograph of the top view of the wheelchair, equipped with a primary antenna at the back. Before multi-purpose monitoring (i.e., posture monitoring for prevention of disc herniation and continuous monitoring of pressure, shear stress, and temperature for prevention of pressure injuries), six wireless sensors were attached to the scapulae (wireless sensors 1–4), lower back (wireless sensors 5), and sacrum (wireless sensors 6) as shown in Fig. 5e. Specifically, four sensors were attached to the left upper, right upper, left lower, and right lower scapulae.

Fig. 5: Posture monitoring of a subject sitting in a wheelchair using continuously monitored data from wireless, battery-free multi-axial sensors.
figure 5

a Schematic illustration of a subject sitting in a wheelchair integrated with a wireless platform that includes wireless sensors, primary antenna, reader, and laptop for real-time monitoring. Photograph of (b) a subject sitting in a wheelchair, c a primary antenna mounted on the back of the wheelchair, and (d) the wheelchair. e Schematic anatomical diagram indicating the locations of six wireless sensors with X and Y axis for directionality of shear stress. f Five sitting postures of a subject during real-time, continuous monitoring of wireless sensors. Continuously monitored data from six wireless sensors with respect to pressure and shear stress (i.e., X and Y axis) at (g) left upper scapula, h right upper scapula, i left lower scapula, j right lower scapula, k lower back, and l sacrum. m Structure of deep neural network (DNN) for classification of five posture types using twenty-four sensor signals. n Accuracy values of five splits and mean accuracy of all splits. o Classification results comparing predicted labels with actual labels for five sitting posture types.

Full size image

Figure 5f shows continuous five sitting postures (e.g., normal, lean forward, lean backward, left sitting, and right sitting postures) of a subject sitting in the wheelchair for more than 5 min for multi-purpose monitoring. As the subject changed sitting postures in sequence, the wireless sensing platform monitored pressure, shear stress, and temperature in real-time, as shown in Fig. 5g–l (See Supplementary Fig. 16 for continuously measured temperature data). For convenience, each numbered section (n, n = 1–8) at the top of Fig. 5f corresponds with the number at the bottom of the graphs in Fig. 5g–l (e.g., (2) indicates lean forward posture). The direction of shear stress (X and Y) in Fig. 5g–l corresponds with the Cartesian coordinate (upper right; X and Y axis) in Fig. 5e. As the subject leaned forward in section (2), the pressure at the left upper, right upper, left lower, and right lower scapulae decreased, while the pressure at the lower back and sacrum slightly increased. In section (4), the pressure at the left upper and right upper scapulae increased due to the lean backward posture of the subject, whereas the pressure at the left lower and right lower scapulae decreased. In addition, the shear stress at the lower back increased. Then, as the subject changed posture from normal posture to left sitting posture in section (6), the pressure at the right upper and lower scapulae decreased while the pressure at the left upper and lower scapulae increased. Conversely, when the subject changed posture from left sitting posture to right sitting posture in section (7), the pressure at the right upper and lower scapulae increased while the pressure at the left upper and lower scapulae decreased. In addition, the skin condition after attaching the wireless sensor at the skin interface was evaluated for 4 h, as shown in Supplementary Fig. 18. Three commercial devices, such as an infrared thermometer (SK-8300 SATO, Japan) for measurement of temperature, a skin analyzer for measurement of moisture and oil (Howskin IC-HSP001, INNOINSIGHT INC., South Korea), and a skin colorimeter (DSM III ColorMeter, Timpac Healthcare, India), were used2. Supplementary Fig. 19 presents measurement results of skin conditions under three conditions: bare skin, skin with a medical dressing attached, and skin with the wireless sensor attached using a medical dressing. No significant differences were found among the three conditions. Additionally, no irritation was observed after 4 h of attachment, which can be attributed to the small and thin form factor of the wireless sensor, the use of biocompatible PDMS, and the application of a commercial, breathable medical dressing.

For the classification of five sitting postures of the subject, A DNN is designed to include (1) an input layer for six wireless sensors (24 nodes), (2) six hidden layers including a batch normalization layer, dense layer, and LeakyReLU layer, (3) an output layer for five postures (5 nodes), and (4) a Softmax function for the classification in Fig. 5m. The experiment for the classification of sitting posture was repeated 12 times, and the subject performed each of five postures 4 times. The dataset consisted of 150–160 samples per test, with 14 to 15 samples allocated to each posture, ensuring class balance. To mitigate overfitting and evaluate accuracy, stratified K-fold cross-validation (K = 5) was employed, where sampling was performed to ensure that the class proportions in each fold reflected the proportions in the whole data set. In this approach, the dataset was divided into five folds, with each iteration using an 80%/20% split for training and testing, respectively. During training, the true labels for the five sitting postures were represented using one-hot encoding. The model was optimized using the cross-entropy loss function and trained for 100 epochs with a batch size of 16. The Adam optimizer was employed with a learning rate of 0.01. After training for each split, validation was conducted in Fig. 5n. Accuracy values for each split were 94.5%, 100%, 98.8%, 98.8%, and 98.8%, with an average accuracy of 98.2%. Figure 5o presents the confusion matrix that shows the comparison results of the predicted and true labels across the five splits, showing accuracies of 98.7%, 98.2%, 94.5%, 98.3%, and 100% for each posture.

In summary, recent studies on the prevention of disc herniation have explored posture monitoring using wearable IMU (inertial measurement unit; accelerometer) and pressure mattress sensors40,41,42,43,44,45,46,47. However, mattress-type pressure sensors are costly, consume significant power, and may suffer from cross-talk, while IMUs cannot directly measure pressure and shear on the spine. Our approach utilizes wireless sensors, which enable practical, continuous monitoring in daily life with a small and thin form factor due to its battery-free design. With sufficient data collection, this system is expected to provide personalized feedback and a promising patient-friendly solution for daily spinal health management (i.e., posture monitoring for the prevention of disc herniation).

Augmented reality (AR)-assisted monitoring of a subject lying in bed using wireless, battery-free multi-axial sensors

An AR system integrated with an NFC-based wireless sensing platform was developed to offer medical staff an immersive AR environment for intuitive and immediate monitoring of physical parameters at the skin interfaces of patients. Figure 6a presents the key components of the AR system, which include AR glasses (XREAL Air 2 Ultra, XREAL, China), a smartphone, a marker, and a wireless sensor. The AR glasses can overlay sensing data onto the user’s field of vision, with the smartphone acting as a control unit and displaying additional information. The marker serves as a reference coordinate for the accurate positioning of the information overlay. In Fig. 6b, the photograph shows a user wearing the AR glasses, demonstrating not only the comfortable fit of the AR glasses, but also the utility of a hands-free approach for medical staff when treating the subject during continuous, immersive monitoring. Figure 6c presents a block diagram divided into two main parts such as the wireless sensing platform and the AR system. The block diagram describes how sensing data obtained from wireless sensors could be visualized on the displays of smart devices (e.g., AR glasses and smartphones) in real-time. The laptop acquires wireless sensing data through the NFC ISO 15693 protocol. Then, it transmits sensing data to the smart devices, equipped with an application programmed using AR software (Unity, Unity Technologies, USA), to display pressure and shear stress information at the skin interfaces in the user’s field of vision through color-coded maps, numerical values, and the direction of forces.

Fig. 6: Augmented reality (AR)-assisted monitoring of a subject lying in bed using wireless, battery-free multi-axial sensors.
figure 6

a Photograph of devices for an immersive AR environment that includes AR glasses (XREAL Air 2 Ultra, XREAL, China), a smartphone, a marker, and a wireless sensor. b Photograph of a user wearing the AR glasses. c Block diagram of a wireless sensing platform integrated with the AR system that includes three parts: (i) wireless sensing platform for real-time monitoring of resultant forces, (ii) Unity software for programming the application, and (iii) a smart device (e.g., AR glasses or smartphone) that receives sensing data from the laptop and overlays sensing data onto the user’s field of vision using the application. d Wireless sensor attached to the wrist for validation of AR system. Photographs of the wrist during AR-assisted monitoring while the wrist moves (e) rightward, f upward, and (g) upper-left. h Photograph of the wrist when pressure is applied to the top of the wrist. Schematic illustrations of AR-assisted monitoring of the subject lying in bed using (i) wireless sensing platform and j AR system including laptop, smartphone, and marker. Photographs of the subject during AR-assisted monitoring while the subject (male, 27-year-old; height, 178 cm; mass, 82 kg) lying in (k) the supine position, l the right lateral recumbent position, m the left lateral recumbent position and (n) the supine position.

Full size image

For validation of the AR system, a wireless sensor was firmly attached to the wrist of the subject using a transparent film dressing as shown in Fig. 6d. Herein, the multi-axial sensor attached to the wrist functions as cutaneous receptors (e-skin), providing medical staff with sensing data (i.e., pressure and shear stress) intuitively with the AR system. For instance, when the subject moves their wrist in a certain direction on a surface, the surface exerts a friction force on the multi-axial sensor in the opposing direction. Thus, the AR system displays the magnitude of the forces applied to the multi-axial sensor according to the Cartesian coordinate system (reference) on the marker in real-time. As the subject moved their wrist rightward (yellow arrow) in Fig. 6e and Supplementary Video 1, the magnitude of forces applied to the sensor from the surface indicated 6.10 kPa (X-axis; red), 0.22 kPa (Y-axis; green), 0.02 kPa (Z-axis; blue) according to the reference coordinate (XYZ-axis) on the marker, while the direction of force applied to the sensor from the surface shows a leftward direction along the X-axis. Likewise, the upward movement (yellow arrow) of the wrist resulted in the magnitude of sensing force of 0.35 kPa (X-axis), 10.22 kPa (Y-axis), 3.5 kPa (Z-axis), leading to a downward direction of sensing force, as shown in Fig. 6f. In sequence, the upper-left movement (yellow arrow) of the wrist applied force to a wireless sensor with a magnitude of 5.72 kPa (X-axis), 9.05 kPa (Y-axis), 5.16 kPa (Z-axis), showing the direction of sensing force toward the lower-right in Fig. 6g. Applying pressure (red arrow) to the wrist resulted in a force magnitude of 0.54 kPa (X-axis), 4 kPa (Y-axis), and 14.4 kPa (Z-axis) in Fig. 6h. Supplementary Fig. 21 shows the time-dependent monitored sensing data of the wireless sensor under pressure and shear stress, corresponding to Fig. 6e–h.

For the application of the AR system in medical fields, the clinical bed is integrated with the wireless sensing platform (primary antenna) for intuitive measurement of pressure and shear stress at the skin interfaces in Supplementary Fig. 22. As shown in Fig. 6i, the schematic illustration presents three wireless sensors attached to the bony prominences of the subject, such as the left scapula, right scapula, and sacrum, followed by wireless power transfer and wireless data communication. The sensing data from the three wireless sensors were transmitted to the smart devices of the AR system for AR-assisted monitoring of pressure and shear stress in Fig. 6j. After the smart device recognized the marker, the sensing data were overlaid onto the real-world environment through the display of the smart device for medical staff. Figure 6k–n and Supplementary Video 2 show the AR-assisted monitoring of pressure and shear stress distribution using the three wireless sensors attached to the bony prominences (i.e., scapulae and sacrum) of the subject (male, 27-year-old; height, 178 cm; mass, 82 kg) lying in bed. In Fig. 6k, as the subject lies in the supine position, pressure values of 3.19 kPa, 4.43 kPa, and 3 kPa were measured and displayed for the left scapula (wireless sensor 1), right scapula (wireless sensor 2), and sacrum (wireless sensor 3), respectively, along with the reference coordinate (XYZ-axis) and color-coded maps. Negligible shear stress values of approximately 0.1 kPa were observed for the three critical sites simultaneously. As the subject turned his body to the right recumbent position in Fig. 6l, shear stress value along the X-axis increased from 0 kPa to 3.04 kPa at the right scapula, while pressure value slightly decreased from 4.43 kPa to 2.44 kPa. On the other hand, pressure and shear stress were not applied at the left scapula due to the generation of an air gap at the skin interface, resulting in decreases in pressure and shear stress values from 3.19 kPa to 0 kPa and from 0.08 kPa to 0 kPa, respectively. At the sacrum, the pressure value increased from 3 kPa to 6.81 kPa, and the shear stress value along the Y-axis increased from 0 kPa to 0.29 kPa. As shown in Fig. 6m, the pressure value increased from 0 kPa to 4.37 kPa, and the shear stress value along the X-axis increased from 0 kPa to 1.75 kPa due to the posture change of the subject from the right recumbent position to the left recumbent position. Likewise, pressure and shear stress values at the right scapula were not measured due to the generation of an air gap, leading to decreases in pressure and shear stress values from 3.04 kPa to 0 kPa and from 2.44 kPa to 0 kPa, respectively. In Fig. 6n, the subject turned his body from the left recumbent position to the supine position, resulting in decreases in shear stress at the three critical sites, ranging from 0.1 to 0.3 kPa. Pressure values were 3.8 kPa, 3.75 kPa, and 4.02 kPa for the left scapula, right scapula, and sacrum, respectively. Supplementary Fig. 23 shows the time-dependent monitored sensing data of the three wireless sensors for the scapulae and sacrum that corresponds to Fig. 6k–n.

In summary, this newly developed wireless, AR-system-integrated sensing platform enables medical staff to immediately and intuitively obtain multiple hidden tactile sensing data through AR, eliminating the need to check separate monitors or numerical tables. This supports not only quick, intuitive decisions for patient repositioning by caregivers to prevent pressure injuries, but also helps caregivers save time and energy in hospitals, thereby alleviating medical staff shortages worldwide. Furthermore, this platform can assist medical staff in compression therapy6, nerve injury prevention7, and prosthetic interface applications14, where real-time measurement of pressure and shear stress values at multiple locations is critical for accurate diagnosis and effective therapy.

Discussion

In this research, we reported a wireless, battery-free multi-axial sensor system integrated with augmented reality (AR) technology for intuitive, real-time monitoring of pressure, shear stress, and temperature at skin interfaces. The wireless sensor provides information on the magnitude and direction of the resultant forces (i.e., pressure and shear stress) by combining the electrical signals from four cantilever strain gauges under applied pressure and shear stress. In addition, the validation trials of the overall system demonstrated the robustness and compliance of the NFC-based wireless sensor due to its thin and small form factor and highlighted the significance of multi-purpose monitoring in healthcare applications (i.e., intuitive AR monitoring, posture monitoring for the prevention of disc herniation, and continuous patient monitoring for the prevention of pressure injuries).

However, several technical challenges remain unresolved. First, AR systems currently face issues with latency and delays, which can hinder the real-time visualization of sensing data. This latency must be minimized to ensure seamless interaction and timely decision-making in clinical environments. Additionally, the integration of wireless sensors with AR systems presents further challenges, such as achieving reliable data synchronization and real-time analysis, particularly in dynamic environments where multiple sensors may be involved. Moreover, while the wireless sensor system has proven effective, achieving perfect conformality with human skin and ensuring full compliance with different body types and movements remain areas for further improvement. The current system is limited to three-axial measurements (XYZ-axis), but future systems will need to expand to analyze more complex, multi-dimensional forces (e.g., up to 6 axes), integrating compliance sensors that can adapt to the body’s natural contours and measure various biomechanical signals with greater precision.

In the future, overlaying sensing data from the wireless sensor array onto the user’s field of vision within an immersive and interactive AR environment will help medical staff understand diverse physical parameters (e.g., pressure, shear stress, temperature, heart rate, humidity, pH, and sweat) at skin interfaces and immediately assess the risk factors related to the diagnosis and prevention of disease (e.g., pressure injuries, disc herniation, venous leg ulcers, nerve injuries, etc.). In particular, the development and validation of sensor-AI integrated prediction models is essential for improving efficiency in hospitals, given that the shortage of medical staff (e.g., caregivers and professionals) is a worldwide issue. Our future work will focus on establishing reliable prediction models, which require scalable and comprehensive procedures, such as collecting high-quality clinical data from numerous subjects during clinical trials conducted under the supervision of medical staff. Furthermore, integrating the AR system with the wireless sensing platform offers significant potential in the healthcare industry, with a wide range of medical applications, including the optimization of prosthetics, rehabilitation, diagnosis, education, and therapy.

Methods

Battery-free, wireless platform

Electronic components such as NFC SoC (RF430FRL152H, Texas Instruments, USA), NTC thermistor (NTCG064EF104FTBX, TDK Corporation, Japan), multiplexer (MUX; TMUX1108, Texas Instruments, USA), amplifier (AMP; INA333AIDRGT, Texas Instruments, USA), resistors (RC0201, Yageo, Taiwan) and capacitors (CL03B, Samsung Electro-Mechanics, South Korea) were surface-mounted on a flexible printed circuit board (fPCB; PI/Cu/PI, 35/44/35 μm thick; Pyralux AP8535 R, DuPont, USA) patterned by a UV laser system (ProtoLaser U4, LPFK Laser & Electronics AG, Germany) as shown in Supplementary Fig. 1. Analog signals obtained from the Wheatstone bridge (cantilever strain gauges and AMP) and the voltage divider (NTC thermistor) were converted to digital signals through ADCs of NFC SoC in Supplementary Fig. 12. These digital signals were then transmitted wirelessly by NFC SoC to a primary antenna (ID ANT310/310, Feig Electronic, Germany) and an antenna reader (ID LRM2500, Feig Electronics, Germany), according to the ISO 15693 protocol for NFC at 13.56 MHz.

Fabrication of a cantilever strain gauge

The fabrication process began with the spin coating of polyimide (PI) solution (PI-1388, VTECTM, USA) at 1000 rpm for 30 s on a 525 μm thick silicon (Si) wafer, followed by heating at 240 °C in a convection oven (Supplementary Fig. 3). The spin coating process was repeated four times in sequence to complete a 75 μm thick PI film, which served as the substrate for a strain gauge. A negative photoresist (negative PR; AZ nLof 2035, MicroChemicals GmbH, Germany) was then spin-coated at 4000 rpm for 30 s, followed by soft baking at 110 °C for 1 min on a hot plate. After photolithography of the negative PR (UV irradiance of 72 mJ/cm2 for 8 s) using a photomask for the strain gauge patterns, post-exposure baking (PEB) was conducted at 110 °C for 1 min on a hot plate. To develop the pattern, the Si wafer was immersed in a developer solution (AZ 300 MIF Developer, Omniscience, South Korea) for 2 min, and then rinsed with distilled (DI) water. A plasma asher operating at 50 W was used to clean residual PR for 1 min to improve adhesion properties before electron beam evaporation (E-beam). Chrome (Cr) and gold (Au) were sequentially deposited using the E-beam to achieve 10 nm and 30 nm thickness, respectively. After E-beam deposition, the Si wafer was immersed in a lift-off solution (mr-Rem 700, micro resist technology GmbH, Germany) for 1 h, and then rinsed with isopropyl alcohol (IPA; J.T. Baker, USA) and DI water. For protective encapsulation, the spin coating process of the PI solution was conducted at 8000 rpm for 30 s on the Cr/Au pattern, forming a thin 3 μm thick PI layer, thus completing the tri-layered cantilever strain gauge (PI/Au/PI, 3/0.03/70 μm in thickness).

Fabrication of a multi-axial sensor

The fabrication process of a multi-axial sensor began with mounting a NTC thermistor (NTCG064EF104FTBX, TDK Corporation, Japan) onto the backside of fPCB (PI/Cu/PI, 35/44/35 μm in thickness) using solder paste (SMDLTLFP10T5, Chip Quick, Canada) for electrical connection (Supplementary Fig. 4). A rigid substrate (silicon wafer; 8 × 8 × 0.25 mm3), a glass substrate (glass wafer, 3 × 2 × 0.25 mm3), a cantilever strain gauge (PI/Au/PI, 2 × 1.2 × 0.07 mm3) and a rigid cover (Poly(methyl methacrylate) (PMMA), 7 × 7 × 0.25 mm3) were cut using the UV laser system. Then, the rigid substrate (silicon wafer; 8 × 8 × 0.25 mm3) was bonded onto the backside of fPCB, and the glass substrate was bonded onto the topside of fPCB using the epoxy resin (1365868-6, Loctite, USA) and silver epoxy (8331-14 G, MG chemicals, Canada). Likewise, four cantilever strain gauges were bonded using the same epoxy resin and silver epoxy. A soft cover (polydimethylsiloxane (PDMS; Sylgard 184, Dow Inc., USA); 8 × 8 mm2) was chemically treated to enhance adhesion using (3-Mercaptopropyl)trimethoxysilane (MPTMS; 175617, Sigma–Aldrich, USA)49. The soft cover (PDMS) and the rigid cover (PMMA) were then bonded using the epoxy resin.

Finite element analysis

Commercial finite element analysis (FEA) software (Abaqus, Dassault Systèmes, France) was used to perform FEA simulations of the multi-axial sensor under mechanical stimuli such as pressure and shear stress. The multi-axial sensor was modeled by C3D10 elements. The elastic modulus (E) and Poisson’s ratio (υ) of the materials in the multi-axial sensor were as follows. For silicon, ESi = 160 GPa and υSi = 0.27; for polyimide, EPI = 2.5 GPa and υPI = 0.34; for glass, EGlass = 50 GPa and υGlass = 0.2; for PDMS, EPDMS = 2.5 MPa and υPDMS = 0.5.

Characterization of the multi-axial sensor

A customized three-axis testing machine, consisting of XYZ-axis actuator and XYZ-axis load cell, was used to apply resultant load (i.e., combined pressure and shear stress) to the multi-axial sensor using software (LabVIEW, National Instruments, USA). As shown in Supplementary Fig. 6, (1) four sensing data from four Wheatstone bridges with four amplifiers (HX711, Avia Semiconductor) and (2) three XYZ-axis force data from XYZ-axis load cell were simultaneously obtained using a development board (Arduino Mega2560, BCMI, Italy).

Near field communication protocol

An Evaluation Module (TRF7970A Evaluation Module (EVM), Texas Instruments, USA) connected to a laptop wirelessly transmitted embedded code to the NFC SoC using software provided by Texas Instruments, operating at a frequency of 13.56 MHz and following ISO 15693 protocol. A small primary antenna (ID ANT310/310, Feig Electronic, Germany) and a large primary antenna (ID ANT800/600, Feig Electronic, Germany) were installed on a benchtop, a wheelchair, and a clinical bed. The antennas, connected to a multiplexer module (ID ANT.MUX.M4, Feig Electronic, Germany) and an antenna reader (ID LRM2500, Feig Electronics, Germany), transferred power and communicated data wirelessly using software (ISOStart 2018, Feig Electronics, Germany). The RF power from the primary antenna was set to 8 W using ISOStart 2018. Then, ISOStart 2018 received the digital signals from the NFC SoC, and Python converted the digital values from hexadecimal to decimal simultaneously.

Deep Neural Network (DNN) based calibration

Signal processing and deep learning models were developed using Python and the TensorFlow framework in the Google Colab Notebook environment (Google, USA). For the calibration of the multi-axial sensor, data from four cantilever strain gauges were collected, and three-axis force data were obtained using a customized three-axis testing machine equipped with an XYZ-axis actuator and an XYZ-axis load cell (Supplementary Fig. 6). This setup allowed for the synchronization of data from four cantilevers, which serve as input data features for the deep neural network (DNN), along with the force data that provides the true labels. The entire dataset was divided into training and test sets in a 4:1 ratio for model training and evaluation. The model was trained for 500 epochs using the Adam optimizer with a learning rate of 0.01, a batch size of 256, and mean squared error (MSE) as the loss function. For the posture classification task, stratified K-fold cross-validation was implemented using the scikit-learn package of Python to evaluate the model’s generalization performance on data with imbalanced class distributions. For the training process, one-hot encoding was used to provide the algorithm with the true labels of five sitting postures. The cross-entropy function was chosen as the loss function. The model was trained with a batch size of 16 for 100 epochs using the Adam optimizer with a learning rate of 0.01. With these tools and parameters, the sensor calibration and posture classification tasks were successfully performed.

Augmented reality (AR) system integrated with a wireless sensing platform

The AR system was designed to operate with a host device, which can be either a laptop or a smartphone. When using a laptop as the host device, the wireless, battery-free multi-axial sensors transmitted data through an NFC-based wireless sensing platform to the laptop. Then, the laptop processed the sensing data and sent it to AR software (Unity, Unity Technologies, USA) running on the laptop. Unity renders the sensing data in real-time, visualizing it within a 3D model of the subject. The processed data was subsequently transmitted from the laptop to the AR glasses (XREAL Air 2 Ultra, XREAL, China), where it was displayed in an immersive and interactive environment. Similarly, when using a smartphone as the host device, the smartphone received the sensing data, processed it through the Unity app, and directly transmitted the visualized data to the AR glasses for display. In both configurations, whether using a laptop or a smartphone as the host device, the system enabled real-time visualization and interaction with the sensing data through the AR glasses, providing a seamless and high-fidelity AR experience.

Validation trial

The validation trials were approved by the institutional review board (IRB) of Korea Advanced Institute of Science and Technology (KAIST) (IRB no. KH2023-001). After understanding the contents of the study and signing consent forms for the validation trial and publication, the recruited volunteers participated in the validation trials. The subjects were a 34-year-old male (180 cm, 72 kg) and a 27-year-old male (178 cm, 82 kg), whose critical sites, including scapulae and sacrum, were targeted by wireless sensors. Then, the subject seated in a wheelchair changed sitting postures (e.g., normal, lean forward, lean backward, left sitting, and right sitting postures) in sequence for more than 5 min during multi-purpose monitoring. For AR-assisted monitoring of physical parameters, as the subject lying in bed changed lying postures (e.g., supine position, right recumbent position, and left recumbent position), the smart devices displayed real-time sensing data from wireless sensors attached to the scapulae and sacrum.