Introduction

Touchscreen technology has become ubiquitous due to its convenience and versatility, making it the popular choice of interface for human–computer interaction, with applications in touchscreen displays, fingerprint sensors, touch buttons, and sliders. With the advent of Internet of Bodies (IoB)1,2,3, touch sensing and touchscreen-based device have further increased in popularity over the last decade. Touchscreen systems have traditionally been used to detect touch and to locate the presence of a touch event. Despite being the most common human-machine interface, touchscreens have no direct impact on the communication between devices. Enabling communication through a touchscreen system opens up a whole range of new applications based on communication strictly through touch. This has motivated the study to expand the functionality of touchscreens in Human-Computer Interaction to include high-speed communication through a touchscreen. In this study, we demonstrate Touchscreen Communication (ToSCom), a first-of-its-kind system that takes touch as an input to not only detect and locate the presence of a touch event but also allows us to simultaneously transfer high-speed data through the touchscreen interface.

Previous attempts at communicating data using capacitive touch sensing systems have had limited success with very low data rates (≤500 bps)4,5,6,7,8 which limits the applications of such technologies. Some studies have analyzed the use of proximity communication where the transceivers need to be placed close to each other or have an additional electrode that needs to be touched each time to communicate data5, which is not an elegant solution and is inconvenient for the user.

To use a touch event as the trigger for communicating data, a communication methodology that has the required selectivity to transfer data through touch is imperative. In this regard, Capacitive Electro-Quasistatic Human Body Communication (EQS-HBC) has been successfully demonstrated to enable data transfer strictly through touch9. In EQS-HBC (Fig. 1c), the human body acts as a wire to connect devices on the body, providing a low path loss communication channel. The ability to use the human body as a communication channel at the Electro-Quasistatic (EQS) frequency regime (≤20 MHz) further enables physically secure data transfer10,11, containing the fields carrying the data securely inside the body, preventing signal leakage away from the body. This signal containment provides the required touch selectivity, enabling data transfer strictly through touch events required in Touchscreen Communication (ToSCom).

Fig. 1: Touchscreen Communication (ToSCom): Combining the applications of Capacitive touch sensing with Capacitive Electro-Quasistatic Human Body Communication (EQS-HBC).
figure 1

a ToSCom enables high-speed data transfer (3Mbps) with applications in viewing personalized app data on touchscreen-based wearables and ground connected touchscreen kiosks like Point of Sale systems. b ToSCom is implemented and demonstrated using commercial-off-the-shelf components. c A capacitive EQS-HBC-based on-body transmitter (Tx) is used to couple signal on to the body. d A capacitive touchscreen interface based receiver (Rx) is used for high-speed communication at Mbps scale data rate. e Enabling high signal-to-noise ratio (SNR) resulting in high-speed communication using communication channel modeling for ToSCom. f An illustration of the ToSCom receiver setup developed.

Full size image

ToSCom enables users to access personalized data for applications via touchscreen kiosks or on personal devices like smartphones (Fig. 1a). ToSCom provides a platform for high-speed data transfer between touchscreen-based devices for a large spectrum of digital data including biopotential sensor data, productivity app data, and audio/image files. We demonstrated ToSCom by re-purposing a commercial off-the-shelf capacitive touch sensing hardware to be used in dual mode to detect touch-based inputs and simultaneously transfer data through the sensing system (Fig. 1b). We use minimal modifications to an existing capacitive touch sensing hardware to achieve a 3 Mbps communication channel between wearable devices connected to the body and a touchscreen device. This development extends the applications of touch-based sensors to perform data transfer between Body Area Network devices, empowering capacitive touch sensors to become a part of the Internet-of-Body (IoB) space.

Capacitive touch sensing systems have been around for decades, being first introduced in 196512 for air traffic control computers. The smartphone and tablet PC boom has further cemented its place in Human–Computer Interaction (HCI). Its ubiquitous influence has motivated this study to increase the uses of capacitive touch sensing systems beyond detection and location of touch to include simultaneous data transfer through the touchscreen.

A typical capacitive touch sensing system, illustrated in Fig. 1d, has two primary functions: touch detection and location. The capacitive touch sensing system consists of a non-conductive overlay that lies on top of a grid of sensing electrodes. The lowest layer houses the printed circuit board (PCB) which contains the system ground and the touch sensing circuit elements. When a finger is placed on the capacitive sensor, the capacitance sensed by the sensing electrodes changes due to the additional capacitance of the finger and the sensor pad. The electrodes with the largest change in capacitance on each layer indicate the position of the finger and provide the coordinates of the touch on the screen.

Owing to the ubiquity of touch sensing systems, various applications of capacitive sensing have been developed across the Human–Computer-Interaction community13. Common applications of capacitive touch sensing systems beyond touch detection and location include gesture recognition14,15, biometric authentication and identification16, indoor localization, and whole-body movements17. Current applications of capacitive touch sensing systems can be categorized into 4 operating modes namely, loading mode, shunt mode, transmit mode, and receive mode13. These operating modes are based on the relative position between the body with the transceiver electrodes. In this study, we propose a method of interaction with capacitive touch sensing technology, where the touch sensing system is utilized as a part of the communication with the touch-based inputs used to communicate high-speed data. Combining data transfer with capacitive touch sensing has been explored previously for very low data rate applications (≤100bps) like simple user authentication4,5,6,7,8. The transmitter node, in some of these studies, communicates data by physically touching the touch sensor with an electrode on the transmitter, forming a proximity communication channel. Previous studies have demonstrated using the body as a medium for data transfer, but even with custom hardware, have been limited to low data rate applications of ≤1kbps18,19. The low data rates achievable for previous attempts to communicate data through a touchscreen primarily result from a suboptimal communication receiver on the touch sensing receiver circuitry. The capacitive touch sensing circuit typically has a high scan time for the sensors, which is the limit on the sampling rate of the receiver. Further, filters and amplifiers that are not optimized for communication combine to limit the communication performance of previous approaches.

Human Body Communication (HBC) is a low-power and energy-efficient methodology20,21,22,23,24,25,26,27,28, where the conductive properties of the body tissues are used to communicate data around the body. Further, HBC has been demonstrated to communicate data among devices around the body for high data rates (10s of Mbps)29,30. Human Body Communication can be operated in two modalities, Capacitive and Galvanic. In Capacitive HBC, a single signal electrode couples the signal to the human body, with a second electrode working as the floating ground plane31. In Galvanic HBC, both electrodes are connected to the body to transmit and receive fringing fields through the body32. It has been illustrated that Capacitive HBC is more suitable for communication between wearable devices with a lower channel loss across larger channel lengths between the transmitter and receiver20,23,33,34. Further, the use of Electro-Quasistatic (EQS) frequency regime of ≤30MHz, where the wavelength of the signals is at least an order of magnitude higher than the dimensions of the human body has shown the additional advantage of being physically secure with low signal leakage away from the body as compared to radiative technologies10,35,36. The low power, energy-efficient, and physically secure nature of EQS-HBC makes it especially suitable for communication in wearable Internet of Bodies (IoB) nodes.

In ToSCom, we leverage the advantages of Capacitive EQS-HBC (Fig. 1c), which provides a physically secure10,11 low-loss channel that enables high-speed communication29 strictly through touch9, with a redesigned touchscreen interface to enable communication reception capability to detect and locate the position of the touch as well as communicate data through touchscreens. A detailed communication channel modeling is performed to design the receiver electrodes which reduces channel loss for the ToSCom channel, allowing high-speed data transfer via the touchscreen interface (Fig. 1e). The touchscreen communication system designed allows simultaneous high-speed communication with touch detection and location, enabling personalized data transfer between wearable nodes like smartwatches and touchscreen kiosks in Point-of-Sale registers enabling touch-based payments (Fig. 1f). Table 1 (Comparison with prior studies) shows the increase in data rate and applications possible with the use of ToSCom over previous studies on communication around touch sensing systems.

Table 1 Comparison with prior studies
Full size table

Results and discussion

Communication system

A dedicated communication module is essential in integrating high-speed data transfer with simultaneous touch sensing. Further, the communication methodology used also has to be enabled only when strictly in contact with the touchscreen system. The communication module used in ToSCom is an Electro-Quasistatic Human Body Communication (EQS-HBC) system. The principle of EQS-HBC is illustrated in Fig. 2a, b. EQS-HBC uses the body’s conductive tissues to transmit data through the body at low frequencies of ≤20MHz. The wavelengths at these operating frequencies are more than an order of magnitude higher than the dimensions of the human body, which allows us to model the human body as a lumped system (Fig. 2b). Detailed biophysical circuit models for wearable Capacitive EQS-HBC systems have been studied extensively20,23,24. A circuit model describing the modulation of the potential of the body (VBody) with respect to the earth’s ground when a transmitter is connected to the body is illustrated in Fig. 2c. Further, the human body being an inefficient radiator at the operating frequencies of 10s of MHz makes it a physically secure system as it confines the signal in a 10 cm bubble around the body and allows communication strictly through touch. Thus, to gain access to the data, the attacker needs to almost be in contact with the person, making it difficult to snoop the signal. A complete analysis of the physical layer security provided by Electro-Quasistatic Human Body Communication has been presented in Supplementary Note 1.

Fig. 2: Electro-Quasistatic Human Body Communication (EQS-HBC): Using EQS-HBC for communicating data between a wearable and a touchscreen-based receiving device for ToSCom.
figure 2

a Biophysical circuit model for a Capacitive Electro-Quasistatic Human Body Communication (EQS-HBC) system. b The operating frequency for EQS-HBC is below 20MHz which makes the wavelengths at least an order of magnitude higher than the dimensions of the devices and the human body. c A simplified circuit model depicting the Capacitive EQS-HBC channel is illustrated. d Concept figure to illustrate an optimal channel loss response for the ToSCom system. e A biophysical circuit model for the complete ToSCom system is created by combining the touch sensing system with the Capacitive EQS-HBC system.

Full size image

Effect of device size and position

EQS-HBC is a voltage-mode communication system, the communication channel loss is a function of the device’s size20,24 as well as the positioning of the device with respect to the environment surrounding the device23. Increasing the size of the transmitter’s or receiver’s floating ground plane decreases the channel loss as the parasitic return path capacitance (CG-Tx, CG-Rx1) increases. Thus, having larger transceivers facilitates easier communication with lower path loss which results in a more robust communication system.

Further, varying the position of the on-body transmitter in a capacitive EQS-HBC system changes the return path capacitance (CG-Tx), thus changing the path loss of the total system. The positional variance in channel loss for Capacitive EQS-HBC has been studied in detail previously23,27. In this study, the ToSCom channel is analyzed without the added complexity of positional variance of the transmitter and receiver. The transmitter and receiver positions are kept constant for the simulations and experiments. However, to illustrate that the ToSCom system may be used with an on-body transmitter placed at different locations on the body, the Finite Element Analysis based Electromagnetic simulations shown in Fig. 3 are performed with the Transmitter and Receiver on the same arm and the experiments shown in Fig. 4 are performed for the transmitter and receiver on opposite arms. The effect of a dielectric clothing layer between the on-body transmitter and the body is analyzed in Supplementary Note 2 where we observe an additional channel loss of 2 dB for a 5mm thick dielectric layer.

Fig. 3: Finite-Element-Analysis (FEA) based electromagnetic simulations for optimal receiver electrode configuration.
figure 3

A For optimal signal reception, 4 configurations of receiver electrodes are analyzed. The ToSCom receiver is connected across electrodes 1 and 2. B Electric field plot for a surface infinitesimally above the system ground plate, showing electric field around the sharp edges of the ground plates have high field strength. C Channel loss across the entire touchscreen surface is analyzed with the finger moving across the leading diagonal. D Channel loss response for the 4 configurations of the ToSCom receiver. E Using 2 small electrodes to further optimize the channel loss. F Optimal, constant channel loss response with 2 small electrode configuration is shown. G Touch selectivity of ToSCom due to its Electro-Quasistatic (EQS) behavior is illustrated. Moving the finger away from the touchscreen shows a sharp increase in channel loss 1cm away from the screen.

Full size image
Fig. 4: Experimental validation of the ToSCom system.
figure 4

a The experiments are performed using a commercially available touchscreen module, TI Captivate BSWP. The positions of touch on the touchscreen surface are illustrated. b Experimental results of channel loss for 4 configurations show that the small electrode configuration has the optimal channel loss response. c The variation of path loss for changing posture and environments is illustrated to show the functioning of the ToSCom system in different operational conditions. d Bit-error-rate (BER) for all 4 configurations are measured for around 107 bits. The small electrode configuration has the lowest average BER. A concept figure for BER as a function of signal-to-noise ratio (SNR) is illustrated to show the trends in BER for all configurations. The channel characteristics are illustrated for edge cases (shaded region) to show the variations in BER across different configurations. e The experiments and demonstrations (Supplementary Video 1 and Supplementary Note 342) were performed for a data rate of 3 Mbps. A sample of the 3 Mbps transmitted waveform measured from the WiR evaluation kit and the same signal after it is passed through the body channel is illustrated.

Full size image

The receiver electrodes in ToSCom are connected to a capacitive touchscreen-based device. The placement of receiver electrodes within the device dictates the eventual channel loss of the ToSCom communication system. In the next section, we discuss the criteria for choosing the receiver electrodes and possible receiver electrode configurations.

Receiver electrode

As the received signal strength is a strong function of the receiver electrode position and size, designing the receiver to receive optimal power is essential. To add a communication module for receiving data along with the existing touch sensing structure, the receiver electrodes must be placed such that the optimal voltage difference is captured across the whole touchscreen. Figure 2d illustrates the desired channel loss response for a ToSCom system. The channel loss response required must be optimized over the entire screen. A localized low path loss on one part of the screen is undesired, as it may not allow us to transfer data when the finger is placed on the entire screen.

To obtain a flat channel loss response across the entire touchscreen, using the existing structure (sensing grid and ground illustrated in Fig. 1d) may not be enough. Thus, additional receiver electrodes must also be analyzed to optimize channel loss. For effective touch sensing and simultaneous communication, the criteria required to be satisfied are:

  • The operation of the touch sensing system must be unhindered to ensure that touch detection is efficiently performed in the system. For this, the parasitic capacitance (CP in Fig. 2e) must be as small as possible. Increasing parasitic capacitance decreases the sensitivity of a touch sensing setup. Any additional elements will contribute to increasing parasitic capacitance. Thus, any additional elements must also be designed to ensure parasitic capacitance added is minimized.

  • Change to the touchscreen setup is minimized to ensure that the ToSCom system resembles the Capacitive touch sensing system as much as possible, and minor modifications can enable simultaneous data transfer.

Circuit modeling

The biophysical channel model for the ToSCom system is illustrated in Fig. 2e. This channel model is developed using biophysical analysis performed to create a circuit model for Capacitive EQS-HBC channel20,23,24 and embed further capacitive elements to accurately represent the touch sensing system present in touchscreen interfaces. The simplified model is developed by approximating the touch sensing grid as a metal surface to lump the capacitances between the sensing electrodes into one node. The capacitances used in the biophysical channel model are:

  • CB: Body capacitance from the lumped point that models the body to the earth’s ground.

  • CG-Tx: Return path capacitance, which models the parasitic capacitance between the floating ground plate of the transmitter and the earth ground.

  • CSIG: Capacitance between the finger and the additional receiver electrode.

  • CF: Capacitance between finger and the metal plane emulating the touch sensing grid electrodes.

  • CP: Capacitance between the metal plane emulating the touch sensing grid and the system ground.

  • CC: Capacitance between the metal plane emulating the touch sensing grid and the additional receiver electrode.

  • CG-Rx1 and CG-Rx2: Return path capacitance from the system ground to the earth’s ground and the additional receiver electrode to the earth’s ground.

The received voltage in a ToSCom system is the differential voltage picked up between the additional communication electrode and the system ground. This differential signal is a function of the capacitances shown in the biophysical channel model. Further, the magnitude of the differential signal is dependent on the amount of signal coupled from the body to the additional communication electrode and the signal coupled to the system ground. The signal coupled to the additional electrode depends on the magnitude of the capacitances CSIG, CC, CG-Rx2 (marked in green in Fig. 2e). Similarly, the amount of signal coupling to the system ground is dependent on the capacitances CP and CG-Rx1 (marked in blue in Fig. 2e). The results obtained using Finite Element Method (FEM) based Electromagnetic simulations and experiments will be analyzed using the biophysical circuit model developed.

To find the optimal receiver configuration, detailed FEM based Electromagnetic analysis is performed using Ansys High Frequency Structure Simulator (HFSS), which are verified using experiments conducted in a lab setting.

Design of receiver electrodes

For a low-loss communication channel, it is essential to design the receiver electrode configuration efficiently for receiving data. The structure of a typical capacitive touch sensing system for a touchscreen-based device is illustrated in Fig. 1d.

Four basic configurations of the receiver electrodes are analyzed to study the dependence of channel characteristics on the structure of the receiver. One of the electrodes (electrode 2) of the receiver is always set as the system ground of the touchscreen system, as it provides the largest surface area electrode from the touch sensing setup. Intuitively, we understand that the signal coupled to the system ground will be large when the finger is close to the touchscreen. Electrode 2 (ground of touch sensing setup) is shown in “green” in Fig. 3a. The four basic configurations of electrode 1 are listed below:

  • Configuration A: Metal strip from the touch sensing system.

  • Configuration B: Additional metal layer above ground plate.

  • Configuration C: Additional metal ring around the touch sensing system.

  • Configuration D: Additional small metal electrode.

Configuration A uses the existing touch sensing setup without any additional electrodes. Electrode 1 in this scenario is one of the metal strips from the touch sensing grid. Configuration B uses an additional metal plate as electrode 1 between the system ground and the touch sensing grid. Configuration C places an additional metal ring around the sensing grid structure which is used as electrode 1. In configuration D, a small metal electrode is introduced, which may be a pin or a metal pad on the PCB for the touch sensing system. The field plot shown in Fig. 3b is the steady state electric field magnitude infinitesimally above the system ground. The placement of the small electrode of size 1mm2 has to be such that the voltage difference captured by the electrode is tthe he highest. This placement of the electrode can be deduced from the field plot shown. Analytically, the small electrode must be placed at a position where the highest field strength is measured such that the voltage difference is maximized. We can represent the voltage difference captured by the receiver mathematically, as shown by

$$\Delta V=-\int\,\overrightarrow{E}.\overrightarrow{{{{\rm{d}}}}l}$$
(1)

To maximize ΔV, we need to maximize the electric field integrated. As observed from the field plot in Fig. 3b, the highest field strength is present at the sharp edges of the ground plane. The field strength is typically higher at the corner of metal surfaces due to a higher charge density. Thus, the electrode must be placed at one of the corners to gain the lowest channel loss across the whole touchscreen surface.

The four configurations chosen are now studied to analyze the channel loss characteristics and their agreement with the criteria listed in Section 2.3.

FEM-based EM simulations

The simulations are performed to compare the 4 configurations of the receivers illustrated in Fig. 3a. To measure the channel loss across the entire screen, the finger is moved across the leading diagonal of the touchscreen system, as illustrated in Fig. 3c. Due to the symmetric nature of the touchscreen setup, moving the finger along the leading diagonal allows us to visualize the channel loss across the whole screen accurately.

Figure 3d illustrates the channel loss on the movement of the finger along the leading diagonal. We observe that in configuration A, where one of the sensing grids is used as electrode 1 of the receiver, the channel loss is lowest when the finger is close to electrode 1 and then drops off as the finger moves away from the electrode connected to the receiver. This provides a localized region of low channel loss around the region where electrode 1 is present. This resembles the non-ideal channel loss characteristic illustrated in Fig. 2d. Further, configurations B and C (additional metal plate and additional ring) provide a flat band loss for most parts of the touchscreen surface, but the channel loss is high (≈70–80 dB) for the complete screen surface. This is due to the fact that both electrode 2 (system ground) as well as the additional electrodes (electrode 1) in configurations B and C have a lot of signal coupled due to a high CSIG, CC and CP. This results in a smaller differential voltage picked up by the receiver.

Figure 3d shows that using a small ground plate as in configuration D has a low and flat band channel loss response about half of the touchscreen surface. A low channel loss is observed in Configuration D when the finger is further away from the small electrode due to a small amount of signal coupled to the small electrode in comparison to the signal coupled to the system ground. This ensures that the differential voltage is higher when the finger is away from the smaller electrode. However, as the finger keeps moving towards the small electrode, CSIG and CC keep increasing, resulting in a higher signal coupled to electrode 1. This reduces the differential signal captured as the finger keeps moving towards electrode 1. Eventually, as the finger keeps moving closer to the smaller electrode, the signal coupled to the smaller electrode keeps increasing beyond the signal coupled to the system ground, which shows up as a notch in the channel loss characteristics at the position “180cm.” Hence, a modification to configuration D must be made to ensure that the channel loss is lowest across the entire screen with a flat response.

This is done by including another small electrode diagonally opposite to the original small electrode described above, as illustrated in Fig. 3e. We connect two receivers between the two small electrodes and the system ground. When the position of the finger is close to one of the electrodes, it is furthest away from the other electrode, and thus the overall channel loss of the received signal is minimized. This is illustrated in Fig. 3f We observe that using two small electrodes provides us a flat channel loss response of  ≈55dB across the entire touchscreen surface, which was the optimal channel loss characteristic desired (Fig. 2d).

Further, configurations B and C have large additional metal electrodes, increasing the parasitic capacitance and thus reducing the sensitivity of the touch sensing setup. Using small electrodes minimizes the parasitic capacitance and has a low channel loss across the entire screen. Electro-Quasistatic Human Body Communication enables strictly touch-based communication, as illustrated by Fig. 3f. The received voltage has a sharp drop of  ≈10 dB when the finger moves 1 cm away from the touchscreen surface.

Experiments

Experiments have been conducted using a commercial off-the-shelf setup to demonstrate that a wearable-to-wearable communication channel can be successfully established using a touch sensing setup and a dedicated communication module. Experiments to validate the channel loss response obtained in simulations are performed using a capacitive touch sensing board, TI Captivate BSWP37 and a pocket spectrum analyzer TinySA38. Further, an EQS-HBC transceiver setup, Wi-R evaluation kit by Ixana39,40, is used to demonstrate successful data transfer along with touch detection and location using the TI captivate BSWP touch sensing system. The experiments are performed to observe and validate the trends illustrated in the simulation results to enable high-speed communication through touchscreen. The channel loss magnitude in the experiments and the simulations vary due to the differences in the dimensions of the two setups, resulting in different values of the capacitances shown in the biophysical channel model illustrated in Fig. 2e.

Figure 4a shows the positions in which the touch sensor (TI Captivate BSWP board) is touched. Fig. 4b shows that the experimental results validate the findings of the FEM-based EM simulation study. We observe that the small electrode configuration (configuration D) of the receiver has the lowest channel loss, as well as a flat channel loss response across the entire touchscreen. Configurations A, B, and C have a higher channel loss as expected from the results obtained in the simulations. Further, to illustrate the variation of channel loss with different operational environments, we demonstrate the path loss variation for varying postures in three different environments - a conference room, an empty parking lot, and an anechoic chamber (Fig. 4c). The variation of path loss due to changing environments is observed to be minimal (<5 dB)27. This is because the variation of return path capacitance with varying environments is typically minimal unless there are strong interference sources around the ToSCom system41. The dimensions of the transmitting and receiving devices are generally much smaller than the distance between the device and the Earth’s ground24. Varying posture has a notable effect on path loss due to the effect of the interaction of the electric field lines between the transmitter and the receiver with the human body. A low path loss is observed when the transmitter and receiver are close to each other, as in the case of the experiments with the “Arms Up” posture. This is due to the effect of inter-device coupling, as previously illustrated by Datta et al.23 and Yang et al.27. Due to the close proximity of the devices in this posture, a direct coupling between the floating ground plane of the transmitter and receiver provides an extra return path, resulting in a lower channel loss of up to 20dB. Further, in the “Arms Down” posture, the receiver and transmitter floating ground planes are shadowed by the body resulting in a lowering of the return path capacitance, increasing the channel loss by up to 10dB. Finally, the “T-Pose”, is a compromise between the previous two postures. Here, the effects of inter-device coupling and body shadowing by the torso are minimized. For all other experiments, we use the “T-Pose” as the posture for this study.

To demonstrate successful communication between a wearable transmitter and a wearable touchscreen-based receiver, a Wi-R transceiver is used. The transmitter module transmits 64-byte pseudo-random binary sequence (PRBS) packages at a data rate of 3 Mbps. The bit-error-rate (BER) is measured by the receiver for all configurations of the ground plane for around 107 bits, as shown by Fig. 4d. The BER for the small electrode setup (configuration D) is observed to be consistently the lowest across all positions of the touchscreen setup. The other configurations (A, B, and C) have a higher average BER over the entire touchscreen. We use a concept figure to further illustrate the trends in the BER results shown. The BER results are calculated for edge cases where the SNR of the system is in the blue shaded region of the concept figure. This allows us to better understand the variations in BER for different positions on the device for the 4 configurations of electrodes used in the experiments. The average BER for all the configurations is estimated on the plot to illustrate the effect of higher path loss on BER of the system. In practical scenarios, channel loss margins are implemented to ensure that these edge cases are avoided to ensure low communication BER is achieved consistently.

A sample transmitted waveform measured using a tabletop oscilloscope from the WiR transmitter is illustrated in Fig. 4e. A zoomed in view of 1 bit data width shows 6 pulses of the carrier signal forming the single data bit, illustrating a 3 Mbps data transfer at 18MHz carrier frequency. The waveform is an On-Off-Keying (OOK) signal with an amplitude of 2.5Vpp. We also illustrate the same signal after it is passed through the body channel. The signal after it is passed through the body is taken by the receiver for further processing and decoding.

We further demonstrate a communication system implemented successfully by adding a transceiver to a touchscreen interface redesigned to optimize the communication channel loss, while simultaneously detecting and locating touch events using the touch sensing setup of TI Captivate BSWP (Supplementary Note 3)42. The transmitter LED data is communicated successfully using a Wi-R evaluation kit, while a software (TI Captivate Design Center) on a computer connected to the receiver setup detects the touch location. Successful communication at 3 Mbps data rate was performed using the communication module, which demonstrates the potential of ToSCom for simultaneous high-speed data transfer with touch detection and location.

Dual small electrode based receiver

The complete ToSCom receiver setup is illustrated in Fig. 5a. As explained using simulations and experimental results, the finalized electrode design consists of two small additional electrodes, which provide the optimal channel loss for all positions of touch detected on a touchscreen system. Using additional small electrodes also ensures that structurally, the touch sensing system is minimally modified, and additional parasitic capacitance is also minimal with no perceivable effect on the touch sensing sensitivity. The small additional electrodes can be an additional pin or metal pad that is connected to the receiver module.

Fig. 5: Simulation and experimental setup for the development of ToSCom.
figure 5

a The complete touchscreen communication (ToSCom) receiver (Rx) setup with two small electrodes is illustrated. b Finite-element-method (FEM) based Electromagnetic simulations are performed using Ansys High Frequency Structure Simulator (HFSS). The structure of the transmitter (Tx) and touchscreen receiver (Rx) is illustrated. The complete structure of the touchscreen is developed using Autodesk Fusion 360. c The experimental setup is shown, with the transceiver setup being an Electro-Quasistatic Human Body Communication (EQS-HBC) based Wi-R evaluation kit by Ixana. The receiver setup for measuring channel loss is created using a buffer for high impedance termination, which is connected to a TinySA spectrum analyzer.

Full size image

ToSCom channel optimization for wider applications

The optimal placement of the small electrodes depends on the structure of the ground plane for the touch sensing setup. In this study, the optimization was performed for a touchscreen of smartphone form-factor due to its wide use cases. The ground plane is a rectangular structure where the vertices are the positions where the highest field strength is observed due to a higher charge density accumulating at the sharp corners. The small additional electrodes ensure that the differential signal captured is maximized by coupling less signal onto the smaller electrode as compared to the system ground of the touch sensing system.

The aim while designing the ToSCom communication system is to decrease the channel loss for the channel across the entire touchscreen surface. In this regard, the size of the transmitter and receiver also becomes important. The larger the size of the receiver, the smaller the channel loss of the ToSCom system. Increasing the size of the receiver effectively means that the system ground plate increases in size and area. This increase in system ground plate area implies an increase in the parasitic return path capacitances (Cret-Rx), thus reducing the channel loss. Further, increasing the load capacitance between the receiver electrodes (small electrode and system ground) decreases the received signal, resulting in additional path loss, reducing the signal-to-noise ratio (SNR). Thus, minimizing the load capacitance across the receiver electrodes is key to ensuring a high SNR (Supplementary Note 4).

The Touchscreen Communication system developed is the first-of-its-kind system, which can perform high-speed data transfer through the touchscreen interface. The ToSCom system is demonstrated for a 3 Mbps data rate communication for a wearable/portable form factor device with an average bit-error-rate of less than 5 × 10−7, enabling seamless data transfer through a touchscreen for any digital data, including but not limited to authentication, biomedical signals, productivity application data, and audio signals.

Future work

The current implementation of the ToSCom system relies on commercial off-the-shelf touchscreen sensors and EQS-HBC communication modules. The use of commercial devices to create the ToSCom platform makes the device bulky. We can further optimize the device size and data rates for Touchscreen Communication with an application-specific custom circuit implementation with an integrated touch sensing and communication module. The use of TI Captivate BSWP setup for touch detection and location also makes it a bench-top setup when the verification of touch detection and location is performed for ToSCom due to the need to connect with a computer to run the TI Captivate Design Center software. A custom implementation of wearable-wearable ToSCom to demonstrate simultaneous touch sensing and high-speed communication in small form factor devices will further optimize the communication channel performance and expand the applications of ToSCom for communication using touchscreens strictly through touch. Furthermore, the ToSCom platform can be used in conjunction with application-specific hardware and software. This allows us to develop detailed application-specific devices which benefit from the use of ToSCom for high-speed data transfer. These applications and devices with experimental validation on multiple human subjects can highlight the Human–Computer-Interaction aspects of this research.

The ToSCom channel has been optimized to provide high data rate from an on-body transmitter to a capacitive touchscreen-based receiver. Increasing the applicability of ToSCom requires bidirectional communication with the touchscreen-based device as the transmitter to an on-body receiver. With the current design of the ToSCom system, an asymmetric bidirectional channel where a low data rate communication is possible between the touchscreen-based device to the on-body wearable can be performed. The higher channel loss in the reversed channel results in reduced channel capacity. Theoretical channel capacity results show that a low data-rate communication of 10–30 kbps is possible with the current design, which allows high data-rate communication from the wearable transmitter to a touchscreen-based receiver with acknowledgements sent in the return channel with a lower data rate. With additional effort on designing a more symmetrical communication channel, we can achieve higher data rates from the touchscreen-based device to the on-body node by using additional transmitting electrodes, which optimize the communication performance of the reverse channel. A complete theoretical analysis of the asymmetric bidirectional ToSCom channel is provided in Supplementary Note 5.

Theoretical calculations of SNR for an Additive White Gaussian Noise (AWGN) dominated channel shows that the channel capacity for a bandwidth of 4MHz (bandwidth of WiR module used) is of the order of 10s of Mbps (Supplementary Note 4). Channel capacity gives the theoretical maximum data rate possible for a given channel bandwidth and SNR. Thus, using a different modulation scheme than On-Off-Keying (OOK) can further enhance data rate, leading to higher data rate communication with the same channel. The analysis of the variation of SNR and Channel Capacity with varying transmitted voltage and load capacitance is provided in Supplementary Note 4.

Conclusion

With the touch sensing systems increasing in popularity with an increasing number of wearable devices that we interact with, enhancing their functionality will improve the user experience and allow us to extend their functions beyond touch detection and location. In this study, we develop and demonstrate Touchscreen Communication, a system that performs simultaneous touch detection, location, and high-speed data transfer. The system developed has been demonstrated to perform 3 Mbps data transfer. To enable, strictly touch-based communication, we use Electro-Quasistatic Human Body Communication (EQS-HBC), where data is transferred using the body as the channel. Finite Element Method based EM analysis is used to perform the initial design for the entire system, which is then validated using experiments performed. ToSCom extends interaction with the electronic world by introducing high-speed data transfer through touch.

Methods

Simulation setup

Simulations are performed using Finite Element Method (FEM) based EM simulations on Ansys HFSS. The simulation setup is described in Fig. 5b. The sensing electrode grid emulating a touchscreen setup is created using AutoCAD Fusion 360 software. The metal layers including the ground plate of the touch sensor are assigned the material “copper”. The additional electrodes (metal sheet, ring, small electrode) for communication are added as “Perfect Electrical Conductor (PEC)” sheets. An arm and finger model is created with a cuboidal structure made of muscle tissue which has the electrical properties of muscle from the Gabriel-Gabriel database43. The cuboidal structure is used to approximate the arm to reduce computational complexity and reduce simulation time. The transmitter is placed on the simplified cuboidal arm model. The transmitter consists of a signal plate and a floating ground plate. A voltage excitation is applied between the two electrodes of the transmitter. The simulations are performed at 20MHz frequency which is within the Electro-Quasistatic (EQS) regime10. The voltage difference between any two points is measured by integrating the electric field along a line connecting the two points.

Human subject experiments

The study requiring measurements of channel loss and BER on human subjects has been approved by the Purdue Institutional Review Board (IRB Protocol no. 1610018370). All guidelines and regulations, as given by the Purdue IRB, were followed during the experiments. Informed consent was obtained from all the participants for the experiments.

Experiments: EQS-HBC transceiver setup

The transceiver used for the experiments and demonstration (Supplementary Video 1 and Supplementary Note 3) is a Wi-R Evaluation Board kit by Ixana which is illustrated in Fig. 5c. The evaluation board uses Capacitive EQS-HBC to transmit data through the body. The transmitter is connected to a smartphone, which powers and programs the communication module. The transceiver is configured to transmit 3 Mbps data rate signal at 18 MHz with a transmitted voltage of 2.5 Vpp. The transmitter sends 64-byte packets of PRBS-5 data during the BER experiments. The experiments to calculate bit-error-rate (BER) as well as the LED demonstrations are performed at a data rate of 3 Mbps. The BER measurement is performed over  ≈107 bits of data. The BER measurements are taken at every position over 5 times, and the mode (signifying the steady-state value) of the result is illustrated in Fig. 4d. The successful high-speed data transfer through the body using the ToSCom system is enabled through a low channel loss due to careful receiver electrode design which increases the Signal-to-Noise ratio at the receiver and with an increased sensitivity receiver optimized for Electro-Quasistatic Human Body Communication. A detailed discussion on the transceiver system with the system schematic, data packet structure, and the transmitted waveforms illustrating high-speed data transfer at 3 Mbps is provided in Supplementary Note 6.

Experiments: touchscreen receiver setup

The touchscreen setup is created using the TI Captivate BSWP capacitive touch sensor system as the touch sensor. Additional electrodes are created using Aluminum foil to emulate the simulations performed. The received signal strength is measured using a pocket spectrum analyzer “TinySA”38 which is connected to a buffer developed in-house using TI OPA283644 to provide a high impedance termination for accurate measurement from the electrodes connected to the body. The complete setup is housed inside a box which is of a wearable/portable form factor as illustrated by Fig. 5c.

Experiments: data transfer setup

The data transfer experiments are created using the custom transmitter node described above and a corresponding custom receiving node. The receiver is of a wearable form factor and is attached to the Captivate BSWP setup for emulating a touchscreen setup. The communication module transmits data at 3 Mbps data rate and the receiving module once done receiving is connected to a smartphone to analyze the collected data to provide the BER and data packets received.

Complete Experimental Setup

For the channel loss studies (Fig. 4b), a single WiR module is used as the transmitter, which is connected to a smartphone using a USB-C connector. The transmitter is configured to continuously transmit data packets at 3 Mbps centered at an EQS frequency of 18 MHz. A commercially available spectrum analyzer is connected to the Captivate BSWP capacitive touchscreen surface housed in a box which acts as the receiver setup. The different electrode configurations for the receiver (A, B, C, and D) are created by using conducting Aluminum foil to model the structure of the additional electrodes resembling the additional electrodes in the simulations. For example, for configuration C (additional ring-shaped electrode), a thin Aluminum foil strip is wound in the shape of a ring along the sides of the receiver box.

For the Bit-Error-Rate (BER) measurements (Fig. 4d), a WiR receiver module is connected to the Captivate BSWP touchscreen interface. The transmitter setup is the same as that for channel loss experiments with a WiR transmitting module connected to a smartphone with a USB-C connector. The transmitter and receiver modules are configured to transmit and receive PRBS-5 data at 3 Mbps with an On-Off-Keying (OOK) modulated waveform centered around 18 MHz to calculate the communication BER, prior to the start of the experiments. The transmitter and receiver are also set up to communicate data for a 2–5 second period, in which time the devices are held steady at the T-pose illustrated in Fig. 5c. Once the transmitter stops transmitting data, the receiver is disconnected from the Captivate BSWP system and connected to another smartphone with a USB-C connector to read out the measured BER result.

Demonstration setup

The demonstration video (Supplementary Video 1 and Supplementary Note 3)42 is presented to show the potential of ToSCom in performing high-speed data transfer along with touch detection and location. The transceiver used is the Wi-R evaluation kit by Ixana. The transmitting node is connected to a smartphone and the receiver node is connected to the touchscreen sensor, TI Captivate BSWP. The TI Captivate BSWP node is connected to the computer via the TI Captivate FR2633 MCU board and TI Captivate-PGMR programmer. The computer runs the software TI Captivate Design Center, which displays the location and detection of touch. The transceiver is set to LED mode, where data packets are transmitted at 3 Mbps data rate to communicate the status of the LED to the receiver, which displays the LED status of the transmitter.

The wearable-wearable setup in the second part of the video consists of the transmitter setup connected to the smartphone as used previously with a commercial off-the-shelf receiver setup consisting of the touchscreen board, TI Captivate BSWP connected to the receiver node (Wi-R evaluation kit by Ixana). The transceiver pair is set up to operate in the LED mode to visualize data transmission through the body.

The wearable-wearable setup demonstrated in the second part of the demonstration video (Supplementary video 1 and Supplementary Note 3) is representative of the envisioned use cases in real applications, whereas the table-top setup was necessary to visualize the simultaneous communication and touch sensing in a large screen.