Introduction

Facial muscles are critical in numerous functional and behavioral human activities, including eating, blinking, communicating, and socializing. For each function or expression, different facial muscles are activated, and the external appearance of the human face is reshaped into distinct static or dynamic patterns1. While vast literature from recent decades associates facial expressions (e.g., smiling, frowning, closing the eyes) with specific muscle activation, inferring muscle activation from facial features in a precise manner is elusive2. The difficulty of associating facial features with muscle action in a simple and non-invasive way remains a major hurdle in medical and psychological research and practice.

This limitation is of special significance in medical diagnostics, where pinpointing facial muscle dysfunction is essential in assessing and treating various conditions. Facial muscle dysfunction is a debilitating condition impacting many physiological and social functions. It can result from either central or peripheral pathologies. Peripheral facial nerve dysfunction severity can range from neurapraxia and axonotmesis to neurotmesis, reflecting increased levels of nerve damage and functional impairment3. Although evident in its appearance, facial muscle dysfunction poses significant diagnostic challenges. In particular, distinguishing between different conditions and severity4. Contemporary medical evaluation of muscle dysfunction consists of direct visual observation or primarily qualitative analysis using invasive needle Electromyography (nEMG). Despite ongoing debates regarding the predictive value of various electrophysiologic tests in acute peripheral facial nerve palsy cases, nEMG continues to be a critical tool for evaluating nerve function and muscle activity5.

In contrast to nEMG, non-invasive facial surface Electromyography (sEMG), which uses surface electrodes to measure muscle activity, offers a less invasive and, therefore, more patient-friendly alternative. sEMG is a versatile technique with applications in medical and psychological research4,6,7,8,9. This technique can provide accurate, real-time insight into facial muscle function across many facial areas, making it a valuable addition or complete alternative in diagnosing and managing facial palsy, a condition characterized by the weakening or paralysis of facial muscles due to nerve damage. sEMG is currently not used in routine diagnostics of patients with acute facial palsy, even though it can offer more detail for further diagnosing patients with persistent nerve paralysis or synkinetic reinnervation, a chronic condition following initial palsy. By comparing the affected to the contralateral side, sEMG can provide detailed information on muscle activation like compensatory movement patterns, and readily quantify facial a-symmetry4. Thus, it can aid in developing objective methods for facial grading based on muscle activity, as opposed to the numerous image-based approaches currently available10,11,12. sEMG also offers potential in facial muscle rehabilitation. By pinpointing areas of muscle weakness and evaluating the effectiveness of exercises, healthcare professionals can craft personalized treatment plans and optimize rehabilitation strategies. One such approach is sEMG biofeedback training, where patients receive visual or auditory cues based on their muscle activity, helping them improve the accuracy and precision of their muscle contractions during exercises13.

Facial sEMG measurements using wired electrodes are limited, typically confined to a laboratory environment, and the occurrence of motion artifacts, which restricts their effectiveness in natural settings14. Additionally, the precise positioning of many sEMG electrodes is time-consuming and is therefore not suitable for clinical routine6,15. Screen-printed electrode arrays on soft support offer an alternative to the cumbersome gelled sEMG electrodes16. These electrodes exhibit ease of operation, fast placement, convenience to the patient, and, as recently established, high-quality data comparable to gelled electrodes in facial EMG applications15,17.

The electrode array uses the eye as a consistent anatomical reference (an arrow on the mask points toward the exocanthion, the outer corner of the eye fissure), ensuring reproducible placement. Even when electrodes do not align exactly due to facial variability, data processing methods allow meaningful information to be extracted. Owing to their unique properties, particularly their suitability for home-based, self-use, they are an ideal platform to explore automated tools to expedite analysis.

Improving the interface with easy-to-use and stable electrode technology15 is an important step toward advancing sEMG technology, but significant challenges remain across all EMG electrode types. One of the most demanding aspects is the precise extraction and interpretation of meaningful data from sEMG signals. These signals are often complex and prone to contamination from various sources, such as white noise, movement artifacts, electrical interference, and cross-talk. Filtering these unwanted components while preserving relevant muscle activity requires sophisticated signal-processing techniques. Cross-talk is a particularly difficult issue, where signals from nearby muscles interfere with the intended measurement18. Isolating the target muscle’s signals from surrounding muscle activities is essential for accurate interpretation16. Another challenge is comparing sEMG signals across different individuals or experimental conditions to establish reliable and consistent measurement standards, which is inherently complex. Developing standardized protocols that account for individual variability while ensuring consistency across settings is a major technical hurdle in this field. Turning raw sEMG data into clinically meaningful information, such as identifying muscle activation patterns or diagnosing neuromuscular disorders, requires advanced analysis methods and expertise. Determining the clinical relevance of sEMG findings and how they inform diagnosis, treatment, or rehabilitation involves integrating data with other clinical information (e.g., facial expression), adding another layer of complexity. Overall, key technical challenges in facial sEMG involve enhancing signal quality, minimizing interference, developing robust processing methods, ensuring measurement consistency, and extracting valuable information from complex and variable data.

Owing to the limitations of contemporary high-resolution sEMG, the link between muscle activity and facial expressions has received relatively little attention1,2. Accounting for the wide use of computer vision analysis of facial expressions in medical and psychological research, achieving precise mapping between facial muscles and facial external features is of immense value. Specifically, which muscles underlie specific facial actions (e.g., lip pulling, smiling, snarling), whether the association between muscles and facial expressions is consistent across expressions and their repetitions, and whether repetitions are stable across participants.

In this paper, we combine state-of-the-art conformal and dry skin-electrode technology with a novel analysis approach to build and test a facial sEMG system capable of robust, non-invasive monitoring of muscle activity. The system was designed to provide accurate data for clinical applications, offering improved accessibility and precision that is compatible with diagnostics and rehabilitation settings. The main goals of the investigation reported here are: (1) to enhance signal quality to improve the accuracy of identified muscle activation sources, (2) to develop a novel system of muscle units, which we refer to as a facial muscle atlas, and (3) to establish a computational model to predict facial expressions from facial sEMG data. To achieve these goals, we implemented a new approach for automated facial sEMG analysis, focusing on precise identification of muscle activation. Specifically, we applied independent component analysis (ICA) to facial sEMG data, creating an automated mapping of facial muscles with high specificity, resulting in the creation of a standardized facial muscle atlas. To further enhance the robustness of the atlas, we denoised the sEMG signals using wavelet transform and thresholding techniques before applying ICA, ensuring more accurate and reliable muscle identification.

Results

Wireless sEMG system design and implementation

The wireless system used in this investigation consists of several key components, as shown in Fig. 1. The system features a wearable and easy-to-apply sEMG technology (Fig. 1a), designed to ensure user comfort and ease of application. The electrode array includes a printed reference electrode specifically designed for placement behind the ear, enhancing signal stability (Fig. 1b). Data were collected wirelessly through a compact DAU (Fig. 1c), enabling real-time transmission of the recorded signals. The system’s performance is demonstrated in Fig. 1d1, d2, showing sEMG data in the time and frequency domains (Fig. 1d1), and further processed with denoised using wavelet transform and hard thresholding (Fig. 1d2).

Fig. 1: Conformal, easy-to-apply sEMG technology.
figure 1

a Demonstration of electrode placement on facial muscles. b Electrode array design, including a printed reference electrode for placement behind the ear. c Wireless DAU. d1, d2 Examples of data acquired by the system during facial muscle activity. d1 Filtered signal in the time and frequency. d2 The filtered and denoised signal using wavelet denoising and hard thresholding.

Full size image

Clustering independent components (ICs) to create the facial muscle atlas

To reduce cross-talk in the sEMG data, we performed ICA. ICA output conventionally lacks a specific order, and assigning components to their spatial locations is needed to understand the physiological meaning of each IC and to compare facial muscle activity between participants19. To address these issues, we have constructed a standardized facial muscle atlas. The atlas was built using data from 32 participants (Fig. 2). Facial sEMG data were first collected with soft facial electrode arrays, and then each participant’s data were filtered, denoised, and analyzed using ICA. Finally, the ICs of all the participants were clustered based on spatial similarity, and an averaged representation of each cluster was calculated. Each heatmap in the atlas is a mean heatmap of all the spatially similar heatmaps assigned to this cluster by the algorithm. Each cluster consists of 25–32 participants. The variable N indicates the number of participants who contributed an IC to each cluster, reflecting the group variability within each pattern. If a participant’s set of ICs did not include a component that matched a particular cluster based on our distance threshold criterion, they were not counted in that cluster’s N value. Figure 2c shows a horizontal bar graph displaying each component’s root mean square error (RMSE) relative to the cluster mean, offering a quantitative measure of the clustering algorithm accuracy, with and without denoising. The atlas heatmaps capture distinct and consistent patterns of facial muscle regions across participants, which can then be applied to investigate different facial actions. Cluster 17 contains 21 components, deviating significantly from the other 16 clusters.

Fig. 2: The facial muscle atlas constructed from sEMG data of 32 participants.
figure 2

Each cluster (presented in b) presents the mean heatmap of a cluster of ICs (presented in a). Each IC is derived from sEMG data. Each heatmap represents a distinct pattern of muscle activation. Clusters are sorted and numbered according to the order of the electrode array channels: From 1 (close to the mouth) to 16 (at the forehead). The variable N (in a) denotes the number of participants who contributed an IC to each cluster. Values range between 25 and 32. c A horizontal bar plot, which illustrates the RMSE between each component in a cluster and the cluster mean, provides a quantitative measure of clustering accuracy. Cluster 17 contains 21 components, deviating significantly from the other 16 clusters.

Full size image

ICA and wavelet denoising for facial expression analysis

To exhibit the effectiveness of ICA and wavelet denoising, we compared the ability to distinguish between different facial actions using filtered sEMG data with and without ICA and wavelet denoising. We analyzed the amplitude per source (either ICA component or sEMG channel) for various functional movements. For each participant, we calculated the RMS of the signal in each channel during the 11 functional facial expressions. To account for amplitude variability both across different muscles within the same participant and between different participants, we divided the RMS of each channel by the RMS of the corresponding channel measured in an equivalent time window preceding the expression. These values were then averaged across participants. The results are presented in Fig. 3a, b. The panels show the results for each approach (from left to right): Filtered sEMG data, ICA without denoising, and ICA with wavelet denoising. The figures in Fig. 3a were normalized by the maximum value in each facial expression, and figures in Fig. 3b were normalized by the maximum value in each IC. These panels reveal the effectiveness of ICA and wavelet denoising in establishing better-localized sources.

Fig. 3: Comparing the ability to distinguish between different facial actions using filtered sEMG data with and without ICA and wavelet denoising.
figure 3

Each matrix shows the Mean RMS values of all 32 participants’ sEMG versus ICA with and without denoising during functional facial expressions. From left to right: Filtered sEMG data, ICA without denoising, and ICA with wavelet denoising. a Normalized amplitude per facial expression. b Normalized amplitude per source. c Confusion matrices which test the discrimination ability of each approach. Every matrix element is the correlation between two facial expressions. Well-separated expressions are typified by no off-diagonal correlation exceeding 0.95.

Full size image

To further evaluate the data pre-processing approach, we calculated the confusion matrix between all functional facial expressions after each data analysis step. The results are shown in Fig. 3c. No off-diagonal correlation typifies well-separated expressions. Significant off-diagonal values, on the other hand, indicate close similarity or inability to achieve separation between the two expressions. Indeed, ICA contributed to reducing off-diagonal values. To quantify this improvement, we evaluated the mean sensitivity of all participants of each method by using the diagonal elements of the confusion matrix as true positives and false positives as off-diagonal correlations exceeding a threshold set to 0.95. Altogether, ICA after denoising has resulted in 0.86 mean sensitivity compared with 0.83 without denoising and 0.77 for filtered sEMG. Despite the improved sensitivity, there is still, on average, confusion between different actions.

To characterize each cluster’s functional and anatomical properties, we cross-referenced these findings with anatomical structures by examining each cluster’s spatial distribution and identifying the corresponding facial muscles in these regions. For each cluster, we noted the facial expressions that elicited activation higher than a threshold that we set to 0.6. The relationship between clusters, functional facial expressions, and anatomical structures is summarized in Table 1. We show the results, omitting the expressions face at rest and closing the eyes gently.

Table 1 Relationship between functional clusters, facial expressions, and underlying anatomical structures
Full size table

Deep learning based expression reconstruction

Finally, after establishing the facial muscle atlas, we demonstrate its utility to explore the link between facial expressions and muscle activation. We used sEMG and video data to train a deep learning model to reconstruct facial expressions from the sEMG data, as shown in Fig. 4. We recorded two additional participants and simultaneously collected sEMG and video data. Then, we applied wavelet denoising and ICA to the sEMG data. We then classified the ICs of each participant using the facial muscle atlas and our classification algorithm. Figure 4a shows 16 ICs of one participant during functional facial movements. Next, we trained the model to predict video data from sEMG data from one participant. Using Grid search optimization, we set the model’s hyperparameters as a hidden dimension of 128, a learning rate of 0.01, and 500 training epochs using Smooth L1 Loss as the objective function. The model demonstrated robust performance, achieving a training loss of 0.109 and a validation loss of 0.130.

Fig. 4: Reconstructing facial expressions from sEMG data using a deep learning model.
figure 4

a 16 ICs of one participant during functional facial movements. b Comparison of the 16 most highly correlated BS between model predictions (blue) and video (red). c Snapshots of a facial avatar controlled by video (left) and predicted BS (right). d Quantification of model accuracy by correlating predicted BS with ground truth and comparison of model performance on the participant the model was trained on and another participant.

Full size image

Figure 4b shows most highly correlated 16 BS and model predictions: Ground truth data (red) collected with video, and the model’s predictions based on preprocessed sEMG input (blue) of the participant. Notably, the predictions show more fluctuations than the ground truth. Figure 4c compares a facial avatar controlled by the video (ground truth) and BS predicted by the model based on the sEMG data for representative expressions. Since our model was trained to predict BS values only for the right side of the face, we mirrored these values to the left side when creating the avatar visualization to produce symmetrical and more natural-looking facial expressions. In Fig. 4d, we quantified the model’s accuracy by calculating the correlation between the predicted BS and the ground truth. For most BS, the model follows the BS data with correlation values ranging from 0.7 to nearly 1 (see Fig. 4d). Importantly, when using the trained model to predict data of the second participant (which we denote as a new participant in Fig. 4d), model performance was very poor, suggesting a participant-specific link between muscle activity and expressions.

Discussion

In this study, we combined advanced conformal skin-electrode technology with an innovative signal analysis approach. Our system is designed for non-invasive monitoring of facial muscle activity, providing enhanced accuracy and accessibility for clinical applications. We demonstrated that printed, dry-screen electrodes enable high-quality sEMG recordings while offering significant practical advantages over conventional gel-based systems, including rapid application and reduced preparation time15,20,21. These features make them promising for clinical use, particularly in long-term or repeated measurements such as sleep monitoring20. Other groups have highlighted the mechanical robustness and comfort of advanced printed or nanomesh-based systems. For instance22, introduced highly stretchable gold-nanowire-based films that retain conductivity under extreme strain, enabling durable and sensitive facial EMG applications. The group also presented ultrathin nanomesh strain gauges that conform closely to the skin, allowing long-term facial motion tracking with minimal interference23. Inzelberg et al. emphasized the practical benefits of soft, printed electrode arrays in eliminating the need for cumbersome gelled electrodes and wires while improving signal quality and user comfort24. Together, these developments support the growing shift toward printed sensor technologies in electrophysiological monitoring and provide context for the clinical applicability of our system.

The long-term wearability and comfort of these electrodes, especially for extended use in home-based or clinical settings was previously discussed. In our prior work20,25 we demonstrated the use of similar electrodes in multi-subject, full-night recordings at home, showing both improved user convenience (manifested by better sleep quality) and strong agreement with medical polysomnography. Additionally, in21, we validated the electrodes’ performance in dynamic conditions by capturing high-fidelity facial sEMG during sports activity, despite extensive mechanical movements. Collectively, these and other studies have substantiated the electrodes’ suitability for high-quality data acquisition in both clinical and non-clinical conditions. While the current electrode design supports widespread adoption, offering both user convenience and overall economic viability, alignment with clinical recommendation practices and insurance remains a challenge.

The primary objectives of this investigation were to improve the signal quality for more accurate identification of muscle activation, to create a comprehensive and standardized facial muscle atlas, and to demonstrate a proof of concept of a computational model capable of linking facial expressions and sEMG data.

To meet these goals, we developed an automated sEMG analysis method that utilizes ICA for precise mapping of facial muscles. By incorporating wavelet transform and thresholding techniques to denoise the signals before ICA application, we enhanced the robustness and accuracy of facial muscle atlas identification. This innovative approach led to the creation of the facial muscle atlas, providing a standardized and precise map of muscle activation patterns6.

While centering and whitening are common pre-processing techniques for ICA, other methods can potentially enhance its performance, particularly in applications like sEMG, where essential signal features are often ambiguous. The wavelet transform’s multiresolution analysis has proven valuable in such contexts, facilitating the detection and extraction of signal characteristics across various scales26. This approach offers a comprehensive signal representation that accentuates core properties by integrating extracted features. Notably, denoising signals using Wavelet Transform and thresholding has shown promising results in improving signal quality27,28. In addition, wavelet denoising effectively isolates muscle-related activity in complex recordings29,30, making it particularly suitable for our goal of extracting distinct facial muscle signals from surface EMG. Compared to alternative methods, wavelet-based approaches better preserve signal morphology while effectively removing noise31,32,33. Furthermore, the combination of wavelet denoising with ICA has shown high effectiveness in reducing EMG artifacts in bioelectrical signals improving model performance34,35. However, to date, the potential of this denoising method as a preprocessing step to enhance ICA performance, particularly in sEMG analysis, still needs to be explored. This study demonstrated how wavelet-based denoising, when applied as a preprocessing technique, can improve the efficacy of ICA in extracting meaningful components from sEMG signals.

While our results demonstrate the functional effectiveness of wavelet denoising for improving ICA-based muscle source separation, future work should include direct quantitative comparisons with alternative denoising methods such as empirical mode decomposition (EMD) and variational mode decomposition (VMD) using objective metrics like SNR or MSE. Such comparisons would be particularly valuable for optimizing real-time analysis applications, where computational efficiency must be balanced with denoising effectiveness.

The improved accuracy of muscle activation identification achieved through our signal processing techniques is crucial for clinical applications. Denoising the sEMG signals using Wavelet Transform and thresholding ensures that the data is not only cleaner but also more representative of true muscle activity, allowing for better diagnostics and rehabilitation outcomes. The facial muscle atlas also enables a systematic approach to studying muscle activity across different individuals. It provides a valuable tool for comparing facial expressions, whether naturally occurring or intentionally produced.

Indeed, some expressions are poorly differentiated owing to anatomical but also subjective variability in execution. The outcome of the current analysis further supports that for closely related movements, complementary methods (e.g., image-based analyses) should be considered for quantitative differentiation, whereas sEMG retains its unique value in qualitatively assessing underlying physiological and electrical activity36.

In the current study, we focused exclusively on healthy participants to establish a normative baseline for facial muscle activation using our proposed facial muscle atlas. As such, direct comparison with clinical diagnostic tools, typically applied in patient populations, was beyond the scope of this work. Validation against standard clinical assessments is essential to evaluate the diagnostic and rehabilitative potential of our system. In addition to the House-Brackmann scale and the Stennert index, future studies will also include comparisons with the Sunnybrook scale and eFACE, particularly to evaluate muscle atlas utility in evaluating facial palsy, post-paralytic synkinesis. Unlike nEMG, the proposed atlas enables simultaneous evaluation of multiple facial regions, allowing for a more comprehensive analysis of co-activation patterns, which is a key direction for future research.

Although this investigation employed an electrode array covering only one side of the face, bilateral facial measurements are critical in clinical diagnostics. Given the focus on healthy individuals in this studys, and prior studies indicating no significant differences in muscle activation between the two sides of the face in such populations37, unilateral recordings were considered sufficient. However, to enable full-face assessments in clinical populations, we have developed a bilateral version of the system that will facilitate more comprehensive facial assessment and accurate evaluation of facial asymmetry.

The sEMG system utilized in this study is equipped with 16 electrodes, allowing for the identification of up to 16 clusters. It is important to highlight that the human face contains about forty muscles, most of them in pairs. This means each cluster is influenced by multiple muscles as associated in Table 1 based on the most active movements for the respective cluster and the anatomical area covered by the cluster’s heatmap. Future work should investigate the optimal sEMG setup to improve muscle representation and enhance movement identification.

While the current study demonstrates a robust approach to facial muscle mapping using ICA, it is based on a relatively small and heterogeneous sample of healthy individuals. Nonetheless, future studies should include larger and more diverse populations. Importantly, to address clinical applicability, ongoing work in our group is focused on expanding the dataset to include individuals with facial motor disorders. Preliminary data from patients with chronic facial palsy have already been collected, and future analyses will explore how pathological activation patterns differ and how source identification can be refined in such cases.

An intriguing aspect of the results presented in Table 1 is the clear muscle co-activation phenomena where participants were instructed to perform functional movements for the first time. For example, cluster two shows high activation in “depress lip”. Although anatomically it is distant from muscles needed for that action (like the depressor anguli oris, and the depressor labii inferioris muscle). The heatmap of cluster two is located at the jawline and covers fibers of the platysma, a neck muscle that extends beyond the jawline and inserts into the skin of the lower face. Some fibers can also partially integrate into other muscles, such as the depressor anguli oris. Solely pulling down the corners of the mouth is not a typical movement in daily life for most individuals, and visual observation indicates that many people inadvertently activate their neck muscles during the depress lip movement as well. The co-activation of the neck muscles during depressing lips, appears to be reflected in the high sEMG activation observed in cluster two.

Additionally, the heatmap for cluster two does not encompass the buccinator muscle but may partially cover it. Still, the cluster indicates high activity during the action of blowing cheeks. Although the buccinator muscle can remain relaxed during this movement, visual observation shows that many people increase the pressure in the oral cavity by activating the muscle when inflating their cheeks. Furthermore, some fibers of the buccinator muscle intertwine with the orbicularis oris muscle, which plays a crucial role in keeping the lips closed while blowing cheeks. This close anatomical and functional relationship may be the reason for the observed co-activation in the sEMG data. Cluster two is just one example of the many insights offered by our technique into the complex dynamics of facial muscle interactions.

In this investigation, we also discussed a regression DNN that links visible facial expressions to muscle activation, as indicated by sEMG signals. Such models offer numerous opportunities for mapping and understanding facial muscle activity. Firstly, they allow for examining variability between individuals as we can observe in Fig. 4d. Secondly, it becomes evident that sEMG data exhibit distinct dynamics compared to facial expressions: muscle intensity fluctuates during facial actions, while the expressions remain relatively stable Fig. 4b. These dynamics and the fact that we could fit a non-linear model may reflect the complex interplay between applying force and the sensory input required to initiate and maintain facial movements. Exploring this dynamic further in healthy individuals versus those with facial palsy could provide valuable insights.

Our computational model, built on the atlas, has significant implications for future real-time applications, such as avatar control and feedback in research, clinical and home-based remote monitoring, and rehabilitation settings, particularly for patients recovering from facial paralysis or neuromuscular disorders. For example, individuals undergoing rehabilitation can use this system to visualize their facial movements on a 3D avatar. Watching an avatar might be more appealing to some participants than watching themselves, especially when combining it with gamification. Real-time feedback could help patients track their recovery and allow therapists to tailor interventions to improve muscle coordination and strength. Additionally, the model can assist in remote rehabilitation by enabling home-based therapy sessions where patients can perform exercises and receive immediate feedback, thus enhancing the accessibility and consistency of facial rehabilitation programs.

While our model demonstrated promising results, we acknowledge that its development might be constrained by the limited sample size from the three repetitions per expression and the predefined set of standardized facial movements. Although our temporal windowing approach (100 ms windows from 5 s recordings) effectively expanded the training data, more sophisticated predictive models would benefit from richer datasets. Future research in ecological settings should incorporate additional repetitions, a wider variety of expressions, including spontaneous and subtle movements, and extended recording sessions that capture natural variations in facial behavior.

Although the approach described in this paper concerns the utility of facial sEMG in medical practice, it also has significant utility in psychology. Distinct muscle activation patterns have been associated with particular emotional states38, and a more precise mapping of facial muscle activation may allow a more robust and consistent comparison between participants. In psychiatry, abnormal facial muscle activation patterns have been correlated to specific disorders such as schizophrenia and major depression39,40. The results presented in this paper may thus contribute to a better investigation of facial expressions of emotions.

Overall, this study demonstrates the potential of combining advanced sensor technology with sophisticated data processing techniques to create a user-friendly, robust system for facial muscle monitoring. By improving the accessibility and precision of sEMG analysis, our approach offers valuable insights for both the scientific community and clinical practitioners. For the first time, we demonstrated an accurate deep learning model, utilizing data collected with a conformal recording setup, allowing systematic and objective comparison between facial muscle action and facial expressions.

Methods

sEMG data collection

Screen-printed carbon electrodes were purchased from X-trodes Ltd. Data were recorded with a miniature wireless data acquisition unit (DAU, X-trodes Inc.), developed to allow electrophysiological measurements under natural conditions. The DAU supports up to 16 unipolar channels with a sampling rate of 4000 S/s, 16-bit resolution, an input range of ±12.5 mV, and an input impedance of 107Ω. A 620 mAh battery supports DAU operation for up to 16 hr. A Bluetooth (BT) module is used for continuous data transfer. An Android application controls the DAU, and the data are stored on a built-in SD card and the cloud for further analysis. The DAU also includes a 3-axis inertial sensor to measure the acceleration of the head during the measurements.

We placed electrodes following the protocol we previously published15. The electrodes contain 16 channels: channels 0–14 on the right side of the face and channel 15 on the left forehead (Fig. 1). A ground electrode was positioned on the right mastoid. The mastoid region was selected as the reference site as it offers several advantages for facial sEMG. This region is anatomically distant from the primary facial expression muscles and exhibits minimal muscle activity, which significantly reduces contamination of the reference signal. This placement is widely used in facial sEMG studies and is considered appropriate for capturing clean and reliable facial muscle signals25. Finally, it offers a convenient position for the placement of the electrode array, allowing for easier application by either clinicians or the subjects themselves.

The physical contact between the soft electrodes and the rigid DAU is achieved with a card connector. To minimize mechanical artifacts, it is critical to avoid relative movement between the DAU and the electrode array. Moreover, the DAU-electrode interface is positioned above the zygomatic bone, where it is minimally affected by facial movements, thereby enabling stable signal acquisition even during rapid facial expressions. Under conditions of proper DAU anchoring, minimized relative movement between the DAU and the array ribbon, and appropriate filtering, artifact-free sEMG data can be achieved, as demonstrated previously21. Importantly, despite global mechanical movements (e.g., head movements) or impulse noise (e.g., due to walking or expression onset), pretty clean sEMG data are readily recorded24.

Participants

35 healthy adults (ages 20–57 years, μ = 28.13, σ = 8.73) participated in the study. Three participants were excluded due to missing annotations. All experiments on human skin were conducted on volunteers and according to relevant guidelines and regulations, under approval from the Institutional Ethics Committee Review Board at Tel Aviv University (no. 0005248-2), following the Helsinki guidelines and regulations for human research. Informed consent was obtained from all participants. The authors confirm that any identifiable participants in this study have given their written consent for publication. The authors have obtained written consent to publish facial images.

Experimental protocol

The study was part of multiple studies conducted sequentially with the experimental setup described previously in ref. 15. In total, the study took about one hour. Briefly, the experimental protocol was implemented using PsychoPy (version 23.1.3)41, an open-source software platform for psychological experiments, and conducted in a controlled environment where participants were positioned in a standardized upright position facing a display monitor. The protocol in this study started about 30 min from the beginning of the recording. It included an automated video tutorial to ensure consistent instruction delivery across all participants (the tutorial was ~12 min long). The tutorial, presented by a human instructor, provided standardized guidance for executing specific facial movements. Each facial movement was guided by a demonstration sequence42,43. Subsequently, participants performed a series of 11 distinct facial expressions in synchronization with the instructor’s video demonstration. The facial movements were: Face at rest (no movement), wrinkling of the forehead, closing the eyes normally (eye gentle), closing the eyes forcefully (eye tight), nose wrinkling, smiling with closed mouth, smiling with open mouth (smile open), lip puckering (lip pucker), blowing out the cheeks, snarling, depressing the lower lip and a natural smile. Each facial expression was repeated three times per participant. As we showed in a recent study15, a statistical analysis comparing three repetitions of each facial expression revealed no significant differences between repetitions. For the analysis, we chose the highest quality repetition from each participant’s set of expressions, selecting the one with the lowest baseline activity and least noise.

Pre-processing

Filtering

In each channel, the noise level was assessed within the frequency ranges of 30–350 Hz for EMG bandwidth. A 4th-order bandpass filter was applied with the respective cutoff frequencies to extract the desired signals. Additionally, notch filters of 50, 100 and 200 Hz were applied, with a Q factor of 30. The filtering was done using the Scipy Python library44.

Denoising using discrete wavelet transform

We used a discrete wavelet transform (DWT) and hard thresholding to denoise the sEMG signals. DWT decomposes the signal into a series of coefficients that represent the signal in both time and frequency domains. We used Debauchies 15 (db15) as the mother wavelet and performed a five-level decomposition. For each 10 s time window, we computed each channel’s wavelet coefficients. To denoise the signal, we applied hard thresholding to the wavelet coefficients. Our threshold was established as the standard deviation of the coefficients within the window. Subsequently, the denoised signal was reconstructed using the Inverse Discrete Wavelet Transform (IDWT). The implementation of DWT, IDWT, and threshold-based denoising was accomplished using the PyWavelets Python library45.

Independent component analysis (ICA)

The essence of Independent Component Analysis (ICA) lies in finding a linear transformation that reduces the statistical dependence between the components of a signal46. When the observed signals are noisy, preprocessing the data is essential to achieve better results with ICA. Common preprocessing techniques before applying ICA include centering and whitening47. We whitened each participant’s signal using Eigenvalue Decomposition (EVD) on the covariance matrix of the centered signals. We then calculated the Independent Component (IC) of each participant with FastICA algorithm48, using the python-picard package49. We implemented ICA using the FastICA algorithm with extended-infomax, which enforces output decorrelation rather than statistical independence. While facial muscles often exhibit synergistic movements that may challenge independence assumptions, FastICA has shown robust convergence even when source independence is not guaranteed.

Creating the facial muscle atlas

We developed a modified K-means clustering algorithm50 to group ICs from all 32 participants into 17 clusters based on their heatmap spatiality. We will refer to cluster 17 as the “other” cluster, which includes the outlier ICs. The initialization phase sets the first 16 centroids using ICs from a single participant, while the “other” cluster’s centroid is initialized as a vector of ones. Until convergence, in each iteration, the algorithm iteratively assigns each participant’s channels to their nearest centroid based on Euclidean distance while updating centroids until convergence. The threshold for convergence was set to 1 × 10−5. We implemented a dynamic thresholding mechanism to determine whether a channel belongs to one of the 16 primary clusters or the “other” cluster. This threshold is calculated as the 99th percentile of the minimum distances between each channel and all centroids in each iteration of the algorithm, computed across the ICs of all participants. For each participant channels with distances from the closest centroid exceeding this threshold are assigned to the “other” cluster, ensuring that only channels with strong spatial similarity to a primary cluster centroid are included in that cluster.

Blendshapes (BS) prediction from sEMG

We recorded two additional participants using Unity’s Live Capture app simultaneously with the sEMG electrodes. These participants weren’t used to build the facial muscle atlas. They performed the 11 facial movements with three repetitions each. Live Capture, leveraging Apple’s ARKit, tracks 52 facial blendshapes (BS) (https://arkit-face-blendshapes.com/). The BS, provided by ARFaceAnchor objects, quantify facial feature movements on a scale from 0.0 (neutral) to 1.0 (maximum displacement). The sEMG signals were sampled at 500 Hz while BS were recorded at 60 Hz. We trained the model to predict the BS of the right side of the face and thus trained the model on 31 corresponding BS.

Using the data collected from one of the participants, we trained a deep neural network (DNN) regression model to reconstruct continuous BS values from preprocessed sEMG signals. The preprocessing pipeline included signal filtering, wavelet denoising, ICA, and automated component classification using the facial muscle atlas we created. The classification of the ICs of each participant to the clusters of the facial muscle atlas was done with our clustering algorithm without updating the resulting clusters of the facial muscle atlas. We then calculated the root mean square (RMS) of the BS and ICs in windows of 100 ms. Finally, to account for amplitude variations both across different muscles within the same participant and between different participants, we normalized each channel by dividing its RMS value by the RMS value of a 100 ms baseline window from the same channel during the resting face expression.

To address potential bias from the limited number of repetitions, we employed a strategic data-splitting approach: one randomly selected repetition from each facial expression’s three repetitions was allocated to the test set, while the remaining two assigned to the training set. To ensure robust model training, we applied several data processing strategies. First, we shuffled the training samples to prevent the model from learning the temporal sequence of facial expressions. Second, we standardized both the input features and target values of the training set and applied the same transformation to the test set. Additionally, we trained the model using RMS values calculated 100 ms time windows instead of using raw preprocessed signals. This was done to mitigate the risk of overfitting to specific muscle activation patterns, given the limited sample size. By focusing on the overall magnitude of muscle activation within each window rather than precise temporal dynamics, this approach aimed to creating a more generalizable representation, less sensitive to minor variations in facial movement execution.

To evaluate the generalizability of our approach across individuals and demonstrate the utility of the facial muscle atlas for cross-participant analysis, we used the DNN that we trained on the data of the first participant to predict BS values from the sEMG signal of the second participant. This cross-participant validation followed identical preprocessing steps, including signal filtering, wavelet denoising, ICA, and the normalization procedure described above.

Our DNN, was designed as a regression model that employed a fully-connected (dense) feedforward architecture with a bottleneck design comprising five hidden layers, with dimensions scaling up and down through the network (hidden dim, hidden dim × 2, hidden dim × 4, hidden dim × 2, hidden dim), each incorporating batch normalization, ReLU activation, and dropout (dropout rate was set to 0.4). Hyperparameter optimization utilized 5-fold cross-validation, exploring learning rates [0.001, 0.01, 0.1], hidden dimensions [32, 64, 128], and epochs [50, 100, 200, 300, 400, 500]. We trained the network using Smooth L1 Loss as the objective function. We used Adam optimizer with a learning rate reduction strategy that decreased the learning rate when performance metrics plateaued (implemented via PyTorch’s ReduceLROnPlateau scheduler).