Introduction

Essential tremor (ET) is the most common movement disorder, characterized by rhythmic tremors in the hands, head, or other parts of the body during voluntary movements1. It affects an estimated 60 million people worldwide and is expected to rise as the population ages2. Regular ET assessments can help monitor disease progression, detect deterioration or treatment response, and adjust management plans accordingly.

Currently, assessing ET is challenging due to the lack of specific tests or biomarkers3,4, typically relying on clinical observation during face-to-face interactions5. Clinicians evaluate tremors based on the clinical rating scale for tremor (CRST)6 through tasks such as hand-to-nose movements, drinking water, or drawing3. Spiral drawing is a commonly used clinical method for assessing ET7. The patient is provided with a paper containing a pre-drawn Archimedean spiral (with guideline templates). Two points are marked at the center and the outer edge of the spiral. The patient is instructed to connect the two points without crossing the guideline. Clinicians evaluate the severity of tremor visually based on the patient’s drawing (as shown in Table 1). This method is widely used for its simplicity and practicality in clinical settings8,9.

Table 1 Rules for tremor severity scoring by Archimedean spiral drawing task in the CRST.
Full size table

However, traditional assessment methods require patients to visit medical institutions, schedule appointments with neurologists, and undergo in-person evaluations. Many ET patients are elderly with limited mobility, making the process cumbersome and time-consuming for both patients and clinicians10. Additionally, doctor-based subjective assessments are prone to significant bias, and the data are often difficult to retain, preventing continuous monitoring of the patient’s condition. Remote intelligent assessment offers a promising solution by reducing costs and improving convenience and accessibility, enabling patients in remote areas or with mobility challenges to receive professional evaluations without geographical constraints11.

The technology for assessing ET has rapidly evolved in recent decades. Devices such as inertial sensors12, EMG13, video equipment14, and electronic handwriting boards15,16 have significantly enhanced the objectivity, quantification, and consistency of tremor detection7. The application of machine learning and deep learning algorithms in ET assessment has gained increasing attention7,17. Ali et al.18 recorded accelerometer signals from 35 subjects while they drew Archimedean spiral diagrams, distinguishing ET from healthy controls. Sole-Casals et al.19 used information from electronic handwriting boards as input to an SVM model to differentiate Parkinson’s Disease (PD) and ET. Wang et al.15 combined convolutional neural networks with electronic handwriting boards to diagnose ET.

Although previous research has highlighted the advantages of objectively assessing and diagnosing ET, these methods also have limitations. Most studies have focused on ET identification, such as distinguishing ET from healthy populations. This binary classification task primarily involves identifying differences in tremor patterns, which are often visually apparent. However, fine-grained assessment of ET tremor severity remains a significant clinical challenge. To our knowledge, no published research has utilized standard CRST-guided handwritten images for grading ET tremor severity. Furthermore, remote home monitoring and self-assessment for ET, essential for tracking disease progression, remain unexplored. To address these gaps, we present the following contributions in this paper.

Proposing a remote ET diagnosis framework

This study introduces an innovative framework for remotely assessing ET severity using handwritten images, overcoming the limitations of traditional clinical settings. By utilizing paper and pen, this approach provides a low-cost, accessible, and scalable solution for ET evaluation, particularly beneficial for elderly patients or those in resource-limited environments.

Creating a high-quality handwritten ET grading dataset

We collected about 1000 high-quality CRST Archimedean spiral handwriting images from more than 300 patients, both from medical institutions and remote home environments, forming a robust dataset with expert ratings. This dataset addresses the shortage of high-quality annotated data for ET severity grading and offers a valuable resource for further research.

Demonstrating the model’s strengths

The proposed ETSD-Net model achieved 88.44% accuracy, surpassing traditional physician assessments and baseline models such as ConvNeXt-Tiny, DenseNet, MobileNet-V2, and ResNet50. By using deep learning for image recognition, our model provides an objective, consistent, and quantifiable method for ET severity assessment, especially when clinical evaluations are constrained by time, resources, or expertise. Furthermore, our model’s use of Grad-CAM visualizations enhances interpretability, ensuring that the focus remains on relevant features of the handwriting images, thereby improving diagnostic reliability.

Related work

We conducted a search in the Web of Science database using the query "Title = essential tremor AND (Topic = drawing OR Topic = writing)", yielding 151 results as of January 9, 2025. After reviewing the titles and abstracts, we excluded 138 studies that focused on genetics, epidemiology, surgery, or pharmacological treatments. Finally, 13 relevant articles were selected for full-text review, and 9 articles focused on ET severity assessment and potentially containing handwriting datasets were summarized in our analysis, as shown in Table 2. Ali et al.20 recruited 17 ET patients and 18 healthy controls. Participants performed guided Archimedean spiral drawing tasks while wearing an Inertial Measurement Unit (IMU) on the forearm. Classification was performed using a Support Vector Machine, achieving only 68.57% accuracy in estimating the severity of ET. Ma et al.21, using a digital writing tablet and pen, collected multi-modal data from 147 ET patients. By utilizing transfer learning and an attention mechanism, their system achieved an accuracy ranging from 97.33% to 97.39% for five-category tremor severity classification. Although this accuracy is quite high, the data collection required supervision from researchers and took place in a laboratory setting, making it unsuitable for remote home assessments.

Table 2 Overview of handwriting studies that assess ET.
Full size table

After summarizing the findings, we observed that current studies are predominantly based on IMUs or electronic handwriting boards. At present, no research has focused solely on using handwritten images to assess the severity of ET, nor has a dedicated handwriting image dataset for ET patients been established. Moreover, existing studies primarily rely on data collected in laboratory settings, without addressing methods for remote assessment. The spiral diagram provided by digital handwriting systems lacks a reference template, which increases the time required for physicians to guide patients during the drawing process. Visual diagnosis by experts often involves observing the number of crossings between the handwriting and the template. Furthermore, compared to electronic handwriting boards, pen and paper offer a more natural writing environment, making the diagnosis and assessment closer to real-life conditions. This approach also avoids the costs associated with purchasing and maintaining electronic handwriting boards or IMU devices, making the assessment more affordable and convenient, with the potential for remote evaluation.

Methods

Subjects and protocol

This study aims to develop a convenient and highly accurate method for assessing the severity of ET using handwritten images. It is based on the clinical trial titled “Efficacy and Safety Study of ExAblate Transcranial MRgFUS Thalamic Disruption for Drug-Refractory Idiopathic Tremor” (trial protocol code ET002J) at PLA General Hospital. The trial was approved by the Ethics Committee of the Chinese PLA General Hospital (S2018-021–00/01). The ET002J clinical trial is part of a prospective, single-arm, multi-center clinical trial sponsored by InSightec (ClinicalTrials.gov Identifier: NCT03253991). Patients with symptoms of tremor were remotely recruited through web-based questionnaires (designed for the ET002J clinical trial), telephone interviews, and video consultations. Multiple neurologists screened patients with typical symptoms of ET. Data collection for this experiment began on September 4, 2020. As of March 5, 2024, 315 ET patients at different stages—who had completed the necessary tests and were confirmed not to have other conditions such as PD, hyperthyroidism, hepatolenticular degeneration, or drug-induced tremors—were included in the study. The participants’ ages ranged from 30 to 78 years. To provide a clearer understanding of the patient profiles, we categorized the patients into four groups based on their CRST Write scores: Group 1 (0–7 points), Group 2 (8–14 points), Group 3 (15–21 points), and Group 4 (22–28 points), as shown in Table 3.

Table 3 Patient demographics and baseline characteristics.
Full size table

Research on digital images has found that drawing Archimedean spirals or straight lines offers greater discriminative power than writing5, likely due to its requirement for continuous movements across multiple planar directions, unlike the predominantly vertical motions associated with writing27. Therefore, we chose to have patients use paper and pen to complete the CRST scale drawing A (large Archimedean spiral) as the method for data collection. Patients were randomly divided into two groups: one undergoing evaluation in medical institutions and the other undergoing remote home-based evaluation, at an 8:2 ratio. After a 24-h medication withdrawal, the medical institutions group consisted of 252 patients who completed the task under the guidance of neurologists. The handwritten images were then scanned using the scanning function of the HP LaserJet Pro MFP M226dw printer (Hewlett-Packard, USA) and saved as JPEG format images with a preset resolution of 300 dpi. The 63 patients in the remote home-based group were instructed to print a PDF template of the drawing task on A4 paper at home, complete the drawing as instructed, and then take photos using smartphones or cameras at the highest resolution and upload the images in full resolution JPEG format.

This resulted in a total of 798 high-definition scanned images and 199 photos of patient-produced drawings from the remote group. An expert panel of neurologists assessed the tremor severity of the patients using the CRST scale based on the handwritten images. Each image was independently scored by three neurologists. In cases of scoring discrepancies, the neurologists discussed and re-evaluated the images to reach a consensus score. This process resulted in a high-quality dataset of 997 images. These consensus scores were then used as labels for training the model. The experimental process is illustrated in Fig. 1.

Fig. 1
figure 1

Schematic diagram of the proposed system for a deep learning approach to remotely assessing ET with handwritten images. This mainly includes: (1) Patients completing the CRST handwriting tasks at home or in medical institutions and collecting handwritten images; (2) Experts scoring and processing images to form a dataset; and (3) modeling and evaluation.

Full size image

Dataset preparation

In this study, we utilized transfer learning techniques for the classification of handwritten spiral images. Transfer learning is a deep learning method where a pre-trained model, originally developed for a specific task, is repurposed as the starting point for a model on a new task28,29,30,31. The pre-trained model leverages the prior knowledge gained by the model on the extensive ImageNet dataset to enhance performance on a specialized, private dataset.

Preprocessing is a critical initial step that ensures the input data is suitable and of high quality for training deep learning models. To make our handwritten images compatible with pre-trained deep learning models, we applied several preprocessing steps, including resizing images, normalizing pixel values, and data augmentation28,29,30,31. Many deep learning models, particularly convolutional neural networks (CNNs) like VGG16 and ResNet, are trained with fixed-size input images, typically 224 \(\times\) 224 pixels29. To ensure compatibility with these pretrained models, we resize our images to 224 \(\times\) 224 pixels. This size strikes a balance between computational efficiency and image detail. Larger images (e.g., 512 \(\times\) 512) increase computational costs and training time, while smaller sizes (e.g., 64 \(\times\) 64) may lose detail, reducing performance. Thus, 224 \(\times\) 224 pixels provides an optimal compromise32.

This resizing ensures that the model can effectively process the images without distortion or loss of critical information. The images were normalized using the mean and standard deviation values derived from the ImageNet dataset ([0.485, 0.456, 0.406] for the means and [0.229, 0.224, 0.225] for the standard deviations, respectively). Normalization and resizing as part of preprocessing ensure that the model focuses on learning relevant features from the images, rather than being influenced by variations in color, brightness, or size. This process helps in stabilizing the training process and improves model convergence.

Beyond preprocessing, data augmentation plays a pivotal role in enhancing the model’s ability to generalize from the training data to unseen data. It artificially expands the training dataset by applying a series of random transformations that produce plausible variations of the input images. Our data groups were naturally imbalanced, with a higher number of patients with mild tremors, leading to an unequal distribution of samples across each group. To address this imbalance, we applied data augmentation techniques before training the model to balance the groups and ensure a more even distribution of samples. Specifically, we used rotation (± 25 degrees), horizontal flipping, vertical flipping, scaling (0.8 to 1.2), additive Gaussian noise (scale 0 to 0.05*255), Gaussian blur (sigma 0 to 3.0), linear contrast adjustment (0.75 to 1.5), brightness multiplication (0.8 to 1.2), and random cropping (up to 10%). By systematically applying these preprocessing and augmentation techniques, we expanded each data group to 800 samples, ensuring that our dataset was robust and capable of training an effective deep learning model for the classification of handwritten spiral images.

Model evaluation

To comprehensively evaluate the applicability of using hand-drawn spiral lines to assess the severity of ET, we incorporated four high-performance deep learning networks as baseline models.

  1. (1)

    ResNet5029: The residual structure alleviates the gradient vanishing problem, making it suitable for extracting complex features. Its mature and stable architecture is a widely used benchmark model in visual tasks.

  2. (2)

    DenseNet30: The dense connectivity mechanism enables efficient feature reuse, achieving good performance with fewer parameters. Its characteristics make it particularly suitable for tasks involving small datasets or requiring deep feature fusion.

  3. (3)

    ConvNeXt-Tiny31: It widely adopts the design principles of modern lightweight convolutional networks, enabling it to capture multi-scale local and global features while balancing performance and efficiency.

  4. (4)

    MobileNet-V228: By utilizing depthwise separable convolutions and an inverted residual structure, it achieves a balance between computational efficiency and predictive performance. Its lightweight design is highly suitable for deployment on mobile or portable devices.

Considering that the remote diagnosis of ET severity relies on the accuracy and efficiency of the model, we propose ETSD-Net, an improved model based on MobileNetV2. The input size of the ETSD-Net is 224 × 224 pixels, which is consistent with the preprocessing step where all images were resized to this dimension. It comprises 2 convolution blocks, N inverted residual blocks and 1 fully connected layers, with specific batch normalization layer and activation functions. Specifically, The input of ETSD-Net is first passthrough a convolution block which contains convolution layer, batch normalization layer, and relu layer to extract shallow features. The the shallow features are sent to several stacked inverted residual blocks to capture spatial details and semantic information in hand-drawn spiral line images, which is benefited from the introduce of a channel-spatio attention mechanism in the inverted residual modules of the network. Then the semantic features are refined by another convolution blocks with average pooling layers. Finally, the classes related features are flattened and sent to a fully connected layers to get the outputs.

Experimental setups

To prevent data leakage and ensure an objective evaluation of both the baseline model and ETSD-Net proposed in this paper, we employed a subject-independent data split strategy, ensuring that the data from the same subject only appears in one of the training, validation, or test sets. Based on this, we divided the data collected in Section. Dataset Preparation in a 6:2:2 ratio.

To reduce the cost of training the model and accelerate the convergence process, both the baseline model and ETSD-Net are trained using transfer learning in this paper. The reason we adopted this approach is because the initial layers in transfer learning models capture generalizable features, while the latter layers are more task-specific.

So, the models underwent a two-phase training process: the first phase involved adding a new classifier, and the second phase focused on fine-tuning the model. In the first phase, for ResNet50, DenseNet, ConvNeXt-Tiny and MobileNet-V2, a fully connected layer corresponding to the five severity levels defined by the CRST was added to the end of each of the four pre-trained models, with the weights of the other layers frozen. For ETSD-Net, since its underlying architecture is MobileNet-V2, we used the pre-trained weights of MobileNet-V2 when loading the model. We froze the weights of the modules that could be matched, and the fine-tuning mainly focused on the spatiotemporal attention module and the final classification layer. In this way, the initial layers of a neural network capture universal features like edges and textures. By keeping these layers unchanged, the model can utilize these learned features without the need for retraining.

During the fine-tuning process, we use a batch size of 64 for iterative training and employed a learning rate schedule with hierarchical decay. The base learning rate was set to 10–3. For the feature extraction layers, the learning rate is decayed by a factor of 0.5 every three layers, while the classification layer is set to the base learning rate without decay. This is because the feature layers contain pre-trained weights and do not require a large learning rate to find optimal weights, whereas the classification layer has not loaded pre-trained weights. The Adam optimizer is used for parameter optimization, and multi-class cross-entropy is used as the loss function for backpropagation. The entire fine-tuning process lasted for 10 epochs. Here, we use a small number of epochs because the model is initialized with pretrained weights from ImageNet. Experimental results show that this initialization method enabled the model to converge within 10 epochs. Therefore, we do not use a larger number of training epochs to avoid overfitting on the small-scale dataset. The model is trained on the training set, and the model with the highest accuracy on the validation set is saved as the best model. The performance metrics used to evaluate the model in this paper are accuracy, F1-score, precision, and recall.

The process is exemplified by ETSD-Net and illustrated in Fig. 2.

Fig. 2
figure 2

Integration of ETSD-Net in severity prediction.

Full size image

Results

The performance of four state-of-the-art transfer learning models—ResNet50, DenseNet, MobileNet-V2, and ConvNeXt-Tiny—against our proposed model using four evaluation metrics: Accuracy, Precision, Recall, and F1-score were compared. The results are summarized in Table 4.

Table 4 Performance comparison of baseline models and ETSD-Net.
Full size table

Our ETSD-Net outperforms the baseline models in several key metrics. Specifically, it achieves the highest Accuracy (88.44%), Recall (88.44%), and F1-score (88.45%), demonstrating its superior ability to correctly classify and balance precision with recall. While ConvNeXt-Tiny shows slightly higher Precision (88.87%), our model achieves a comparable result (88.64%) while maintaining better overall performance across the other metrics.

To evaluate the impact of data augmentation on the performance of our model, we compared the results of ETSD-Net trained on the original dataset and the augmented dataset. The model trained on the augmented dataset achieved an accuracy of 88.44%, compared to 86.43% on the original dataset. The precision, recall and F1-score also showed significant improvements, with the precision increasing from 86.39% to 88.64% and the F1-score from 86.18 to 88.45%.

The performance of our model is demonstrated through the ROC curve and confusion matrix. The ROC curves for each severity level show high areas under the curve (AUC) values, ranging from 0.98 to 0.99, indicating excellent classification capability across all categories. The confusion matrix further highlights the model’s robust performance, with most predictions falling along the diagonal, signifying correct classifications (as shown in Fig. 3). These results indicate that our proposed model successfully leverages transfer learning techniques and outperforms existing architectures in multiple evaluation criteria, making it more robust and reliable for the task at hand.

Fig. 3
figure 3

Performance evaluation of baseline models and ETSD-Net: confusion matrices, ROC curves, and t-SNE visualization for ET severity prediction.

Full size image

ETSD-Net, built upon the MobileNet-V2 architecture, maintains a lightweight design with comparable computational efficiency to MobileNet-V2. As shown in Table 5, both models exhibit similar FLOPs (0.33 GFLOPs for ETSD-Net vs. 0.30 GFLOPs for MobileNet-V2) and parameter sizes (2.26M for ETSD-Net vs. 2.20M for MobileNet-V2). This similarity reflects the architectural choices aimed at retaining MobileNet-V2’s computational efficiency while incorporating improvements to enhance performance.

Table 5 Computational efficiency and inference time of baseline models and ETSD-Net.
Full size table

However, our model’s inference time (20.17 ± 10.56 ms) is notably higher than that of MobileNet-V2 (7.21 ± 0.93 ms). This increase in latency can be attributed to the additional modifications designed to improve feature extraction and overall performance. In contrast, other baseline models, such as ResNet50 and ConvNeXt-Tiny, achieve faster inference times (e.g., 5.34 ± 1.28 ms for ConvNeXt-Tiny), but at the cost of significantly higher computational demands (4.45 GFLOPs and 27.80M parameters for ConvNeXt-Tiny).

Figure 4 and Fig. 5 illustrate the Grad-CAM33 and saliency map34 visualizations, respectively, for the different models across the CRST input images (0–4). These visualizations collectively reveal how each model allocates attention to specific regions of the input images during classification and provide complementary insights into their focus mechanisms. The Grad-CAM results highlight the broader attention distribution, while the saliency maps emphasize sensitivity to task-relevant regions at a more granular level.

Fig. 4
figure 4

Grad-CAM heatmap visualizations for baseline models and ETSD-Net.

Full size image
Fig. 5
figure 5

Saliency map visualizations for baseline models and ETSD-Net.

Full size image

ResNet50 and DenseNet show limited and localized attention distributions in both Grad-CAM and saliency maps, primarily focusing on isolated segments of the spiral. This incomplete coverage suggests that these models struggle to capture the global geometric pattern of the spiral, which is critical for accurate classification. ConvNeXt-Tiny exhibits attention that is heavily concentrated on disconnected, small regions, as seen in both visualizations, indicating a tendency to overfit to local details rather than understanding the spiral’s overall structure. MobileNet-V2 demonstrates a more balanced attention pattern compared to the earlier models, with Grad-CAM and saliency maps showing broader coverage of the spiral. However, its focus is still insufficiently global, leaving portions of the structure underrepresented.

In contrast, ETSD-Net achieves the most comprehensive and consistent attention distribution, as evidenced by both Grad-CAM and saliency maps. The model effectively learns the full geometric structure of the spiral, with attention covering both central and peripheral regions. This global focus ensures robust feature extraction and better generalization across varying input complexities. The combined results from Grad-CAM and saliency maps strongly support ETSD-Net’s superior performance, as its ability to capture both local and global features is unmatched by the baseline models.

Discussion

Clinical value of this framework

This study innovatively uses handwritten images to enable remote assessment of ET, breaking the current limitation where ET evaluation is confined to clinical settings. We collected and processed 997 high-quality handwriting images from medical institutions and remote home environments, creating a high-quality dataset through rigorous evaluation by a panel of neurological experts. Using a transfer learning approach, we proposed the ETSD-Net model for ET severity assessment, which achieved the best performance with an accuracy of 88.44%, surpassing the baseline models ConvNeXt-Tiny, DenseNet, MobileNet-V2, and ResNet50.

Studies show that scoring tremor severity with scales like the CRST requires trained raters to achieve reliable results35. Grimaldi et al.36 reported an intraclass correlation coefficient of 85.9% for CRST total scores, with variability across subcomponents (e.g., 88.2% for tremor amplitude and 67.1% for daily activity). Similarly, Elble et al.37 reported validity testing with spiral ratings (80.4%) and handwriting ratings (76.2%) in ET patients. In comparison, our model provides a consistent, objective, and quantifiable method for tremor severity assessment, which is especially useful when clinical evaluations are limited by time, resources, or expertise. Standardizing assessments through algorithms can complement clinical evaluations and improve diagnostic reliability, particularly in remote or resource-limited settings.

Given that current ET assessment research is primarily based on electronic handwriting boards and IMU devices, our study demonstrates the feasibility of using only paper and pen for remote ET evaluation. This approach is low-cost, preserves the natural writing habits of patients, and is particularly beneficial for elderly ET patients. We believe our study is clinically meaningful and represents a significant step toward integrating remote, automated methods into routine ET assessment.

Model performance and interpretability

In the previous Sect “Related work”, we conducted a detailed review of research on handwriting-based ET severity assessment. We found that current studies using handwriting to assess ET severity are primarily based on IMU or electronic tablets. The lowest accuracy reported was 68.57% by Ali et al.20, who used an IMU-based approach, while the highest accuracy was 97.39% from Ma et al.21 from our research group, which used an electronic tablet. McGurrin et al.24, also using an IMU, achieved an accuracy of 93.00%. However, it is important to note that these studies using IMU or electronic tablets involve more complex paradigms, with accelerometer, electronic trajectory, or pressure signals that differ significantly from our handwriting image-based approach. In contrast, our model achieved 88.44% accuracy based solely on handwriting images, far surpassing the clinical accuracy of 76.20%-80.40% reported in handwriting-based studies.

We use a deep learning model for image recognition that processes the input handwriting images directly and extracts relevant features through convolutional layers, without requiring additional input characteristics. The model is designed to autonomously learn important patterns and representations from the input images. However, to enhance interpretability and understand what the model focuses on during its decision-making process, we incorporated Grad-CAM visualizations. These visualizations highlight the specific regions of the handwriting images that the model attends to, providing insights into how the model interprets the handwriting. This approach ensures that the focus remains on the image data. We also used t-SNE for visualization, which shows that the model effectively classifies tremor severity based on the handwriting images.

Limitations and future directions

However, we acknowledge several limitations in this study. Our research is based on handwriting images, and the quality of the images affects the performance of our model. Ink flow issues during writing could have influenced the results. Most patients carefully completed the drawings under the supervision of neurologists or at home. However, some patients, due to urgency for medical attention or other distractions, did not complete the drawings as carefully. This may have led to less accurate drawings compared to those done more attentively. Using a deep learning model to analyze 2D static images, while practical, also limits the study of clinical physiology and the interpretability of the model. In the future, we plan to expand the collection of handwritten images to include those from PD patients, which will help facilitate the early differential diagnosis between PD and ET.

Conclusion

This study presents an effective approach to remotely assessing the severity of ET using handwriting images, offering a practical and accessible method for evaluation. We collected about 1000 high-quality CRST Archimedean spiral handwriting images from more than 300 patients, establishing a robust dataset with expert ratings. Using a transfer learning approach, we developed the ETSD-Net model for ET severity assessment. Our model not only achieves objective and accurate evaluations but also enables remote, low-cost assessments. With an accuracy of 88.44%, ETSD-Net outperforms existing methods and shows great potential for integrating into clinical practice, especially in remote or resource-limited settings. This study represents a meaningful contribution to improving the accessibility and reliability of ET assessment, particularly for elderly patients who may benefit most from home-based evaluations.