Introduction

The motor tasks included in the Movement Disorder Society-Unified Parkinson’s Disease Rating Scale (MDS-UPDRS)1 Part 3 constitute a major portion of the clinical examination during regular follow-ups and are instrumental in planning treatment schedules. Among various motor tasks, those measuring bradykinesia severity in the upper and lower extremities, such as finger taps and leg agility, are commonly used in clinical settings for a rapid and sensitive check for Parkinsonism. Importantly, the finger tapping (FT) task has been shown to be effective in distinguishing patients with Parkinson’s disease (PD) from healthy controls2, in the differential diagnosis of atypical parkinsonism3, and in assessing dopaminergic medication effects4,5,6. However, the semi-quantitative and subjective nature of the scaling system prevents clinicians from accurately assessing patients’ current status and detecting changes.

Recent advances in machine learning (ML) and deep learning (DL) have led to innovative methods for quantifying motor severity in patients with PD. These methods encompass a wide range of technologies, including smartphone applications that measure FT, gait, voice, balance, and reaction time7, as well as touchscreens that assess FT tasks to evaluate treatment responses6 and monitor motor fluctuations4. Wearable devices, such as accelerometers and sensors, have been developed for detailed assessments of FT8, gait monitoring9, and detection of freezing of gait episodes10. More recently, vision-based motion estimation models have gained attention, with studies exploring their use in analyzing retrospectively collected videos of patients with PD filmed against noisy backgrounds11,12 and even in videos recorded at home using webcams or any video recording device13,14. Vision-based models are increasingly utilized, as they can be applied to previously filmed routine video recordings obtained in common clinical settings, without any need for specialized devices or environmental setups.

Researchers have also applied those automatic methods for quantifying motor symptom severity in patients with advanced PD who have undergone deep brain stimulation (DBS). More specifically, numerous studies have examined and compared the effects of DBS and medication in ameliorating motor symptoms among patients with PD15,16,17,18,19,20,21,22,23. However, most of these studies have been constrained by small sample sizes, often attributed to the challenges of conducting prospective research utilizing wearable sensors or devices15,16,17,18,19,21,23. Moreover, these researchers collected data in the post-operative setting, lacking baseline assessments of patients prior to any surgical intervention. Most importantly, all patients in these studies underwent DBS targeting the subthalamic nucleus (STN) rather than the globus pallidus internus (GPi), limiting our understanding of the potential benefits associated with GPi stimulation.

This study comprises two parts. In the first part, we utilized a state-of-the-art DL model for 3D hand pose reconstruction to estimate FT motion. From the reconstructed hand keypoints, we extracted 21 motion parameters from FT videos to predict the MDS-UPDRS Part 3 FT score, ensuring that these parameters accurately quantify the severity of bradykinesia. In the second part, we compared motion parameters from FT videos recorded in four different states: preoperatively with and without medication, and postoperatively before and after DBS programming, to evaluate the distinct treatment effects of medication and GPi DBS.

Methods

Patients

Patients diagnosed with PD according to the United Kingdom PD Society Brain Bank criteria24 and who underwent GPi DBS for their advanced stage of PD in the Departments of Neurology and Neurosurgery at the Asan Medical Center (AMC) between 2014 and 2022, were included in the study. These patients experienced motor fluctuations or levodopa-induced dyskinesias despite following carefully planned dopaminergic medication schedules. Patients with video recordings of the FT task during the preoperative levodopa challenge test and postoperative initial DBS programming were included.

At our center, GPi DBS was generally preferred over STN DBS, except when the primary goal was to reduce dopaminergic medication or when patients had significant medication-related psychiatric symptoms. Although both GPi DBS and STN DBS have been reported to improve motor symptoms to similar degrees, with distinct advantages depending on patients’ disease characteristics25,26,27,28,29, neurosurgeons at our center more often selected GPi as the surgical target due to the higher risk of intracerebral hemorrhage associated with permanent neurological deficits in STN DBS, despite similar incidence rates of intracerebral hemorrhage between the two targets30. Consequently, patients with STN DBS were excluded from the analyses due to the smaller sample size compared to those receiving GPi DBS, in order to avoid data imbalance and ensure reliable statistical analysis. The study adhered to the principles outlined in the Declaration of Helsinki and was approved by the institutional review board of the AMC (IRB 2023 − 0506, April 25, 2023). All patients provided written informed consent.

Disease duration calculated from symptom onset, levodopa equivalent daily dose (LEDD)31, and cognitive status assessed using the Korean Mini-Mental State Examination (K-MMSE) and Clinical Dementia Rating (CDR), were collected prior to surgery. As part of the routine preoperative evaluation, patients were video recorded while performing tasks from the standard UPDRS Part 332, once in the “off state,” at least 12 h after the last dose of anti-parkinsonian medication (“med off”), and a second time in the “on state,” following administration of 1.5 times the usual first morning levodopa equivalent dose (“med on”). DBS surgery was performed in accordance with institutional guidelines. Briefly, GPi DBS leads were stereotactically implanted under local anesthesia, guided by microelectrode recording. The optimal target coordinates for the GPi were individually determined using Framelink/StealthCranial (Medtronic, Dublin, Ireland) and SurgiPlan (Elekta, Stockholm, Sweden), based on direct target delineation from T2-weighted short-tau inversion recovery images. Final electrode positioning was confirmed using intraoperative cone-beam O-arm computed tomography.

During the initial postoperative DBS programming session, typically scheduled two to three weeks after surgery, patients were instructed to attend without having taken their medication overnight. Videos of patients performing the same tasks were recorded again, both before the initial programming (“DBS off”) and approximately 30 min after DBS optimization (“DBS on”). This optimization was conducted by an experienced movement disorder specialist, following the expert consensus established in a previous study33. Only patients with videos recorded in all four states — “med off,” “med on,” “DBS off,” and “DBS on”— were included (Fig. 1a).

Fig. 1
figure 1

Study design. (a) Timeline of finger tapping video recording time points presented in chronological order. (b) Process of motion parameter extraction from finger tapping videos: (left) 3D hand pose reconstruction and prediction of 21 hand keypoints using Mesh Graphormer; (right) time series signal of finger tapping with detected peaks and troughs marked as red dots; (far-right) variations in distance, speed, and acceleration throughout a cycle of finger closing, opening, and then closing, scaled to facilitate understanding. (c) Prediction of MDS-UPDRS Part 3 finger tapping score using machine learning models. (d) Statistical analyses comparing motion parameters across four states.

Full size image

Data selection

From the tasks in the MDS-UPDRS Part 3, item 3.4, FT, was selected for this study. The videos were reviewed, and parts showing the FT task were clipped. Videos were excluded if they exhibited incorrect FT motion, severe upper extremity dyskinesia, the hand being outside the frame, blurriness, sudden zooming in or out, or if they were both shorter than 5 s in duration and had fewer than 10 detected finger taps. Videos were recorded with camcorders (HDR-CX360, FDR-AX700, HDR-PJ790, and HDR-CX560, Sony Electronics Inc., New York, NY) at frame rates of 30 or 60 frames per second and with resolutions of at least 480\(\:\times\:\)640.

Preprocessing and motion analysis

Videos of the hand contralateral to the implantation site were collected for each patient in four distinct states, forming a complete set. In cases of bilateral DBS, this resulted in two separate sets of videos, one for each hand. A smaller subset of patients with unilateral DBS provided only one set of videos.

The DL model Mesh Graphormer (https://github.com/microsoft/MeshGraphormer)34, pre-trained on the multi-view, large-scale FreiHAND Dataset35 and achieving top performance in 3D hand reconstruction, was used out-of-the-box for hand pose reconstruction without model tuning. As the model was trained on right hands centered in 224 × 224 pixel frames, our videos were first cropped to center the hand and horizontally flipped for left hands.

To automate the cropping process, another hand pose estimation model, OpenPose version 1.7.0 (https://github.com/CMU-Perceptual-Computing-Lab/openpose)36 identified the 21 keypoints of each hand (Fig. 1b). Keypoint 9, the most proximal point of the middle finger, was used as the reference center for cropping each video frame to a size of 224 × 224 pixels. Subsequently, frames with the left hand were horizontally flipped. Next, the cropped frames were processed using Mesh Graphormer to obtain a 3D reconstructed hand pose and mesh for each frame. In case of frames with poor reconstruction quality, they were manually recropped and resampled to 224 × 224 pixels if needed, then reprocessed. We recognize that manual quality control could introduce bias into the experimental results; however, we retained this selection process to avoid the potential weakening of outcome implications due to poor model predictions. One NVIDIA Tesla P40 GPU was used for employing OpenPose and Mesh Graphormer, requiring 3 and 5 h for completion, respectively.

Mesh Graphormer predicted 21 keypoints, adhering to the same keypoint numbering system of the hand as utilized in OpenPose, but in 3D coordinates. Of these, keypoints 4 and 8, corresponding respectively to the most distal ends of the thumb and index finger, were utilized to calculate the 3D Euclidean distance between the two fingertips. Since the model outputs scale-normalized keypoint coordinates by default37, no additional normalization was needed. The calculated distances were multiplied by 100 to compensate for their small values, producing a time series signal with periodic patterns representing finger opening and closing motions (Fig. 1b, Supplementary Video S1).

Feature extraction

A custom peak and trough detector from previous works11,13 was used to identify frames where the fingers reached their maximum and minimum separation. To accommodate the varying lengths of the videos, 543 videos out of 556 were limited to the first 10 taps, and 13 videos with fewer than 10 taps were included.

To capture the diverse aspects of hand bradykinesia represented in the MDS-UPDRS Part 3 FT rating criteria (i.e., amplitude, speed, rhythm, interruptions, and amplitude decrement), we selected 21 corresponding motion parameters. These parameters were divided into two groups: those derived from the distance measured in each frame (speed, acceleration, and aperiodicity) and others extracted from the peaks (amplitude, period, fatigue, and freeze). Speed and acceleration were calculated using standard physical formulas. Amplitude refers to the peak magnitude of the distance, measured in arbitrary units (a.u.), while the period refers to the interval between successive peaks, measured in seconds (s). The minimum, maximum, median, and IQR values were calculated for speed, acceleration, amplitude, and period. To assess the arrhythmicity of FT, we calculated aperiodicity and entropy for both amplitude and period, employing the methodology outlined in prior research13.

The fatigue parameter was defined using the gradient of an interval characterized by a gradually decreasing amplitude and the freeze parameter was calculated by summing the time duration of all intervals including the predefined freeze definition. The hyperparameters for detecting freeze and fatigue were optimized through repetitive comparisons between the detected and actual intervals, as evaluated by a neurologist. For detailed description of the motion parameters, please refer to Supplementary Methods.

Gold standard rating

The FT videos were reviewed and rated by two movement disorder specialists (JL and SL) and one neurologist (GYL) according to the scoring guideline of the MDS-UPDRS Part 3 item 3.4. The gold standard ratings were established based on the majority rule. Specifically, when two or more raters agreed on a rating, it was designated as the gold standard. In the rare instances where all three raters disagreed, the median score was selected.

ML-based automatic rating

A ML model was trained with all 21 parameters to predict the gold standard ratings (Fig. 1c). Given that the gold standard ratings exhibited significant class imbalance, we conducted experiments in two scenarios: one where the model predicted all five classes as they were, and another where scores of 0 were grouped with 1, and scores of 4 were grouped with 3, thus predicting three classes (0/1, 2, and 3/4).

The videos were initially divided randomly per patient into a training set and a test set, maintaining an approximate 8:2 ratio. To ensure reproducibility, a random seed of 42 was used. However, this initial split resulted in the test set lacking patients with scores of 0 or 4. To address this issue, one patient was moved from the training set to the test set to include these scores. RandomizedSearchCV (Scikit-learn version 1.3.0) with 5-fold cross validation was employed on the training set to optimize the hyperparameters.

We trained and compared five regression models and four classification models, rounding off the predictions of regression models to the nearest integers. For evaluation metrics, accuracy, Spearman’s rank correlation coefficient (Spearman’s \(\:\rho\:\)), Kendall rank correlation coefficient (Kendall’s \(\:\tau\:\)), and mean absolute error (MAE) were used. Prior to model training, Spearman’s rank correlation coefficient (Spearman’s r) was calculated to assess the relationship strength between the gold standard ratings and each parameter. The intraclass correlation coefficient (ICC) (2,1) was evaluated between the predictions from the highest accuracy model and the gold standard ratings in the test set, as well as among the three human raters. Python version 3.8.5, Scikit-learn version 1.3.0, and SciPy version 1.10.1 were used for ML and evaluation metrics.

Statistical analysis

A total of 21 parameters are presented as mean (standard deviation, SD) or median (interquartile range, IQR). The differences in each parameter across four states were examined using a one-way repeated measures analysis of variance (ANOVA) for parametric variables and a Friedman test for nonparametric variables (Fig. 1d). For ANOVA, Mauchly’s test was employed to assess the assumptions of sphericity; based on its results, the degrees of freedom were adjusted using Greenhouse–Geisser estimates to correct for any violations. Additionally, a paired t-test for parametric and a Wilcoxon test for nonparametric variables were conducted for pairwise comparisons between states. A Bonferroni correction was applied by dividing the conventional alpha threshold of 0.05 by 21 (the number of parameters) for the ANOVA and Friedman tests, and by 5 (the number of pairwise comparisons between two states) for the paired t-tests and Wilcoxon tests. The paired t-tests and Wilcoxon tests were performed only for parameters that exhibited significant p values in the ANOVA and Friedman tests.

We also conducted two subgroup analyses. First, since more than half of the patients exhibited levodopa-induced dyskinesia (LID) in the “med on” state, defined as dose-dependent hyperkinetic involuntary movements caused by dopaminergic medication, a subgroup analysis was performed by categorizing video sets into those from patients with LID affecting any body part, including the trunk and neck, and those without. For parameters with significant p values in the ANOVA and Friedman tests using the full dataset, paired t-tests or Wilcoxon tests were used to compare the “med off” and “med on” states, applying an adjusted alpha threshold of 0.05 divided by 5 to maintain consistency with the full dataset analyses.

The second subgroup analysis was employed to assess whether motion parameter differences across four states remain consistent after controlling for the effects of LEDD and DBS amplitude. Videos from patients with pulse widths other than 60 \(\:\mu\:s\) or frequencies other than 130 \(\:Hz\) were excluded due to their small sample size. For nonparametric motion parameters, we applied a rank transformation before analysis. A linear mixed-effects model was then applied to both parametric and nonparametric motion parameters, with state, LEDD, and DBS amplitude as fixed effects, each subject’s hand as a random intercept, and motion parameters as the outcome variable, using restricted maximum likelihood estimation. The significance of state, a fixed effect, was assessed using ANOVA, followed by post hoc pairwise comparisons using estimated marginal means. The same Bonferroni correction was applied as before, with a threshold of 0.05/21 for ANOVA and 0.05/5 for pairwise comparisons.

The means (SD) and medians (IQR) of changes in parameter values resulting from medication (“med on” minus “med off”) and DBS programming (“DBS on” minus “DBS off”) were also calculated. The effects of medication and DBS programming were compared using a paired t-test for parametric variables and a Wilcoxon test for nonparametric variables, with Bonferroni correction applied by dividing the alpha threshold of 0.05 by 21 (the number of parameters). The same subgroup analysis controlling for covariates was applied in this setting, but instead of comparing the four states directly, it was conducted on the changes in parameter values (“med on” minus “med off”) and (“DBS on” minus “DBS off”).

All tests were two-sided. All statistical analyses were performed with R Studio (2023.06.1) and R (version 4.3.1, R Foundation for Statistical Computing, Vienna, Austria).

Use of large language models

ChatGPT 4 (https://chat.openai.com/) developed by OpenAI (https://openai.com/), was employed solely for grammatical revisions and to enhance the clarity of existing texts. No new information or content was generated by ChatGPT.

Results

Patient characteristics

Video recordings were initially collected from 128 patients with PD who underwent DBS during the study period. After excluding patients with DBS implantation in the STN and removing videos of inadequate quality, 148 sets of videos from 89 patients were retained. Subsequently, nine sets of videos were further removed due to inaccurate hand pose estimations from Mesh Graphormer. These estimations did not meet our visual quality control standards because instances of finger overlap obscured hand motion. This overlap occurred either between fingers, hindering clear visibility, or between fingers and the patient’s face, making it difficult to distinguish the hand from the background. Thus, for this study, a total of 556 videos, organized into 139 sets, representing 87 patients, were used (Fig. 2). The clinical characteristics of the patients are provided in Table 1.

Fig. 2
figure 2

Flowchart for patient selection.

Full size image
Table 1 Clinical characteristics.
Full size table

Performance of motion analysis

The gold standard rating of the FT task by three neurologists was as follows: 5 clips rated as 0, 152 as 1, 260 as 2, 131 as 3, and 8 as 4. The training set comprised 428 videos from 68 patients and the test set included 128 videos from 19 patients (Supplementary Fig. S1 and Table S1).

In the original 5-category rating, linear regression (LR) showed the best results, while for the modified 3-category rating, support vector regression demonstrated the highest performance in all evaluation metrics (Fig. 3; Table 2). Among the 21 parameters, the median and maximum values of amplitude and the IQR of speed ranked first, second, and third, respectively, in the Spearman’s r values. Notably, 15 of the 21 parameters exhibited p values less than 0.005 (Supplementary Table S2).

Fig. 3
figure 3

Confusion matrix of automatic MDS-UPDRS Part 3 ratings of finger tapping. (a) Linear Regression on the original 5-category rating. (b) Support Vector Regression on the modified 3-category rating.

Full size image
Table 2 Comparison of model performances.
Full size table

In the evaluation of 556 videos, full expert agreement occurred in 34.9% of cases, while two experts agreed in 58.6%. Disagreement across all raters was observed in only 6.5%. For the test set of 128 videos, the ICC for the predictions by LR compared to the gold standard rating was 0.72 (95% confidence interval, CI = 0.63–0.8), whereas the ICC among the three raters was 0.60 (95% CI = 0.5–0.69).

Box plots for 21 parameters, grouped by gold standard ratings and four states, are shown in Supplementary Figs. S2–5.

Comparison of FT severity across four states

The MDS-UPDRS Part 3 FT scores were generally lower in the “med on” and “DBS on” states compared to the “med off” and “DBS off” states, respectively (Table 3). Medication improved the FT score by − 0.52 ± 0.78 (mean ± SD) (“med on” – “med off”), while DBS programming resulted in an improvement of -0.67 ± 0.83 (“DBS on” – “DBS off”).

Table 3 Number of videos (percentages) categorized by the MDS-UPDRS part 3 finger tapping scores across the four states.
Full size table

The mean (SD) or median (IQR) values along with the p values from statistical tests of the 21 parameters are shown in Table 4. All parameters, except fatigue and freeze, were calculated using 139 sets of videos. A set was classified as exhibiting fatigue if at least one out of its four clips showed signs of fatigue; similarly, a set was classified as exhibiting freeze if any clip indicated freeze. Fatigue was identified in 90 sets, and freeze in 69 sets.

Table 4 Comparison of finger-tapping severity across four states: medication (on, off) and DBS (on, off).
Full size table

Significant differences were noted across the four states for most parameters. However, the IQR of amplitude, minimum speed, median acceleration, fatigue, and freeze parameters showed no significant difference. Following medication intake (“med off” vs. “med on”), parameters indicating motion arrhythmicity, such as entropy of amplitude, entropy of period, and aperiodicity, increased; in contrast, DBS programming (“DBS off” vs. “DBS on”) yielded no significant changes in these parameters. However, subgroup analyses of patients without LID revealed no significant increase in the entropy of amplitude or aperiodicity with medication, suggesting that the observed increases in these parameters were likely confounded by the effects of LID induced by medication (Supplementary Tables S3 and S4). In the “DBS off” state, where the implanted leads were not yet stimulated, significant improvements were observed in speed and acceleration parameters compared to the “med off” state, as well as in period-related parameters. Notably, minimum speed and median acceleration values were typically zero in all four states, reflecting the periodic opening and closing motion of FT.

Upon excluding parameters that did not show significant variances across states, most parameters displayed improvements after treatments compared to the “med off” baseline state (“med off” vs. “med on” and “med off” vs. “DBS on”). Importantly, most speed and acceleration parameters were more favorable following DBS than medication (“med on” vs. “DBS on”).

Comparison of FT severity across four states with covariate control

A total of 122 video sets were included in the subgroup analysis to control for covariates (LED and DBS amplitude), with fatigue observed in 78 sets and freeze in 56 sets. P values from statistical tests and estimated marginal means are presented in Table 5 and Supplementary Table S5, respectively. The general pattern of statistical significance was consistent with the results from tests conducted without covariate control on the full dataset. Exceptions included aperiodicity and minimum period, which showed no significant differences across four states despite being significant without covariate control. Additionally, one speed parameter, maximum speed, became nonsignificant in the pairwise comparison of the “med on” and “DBS on” states, though its p value remained close to the significance threshold.

Table 5 Covariate-adjusted comparison of finger-tapping severity across four states: medication (on, off) and DBS (on, off).
Full size table

Comparison of treatment effects of medication and DBS

Table 6 describes the changes in parameter values resulting from each treatment. The treatment effects from medication and DBS programming were considerably different across several period-related parameters: median, minimum, and maximum of period. All of these parameters showed greater improvement with medication, consistent with the findings in Table 4, where significant improvements were observed with medication (“med off” vs. “med on”), while no significant difference was found with DBS programming (“DBS off” vs. “DBS on”).

Table 6 Comparison of treatment effects of medication and DBS.
Full size table

Comparison of treatment effects of medication and DBS with covariate control

Similar to the results from the analysis without covariate control, the treatment effects of period-related parameters showed significant differences between medication and DBS programming (Table 7 and Supplementary Table S6). The IQR of speed, which showed no significant difference in the analysis without covariate control, improved more with medication than with DBS programming.

Table 7 Covariate-adjusted comparison of treatment effects of medication and DBS: medication change (“med on” – “med off”) and DBS programming change (“DBS on” – “DBS off”).
Full size table

Discussion

In this study, we first applied a state-of-the-art 3D hand pose estimation DL model to quantify the severity of bradykinesia in the FT task in patients with advanced PD, which enabled component-wise analysis of distal hand bradykinesia observed in the FT movements. We then presented and compared changes in motion parameters after dopaminergic medication intake, GPi DBS lead implantation, and DBS programming to better understand each treatment’s specific role in mitigating bradykinesia.

Before employing the 21 parameters for comparative analyses, we validated their effectiveness in capturing bradykinesia by using these parameters to predict the MDS-UPDRS Part 3 FT scores. The LR model achieved an accuracy of 0.70 and an acceptable accuracy of 1 (rating difference within ± 1). This is noteworthy, especially when considering that the three neurologist raters reached complete consensus in only 34.9% of cases. Their agreement improved to 93.5% when adopting the acceptable accuracy criterion, marking a competent performance that, however, did not surpass the model’s. Automated quantification of disease severity in PD has been actively explored, with several approaches already commercialized38. Methods range from open-source accelerometer-based algorithms, such as ReTap39 to vision-based libraries like OpenPose or Google MediaPipe40,41, aiming to improve the prediction accuracy of expert-labeled MDS-UPDRS Part 3 scores. The generalizability of vision-based models for whole-body bradykinesia has been validated across multi-site populations under various medication and DBS stimulation states42. However, methods dependent on wearable sensors require sensor calibration and sensors may restrict finger motion12,43, limiting their ability to capture natural movement. While vision-based models avoid these issues, most rely on 2D joint position keypoints11,41,44,45,46, which can reduce accuracy in measuring distances between keypoints due to their variations in depth. Mesh Graphormer we applied reconstructs hand mesh and keypoints in 3D from 2D footage. As a highly generalizable pre-trained model demonstrating competent reconstruction even on internet hand images, it showed effective results on our dataset without finetuning. Our model’s performance is either superior to or on par with the most recent advancements in MDS-UPDRS score prediction13,47. While one study reported a higher accuracy of 0.812, the input videos were filmed using a depth camera, which records both RGB and depth46.

Comparative analyses of the motion parameters reveal the distinct contributions of medication or dopaminergic effect, the microlesion effect (MLE) from DBS lead implantation, and the stimulation effect from optimized DBS programming (Supplementary Fig. S6). However, several motion parameters, including the IQR of amplitude, fatigue, and freeze, did not show significant changes across the four states (refer to the first p value column in Table 4). The minimal variance in the IQR of amplitude suggests that both medication and GPi DBS enhance amplitude with little impact on its variability. Regarding the fatigue and freeze parameters, few studies have addressed the detection and measurement of freeze intervals as we have in our research, although several previous works have reported on the treatment effects on fatigue. A previous study found no noticeable improvement in the speed and amplitude decrement of FT, hand grasping (MDS-UPDRS Part 3 item 3.5), and pronation-supination (item 3.6) tasks after medication intake when these tasks were considered together48. However, for a task focusing only on proximal upper extremity movements (e.g., tapping the ‘S’ and ‘;’ keys on the keyboard with one index finger), the medication did improve the decrement in speed49. Furthermore, increasing the intensities of STN open-loop DBS led to a noticeable improvement in the reduction of amplitude18. When controlling for covariates, aperiodicity and minimum period showed no significant differences across states, suggesting that both parameters may be influenced by covariates rather than reflecting intrinsic treatment effects of medication or GPi DBS. For aperiodicity, subgroup analysis in patients without LID further supports the influence of covariates, as aperiodicity did not significantly increase after medication in this group. Given that LID is often associated with higher levodopa doses, the observed increases in aperiodicity after treatment may be attributed to variability in levodopa dosage rather than the direct effects of medication or DBS.

Both the MLE and the stimulation effects contributed to improvements in speed and acceleration. However, changes in amplitude were solely attributed to the stimulation effect, while modifications in period were exclusively linked to the MLE. The MLE, characterized by improvements in motor symptoms following lead implantation but prior to the initiation of stimulation, has been observed in both STN DBS and GPi DBS procedures50,51,52,53. Beyond a significant reduction in the MDS-UPDRS Part 3 score, the extent of this decrease after lead implantation has been associated with long-term amelioration of motor symptoms50,52,54. Two functional MRI (fMRI) studies conducted during FT exercises further support the notion that lead implantation alters brain network activity. One study reported signal reductions in the bilateral basal ganglia, thalamus, motor cortex, and insula following STN DBS surgery without stimulation, compared to baseline levels in the preoperative medication-off state55. Another study found similar reductions near the anterior thalamus and GPi56. Additionally, in a resting-state fMRI, STN DBS stimulation enhanced connectivity within the left premotor area, whereas the MLE was linked to increased functional connectivity in the brainstem57. This suggests that MLE and stimulation may exert different roles in modifying neural networks, which may account for the varied improvements in motion parameters.

Comparing the cumulative effects of MLE and DBS programming resulting from surgical intervention to medication, we observed similar improvements in amplitude and period. However, DBS provided superior enhancements in speed and acceleration (“med on” vs. “DBS on”). This difference may be related to the distinct mechanisms by which medication and DBS affect motor control, particularly the involvement of the cerebellum. Previous studies have highlighted the cerebellum’s role in regulating movement speed. An fMRI study involving healthy participants showed that activity in the anterior lobe of the right cerebellum (lobule V) strongly encoded movement speed during a wrist flexion and extension task58. Similarly, in athletes, the maximum velocity of plantar flexion correlated with fMRI activity in the cerebellar anterior lobe59. Notably, athletes requiring rapid foot movements exhibited a greater grey matter volume in this region. Interestingly, one study suggested that GPi DBS can modulate cerebellar function. A nonhuman primate study found that GPi DBS modulated the cerebellothalamic tract, altering neural signals from the cerebellum to the motor thalamus, specifically the nucleus ventralis posterior lateralis pars oralis (VPLo)60. Furthermore, a study in patients with PD showed that GPi DBS improved both static and dynamic balance, effects not fully replicated by levodopa alone61. These findings suggest that GPi DBS influences not only the basal ganglia-thalamo-cortical circuit but also the cerebello-thalamic-cortical circuit. Although the precise mechanisms underlying the differential motor effects of medication and GPi DBS remain unclear, these observations suggest that GPi DBS may engage cerebellar pathways to a greater extent, contributing to its superior effects on speed and acceleration.

This study has several limitations. First, FT videos were recorded 2–3 weeks post-surgery, potentially conflating treatment effects with MLE. Some patients reported diminished DBS effects at ~ 1 month, necessitating voltage increases, suggesting the initial programming may not have fully captured lead stimulation effects. Future work will investigate long-term effects using 1- and 5-year follow-up videos. Second, no videos captured the combined medication and DBS “on” state, limiting interpretation of cumulative treatment effects. Third, although the DBS settings were optimized after extensive testing and video recording, the total MDS-UPDRS Part 3 score during DBS programming was unavailable. This score could have been used to validate the surgical process, including the accuracy of lead placement. Fourth, motion features were extracted through multiple steps rather than in an end-to-end fashion. As the analysis focused on thumb-index finger motion from variably zoomed videos showing the full body or upper body, cropping to center the hands was crucial for accurate reconstruction. Additionally, we had to visually inspect videos after each preprocessing step to mitigate potential noise from preprocessing errors affecting the motion signal. Fifth, patients with severe upper extremity dyskinesia were excluded prior to hand reconstruction, as rapidly changing hand positions were expected to hinder both accurate FT rating by experts and hand reconstruction. This exclusion may have introduced bias into the statistical results, although the dataset still included over half of the patients exhibiting some level of LID. Sixth, although hand keypoint coordinates were automatically normalized during hand pose reconstruction, the lack of individual hand size data may have introduced inaccuracies in the distance calculations between the thumb and index finger, which were used to derive motion parameters, such as those related to amplitude. However, our statistical analyses focused on within-subject changes in motion parameters across four states, thereby mitigating the potential impact of these inaccuracies. Seventh, the patients’ videos were taken from a single center, which may limit the generalizability of our results and interpretations. However, the study period covered a relatively long period of about nine years and the videos were recorded with multiple camcorders of varying resolutions in different background settings. Eighth, our covariate analyses may have overlooked other influential factors, such as detailed DBS settings beyond amplitude or disease conditions apart from hand bradykinesia. Given the potential variability of these unaccounted factors, our findings should be interpreted with caution, as treatment effects may vary accordingly. Lastly, the motion analysis was limited to FT, precluding assessment of the treatment effects on the proximal upper or lower limbs. Future investigations will include additional video analyses of pronation–supination, leg agility, and gait to provide a more comprehensive evaluation.

In conclusion, our research demonstrates that a vision-based DL model designed for general hand pose estimation can quantify and differentiate various aspects of hand bradykinesia in patients with advanced PD. We identified distinct treatment effects of medication and GPi DBS, as well as potential roles of GPi in mechanisms underlying bradykinesia in PD. Crucially, employing a DL model facilitated component-wise movement analysis, offering insights that the traditional semi-quantitative scoring system might have missed or overlooked. Our findings contribute to a more in-depth understanding of the components of bradykinesia that respond to treatments and, if responsive, whether dopamine replacement therapy or surgical intervention is more effective. This may help clinicians in planning treatment strategies for patients with PD, including determining DBS targets and anticipating its therapeutic effects. Further research is needed to elucidate the pathophysiology underlying the differential treatment effects and to understand how lead stimulation affects the neural circuits involving the GPi.