Movement of deception in motion capture

Abstract

Deception detection has attracted broad interest in professional practice and academic research, and body movement is considered one of the key aspects in deception detection. Previous work has focused on certain body parts (i.e., hand, head, leg) or gestures (i.e., gaze aversion, leg unnatural movement, etc.), which were manually coded by human judges. However, manual coding of nonverbal behavior is time-consuming and painstaking to the coders, as well as possibly vulnerable to observation biases. To overcome the challenges associated with manual coding, we employed a motion capture system to collect the body movements, in which a total of 80 participants were engaged in two interviews about their holiday experiences. In interviews about their vacation experiences, participants either told the whole truth or lied completely. The results revealed distinct movement patterns across conditions. Lower body movement differentiated honest from deceptive responses, and this association was moderated by task order, whereas upper body movement showed no reliable effects once task order was taken into account. Notably, the pattern does not support a generalized “rigidity effect” (i.e., uniformly reduced movement during deception); instead, the observed movement differences were order-dependent. Exploratory analyses using machine learning approaches further delineated the temporal dynamics of deceptive movement, providing complementary insights into nonverbal markers of deception.

Introduction

Deception is defined as “an act that is intended to foster in another person a belief or understanding which the deceiver considers to be false”(p. 3)¹. Previous studies have illustrated that people are poor lie detectors². This is often because they are focusing their attention on the wrong cues. The present study aims to further our understanding of how deceivers behave when they lie.

Zuckerman and his colleagues summarized emotional and cognitive factors that influence deceiver’s nonverbal behavior¹. As for the emotional account, researchers have focused on three emotions: guilt, fear, and “duping delight”^3,4,5. These emotions are theorized to manifest in response to deceitful behavior, and they can also influence the manner in which deceivers behave². Guilt may lead to gaze aversion because deceivers do not dare to look into receivers’ eyes when lying. Then, fear can lead to eye blinks, speech hesitations, speech pauses, speech errors, and self-adaptor behavior⁶. Duping delight may lead to behavior signs similar to joy, such as gestures, movements, and smiles increase. In addition to positive emotions such as delight, negative emotions resulting from deception also manifest as withdrawal behaviors, including a reduction in eye contact, less body orientation, and gestures². In the cognitive effort account, the extra mental effort or increased cognitive load required to construct a lie has been investigated^1,7. In order to prevent their fabrications from being exposed, deceivers must exert a considerable degree of effort to deceive their targets^8,9. Deceiving costs more time to prepare than telling the truth¹⁰, but deceivers cannot take unnaturally long to avoid being detected in their lies. They overcome this problem by extra cognitive effort, even if it may create distortions in other aspects. Indeed, early literature has indicated that participants started to have latency and pause more frequently when they were asked to make greater cognitive complexity required statements¹¹. Also, a high level of concentration and absorption in a speech would result in a reduction in the frequency of illustrations⁶. Thus, Zuckerman et al. suggested that the higher cognitive complexity of lying may result in more speech pauses or hesitations, longer response latency, and fewer illustrations¹.

Another powerful theoretical approach that predicts a series of nonverbal cue indicators is the rigidity effect¹. Interpersonal deception theory (IDT) argues that deceivers attempt to regulate their nonverbal behavioral conduct and overall image to appear credible while simultaneously attempting to control behaviors that could be detrimental to their performance⁸. According to the IDT, if efforts to appear natural, expressive, and relaxed are overridden by attempts to suppress signs of shame, the over-control will have the opposite effect. The rigidity effect postulates that extemporaneous deception under high stakes leads to an initial freeze response^12,13. Zuckerman et al. also indicate similar findings that liars showed rigid, wooden, unnatural postures through the experiment¹. Other research showed that liars maintain fewer movements of arm, leg, foot, and overall body movements compared to truth-tellers during deception^{7,12,14,15,16,17}. The study of Pentland et al. demonstrated that testing rigidity effect directly with videotapes from high-stakes experiments, and captured emotional and micro-expressions¹⁸. When responding to target questions, guilty participants completing a concealed information test (which controls for cognitive load) showed significantly fewer differences than innocent participants in four deception-relevant emotions (disgust, fear, sadness, and surprise). In the second experiment that measured differences in ten facial movements, the deceivers showed rigidity effects in eight of them when target pictures were presented. In another experiment by Twyman et al., despite participants being requested to employ compensating measures, rigidity still persisted¹⁹. Research on the mechanisms of the rigidity effect reveals it as an involuntary reflex associated with a defensive reaction to threat or demanding situations,¹⁹ or a result of excess demand on cognitive resources and working memory^20,21. Alternatively, if the inhibition of movement is only temporary while the deceiver decides how to respond, it may be more accurately described as an adaptive response in line with IDT, which can only be observed if there is sufficient time for dynamics to be observed^21,22,23. The rigidity effect showed that it has the potential to serve as an indicator of deceiving behavior, and has a possibility to use it value the stakes of deception, and being a demonstration of the increasing cognitive load.

DePaulo and her colleagues’ quantitative meta-analysis has revealed a number of nonverbal cues of deception⁷. Examples of behaviors that are commonly assumed to be linked to deception are gazing, smiling, and self-adaptors (scratching the head, wrists, etc.)² However, those “major” nonverbal cues of deception are not actually related to deception. Yet, they showed, compared to the truth-tellers, the deceivers have more gaze ($d =.03$), smiling ($d =.00$), eye blinks ($d =.07$), on the other hand, fewer self-adaptors ($d = -.01$), illustrators (hand and arm movements designed to modify and/or supplement what is being said verbally², $d = -.14$), hand/finger ($d = -.36$), leg/foot movements ($d = -.09$) and head movements ($d = -.02$), whereas their effect sizes were all considered small, and only illustrators and hand/finger were statistically significant⁷. More importantly for this study, the meta-analysis reported an erratic pattern and many findings have conflicting results⁷; some might find liars gaze more, but others have opposite results. A possible reason for inconsistent findings or small effect sizes in previous studies is the complex interplay of emotional and cognitive factors involved in deception¹.

The other potential reason for the inconsistent results is the inadequate scoring systems between past studies and the limited focus on the combination of cues that serve as indicators of deception². In the classic literature, the detection results of deception were obtained using manual coding techniques; namely, human observers have used a classification scheme²⁴ to code or rate speaker’s nonverbal behavior when lying^25,26,27. However, previous studies^28,29 have indicated that observer ratings could be vulnerable to bias. For example, no matter how focused the observers are, it is impossible for them to capture all the behaviors the participants make, and they can miss the slightest movements. Therefore, in order to overcome the issue of bias, researchers have initiated the use of an automated coding technique to measure nonverbal behaviors, including the posture of the speaker³⁰. As a result, researchers can obtain higher accuracy in capturing slight movements, which benefits the identification of which posture, movement, or behavioral pattern is dedicated to deception.

In this regard, Van der Zee et al. demonstrated how a motion capture system can be applied to deception detection²⁷. They proceeded with an experiment with two tasks; the first was to play a computer game called ‘Never End’ for seven minutes, and the second task involved handling a lost wallet. Half of the interviewees were instructed to respond truthfully, while the other half were instructed to lie. After completion of the two tasks, the participants were prepared for motion capture and responded to the questions posed by the experimenter. The questions were designed to increase their cognitive load by requesting details about the task in reverse order. In addition to recording movement data, the experimenters investigated the effect of cognitive load and emotions using self-report questionnaires to express how the participants felt and how hard the task was for them during the experiment. In order to ascertain the discriminative capacity of full-body movement when applied at the individual level, Van der Zee et al. performed a binary logistic regression to calculate the predictive value of full-body movement for the purpose of deception detection. The research revealed that full-body motion—defined as the sum of joint displacements—exhibited a 74.4% sensitivity in detecting lies. Furthermore, incorporating individual limb data into full-body motion measurements enhanced its discriminatory capacity, elevating it to an 82.2% precision level. Additionally, the researchers found that individuals who were not being truthful exhibited increased mobility. In contrast, Duran proposed that humans exhibit diminished movement in a state of deceit, as evidenced by their reanalysis of the motion capture data collected by Eapen et al.^14,15. However, their study was constrained to the investigation of specific anatomical regions, i.e., arms and head. Second, in Van der Zee et al., participants sit in an environment similar to an interview²⁷; whereas in Eapen et al.’s study, participants were confronted while standing¹⁵. Finally, Van der Zee et al. proposed that self-reported difficulty, or cognitive load, exhibited significant differences in questionnaire results but demonstrated no correlation with movement during either of the tasks.

In light of the two contradictory findings on the behavioral characteristics of deception in motion capture (i.e., liars move more versus liars move less), the present study was undertaken. The hypotheses under consideration are rooted in classical deception literature, as previously discussed.^1,2,7,8,12. However, they are tested through the lens of the methodology developed by Van der Zee et al.²⁷. Here, we formally propose three hypotheses as follows:

H1. Deceivers move less than truth-tellers.

Furthermore, deceivers experience greater negative emotions such as guilt and fear during deception^1,2. This study utilizes the Positive and Negative Affect Schedule (PANAS)³¹ to measure emotional states during deception (see Methods for details). Thus, as for emotions, we propose the second hypothesis:

H2. Deceivers obtain (2a) lower positive emotions scores and (2b) higher negative emotions scores on PANAS questionnaire than truth-tellers.

Regarding the cognitive aspect, the cognitive load is expected to increase when individuals try to process a complex, cognitively demanding task like deception, which would lead to a neglect of body language, reducing overall body movements, lesser finger, hand, and arm movement, and lesser leg and foot movement^6,32,33,34. In order to experimentally induce cognitive load, researchers have utilized unexpected questions as a method of detection in previous studies^35,36,37,38. To verify that participants indeed perceived these questions as unexpected, Monaro et al. developed and employed the Unexpectedness Questionnaire, a 7-point Likert-scale questionnaire in order to assess whether the interviewees perceived the examiner’s questions as unexpected or not^37,38. Accordingly, the third hypothesis is formulated as follows:

H3. Deceivers obtain higher scores on the Unexpectedness Questionnaire than truth-tellers, indicating greater subjective perceptions of unexpectedness.

Methods

In this experiment, a within-subjects design was employed. Participants wore motion capture suits to complete two vacation experience interviews^37,38; namely, vacation experience interview (honest), and vacation experience interview (lying). The order of vacation experience interviews was counterbalanced. The ethics committee for human research of National Chung Cheng University approved the experimental procedure. All methods in this study were performed in accordance with the relevant guidelines and regulations, as approved by the same committee. All participants provided informed written consent prior to study enrollment.

Sample size

This study recruited 80 participants (58 female, 22 male) aged 18–57 years ($M_{age}$ = 21.038, $SD_{age}$ = 4.790). However, data from 10 participants were damaged during the export process, and thus, they were excluded from the analysis. Reducing the number of participants to 70. Participants were recruited in courses offered by the Psychology Department that included a large number of students from various majors and paid NTD$300 for their participation.

Instrument

In this study, we employed a motion capture system VICON in order to obtain the highest accuracy and slightest movements as possible of participants’ motion. A motion capture system is one of the best accurate approaches to recording full body movements using automated coding techniques. By recording the 3D coordinate of the 69 markers attached to the subject’s body, researchers are able to collect the participant’s position, orientation, and their velocity of movement. Also, the time resolution (the sampling rate) of the motion recording is also high such as over 120 Hz (i.e., 120 times per second). A motion capture system can offer greater accuracy, view angle, and time series pattern than human manual coding in terms of precision, making it one of the best and most suitable techniques for observing and recording the dynamic of deception behavior.

Even in the deception literature, Van der Zee and her colleague suggest the potential validity and benefit of a motion capture system²⁷, as it is able to collect the information of magnitude and direction of the movement, which is typically not taken into account by past researchers, even findings showed that such differences carry the “meaning” of the movement³⁹. Motion capture techniques can also investigate combinations of cues than a single cue can increase the accuracy of detecting deception^2,40. Thus, through a motion capture system, bodily movement can be a reliable detector of deception in nonverbal behavior. As previously mentioned, studies using motion capture have shown contrasting results to those that have relied on manual coding. The results of major past studies using manual coding have demonstrated that deceivers move more when they are lying, which is confirmed in all parts of the body^2,7. On the contrary, studies with a motion capture system showed that individuals move less when lying^14,15, though the subject of focus in those studies was limited to certain body parts, i.e., arms and head. Yet, even among studies using motion capture, it seems that the results are not always consistent, as Van der Zee et al. again suggested liars moved more in motion capture than truth-tellers²⁷. Even with inconsistent and contrary findings among nonverbal deception cues, in terms of the accurate measurement environment, it is still beneficial to utilize a motion capture system to record full-body movements in order to identify the distinct motion patterns associated with deceiving and truth-telling, which should contribute to a comprehensive understanding of the behavioral dynamics between deceivers and truth-tellers.

Materials

Positive and negative emotions questionnaire. We used PANAS, a 5-point Likert scale with 10 positive emotions and 10 negative emotions questions³¹, to evaluate their emotions in the previous interviews. The positive emotions include: interested, excited, strong, enthusiastic, proud, alert, inspired, determined, attentive, active; $\alpha =.85$. The negative emotions include: depressed, upset, guilty scared, hostile, irritable, ashamed, nervous, jittery, afraid; $\alpha =.91$. This study used the scale version translated into traditional Chinese characters⁴¹.

Unexpectedness Questionnaire. In the culminating phase of the interview, participants were tasked with completing a post-interview questionnaire. This questionnaire comprised a series of 10 inquiries (five questions for each interview), to which respondents were required to respond using a 7-point Likert scale. The questionnaire was designed to assess the unexpectedness of the unanticipated inquiries. It is imperative to acknowledge that the Unexpectedness Questionnaire evaluated subjective perceptions of unexpectedness, not cognitive load. This distinction underscores the questionnaire’s role as a mechanism to detect expectancy violations, rather than a direct indicator of cognitive load. The questionnaire was modeled after the Unexpectedness Questionnaire from Monaro et al.’s study^37,38. The items included the following: “ Overall, I did not expect the experimenter to ask about something I wasn’t asked to prepare.” “I did not expect to be asked about specific details of my travel plans.” “I did not expect to be asked about unpleasant experiences during my trip.” “I did not expect to be asked about unexpected incidents that occurred during my trip.” “I did not expect to be asked to compare this trip with other trips I’ve taken.”

Data collection

On the experiment day, the experimenters provided participants with information about the experiment, including its duration and the right to withdraw from the study at any time. Participants were also verbally and in writing informed about the protection of their personal information.

After obtaining written consent, participants were asked to draw lots as instructed to determine the order of the next two tasks of the vacation experience (lie first, then be honest, or be honest then lie), making the experimenter blind to which condition (lie or truth) the participant was speaking under. Then fill in two forms (one for the true vacation experience; and the other one for the fake vacation experience) of questions about the vacation experience, including details about the vacation; namely, where they went, how they traveled, how long they stayed there, with whom they went, how the weather was, and a daily activity plan. For the honest condition, participants were asked to fill in experiences that occurred within one and a half years and lasted from one to seven days. They are also suggested to use supplementary tools, such as photographs and videos, to make sure that what they remembered was correct and omit details if they were unable to remember them. For the lying condition, they were suggested to search for information online to create the experience without any true details. The total time for filling out forms is controlled within 30 minutes.

The participants were dressed in a specific motion capture suit during the interviews and received help from the experimenter placing markers. Following the preparation of the motion capture setup, participants were directed to sit on a chair facing a camera for the upcoming experiment. During the vacation experience interview, participants were asked to (1) self-introduce (one minute, as a baseline), (2) introduce their vacation (two minutes), and (3) answer 15 questions asked by the experimenter. These 15 questions included reverse order and unexpected questions that participants were not prepared for, which are designed to increase cognitive load⁴², e.g., “What did you have for the last dinner of the trip?” “How much did you budget for this trip?” “What is the name of the hotel you stayed at?”, “What time did you leave on the first day?”

After each interview, participants were asked to complete the PANAS questionnaire^31,41. Finally, the participants were requested to complete the final questionnaire, the Unexpectedness Questionnaire, consistent with prior work to assess whether they perceived the examiner’s questions as expected or unexpected^37,38. After completing the Unexpectedness Questionnaire, the experiment was then completed. The total duration of the experiment, including setting a motion capture environment with the participants, was approximately 90 minutes.

Measuring body motion

In order to measure the body movement differences between the conditions, we transfer the absolute point data to the relative displacement records of each trial. Then, we examined an exploratory measuring by grouping the 75 estimate points into two parts, “Upper body” and “Lower body.” The reason why we did not align estimate points more specifically (i.e., arm/hand/torso/leg) like the previous study did²⁷, is because, in the results of this study, most of the connecting estimate body points are relevant, e.g., raising hand caused nearly the same motion displacement to finger, hand, forearm, arm, and shoulder. By categorizing them into “Upper body” and “Lower body,” eliminates the possibility that estimate points with similar motion displacement are analyzed as different factors.

Analysis strategy

The motion capture data was then analyzed to calculate the upper and lower body movements of each participant for each question and task. Although t-test or ANOVA could be our options, for the sake of comparison, this study followed the analysis method of Van der Zee et al.²⁷ and predicted whether participants were lying or not based on their bodily movements. More specifically, a binary logistic regression model was employed for statistical analysis, which utilized the experimental condition (1 = honest, 0 = lying) as the dependent variable and the amount of upper and lower body movements as the independent variables. To assess whether the order in which participants completed the honest and lying tasks influenced these relationships, we included task order (lie first vs. truthful first) and its interactions with the movement variables in the model for statistical analysis. For the computation, we used the glm function in R. More specifically, the model was formulated as follows:

glm(Condition $\sim$ UpperBody + LowerBody + TaskOrder +

UpperBody:TaskOrder + LowerBody:TaskOrder, DATA, family = binomial)

Here, Condition refers to a categorical variable indicating the experimental condition (honest or lying), while TaskOrder represents the sequence in which participants completed the task. The dataset used for this analysis is specified in DATA. By fitting a model to the data, it is possible to ascertain the existence of different patterns between honest and lying conditions simultaneously, provided that the relationship between them is understood. This knowledge allows for the prediction of the performance of participants under two conditions. Furthermore, effect sizes were reported in conjunction with all hypothesis results. Cohen’s d is employed for the purpose of conducting condition comparisons across all hypotheses, while Nagelkerke’s $R^2$ and odds ratio are utilized for binary logistic regression models. Please note that we tried a GLMM model with the random intercept of participants and interview questions, $glmer(Condition \sim UpperBody + LowerBody + TaskOrder + UpperBody:TaskOrder +$ $LowerBody:TaskOrder +(1|Participants) + (1|Questions), \, DATA, \, family = binomial)$. However, the results were almost identical to the GLM model because the random effects were very small (nearly zero).

In addition to the hypothesis tests, we also followed Burgoon et al.⁴³ and tried machine learning algorithms for an exploratory purpose. As in Burgoon et al.⁴³, six machine learning algorithms were employed to predict the binary condition of the study, namely, Logistic Regression, Random Forest, Naïve Bayes, Support Vector Machine (SVM), Bagging and Boosting. These algorithms were implemented by Burgoon and her colleagues in their study to predict nonverbal cues and have gained wide popularity due to their superior performance⁴³. In accordance with the methodology established by Burgoon et al., we employed a set of models to predict the binary condition (Honest vs. Lie) using two body movement measurements, namely, “Upper body” and “Lower body,” as independent variables⁴³. An 80/20 split was applied to partition the dataset into a training set and a testing set. To obtain a more robust estimate of each model’s prediction performance, a repetitive random split strategy was adopted, in which the full dataset was divided 100 times at random. (Due to Naïve Bayes’ assumption of independence among predictors, its formula was modified to include only main effects, $Condition \sim UpperBody + LowerBody + TaskOrder$.) The F1 score averages accuracy in predicting honesty and lying. For each train-test split, the accuracy and F1 score were computed, and the mean values across the 100 splits were used as the final evaluation metrics.

In order to analyze the PANAS questionnaires completed by participants following each trial, a paired t-test was employed to compare positive emotions among honest and lying conditions. The same analysis was also performed for the negative emotion reports in the two conditions. In addition, two 2$\times$2 ANOVAs were conducted to examine potential effects of task order, one for positive emotions and one for negative emotions. The unexpectedness reported by participants in response to the unexpected questions served as a manipulation check for the cognitive load-inducing interview strategy. It was also compared across the two conditions using a paired t-test, and additionally analyzed using ANOVA to examine potential task order effects.

The present study focused exclusively on upper–body and lower–body motion, PANAS scores, and unexpectedness scores. No other variables were measured, and all analyses conducted are reported above; no additional exploratory or post–hoc tests were performed. The present report includes only two conditions: honest and lying, with the order counterbalanced. An earlier design included a third condition, designated as “mixed,” which represented a mixture of truthful and dishonest responses. However, in accordance with the feedback provided by the reviewers, we removed the third condition entirely because it was incompatible with the other two. For transparency, it should be noted that the early variant was discontinued prior to the present analyses and did not contribute any observations to the reported dataset.

Results

Bodily motion

To examine whether participants exhibited different bodily movements between honest and lying conditions, we analyzed their relative displacement movement (i.e., distance moved per 1/120 second) using a binary logistic regression model. The analysis compared movement across two conditions: honest versus lying. Results for upper and lower body motion differences between conditions are presented in Table 1. The model’s Nagelkerke’s $R^2$ was 0.015, indicating a small effect size.

Table 1 Summary of the bodily motion. “N” represents the total number of questions for each condition (baseline excluded). Honest: 70 participants $\times$ 16 questions = 1120. Lying: 70 $\times$ 16 = 1120. Due to data damage, the number of participants in the task order was unbalanced.

Upper body motion

The findings reveal no significant effect of upper body motion on deception classification in the model that included task order and interaction (b = 0.24, SE = 0.37, p = .518, OR = 1.27; see Table 2). In addition, the interaction between upper body motion and task order proved to be non-significant (b = 0.40, SE = 0.55, p = .464). This finding suggests that there are no differences in the effect of upper body movement, regardless of whether those who lied first or those who told the truth first. As illustrated in Fig. 1, participants demonstrated marginally elevated levels of upper body movement in the deceitful condition relative to the honest condition. However, this observed tendency did not attain statistical significance. Consequently, the hypothesis H1 was not supported.

The effect size was negligible (d = 0.03), and the overall model fit for both body parts was found to be weak (Nagelkerke’s $R^2$ = .015). This phenomenon can be attributed to the participants’ attempts to persuade the interviewer to believe their false narrative, resulting in an increased prevalence of upper body gestures. However, this pattern was not sufficiently strong or consistent to reach statistical significance.

Table 2 Summary of the binary logistic regression of bodily motion. Task order was coded as a factor (reference: Lie First; comparison: Truth First).

Fig. 1

Upper body motion by task order and veracity. No significant difference was found between the honest and lying conditions in either task order.

Lower body motion

The analysis indicated a significant interaction between task order and lower body motion (b = 15.20, SE = 6.70, p = .023, OR = 3,972,709.00), as shown in Table 2. The participants in the “Lie First” condition demonstrated a reduction in lower body movement while lying compared to their responses when being truthful ($b = -13.31$). In contrast, those in the “Truth First” condition exhibited an opposite pattern, with a slight increase in movement during the act of lying ($b = -13.31 + 15.20 = 1.89$). These results suggest that the relationship between deception and lower body movement was moderated by task order (see Fig. 2). Therefore, H1 was partially supported. A reduction in lower body movement during deception was observed exclusively among participants who began with the lying task, thereby aligning with the rigidity effect. In this context, individuals may have experienced a redistribution of behavioral resources and suppressed unnecessary lower body movements. This resulted in less spontaneous movement, particularly in the lower limbs. In contrast, participants who began with a truthful task displayed greater lower body movement when later engaging in deception. This pattern aligns with findings by Van der Zee et al.²⁷, which suggested people move more when they’re lying.

It is important to note that some odds ratios that were extremely large or close to zero are likely explained by two factors. First, the displacement metric was recorded at a very fine temporal resolution (distance moved per 1/120 second), so even minimal raw changes could produce large shifts in the estimated regression coefficients. Second, certain motion-by-task order combinations showed quasi-complete separation, which we identified through quartile-based grouping and cross-tabulation, resulting in unstable coefficient estimates due to very small cell counts in some predictor categories. Together, these factors can inflate odds ratios without reflecting an extreme effect on underlying behavior.

Fig. 2

Lower body motion by task order and veracity. In the “Lie First” condition, participants showed less lower body movement when lying than when telling the truth. In the “Truth First” condition, the pattern reversed: lying was associated with greater lower body movement.

In summary, the overall variance explained by the binary logistic regression model was small. The upper body’s motion did not support H1, as no significant effects were observed and the effect size was negligible. In contrast, the relationship between lower body motion and veracity was moderated by task order, such that participants who lied first exhibited reduced lower body movement when deceiving, which partially supports the rigidity hypothesis.

Exploratory analysis

As a further step towards prediction, we intend to explore macro-level patterns of body motion between two conditions. The prediction mean accuracy and mean F1 score of the six machine learning models are displayed in Table 3. The results of both the accuracy and F1 scores indicated that the SVM model demonstrated the optimal performance among the six models examined, suggesting that this model is the preferred model on these data. With regard to accuracy, the performance of all models was around $.45-.55$, majority were only slightly above stochastic. In terms of the F1 score, the performance of the models was more varied. The Logistic Regression and the Naïve Bayes both performed below stochastic, while the others showed slightly above stochastic performance. Naïve Bayes yielded a markedly lower F1 score (.348) despite accuracy (.509) comparable to the other models. This finding suggests the presence of an imbalanced precision-recall trade-off, potentially driven by the model’s main-effects-only specification (absence of interactions) and distributional/independence mismatches, which result in depressed recall for the minority class.

Table 3 Prediction accuracy and F1 score of the machine learning models. Bolded numbers indicate the best performance.

Positive and negative emotions

The PANAS scales collected from the participants were analyzed with paired t-test (see Table 4). Positive emotion among the two conditions showed non-significant results, $t(69) = 1.08, \, p =.282, \, d = 0.13$. Furthermore, a 2 (honest vs. lying) $\times$ 2 (lie first vs. truth first) ANOVA was conducted to examine whether task order influenced the result, no significant main effect of condition ($F(1, 68) = 1.11$, $p =.295$, $\eta ^2_p =.016$), no effect of task order ($F(1, 68) = 0.71$, $p =.404$, $\eta ^2_p =.010$), and no interaction ($F(1, 68) = 0.93$, $p =.339$, $\eta ^2_p =.013$). This indicates that positive emotion did not differ from conditions (Figure 3a), therefore H2a was not supported.

On the contrary, a significant difference was found in negative emotion between conditions ($t(69) = -11.78, \, p <.001, \, d = -1.06$), The negative emotion scores from participants in the lying condition (M = 2.51, SD = 0.73) were found to be significantly higher than those in the honest condition ($M = 1.79$, $SD = 0.56$). This effect remained robust when including task order in an ANOVA test. Condition remained significant ($F(1, 68) = 136.53$, $p <.001$, $\eta ^2_p =.668$), the task order ($F(1, 68) = 0.42$, $p =.521$, $\eta ^2_p =.006$) and the interaction ($F(1, 68) = 0.0002$, $p =.988$, $\eta ^2_p <.001$) were not significant. This showed that participants had stronger negative emotion during lying trials, in which they might have experienced guilt and fear, and tried to lie, bluff, and persuade the opponent (Fig. 3b), which supports H2b.

Table 4 Descriptive statistics of positive and negative emotions and unexpectedness across conditions. Due to data damage, the number of participants in the task order was unbalanced.

Fig. 3

Positive and negative emotions across honest and lying conditions.

Unexpectedness

To this end, a paired t-test was conducted to examine the difference in participants’ perceived unexpectedness between conditions (see Table 4). The investigation did not reveal any statistically significant differences between the two conditions ($t(69) = -1.63, \, p =.108$), and the effect size was small ($d = -0.22$). The mean score of the honest condition (M = 3.89, SD = 1.31) and the lying condition (M = 4.18, SD = 1.36) did not demonstrate a significant difference, although the lying condition was marginally higher than the honest condition. These findings do not support H3.

We also conducted a 2 (honest vs. lying) $\times$ 2 (lie first vs. truth first) ANOVA. The condition was marginally significant ($F(1, 68) = 3.92$, $p =.052$, $\eta ^2_p =.055$), and the interaction between condition and task order was significant ($F(1, 68) = 21.06$, $p <.001$, $\eta ^2_p =.237$). This indicates that the difference in unexpectedness between the honest and lying conditions depended on task order. Post hoc analyses suggested that participants who encountered the lying condition first (“Lie First”) experienced significantly greater unexpectedness when lying than when telling the truth ($F(1, 33) = 20.27$, $p <.001$, $\eta ^2_p =.381$). In contrast, no such difference in unexpectedness was observed for participants who engaged in the honest condition first ($F(1, 35) = 3.62$, $p =.065$, $\eta ^2_p =.094$), see Fig. 4.

Furthermore, regarding between-group comparisons, task order showed no significant effect on unexpectedness in the lying condition ($F(1, 68) = 0.17$, $p =.678$, $\eta ^2_p =.003$); however, in the honest condition, task order had a significant effect: participants in the “Truth First” group reported significantly higher unexpectedness than those in the “Lie First” group ($F(1, 68) = 22.95$, $p <.001$, $\eta ^2_p =.252$).

To sum up, H3 was partially supported: deceivers reported higher unexpectedness than truth-tellers, but only in the “Lie First” order. Furthermore, between-group comparisons revealed that task order significantly influenced unexpectedness in the honest condition, participants in the Truth First group reporting notably higher unexpectedness than those in the “Lie First” group. These findings suggest that the occurrence of unexpectedness during the act of lying is contingent upon the sequence of tasks.

Fig. 4

Unexpectedness by task order and veracity.

Discussion

The purpose of this study was to investigate the full body motion behavior differences in truth, and deception. To reduce the potential bias from manual observations, data were collected via an automated coding technique using a motion capture system, which enables the collection of the participant’s full body position, orientation, and velocity of movement. Using a two-condition interview task recorded with the VICON motion capture system and PANAS, the Unexpectedness Questionnaire, three main findings were presented^31,37,38,41.

First, with regard to bodily motion, no significant differences were found in upper body movement between liars and truth-tellers. In contrast, a significant interaction was observed between condition and task order for lower body movement. In particular, when participants performed the lying task first, they demonstrated a reduction in lower body movement, which supports H1. This finding aligns with prior research on the initial freeze response under high-stakes deception^12,13, where liars might become physically inhibited due to the rigidity effect. Conversely, when the participants completed the truth-telling task first, the liars subsequently exhibited increased lower body movement. This result align with Van der Zee et al.²⁷, liars move more. This contradictory pattern might be indicative of a decrease in rigidity since participants initiated the task with a milder task order (i.e., the honest task). Consequently, they did demonstrate a notable degree of rigidity in the subsequent task. Alternatively, this shift could be attributed to a behavioral strategy modification following familiarization with the task. These results suggest that task sequencing plays a critical role in how deceptive behavior manifests and that support for H1 is conditional upon this sequence.

In previous studies, numerous inconsistencies have been reported regarding motion in deception. Van der Zee et al. suggested that liars move more with the investigation of sitting full-body movements²⁷, while Duran et al. demonstrated that they move less by asking the participants standing and only investigated head and hand movements¹⁴. Interestingly, our finding of increased lower body motion in the truth-telling task first order is consistent with the result of Van der Zee et al.’s study²⁷ which refers to liars moving more. The participants first engaged in the truthful task, which did not elicit the same level of pressure as the other task. Upon engaging in the latter task, the rigidity effect was not observed. The possibility that participants exhibited more lower body movement may be attributable to their preparation for the task from the previous task. Consequently, the lying task did not trigger the rigidity effect, as the risk was mitigated. Alternatively, with regard to H1, it is our contention that the observed lower body motion outcome in the lying task first order can be attributed to the rigidity effect, in which engaged deception under high stakes initially as the start of the tasks leads to an initial freeze response^12,13, which resulted in an overload of cognitive resources and working memory^12,13,20,21, thus deceivers maintained fewer bodily movements of the leg and foot. As aforementioned, liars could move less^14,15. However, it is worth considering that their finding only extracted body parts of the arms, hand, and head, on the other hand, the current result from our study suggested that the result only happened on the lower body. Although it is unlikely that a single study will conclude a link between deception and nonverbal behavior, researchers may need to focus on the interplay of multiple nonverbal channels. Automated coding methods would be an effective technique for this purpose.

In the exploratory analysis, prediction was performed through six machine learning models⁴³. However, even the most effective model, SVM, attained only 54.6% accuracy and 57.4% F1 score, which are only modestly above the chance level. The discrepancy between the prediction result and the findings of Van der Zee et al. (74.4% accuracy) is substantial. Firstly, the design of our study incorporated a within-subjects approach, whereas Van der Zee and Monaro implemented between-subjects protocols^27,37. A within-subjects design is intended to better isolate manipulation effects; however, task order must be explicitly modeled to address potential order or carryover effects. Therefore, we included task order and its interactions with movement measures in all models. Secondly, the operationalization of movement that was employed herein reduced full body motion into two segments (upper body and lower body). It is possible that this reduction in the number of segments is too extensive to capture the fine-grained kinematic cues reported in prior work. The design and measurement choices that were implemented likely attenuated the discriminative signal, which helps to explain the modest predictive performance that was observed.

Second, on the matter of positive and negative emotions responses, according to participants’ self-reported PANAS^31,41, no significant difference was found in positive emotions between conditions and task order did not produce significant main or interaction effects. (H2a was not supported), This finding suggests that the emotional outcomes resulting from deception are consistently insignificant, regardless of the order in which tasks are presented. In contrast, support with H2b revealed that the participants exhibited considerably elevated levels of negative emotion when lying rather than true. This lends credence to the hypothesis that deceitful behavior is psychologically demanding, potentially inducing internal emotional states such as guilt and fear^1,2, which are often associated with deceptive conduct. Guilt, fear, and delight are typically associated with deception^3,4,5. Negative emotions such as guilt and fear that occurred during deception can also affect their bodily movements to gaze aversions or self-adaptor behavior, such as grooming, scratching, and touching their own clothes and body^1,2,6. These findings are consistent with the conclusions of previous studies that indicated deception may amplify negative emotions without necessarily reducing positive emotions.

Third, with respect to unexpectedness (H3), the evidence was partial and order dependent. The study’s findings indicated that participants reported a greater degree of unexpectedness when lying in comparison to truth-telling, particularly when deception was initiated first (“Lie First” group). Conversely, the “Truth First” group exhibited no significant difference, with ratings of unexpectedness remaining consistently high and stable across both conditions. Between-group comparisons further showed no order effect during lying, whereas during honesty, participants in the “Truth First” group reported greater unexpectedness compared to those in the “Lie First” group. This pattern suggests that the experience of unexpectedness may not be a stable characteristic of lying itself. Instead, it reflects reflects participants’ evolving sense of question unpredictability, which varies depending on task sequencing. The first task, irrespective of its veracity, tends to feel more unexpectedness, given that the interview format and question style are still being familiarized with. By the time the second task is conducted, participants have adapted, thereby reducing the unexpectedness. When lying is initiated first (“Lie First”), this results in a greater preference for “Lie” compared to “Honest” within that particular cohort. On the other hand, when the act of telling the lie is prioritized second (“Truth First”), the contrast between these options was attenuated and did not reach significance.

These findings align with Monaro et al.³⁷ who demonstrated that the unexpected questions are efficient in increasing the cognitive load of the participants, as reflected in reduced blink rates during deception tasks. While the aforementioned study concentrated on physiological indicators, the present findings suggest that similar increases in cognitive effort may be subjectively perceived, particularly before participants have adapted to the questioning and in within-subject designs that require switching between honesty and deception. Thus, perceived unexpectedness may offers a complementary, introspective marker of cognitive load in deception paradigms.

In summary, the present study revealed that the behavioral and emotional responses to deception are shaped not only by the act of lying itself but also by the sequence in which truthful and deceptive tasks are presented. The findings indicated that lower body movement and unexpectedness were both significantly moderated by task order, thereby emphasizing the critical role of condition and task order in deception research. These findings underline the importance of examining interaction effects and suggest that deception-related cues are dynamic.

Limitation and future direction

The current study lends support to a variety of contradictory findings regarding the movements of deception. Some studies have indicated that liars move more²⁷, while others have indicated that liars move less^14,15. This study indicated that liars exhibit differential movement patterns in response to varying task orders, with these effects being observed exclusively in lower body regions. This finding aligns with a previous study that produced contradictory findings. These findings demonstrate that there has not yet been established a standard methodology for data collection, including the specific scenarios to be used, the positioning of the participants, and the grouping and counting of body parts for analysis. The previous study²⁷ took the differences for the subset of relevant joints into account, then aligned the subset of joints on the body part root. This effectively eliminates the movement due to movement in other body parts. The current study performed an exploratory data analysis by grouping body parts into two categories: “upper body” and “lower body”. Utilizing this methodology, it was ascertained that individuals engaging in deception exhibit increased upper body movement but diminished lower body movement during the act of deceiving, with a significant result. However, when incorporating machine learning prediction models, the prediction of new individuals may be hindered by the absence of IV numbers, the presence of smaller observed effect sizes, and the existence of other influences that result in the model’s classification efficacy being marginally superior to that of a random model. The methodology of calculating body parts warrants further discussion and experimentation, thus we do not reject the possibility of other distinctions for the movements of deception, such as combining or dividing body segments.

This experiment was conducted in Taiwan, where the participants were all Asian and primarily Taiwanese (with one Malaysian participant). This is a distinctive data set, as there has been no previous motion capture research targeting Asians, particularly the Taiwanese demographic. The Asian data can inform future studies that require cross-cultural comparisons, and it represents a crucial next step in the field’s evolution.