Introduction

The sense of touch is an essential aspect of social interaction among humans and animals, finding relations to attachment, bonding, stress, and even memory1,2. The literature focuses on understanding the role that touch plays in social interaction. The term social touch began to be coined as the act of touching another person in a social context3. The use of social touch is diverse, ranging from its use during greetings to showing affection and support4.

Interpersonal touch is inherently reciprocal, as one cannot touch without being touched in return. While studies have explored the psychological implications of caressing others5, particularly in non-affective touch, research on the effects of active social touch from the toucher’s (In the manuscript, we use the term toucher to describe the person or the agent who is actively touching, as opposed to the one who is touched) perspective is still limited, particularly from an interoceptive standpoint.

Touch is a reciprocal interaction that elicits reactions from the recipient. These reactions can be shaped by the expressive qualities of both parties, as expressiveness plays a crucial role in interpersonal relationships, with extroverted individuals often making better impressions on people6. In this sense, it can be expected that during a tactile interaction initiated by the toucher, the subsequent expressive reaction of the recipient might have an influence the toucher’s behaviour. Therefore, studying how the recipient’s responsiveness and demeanour affect the interaction’s quality and outcome could provide valuable insights into tactile communication’s nuances and its role in fostering social connections.

Social robots are designed to facilitate natural and affective interactions between humans and artificial agents. To achieve these natural interactions, it is essential for the robot to be recognized as a valid social actor7. Humans tend to feel affiliation towards living beings, suggesting a potential benefit of giving robots an animate appearance to create natural interactions8. In the context of tactile interaction, considering both verbal and multimodal responses when crafting robots’ reactions to touch may be valuable. Research shows that 45% of communication content is conveyed through non-verbal channels, which significantly influences the impression formed by the speaker9,10.

Assessing the impact of actively touching a robot in human–robot interaction (HRI) presents several challenges. Firstly, it’s crucial to determine an interaction context where touch plays a significant role. We aimed to create a game that incorporates touch, raising the need to establish a baseline for comparison with activities devoid of touch. Additionally, defining appropriate expressive responses to touch activities and implementing these behaviours presents another challenge to address.

Therefore, the study presented in this work aims to evaluate the effects of active social touch and the robot’s expressiveness on the user attitudes and behaviour in human–robot interaction in the context of a memory game. Social touch has been shown to influence attitudes and behavioural responses, as even a brief touch on the hand, arm, or shoulder can enhance a person’s perception of the toucher, their emotional state, and their attitude towards the interaction setting11. In this study, these effects are explored through measures of engagement, fun, and intrinsic motivation, assessing how active social touch impacts user experience in human–robot interaction. For this purpose, we developed a game in which the user has to memorise and reproduce a sequence. The sequence incorporated a new element in each round after the user completed it and had to be repeated from the beginning at each turn.

In this study, we established a baseline condition using a custom external peripheral with buttons that emit sounds when pressed. For the second condition involving touch interaction with the robot, we developed an application leveraging the Acoustic Touch Recognition (ATR) system integrated into the robotic platform12. This application mirrored the button version’s rules but required users to touch different areas of the robot and remember the specific gestures to perform. These conditions (buttons and ATR) were combined with another variable: the robot’s expressiveness. This resulted in four distinct conditions: using the external peripheral with or without the robot’s expressiveness and employing the ATR system with the robot’s expressiveness active or inactive.

The study concluded by evaluating the combined effects of both factors on user attitudes and behaviour. We introduced the concept of user engagement, as defined by O’Brien13, which measures the depth of user experience during Human-Computer Interaction (HCI). Additionally, intrinsic motivation, as described by Deci and Ryan14, was assessed to determine if users interacted with the robot for inherent satisfaction or external outcomes. Furthermore, we considered the fun aspect of the activity, as highlighted by Tisza et al.15, and its role in catalyzing behavioural change and promoting enjoyment, as discussed by Gerow et al.16. These aspects related to user attitudes and behaviour will be evaluated through post-experimental questionnaires administered to the participants.

Background

This section lays the foundations for the multidisciplinary approach presented in this study. We merge concepts from robotics and computer science with concepts from the analysis of human entertainment, everything in the context of social touch. First, we will address how social touch impacts human attitudes and behaviour, both in human-human and human–robot interactions, emphasising its applications to social robotics. Secondly, we will explore the effects of the robot’s expressiveness on the user.

Human–robot touch interaction and its effects on the user

To properly frame our contribution, we first need to address the role of the sense of touch in HRI, specifically the effects of social touch. In tactile HRI, physical interactions between the robot and a human can be studied from two perspectives: the perspective of behaviour development and execution by the robot17, or the perspective of safety (accidental contacts, for example). In this work, we are interested in the former.

The attitudes and behaviour of the person who another person is touching may be affected by social touch. A light touch to the hand, arm or shoulder can positively affect how the recipient feels about the peer, how they are feeling emotionally, and how they feel about the environment in which the touch is occurring18. A robot’s simulation of social touch might also positively influence the user’s attitudes and behaviour towards the machine. Nakanishi et al. demonstrated by conducting multiple studies that conversations through a physically huggable humanoid cushion device, called Hugvie19, enhanced interest and maintained trust in the partner20, reduced negative imagination about a topic21, and improved attention and memory retention22 when reading to children. Later, in 2021, Hoffmann and Krämer23 examined the persuasive effects of non-functional touch from a humanoid robot, showing that even brief contact can increase compliance and improve subjective experiences.

The literature is scarce concerning the works that strive to understand the role of active social touch in human–robot interaction, mostly because the research topic is fairly recent. Much of the existing research has focused on the functional and industrial applications of touch in HRI. For instance, Iskandar et al.24 developed an intrinsic robotic sense of touch based on force-torque sensing, designed to improve physical interactions in collaborative industrial environments, manufacturing, and assistive robotics. These approaches emphasize touch as a means to enhance precision and efficiency in task-oriented settings rather than as a mechanism for social bonding. In contrast, other studies have begun exploring the role of active social touch in shaping human–robot relationships. Regarding user’s attitudes and behaviour, the work by Midorikawa et al.25 aimed to demonstrate that eliciting contact from people could be a first step in building better human–robot relationships. Their research showed that an initial handshake produced a positive emotional effect on the subjeqcts. Following this direction, our study further explores how active social touch can shape user attitudes and behaviour in human–robot interactions, reinforcing the idea that physical contact is not only a functional input but also an important factor in creating social bonds with robots.

Robot expressiveness and its effects on the user

Extensive research has been conducted on endowing social robots with the ability to use expressive behaviours. While traditionally, these solutions relied on predefined expressions (in this work, we will use both gesture and expression to refer to combinations of unimodal actions like moving a limb or uttering a sentence seeking to convey a standard communicative message), nowadays there has been a change towards generating the robot’s behaviours dynamically. In 2012, Meena et al.26 proposed aligning the stroke of the gesture (part conveying meaning) to the script’s keywords. Their experiments suggest that open arm motions can help to structure the robot’s discourse and gaze-following and head motions can help to maintain user engagement. Aly et al.27 proposed in 2016 adapting the robot’s verbal expressiveness to the user’s personality trait (extroverted/introverted) and then using predefined rules for generating expressions based on the verbal message. Adding the non-verbal component made their robot perceived as more engaging, appropriate, and social than a robot that only uses verbal messages.

In 2018, Hasegawa et al.28 proposed generating gestures from a robot’s speech audio features using a Long-Short Term Memory network. They compared this approach with handcrafted and mismatched expression-speech combinations. Their approach was perceived as being more natural, timely, and semantically consistent with speech than the mismatched condition. Besides conveying task-related messages, expressiveness research in robotics has also sought how to appropriately convey the robot’s internal state, particularly its affective state (mood and emotions). For example, Hong et al.29 observed in 2020 that users considered interacting with a robot capable of expressing their affective state more pleasant, and chose vocal intonation as the robot’s primary source of affective expression.

Materials

This section breaks down the essential components and technologies used in our experimental setup to explore how touch can enhance human–robot interaction. These elements are (i) the robotic platform used, (ii) a peripheral that contains several buttons used to play one of the games, and (iii) the touch recognition system that captures when and how the user touches the robot.

The social robot Mini

Our social robot, Mini30, is a desktop robotic platform engineered at Universidad Carlos III de Madrid (see Fig. 1). Its primary purpose is to assist elderly people with mild cognitive impairment. Mini has been designed to serve as an assistive technology, offering its users entertainment and informative functionalities, thereby enhancing their quality of life through interactive engagement and support.

Fig. 1
figure 1

The social robot Mini with the tablet and the buttons peripheral.

Full size image

Mini has LED in its cheeks and heart, speakers for emitting voice and non-verbal sounds, OLED screens for eyes, five degrees of freedom (two in the neck, one in each arm and another in the waist), and a touch screen used both for interacting with the users through menus and for displaying multimedia content. We have enhanced the robot by adding a peripheral with five coloured buttons and a touch gesture recognition system.

Regarding software (see Fig. 2), Mini’s modular architecture has been built using ROS31. The tasks the robot is able to perform are provided by the Skills (capabilities), which include different activities, like games, multimedia, information and cognitive stimulation exercises. A Decision-Making System (DMS) is responsible for initiating or stopping these Skills. In our work, the game involved in the study has been integrated as two separate skills, each associated with a different sensor.

Fig. 2
figure 2

Main modules of the Robot’s software architecture. The blue modules show the Touch Simon Game (touch sensors and skill), and the light blue modules correspond to the buttons peripheral and the Simon Game skill.

Full size image

The Sensors collect information from the robot’s environment. Among these are the button-based peripheral and the touch system mentioned earlier, each serving as an alternative interaction interface for gathering data, thereby providing diverse means for the robot to perceive and respond to its surroundings. This information is passed to the Perception Manager (PM), where it is packaged and formatted appropriately. The Human Robot Interaction (HRI) controls the flow of communication between the robot and the user. The Expression Manager (EM) orchestrates the robot’s actuators. Finally, a Liveliness module generates random behaviours aimed at giving Mini an appearance of animacy.

Interaction interfaces

HRI is evolving rapidly, driven by technology advancements and novel interaction paradigms. Developing peripherals and systems that facilitate intuitive interactions is crucial in this context. After presenting the general software architecture of the robot, the next step is to describe in more detail the two interfaces that take part in this experiment. On the one hand, we have a peripheral consisting of coloured buttons designed for cognitive and physical stimulation games. On the other hand, the tactile system enables touch recognition on the robot’s surface using piezoelectric pickups. Further details are explored in subsequent sections.

Buttons peripheral

The buttons peripheral (see Fig. 1) has been designed to develop cognitive and physical stimulation games that motivate the user to perform movement activities with the robot. Additionally, it enables a more direct interaction with the robot because each button can be associated with a different action or function, providing an interactive experience. Its main objective is to increase the user’s attention by using different coloured buttons and sounds.

This device is connected to the robot, which decides what actions the device should perform and receives user interactions with the peripheral. As discussed in ’Materials” Section, the communication between the robot and the buttons peripheral is done through a ROS node that transmits the information from the peripheral to the robot’s perception and vice versa. Once the information is transmitted, the robot distributes it to the corresponding modules to carry out the actions assigned to each button.

The variety of colours (yellow, red, green, blue, and white) adds an aesthetic element and enhances sensory stimulation, enriching the experience. Additionally, the peripheral is equipped with a buzzer capable of playing simple sounds, thus adding an auditory dimension to the experience by providing audio feedback on the actions performed.

The acoustic touch recognition system

The final component of the setup is the touch recognition system. The Acoustic Touch Recognition (ATR) system to detect, identify and locate touch gestures on a robot surface12. A distinctive aspect of this system is its innovative use of piezoelectric pickups in social robotics, specifically within human–robot touch interaction. These pickups detect sound vibrations produced when a user touches the robot’s surface. The resulting perturbations propagate through the robot’s rigid components (shell and inner structure), creating a unique signature that enables the system to recognize the type of contact and its location on the robot’s surface.

The operational principles of the system are as follows: when multiple sensors register an interaction, the system initiates the individual processing of their signals. It extracts features in both the time and frequency audio domains. Upon detecting the conclusion of the contact, the system calculates the average values of the extracted features during this duration. These values contribute to a labelled instance representing the specific contact. A dataset comprising these instances is subsequently employed as training data for classifying the gesture and its location using machine learning techniques. The system is integrated into the robot’s software architecture as a ROS node in the Detectors module as shown in Fig. 2 and sends its output to the Perception Manager.

The gestures recognised by the system are stroke, moving the hand with gentle pressure over the surface, tap, striking the surface with a quick light blow or blows using one or more fingers; tickle, touching the surface with light finger movements; scratch, rubbing the surface with the fingernails; slap, quickly and sharply striking the surface, and rub, moving the hand repeatedly over the robot’s surface. The robot’s arms and belly are where the system is able to locate physical contact.

Expressiveness capabilities of the Mini robot

The Expression Manager is the Mini software architecture module that orchestrates the robot’s expressiveness capabilities based on the other modules’ needs32. In particular, this element performs three main tasks. First, it receives requests for performing expressions from the rest of the architecture and plans their execution. The two main sources of expressiveness requests are the HRI Manager and the Liveliness.

The second task is managing any conflicts that might arise while handling the requests for executing gestures. By conflict, we understand a situation where two modules request expressions that require the same interfaces (for example, two expressions that need to move the same arm). Whenever a new expression is requested, the Expression Manager compares the list of interfaces the new gesture will need with the ones currently used by the active expressions. If a conflict arises, the system checks the priorities (either low, medium, or high) of the expressions involved to decide which one to execute immediately and which one to discard (if it has low priority) or store for being performed later (if medium priority).

Finally, the last task involves controlling the correct execution of the robot’s gestures. For this, the Expression Manager loads the expressions that must be performed from a gesture library, configures them appropriately, and ensures their execution is completed successfully by sending the appropriate commands to the robot’s actuators.

Memory game

The game that will be used during our evaluation is similar to Simon, an electronic game created in 1978 based on the traditional game, in which one of the participants says an action and the rest must perform it. The original electronic game consisted of four buttons, each of a different colour, which light up randomly while emitting a sound. Once the sequence finishes, the player must reproduce the sequence in the correct order. We adapted the game to be played with the Mini robot, integrating it as two distinct Skills, each employing a different interaction modality. One approach is based on the uses of the buttons peripheral and the other uses the touch recognition system. The following sections discuss these interfaces and their role in the experimental setup.

Button-based version

We adapted the game to be played with Mini’s button peripheral, having the robot light up the buttons to show the sequence and asking the user to repeat it. The length of the sequence determines the difficulty of the game. While the instructions and dynamics are identical to the original version, our peripheral has five buttons instead of four. Figure 3 shows the flowchart for this version of the game.

Fig. 3
figure 3

Flowchart of the Simon game when using the buttons peripheral.

Full size image

The game proceeds as follows. First, it starts and the robot asks the user if they need a reminder about the game rules and dynamics. If the answer is affirmative, the robot explains the rules. Once this step is completed, the robot shows the sequence by lighting up one of the buttons in the peripheral randomly and playing the associated sound. Then, the robot waits for the user input to replicate the sequence. Once a button has been pressed, the robot checks if it matches the corresponding colour in the sequence (the right input). If the user presses the correct button, the robot verbally cheers the user (an example of utterance used for this is “Let’s move to the next level”), adds a new colour at the end of the sequence, plays the new sequence for the user, and waits for them to replicate it. In sequences with multiple steps, the robot checks after the user presses every button to see if it matches the corresponding colour in the sequence and if it is the last element in the sequence. If it is not the last one, the robot waits for the next button press and repeats the process until all steps in the sequence are completed. If the user presses the wrong button, the robot stops the game and tells the user that the game is over (with an utterance like “You have lost”).

Touch-based version

A new version of the previous game involving the touch system has been specifically designed for this experiment. While the game’s rules are the same, the mechanics have been modified. Instead of a sequence of colours on the peripheral, the user has to repeat a sequence of tactile gestures by touching the robot on its arms and belly. The tactile gestures integrated into the game are tap, slap, stroke, tickle, rub, and scratch, as shown in “Materials” Section. Instead of generating a sequence of colours, the robot generates a sequence of tactile expressions by randomizing the location and type of contact for each element in the sequence. Also, since the robot does not have LED in the contact locations mentioned before, the sequence is shown by voicing the type of contact while moving the location where the user has to touch the robot (i.e. for asking the user to slap the left arm, the robot would say “slap” while moving the left arm). We combine verbal messages and motions to shorten the explanation.

Figure 4 shows the complete flowchart of the version of the game that uses the touch system. The boxes in blue are the steps specific to the touch-based version of the game, while the remaining boxes are the same ones in the button-based version. Besides the method for communicating the sequence to the user and capturing the user’s input, there are two main differences from the previous version of the game. First, when explaining the game’s rules to the user, the robot explains the touch gestures. This is done in two ways to ensure the user understands the meaning of each gesture. The first way is by defining the gesture verbally and textually on the tablet, in line with the ones presented in Yohanan’s dictionary33. The second way is displaying a video demonstration of the gesture on its tablet. The videos are intended to prevent possible conflicts between gestures that are a priori similar or whose formal definition alone might not solve ambiguous concepts related to them. For example, the duration or strength of the gestures involved in performing them.

Fig. 4
figure 4

Flowchart of the memory game using the touch recognition system. The elements included in the pipeline are in blue.

Full size image

The second difference involves the feedback given to the user when they are repeating the sequence. Since touching the robot is not equivalent to pushing a button, in this game version, the robot gives the user feedback verbally and explicitly on whether the gesture was correct. In addition, the game includes a threshold to evaluate whether the touch gesture was incorrect or a false negative. This feature was based on whether the touch gesture recognition system classified the gesture with enough confidence. If a false negative is detected, the robot indicates that it cannot understand the gesture and asks the user for repetition.

Methods

We established that, during the game, the robot would play the role of animator-guide rather than that of a rival. This decision was motivated by the game’s rules. Therefore, as far as the robot’s expressiveness is concerned, this was manifested through encouraging comments in each successful round, joking comments and signs of disappointment during the defeat. In addition, the robot kept track of the user’s successful rounds, and finally, when the user lost, it indicated the number of successful rounds and the correct sequence. As indicated before, we theorised that this behaviour by the robot might influence the user during the test. In particular, based on the ideas presented in works like Riggio et al.6 or Fromm et al.8, we theorise that adding these expressiveness cues will improve how users perceive Mini. Besides the effects of the robot’s expressiveness on the interaction with the users, we believe that these effects could be different when comparing the button and touch systems. For example, the user might focus on the button system, not the robot’s feedback. Equivalently, in the case of the touch system, expressiveness could influence the user’s attitudes and behaviour since the user interacts directly with the robot during the touch-based game. Therefore, we have decided to treat expressiveness as a factor with two different conditions: expressiveness present, i.e. the natural state of the robot, and minimising expressiveness, giving minimal information to the user to continue the game and be aware of events such as making a mistake and ending the game.

We designed a 2x2 between-subject design user study to test four experimental conditions: the presence or absence of the robot’s expressiveness and the choice of peripheral. The peripherals were either the button-based game or the game based on the ATR system. We chose to conduct a between-subjects experiments because they avoid the carryover effect, where the evaluation of a condition by a participant will be biased by their experience with previous conditions34. Also, this type of experiment results on shorter tests, which will help to minimize fatigue among participants. On the other hand, they require a larger number of participants, are more prone to errors due to differences between participants if the groups are not well balanced, and it can be harder to detect differences in responses to the robot. The four conditions in our experiment are the following:

  1. 1.

    Buttons and No expressiveness condition (BN): We use the button-based game and minimum expressiveness from the robot interaction. The robot will only conduct the game and give the user essential guidelines to play.

  2. 2.

    Touch and No expressiveness condition (TN): The same as before, but instead of using the button-based peripheral, the user will play directly with the robot using the touch system.

  3. 3.

    Buttons and Expressiveness condition (BE): The expressiveness is added to the button-based game. Under this condition, the sentences that the robot uses to give feedback to the user are longer and richer. Also, the robot accompanies these utterances with nonverbal expressions that match the sentiment of the sentences. This means that a happiness expression will performed alongside positive sentences (for example, when the user gets the colour sequence right), while a sadness expression will be used alongside negative sentences (for example, if the user presses the wrong button)

  4. 4.

    Touch and Expressiveness condition (TE): We include expressiveness in the button-based game. Similar to the addition of expressiveness to the button-based game, we enrich the utterances that the robot uses (each game uses different sentences that relate to the interaction modality used in the game, this is, the touch system or the peripheral), and add happiness or sadness gestures depending on the sentiment of the utterance.

The peripheral choice and the addition of expressiveness were the independent variables of our experiment, while engagement, intrinsic motivation and fun were the dependent variables. Now that the variables are set, we can formulate the following hypotheses:

  • For Engagement:

    1. 1.

      \(\mathbf {H_{eng1}}\): There are significant differences in engagement between using the touch system and the button system.

    2. 2.

      \(\mathbf {H_{eng2}}\): Significant engagement differences exist between expressiveness activation and non-activation.

    3. 3.

      \(\mathbf {H_{eng3}}\): There is an interaction effect between peripheral type and expressiveness activation on engagement.

  • For Intrinsic Motivation:

    1. 1.

      \(\mathbf {H_{im1}}\): There are significant differences in intrinsic motivation between using the touch system and the button system.

    2. 2.

      \(\mathbf {H_{im2}}\): There are significant differences in intrinsic motivation between expressiveness activation and non-activation.

    3. 3.

      \(\mathbf {H_{im3}}\): There is an interaction effect between peripheral type and expressiveness activation on intrinsic motivation.

  • For Fun:

    1. 1.

      \(\mathbf {H_{fun1}}\): There are significant differences in fun between using the touch system and the button system.

    2. 2.

      \(\mathbf {H_{fun2}}\): There are significant differences in fun between expressiveness activation and non-activation.

    3. 3.

      \(\mathbf {H_{fun3}}\): There is an interaction effect between the peripheral type and expressiveness activation on the fun.

Participants

A total of 83 people volunteered in the study. Because this research is an initial step into understanding the effect of active touch during HRI, we decided to test it with the general public. This is because results obtained for a specific population might not generalize to other populations35. Regarding gender, 37 participants identified themselves as female and 46 as male (45/\(55\%\)). In terms of age, 26 belonged to the 18–24 group (\(31\%\)), 14 were in the 25–34 group (\(17\%\)), three were in the 35–44 group (\(4\%\)), 8 participants were in the 45–54 age group (\(10\%\)), five belonged to the 55–64 age group (\(6\%\)), and 27 were older adults, with 65 years or more (\(32\%\)). All the volunteers were assigned in a random manner to the four conditions, resulting in \(n_{BN}=21\), \(n_{TN}=21\), \(n_{BE}=20\) and \(n_{TE}=21\). The Research Ethics Committee (CEI) of the Carlos III University of Madrid approved this experiment (under the name PI_engagement_HRI), it was performed under the European General Data Protection Regulation (Regulation (EU) 2016/679, abbreviated GDPR), and informed consent was obtained from all participants and/or their legal guardians.

Procedure

We used the Mini robot placed on a table for this experiment. The robot’s location varied slightly depending on the peripheral involved in the interaction. In the case of the button device, the robot was positioned just behind it. In the case of using the touch system embedded in the robot, Mini was positioned close to the edge of the table to facilitate the user’s physical contact with the robot. Both setups are displayed in Fig. 5.

Fig. 5
figure 5

Experimental setups of the user’s attitudes and behaviour experiment.

Full size image

The experiment was carried out as follows. The user entered a room where the robot and the experimenter were located. The experimenter briefly explained to the participant what the experiment consisted of. After this, the participant was asked if they had any questions. The participant then took a seat and filled out a data protection document, including personal and contact details and the identifier that serves as a pseudonym. Once this document was completed, the experimenter left, and the user proceeded to interact with the robot.

The experiment consisted of playing the memory game explained in “Materials” Section. Although there were different conditions, none fundamentally altered the game’s mechanics, which continued until the player made a mistake. At that point, the player had the option to continue playing or proceed to complete the questionnaire. Participants were free to play for as long as they wished, without a fixed time frame. On average, each game lasted approximately 8.64 min, with durations ranging from 1.00 to 18.92 min.

Post-experimental questionnaire

The objective was to create a unified questionnaire to gather information regarding the metrics presented in the introduction. Our main priority when designing the unified questionnaire was gathering enough data to ensure the reliability of the responses while minimising assessment time and respondent fatigue. To assess engagement from the user, we used the short form of the User Engagement Scale (UES-SF). This questionnaire has proven to be sufficiently accurate and valid and is frequently used in digital contexts36,37,38. It contains 12 items that correspond to four different categories, with three items each: the feeling of being absorbed in the interaction and losing track of time (Focused Attention), the negative affect experienced as a result of the interaction and the degree of control and effort expended (Perceived Usability), the attractiveness and visual appeal of the interface (Aesthetic Appeal) and the Reward Factor. Next, we measured intrinsic motivation with the Intrinsic Motivation Inventory (IMI)39. To introduce it in our study, according to its authors, we first had to decide which of the variables (categories) we wanted to use based on the theoretical questions we were addressing. We used the items from the Interest/Enjoyment category since it is regarded as the self-report measure of intrinsic motivation. Finally, to assess fun, we used the FunQ questionnaire, with 18 items divided into six dimensions: Autonomy, Challenge, Delight, Immersion, Loss of Social Barriers and Stress. We decided to remove Stress, Autonomy and Loss of Social Barriers from these dimensions because they had a less significant effect on the ‘Experienced Fun’ concerning the other categories, according to the authors40.

Next, since most of the people who participated in the experiment only understood Spanish, we followed a set of steps to adapt the questionnaire: (i) Each of the items used in the final questionnaire was translated into this language, trying to preserve the original meaning of the question in the translation, (ii) backward translation from Spanish, and finally, (iii) comparison of the original and the backwards translated English text, solving the discrepancies. Lastly, for each item in the final form, we used a 5-point Likert scale, ranging from 1-‘strongly disagree’ to 5-‘strongly agree’. To score engagement and intrinsic motivation, we computed the average of the scores of each of its items. For the FunQ, however, the resulting score was obtained as the sum of the scores of the items without averaging. We also collected demographic data about age, gender and mood. At the end of the form, we left an open and non-compulsory ‘Comments’ question, where the participants could give their opinion about the study and the elements involved, such as the peripherals, the system’s performance or the expressiveness of the robot.

Results

This section covers the statistical analyses carried out using the IBM SPSS Statistics software (version 26) and the information extracted from the comments left by 44 participants of the experiment. An alpha significance level of \(\alpha =0.05\) was used for the statistical analyses.

Quantitative results

The first step before the analysis was to detect and remove all possible outliers the dataset might contain. By analysing the interquartile range for each dependent variable, we removed seven cases from the dataset that SPSS considered outliers. The final dataset comprised 77 samples, 19 from the BN condition, 20 from the BE condition, and 20 and 18 for the TN and TE conditions, respectively.

We planned to perform a 2-way MANOVA involving engagement, intrinsic motivation, and fun with this dataset. We had to test whether the assumptions were met to do this analysis. The first assumption implies that the data should be normally distributed. A set of Shapiro-Wilk tests verified that, on the one hand, the fun and the intrinsic motivation met this premise by returning their respective non-significant tests \(p=0.213\) and \(p=0.428\). Nonetheless, deviation from normality was significant in the engagement case (\(p=0.021\)). The following condition was the correlation between variables. In this case, all possible combinations between the variables showed a significant correlation: engagement-motivation (\(r=0.711\), \(p<0.001\)), engagement-fun (\(r=0.692\), \(p<0.001\)), and fun-motivation (\(r=0.713\), \(p<0.001\)). The last assumption implies verifying the homogeneity through Levene’s tests. All three variables, engagement (\(p=0.639\)), motivation (\(p=0.132\)) and fun (\(p=0.219\)), showed no significance, thus revealing homogeneity of variances.

The 2-way MANOVA for the intrinsic motivation and the fun showed a significant effect of using the touch system over the buttons on the engagement and the motivation variables at once, with a Wilks’ \(\Lambda =0.917\), \(F(2, 72)=3.247\), \(p=0.045\), \(\eta _{p}^{2}=0.083\), \(\text {Power}=0.601\). The expressiveness condition, however, showed no combined effect on intrinsic motivation and fun, with a Wilks’ \(\Lambda =0.926\), \(F(2, 72)=2.871\), \(p=0.063\), \(\eta _{p}^{2}=0.074\), \(\text {Power}=0.545\). There was no significant interaction between the peripheral choice and the robot’s expressiveness for intrinsic motivation and fun combined with a Wilks’ \(\Lambda =0.975\), \(F(2, 72)=0.914\), \(p=0.405\), \(\eta _{p}^{2}=0.025\), \(\text {Power}=0.202\). The subsequent univariate ANOVA for the intrinsic motivation revealed a non-significant effect of the touch system over the buttons \(F(1, 73)=3.202\), \(p=0.078\), \(\text {Power}=0.423\), and neither when the expressiveness was introduced \(F(1, 73)=2.346\), \(p=0.130\), \(\text {Power}=0.327\), and also no interaction between these two factors \(F(1, 73)=1.792\), \(p=0.185\), \(\text {Power}=0.262\). These results reject \(\mathbf {H_{im1}}\), \(\mathbf {H_{im2}}\) and \(\mathbf {H_{im3}}\). For the fun variable, the univariate ANOVA showed a significant effect of the touch system over the buttons \(F(1, 73)=6.582\), \(p=0.012\), \(\eta _{p}^{2}=0.083\), \(\text {Power}=.716\) (validating \(\mathbf {H_{fun1}}\)) and the expressiveness \(F(1, 73)=5.799\), \(p=0.019\), \(\eta _{p}^{2}=0.074\), \(\text {Power}=0.661\) (also validating \(\mathbf {H_{fun2}}\)), but no interaction between these two factors \(F(1, 73)=20.918\), \(p=0.277\), \(\text {Power}=0.191\), rejecting \(\mathbf {H_{fun3}}\). Figure 6 shows a bar chart comparing the averages of the intrinsic motivation and total fun-dependent variables for each condition.

Fig. 6
figure 6

Charts with the average values for the intrinsic motivation (left) and the total fun (right). Significance levels are indicated: * for \(p <0.05\).

Full size image

We used two Independent-Samples Mann-Whitney U Test for the engagement dependent variable, one for the peripheral and another for the expressiveness conditions. The first test indicated that the touch system group had a significantly higher engagement level than the group that used the buttons with a Mann-Whitney \(U=948.5\), \(p=0.034\), \(d=0.501\), \(\text {Power}=0.685\). However, this was not true when comparing the groups based on whether the robot’s expressiveness was present, with a Mann-Whitney \(U=875\), \(p=0.171\), \(\text {Power}=0.401\), thus rejecting \(\mathbf {H_{eng2}}\). A subsequent two-way ANOVA supported these results, showing a significantly higher level of engagement for the touch system factor \(F(1, 73)=5.098\), \(p=0.027\), \(\eta _{p}^{2}=0.065\), \(\text {Power}=0.606\), but not for the expressiveness factor \(F(1, 73)=2.409\), \(p=0.125\), \(\text {Power}=0.334\). Therefore, the combined U-test and ANOVA results proved \(\mathbf {H_{eng1}}\). Lastly, concerning the \(\mathbf {H_{eng3}}\) hypothesis, the ANOVA did not show an interaction between the peripheral choice and the inclusion of the expressiveness over the level of engagement \(F(1, 73)=0.154\), \(p=0.696\), \(\text {Power}=0.067\), which rejects it. A descriptive bar chart comparing the averages of the engagement variable for each condition is shown in Fig. 7.

Fig. 7
figure 7

Charts with the average values for engagement. Significance levels are indicated: * for \(p < 0.05\).

Full size image

Qualitative results

Of the 83 participants, 44 used the ‘Comments’ 3 of the questionnaire, consisting of an open text box, to provide qualitative feedback on the experiment. The main aspects commented on were related to (i) the application’s usefulness, (ii) the interaction platform (robot and peripherals), (iii) the design of each tested application’s design and (iv) the instructions’ wording. Eight of the participants mentioned the touch interaction scenario positively, with an emphasis on human–robot interaction. Comments such as, “It was straightforward and convenient to touch it and listen to the answers. I like and find it interesting to touch the robot. Two pieces of iron that I can touch and hug, I loved it” (\(ID=025\)). Another comment was, “I found it very nice to touch it. Touching it feels like you get familiar with it; it is going to be easier” (\(ID=027\)).

We also gathered feedback regarding the design and use of both platforms. For example, some of the users who played using the button peripheral commented that the tone of the sounds, the intensity of the light or the speed of the game should be improved. On the other hand, the participants who experimented with the touch system said that there are sometimes false positives in the recognition. For example, one subject commented, “I had some problems with the recognition of the tickling in the stomach, otherwise [the system is] quite reliable and correct” (\(ID=037\)). Finally, it could be observed that users under the conditions containing robot expressiveness (BE and TE) included in some of their comments that the robot was attractive. For example, one volunteer commented, “The eyes are very attractive. I felt very comfortable. Looking at the eyes attracts me. Depending on who the activity is for, it would be interesting. This little guy could visit me more often” (\(ID=026\)).

Discussion

The results of the experiments showed a significant increase in two parameters, engagement and fun, when users used the touch system to participate in the game. These results validated our hypotheses \(\mathbf {H_{eng1}}\) and \(\mathbf {H_{fun1}}\), indicating a positive effect on the user’s attitudes and behaviour when interacting with the robot physically rather than through the button system. The effect sizes for these comparisons were moderate to high, with \(d=0.501\) for engagement and \(\eta _p^2=0.083\) for fun, suggesting that the impact of the touch system was substantial. Additionally, the observed power for these effects was 0.685 and 0.716, respectively, indicating a reasonable probability of detecting these differences with the current sample size. The case of intrinsic motivation is more complex to analyse since, despite the refutation of hypothesis \(\mathbf {H_{im1}}\) in the univariate ANOVA, a significant increase could be seen when the interaction of this variable was analysed together with fun through a MANOVA. This analysis was supported by the fact that both dependent variables were correlated. However, the observed power was low (0.423), suggesting that the lack of significance might be partially due to insufficient statistical power rather than the absence of a real effect.

The effect of expressiveness was significant only in terms of the fun experienced by the user (proves \(\mathbf {H_{fun2}}\) but rejects \(\mathbf {H_{eng2}}\) and \(\mathbf {H_{im2}}\)). This observation is relatively new since in other contexts, such as learning environments, the robot’s presence was a distracting element that negatively impacted the user’s experience38. We made this observation by introducing more complex measures of fun through the FunQ questionnaire. In this case, expressiveness significantly influenced fun (\(\eta _p^2=0.074\)) with an observed power for this effect of 0.661, suggesting a moderate ability to detect this result reliably. Conversely, for engagement and intrinsic motivation, the power was lower (0.401 and 0.327, respectively), reinforcing the idea that these non-significant results may be due to sample size constraints.

Regarding \(\mathbf {H_{eng3}}\), \(\mathbf {H_{im3}}\) and \(\mathbf {H_{fun3}}\), there was no significant interaction effect between touch and expressiveness independent variables. This indicates that the effects of interface type and expressiveness activation on engagement, intrinsic motivation and fun are apparently independent. In other words, the impact of peripheral type on the dependent variables does not change depending on whether expressiveness is active or not, and vice versa. However, the observed power for these interaction effects was low (0.067 for engagement, 0.262 for intrinsic motivation, and 0.191 for fun), suggesting that the lack of statistical significance may be partially attributed to limited sample size rather than a complete absence of interaction effects. Future studies with larger samples could provide further insights into potential interactions between touch and expressiveness.

We also obtained relevant information through the open-ended question in which we asked participants for feedback on the experiment. In these comments, users could express their opinions about interacting with the user through the touch system. Some of the comments were positive, indicating that qualitatively, for them, interacting with the robot directly with games was pleasant and interactive. Some of the feedback also evaluated the performance of the touch system in some cases where there were problems detecting contact. This has allowed us to know that, although the performance in general terms has been good, there is still work to be done in terms of detection in some cases. In particular, the ATR, while effective in recognising most gestures, has limitations in terms of accuracy and response time when compared to the button interface. Previous studies12 have reported an average recognition accuracy of 86% for ATR-based touch gesture classification, meaning that occasional misclassifications occur. These misclassifications were especially notable in gestures that involved more subtle or prolonged contact, such as tickling, which some participants reported as not always being reliably detected. Additionally, response times for ATR are slightly longer compared to direct button input, as the system requires real-time audio processing to classify gestures. While participants did not explicitly mention this delay as an issue, it remains an aspect to consider in future system iterations. Regarding the other peripheral, the users also appreciated the button game, although some participants indicated they found it monotonous sometimes.

We believe that some elements in the work help contextualise it and might also help design future follow-up studies. Although the results showed that the introduction of touch in human–robot interaction was a factor that significantly improved the interaction, the results have to be framed in the context of the memory game. In this sense, some factors could be considered in future work, such as knowing the game mechanics beforehand in cases where the participant is playing the touch game. The Simon game was particularly popular in the ’90s and 2000s; therefore, this may cause the participant to compare the touch-based game with the original version subconsciously. However, we tried to minimise this factor by hiding to these participants the buttons peripheral when collecting the data. Preference for memory games may also be a relevant factor and, in some cases, may lead to a more or less positive rating of the game. This factor could also be included in a future questionnaire because it has been used for this experiment.

Regarding participant distribution, participants were randomly assigned to the four experimental conditions, but this process resulted in some imbalance in the distribution of age groups across conditions. Examining the choice of peripheral condition, some age groups were relatively balanced. In the BN condition, younger participants (18–24) accounted for 5 (26.3% within the condition) participants, while the majority of the older adults (64+) were in this group 11 (57.9%). A similar trend was observed in the BE condition, where the 18-24 group had 7 (35.0%), but again, the older adults were overrepresented 10 (50.0%). However, other groups showed a more noticeable imbalance. In the TN condition, younger adults were more prevalent, with 7 (35.0%) in 18–24 and 7 (35.0%) in 25–34, while no participants from the older adults (64+) were assigned to this group. In the TE condition, the distribution was slightly more balanced, with 5 (27.8%) in 18–24, 4 (22.2%) in 25-34, and 6 (33.3%) in older adults (64+). Although the age distribution was not perfectly balanced, gender distribution was controlled. In BN, the gender split was 10 (52.6%) male, 9 (47.4%) female, while in BE, it was 9 (45.0%) male, 11 (55.0%) female. The TN condition was more skewed, with 15 (75.0%) male, 5 (25.0%) female, whereas TE was more balanced at 8 (44.4%) male, 10 (55.6%) female. Future studies should consider a more evenly distributed sample to improve the generalizability of the findings.

Also, while the experiment was oriented towards the general public, it was conducted in a single location, with participants who come from similar cultural backgrounds. It would be interesting to conduct further testing in several locations with a more diverse population sample. Conducting long-term testing would also help validate the results obtained by removing the novelty effect and assessing whether the positive effects of active touch diminish over time. One final limitation regarding the population of the study has to do with the lack of a specific target public. While evaluating our work with the general public gives us a more general understanding of the effect of active touch, our robotic platform has been designed with a specific user in mind (seniors with mild cases of cognitive impairment). Before implementing our system in the robot, it would be important to validate that the general results observed in this study apply to the population that will interact with the robot.

Conclusions

In this study, we conducted an investigation involving 83 participants to assess the impact of actively touching a robot during a play activity that heavily involves tactile interaction. The activity entailed memorizing and replicating a sequence that progressively increased in length and complexity. The first factor this study aimed to evaluate was the device utilized for gameplay, which featured two distinct conditions. Firstly, participants engaged in a button-based game inspired by the popular Simon game. Alternatively, participants interacted with the ATR touch system, executing a sequence of touch gestures in a specific order. Additionally, we also explored whether the expressiveness of the robot during gameplay could serve as a factor influencing the participants’ attitudes and behaviour. For that, we introduced two additional conditions: one showcasing expressiveness and reactivity during interactions with the user, while the other omitted this element. To evaluate the impact of both conditions on this aspect, we administered a questionnaire to measure participants’ engagement, intrinsic motivation, and fun during the activity.

The study’s results have shown, on the one hand, that interacting physically with the robot significantly affects the user’s attitudes and behaviour in terms of engagement and fun and, to a lesser extent, intrinsic motivation. The study also allowed us to prove that this effect occurs independently of the expressiveness and interactivity displayed by the robot. In addition, this work was a test for the ATR touch system in the context of real-time social touch research, being a valuable tool for touch-interaction-related experiments. The touch system made conducting a social touch experiment possible mainly through the robotic platform without employing mock-ups or teleoperation. Finally, another advantage arose through integrating the touch system and creating the memory game: we could completely replace the button system, allowing us to reduce the number of external devices the robot requires to provide fun and interactivity to the user. The results from this study also unveiled possibilities for future work. Firstly, more playful activities could be designed that use the touch system and test other user skills, such as games that test the user’s reflexes. From this work, it could also be explored whether direct interaction with the robot through such games can positively affect cognition, extending the study to participants with different levels of cognitive impairment. The study could also be improved by integrating different systems (e.g. vision-based perception systems) to assess whether the user is engaged during these tests or by extending it to assess effects related to the participant’s cultural background, age or gender.