Shared human-machine control of an intelligent bionic hand improves grasping and decreases cognitive burden for transradial amputees

Trout, Marshall A.; Mino, Fredi R.; Olsen, Connor D.; Hansen, Taylor C.; Teramoto, Masaru; Warren, David J.; Segil, Jacob L.; George, Jacob A.

doi:10.1038/s41467-025-65965-9

Download PDF

Article
Open access
Published: 09 December 2025

Shared human-machine control of an intelligent bionic hand improves grasping and decreases cognitive burden for transradial amputees

Nature Communications volume 16, Article number: 10418 (2025) Cite this article

6276 Accesses
815 Altmetric
Metrics details

Subjects

Abstract

Bionic hands can replicate many movements of the human hand, but our ability to intuitively control these bionic hands is limited. Humans’ manual dexterity is partly due to control loops driven by sensory feedback. Here, we describe the integration of proximity and pressure sensors into a commercial prosthesis to enable autonomous grasping and show that continuously sharing control between the autonomous hand and the user improves dexterity and user experience. Artificial intelligence moved each finger to the point of contact while the user controlled the grasp with surface electromyography. A bioinspired dynamically weighted sum merged machine and user intent. Shared control resulted in greater grip security, greater grip precision, and less cognitive burden. Demonstrations include intact and amputee participants using the modified prosthesis to perform real-world tasks with different grip patterns. Thus, granting some autonomy to bionic hands presents a translatable and generalizable approach towards more dexterous and intuitive control.

Multichannel haptic feedback unlocks prosthetic hand dexterity

Article Open access 11 February 2022

Slip-actuated bionic tactile sensing system with dynamic DC generator integrated E-textile for dexterous robotic manipulation

Article Open access 30 July 2025

A lightweight prosthetic hand with 19-DOF dexterity and human-level functions

Article Open access 22 January 2025

Introduction

Commercially available bionic hands can now replicate many movements and grip patterns of our innate biological hands^1,2,3. However, controlling a multiarticulate bionic hand is not an easy or intuitive task; individuals with hand amputation abandon these commercially available bionic hands at an alarming rate^4,5, often citing poor control^6,7 and cognitive burden^8,9 as primary reasons.

Most commercial prostheses are controlled using pattern recognition or dual-site control that requires the user to select among predetermined grip patterns, often with a fixed force output. In contrast, our native hands are capable of seamlessly transitioning between numerous grip patterns and generating a wide range of forces uniquely tailored to individual objects. This dexterity is made possible by an intricate neural encoding of hand control in terms of joint position^10,11,12. Ideally, prosthesis users could similarly control the individual position of each prosthetic digit to generate any arbitrary grasp and exert any desired force. However, simultaneous and proportional control of multiple degrees of freedom is difficult to achieve; as the number of degrees of freedom increases, errors compound, cognitive burden grows, and task performance decreases^13,14.

How, then, does the brain fluidly control the human hand, which features over 20 biomechanical degrees of freedom^15,16? One approach taken by the human nervous system is to simplify higher-level control by reducing the dimensionality of the movements on a task-by-task basis^12,17,18,19. Another approach is to utilize cutaneous feedback to automate lower-level grasping capabilities like the slip-detection reflex^{20,21,22,23,24,25}. These biological tactics rely on multimodal sensory feedback from the environment – visual and somatosensory.

To this end, one biomimetic approach to make bionic arms more dexterous and intuitive would be to leverage multimodal sensors to aid with some aspects of manual dexterity, thereby offloading some of the cognitive demand to the bionic arm itself. Indeed, researchers have used optical sensors to preposition robotic hands^26,27,28,29, force sensors to guide force output^30,31, and even acoustic sensors to detect and prevent slip³¹. These prior works demonstrate that even a single sensing modality can improve functional outcomes on specific tasks in controlled environments. However, among all of these works, there is a distinct lack of testing with users performing real-world tasks with physical prostheses. Additionally, the algorithms used to leverage multimodal sensor data to offload aspects of dexterity are often task-specific and unable to generalize to the diversity of objects and grasps seen in the real world.

One common approach is to develop a semi-autonomous prosthesis where control switches between the human user and an autonomous machine controller. For example, Zhuang et al. developed an autonomous machine controller that moved the individual fingers of a virtual prosthesis to maximize finger contact with virtual objects³⁰. The human user controlled the prosthesis with surface electromyography (EMG) to initiate initial contact with the object, at which point the autonomous controller would take over and move the digits autonomously to create a multidigit grasp with a fixed force output. Although this approach was shown to improve grip stability and functional performance, it results in a single fixed force being applied to all objects, making it unlikely to generalize to broader activities of daily living. Indeed, Zhuang et al. noted this limitation and hypothesized that continuously sharing control between the human and machine, rather than toggling, would improve functionality and make the control more intuitive for the user³⁰.

To this end, another common approach to share control between a human and a machine is to blend the human control and the machine control using a simple linear coefficient, β. Under this approach, the prosthesis position is determined by human control multiplied by β plus the machine control multiplied by (1-β). For example, Dantas et al. used this approach with a virtual prosthesis to demonstrate improved control on a virtual target touching task³². However, this approach requires the machine to be able to consistently provide a meaningful goal, not just under controlled conditions when triggered by the user. Indeed, Dantas et al. punted this issue by simply setting the machine goal equal to the location of the virtual target. It remains an open question as to how an autonomous machine could use real-world sensors to provide a meaningful control signal to users at all times across a range of tasks.

Here, we build on these past works by incorporating multimodal sensors, capable of detecting both proximity and pressure, into the fingertips of a commercial prosthetic hand for generalizable and translational use. We also establish a new autonomous closed-loop machine controller for the physical prosthesis to enhance grasping precision and stability. Importantly, to support generalizability, we introduce a novel shared control framework that continuously blends human and machine intent throughout the grasp, such that the human remains in control at all times while still receiving subtle assistance from the machine. In stark contrast to prior works, we show that this shared control framework allows the machine to assist the user with a variety of real-world tasks and grasps that demand unique positions and force contributions from each digit. Importantly, as a further demonstration of generalizability and translational potential, we demonstrate greater grip precision, improved grip security, and reduced cognitive burden through a series of experiments with multiple intact-limb participants and four transradial amputees using a physical prosthesis. These results serve as proof-of-concept that sharing control between the user and an autonomous machine can provide immediate functional benefits while decreasing the cognitive cost of using a prosthetic hand. As such, this work constitutes an important step towards more intuitive and dexterous control of multiarticulate bionic arms and has broad implications for prosthetic and robotic arms.

Results

Multi-modal finger-tip sensors allow for zero-force contact detection

Our innate biological fingertips are innervated by a dense network of sensory afferents that can detect even the gentlest interactions with our environment. Our sense of touch also enables instinctive motor actions that enhance our dexterity. However, given the delay associated with cutaneous reflexes²⁴, our dexterity is not simply due to sensory feedback³³. Indeed, our dexterity is also made possible in part due to internal models in the brain that simulate and anticipate hand-object interactions, and that update with experience over time³⁴.

In contrast, most commercial prostheses are not sensorized and cannot provide the user with the feedback necessary for efficient motor planning and learning. To bridge this gap, we developed a multimodal sensor for bionic hands capable of “seeing” and “feeling” objects. We developed a printed circuit board (PCB) in the shape of a fingertip that contained an infrared proximity sensor (VCNL4010, Vishay Semiconductor) and a barometric pressure sensor (MS5637-02BA03, TE Connectivity) (Fig. 1A). We then encased the PCB in silicone in a form factor that mimicked the removable fingertips of a widely used commercial prosthesis (TASKA Hand, TASKA; Fig. 1B). The fingertips of the prosthesis were then replaced with our custom sensors to provide wireless proximity and pressure data from each digit in real-time (Fig. 1C).

**Fig. 1: Sensorized fingertips can detect pressure and proximity.**

As an initial test of the sensors, a cotton ball with essentially no weight was dropped onto the fingertip while recording the proximity and pressure readings (Fig. 1D). A sudden rise in the proximity sensor reading corresponded with the cotton falling onto the sensor, validating the ability to detect proximity. The steady-state elevated proximity signal validated the ability to also use proximity as a near-zero force contact signal. Being able to detect near-zero force contact is crucial if one wishes to emulate the precision and dexterity of the intact human hand, as our hand control is based heavily on feedback from highly tuned mechanoreceptors that encode subtle and fast deformations in the skin³⁵. Finally, increasing proximity and pressure data when applying external force to the cotton ball validated the ability to detect external forces on the hand. Further testing demonstrated that the proximity sensor had a range of 0–1.5 cm, and the pressure sensor embedded in the silicon could measure up to 35 N (Supplementary Fig. 1). Additionally, the use of pressure sensors embedded in silicone enable the sensors to detect forces applied in different directions, as opposed to load cells that only accurately measure force in specific directions³⁶.

Machine control grasps objects with minimal force

With our native hands, we are exquisitely proficient at exerting just enough pressure on an object to grasp it³⁷. In contrast, commercial prostheses tend to select among predetermined grasps and flex until reaching a preset position or motor current, regardless of the situation or object being grasped. To address this, we developed an autonomous grasping controller that automatically grasps objects with minimal force using the embedded proximity and pressure sensors as feedback (Fig. 2A). Once an object was detected as being in proximity to a finger, that digit moved toward the object until pressure was detected, at which point the digit would hold its position. To retain control over the bionic hand and ensure participants could still release objects, participants enabled and disabled the autonomous controller by contracting their extrinsic hand flexors or extensors, respectively.

**Fig. 2: Machine control outperforms human control of bionic hand.**

We evaluated the performance of this autonomous “machine” controller against a more traditional myoelectric “human” control strategy on a fragile-object transfer task^38,39. This fragile-object task requires an individual to generate enough grip force to pick up and transfer the object, while also minimizing their grip force to avoid “breaking” the object (Fig. 2B). To ensure the human control was capable of producing the variable force necessary to complete the task, the comparative “human” control benchmark was a one degree-of-freedom (DOF), myoelectric proportional controller. To this end, a modified Kalman filter⁴⁰ was used to regress the kinematic position of the thumb using surface electromyography (sEMG) measured at the forearm using a compression sleeve with embedded electrodes⁴¹.

Under machine control, the hand successfully detected the object and moved the fingers to the point of making contact and exerting measurable pressure before stopping (Fig. 2C). Relative to human control, machine control reduced the normalized pressure measured by the sensor embedded in the hand (53 ± 24% vs. 40 ± 13%; p < 0.001; generalized linear model; Fig. 2E), suggesting the machine exerted less force on the object. This is further supported by the fact that, with machine control, participants were more successful at transferring the fragile object over the barrier without breaking (23 ± 21% vs. 99 ± 3%; p < 0.001; permutation test; Fig. 2D).

We also surveyed the participants on the subjective workload required for machine and human control after the fragile-object transfer task using the NASA Task Load Index (TLX)⁴². As expected, participants reported less subjective workload with machine control (70.70 ± 20.93 vs. 21.15 ± 7.72; p < 0.01; permutation test; Fig. 2F).

Surprisingly, machine control resulted in a greater physical burden to the user, as indicated by greater mean absolute sEMG activation (0.34 ± 0.13 µV vs. 0.62 ± 0.18 µV; p < 0.001; generalized linear model; Fig. 2G). This is likely attributed to the binary sEMG threshold used to toggle the machine control on and off; consistently turning on and off the machine control with distinct muscle contractions during quick transfers requires more energy than proportional control, which uses small muscle activation to produce gentle forces.

From a practical perspective, this experiment demonstrates that multimodal sensors can be embedded into a commercial prosthesis to create a semi-autonomous prosthesis capable of automating dexterous tasks like fragile object manipulation. However, as noted in the introduction and in prior works³⁰, simply toggling between human and machine control limits the user to a single fixed force output determined by the machine. Thus, although this is a compelling proof of concept that a machine can aid in a dexterous task, it is unlikely to generalize to real-world use, where different objects demand different forces. To this end, we next explored how control of a bionic hand could be continuously shared between the human and machine to enable more generalizable control and translatable control.

Control is shared between the user and the autonomous bionic hand

Although the autonomous machine control presented in the previous section outperformed human control, it prevented the user from intentionally modulating their grip force. Similarly, prior work using embedded sensors to minimize grip force provided only one set of target forces for applying pressure on grasped objects³⁰. In contrast, our native hands produce different grip forces for different tasks: heavy and slippery objects are gripped with more force than are light, high-friction ones³³. Thus, user-modulated forces are vital to make autonomous control applicable to a wide range of manual activities³⁰.

To this end, we developed a novel shared control framework that continuously merges human and machine intent. Our framework is inspired by the intact biological hand’s ability to seamlessly conform to an object, as opposed to prosthesis users who rely on their vision to properly grasp objects⁴³. The shared control framework leverages a machine controller to conform each independent finger to make minimal-force contact with an object (Supplementary Fig. 3, Supplementary Video 1) while allowing the human user to modify the grasp about the point of contact by controlling the position of each digit decoded from surface EMG. By continuously sharing control, the human maintains the ability to regulate their grasp while offloading the conformation of the prosthetic hand around a given object to the machine. Importantly, the human user controls each independent digit simultaneously and proportionally, a task which would otherwise be error-prone and cognitively burdensome without machine assistance (Fig. 3A)^40,44.

**Fig. 3: Machine assistance using shared control.**

In contrast to prior works that toggle between human and machine control³⁰, or use a fixed proportion of human and machine control^32,45, our shared control framework uses a dynamically weighted sum of human and machine control that is dependent on the relative position of each digit (Fig. 3B). The control of the hand is shared continuously such that neither the machine nor human is entirely in control of the hand and the human’s control becomes more precise as the machine moves to contact smaller objects. For example, when an autonomous digit contacts a small object at 75% of its range of motion, the human’s intent is then remapped to the remaining 25% range of motion, thereby quadrupling the effective precision (Fig. 3C). Since we naturally exert only the minimum amount of force necessary to hold an object securely with our native hands³³, we further amplified human precision by mapping human intent exponentially to the remaining kinematic range of motion (Supplementary Fig. 4). Such control is consistent with our endogenous recruitment of motor neurons⁴⁶. This shared control framework was used for all the following experiments. No parameters were tuned per experiment or object; the same framework was explored across multiple grips and tasks to quantify generalizability and translatability.

To establish a machine controller that can autonomously conform the grasp to objects (Supplementary Fig. 3, Supplementary Video 1), we trained a multilayer perceptron (MLP) to estimate the kinematic distance to an object on a digit-by-digit basis using the embedded proximity and pressure sensors (Supplementary Fig. 4). The MLP predicted kinematic distance with high accuracy throughout the entire kinematic range of the sensors. The MLP predictions were then low-pass filtered and added to the current position of each digit to serve as the machine intent such that the machine goal corresponds to the position at which the fingertip would make contact with the observed object (Fig. 3B). Both the proximity sensors and the pressure sensors are used to detect when contact is made, at which point the machine goal is frozen until contact is broken.

Human intent was the output of a modified Kalman filter⁴⁰ used to regress the kinematic position of each digit using surface EMG⁴¹. The Kalman filter has been used extensively to proportionally control prosthetic hands with multiple independent DOFs^40,47, but control becomes jittery, error-prone, and cognitively burdensome when controlling multiple DOFs simultaneously^14,44. The resultant shared control allows for quick and precise position control of each digit individually, such that the user can produce a wide range of forces on a given object (Fig. 3D). Allowing the user to control the position of the hand, in contrast to directly controlling force output, is consistent with the brain’s natural position-based encoding scheme for the hand⁴⁸.

Shared control improves grip precision

As an initial validation of our shared controller, we had participants repeat the fragile-object transfer task described before (Supplementary Video 2). This time, however, we embedded a strain gauge into the “fragile” object^49,50 to track the cumulative force exerted on the object by the hand (Fig. 4A). Measuring the grip force enabled us to analyze the range of grip forces under shared or human control, rather than just whether the participant exceeded a predetermined force. Grip force measurements from the “fragile” object were used solely for analysis purposes and were not used by the controller in any way. A diverse range of forces exerted on the object would imply the user can still modulate their grip force, and improved performance on the task would suggest the machine is still meaningfully contributing.

**Fig. 4: Shared control improved grip precision during fragile-object transfer task.**

As expected, forces recorded during the task showed participants were able to produce a wide range of grip forces under shared control (Fig. 4B). Participants approached the object while opening their hand, and then grasped around the object. Once the object was in proximity to the digits, the machine automatically adjusted its target position to the exact joint position necessary to make contact with the object. This stabilized the user’s grip around the object, effectively offloading the positioning of the digits to the machine and allowing the user to focus exclusively on the force they intended to produce. Indeed, we observed multiple events in which the user’s intent to minimize force on the object would have resulted in dropping or breaking the object, but the machine’s contribution to the shared control maintained a secure grip on the object (Fig. 4B). In comparison, user’s attempting the task with human control struggled to successfully transfer the object without breaking or dropping it as relatively small changes in the decoded position resulted in large changes in the grasp force (Supplementary Fig. 5A).

Relative to human control, shared control resulted in greater task performance, implying the machine contribution improved grip precision. As opposed to human control, shared control increased the participant’s success rate when transferring the fragile object without dropping it or exceeding the break threshold (59 ± 23% vs. 89 ± 10 %; p < 0.01; permutation test; Fig. 4C). Participants produced a similar variety of forces under shared control and human control (Fig. 4D), but they tended to use less force under shared control, in line with the task requirements (9.68 ± 4.95 N vs. 7.60 ± 3.53 N; p < 0.001; generalized linear model; Fig. 4D). Participants transferred the object at a similar rate under each control strategy (10.08 ± 2.87 transfers per min vs. 9.04 ± 2.92 transfers per min.; p = 0.07; permutation test; Fig. 4E).

Shared control improves grip security

When grasping an object, we instinctively maximize the contact area with the object while minimizing our grip force to ensure a stable and efficient grasp⁵¹. Importantly, our native hands regulate grip force on a digit-by-digit basis with millisecond precision⁵². In contrast, commercial prostheses typically move all digits in synchrony, following a predetermined trajectory to form a specific grasp. This ‘robotic’ motion results in an unequal distribution of pressure across the digits and an unstable grip.

We therefore sought to next validate the grip security afforded by our shared controller on a holding task with a larger spherical object (Fig. 5A). With our native hands, we tend to grasp small rectangular fragile objects, like those used in our fragile-object transfer tasks, with a prismatic precision pinch, often involving only two digits (e.g., the thumb and index)⁵³. In contrast, for large spherical objects, we typically employ a spherical power grasp involving all of the digits⁵³. Thus, a holding task³⁰ involving a large ball assesses performance on the opposite end of the grasping spectrum, and similar improvements in performance would support the potential of the shared controller to generalize across a variety of activities of daily living.

**Fig. 5: Shared control improved grip security and reduced cognitive burden on a holding task.**

To this end, intact-limb participants were provided with control of each digit individually and were tasked to pick up and hold a large ball for two minutes (Supplementary Video 3). With shared control, participants would approach the ball while opening their hand, and then grasp the ball. As the digits came into proximity to the ball, the machine automatically adjusted its target position and continued to make positional adjustments for each digit as the participant fatigued or adjusted their grip over time (Fig. 5B). Such adjustments are consistent with the biological corrections in grip force seen in all phases of lifting and grasping an object²³. However, the same degree of control of the four independent digits was not observed with human control, as noise and cross-talk between sEMG channels make precise, stable control difficult (Supplementary Fig. 6).

Relative to human control, shared control resulted in greater grip security. When using shared control, participants were less likely to drop the ball (2.33 ± 3.50 drops vs. 0.25 ± 0.45 drops; p < 0.05; permutation test; Fig. 5C). and were able to hold the ball longer without dropping it (54.01 ± 46.02 s vs. 77.61 ± 45.47 s; p < 0.05; permutation test; Fig. 5D). Participants also tended to have more fingers making contact with the ball, although this was not statistically significant (2.40 ± 0.48 fingers vs. 2.73 ± 0.65 fingers; p = 0.28; permutation test; Fig. 5E).

Shared control reduces cognitive burden

Activities of daily living often involve dividing attention between multiple simultaneous subtasks; for example, holding a jar while twisting off its lid. As such, a top priority for prosthesis users is less attention-demanding control⁸. To this end, we had participants complete a secondary detection response task (DRT) while performing the fragile-object transfer task and the holding task. The DRT provides a simple yet sensitive measure of cognitive resource allocation based on response time to a stimulus⁵⁴, and has also been used to assess the cognitive load of prostheses⁵⁵.

For the holding task, shared control resulted in less cognitive demand than human control. Participants had significantly faster response times on the secondary DRT when using shared control (0.46 ± 0.26 s vs. 0.40 ± 0.26 s; p < 0.05; generalized linear model; Fig. 5F). While there are limited research studies using the DRT to quantify the cognitive burden of using a myoelectric hand⁵⁶, research into the effects of cognitive load on driving indicates that a change of 50 ms in the response times corresponds to a noticeably more difficult task, as human response times to the tactile DRT generally range from approximately 300 ms to 1 s^57,58,59. Without an additional task, the response time to the DRT was measured to be 250 ms (Supplementary Fig. 7). Given this, the measured decrease of 60 ms corresponds to a 29% decrease in cognitive burden with respect to the range of response times when using shared control, which is similar to the 30% decrease in cognitive load observed when switching from completing a task with a 1-DOF bionic hand to an intact hand (Supplementary Fig. 7). Shared control also resulted in less muscle activity, although this was not statistically significant (60.16 ± 29.57 µV vs. 46.96 ± 8.95 µV; p = 0.17; permutation test; Fig. 5G).

No significant differences were observed for cognitive or physical burden for the previously discussed fragile-object task (Fig. 4F and 4G; p > 0.05). This may be due to the time-based component of the task; transfers are performed so quickly that the user stays engaged throughout the entire task, making the contributions from the machine more subtle.

Shared control improves grip security and precision for upper limb amputee participants

Having validated the shared controller with intact-limb participants on two tasks involving different grips and forces, we next sought to validate the shared controller with upper-limb amputees. To reduce experimental time and document performance on yet another grip, we had four transradial amputees perform a fragile-object holding task, in which they picked up and held an instrumented fragile object^49,60 using an intermediate tripod pinch⁵³ for up to two minutes (Fig. 6A). The force regulation used in this task is akin to that used when holding a plastic cup of water without dropping or crushing it. The amputee participants also completed the secondary DRT during the fragile-object holding task to measure cognitive load.

**Fig. 6: Shared control improves grip security, enhances grip precision, and reduces cognitive burden for amputee participants.**

Relative to human control, shared control resulted in greater grip security and greater grip precision. Participants were able to hold the object without dropping or breaking it for a longer duration with shared control (7.81 ± 12.33 s vs. 51.56 ± 45.0 s; p < 0.01; permutation test; Fig. 6C). Participants were still able to produce a wide range of grip forces with shared control, but their average grip force was lower when using shared control (14.86 ± 4.54 N vs. 9.74 ± 4.27 N; p < 0.001; generalized linear model; Fig. 6D). The participants also dropped the object less with shared control, although this was not statistically significant (9.64 ± 6.46 drops vs. 4.64 ± 4.88 drops; p = 0.10; permutation test; Fig. 6E).

Shared control also reduced the cognitive burden of the amputee participants. Response times to the secondary DRT were significantly lower when using shared control (0.74 ± 0.41 s vs. 0.62 ± 0.38 s; p < 0.001; generalized linear model; Fig. 6F). This 120-ms change in their response times corresponds to a 24% decrease in cognitive burden by using shared control. To frame this change with respect to other real-world scenarios, a 120-ms improvement was also observed when drivers operated their car radio at a standstill versus operating their radio while driving⁶¹. A comparison between the intact-limb and amputee participant’s response times indicates that the amputee participants were slower to respond than the intact-limb participants (Supplementary Fig. 8), which could be attributed to the different task, or to their amputation or age. Consistent with the results from the intact-limb participants, we observed no difference in physical burden between the control strategies (49.89 ± 43.28 µV vs. 45.89 ± 39.0 µV; p = 0.54; permutation test; Fig. 6G).

Shared control generalizes to activities of daily living

An important concern with controlled laboratory experiments is whether the tasks used to assess the performance of the prosthesis are ecologically valid. For example, do improvements in manipulating fragile objects and holding oddly shaped objects translate to improved performance on activities of daily living (ADLs)? Inherent to that question is whether the shared controller can generalize to new and diverse conditions, as would be experienced with ADLs. We evaluated this by having the participants complete a variety of ADLs with shared control and human control (Supplementary Video 4). Improvements are difficult to quantify with ADLs, but the participants noted that shared control was particularly helpful for tasks involving precise hand positioning and force regulation. Three tasks stood out: bringing a disposable cup to their mouth, transferring an egg between two plates, and picking up a piece of paper (Fig. 7). Each of these tasks corresponded to one of the grasps used in the prior experiments. Drinking from the disposable cup used a power grasp (Fig. 5A), moving the egg used a tripod grasp (Fig. 6A), and moving the piece of paper was completed with a pinch grasp (Fig. 4A). One participant specifically noted that these tasks were incredibly difficult, if not impossible, with human control, but that shared control made the tasks reasonable. Indeed, with only human control, participants repeatedly broke/dropped the egg, crushed/dropped the cup, and crinkled/dropped the piece of paper; under shared control, they were able to consistently complete all the tasks (Supplementary Video 4).

**Fig. 7: Shared control supports activities of daily living.**

Occupational therapists often rate amputee patients’ performance on activities of daily living using subjective graded scales. For example, the Activities Measure for Upper-Limb Amputees (AM-ULA) scores amputees on 5 categories: extent of completion, speed of completion, movement quality, skillfulness, and independence. Although the activities completed with AM-ULA are not well-suited for next-generation prostheses capable of a delicate touch^39,62,63, the rate and speed of completion score criteria are quantitative and can easily be applied to other activities of daily living. To this end, we quantified the speed and success rate of one participant as they completed six trials of the drinking task with disposable cups, during which they attempted to pick up the cup from the table, bring it to their mouth and mime a sip, and then return it to the table. This real-world activity of daily living is considerably difficult. These common cups break with approximately 7 N; in contrast, the fragile holding task required the amputees to maintain forces below 15 N. Under human control, the participant crushed the cups in five out of the six trials and dropped it twice (Fig. 7D). However, with shared control, the participant successfully completed the task four out of the six times. No significant difference in the task speed was observed; the participant took 9.67 ± 3.27 s to make a grasp and lift the cup to their mouth with human control, and only 7.60 ± 2.59 s with shared control (p = 0.26; unpaired t-test; Fig. 7E).

Discussion

This work introduces a generalizable framework for enhancing the performance of bionic arm users through embedded multimodal sensors and autonomous control. We show generalizability through a variety of different grasps and tasks and facilitate translation through integration with a widely used commercial prosthesis and by validating the system with four amputees, a sizable cohort for the transradial population. Indeed, this work represents the first demonstration of shared control with multiple amputee participants using a physical prosthesis. Importantly, we show enhanced performance both in terms of physical and cognitive function; participants had better grip security, enhanced grip precision, and reduced cognitive burden with no increase in physical burden.

These results build on prior sensorized autonomous bionic arms by leveraging distributed multimodal sensors within a commercial prosthesis. In contrast to camera-based systems^26,27,28,29, our distributed proximity sensors require substantially less power for operation and data analysis. Prostheses should ideally operate continuously for an entire day; greater power requirements led to larger, heavier batteries, and weight is a critical priority for prosthesis adoption⁸. Distributed proximity sensors also provide more robust operating conditions. With a single camera, users must take time to methodically approach objects to ensure the camera has a line of sight to determine the position of each digit²⁹. In contrast, a distributed system provides several viewing angles and allows each digit to operate independently of the others.

In principle, the shared control algorithm presented here could operate with only a single modality that detects contact via fingertip pressure sensors already available in commercial hand prostheses^1,64,65. However, in practice, the multimodal configuration provided more robust control, given that sensor data is often non-ideal in the real world. Using pressure sensors alone requires the machine control to operate entirely in a reactive manner. The pressure sensors would need to be sampled at a much faster rate than the actuation speed of the prosthesis to ensure the machine stops quickly enough to provide near-zero contact force. In practice, this was not feasible due to the users’ desire to operate the prosthesis quickly, the momentum of the prosthesis once in motion, the delays associated with power-efficient sensor readings, and wireless communication. In contrast, the feedforward nature of the infrared sensors allows for a more seamless blending of the human and machine intent. Additionally, the infrared sensors have the added benefit of initiating movement on their own to maximize the number of digits making contact with the object. Combined, the multimodal infrared and pressure sensors provide both a feedforward and a feedback approach to ensure robust operation. With this in mind, the fingertip sensor modules used in this work have been designed to be retrofitted onto different commercial prosthetic hands, providing a straightforward means to translate the shared control algorithm³⁶.

Nevertheless, even the most advanced sensorized autonomous bionic arms are unlikely to capture a user’s intent perfectly. Prosthesis agency, the sense that you are the author of your actions, is an important factor for prosthesis design^66,67. To this end, shared control has the potential to maintain a strong sense of agency while offloading some of the cognitive burden of grasping. Here, we introduce a new shared control framework that uses a dynamically weighted sum to allow simultaneous and independent contributions from the user and the autonomous bionic arm. This approach was inspired by biological grasping that maximizes contact area while limiting grasping forces⁶⁸. This is a demarcation from prior works that toggle between human and machine control^29,30, or use a fixed proportion of human and machine control^32,45. Our decision to pursue dynamic weighting was guided by several failed attempts to leverage these past frameworks and past literature that suggested that continuously sharing control would improve performance³⁰. Toggling between human and machine control requires the machine to be able to operate autonomously at a level greater than that of a user. Given the variety of objects grasped and forces exerted in daily activities, this framework fails when attempting to generalize across multiple tasks. Indeed, the most complete shared control work to date noted this critical problem³⁰. The alternative, sharing some fixed proportion of control between the human and user, also failed to generalize well because it requires the machine to be consistently useful across a range of conditions. From our experience, fixed support from a machine designed to enhance grip security and grip precision (a challenge for modern prostheses) ultimately hindered performance on simple tasks (that would not otherwise be a challenge for modern prostheses). The dynamic framework introduced here allows for generalizable control by ensuring the user maintains agency over the bionic arm at all times.

One could imagine a few scenarios where the shared control framework proposed in this work might impede functionality; for example, when the machine control either lags or precedes the human’s intent. In theory, the machine controller could lag behind the human’s intent or fail to detect an object if the line of sight to all of the digits is limited. In this event, the user will have decreased precision until contact is made, at which point the machine’s goal will update, and the shared control scheme will operate as usual. In theory, it is also possible for the machine controller to precede the human’s intent. For this to happen, the user would have to reach for an object and surround the object with the prosthetic fingers, but not desire to grasp it. This scenario is uncommon and unusual in activities of daily living, where grasping and reaching are performed simultaneously⁶⁹. Nevertheless, in this event, the machine would attempt to move the fingers closer to the object to make initial near-zero contact with the object. Then the user could still achieve their intent, but would have to exert more physical and cognitive effort. Given the rarity and minimal impact of the machine lagging or preceding the human’s intent, we believe the shared control framework presented here provides a generalizable approach towards offloading some of the cognitive burdens of dexterous control while maintaining user autonomy and robust performance when working with non-ideal sensor data in the real world. These examples provide a rough baseline for how the system might behave under relatively simple conditions. However, the impact of more complex sensorimotor tasks, like opening a bottle cap or tying shoelaces remains an important question worthy of future research.

We demonstrated that the shared control framework can generalize to a variety of objects and grip patterns common in activities of daily living. Indeed, the tasks performed mimicked standard clinical tasks to assess patient functionality. For example, the fragile-object transfer task is effectively a more difficult version of the box and blocks tasks, and the holding tasks with different-shaped objects are similar to portions of the Southhampton Hand Assessment Procedure⁷⁰. However, this does not necessarily mean that shared control will generalize well to everyday real-world use. A long-term at-home clinical trial, along with modern clinical assessments, like the Activities Measure for Upper Limb Amputees (AMULA)⁷¹, is necessary to validate real-world generalizability and to demonstrate improvements in independence and quality of life. Such a study could also answer questions regarding the impact of shared control on prosthesis adoption and learnability.

Shared control resulted in immediate improvements in functional and cognitive performance in naïve users. An important question remains as to whether shared control can provide similar, or even greater, benefits to experienced users. As a simple first-order assessment of learning, we explored the change in performance for human control and shared control between the first and last experimental blocks of the fragile-object transfer task, holding task, and fragile-holding task (Supplementary Fig. 9). No learning took place under shared control; there were no significant differences for any of the tasks. Under human control, no learning took place for the holding task or fragile-holding task, but there was a significant improvement in success rate on the fragile object transfer task. Given the timeframe of the tasks performed herein, we suspect that the limited learning that did take place would mostly be attributed to task familiarization. Future work should explore long-term motor learning associated with shared control. Shared control could potentially improve motor learning, as decreased cognitive burden is associated with an increase in motor skill acquisition^72,73. In contrast, the machine contribution could also act as an additional hidden variable within the user’s internal model that impedes learning⁷⁴. Regardless of long-term outcomes, the short-term benefit of enhanced dexterity and reduced cognitive load could potentially improve prosthesis adoption during the critical golden window after amputation⁷⁵.

Among the amputee patients, shared control provided similar benefits to those who used a myoelectric prosthesis and those who used a body-powered prosthesis. Future work should explore the impact of shared control among myoelectric and body-powered users with a greater number of patients. Shared control is most straightforwardly applicable to myoelectric users since shared control could be readily implemented into commercial multiarticulate bionic arms like the TASKA hand used in this study. However, the ability of shared control to reduce the cognitive load of controlling a multiarticulate bionic arm may also make myoelectric control more appealing to patients who would have otherwise opted for a body-powered prosthesis. Although there are many differences between body-powered and myoelectric prostheses (e.g., weight), body-powered prostheses are often considered to require less attention, and this is a major contributing factor in prosthesis selection⁷⁶. Indeed, body-powered prostheses require less training time⁷⁷ and provide residual sensory feedback that has been hypothesized to reduce cognitive and visual demand⁷⁸. Given that shared control can reduce the cognitive demand of a myoelectric prosthesis, future work should explore the ability of shared control to reduce prosthesis abandonment among myoelectric users, as myoelectric prostheses have yet to improve prosthesis adoption over body-powered alternatives⁵.

Commercial myoelectric prostheses are usually controlled in an open loop with no explicit feedback. The few commercial prostheses with feedback typically use motor current to indirectly estimate and limit exerted force. However, there is a recognizable trend of more commercial bionic hands becoming more sensorized with multiple pressure sensors across the hand^1,64,65. Embedded pressure sensors allow for faster reactions and gentler forces and maintain functionality during passive interactions in which the motors are not active. As sensor arrays become more common in commercial bionic hands, shared control work will become more relevant and impactful. Related work has shown that integrated force sensors can also close the loop by driving haptic feedback provided directly to a user^{39,79,80,81,82,83,84,85,86,87,88,89}. The present work complements these related works by demonstrating another value of integrating force sensors into prostheses.

Commercial myoelectric prostheses are also typically controlled using either pattern recognition⁹⁰ or dual-site control^91,92, both of which require the user to sequentially select among grip patterns with a fixed force output⁹¹. In contrast, here we demonstrate simultaneous and proportional control of each digit in real-time. By eliminating the need to sequentially switch between grip patterns, simultaneous and proportional control has been shown to improve performance relative to pattern recognition for a 3-DOF prosthesis⁹³. Simultaneous and proportional control of more than 3 DOFs has been achieved previously using a modified Kalman filter⁴⁰. Indeed, a modified Kalman filter has been used to control up to 10 DOFs⁹⁴, deployed on computationally limited hardware^95,96,97, and used to complete various activities of daily living^39,40,98. Although promising, it is recognized that the performance and intuitiveness of simultaneous and proportional control algorithms decrease as the number of controllable DOFs increases, and this is also true for the modified Kalman filter⁹⁵. In the present work, we demonstrate the ability of shared control to enable simultaneous and proportional control of a greater number of DOFs while decreasing the overall cognitive burden.

An important question remains as to whether the shared control framework employed herein can be readily extended to other, more traditional, control strategies like pattern recognition or dual-site control. Explicit validation is required, but in theory, both pattern recognition and dual-site control could benefit from the shared control framework outlined here. Parallel dual-site control of multiple DOFs⁹³ is akin to the modified Kalman filter, and we would likely see similar improvements in grip security, grip precision, and cognitive load. In the case of pattern recognition, the shared control framework could increase grip security and/or precision by helping maintain digit contact and distributing forces more equally among the active DOFs, but would likely have minimal impact on cognitive load. Future shared control frameworks with pattern recognition in mind could leverage the embedded sensors to inform grip selections, which, if done well, could improve the speed and reduce the cognitive demand of coordinated multi-gesture tasks.

The Southampton Hand was one of the first prostheses to utilize integrated sensors to regulate grip force⁹⁹. The Southampton Hand was controlled by the user toggling a “touch” mode in which the hand would move towards an object and stop upon force feedback. The user would then regain control to trigger a subsequent increase or decrease in grip force as needed. In contrast, the present work leverages proximity sensors to optimize grip configuration and force prior to making contact and blends human and machine intent for precise position control rather than toggling between the human and machine for iterative refinement. During experiments with the Southampton Hand, researchers observed that participants had to dedicate less attention to using the Southampton Hand compared to a conventional myoelectric hand¹⁰⁰. Since then, other research has detected a decrease in cognitive load when using shared control via the NASA TLX survey²⁸. The present work also builds on these past works by quantifying a reduction in cognitive load through a nonsubjective metric, the detection-response task. Notably, the detection response task used in the present work provides an objective measure of cognitive load, establishing a benchmark for the intuitiveness of prosthetic control strategies. More intuitive control may ultimately increase prosthesis usage, given that poor and unintuitive control are primary reasons for abandonment⁶.

Methods

Human testing

Nine intact-limb participants (22.3 ± 2.40 years old; 44% female) and four participants with transradial amputations (45.75 ± 14.36 years old; 100% male) were recruited for this study. The intact limb participants were 89% right-hand dominant, and the amputee participants were 100% right-hand dominant before amputation. Table 1 contains further details about the amputee participant population. None of the intact limb participants had any prior experience controlling myoelectric devices. Informed consent, experiment protocols, and patient image collection were carried out in accordance with the University of Utah Institutional Review Board (IRB 00098851), the Department of the Navy Human Research Protection Program, and the Declaration of Helsinki.

Table 1 Participant Information

Full size table

Signal acquisition

Surface electromyography (sEMG) was collected from the participants using 32 sEMG electrodes. sEMG was sampled at 1 kHz and filtered using the Summit Neural Interface Processor (Ripple Neuro Med LLC). sEMG signals were band-pass filtered with a lower cutoff frequency of 15 Hz (sixth-order high-pass Butterworth filter) and an upper cutoff frequency of 375 Hz (second-order low-pass Butterworth filter). Notch filters were applied at 60, 120, and 180 Hz. Differential sEMG signals were calculated for all possible pairs of channels, resulting in 496 differential pair recordings⁴⁰. The mean absolute value (MAV) was then calculated for the 32 single-ended and 496 differential recordings at 30 Hz using an overlapping 300-ms window⁴⁰. sEMG was measured using a 32-channel, dry electrode compression sleeve for intact-limb participants⁴¹ and from 32 adhesive electrodes (Nissha Medical Technologies) placed in a grid pattern on the residual forearm for amputee participants. The embedded fingertip sensors were sampled at 30 Hz and passed through a median filter with a time window of 3 samples.

Sensorized fingertips

The sensorized fingertips (Point Designs, Broomfield, CO, USA) contained a barometric pressure sensor (MS5637-02BA03, TE Connectivity) and a proximity sensor (VCNL4010, Vishay Semiconductor)³⁶. The proximity sensors operate by emitting a specific wavelength of infrared light and measuring the intensity of any reflections. The sensors are mounted in a plastic housing and embedded in silicone. The proximity sensor is capable of detecting surfaces up to 1.5 cm away. The pressure sensor embedded in the silicone is capable of reliably detecting forces less than 1 N and up to 30 N (Supplementary Fig. 1). Each of the fingertip sensors communicates with a central microcontroller (ESP32, Espressif, Shanghai, China) mounted on the back of the prosthetic hand. The housing of the fingertip sensors was designed in multiple form factors to fit different commercial bionic hands, but only the TASKA form factor was used in this work.

Bionic hand interface

For intact-limb participants, the sensorized bionic hand was mounted to a bypass socket that was worn on the left arm¹⁰¹. For amputee participants, the sensorized bionic hand was mounted to an adaptable 3D-printed functional test socket¹⁰². After applying the sEMG electrodes, the residual limb was wrapped using a self-adherent wrap to apply compression and manage cables. The adaptable socket was then placed over the wrapped electrodes and secured by wrapping an elastic bandage around the socket to provide compression and around the bicep of the upper arm to prevent slipping. All participants used a left-handed TASKA hand (TASKA, Christchurch, New Zealand).

Initial fragile transfer task with autonomous machine control

To assess how incorporating information from the sensorized fingertips might affect bionic arm control, three intact limb participants completed a fragile-object transfer task with a mechanical fragile object¹⁰³ (Fig. 2B). The fragile-object transfer test is a derivative of the widely used box and blocks test of hand dexterity¹⁰⁴ in which the participants must limit the grip force applied to the block. The fragile-object test provides additional information beyond the box and blocks test and has been used extensively to validate the fine dexterity of prostheses^39,105,106. The transfer task involved picking up a single fragile object and translating it over a barrier without exceeding a predetermined grip force. Once the object was broken, dropped, or successfully transferred, or if the trial timed out, then the trial was ended, and the object was reset. Participants had up to 45 seconds to complete the trial with a verbal warning when 15 seconds remained. The barrier was 62.5 mm tall.

The mechanical fragile object consisted of a steel plate, a steel lever, a magnet that held the steel lever in place, and a weight. When grasped above the holding force of the magnet, the fragile object would “break,” and the lever would emit a clicking sound. The break force could be adjusted by sliding the magnet along the lever. The object weighed 496 g.

Each of the three intact-limb participants completed the fragile object transfer task using a prismatic precision pinch grasp with human control and with autonomous machine control. Autonomous machine control was engaged when the participant flexed and disengaged when the participant extended their hand. When autonomous machine control was engaged, the fingers would close when a surface was detected by the proximity sensors and stop when contact was detected via the pressure sensors (Fig. 2A). A proximity measurement of 30 indicated that an object was detected, and a pressure measurement of 30 indicated that contact had been made.

The experiment consisted of the participants completing two blocks of the fragile object transfer task with each control scheme, for a total of four blocks, in a pseudo-randomized counterbalanced format. Each block consisted of 10 trials. After the second block of each control scheme, participants completed the NASA TLX survey. Each participant completed the experiment twice, once with a break force of 19 N and another with 17 N (Supplementary Fig. 2). Data from the two experiments were aggregated. The pressure readings from the fingertips from each trial and the success rate from each block were recorded as measures of grip precision. Due to the unitless nature of the pressure sensors, pressure measurements were normalized by the maximum recorded value for each participant. The sEMG MAV was recorded and averaged across channels and each trial as a measure of physical exertion, and the NASA TLX served as a measure of perceived workload.

Shared control algorithm

Machine control

The machine control algorithm predicts the kinematic position of each digit in real time using proximity and pressure values from a sensorized prosthetic hand. A left-handed TASKA hand was retrofitted with fingers containing pressure and infrared proximity sensors (Point Designs LLC, Lafayette, CO, USA) (Fig. 1A)^26,107. A multilayer perceptron (MLP) was designed with a single hidden layer to predict object distance from the proximity and pressure sensors¹⁰⁷. Training data for the MLP was collected by oscillating each digit towards and away from 3D-printed objects. The ground-truth distance from the object for the training data was computed by measuring the kinematic position of the finger upon initial contact with the object (i.e., when pressure was first recorded) and then retroactively using the difference between that kinematic contact position and the current position to determine the current distance to the object

The machine's position goal was the predicted kinematic position of each digit that would result in contact with the object, computed as the sum of the current position of the digit and the predicted distance to the object. A first-order lowpass filter with a cutoff frequency of 90 Hz was used to smooth the resulting predicted kinematic position for the machine goal, ${u}_{m}$ (Fig. 1B). The digits would only move to ${u}_{m}$ if the proximity sensors exceeded three times the standard deviation of their noise. A proximity measurement of 30 indicated that an object was detected. This threshold was implemented to prevent the machine control from responding to random noise. Once contact was detected, the machine predictions were frozen until the grasp was released to prevent undesired jitter. Contact was defined as the pressure measurement exceeding 30 or the proximity measurement exceeding 1000 (Supplementary Fig. 1A and B). By developing the machine controller around achieving contact, rather than modulating force, there is no dependence upon the range of the proximity or pressure sensor beyond the detection of contact. The respective thresholds were set such that the fingers would touch a given object but not exert a measurable force on it. ${u}_{m}$ was limited to a range of −1 to +1, where −1 represents maximum digit extension, and +1 represents maximum digit flexion.

Human control

The human position goal, ${u}_{h}$, was computed using a modified Kalman filter (MKF)⁴⁰. The MKF was trained to correlate sEMG features measured from the forearm to kinematic positions. Training data was collected as the participants actively attempted to move their (intact or amputated) hand alongside preprogrammed movements of the prosthesis. Preprogrammed movements consisted of both flexion and extension for the thumb, index, middle, ring, and pinky fingers. Participants completed four trials of flexion and four trials of extension for each digit. The resultant MKF provided the user with proportional position control of all five digits on the TASKA hand. sEMG features used for estimating motor intent consisted of the 300-ms smoothed mean absolute value (MAV) of 528 sEMG channels (32 single-ended channels and 496 calculated differential pairs) calculated at 30 Hz. To reduce computation complexity, the 528 sEMG features were down-selected to 48 sEMG features via Gram-Schmidt orthogonalization^45,108. The human position goal, ${u}_{h}$, was limited to a range of −1 to +1, where −1 represents maximum digit extension, and +1 represents maximum digit flexion. A nonlinear, latching filter was applied to the human position goal to limit jitter and smooth the human intent for the ADL assessment⁴⁴.

Shared control

We aimed to develop a shared-control algorithm that continuously blends human and machine control. Unlike prior work, where control of the hand was toggled between human and machine control^26,29, the shared-control law is fixed³², or the grip force is limited³⁰, the shared-control algorithm we present in this paper uses machine control to modify human control dynamically while still allowing the human to modify the grip force¹⁰⁷. The motivation for this design stems from the idea that the machine controller should continuously assist the human operator with the grasp, but never be in complete control of the hand.

The shared control framework was designed such that the machine control would conform each digit to the surface of a detected object. If no object was detected, then the user had full control of the hand, enabling them to flex the fingers towards an object if the sensors failed to detect the surface. Once the machine controller detected the object, it would move the fingers to make contact with the surface and hold its position. The user could then flex and extend the grasp about the point of contact to make a tighter or more relaxed grasp. The mathematical implementation of this design is explained below.

Control of each DOF was shared between the human position goal, ${u}_{h}$, and the machine position goal, ${u}_{m}$. The shared goal, ${u}_{s}$, was computed as the following:

$${u}_{s}={u}_{m}+\left(1-{u}_{m}\right)\frac{{e}^{\alpha {u}_{h}}-1}{{e}^{\alpha }-1}$$

(1)

This effectively attenuates the human’s control of the hand in proportion to the machine’s control output. The human control signal is also modified by a normalized exponential to give the user greater precision during small movements in exchange for less precision during large movements. The exponential attenuation was implemented to assist the user in making finer motions and to limit the user from over-flexing. The damping parameter α was set to α = 3 for intact limb participants, such that a human position goal of 0.5 would be remapped to approximately 0.2. Preliminary data indicated that a more aggressive scaling could further improve control, so for amputee participants, α was set to α = 4, remapping a human position goal of 0.5 to approximately 0.1 (Supplementary Fig. 4C). The scaling factor was increased to α = 7 for the ADL testing.

Because shared control is only useful for grasping, a kinematic threshold was used to disable the shared-control algorithm during extension²⁶. For example, if the MKF predicted that the human was attempting to extend to a position greater than 75% of the extension range, shared control was disabled to allow fluid hand opening. To return to the shared-control state, the user had to flex 1% of the flexion range. The 1% threshold was chosen so that shared control would be effectively re-enabled as soon as the user stopped opening the hand. This aided the participant in releasing the object without altering the evaluation of the shared controller’s performance during grasping and gave the user the ability to extend DOFs if ${u}_{m}=1$.

Cognitive and physical burden

To determine the cognitive burden associated with each control strategy, we measured cognitive load with a secondary tactile detection-response task (DRT) that the participant simultaneously completed while using the prosthesis^55,109,110. The DRT requires the participant to push a button with their intact or contralateral hand in response to a small vibrating motor on their collarbone. The bilateral amputee had the button secured to their hip and pressed it using their residual limb. Both the response rate (i.e., how often they respond to the vibratory stimuli) and response time (i.e., how long it takes to press the button after the vibratory stimulus) are used as direct measures of cognitive load. In accordance with ISO 17488:2016, vibrations were applied according to a Gaussian probability distribution function with a mean of three seconds and a variance of two seconds. The response rate to the DRT has been correlated with increased cognitive load and task difficulty^57,58,59. In a separate experiment, the baseline response times to the tactile DRT without an additional task were measured to be approximately 250 ms (Supplementary Fig. 3). In driving tasks, average response times vary with task difficulty, ranging from baseline to approximately 800 ms, where a ceiling effect is observed⁵⁷.

We also analyzed the shared control’s effect on the physical burden of using the bionic hand. The mean absolute value of the sEMG magnitude was used to measure the user’s physical effort (muscle activity). This measure included all 32 electrodes on the user’s forearm or residual limb.

Fragile transfer task with shared control

Six intact-limb participants completed a fragile-object transfer task with the Electronic Grip Gauge (EGG)⁶⁰. The EGG consists of a weighted 3D-printed block containing a load cell and accelerometer to measure grip force and lift force, respectively. Similar to the mechanical fragile object, the EGG would emit a sound when the grip force exceeded a threshold. As opposed to the pass-fail nature of the mechanical fragile object used to evaluate the autonomous controller, the EGG provides time series data, allowing for a deeper analysis of how different control schemes might affect grip force, such as analyzing the average or peak grip forces.

During the fragile-object task, participants used a prismatic precision pinch grasp to pick and transfer the object over a barrier as many times as possible in 120 seconds without dropping or “breaking” the object by exceeding a grip force of 17 N. Data collected from the experiment involving the autonomous controller (Supplementary Fig. 2) indicated that the break force should be set to 17 N rather than 19 N to make the task more difficult. Participants also completed the detection response task (DRT) as a secondary task during the experiment^56,59,110. In order to properly measure cognitive load, the DRT and fragile-object transfer task needed to be run continuously rather than on a per-trial basis⁵⁹. Each participant completed two trials of the fragile object transfer task using human control and two trials using shared control in a pseudo-randomized counterbalanced format. Participants were given up to 1 minute to practice before each experimental block. The number of times the participants broke or dropped the EGG from each trial, the number of successful transfers from each trial, and the grip force during each grasp attempt were recorded as measurements of motor precision. The DRT response time was recorded as a metric of cognitive burden, and the average sEMG activation during the trial was recorded as a metric of physical burden.

Holding task

To assess the shared controller’s ability to assist with gross, prolonged movements, six intact-limb participants completed a holding task in which they were instructed to use the prosthesis to pick up a 3D-printed sphere and hold it for two minutes without dropping it. Participants controlled four independent DOFs for this task: the thumb, index, middle, and ring fingers. The pinky finger was not sensorized and was linked to the control of the ring finger. The participants were instructed to pick the object up using a power circular grasp as quickly as possible and to pick it back up as quickly as possible if, at any point, they dropped it. Each participant completed two 120-second trials of the holding task using human control and two 120-second trials of shared control in a pseudo-randomized counter-balanced format to prevent order effects such as fatigue or learning. Participants were given up to 1 minute to practice before each experimental block. Participants also completed the DRT as a secondary task during the experiment. The number of times the object was dropped, and the average number of fingers used when grasping (determined by non-zero pressure sensor data), were recorded as measures of grip security. Contacting an object with more digits results in a more secure grasp³⁰. The response times to the DRT were recorded as a measure of cognitive burden, and the average sEMG activation during the trial was recorded as a metric of physical burden.

Fragile holding task

Four amputee participants completed a novel fragile holding task in which they were instructed to use the prosthesis to pick up the EGG and hold it for two minutes without dropping or breaking it⁶⁰. Due to limited time with the amputee participants, the fragile holding task was designed to measure grip precision and grip security simultaneously, as a combination of both the aforementioned fragile object transfer task and the holding task. Exceeding a grip force of 15 N resulted in a sound notifying the participant to lower their grip force. Participants were instructed to prioritize holding the instrumented object and secondarily prioritize keeping their grip force in the given range. An intermediate tripod grasp was used for the holding task. In this manner, the participant’s ability to maintain a precise, constant grip could be evaluated by a single task requiring less time from the amputee participants.

Each participant completed two 120-second trials of the holding task using human control and two 120-second trials of shared control in a pseudo-randomized counter-balanced format to prevent order effects such as fatigue or learning. Participants were given up to 15 minutes to practice before each experimental block. The additional practice time enabled participants to learn how to use the shared and human control, as both control schemes were different from the direct-drive control that many of them were familiar with. If participants felt comfortable with the given control scheme, then they could forgo the remainder of their practice time. Participants 1 and 2 completed the task twice, and participants 3 and 4 completed the task only once due to limited availability. The number of times the EGG was dropped during a trial and the time spent holding the EGG were recorded as measures of grip security. The average force exerted during each grasp attempt and the maximum amount of time holding the EGG within the given force threshold were recorded as metrics of grip precision. Only grasp attempts greater than one second long were analyzed to remove any measured forces of the participant bumping into the object before grasping it. The response times to the DRT were recorded as a measure of cognitive burden, and the average sEMG activation during the trial was recorded as a metric of physical burden.

Drinking task

A single amputee participant completed six trials of a simulated drinking task with shared control, followed by six trials with human control to assess the validity of shared control for activities of daily living (ADLs). The participant attempted to pick up a disposable foam cup off a table, bring it to their mouth, mime a sip, and return the cup to the table without crushing or dropping it. The cups broke under a compressive force of approximately 7 N. The amount of time it took the participants to pick up the cup and bring it to their mouth was recorded as a metric of speed. Additionally, an experimenter noted if the participant crushed or dropped the cup throughout each trial as a metric of success. The participant was allowed to practice for 5 minutes with each controller before the respective trials.

Statistical methods

Descriptive statistics, including mean and standard deviation (SD) for continuous variables, and frequency and percentage for categorical variables, were calculated for participant demographics and outcome variables. In order to examine the effect of shared control on each outcome variable, a generalized linear model with a permutation test of the coefficient of intervention was fit to the data on the outcome variables with ≤ 30 total observations^{111,112,113,114}. Specifically, a permutation test was performed by enumeration of all possible distinct permutations or using 10,000 Monte Carlo permutations, depending on the total number of observations. An exact Monte Carlo p-value for the coefficient of intervention was calculated in each model. A permutation test was used for these variables since it does not assume a particular distribution of residuals and since it is specifically suitable for the analysis of data with a smaller sample size as long as sufficient permutations can be obtained. Meanwhile, the outcome variables with > 30 total observations were analyzed using a generalized linear model with the robust or sandwich estimator of variance, allowing for heteroscedastic residuals^115,116,117. All models were adjusted for participant, trial, difficulty (if used in data collection), and block (if used in data collection), by using these variables as fixed-effect covariates. All significance tests were conducted as two-sided tests, with an α level of 0.05 for statistical significance.

Data availability

All data required to evaluate the conclusions in the paper are presented in the main manuscript and/or the Supplementary Materials. Any additional requests for information can be directed to and will be fulfilled by the corresponding authors. Source data are provided with this paper.

References

M. Bionics LUKE Arm Detail Page – Mobius Bionics. http://www.mobiusbionics.com/luke-arm/.
Ottobock. bebionic Technical Manual. http://bebionic.com/distributor/documents/Tech_Manual_Medium_Large_Hand1.pdf.
T. Bionics User Brochure: i-limb quantum. http://www.touchbionics.com/sites/default/files/files/MA01374US%20rev.%202%2C%20January%202017%20User%20Brochure_i-limb%20quantum.pdf.
Biddiss, E. A. & Chau, T. T. Upper limb prosthesis use and abandonment: A survey of the last 25 years. Prosthet. Orthot. Int 31, 236–257 (2007).
Article PubMed Google Scholar
Salminger, S. et al. Current rates of prosthetic usage in upper-limb amputees – have innovations had an impact on device acceptance? Disabil. Rehabil. 44, 3708–3713 (2022).
Article PubMed Google Scholar
Biddiss, E. & Chau, T. Upper-limb prosthetics: critical factors in device abandonment. Am. J. Phys. Med Rehabil. 86, 977–987 (2007).
Article PubMed Google Scholar
Smail, L. C., Neal, C., Wilkins, C. & Packham, T. L. Comfort and function remain key factors in upper limb prosthetic abandonment: findings of a scoping review. Disabil. Rehabil.: Assist. Technol. 16, 821–830 (2021).
PubMed Google Scholar
Atkins, D. J. O., Heard, D. C. Y. M. & Donovan, W. H. Epidemiologic overview of individuals with upper-limb loss and their reported research priorities. J. Prosthet. 8, 2–11 (1996).
Article Google Scholar
Cordella, F. et al. Literature review on the needs of upper limb prosthesis users. Front. Neurosci. 10, 1–14 (2016).
Goodman, J. M. et al. Postural representations of the hand in the primate sensorimotor cortex. Neuron 104, 1000–1009.e7 (2019).
Article CAS PubMed PubMed Central Google Scholar
Sobinov, A. R. & Bensmaia, S. J. The neural mechanisms of manual dexterity. Nat. Rev. Neurosci. 22, 741–757 (2021).
Article PubMed PubMed Central Google Scholar
Yan, Y., Goodman, J. M., Moore, D. D., Solla, S. A. & Bensmaia, S. J. Unexpected complexity of everyday manual behaviors. Nat. Commun. 11, 3564 (2020).
Article ADS CAS PubMed PubMed Central Google Scholar
Nowak, M., Vujaklija, I., Sturma, A., Castellini, C. & Farina, D. Simultaneous and proportional real-time Myocontrol of up to three degrees of freedom of the wrist and hand. IEEE Trans. Biomed. Eng. 70, 459–469 (2023).
Article PubMed Google Scholar
Warren, D. J. et al. Recording and decoding for neural prostheses. Proc. IEEE 104, 374–391 (2016).
Article ADS Google Scholar
Koura, G. & Singh, K. Handrix: Animating the Human Hand. Eurographics/SIGGRAPH Symposium on Computer Animation 11, 1–11 (2003).
Lee, J. C. & Healy, J. C. Normal Sonographic anatomy of the wrist and hand. RadioGraphics 25, 1577–1590 (2005).
Article PubMed Google Scholar
Tresch, M. C. & Jarc, A. The case for and against muscle synergies. Curr. Opin. Neurobiol. 19, 601–607 (2009).
Article CAS PubMed PubMed Central Google Scholar
Flash, T. & Hochner, B. Motor primitives in vertebrates and invertebrates. Curr. Opin. Neurobiol. 15, 660–666 (2005).
Article CAS PubMed Google Scholar
Bernshteĭn, N. A. The Co-Ordination and Regulation of Movements. (Pergamon Press, Oxford; New York, 1967).
Westling, G. & Johansson, R. S. Factors influencing the force control during precision grip. Exp. Brain Res. 53, 277–284 (1984).
Article CAS PubMed Google Scholar
Roudaut, Y. et al. Touch sense: functional organization and molecular determinants of mechanosensitive receptors. Channels (Austin) 6, 234–245 (2012).
Article CAS PubMed Google Scholar
Johansson, R. S. & Westling, G. Signals in tactile afferents from the fingers eliciting adaptive motor responses during precision grip. Exp. Brain Res. 66, 141–154 (1987).
Article CAS PubMed Google Scholar
Westling, G. & Johansson, R. S. Responses in glabrous skin mechanoreceptors during precision grip in humans. Exp. Brain Res 66, 128–140 (1987).
Article CAS PubMed Google Scholar
Johansson, R. S. & Westling, G. Programmed and triggered actions to rapid load changes during precision grip. Exp. Brain Res. 71, 72–86 (1988).
Article CAS PubMed Google Scholar
Johansson, R. S. & Flanagan, J. R. Coding and use of tactile signals from the fingertips in object manipulation tasks. Nat. Rev. Neurosci. 10, 345–359 (2009).
Article CAS PubMed Google Scholar
Hansen, T. C., Trout, M. A., Segil, J. L., Warren, D. J. & George, J. A. A Bionic Hand for Semi-Autonomous Fragile Object Manipulation via Proximity and Pressure Sensors. In 2021 43rd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC) 6465–6469 (2021). https://doi.org/10.1109/EMBC46164.2021.9629622.
Heo, S.-H. & Park, H.-S. Proximity perception-based grasping intelligence: toward the seamless control of a dexterous prosthetic hand. IEEE/ASME Trans. Mechatron. 1–12 https://doi.org/10.1109/TMECH.2023.3324051 (2023).
Starke, J., Weiner, P., Crell, M. & Asfour, T. Semi-autonomous control of prosthetic hands based on multimodal sensing, human grasp demonstration and user intention. Robot. Auton.s Syst. 154, 104123 (2022).
Article Google Scholar
Castro, M. N. & Dosen, S. Continuous semi-autonomous prosthesis control using a depth sensor on the hand. Front. Neurorobot. 16, 1–17 (2022).
Zhuang, K. Z. et al. Shared human–robot proportional control of a dexterous myoelectric prosthesis. Nat. Mach. Intell. 1, 400–411 (2019).
Article Google Scholar
Kyberd, P. Slip Detection Strategies for Automatic Grasping in Prosthetic Hands. Sensors 23, 4433 (2023).
Article ADS PubMed PubMed Central Google Scholar
Dantas, H. et al. Shared human-machine control for self-aware prostheses. 2018 IEEE International Conference on Acoustics, Speech, and Signal Processing, (2018).
Augurelle, A.-S., Smith, A. M., Lejeune, T. & Thonnard, J.-L. Importance of cutaneous feedback in maintaining a secure grip during manipulation of hand-held objects. J. Neurophysiol. 89, 665–671 (2003).
Article PubMed Google Scholar
Jordan, M. I. & Wolpert, D. M. Computational motor control. In The Cognitive Neurosciences Cambridge. 616–633 (MIT Press, Cambridge, MA, USA, 1999).
Delhaye, B. P., Long, K. H. & Bensmaia, S. J. Neural basis of touch and proprioception in primate cortex. Compr. Physiol. 8, 1575–1602 (2018).
Article PubMed PubMed Central Google Scholar
Segil, J., Patel, R., Klingner, J., Weir, R. F. & Correll, N. Multi-modal prosthetic fingertip sensor with proximity, contact, and force localization capabilities. Adv. Mech. Eng. 11, 1687814019844643 (2019).
Article Google Scholar
Flanagan, J. R. & Wing, A. M. The stability of precision grip forces during cyclic arm movements with a hand-held load. Exp. Brain Res. 105, 455–464 (1995).
CAS PubMed Google Scholar
Engeberg, E. D. & Meek, S. Improved grasp force sensitivity for prosthetic hands through force-derivative feedback. IEEE Trans. Biomed. Eng. 55, 817–821 (2008).
Article PubMed Google Scholar
George, J. A. et al. Biomimetic sensory feedback through peripheral nerve stimulation improves dexterous use of a bionic hand. Sci. Robot. 4, eaax2352 (2019).
Article PubMed Google Scholar
George, J. A., Davis, T. S., Brinton, M. R. & Clark, G. A. Intuitive neuromyoelectric control of a dexterous bionic arm using a modified Kalman filter. J. Neurosci. Methods 330, 108462 (2020).
Article PubMed Google Scholar
George, J., Neibling, A., Paskett, M. & Clark, G. Inexpensive surface electromyography sleeve with consistent electrode placement enables dexterous and stable prosthetic control through deep learning. In Myoelectric Contol Symposium. 1–4 (UNB Library, Fredericton, New Brunswick, Canada, 2020).
Hart, S. G. & Staveland, L. E. Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research. In Advances in Psychology (eds Hancock, P. A. & Meshkati, N.) vol. 52 139–183 (North-Holland, 1988).
Hebert, J. S. et al. Quantitative eye gaze and movement differences in visuomotor adaptations to varying task demands among upper-extremity prosthesis users. JAMA Netw. Open 2, e1911197 (2019).
Article PubMed PubMed Central Google Scholar
Nieveen, J., Brinton, M., Warren, D. J. & Mathews, V. J. A nonlinear latching filter to remove jitter from movement estimates for prostheses. IEEE Trans. Neural Syst. Rehabil. Eng. 28, 2849–2858 (2020).
Article PubMed Google Scholar
Dantas, H., Hansen, T. C., Warren, D. J. & Mathews, V. J. Shared prosthetic control based on multiple movement intent decoders. IEEE Trans. Biomed. Eng. 68, 1547–1556 (2021).
Article PubMed Google Scholar
Henneman, E. Relation between size of neurons and their susceptibility to discharge. Science 126, 1345–1347 (1957).
Article ADS CAS PubMed Google Scholar
George, J. A., Brinton, M. R., Duncan, C. C., Hutchinson, D. T. & Clark, G. A. Improved Training Paradigms and Motor-decode Algorithms: Results from Intact Individuals and a Recent Transradial Amputee with Prior Complex Regional Pain Syndrome. in 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) 3782–3787 (2018). https://doi.org/10.1109/EMBC.2018.8513342.
Goodman, J. M. et al. Postural Representations of the Hand in the Primate Sensorimotor Cortex. Neuron 104, 1000–1009.e7 (2019).
Buczak, M., Baum, B. S., Olsen, C. D. & George, J. A. Disentangling Sensory and Motor Deficits of Fine Hand Function Using an Electronic Grip Gauge (EGG) to Simulate Transferring Fragile Objects. in (Fredericton, New Brunswick, Canada, 2022).
Controzzi, M., Clemente, F., Pierotti, N., Bacchereti, M. & Cipriani, C. Evaluation of hand function transporting fragile objects: the virtual eggs test. In Myoelectric Control Symposium 83 (UNB Library, Fredericton, New Brunswick, Canada, 2017).
Johansson, R. S. & Cole, K. J. Grasp stability during manipulative actions. Can. J. Physiol. Pharm. 72, 511–524 (1994).
Article CAS Google Scholar
Edin, B. B., Westling, G. & Johansson, R. S. Independent control of human finger-tip forces at individual digits during precision lifting. J. Physiol. 450, 547–564 (1992).
Article CAS PubMed PubMed Central Google Scholar
Cini, F., Ortenzi, V., Corke, P. & Controzzi, M. On the choice of grasp type and location when handing over an object. Sci. Robot. 4, eaau9757 (2019).
Article PubMed Google Scholar
Strayer, D. L., Cooper, J. M., Turrill, J., Coleman, J. R. & Hopman, R. J. The smartphone and the driver’s cognitive workload: A comparison of Apple, Google, and Microsoft’s intelligent personal assistants. Can. J. Exp. Psychol. 71, 93–110 (2017).
Article PubMed Google Scholar
Paskett, M. D. et al. Improving Upper-limb Prosthesis Usability: Cognitive Workload Measures Quantify Task Difficulty. 2022.08.02.22278038 Preprint at https://doi.org/10.1101/2022.08.02.22278038 (2022).
Olsen, C. D. et al. Electromyographically controlled prosthetic wrist improves dexterity and reduces compensatory movements without added cognitive load. Sci. Rep. 14, 23248 (2024).
Article ADS PubMed PubMed Central Google Scholar
Bruyas, M.-P. & Dumont, L. Sensitivity of Detection Response Task (DRT) to the Driving Demand and Task Difficulty. In Driving Assessment Conference. Vol. 7, 64–70 (University of Iowa, 2013).
Stojmenova, K. et al. Sensitivity evaluation of the visual, tactile, and auditory detection response task method while driving. Traffic Inj. Prev. 18, 431–436 (2017).
Article PubMed Google Scholar
Stojmenova, K. & Sodnik, J. Detection-Response Task—Uses and Limitations. Sensors 18, 594 (2018).
Article ADS PubMed PubMed Central Google Scholar
Adkins, M. D., Buczak, M. K., Olsen, C. D., Iversen, M. M. & George, J. A. Automated Quantifiable Assessments of Sensorimotor Function Using an Instrumented Fragile Object. in 2023 International Conference on Rehabilitation Robotics (ICORR) 1–6 https://doi.org/10.1109/ICORR58425.2023.10304693 (2023).
Chang, C.-C., Boyle, L. N., Lee, J. D. & Jenness, J. Using tactile detection response tasks to assess in-vehicle voice control interactions. Transp. Res. Part F: Traffic Psychol. Behav. 51, 38–46 (2017).
Article Google Scholar
Marasco, P. D. et al. Neurorobotic fusion of prosthetic touch, kinesthesia, and movement in bionic upper limbs promotes intrinsic brain behaviors. Sci. Robot. 6, eabf3368 (2021).
Article PubMed Google Scholar
Thumser, Z. C., Slifkin, A. B., Beckler, D. T. & Marasco, P. D. Fitts’ Law in the Control of Isometric Grip Force With Naturalistic Targets. Front. Psychol. 9, 1–13 (2018).
PSYONIC. https://www.psyonic.io/.
COVVI’s Technical Specifications: The Key Details. COVVI Ltd https://www.covvi.com/covvi-hand/product-overview/.
Zbinden, J., Lendaro, E. & Ortiz-Catalan, M. A multi-dimensional framework for prosthetic embodiment: a perspective for translational research. J. Neuroeng. Rehabil. 19, 122 (2022).
Article PubMed PubMed Central Google Scholar
Segil, J. L., Roldan, L. M. & Graczyk, E. L. Measuring embodiment: A review of methods for prosthetic devices. Front. Neurorobot. 16, 1–26 (2022).
Kargov, A., Pylatiuk, C., Martin, J., Schulz, S. & Döderlein, L. A comparison of the grip force distribution in natural hands and in prosthetic hands. Disabil. Rehabil. 26, 705–711 (2004).
Article PubMed Google Scholar
Vergara, M., Sancho-Bru, J. L., Gracia-Ibáñez, V. & Pérez-González, A. An introductory study of common grasps used by adults during performance of activities of daily living. J. Hand Ther. 27, 225–234 (2014).
Article PubMed Google Scholar
Light, C. M., Chappell, P. H. & Kyberd, P. J. Establishing a standardized clinical assessment tool of pathologic and prosthetic hand function: Normative data, reliability, and validity. Arch. Phys. Med. Rehabil. 83, 776–783 (2002).
Article PubMed Google Scholar
Resnik, L. et al. Development and Evaluation of the Activities Measure for Upper Limb Amputees. Arch. Phys. Med. Rehabil. 94, 488–494.e4 (2013).
Article PubMed Google Scholar
James, D. R. C. et al. Cognitive Burden Estimation for Visuomotor Learning with fNIRS. in Medical Image Computing and Computer-Assisted Intervention – MICCAI 2010 (eds Jiang, T., Navab, N., Pluim, J. P. W. & Viergever, M. A.) 319–326 (Springer, Berlin, Heidelberg, 2010).
Paas, F. G. W. C. & Van Merriënboer, J. J. G. Instructional control of cognitive load in the training of complex cognitive tasks. Educ. Psychol. Rev. 6, 351–371 (1994).
Article Google Scholar
Kawato, M. & Wolpert, D. Internal Models for Motor Control. In Novartis Foundation Symposium 218 - Sensory Guidance of Movement 291–307 (John Wiley & Sons, Ltd). https://doi.org/10.1002/9780470515563.ch16.
Resnik, L., Hamner, T. & Borgia, M. The association between upper limb prosthesis receipt timing and prosthesis adoption and satisfaction: Findings from a cross-sectional study. Prosthetics and Orthotics International (2022) https://doi.org/10.1097/PXR.0000000000000474.
Carey, S. L., Lura, D. J. & Highsmith, M. J. Differences in myoelectric and body-powered upper-limb prostheses: systematic literature review. JPO: J. Prosthet. Orthot. 29, P4 (2017).
Google Scholar
Huang, M. E., Levy, C. E. & Webster, J. B. Acquired limb deficiencies. 3. Prosthetic components, prescriptions, and indications. Arch. Phys. Med. Rehabil. 82, S17–S24 (2001).
Article CAS PubMed Google Scholar
Engdahl, S. M., Meehan, S. K. & Gates, D. H. Differential experiences of embodiment between body-powered and myoelectric prosthesis users. Sci. Rep. 10, 15471 (2020).
Article ADS PubMed PubMed Central Google Scholar
Schiefer, M., Tan, D., Sidek, S. M. & Tyler, D. J. Sensory feedback by peripheral nerve stimulation improves task performance in individuals with upper limb loss using a myoelectric prosthesis. J. Neural Eng. 13, 016001 (2016).
Article ADS PubMed Google Scholar
Tan, D. W. et al. A neural interface provides long-term stable natural touch perception. Sci. Transl. Med. 6, 257ra138 (2014).
Article PubMed PubMed Central Google Scholar
Dhillon, G. S. & Horch, K. W. Direct neural sensory feedback and control of a prosthetic arm. IEEE Trans. Neural Syst. Rehabil. Eng. 13, 468–472 (2005).
Article PubMed Google Scholar
Raspopovic, S. et al. Restoring natural sensory feedback in real-time bidirectional hand prostheses. Sci. Transl. Med. 6, 222ra19–222ra19 (2014).
Article PubMed Google Scholar
Oddo, C. M. et al. Intraneural stimulation elicits discrimination of textural features by artificial fingertip in intact and amputee humans. eLife 5, (2016).
Petrini, F. M. et al. Six-Month assessment of a hand prosthesis with intraneural tactile feedback: hand prosthesis. Ann. Neurol. 85, 137–154 (2019).
Article PubMed Google Scholar
Zollo, L. et al. Restoring tactile sensations via neural interfaces for real-time force-and-slippage closed-loop control of bionic hands. Sci. Robot. 4, eaau9924 (2019).
Article PubMed PubMed Central Google Scholar
D’Anna, E. et al. A closed-loop hand prosthesis with simultaneous intraneural tactile and position feedback. Sci. Robot. 4, eaau8892 (2019).
Article PubMed Google Scholar
Davis, T. S. et al. Restoring motor control and sensory feedback in people with upper extremity amputations using arrays of 96 microelectrodes implanted in the median and ulnar nerves. J. Neural Eng. 13, 036001 (2016).
Article ADS CAS PubMed Google Scholar
Page, D. M. et al. Motor control and sensory feedback enhance prosthesis embodiment and reduce phantom pain after long-term hand amputation. Front. Hum. Neurosci. 12, 1–16 (2018).
Page, D. M. et al. Discriminability of multiple cutaneous and proprioceptive hand percepts evoked by intraneural stimulation with Utah slanted electrode arrays in human amputees. J. Neuroeng. Rehabil. 18, 12 (2021).
Article PubMed PubMed Central Google Scholar
Coapt - Myoelectric Pattern Recognition for Upper Limb Prostheses. Coapt Myo Pattern Recognition https://coaptengineering.com/.
Roche, A. D., Rehbaum, H., Farina, D. & Aszmann, O. C. Prosthetic myoelectric control strategies: a clinical perspective. Curr. Surg. Rep. 2, 44 (2014).
Article Google Scholar
Resnik, L. et al. Evaluation of EMG pattern recognition for upper limb prosthesis control: a case study in comparison with direct myoelectric control. J. Neuroeng. Rehabil. 15, 23 (2018).
Article PubMed PubMed Central Google Scholar
Smith, L. H., Kuiken, T. A. & Hargrove, L. J. Real-time simultaneous and proportional myoelectric control using intramuscular EMG. J. Neural Eng. 11, 066013 (2014).
Article ADS PubMed PubMed Central Google Scholar
Wendelken, S. M. Dexterous control of a hand prosthesis using neuromuscular signals from implanted or surface electrodes. (University of Utah, 2018).
George, J. A., Radhakrishnan, S., Brinton, M. & Clark, G. A. Inexpensive and Portable System for Dexterous High-Density Myoelectric Control of Multiarticulate Prostheses. in 2020 IEEE International Conference on Systems, Man, and Cybernetics (SMC) 3441–3446 (2020). https://doi.org/10.1109/SMC42975.2020.9283086.
Brinton, M. R. et al. Portable take-home system enables proportional control and high-resolution data logging with a multi-degree-of-freedom bionic arm. Front. Robot. AI 7, 1–12 (2020).
Paskett, M. D., Davis, T. S., Tully, T. N., Brinton, M. R. & Clark, G. A. Portable System for Home Use Enables Closed-Loop, Continuous Control of Multi-Degree-of-Freedom Bionic Arm. in 2021 43rd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC) 6608–6612 (2021). https://doi.org/10.1109/EMBC46164.2021.9631087.
Paskett, M. D. et al. Activities of daily living with bionic arm improved by combination training and latching filter in prosthesis control comparison. J. Neuroeng. Rehabil. 18, 45 (2021).
Article PubMed PubMed Central Google Scholar
Kyberd, P. J., Mustapha, N., Carnegie, F. & Chappell, P. H. A clinical experience with a hierarchically controlled myoelectric hand prosthesis with vibro-tactile feedback. Prosthet. Orthot. Int. 17, 56–64 (1993).
Article CAS PubMed Google Scholar
Kyberd, P. J. & Chappell, P. H. The Southampton Hand: an intelligent myoelectric prosthesis. J. Rehabil. Res. Dev. 31, 326–334 (1994).
CAS PubMed Google Scholar
Paskett, M. D. et al. A modular transradial bypass socket for surface myoelectric prosthetic control in non-amputees. IEEE Trans. Neural Syst. Rehabil. Eng. 27, 2070–2076 (2019).
Hansen, T. C. et al. A multi-user transradial functional-test socket for validation of new myoelectric prosthetic control strategies. Front. Neurorobot. 16, (2022).
Meek, S. G., Jacobsen, S. C. & Goulding, P. P. Extended physiologic taction: design and evaluation of a proportional force feedback system. J. Rehabil. Res. Dev. 26, 53–62 (1989).
CAS PubMed Google Scholar
Haverkate, L., Smit, G. & Plettenburg, D. H. Assessment of body-powered upper limb prostheses by able-bodied subjects, using the Box and Blocks Test and the Nine-Hole Peg Test. Prosthet. Orthot. Int 40, 109–116 (2016).
Article PubMed Google Scholar
Valle, G. et al. Hand control with invasive feedback is not impaired by increased cognitive load. Front. Bioeng. Biotechnol. 8, (2020).
Valle, G. et al. Biomimetic intraneural sensory feedback enhances sensation naturalness, tactile sensitivity, and manual dexterity in a bidirectional prosthesis. Neuron 100, 37–45.e7 (2018).
Article CAS PubMed Google Scholar
Trout, M. A. et al. Shared-control decreases the physical and cognitive demands of maintaining a secure grip. in Myoelectric Controls Symposium https://doi.org/10.57922/mec.1970 (2022).
Mao, K. Z. Orthogonal forward selection and backward elimination algorithms for feature subset selection. IEEE Trans. Syst., Man Cybern. B (Cybern.) 34, 629–634 (2004).
Article CAS PubMed Google Scholar
Čegovnik, T., Stojmenova, K., Jakus, G. & Sodnik, J. An analysis of the suitability of a low-cost eye tracker for assessing the cognitive load of drivers. Appl. Ergon. 68, 1–11 (2018).
Article PubMed Google Scholar
Innes, R. J., Evans, N. J., Howard, Z. L., Eidels, A. & Brown, S. D. A broader application of the detection response task to cognitive tasks and online environments. Hum. Factors 63, 896–909 (2021).
Article PubMed Google Scholar
Good, P. Resampling Methods: A Practical Guide to Data Analysis. (Birkhauser, Boston, MA, 2006).
Edington, E. S. & Onghena, P. Randomization Tests. (Chapman & Hall/CRC, Boca Raton, FL, 2007).
Higgins, J. J. An Introduction to Modern Nonparametric Statistics. (Brooks/Cole-Thomson Learning, Pacific Grove, CA, 2004).
Anderson, M. J. Permutation tests for univariate or multivariate analysis of variance and regression. Can. J. Fish. Aquat. Sci. 58, 626–639 (2001).
Article Google Scholar
Huber, P. J. The behavior of maximum likelihood estimates under nonstandard conditions. In Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability 1 221–223 (University of California Press, Berkeley, CA, 1967).
White, H. A heteroskedasticity-consistent covariance matrix estimator and a direct test for heteroskedasticity. Econometrica 48, 817–838 (1980).
Article MathSciNet Google Scholar
White, H. Maximum likelihood estimation of misspecified models. Econometrica 50, 1–25 (1982).
Article MathSciNet Google Scholar

Download references

Acknowledgements

We thank the many participants who assisted with this research. Additionally, we thank our funding sources, the NIH and NSF, for making this work possible. Funding came from NIH grant DP5OD029571 (JAG), NSF grant 1901236 (DJW) and NSF grant 1901492 (DJW).

Author information

Authors and Affiliations

Department of Electrical and Computer Engineering, University of Utah, Salt Lake City, UT, USA
Marshall A. Trout, Fredi R. Mino, Connor D. Olsen, David J. Warren & Jacob A. George
Department of Biomedical Engineering, University of Utah, Salt Lake City, UT, USA
Taylor C. Hansen, David J. Warren & Jacob A. George
Department of Orthopedics, University of Utah, Salt Lake City, UT, USA
Masaru Teramoto
Department of Physical Medicine and Rehabilitation, University of Utah, Salt Lake City, UT, USA
Masaru Teramoto & Jacob A. George
Department of Mechanical Engineering, University of Colorado, Boulder, CO, USA
Jacob L. Segil
Department of Mechanical Engineering, University of Utah, Salt Lake City, UT, USA
Jacob A. George

Authors

Marshall A. Trout
View author publications
Search author on:PubMed Google Scholar
Fredi R. Mino
View author publications
Search author on:PubMed Google Scholar
Connor D. Olsen
View author publications
Search author on:PubMed Google Scholar
Taylor C. Hansen
View author publications
Search author on:PubMed Google Scholar
Masaru Teramoto
View author publications
Search author on:PubMed Google Scholar
David J. Warren
View author publications
Search author on:PubMed Google Scholar
Jacob L. Segil
View author publications
Search author on:PubMed Google Scholar
Jacob A. George
View author publications
Search author on:PubMed Google Scholar

Contributions

Controller designs were conceptualized and implemented by M.A.T., F.R.M., C.D.O., T.C.H., J.L.S., D.J.W. and J.A.G. Experimental methodologies were developed by M.A.T., J.A.G. and M.T. Experiments were performed by M.A.T., F.R.M., C.D.O. and T.C.H. Figures were generated by M.A.T. and J.A.G. Funding was secured by J.A.G. and D.J.W. Project administration and supervision were provided by J.A.G. and D.J.W. The original manuscript was written by M.A.T., M.T. and J.A.G. Figures and illustrations were produced by M.A.T. Edits and revisions of the manuscript were provided by M.A.T., F.R.M., C.D.O., T.C.H., J.L.S., D.J.W., J.A.G. and M.T.

Corresponding authors

Correspondence to Marshall A. Trout or Jacob A. George.

Ethics declarations

Competing interests

J.L.S. declares a competing interest as he is a founder and shareholder of Point Designs LLC and an inventor on a non-provisional patent application on the prosthetic fingertips used in this work. The remaining authors declare no competing interests.

Peer review

Peer review information

Nature Communications thanks Silvestro Micera and the other anonymous reviewer(s) for their contribution to the peer review of this work. A peer review file is available.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Supplementary Information

Description of Additional Supplementary Files

Supplementary Movie 1

Supplementary Movie 2

Supplementary Movie 3

Supplementary Movie 4

Transparent Peer Review file

Source data

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/.

Reprints and permissions

About this article

Cite this article

Trout, M.A., Mino, F.R., Olsen, C.D. et al. Shared human-machine control of an intelligent bionic hand improves grasping and decreases cognitive burden for transradial amputees. Nat Commun 16, 10418 (2025). https://doi.org/10.1038/s41467-025-65965-9

Download citation

Received: 18 February 2025
Accepted: 24 October 2025
Published: 09 December 2025
Version of record: 09 December 2025
DOI: https://doi.org/10.1038/s41467-025-65965-9