Fig. 4: Quantitative analysis and resilience test.

a, We modified the dynamics of the HSA platform. We attached a rod to the platform and appended 350-g weights at a controlled location, causing the platform to tilt in its resting position. b, Top, our framework enabled the HSA system with changed dynamics to complete the rotation motion. Bottom, the graph shows the distance from goal over time. c, Using a bird’s-eye view, we overlaid the completed 3D trajectory (traj.) on top of the starting configuration of the HSA platform. We compared the execution trajectory of our approach with the reference trajectory. This visualization confirmed that our method is able to counteract the physical effects of the weight and stabilize the motion trajectory towards the target path. d, The distance from goal of the Allegro hand decreased over time as we executed the motion plan. We measured the distance from goal using both joint errors in degrees and fingertip positions in millimetres. e, Top, the reference trajectory is shown in white and the completed trajectory in colours during a square drawing task. Bottom, distance from goal over time using the Poppy robot arm in four trajectory segments. f, Comparison of our Jacobian predictions with analytical counterparts computed using physics simulations^47,48. Our method learnt consistent Jacobian measurements from raw RGB observations.