Fig. 5: Algorithmic realizations of hierarchical control as inference.

a, Schematic of a (high-level) generative model that underwrites human motor control. b, The implicit generative model for a robotics system. The green nodes in a and green boxes in b refer to the highest levels of human motor control and our implicit generative model, respectively. In the generative model, high-level decision-making is realized as a neural network learned through deep reinforcement learning. The blue nodes in a correspond to the middle level of human motor control and the blue boxes in b are intermediate level realizations, implemented as a deep neural network policy learned through deep reinforcement learning for locomotion and an inverse kinematics and dynamics policy for manipulation. On the lowest level, depicted in grey nodes and boxes, a joint impedance controller calculates the torques required for the actuation of the robot. Yellow and light red denote sensor information and motor control, respectively. For clarity, we limit our exposition to key regions in a, based on prior literature, where these are drawn using the solid lines. The dotted lines represent the processing of a separate outcome modality for human motor control, that is, the visual input. Lastly, the prefrontal cortex is connected via the dashed lines to denote its supporting role during human motor control. Dotted lines in b indicate the realizations of the corresponding principles, while dashed lines indicated message parsing. Please refer to ‘Implicit hierarchical generative model for a robotics system’ for the algorithmic implementation of b. SE, state estimation; PDF, probability density function; IMU, inertial measurement unit; τ, torque of individual joints; d_L and d_M, damping parameters; j_L and j_M, inertia parameters; k, stiffness parameter; F/T, force/torque; τ_L and τ_M, torques on L and M, respectively.