Table 1 Summary of key principles of hierarchical control.

Information factorization

Different information is routed to different subsystems.

Factored learning can require less experience per subsystem. Subsystems are invariant to hidden information and therefore are reusable across contexts.

Partial autonomy

Lower-level systems function somewhat autonomously, with modulation from higher-level systems.

System is more robust and lower-level does not require costly micromanagement.

Amortized control

Movements that have been successfully executed multiple times are compressed into a system that can rapidly reproduce them.

Re-execution of frequently repeated movements should be more computationally efficient than novel variations.

Modular objectives

Specific subsystems may be trained to optimize specific objectives, distinct from the global task objective.

Training of subsystems can leverage error signals that are denser or more well known than the global task objective.

Multi-joint coordination

Movement is produced in a manner that reflects common patterns across the body.

Exploration and action-selection can exploit commonly co-occurring multi-joint patterns.

Temporal abstraction

Common temporal motifs are abstracted.

Behavior specification or planning can occur at a coarser timescale.