Information Decomposition in Complex Systems

Complex systems—from neural circuits and social networks to amorphous materials and economic markets—rely on intricate patterns of information flow among their components. Information decomposition offers a principled way to dissect how individual parts contribute uniquely, redundantly or synergistically to collective behaviour. By partitioning mutual information into distinct components, researchers can identify which variables carry overlapping signals, which offer exclusive insights and which combine to yield novel information that no single source provides alone. Advances in theoretical frameworks, such as partial information decomposition and related metrics like O-information, have transformed our ability to quantify high-order interactions among large ensembles of variables. These tools enable the localisation of critical nodes in a network, reveal the emergence of macroscopic phenomena from microscopic dynamics and inform strategies for control, inference and design in domains as diverse as materials science, machine learning and systems biology. Practical applications range from optimising material properties under stress to decoding cognitive processes in the brain, underscoring the global significance of understanding information circuits within complex systems.

Research from Nature Portfolio

No recent Nature Portfolio content available.

Research from all publishers

One recent study introduces a practical machine-learning approach to decompose the information content of multiple measurements by jointly optimising a lossy compression of each signal. Guided by a distributed information bottleneck objective, this method unpacks the entropy of system states—demonstrated on Boolean circuits and deforming amorphous solids—to isolate the bits most relevant to macroscale behaviour. A complementary investigation in neural networks solving multiple tasks applies an information-decomposition framework to reveal that synergistic information increases as networks learn diverse problems and supports the integration of different input modalities. This work shows that flexible learning hinges on neurons whose joint activity encodes information unobtainable from any subset alone. Another contribution proposes the gradients of O-information as low-order descriptors of high-order dependencies, localising the roles of specific variables within systems ranging from frustrated Ising models to macroeconomic indicators. By mapping how redundancy and synergy vary across components, this approach highlights the formation of informational circuits that underpin emergent phenomena.

Information Decomposition in Complex Systems publication trend

The graph below shows the total number of articles in information decomposition in complex systems across all publications each year (not limited to Nature Index journals).

Technical terms

Mutual information: Measure of the shared information between two variables, reflecting how knowledge of one reduces uncertainty about the other.

Entropy: Quantitative indicator of the uncertainty or disorder within a system’s state distribution.

Partial information decomposition: Framework that breaks down the mutual information between multiple sources and a target into unique, redundant and synergistic contributions.

Synergy: Information about a target variable that emerges only when two or more sources are considered together.

Redundancy: Overlapping information about a target that is provided independently by multiple sources.

O-information: Metric that captures the global balance between redundancy and synergy across groups of variables.

References

1. Information decomposition in complex systems via machine learning. Proceedings of the National Academy of Sciences of the United States of America (2024).
2. Synergistic information supports modality integration and flexible learning in neural networks solving multiple tasks. PLOS Computational Biology (2024).
3. Gradients of O-information: Low-order descriptors of high-order dependencies. Physical Review Research (2023).