CS Paper Radar

Statistical Dark Matter: Synergy is everywhere but is hard to capture

Alberto Liardi, Fernando E. Rosas, Daniele Marinazzo, Thomas F. Varley, Michael Gastpar, Pedro A. M. Mediano

cs.IT physics.data-an ·submitted 2026-09-27 16:56 UTC · v2
unusual

Abstract

The information-theoretic construct of synergy refers to the statistical structure that is contained in three or more variables, but not in any subset of them. Here we leverage recent advances in multivariate information theory to show that synergy is far more prevalent in complex systems than previously thought. In particular, we show that many-body systems tend to become strongly dominated by synergy as the number of subcomponents grows. At the same time, our results also reveal that commonly used tools for statistical modelling severely underestimate synergistic structures. These findings imply that synergy constitutes a prevalent, yet often invisible, informational component of complex systems. With a certain poetic licence, we interpret these results as suggesting that synergy is the dark matter of statistics - interdependencies that we know exist, but standard instruments fail to detect.

Closest papers

Nearest neighbours by meaning across the whole index.

An information-geometric formulation of distributed computation

cs.IT ·2026-11-02 ·Alberto Liardi, George Blackburne, Pedro A. M. Mediano
similarity 0.57
Abstract · 16 pages, 6 figures + supplementary material

Understanding how complex systems perform computations, i.e. how they store, transmit, and transform information across their components, is a major goal across the natural sciences. However, characterising genuinely collective computations across multiple inputs and outputs that cannot be reduced to contributions from individual components remains an open problem. In this work, we introduce $Φ\text{ID}^{IG}$, a geometric theory of information dynamics that captures irreducible collective phenomena and enables a complete decomposition of multi-input multi-output operations into interpretable computational modes, providing a complete informational profile of a system. We first validate our framework to identify and quantify distinct computational modes in discrete and Gaussian systems with known ground truth. We then apply it to uncover the computational architecture of complex systems across synthetic and real-world scenarios, including chaotic maps, where distinct universality classes are associated with characteristic modes of computation; brain anatomy of over 100 mammalian species, revealing an evolutionary shift towards more distributed computation in the human neocortex; and financial markets, where collective computations recover the modular organisation of market sectors. Overall, our results show how complex systems can be quantitatively investigated through multidimensional computational portraits derived from the geometry of their underlying probability distributions.

0.28

Claims of emergence in AI should be grounded in information decomposition

cs.AI cs.HCcs.ITcs.LG+1 ·2026-10-20 ·Christoph Riedl, Fernando E. Rosas
top venueposition similarity 0.55
Abstract · NeurIPS Spotlight Paper

When a language model develops capabilities not traceable to its individual components, when a multi-agent AI system outperforms each agent on its own, or when a human-AI pair succeeds where either fails in isolation, AI researchers describe it as a case of ``synergy'' or ``emergence''. However, these are usually informal assessments not supported by a theoretical framework, making these claims impossible to compare or even meaningfully dispute. Here we argue that \emph{information decomposition can serve as a unifying formal framework to operationalize assessments about synergy in AI systems}. Information decomposition provides a way to distinguish qualitatively different \emph{types} of information, including unique information available from individual components, redundant information shared across multiple components, and synergistic information accessible only from parts taken together. This gives claims of emergence a precise, falsifiable criterion: a system's behavior is emergent only if it carries synergistic information about task-relevant outcomes. We apply this lens to attention heads, autonomous agents, and human-AI teams, showing how it organizes existing knowledge across scales and substrates, while also generating cross-level conjectures that siloed fields would miss.

0.49

The Logic of Emergence

cs.LO math.LOphysics.hist-ph ·2026-11-04 ·David H. Wolpert, David C. Krakauer
similarity 0.53
Abstract · 52 pages, 44 pages main text, 3 figures

``Emergence'' refers to the fact that different mathematical languages and theories seem to be required to describe scientific theories that operate at different spatial and temporal scales. Highly circumscribed, special cases of emergence have been investigated using many different formal tools, including coarse-graining (over space and / or time), renormalization, dimension reduction, broken symmetry, and various formulations of causality. Here we extend the mathematics of model theory to unify and extend these earlier investigations. As a preliminary application of our framework, we investigate canonical examples of emergence, including the emergence of genetics from chemistry, of psychology from neurobiology, and of different forms of collective behavior from the laws governing their constituent parts. By exploiting the power of model theory we also uncover many aspects of emergence that were previously obscured. In particular, we uncover some of the rich hierarchical structure that relates different scientific fields. For example, we show that scientific fields can be related via a collection of multiple partial orders, all defined over the same set. On the other hand, under an alternative relation among scientific fields, one which allows cycles, we show that it is possible that the set of all of scientific fields must collapse into a single scientific field. Going forward, these investigations point to the formal tools that might need to be developed in order to arrive at a deeper understanding of this emergent hierarchy of scientific reality.

0.38

Integrated Information in the Active Inference Framework

cs.IT ·2026-08-14 ·Carlotta Langer, Jesse van Oostrum, Nihat Ay
similarity 0.52
Abstract

The active inference framework provides a principled approach to modeling sentient behavior. In this framework perception and action selection are treated in a unified way. The resulting agents form an internal generative model of the relevant dynamics of the world in order to infer their future observations, their internal states and to select actions. We combine this modeling framework with the Integrated Information Theory of consciousness and are therefore able to analyze the active inference agents from the perspective of integrated information. The Integrated Information Theory aims at quantifying the level of consciousness of a system by assessing its capability to integrate information. Here, we define a measure of integrated information for the generative model by making an additional structural assumption. Experiments with simulated agents reveal a correlation between integrated information measures and the free energy of the active inference agents that increases with the size of the generative model.

0.32

Same Behavior, Different Mechanism: What Finite Experiments Can Certify About Integrated Information

cs.LG cs.AIq-bio.NC ·2026-11-05 ·Linqian Wu, Diaoxuan He
theory similarity 0.51
Abstract · 30 pages, 7 figures. Code available as ancillary files

Interpretability, AI auditing and assessments of machine consciousness increasingly rest on mechanism-level quantities: functionals of a system's causal transition law that describe how it computes, not what it outputs. Experiments see that law only through finite, partial and noisy interventions. We ask when such a quantity is identifiable from an experiment's access, continuous in the mechanism and certifiable from finite data, and answer exactly for a deliberately demanding case - the integrated information $\varphi$ of integrated information theory - in stochastic networks of four and five units with known ground truth. Behavior can be silent: a recurrent and a feed-forward network with identical input-output laws at their response delay carry 0.28 and 0 bits, yet internal recordings separate them within 4-53 trials. Partial access can be just as blind: in our examples, units left out of a one-step recording impose precision floors of 0.30-1.02 bits that no sample size lowers, and in all-but-self parity networks, recordings of any length see several hidden units only through the product of their fidelities - a symmetry that one intervention broke in exact checks. The measure has sharp edges: because $\varphi$ selects a partition, and the cause and effect it specifies, by one criterion and reports another, infinitesimal changes of mechanism that cross an edge change it by 25-100% at 99% of the edges we found in random networks, and about 1% of surrogate models that fit interventional data almost perfectly misreport it by more than 20%. One-sided claims survive: the same data certify a lower bound on both sides of an edge, and we prove $\varphi>1.004$ bits throughout a 64-dimensional region of mechanisms and certify it from about $10^7$ controlled transitions with false-certificate probability at most $α$. Measurability is thus a design question with computable answers.

0.27

Emergence in Network Systems is Bounded by Boundary-Crossing Paths

cs.SI physics.soc-phnlin.AO ·2026-09-29 ·Johnny Jingze Li, Gabriel A. Silva
similarity 0.51
Abstract

Emergence, features of a whole that no part shows alone, is central to complex systems but rarely measured so as to locate its source. Viewing a system structurally, as a network of interacting parts, we evaluate emergence as what observing the coupled whole reveals beyond, or erases from, the combined observations of the parts. For observations that build what they see route by route, we show that every emergent feature is produced by its own route across the interface between parts, a route that traces it to the components and interactions responsible. We show the number of all such boundary-crossing paths bounds the total emergence discrepancy, and equals the emergence of measures that count routes. The bound is computed locally around the interface, validated through network examples, and in a nervous system the routes show that interneurons relay most of the emergence. Our work associates emergence with path structure in the network system that can be anticipated, attributed, and engineered.

0.33

Existence of an infinite family of substrates satisfying all the postulates of integrated information theory, exclusion included, with arbitrarily large integrated information

cs.IT q-bio.NC ·2026-09-09 ·A. Mayeux
similarity 0.50
Abstract

For infinitely many sizes we exhibit a cycle of binary units, each taking the exclusive-or of itself and its two clockwise neighbours, that is a complex in every state, no proper subsystem being more integrated, and whose integrated information grows as two raised to a power quadratic in the number of units.

0.24

Latent geometry organizes higher-order interactions

cs.SI physics.soc-ph ·2026-09-07 ·Berné L. Nortier, Jasper van der Kolk, Robert Jankowski, Simon Dobson, M. Ángeles Serrano +2 more
similarity 0.49
Abstract · Main: 11 pages, 4 figures. Supplemental material: 18 pages, 6 figures

Higher-order structures offer a natural representation of complex systems that involve interactions between groups of different sizes. A widespread feature of their higher-order structure is nestedness, whereby interactions involving smaller groups are contained within larger ones. Yet, why interactions of different orders organise into nested structures remains largely unexplained. Here, we introduce an analytically tractable geometric model of higher-order networks in which a single latent geometric space couples interactions across orders, leading to the spontaneous emergence of nestedness. We show analytically and numerically that nestedness undergoes a transition between a nested geometric regime, where it remains finite in the thermodynamic limit, and a regime where it vanishes with system size. In this regime, we uncover a weakly geometric range characterized by an anomalously slow finite-size decay, allowing substantial nestedness to persist in finite systems even when its asymptotic value vanishes. Finally, with a single geometric coupling parameter, the model reproduces the nestedness profiles observed in real-world hypergraphs across different domains. Our results reveal latent geometry as a simple organising principle underlying the nested organisation of higher-order interactions.

0.30