CS Paper Radar

A General Capacity Frontier of Complex Networks

Nicholas Kunz, H. Oliver Gao

cs.SI physics.soc-ph ·submitted 2026-10-07 07:16 UTC · v1

Abstract

From cells to cities, ecosystems to economies, complex networks exhibit peak intensities that define their empirical limit. Estimating these limits traditionally requires domain-specific knowledge. We demonstrate that a wide range of complex networks share a capacity frontier that constrains these maxima. We formalize this regularity by proposing Structural Capacity Theory for Network Substrate Structured Systems. These systems couple a fixed network substrate with an observable dynamical process. The frontier is learned from a vector representation of the network substrate and modulated log-additively by a vector representation of the dynamical process. Frontiers trained on one set of domains reliably bound empirical maxima of entirely different ones without observing their peak intensities or simulating their dynamics. Independent learning paradigms converge on consistent upper bounds. Frontier performance is stable under perturbation, yet degrades under deliberate falsification, with no performance gain under ablation. These findings support the capacity frontier as a generalizable constraint that describes how network structure bounds dynamical processes across otherwise unrelated systems.

Comment
38 pages, 4 figures, 9 supplementary tables, 2 supplementary notes

Closest papers

Nearest neighbours by meaning across the whole index.

NGN: Learning Neural Network Size as a Differentiable Count

cs.LG ·2026-09-23 ·Lixing Li
similarity 0.60
Abstract · 20 pages, 7 figures

Neural network size is usually chosen before training, separating architecture selection from weight optimization. We introduce the Neurogenesis Network (NGN), a differentiable parameterization for learning how many ordered structural components a model should use. For each ordered component group, one learnable boundary selects an active prefix while the model parameters are trained. The boundary can grow from a compact initialization and can be deployed by discarding components beyond the learned boundary. Controlled experiments examine convergence of the learned boundary, the performance of deployed prefixes, and comparisons with fixed-size models and alternative approaches to learning capacity. We then apply the same mechanism to MLPs, convolutional and graph networks, Transformers, state-space models, LoRA, and adapters. Across these settings, deploying only the learned prefix usually changes performance little, and the selected architectures perform similarly to fixed models trained at the same size. These results show that structural capacity can be optimized directly as a count.

0.22

Stability and Diversity of Networked Self-Consuming Generative Ecosystems

cs.LG cs.CY ·2026-10-07 ·Xiukun Wei, Yang Zhang, Xueru Zhang
published similarity 0.59
Abstract · Published as a conference paper at NeurIPS 2026

The widespread deployment of generative AI has made it increasingly difficult to distinguish synthetic content from real data. Consequently, synthetic data is inevitably incorporated into the training pipelines of future model generations, forming a self-consuming training loop. Prior work has studied the effects of such recursive self-consuming training, but analyses have largely been limited to isolated models, where a model consumes only its own synthetic data, or to simplified interactions between two models. This paper takes a first step toward understanding networked self-consuming generative models, in which multiple models consume synthetic data generated by one another through complex interaction pathways. We introduce a theoretical framework representing models as nodes in a directed, weighted graph, with edge weights governing the flow of synthetic data among models. Using this framework, we analyze the long-term behavior of networked models under retraining dynamics, establishing conditions for convergence and characterizing the resulting fixed points. We further investigate how the system's long-term stability and diversity are shaped by each model's access to real data, cross-model data consumption, and the structure of the interaction graph.

0.69

The Capability Manifold and ML Scaling Laws

cs.LG cs.AI ·2026-09-23 ·Syed Ali Raza Zaidi, Maryam Hafeez
similarity 0.56
Abstract

Existing machine learning (ML) scaling laws relate predictive loss to compute, model parameters, and data. However, as models are increasingly deployed through agentic harnesses, loss alone is insufficient to characterize downstream performance: models with similar loss can exhibit different capabilities in reasoning, retrieval, planning, and adaptation. Yet, no unified framework connects such capabilities to the coupled resources available across the ML lifecycle. We bridge this gap by introducing a capability manifold, a multidimensional framework mapping downstream capabilities to pre-training, post-training, and test-time resources through bounded scaling functions. Analytical Jacobians quantify capability sensitivity to resource changes and interactions. As an initial application, we embed Kaplan- and Chinchilla-type scaling laws and test-time compute within the framework, demonstrating how existing scaling relationships can be unified as trajectories on a common capability manifold.

0.24

SoftModel: A Neural Model That Grows Its Own Topology -- Governed Structural Growth for Continual In-Service Learning

cs.LG ·2026-08-17 ·Zhoumin Xie
similarity 0.55
Abstract · 99 pages, 16 figures, 12 tables

Today, a neural system is almost always used in two phases -- trained, then deployed -- and in that regime it freezes twice: training ends, and the topology itself was never a degree of freedom. We take the opposite premise as an axiom -- total plasticity: no part of a model, including its structure, is ever frozen -- and derive the governance a lifelong learner then requires. The design's target regime is continual, in-service learning: a long-lived model on a non-stationary stream, whose stability comes from governance rather than immobility and whose capacity follows demand. The result is a growable soft model: an algebra of structural operators (width, hierarchy, composition, input interface, grown cycles, attention heads), each exact at application, budgeted, and audited, with adoption decided solely by a held-out reality gate that treats parametric and structural change uniformly. A complete from-scratch system realizes the whole account; its factory surface is operated end-to-end by a production LLM. Two conclusions follow from the axiom by construction: stability under lifelong change becomes an audit property of the lifecycle, and structure that follows demand removes the silent cap a fixed topology places on later capability where the capacity floor binds. A third is measured: in the worlds where this was measured, the marginal value of new capacity was unobservable before adoption, so workable growth governance took its ex-post form. The same governance extends to evaluative signals, and the core method is evaluated on standard continual-learning benchmarks, where governed growth preserves the ability to keep learning along long task sequences. A pre-registered experimental program adjudicates the mechanism and value claims on the tested problems and reports its failures at full prominence; the map -- positive and negative -- is the contribution.

0.36

Mesoscopic learning dynamics of extensive-width neural networks

cs.LG stat.MLcond-mat.dis-nncond-mat.stat-mech ·2026-10-16 ·Shu Zhou, Jean Barbier, Sebastian Goldt
similarity 0.55
Abstract

Understanding how stochastic gradient descent learns internal representations in wide neural networks trained on rich target functions is a central problem in machine learning. We study this question in two-layer neural networks learning target functions represented by networks of the same architecture, with arbitrary nonlinearity, and with network width and input dimension growing proportionally. In this regime, standard approaches for describing the learning dynamics require tracking a diverging number of overlaps, which measure correlations between learned and target features, leaving no obvious description in terms of a few order parameters. Here, we develop a mesoscopic description of learning by characterizing the distribution of overlaps. In the early stages of learning, we find that a Gaussian description of overlap statistics reduces the learning dynamics to a few collective variables that track learning of the target function's linear and quadratic components. We then estimate the timescale of specialization, when Gaussian closure breaks down and individual learned features begin to strongly correlate with specific target features. We find that specialization in extensive-rank networks is accelerated via a quadratic shielding mechanism, whereby learning the linear and quadratic components of the target function filters gradient fluctuations. We finally describe how the activation function determines the pattern of specialization and characterize the subsequent refinement of learned features. Together, these results show how collective learning gives way to individual feature discovery and clarify when learning in large neural networks admits a simple statistical description.

0.32

Emergence in Network Systems is Bounded by Boundary-Crossing Paths

cs.SI physics.soc-phnlin.AO ·2026-09-29 ·Johnny Jingze Li, Gabriel A. Silva
similarity 0.55
Abstract

Emergence, features of a whole that no part shows alone, is central to complex systems but rarely measured so as to locate its source. Viewing a system structurally, as a network of interacting parts, we evaluate emergence as what observing the coupled whole reveals beyond, or erases from, the combined observations of the parts. For observations that build what they see route by route, we show that every emergent feature is produced by its own route across the interface between parts, a route that traces it to the components and interactions responsible. We show the number of all such boundary-crossing paths bounds the total emergence discrepancy, and equals the emergence of measures that count routes. The bound is computed locally around the interface, validated through network examples, and in a nervous system the routes show that interneurons relay most of the emergence. Our work associates emergence with path structure in the network system that can be anticipated, attributed, and engineered.

0.33

RiverScan: Learning to Accumulate on River Networks

cs.LG ·2026-11-13 ·He Zhu, Zhiyi Gao, Youhuan Yang, Han Quan, Ming Li
similarity 0.55
Abstract · 17 pages including references and appendices; 6 figures, 5 tables

Accurate runoff prediction across large river basins requires models to capture heterogeneous local hydrological responses and their accumulation downstream. River-network connectivity provides a natural structure for this task, but using it effectively requires learning how to combine contributions from subcatchments of different sizes. We frame this challenge as learning to accumulate and introduce RiverScan, a spatial operator based on area-weighted selective state-space recurrence. RiverScan recursively combines local temporal representations with accumulated upstream states in topological order. Contributing-area fractions account for subcatchment size, while learned retention controls how upstream information persists downstream. Experiments on 608 stations in the Danube basin show that RiverScan outperforms the evaluated temporal and graph baselines in overall predictive performance. Structural controls and ablations demonstrate that composing heterogeneous subcatchment representations in a manner aligned with hydrological accumulation substantially improves runoff prediction, with the benefits of correct topology growing along the river-network hierarchy. These findings establish physically structured accumulation as an effective inductive bias for hydrological modeling and motivate finer-grained, fully distributed AI-based hydrological models.

0.20

Strategic Origins of Network Structure

cs.GT physics.soc-phcs.SIecon.GN ·2026-10-17 ·Christoph Riedl, Rory Smead
theory similarity 0.55
Abstract

The structure and dynamics of complex networks are intimately linked. Despite immense progress in statistically characterizing these systems, a generalized mechanistic framework explaining why specific mesoscale structures emerge remains elusive. Here we introduce the game-graph equilibrium, the stable rest point of a payoff-driven partner-choice dynamic, analogous to a Nash equilibrium. This reveals network structure as the fossilized record of strategic interaction. We prove analytically that two-type games have exactly four network topologies, which are their exhaustive thermodynamic phases: assortative, disassortative, and two core-peripheries. This result generalizes to $K^K$ topological classes for $K$-type interactions. Furthermore, four distinct Nash equilibria produce core-periphery topologies, yet their payoffs reveal whether the underlying dynamic is cooperative or exploitative. To recover the generating dynamics from observed networks, we develop mechanistic inference by constraining a block model to lie on a low-dimensional, game-theoretic manifold. Applied to real-world networks, the model yields interpretable quantities, distinguishes states that look identical to conventional methods, predicts network topologies it has never seen without retraining, and controls its own complexity without hand-tuned penalties. Networks such as international trade are well described by the model despite the hard constraint, suggesting that real-world systems are organized by payoff-driven partner choice. Our framework resolves a foundational ambiguity by offering a precise, mechanistic definition of communities: strategic equivalence.

0.44