CS Paper Radar

Breakdown of Local Denoising as Semantic Speciation

Guangkuo Liu, Mert Okyay, Yifan F. Zhang, Fangjun Hu, Rahul Nandkishore, Xun Gao

cs.LG cond-mat.dis-nncond-mat.stat-mech ·submitted 2026-09-29 17:59 UTC · v1
theoryunusual

Abstract

The dynamics of generative models exhibit two apparently distinct temporal windows: a speciation window, in which a sample commits to a semantic class, and a nonlocality window, in which local context windows become insufficient for generation. Motivated by evidence of their near-concurrence in a variety of frontier models, we investigate their relationship through the spatial distribution of semantic information. Under a "common cause" hypothesis, we prove that the nonlocality window must lie in the speciation window. This hypothesis postulates that semantic labels explain a fraction of the correlations between distant tokens, a condition that is natural for many real datasets. We further give conditions under which both windows shrink to a single limiting time as system size grows, defining a "phase transition", and verify this behavior analytically in Gaussian mixtures. Together, these results identify conditions under which semantic information explains the concurrence of speciation and nonlocality, connecting two complementary perspectives on the emergence of semantic structure in generative modeling.

Comment
9 pages main, 13 pages appendix, 4 figures. Comments very welcome

Closest papers

Nearest neighbours by meaning across the whole index.

Generative diffusion in long-range correlated landscapes: Speciation cascade and the foam structure of the score

cs.LG cond-mat.dis-nncond-mat.stat-mech ·2026-11-02 ·Jerome Garnier-Brun, Giulio Biroli, Marc Mézard, Jean-Philippe Bouchaud
similarity 0.60
Abstract · 25 pages, 7 figures

While state of the art for image and video generation, the fundamental reasons for the effectiveness of generative diffusion remain unclear. In particular, the observed advantage of following an entire reverse process with respect to single-shot or autoregressive methods, and for what type of data it may hold, is not well understood. In this work, we aim to better understand the role of the reverse process by considering a complex yet analytically tractable data model, the Gibbs measure of a high-dimensional Gaussian random potential with long-range correlations. Focusing on the exact score, we show that the speciation threshold, in which a trajectory commits to one class of the data, becomes a speciation window, in which the reverse process progressively resolves the Parisi tree of pure states of the full replica-symmetry breaking measure, committing to ever finer branches down to a single pure state. Importantly, these commitments take place at noise levels where the noisy sample distinguishes the pure states only through an energy of order one, beyond the reach of standard replica saddle points. Constructing a Larkin argument suited to such $O(1)$ energy gaps, we describe the score as a foam, with slab-like cells separated by shock-like boundaries. In the variance-exploding process we consider, this window is followed by a shrinkage of the samples towards the data support, and finally by reconstruction within the selected pure state. The denoising error of the exact score follows these stages, dropping during the speciation window with a profile set by the Parisi function alone, and then within the selected state. We believe that the generative foam we describe, and in particular the boundaries between cells, could play an important role in determining where the errors of trained diffusion models redistribute the mass of the generated samples when the data are long-range correlated.

0.29

Gluing Local Contexts into Global Meaning: A Sheaf-Theoretic Decomposition of Transformer Representations

cs.CL cs.LG ·2026-10-12 ·Bryce Grant, Peng Wang
publishedcode/data similarity 0.57
Abstract · NeurIPS 2026 Poster, ICLR Unifying Concept Representation Learning

Prior research into interpretability does not test whether a representation stays consistent under paraphrasing. Each pair of paraphrases can be thought of as a local witness that two activations should be the same. From these local witnesses we can build a cellular sheaf on the paraphrase graph. The question of what is consistent under paraphrasing can be answered by the Laplacian $L_\mathcal{F}$ given a frozen model. No training required. Our kernel $H^0$ gives us a content subspace. We show that with only 20 dims the kernel outperforms the subspace that LEACE retains with all dims on held-out CounterFact fact retrieval by $+17.1$\,pp on average across 8 models. By constraining all paraphrases of a fact together we see an additional increase of $+6.5$\,pp. Our spectral complement shows us where the causal influence is concentrated. Ablating the coordinates that vary the most when paraphrased (which approximates ablating the spectral complement at the coordinate level) leads to $5.6$ to $26.5\times$ larger changes in model outputs compared to ablating coordinates with matched per-coordinate variability. We find this across 9 models from 124M to 13B params. By stacking layers we can create a grid which has $b_1 = N(L-1)$ algebraic 1-cycles. The harmonic mass derived from the model varies by nearly 3 orders of magnitude between the 6 architectures. It also tracks the order of steering fragility (Spearman's $ρ= 0.90$, exact one-sided permutation $p = 0.014$). The kernel, its spectral complement and the harmonic mass all come from the same paraphrase graph. Project page: https://cwru-aism.github.io/gluing-lc-page/.

0.71

A phase diagram for training and inference in language models

cs.LG cond-mat.dis-nncond-mat.stat-mechcs.CL ·2026-11-15 ·Zhuo-Yang Song, Zeyu Li, Qing-Hong Cao, Ming-xing Luo, Hua Xing Zhu
similarity 0.56
Abstract · 20 pages, 12 figures, 4 tables

A language model is a complex system whose behavior is determined jointly by the Transformer and the corpus, yet how the representation geometry in the Transformer relates to the actual behavior of language models remains open. We analyze how the representation geometry changes along the forward propagation of actual language models and characterize the representation geometry of a document at a given layer by two collective coordinates, document distinguishability and document dimension. On the phase diagram spanned by these collective coordinates, inference is represented as a flow along depth. We find that, for untrained models, depending on their initialization scale, most of the 156 inference flows move toward the collapse and noise regions of the phase diagram. By contrast, the inference flows of 8 trained models from 5 architecture families all enter and reside in the central region of the phase diagram. Describing the evolution of the flows with a discrete Langevin equation, we find that the difference between the inference flows of untrained runs and those of trained models reduces to whether the effective force on the flows in this region is of saddle or stable-node type. On Pythia-1.4B, we interpret the training process as a phase transition. These results give a unified gateway to the relation between representation geometry and language-model behavior.

0.30

Language Model Activations Inhabit Privileged Error-Correcting Basins

cs.AI cs.CLcs.LG ·2026-10-03 ·Matthew Finlayson, Francisco Pernice, Eric Todd, Amir Zur, Daniel Wurgaft +14 more
similarity 0.55
Abstract

Language models exhibit remarkable robustness, continuing to produce coherent text even when their activations are perturbed by interventions like linear steering. We hypothesize that this robustness is a result of passive dynamics, i.e., constraining mechanisms in the forward pass that funnel activations toward "good" regions that produce coherent outputs. To investigate these hypothesized error-correcting mechanisms, we probe the geometry of language model activation space by observing the action of model layers on low-dimensional curves. In doing so, we discover that model activations occur within a cluster of distinct attracting basins, which differentiate natural activations geometrically from distributionally similar synthetic activations. Applying this lens to language model steering, we observe feature-specific basins along semantic steering directions, and find that steering moves activations between these basins. To demonstrate the active role of this geometry in neural computation, we show that adaptively modulating steering strength to transport activations across basins improves inter-language steering, significantly increasing the probability of sampling tokens from the target language compared to fixed-strength steering. Our findings establish analysis of activation space geometry as a promising approach to interpreting and controlling language models.

0.27

Not All Answers Are Contextually Persuadable: Inference Dynamics in Large Language Models under Contextual Influence

cs.AI ·2026-10-03 ·Zongye Hu, Weiqing Luo, Yanjie Fu, Yu Gan, Haofeng Zhang +1 more
top venue similarity 0.54
Abstract · Published in the Proceedings of the 43rd International Conference on Machine Learning (ICML 2026)

At the core of modern prompting techniques is contextual sensitivity, the ability of large language models to adapt their predictions based on inference-time context. Despite its central role, inference behavior under strong contextual influence remains poorly understood, particularly at the level of internal inference dynamics. We introduce a theoretical framework for analyzing contextual influence through inference dynamics, enabling quantitative characterization of inference behavior beyond output-level answer changes. Our analysis shows that inference dynamics do not exhibit unbounded drift under repeated contextual assertions. Instead, predictive representations converge to stable, query-dependent regimes that fundamentally constrain whether contextual signals can alter a model's prediction. This leads to a surprising finding: Repeated contextual assertions do not act as accumulating evidence during inference and may therefore fail to alter a model's prediction even under unbounded repetition, while in other cases a prediction change becomes inevitable. We empirically validate our theoretical predictions, demonstrating strong alignment between theory and observed inference behavior. These contributions offer a principled pathway toward characterizing the limits of contextual influence during inference, providing practical implications for model development.

0.49

Measuring Collective Semantic Change in Populations of Language Model Agents

cs.MA physics.soc-phcs.SI ·2026-08-13 ·Elena Kopteva
similarity 0.54
Abstract · 43 pages, 13 figures

Collective semantic change in populations of language model agents is a measurable dynamical phenomenon. We present a passive longitudinal instrument called Kopterix that observes the semantic state of an agent population as a sequence of bounded observations under a protocol defined before the observations begin. Each observation divides the sampled feed by post age into surface, mid-stream, and residue layers, which makes semantic differences across content age measurable alongside run-to-run change. We validate the instrument on Moltbook, an agent-native social platform, over a two-month window of scheduled observations, with the periodicity check extended across approximately four months. At the lexical level, rarefied entropy resolves an April-May difference in the evenness of the stored top 200 unigram distributions, and adjacent states are lexically closer than states paired after timestamp shuffling. At the geometric level, grand mean centering exposes the scale of a common embedding direction, and scheduled shuffle checks support a recurring excess in the mid-stream to residue separation relative to the shuffled reference. At the temporal level, detrended scalar quantities and centered layer centroids lose much of their similarity over several hours, and a weaker positive component declines across longer separations with no strong weekly recurrence. Several attractive apparent structures failed their controls, and each reading is limited to the level its controls support. The design applies wherever a population of agents produces a timestamped language environment that can be observed repeatedly and divided by content age.

0.10

Semantic Lenia: Emergence of Homeostatic Solitons within the Semantic Space of Large Language Models

cs.CL cs.AInlin.CG ·2026-08-12 ·Yoshihiko Kayama
code/data similarity 0.53
Abstract · 18 pages, 6 figures. Code, datasets, and interactive phase diagrams are available at https://y-kayama.github.io/semantic-lenia/

We introduce Semantic Lenia, an artificial life framework that transforms Large Language Model (LLM) inference from a static optimization problem into a continuous, closed-loop dynamical system. By establishing a non-linear homeostatic feedback loop to dynamically balance semantic attraction and syntactic repulsion, we demonstrate the emergence of ``Homeostatic Solitons''-metastable semantic structures that actively resist repetitive crystallization. Our exhaustive parameter sweeps map a critical ``Habitable Ridge'' where applied steering forces balance the model's intrinsic syntactic inertia. This approach successfully maintains generative trajectories in a numerically sensitive critical regime, triggering profound abductive leaps without structural collapse, and reveals a capacity-dependent scaling trend in the syntactic inertia across different model sizes.

0.26

Dancing Tokens on the Vocabulary Network from LLM Generation

cs.AI cond-mat.dis-nn ·2026-11-13 ·Yuan-Chao Hu
similarity 0.53
Abstract · 24 pages, 6 figures

Large language models (LLMs) generate text one token at a time, yet how a particular generation relates to the geometry learned in pre-training remains unclear. The fundamental explainability of LLMs is elusive. Here, we build a similarity network from the input embeddings of six open LLMs and map generation onto it. The token-embedding network is scale-free while heavy-tailed, as are random embeddings with matched covariance, but far more fragmented. Distillation barely changes this topology because input embeddings are largely inherited. The knowledge space is thus deterministic from pre-training. Consecutive generated tokens are rarely graph neighbours and, in most models, no more often than in shuffled order, so generation is not a walk on the predefined network, although the tokens it visits are concentrated within it. Through the logit lens in a layer- wise manner, tokens settle late in depth in most models, and repetitive models remain confined to small domain-specific sub-graphs. These observables could serve as inference-time monitors of degeneration. We thus provide a fresh perspective on understanding LLM generation in the microscopic way from node or link prediction by graph learning.

0.14