CS Paper Radar

Deep neural networks as lattice gauge theories

Ro Jefferson, Shradha Ramakrishnan

cs.LG hep-thcond-mat.dis-nn ·submitted 2026-08-19 18:00 UTC · v1

Abstract

We modify the NN/QFT duality [1] to incorporate the layerwise permutation symmetry of the network, resulting in a $(0\!+\!1)$-dimensional lattice gauge theory, in which each layer of $N$ neurons acts as an $N$-component lattice site, and the weight matrices play the role of gauge fields living on the links. In this framework, we compute the tree-level neuron-neuron propagator which describes the evolution of layer variance in the network, and develop the Feynman diagram machinery to compute interactions in the perturbative expansion in $1/N$. In particular, we obtain a recursive expression for all corrections to the exact propagator at $O(1)$, representing statistical fluctuations in the ensemble of networks, including infinitely-many loop diagrams mediating the interactions from previous layers. We also present a preliminary analysis of neuron scattering amplitudes that contribute order-by-order in $1/N$, which provides a field-theoretic framework for studying higher-point correlations, and by extension information propagation, in deep networks. We remark on some interesting directions for future work at the intersection of neural networks and quantum field theory.

Comment
40 pages, infinite figures

Closest papers

Nearest neighbours by meaning across the whole index.

Emergent Loop Geometry in Gauge-Equivariant Graph Neural Networks

cs.LG cond-mat.stat-mechcond-mat.str-elhep-lat ·2026-10-13 ·Ali Rayat, Gia-Wei Chern
similarity 0.68
Abstract · 16 pages, 8 figures

Gauge-equivariant graph neural networks generate ordered products of gauge links through local message passing, but the geometry of these internal structures is not captured by the receptive field alone. We investigate the statistical ensemble of closed lattice walks generated by recursive node- and edge-feature operations, independently of network training. Analytical and numerical results reveal a separation between rapid growth of contour length and much slower spatial expansion: deeper compositions generate increasingly long paths through repeated traversal of relatively compact regions. Loop expansions of fermionic observables provide a physical interpretation of this hierarchy, showing how spatial locality remains compatible with high-order dependence on nearby gauge fields. These findings identify complementary roles of network depth in extending spatial reach and enriching the representations constructed within that reach, providing a geometrical perspective on learning lattice gauge observables.

0.19

Scaling and Renormalization in Neural Networks: An Effective Renormalization-Group Description of Grokking

cs.LG cs.AIhep-thquant-ph ·2026-10-26 ·Min Chen, Junqi Wang, Zeguan Wu, Wenhao Huang, Junyu Liu
hot direction similarity 0.66
Abstract · 22 pages, 10 figures

Grokking is a delayed transition from memorization to generalization in deep neural networks (NNs). We examine whether scaling and renormalization group (RG) theories can describe this dynamics, establishing a quantitative physical framework for neural generalization transitions. For modular addition on a two-layer NN, an ensemble of trained models defines an order parameter (the squared weight norm) and fluctuation proxies for the heat capacity and the susceptibility, with training epoch and weight decay serving as thermal and field control coordinates. To bridge neural architectures to real-space RG, we construct an exact coupling matrix via a second-order expansion of the loss with respect to per-neuron norms and use its Fiedler vector to define an effective geometry. This spectral ordering yields a monotonic spatial decay of coupling strength that is absent under random permutations. We then execute real-space RG by block-spin aggregating adjacent neurons and parameterizing the coupling matrix into off-diagonal band strengths. Linearizing this local RG map yields relevant scaling exponents and five derived critical exponents, with direct empirical estimates matching RG predictions within 1-10%. Overall, our framework provides a consistent effective description of grokking, suggesting the scaling and renormalization group formalism as a tool to investigate NN dynamics.

0.31

Neural Renormalization Group Flow for Percolation

cs.LG cond-mat.dis-nn ·2026-08-27 ·Anaclara Alvez, Luca Camagna, Sergio Chibbaro, Cyril Furtlehner, François Landes +2 more
similarity 0.61
Abstract · 7 pages, 5 figures

Machine learning offers a possible route to data-driven real-space renormalization when the relevant observables are nonlocal and difficult to prescribe explicitly. We explore this idea for two-dimensional site percolation developping a supervised, scale-shared neural architecture. The model recursively applies the same learned coarse-graining rule across scales, producing a latent field from which the crossing probability is predicted, while a corresponding fine-graining decoder reconstructs the largest-cluster mask. Trained only on small lattices, the model extrapolates to substantially larger systems, recovers the spanning cluster with high fidelity, and produces observables obeying the expected finite-size scaling near the critical point. We observe that to get such performance it is key that the learned latent representation exhibits critical fluctuations and scale-dependent flows consistent with the renormalization-group structure of percolation.

0.38

The Continuum Limit of the Deep Linear Network

cs.LG math.DG ·2026-10-31 ·Catherine Eva Pfaff, Karen Uhlenbeck
similarity 0.59
Abstract

The Deep Linear Network (DLN) is a simple version of a Multilayer Perceptron (MLP) in which the activation function is the identity and the bias vector is the zero vector. We examine the behavior of the model when the number of weight matrices, which is finite in the original model, goes to infinity. We show that a plausible continuum limit is a dimensionally reduced version of the Yang-Mills equation on the torus, revealing an unexpected geometric interpretation of infinite-depth DLNs. While we do not prove this rigorously, it is a very pretty calculation. It also gives some insight into several features of the construction. We note that the choices of boundary conditions on the ends of the discrete series survive better in the continuum limit. Also, the regularizing functional which uses the moment map survives nicely in the continuum limit, whereas the regularizing functional using the size of approximating terms has no sensible limit. This surely is useful information for the choices we make in the finite-dimensional theory, and might also be useful in the more complicated MLP setting.

0.52

Percolation Dynamics in Optimization: Variance Cascades and Nested Symmetry

cs.LG cond-mat.dis-nncond-mat.stat-mechcs.AI ·2026-09-02 ·Sai Niranjan Ramachandran, Suvrit Sra
similarity 0.58
Abstract · 49 pages, 9 in the main text, v2 corrected theoretical results for Adam with verified additional experiments in the appendix

We study the dynamics of Stochastic Gradient Descent (SGD), which is known to steer deep neural networks toward invariant sets that correspond to simpler subnetworks. How this steering unfolds over time remains poorly understood. We answer this by modeling the stochastic gradient flow (SGF) as a percolation process, in which nested architectural symmetries force subnetworks to merge in discrete blocks rather than by single-edge attachment. These structural transitions register as variance spikes in a macroscopic order parameter echoing physical phase transitions. We further state sufficient conditions under which the trapping argument carries over to Adam and AdamW under heavy-tailed gradient noise and measure them on a trained Transformer.

0.14

What Neural Network Field Theory Can and Cannot Realise on a Computer

cs.LG hep-thhep-lathep-ph+1 ·2026-08-21 ·Thomas R. Harvey
theoryhighly cited similarity 0.58
Abstract

One aim of neural network field theory is to put a quantum or effective field theory on a computer, with the network ensemble itself as the theory. We ask how far that aim can be pushed for a function class regular enough to be computed with. Our main result is a no-go theorem with assumptions that hold for standard network architectures. We use it to separate four versions of neural network field theory, according to whether the defining object is the finite width ensemble or its infinite width limit, and whether the target we want to compute is a quantum or an effective field theory. Neither finite width interpretation is straightforwardly consistent. For finite width ensembles with finite variance at each point, the QFT interpretation fails reflection positivity, while the EFT interpretation establishes no scale separation by which the positivity violation can be placed outside its domain of validity. Of the two limit versions, one can be simulated in full and the other only in part, as only its smeared correlators are computable with a controlled error. As such, at the level of a controlled numerical computation, the QFT and EFT versions cannot be distinguished. One dimension escapes the obstruction, yet reflection positivity is shown to still fail there at every finite width for the cosine network. Two escapes from the theorem remain, giving up either finite variance at a point or exact rotation invariance, and we discuss both of these possibilities.

0.58

Learning from Duality in Neural Network Field Theories

cs.LG hep-th ·2026-11-16 ·Pietro Capuozzo, Brandon Robinson, Benjamin Suzzoni
similarity 0.58
Abstract · 1+76 pages, 7 figures

We develop neural realizations of positive-level Wess-Zumino-Witten models using bosonic Wakimoto and fermionic Fourier-Berezin fields. Level-rank duality connects the charged sectors of the fermionic construction, where explicit neural observables recover conformal block data and enable the transfer of information about unknown correlators between paired theories. We also show that distinct neural presentations of the same continuum current algebra exhibit different finite-width behavior, with approximation accuracy depending on architecture, regulator, and parameter distribution. These results connect the exact mathematical structure of conformal field theory with the expressive and computational properties of neural field theories.

0.31

Neural Network Field Theory at Finite Width

cs.LG hep-thcond-mat.dis-nn ·2026-08-21 ·Christian Ferko, Aaron Mutchler
highly cited similarity 0.57
Abstract · 37 pages

Under mild assumptions, any quantum mechanical (QM) model or quantum field theory (QFT) admits a representation in terms of an ensemble of neural networks with countably many random parameters. We investigate the features of NN-QM and NN-FT models with finitely many parameters, such as a feedforward network of width $N < \infty$. We find that, generically, such models must violate one of the properties of conventional Euclidean QFTs, such as reflection positivity or cluster decomposition. We present several complementary ways of understanding which features can and cannot be preserved at finite $N$, both in QM and in QFT.

0.63