CS Paper Radar

Evaluating generative samplers for lattice field theory by scale

Satoshi Iso, Lingxiao Wang

cs.LG hep-lat ·submitted 2026-11-08 03:39 UTC · v1

Abstract

Generative samplers for lattice field theory are commonly evaluated by their effective sample size (ESS). We show that a small global ESS need not imply inaccurate long-distance observables: errors spread over many short-distance modes can dominate this diagnostic. We introduce a scale-resolved diagnostic based on the overlap between coarse-grained model and target distributions. At the lattice cutoff, this overlap determines the asymptotic ESS fraction; lower cutoffs isolate long-wavelength errors. In the exactly solvable free theory, two samplers with comparable ESS yield a $32\%$ bias and no bias in the susceptibility, respectively. In two-dimensional $φ^4$ theory near criticality, trained diffusion samplers show the same separation of scales: their path-weight ESS fraction falls rapidly over the volumes studied, while low moments of the magnetization remain close to their target values. Varying the reference parameters substantially changes infrared observables, with little effect on the log-weight variance. Given the target magnetization distribution, a marginal Metropolis test maintains an acceptance rate of order one in our tests. Combined with exact local updates, it reduces the remaining mismatch, although residual bias remains at finite relaxation length. In the free field, uniformly bounded relative errors in the proposal mode variances give a volume-independent bound on the number of updates needed for a prescribed accuracy of the chain's stationary mode variances at fixed correlation length. The target marginal must be supplied or computed separately; estimating it from the samplers' own weights can inherit the poor global overlap.

Comment
27 pages, 11 figures, comments are welcome!

Closest papers

Nearest neighbours by meaning across the whole index.

Data Free Training of Diffusion Models for Lattice Field Theories

cs.LG hep-lat ·2026-10-13 ·Dennis Grinwald, Ankur Singha, Khaled Kahouli, Karl Jansen, Shinichi Nakajima
benchmark similarity 0.69
Abstract

Sampling equilibrium field configurations remains a major computational bottleneck in lattice field theory, motivating the development of generative samplers that can produce independent configurations. Diffusion models are promising for this purpose, but their standard denoising score matching objective requires samples from the target distribution, seemingly reintroducing the original sampling problem. Here, we show that diffusion models can instead be trained from their own generated samples using the target score identity, provided access to the unnormalized probability density. Importance-weighted formulations of this identity yield a consistent target throughout training, while unweighted variants define a self-consistent fixed-point training scheme. We benchmark the different approaches in two-dimensional \(φ^4\) theory and identify effective choices that enable efficient training and sampling across the symmetric, critical, and broken phases. Ultimately, our data-free training approach achieves comparable performance to training on HMC reference data.

0.40

Sampling the XY model with an attention-based neural network

cs.LG cond-mat.dis-nncond-mat.stat-mechhep-lat ·2026-10-16 ·Piotr Białas, Christian D. Morales, Tomasz Stebel, Dawid Zapolski
similarity 0.55
Abstract · 19 pages, 6 figures

We present a transformer-based variational autoregressive network for sampling lattice systems with continuous degrees of freedom and apply it to the two-dimensional XY model near the Berezinskii-Kosterlitz-Thouless transition. The architecture is combined with physics-informed ingredients: locality-informed attention and approximate probabilities. The resulting sampler significantly outperforms existing neural samplers. We train the model on lattices up to $96\times96$, verify with a mode-dropping estimator that no parts of the configuration space are missed, and reproduce the energy, magnetization and helicity modulus obtained with the traditional Markov Chain Monte Carlo algorithm.

0.22

High-Magnetization Sampling at Low Temperatures: Ising Models and Bayesian Sparse Linear Regression

cs.DS cs.LGmath.PRmath.ST+1 ·2026-09-08 ·Syamantak Kumar, Purnamrita Sarkar, Kevin Tian, Yusong Zhu
similarity 0.55
Abstract · 61 pages

Sparsity is a powerful structural resource in optimization and statistics. We develop frameworks for leveraging sparsity in sampling problems over the Hamming slice $\mathcal{X}_k^d:=\{\mathbf{x}\in\{\pm 1\}^d:|\{i:\mathbf{x}_i=1\}|=k\}$, in high-dimensional regimes where $k\ll d$ (i.e., where $\mathcal{X}_k^d$ is \emph{highly magnetized}). We use our frameworks to design improved samplers for canonical problems in the study of \emph{Ising models} and \emph{Bayesian sparse linear regression}. Our first main result considers the \emph{Sherrington--Kirkpatrick} (SK) model restricted to fixed-magnetization slices $\mathcal{X}_k^d$. We give a polynomial-time sampler for fixed-magnetization SK models at any inverse temperature $β>0$, under arbitrary external fields, provided that $k\le c_βd$ for an appropriate constant $c_β$. By combining this result with an annealing strategy for estimating normalizing constants, we obtain polynomial-time samplers for the SK model at arbitrarily low temperatures under a sufficiently strong external field of strength $h$. In the large-$β$ limit, our framework permits sampling at field strengths within constant factors of the \emph{Almeida--Thouless line} delineating the replica-symmetric and replica-symmetry-breaking regions ([dAT78]), improving polynomially over the field strength $h(β)$ required by the recent work of [BAR26]. Our second main result concerns the measurement complexity of polynomial-time Bayesian sparse linear regression. Recent work by [KSTZ25] shows how to sample from the canonical \emph{Gaussian spike-and-slab posterior} with expected sparsity $k$, at any signal-to-noise ratio, given $n\gtrsim k^3\log^3 d$ Gaussian measurements. We improve this requirement to $n\gtrsim k^{3/2}\log^2 d+k\log^3 d$, using a common sparsity-aware framework underlying both our results.

0.44

Diffusion models for the $O(3)$ nonlinear sigma model

cs.LG cond-mat.stat-mechhep-lat ·2026-10-29 ·Daniel Schubring
similarity 0.55
Abstract · 36 pages, 9 figures

A score-based diffusion model is trained on the two-dimensional classical Heisenberg model, the lattice representation of the $O(3)$ nonlinear sigma model. Neural networks are trained on three different coarse-grainings of the same training set to implement a multiscale sampling scheme that modestly improves both the accuracy and computational cost of direct sampling on the finest lattice. Stein (or Schwinger-Dyson) diagnostics for the two-point correlation function are introduced to assess the quality of the learned score without relying on reverse diffusion sampling. These diagnostics are also incorporated into an additional training loss. We discuss several technical considerations relevant to implementing diffusion models on $S^2$. These include a consideration of lifting the network input to $S^3$, as in the $CP^1$ representation, and developing a boundary-layer approximation to the short-time spherical heat kernel that extends its usefulness to larger diffusion times.

0.48

Quenched Ensemble Sampling

cs.LG stat.MLstat.CO ·2026-09-14 ·David Yallup
similarity 0.55
Abstract · 27 pages, 8 figures

Some of the sharpest challenges in sampling from the energy functions of physical systems arise at phase transitions, where the density of states changes abruptly and many sampling algorithms stall. Nested sampling is a particle method that traverses the density of states under a hard energy constraint and is known to be robust to such transitions, but its application in high dimension is limited by the difficulty of sampling under that constraint. In this work we introduce Quenched Ensemble Sampling, which generalises the hard constraint to a family of repulsive potentials at the energy boundary. This preserves the quenched path of monotonically decreasing energy while making the constrained target amenable to scalable gradient-based kernels. We demonstrate on synthetic models of phase transitions that our method estimates the marginal likelihood and draws posterior samples across a first-order transition where popular alternatives such as tempering fail. We apply the procedure to marginal likelihood estimation in Bayesian neural networks, enabling model comparison between network architectures. Finally, in a high-dimensional continuous lattice field theory, we show that this method traverses a first-order transition and estimates the partition function.

0.35

Discrete Diffusion Models via Evolving Variational Autoregressive Networks

cs.LG cond-mat.dis-nnstat.ML ·2026-09-23 ·Kewen Pan, Ying Tang
similarity 0.53
Abstract

Conventional score-based diffusion models learn scores without representing normalized densities, whereas tractable normalized models support both sampling and direct likelihood evaluation. A recent tensor-network approach provides such a representation but is largely restricted to low-dimensional lattices. Here we introduce a discrete diffusion model that parameterizes normalized probability distributions using variational autoregressive networks. Explicit Markov jump operators govern the forward noising and reverse denoising dynamics, extending discrete diffusion models with normalized distributions to spin systems on higher-dimensional lattices. We apply this framework to the two- and three-dimensional Ising models across ordered, critical, and disordered regimes, accurately computing thermodynamic quantities including free energy, energy, and magnetization. We further integrate the framework with Monte Carlo sampling, using adaptive diffusion steps to maintain high acceptance rates even at low temperatures while enhancing sample diversity. These results establish a neural-network framework for the discrete diffusion model with normalized probability distributions.

0.17

Implicit and explicit reparameterizations of the Folded Normal and Rice distributions

cs.LG stat.MEphysics.data-an ·2026-10-12 ·Luis Aldama, Kevin Dalton, Minhuan Li, Doeke Hekstra
code/dataunusual similarity 0.53
Abstract · 45 pages, 14 figures, 3 tables. Includes supporting information (pages 29-45)

Variational inference is a flexible tool for modeling experimental data and has proven useful in crystallography. Pathwise gradient estimators provide low-variance gradients for reparameterized variational inference, but are not readily available for some distributions central to crystallography. This work derives and implements implicit and explicit reparameterized gradient estimators for the folded Normal and Rice distributions in \prog{PyTorch}, and compares their variance analytically and empirically. In each case the implicit estimator achieves lower variance for the location derivative, particularly at small location-to-scale ratios, while the scale-derivative variances are nearly equal. For both distributions, implicit estimator is proved to be the conditional expectation of the explicit estimator given the sampled magnitude, so its variance is never larger. As a practical application, French-Wilson scaling is formulated as a variational inference problem, using the folded Normal and Rice distributions as variational families for centric and acentric reflections, respectively. Both estimators recover the exact posterior to better than $1\%$ on the amplitude, and converge at the same rate. The implementations are available in the open-source package \prog{rs-distributions}.

0.35

Recovering Missing Degrees of Freedom in Simulation-Trained Field-Level Generative Models

cs.LG astro-ph.CO ·2026-11-02 ·Masato Shirasaki, Shiro Ikeda
similarity 0.53
Abstract · 4+14 pages, 1+8 figures

Simulation-trained generative models are emerging as a powerful framework for field-level inference in cosmology, particularly in strongly nonlinear regimes where analytical approaches become intractable. However, the finite size of training datasets often leads to variance shrinkage, indicating that part of the realization-level variability of the underlying physical process is not captured by the learned generative dynamics. In this Letter, we interpret variance shrinkage as a diagnostic of missing effective degrees of freedom and introduce physics-informed latent modifications to the generative dynamics of a continuous normalizing flow. We show that these latent corrections recover a substantial fraction of the missing variability without requiring additional training simulations. Using weak-lensing field-level inference as a representative cosmological application, we further demonstrate through posterior analysis that the learned latent directions are actively exploited during inference and behave as effective nuisance parameters that absorb part of the mismatch between the training and target distributions. As a result, the model exhibits improved robustness to out-of-distribution shifts in field-level inference. Our work establishes variance shrinkage as a practical diagnostic for identifying missing degrees of freedom in simulation-trained generative models and provides a framework for recovering them through physics-informed modifications of generative dynamics.

0.28