CS Paper Radar

Finite-Sample Guarantees for Fully Discrete Conditional Diffusion Models

Qiuyun Zhu

cs.LG stat.ML ·submitted 2026-10-09 16:59 UTC · v1

Abstract

We establish finite-sample guarantees for conditional diffusion models learned from data and implemented with finitely many denoising steps. Our approach aligns the score-training objective with the sampling grid, allowing statistical estimation and numerical discretization errors to be analyzed together. Under a smooth conditional density model, we derive high-probability bounds on the generated distribution's error and quantify the required number of score evaluations. For every smoothness index $β>0$, a suitable grid yields averaged conditional Kullback--Leibler (KL) divergence $\widetilde O_p(n^{-2β/(d_x+d_y+2β)})$ relative to the original target with or without early stopping. Both the full and early-stopped samplers attain average target total-variation (TV) error $\widetilde O_p(n^{-β/(d_x+d_y+2β)})$. Here $n$ is the sample size, $d_x,d_y$ are the data and conditioning dimensions, and errors are averaged over the conditioning variable. The total-variation bound matches the rate established in existing conditional distribution-estimation theory, up to logarithmic factors.

Comment
31 pages

Closest papers

Nearest neighbours by meaning across the whole index.

Convergence of Denoising Diffusion Probabilistic Models: A Rényi Divergence Perspective

cs.IT ·2026-10-18 ·Wenliang Liu, Santiago Mazuelas, Guanding Yu, Renjie Liao, Lele Wang +1 more
similarity 0.70
Abstract

Denoising diffusion probabilistic models (DDPMs) have emerged as a powerful framework for generative modeling by transforming standard Gaussian inputs into outputs that approximate a target distribution. While DDPMs have been proved to converge in total variation, KL divergence, and Wasserstein distance, these discrepancy measures share a fundamental limitation of being insensitive to distributional mismatch in low-probability regions. To address this, we adopt the stronger Rényi divergence as the discrepancy measure. Leveraging a Girsanov-type change-of-measure argument, we establish the first Rényi characterization of DDPMs to the best of our knowledge, showing that the total error decomposes into a mixing error, a discretization error, and a score estimation error. We assume the target distribution is sub-Gaussian and characterize the score error via the cumulant generating function, relaxing both the log-Sobolev-inequality and $L^\infty$-score-error conditions used in prior Rényi divergence analyses of inexact Langevin Monte Carlo, a score-based sampling algorithm closely related to DDPM. Our results also establish an iteration complexity for achieving $ε$-accuracy in $α$-Rényi divergence that is asymptotically proportional to $α^3/ε$. Finally, by extending our analysis, we establish the first Rényi divergence convergence guarantee for critically damped DDPMs, a kinetic variant governed by critically damped Langevin dynamics. Similar to existing characterizations of critically damped DDPMs, this result shares the structural scaling of standard DDPMs, differing by at most a logarithmic factor in the mixing error exponent, and supports the conjecture that critically damped DDPMs do not yield a theoretical improvement.

0.21

Sobolev Regularized Score Difference Estimation in Diffusion Models

cs.LG stat.MLmath.OC ·2026-08-18 ·Chenghan Xie, Jose Blanchet, Renyuan Xu
publishedtheory similarity 0.67
Abstract · Accpeted by ICML 2026

Estimating the difference of two Stein's score functions is a fundamental problem in generative modeling. In particular, score differences arise naturally in transfer learning, where the score difference provides the mechanism for adapting a pre-trained model to a new target distribution, and in diffusion model-based post-training methods such as discriminator guidance. Existing estimators for score differences in these settings either lack of statistical consistency or are difficult to scale up in high-dimensions. We propose a statistically consistent and scalable estimator for score differences based on Sobolev regularization, which plays a crucial role in ensuring consistency and stablizing the training in the small-sample regime. Mathematically, we establish a convergence rate of $O(n^{-\frac{s-1}{d+2s-2}})$ where $d$ is the dimension and $s$ denotes the smoothness of the underlying densities, and provide a minimax lower bound of $\tildeΩ(n^{-\frac{2(s-1)}{d+2s}})$ (in mean-squared error). Empirically, our estimator exhibits significantly improved stability in small-sample regimes compared to existing methods. We demonstrate its effectiveness on real-world tasks, including transfer learning for ECG signal generation, where it substantially outperforms non-regularized score difference estimators in downstream classification performance.

0.64

Finite-Time Online Sample Complexity for Conditional Diffusion Models

cs.LG cs.AI ·2026-11-01 ·Mudit Gaur, Prashant Trivedi, Sasidhar Kunapuli, Amrit Singh Bedi, Vaneet Aggarwal
similarity 0.66
Abstract

We study the finite-time training and sampling error of conditional diffusion models trained by online stochastic-gradient denoising score matching. Rather than assuming access to an exact empirical or population risk minimizer, we analyze the score network produced after a finite number of stochastic-gradient updates under an explicit localized Polyak--Łojasiewicz (PL) condition, stochastic-gradient regularity, and a trajectory-containment assumption. Our analysis separates the network approximation complexity from the online training-sample budget and propagates both errors through the reverse diffusion process. Under anisotropic smoothness, a joint intrinsic-dimensional structure for the data and conditioning variables, sub-exponential tails, and explicit approximation-class compatibility, we obtain a conditional score approximation error of order \[ \widetilde{\mathcal O}\!\left(M^{-η_{\mathrm{app}}}\right), \qquad η_{\mathrm{app}} = \frac{2α_x} {2α_x+d_x'+(α_x/α_y)d_y'}, \] where \(M\) denotes the approximation complexity, \(d_x'\) and \(d_y'\) are the intrinsic response and conditioning dimensions, and \(α_x,α_y\) are their corresponding smoothness exponents. For \(N=RB\) online denoising samples used by SGD, the finite-time optimization error is controlled explicitly in terms of the PL, smoothness, gradient-noise, and initialization constants. Combining these ingredients with reverse-process stability yields a total-variation bound that decomposes into terminal-initialization, time-discretization, approximation/localization, and finite-time optimization errors.

0.11

Asymptotically Exact Negative Guidance of Diffusion Models via Positive-Unlabeled Learning

cs.LG stat.ML ·2026-10-14 ·Hirohane Takagi, Masashi Sugiyama
top venue similarity 0.65
Abstract · Accepted to NeurIPS 2026

Negative guidance of diffusion models generates samples that avoid an undesired condition, specified either by reference samples or by a conditional score alongside an unconditional model. Existing methods inject repulsive terms derived from local information at each sample, so the resulting sampler does not target an explicit density, and its bias cannot be systematically reduced by increasing computation. We instead present a negative-guidance method that samples from an explicit target density, which uniformly handles both reference-sample and score-based specifications and suppresses probability mass near the undesired condition with an avoidance strength that parametrizes the target density itself. Realizing this target requires estimating, at each diffusion timestep, how likely each sample is to belong to the undesired condition. We cast this estimation as a Positive-Unlabeled learning problem that admits lightweight online training along the diffusion trajectory, using condition samples as positives and diffusion particles as unlabeled data. Because the guided particles drift from the unconditional marginal, we further correct the residual mismatch via Sequential Monte Carlo, yielding a sampler that is asymptotically exact in the particle limit and that turns any existing negative-guidance method into its proposal kernel. Empirically, our method approaches the ground-truth target on Gaussian mixtures and Pareto-dominates existing methods on trade-offs between avoidance and generation quality, distributional bias, and diversity, including in an image-generation setting with a pretrained score network.

0.31

Two-Stage Score Estimation for Group-Specific Diffusion Models

cs.LG stat.MLmath.ST ·2026-10-14 ·Huitao Yang, Oscar Hernan Madrid Padilla
theory similarity 0.64
Abstract

Generative modeling across related distributions raises a question: how can shared structure support accurate generation for a target distribution with limited observations? We develop a statistical framework for diffusion transfer that analyzes a shared pooled score together with a target-specific offset. Under Besov regularity of the pooled density and a stable hierarchical composition model for the offset, we derive convergence rates that separate the costs of learning these components and identify when a simpler offset yields a faster risk-bound rate than separate fitting. Under suitable boundary and regularity assumptions, we prove that the offset remains bounded and jointly regular as diffusion time approaches zero. These results allow the lower time cutoff to vanish polynomially and provide total-variation guarantees for diffusion models as density estimators. Synthetic simulations and image benchmarks provide a proof of concept for the framework.

0.19

Uniform Discrete Diffusion Models are Minimax Optimal for Estimating Distributions with Small Effective Support Size

cs.LG stat.ML ·2026-10-06 ·Dongsun Yoon, Saptarshi Chakraborty
similarity 0.62
Abstract

Discrete diffusion models have emerged as a practically successful framework for generative modeling on discrete product spaces, yet their statistical generalization properties remain poorly understood. Discrete real-world data such as text or biological sequences often concentrate on a small fraction of the astronomically large ambient space because of semantic or physical constraints, but existing bounds fail to capture this distributional structure and instead scale with the size of the ambient space, giving rise to almost vacuous error bounds. We address this gap for uniform discrete diffusion, one of the two dominant discrete diffusion paradigms alongside masking diffusion, by deriving statistical guarantees governed by the effective support size $s_n(P_0)$, a sample-size-dependent measure of distributional complexity. Given $n$ independent and identically distributed (i.i.d.) samples from an unknown data distribution $P_0$ on $[K]^d$, we show that, with appropriate choices of network size and hyperparameters, the expected total variation (TV) loss scales as $O(\sqrt{s_n(P_0)/n})$, while the expected Kullback--Leibler (KL) divergence is bounded by $O(\frac{1}{n}s_n(P_0)\log(eK^d/s_n(P_0))\log n)$. Furthermore, we show that the TV rate is minimax optimal and that the KL rate is minimax optimal up to a factor of $\log n$. Together, these upper and lower bounds show that uniform discrete diffusion successfully avoids the curse of dimensionality for distributions with small effective support size: the TV error rate depends on the ambient state-space size only through $s_n(P_0)$, while the corresponding KL rate incurs only an additional logarithmic dependence on the ambient state-space size.

0.28

Rényi Tracking Bounds for Langevin Dynamics with Moving Targets

cs.LG stat.ML ·2026-08-25 ·Yuchen Xin, Jingxin Zhan, Zhihua Zhang
similarity 0.62
Abstract

We study Langevin diffusion and Langevin Monte Carlo (LMC) when the target distribution changes over time. Under a log-Sobolev inequality (LSI), we derive non-asymptotic Rényi-divergence guarantees for tracking the current target. The framework covers continuous-time Langevin diffusion and its discretizations. We then apply the results to nonsmooth sampling based on successive Moreau envelopes. For this scheme, we give explicit choices of the smoothing parameters and step sizes, together with corresponding complexity bounds. To our knowledge, these are the first non-asymptotic Rényi-divergence tracking bounds for Langevin dynamics with discrete target updates.

0.33

Diffusion Removes Langevin's Conditioning Dependence: A Sharp Gaussian Analysis

cs.LG stat.ML ·2026-10-08 ·Adam Perbost, Francis Bach, Pierre Marion
hot direction similarity 0.62
Abstract · 49 pages (10 main + appendix), 3 figures

Despite their empirical success, why diffusion models overcome the bottlenecks of classical score-based samplers remains unclear. In this work, we leverage Gaussian distributions to isolate this phenomenon. We establish 2-Wasserstein convergence bounds for optimized hyperparameters, showing that diffusion processes achieve a sampling error of $O(\sqrt{dλ_{\max}}\log N/N)$, where $d$ is the dimension, $N$ the number of sampling steps, and $λ_{\max}$ the largest eigenvalue of the target covariance matrix. Unadjusted and underdamped Langevin dynamics suffer from an additional $\sqrtκ$ factor, where $κ$ is the condition number. These rates follow from spectral bounds which are sharp: we confirm them via matching first-order asymptotics as $N\rightarrow\infty$. Our analysis provides a rigorous characterization, in the Gaussian setting, of how time-dependent score trajectories remove condition-number dependence during sampling. By contrast, in the learning phase, we show that estimating the unnoised score by gradient descent leads to essentially the same estimator as estimating a noisy score, which suggests that the benefits of noising do not come from the learning phase.

0.35