CS Paper Radar

Minimax Rate for Diffusion-Guided Local Intrinsic Dimension Estimation based on Fokker-Planck Equation

Jaehee Seo, Wontae Jeong, Jisu Kim

cs.LG stat.MLmath.ST ·submitted 2026-09-04 07:18 UTC · v2
theory

Abstract

Diffusion-guided methods have recently emerged as effective tools for probing the intrinsic geometry of high-dimensional data. Understanding their accuracy requires quantifying both (i) finite-scale approximation error relative to the true local intrinsic dimension (LID) and (ii) estimation error from finite samples. We address these two questions for the FLIPD field (Kamkari et al., 2024; arXiv:2406.03537), defined at the population level through the logarithmic scale derivative of a Gaussian-smoothed density and admitting a Fokker--Planck representation. For $α$-smooth densities on a regular $d$-dimensional manifold, we establish a uniform $O(σ^2)$ approximation bound relative to the manifold dimension $d$. Under expected integrated squared loss, we prove a minimax lower bound of order $(nσ^d)^{-1}$ for estimating the FLIPD field from $n$ observations when $n^{-1/(2α+d)}\leσ\leσ_0$. A plug-in FLIPD estimator based on the empirical denoising score matching (DSM) solution attains a matching upper bound. Thus the minimax rate is of order $(nσ^d)^{-1}$ and it becomes $n^{-2α/(2α+d)}$ at its smallest scale.

Comment
35 pages, 1 figure

Closest papers

Nearest neighbours by meaning across the whole index.

Low-Dimensional Embeddings for Gaussian Kernels on Manifolds

cs.CG cs.LG ·2026-09-14 ·Soumik Dutta, Kunal Dutta
theory similarity 0.62
Abstract · 57 pages, 3 figures

The Gaussian kernel is a widely used similarity measure underlying kernel methods such as kernel PCA and spectral clustering, but computing Gaussian kernel distances for many pairs of points can be expensive. Using Random Fourier Features (RFF), Chen and Phillips [ALT 2017] showed that for points in a $d$-dimensional Euclidean ball in ${\mathbb R}^N$, $t=Ω((d/\varepsilon^2)\log(dR/\varepsilon))$ features suffice to preserve all pairwise Gaussian kernel distances within a $(1\pm\varepsilon)$ factor with high probability. We establish a uniform relative-error embedding theorem for the more general setting of an arbitrary positive-reach submanifold $\mathcal M\subset{\mathbb R}^N$ of intrinsic dimension $d$. We show that $t=O((d/\varepsilon^2)\log(\operatorname{vol}(\mathcal M)^2N^{2d}/(\operatorname{vol}(B_1^d(0))^2\operatorname{rch}(\mathcal M)^{2d}\varepsilon^{2d+1}δ)))$, or approximately $O((d^2/\varepsilon^2)(\log N+\log(1/(\varepsilonδ))))$, RFFs suffice, with probability $1-δ$, to preserve the Gaussian kernel distance between every pair of manifold points up to relative error $\varepsilon$. Thus the bound depends only logarithmically on the ambient dimension and on manifold parameters such as volume and reach, while retaining the $1/\varepsilon^2$ Euclidean rate. We also prove a topological consequence: under the same RFF embedding, persistent homology is preserved in the sense that weighted Cech and Rips filtrations built from Gaussian kernel power distance are $(1\pm\varepsilon_\star)$-interleaved, where $\varepsilon_\star$ accounts for both distance distortion and kernel-weight approximation.

0.22

Interpretable intrinsic dimension estimation through componentwise calibration of distance and angle

cs.LG stat.ML ·2026-09-29 ·Chih-Hsuan Huang, Chih-Wei Chen, Szu-Chi Chung
similarity 0.61
Abstract · 35 pages, 5 figure, 2 tables

DANCo (Dimensionality from Angle and Norm Concentration) jointly calibrates nearest-neighbor distance and angular statistics and consistently reaches state-of-the-art accuracy on clean intrinsic-dimension (ID) benchmarks. Practical data, however, introduce neighborhood-relative noise and sample-amplitude heterogeneity that can distort these geometric signals. We reformulate DANCo componentwise, retaining separate distance and angular discrepancy curves so that the source of an estimate can be identified and interpreted. For the distance component, we derive a closed-form Kullback-Leibler divergence for the generic-order ratios of the generalized ratios ID estimator (Gride); when both angular parameters are matched (Full), Gride reduces mean percentage error from $27.7\%$ to $17.6\%$ at noise equal to $40\%$ of typical neighbor spacing on 24 manifolds. For the angular component, two sampling regimes motivate aligning mean direction while retaining concentration matching (Profiled). On a Gaussian scale mixture with generating dimension 70 embedded in 100 dimensions, profiling raises the Minimum Neighbor Distance (MiND) estimate from $22.8$ to $66.7$, while removing the known amplitudes restores MiND-Full to $71.9$; the control thus attributes the Full shortfall to amplitude heterogeneity. On CIFAR-10 and ImageNet, amplitude-reducing normalizations move angular location toward the references and narrow the Full-Profiled gap, an observational counterpart to the controlled mixture. Across four pretrained convolutional neural networks, Gride-Profiled, the two-nearest-neighbor estimator (TWO-NN), and the maximum-likelihood estimator (MLE) exhibit similar rise-and-fall profiles, while Full-Profiled differences identify the layers most sensitive to angular calibration.

0.26

Robustness of Diffusion Models under Distribution Shift

cs.LG math.STmath.PRstat.ML ·2026-09-23 ·Wei Luo, Neil K. Chada, Shijie Zhang, Lu Yu
similarity 0.60
Abstract · 13 pages, 2 figures

Score-based diffusion models are increasingly considered in settings where the underlying data distribution may differ from the training distribution, yet existing theoretical guarantees largely focus on the no-shift setting. In this work, we study robust score estimation under Wasserstein perturbations of a reference distribution. For the Ornstein--Uhlenbeck diffusion, we show that robust estimation decomposes into two fundamental components: the statistical cost of learning the reference distribution and the intrinsic cost of distribution shift. The latter scales quadratically with the Wasserstein radius, and this dependence is minimax optimal. We construct an explicit finite-sample estimator achieving the resulting robust minimax rate without knowing the shift radius. When the reference distribution lies on an unknown low-dimensional subspace, the statistical term adapts to the intrinsic dimension while the shift cost remains unchanged. Finally, we show that the same decomposition governs positive-time reverse sampling and obtain matching minimax guarantees in KL divergence. Together, these results characterize how finite data, intrinsic dimension, and distribution shift affect the robustness of score-based diffusion models.

0.22

Dimension-free regimes for unadjusted Langevin in ridge-separable models: A subspace robust Wasserstein analysis

cs.LG stat.MLmath.PRmath.ST+1 ·2026-10-22 ·Yuansi Chen, Francesco Pedrotti, Peter A. Whalley
theory similarity 0.59
Abstract · 30 pages, 4 figures

The Unadjusted Langevin Algorithm (ULA) is widely used for high-dimensional Bayesian inference. Classical analyses in Wasserstein-2 distance predict an asymptotic bias and a mixing time that grow with dimension, but this metric is stronger than necessary for many statistical tasks. We instead analyze the bias in subspace robust Wasserstein distance, which controls the mean and arbitrary low-dimensional marginals and is therefore well suited to observables typically estimated in practice. For ridge-separable potentials formed with i.i.d. data, we prove high-probability bounds showing a bell-shaped mixing profile as the number of data points $N$ varies relative to the dimension $d$. In the overparameterized regime, the effective dimension is the rank $N$ of the data-dependent perturbation, leading to complexity depending on $N$ rather than $d$. In the large-sample regime, Gaussian-perturbation estimates yield dimension-free complexity once $N$ is sufficiently large. In contrast, near the proportional regime $\asymp d$, we prove that dimension-free guarantees are impossible in general.

0.14

Sample-Optimal Estimation of the Fréchet Inception Distance

cs.LG cs.CVcs.DSmath.ST+1 ·2026-10-05 ·Ziyun Chen, Jerry Li, Kevin Tian, Yusong Zhu
code/datahot direction similarity 0.59
Abstract · Our code is available at https://github.com/zys996/sample-optimal-fid-estimation

The Fréchet Inception Distance (FID) is widely used to evaluate generative models, but its empirical plug-in estimator suffers from finite-sample bias [BSAG18, CF20]. We study the sample complexity $n$ of estimating FID to error $ε$ between $d$-dimensional Gaussians with bounded mean distance and covariances, when one distribution is known. Our contributions are threefold. (1) We establish tight finite-sample $Θ(\frac{d^2}{n})$ bias and $Θ(\frac{d}{n} + \frac {d^2} {n^2})$ variance bounds for the empirical plug-in estimator, establishing a $\gtrsim d^2$ sample complexity. (2) To debias the empirical plug-in estimator, we generalize the ${\rm FID}_\infty$ estimator of [CF20] to extrapolation methods of arbitrary order $k$. We further prove tight bias and variance bounds of $Θ(\frac{d^{k + 2}}{n^{k + 1}})$ and $Θ(\frac d n + \frac{d^2}{n^2})$ for any order-$k$ extrapolation under our framework. (3) We introduce Relative Taylor Debiasing (RTD), a new, computationally efficient FID estimation algorithm using debiasing techniques inspired by U-statistics. We show that RTD achieves an $O(\frac d {ε^2})$ sample complexity, and prove that this is optimal. We provide a complementary empirical evaluation of our new estimators. Our experiments on synthetic Gaussians validate the predicted residual bias and support the tightness of our bounds. On ImageNet with Inception embeddings, RTD achieves the lowest mean estimation error at the standard 50K sample budget, while our second-order variance-aware extrapolation estimator (VALE$_2$) uses only 10K samples to achieve accuracy comparable to FID$_\infty$ at 50K samples.

0.46

Minimax Rates and Improved Spectral Convergence for Multi-Manifold Clustering

cs.LG stat.ML ·2026-10-11 ·Haoyu Chen, Chenghui Li
similarity 0.59
Abstract

We study eigenpair estimation for the tensorized, density-weighted Laplace--Beltrami operator from $n$ independent data points sampled from a multi-manifold mixture distribution where manifolds may intersect. Let $d$ denote the largest intrinsic dimension of the manifold components and $s\geq2$ the smoothness of the sampling densities. For simple, nontrivial eigenpairs, we establish the minimax rate $n^{-s/(d+2s)}$ for approximating the eigenvalue and eigenfunction of the tensorized Laplace--Beltrami operator, and show that a continuum plug-in estimator based on kernel density estimation attains the same rate when the supporting manifolds are known. We further analyze a manifold-agnostic graph Laplacian estimator and prove that its eigenpairs converge to those of the tensorized Laplace--Beltrami operator at rate $n^{-1/(2.5d-1)}$, up to logarithmic factors, improving upon previously known spectral convergence rates for multi-manifold learning. Numerical experiments on synthetic multi-manifold data corroborate the rates guaranteed by our theorems. These results characterize the information-theoretic difficulty of multi-manifold spectral estimation and quantify the additional cost of learning without oracle component information.

0.23

An Optimization Framework for Denoising Generation

cs.LG ·2026-11-17 ·Xinyu Zhou, Jiawei Zhang, Stephen J. Wright
similarity 0.57
Abstract

Noise-unconditioned denoising learns a single score, so generation can be viewed as gradient ascent on a fixed log-density landscape. This optimization interpretation leaves two questions: do trajectories approach meaningful data, and do different initializations retain diverse outcomes? We establish both phenomena for empirical and population multi-scale Gaussian-mixture objectives. Under explicit posterior and landscape conditions, ascent converges almost surely within $O(mσ_{\min}^2/τ)$ of an $m$-dimensional reference manifold with reach at least $τ$. We derive a source--sink identity for the same fixed field and use it to prove finite-resolution regional coverage. In an affine construction, we further identify a positive-bandwidth regime where even empirical-score optimization can avoid training-sample memorization while preserving diversity at the training resolution. The same fixed objective naturally supports penalty-guided controllable generation. Experiments support our predicted mechanisms, show improved robustness under large-step guidance, and demonstrate downstream gains consistent with finite-data generative generalization.

0.15

The Manifold-Index Model: provably and efficiently avoiding the statistical and computational curse of dimensionality

cs.LG stat.MLmath.ST ·2026-11-15 ·Yantao Wu, Mauro Maggioni
similarity 0.57
Abstract · 89 pages, 17 figures, 8 tables

We study vector-valued regression with predictors filling a tube around an unknown $m$-dimensional manifold $\mathcal{M}$ in $\mathbb{R}^D$. In the Manifold-Index Model, the mean response is $F=f\circΠ_{\mathcal{M}}$, where $Π_{\mathcal{M}}$ is the nearest-point projection onto $\mathcal{M}$ and $f:\mathcal{M}\to\mathbb{R}^p$ is a possibly non-injective link function. Multiscale ANisotropic lOcal Regression (MANOR) estimates tangent planes by multiscale PCA of predictors with similar responses, then refines them by pooling outer products of local slopes. The neighborhoods narrow tangentially while retaining complete normal fibers. We assume a Lipschitz link differential and bounded response noise. For $p\ge m$, under our geometric and sampling conditions, $n$ independent observations give mean tangent-space error of order $n^{-2/(m+4)}$ and integrated squared prediction error at the $m$-dimensional nonparametric rate $n^{-4/(m+4)}$, up to logarithmic factors. In a scaling regime with fixed intrinsic dimension, model and scale parameters, and ratio of manifold volume to the $m$-th power of its reach, the required sample size grows polynomially in $D+p$, and computation is polynomial in $n$, $D$ and $p$. Numerical experiments on curves with non-monotone links and surfaces with vector-valued responses compare observed convergence rates with the theoretical bounds; a Langevin dynamics example tests prediction of hitting probabilities under an approximate manifold-index model.

0.36