CS Paper Radar

Recovering Weighted Tangent Geometry from a Single-Scale Score Field

Ziqi Zhao, Qingjian Ni

cs.LG stat.ML ·submitted 2026-08-23 10:07 UTC · v1
theory

Abstract

Near a smooth data manifold, one tangent space summarizes local geometry. At a branch point, the corresponding first-order object is instead a measure over tangent directions, whose normalized masses record the local share of each branch under the chosen data measure. We ask whether a score field at one noise level determines this weighted tangent geometry when the branch center and homogeneity degree $d$ are unknown. In this tangent-measure model, $d$ is the local measure dimension. Gaussian smoothing of a homogeneous tangent measure satisfies an Ornstein--Uhlenbeck eigenfunction equation. Its weak form turns score values---without score derivatives---into a linear system for the center and homogeneity degree, with an explicit rank condition and perturbation bound. After this calibration, the tangential score on one sphere is the spherical log-gradient of a scalar Gaussian--cone transform. Integration recovers that transform up to scale, and all its spherical-harmonic multipliers are positive. Thus one exact shell identifies the normalized angular measure in every ambient dimension $D\geq2$. For at most $K$ positive rays, moments through degree $2K-1$ constructively recover count, directions, and weights in arbitrary dimension. Any fixed observation scheme needs at least $KD-1$ scalar tangential components. In the plane, degree $K$ is both sufficient and necessary, and we give quantitative finite-query certificates. For finite planar $C^{1,β}$ branches with positive $C^{0,β}$ densities, we prove $O(σ^β)$ convergence from the finite-noise score to its tangent model. In controlled experiments, 50k-step training lowers validation normalized-score error across four geometries yet raises angular-moment error, separating ordinary score fit from geometry recovery.

Closest papers

Nearest neighbours by meaning across the whole index.

Matched Queries for Curvature and Density at Branching Junctions

cs.LG stat.ML ·2026-09-01 ·Ziqi Zhao, Qingjian Ni
similarity 0.61
Abstract

At a junction, a score field can reveal weighted tangent rays, yet these first-order quantities do not determine how individual branches bend or how their densities change away from the center. Recovering this missing information is necessary for describing local continuation beyond a single point, but finite observations must separate branchwise second-order effects while allowing error in the estimated center. We address this inverse problem using matched score queries at noise scales $σ$ and $λσ$. For a finite union of $C^{2,α}$ half-branches in $\mathbb{R}^D$, the normalized score has the expansion $F_σ=F_0+σG+O(σ^{1+α})$. Matched subtraction cancels the tangent contribution and exposes $G$, which depends linearly on branchwise curvature and log-density slope. Given tangent directions and weights on distinct rays, $G$ uniquely identifies all $sD$ branch parameters, and $sD$ scalar component observations are necessary. An $O(σ^2)$ center error introduces $D$ translation modes, leading to $(s+1)D$ observations under full-rank calibration, except for a translation-invariant full line. We also establish a perturbation bound and a conditional kernel-density-estimation rate. Experiments reproduce the predicted population and $N^{-1/5}$ trends and remain full rank up to $D=20$ with 16 supplied branches. In end-to-end tests for $D=3$--$5$, a known-count first-order frontend yields full rank in all 135 population systems and a median relative jet error of 0.132. With strong first-order error, matched responses reduce median parameter error by a factor of 49.4 relative to naive tangent subtraction.

0.27

The Manifold-Index Model: provably and efficiently avoiding the statistical and computational curse of dimensionality

cs.LG stat.MLmath.ST ·2026-11-15 ·Yantao Wu, Mauro Maggioni
similarity 0.55
Abstract · 89 pages, 17 figures, 8 tables

We study vector-valued regression with predictors filling a tube around an unknown $m$-dimensional manifold $\mathcal{M}$ in $\mathbb{R}^D$. In the Manifold-Index Model, the mean response is $F=f\circΠ_{\mathcal{M}}$, where $Π_{\mathcal{M}}$ is the nearest-point projection onto $\mathcal{M}$ and $f:\mathcal{M}\to\mathbb{R}^p$ is a possibly non-injective link function. Multiscale ANisotropic lOcal Regression (MANOR) estimates tangent planes by multiscale PCA of predictors with similar responses, then refines them by pooling outer products of local slopes. The neighborhoods narrow tangentially while retaining complete normal fibers. We assume a Lipschitz link differential and bounded response noise. For $p\ge m$, under our geometric and sampling conditions, $n$ independent observations give mean tangent-space error of order $n^{-2/(m+4)}$ and integrated squared prediction error at the $m$-dimensional nonparametric rate $n^{-4/(m+4)}$, up to logarithmic factors. In a scaling regime with fixed intrinsic dimension, model and scale parameters, and ratio of manifold volume to the $m$-th power of its reach, the required sample size grows polynomially in $D+p$, and computation is polynomial in $n$, $D$ and $p$. Numerical experiments on curves with non-monotone links and surfaces with vector-valued responses compare observed convergence rates with the theoretical bounds; a Langevin dynamics example tests prediction of hitting probabilities under an approximate manifold-index model.

0.29

Minimax Rate for Diffusion-Guided Local Intrinsic Dimension Estimation based on Fokker-Planck Equation

cs.LG stat.MLmath.ST ·2026-09-04 ·Jaehee Seo, Wontae Jeong, Jisu Kim
theory similarity 0.54
Abstract · 35 pages, 1 figure

Diffusion-guided methods have recently emerged as effective tools for probing the intrinsic geometry of high-dimensional data. Understanding their accuracy requires quantifying both (i) finite-scale approximation error relative to the true local intrinsic dimension (LID) and (ii) estimation error from finite samples. We address these two questions for the FLIPD field (Kamkari et al., 2024; arXiv:2406.03537), defined at the population level through the logarithmic scale derivative of a Gaussian-smoothed density and admitting a Fokker--Planck representation. For $α$-smooth densities on a regular $d$-dimensional manifold, we establish a uniform $O(σ^2)$ approximation bound relative to the manifold dimension $d$. Under expected integrated squared loss, we prove a minimax lower bound of order $(nσ^d)^{-1}$ for estimating the FLIPD field from $n$ observations when $n^{-1/(2α+d)}\leσ\leσ_0$. A plug-in FLIPD estimator based on the empirical denoising score matching (DSM) solution attains a matching upper bound. Thus the minimax rate is of order $(nσ^d)^{-1}$ and it becomes $n^{-2α/(2α+d)}$ at its smallest scale.

0.31

Interpretable intrinsic dimension estimation through componentwise calibration of distance and angle

cs.LG stat.ML ·2026-09-29 ·Chih-Hsuan Huang, Chih-Wei Chen, Szu-Chi Chung
similarity 0.53
Abstract · 35 pages, 5 figure, 2 tables

DANCo (Dimensionality from Angle and Norm Concentration) jointly calibrates nearest-neighbor distance and angular statistics and consistently reaches state-of-the-art accuracy on clean intrinsic-dimension (ID) benchmarks. Practical data, however, introduce neighborhood-relative noise and sample-amplitude heterogeneity that can distort these geometric signals. We reformulate DANCo componentwise, retaining separate distance and angular discrepancy curves so that the source of an estimate can be identified and interpreted. For the distance component, we derive a closed-form Kullback-Leibler divergence for the generic-order ratios of the generalized ratios ID estimator (Gride); when both angular parameters are matched (Full), Gride reduces mean percentage error from $27.7\%$ to $17.6\%$ at noise equal to $40\%$ of typical neighbor spacing on 24 manifolds. For the angular component, two sampling regimes motivate aligning mean direction while retaining concentration matching (Profiled). On a Gaussian scale mixture with generating dimension 70 embedded in 100 dimensions, profiling raises the Minimum Neighbor Distance (MiND) estimate from $22.8$ to $66.7$, while removing the known amplitudes restores MiND-Full to $71.9$; the control thus attributes the Full shortfall to amplitude heterogeneity. On CIFAR-10 and ImageNet, amplitude-reducing normalizations move angular location toward the references and narrow the Full-Profiled gap, an observational counterpart to the controlled mixture. Across four pretrained convolutional neural networks, Gride-Profiled, the two-nearest-neighbor estimator (TWO-NN), and the maximum-likelihood estimator (MLE) exhibit similar rise-and-fall profiles, while Full-Profiled differences identify the layers most sensitive to angular calibration.

0.26

Geometry-Aware Score-Repellent Monte Carlo

cs.LG stat.ML ·2026-11-06 ·Jie Hu, Lingyun Chen, Do Young Eun
top venuetheory similarity 0.52
Abstract · To appear at NeurIPS 2026

The recently proposed Score-Repellent Monte Carlo (SRMC) method improves MCMC sampling by tilting the target density $π$ in $\mathbb{R}^d$ with the factor $\exp(-α\,θ_n^\top s(x))$, where the score function $s(x) = \nabla_x \log π(x)$, and $θ_n$ is the score average from past samples. However, a single scalar repellence strength $α\ge0$ cannot handle both the steep and the flat directions in anisotropic energy landscapes at once. We introduce \emph{Geometry-Aware SRMC} (GA-SRMC), which generalizes the Euclidean alignment to $θ^\top M s(x)$ for any symmetric positive-definite matrix $M$. A stochastic-approximation central limit theorem under a trace constraint on $M$ identifies the trace-normalized inverse Fisher $M_{\rm opt}\propto S^{-1}$ as the unique minimizer of the worst-direction covariance bound, elevating the Fisher information $S=\text{Cov}_π(s,s)$ from a proposal-side preconditioner (e.g., FisherMALA, natural gradient) to the bound-optimal repulsion-side matrix, with a single online estimate $\widehat S_n$ serving both roles at no additional computational cost. The full-Fisher online metric $M_n$ converges almost surely to $M_{\rm opt}$ and satisfies a CLT, providing a theoretical guarantee for the online implementation of the GA-SRMC framework. On an $8$-target simulation spanning the condition number $κ\in[1.3,100]$ and dimension $d\in[2,785]$, GA-SRMC dominates SRMC in ESS by $\boldsymbol{4.3\times}$ on anisotropic Gaussian ($κ=100$) and $\boldsymbol{6.8\times}$ on MNIST under the MALA baseline, by $\boldsymbol{16\text{--}32\%}$ on three well-conditioned logistic-regression posteriors, and by up to $\boldsymbol{+48\%}$ under FisherMALA; gains peak on the most anisotropic targets, and posterior-mean MSE is lower than with SRMC in all $16$ full-Fisher implementations.

0.33

How Geometry Shapes Generative Model Evaluation on Riemannian Manifolds

cs.LG stat.ML ·2026-10-24 ·Misha Oberski, Valentijn Oldenburg, Xanthos Kekkos, Soham Chaudhuri, Emma Oosterhuis +4 more
published similarity 0.52
Abstract · NeurIPS 2026, Geometric Distributional Deep Learning workshop

On Riemannian manifolds, generative-model evaluation is geometry-dependent since curvature and dimension jointly determine distances and volume growth. Our framework systematically compares geodesic, ambient, and tangent-space versions of Wasserstein and MMD across different geometries. Using controlled-noise perturbations and Riemannian Flow Matching samples, we study metric agreement, noise sensitivity, and their relation to geometry-induced volume scaling. We find that no metric is uniformly reliable: evaluation distance can affect both measured discrepancies and noise sensitivity, varying across curvature and dimension. Our analysis leads to practical recommendations for reporting evaluation distances, comparing metric types, and calibrating metrics to the geometry of the samples.

0.22

EGGROLL, Unrolled: Understanding and Improving Low-Rank Evolution Strategies at Scale

cs.LG cs.AImath.OC ·2026-09-10 ·Ege C. Kaya, Abolfazl Hashemi
similarity 0.52
Abstract · 38 pages, 5 figures

EGGROLL (Sarkar et al., 2026) makes evolution strategies (ES) practical for LLMs by replacing dense Gaussian weight perturbations with low-rank Gaussian products, often of rank one. This choice is computationally attractive but geometrically severe: Each rank-one perturbation lies in a zero-volume subset of the ambient matrix space, despite having identity covariance. We characterize the EGGROLL update mean field at finite rank and nonzero perturbation radius as a resolvent applied to the gradient of the perturbation-smoothed objective. This transformation can make the mean field nonconservative and reverse the local stability of an optimum. EGGROLL nevertheless recovers the gradient exactly on quadratic objectives at every rank and radius. In finite populations, the additional sampling variance of rank-one perturbations relative to dense Gaussian ES decays inversely with matrix width under a local affine model, and is only $0.098\%$ at width $4096$. Finally, we introduce LOO-ROLL, a leave-one-out estimator that replaces EGGROLL's two antithetic evaluations per direction by one. At equal evaluation cost, LOO-ROLL halves estimator MSE in transformer blocks. Across fourteen post-training settings up to 14B parameters, matched-time comparisons with EGGROLL yield eleven improvements in individual paired tests and no significant loss. At 1.7B, 8B, and 14B parameters, matched-time gains are $2.9$, $14.1$, and $7.9$ percentage points on GSM8K and $12.2$, $8.4$, and $8.1$ points on MATH-500.

0.32

The information geometry of the Sinkhorn divergence on a Hilbert manifold

cs.IT math.ST ·2026-10-29 ·Paul Navas
similarity 0.51
Abstract

The Sinkhorn divergence, the debiased entropic optimal transport cost, determines a metric tensor on probability measures defined along curves, due to Lavenant et al. We construct a real-analytic Hilbert manifold of measures, the smooth stratum, on which this tensor is a weak Riemannian metric, and we continue Eguchi's construction of a geometry from a divergence to the third order. The stratum consists of the measures with a positive Sobolev density on a closed Riemannian manifold, and the strata form a scale whose intersection is the Fréchet manifold of smooth positive densities of Ay. Following it, we characterize the metric through the gradients of mean values: its cotangent form is a spectral reweighting of the Dirichlet form of a Markov chain, so the metric is of the same kind as the Fisher-Rao and the Otto metric, and on the stratum it is bounded by a multiple of the Fisher-Rao metric (but not conversely). At third order, the Sinkhorn divergence is self-dual: both connection forms of Eguchi's construction coincide with the Levi-Civita form of the metric, and the cubic tensor vanishes, as a consequence of the transposition symmetry of the transport problem. On every stratum, the Levi-Civita form is represented, so the stratum has a real-analytic Levi-Civita connection, which on the Fréchet manifold is the connection of Ay and Schwachhöfer.

0.17