CS Paper Radar

Exact Transport-Information Correspondence for Distribution-Adaptive Discretization

Koretaka Yuge

cs.IT cond-mat.stat-mech ·submitted 2026-11-04 07:03 UTC · v1

Abstract

Discretization of continuous probability distributions is an operation on the sample space, and the structure induced by this operation cannot generally be determined solely from information geometry. In our recent work on unavoidable canonical nonlinearity (UCN), we established a non-arbitrary connection between the transport cost associated with discretization and the information-geometric indistinguishability of continuous probability distributions. Through the transport-information correspondence, the UCN quantify the information-theoretic cost due to discretization as the expected Kullback-Leibler divergence between nearby continuous distributions associated with the discretization geometry. In the original UCN construction, discretization is implemented by transporting the probability mass within each cell to its geometric centroid. The resulting transport-information correspondence, however, is derived under specific conditions involving the cell geometry, spatial coverage, and the vanishing-cell-scale limit. In this work, we introduce an alternative local-map discretization scheme in which the probability mass within each cell is transported to its own probability-weighted centroid. This change in the representative-point selection enables an exact expression for the transport cost of each cell in terms of the covariance matrix of its conditional probability distribution and the Fisher metric of the underlying continuous family. We establish an exact correspondence between the resulting transport cost and the expectation of the associated Fisher-induced quadratic form, without requiring uniform cell geometry, complete coverage of the sample space, or an infinitesimal cell-scale limit. The proposed construction thus generalizes the transport-information correspondence to finite-scale discretization using arbitrary collections of non-overlapping bounded convex sets.

Comment
6 pages

Closest papers

Nearest neighbours by meaning across the whole index.

Closed-Form Gaussian Mixture Representations of d-Dimensional Triangulated Domains

cs.IT math.NAmath.PR ·2026-10-31 ·Ali Darijani, Benedikt Stratmann, Uwe D. Hanebeck, Ratan Bahadur Thapa, Jürgen Beyerer +1 more
publishedtheory similarity 0.57
Abstract · Accepted to "2026 IEEE Sensor Data Fusion: Trends, Solutions, Applications (SDF)". 17 pages and 6 figures

We derive a Gaussian Mixture (GM) representation of a full-dimensional subset of Euclidean space with finite volume from a finite simplicial triangulation in closed form. On each simplex, we replace the uniform probability density function (PDF) with a Gaussian PDF that minimizes the Kullback--Leibler (KL) divergence, arriving at the information projection onto the family of Gaussian PDFs. By exploiting affine isomorphisms between simplices, the joint convexity and bijection invariance properties of the KL divergence, identities for Dirichlet PDFs, and a partition-of-unity argument, we establish a dimension-dependent but geometry-independent sharp upper bound on the global KL divergence between the resulting GM PDF and the uniform PDF on the original domain. Numerical examples illustrate the construction and demonstrate its potential in practical applications.

0.59

Density-Reweighted Entropic Optimal Transport: Decoupling Geometry from Sampling Density

cs.LG stat.MLstat.APstat.ME ·2026-08-17 ·Keyi Li, Yuval Kluger, Boris Landa
similarity 0.57
Abstract

Dataset alignment is a central step in data analysis across science and engineering, where the goal is to match observations between datasets. Entropic Optimal Transport (EOT) offers a computationally tractable framework for this task by encoding cross-dataset affinities in a transport plan. However, when two datasets are sampled from geometrically similar low-dimensional structures with substantially different sampling densities, the EOT plan may match points by relative sampling density rather than geometric proximity, yielding geometrically misleading correspondences. To address this issue, we propose a density-reweighted EOT framework in which the influence of sampling density on the transport plan can be discounted to a desired degree, ranging from standard EOT to alignment driven purely by underlying geometry. Under suitable regularity conditions, we establish convergence of the reweighted EOT plan to a family of population-level plans whose dependence on sampling density is made explicit. Through simulations, we show that our approach recovers geometrically faithful correspondences, improving over related EOT-based frameworks when datasets exhibit substantial sampling density disparity.

0.31

Bregman geometry in regular exponential families: dual representations and relative smoothness

cs.LG stat.MLmath.OC ·2026-11-13 ·Mame Diarra Fall, Éric Barat, Nicolas Dobigeon
similarity 0.57
Abstract

Regular exponential-family distributions are ubiquitous in statistics due to their appealing and convenient properties. Notably, they can be expressed in terms of Bregman divergences, revealing fundamental connections between statistics, convex analysis, and information geometry. However, this geometric structure has not been systematically exploited to characterize these distributions with respect to their expectation parametrization. Building on a projective characterization, we establish a general construction of equivalent dual Bregman representations, together with normalization and uniqueness results. These representations express log-densities as ordinary Bregman divergences in dual coordinate systems, providing geometric characterizations of the parameter space. So-called $β$- and $α$-scaled dual representations emerge as particular instances of this framework. We further derive dual Bregman representations of conjugate priors and establish that they preserve conjugacy. Applications to classical distributions illustrate the geometric structures underlying seemingly different statistical models. Moreover, we show that these representations provide natural reference geometries for establishing relative smoothness of the primal Bregman representation of the distribution, with explicit bounds that are sharp in the scalar and separable settings. An illustrative application to the Poisson distribution sheds new light on several works in the literature, providing complementary geometric justifications for reference geometries while deriving existing algorithms.

0.27

Coupled Optimal Transport with Landmark Constraints

cs.CV ·2026-08-20 ·Xiang Gu, Jian Sun, Zongben Xu
similarity 0.56
Abstract

Existing optimal transport (OT) models primarily seek an OT map or plan between distributions by minimizing a prescribed transport cost or distortion. However, minimizing transport cost or distortion alone may fail to identify a geometrically meaningful transformation between the two distributions. To address this limitation, this paper proposes a novel coupled OT framework that leverages a small number of annotated landmarks to guide the recovery of an underlying deformation governing the distribution transformation. The coupled OT framework integrates the optimization of the transport plan and the deformation field into a unified model, where the landmark-guided deformation field and the cost-driven transport plan are coupled through a mutual-consistency constraint. As a result, the deformation is jointly determined by the annotated landmarks and cost-driven distribution matching. The proposed framework provides a principled connection between landmark-based registration and transport-based distribution matching, enabling the recovery of transport maps from sparse geometric supervision. We establish the well-definedness of the proposed model in a general variational setting and develop a finite-element-based numerical algorithm for computation whose convergence properties are systematically analyzed. The practical effectiveness of the proposed approach is verified in shape matching.

0.24

Closed-Form Gaussian Information Projection for Embedded Simplices and Simplicial Mixtures

cs.IT ·2026-11-09 ·Ali Darijani, Emadedin Soufian, Jürgen Beyerer
similarity 0.55
Abstract · 27 Pages, 3 Figures

We study Gaussian approximation of Dirichlet--Gaussian probability density functions on embedded simplices. We derive the unique information projection onto the family of nondegenerate ambient Gaussian probability density functions, obtaining closed-form expressions for the optimal Gaussian and the minimum divergence for arbitrary positive Dirichlet parameters. The minimum divergence depends only on the intrinsic simplex dimension and Dirichlet parameters, is invariant up to an affine isomorphism, independent of the ambient dimension, and independent of the normal covariance. Furthermore, by joint convexity of Kullback--Leibler divergence, we establish explicit intrinsic dimension-dependent sharp upper bounds for finite Gaussian Mixture with shared component weights. For uniform probability density functions, these bounds reduce to dimension-dependent constants. The results provide closed-form Gaussian Mixture constructions and rigorous approximation guarantees for probability density functions associated with embedded simplicial complexes. A numerical example on a circular curve illustrates the theoretical results and their connections to geometric Gaussian Mixture representations.

0.24

Diffusion Models for High-Dimensional Clustered Data: Intrinsic-Dimension Adaptivity via Bayesian Classification

cs.LG stat.MLmath.ST ·2026-08-19 ·Yuga Iguchi, Paul Fearnhead
theory similarity 0.55
Abstract

The empirical success of diffusion models in generative modelling has motivated theoretical work, including quantitative error bounds and qualitative analyses that characterise the different phases of denoising. We bring these two areas together by studying the adaptivity of diffusion models to the structured geometry of multimodal high-dimensional data that consists of multiple clusters in $\mathbb{R}^D$, each with its own low-dimensional structure, and inter-cluster separation depending on $D$. We employ $K$-mixture Gaussian distributions as a canonical framework to capture this geometry and establish two theoretical results. First, we interpret denoising as a dynamical Bayesian classifier: the mixture score is a posterior-weighted average of cluster-wise scores, and we show that, with high probability, the posterior class probabilities concentrate on a single cluster once the signal-to-noise ratio reaches the scale $Θ(\log (KD)/D)$. Second, by separately analysing the denoising process in its mixing and cluster-commitment phases, we prove that the KL error bound depends linearly on the maximum intrinsic dimension of a cluster, up to a logarithmic factor, even when $K$ grows polynomially with $D$. This improves on ambient-dimensional bounds and extends existing low-dimensional adaptivity analyses to multimodal distributions with heterogeneous, approximately low-rank covariances.

0.26

Physics of Information Geometry - Part II: Small-Step Active Inference on the Probability Simplex

cs.IT cond-mat.stat-mech ·2026-09-10 ·C. Emre Koksal, Deniz Sargun
theory similarity 0.55
Abstract

This paper is the second in a two-part investigation of the physics of information geometry. While Part I develops a physical foundation for distributional motion on the probability simplex, the present paper studies how that framework manifests in active inference. The treatment is fully self-contained and does not require familiarity with Part I. We focus in particular on active inference through small distributional steps and the geometric structure induced by such local motion. Starting from an initial distribution, an agent evolves its belief state toward a final target distribution through a sequence of constrained updates. We define a relative free energy functional with respect to the preferred distribution and extend it to a relative potential energy analogous to the Helmholtz/Gibbs free-energy decomposition. The evolution is subject to a per-step kinetic constraint expressed through the Kullback-Leibler (KL) divergence between consecutive distributions, which serves as a discrete kinetic energy on the probability simplex. Using the information-geometric Pythagorean theorem on KL balls, we show that sufficiently small local moves dominate large direct jumps, and that greedy maximization of free-energy reduction is globally optimal under the kinetic constraint. This leads to a sequential variational principle in which the optimal trajectory minimizes the associated Lagrangian of the optimization problem. Similar to classical mechanics, the Lagrangian takes on the form as the difference between the kinetic and potential terms, establishing a least-action principle for distributional motion on the simplex. The resulting optimal update admits a closed form as an exponentially tilted version of the current distribution toward the preferred distribution, parametrized by an inverse-temperature-like multiplier. We further extend the framework to incorporate state-dependent geodesic...

0.17

Streaming Algorithms for Gaussian Kernel Density Statistics

cs.DS ·2026-09-09 ·Qin Zhang
similarity 0.55
Abstract · 37 pages

Motivated by data produced by generative systems, \cite{LZ26b} formulates similarity-aware statistics via a weighted similarity graph, replacing equality with similarity in classical frequency-based statistics. Although this framework captures semantic relationships between nonidentical items, under general similarity functions even coarse one-pass approximation can require linear space. We therefore ask whether the geometric structure present in natural vector similarities can overcome this barrier. We answer this question affirmatively for the Gaussian kernel. For fixed-dimensional Euclidean vector streams, we study similarity-aware analogues of classical frequency statistics, including the number of distinct elements and frequency moments, through the diversity index and Gaussian density moments. We give one-pass sublinear-space approximation algorithms that exploit the geometric and analytic properties of the Gaussian kernel, and complement them with lower bounds. Our results show that geometric structure can fundamentally change the streaming complexity of similarity-aware statistical analysis.

0.20