CS Paper Radar

Fisher Information and Support Complexity of Gaussian Mixtures

Luca Barletta, Alex Dytso

cs.IT ·submitted 2026-10-15 11:44 UTC · v1
theory

Abstract

We study lower bounds on the Fisher information of Gaussian location mixtures and their applications to estimation problems. First, for a discrete mixing distribution, we derive a universal positive-semidefinite lower bound on the Fisher information matrix in terms of the Rényi entropy of order two. For a mixing distribution supported on $K$ points, this bound scales as $\log(K)/K^2$, improving by a logarithmic factor on the support-size dependence obtained from Stam's inequality. We then develop dimension-aware trace bounds using classical Sobolev inequalities. These bounds are expressed in terms of Sibson's $α$-mutual information and yield the scaling $K^{-2/n}$ in dimension $n$. A complementary bound involving the $α$-mutual information of order $1/2$ is obtained using the sharp Young convolution inequality. The derived inequalities are further specialized to rotationally invariant mixing distributions supported on discrete spherical shells. In this setting, an intrinsic-dimensional bound on Sibson mutual information leads to lower bounds on the Fisher information in terms of the number of occupied shells. As applications, we study least favorable priors for bounded Gaussian mean estimation. In the scalar case, we show that the least favorable prior on $[-R,R]$ has at least $Ω(R\sqrt{\log R})$ support points and that each of its probability masses is of order at most $1/R$. In the vector case, we show that the number of occupied shells grows at least linearly with $R$. Finally, Brown's identity and the I-MMSE relationship are used to translate the Fisher-information inequalities into upper bounds on the MMSE and mutual information.

Comment
24 pages, single-column

Closest papers

Nearest neighbours by meaning across the whole index.

Conditional Fisher-Information Central Limit Theorems under Log-Concavity with Information-Theoretic Consequences

cs.IT math.PR ·2026-09-07 ·Tong Ye, Liu-Quan Yao
benchmarktheory similarity 0.71
Abstract · 47 pages

We establish conditional central limit theorems in Fisher information under log-concavity in every fixed dimension. For conditionally centered normalized sums, after whitening by the averaged conditional covariance, the averaged conditional Fisher information converges to the dimension if and only if it is finite at one convolution level. The scalar criterion follows as the one-dimensional case; we also provide an independent scalar proof based on a second-order continuity theorem for Fisher production on Gaussian-smoothed, tail-controlled classes. For the original sums, the averaged Fisher information matrix converges in operator norm to the inverse averaged conditional covariance. Consequently, the conditional relative Fisher information with respect to the limiting Gaussian law vanishes, and the Gaussian logarithmic Sobolev inequality yields convergence in conditional relative entropy and conditional entropy. We give two operational consequences. For any fixed finite-constellation low-power input, the first-order conditional mutual-information slope converges to the Gaussian-noise benchmark. For Gaussian signaling at any fixed signal covariance, the mutual-information gap from that benchmark is bounded by the conditional relative Fisher deficit and hence vanishes asymptotically.

0.38

Hermite-Fisher bounds and stability for min-entropy power inequalities

cs.IT math.FAmath.MGmath.PR ·2026-09-18 ·Silouanos Brazitikos, Martin Rapaport, Tomasz Tkocz
similarity 0.69
Abstract

We derive explicit lower bounds for relative Fisher information by combining a variational principle with suitably orthogonalized Hermite-polynomial test functions. The resulting cumulant bounds are asymptotically sharp and yield lower bounds for Gaussian entropy deficits. We also establish quantitative versions of sharp min-entropy power inequalities in all dimensions. En route, we develop a stability result for Brzezinski's sharp bound for block sections of products of Euclidean balls, which may be of independent interest.

0.39

Polylogarithmic Sparsity of Randomly Reweighted NPMLEs for Gaussian Mixtures

cs.LG stat.MEmath.ST ·2026-10-01 ·Hansheng Jiang
similarity 0.69
Abstract

The nonparametric maximum likelihood estimator (NPMLE) of a Gaussian location mixture maximizes the likelihood over the infinite-dimensional space of mixing distributions. The maximizing mixing distribution can be nonunique, and the classical bound on its number of atoms grows linearly with the sample size $n$. We show that a vanishingly small random perturbation of the likelihood yields exact polylogarithmic sparsity. The resulting randomly reweighted NPMLE maximizes a weighted likelihood whose independent weights, taken to be Gamma in our analysis, concentrate around one as $n$ grows. With high probability, it is unique, has $O\{(\log n/\log\log n)^d+\log n\}$ atoms in dimension $d$, nearly maximizes the ordinary likelihood, and estimates the mixture density at a Hellinger rate that is parametric up to logarithmic factors. This sparsity holds for the estimator itself, not for an approximation of it, and requires no support penalty. The proof rests on an effective-dimension principle for positive kernel mixtures: low-dimensional variation of the fitted values controls the support of every extreme point of the set of maximizers. Numerical illustrations verify that the reweighted NPMLE has Hellinger risk and support size comparable to those of the ordinary NPMLE.

0.47

Minimax Optimality of the NPMLE for Gaussian Mixtures in Hellinger Distance

cs.IT math.STcs.LG ·2026-10-21 ·Shayan Hundrieser, Tudor Manole
similarity 0.68
Abstract

We analyze the problem of estimating univariate Gaussian mixture densities using the nonparametric maximum likelihood estimator (NPMLE) based on $n$ independent and identically distributed samples. We show that the unconstrained NPMLE achieves the minimax optimal rate $\log(n)/n$ in squared Hellinger distance, when the true mixing measure is sub-Gaussian. In addition, we show that when the true mixing measure has compact support, the NMPLE constrained to a compact interval also achieves the minimax optimal rate, which is slightly faster and of order $\log(n)/(n \log \log n)$. Our proof consists of sharply characterizing the local Hellinger bracketing entropy of the class of finite Gaussian mixtures. One of our key technical innovations is to sharply bound the $L^2$ bracketing entropy of balls of bounded-degree polynomials, which may be of independent interest. As an application, we establish the regret-optimality of $g$-modeling procedures based on the NPMLE in the Gaussian Empirical Bayes problem with bounded priors.

0.31

Dimension-free comparison inequalities between $f$-divergences for Gaussian location mixtures

cs.IT math.STmath.PRstat.ML ·2026-10-18 ·Jeonghwan Lee, Cong Ma
similarity 0.67
Abstract · 21 pages; Comments would be appreciated!

We establish dimension-free comparison inequalities between the Kullback-Leibler (KL) and $χ^2$-divergences and the squared Hellinger distance for Gaussian location mixtures with compactly supported mixing distributions. We show that both divergences are bounded by the squared Hellinger distance up to constants depending only on the radius of the support, independently of the ambient dimension and the number of mixture components. These results resolve an open question raised by \cite{jia2023entropic} and strengthen existing comparison inequalities. Our proof relies on a Gaussian interpolation argument that relates the derivatives of these divergences to differences between posterior means.

0.29

Closed-Form Gaussian Information Projection for Embedded Simplices and Simplicial Mixtures

cs.IT ·2026-11-09 ·Ali Darijani, Emadedin Soufian, Jürgen Beyerer
similarity 0.66
Abstract · 27 Pages, 3 Figures

We study Gaussian approximation of Dirichlet--Gaussian probability density functions on embedded simplices. We derive the unique information projection onto the family of nondegenerate ambient Gaussian probability density functions, obtaining closed-form expressions for the optimal Gaussian and the minimum divergence for arbitrary positive Dirichlet parameters. The minimum divergence depends only on the intrinsic simplex dimension and Dirichlet parameters, is invariant up to an affine isomorphism, independent of the ambient dimension, and independent of the normal covariance. Furthermore, by joint convexity of Kullback--Leibler divergence, we establish explicit intrinsic dimension-dependent sharp upper bounds for finite Gaussian Mixture with shared component weights. For uniform probability density functions, these bounds reduce to dimension-dependent constants. The results provide closed-form Gaussian Mixture constructions and rigorous approximation guarantees for probability density functions associated with embedded simplicial complexes. A numerical example on a circular curve illustrates the theoretical results and their connections to geometric Gaussian Mixture representations.

0.23

On the discrete Fisher information

cs.IT math.PR ·2026-10-13 ·Keith M. Ball, Barnabás Gárgyán
similarity 0.65
Abstract · 22 pages

We investigate a discrete analogue of the Fisher information. By using a discrete Laplacian operator, we can construct a discrete analogue of the score function and hence of the information. This provides a natural framework for establishing several information-theoretic inequalities in the discrete setting.

0.32

An exact computer-assisted lower bound for Fisher-information convexity along the two-dimensional heat flow

cs.IT hep-thmath.PR ·2026-10-12 ·Fayez Abu-Ajamieh, Xiaoyan Huo
theory similarity 0.63
Abstract · 12 pages

We study the sharp constant in the differential inequality \[ I(t)I''(t)\geq c\,[I'(t)]^2 \] for Fisher information along the probability-normalized heat flow on $\mathbb R^2$. The unrestricted log-convexity value $c=1$ is false in two dimensions, whereas the known square-root convexity theorem gives the universal value $c=1/2$. We construct an exact, machine-verifiable sum-of-squares certificate proving the stronger universal bound \[ c\geq \frac{231691}{250000}=0.926764. \] Together with an independent exact Fourier/Gaussian-envelope construction, this gives \[ 0.926764\leq c_*<0.94176665012062, \] where $c_*$ is the sharp constant for positive-time Gaussian convolutions of arbitrary probability measures on $\mathbb R^2$. We formulate the proof in explicit two-dimensional index notation. The lower bound follows from an exact rational polynomial identity involving a strictly positive denominator, a weighted total derivative, and eleven positive rational Gram matrices. All $491$ recorded leading principal minors are strictly positive and the complete polynomial identity has zero residual. No symmetry, log-concavity, finite-variance, or finite-entropy assumption is imposed on the initial measure. The sharp value remains open.

0.35