CS Paper Radar

Functional inequalities from the fundamental limits of Bayesian estimation

Shuchen Li, Ilias Zadik

cs.IT math.PRmath.ST ·submitted 2026-11-04 02:17 UTC · v1
theory

Abstract

We develop a Bayesian information-theoretic framework for proving functional inequalities for log-concave measures. Motivated by Klartag and Ordentlich, we establish a unified equivalence between $Φ$-Sobolev inequalities and Bayesian estimation lower bounds: for a large class of smooth convex $Φ$, a log-concave measure $μ$ satisfies a $Φ$-Sobolev inequality with constant $C>0$ if and only if for $X\simμ$, any $t>0$, and any real-valued measurable $f$, the Bayes risk of estimating $f(X)$ under the $Φ$-Bregman loss from the Gaussian additive observation $Y_t=\sqrt tX+G, G\sim N(0,I_n)$ is at least a $1/(1+Ct)$-fraction of that of the trivial estimator. Combining it with direct information-theoretic arguments, such as testing-to-estimation reductions, we obtain old and new functional inequalities: - For the Poincaré inequality, corresponding to $Φ(x)=(x-1)^2$, we give a simple static information-theoretic proof that the Poincaré constant of an isotropic log-concave measure on $\mathbb{R}^n$ is $O(n^{1/2})$, first proven via stochastic localization in the seminal work of Lee and Vempala. - For the log-Sobolev inequality, corresponding to $Φ(x)=x\log x$, we prove that all smooth $K$-subgaussian log-concave measures $μ$ satisfy $C_{\mathrm{LS}}(μ)\lesssim K^2n^{1/4}$, surpassing the recent state-of-the-art bound of $O(K^2n^{1/2})$ by Bizeul. In parallel and independent work, OpenAI claimed the optimal bound $O(K^2)$ in their October 6, 2026 release of 722 papers. - Finally, we give a Bayesian reformulation of the common structure of the three recent proofs of the Kannan--Lovász--Simonovits conjecture, a dimension-free Poincaré bound for isotropic log-concave measures [BK26; BKL26; SZ26]. Tools from the study of the "low-degree MMSE" in computational complexity of statistical inference motivate and recast key steps and quantities of these proofs.

Closest papers

Nearest neighbours by meaning across the whole index.

An exact computer-assisted lower bound for Fisher-information convexity along the two-dimensional heat flow

cs.IT hep-thmath.PR ·2026-10-12 ·Fayez Abu-Ajamieh, Xiaoyan Huo
theory similarity 0.61
Abstract · 12 pages

We study the sharp constant in the differential inequality \[ I(t)I''(t)\geq c\,[I'(t)]^2 \] for Fisher information along the probability-normalized heat flow on $\mathbb R^2$. The unrestricted log-convexity value $c=1$ is false in two dimensions, whereas the known square-root convexity theorem gives the universal value $c=1/2$. We construct an exact, machine-verifiable sum-of-squares certificate proving the stronger universal bound \[ c\geq \frac{231691}{250000}=0.926764. \] Together with an independent exact Fourier/Gaussian-envelope construction, this gives \[ 0.926764\leq c_*<0.94176665012062, \] where $c_*$ is the sharp constant for positive-time Gaussian convolutions of arbitrary probability measures on $\mathbb R^2$. We formulate the proof in explicit two-dimensional index notation. The lower bound follows from an exact rational polynomial identity involving a strictly positive denominator, a weighted total derivative, and eleven positive rational Gram matrices. All $491$ recorded leading principal minors are strictly positive and the complete polynomial identity has zero residual. No symmetry, log-concavity, finite-variance, or finite-entropy assumption is imposed on the initial measure. The sharp value remains open.

0.35

Near-Gaussian counterexamples to the Ball-Nayar-Tkocz entropy concavity conjecture

cs.IT math.PR ·2026-09-10 ·Congyi Luo
similarity 0.59
Abstract · 37 pages

The Ball-Nayar-Tkocz conjecture asserts that, for independent real random variables $X,Y$ with a common log-concave density, the function $t\mapsto h(\sqrt tX+\sqrt{1-t}Y)$ is concave on $[0,1]$. Gaussian distributions have this property. We construct strongly log-concave counterexamples arbitrarily close to the Gaussian and study their persistence under Gaussian smoothing. Let $X$ have mean zero and variance one, let $G$ be an independent standard Gaussian, and let $q\in[0,1]$ denote the signal variance proportion in the smoothed variable $\sqrt qX+\sqrt{1-q}G$. In the asymmetric case, for every fixed $0<q\le1$, there are counterexamples for which the entropy of the weighted sum of two independent copies of the smoothed variable has strictly positive second derivative near the endpoints. In the symmetric case, for every $4/7<q\le1$, there are counterexamples for which an unequally weighted sum has strictly greater entropy than the equally weighted sum. Both families have smooth, strictly positive densities and are strongly log-concave before and after smoothing. The density ratios $f/\varphi$ converge uniformly to one, and each derivative of fixed positive order converges uniformly to zero, where $\varphi$ is the standard Gaussian density. The symmetric counterexamples can also match any prescribed finite number of Gaussian moments exactly. The asymmetric construction uses an endpoint expansion of entropy, whereas the symmetric construction exploits the different decay rates of high-order Hermite perturbations under different weights. Thus, densities violating concavity approach the Gaussian density in the strong sense described above. The question of uniform concavity in the symmetric class for $0<q\le4/7$ remains open.

0.25

Minimax Quantile Bounds via Information Measures

cs.IT math.ST ·2026-08-21 ·Amedeo Roberto Esposito
similarity 0.59
Abstract

We develop a unified information-theoretic framework for lower bounding minimax quantiles. The starting point is a loss-adapted Neyman--Pearson metaconverse that bounds the minimax success probability at every loss threshold and confidence level. The bound separates the small-ball behaviour of the prior under the loss from the statistical distinguishability of the observation model, and is optimised over an auxiliary output distribution. Different relaxations of this Neyman--Pearson bound yield converses based on \(f\)-informativity, Sibson mutual information \(I_α\), Maximal Leakage, and Amemiya norms. Classical Fano and Le Cam lower bounds are recovered as special cases. The framework also clarifies why different information measures are suited to different recovery criteria. Maximal Leakage is exact for a class of symmetric exact-recovery problems. We use this identity to derive finite-sample bounds on the full minimax exact-recovery risk in the balanced Gaussian weighted stochastic block model, as well as two-sided finite-sample minimax-quantile bounds for low-rank matrix estimation under isotropic bounded-energy noise. For approximate Hamming recovery, we exhibit a heterogeneous binary model in which an optimised finite Sibson order yields a strong converse while the Maximal Leakage specialisation is trivial. Finally, for one-coordinate Poisson localisation, a Bennett-type Young function used through its Amemiya norm recovers the exact success-probability scale, whereas classical Fano and fixed-power relaxations are strictly weaker. These results show that sharp converses for minimax quantiles require adapting the information measure to the recovery resolution, whether exact or approximate, and to the tail behaviour of the likelihood ratio.

0.45

Conditional Fisher-Information Central Limit Theorems under Log-Concavity with Information-Theoretic Consequences

cs.IT math.PR ·2026-09-07 ·Tong Ye, Liu-Quan Yao
benchmarktheory similarity 0.59
Abstract · 47 pages

We establish conditional central limit theorems in Fisher information under log-concavity in every fixed dimension. For conditionally centered normalized sums, after whitening by the averaged conditional covariance, the averaged conditional Fisher information converges to the dimension if and only if it is finite at one convolution level. The scalar criterion follows as the one-dimensional case; we also provide an independent scalar proof based on a second-order continuity theorem for Fisher production on Gaussian-smoothed, tail-controlled classes. For the original sums, the averaged Fisher information matrix converges in operator norm to the inverse averaged conditional covariance. Consequently, the conditional relative Fisher information with respect to the limiting Gaussian law vanishes, and the Gaussian logarithmic Sobolev inequality yields convergence in conditional relative entropy and conditional entropy. We give two operational consequences. For any fixed finite-constellation low-power input, the first-order conditional mutual-information slope converges to the Gaussian-noise benchmark. For Gaussian signaling at any fixed signal covariance, the mutual-information gap from that benchmark is bounded by the conditional relative Fisher deficit and hence vanishes asymptotically.

0.38

Subpolynomial-Query Proximal Sampling Beyond Log-Concavity

cs.CC stat.MLcs.LGmath.ST ·2026-11-02 ·Wei Guo, Bo Yuan, Ye He, Jiaming Liang, Molei Tao +1 more
theory similarity 0.58
Abstract

We study the gradient query complexity of sampling from a density $π(x)\propto\mathrm{e}^{-V(x)}$ on $\mathbb{R}^d$, where $V$ has a $β$-Lipschitz gradient but need not be convex. Our approach uses the proximal sampler, which alternates Gaussian perturbation with sampling a position conditional on the noisy observation. We implement its observation chain using the high-order noisy conditional sampler of [OpenAI 2026], obtaining improved dimension dependence under functional inequalities. If $π$ satisfies a log-Sobolev inequality (LSI) with constant $α$, then for every fixed $ν>0$, there is a finite constant $C_ν$ depending only on $ν$ such that the algorithm returns a sample within total-variation distance $\varepsilon$ using at most $C_νκ(d+κ+\varepsilon^{-1}+\|\nabla V(x_0)\|^2/β)^ν$ queries, where $κ=β/α$ and $x_0$ is an arbitrary reference point for initialization. The analysis controls center-dependent local errors along exact proximal histories and removes the final observation noise by conditional decoding. Under a Poincaré inequality (PI), we obtain a corresponding guarantee from a warm start with a known bound on its Rényi divergence. An oracle lower bound shows that a stationary point alone, which suffices under LSI, is insufficient under PI.

0.17

On the Subadditivity of Mutual f-Information

cs.IT ·2026-10-15 ·Hamidreza Abin, Amin Gohari
similarity 0.58
Abstract

We address an open problem raised by Polyanskiy and Wu in their textbook concerning the subadditivity of mutual \(f\)-information. We study two notions of generalized mutual information: mutual \(f\)-information, defined by \(\widetilde{I}_f(X;Y)=D_f(p_{XY}\|p_Xp_Y)\), and variational \((G,f)\)-information, obtained by minimizing an average of transformed \(f\)-divergences. For the first notion, we derive a sufficient condition on \(g(x)=x^2f''(x)\) that guarantees subadditivity under conditionally independent observations. This condition yields explicit families beyond the previously known examples generated by \(x\log x\) and \((x-1)\log x\). Our proof relies on the concavity of a generalized variance functional. For the variational notion, we extend existing sufficient conditions beyond the transforms \(G(x)\in\{x,\log(1+x),-\log(1-x)\}\). Using a comparison inequality for \(G^{-1}(G(A)+G(B))\), we obtain differential criteria for concave and convex transforms. Explicit constructions illustrate the resulting classes of subadditive information measures.

0.31

On the SoS Certifiability of Log-Concave Distributions

cs.LG cs.CCmath.PR ·2026-09-24 ·Aleksandr Storozhenko
theoryhot direction similarity 0.58
Abstract

We prove that for every isotropic log-concave distribution $P$ on $\mathbb{R}^d$ and every even $m\ge2$, the polynomial $(Cm)^m\|v\|_2^m - \mathbb{E}_{X\sim P}\langle X,v\rangle^m$ is a sum of squares, where $C>0$ is a universal constant. This improves on the Poincaré-dependent bounds (Kothari and Steinhardt, 2017), recovering the optimal moment bounds for log-concave distributions. As an immediate corollary, we obtain computationally efficient algorithms with dimension-free error guarantees for a wide range of statistical estimation problems. Our proof proceeds by using stochastic localization to decompose $P$ as an average of strongly log-concave measures, whose centered moments admit the subgaussian certificates (Diakonikolas et al., 2025). With a covariance-adapted choice of localization, we show that a fourth-moment certificate derived from the variance inequality for quadratic forms (Letwin, 2026) suffices to control this averaging at every even degree.

0.54

Certified Bayesian optimal experimental design: from Expected Information Gain to Signal-to-Noise Ratio

cs.IT ·2026-09-26 ·Mohamed Doumbouya, Arthur Vidard, Olivier Zahm
code/datahot direction similarity 0.57
Abstract

Optimal experimental design is often formulated as the maximization of the Expected Information Gain (EIG), which measures the expected reduction in uncertainty about a parameter of interest after observing the data. For nonlinear forward models and nonGaussian priors, computing the EIG typically requires costly nested Monte Carlo estimators or sophisticated density-approximation techniques. In this work, we derive new upper and lower bounds on the EIG using dimensional logarithmic Sobolev inequalities. The proposed bounds are expressed in terms of signal-to-noise ratios (SNR), preserving the simple matrix-based structure of the EIG in the linear-Gaussian setting and depending solely on covariance matrices and Fisher information matrices. We introduce two families of bounds: conservative bounds, designed to retain the entire information content, and incremental bounds, that prioritize early information acquisition for sequential experimental design. When model gradients are available, we further derive computable approximations of the Fisher-information terms that require only a small number of gradient evaluations, yielding substantial computational savings. The tightness of these bounds is controlled by the nonlinearity of the forward model, and in the linear-Gaussian setting, these bounds recover the exact value of the EIG. By eliminating the nested-sampling bottleneck, the proposed framework offers a tractable and theoretically grounded alternative to direct EIG estimation for large-scale Bayesian experimental design, while retaining the interpretability of the SNR-based bounds.

0.41