CS Paper Radar

A Self-Coupling Gaussian Extremal Inequality for Information-Constrained Optimal Transport with Applications to Rate-Distortion-Perception

Jiayang Zou, Jun Chen, Ayfer Özgür, Jia Wang

cs.IT ·submitted 2026-10-18 06:00 UTC · v1
theory

Abstract

How much mutual information is needed to achieve a target correlation between two copies of a prescribed distribution? We prove a sharp Gaussian extremal inequality for this self-coupling problem: among distributions with a fixed mean and covariance, the Gaussian has the greatest minimum information cost for meeting a positive semidefinite cross-covariance target. The inequality is strict for non-Gaussian distributions when the target is positive definite. The proof uses a midpoint gluing construction. Two Ornstein--Uhlenbeck half-bridges, each started from the prescribed endpoint law, meet at a common intermediate distribution and are joined by conditional gluing. Changing the Gaussian reference then turns the resulting relative-entropy bound into the desired extremal inequality. As a first application, scaling one endpoint of the self-coupling establishes Gaussian extremality for rate-distortion-perception with unlimited common randomness, under quadratic distortion and squared Wasserstein-2 perception measures. We also establish an analogous result for finite common randomness, where the problem involves two mutual information constraints associated with an auxiliary random variable. In that setting, we retain an intermediate observation on one half-bridge and balance the two information budgets simultaneously using the Poincaré-Miranda theorem.

Comment
31 pages. Submitted to IEEE Transactions on Information Theory

Closest papers

Nearest neighbours by meaning across the whole index.

Rate-Distortion-Perception Tradeoff for Vector Gaussian Sources with Limited Common Randomness: Exact Characterization and Approximate Universality

cs.IT ·2026-11-03 ·Wentao Wang, Shuo Shao, Ruibin Li, Jingjing Qian, Jun Chen +1 more
theory similarity 0.66
Abstract

The distortion-rate-common randomness-perception (DRCP) theory characterizes how limited common randomness (CR) improves reconstruction fidelity under perceptual constraints, thereby providing an information-theoretic foundation for generative compression. In this paper, we characterize the DRCP tradeoff for vector Gaussian sources with limited CR under the squared-error distortion measure and the squared Wasserstein-2 perception measure. Specifically, we first establish a generalized reverse waterfilling characterization of globally optimal allocations. Second, we derive an analytical upper bound on the DRCP function through sequential greedy allocation of rate and CR. For fixed rate and CR budgets, the resulting representation is independent of the perception threshold. Finally, we establish its approximate universality through uniform performance guarantees: the achievable distortion exceeds the optimal distortion by at most $2ρ^*\approx0.2897$ times the total source variance, whereas the optimal-to-greedy distortion ratio is at least $η^*\approx0.7517$.

0.32

Mutual Information Constrained Chernoff Bottleneck

cs.IT cs.LGmath.OCmath.ST+1 ·2026-09-29 ·Dier Tang, Guangyue Han
similarity 0.65
Abstract · 35 pages, 4 figures, 3 tables. Feedback and comments are welcome!

The classical information bottleneck (IB) measures the relevance of a representation $U$ of $X$ to a target $Y$ by $I(U;Y)$, which does not directly characterize the error of downstream decisions. For a binary hypothesis $Y$ inferred from many separately encoded observations, the optimal error exponent is the Chernoff information between the two conditional distributions of $U$ given $Y$. We study the mutual information constrained Chernoff bottleneck, which seeks an encoder that maximizes this Chernoff information subject to a rate constraint $I(U;X) \leq R$. We show that its optimal value $C(R)$ increases strictly up to $R = H(V)$, where $V$ merges the symbols of $X$ with equal likelihood ratio, remains at the uncompressed exponent beyond, and, unlike the IB curve, need not be concave. We further show that $k+1$ outputs suffice to attain $C(R)$, where $k$ is the cardinality of $V$. We propose an alternating algorithm that updates the encoder via a generalized Blahut--Arimoto algorithm and the Chernoff parameter $s$ via a nonlinear equation, and prove that its iterates remain feasible, with nondecreasing and convergent Chernoff information. Comparing with the classical IB for $n$ samples, we show that, whenever the two objectives disagree, the IB encoder has the smaller equivocation $H(Y|U^{n})$ at $n = 1$, whereas the Chernoff bottleneck has the smaller error probability and equivocation for all large $n$. Numerical experiments confirm the theory, and on real topic-detection data from the 20 Newsgroups corpus, compressing each word to only $17\%$ of its entropy retains $90\%$ of the error exponent and nearly the accuracy of the uncompressed classifier.

0.40

Exact Common Information and Exact Channel Synthesis for Correlated Gaussian Sources

cs.IT ·2026-08-26 ·Lei Yu
similarity 0.64
Abstract · 16 pages, 2 figures

In this paper, we resolve two conjectures posed by Yu and Tan in 2020 (in two separate papers published in the IEEE Trans. Inf. Theory). Specifically, we establish that: 1) the exact common information for a pair of $ρ$-correlated Gaussian sources is given by the conjectured expression $\frac{1}{2}\log\frac{1+ρ}{1-ρ}+\fracρ{1+ρ}$; and 2) the admissible region for the shared randomness rate and the communication rate in exact channel synthesis is exactly the conjectured one. These results yield two important consequences. First, for any $ρ>0$, the exact common information of a correlated Gaussian pair strictly exceeds Wyner's common information. Second, for $ρ>0$, the exact channel synthesis of such a pair requires strictly higher rates than the total-variation version. The proof combines an exact optimal-transport representation of the worst-case Gaussian cross-entropy, Fathi's Gaussian transport inequality, and a determinant inequality arising from the covariance structure of the conditional means.

0.36

Sequential Lossy Compression With Causal Conditional Perception

cs.IT eess.SP ·2026-09-08 ·Photios A. Stavrou, Zixuan He
theory similarity 0.62
Abstract · 6 pages, 1 figure, submitted paper

In this paper, we study sequential lossy compression under a causal conditional perception criterion comparing source and reconstruction distributions given the same reconstruction history. For first-order Markov sources, we formulate the finite-horizon nonanticipative rate-distortion-perception function (NRDPF) with stagewise constraints and establish one-shot lower and upper bounds on the minimum variable-length sum rate using a strengthened strong functional-representation lemma (SFRL) and common randomness. For time-varying scalar Gauss--Markov sources under pointwise mean-squared error (MSE) and conditional squared Wasserstein-$2$ fidelity, we prove Gaussian optimality, derive a log-variance characterization, and obtain a closed-form solution that recovers the classical Gaussian nonanticipative rate-distortion function (NRDF) when perception is unconstrained and the classical Gaussian RDPF when the source is stationary and memoryless.

0.23

On the Cardinality of Optimal Representations in the Binary-Source Information Bottleneck

cs.IT cs.LGmath.OC ·2026-10-05 ·Dier Tang, Jun Chen
similarity 0.61
Abstract · 9 pages. Feedback and comments are welcome!

The information bottleneck (IB) seeks a representation $U$ of a source $X$ that retains as much information as possible about a target $Y$, subject to a constraint on $I(U;X)$. A classical argument shows that it suffices to consider representations with at most $|\mathcal{X}|+1$ symbols, and this bound is known to be tight whenever $|\mathcal{X}| \geq 3$. We show that the binary case behaves differently: if $X$ is binary and $Y$ is finite, then for every joint distribution of $(X,Y)$ and every rate constraint, the IB optimum is attained by a binary $U$. Hence the bound $|\mathcal{U}| \leq |\mathcal{X}|+1$ sharpens to $|\mathcal{U}| \leq |\mathcal{X}|$ for binary sources. The proof combines a separating hyperplane argument with the observation that, for a binary source, the ratio of the second derivatives of the two entropy functions involved is concave.

0.16

Conditional Fisher-Information Central Limit Theorems under Log-Concavity with Information-Theoretic Consequences

cs.IT math.PR ·2026-09-07 ·Tong Ye, Liu-Quan Yao
benchmarktheory similarity 0.61
Abstract · 47 pages

We establish conditional central limit theorems in Fisher information under log-concavity in every fixed dimension. For conditionally centered normalized sums, after whitening by the averaged conditional covariance, the averaged conditional Fisher information converges to the dimension if and only if it is finite at one convolution level. The scalar criterion follows as the one-dimensional case; we also provide an independent scalar proof based on a second-order continuity theorem for Fisher production on Gaussian-smoothed, tail-controlled classes. For the original sums, the averaged Fisher information matrix converges in operator norm to the inverse averaged conditional covariance. Consequently, the conditional relative Fisher information with respect to the limiting Gaussian law vanishes, and the Gaussian logarithmic Sobolev inequality yields convergence in conditional relative entropy and conditional entropy. We give two operational consequences. For any fixed finite-constellation low-power input, the first-order conditional mutual-information slope converges to the Gaussian-noise benchmark. For Gaussian signaling at any fixed signal covariance, the mutual-information gap from that benchmark is bounded by the conditional relative Fisher deficit and hence vanishes asymptotically.

0.38

Fisher Information and Support Complexity of Gaussian Mixtures

cs.IT ·2026-10-15 ·Luca Barletta, Alex Dytso
theory similarity 0.60
Abstract · 24 pages, single-column

We study lower bounds on the Fisher information of Gaussian location mixtures and their applications to estimation problems. First, for a discrete mixing distribution, we derive a universal positive-semidefinite lower bound on the Fisher information matrix in terms of the Rényi entropy of order two. For a mixing distribution supported on $K$ points, this bound scales as $\log(K)/K^2$, improving by a logarithmic factor on the support-size dependence obtained from Stam's inequality. We then develop dimension-aware trace bounds using classical Sobolev inequalities. These bounds are expressed in terms of Sibson's $α$-mutual information and yield the scaling $K^{-2/n}$ in dimension $n$. A complementary bound involving the $α$-mutual information of order $1/2$ is obtained using the sharp Young convolution inequality. The derived inequalities are further specialized to rotationally invariant mixing distributions supported on discrete spherical shells. In this setting, an intrinsic-dimensional bound on Sibson mutual information leads to lower bounds on the Fisher information in terms of the number of occupied shells. As applications, we study least favorable priors for bounded Gaussian mean estimation. In the scalar case, we show that the least favorable prior on $[-R,R]$ has at least $Ω(R\sqrt{\log R})$ support points and that each of its probability masses is of order at most $1/R$. In the vector case, we show that the number of occupied shells grows at least linearly with $R$. Finally, Brown's identity and the I-MMSE relationship are used to translate the Fisher-information inequalities into upper bounds on the MMSE and mutual information.

0.28

On the Information Bottleneck for Stable Variables

cs.IT ·2026-10-06 ·Dongmin Park, Jihad Fahs, Mohamad Assaad
similarity 0.58
Abstract · 13 pages, 3 figures

We consider a generalization of the Gaussian information bottleneck problem to jointly stable variables $X$ and $Y$. Specifically, for $X = Y + A$, where $A$ is stable and independent of $Y$, we find optimal stable linear encoders of the form $T = kX + N$, with $N$ stable and independent of $X$. We characterize the linear stable information bottleneck in closed-form and its critical value. Furthermore, we show that, except for the Gaussian case, such stochastic linear encoders are strictly suboptimal, while becoming asymptotically optimal at large compression rates. Our linear solution recovers the Gaussian information bottleneck for jointly Gaussian variables as a special case.

0.45