CS Paper Radar

Third-Order Asymptotics for Fixed-Length Lossy Compression of Discrete Memoryless Sources

Vincent Y. F. Tan

cs.IT ·submitted 2026-11-10 04:40 UTC · v1
hot direction

Abstract

This paper determines the third-order asymptotics of the logarithm of the minimum codebook size for fixed-length lossy compression of discrete memoryless sources. For an excess-distortion probability constraint $\varepsilon\in(0,1)$ and every interior distortion level, we establish a third-order correction of $\frac12\log n+O(1)$, where $n$ is the blocklength. This sharpens the general bounds of Kostina and Verdú [IEEE Trans. Inf. Theory, 2012]. The main technical innovation lies in the converse for positive dispersion. Under a reverse change of measure, we retain the exponential weights associated with tilted information and distortion slack and control them jointly using the multivariate weighted-strip estimate of Scarlett, Martinez, and Guillén i Fàbregas [Proc. IEEE ISIT, 2015]. A pairing argument verifies the required covariance nondegeneracy uniformly over all reconstruction words whose distortion balls meet a typical set, yielding the matching converse.

Comment
16 pages, 2 figures

Closest papers

Nearest neighbours by meaning across the whole index.

Joint Source-Channel Coding of Gaussian Sources over Block Erasure Channels: Nonasymptotic Bounds and Channel-Uniform Normal Approximations

cs.IT ·2026-09-30 ·Adeel Mahmood
benchmark similarity 0.70
Abstract · Added an SSCC achievability benchmark and expanded numerical results

We study finite-blocklength lossy transmission of a Gaussian memoryless source over memoryless, possibly nonstationary block erasure channels under an excess mean-squared distortion criterion. We derive computable nonasymptotic achievability and converse bounds and establish matching third-order, channel-uniform normal approximations. Our achievability bound exactly evaluates the ensemble-average excess-distortion probability of a specified random coding scheme and improves corresponding specializations of known general one-shot bounds. Our converse conditions on the erasure pattern and source energy and combines spherical-cap and volume bounds, with the spherical-cap term providing the geometric prefactor needed for the matching third-order term. In our channel-uniform normal approximation, we show that for fixed distortion ratio, target excess-distortion probability, and block size, the sufficient and necessary information-balance conditions have the same dispersion and $\frac{1}{2}\log k$ terms, where $k$ is the source blocklength, and differ only by bounded remainders that are uniform over the channel blocklength and erasure-probability profile. Numerical evaluations compare the nonasymptotic JSCC bounds and their common third-order normal approximation with an optimized symmetrized SSCC achievability benchmark.

0.32

Exact Second-Order Asymptotics for the Gaussian Gray--Wyner Problem

cs.IT ·2026-10-28 ·Lin Zhou, Lei Yu
theory similarity 0.68
Abstract

We revisit the Gray-Wyner problem of lossy compression of bivariate Gaussian sources under the quadratic distortion measure and derive the exact second-order asymptotics. Specifically, we first identify the boundary rate tuples using the explicit first-order asymptotics by Yu (TIT 2023) and then characterize the tradeoff among three second-order coding rates subject to a constraint on the joint excess-distortion probability. Our main contribution is the achievability proof, where we derive a covariance shell covering lemma that is analogous to the type covering lemma for the discrete case. In particular, our covering lemma classifies the source sequences into power shell types via their empirical covariance matrix. For each such power shell type, we derive sufficient conditions on the compression rates so that covering within distortion levels are achievable. Finally, asymptotical analysis via the Berry-Esseen theorem yields the desired result.

0.47

A Tight Second-Order Converse Bound for Variable-Length Feedback Codes

cs.IT ·2026-09-10 ·Recep Can Yavas
similarity 0.66
Abstract · 26 pages, submitted to Information Theory Transactions

We study variable-length feedback (VLF) codes over a discrete memoryless channel under average decoding-time and error-probability constraints. In the non-vanishing error probability regime, Polyanskiy, Poor, and Verdú (2011) derive achievability and converse bounds on the logarithm of the maximum achievable codebook size. These bounds establish the $ε$-capacity but leave an order-$\log N$ gap in the second-order expansion, where $N$ is the average decoding time. Yavas and Tan (2025) improve the coefficient of $\log N$ in the achievability bound from $-1$ to $-\frac{C}{C_1}$, where $C$ is the channel capacity and $C_1$ is the largest Kullback--Leibler divergence between two conditional output distributions. We derive a converse with the same coefficient, establishing the second-order fundamental limit for every positive-capacity discrete memoryless channel with finite $C_1$. The result also covers the moderate-deviations and error-exponent regimes, including polynomially decaying error probabilities. The converse uses Rényi entropy and the extrinsic Jensen--Shannon divergence. We also derive necessary properties of asymptotically optimal VLF codes. First-order-optimal codes must have an early-stopping branch, and second-order-optimal codes must additionally exhibit communication and confirmation behavior. Finally, for the binary erasure channel, we determine the exact minimum expected decoding time for every message-set size and admissible error probability.

0.11

Chi-Squared Geometry for Robust Finite-Blocklength Information and Dispersion Analysis

cs.IT ·2026-08-18 ·Hassan Tavakoli, Thinh Nguyen, Bella Bose
theory similarity 0.65
Abstract · Accepted for publication at Information Theory Workshop 2026, ITW 2026

We develop a column-wise chi-squared geometry for discrete memoryless channels (DMCs) yielding tight, logarithm-free bounds on mutual information, channel dispersion, and finite-blocklength coding rates without evaluating logarithms of the channel matrix. The key parameter is~\(η\)---the worst-case relative deviation of a transition probability from its output marginal, which is small precisely when the channel is close to the fully noisy channel $t_{ij}=s_j$. We prove three main results: (1) a third-order ratio expansion showing \(I(X;Y)/χ^2(X;Y)\to 1/2\) as \(η\to 0\) with an \(O(η)\) skewness correction; (2) a two-sided dispersion equivalence bounding \(V(X;Y)\) above and below by \(χ^2(X;Y)\) with explicit constants \(c_{\pm}(η)\to 1\); and (3) a certified robust design rate \(R_{\mathrm{cert}}(n,\varepsilon)\) with total certification gap \(O(η)+O(η/\sqrt{n})+O(\log n/n)\). The certified bounds on \(I\) and \(V\) require only addition, multiplication, division, and square roots; the final rate also uses \(Q^{-1}(\varepsilon)\).

0.36

Exact Redundancy under Average and Almost-Sure Distortion Constraints

cs.IT ·2026-11-15 ·Yijun Fan
hot direction similarity 0.64
Abstract

We characterize the exact redundancy of lossy source coding under average and almost-sure distortion constraints. For general source and output alphabets, we establish the asymptotic redundancy coefficients under suitable regularity conditions. Under the average distortion constraint, the coefficient is half of the minimum probability that an output reveals information about the source, among all rate-distortion optimal channels. Under the almost-sure distortion constraint, the coefficient is either $1/2$ or $1$, depending on whether the $D$-tilted information is constant or not. Both converse results follow from a common change-of-measure inequality. Finally, we verify the required conditions under both distortion requirements always hold for finite alphabets.

0.23

Exact Minimax One-Bit Unbiased Compression: Heavy-Tail Necessity and Finite-Randomness Approximation

cs.LG ·2026-08-20 ·Tao Jiang, Minbo Gao, Shaowei Cai
theoryhot direction similarity 0.62
Abstract

A pointwise-unbiased one-bit compressor reconstructs every real input in expectation while transmitting one bit. For a scalar source $P$ with CDF $F$, mean $m$, and $\mathcal J(P)=\int_{\mathbb R}\sqrt{F(r)(1-F(r))}\,dr$, we prove that the infimum of the source-averaged reconstruction second moment over all public-coin one-bit codes unbiased on $\mathbb R$ is $m^2+\mathcal J(P)^2$. For regular full-support sources, a distribution-centered random-threshold code attains this value; a converse over arbitrary randomized binary encoders and an equality analysis characterize every attaining code up to null sets, bit relabeling, and public-seed refinement. For the Gaussian location family $\mathcal N(μ,σ^2)$ with $|μ|\le cσ$, the equal prior on the endpoint means is least favorable and the minimax value is $σ^2Λ_c^2$. Exact Gaussian minimax optimality forces a critical heavy tail: at the endpoint means, absolute moments are finite exactly for $p<3$, and $\Pr(W>t)=Θ(t^{-3}/\sqrt{\log t})$. A Cauchy-mixture robustification inflates the second moment by at most $1/(1-η)$ while making every positive-order absolute moment finite. Finite-support public randomness with finite decoder means cannot achieve exact unbiasedness on $\mathbb R$, but a bounded-output approximation using exactly $R$ shared random bits has explicit bias and second-moment bounds converging to the minimax constant. Finally, coordinate allocation communicates exactly $B$ bits per Gaussian-gradient query. On Kim's continuous quadratic hard family, the expected optimization guarantee matches the lower bound in its dependence on $(σ,d,B,\varepsilon)$, and a finite-variance high-probability bound incurs only a logarithmic confidence factor.

0.24

An Agnostic Sample Compression Scheme for Squared Loss of Near-Linear Size in the Fat-Shattering Dimension

cs.LG ·2026-08-31 ·Guangjian Zhang, Zhengyi Yang, Xiaoyang Wang
theory similarity 0.62
Abstract · 43 pages

We study the squared-loss sample compression question posed by Attias, Hanneke, Kontorovich, and Sadigurschi in their ICML 2024 paper. For every nonempty class $\mathcal{F}\subseteq[0,1]^{\mathcal{X}}$ and accuracy $0<α\le1$ with $d=\operatorname{fat}(\mathcal{F},c'α)<\infty$, where $c'>0$ is universal, we prove the existence of a deterministic agnostic scheme of total size $O(d\log^3(2/α))$. Its code contains original labeled observations and auxiliary bits, and its reconstruction has empirical loss at most $α$ above the infimum over $\mathcal{F}$ on every finite sample. This establishes the polylogarithmic formulation in Section 5 of arXiv:1810.01864v2. The construction approximates a near-optimal internal target by a fixed-step boosting ensemble and takes its equal-weight median. Average loss permits an accuracy-dependent number of rounds, so the entire ensemble can be retained without dual-dimension sparsification. Quantized target constraints and round-membership bits determine the reconstructed function exactly. We also prove that, when retained observations must come from the sample, a fixed class of fat-shattering dimension one requires total size $Ω(\log(1/α))$. With observations and auxiliary bits charged together, this rules out the proceedings version's log-free formulation under its original-example restriction. The framework extends to average power loss at scale $α/p$.

0.24

The Sample Complexity of Channel Coding

cs.IT math.ST ·2026-11-16 ·Terence Viaud, Ioannis Kontoyiannis
similarity 0.62
Abstract · 31 pages, 3 figures

We introduce a new framework for the study and evaluation of the fundamental limits of channel coding, emphasizing genuinely nonasymptotic results. The {\em channel coding sample complexity} $n^*({\cal W},ε)$ of a channel ${\cal W}$ is defined as the smallest blocklength at which it is possible to transmit information through that channel with error probability below $ε$ and at a rate at least as high as that given by an appropriate constraint also in terms of $ε$. This formulation follows analogous developments in statistics, computer science and, more recently, in lossless source coding. For any discrete memoryless channel we derive nonasymptotic bounds on the sample complexity, with explicit constants. These bounds show that the sample complexity is characterized not by the usual channel capacity, but by the channel's Augustin--Csiszár capacity of order $1/2$, which coincides with the cutoff rate as well as with the channel's Sibson capacity of order $1/2$. The proof is partly based on a new channel-coding one-shot converse in terms of hypothesis testing. We also examine extensions of the sample complexity framework to general target rates. Moreover, we derive corresponding results for the sample complexity of the additive white Gaussian noise channel with a quadratic power constraint. Their proof relies on new, explicit upper bounds on the best achievable error probability, which may be of independent interest.

0.20