CS Paper Radar

Lossless Source Coding with Marginal Distribution Constraint

Cheuk Ting Li, Jun Chen, Shahab Asoodeh

cs.IT ·submitted 2026-11-04 21:29 UTC · v1

Abstract

We consider lossless source coding with a marginal distribution constraint on the encoding symbols (SC-MDC). Unlike distribution matching, signal shaping and homophonic coding which require the symbols to be (approximately) i.i.d. following a target distribution, SC-MDC requires each symbol to have a marginal distribution that exactly matches the target distribution, which can be regarded as a perfect perceptual quality constraint on the symbols, and may have applications to wear leveling, source coding with cost and steganography. We give nonasymptotic lower and upper bounds on the optimal encoding length, and exactly characterize the coding rate (or the first-order term of the length) when the source distribution or the target distribution is uniform, or when the source is an i.i.d. sequence.

Comment
15 pages

Closest papers

Nearest neighbours by meaning across the whole index.

Exact Redundancy under Average and Almost-Sure Distortion Constraints

cs.IT ·2026-11-15 ·Yijun Fan
hot direction similarity 0.75
Abstract

We characterize the exact redundancy of lossy source coding under average and almost-sure distortion constraints. For general source and output alphabets, we establish the asymptotic redundancy coefficients under suitable regularity conditions. Under the average distortion constraint, the coefficient is half of the minimum probability that an output reveals information about the source, among all rate-distortion optimal channels. Under the almost-sure distortion constraint, the coefficient is either $1/2$ or $1$, depending on whether the $D$-tilted information is constant or not. Both converse results follow from a common change-of-measure inequality. Finally, we verify the required conditions under both distortion requirements always hold for finite alphabets.

0.23

Rate-Distortion-Perception Tradeoff for Vector Gaussian Sources with Limited Common Randomness: Exact Characterization and Approximate Universality

cs.IT ·2026-11-03 ·Wentao Wang, Shuo Shao, Ruibin Li, Jingjing Qian, Jun Chen +1 more
theory similarity 0.65
Abstract

The distortion-rate-common randomness-perception (DRCP) theory characterizes how limited common randomness (CR) improves reconstruction fidelity under perceptual constraints, thereby providing an information-theoretic foundation for generative compression. In this paper, we characterize the DRCP tradeoff for vector Gaussian sources with limited CR under the squared-error distortion measure and the squared Wasserstein-2 perception measure. Specifically, we first establish a generalized reverse waterfilling characterization of globally optimal allocations. Second, we derive an analytical upper bound on the DRCP function through sequential greedy allocation of rate and CR. For fixed rate and CR budgets, the resulting representation is independent of the perception threshold. Finally, we establish its approximate universality through uniform performance guarantees: the achievable distortion exceeds the optimal distortion by at most $2ρ^*\approx0.2897$ times the total source variance, whereas the optimal-to-greedy distortion ratio is at least $η^*\approx0.7517$.

0.32

Exact Minimax One-Bit Unbiased Compression: Heavy-Tail Necessity and Finite-Randomness Approximation

cs.LG ·2026-08-20 ·Tao Jiang, Minbo Gao, Shaowei Cai
theoryhot direction similarity 0.63
Abstract

A pointwise-unbiased one-bit compressor reconstructs every real input in expectation while transmitting one bit. For a scalar source $P$ with CDF $F$, mean $m$, and $\mathcal J(P)=\int_{\mathbb R}\sqrt{F(r)(1-F(r))}\,dr$, we prove that the infimum of the source-averaged reconstruction second moment over all public-coin one-bit codes unbiased on $\mathbb R$ is $m^2+\mathcal J(P)^2$. For regular full-support sources, a distribution-centered random-threshold code attains this value; a converse over arbitrary randomized binary encoders and an equality analysis characterize every attaining code up to null sets, bit relabeling, and public-seed refinement. For the Gaussian location family $\mathcal N(μ,σ^2)$ with $|μ|\le cσ$, the equal prior on the endpoint means is least favorable and the minimax value is $σ^2Λ_c^2$. Exact Gaussian minimax optimality forces a critical heavy tail: at the endpoint means, absolute moments are finite exactly for $p<3$, and $\Pr(W>t)=Θ(t^{-3}/\sqrt{\log t})$. A Cauchy-mixture robustification inflates the second moment by at most $1/(1-η)$ while making every positive-order absolute moment finite. Finite-support public randomness with finite decoder means cannot achieve exact unbiasedness on $\mathbb R$, but a bounded-output approximation using exactly $R$ shared random bits has explicit bias and second-moment bounds converging to the minimax constant. Finally, coordinate allocation communicates exactly $B$ bits per Gaussian-gradient query. On Kim's continuous quadratic hard family, the expected optimization guarantee matches the lower bound in its dependence on $(σ,d,B,\varepsilon)$, and a finite-variance high-probability bound incurs only a logarithmic confidence factor.

0.24

Joint Source-Channel Coding of Gaussian Sources over Block Erasure Channels: Nonasymptotic Bounds and Channel-Uniform Normal Approximations

cs.IT ·2026-09-30 ·Adeel Mahmood
benchmark similarity 0.62
Abstract · Added an SSCC achievability benchmark and expanded numerical results

We study finite-blocklength lossy transmission of a Gaussian memoryless source over memoryless, possibly nonstationary block erasure channels under an excess mean-squared distortion criterion. We derive computable nonasymptotic achievability and converse bounds and establish matching third-order, channel-uniform normal approximations. Our achievability bound exactly evaluates the ensemble-average excess-distortion probability of a specified random coding scheme and improves corresponding specializations of known general one-shot bounds. Our converse conditions on the erasure pattern and source energy and combines spherical-cap and volume bounds, with the spherical-cap term providing the geometric prefactor needed for the matching third-order term. In our channel-uniform normal approximation, we show that for fixed distortion ratio, target excess-distortion probability, and block size, the sufficient and necessary information-balance conditions have the same dispersion and $\frac{1}{2}\log k$ terms, where $k$ is the source blocklength, and differ only by bounded remainders that are uniform over the channel blocklength and erasure-probability profile. Numerical evaluations compare the nonasymptotic JSCC bounds and their common third-order normal approximation with an optimized symmetrized SSCC achievability benchmark.

0.32

Functional Information, Optimal Channel Simulation and Lossy Source Coding

cs.IT ·2026-11-15 ·Gergely Flamich, Spencer Hill, Fady Alajaji, Tamás Linder
similarity 0.62
Abstract

We characterize, within a constant number of bits, the minimum achievable rate for several one-shot coding problems, such as channel simulation and lossy source coding. The basis of our work is a new information measure between random variables $X, Y$ that we call the functional information $I_F(X \to Y)$. We show that this quantity characterizes the minimum one-shot rate $R^*(X \to Y)$ of channel simulation, in which a sample $Y \sim P_{Y \mid X}$ is communicated in the presence of common randomness: $I_F(X \to Y) \leq R^*(X \to Y) < I_F(X \to Y) + 1.45$. For the achievability result, we construct an explicit code using the Poisson functional representation. Using these results, we develop new bounds for one-shot variable-rate lossy source coding: given some distortion measure $Δ$, we use $I_F(X \to Y)$ to define the functional rate-distortion function $R_F$. Then, letting $R_O$ denote the one-shot operational rate-distortion function with private encoder randomness and distortion measure $Δ$, we make use of our channel simulation result to show that $R_F(D) \leq R_O(D) < R_F(D) + 2.45$ at every positive distortion level $D$. Finally, we generalize our channel simulation result by introducing the Rényi $α$-functional information $I_{F, α}(X \to Y)$ and show that it characterizes the minimum Arimoto-Rényi conditional entropy for channel simulation, ${R^*_α(X \to Y)}$, associated with the Campbell cost: ${I_{F, α}(X \to Y) \leq R^*_α(X \to Y) \leq I_{F, α}(X \to Y) + O(1/α)}$.

0.22

Lossy Compression via Sparse Regression Codes: Generalized Construction and Finite-length Bounds

cs.IT ·2026-08-14 ·Galen Reeves, Ramji Venkataramanan
similarity 0.62
Abstract

We study sparse regression codes (SPARCs) for lossy compression under simple greedy encoding rules, including both correlation-based and distance-based methods. We generalize the SPARC construction, and consider the class of \emph{additive orthogonal} regression codes, of which standard SPARCs are a special case. For this class of codes, we derive nonasymptotic bounds on the squared-error distortion by tracking the evolution of the encoding residual across stages. Our results highlight the role of power allocation in controlling the distortion, allowing us to optimize the allocation based on the parameters of the code. The optimized allocation improves the finite-length compression performance of SPARCs, and our bounds provide distortion guarantees for lower complexity variants of SPARCs, like signed SPARCs and $K$-sparse SPARCs.

0.04

Lossy Compression, Realism, and Coordination

cs.IT ·2026-08-12 ·Yassine Hamdi, Deniz Gündüz
publishedsurvey similarity 0.61
Abstract · Published in the IEEE BITS Information Theory Magazine

Classical rate-distortion theory characterizes the fundamental limits of lossy compression under fidelity constraints, but minimizing distortion often yields perceptually unsatisfying reconstructions - blurry images, over-smoothed textures, and unnatural artifacts. This has motivated a growing body of work on compression with realism constraints, which require reconstructions to be statistically indistinguishable from natural signals, giving rise to the three-way rate-distortion-perception (RDP) trade-off. This paper provides an accessible overview of this emerging area and reveals deep connections to another fundamental problem: distributed coordination under rate-limited communication. Under strong distribution matching formulations, both problems lead to nearly identical information-theoretic characterizations, both require common randomness (CR) for optimal performance, and both rely on similar analytical tools such as the soft covering lemma. Beyond a unifying perspective, we survey recent developments in formalizing realism, including batched critics and algorithmic realism, and propose to transfer such paradigms to coordination - illustrating how the connection continues to generate new problems.

0.52

Third-Order Asymptotics for Fixed-Length Lossy Compression of Discrete Memoryless Sources

cs.IT ·2026-11-10 ·Vincent Y. F. Tan
hot direction similarity 0.61
Abstract · 16 pages, 2 figures

This paper determines the third-order asymptotics of the logarithm of the minimum codebook size for fixed-length lossy compression of discrete memoryless sources. For an excess-distortion probability constraint $\varepsilon\in(0,1)$ and every interior distortion level, we establish a third-order correction of $\frac12\log n+O(1)$, where $n$ is the blocklength. This sharpens the general bounds of Kostina and Verdú [IEEE Trans. Inf. Theory, 2012]. The main technical innovation lies in the converse for positive dispersion. Under a reverse change of measure, we retain the exponential weights associated with tilted information and distortion slack and control them jointly using the multivariate weighted-strip estimate of Scarlett, Martinez, and Guillén i Fàbregas [Proc. IEEE ISIT, 2015]. A pairing argument verifies the required covariance nondegeneracy uniformly over all reconstruction words whose distortion balls meet a typical set, yielding the matching converse.

0.31