CS Paper Radar

A Constructive Theory of Closed-Form Rate-Distortion Functions

Dazhuan Xu, Yongming Huang, Xiaohu You

cs.IT ·submitted 2026-11-12 03:56 UTC · v1

Abstract

Rate-distortion functions are defined by a variational problem whose analytic solution depends on the source distribution and distortion measure. We develop a posterior-based construction framework using conditional entropy as a common distortion coordinate. Building on the rate-entropy formulation, we establish entropy-level attainability for discrete sources with finite entropy and absolutely continuous sources with integrable log densities. We give a constructive rate-entropy coding proof for finite-alphabet sources and such continuous sources, without a second-moment assumption. Marginal compatibility ensures that a candidate posterior and output distribution recover the source, while a Gibbs representation and a kernel mass bound certify attainment of maximum conditional entropy. The negative log posterior identifies candidate fixed distortions. Excess rate equals a conditional entropy gap and decomposes into budget slack and posterior mismatch. Convolution decompositions and conjugate posteriors provide two construction paths. They recover established results, including the Gamma-source rate-distortion function under Itakura-Saito distortion, and yield a closed-form parametric expression for zeta sources under divisibility-constrained logarithmic multiplicative distortion. Exact expressions are obtained on specified intervals for negative binomial sources under one-sided counting distortion and Gamma sources under one-sided mean error. A residual-count coordinate for Poisson entropy budgets and rate-entropy parametrizations of Gamma and Dirichlet counting channels connect model parameters to remaining uncertainty. The framework unifies distortion selection, compatible-channel construction, and optimality verification.

Comment
26 pages, 3 figures, 1 table

Closest papers

Nearest neighbours by meaning across the whole index.

Rate-Distortion Theory of Gaussian Mixtures via Posterior Label Sampling

cs.IT ·2026-10-11 ·Weichen Wang, Deniz Gündüz
theory similarity 0.62
Abstract

We study the mean-squared error (MSE) rate-distortion function (RDF) of finite Gaussian mixture models (GMMs) in $\mathbb{R}^d$, for which neither the RDF nor an optimal test channel is known in closed form. It can be sandwiched between the genie-aided conditional RDF $R_{\mathbf{X} | T}(D)$, which can be characterized in closed-form for a GMM, and $R_{\mathbf{X} | T}(D)+I(\mathbf{X};T)$, where $T$ denotes the latent component label. We show that the upper bound can be achieved operationally by the proposed Gibbs-relabelling method (GRM), which conveys a label from the posterior $P_{T|\mathbf{X}}$ through \textit{channel simulation} at rate $I(\mathbf{X};T)$, followed by vector quantization of the resulting exactly Gaussian conditional sources by reverse water-filling. Using the Shannon lower bound (SLB), we prove that GRM attains the SLB, and hence the RDF, of a general finite mixture if and only if every component attains its own SLB. For a GMM this holds throughout the fully active interval $D/d<\min_{k,m}λ_{k,m}$, where $λ_{k,m}$ are the eigenvalues of the component covariances, and for scalar GMMs the condition is also necessary. Since GRM cannot operate below the label cost $I(\mathbf{X};T)$, we propose a complementary stochastic scalar quantization (SSQ) scheme for low rates, which simulates an $M$-ary reconstruction index. We show that at low rates SSQ with $M\geq 2$ matches the Shannon RDF to first order: $R(D_{\max}-δ)=δ/(2τ^2)+o(δ)$, where $τ^2$ is a directional sub-Gaussian proxy of the mixture, and quantify the price of a finite codebook through a strict rate gap at high rates and a distortion floor whose joint scaling in $M$ and number of mixture components is characterized. Numerical results illustrate both regimes and the finite-blocklength cost of channel simulation.

0.28

Sequential Lossy Compression With Causal Conditional Perception

cs.IT eess.SP ·2026-09-08 ·Photios A. Stavrou, Zixuan He
theory similarity 0.62
Abstract · 6 pages, 1 figure, submitted paper

In this paper, we study sequential lossy compression under a causal conditional perception criterion comparing source and reconstruction distributions given the same reconstruction history. For first-order Markov sources, we formulate the finite-horizon nonanticipative rate-distortion-perception function (NRDPF) with stagewise constraints and establish one-shot lower and upper bounds on the minimum variable-length sum rate using a strengthened strong functional-representation lemma (SFRL) and common randomness. For time-varying scalar Gauss--Markov sources under pointwise mean-squared error (MSE) and conditional squared Wasserstein-$2$ fidelity, we prove Gaussian optimality, derive a log-variance characterization, and obtain a closed-form solution that recovers the classical Gaussian nonanticipative rate-distortion function (NRDF) when perception is unconstrained and the classical Gaussian RDPF when the source is stationary and memoryless.

0.23

Rate-Distortion-Perception Tradeoff for Vector Gaussian Sources with Limited Common Randomness: Exact Characterization and Approximate Universality

cs.IT ·2026-11-03 ·Wentao Wang, Shuo Shao, Ruibin Li, Jingjing Qian, Jun Chen +1 more
theory similarity 0.61
Abstract

The distortion-rate-common randomness-perception (DRCP) theory characterizes how limited common randomness (CR) improves reconstruction fidelity under perceptual constraints, thereby providing an information-theoretic foundation for generative compression. In this paper, we characterize the DRCP tradeoff for vector Gaussian sources with limited CR under the squared-error distortion measure and the squared Wasserstein-2 perception measure. Specifically, we first establish a generalized reverse waterfilling characterization of globally optimal allocations. Second, we derive an analytical upper bound on the DRCP function through sequential greedy allocation of rate and CR. For fixed rate and CR budgets, the resulting representation is independent of the perception threshold. Finally, we establish its approximate universality through uniform performance guarantees: the achievable distortion exceeds the optimal distortion by at most $2ρ^*\approx0.2897$ times the total source variance, whereas the optimal-to-greedy distortion ratio is at least $η^*\approx0.7517$.

0.32

Second-Order Image Characterization with Distortion and Secrecy

cs.IT ·2026-10-07 ·Cao Daming
theory similarity 0.60
Abstract · Draft manuscript

Image characterization relates coding budgets to the probability carried by source cells and their output images. At second order, refining a cell can reveal unavailable information, while a change of reference measure can alter the relevant likelihood fluctuations. We establish image bounds that account for both effects. A finite-block partition theorem bounds the cost of merging component images by the joint conditional information spectrum of their label. It preserves continuous second-order profiles when that cost is negligible on the square-root blocklength scale. For reconstruction constraints, cell-dependent reference measures retain the actual distortion deficits in a single likelihood. A Gaussian evaluation gives matching second-order rates for Heegard--Berger coding with one uninformed decoder in the strict both-active regime, common reconstruction under a joint agreement and distortion criterion, and two-stage refinement with side information absent at the first decoder. A structured binary source illustrates partition stability even when its global density and conditional-divergence deficits are linear in blocklength. Posterior-mass images also sharpen extraction after a public refinement by charging its residual information rather than its alphabet size. These results preserve the joint constraints and the observations available for choosing the local coding experiment.

0.47

Third-Order Asymptotics for Fixed-Length Lossy Compression of Discrete Memoryless Sources

cs.IT ·2026-11-10 ·Vincent Y. F. Tan
hot direction similarity 0.59
Abstract · 16 pages, 2 figures

This paper determines the third-order asymptotics of the logarithm of the minimum codebook size for fixed-length lossy compression of discrete memoryless sources. For an excess-distortion probability constraint $\varepsilon\in(0,1)$ and every interior distortion level, we establish a third-order correction of $\frac12\log n+O(1)$, where $n$ is the blocklength. This sharpens the general bounds of Kostina and Verdú [IEEE Trans. Inf. Theory, 2012]. The main technical innovation lies in the converse for positive dispersion. Under a reverse change of measure, we retain the exponential weights associated with tilted information and distortion slack and control them jointly using the multivariate weighted-strip estimate of Scarlett, Martinez, and Guillén i Fàbregas [Proc. IEEE ISIT, 2015]. A pairing argument verifies the required covariance nondegeneracy uniformly over all reconstruction words whose distortion balls meet a typical set, yielding the matching converse.

0.31

Operational Indirect Rate-Distortion Functions for Classification Tasks

cs.IT ·2026-11-05 ·Yann Miguet, Elsa Dupraz, Philippe Mary
position similarity 0.59
Abstract · 8 pages, 4 figures

In goal-oriented communications, indirect source coding involves two sources: a semantic source $X$ and an observation $Z$, which is the only source available at the encoder. In this paper, we consider this framework for classification, where the decoder only needs to reconstruct $X$. We argue that the gap between the indirect rate-distortion function and the performance of practical learned coding schemes, such as variational information bottleneck (VIB) and variational autoencoder (VAE) architectures, comes from three effects: the latent representation, the choice of the deep neural network (DNN) architecture and training loss, and whether the encoder is aware of the task. To quantify these effects, we introduce four operational rate-distortion functions that account for the DNN architecture and the training loss, and we establish a hierarchy among them. We then derive these functions for a Gaussian mixture source, where $X$ is the mixture state. Numerical results, both for the Gaussian mixture and for practical coding schemes on the CIFAR-100 dataset, quantify the rate loss incurred by each effect.

0.19

Exact Redundancy under Average and Almost-Sure Distortion Constraints

cs.IT ·2026-11-15 ·Yijun Fan
hot direction similarity 0.58
Abstract

We characterize the exact redundancy of lossy source coding under average and almost-sure distortion constraints. For general source and output alphabets, we establish the asymptotic redundancy coefficients under suitable regularity conditions. Under the average distortion constraint, the coefficient is half of the minimum probability that an output reveals information about the source, among all rate-distortion optimal channels. Under the almost-sure distortion constraint, the coefficient is either $1/2$ or $1$, depending on whether the $D$-tilted information is constant or not. Both converse results follow from a common change-of-measure inequality. Finally, we verify the required conditions under both distortion requirements always hold for finite alphabets.

0.23

Opportunistic Conditional Entropy Coding with Frozen Analysis and Synthesis Transforms

eess.IV cs.IT ·2026-09-18 ·Vincent Corlay, Maxime Rousselot, Andriy Enttsel
similarity 0.58
Abstract

In many delivery settings, a receiver may already hold a lower-quality or lower-resolution representation of an image, obtained through an independent transmission. Conventional codecs encode a subsequently requested higher-quality representation without exploiting this incidental side information, whereas conditional codecs generally assume a prescribed source of side information that is always available. We instead consider an opportunistic setting in which side information may or may not be present. We introduce a single entropy model that conditions on a previously decoded latent when available and falls back to a standard hyperprior otherwise. The proposed adapter maps the side-information latent to the prior signal required by the entropy model, allowing the same model to support multiple target and side-information quality combinations. The analysis and synthesis transforms remain frozen, enabling retrofitting of an existing learned codec while preserving its latent representation and reconstruction path. When the receiver holds the quality immediately below the target, the proposed method reduces the rate of the subsequent transmission by up to 46%, or by 52% when an additional hyper-latent is transmitted. In the absence of side information, the rate penalty remains below 4%, and the reconstructions are bit-identical across the conditional and fallback modes.

0.46