CS Paper Radar

Universal Decoding via the Pairwise Error Probability

Nir Elkayam, Meir Feder

cs.IT ·submitted 2026-08-20 20:18 UTC · v1
theory

Abstract

We develop a theory of universal decoding built on the pairwise error probability (PEP) primitive of a companion paper. The PEP and its error spectrum are defined for an arbitrary decoding metric, and while raw metric values across a family share no common scale, the PEP supplies one. Universal decoding -- decoding well simultaneously against a whole family of metrics or channels -- becomes the problem of merging the per-metric spectra into a single rule, which we construct from a clipped inverse-PEP statistic. The construction rests on a Kraft-type inequality for decoding, valid for any input prior: per output, every metric canonicalizes into a conditional probability assignment on the codewords, and the merge is the normalized-maximum-likelihood envelope of the induced family. We prove the rule is random-coding universal (it loses only a vanishing rate relative to the best metric in the family, against every channel), show it is an asymptotic minimax/equalizer rule, and derandomize it: over a subexponential channel family a single deterministic code inherits the guarantee. For discrete memoryless channels the construction reduces to types and recovers the maximum-mutual-information decoder, its tilted variant for non-uniform memoryless input, and finite-state universal decoders. It extends to decoding with an erasure option (deterministically, uniformly over erasure margins) and, via a discretization of separable metric families, to continuous alphabets: for AWGN with deterministic interference it attains the matched-ML exponent up to an explicit typical-set cap, and for ISI channels the best exponent in a family of equalize-and-decode rules, under explicit assumptions on the equalizers and the channel spectrum and the same cap. Throughout, universality is a corollary of the PEP analysis rather than a separate theory.

Closest papers

Nearest neighbours by meaning across the whole index.

Universal Decoding for Unequal Error Protection

cs.IT ·2026-10-11 ·Seyed AmirPouya Moeini, Albert Guillén i Fàbregas
similarity 0.71
Abstract

We study universal unequal error protection of a single special message over discrete memoryless channels. We construct a sequence of codes with a decoding rule that does not depend on the channel transition law. We derive an achievable error exponent for the special message while ensuring reliable decoding of the ordinary messages. The resulting exponent strictly improves on the natural equal-error-protection random-coding baseline and remains positive as the rate approaches capacity. For symmetric channels, it converges to the optimal red-alert exponent at capacity.

0.17

Communication Complexity of Universal Features

cs.DS stat.MLcs.LG ·2026-10-19 ·Lunjia Hu, Anish Jayant, Peter Li, Princewill Okoroafor, Abhishek Shetty
theory similarity 0.69
Abstract

Motivated by the notion of features from machine learning, we study the complexity of representing data that allows for evaluation with a given family of functions. Concretely, we consider a deterministic encoder that observes $X\sim D$ and produces a finite encoding before a task $f\in\mathcal{F}\subseteq\{0,1\}^{\mathcal{X}}$ is specified. For each task, a decoder must predict $f(X)$ with error at most $\varepsilon$ under $D$. We construct an encoder that outputs a representation with $O(d^{*}\log(1/\varepsilon))$ bits, where $d^{*}$ is the dual VC dimension of $\mathcal{F}$. Our encoder greedily selects $O((d^{*}/\varepsilon)\log(1/\varepsilon))$ functions from the class and uses their joint label pattern to produce the encoding. The key idea in our upper bound is a novel potential function argument, where we compare the entropy gained from each greedy selection with the number of patterns allowed by the dual VC dimension. We present lower bounds showing that the representation length achieved by our encoder is worst-case optimal for every $d^* \ge 1$ and $\varepsilon < 1/4$ up to a universal constant factor, and instance-wise optimal for every family $\mathcal{F}$ up to an $O(\log(1/\varepsilon))$ factor. We give two ways to learn the representation from unlabeled samples, with different guarantees on the representation size and sample complexity. Experimentally, we verify the greedy construction's accuracy-size tradeoff and explore its connection to neural sparse autoencoders.

0.21

One-Shot Information Theory via the Pairwise Error Probability: Lossy, Joint Source-Channel, Erasure, and Multiuser Coding

cs.IT ·2026-08-15 ·Nir Elkayam, Meir Feder
similarity 0.67
Abstract

This paper extends a one-shot (finite-blocklength) information-theoretic framework built on a single primitive: the pairwise error probability (PEP) of a randomized, dither-broken decoding rule, and the error spectrum it induces. A companion paper developed the framework for point-to-point channel coding -- uniformity of the PEP, the error-spectrum representation of achievability and converse, and the linear-programming form of the prior-optimized minimax meta-converse. Here we show that the same primitive, read on an enlarged candidate space, governs four further settings: lossy source coding under average distortion, joint source-channel coding with list decoding, channel coding with an erasure/undetected-error option, and the two-user multiple-access channel. In each case a single spectrum yields a random-coding achievability bound and exact fixed-code identities, and we indicate how the convex -- indeed linear-programming -- prior optimization of the channel-coding case extends under matched decoding. The development recovers the one-shot lossy bound of Matsuta-Uyematsu and the joint source-channel bounds in the Csiszar tradition, complements the lossy bounds of Kostina-Verdu, and connects the multiuser case to the comparable achievability/converse pair through three pairwise error events.

0.38

A Continuous Projection Converse and a Weighted Construction for Uniquely Decodable Code Pairs

cs.IT math.CO ·2026-09-27 ·Lei Yu
theory similarity 0.65
Abstract · 12 pages

We study the maximum sum rate of uniquely decodable code pairs. A weighted refinement of a complement-gluing construction yields an explicit code of length 672 and sum rate exceeding $1.318639029203$. An iterated projection argument, combined with a conditional entropy estimate and a justified continuous limit, gives an upper bound of $1.480063425539$.

0.15

Universal Decoding for Gaussian MIMO-ISI: Spherical Ensembles and Symmetric Neighborhoods

cs.IT ·2026-11-06 ·Nir Elkayam, Meir Feder
theory similarity 0.64
Abstract · 24 pages, 1 figure

We construct a universal decoder for cyclic Gaussian multiple-input multiple-output channels with intersymbol interference and unknown channel taps and spatial noise covariance. Codewords are drawn independently and uniformly from a sphere of fixed total energy. Under stated bounds on frequency variation, the decoder attains the ensemble-average error exponent of informed maximum-likelihood decoding for fixed channels and channels with growing memory. We approximate the frequency response by a constant matrix within each frequency block. For this auxiliary channel, every matched log-likelihood is affine in a common sufficient statistic. At fixed received output, we compute the statistic density of an independent competitor. The decoder selects the candidate with the smallest statistic density, equivalently minimizing a weighted sum of residual log-determinants. The density and its support volume give an upper bound on the universal competitor probability. Affinity ensures that at least half of a suitable symmetric neighborhood of the transmitted statistic has matched likelihood no smaller than that of the transmitted word. Integrating a density lower bound over this half gives the matched competitor bound. These bounds establish equality of error exponents, which a Gaussian channel comparison transfers to the original channel. The same decoder attains the matched exponent for independent frequency blocks with prescribed energies satisfying the stated conditions. For energies proportional to block length, an additional gain condition ensures that this exponent is at least that of the total-energy spherical ensemble.

0.23

Universality Sacrifices Reliability in Classical-Quantum Channel Coding

cs.IT quant-ph ·2026-10-01 ·Kaito Watanabe, Masahito Hayashi, Takaya Matsuura, Hao-Chung Cheng
similarity 0.63
Abstract · 5+23 pages

Universal channel coding enables communication without a complete description of the channel. For classical channels, universal codes can attain both capacity and the optimal high-rate reliability. We show that this compatibility fails for classical-quantum channels in general; that is, the optimal reliability in the channel-aware scenario is not always achievable with universal coding due to the ignorance of the unitary rotation of the output system. We exhibit a family of classical-quantum channels for which one cannot achieve the channel-aware optimal reliability by a fixed coding scheme. We further derive a converse bound on the reliability for unitary-invariant decoders, a natural assumption for the universal coding scheme, that can be strictly smaller than the optimal channel-aware error exponent. Conversely, we construct a channel-independent encoder-decoder pair and establish a universally achievable bound on the reliability that matches this converse bound in the high-rate regime, thereby characterizing the optimal universal reliability. Specifically, the channel-aware and universal exponents are governed by the Petz and sandwiched Rényi divergences, respectively. These divergences coincide for commuting outputs but differ for noncommuting ones, explaining why universality preserves optimal reliability classically but can reduce it quantumly. Our results showcase the fundamental reliability cost of performing the classical-quantum channel coding task universally.

0.30

Soft GRAND under Channel Uncertainty: Minimax Posterior-Envelope Ordering and Gallager-Exponent Preservation

cs.IT ·2026-09-03 ·Behrooz Razeghi
similarity 0.63
Abstract

Soft Guessing Random Additive Noise Decoding queries corrections in an observation-dependent order. For a known channel, the matched order ranks corrections by nonincreasing conditional posterior probability. Under channel uncertainty, admissible channels may induce matched orders that assign different ranks to the same correction at the same observation. We instead use a posterior-envelope order, defined by the pointwise supremum of codebook-independent uniform-input correction posteriors. For fixed measurable posterior representatives and a measurable envelope, this order minimizes, at each observation, the worst-case logarithm of the rank--posterior product over admissible channel--correction pairs with positive posterior probability. For uniform-subset codebooks, this finite-block minimax property does not by itself determine the ensemble-average error exponent, which depends on the distribution of the realized correction rank. For i.i.d. correction--observation pairs with a finite correction alphabet whose joint distribution is dominated by the product of counting measure on that alphabet and a $σ$-finite measure on the observation space, Arimoto conditional Rényi entropy characterizes the matched rank-moment spectrum. An auxiliary posterior-envelope order preserves this spectrum on $[0,1]$ when its posterior family contains the corresponding conditional power tilts and has subexponential conditional Shtarkov complexity on observation sets whose complement probabilities decay exponentially faster than the reciprocal correction-space cardinality. Under additional differentiability and entropy nondegeneracy conditions, the auxiliary order attains the matched uniform-subset ensemble-average error exponent at every fixed normalized code rate strictly between zero and one. For uniform-input memoryless channels, this exponent equals Gallager's uniform-input random-coding exponent.

0.29

Optimal Universal Coding of Integers

cs.IT ·2026-10-01 ·Wei Yan, Yunghsiang S. Han, Leqian Zheng
similarity 0.63
Abstract

Universal coding of integers (UCI) provides prefix codes for positive integers such that, for every nonincreasing source distribution $P$, the average codeword length under $P$ stays within $K$ times $\max\{1,H(P)\}$. For a given UCI $\mathcal{C}$, the smallest such constant $K$ is called the minimum expansion factor of $\mathcal{C}$, denoted $C_{\mathcal{C}}^{*}$. Let $C^{*}\triangleq \inf \{C_{\mathcal{C}}^{*}\}$ over all UCIs. A UCI attaining $C^{*}$, or a family of UCIs attaining $C^{*}$ in the limit, is called optimal. It is currently known that $2\le C^{*}\le 2.0386$. In this paper, we construct a family of one-point plus uniform-tail distributions and prove that, for every UCI, a distribution in this family attains the worst-case ratio; thus, the family is least favorable for the UCI problem. We further establish an inequality, called the \emph{UCI inequality}, which plays the same role for UCI as the Kraft inequality does for prefix codes: for any real number $B\ge 1$, it decides whether $B$ lies below or above $C^*$. The UCI inequality gives an equivalent definition of $C^*$. By numerical computation, we determine $C^* = 2.00012475703610197861701176297579886254\cdots$, the first $38$ decimal digits being certified. Once $C^*$ is known, we can theoretically construct the optimal UCI.

0.19