CS Paper Radar

Lossy Compression via Sparse Regression Codes: Generalized Construction and Finite-length Bounds

Galen Reeves, Ramji Venkataramanan

cs.IT ·submitted 2026-08-14 17:08 UTC · v1

Abstract

We study sparse regression codes (SPARCs) for lossy compression under simple greedy encoding rules, including both correlation-based and distance-based methods. We generalize the SPARC construction, and consider the class of \emph{additive orthogonal} regression codes, of which standard SPARCs are a special case. For this class of codes, we derive nonasymptotic bounds on the squared-error distortion by tracking the evolution of the encoding residual across stages. Our results highlight the role of power allocation in controlling the distortion, allowing us to optimize the allocation based on the parameters of the code. The optimized allocation improves the finite-length compression performance of SPARCs, and our bounds provide distortion guarantees for lower complexity variants of SPARCs, like signed SPARCs and $K$-sparse SPARCs.

Closest papers

Nearest neighbours by meaning across the whole index.

A Continuous Projection Converse and a Weighted Construction for Uniquely Decodable Code Pairs

cs.IT math.CO ·2026-09-27 ·Lei Yu
theory similarity 0.63
Abstract · 12 pages

We study the maximum sum rate of uniquely decodable code pairs. A weighted refinement of a complement-gluing construction yields an explicit code of length 672 and sum rate exceeding $1.318639029203$. An iterated projection argument, combined with a conditional entropy estimate and a justified continuous limit, gives an upper bound of $1.480063425539$.

0.15

Lossless Source Coding with Marginal Distribution Constraint

cs.IT ·2026-11-04 ·Cheuk Ting Li, Jun Chen, Shahab Asoodeh
similarity 0.62
Abstract · 15 pages

We consider lossless source coding with a marginal distribution constraint on the encoding symbols (SC-MDC). Unlike distribution matching, signal shaping and homophonic coding which require the symbols to be (approximately) i.i.d. following a target distribution, SC-MDC requires each symbol to have a marginal distribution that exactly matches the target distribution, which can be regarded as a perfect perceptual quality constraint on the symbols, and may have applications to wear leveling, source coding with cost and steganography. We give nonasymptotic lower and upper bounds on the optimal encoding length, and exactly characterize the coding rate (or the first-order term of the length) when the source distribution or the target distribution is uniform, or when the source is an i.i.d. sequence.

0.26

Pairwise distortion distribution for compression and quantization

cs.LG ·2026-11-09 ·Carl Nuzman, Jeongmin Chae, Kursat Rasim Mestav, Urbashi Mitra
publishedtheory similarity 0.61
Abstract

Choosing the latent dimension is a critical challenge in data compression, where conventional hyperparameter optimization is costly and ad hoc. Intrinsic dimension provides a good alternative, which connects information-theoretic notions such as Renyi information dimension (RID) to rate-distortion dimension and random codebook compression. In this work, we introduce the pairwise distortion distribution (PDD) and its asymptotic limit, pairwise dimension (PD), which generalize the correlation integral to general distortion measures. We show that PD bounds RID, and PDD provides a lower bound for random codebook quantization performance, which can guide a practical use for compression, quantization, and dimensionality reduction applications.

0.45

Entropy/Length Profile Bounds for the Conversion of (Non)linear Codes in the Merge Regime

cs.IT ·2026-10-11 ·Songping Ge, Han Cai, Xiaohu Tang
similarity 0.60
Abstract · 26 pages, 1 figure

Code conversion changes the parameters of encoded data while avoiding complete decoding and re-encoding. Existing general bounds largely rely on the algebraic structure of linear codes. This paper develops an information-theoretic framework for nonlinear code conversion in the merge regime. Using the entropy/length profile (ELP), we derive upper bounds on the number of unchanged symbols and lower bounds on the number of read symbols for arbitrary $(n,M)_q$ codes, without assuming linearity or requiring $\log_q M$ to be an integer. We also obtain joint bounds for multiple initial codes and consequences involving minimum distance. For linear codes, the proposed bounds reduce to the corresponding generalized Hamming weight (GHW) bounds. The construction based on Hamming codes converts Hamming codes of redundancy two into a Hamming code of any larger redundancy and simultaneously attains the general bounds on unchanged and read symbols. The construction based on quasigroups shows that new bounds remain tight for genuinely nonlinear codes of nonintegral dimension and at a nonintegral entropy threshold, which, to the best of our knowledge, is the first construction of convertible codes for nonlinear codes.

0.29

On Optimal Encodings and Systematicity in Function-Correcting Codes

cs.IT ·2026-10-06 ·Charul Rajput, Kanchana Lokshmii Jagatti, B. Sundar Rajan
theory similarity 0.60
Abstract

Function-correcting codes (FCCs) protect the value of a function $f$ of the message against $t$ errors. In the original formulation of Lenz et al. (2023), the encoding is systematic, and the lower bound of $2t$ on the redundancy relies on this form. Recently, D. Ho (arXiv, 2026) showed, for linear functions and linear encodings, that systematicity can cost redundancy. We begin with the OR function to show that the cost is not confined to the linear setting: a non-systematic encoding attains redundancy $1$ while every systematic encoding needs $2t$. For a linear function $f$ and a fixed linear code $C$, let $d_f$ denote the minimum distance between codewords of messages with different function values. Different generator matrices of $C$ assign different codewords to the messages and can give different values of $d_f$. We study which generator matrix of $C$ gives the largest $d_f$. We give an algorithm that constructs an optimal generator matrix and determines the optimal value as the weight of a codeword in a greedy basis of $C$. We then characterize, in terms of information sets, when a generator matrix in systematic form attains this optimum, and give a necessary condition that is checked on the low-weight codewords of $C$ alone. For two-valued functions, we drop both linearity and systematicity. Using the minimality of initial segments of the simplicial order with respect to Hamming neighbourhoods, we show that a non-systematic $(f,t)$-FCC of length $n$ exists if and only if a condition depending on $f$ only through the size of its smaller preimage holds. For $t=1$, redundancy $1$ is sufficient for every nonconstant two-valued function on $\mathbb{F}_2^k$ with $k\ge 10$. For general $t$, redundancy $1$ suffices for all two-valued functions once $k$ is large enough, and for each $s<2t$ we give an upper bound on the threshold in $k$ beyond which redundancy $s$ suffices.

0.15

MacKay-Neal Codes Achieve Capacity under MAP Decoding

cs.IT ·2026-10-07 ·Kenta Kasai
theory similarity 0.60
Abstract · 22 pages

Statistical-mechanical analyses predict that MacKay-Neal codes can achieve channel capacity at fixed degrees. We prove this prediction for the uncoupled $(\ell,3,3)$ MN socket ensemble for every fixed integer $\ell\ge4$. For each binary-input memoryless symmetric channel of capacity strictly greater than $3/\ell$, the ensemble-average block error probability under maximum a posteriori (MAP) decoding is $O(\log N/N)$, where $N$ is the transmitted blocklength. The actual transmitted rate converges to $3/\ell$. The result includes both punctured and transmitted variables and does not condition the sparse square matrix on invertibility. The proof establishes an entropy inequality for six bits subject to even parity by an analytic argument that reduces the domain by symmetry, restricts its interior stationary points, where both partial derivatives vanish, to the diagonal, and controls the resulting scalar functions by explicit polynomial bounds. Exact configuration counts then bound the conditional entropy on the binary symmetric channel. A standard comparison by mutual information extends the entropy bound to general symmetric channels; independent output erasures and a minimum-distance estimate for the transmitted code yield the bit and block error bounds.

0.42

An SCL-Aware Design of PAC Codes

cs.IT ·2026-10-29 ·Tyler Kann, Shrinivas Kudekar, Matthieu R. Bloch
similarity 0.60
Abstract

We develop design methodologies for the rate profile of polarization-adjusted convolutional (PAC) codes tailored to successive cancellation list (SCL) decoding. The crux of our approach is two-fold: first is the creation of a modified density evolution Gaussian approximation (DEGA), called conditional DEGA (CDEGA), to analyze the decoding process in the presence of errors; second is the creation of a dangerous-path occupancy metric to find the conditional occupancy of bit-errors in a finite list. Our approach not only illuminates the role of rate-one precoding, but also provides insight into why certain information sets are particularly well suited to PAC codes. We demonstrate the usefulness of CDEGA and dangerous-path metrics with algorithms that create new information sets, as well as precoders that are extremely sparse yet competitive. In particular, we design codes of length $N \leq 512$ with various rates and list sizes, all of which offer at least half a dB gain over traditional PAC codes at an FER of $10^{-3}$ and improve upon state-of-the-art results.

0.30

Exact Redundancy under Average and Almost-Sure Distortion Constraints

cs.IT ·2026-11-15 ·Yijun Fan
hot direction similarity 0.59
Abstract

We characterize the exact redundancy of lossy source coding under average and almost-sure distortion constraints. For general source and output alphabets, we establish the asymptotic redundancy coefficients under suitable regularity conditions. Under the average distortion constraint, the coefficient is half of the minimum probability that an output reveals information about the source, among all rate-distortion optimal channels. Under the almost-sure distortion constraint, the coefficient is either $1/2$ or $1$, depending on whether the $D$-tilted information is constant or not. Both converse results follow from a common change-of-measure inequality. Finally, we verify the required conditions under both distortion requirements always hold for finite alphabets.

0.23