CS Paper Radar

Semantic Transferability of Generative Rate-Distortion-Perception Systems

Jinwan Jeon, Seoyeon Hwang, Sung Hoon Lim

cs.IT ·submitted 2026-10-26 01:41 UTC · v1

Abstract

Generative lossy compression combines a rate-limited representation with a stochastic decoder that preserves source-dependent information while generating unresolved detail. We ask what remains reusable when a rate-distortion-perception (RDP) system selected for one operating point is transferred to a prescribed conditional-generation task. We distinguish the single-letter RDP factorization from its block-code implementation and compare Full Redesign with three increasingly restrictive transfer strategies. Renderer-Only Transfer retains only the renderer and redesigns semantic steering. Representation-Induced Transfer additionally retains the semantic latent law induced by the original representation. Full-Model Transfer retains a compatible RDP code, marginalizing over the task conditional law to form its semantic encoder. For finite alphabets, we characterize Renderer-Only Transfer's exact compression-common-randomness rate region, renderer-hull controllability condition, and compression-rate penalty relative to Full Redesign. Under unlimited common randomness, this penalty is the minimum residual semantic information $I(S;U|X)$ forced by the inherited renderer. Representation-Induced Transfer is exact if and only if the semantic condition depends on the source only through the recurrent-class label $C$ of the end-to-end RDP test channel. Every compatible RDP code reused under Full-Model Transfer preserves $C$ and supports every task satisfying $X-C-S$. Any additionally transferable within-class semantic dependence must satisfy $I(S;X|C)\le R_0-H(C)$, where $R_0$ is the inherited compression rate. The bound is exact when $R_0=H(C)$. A scalar Gaussian specialization yields a renderer-noise controllability threshold, a closed-form transfer penalty, and examples of identical Gaussian RDP test channels with different Renderer-Only Transfer capabilities.

Comment
38 pages, 2 figures. Submitted to IEEE Transactions on Information Theory

Closest papers

Nearest neighbours by meaning across the whole index.

Operational Indirect Rate-Distortion Functions for Classification Tasks

cs.IT ·2026-11-05 ·Yann Miguet, Elsa Dupraz, Philippe Mary
position similarity 0.62
Abstract · 8 pages, 4 figures

In goal-oriented communications, indirect source coding involves two sources: a semantic source $X$ and an observation $Z$, which is the only source available at the encoder. In this paper, we consider this framework for classification, where the decoder only needs to reconstruct $X$. We argue that the gap between the indirect rate-distortion function and the performance of practical learned coding schemes, such as variational information bottleneck (VIB) and variational autoencoder (VAE) architectures, comes from three effects: the latent representation, the choice of the deep neural network (DNN) architecture and training loss, and whether the encoder is aware of the task. To quantify these effects, we introduce four operational rate-distortion functions that account for the DNN architecture and the training loss, and we establish a hierarchy among them. We then derive these functions for a Gaussian mixture source, where $X$ is the mixture state. Numerical results, both for the Gaussian mixture and for practical coding schemes on the CIFAR-100 dataset, quantify the rate loss incurred by each effect.

0.19

Rethinking Generative Image Compression at Extremely Low Bitrates

cs.CV ·2026-09-30 ·Tianyu Zhang, Zhaoyang Jia, Houqiang Li, Dong Liu
code/data similarity 0.61
Abstract

Generative image compression produces visually plausible reconstructions at low bitrates, yet their behavior as the rate approaches zero remains largely unexplored. When pushed below normal operating rates, representative codecs undergo semantic collapse: rather than gracefully losing source-specific detail, they produce malformed or unrecognizable content. Our analysis identifies two factors. As the bitrate decreases, reconstruction losses increasingly conflict with semantic objectives on gradients and visual results, while pixel-space and reconstruction-oriented VAE diffusion models become less efficient on semantic preservation. Guided by these findings, we introduce RAE-CoD, a compression-oriented diffusion (CoD) built in a representation autoencoder (RAE) space with direct alignment between compressed and source representations, preserving recognizable, naturally structured content for a $256\times256$ image with as few as 16 bits. We evaluate this framework using five vision foundation models (VFM) and a blinded vision-language model protocol. On MSCOCO-30K, RAE-CoD stands out from all evaluation. At 0.001-0.008 bpp, it reduces relative VFM feature MSE and Fréchet Distance ratio by at least 25.7% and 69.1% over the best competitors. Meanwhile, semantic recognizability and quality of the reconstructions remain nearly constant while source consistency falls smoothly, replacing abrupt semantic collapse with a graceful transition toward unconditional generation. Code will be released at https://github.com/LuizScarlet/RAE-CoD.

0.32

The Hidden Perception Constraint in Task-Aware Compression

cs.IT cs.LG ·2026-09-28 ·Sahan Liyanaarachchi, Semih Akkoc, Sennur Ulukus, Aylin Yener
similarity 0.61
Abstract

With the recent advancements of neural compressors, explicitly incorporating perception constraints into the design of compression schemes has gained significant attention. Traditionally, these perception constraints ensure that the distribution of the reconstruction does not significantly deviate from the distribution of the source, thus attesting to the perceptual quality of the reconstruction. In this work, we uncover several perception constraints that are naturally present in task-aware compression. In particular, we consider a problem where the primary task is reconstruction and the secondary task is classification (i.e., a statistical test). We study this problem at varying levels of domain information available to us and discuss how to utilize the naturally emerging perception constraints to design rate-minimal compression schemes that also maximize the utility of our secondary task. We show that in this setting, if the decision boundaries of the classifier are ill-defined (mismatch) for our source distribution, then matching onto a target distribution enhances our classification accuracy.

0.26

From Source Reconstruction to Predictive State Preservation: An Information-Theoretic Framework for AI-Native Communication

cs.IT ·2026-09-01 ·Yi Wang, Linglong Dai
similarity 0.60
Abstract

AI-native communication increasingly aims to support prediction rather than reproduce every detail of the source. This shift raises a basic question left implicit by conventional source coding: what should be preserved when the terminal goal is prediction? We take the source-induced predictive state as the fidelity object. It is the distribution of the specified future conditioned on the source observation and shared context. We show that this state is sufficient and minimal for exact predictive preservation. The terminal prediction loss then defines communication distortion as lost predictive performance rather than source reconstruction error. Under logarithmic loss, this distortion equals the conditional mutual information lost through communication. Using the receiver-side predictive state as a Bayes reference, we separate AI-receiver error into predictive value lost in communication, receiver-available value unusable by the model family, and family capability not realized by the deployed model. The same predictive state also suffices for matched compression. For finite-alphabet memoryless sources, access to the raw source gives no rate-distortion advantage over coding the state directly. Applying the same target-conditioned construction to sequential prediction reveals a dynamic boundary. A state induced by a fixed horizon is minimal for that horizon but may not support recursive updating as the target window shifts. Taking the entire future as the target yields a minimal full-future state that updates recursively and admits a Markov representation. Together, these results shift AI communication from source reconstruction to predictive-state preservation.

0.41

Recasting DiT for Extreme Image Compression with Hierarchical Visual Priors

cs.CV ·2026-10-13 ·Yunuo Chen, Jingnan Gao, Bing He, Jinjing Zhao, Chuqin Zhou +4 more
similarity 0.60
Abstract

Pretrained diffusion models provide powerful generative priors for extreme image compression, but their stochastic, iterative generation processes are not designed to decode compressed representations directly. We present PrismCodec, which recasts a pretrained Diffusion Transformer (DiT) as a bitstream-grounded one-step generative decoder. PrismCodec uses the decoded representation in two complementary roles: a projected codec state defines the starting state for generation, while hierarchical visual priors derived from the same decoded latent provide semantic and structural guidance. To bridge the gap between stochastic, iterative generation and deterministic one-step decoding, we develop a progressive adaptation strategy. We first learn codec-conditioned transport from a noisy neighborhood centered on the codec state, distill it into a one-step mapping, and finally anneal the source perturbation to zero during end-to-end optimization. The resulting decoder reconstructs deterministically from the received representation without additional conditioning bits. Across Kodak, CLIC, and DIV2K, PrismCodec consistently improves rate-perception performance over prior generative codecs, achieving BD-rate reductions of 70-75% in LPIPS and 75-81% in DISTS over StableCodec. Code will be released upon acceptance.

0.20

DARTS: Decoder-Aware Representation Tuning via Surgery for Model Merging

cs.LG ·2026-08-28 ·Aaryan Ajay Sharma, Sai Nishanth Padala, Seganrasan Subramanian
top venue similarity 0.60
Abstract · Accepted to EMNLP 2026 Main Conference

Model merging combines multiple task-specific fine-tuned LLMs into a single multi-task model without additional training. However, merged models are known to suffer from representation bias: systematic drift between the merged model's hidden states and those of each individual source model. Prior work (Yang et al., 2024a) study and mitigate this bias for encoder-based vision models using a lightweight correction module trained with L1 loss. However, such bias is not studied for decoder models due to their autoregressive nature. We analyze the problem of representation bias in decoder models, and show two challenges absent in encoders: (1) the causal attention mask causes bias to accumulate across token positions, requiring position-dependent correction; and (2) not all token positions are equally important, i.e., high-entropy (decision-critical) positions matter far more than low-entropy ones. To address these challenges, we propose Decoder-Aware Representation Tuning via Surgery (DARTS). DARTS employs a novel entropy-weighted L1 loss to upweight correction at high-entropy positions where errors most affect generation quality, and a per-position additive bias that captures position-dependent error without overparameterization. We perform extensive evaluation on three domains: code generation (HumanEval), mathematical reasoning (GSM8K), and instruction following (AlpacaEval) on Llama-2-7B models, and show DARTS achieves significant improvement over the standard surgery approach while adding negligible parameters ($0.1\%$ of total parameters).

0.39

Beyond Task Success: A Theory of Representation Reuse

cs.LG ·2026-10-13 ·Zibo Meng, Andrew C. M. Austin, Alin Achim
similarity 0.60
Abstract · 42 pages, 10 figures

A learned representation is useful beyond its source task only if later tasks can make use of what the encoder has already learned. Source-task performance, however, gives no guarantee that the representation itself will remain useful once the original head is removed. We study representation reuse with respect to a specified family of future tasks. For such a family, we define the task-relevant state as the coarsest deterministic state sufficient for all of its tasks. A larger task family can require a finer state, and losing a required distinction makes at least one future task unrecoverable on a target support containing that conflict. Exact reuse also depends on how the retained information is presented to the downstream rule. We separate this into structural accessibility, which concerns whether the required distinctions remain recoverable, and coordinate consistency, which concerns whether those distinctions can be interpreted by the same downstream rule across samples. Both failure modes appear separately in the experiments. Reuse improves as task-relevant distinctions become more accessible. Reversible sample-dependent coordinate changes can sharply reduce reuse even when the representation remains recoverable, and inverting the applicable coordinate change restores performance. The same accessibility-reuse ordering appears in prospectively specified CIFAR-100 experiments with both ResNet-18 and DenseNet-121. Finally, preserving a candidate state that contains the distinctions needed by the anticipated task family improves reuse on compatible held-out tasks, with the clearest benefit when target supervision is limited. Representation reuse is therefore relative to the tasks that follow source learning: a useful representation must retain the distinctions those tasks need and make them available in a form that a shared downstream rule can use.

0.19

Opportunistic Conditional Entropy Coding with Frozen Analysis and Synthesis Transforms

eess.IV cs.IT ·2026-09-18 ·Vincent Corlay, Maxime Rousselot, Andriy Enttsel
similarity 0.60
Abstract

In many delivery settings, a receiver may already hold a lower-quality or lower-resolution representation of an image, obtained through an independent transmission. Conventional codecs encode a subsequently requested higher-quality representation without exploiting this incidental side information, whereas conditional codecs generally assume a prescribed source of side information that is always available. We instead consider an opportunistic setting in which side information may or may not be present. We introduce a single entropy model that conditions on a previously decoded latent when available and falls back to a standard hyperprior otherwise. The proposed adapter maps the side-information latent to the prior signal required by the entropy model, allowing the same model to support multiple target and side-information quality combinations. The analysis and synthesis transforms remain frozen, enabling retrofitting of an existing learned codec while preserving its latent representation and reconstruction path. When the receiver holds the quality immediately below the target, the proposed method reduces the rate of the subsequent transmission by up to 46%, or by 52% when an additional hyper-latent is transmitted. In the absence of side information, the rate penalty remains below 4%, and the reconstructions are bit-identical across the conditional and fallback modes.

0.46