CS Paper Radar

Information Capacity of Generative Video Compression: Quantifying the Rate-Compute Exchange at Identical Quality

Cheng Yuan, Jiawei Shao, Xuelong Li

cs.CV ·submitted 2026-09-23 07:56 UTC · v1

Abstract

Under the AI Flow framework, communication networks distribute intelligence across devices, edge servers, and clouds, and computation at the receiver becomes a resource that can substitute for transmitted bits. Generative video compression (GVC) embodies this exchange by sending compact tokens with ultra-low bitrate and letting a generative decoder synthesize the video, yet how much bandwidth savings a unit of decoder compute actually achieves has never been quantified. To fill this vacancy, we model reconstruction quality as a two-factor power law in data rate and decoder compute, which fits measured DISTS of two GVC decoders with a mean error below 3%, and define the information capacity (IC) as the negative logarithmic slope along an iso-quality contour, namely the fraction of rate saved per fractional increase in compute at identical quality. IC is dimensionless and unit-invariant, thus enabling an architecture-agnostic comparison. It forms a field over the operating plane, locating where additional denoising steps are worth their cost. Across five datasets, the 14B decoder trades more compute for fewer rate about ten times more efficiently than the 1.3B decoder. IC also varies significantly across datasets, indicating imbalanced performance on the rate-compute trade-off in GVC methods.

Closest papers

Nearest neighbours by meaning across the whole index.

Perceptual Refinement of an End-to-End Video Streaming Pipeline via Generative AI Layers

eess.IV cs.AIcs.CVcs.MM ·2026-09-16 ·Emanuele Artioli, Farzad Tashtarian, Christian Timmerer
theorycode/data similarity 0.69
Abstract · 28 pages. Submitted to ACM Transactions on Multimedia Computing, Communications and Applications (TOMM), special issue on MMSys and co-locat…

Traditional codecs treat every region of a frame alike; a generative layer can instead degrade the regions a viewer attends to least and reconstruct them at the client. We present PRESLEY, which extends the prior conference work ELVIS by replacing destructive block removal with adaptive in-place degradation under a removability mask, signaling per-block strength in a bit-packed side channel, and restoring via generative backbones conditioned on transmitted visual priors rather than unconditioned in-painting. We separate the problem into three goals: choosing which blocks to degrade, degrading them so the encoder spends fewer bits, and restoring them. Against its predecessor at matched rate, PRESLEY achieves a decisive mean -56.4% BD-rate reduction on delivered background quality across 13 rate ladders spanning multiple codecs and dataset families. Against pristine baselines, PRESLEY defines the operating regime of generative transport: delivering substantial bitrate savings (up to -29.4% BD-rate) and superior background quality (17/23 sequences) in the target bit-starved regime, while maintaining foreground fidelity bit-exact. We further map where the theoretical headroom in this class of architecture lies. Using an exact leave-one-superblock-out combinatorial oracle as an additive empirical bound, we show that existing complexity heuristics already capture 83.3% of bit-cost savings, bounding remaining cost-axis headroom at about 5% of total bitrate. We then identify and model the primary unaddressed axis -- post-restoration damage -- which disperses widely (4.9-8.4 dB). We prove that this damage is predictable before transmission (held-out rho = +0.400), establishing the feasibility of transmit-time restorability modeling and defining the roadmap for joint rate-distortion-restoration selection rules.

0.38

Towards High-Fidelity Learned Video Compression

cs.CV ·2026-11-19 ·Bumyoon Kim, Yongseong Kim, Inhyuk Jeong, Byeungwoo Jeon
similarity 0.68
Abstract · 11 pages, 5 figures

Professional video workflows operate at qualities that learned video codecs do not reach. We measure how much of a professional quality range a codec covers, from the lowest quality of the standard test conditions, VVC at QP 37, to the highest 4:2:2 grade of the production codec ProRes. To compare inter-frame coding alone, every codec receives original frames as references at the intra positions. On 10-bit YCbCr 4:2:2 test sequences, four recent learned codecs cover 24 to 38% of this range. Their encoders are feed-forward networks with a fixed latent capacity, the number of latent values per sample, of at most 0.25, and removing their quantization raises the quality by at most 1 dB. We propose a learned inter-frame codec whose encoder searches, as the encoders of conventional codecs do. It searches block motion, which a learned network refines, and it searches among learned block transforms whose latent capacity ranges up to 1, selecting the partition, the skip decisions, and the quantized coefficients by a rate-distortion cost. Our codec reaches the upper end of the range on all six sequences. With all residual blocks fixed to 16x16, whose latent capacity is 0.25, the same codec covers 55% of the range, with or without quantization.

0.27

Rethinking Generative Image Compression at Extremely Low Bitrates

cs.CV ·2026-09-30 ·Tianyu Zhang, Zhaoyang Jia, Houqiang Li, Dong Liu
code/data similarity 0.67
Abstract

Generative image compression produces visually plausible reconstructions at low bitrates, yet their behavior as the rate approaches zero remains largely unexplored. When pushed below normal operating rates, representative codecs undergo semantic collapse: rather than gracefully losing source-specific detail, they produce malformed or unrecognizable content. Our analysis identifies two factors. As the bitrate decreases, reconstruction losses increasingly conflict with semantic objectives on gradients and visual results, while pixel-space and reconstruction-oriented VAE diffusion models become less efficient on semantic preservation. Guided by these findings, we introduce RAE-CoD, a compression-oriented diffusion (CoD) built in a representation autoencoder (RAE) space with direct alignment between compressed and source representations, preserving recognizable, naturally structured content for a $256\times256$ image with as few as 16 bits. We evaluate this framework using five vision foundation models (VFM) and a blinded vision-language model protocol. On MSCOCO-30K, RAE-CoD stands out from all evaluation. At 0.001-0.008 bpp, it reduces relative VFM feature MSE and Fréchet Distance ratio by at least 25.7% and 69.1% over the best competitors. Meanwhile, semantic recognizability and quality of the reconstructions remain nearly constant while source consistency falls smoothly, replacing abrupt semantic collapse with a graceful transition toward unconditional generation. Code will be released at https://github.com/LuizScarlet/RAE-CoD.

0.32

Rate-Adaptive One-Step Diffusion Compression for AIGC Images

cs.CV ·2026-09-25 ·Nitiz Khanal
benchmark similarity 0.67
Abstract

We describe our entry to the LoViF 2026 AIGC Image Compression Challenge, a benchmark for ultra-low-bitrate coding of AI-generated images under a strict global rate budget of 0.025 bits per pixel (BPP). Generated imagery poses a distinct challenge for compression: it frequently contains rendered typography, synthetic edges, repeated motifs, UI-like layout, and stylized micro-texture that conventional distortion-oriented codecs erase at this rate, while unconstrained generative decoders can restore plausible-looking detail that no longer matches the source geometry or symbols. We treat this as a rate-perception allocation problem. Our system fine-tunes four rate-specialized checkpoints of the AEIC one-step diffusion codec, generates per-image candidates from all four, including one latent refined through encoder-side test-time optimization (TTO) with periodic entropy-conditioning refresh, entropy-codes every candidate with practical rANS coding, and selects exactly one bitstream per image with an exact multiple-choice knapsack solved over true coded file sizes. A fixed, zero-additional-bit residual restoration network is applied at decode time. Every submitted bitstream is independently decodable by the shipped decoder, which uses no source image or external side information. We report the full pipeline, an ablation history spanning 77 logged experiments, and a set of negative results, including why PSNR could not be pushed to parity with rate-distortion-oriented competitors under this architecture, useful to future participants. This is a challenge report: our entry scored 31.527739 (PSNR 27.02 dB, MS-SSIM 0.9176, LPIPS 0.0778, DISTS 0.0390 at 0.02495 BPP), the second-best DISTS on the leaderboard, ranking 5th on the final test-phase leaderboard announced August 4, 2026.

0.15

The Hidden Cost of MoE Diffusion Decoding: Profiling and Fixing the Sampler

cs.LG cs.AIcs.CL ·2026-11-16 ·Harshit Samrat, Ramakrishna Paramahamsa Kompella
publishedbenchmarkcode/data similarity 0.67
Abstract · Accepted to NeurIPS workshop DiffuLM 2026

Masked diffusion language models (dLLMs) decode by iteratively denoising a full canvas of tokens, so every forward pass runs a per-token sampling and stopping pipeline over the entire canvas-by-vocabulary logits tensor. We profile single-user decoding of a 26B-parameter diffusion Mixture-of-Experts model on a bandwidth-bound edge accelerator (NVIDIA GB10) and report a result that is easy to miss: after weight quantisation, this logits pipeline, not the expert GEMMs, is the single largest consumer of decode time. Unlike autoregressive decoding, where sampling touches one position's logits per committed token, diffusion decoding processes S times as many logit entries per output token, where S is the number of denoising steps. We show this pipeline is largely redundant or fusable and remove it in two lossless tiers: a bit-exact tier whose output is byte-identical to the reference decoder, and an accuracy-parity tier using a fused single-pass entropy kernel and Gumbel-max sampling. Combined with a W4A16 expert kernel, the full stack reaches 2.4-2.7 times end-to-end speedup over bf16 at unchanged benchmark accuracy on GSM8K, HumanEval, and MATH, on a single 273 GB/s device. We confirm the finding on a second, independently built diffusion MoE (LLaDA2.0-mini: pipeline 28.8% of decode, fused W4A16 3.3 times at accuracy parity) and show it disappears on a dense diffusion model (3.6%). The cost is specific to MoE diffusion, where quantisation makes the backbone cheap. Code and quantised weights are released.

0.42

End-to-End Learned Image Coding: From Rate-Distortion Optimization to Practical Deployment

eess.IV cs.MM ·2026-10-15 ·Ming Lu, Junqi Shi, Zhan Ma
benchmark similarity 0.66
Abstract

End-to-end learned image coding (LIC) has achieved substantial improvements in rate--distortion (R--D) performance, yet practical deployment requires addressing a broader set of system-level constraints. This article provides a deployment-oriented review of LIC, with particular attention to the transition from high-performance neural compression models to practical codecs. We first summarize the basic end-to-end coding framework, including learned transforms, quantization, entropy modeling, and R--D optimization. We then review practical extensions that are increasingly important for deployment, including variable-rate coding and rate control, integer-precision inference and cross-platform consistency, robustness and communication resilience, lightweight and hardware-aware implementation, and standardization and interoperability. To study these issues under a common setting, we further introduce TinyLIC as an open benchmark and reproducible case study for rate--distortion--complexity analysis. Experimental results illustrate the trade-offs among compression efficiency, computational complexity, numerical consistency, and robustness. In particular, TinyLIC with approximately 10 KMACs/pixel decoding complexity substantially outperforms the H.265/HEVC image-profile anchor on Kodak, while the 50 KMACs/pixel configuration achieves performance comparable to the H.266/VVC image-profile anchor on several test sets. The review highlights practical constraints and design trade-offs that remain central to the deployment and interoperability of learned image codecs. Materials are accessible at https://njuvision.github.io/TinyLIC.

0.14

VoRTeC: Taming Foundation Flow for One-step Real time Video Compression

cs.CV cs.AI ·2026-09-02 ·Yichong Xia, Qinhong Wu, Bin Chen, Jinpeng Wang, Zeyuan Chen +1 more
highly cited similarity 0.64
Abstract

Ultra-low bitrate video compression still faces critical challenges: traditional neural video compression inevitably introduces blurring artifacts, while diffusion-based generative video compression suffers from excessive decoding latency and poor temporal consistency. To address these issues, we propose $\mathtt{VoRTeC}$, a Video Compression framework built upon a foundational flow model (Wan2.1). By compactly encoding latent video representations, predicting the positions of compressed representations along flow trajectories, and integrating multi-scale priors, $\mathtt{VoRTeC}$ enables the compressor to harness generative video flow priors effectively. Without accessing the parameters or gradients of flow matching networks, our framework achieves one-step decoding and reconstructions with high perceptual fidelity. Meanwhile, we maintain consistency across frame groups via tail-frame reuse and prior caching. Extensive experiments demonstrate that our method reduces bit consumption by 58\% compared to prior diffusion-based approaches, with decoding speed boosted by 3 to 197 times: $\mathtt{VoRTeC}$ achieves a decoding speed of 13 FPS at 720p and 32 FPS at 480p.

0.39

PerQ: Inverse Generative Modeling for Neural Image Compression via Quantization Error Compensation

cs.CV ·2026-10-19 ·Tianhao Peng, Ho Man Kwan, Fan Zhang, Shan Liu, Dave Bull
code/data similarity 0.64
Abstract

Neural image codecs face a fundamental trade-off between pixel fidelity and perceptual quality. Distortion-optimized codecs preserve pixel-wise fidelity but produce blurred reconstructions at low bitrates, whereas generative codecs leverage learned priors to improve perceptual quality but tend to exhibit saturating PSNR at higher bitrates and require separate models for different bitrate targets. We propose PerQ, which reframes perceptual reconstruction as inverse generative modeling with bounded support. Specifically, we define the quantization cell as the set of latents that yield the same transmitted symbols and derive a transport bound showing that reconstructions from this cell remain within a rate-determined radius of the codec output. This motivates performing generative reconstruction within the codec-induced quantization cell, rather than sampling from an unconstrained image prior. PerQ implements this principle by attaching a rate-conditioned flow-matching compensator to a frozen, multi-rate distortion-optimized backbone, enabling perceptual enhancement while preserving the backbone's coding performance. Experiments on Kodak and CLIC2020 show that PerQ achieves perceptual performance comparable to generative codecs such as MS-ILLM in LPIPS and FID, while maintaining competitive PSNR relative to distortion-optimized codecs such as ELIC. A single model checkpoint supports a wide bitrate range with fast decoding, and the compensator provides consistent gains across three different backbones. The implementation is available at https://jasminepp.github.io/PerQ/.

0.31