CS Paper Radar

HR-NVC++: Unified Ultra-High-Resolution Neural Video Coding with Confidence-Guided Spatial-Temporal Cues

Feng Ye, Lijiong Chen, Kai Zhang, Li Zhang, Chuanmin Jia

eess.IV ·submitted 2026-11-11 16:14 UTC · v1

Abstract

Ultra-high-resolution neural video coding (NVC) remains challenging because temporal correspondence becomes increasingly unreliable as spatial resolution increases. Although recent neural B-frame codecs have improved bi-directional temporal prediction, complex motion and fine-grained structures still lead to unreliable temporal references, while spatial information of the current frame remains underexploited. In this work, we propose HR-NVC++, a unified ultra-high-resolution NVC approach that supports both fidelity-oriented and perceptual-quality-oriented compression with a shared spatial-temporal context. The context integrates bi-directional temporal cues with complementary spatial information: confidence-guided fusion adaptively balances the forward and backward temporal cues according to their estimated correspondence reliability, while a spatial complement branch enriches regions that temporal references fail to explain. The resulting context serves as a common basis for two compression modes. Built upon this shared context, the fidelity mode retains entropy-based latent coding for pixel-accurate compression, whereas the perceptual mode removes additional latent coding and performs reconstruction directly from the context. This design demonstrates that a sufficiently informative spatial-temporal context already carries the dominant content required for perceptually favorable reconstruction, allowing different compression objectives to share the same contextual representation. Extensive experiments on UVG-4K and JVET Class A1/A2 show that HR-NVC++ achieves an average PSNR BD-rate saving of 13.01% over the H.266/VVC reference model VTM-13.2 under the random access configuration, while delivering substantially better perceptual quality and temporal consistency than existing neural video codecs.

Closest papers

Nearest neighbours by meaning across the whole index.

Event-guided Neural Video Compression

eess.IV cs.CVcs.MM ·2026-10-01 ·Jiyun Kong, Jungwoo Kim, Enes Eray Demirtas, Touradj Ebrahimi, Jong-Seok Lee
code/data similarity 0.76
Abstract · 28 pages. 21 figures

Neural video codecs derive motion and temporal contexts mainly from RGB frames, leaving room for cross-modal guidance from complementary temporal observations. Event streams can provide such observations by recording brightness changes between frames. In this work, we propose an Event-guided Neural Video Codec (ENVC) that uses events shared by the encoder and decoder to improve RGB compression efficiency. For motion coding, ENVC forms an event-guided motion prior and codes the remaining motion residual. For frame coding, an event-conditioned predictor supplies multi-scale features for gated temporal context refinement. To support training and evaluation on standard video datasets, we synthesize paired RGB-event data and assess its predictive utility through comparisons with real events. Across six benchmarks, ENVC achieves average BD-rate savings of 39.13% using PSNR-RGB and 67.63% using LPIPS relative to DCMVC. Further analyses show that our gains persist on large-motion sequences and that ENVC effectively learns to integrate event information. These results demonstrate the potential of events as a complementary modality for reducing the RGB coding rate. Our model and code are available at https://github.com/kjungwoo03/ENVC.

0.39

BiCRVC: An Efficient Bidirectional Neural Video Compression Framework via Coupled Representation Coding

eess.IV ·2026-08-17 ·Wei Jiang, Junru Li, Kai Zhang, Li Zhang
similarity 0.73
Abstract · Fix some typos

Neural video compression (NVC) has achieved strong compression performance, but practical random-access coding still faces two technical challenges: existing bidirectional NVCs (BVCs) usually require costly motion-first decoding, and reliable motion estimation is difficult under long-range bidirectional prediction. To address these issues, we present BiCRVC, an efficient bidirectional neural video compression framework based on coupled representation coding. Instead of coding motion and frame information with two separate codecs, BiCRVC transforms the motion representation and the current-frame latent into a unified latent representation for entropy coding. This design enables motion and frame information to be decoded from the same bitstream with one unified codec, while still reconstructing motion-aligned contexts for frame decoding. To improve motion accuracy, we introduce multi-candidate motion estimation (MCME), which combines multi-scale motion estimation and parallel accumulated motion estimation to better handle diverse and long-range motions. To reduce motion coding overhead, we further propose bidirectional motion feature propagation (BMFP), which reuses previously decoded motion features at both the encoder and decoder as temporal priors for conditional motion coding. In addition, coupled distortion training and random GOP structure training are used to encourage joint motion-frame coding and improve adaptation to hierarchical random-access structures. Experiments show that BiCRVC achieves better compression performance than state-of-the-art BVCs while providing about 30 times faster 1080p decoding than recent BVCs.

0.30

tcnerv:dual-domain temporal context modeling for implicit neural video compression

cs.CV ·2026-09-15 ·Xuezhi Xiang, Yixin Zhao, Heqi Xiang, Jiayao Liu, Shanjun Zhang
similarity 0.71
Abstract

Video compression aims to minimize reconstruction distor tion under a constrained bit rate. Existing video implicit neural representations (INRs) often decode frames independently, leaving intermediate features unconditioned on previous reconstructions and content embeddings without explicit temporal prediction. We propose TCNeRV, which exploits reconstructed context in both feature and embedding domains. Its multi-scale temporal-context fusion (MTCF) module injects gated historical features at multiple decoder scales, while temporal embedding-residual coding (TERC) predicts each content embedding and codes only its residual. With approximately 3M parameters, TCNeRV achieves an average PSNR of 36.08 dB on the UVG dataset, outperforming HNeRV-Boost by 2.20 dB. It reduces BD-rate by 22.06%, 66.73%, and 29.85% relative to HM, DCVC, and HiNeRV, respectively, demonstrating competitive rate-distortion performance with limited model capacity.

0.18

CoANeRV: Coordinate-Aware Token-Space Neural Video Representation

cs.CV ·2026-08-14 ·Jialong Guo, Ke Liu, Mengxuan Li, Jiajun Bu, Haishuai Wang
code/data similarity 0.70
Abstract · 27 pages, 13 figures. Code is available at https://github.com/jialong2023/CoANeRV

Neural representations for videos (NeRV) have shown strong reconstruction fidelity by storing video-specific information in network weights. However, existing formulations typically require either costly per-video optimization or video-specific weight generation, making it difficult to scale to efficient amortized video representation. We propose CoANeRV, a coordinate-aware token-space framework that adapts the broader token-conditioned neural-field paradigm to amortized video representation. CoANeRV forms compact video tokens in one feed-forward pass and uses a shared coordinate-conditioned decoder to reconstruct continuous spatio-temporal queries, avoiding per-video decoder optimization or generation while retaining coordinate-level reconstruction flexibility. To make token-space reconstruction effective, CoANeRV introduces a coordinate-aware decoding architecture that aligns spatio-temporal queries with video tokens through axis-adaptive positional encoding and temperature-modulated cross-attention. Block-wise coordinate querying further reduces peak attention memory, making high-resolution reconstruction practical. Experiments on diverse video datasets show that CoANeRV consistently improves reconstruction quality over prior feed-forward NeRV and INR baselines, reduces peak memory compared with attention-based coordinate decoders, and provides efficient amortized encoding without per-video optimization. These results support the proposed video-specific combination of feed-forward token formation, spatio-temporal coordinate retrieval, and memory-bounded dense querying. The code is available at https://github.com/jialong2023/CoANeRV.

0.30

RIFT: Residual-Informed Flow Transfer for One-Step Generative Video Compression

cs.CV ·2026-11-01 ·Qizhe Wang, Jiaye Fu, Feng Ye, Haocheng Tang, Zetian Song +3 more
code/data similarity 0.69
Abstract

Recent advances in video generation have enabled generative video compression to reconstruct details lost during coding using pretrained spatiotemporal priors. However, a substantial gap remains between codec-induced degradation and the perturbation trajectories learned by flow-matching models. In this paper, we identify a compression-dependent directional mismatch that degradation-strength-based timestep calibration alone cannot resolve. Compressed latents remain strongly aligned across codec operating points, while their reconstruction residuals vary substantially in magnitude and direction. To bridge this gap, we propose RIFT, a Residual-Informed Flow Transfer framework for one-step generative video compression. At its core, RIFT introduces a Residual Geometry Inducer (RGI) that converts the pretrained flow prediction into a content-adaptive restoration update using decoder-visible codec cues. RGI integrates flow-timestep conditioning with a learned full-band residual-strength estimate and content-conditioned direction and gain modulation, aligning the orientation and magnitude of the update with the compression-dependent residual geometry. Extensive experiments on three public video compression benchmarks show that RIFT outperforms competing generative video compression methods. The code is publicly available on RIFT.

0.19

SGF+: Decoupling Gradient Flows for Autoregressive Video Generation

cs.CV ·2026-10-07 ·Zihan Su, Junhao Zhuang, Yaowei Li, Siwen Lu, Haoran Li +8 more
code/datahighly cited similarity 0.69
Abstract · Project page: https://zihan-su.github.io/self-gradient-forcing-plus

Autoregressive video generation requires denoising the current frames while writing their key-value representations as context for future predictions. However, these two roles typically share parameters, and we find that their gradients exhibit distinct patterns and systematic negative alignment, hindering the joint optimization of visual quality and temporal consistency. We introduce Self Gradient Forcing Plus (SGF+), which assigns separate parameters to context writing and denoising while preserving their interaction through causal attention. Both roles are jointly optimized using the original generation objective without auxiliary losses, with context writing supervised through its contribution to future predictions. This simple change improves visual quality and long-horizon consistency over the evaluated baselines in both framewise and chunkwise generation, without additional video training data or a longer training horizon. Trained on only 5s rollouts, SGF+ supports continuous generation for up to 24 hours without long-video fine-tuning. These results highlight role-specific parameterization as an effective design principle for high-quality autoregressive video generation and native long-horizon extrapolation.

0.51

VoRTeC: Taming Foundation Flow for One-step Real time Video Compression

cs.CV cs.AI ·2026-09-02 ·Yichong Xia, Qinhong Wu, Bin Chen, Jinpeng Wang, Zeyuan Chen +1 more
similarity 0.68
Abstract

Ultra-low bitrate video compression still faces critical challenges: traditional neural video compression inevitably introduces blurring artifacts, while diffusion-based generative video compression suffers from excessive decoding latency and poor temporal consistency. To address these issues, we propose $\mathtt{VoRTeC}$, a Video Compression framework built upon a foundational flow model (Wan2.1). By compactly encoding latent video representations, predicting the positions of compressed representations along flow trajectories, and integrating multi-scale priors, $\mathtt{VoRTeC}$ enables the compressor to harness generative video flow priors effectively. Without accessing the parameters or gradients of flow matching networks, our framework achieves one-step decoding and reconstructions with high perceptual fidelity. Meanwhile, we maintain consistency across frame groups via tail-frame reuse and prior caching. Extensive experiments demonstrate that our method reduces bit consumption by 58\% compared to prior diffusion-based approaches, with decoding speed boosted by 3 to 197 times: $\mathtt{VoRTeC}$ achieves a decoding speed of 13 FPS at 720p and 32 FPS at 480p.

0.39

CoGenRec: A Unified Generative Framework for Complementary Vision Reconstruction

cs.CV ·2026-11-05 ·Yapeng Meng, Yuguo Chen, Xiangru Chen, Shaojun Xu, Xuezhen Wang +5 more
code/data similarity 0.68
Abstract

Capturing scenes with extreme motion and high dynamic range is fundamentally constrained by trade-offs among temporal resolution, spatial detail, signal-to-noise ratio, and dynamic range under practical bandwidth, power, and quantum-efficiency limits. Conventional frame-based cameras therefore suffer from temporal undersampling, exposure-induced motion blur, and sensor saturation. The Complementary Vision Sensor (CVS) addresses these limitations by decomposing visual information across two complementary pathways, a spatially dense, temporally sparse, long-exposure RGB pathway and a temporally dense, spatially sparse, short-exposure patiotemporal difference pathway. Their complementary measurements provide appearance, structural, and motion cues across a wide dynamic range that enable reconstruction beyond conventional imaging. We present CoGenRec, a complementarity-driven generative reconstruction framework that unifies video deblurring, high-temporal-resolution frame interpolation, and HDR fusion. CoGenRec encodes complementary measurements into multi-scale spatiotemporal features using exposure-aware appearance modeling and bidirectional temporal propagation. These features constrain a video diffusion model and are injected into the VAE decoder to recover fine structures lost during latent-space compression, combining the scene fidelity of physical measurements with the appearance recovery capabilities of learned video priors. On SportsSloMo-CVS, CoGenRec improves PSNR over the strongest general-purpose baseline by 4.40 dB for deblurring, 3.89 dB for frame interpolation, and 5.04 dB for HDR fusion, respectively. It also achieves strong perceptual quality on 129 unseen scenes captured with real CVS. These results establish a unified approach to advance high-quality reconstruction under challenging motion and illumination conditions. Project page: https://cogenrec.github.io/.

0.33