CS Paper Radar

Lossy Event Compression: From Event Stream Distortion to Task Performance

Zahra Rezaee, Catarina Brites, João Ascenso

cs.CV eess.IV ·submitted 2026-08-28 15:12 UTC · v1
published

Abstract

Event cameras generate asynchronous, sparse data streams with microsecond temporal resolution, but in moderate-to-high motion scenes they can produce as many as hundreds of millions of events per second, creating significant bandwidth and storage challenges. Lossy compression is therefore essential for practical deployment, yet existing event stream distortion metrics fail to reliably predict compression-induced degradation at the task level, forcing codec optimization to rely on expensive task-specific evaluations. To address this gap, this paper introduces two fundamentally different event compression pipelines: i) an aggregation-based pipeline that converts the event stream into polarity-based histogram frames for compression with the conventional image codec JPEG 2000, and ii) a frame-free point cloud-based pipeline that codes events natively as 3D points using the octree-based codec G-PCC. Both pipelines are then assessed within a unified task-driven evaluation framework that relates event stream distortion to downstream application performance across four representative tasks: i) video reconstruction, ii) object detection, iii) optical flow estimation, and a delay-sensitive task iv) asynchronous feature tracking under a reference-relative protocol. Building on this framework, five classification-based distortion metrics are applied to event compression for the first time, to the best of the authors' knowledge, and benchmarked against existing event stream metrics. Experimental results demonstrate that the proposed metrics reliably predict compression-induced task degradation across different coding frameworks. This demonstrates that event stream distortion assessment can be an efficient alternative to repeated task-specific evaluation, providing direct guidance for the development and optimization of future event data coding solutions.

Closest papers

Nearest neighbours by meaning across the whole index.

Event-guided Neural Video Compression

eess.IV cs.CVcs.MM ·2026-10-01 ·Jiyun Kong, Jungwoo Kim, Enes Eray Demirtas, Touradj Ebrahimi, Jong-Seok Lee
code/data similarity 0.73
Abstract · 28 pages. 21 figures

Neural video codecs derive motion and temporal contexts mainly from RGB frames, leaving room for cross-modal guidance from complementary temporal observations. Event streams can provide such observations by recording brightness changes between frames. In this work, we propose an Event-guided Neural Video Codec (ENVC) that uses events shared by the encoder and decoder to improve RGB compression efficiency. For motion coding, ENVC forms an event-guided motion prior and codes the remaining motion residual. For frame coding, an event-conditioned predictor supplies multi-scale features for gated temporal context refinement. To support training and evaluation on standard video datasets, we synthesize paired RGB-event data and assess its predictive utility through comparisons with real events. Across six benchmarks, ENVC achieves average BD-rate savings of 39.13% using PSNR-RGB and 67.63% using LPIPS relative to DCMVC. Further analyses show that our gains persist on large-motion sequences and that ENVC effectively learns to integrate event information. These results demonstrate the potential of events as a complementary modality for reducing the RGB coding rate. Our model and code are available at https://github.com/kjungwoo03/ENVC.

0.39

E-WAVE: Event-based Continuous Optical Flow via Warping-Aligned Visual Encoding

cs.CV ·2026-09-28 ·Jiale Wu, Xiaoyang Bai, Haoming Yu, Yiwei Chen, Yifan Peng +1 more
similarity 0.69
Abstract

Temporally dense optical flow is essential for dynamic perception in immersive VR/AR systems, where rapid head, hand, and object motion must be continuously captured and tracked. Existing frame-based optical flow estimation methods are constrained by the tradeoff between temporal resolution and computational cost; while event cameras, with their high temporal resolution and energy efficiency, serve as a natural solution to the dilemma. However, event-based approaches commonly rely on correlation volumes to capture pairwise voxel correspondences, which incur substantial memory and computation overhead. We present E-WAVE, a correlation-free framework for high-temporal-resolution (HTR) optical flow estimation from event streams. Instead of constructing all-pairs correlation volumes, E-WAVE employs global attention mechanism to model long-range feature dependencies and performs trajectory guided feature warping using Bézier curve. Through iterative updates, it predicts trajectories that allow for querying at arbitrary timestamps without repeated inference. Experiments on MultiFlow and DSEC-Flow demonstrate a 25% lower trajectory error and comparable endpoint flow estimation accuracy relative to state-of-the art baselines. Additional evaluations on self-captured data using a head-mounted prototype validate that E-WAVE remains robust under challenging real-world conditions.

0.21

OASIS: Online Adaptive Video Compression via Closed-loop Feedback Control

eess.IV ·2026-09-14 ·Rentao Wan, Jinho Park, Mingoo Seok
similarity 0.68
Abstract

Computer vision systems are a key building block in an autonomous vehicle, responsible for a range of perception tasks. However, they incur massive data transmission over long communication links from multiple cameras, creating a critical bandwidth and energy bottleneck. Although conventional codecs such as H.264 can reduce data rates, they are ill-suited for real-time vision systems due to high processing latency and energy consumption, as well as their reliance on static user-defined compression settings. In light of these challenges, we propose OASIS, an adaptive video compression framework that integrates lightweight in-sensor compression with task-aware compression ratio control. Based on the real-time task performance, it dynamically updates the optimal compression ratio. Experimental results demonstrate that OASIS generalizes across multiple vision tasks, achieving on average a 6x data compression, 5.8x reduction in link power consumption, and 2.5x reduction in link latency, with at most 1.5% performance degradation.

0.25

4D Reconstruction with Events & RGB

cs.CV ·2026-10-27 ·Dulanga Weerakoon, Darshana Rathnayake, Van Minh Do, Hemanth Sabbella, Kaichen Zhou +5 more
publishedbenchmark similarity 0.67
Abstract · Accepted at NeVi 2026 workshop at the 19th European Conference on Computer Vision (ECCV 2026)

Reconstructing dynamic 3D scenes over time (4D reconstruction) from a single moving camera is central to robotics and AR/VR applications, yet state-of the-art feed-forward methods rely solely on RGB frames and therefore inherit the failure modes of conventional cameras, including motion blur and temporal aliasing under fast or non-uniform camera and object motion. Event cameras report per-pixel brightness changes asynchronously at microsecond resolution, offering a complementary signal that is largely immune to these artifacts, but whether this modality can be incorporated into existing RGB based 4D reconstruction pipelines without a full architectural redesign remains an open question. In this preliminary study, we investigate a minimal extension: we augment Monst3R, an existing RGB-only 4D reconstruction model, with an additional event-camera input branch, leaving the rest of the architecture unchanged. Across 3 benchmark datasets, this simple RGB-event fusion reduces Abs Rel by 3.4% on average relative to the RGB-only Monst3R baseline, and a sensitivity analysis shows that the improvement grows with the magnitude of camera and object motion, with the largest gains observed under high object motion conditions where RGB-only reconstruction degrades most. These early results suggest that even a lightweight, non-invasive fusion of event data can meaningfully improve 4D reconstruction robustness, motivating a more thorough architectural investigation in future work.

0.35

Spatial-Temporal Multi-scale Network for Screen Content Video Quality Enhancement

cs.CV ·2026-09-30 ·Ziyin Huang, Sik-Ho Tsang, Xinyuan Qin, Yui-Lam Chan, Xueling Zhou +1 more
code/data similarity 0.67
Abstract · 5 pages, 4 figures

Different from natural videos, Screen Content Videos (SCVs) are characterized by abrupt motion, scene switches, and high-frequency details such as text and graphics. Conventional video enhancement methods, which rely heavily on temporal continuity, often suffer from performance degradation when processing SCVs due to the disruption of temporal correlations. To address these challenges, we propose the Spatial-Temporal Multi-scale Network (STM-Net), a novel framework specifically tailored for compressed SCV enhancement. Our approach integrates three complementary components: a Prior-Guided Spatio-Temporal Dispatcher (PG-STD) that routes input into three parallel streams to avoid feature contamination, a Bidirectional Temporal Feature Extraction (BTFE) module that adaptively handles abrupt transitions without explicit detection, and a Cascaded Multi-scale Feature Distillation (CMFD) module that preserves critical high-frequency details. Experimental results demonstrate that STM-Net outperforms state-of-the-art methods in both objective metrics and subjective visual quality, providing a robust solution for screen content artifacts. Code is available at https://github.com/HUANGZiyin1/STM-Net.

0.43

RIFT: Residual-Informed Flow Transfer for One-Step Generative Video Compression

cs.CV ·2026-11-01 ·Qizhe Wang, Jiaye Fu, Feng Ye, Haocheng Tang, Zetian Song +3 more
code/data similarity 0.66
Abstract

Recent advances in video generation have enabled generative video compression to reconstruct details lost during coding using pretrained spatiotemporal priors. However, a substantial gap remains between codec-induced degradation and the perturbation trajectories learned by flow-matching models. In this paper, we identify a compression-dependent directional mismatch that degradation-strength-based timestep calibration alone cannot resolve. Compressed latents remain strongly aligned across codec operating points, while their reconstruction residuals vary substantially in magnitude and direction. To bridge this gap, we propose RIFT, a Residual-Informed Flow Transfer framework for one-step generative video compression. At its core, RIFT introduces a Residual Geometry Inducer (RGI) that converts the pretrained flow prediction into a content-adaptive restoration update using decoder-visible codec cues. RGI integrates flow-timestep conditioning with a learned full-band residual-strength estimate and content-conditioned direction and gain modulation, aligning the orientation and magnitude of the update with the compression-dependent residual geometry. Extensive experiments on three public video compression benchmarks show that RIFT outperforms competing generative video compression methods. The code is publicly available on RIFT.

0.19

GenStream: Semantic Streaming Framework for Generative Reconstruction of Human-centric Media

cs.CV cs.AIcs.MM ·2026-09-16 ·Emanuele Artioli, Daniele Lorenzi, Shivi Vats, Farzad Tashtarian, Christian Timmerer
top venuecode/datahighly cited similarity 0.66
Abstract · 9 pages. Published at ACM MM 2025. Code: https://github.com/emanuele-artioli/genstream

Video streaming dominates global internet traffic, yet conventional pipelines remain inefficient for structured, human-centric content such as sports, performance, or interactive media. Standard codecs re-encode entire frames, foreground and background alike, treating all pixels uniformly and ignoring the semantic structure of the scene. This leads to significant bandwidth waste, particularly in scenarios where backgrounds are static and motion is constrained to a few salient actors. We introduce GenStream, a semantic streaming framework that replaces dense video frames with compact, structured metadata. Instead of transmitting pixels, GenStream encodes each scene as a combination of skeletal keypoints, camera viewpoint parameters, and a static 3D background model. These elements are transmitted to the client, where a generative model reconstructs photorealistic human figures and composites them into the 3D scene from the original viewpoint. This paradigm enables extreme compression, achieving over 99.9% bandwidth reduction compared to HEVC for the continuous data stream. We partially validate GenStream on Olympic figure skating footage and demonstrate potential for high perceptual fidelity under minimal data. While acknowledging the significant computational costs shifted to the client and challenges in generalization, GenStream opens new directions in volumetric avatar synthesis, canonical 3D actor fusion across views, and personalized viewing experiences, laying the groundwork for scalable, intelligent streaming in the post-codec era.

0.81

CodecMotion-OAD: Efficient Online Action Detection with Compressed-Domain Motion

cs.CV ·2026-10-17 ·Zepeng Sun, Naichuan Zheng, Hailun Xia, Junjie Wu
similarity 0.66
Abstract

Online action detection (OAD) identifies ongoing actions in streaming videos. Many existing OAD systems combine RGB appearance with optical flow, but dense-flow extraction remains a major computational bottleneck. Codec motion vectors (MVs) are available during decoding, avoiding a separate optical-flow computation. On THUMOS'14, optical flow exceeds MVs by 11.2 mAP points with motion-only inputs, whereas the gap narrows to 3.7 points after RGB fusion, with the largest reduction in action-category discrimination. We propose CodecMotion-OAD, which replaces optical flow with decoder-provided MVs while retaining existing temporal detector architectures. Its two-stage training first aligns MV features with a pretrained flow-feature interface without action labels and then trains RGB+MV detectors using ground-truth labels and soft predictions from a frozen RGB+Flow teacher. To the best of our knowledge, this is the first application of codec MVs to frame-level OAD. On THUMOS'14 with MiniROAD, CodecMotion-OAD achieves 68.3 mAP versus 70.9 mAP for matched-distillation RGB+Flow, while reducing decoder-to-prediction processing time by 11.48-fold on CPU and 2.04-fold on GPU.

0.14