CS Paper Radar

How Much Does Saliency Accuracy Matter for Saliency-Guided 360-degree Image Compression?

Sakib Ahmed, Sidi-Mohammed Senouci, Philippe Brunet, Guillaume Caron, Nevrez Imamoglu

eess.IV cs.CV ·submitted 2026-11-16 18:23 UTC · v1
published

Abstract

Saliency-guided coding allocates bits to the regions viewers are most likely to attend, but accurate 360-degree saliency prediction is computationally expensive, an obstacle for low-latency immersive streaming applications such as robot teleoperation. We quantify how much saliency-prediction accuracy actually translates into compression gain. Building on a saliency-guided rate-distortion-optimized (RDO) learned codec, modified so that the training-time latent mask is driven by ground-truth saliency, which yields a single, predictor-agnostic codec, we evaluate saliency predictors spanning two orders of magnitude in compute on both the learned codec and H.265/HEVC. The results show that higher saliency-prediction accuracy yields diminishing compression gains: a lightweight predictor fast enough for live use (52 frames per second) captures 65% of the ground-truth oracle's bit-rate saving, a 3.3x heavier one reaches 74%, and a further 38x increase in compute adds no measurable benefit, on either codec. Therefore, lightweight predictors might also be a practical choice for latency-constrained 360-degree streaming.

Comment
8 pages, 7 figures, 1 table. Accepted at the 2027 IEEE/SICE International Symposium on System Integration (SII 2027)

Closest papers

Nearest neighbours by meaning across the whole index.

Moving Forward with Video Saliency: A New Dataset and Benchmark where Motion Matters

cs.CV ·2026-10-02 ·Susmit Agrawal, Rebecca Wanner, Juliane Verwiebe, Matthias Tangemann, Matthias Bethge +1 more
benchmarkdatasetcode/data similarity 0.66
Abstract

Video saliency prediction is inherently harder to model than static image saliency due to the additional temporal dimension. Video saliency benchmarks rest on the premise that predicting gaze on video requires utilizing temporal activity distributed across frames. Prior work has challenged this, showing that static baselines recover a significant fraction of the explainable gaze information on LEDOV, a popular video saliency dataset, and that video saliency models fail in the same places as this static baseline. We verify that this diagnosis still stands: under a more capable gold standard than the original analysis, and an updated panel of recent architectures, the strongest temporal architecture in the panel still does not substantially improve over a fine-tuned static baseline. However, it remains unclear whether the marginal gain reflects limitations of current temporal architectures or a lack of temporal patterns in the benchmark itself. We introduce SalTempto, a video saliency benchmark with greater dynamism: 224 clips of highly dynamic content, sourced from the HACS-Segments dataset so that each clip contains an event together with its lead-up and aftermath, with gaze recordings from up to 16 subjects and a training split for adapting pretrained models. On SalTempto, the static baseline recovers only about 13\% of the headroom above the centerbias, against more than half on LEDOV. A fine-tuned temporal architecture shows a substantial gain in performance over the static baseline, indicating that it does capture meaningfully more temporal information, which LEDOV fails to measure. Yet, even this SoTA model still leaves nearly half of SalTempto's headroom unexplained, indicating room for improvement in video saliency modelling. Examination of SalTempto also lets us describe human tendencies that models miss. SalTempto link: https://huggingface.co/datasets/bethgelab/video_saliency.

0.32

OpenSAL360: Open-Source Crowdsourcing Platform for Omnidirectional Video Saliency Collection

cs.CV ·2026-09-18 ·Alexey Bryncev, Andrey Moskalenko, Kira Shilovskaya, Ivan Kosmynin, Dmitriy Vatolin
top venuedatasetcode/data similarity 0.66
Abstract · Accepted by ACM MM 2026

Omnidirectional video saliency prediction plays an important role in many immersive multimedia applications, including viewport-adaptive streaming and compression, foveated rendering, mesh simplification, perceptual quality assessment. Yet progress in this area remains constrained by the cost and complexity of collecting eye-tracking data with VR headsets, which makes large-scale dataset creation difficult to extend. We present OpenSAL360, the first open-source platform for scalable, low-cost 360° video saliency collection. Unlike conventional VR-based protocols, it requires only a standard screen, mouse, and internet connection, enabling parallel saliency data collection from common crowdsourcing assessors without specialized hardware. We validate our collection protocol against seven well-established VR eye-tracking datasets and conduct ablation studies on key interface, pre-, and post-processing parameters. To demonstrate the effectiveness and scalability of the proposed methodology, we collect and publicly release a saliency dataset covering 500 omnidirectional videos annotated by 2,000+ crowdsourcing assessors, making it, to the best of our knowledge, the largest dataset in this field. We make OpenSAL360 publicly available at https://github.com/msu-video-group/OpenSAL360.

0.61

You've Seen Enough: Quality-Constrained Image Coding for Machines

cs.CV cs.AI ·2026-09-20 ·Khoa Pham-Dinh, Sanaz Nami, Hamed Rezazadegan Tavakoli, Moncef Gabbouj, Farhad Pakdaman
similarity 0.64
Abstract

Visual data is increasingly consumed by machine-vision systems rather than by human observers. Image Coding for Machines (ICM) compresses images assuming the main observer is a computer vision application and that the human observer needs to inspect or validate the decisions. Inspired by just-noticeable distortion, we cap human-observed quality at a desired level and devote the remaining bits to machine performance. Specifically, joint compression-segmentation training is recast as a constrained optimization problem in which the codec must meet a predefined acceptable target visual quality while a task term consumes the remaining coding capacity. This paper proposes two variants of a penalty function that guides the quality toward the target: an absolute function and a bilinear function, the latter applying a steeper slope once the target visual quality is exceeded. Experimental results show that, under the quality constraint, the proposed method achieves BD-rates of $-22.82\%$ and $-29.81\%$ relative to an unconstrained joint rate--distortion--task optimization and a simple rate--distortion baseline, respectively, showcasing bitrate reduction with the same task performance. This is achieved while the codec also meets the target visual quality with a reasonable error and without adding any complexity overhead.

0.54

Blind Stereoscopic Omnidirectional Image Quality Assessment Using Predictive Coding Hierarchy

cs.CV ·2026-08-28 ·Wei Zhou, André Kaup
top venue similarity 0.64
Abstract · IEEE TMM

Stereoscopic omnidirectional images (SOIs) have provided users with newly immersive quality of experience in virtual reality environments. However, developing efficient and accurate perceptual quality assessment metrics for SOIs remains challenging due to many factors such as freely changeable field of views and binocular vision. In this paper, based on the characteristics of the human visual system (HVS), we propose a Predictive Coding Hierarchy-inspired metric (PCH) for blind/no-reference stereoscopic omnidirectional image quality assessment. Motivated by the viewing process of SOIs, the proposed PCH includes a local cyclopean perception module, a global predictive perception module, and a visual quality regressor. First, observers browse different spherical sceneries from viewports, and aggregate the local visual information to infer the perceptual quality of SOIs. Therefore, we extract various viewports, followed by cyclopean conversion and saliency detection to approach the perception and attention of the human brain. After the local aggregation, viewers then infer the global scene in their minds. Based on the binocular mechanism, we fuse left and right views to perform predictive coding hierarchy modelling. Finally, the visual quality regressor is exploited to obtain the ultimate quality score related to both local and global perceptual cues. Extensive experiments demonstrate that the proposed PCH achieves competitive and consistently improved performance compared with state-of-the-art quality assessment methods.

0.39

Event-guided Neural Video Compression

eess.IV cs.CVcs.MM ·2026-10-01 ·Jiyun Kong, Jungwoo Kim, Enes Eray Demirtas, Touradj Ebrahimi, Jong-Seok Lee
code/data similarity 0.64
Abstract · 28 pages. 21 figures

Neural video codecs derive motion and temporal contexts mainly from RGB frames, leaving room for cross-modal guidance from complementary temporal observations. Event streams can provide such observations by recording brightness changes between frames. In this work, we propose an Event-guided Neural Video Codec (ENVC) that uses events shared by the encoder and decoder to improve RGB compression efficiency. For motion coding, ENVC forms an event-guided motion prior and codes the remaining motion residual. For frame coding, an event-conditioned predictor supplies multi-scale features for gated temporal context refinement. To support training and evaluation on standard video datasets, we synthesize paired RGB-event data and assess its predictive utility through comparisons with real events. Across six benchmarks, ENVC achieves average BD-rate savings of 39.13% using PSNR-RGB and 67.63% using LPIPS relative to DCMVC. Further analyses show that our gains persist on large-motion sequences and that ENVC effectively learns to integrate event information. These results demonstrate the potential of events as a complementary modality for reducing the RGB coding rate. Our model and code are available at https://github.com/kjungwoo03/ENVC.

0.39

PULSE: Unlocking Practical Image Compression on Single-Thread CPU

cs.CV ·2026-09-16 ·Zhaoyang Jia, Tianyu Zhang, Zihan Zheng, Wenxuan Xie, Jiahao Li +3 more
code/data similarity 0.62
Abstract

Despite recent progress in learned image compression, existing methods remain computationally expensive on resource-constrained hardware, particularly CPUs. We introduce PULSE, a practical codec that enables (1) low-latency decoding on diverse hardware platforms with an ultra-low-complexity 5.2 kMAC/pixel neural receiver, and (2) efficient bit-exact entropy coding with an integer linear CDF predictor and a meta prior. To recover compression performance under this tight budget, we introduce an agentic evolution process guided by heuristic probes that iteratively improves the architecture through human-LLM collaboration. PULSE decodes a 1080p image in 126 ms on a single CPU thread while achieving compression performance comparable to HM. After perceptual optimization, PULSE competes with larger perceptual codecs like MS-ILLM. Codes are at https://github.com/microsoft/GenCodec/tree/main/PULSE

0.41

SphereSOD: Geometry-Structure Coupled Learning for 360 Salient Object Detection

cs.CV ·2026-09-07 ·Junsong Zhang, Zhijie Shen, Shuai Zheng, Feng Li, Runmin Cong +2 more
similarity 0.62
Abstract · 12 pages

360° salient object detection (SOD) aims to accurately segment salient regions across a full field of view. However, equirectangular projection (ERP) introduces severe spatial distortion when mapping the spherical domain onto a planar representation. Existing methods mainly focus on compensating projection distortion while overlooking the interaction between panoramic geometry and salient object structure during feature perception and prediction refinement. To this end, we propose SphereSOD, an ERP-native framework that couples panoramic geometry with evolving salient structures. Specifically, spherical geometry governs feature sampling and spatial weighting, while coarse-grained saliency and contour prediction influence context aggregation during the progressive decoding process. SphereSOD first initializes deformable sampling based on spherical projection geometry and then employs bounded, content-adaptive offsets, yielding features that are better aligned with the underlying panoramic geometry. Subsequently, the decoder performs structure-guided context aggregation and progressive refinement to recover complete salient regions and accurate boundaries. Extensive experiments on three public 360° SOD benchmarks demonstrate state-of-the-art performance and a favorable accuracy-efficiency trade-off, supporting structurepreserving inference directly in ERP space as a promising alternative to projection-heavy panoramic pipelines.

0.13

High-Fidelity Video Quality Assessment with VQA-Specific Saliency

cs.CV ·2026-09-15 ·Hakan Emre Gedik, Shashank Gupta, Alan Bovik
published similarity 0.61
Abstract · Accepted to WACV 2027

No-reference video quality assessment (NR VQA) has recently seen promising progress with deep learning. However, video data is inherently large, and processing them with deep models incurs high computational cost. This challenge is particularly acute in VQA, where preserving original-resolution cues and dense temporal information is critical for accuracy. Existing efficiency-driven preprocessing strategies, such as fragmenting, reduce computation but alter the input data distribution, limiting effective reuse of pretrained video foundation models (ViFMs). To address these challenges, we propose \textbf{H}igh-\textbf{F}idelity \textbf{V}ideo \textbf{Q}uality \textbf{A}ssessment (\textbf{HFVQA}), a framework built on fixed-size spatio-temporal (ST) patches that is fully compatible with pretrained ViFMs. HFVQA samples ST patches across multiple scales, including the original resolution, with minimal temporal subsampling to preserve low-level quality cues and semantic context. To limit computation, HFVQA introduces a lightweight auxiliary network trained end-to-end with the ViFM encoder to learn \textit{VQA-specific saliency}. Distilled directly from quality supervision, this saliency captures task-specific importance patterns, reflecting that video quality perception is dominated by a small subset of spatio-temporal regions. By combining high-fidelity spatio-temporal cues with learned, task-specific saliency, HFVQA achieves SOTA performance on standard NR VQA benchmarks while processing as little as 12\% of candidate ST patches, making high-fidelity ViFM-based VQA computationally tractable.

0.34