CS Paper Radar

JEVQA - Video Quality from Metadata, Bitstream, and Pixel Features with a General-Purpose Decision Model

Werner Robitza

eess.IV ·submitted 2026-09-21 10:45 UTC · v1
highly cited

Abstract

Instrumental quality models for video quality prediction are usually trained for a fixed set of codecs or other input features, and every new input variable requires retraining. Novel, general-purpose decision models can answer questions without task-specific training, but it is unclear whether they can judge video quality. We evaluate Jev, a commercial ``System One'' model that returns probability distributions over a provided answer scale, as a zero-shot video quality model. We call the resulting method JEVQA. In a first study on 1,936 AV1, H.264, HEVC, and VP9 encodes of 22 sources, scored against VMAF as ground truth, using encoding metadata only, JEVQA reached a Pearson correlation of 0.737, on par with the standardized ITU-T P.1204.1 model (0.733). Giving the model bitstream data raised the accuracy to 0.797, and combined pixel-based and bitstream features raised it to 0.824. A pixel-only variant failed in our tests. In a second study, using H.264, HEVC, and VP9 encodes in the AVT-VQDB-UHD-1 database, the metadata-only model reached a correlation of 0.879 with MOS, close to P.1204.1 (0.898). Bitstream statistics did not help there. Our results show that trained models on the same features remain clearly ahead in both studies, but that zero-shot classifiers are promising.

Closest papers

Nearest neighbours by meaning across the whole index.

Neural video codecs quality assessment dataset and benchmark

cs.CV ·2026-08-29 ·Nikolay Safonov, Nikita Gornostaev, Alexandra Dubonos, Dmitriy Vatolin
top venuebenchmarkdatasetcode/data similarity 0.64
Abstract

Video traffic constitutes a significant share of global web traffic. To reduce its volume, video codecs have been developed and continuously improved. While the industry has achieved substantial progress in traditional video coding, neural video codecs (NVCs) have recently emerged as a new approach that applies deep learning to video compression. This creates new challenges for compression quality assessment, which is essential for the further development and improvement of such codecs. In particular, it is important to evaluate the novel temporal compression paradigms introduced by NVCs. In this work, we present a large-scale subjective dataset of videos compressed with both neural and traditional video codecs. The subjective scores were collected through crowd-sourced pairwise comparisons. The proposed dataset provides a valuable resource for the development and benchmarking of video quality metrics tailored to neural video codecs. The dataset is available at the following link: https://videoprocessing.github.io/nvc-dataset-benchmark

0.45

Towards High-Fidelity Learned Video Compression

cs.CV ·2026-11-19 ·Bumyoon Kim, Yongseong Kim, Inhyuk Jeong, Byeungwoo Jeon
similarity 0.62
Abstract · 11 pages, 5 figures

Professional video workflows operate at qualities that learned video codecs do not reach. We measure how much of a professional quality range a codec covers, from the lowest quality of the standard test conditions, VVC at QP 37, to the highest 4:2:2 grade of the production codec ProRes. To compare inter-frame coding alone, every codec receives original frames as references at the intra positions. On 10-bit YCbCr 4:2:2 test sequences, four recent learned codecs cover 24 to 38% of this range. Their encoders are feed-forward networks with a fixed latent capacity, the number of latent values per sample, of at most 0.25, and removing their quantization raises the quality by at most 1 dB. We propose a learned inter-frame codec whose encoder searches, as the encoders of conventional codecs do. It searches block motion, which a learned network refines, and it searches among learned block transforms whose latent capacity ranges up to 1, selecting the partition, the skip decisions, and the quantized coefficients by a rate-distortion cost. Our codec reaches the upper end of the range on all six sequences. With all residual blocks fixed to 16x16, whose latent capacity is 0.25, the same codec covers 55% of the range, with or without quantization.

0.27

MDFI: A Multi-Domain Features Integration for Compressed Video Quality Enhancement

eess.IV cs.CV ·2026-08-21 ·Sang NguyenQuang, Hieu Bui Minh, Dang BuiDinh, Xiem HoangVan
top venuedatasetcode/data similarity 0.61
Abstract

The latest video coding standard, H.266/VVC, has demonstrated significant improvements in compression efficiency compared to H.265/HEVC. Despite its advanced coding techniques, H.266/VVC still faces challenges in meeting the increasing demand for higher perceptual quality and enhanced compression performance. To address these limitations, we propose MDFI (Multi-Domain Features Integration), a compressed video quality enhancement approach that features a novel Frame-Prediction Feature Transform (FPFT) module to process prediction information. Moreover, MDFI integrates a multi-domain feature fusion strategy that effectively combines spatiotemporal characteristics, cross-frequency representations, and compressed-domain prediction information to enhance decoded video quality. Additionally, we introduce a comprehensive dataset that encompasses uncompressed video sequences, corresponding reconstructed versions at multiple QP levels, and predicted frames generated from H.266/VVC compressed bitstreams, providing essential resources for developing and benchmarking video enhancement approaches. Extensive experiments demonstrate that our MDFI approach achieves superior performance to state-of-the-art methods in both objective metrics and visual quality, effectively mitigating video compression artifacts. The code is available at: https://github.com/dangdinh17/MDFI.git.

0.75

Scalable SSIM Estimation from PSNR for Per-Title and Context-Adaptive Encoding Workflows

eess.IV cs.MM ·2026-09-19 ·Luc Trudeau, Maria G. Martini
publishedcode/datahot direction similarity 0.61
Abstract · Presented at IBC 2026, Amsterdam, The Netherlands. 2026 IBC Technical Paper

Modern streaming pipelines run hundreds of candidate encodes per asset to support per-title encoding, shot-based optimization, and context-adaptive ABR ladder construction. These techniques have moved perceptual quality metrics into the critical path: SSIM and VMAF now guide encoding decisions rather than passively monitor them. We measure that SSIM evaluation accounts for 7-35% of x264 encode time at production speed presets, with the cost ratio rising as encoders run faster. We propose ApproxSSIMate, a low-complexity method for estimating SSIM from PSNR combined with reference-sequence statistics computed once per sequence and reused across every candidate encode. This decouples quality estimation from the encode-decode-compare loop, enabling perceptual quality feedback in live encoding and amortizing quality measurement across candidate encodes in per-title workflows. We validate the approach across H.264/AVC, H.265/HEVC, and AV1 on the Objective-1-fast dataset and release the implementation as free and open-source software.

0.54

GS-PQM: A Parameter-Domain Quality Metric for Compressed Gaussian Splatting

cs.GR cs.CVcs.MM ·2026-09-18 ·Pedro Martin, António Rodrigues, João Ascenso, Maria Paula Queluz
similarity 0.59
Abstract

Recent advances in Gaussian Splatting (GS) compression have enabled substantial reductions in GS model size. Reliable objective quality assessment is therefore essential for comparing compression methods and guiding the development of more efficient GS codecs. Existing GS quality assessment typically relies on image and video quality metrics, requiring rendering of predefined viewpoints and making the quality estimate dependent on the selected views. This paper introduces GS-PQM, a novel full-reference quality metric for post-training GS compression that operates directly in the GS parameter domain. GS-PQM estimates perceptual quality from a set of parameter-domain distortion errors using a Support Vector Regression model. Experimental results show that GS-PQM outperforms 25 existing image, video, and point-cloud quality metrics in assessing compressed GS content, providing an accurate and computationally efficient alternative to rendering-based quality assessment.

0.34

Explicit Regional Representations For Video Quality Assessment

cs.CV ·2026-10-27 ·Fuyi Yang, Lei Yang, Andrew Liu, Chenyu Wei, Xiaoyi Guo +1 more
similarity 0.59
Abstract · Preprint. This work is an initial version and may be updated in future revisions

End-to-end video quality assessment (VQA) methods leave the regional evidence underlying their predictions largely implicit. We introduce an explicit pixel-level regional representation for VQA, instantiated as a Quality-Relevance Heatmap (QRH) that combines localized degradation evidence with perceptual priors under vision-language model guidance, without requiring pixel-level annotations. We apply QRH to existing VQA models through two deterministic fragment-sampling strategies: distribution-preserving sampling approximates the original sampler's QRH-score distribution without retraining, while quality-aware sampling prioritizes regions with high QRH scores and is paired with fine-tuning. Experiments with DOVER on multiple benchmarks show that the former closely approximates predictions averaged over random samples, while the latter improves most evaluated metrics across popular VQA benchmarks. These results support explicit regional representations as a useful interface for downstream VQA.

0.15

High-Fidelity Video Quality Assessment with VQA-Specific Saliency

cs.CV ·2026-09-15 ·Hakan Emre Gedik, Shashank Gupta, Alan Bovik
published similarity 0.59
Abstract · Accepted to WACV 2027

No-reference video quality assessment (NR VQA) has recently seen promising progress with deep learning. However, video data is inherently large, and processing them with deep models incurs high computational cost. This challenge is particularly acute in VQA, where preserving original-resolution cues and dense temporal information is critical for accuracy. Existing efficiency-driven preprocessing strategies, such as fragmenting, reduce computation but alter the input data distribution, limiting effective reuse of pretrained video foundation models (ViFMs). To address these challenges, we propose \textbf{H}igh-\textbf{F}idelity \textbf{V}ideo \textbf{Q}uality \textbf{A}ssessment (\textbf{HFVQA}), a framework built on fixed-size spatio-temporal (ST) patches that is fully compatible with pretrained ViFMs. HFVQA samples ST patches across multiple scales, including the original resolution, with minimal temporal subsampling to preserve low-level quality cues and semantic context. To limit computation, HFVQA introduces a lightweight auxiliary network trained end-to-end with the ViFM encoder to learn \textit{VQA-specific saliency}. Distilled directly from quality supervision, this saliency captures task-specific importance patterns, reflecting that video quality perception is dominated by a small subset of spatio-temporal regions. By combining high-fidelity spatio-temporal cues with learned, task-specific saliency, HFVQA achieves SOTA performance on standard NR VQA benchmarks while processing as little as 12\% of candidate ST patches, making high-fidelity ViFM-based VQA computationally tractable.

0.34

SemABR: Measuring Video Semantic Fidelity with Multimodal LLMs for Adaptive Bitrate Streaming

cs.MM cs.NI ·2026-09-16 ·Shiqi Xu, Soung Chang Liew, Yuyang Du
similarity 0.59
Abstract

Conventional video metrics such as PSNR, SSIM, and VMAF measure visual distortion or perceptual quality, but they do not directly capture semantic preservation: whether compression retains a video's objects, actions, and temporal narrative. Existing Quality-of-Experience (QoE)-driven bitrate-selection and resource-allocation methods primarily aim to minimize rebuffering and bitrate switching while maximizing perceptual video quality, without explicitly considering semantic preservation. To address this gap, we introduce video semantic fidelity (SF), a metric that quantifies how well a compressed video preserves the semantic content of its source. An offline multimodal large language model (MLLM) generates structured descriptions of the reference and compressed versions of the video, and a separate text-only large language model (LLM) evaluates their semantic correspondence. The resulting content-dependent SF--bitrate profiles are cached and queried by the online bitrate selector without invoking MLLMs at runtime. Evaluations on three subjective QoE benchmarks show a consistent positive association between SF and mean opinion scores (MOS). A separate human semantic-rating study evaluates semantic preservation and shows that SF correlates more strongly with human judgments than conventional video metrics. We then embed these profiles into a 5G MEC-assisted video-on-demand (VoD) resource-allocation framework at the base station. When wireless resources cannot support high bitrate levels for all users, the framework uses the SF--bitrate profiles to select bitrate levels jointly across users and reduce the semantic loss caused by the required bitrate reductions. NS-3 simulations with the 5G NR module show that the proposed framework achieves higher average and worst-user SF than the evaluated baselines, with a widening advantage as the wireless resources available to each user decrease.

0.52