CS Paper Radar

MCIQA-2K: A Multi-Dimensional Dataset and No-Reference Quality Assessment Benchmark for Colorized Images

Yunkai Zhuang, Qihang Yan, Zicheng Zhang, Guangtao Zhai

cs.CV ·submitted 2026-09-13 13:05 UTC · v1
benchmarkdatasetcode/data

Abstract

Image colorization is an inherently ill-posed task, since a single grayscale image may correspond to multiple plausible colorized results. Consequently, conventional full-reference image quality assessment (IQA) metrics fail to accurately reflect human perceptual preferences for colorized images. In this paper, we present MCIQA-2K, a large-scale multi-dimensional benchmark specifically designed for no-reference quality assessment of colorized images. We construct a dataset containing 2,000 colorized images generated by five representative colorization models, together with human annotations across three perceptual dimensions: color smearing, semantic color misalignment, and global naturalness. Building upon the proposed benchmark, we further introduce MCIQA, a dedicated multi-branch NR-IQA framework for colorized images. Extensive experiments demonstrate that MCIQA significantly outperforms existing full-reference and no-reference IQA methods on the proposed benchmark, while also exhibiting competitive generalization capability on several widely-used IQA datasets. The dataset and code are publicly available at https://github.com/ARBEZ-ZEBRA/MCIQA.

Comment
15 pages, 5 figures, 2 tables

Closest papers

Nearest neighbours by meaning across the whole index.

Unifying Image Quality Assessment Datasets: MOSAIQ-500K and MOSAIQ-Bench

eess.IV ·2026-09-17 ·Wenbo Yang, Zhongling Wang, Jialu Xu, Jinghan Zhou, Zhou Wang
benchmark similarity 0.74
Abstract · 6 pages, 5 figures. This work has been submitted to the IEEE for possible publication

Image quality assessment (IQA) datasets use different subjective protocols and rating scales, so their scores are not directly comparable. The lack of a common perceptual scale hinders multi-dataset training and precludes direct inter-dataset evaluation. We address this by conducting a new subjective experiment and using its ratings as perceptual anchors to fit monotonic mappings that place the existing scores of 23 IQA datasets on a common quality scale while preserving within-dataset rankings. The resulting dataset, MOSAIQ-500K, contains over 500,000 images and is, to our knowledge, the largest IQA dataset with perceptually aligned subjective scores. We also propose MOSAIQ-Bench, an inter-dataset benchmark, and use it to evaluate 31 IQA methods, revealing substantial gaps between intra- and inter-dataset performance, particularly on authentic distortions. The aligned scores offer a key advantage: replacing specialized multi-dataset training mechanisms with standard regression losses yields comparable intra-dataset accuracy and better inter-dataset performance. MOSAIQ thus provides a practical foundation for combining heterogeneous IQA datasets to train more generalizable IQA models. Code and dataset will be made public at ivc.uwaterloo.ca/projects/unifying_iqa_datasets/ upon acceptance.

0.26

FoMo: Forking Moment in Generative Trajectory as a Perceptual Distance

cs.CV ·2026-09-22 ·Jaihyun Lew, Mingi Jung, Minjun Park, Wooseok Song, Sungroh Yoon
similarity 0.67
Abstract

Reference-based image quality assessment (IQA) metrics aim to reflect how humans perceive the perceptual distance between a pair of images. To learn how the human visual system (HVS) operates, recent reference-based IQA metrics heavily rely on human-annotated data. Mean opinion score (MOS)-based pointwise scoring, which assigns a scalar quality value per image, is preferable for annotation but is prohibitively expensive to collect at scale and is known to be noisy due to inconsistent human judgments. As an alternative, two-alternative forced choice (2AFC) pairwise labels have gained popularity due to their reliability and efficiency, but they capture only relative comparisons between pairs. In this paper, we propose a fully automated data generation pipeline that generates pointwise perceptual distance labels between image pairs without any human annotation. Our approach exploits the generative dynamics of diffusion models as a perceptual distance proxy, where the coarse structure of an image is generated in the early timesteps and the fine details are generated in the later timesteps. Images that fork early in the generation process share only coarse structure and are perceptually far apart; images that fork late differ only in fine detail. We demonstrate that the diffusion trajectory aligns well with the human visual system, and use this forking moment, FoMo, as a reference-grounded distance label to supervise the training of a reference-based IQA metric. The pointwise labels, which support universal comparison between arbitrary image pairs, enable an information-rich training objective. Extensive experiments across diverse backbone architectures confirm the effectiveness of our generation pipeline, outperforming human-annotated datasets in multiple benchmarks.

0.16

VGIQA: Fine-Grained Image Quality Assessment with Visual Grounding Rewards

cs.CV ·2026-10-27 ·Weichang Wang, Liyu Chen, Shuyue Guan, Jie Hu, Fei Gao +2 more
datasetcode/data similarity 0.67
Abstract · 20 pages, 5 figures, 6 tables; main paper: 4 figures and 4 tables, appendix: 1 figure and 2 tables

Image quality assessment (IQA) aims to evaluate the perceptual visual quality of images. Recently, the rapid development of Multi-modal Large Language Models (MLLMs) has significantly advanced the IQA field. By integrating chain-of-thought (CoT) reasoning and reinforcement learning (RL), these models can generate more accurate and interpretable results while relying on less training data. However, existing MLLM-based methods are predominantly constrained to holistic, image-level evaluations and lack the capability to produce fine-grained, localized assessments, which substantially limits their practical applicability. To address this limitation, we propose VGIQA, a novel IQA model endowed with both local and global quality assessment capabilities. VGIQA is trained via a two-stage paradigm comprising a cold-start phase followed by reinforcement learning. Driven by carefully designed local quality assessment reward functions, our model exhibits exceptional image quality understanding on VGSR-2K, a new dataset we propose specifically for local quality evaluation, significantly outperforming existing open-source MLLMs such as Qwen2.5-VL-72B-Instruct and Qwen3-VL-32B-Instruct. Furthermore, the local quality evaluation task can effectively enhance the model's performance on the degradation perception task.

0.20

From Legacy to Contemporary: Scalable Color Modernization for Long Videos

cs.CV ·2026-10-19 ·Dongyun Kim, Jiwoo Choi, Jaeseong Lee, Seon Joo Kim
code/data similarity 0.67
Abstract · Preprint

Bringing legacy video back to contemporary screens does not necessarily make it look contemporary. Earlier imaging pipelines often leave color casts, muted chroma, weak con- trast, and tonal imbalance, while correcting these character- istics still relies heavily on expert colorists and scales poorly to long, multi-shot videos. We formulate long video color modernization, a task that updates the chromatic and tonal appearance of legacy color footage while preserving scene content and coherence across frames and shots. We introduce a scalable, training-free framework that jointly modernizes grids of temporally adjacent keyframes using a pretrained text-guided image editing model, producing coherent con- temporary references without paired supervision. Compact shot-specific color transforms then propagate these adjust- ments throughout each shot, preserving spatial structure and temporal stability. We conduct a multidimensional evaluation on diverse legacy videos spanning five decades. We combine a human evaluation of perceived modernness with quantita- tive measures of content fidelity, temporal stability, percep- tual quality, and efficiency, and introduce Source-Weighted Inter-shot Grade Consistency (SW-IGC) to assess grade co- herence across shots. On our evaluation set, our frame- work achieves the highest perceived modernness while re- maining competitive across complementary measures, open- ing a path toward the systematic study of long video color modernization. Code is available at https://github.com/jiwoomood/ Video_Color_Modernization.git

0.32

ScenePercept: Scenario-Conditioned Perceptual Assessment of Real-World User-Generated Content Images

cs.CV ·2026-10-13 ·Shuhan Xia, Yu Chen, Xiang Wang, Shuai Li, Peipie Li +1 more
benchmark similarity 0.66
Abstract · Revise some experimental results

The rapid development of mobile imaging and social platforms has led to explosive growth in user-generated content images, making their systematic and comprehensive perceptual assessment increasingly important. Existing image perception benchmarks are typically task-oriented, curating specific scenarios for quality or aesthetic assessment and applying a uniform perceptual criterion across diverse scenarios. This limits their ability to capture the scene diversity and perceptual complexity of real-world UGC images. To address these limitations, we introduce ScenePercept-Bench, a benchmark designed to reflect the distribution of real-world UGC images, comprising 23,100 images across 21 categories and 105 fine-grained scenes. To account for the influence of scene context on human perception, we design a hierarchical, scene-aware annotation protocol, covering visual quality, aesthetic perception, and scene semantics, with 15 attributes in total. Each image is annotated with an overall score, 15 attribute-level scores, and structured explanatory reasoning. Building on this benchmark, we further introduce Scenario-Conditioned Perceptual Alignment, a training framework that tackles the heterogeneity of perceptual criteria across scenarios. SCPA jointly optimizes absolute score prediction, preference ranking, and cross-scenario calibration, preserving the model general perceptual assessment ability while encouraging it to explore and adapt perceptual criteria across different scenarios. Extensive experiments demonstrate that SCPA achieves state-of-the-art performance in UGC image perception and generalizes effectively to established IQA and IAA benchmarks.

0.20

NightIQA: A Comprehensive Dataset and Benchmark for MLLM-Based Nighttime Image Quality Assessment

cs.CV ·2026-11-03 ·Yiding Tian, Jiaojiao Yi, Jiayuan Lu, Pufan Xu, Jinlin Wu +2 more
publishedbenchmarkdatasetcode/data similarity 0.66
Abstract · Accepted to WACV 2027. 25 pages including supplementary material

Nighttime images are inherently susceptible to complex, compound degradations, making accurate quality assessment exceptionally challenging. Score-based IQA methods condense quality into a single number, lacking the multi-dimensional, language-grounded reasoning required for nighttime evaluation. Multimodal Large Language Models (MLLMs) offer a more comprehensive alternative, yet remain constrained by a critical data gap: existing real-world IQA datasets are overwhelmingly daytime-centric and often built from outdated captures, while synthetic alternatives lack nighttime-specific degradation modeling, yielding artifacts far removed from authentic nighttime conditions. To bridge this gap, we present NightIQA, a dataset and benchmark suite purpose-built for MLLM-based nighttime IQA. It comprises three core components: NightIQA-Real, a real-world dataset of 1,769 high-resolution images that we captured ourselves with modern equipment, encompassing six canonical distortion types and structured multi-dimensional annotations; NightIQA-Bench, a benchmark of 1,078 visually grounded multiple-choice questions tailored to MLLM-based nighttime IQA, organized around a four-level evaluation hierarchy spanning Perception, Technical Assessment, Localization, and Reasoning; and NightIQA-Syn, an inherently scalable, scene-aware synthesis pipeline that jointly models global-level and instance-level degradations, with an initial release of 80k high-fidelity images alongside the full generation code. Fine-tuned on NightIQA with Supervised Fine-Tuning (SFT) and Group Relative Policy Optimization (GRPO), Qwen3-VL-8B outperforms open-source IQA methods and achieves higher overall accuracy than leading closed-source models, highlighting the value of domain-specific data and structured supervision.

0.49

Benchmarking color judgment in multimodal LLMs with CIE-computable ground truth

cs.CV ·2026-10-29 ·Shuwei Yue, Kaida Xiao, Zhihua Wang, Qiao Chen
benchmark similarity 0.65
Abstract

Multimodal large language models (MLLMs) are now used as zero-shot judges of image quality, but their color judgments are rarely tested with colorimetric ground truth. We present a five-task benchmark based on the CIE colorimetric framework: AWB pair judgment, skin-tone bias direction, CCT estimation, CIE Delta E 2000 pair ordering, and color naming under changing illuminants. We test eight MLLM endpoints, including four Qwen3-VL variants, GLM-4.5V, GPT-5.4, Claude-Opus-4.7, and Gemini-3.1-Pro. The models often solve coarse binary AWB judgments, but they remain weak on numeric CCT estimation and on some fine color-difference judgments. On ColorChecker-24 stimuli, the three closed-weight frontier models reach CCT MAE of 715-1056 K, compared with 1358-2133 K for the five open-weight models. A non-ColorChecker control shows that this advantage is partly template-dependent. The results suggest that current MLLMs have useful color words and coarse visual priors, but their color judgments are not reliably calibrated to CIE-computable color differences.

0.21

From Perception to Pixels: Human-Derived Visually Distinct Colors for Improved Dominant Color Extraction

cs.HC cs.CV ·2026-10-23 ·Uldana Shyndali, Nurdaulet Toleugaliyev, Kuat Zharykbayev, Zarina Abildasheva, Pakizar Shamoi
similarity 0.65
Abstract

Dominant color extraction is a fundamental task in computer vision, with applications in design, image retrieval, and content recommendation. However, conventional algorithms based on device-oriented color spaces (e.g., RGB) and fixed quantization often fail to capture how humans perceive color differences. In this study, we propose a human-perception-based framework for dominant color extraction, built on a novel Human Color Dictionary (HCD) derived from a perceptual experiment. Participants (n=110) were sequentially shown 120 colors and asked to identify those they perceived as unique. Based on their responses, we constructed a 120 by 120 co-occurrence matrix and applied a greedy selection algorithm to derive 64 perceptually distinct colors, forming the HCD. This dictionary was integrated into RGB color histogram and K-means color extraction algorithms by initializing centroids and mapping pixel values to their nearest perceptual color using the deltaE 00 distance. Evaluation on a dataset of 4,562 images paired with expert-designed palettes demonstrated that our method improves perceptual alignment, as measured by Earth Mover's Distance (EMD) and best-match distance. Our findings suggest no meaningful gender differences in the number of uniquely perceived colors, indicating that sex does not significantly influence color perception in this task. Another key finding is that highly saturated colors are perceived as the most distinguishable, whereas dark and deep colors tend to perceptually collapse and are less reliably differentiated. The results confirm that incorporating experimentally derived human color representations ensures the generated color clusters align more closely with human perception, thus improving the usability of color palette generation in practical applications.

0.16