CS Paper Radar

Multidimensional Observer Model and Perceptual Dimensions of Human Image Quality Assessment

Sheng Zhao, Weikai Lin, Yuhao Zhu

cs.CV ·submitted 2026-09-29 20:15 UTC · v1
top venue

Abstract

Judging image quality is not only ecologically relevant to everyday human tasks, but also underpins many machine vision tasks such as image generation. This paper proposes a framework to understand the inherent perceptual space underlying image quality judgment in humans. We propose a multi-dimensional observer model that represents images as distributions in a latent perceptual space and that models human judgment as comparing noisy samples. Being constrained by neural representations in the primate ventral stream and fit to large-scale behavioral data, the model enables analysis of perceptual structure while matching the predictive power of existing metrics. Using this model, we find that the perceptual spaces needed to account for image quality judgment in humans are extremely low-dimensional compared to the image space even when considering its sparsity. The exact structure of the space (e.g., dimensionalities, information encoded) varies between low-level and high-level quality judgments, suggesting that, despite a shared retinal encoding in the beginning, humans selectively construct task-dependent perceptual spaces in visual decision making.

Comment
Accepted at NeurIPS 2026. 25 pages

Closest papers

Nearest neighbours by meaning across the whole index.

LLaVA-Assessor: Building the Foundation LMM For Visual Quality Assessment

cs.CV ·2026-08-11 ·Ziheng Jia, Zicheng Zhang, Jiaying Qian, Guangtao Zhai, Xiongkuo Min
code/data similarity 0.65
Abstract

Aligning with the human visual system~(HVS) in perceiving and evaluating the quality of visual signals is a central objective of machine-vision-based visual quality assessment systems. With the rapid progress of large multi-modal models~(LMMs), visual question answering provides a promising paradigm for building unified foundation models for visual quality assessment under multi-modal and multi-task scenarios. Inspired by the classical ``perception-decision" process in HVS-based quality evaluation, we formulate visual quality assessment for LMM-based machine vision as two complementary tasks: ``quality interpretation'' and ``quality scoring". Centered on these objectives, we propose LLaVA-Assessor, a unified data construction and model training system. To support multi-modal inputs, we design an adaptive model architecture that enables efficient processing of both images and videos. For data construction, we develop rigorous human annotation protocols and a novel machine-synthesis-dominated data expansion pipeline to build a large-scale and high-quality datasets. Furthermore, we introduce a simple yet effective prompt disentanglement strategy to alleviate training-objective confusion in multi-task learning, thereby enabling stable and coherent joint training. The resulting all-in-one LMM LLaVA-Assessor-GIGA achieves superior performance on $11$ image/video quality scoring test sets and 4 visual quality interpretation benchmarks. Extensive results demonstrate the effectiveness of integrating structured data construction, adaptive model design, and multi-task joint training for automated visual quality assessment. Our work provides compelling insights for developing foundation LMMs for automatic visual quality assessment. Project page at https://github.com/jzhws/LLaVA-Assessor.

0.24

Do Generative Priors Align with Human Naturalness Perception?

cs.CV q-bio.NC ·2026-10-07 ·Taiki Fukiage
similarity 0.64
Abstract · 62 pages, 35 figures, including appendices

Visual generative models are trained to capture the probability distributions of natural images, yet whether their native priors reflect the regularities governing human perception of image naturalness remains an open question. Here, we probe these priors through native prediction errors across 25 open image and video generators. Because raw single-image losses are dominated by scene content and visual complexity, we evaluate directional loss differences using content-preserving, paired relational interventions that selectively disrupt facial configurations or physical illumination consistency while limiting changes in low-level image statistics. Across both domains, these loss differences reproduce human-like selective sensitivities and tolerances, capturing the classic Thatcher effect on faces and shape-dependent responses to illumination inconsistencies. Notably, these loss differences reliably track continuous gradations of human naturalness judgments across individual stimulus pairs (peaking at $r = .84$ on faces and $.64$ on physical scenes) and retain unique human-aligned signals even after controlling for feature distances from frozen vision encoders and standard image quality metrics. We also find that while overall sensitivity to these violations broadly covaries with human alignment across models, the two systematically decouple along denoising schedules, with alignment peaking earlier than sensitivity, revealing that human-like naturalness judgments dissociate from generic violation detection. Together, these findings demonstrate that learning visual distributions yields generative loss landscapes that capture distinct aspects of human naturalness perception.

0.33

Psychometrics-Inspired Coreset Selection for Image Quality Assessment

cs.CV ·2026-11-18 ·Susu Jiang, Xuelin Shen, Baoliang Chen
code/data similarity 0.64
Abstract

Collecting large-scale subjective annotations remains a critical bottleneck in training accurate image quality assessment (IQA) models. Coreset selection reduces this burden by selecting representative images before annotation. However, diversity in a visual feature space does not necessarily reflect diversity in perceptual quality, raising a fundamental question about which quality-related factors the representation should encode. We propose GRAM, a quality-aware representation learning framework inspired by the graded response model (GRM) from psychometrics. GRM describes how an individual's latent ability determines the probabilities of reaching successive assessment grades, with response sensitivity and grade thresholds governing this relationship. In this paper, we find that this principle naturally aligns with perceptual quality assessment, where the impact of image degradation depends on image content and distortion characteristics. Building on this connection, GRAM models quality judgments through three complementary factors: latent quality, content-dependent response sensitivity, and distortion-dependent grade thresholds. These factors jointly determine the probabilities of attaining different quality grades and the resulting expected quality score. The learned parameters provide a structured quality-aware representation that complements general visual features for coreset selection. Experiments on nine datasets demonstrate improvements across different selection strategies, downstream IQA models, and annotation budgets. The code will be made publicly available.

0.35

FoMo: Forking Moment in Generative Trajectory as a Perceptual Distance

cs.CV ·2026-09-22 ·Jaihyun Lew, Mingi Jung, Minjun Park, Wooseok Song, Sungroh Yoon
similarity 0.63
Abstract

Reference-based image quality assessment (IQA) metrics aim to reflect how humans perceive the perceptual distance between a pair of images. To learn how the human visual system (HVS) operates, recent reference-based IQA metrics heavily rely on human-annotated data. Mean opinion score (MOS)-based pointwise scoring, which assigns a scalar quality value per image, is preferable for annotation but is prohibitively expensive to collect at scale and is known to be noisy due to inconsistent human judgments. As an alternative, two-alternative forced choice (2AFC) pairwise labels have gained popularity due to their reliability and efficiency, but they capture only relative comparisons between pairs. In this paper, we propose a fully automated data generation pipeline that generates pointwise perceptual distance labels between image pairs without any human annotation. Our approach exploits the generative dynamics of diffusion models as a perceptual distance proxy, where the coarse structure of an image is generated in the early timesteps and the fine details are generated in the later timesteps. Images that fork early in the generation process share only coarse structure and are perceptually far apart; images that fork late differ only in fine detail. We demonstrate that the diffusion trajectory aligns well with the human visual system, and use this forking moment, FoMo, as a reference-grounded distance label to supervise the training of a reference-based IQA metric. The pointwise labels, which support universal comparison between arbitrary image pairs, enable an information-rich training objective. Extensive experiments across diverse backbone architectures confirm the effectiveness of our generation pipeline, outperforming human-annotated datasets in multiple benchmarks.

0.16

Vision Models Predict Urban Scene Appraisal with Limited Neural Alignment

cs.CV ·2026-08-31 ·Kaizhen Tan, Yuantao Deng
similarity 0.63
Abstract

Pretrained vision embeddings are widely used to model how people appraise urban scenes, and they are validated almost entirely by how well they predict human ratings. Accurate prediction shows that an embedding contains the information needed to recover the ratings. It does not show that the embedding arranges scenes as the human visual system does, which is assumed when distances or dimensions in the embedding are read as perceptual. Using openly released EEG recorded while 63 adults viewed and rated street scenes of Berlin, we compared the neural representational geometry of the scenes with the geometry of pretrained vision models that differ in training objective and size, of simple image descriptors and of the ratings themselves, each relative to a noise ceiling given by the agreement between participants. No feature space reached more than about half of the noise ceiling, which corresponds to about a fifth of the reliable variance in the neural geometry. A descriptor of oriented edge energy reached the level of most pretrained models while capturing a different part of the neural geometry, and deeper layers corresponded to later neural responses. The same embeddings predicted held-out ratings well, up to r = 0.87, yet prediction accuracy and neural correspondence were not reliably related across models, and the best predictor was among the least aligned. Predicting how a street is appraised is therefore weak evidence that a model represents the street as the brain does.

0.25

An Evolutionary Agentic Approach for Open-ended Image Quality Perception

cs.CV cs.AI ·2026-09-19 ·Zhenchen Tang, Bo Peng, Zichuan Wang, Songlin Yang, Leilei Cao +2 more
similarity 0.63
Abstract

Generative models are rapidly expanding image quality assessment (IQA) beyond traditional fidelity factors to emerging dimensions such as physical plausibility and text-rendering correctness. However, existing IQA models rely on fixed definitions and heavy supervision, making them difficult to extend to open-ended perceptual dimensions. We identify holistic bias as an important limitation: when scoring an unseen dimension, models reuse generic quality priors, leading to scoring errors and rank inversion. To address this, we propose PACE (Perceptual Agentic Collaborative Evolution), a training-free multi-agent framework that formulates open-ended IQA as explicit protocol construction. Given a target dimension, PACE uses collaborative agents to construct an evaluation protocol composed of verifiable Visual Question Answering (VQA) probes, grounding evaluation in concrete visual evidence rather than holistic impressions. The resulting protocol is calibrated using only four human-annotated images per dimension, while a dual-track scoring mechanism aligns model perception with human scoring scales. Across traditional IQA, structural fidelity, context-aware aesthetics, and newly defined open-ended dimensions, PACE consistently improves its MLLM backbone, achieving competitive performance across diverse IQA settings, and reduces the Holistic Override Rate (HOR) from 44.4\% to 8.6\%.

0.30

Task-Utility IQA: Rethinking Image Quality Assessment in Embodied Tasks

cs.CV cs.AI ·2026-09-30 ·Jirong Zha, Yanwei Liao, Yemin Wang, Xiangmin Yi, Siqi Peng +3 more
position similarity 0.62
Abstract

Image quality assessment (IQA) underpins embodied imaging pipelines by judging whether visual quality satisfies downstream tasks, yet most existing methods learn task-agnostic scores aligned with generic human ratings on static benchmarks. This objective mismatches embodied and interactive settings, where image adequacy depends on task goals, context, and action requirements that shape an agent's decisions. We argue that IQA should shift from score regression to goal-conditioned judgment defined by embodied task utility. To this end, we propose an MLLM-based embodied IQA agent framework that reasons about task requirements, grounds quality evidence, and produces decision-facing assessments. A compact VisDrone evidence study shows that conventional perceptual metrics and generic MLLM-based IQA scores are weak proxies for downstream VLM task success, while a task-conditioned utility score better predicts whether visual input supports the target task. These findings support a task-utility-oriented reframing of IQA for embodied, explainable, and decision-aware visual perception.

0.30

Alignment of visual brain encoding models with individual perceptual similarity judgments

cs.CV q-bio.NC ·2026-10-15 ·Daniel Chong, Haomiao Chen, Keith W Jamison, Raja Marjieh, Ilia Sucholutsky +2 more
similarity 0.62
Abstract

Encoding models that predict brain activation responses to images have traditionally been trained to maximize accuracy when predicting observed brain data. However, their internal representations may not reflect individual human perception. In this work, we investigate whether we can fine-tune brain encoding models to better align their representations with individuals' similarity judgments. Using similarity judgments of 100 images from 8 participants in the Natural Scenes Dataset (NSD), we adapt a generic human similarity judgment model (DreamSim) separately for each participant to obtain personalized perceptual similarity models. These models are then used to obtain subject-specific similarity structure across images in their encoding model training sets. We fine-tune each participant's baseline encoder in two conditions, aligning its representations with either generic or personalized similarity judgments, with the same initialization and objective. We find that baseline brain encoders reflect within-subject perceptual similarity more strongly than measured fMRI responses and preserve intersubject correlations present in measured fMRI, but fail to reflect those of human perceptual similarity judgments. Fine-tuning with generic, population-level perceptual similarity information results in better alignment with human judgments than baseline encoders, but these models still fail to capture intersubject correlations of human judgments. However, fine-tuning with personalized perceptual similarity information results in models that are the most aligned with human perceptual similarity judgments and also reflect intersubject correlations of human judgments. Our framework provides an approach for bootstrapping human similarity judgment data into more individualized, human-aligned brain encoding models of vision.

0.19