CS Paper Radar

Characterizing Age-Conditioned Affective Reasoning in Vision-Language Models

Shahriyar Zaman Ridoy, Azmine Toushik Wasi, Lin Gu, Ruogu Fang

cs.CV ·submitted 2026-11-19 06:39 UTC · v1
code/data

Abstract

An age prompt can change how a vision-language model (VLM) rates the emotion evoked by an image. Does the change predict how people of different ages rate that same image? We evaluate 17 VLMs on 900 OASIS images using young-, middle-, and older-adult prompts and estimate image-specific age differences from ratings by 822 participants. Without an age cue, models closely track average human valence (median Pearson $r=0.891$). Under an older-adult prompt, 16 of 17 models assign lower valence on average than under a young-adult prompt. Yet after adjusting for image pleasantness, correlations between model and human age effects range from $-0.030$ to $0.093$, against an estimated ceiling of $0.595$. The discrepancy is clearest on 54 sensitive images rated as pleasant. Across models, the older prompt reduces valence by $0.845$ points on average, while older participants rate these images $0.443$ points higher than younger participants. Providing human ratings from other images improves agreement on the direction of age differences, although image-specific fidelity remains modest on 600 held-out images ($r_{\mathrm{part}}=0.105$). These results show why evaluations of demographic personalization in foundation models need human comparisons at the level of individual stimuli. Project page available at https://shahriyar-zaman.github.io/age-conditioned-emotion-reasoning/

Comment
Preprint. Under review

Closest papers

Nearest neighbours by meaning across the whole index.

Organization of Valence and Arousal in Vision-Language Representations of Built Environments: Insights from the EMOIS Dataset

cs.CV ·2026-09-06 ·Madoka Yonekura, Katsunori Kohda, Nobuhiko Muramoto, Takahiro Yamaguchi
benchmark similarity 0.65
Abstract

Visual perception of built environments contributes to the affective impressions that people form in everyday life. However, how these impressions are represented within vision foundation models remains largely unexplored. To support the systematic investigation of this subject, we introduce the Emotional Impression of Spaces (EMOIS) dataset, comprising 1,544 real-world built-environment images. Each image is annotated with image-evoked valence and arousal ratings collected from Japanese adults by conducting a large-scale web-based survey, with approximately 120 ratings per image. Using Contrastive Language--Image Pre-training (CLIP) representations, we perform predictive and geometric analyses to systematically investigate how valence and arousal are encoded and organized within the representation space. These analyses reveal that valence exhibited stronger and more coherent organization than arousal. Cross-dataset analyses with the Open Affective Standardized Image Set (OASIS), a benchmark dataset of general affective photographs, reveal differences in affective organization between the two datasets. Regression analyses demonstrate high predictive performance for valence and arousal within EMOIS, with mean coefficients of determination of 0.865 and 0.807, respectively, across repeated internal hold-out evaluations. Finally, we present an example-based interface illustrating how learned representations can support qualitative interpretation of predicted affective values. These findings can help elucidate affective representations of built environments and establish EMOIS as a densely annotated resource for future affective computing research in this domain.

0.35

Parsed Text Understates Vision-Language Models: From Below to Above Expert Agreement on Fine-Grained Facial Emotion

cs.CV cs.CL ·2026-10-06 ·Tobias Hallmen, Fabian Deuser, Robin-Nico Kampa, Norbert Oswald, Elisabeth André
benchmark similarity 0.64
Abstract · Preprint. 10 pages, 2 figures; 10-page supplementary material as ancillary file. This work has been submitted to the IEEE for possible publi…

EmoNet-Face-HQ is a benchmark of 2,500 generated portraits, each rated by four psychology experts on 40 emotion categories. It concludes that vision-language models (VLMs) on average offer no dependable improvement over random guessing and that a fine-tuned model is required. The benchmark reads each model's answer from generated text, and its zero-shot prompt demands a fixed number of emotions per face. Its own results already show that this protocol can decide the outcome: one VLM moves from chance to close to expert agreement when only the prompt changes. We keep the images, expert ratings and metric and change only how the answer is asked for and read: one yes/no question per category, scored by the probability of "yes". Scored from parsed text on the five categories experts agree on most, eleven open-weight VLMs average a quadratic-weighted $κ$ of 0.425 against individual experts, below the 0.468 at which two experts agree. Scored from yes-probabilities, the same models average 0.553, and all eleven agree with an individual expert better than a second expert does. The VLMs' mean is level with the benchmark's fine-tuned model (0.555), and three of them significantly exceed it without any training. Only one exceeds a stricter reference, the mean of three experts scored against the fourth (0.563). Verdicts that VLMs fail at graded facial affect ratings may therefore describe the readout more than the model. A fine-tuned model should have to beat a pretrained VLM read this way before it is called necessary, and the benchmark's own does not clearly do so.

0.19

Frontier vision-language models have overtaken young adults at detecting AI-generated portraits -- but not their calibration

cs.HC cs.CV ·2026-08-31 ·Sunwhi Kim, Sunyul Kim, Meounggun Jo, Jini Tae
similarity 0.64
Abstract · 35 pages, 8 figures, 5 supplementary figures. Human reference data reused (not newly collected) from arXiv:2603.24048. Data and code: https:…

AI image generators now create face portraits that are hard to tell from real photographs. Vision-language models (VLMs) are increasingly proposed to flag such images. We benchmarked 19 VLMs on the same 198 face portraits -- real photographs and identity-matched ChatGPT-4o and Imagen 3 versions -- under the same task as our earlier study of 1,667 adults (85% correct overall; accuracy fell steeply with age). The June-2026 cohort of 14 models only matched adults in their 20s-30s. Four weeks later the ceiling broke. Among five July-2026 releases under the identical protocol, gpt-5.6-sol reached 92.8% balanced accuracy (five-draw mean 92.1%), clearly above adults in their 20s (88.5%), and claude-fable-5 detected every AI image while averaging 91.9%. Model sensitivity now exceeds young adults decisively (d' up to 3.4 versus ~ 2.4). What has not been overtaken is human calibration. Model criteria spread from c = -1.10 to +1.45 while humans sit near zero at every age; both new leaders are biased (+0.44, -0.97), and only a few mid-ranked models approach the human balance. Changing the labelled examples still flipped about one answer in four. The best machines now out-see young adults here, without matching the human balance between suspicion and trust.

0.35

Faces or Models? Where the Age Gap in Facial Expression Recognition Comes From

cs.CV ·2026-10-08 ·Jessica Wang
code/data similarity 0.63
Abstract

Facial expression recognition (FER) systems are increasingly used in elder care, yet their accuracy on older faces is rarely measured. This study evaluated five open-source FER models on the FACES database, which holds 2,052 posed photographs of 171 adults aged 19 to 80. The models cross three architectures (ResNet-18, POSTER++ and QCS) with two training datasets (RAF-DB and AffectNet). All models lost accuracy on older faces by 11.7 to 15.2 percentage points. Older faces were harder for people too: human raters lost 8.6 points, a classifier using only facial geometry lost 8.3, and for anger, fear, and happiness older adults differed more from one another when posing. On specific expressions the models fell much further than people did. AffectNet-trained models misread older neutral faces as sad or angry 62% of the time. RAF-DB-trained models lost fear, disgust, and anger instead. The gap persisted after controlling for architecture, image quality, cropping, and expression intensity. When only the final classification layer was refitted on older faces, the gap shrank from 11 to 21 points to 4 points or less, suggesting that the models' learned features can represent older faces and that the deficit lies in how the final layer was trained.

0.29

EmoGenie: Benchmarking and Evaluating Expressed and Evoked Emotion in T2I Generation with MLLMs

cs.CV ·2026-10-13 ·Zixuan Xing, Lancheng Gao, Shengyan Li, Huiyu Duan, Zhaolin Cai +2 more
benchmark similarity 0.60
Abstract · Comments: Corresponding author: Xiongkuo Min. Equal contribution: Zixuan Xing and Lancheng Gao

Rapid advances in artificial intelligence-generated content (AIGC) have significantly promoted using text-to-image (T2I) models to generate images with specific affective intentions. However, the emotional effects conveyed by generated images frequently deviate from target expectations, and it remains unclear whether multimodal large language models (MLLMs) can reliably interpret the emotion of AI-generated visual content. Existing affective benchmarks are predominantly built on natural images, while affective studies of generated images have largely focused on emotions expressed in portraits, lacking a unified benchmark for studying the emotions expressed and evoked by generated images. We therefore introduce EmoGenie, a benchmark of emotion for AI-generated images that evaluates capabilities in T2I generation and MLLM understanding across expressed and evoked emotions. It comprises 6,400 images generated by eight T2I models using 800 emotion-targeted prompts, annotated with question-answer (QA) pairs, termed EmoGenie-QA, and continuous human ratings, termed EmoGenie-Assess. EmoGenie-QA contains 90,000 closed-set questions in single-image and image-pair settings, as well as 1,057 open-ended questions for emotion description. EmoGenie-Assess benchmarks continuous affective and aesthetic prediction using 13,175 mean opinion scores (MOSs) aggregated from human ratings in terms of valence, arousal, dominance, and aesthetic quality. We further develop EmoProbe, which combines emotion question answering with response-aligned, dimension-specific regression heads for continuous score prediction. Experimental results show that EmoProbe achieves superior results on emotion-related QA tasks, while its affective and aesthetic predictions correlate better with human ratings, demonstrating its state-of-the-art capabilities of unified coarse-to-fine affective understanding for AI-generated images.

0.11

Do Emotion Concepts Generalize Across Sources, Modalities, and Architectures in Vision-Language Models?

cs.CV cs.LG ·2026-09-28 ·Bohao Xing, Xin Liu, Kaishen Yuan, Deng Li, Rong Gao +3 more
similarity 0.59
Abstract

Recent studies suggest that large language models encode emotion concepts as structured internal representations, but most existing work focuses on text and a single architecture. Therefore, we ask, do emotion concepts generalize across sources, modalities, and architectures in vision--language models (VLMs)? To address this, we construct CMES (Cross-Modal Emotion Stimuli), a multi-source collection of emotion-conditioned stories, real facial expressions, synthetic portraits, and synthetic emotion-evoking scenes. For each stimulus source, we extract a separate set of six Ekman emotion vectors from each of three VLMs. We report four main findings as follows: 1) Image-derived emotion vectors form a low-dimensional geometry similar to that of text-derived vectors. Valence is relatively stable across sources, while arousal varies more. 2) Text- and image-derived emotion vectors have modest cosine similarity but still show held-out cross-modal correspondence. Text-derived vectors can also steer image interpretation. 3) Cross-architecture correspondence remains even when native cosine is near zero. Transformations estimated from generic ImageNet activations recover both correspondence and causal transfer without using the six emotion vectors or their labels. 4) After aligning representations across architectures, we construct a shared emotion subspace that preserves affective geometry and selective steering effects. The corresponding consensus emotion vectors also generalize to a held-out fourth architecture at two model sizes. These results suggest that emotion representations can share relational structure and causal effects across sources, modalities, and architectures, even when individual vector directions differ.

0.19

Do Vision-Language Models Agree on the Affective Qualities of Shape? A Cross-Model Audit for Generative Design Interfaces

cs.HC cs.CV ·2026-08-26 ·Luca Bux, Thiago Rios, Ingo Scholtes, Stefan Menzel, Bernhard Sendhoff
similarity 0.59
Abstract · 13 pages, 5 figures, 7 tables

Generative design interfaces increasingly expose semantic controls that let users steer output with concepts such as "more elegant" or "more minimalist," typically encoded by a vision-language model (VLM). A practical question is whether state-of-the-art VLMs represent objects consistently in terms of the same concept. We audit 6 VLMs by ranking untextured 3D objects along Kansei adjective pairs, where Kansei describes affective impressions of product form, with each axis defined as the difference between the text representations of its two poles. Geometric pairs serve as positive controls, and pairs of unrelated adjectives establish an empirical null. Across 10 categories of ShapeNet database, affective axes converge above the null (mean pairwise rank correlation 0.36 vs. 0.14) but below the geometric ceiling (0.44). The agreement between models is partial and highly uneven: on the three axes shared by all categories, mean convergence ranges from 0.21 for bookshelves to 0.51 for jars. Convergence depends primarily on whether a category's representational variation aligns with the semantic direction being evaluated, rather than simply on how much the objects vary in shape overall. Cross-model convergence does not imply agreement with human judgments. Based on our findings, we implement a UI prototype that shows how the audit can inform which Kansei descriptors to expose as controls for a given object class and which to withhold.

0.25

Undertone: Personalizing Frozen VLM Aesthetic Judges from Few Ratings

cs.CV cs.CLcs.LG ·2026-09-22 ·Amit Jadhav, Shaurya Beriwala, Beomjin Kim
benchmark similarity 0.57
Abstract · v2: substantially revised and retitled; supersedes v1 ("Feed the Panel Dimensions, Not Verdicts: Rubric-Decomposed Fusion of Vision-Language…

Image quality assessment predicts a mean opinion score (MOS), but for image aesthetics the mean hides the individual, so personalized assessment must start from the few images a new user rates. Fitting each person from scratch fails with ten ratings; networks fine-tuned per person carry over poorly to new kinds of images. We present Undertone-P, which learns from other raters a Gaussian prior over per-person regression weights on read-outs of frozen vision-language model (VLM) judges: the probability-weighted rating and the hidden state at the score token. Since people differ from the crowd in ways other raters reveal, a few ratings need only choose among those ways: a new user costs one closed-form solve, and no network is trained. On Ryu and Yanaka's PARA benchmark, the prior alone reaches a Spearman correlation of .640, above the best published 10- and 100-rating results (.581 and .620); 10 and 100 of a person's ratings raise it to .648 and .676. These ratings add most where tastes depart from the crowd: with 100 ratings, +.067 for the raters who agree least with it and +.206 on LAPIS artworks, against +.037 overall. Without labels, the judges' expected scores predict PARA's MOS at .835 (best published model we ran: .694). PARA-trained networks still lead under PARA's original protocol, likely through a stronger end-to-end MOS model.

0.36