CS Paper Radar

Faces or Models? Where the Age Gap in Facial Expression Recognition Comes From

Jessica Wang

cs.CV ·submitted 2026-10-08 23:07 UTC · v1
code/data

Abstract

Facial expression recognition (FER) systems are increasingly used in elder care, yet their accuracy on older faces is rarely measured. This study evaluated five open-source FER models on the FACES database, which holds 2,052 posed photographs of 171 adults aged 19 to 80. The models cross three architectures (ResNet-18, POSTER++ and QCS) with two training datasets (RAF-DB and AffectNet). All models lost accuracy on older faces by 11.7 to 15.2 percentage points. Older faces were harder for people too: human raters lost 8.6 points, a classifier using only facial geometry lost 8.3, and for anger, fear, and happiness older adults differed more from one another when posing. On specific expressions the models fell much further than people did. AffectNet-trained models misread older neutral faces as sad or angry 62% of the time. RAF-DB-trained models lost fear, disgust, and anger instead. The gap persisted after controlling for architecture, image quality, cropping, and expression intensity. When only the final classification layer was refitted on older faces, the gap shrank from 11 to 21 points to 4 points or less, suggesting that the models' learned features can represent older faces and that the deficit lies in how the final layer was trained.

Closest papers

Nearest neighbours by meaning across the whole index.

Whose Face Is It Anyway? A Multi-Model Audit of Facial Affect Recognition on Children, and Why the Gap Is the Head, Not the Features

cs.CV ·2026-10-06 ·Tobias Hallmen, Robin-Nico Kampa, Elisabeth André
similarity 0.70
Abstract · Preprint. 10 pages, 5 figures

Facial affect models are trained almost entirely on adults, yet are increasingly applied to children in education, health, and developmental research. We present a controlled, multi-model audit of five AffectNet-pretrained expression models (EmoNet, EmotiEffLib, DDAMFN++, OpenFace 3.0, LibreFace) on children, across four child image datasets, the AffectNet-8 validation set, and two spontaneous child video datasets, through one shared harness. Three findings emerge. First, the child gap is model-agnostic: every architecture degrades from posed to naturalistic faces and shares the fear$\rightarrow$surprise confusion. Second, it is concentrated and corroborated across all five models: open-mouth faces (read as surprise, correlating with the AU26 jaw drop) and South-Asian children degrade systematically, with a smaller averted-gaze penalty, while closed-mouth faces, White and Black children, and direct gaze do not; the bias tracks expression morphology and specific populations, not skin tone. Third, the gap is diagnosable: a linear probe on frozen features reaches 0.75-0.91 on unseen children versus 0.48-0.66 zero-shot, so it lies largely in the classifier head, not the representation, whereas dimensional valence/arousal regression degrades sharply under domain shift. Building on this, recalibrating only the head on a little target data recovers $+0.13$ to $+0.28$ on the two largest child sets across all five models at negligible adult cost, though the gain is in-distribution and does not transfer across child collections. We will release the harness, per-sample predictions, and analysis code; the child face data stays license-locked and is never redistributed.

0.39

Characterizing Age-Conditioned Affective Reasoning in Vision-Language Models

cs.CV ·2026-11-19 ·Shahriyar Zaman Ridoy, Azmine Toushik Wasi, Lin Gu, Ruogu Fang
code/data similarity 0.63
Abstract · Preprint. Under review

An age prompt can change how a vision-language model (VLM) rates the emotion evoked by an image. Does the change predict how people of different ages rate that same image? We evaluate 17 VLMs on 900 OASIS images using young-, middle-, and older-adult prompts and estimate image-specific age differences from ratings by 822 participants. Without an age cue, models closely track average human valence (median Pearson $r=0.891$). Under an older-adult prompt, 16 of 17 models assign lower valence on average than under a young-adult prompt. Yet after adjusting for image pleasantness, correlations between model and human age effects range from $-0.030$ to $0.093$, against an estimated ceiling of $0.595$. The discrepancy is clearest on 54 sensitive images rated as pleasant. Across models, the older prompt reduces valence by $0.845$ points on average, while older participants rate these images $0.443$ points higher than younger participants. Providing human ratings from other images improves agreement on the direction of age differences, although image-specific fidelity remains modest on 600 held-out images ($r_{\mathrm{part}}=0.105$). These results show why evaluations of demographic personalization in foundation models need human comparisons at the level of individual stimuli. Project page available at https://shahriyar-zaman.github.io/age-conditioned-emotion-reasoning/

0.38

Event-Level Emotion Recognition in the Wild Using Deep Facial Expression Analysis

cs.CV ·2026-09-12 ·Aleksandr Semerikov, Pakizar Shamoi
published similarity 0.59
Abstract · The paper has been submitted to IEEE conference

Facial emotion recognition (FER) in real-world environments remains challenging due to unconstrained imaging conditions, including multiple faces, occlusions, pose variations, and complex lighting. Most existing studies focus on individual facial emotion classification and do not address the analysis of collective emotional states at the event level. This paper proposes an end-to-end pipeline for event-level emotion recognition from photographs. The approach detects faces in each image, classifies facial expressions using a deep convolutional neural network, and aggregates face-level emotion probabilities to estimate the overall emotional distribution of a public event. A comparative evaluation of several CNN architectures on the FER- 2013 and RAF-DB datasets demonstrates that transfer learning with EfficientNet-B2 trained on RAF-DB is more suitable for real-world RGB data. The proposed method is evaluated on a real-world event dataset containing 1658 images. Experimental results show stable emotion distributions across event subsets, confirming the effectiveness of event-level aggregation for emotion analysis in the wild.

0.45

Input-Space versus Feature-Level Conditioning for Robust Facial Expression Recognition

cs.CV ·2026-11-15 ·Weihang Yuan
similarity 0.58
Abstract · 8 pages, 2 figures, 4 tables

Facial expression recognition (FER) degrades sharply when illumination, exposure or resolution departs from the training distribution. Two remedies dominate practice: enriching the input signal, or conditioning intermediate features on an estimated image quality. The two are rarely compared under matched backbones, training budgets and condition annotations, and their cost is seldom reported. We present a controlled comparison on RAF-DB. Input-space conditioning uses four deterministic, label-free views (logarithmic stretching, inverse-logarithmic stretching, down-up resampling and identity), each processed by a shared backbone with a branch-specific BatchNorm and linear head, with the branch logits averaged. Feature-space conditioning is implemented as FiLM and Conditional BatchNorm under an identical protocol. Across six backbone architectures the input-space variant improves top-1 accuracy on every one (mean +0.94 percentage points; bootstrap interval [0.51, 1.63]; sign test p = 0.016), with a seed-controlled estimate of +0.42 percentage points on ResNet-18, whereas feature-level conditioning yields +0.10 and +0.03 percentage points. We attribute the gap to the information content of the heuristic condition label rather than to the conditioning mechanism. The gain costs 4.00x FLOPs, 4.18x latency and 3.79x training time, i.e. 0.26 percentage points per additional GFLOP, and we state an explicit rule for when the trade is worth making.

0.22

STAR-FER: Adaptive Selective Restoration for Edge-Based Low-Resolution Facial Expression Recognition

cs.CV ·2026-10-30 ·ZhongYang Wu, ZhenFeng Gan, Di Zhao, XiaoJun Zhu, HouDe Liu
similarity 0.57
Abstract

Variable interaction distances continuously degrade facial resolution in edge-based affective interaction, making low-resolution facial expression recognition (FER) difficult under limited computation. Full-face image or feature restoration incurs substantial overhead, whereas lightweight recognition pipelines may miss local cues that remain discriminative after degradation. We propose STAR-FER, a scale-and-task-adaptive selective restoration framework that balances recognition performance and resource constraints. A lightweight image-scale conditioned Gate jointly predicts a pixel-wise spatial ranking and a continuous restoration budget, then applies exact-support projection to obtain an input-dependent free-form mask. The mask-conditioned restorer retains complete facial context while concentrating sparse restoration and residual insertion on the selected support. Across seven degradation scales from x2 to x8 on the complete RAF-DB test set, STAR-FER achieves 80.92% mean accuracy and 71.64% Macro-F1, improving the low-resolution ResNet-18 baseline by 2.25 and 2.71 percentage points, respectively. It surpasses SR-FER by 1.04 accuracy points while requiring 8.84 GFLOPs, a 54.29% reduction, with 28.01% mean mask coverage, and maintains effective recognition at unseen non-integer scales. These results show that spatial-ranking and continuous-budget driven selective restoration reduces spatial modification and computation while maintaining low-resolution FER performance, offering an efficient path to resource-constrained affective interaction.

0.26

Parsed Text Understates Vision-Language Models: From Below to Above Expert Agreement on Fine-Grained Facial Emotion

cs.CV cs.CL ·2026-10-06 ·Tobias Hallmen, Fabian Deuser, Robin-Nico Kampa, Norbert Oswald, Elisabeth André
benchmark similarity 0.54
Abstract · Preprint. 10 pages, 2 figures; 10-page supplementary material as ancillary file. This work has been submitted to the IEEE for possible publi…

EmoNet-Face-HQ is a benchmark of 2,500 generated portraits, each rated by four psychology experts on 40 emotion categories. It concludes that vision-language models (VLMs) on average offer no dependable improvement over random guessing and that a fine-tuned model is required. The benchmark reads each model's answer from generated text, and its zero-shot prompt demands a fixed number of emotions per face. Its own results already show that this protocol can decide the outcome: one VLM moves from chance to close to expert agreement when only the prompt changes. We keep the images, expert ratings and metric and change only how the answer is asked for and read: one yes/no question per category, scored by the probability of "yes". Scored from parsed text on the five categories experts agree on most, eleven open-weight VLMs average a quadratic-weighted $κ$ of 0.425 against individual experts, below the 0.468 at which two experts agree. Scored from yes-probabilities, the same models average 0.553, and all eleven agree with an individual expert better than a second expert does. The VLMs' mean is level with the benchmark's fine-tuned model (0.555), and three of them significantly exceed it without any training. Only one exceeds a stricter reference, the mean of three experts scored against the fourth (0.563). Verdicts that VLMs fail at graded facial affect ratings may therefore describe the readout more than the model. A fine-tuned model should have to beat a pretrained VLM read this way before it is called necessary, and the benchmark's own does not clearly do so.

0.19

The shape of individual differences in judgment: shared axes, personal weights, and when their spread crosses the consensus

cs.CV q-bio.NC ·2026-11-17 ·Hiroshi Higashi
similarity 0.54
Abstract

People differ in how they judge faces, artworks, photographs and jokes. Previous work measured the size of these differences; we ask what shape they take. We wrote three candidate shapes as nested generative models (same-sign reweighting of shared dimensions, a linear transform that allows reversal, and stimulus-specific offsets) and compared how well they predict people unseen in training, using ten public datasets (about 20,000 raters). The models recovered a person's judgment space from 10-20 judgments and predicted judgments of unseen stimuli. For faces, reweighting was sufficient, and weights opposing the population were not statistically detected. For artworks, weights spread continuously along the shared abstract-figurative direction, and 28 percent of raters fell on the opposite side from the population. This tail was much smaller when a different rater sample judged the same artworks. Individual differences thus differ in shape as well as size, and the shape depends on domain and raters.

0.23

Do Emotion Concepts Generalize Across Sources, Modalities, and Architectures in Vision-Language Models?

cs.CV cs.LG ·2026-09-28 ·Bohao Xing, Xin Liu, Kaishen Yuan, Deng Li, Rong Gao +3 more
similarity 0.54
Abstract

Recent studies suggest that large language models encode emotion concepts as structured internal representations, but most existing work focuses on text and a single architecture. Therefore, we ask, do emotion concepts generalize across sources, modalities, and architectures in vision--language models (VLMs)? To address this, we construct CMES (Cross-Modal Emotion Stimuli), a multi-source collection of emotion-conditioned stories, real facial expressions, synthetic portraits, and synthetic emotion-evoking scenes. For each stimulus source, we extract a separate set of six Ekman emotion vectors from each of three VLMs. We report four main findings as follows: 1) Image-derived emotion vectors form a low-dimensional geometry similar to that of text-derived vectors. Valence is relatively stable across sources, while arousal varies more. 2) Text- and image-derived emotion vectors have modest cosine similarity but still show held-out cross-modal correspondence. Text-derived vectors can also steer image interpretation. 3) Cross-architecture correspondence remains even when native cosine is near zero. Transformations estimated from generic ImageNet activations recover both correspondence and causal transfer without using the six emotion vectors or their labels. 4) After aligning representations across architectures, we construct a shared emotion subspace that preserves affective geometry and selective steering effects. The corresponding consensus emotion vectors also generalize to a held-out fourth architecture at two model sizes. These results suggest that emotion representations can share relational structure and causal effects across sources, modalities, and architectures, even when individual vector directions differ.

0.19