CS Paper Radar

Input-Space versus Feature-Level Conditioning for Robust Facial Expression Recognition

Weihang Yuan

cs.CV ·submitted 2026-11-15 03:35 UTC · v1

Abstract

Facial expression recognition (FER) degrades sharply when illumination, exposure or resolution departs from the training distribution. Two remedies dominate practice: enriching the input signal, or conditioning intermediate features on an estimated image quality. The two are rarely compared under matched backbones, training budgets and condition annotations, and their cost is seldom reported. We present a controlled comparison on RAF-DB. Input-space conditioning uses four deterministic, label-free views (logarithmic stretching, inverse-logarithmic stretching, down-up resampling and identity), each processed by a shared backbone with a branch-specific BatchNorm and linear head, with the branch logits averaged. Feature-space conditioning is implemented as FiLM and Conditional BatchNorm under an identical protocol. Across six backbone architectures the input-space variant improves top-1 accuracy on every one (mean +0.94 percentage points; bootstrap interval [0.51, 1.63]; sign test p = 0.016), with a seed-controlled estimate of +0.42 percentage points on ResNet-18, whereas feature-level conditioning yields +0.10 and +0.03 percentage points. We attribute the gap to the information content of the heuristic condition label rather than to the conditioning mechanism. The gain costs 4.00x FLOPs, 4.18x latency and 3.79x training time, i.e. 0.26 percentage points per additional GFLOP, and we state an explicit rule for when the trade is worth making.

Comment
8 pages, 2 figures, 4 tables

Closest papers

Nearest neighbours by meaning across the whole index.

STAR-FER: Adaptive Selective Restoration for Edge-Based Low-Resolution Facial Expression Recognition

cs.CV ·2026-10-30 ·ZhongYang Wu, ZhenFeng Gan, Di Zhao, XiaoJun Zhu, HouDe Liu
similarity 0.70
Abstract

Variable interaction distances continuously degrade facial resolution in edge-based affective interaction, making low-resolution facial expression recognition (FER) difficult under limited computation. Full-face image or feature restoration incurs substantial overhead, whereas lightweight recognition pipelines may miss local cues that remain discriminative after degradation. We propose STAR-FER, a scale-and-task-adaptive selective restoration framework that balances recognition performance and resource constraints. A lightweight image-scale conditioned Gate jointly predicts a pixel-wise spatial ranking and a continuous restoration budget, then applies exact-support projection to obtain an input-dependent free-form mask. The mask-conditioned restorer retains complete facial context while concentrating sparse restoration and residual insertion on the selected support. Across seven degradation scales from x2 to x8 on the complete RAF-DB test set, STAR-FER achieves 80.92% mean accuracy and 71.64% Macro-F1, improving the low-resolution ResNet-18 baseline by 2.25 and 2.71 percentage points, respectively. It surpasses SR-FER by 1.04 accuracy points while requiring 8.84 GFLOPs, a 54.29% reduction, with 28.01% mean mask coverage, and maintains effective recognition at unseen non-integer scales. These results show that spatial-ranking and continuous-budget driven selective restoration reduces spatial modification and computation while maintaining low-resolution FER performance, offering an efficient path to resource-constrained affective interaction.

0.26

GM-GReFEL: Geometry-Guided Spatiotemporal State-Space Learning for Robust Dynamic Facial Expression Recognition

cs.CV cs.LG ·2026-10-22 ·Yudhistira Arditya Pratama, Yi-Zeng Hsieh
code/data similarity 0.63
Abstract

Recognizing facial expressions in unconstrained video suffers from two coupled problems: self-attention grows quadratically in cost with clip length, and crowdsourced labels are noisy. We present GM-GReFEL, a spatiotemporal state-space network with a geometry-aware decision stage. Its Modulated GroupMamba backbone replaces a spatiotemporal Vision Transformer. Six channel groups are scanned separately, four over the frame plane and two over time, so the cost grows linearly with sequence length ($\mathcal{O}(N)$); after each layer, a Channel Affinity Modulation gate mixes the groups again. On top of the backbone, a GReFEL reliability module turns the normalized entropy of each prediction into a mixing weight between the classifier output and a vote from trainable geometric anchors, so that facial geometry counts for more when the network is unsure. The model is initialized by static-to-dynamic weight transfer and trained with a Triple Loss. In five-fold cross-validation, the best fold scores 66.57\%/76.46\% UAR/WAR on DFEW, 0.3 points or less below the strongest compared method, and 45.66\%/62.27\% on MAFW, ahead of all compared methods; on FERV39k the mean over five splits, 42.30\%/52.66\%, is competitive. The five-fold means are 64.07\%/74.54\% on DFEW, on par with recent large pre-trained models, and 38.90\%/54.37\% on MAFW, below the best of them. A paired $t$-test on DFEW indicates a modest benefit of temporal modeling (+1.61 UAR points, one-sided $p=0.031$). Attention stays on plausible facial regions, and latency is close to a ViT-B/16 baseline (47.4 vs.\ 46.8 ms per clip). Code: https://github.com/ardiytama/GroupMamba-DFER.

0.34

A Privacy-Aware Deep Learning Benchmark for Facial Emotion Recognition

cs.CV cs.AI ·2026-11-06 ·Larissa Palhares, Maria Machado, Hudson Bruno, Raphael Ramos, Andrew Vale +3 more
benchmark similarity 0.62
Abstract · 7 pages, 2 figures

Facial Expression Recognition (FER) traditionally relies on raw RGB images, introducing privacy risks, sensitivity to illumination variations, and susceptibility to identity-confounding visual shortcuts. As an alternative, this work proposes a lightweight, privacy-aware pipeline using exclusively geometric representations-facial landmarks (478 points) and blendshapes (52 coefficients, ARKit standard)-eliminating persistent raw video storage. Six neural architectures trained from scratch on fused CREMA-D and RAVDESS datasets were benchmarked against an ImageNet-pretrained ResNet50 baseline under a 5-fold subject-independent cross-validation protocol. Geometric models consistently outperformed the pixel baseline on unseen subjects with 12-124 times fewer parameters, while demonstrating superior robustness on subtle emotional expressions such as sadness. Ablation studies reveal that blendshapes capture the primary discriminative signal, while landmarks provide essential complementary spatial topology. These results demonstrate that compact geometric representations preserve affective semantics, eliminate downstream transfer learning dependency on visual backbones, and enable efficient, privacy-aware emotion recognition.

0.32

Aggregation-Induced Calibration Failure in Sum-Based Label Expansion for Multi-Label Facial Expression Recognition: A Diagnostic Study

cs.CV ·2026-11-09 ·Weihang Yuan
code/data similarity 0.62
Abstract · 13 pages, 3 figures, 14 tables. Code and data: https://github.com/yuanweihang810/CALE

Label-space expansion - predicting a larger joint space and summing back to the original labels - is a natural design for multi-label facial expression recognition, but its failure modes have not been characterised. We evaluate Condition-Aware Label Expansion (CALE) on a self-built dataset (30,240 training images, 3,460 multi-label test images), on seven backbones, and on the public RAF-DB and RAF-ML benchmarks. The central finding is that the aggregation rule, not the representation, decides whether the method works. With a fixed threshold of 0.5, the aggregate score spans 28x across five backbones at 10% of the training data (0.24 to 6.79) and collapses to a 1.9x range at full data, while threshold-free mAP stays within 1.28x over the same backbones, which localises the fault to the summation step. The mechanism is the uncalibrated aggregation rule, which adds the main-branch score to half the sum of the three non-main branches; we verify it by measuring main- and non-main-branch activations. Two plain seven-dimensional sigmoid baselines (BCE, ASL) reach mAP 0.5055 and 0.5175; ASL exceeds three of the five CALE configurations and falls within 0.03 of the best. On single-label FER the gain over a seven-dimensional baseline is nominally significant (paired t-test, p = 0.0436) but does not survive Benjamini-Hochberg correction (p_BH = 0.2055) or a Wilcoxon test (p = 0.1250). We conclude that sum-based label expansion requires threshold calibration at the deployment data scale, or a learned aggregation, before it can support multi-label decisions.

0.40

A Comparative Transfer-Learning Study of CNN Backbones for Partial Face Recognition on the SoF Dataset

cs.CV cs.AI ·2026-09-16 ·Ahmed Kubba, Ali Alsalama, Abdelrahman Abdalla, Qassim Nasir, Manar Abu Talib
top venue similarity 0.61
Abstract · 6 pages, 1 figure, 3 tables, accepted at The 6th International Conference on Electrical, Computer, Communications and Mechatronics Engineeri…

Face recognition is widely deployed in surveillance, access control, and forensic workflows, yet accuracy degrades sharply once the face is occluded by accessories, foreground objects, or the frame edge. Because most faces in the wild are partial, robust partial face recognition (PFR) remains open. This paper compares three pretrained convolutional backbones, ResNet-50, VGG-16, and FaceNet, fine-tuned for PFR by transfer learning under identical preprocessing, splitting, and optimization protocols on the Specs-on-Faces (SoF) dataset. All three arms use a common 160x160 input and a frozen backbone with a trainable head under a fixed epoch budget and no per-backbone hyperparameter search. The FaceNet configuration, denoted PFN (Partial FaceNet), substantially outperforms the other two, reaching 97.4% test accuracy with macro-averaged 87.04% precision, 84.61% recall, and 84.17% F1 over the 112 identity classes, the highest accuracy and recall reported on SoF.

0.34

EXPL-FR: Explaining Face Recognition Models via Vision-Language Alignment

cs.CV ·2026-08-21 ·Guray Ozgur, Mustafa Efe Tamyapar, Naser Damer, Fadi Boutros
top venuebenchmark similarity 0.61
Abstract · Accepted at the ECCV 2026 Workshops

Deep face recognition (FR) models reach near-saturated accuracy but remain opaque: a practitioner cannot ask which semantic attributes a similarity score relied upon. EXPL-FR answers this inside the FR model's own embedding space. A lightweight adapter aligns a vision-language model's (VLM) image encoder with the frozen FR space, trained on face images alone and never on text. Because the VLM's encoders share one space, the same adapter applies to the text encoder, turning 978 attribute prompts in 22 categories, also extendable, into FR-space anchors at no extra cost. We do not assume this transfer works: a face-verification protocol measures it, and an ablation changing only the adapter isolates its contribution. Not every concept survives, because an FR model earns its invariances by discarding the factors it must verify identities across. A label-free detectability measure compares each concept's separability in FR space against the VLM space, and the 100 most detectable form the model's readable semantic signature, which separates identities better than the full vocabulary. We cover four FR backbones and two VLM encoders, EXPL-FR needs no architecture access, and supports identity-level, per-image, and differential explanations. We benchmark attribute-level auditing under three supervision settings, human labels (current practice), VLM pseudo-labels, and our fully prompt-driven audit, against real verification behavior. With no labels, the prompt-driven audit ranks four FR models by their measured per-ethnicity RFW errors and ranks controlled attribute changes by their true verification cost.

0.44

DIME: Scaling Facial Representation Learning via Differential Masked Autoencoding

cs.CV ·2026-10-12 ·Hao Yu, Haoyu Chen, Hui Wei, Yan Jiang, Nicu Sebe +1 more
similarity 0.61
Abstract

Scaling up data and model capacity has driven substantial progress in visual representation learning. However, conventional pretext tasks exhibit diminishing returns on fine-grained facial analysis. We attribute this to a scaling bottleneck: these tasks model each image's appearance and are dominated by identity, pose, and illumination, so subtle within-identity facial differences contribute little to the training signal. We introduce DIME, a differential masked autoencoder for large-scale facial representation learning. DIME builds on a simple observation: two images of the same person share identity-specific appearance, so their difference exposes facial changes. DIME reconstructs both masked images in an identity-matched pair and, through an Explicit Differential Direction Supervision (EDDS) objective, aligns the direction of their reconstructed difference with the true difference, while source isolation makes each reconstruction depend only on its own image. Shared appearance cancels in this difference, and because EDDS ignores magnitude, subtle facial changes count as much as large pose or illumination changes. DIME uses identity labels only to form pretraining pairs, requires no dense annotations, and learns dense facial representations with emergent part-level semantics. We pretrain DIME on 20 million high-resolution facial images consolidated from existing face datasets. DIME's AU detection performance continues to improve as pretraining data scales from 6M to 20M images, with its BP4D F1 increasing by 5.9 points, while MAE gains only 0.5 points on the same subsets. DIME also outperforms specialized facial methods on AU detection, AU intensity estimation, cross-domain anti-spoofing, landmark localization, head pose estimation, face parsing, and face generation.

0.21

Different Defects, Different Effects: A Controlled Audit of Face Recognition Training Data

cs.CV ·2026-10-20 ·Ana Dias, João Ribeiro Pinto, Hugo Proença, João C. Neves
published similarity 0.60
Abstract · Accepted at WACV 2027

The development of face recognition has been strongly supported by the availability of large-scale training datasets. Among them, MS-Celeb-1M has played an important role by providing millions of facial images from nearly 100K identities. However, the size and heterogeneity of such datasets increase the likelihood of annotation errors, redundant samples, and other forms of data inconsistency. This motivated the introduction of refined versions, such as MS1MV2 and MS1MV3, aimed at improving the quality of the training data. Despite these cleaning efforts, we show that several types of training-data defects remain in MS1MV3. By proposing complementary data-auditing strategies, we identify five defect categories: exact and near-duplicate images, split identities, crop errors, multiple-face ambiguity, and invalid content. We then assess the effect of correcting these defects under a controlled face-recognition training protocol based on IResNet-50 and ArcFace. The results show that targeted interventions can preserve, and in some cases improve, recognition performance while requiring fewer optimization steps to reach the performance of the original training set. Duplicate removal provides the clearest overall benefit, while split-identity merging and invalid-content removal also improve training efficiency. Overall, the results indicate that the effect of dataset cleaning depends strongly on the type of defect being addressed and provide further evidence that training-data quality remains an important and comparatively underexplored factor in large-scale face recognition. The training manifests and code used in this work will be released upon acceptance.

0.66