CS Paper Radar

A Multi-Task Deep Learning Framework for Pose-Invariant Face Recognition and Facial Landmark Localization

Ali Ashkeyev, Zukhra Abdiakhmetova

cs.CV ·submitted 2026-10-12 14:23 UTC · v1
benchmark

Abstract

Face recognition systems often experience performance degradation under large pose variations due to changes in facial appearance and visibility of discriminative features. This study presents a multi-task deep learning framework that jointly addresses pose-invariant face recognition and facial landmark localization within a unified architecture. To enable simultaneous optimization of identity recognition and facial structure learning, the original Cross-Pose Labeled Faces in the Wild (CPLFW) dataset was adapted into a multi-task learning dataset by associating facial images with identity labels and five-point landmark annotations. The proposed framework consists of two task-specific branches operating on a common facial input: a ResNet50-based branch for identity recognition and an EfficientNetV2B0-based branch with spatial attention for facial landmark localization. The two objectives are jointly optimized using a weighted multi-task loss, allowing the model to learn complementary semantic and spatial facial representations. The proposed framework was evaluated on the prepared CPLFW-based multi-task dataset. Unlike the original CPLFW benchmark, which focuses primarily on identity recognition, the proposed dataset adaptation incorporates five-point facial landmark annotations to enable joint recognition and localization learning. Experimental results demonstrate that the model achieves a validation recognition accuracy of 81.58% and a facial landmark localization mean squared error of 96.38. The results indicate that combining identity classification and landmark regression within a single optimization framework provides an effective approach for learning robust facial representations under cross-pose conditions The proposed method demonstrates the potential of multi-task learning and attention-based feature refinement for unconstrained facial analysis.

Comment
10 pages, 7 figures, 1 table

Closest papers

Nearest neighbours by meaning across the whole index.

Towards Unified Dynamic Face Landmark Detection

cs.CV cs.AI ·2026-08-11 ·Sebastian Regalado, Varshanth R. Rao, Ruowei Jiang, Parham Aarabi, Igor Gilitschenski
top venuebenchmark similarity 0.72
Abstract · Accepted at NeurIPS 2026. 9 pages, 6 figures in Main Paper. 13 pages, 3 figures in Appendix

Although advancements in face landmark detection (FLD) methods continue to push performance boundaries, they overlook two major functional limitations: (1) different network parameters need to be trained independently for each ``$N$-point'' benchmark dataset, and (2) a model trained on an ``$N$-point'' dataset reliably outputs only the $N$ landmarks. In our work, we first conceptualize Face Part-Anchored Landmark Positions (FPALPs), wherein each landmark is treated as a progression value between zero (start) and one (end) along a face part's contour. Every landmark can be expressed in the FPALP format, irrespective of its source dataset, hence unlocking the ability to unify all ``$N$-point'' datasets into a single dataset. Secondly, we represent each landmark with an FPALP-based query, refine it progressively with a cross-modality decoder, and predict its coordinates based on the final representation. Our approach, called Unified Dynamic FLD, embodies these two design choices and streamlines the landmark detection pipeline by enabling (1) a single model to learn on any number of ``$N$-point'' datasets, and (2) yield any number of specific landmark predictions by loading the designated landmark queries at runtime. Extensive experiments on multiple benchmark datasets show that our method delivers these benefits while remaining competitive with, and in several cases outperforming existing state-of-the-art methods.

0.37

Multi-scale Decomposed Convolution Refinement Network for Visible-Infrared Person Re-Identification

cs.CV ·2026-08-17 ·Mingsheng Zheng, Zirui Jiang, Bo Liu, Yupeng Chen, Jun Zhang +1 more
publishedcode/data similarity 0.67
Abstract · 15 pages, 4 figures. Accepted for publication in the LNCS proceedings of ICONIP 2026

Visible-infrared person re-identification (VI-ReID) suffers from cross-modal discrepancies and limited discriminative capabilities, leading to suboptimal recognition performance. Current approaches exhibit limitations in semantic mining, cross-modal fusion and feature constraints. To tackle these challenges, we propose MDCRNet, a Multi-scale Decomposed Convolution Refinement Network that enhances cross-modal feature learning and discriminative metric learning. Specifically, we introduce a Hierarchical Learning Module (HLM) containing four Hierarchical Decomposed Convolution Attention (HDCA) modules, each equipped with lightweight channel attention and multi-scale spatial perception blocks to capture multi-scale spatial dependencies. Moreover, we develop a Joint Discriminative Metric Loss (JDML) incorporating a novel Granularity Discriminative Loss (GDL) that simultaneously optimizes intra-identity compactness and inter-identity separability across modalities. Extensive experiments on SYSU-MM01 and RegDB datasets demonstrate that MDCRNet achieves state-of-the-art performance on both benchmarks. Code is available at https://github.com/Kevin-zms/MDCRNet.

0.38

DIME: Scaling Facial Representation Learning via Differential Masked Autoencoding

cs.CV ·2026-10-12 ·Hao Yu, Haoyu Chen, Hui Wei, Yan Jiang, Nicu Sebe +1 more
similarity 0.67
Abstract

Scaling up data and model capacity has driven substantial progress in visual representation learning. However, conventional pretext tasks exhibit diminishing returns on fine-grained facial analysis. We attribute this to a scaling bottleneck: these tasks model each image's appearance and are dominated by identity, pose, and illumination, so subtle within-identity facial differences contribute little to the training signal. We introduce DIME, a differential masked autoencoder for large-scale facial representation learning. DIME builds on a simple observation: two images of the same person share identity-specific appearance, so their difference exposes facial changes. DIME reconstructs both masked images in an identity-matched pair and, through an Explicit Differential Direction Supervision (EDDS) objective, aligns the direction of their reconstructed difference with the true difference, while source isolation makes each reconstruction depend only on its own image. Shared appearance cancels in this difference, and because EDDS ignores magnitude, subtle facial changes count as much as large pose or illumination changes. DIME uses identity labels only to form pretraining pairs, requires no dense annotations, and learns dense facial representations with emergent part-level semantics. We pretrain DIME on 20 million high-resolution facial images consolidated from existing face datasets. DIME's AU detection performance continues to improve as pretraining data scales from 6M to 20M images, with its BP4D F1 increasing by 5.9 points, while MAE gains only 0.5 points on the same subsets. DIME also outperforms specialized facial methods on AU detection, AU intensity estimation, cross-domain anti-spoofing, landmark localization, head pose estimation, face parsing, and face generation.

0.21

SapiensID 2.0: Aligning Human Recognition Foundation Models with Human Perception

cs.CV ·2026-08-11 ·Yiyang Su, Jie Zhu, Feng Liu, Anil K. Jain, Xiaoming Liu
similarity 0.66
Abstract

While foundation models have significantly advanced human recognition across diverse modalities, they predominantly rely on static, geometric feature extraction. This approach fundamentally diverges from human perception. Consequently, current models often suffer from "semantic blindness," overfitting to transient noise while failing to leverage invariant soft biometrics, and struggle to capture temporal motion signatures. To bridge this gap, we propose SapiensID 2.0, a human recognition framework enriched with both semantic and temporal awareness. To overcome the lack of soft-biometric annotations, we transfer zero-shot semantic knowledge from Multimodal Large Language Models (MLLMs) into a discriminative embedding space. We resolve the dimensional mismatch between these spaces using Invariant Trait Alignment (ITA) to distill core persistent traits, and Transient Noise Disentanglement (TND) to decouple artifacts like clothing. Furthermore, we design a Kinematic Semantic Attention Head (K-SAH) that extends spatial attention across temporal windows. By tracking semantic patches over time, K-SAH captures rich kinematic signatures without requiring large-scale video datasets. Extensive experiments demonstrate that SapiensID 2.0 achieves state-of-the-art performance across image- and video-based person re-identification and gait recognition, while maintaining robust face recognition capabilities.

0.34

FreqFLD: Towards All-in-One Facial Landmark Detection via Frequency Modulation

cs.CV ·2026-09-09 ·Shun Ren, Kaijie Jin, Shengkai Hu, Beihang Song, Hang Sun +3 more
publishedcode/data similarity 0.66
Abstract

Recent progress in deep learning has significantly advanced facial landmark detection. However, most existing methods process features in a spatial-domain manner under a dataset-specific training paradigm, which overlooks the fact that facial landmark detection is inherently geometry-driven and sensitive to frequency variations, thereby limiting cross-dataset generalization under complex scenarios and hindering the development of a facial landmark detection model. To address this issue, we propose \textbf{FreqFLD}, a \textbf{freq}uency-modulated framework towards All-in-One \textbf{f}acial \textbf{l}andmark \textbf{d}etection. Specifically, FreqFLD introduces a Frequency Modulation Module (FreqMoM) to explicitly induce the frequency prior by decoupling and modulating low- and high-frequency components, which is then injected into subsequent feature modeling to enable balanced modeling of global facial structure and local landmark details. Furthermore, FreqFLD employs a Frequency-Modulated Mixture-of-Experts (FreqMoE), with expert selection adaptively conditioned on frequency-modulated priors, enabling flexible modeling of heterogeneous facial landmark patterns under diverse and challenging scenarios. To regularize frequency-consistent modeling under the All-in-One paradigm, we further introduce a Frequency-Consistent Routing (FreqCR) loss, which constrains the routing and assignment of frequency-aware experts to promote balanced expert utilization across diverse facial scenarios, thereby enabling stable expert specialization and achieving robust facial landmark detection. Extensive experiments demonstrate that the proposed FreqFLD achieves comparable performance on popular datasets. The code is available at: https://github.com/jkj1059657014/FreqFLD.

0.55

STAG-Net: Structure-Aware Attention over Landmark Groups for Lightweight Head Pose Estimation

cs.CV cs.AIcs.HC ·2026-11-05 ·Asiri Gawesha Lindamulage, Sanka Mohottala, Dharshana Kasthurirathna, Nuwan Kodagoda
similarity 0.65
Abstract

Head pose estimation is important for human computer interaction and driver monitoring, but many image based methods remain too costly for resource constrained deployment. We propose STAG-Net, a structure-aware attention over landmark groups for lightweight head Pose estimation that estimates head pose from facial landmarks using movement aware selection, spatial clustering, compact DeepSet encoders, self attention, and a global geometry descriptor. We develop variants using FaceMesh and Face Alignment Network landmarks and evaluate them under cross dataset and subject independent protocols on 300W-LP, AFLW2000, and BIWI. STAG-Net achieves the lowest mean absolute error among all compared landmark based methods in every evaluated setting. On BIWI frames with valid detections, the FAN variant also records the lowest overall error among all compared methods, reaching $3.51^\circ$ under cross dataset evaluation and $2.39^\circ$ under subject independent evaluation. The regressors use only 0.012 to 0.043 million parameters and 24.1 to 83.6 thousand FLOPs, about three orders of magnitude fewer FLOPs than FSA-Net. Despite including landmark extraction and regression, the FaceMesh pipeline reaches 322.9 frames per second on a Tesla T4 GPU, nearly twice the 164.5 frames per second of the fastest measured image based baseline, and 111.0 frames per second on CPU. After normalization by dataset size, STAG-Net processes training samples seven to eight times faster than FSA-Net and more than one hundred times faster than HopeNet and TokenHPE, while complete training finishes within 28 minutes. These results show that STAG-Net combines leading landmark based accuracy with substantially lower training and inference cost.

0.18

VTF900: A 900-Subject Visible-Thermal Face Recognition Dataset and Benchmark

cs.CV ·2026-10-17 ·Jacob Abaare, Bianca Ianetta, Cedric Nimpa Fondje, Shuowen Hu, Benjamin S. Riggan
publishedbenchmarkhot direction similarity 0.64
Abstract · Accepted at IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) 2027

Despite advancements in visible-thermal face recognition, where thermal probe (query) face images are matched against visible enrollment templates, existing benchmarks are currently limited by small gallery sizes (N $\approx$ 100 subjects). This limitation inhibits the ability to accurately evaluate visible-thermal facial recognition at scale and to report significant recognition performance metrics, which have better operational relevance. Therefore, we introduce VTF900, a large-scale visible-thermal face recognition benchmark with 900 subjects and paired imagery that includes expression, pose, and occlusion variations. VTF900 is designed to support large-scale (i.e., large gallery) evaluations across verification (1:1 matching) and closed-set identification (1:N matching) for visible-thermal face recognition. Furthermore, VTF900 specifically addresses the challenging combination of cross-spectral and cross-dataset analysis. In this paper, after training models on the ARL-VTF dataset, we provide benchmark analyses across convolutional (baseline), vision transformers, and domain-adaptive neural network architectures to assess how these methods scale. At full gallery scale (N = 900), transformer models achieve strong discriminative performance under frontal conditions (e.g., ViT-B: 97.68% AUC, 51.10% TAR@1%FAR), while domain-adaptive methods such as DPIT-ResNet50 demonstrate greater robustness under off-pose conditions (94.68%AUC, 36.08% TAR@1%FAR). Our results show that high AUC at small scale does not translate to reliable performance at deployment scale, and that pose variation significantly amplifies performance degradation. VTF900 exposes these limitations, enabling more rigorous evaluation of cross-spectral face recognition systems.

0.32

A Privacy-Aware Deep Learning Benchmark for Facial Emotion Recognition

cs.CV cs.AI ·2026-11-06 ·Larissa Palhares, Maria Machado, Hudson Bruno, Raphael Ramos, Andrew Vale +3 more
benchmark similarity 0.64
Abstract · 7 pages, 2 figures

Facial Expression Recognition (FER) traditionally relies on raw RGB images, introducing privacy risks, sensitivity to illumination variations, and susceptibility to identity-confounding visual shortcuts. As an alternative, this work proposes a lightweight, privacy-aware pipeline using exclusively geometric representations-facial landmarks (478 points) and blendshapes (52 coefficients, ARKit standard)-eliminating persistent raw video storage. Six neural architectures trained from scratch on fused CREMA-D and RAVDESS datasets were benchmarked against an ImageNet-pretrained ResNet50 baseline under a 5-fold subject-independent cross-validation protocol. Geometric models consistently outperformed the pixel baseline on unseen subjects with 12-124 times fewer parameters, while demonstrating superior robustness on subtle emotional expressions such as sadness. Ablation studies reveal that blendshapes capture the primary discriminative signal, while landmarks provide essential complementary spatial topology. These results demonstrate that compact geometric representations preserve affective semantics, eliminate downstream transfer learning dependency on visual backbones, and enable efficient, privacy-aware emotion recognition.

0.32