CS Paper Radar

Beyond Landmark Extraction: A Framework for Robust Geometric Feature Evaluation in Structured Image Classification

Saravana Mauree, Sakshi Arya

cs.CV ·submitted 2026-09-01 03:10 UTC · v3
code/data

Abstract

Much of the literature on structured image recognition has disproportionately focused on the comparison of classification algorithms. Rather than investigating which classifier performs best, this paper instead asks: what should a classifier know before it ever makes a prediction? In structured vision problems such as gesture recognition, facial expression categorization, and medical image analysis, discriminative information lies less in individual pixels and more in spatial relationships between semantic parts. Raw pixel spaces are high-dimensional, sensitive to nuisance variation, and often obfuscate the geometric structures that make visual tasks interpretable. Landmark extraction provides one form of dimension reduction, but it does not by itself determine the information preserved. This paper studies the post-landmark feature map as the central object of analysis and proposes a systematic framework for constructing and interpreting landmark-derived representations as an, informed, feature-based "dimension reduction" step. Using static hand gesture recognition as a case study, we evaluate coordinate, distance, angle, and hybrid representations through perturbation and ablation experiments. The results show that visually variable data exposes substantial gaps between raw coordinate features and their geometrically invariant counterparts, while hybrid representations achieve the strongest overall performance by combining complementary geometric components. These findings frame feature construction as a fundamental modeling decision and ultimately suggests that the question of what representation should a classifier learn from is one worth asking. The code used for feature construction and evaluation is available at https://github.com/ShivMaureeCWRU/Feature_based_dimension_reduction

Comment
Under consideration at Pattern Recognition Letters

Closest papers

Nearest neighbours by meaning across the whole index.

VERA-Hand: A Verified Multidimensional Benchmark for 2D Hand Pose Estimation in Third-Person Dynamic Gesture Videos

cs.CV ·2026-10-23 ·Gibran Benitez-Garcia, Hiroki Takahashi
publishedbenchmark similarity 0.64
Abstract · Accepted to WACV 2027

Hand landmarks serve as input for gesture recognition, sign language understanding, and touchless human-computer interaction. Yet hand pose estimators are typically evaluated with frame-wise localization metrics such as PCK or MPJPE, which are computed over matched predictions and overlook missed or spurious hands, handedness errors, and temporal instability. To address these limitations, we present VERA-Hand, a benchmark that evaluates estimators using manually verified, frame-level 2D annotations for 10,690 hands across four complementary dimensions (detection, localization, identification, and temporal stability), as well as a joint Pose-F1 metric that requires both correct detection and accurate localization. Evaluating six estimators spanning hand-specific, whole-body, reconstruction-based, and foundation-model paradigms, we find that no estimator dominates: rankings reorder sharply across dimensions. VERA-Hand also exposes failures that localization-only and downstream evaluations may not reveal, including cross-hand overlap in over 85% of frames for a recent foundation model. A post-hoc cleanup case study demonstrates how the benchmark can validate corrective interventions, while stratified analyses show that robustness varies across motion-blur classes and hand regions. We release the verified annotations, protocol, and code at https://gibranbenitez.github.io/VERA_Hand/

0.42

Towards Unified Dynamic Face Landmark Detection

cs.CV cs.AI ·2026-08-11 ·Sebastian Regalado, Varshanth R. Rao, Ruowei Jiang, Parham Aarabi, Igor Gilitschenski
top venuebenchmark similarity 0.63
Abstract · Accepted at NeurIPS 2026. 9 pages, 6 figures in Main Paper. 13 pages, 3 figures in Appendix

Although advancements in face landmark detection (FLD) methods continue to push performance boundaries, they overlook two major functional limitations: (1) different network parameters need to be trained independently for each ``$N$-point'' benchmark dataset, and (2) a model trained on an ``$N$-point'' dataset reliably outputs only the $N$ landmarks. In our work, we first conceptualize Face Part-Anchored Landmark Positions (FPALPs), wherein each landmark is treated as a progression value between zero (start) and one (end) along a face part's contour. Every landmark can be expressed in the FPALP format, irrespective of its source dataset, hence unlocking the ability to unify all ``$N$-point'' datasets into a single dataset. Secondly, we represent each landmark with an FPALP-based query, refine it progressively with a cross-modality decoder, and predict its coordinates based on the final representation. Our approach, called Unified Dynamic FLD, embodies these two design choices and streamlines the landmark detection pipeline by enabling (1) a single model to learn on any number of ``$N$-point'' datasets, and (2) yield any number of specific landmark predictions by loading the designated landmark queries at runtime. Extensive experiments on multiple benchmark datasets show that our method delivers these benefits while remaining competitive with, and in several cases outperforming existing state-of-the-art methods.

0.37

Dense Representation Learning and Differentiable Shape Modeling for Spine Landmark Detection

cs.CV cs.AI ·2026-10-27 ·Augustin Daridon, Sofiane Sifaoui, Ayman Assi, Mohamad Karam, Wafa Skalli +2 more
code/data similarity 0.63
Abstract

Spine landmark detection from spinal X-ray is a key step for estimating spine deformation severity, for building biomechanical models, and for pre-operative planning. We reframe anatomical landmark detection as a pixel-level contrastive representation learning problem, matching dense pixel embeddings with landmark prototypes. Surprisingly, even with a generic U-Net backbone, this formulation results in more discriminative heatmaps compared to the traditional heatmap regression paradigm, leading to improved discrimination between vertebral corners on distinct vertebra. Furthermore, we combine the resulting heatmaps with learnable shape priors over the global configuration of spinal landmarks to resolve ambiguities in landmark locations, yielding a simple and practical approach to spinal landmark detection. Our approach is competitive, in terms of landmark accuracy and clinical parameter estimation, with the state-of-the-art on the public AASCE dataset and on two private spinal X-ray datasets. Our code and pre-trained weights are available at https://github.com/BioMedTP/Pix2Point.

0.33

A Multi-Task Deep Learning Framework for Pose-Invariant Face Recognition and Facial Landmark Localization

cs.CV ·2026-10-12 ·Ali Ashkeyev, Zukhra Abdiakhmetova
benchmark similarity 0.62
Abstract · 10 pages, 7 figures, 1 table

Face recognition systems often experience performance degradation under large pose variations due to changes in facial appearance and visibility of discriminative features. This study presents a multi-task deep learning framework that jointly addresses pose-invariant face recognition and facial landmark localization within a unified architecture. To enable simultaneous optimization of identity recognition and facial structure learning, the original Cross-Pose Labeled Faces in the Wild (CPLFW) dataset was adapted into a multi-task learning dataset by associating facial images with identity labels and five-point landmark annotations. The proposed framework consists of two task-specific branches operating on a common facial input: a ResNet50-based branch for identity recognition and an EfficientNetV2B0-based branch with spatial attention for facial landmark localization. The two objectives are jointly optimized using a weighted multi-task loss, allowing the model to learn complementary semantic and spatial facial representations. The proposed framework was evaluated on the prepared CPLFW-based multi-task dataset. Unlike the original CPLFW benchmark, which focuses primarily on identity recognition, the proposed dataset adaptation incorporates five-point facial landmark annotations to enable joint recognition and localization learning. Experimental results demonstrate that the model achieves a validation recognition accuracy of 81.58% and a facial landmark localization mean squared error of 96.38. The results indicate that combining identity classification and landmark regression within a single optimization framework provides an effective approach for learning robust facial representations under cross-pose conditions The proposed method demonstrates the potential of multi-task learning and attention-based feature refinement for unconstrained facial analysis.

0.20

Learning a Task-Adaptive Low-Dimensional Semantic Space for Improved Visual Classification

cs.CV ·2026-11-16 ·Nikalal Kaldera, Wanqing Li, Chris Bunn, Philip O. Ogunbona
top venue similarity 0.59
Abstract · 20 pages, 2 figures. Accepted at NeurIPS 2026

Semantic visual classification commonly aligns visual embeddings with semantic spaces induced by pretrained text encoders. However, such spaces are typically high-dimensional, weakly aligned with visual feature geometry, and insufficiently discriminative for downstream classification tasks. We propose TaSS, a framework for learning a task-adaptive low-dimensional semantic space that preserves semantic structure while explicitly optimizing class separability. TaSS is constructed through a parameterized geometry transformation of pretrained text embeddings that enlarges inter-class margins via distance reshaping and improves intra-class compactness through prototype-preserving semantic constraints. The resulting semantic space maintains meaningful semantic relationships while producing a discriminative geometry tailored for classification. Frozen visual encoder features are subsequently aligned to TaSS using a lightweight MLP, enabling improved discriminative learning without fine-tuning pretrained backbones. We further provide theoretical analysis showing that TaSS preserves semantic consistency while enforcing enhanced inter-class separation and controlled intra-class variance. Extensive experiments on skeleton-based human action recognition, video classification, and image classification benchmarks demonstrate consistent improvements over existing semantic alignment approaches, with gains exceeding 3 percentage points on average and up to 8.5 points on challenging datasets.

0.38

DINO Eats Geometry: Self-Supervised Sign Representations that Absorb Articulated Hand Geometry

cs.CV cs.CL ·2026-09-05 ·Junyi Hu, Zhewen He, Haomian Huang, Zhenhua Li, Zhifei Li +2 more
hot direction similarity 0.59
Abstract · 22 pages, 7 figures

Handshape and orientation are geometric components of sign language, yet many sign encoders learn from appearance alone. We introduce SignDEG (DINO Eats Geometry), a temporal sign encoder that uses articulated hand geometry as a pre-training target and requires only video at inference. Frozen DINOv3 features of tracked hands and faces feed per-stream temporal Transformers. Geometric latent prediction (GLP) supervises an intermediate block with WiLoR latents, MANO articulation, and 3D joints, including at masked frames. Geometric relation transfer (GRT) matches cross-video frame-similarity distributions at the final block. Both objectives are learned jointly with temporal self-distillation; WiLoR and the geometry heads are then discarded. Under the SHuBERT public-data evaluation protocol, SignDEG reaches 17.9/24.5 BLEU-4 on How2Sign/OpenASL, compared with 16.2/23.2 for SHuBERT. Rank-1 adaptation improves ASL Citizen and Sem-Lex recognition by 0.02 Recall@1 each, and phonological classification improves across 16 attributes. Removing GRT while retaining GLP reduces How2Sign BLEU-4 by 1.4 and ASL Citizen Recall@1 by 0.034. The results support transferring articulated structure into an appearance encoder without running a hand reconstruction model at deployment.

0.37

UniCon-Former: Unified Convolution Transformer is All You Need for Hand Gesture Recognition

cs.CV cs.HC ·2026-08-13 ·Mallika Garg, Debashis Ghosh, Pyari Mohan Pradhan
similarity 0.58
Abstract

Convolutional Neural Networks (CNNs) capture local features efficiently but struggle with global context due to their limited receptive field. On the other hand, transformers effectively capture global dependencies through self-attention but suffer from high redundancy and computational costs. Thus, to leverage the advantages of both CNNs and transformers, we propose a unified model (UniCon-Former) that aims to provide robust and efficient performance on dynamic hand gesture recognition. The unified approach helps the model to learn both local and global features. At the beginning of each transformer stage, the convolution projections help in decreasing the dimension of the input vectors of the transformer block. This creates a pyramidal structure at each transformer stage. These features enable the UniCon-Former to reduce resource usage than vanilla transformers, making it flexible for learning multi-scale and high-resolution features, which is required in hand gesture recognition. We have performed experiments with NVGesture and Briareo datasets and achieved state-of-the-art results with fewer parameters and MACs.

0.04

Do vision models know that three is three?

cs.CV ·2026-10-26 ·Ankit Sonthalia, Arnas Uselis, Seong Joon Oh
similarity 0.58
Abstract

Do vision models encode the transition $2{\to}3$ the same way whether it appears in handwritten digits or in hand gestures, and do they "understand" that clean${\to}$dirty is the same transformation regardless of whether it is applied to a kitchen or a carpet? The linear representation hypothesis (LRH) answers this for language models: concept changes correspond to directions in (language) embedding space. In vision, the same concept can manifest in visually unrelated "regimes" that rarely co-occur: images of digits and hands do not share much in common in pixel space. To understand whether vision models still encode the underlying abstract concepts, we study 46 vision encoders across architectures, training objectives, and scales, on 10 concept families including numerosity, age, object identity, and MIT-States attributes. Within each regime, we find statistically significant concept directions near-universally across our tested models. However, two distinct regimes do not always share the same concept direction. Several models learn multiple linear structures for the same concept transition, split across regimes. We call this phenomenon "fragmented linearity": strong local linearity coexisting with near-absent global linearity. Fragmentation is, however, not fixed: models with stronger local linearity progressively escape it (local-global linearity correlated with $τ\in [0.38, 0.89]$ across all concepts), and where global linearity does emerge, concept directions can transfer as linear probes across regimes.

0.15