CS Paper Radar

UniCon-Former: Unified Convolution Transformer is All You Need for Hand Gesture Recognition

Mallika Garg, Debashis Ghosh, Pyari Mohan Pradhan

cs.CV cs.HC ·submitted 2026-08-13 13:19 UTC · v1

Abstract

Convolutional Neural Networks (CNNs) capture local features efficiently but struggle with global context due to their limited receptive field. On the other hand, transformers effectively capture global dependencies through self-attention but suffer from high redundancy and computational costs. Thus, to leverage the advantages of both CNNs and transformers, we propose a unified model (UniCon-Former) that aims to provide robust and efficient performance on dynamic hand gesture recognition. The unified approach helps the model to learn both local and global features. At the beginning of each transformer stage, the convolution projections help in decreasing the dimension of the input vectors of the transformer block. This creates a pyramidal structure at each transformer stage. These features enable the UniCon-Former to reduce resource usage than vanilla transformers, making it flexible for learning multi-scale and high-resolution features, which is required in hand gesture recognition. We have performed experiments with NVGesture and Briareo datasets and achieved state-of-the-art results with fewer parameters and MACs.

Closest papers

Nearest neighbours by meaning across the whole index.

Real-Time Hand Gesture Recognition for OpenXR Using Transformer-Based Machine Learning

cs.AI ·2026-09-21 ·Salar Rezayani, Russell Butler
similarity 0.70
Abstract

Hand gesture recognition is a key component in human-computer interaction (HCI), enabling intuitive interfaces for applications in gaming, virtual reality (VR), robotics, and more. This study integrates transformer-based machine-learning models for real-time hand gesture recognition, using hand-tracking data captured through the OpenXR standard in Unity. We leverage positional data of hand joints and wrist rotation angles to train a custom gesture recognition system. By utilizing the sequential modeling capabilities of transformers, the system captures temporal dependencies within short gesture windows and classifies gestures robustly across hand orientations and sizes. The results show a significant improvement in gesture classification accuracy. Building on this, we outline how the approach can be extended toward detecting the flow of movement, i.e., the transitions between gestures, as future work.

0.14

MambaHand: Local Identity and Behavioral Sequence Learning with Mamba for Dynamic Hand Gesture Authentication

cs.CV ·2026-11-03 ·Zenan Lin, Junduan Huang, Xilai Wang, Wenxiong Kang
benchmark similarity 0.66
Abstract

Dynamic hand gesture authentication (DHGA) is an emerging biometric technique that contains both physiological and behavioral identity characteristics. However, existing methods primarily rely on convolutional networks or manually designed motion representations, making it difficult to jointly capture fine-grained local identity differences and long-range behavioral dependencies during hand gesture execution. To address these challenges, we propose a two-stream identity representation learning framework, MambaHand Net, to combine CNNs with Mamba for DHGA. The proposed framework consists of a Local Identity Stream (LIS) and a Behavioral Sequential Stream (BSS). The LIS extracts frame-wise local identity cues with a convolutional network, while the BSS preserves spatial tokens from intermediate visual features and constructs first-order and second-order temporal difference sequences to model multi-order behavioral variations. Moreover, the bidirectional Mamba is incorporated into the BSS to model semantic motion across different spatial locations, enabling efficient long-range modeling of hand gesture execution habits. Finally, attention-guided spatial token aggregation and a two-stream fusion strategy are used to integrate local identity cues and multi-order behavioral patterns into a discriminative identity representation. Our method achieves state-of-the-art (SOTA) average performance on the SCUT-DHGA benchmark dataset, demonstrating that combining CNNs with Mamba is an effective new solution for DHGA.

0.30

TransHands: Repurposing Human Pose Encoders as Hand Pose Encoders

cs.CV cs.AI ·2026-08-23 ·Milo Piccioli, Gianluca Amprimo, Claudia Ferraris, Gabriella Olmo
similarity 0.62
Abstract

Lifting 3D hand poses from 2D monocular representations remains challenging due to the limited availability of large-scale, diverse 3D-annotated hand datasets, in contrast to the abundance of human body motion data. We address this limitation by transferring motion representations learned from large body pose corpora to the hand domain. We introduce TransHands, a backbone-agnostic transfer learning framework that enables pre-trained human motion encoders to be effectively adapted for 3D hand pose estimation from 2D pose inputs. Rather than training hand-specific biomechanical models from scratch, TransHands combines a two-stage training and fine-tuning strategy with a lightweight hand-specific input adaptation module that aligns hand kinematics with the representation space learned for full-body motion. We evaluate TransHands across four state-of-the-art motion modeling architectures, including transformer-based, graph-based, and frequency- domain models. Results demonstrate that motion priors learned from body pose data transfer consistently across architectures, yielding consistent accuracy gains, strong cross-domain generalization, particularly in challenging egocentric settings, and applicability for downstream tasks in real-world contexts.

0.14

Comparing Temporal Deep Learning Architectures for Discrete Cross-User Myoelectric Control

cs.HC ·2026-10-27 ·Ethan Eddy, Evan Campbell, Scott Bateman, Erik Scheme
dataset similarity 0.62
Abstract

The recognition of dynamic gestures via discrete myoelectric control is becoming an attractive input modality for event-based human-machine interaction. While recent work suggests that discrete cross-user models are possible (i.e., no calibration data is required from the end user), few have evaluated techniques to optimize these systems and their generalizability to new randomly prompted data simulating real-world discrete control is relatively unknown. Using the 612-user EMG-EPN dataset, this work benchmarks five temporal deep-learning approaches, evaluates the tradeoff between performance and model complexity, and compares hand-crafted and machine-learned feature extraction techniques. The CNN-GRU achieved the best accuracy of 93% when tested on 184 previously unseen users. These models were then evaluated on a different, more realistic, unsegmented dataset, wherein users were randomly prompted to elicit discrete gestures during a continuous data collection. This dataset was recorded five years apart and in a separate country from the data used to train the original model. Results show that the cross-user results scale to this new dataset (accuracies of 96%) even though no feedback was given to users during the elicitation of gestures. These findings highlight the exciting potential of discrete zero-shot cross-user myoelectric control and offer valuable insights for the continued advancement of this promising technology.

0.16

AD-Mamba: Architecture-Complementary Knowledge Distillation for Efficient and Robust Audio Classification

eess.AS ·2026-10-20 ·Junyu Wang, Tianrui Wang, Meng Ge, Longbiao Wang, Xiaobao Wang +1 more
similarity 0.61
Abstract

Recent advancements in large-scale audio classification have been primarily driven by Transformer-based architectures due to their strong capacity in modeling long-range dependencies. However, their inherent quadratic complexity limits scalability and efficiency, especially when processing long or sparsely informative audio streams. Mamba, a recently proposed state space model (SSM), offers a promising alternative by enabling efficient long-range modeling with linear sequence scaling. Nevertheless, existing Mamba-based audio classification methods still lag behind mainstream Transformer-based models, suggesting that effective training strategies for fully exploiting the audio modeling potential of SSMs remain underexplored. To address this issue, we propose AD-Mamba, an architecture-complementary multi-stage knowledge distillation framework that transfers the local modeling strength of CNNs and the global contextual modeling capability of Transformers into a Mamba-based student model. Specifically, the CNN-guided stages encourage the student to adapt to local spectro-temporal structures, while the Transformer-guided stage further refines global semantic discrimination and contextual modeling. Experimental results on AudioSet show that AD-Mamba achieves 50.3% mean average precision (mAP) with only 30M parameters, outperforming representative Transformer-based and Mamba-based audio classification baselines with lower computational cost. Further evaluations under long-form audio, sparse-information, and noise-interference conditions show that AD-Mamba consistently outperforms representative CNN and Transformer baselines. When transferred to FSD50K, ESC-50, and DCASE16, the proposed model also achieves superior performance, demonstrating strong generalization across audio classification tasks.

0.10

RIECNN: Real-Time Image Enhanced CNN for Traffic Sign Recognition

cs.CV cs.AIcs.LG ·2026-10-10 ·Reem Abdel-Salam, Rana Mostafa, Ahmed H. Abdel-Gawad
publishedbenchmarkhighly cited similarity 0.61
Abstract · Published in Neural Computing and Applications, Volume 34, pages 6085-6096 (2022)

Traffic sign recognition plays a crucial role in the development of autonomous cars to reduce the accident rate and promote road safety. It has been a necessity to address traffic signs that are affected significantly by the environment as well as poor real-time performance for deep-learning state-of-the-art algorithms. In this paper, we introduce Real-Time Image Enhanced CNN ({\our}) for Traffic Sign Recognition. {\our} is a real-time, novel approach that tackles multiple, diverse traffic sign datasets, and out-performs the state-of-the-art architectures in terms of recognition rate and execution time. Experiments are conducted using the German Traffic Sign Benchmark (GTSRB), the Belgium Traffic Sign Classification (BTSC), and the Croatian Traffic Sign (rMASTIF) benchmark. Experimental results show that our approach has achieved the highest recognition rate for all Benchmarks, achieving a recognition accuracy of 99.75\% for GTSRB, 99.25\% for BTSC and 99.55\% for rMASTIF. In terms of latency and meeting the real-time constraint, the pre-processing time and inference time together do not exceed 1.3 ms per image. Not only have our proposed approach achieved remarkably high accuracy with real-time performance, but it also demonstrated robustness against traffic sign recognition challenges such as brightness and contrast variations in the environment.

0.53

ACSAN: All-Convolutional Self-Attention Network Integrating Convolutional Operations with Vision Transformer

cs.CV cs.AI ·2026-10-26 ·Zimian Wei, Tianqi Zhang, Hengyue Pan
similarity 0.61
Abstract

In the past few years, Vision Transformer (ViT) has achieved competitive performance in the field of computer vision. However, there still remain some problems. For instance, ViTs rely on large-scale training data since they simply contain a large number of trainable parameters, which slows down their learning efficiency and limits their applicability in resource-constrained scenarios. In this paper, we propose a novel deep learning model, namely All-Convolutional Self-Attention Network (ACSAN), which applies convolution operations to replace fully-connected matrices multiplications in the ViT framework. To implement ACSAN, we firstly design Dynamic Convolutional Self-Attention (DCSA) as the basic module, which employs multi-scale dilated convolutions to generate Query, Key, and Value. We define spatial self-attention and channel self-attention to generate attention features respectively. The two attention features are finally merged as the output of the DCSA module. Based on DCSA, we construct the basic ACSAN block. Specifically, we use DCSA to replace the regular self-attention operations, and use convolution-based Equivalent Convolutional Fully Connected (EqConvFC) modules to replace MLPs. To evaluate the performance, we train ACSAN on CIFAR-10, CIFAR-100 and ImageNet-1k datasets from scratch. Experimental results show that ACSAN achieves good classification accuracy (the best variants achieve 93.52\% on CIFAR-10, 73.38\% on CIFAR-100, and 83.3\% on ImageNet-1k). Moreover, in comparison with standard ViT and other state-of-the-art methods, ACSAN uses fewer parameters to achieve comparable or better accuracy.

0.16

Single-Query Person-Centric Bimanual Hand-Object Interaction Detection

cs.CV ·2026-09-10 ·Jonghyun Kim, Junho Roh, Yubin Yoon, Hyotae Lee, Jongkuk Park +3 more
top venuecode/data similarity 0.61
Abstract · Accepted to ECCV2026, Project page: https://lgecto-ail-vil.github.io/SingleQuery-BHOI/

Understanding person-level bi-manual interactions requires not only detecting hands, but also identifying which two hands belong to the same person and what each hand interacts with. Existing hand--object interaction methods are mostly hand-centric: they treat each hand as an independent instance, which can lead to ambiguous ownership in multi-person scenes. We propose a person-centric formulation in which a single query predicts a structured output for one person, including the human box, body pose, hand boxes and states, and interaction targets. We introduce part-aware deformable attention to allocate attention across human, hand, and pose-specific reference regions, enabling one query to capture the full person structure. We further unify detection and interaction reasoning with a hand-to-query relationship matrix, where each hand selects its interaction target from the detected query set plus a learnable off token, directly recovering the target's box and class without separate object regression. We build a COCO-based dataset with person-centric bi-manual interaction annotations and define structured metrics for evaluating hand states and complete hand--object tuples. Experiments with a transformer-based detector show that our formulation improves person-level bi-manual interaction parsing and provides an effective unified framework for joint detection, pose estimation, and hand reasoning.

0.52