CS Paper Radar

Geometry-Constrained Dynamic Hypergraph Convolutional Network with Contrastive Score Refinement for Skeleton-Based Action Recognition

Minh Ta Tran Nhat, Yuning Ye, Yuseok Ban

cs.CV ·submitted 2026-11-02 07:37 UTC · v1
publishedcode/data

Abstract

Skeleton-based action recognition is a pivotal task in computer vision; however, conventional Graph Convolutional Networks (GCNs) are often constrained by pairwise joint modeling, which fails to capture the intricate high-order interactions inherent in human movement. While recent hypergraph-based methods attempt to address this limitation, they typically rely on static structures derived from anatomical priors, lacking the flexibility to capture dynamic, action-specific joint correlations. To this end, we propose DyGCN, a Geometry-Constrained Dynamic Hypergraph Convolutional Network for skeleton-based action recognition. At the core of DyGCN is the Dynamic Hypergraph Construction Network with Contrastive Score Refinement (DyGCT) block, which comprises two novel components: the Geometry-Constrained Dynamic Hypergraph Construction (GeoHC) module and the Contrastive Score Refinement (CSR) module. Specifically, GeoHC dynamically constructs action-specific hyperedges by leveraging geometry-constrained second-order compatibility, transcending the limitations of static hyperedge structures. Furthermore, CSR explicitly refines the pairwise compatibility scores during hypergraph construction through prototype-based contrastive learning, fostering more discriminative action-specific hyperedges, with zero inference-time overhead as CSR operates only during training. Extensive experiments on NTU RGB+D, NTU RGB+D 120, and Northwestern-UCLA validate the effectiveness of DyGCN, achieving state-of-the-art performance on widely used benchmarks. The source code is available at https://github.com/marvineta0405/DyGCN.

Comment
19 pages, 4 figures, BMVC 2026

Closest papers

Nearest neighbours by meaning across the whole index.

NeuroPath: Brain-Inspired Dual-Pathway Graph Convolutional Networks for Skeleton-Based Action Recognition

cs.CV ·2026-08-18 ·Kanglei Zhou, Ruizhi Cai, Hubert P. H. Shum, Frederick W. B. Li, Xiaohui Liang
published similarity 0.76
Abstract · Accepted to Pattern Recognition

Skeleton-based action recognition aims to recognize human actions from sequences of human joint coordinates. Most existing Spatial-Temporal Graph Convolutional Networks (STGCNs) have achieved promising results by modeling skeletal structures with implicit spatial-temporal representations. However, our empirical study reveals a clear performance imbalance across different skeletal modalities, indicating that implicitly coupling spatial and temporal information limits the full exploitation of complementary structural and motion cues. Inspired by the ventral and dorsal pathways in human perception, we propose Dual-Pathway Graph Convolutional Networks (NeuroPath), which adopt a dual-pathway architecture for separate yet collaborative modeling of spatial and temporal information. Specifically, transformation units first convert the input into pathway-specific skeletal representations, allowing each pathway to focus on complementary aspects of human motion. To further capture coordinated joint behaviors and their interrelationships, we introduce a group graph convolution block that dynamically identifies key body parts and models their spatial-temporal dependencies. In addition, inter-pathway dynamic fusion modules integrate complementary inter-modal information across pathways, facilitating higher-level semantic interpretation of actions. Extensive experiments on Kinetics Skeleton 400, NTU RGB+D 60, and NTU RGB+D 120 demonstrate consistent performance improvements, validating the effectiveness of dual-pathway spatial-temporal modeling for skeleton-based action recognition.

0.41

DSR-Net: Learning Dynamic Skeletal Relations for 3D Human Pose Estimation

cs.CV ·2026-10-10 ·Fan Li, Jian Xiang, LinLin Xue, Yuwei Gao
similarity 0.71
Abstract · 16 pages, 10 figures

3D human pose estimation requires accurate modeling of motion-dependent skeletal structures and inter-joint dependencies. Existing methods typically rely on fixed skeletal topologies in Graph Convolutional Networks (GCNs) or implicitly capture joint relations through attention mechanisms, limiting their adaptability to diverse motion patterns. To address this issue,we proposeDSR-Net, a dynamic structural representation learning framework for 3D human pose estimation. Specifically, a Dynamic Skeletal Relation (DSR) module is introduced to learn adaptive inter-joint dependencies beyond predefined skeletal connections. Based on the learned relations, a hierarchical relation-aware fusion strategy is developed to facilitate effective interaction between structural and temporal representations. In addition, a lightweight Residual Skeletal Recalibration Module (RSRM) is designed to enhance the stability of structure-aware features with minimal computational overhead. Extensive experiments on Human3.6M and MPI-INF-3DHP demonstrate that DSR-Net achieves MPJPEs of 37.9 mm and 14.9 mm, respectively, consistently improving over strong baselines. The results validate the effectiveness of dynamic skeletal relation modeling for robust 3D human pose estimation.

0.14

HSTGFormer: Hyper Spatial-Temporal Graph Transformer for 3D Human Pose Estimation

cs.CV cs.AI ·2026-08-12 ·Ruochen Li, Shuang Chen, Wenke E, Farshad Arvin, Amir Atapour-Abarghouei
published similarity 0.65
Abstract · Accepted to BMVC 2026, full paper

Transformer-based methods have achieved strong performance in monocular 3D human pose estimation, but most existing approaches organise spatial and temporal reasoning as separate stages, which may weaken unified spatial-temporal interdependencies inherent in human motion and compress frame-level structural information before temporal modelling. In this paper, we propose HSTGFormer, a graph-enhanced Transformer framework that reformulates spatial-temporal reasoning as localised coupled graph aggregation over joint-time nodes. Specifically, HSTGFormer introduces a Hyper Spatial-Temporal Graph (HSTG), which decomposes global spatial-temporal reasoning into local spatial-temporal receptive fields around individual joint-time nodes by extending per-frame skeleton graphs into temporal neighbourhoods, thereby enabling structure-aware coupled reasoning while preserving local structural motion information. It further incorporates an Adaptive Dual-Scale Temporal Graph (ADSTG) to capture joint-specific temporal dependencies over complementary short- and long-range windows. A lightweight node-wise fusion module further adaptively integrates the two graph representations for each joint-time node. Experiments on Human3.6M and MPI-INF-3DHP show that HSTGFormer achieves strong accuracy with high computational efficiency.

0.27

Fine-Grained Action Recognition with Cross-Attentive Latent Sparse Experts

cs.CV ·2026-08-13 ·Imtiaz Ul Hassan, Tasweer Ahmad, Nik Bessis, Ardhendu Behera
published similarity 0.64
Abstract · Accepted manuscript: Workshop on Affective & Behavior Analysis in-the-wild (ABAW), as part of European Conference on Computer Vision (ECCV) …

Fine-grained human action recognition (FHAR) must distinguish visually similar actions that differ mainly in body configuration, timing, or local appearance. RGB representations retain visual context but often suppress joint-level geometry, whereas skeleton representations encode kinematics but discard dense spatial detail. We introduce FineX, which factorizes fine-grained cues into RGB appearance, pose heatmap geometry, and skeletal-graph topology. Pairwise cross-attention enables symmetric, stream-preserving information exchange, followed by a streamwise latent sparse Mixture-of-Experts that routes each representation to a content-dependent subset of shared experts, regularized by a load-balancing objective. FineX achieves state-of-the-art results on Gym99, Gym288, and Diving48. On the long-tailed Gym288, it raises mean class accuracy from 68.6% to 76.2% (+7.6 points) without textual supervision or large-scale vision-language pre-training, demonstrating the benefit of structured visual-pose-graph fusion and conditional expert refinement for FHAR.

0.20

UniMate: One Unified Model to Animate Diverse Skeletons

cs.CV cs.GRcs.LG ·2026-09-04 ·Linzhan Mou, Jiahui Lei, Zhiyang Dou, Chenyue Cai, Chaoyue Song +2 more
publishedcode/data similarity 0.64
Abstract · SIGGRAPH Asia 2026. Project page: https://linzhanmou.com/unimate/

Recent advances in automatic rigging now deliver animation-ready 3D assets at scale, yet generating the motion to drive them remains a bottleneck. Existing learned animators are topology-constrained: they rely on category-specific templates or require per-skeleton fine-tuning and reference motions at inference. We present UniMate, a unified foundation model that synthesizes articulated motion for arbitrary skeletons from a rigged 3D asset and a text prompt, with no test-time optimization or per-skeleton retraining. UniMate introduces a topology-aware diffusion transformer, which integrates skeletal topology into attention via three mechanisms: (1) a graph-aware attention bias from pairwise joint relations and geodesic distances; (2) a spectral rotary position embedding generalizing RoPE to arbitrary kinematic trees via the graph Laplacian; and (3) a global topological conditioner attention-pooled from the rest-pose skeleton. We also curate UniML3D, 13,006 motion sequences spanning bipedal, quadrupedal, avian, marine, insectoid, serpentine, and articulated rigid objects with unified canonicalization and text pairing. Trained on this dataset, UniMate outperforms state-of-the-art baselines in quality, generalization, and efficiency, and supports zero-shot cross-topology transfer, in-betweening, expansion, and text-guided editing. Our project page is available at https://linzhanmou.com/unimate/.

0.61

DiMoP: Diffusion-Driven Motion Representation Learning With Frame-Level Pseudo-Classification for Skeleton-Based Action Recognition

cs.CV ·2026-09-28 ·Shanaka Ramesh Gunasekara, Wanqing Li, Nikalal Kaldera, Philip Ogunbona, Jack Yang
top venue similarity 0.64
Abstract · Accepted to IEEE TRANSACTIONS ON BIOMETRICS, BEHAVIOR, AND IDENTITY SCIENCE

Robust skeleton-based action recognition requires representations that capture a wide spectrum of motions, from subtle to moderate and strong ones. Existing methods often focus on strong motions. This paper introduces DiMoP, a masking- and diffusion-driven motion representation learning method with frame-level pseudo-classification to explicitly learn the distribution of joint motions rather than regressing deterministic coordinates, as existing methods often do. By diffusing masked joints with progressive noise and denoising them conditioned on visible joints, DiMoP learns through controllable noising and denoising processes, enabling uniform learning of weak, moderate, and strong dynamics. To enable the masking-based generative diffusion learning with a discriminative capability, a pseudo-frame classifier is proposed that enforces the learning towards sequence-consistent and temporally coherent pseudo-labels without manual annotations. Together, these strategies provide a principled mechanism for joint generative and discriminative motion modeling. DiMoP achieves state-of-the-art performance across NTU RGB+D 60/120, and PKUMMD, including a 1.1 percentage point gain over prior works on NTU RGB+D 120 with the cross-subject protocol.

0.36

ProMoDD: Dataset Distillation for 3D Human Motion via Prototype-Conditioned Diffusion Models

cs.CV ·2026-10-12 ·Yijin Wei, Yawen Zou, Wenqi Cai, Guang Li, Hangli Ge +3 more
similarity 0.63
Abstract

Dataset distillation (DD) aims to retain the training utility of a large dataset in a much smaller set of examples. For skeleton-based action recognition, this requires capturing discriminative motion variation without disrupting the geometry and temporal continuity of articulated bodies. Directly optimizing joint coordinates can violate these constraints, while plausible motion generation alone does not ensure useful training examples. We propose Prototype-Conditioned Motion Dataset Distillation (ProMoDD), which conditions a motion diffusion model on class-wise prototypes of temporally resolved recognition features. We further tailor diffusion training to motion data, enabling the generator to model both skeletal configurations and their temporal evolution. We then construct each compact training set through budget-aware hybrid selection: representative real sequences cover densely populated regions of the pooled recognition space, while selected generated candidates complement them in less-populated regions. This couples prototype-guided generation with feature-space selection, balancing the reliability of observed motions with the variation of synthesis under a fixed per-class sequence budget. Across five difficulty-stratified subsets of the single-person classes in NTU RGB+D 60, ProMoDD achieves its largest gains under tight distillation budgets, reaching 40.32% top-1 accuracy on Full49 with three sequences per class, 4.57 percentage points above class-wise k-means. These gains reflect the complete framework. The same distilled sets train ST-GCN, MS-G3D, and CTR-GCN without regeneration.

0.15

De-biasing Skeleton-based Action Recognition with Convex Hull Adaptive Shift

cs.CV cs.AIcs.LG ·2026-09-26 ·Mengyuan Liu, Yuhang Wen, Yi Zhang, Songtao Wu, Hong Liu +2 more
top venuecode/data similarity 0.63
Abstract · Accepted for publication in International Journal of Computer Vision (IJCV). Our code is publicly available at https://github.com/Necolizer/…

Skeleton sequences can represent both individual actions and multi-entity interactions, encompassing human bodies, hands, objects, and robots. Existing approaches to recognize skeleton-based actions and interactions usually adopt a late fusion strategy, which expects individuals are independent and identically distributed to train a robust weight-shared entity encoder. However, observed entity bias in various skeletal data violates this assumption, leading to suboptimal optimization of backbone models that might produce wrong recognition results. This bias arises from the world coordinate system's initial configuration, where the choice of origin often creates bias in representation. To this end, we propose a Convex Hull Adaptive Shift based normalization method to reduce Entity bias (CHASE), improving performance across a variety of skeleton-based action and interaction recognition tasks. To adaptively apply plausible shifts to the input skeletons, we formulate a plug-and-play parameterized network that ensures the relocated world origin lies within the skeleton convex hull, which avoids non-convergence by limiting the search space. To further minimize entity bias, we incorporate an auxiliary objective that leverages pair-wise distribution distances to guide network optimization. To support both single- and multi-entity actions, we propose a sub-entity strategy that offers a consistent formulation for both scenarios. Moreover, CHASE demonstrates compatibility with various intra-skeleton modalities, such as bones and velocities, highlighting its adaptability. Essentially, our method works as a normalization approach to reduce entity bias, enabling subsequent classifiers to achieve improved recognition performance across diverse settings. Extensive experiments on 7 datasets verify our approach by seamlessly integrating with various backbones and significantly boosting their performance.

0.57