CS Paper Radar

V-JEPA 2 Meets AQA: Phase-Based Scoring and Reference-Based Feedback for Mountain Biking

Santhiya Rajan, Savio Cardoz

cs.CV cs.AI ·submitted 2026-09-27 18:26 UTC · v1

Abstract

We study mountain-bike action quality assessment (AQA) with weak labels and candidate feedback. The training corpus comprises 95 field-recorded jump clips with binary labels denoting good or poor technique. These are coarse categorical labels, not continuous expert quality scores; phase-boundary and aspect-level annotations are unavailable. Our pipeline summarizes four heuristic phases using frozen V-JEPA 2 video features and two-dimensional pose and motion measurements. A small transformer predicts a binary-label score. A separate reference layer compares phase-specific measurements with those of clips labelled as good technique to propose phase, body-parameter, and direction cues. We detail the measurements, network, and evaluation so that the limits of each output are explicit. In five-fold clip-level cross-validation, the full scorer obtained pooled out-of-fold AUROC 0.768 and average precision 0.812. These are exploratory figures because the reporting fold also selected the neural training epoch and riders were not separated between folds. Removing biomechanical scoring inputs raised AUROC to 0.809. A separate inner-selection run obtained AUROC 0.678, but used about 57 rather than 76 training clips per outer fold, so its difference from the test-selected run does not quantify selection bias. The unsupervised aspect head did not learn distinct factors. On a 17-clip external probe, an earlier checkpoint obtained binary accuracy 0.412 and Spearman association -0.135 with expert scores. Candidate cues had precision 0.118 and recall 1.0 against each of two coaches, with substantial disagreement about direction. These findings do not establish rider-independent scoring or reliable corrective feedback; they identify the empirical work needed to evaluate either use.

Comment
27 pages, 10 figures

Closest papers

Nearest neighbours by meaning across the whole index.

Autonomous Telerehabilitation via Skeletal Motion Prediction and Joint-Level Performance Assessment

cs.CV cs.LG ·2026-08-12 ·Lara Pereira, João Ruivo Paulo, Pedro Santos, Paulo Peixoto
published similarity 0.60
Abstract · Accepted at IEEE RO-MAN2026

Autonomous rehabilitation systems must not only recognize human motion but also provide structured feedback to support users without continuous therapist supervision. This paper presents a telerehabilitation pipeline that integrates skeleton-based exercise quality assessment and short-term motion prediction into a two-module system operating on marker-free RGB video. A self-attentive Bidirectional LSTM performs exercise quality classification using MMD-NCA metric learning, while a graph-based motion prediction module computes per-joint position errors between predicted and observed poses, generating spatially localized deviation signals. Each module is evaluated independently on established benchmarks: the classifier achieves 96.45% mean-class accuracy on squat sequences from the PROZIS dataset, and the adopted STARS predictor achieves a mean MPJPE of 75.8 mm at 560 ms on Human3.6M, outperforming graph and recurrent baselines across all prediction horizons. The framework is designed for eventual deployment in assistive robotics and home-based rehabilitation contexts; end-to-end integration and clinical validation are important directions for future work. By combining motion recognition and prediction in a single system, this work contributes a step toward autonomous, feedback-driven telerehabilitation, for more accessible and scalable rehabilitation solutions.

0.23

Can Vision-Language Models Judge Olympic Diving? From Reasoning to Scores in Zero-Shot Action Quality Assessment

cs.CV cs.AIcs.LG ·2026-09-16 ·Henry O. Velesaca, David Freire-Obregon, Luigi Miranda, Abel Reyes-Angulo
publishedbenchmarkcode/data similarity 0.59
Abstract

Automated action quality assessment (AQA) in Olympic sports remains a challenging task due to the complexity of human motion and the subjectivity inherent in expert judging. This work evaluates the capability of open-source Vision-Language Models (VLMs) to perform zero-shot action quality assessment on Olympic diving videos using the AQA-7 benchmark dataset. In this regard, a regression-based framework is pro-posed to leverage both the semantic reasoning and phase-level sub-scores generated by the VLMs, combining TF-IDF vectorization, dimensionality reduction, and ensemble learning to predict final competition scores. Experimental results show that standalone VLMs achieve moderate Spearman correlations below 0.32, while the proposed ensemble regression framework substantially improves performance in the reported evaluation, reaching a Spearman correlation of 0.67 with a four-model configuration. Textual reasoning features con-sistently outperformed raw numerical sub-scores, highlighting the richness of VLM-generated explanations for action quality analysis. These findings suggest that VLMs hold strong potential as assistive tools for explainable and semi-automated sports performance evaluation. The code is publicly available on GitHub https://github.com/hvelesaca/olympic diving judge vlm

0.46

Autonomous Trail Racing with a Bicycle Robot

cs.RO ·2026-11-11 ·Fabian Jenelten, Deepana Ishtaweera, Pablo Garcia, Liana Margolese, Igor Bogoslavskyi +8 more
code/data similarity 0.59
Abstract · 35 pages, 10 figures, 3 tables (incl. appendix with supplementary figures and tables). Submitted to The International Journal of Robotics Re…

Navigating off-road trails on a mountain bike requires diverse locomotion skills, including aerial maneuvers, cornering, and slip recovery, paired with sophisticated perception for scene understanding. Translating these capabilities to autonomous, two-wheeled robots remains a formidable challenge. We show that autonomous, high-speed trail navigation can be achieved through a single idea: encoding the trail as a semantic layer of the terrain. In simulation, this layer embeds the navigation task into the training environment; during deployment, it is reconstructed onboard from camera and LiDAR data. First, we introduce a unified learning framework that treats local navigation and locomotion as a joint optimization problem, bypassing the twist or pose command interface of classical control hierarchies. With fixed reward weights, the same framework produces both trail racing and agile parkour, shaped only through terrain semantics and episode length. Second, our perception pipeline combines a frozen vision foundation model with a lightweight dual-head decoder that predicts per-pixel terrain semantics and confidence scores, which are fused with LiDAR range measurements into an egocentric semantic elevation map at 30 Hz onboard an NVIDIA AGX Orin. We validate our framework on the Ultra Mobility Vehicle (UMV), a bicycle robot that autonomously raced across all sections of a kilometer-long, unstructured mountain bike trail at speeds exceeding 5 m/s and averaging 3.26 m/s. In simulation, our policy fails up to an order of magnitude less often than hierarchical baselines while driving faster.

0.41

Ampe-DB: A Dataset for Human Activity and Interaction Analysis Using the Ghanaian Game Ampe

cs.CV cs.LG ·2026-10-23 ·Ruth Senior Biney, Isaac Kojo Yedonu Aboah
benchmarkdatasetcode/data similarity 0.58
Abstract · 74 pages, 9 figures, 7 numbered tables. Ampe-DB V1 dataset (637 interaction clips) available at https://huggingface.co/datasets/RuthBiney/Am…

Human activity recognition benchmarks underrepresent culturally situated forms of movement and interaction. We introduce Ampe-DB, a multimodal dataset based on Ampe, a traditional Ghanaian game involving coordinated jumping, leg movements, and two-person interaction. It contains 637 canonical clips from 15 participants, with RGB video, two-person pose representations, participant and pair metadata, four-class movement labels, and derived binary movement-relation labels. We evaluate four-class movement recognition and binary movement-relation classification using three protocols: a duplicate-safe fixed benchmark, four-fold pair-held-out evaluation, and participant-held-out diagnostics for participants with complete class coverage. Baselines comprise a frozen Kinetics-400-pretrained R(2+1)D-18 RGB encoder, a two-layer bidirectional GRU for pose sequences, and validation-selected RGB-pose late fusion. For four-class recognition, RGB macro-F1 falls from 0.955 on the fixed benchmark to 0.418 under pair-held-out evaluation and 0.384 under participant-held-out evaluation. Pose is more robust, achieving 0.957, 0.811, and 0.839 across the same protocols. Fusion reaches 0.976 on the fixed benchmark and 0.824 under pair-held-out evaluation, but does not consistently improve on pose under participant shift. Binary movement-relation recognition also becomes substantially harder under held-out evaluation, especially for different-leg configurations. These findings show that strong fixed-benchmark scores can mask failures to generalize across people and pairings. Ampe-DB supports research on fine-grained movement recognition, human-human interaction, multimodal learning, and identity-aware generalization.

0.33

Throwing4: Phase-Aware Pose Modeling for Throw-Distance Prediction in Elite Athletics

cs.CV ·2026-10-12 ·Bikash Kumar Badatya, Kartike Tiwari, Jyotirmoy Amin, Ravi Hegde
published similarity 0.58
Abstract · Accepted for publication in Sports Engineering

Elite athletics throwing provides a structured biomechanical action sequence together with an objective continuous outcome, namely throw distance, making it a suitable setting for performance-oriented sports analytics. In this work, we investigate whether explicit phase alignment of pose dynamics improves throw-distance prediction under realistic broadcast conditions. To support this study, we curate Throwing4, a dataset of 500 elite throwing clips spanning javelin, discus, hammer, and shot put, with corresponding two-dimensional pose sequences, frame-level phase annotations, and official distance labels. We formulate throw-distance prediction from broadcast-derived pose as a regression problem and compare full-sequence modeling against a phase-conditioned approach that partitions each attempt into four technique stages and fuses phase-specific motion representations. Experiments across multiple graph-based pose encoders show a consistent reduction in prediction error when phase structure is incorporated, indicating that phase-aligned modeling provides a more stable representation than treating the entire motion as an unstructured sequence. These findings suggest that explicit biomechanical phase information is a useful inductive prior for pose-based performance modeling in elite athletics throwing. Beyond score prediction, the dataset and protocol provide a basis for future work on fine-grained technique analysis, interpretable sports analytics, and robust modeling from broadcast video.

0.31

Toward Postural State Classification in Immersive VR with Multimodal Data and Explainability Analysis

cs.HC cs.AIcs.LG ·2026-08-28 ·Nipa Anjum, Md Irfan Pavel, Robert Gonzalez, Kevin Desai, Alberto Cordova +2 more
top venuecode/data similarity 0.57
Abstract · Accepted in The 25th IEEE International Symposium on Mixed and Augmented Reality (ISMAR)

Ensuring a safe virtual reality (VR) experience requires systems that can predict and respond when users lose their balance. Although prior work has examined fall prediction and motion sickness, many approaches are regression-based and postural state classification remains less explored. This study compares machine learning (ML) and deep learning (DL) models for classifying postural states in VR under visual perturbations. We used a multimodal dataset containing kinematic, electromyographic (EMG), and electrodermal activity (EDA) signals. The data were prepared for a binary task to distinguish balanced from imbalanced postural states, and participant-wise downsampling addressed class imbalance. All models were evaluated with Leave-One-Participant-Out (LOPO) cross-validation to test generalization to unseen participants. Among the models, the Mamba-inspired CNN (MI-CNN) achieved the highest accuracy of 96.76%. SHapley Additive exPlanations (SHAP) analysis improved interpretability and identified the most influential classification factors. The SHAP results showed that kinematic features were dominant, indicating that body-motion patterns are informative for detecting imbalance in VR. We also evaluated MI-CNN using only the top two-thirds of features ranked by SHAP importance. Despite a 33% reduction in input dimensionality, the model maintained performance, achieving 0.957 accuracy and 0.957 F1-score, with about a 1% decrease compared with the full-feature model. These findings suggest that multimodal sensing, temporal deep learning, and explainable AI can support reliable classification of balance-related instability in VR. Accurate recognition of imbalanced postural states may raise awareness of fall risk and guide safer, adaptive VR systems that respond to instability while improving user safety and experience. Code is available at: https://github.com/NipaAnjum/MI-CNN.

0.45

3rd Place Solution to Human Motion Challenges in Real-World and Clinical Settings (MoCha) @ECCV2026: Language-Aligned Motion Representations for Domain-Generalizable UPDRS-Gait Severity Estimation

cs.CV ·2026-09-09 ·Soojie Kim, Muhammad Munsif, Minkyung Kim, Seungryul Baek
publishedbenchmark similarity 0.57
Abstract · 3rd Place Solution to the MoCha 2026 Challenge at ECCV 2026

In this work, we introduce language-aligned motion representations for domain-generalizable UPDRS-Gait severity estimation, aiming to learn semantically structured motion features that generalize across heterogeneous clinical domains. We first learn motion representations using a Bi-GRU backbone that captures the temporal dynamics of SMPL sequences. Prior to model training, motion captions are generated offline using Qwen2.5-7B-Instruct. The backbone is then trained with both classification and text-alignment objectives to learn discriminative and semantically structured motion representations while accounting for the class imbalance present in the training data. We subsequently adapt the learned backbone independently to each source domain so that the model can capture domain-specific motion characteristics. The resulting source-specific models are then merged at the parameter level to consolidate complementary knowledge across source domains into a single domain-generalized model. To further mitigate class imbalance, we perform GPT-5.5-based pseudo labeling, and our final merged models for each site do not use any class-prior correction during inference. The resulting model is evaluated under the unseen-site setting of the MoCha Challenge, using Macro F1 as the primary evaluation metric. Our method achieves a macro-F1 of 0.57 on the hidden test set with only 637K active parameters at inference, ranking 3rd among 58 leaderboard entries in the MoCha 2026 Challenge. The challenge attracted 1,669 submissions from 112 participants and offered monetary prizes sponsored by Machine Medicine Technologies.

0.22

Orthogonal Ensembles and Tested Explanations for Performer-Independent Body-Motion Emotion Recognition

cs.CV cs.HCcs.LG ·2026-09-02 ·Naoto Nishida, Yoshio Ishiguro
publishedcode/data similarity 0.57
Abstract · 8 pages, 3 figures, 3 tables. Accepted to ACII2026 workshop.received the Best Performance Award of the MMAC Challenge 2026

We study body-only, 12-class acted-emotion classification from skeleton motion under leave-performer-out (LPO) evaluation, a hard, underdetermined setting: chance is 8.3%, and a protocol-matched reproduced STGCN++ baseline reaches only 25.73 +/- 4.03% Macro-F1. We show that reliable gains come not from a new architecture but from combining eleven models with orthogonal error modes: under 10-fold LPO cross-validation on the labeled training performers, an equal-weight logit-mean ensemble reaches 36.80 +/- 4.00% per-fold Macro-F1, a protocol-matched +11.07 pp (+43% relative) over the same-split reproduced baseline. Our central contribution is a tested explanation suite: for a strong ensemble member, part-masking and counterfactual edits show (rather than assert) that its decisions depend on motion-grounded body-region evidence, and this region saliency aligns with rule-based Laban Movement Analysis (LMA) attributes far more than with classical kinematics: region-level saliency-LMA Spearman rho = +0.500 versus +0.033, roughly 15x, and the alignment holds for the submitted 11-way ensemble itself at rho = +0.517; the audit is post hoc and needs no retraining. The same suite faithfully reports a negative: within-window temporal saliency is diffuse rather than localized. On the hidden challenge test set the submitted ensemble scored 37.23 % Macro-F1 and received the Best Performance Award of the MMAC Challenge 2026 (Human score is 39 %). Code is available at https://github.com/nawta/diema-challenge and the presentation at https://nawta.github.io/mmac2026/.

0.47