CS Paper Radar

Supervised Cross-Modal Feature Alignment for Zero-Wearable Freezing of Gait Detection in Parkinsonism

Aryan Singh, Chandan Biswas

cs.CV ·submitted 2026-09-08 06:40 UTC · v1

Abstract

Objective assessment of Freezing of Gait (FoG) in Parkinson's disease (PD) relies predominantly on wearable Inertial Measurement Units (IMUs). While IMUs provide optimal kinematic precision, mandatory sensor attachment restricts continuous clinical deployment. Conversely, unobtrusive vision-based alternatives suffer substantial classification errors during turning-in-place tasks, where geometric self-occlusion degrades deterministic skeletal coordinates and obscures the high-frequency precursors required for FoG detection. To resolve these physical observation limits, we propose a supervised cross-modal subspace distillation framework. During optimisation, pre-trained kinematic data from IMU sensors and contextual clinical metadata act as oracles to guide a deployable visual architecture. By incorporating joint velocity and acceleration derivatives, utilising a confidence-based gating mechanism, the visual model mitigates some of the tracking errors during occlusion events. Empirical evaluations confirm this latent alignment transfers the predictive fidelity of hardware sensors directly into the visual representation, yielding $85.5\%$ accuracy, and $82.4\%$ balanced accuracy. All the while maintaining a vision only model at inference.

Comment
10 pages, 4 figures

Closest papers

Nearest neighbours by meaning across the whole index.

Freezing of Gait Prediction Under Spatial Occlusion: An IMU-Supervised Cross-Modal Distillation Approach

cs.CV ·2026-09-09 ·Chandan Biswas, Aryan Singh, Anabik Pal
hot direction similarity 0.87
Abstract · 9 pages , 2 figures

Parkinson's disease is a progressive neurodegenerative disorder characterised by gradual deterioration of movement control. Automated freezing-of-gait (FOG) detection supports the objective assessment of gait-related motor impairment. Two common approaches are used for FOG prediction: (i) analysing video recordings of the patient's movements and (ii) analysing data collected using inertial measurement unit (IMU) wearable sensors attached to the patient's lower limbs. Video-based approaches may suffer detection errors during continuous turning-in-place tasks because the lower limbs undergo substantial geometric self-occlusion, degrading pose-estimation accuracy. IMU-based approaches are generally less affected by visual occlusion; however, they are difficult to deploy outside clinical or laboratory settings, as the sensors must be attached securely and remain in place throughout the assessment. Motivated by this, we propose a cross-modal subspace distillation framework to mitigate the limitations of unimodal FOG detection by combining IMU accuracy with video-based practicality. We extract invariant latent topologies from a pre-trained kinematic oracle to structurally supervise a non-encoded visual architecture during training. To resolve periods of severe spatial occlusion, a dual-stream visual model probabilistically fuses skeletal graph nodes and continuous spatial pixels, dynamically shifting reliance to uninterrupted pixel boundaries as joint tracking confidence drops. Evaluated against a public, multi-modal sequence dataset of Parkinson's individuals executing continuous $360^\circ$ turns, empirical results demonstrate that applying sensory boundary topologies strictly mitigates tracking evaluation entropy. Our constrained optimisation confirms that highly precise FOG prediction bounds can be achieved over zero-wearable inference environments.

0.25

Towards Context-Aware Clinical Motion Understanding in Daily Living at Home: Freezing of Gait Detection with Egocentric Vision

cs.AI ·2026-08-13 ·Vayalet Stefanova, Diwas Lamsal, Margot Genbrugge, Maxim Yudayev, Christian Schlenstedt +3 more
published similarity 0.77
Abstract · Accepted to ECCV Workshop 2026 (Human Motion Challenges in Real-World and Clinical Settings)

Understanding motion in daily living requires context beyond kinematics, because similar inertial patterns during activities of daily living (ADLs) can reflect intentional stopping, object interaction, or pathological movement impairment. Egocentric vision provides task-related context that may help disambiguate these cases. We investigate this challenge through freezing of gait (FOG) detection in Parkinson's disease (PD), a symptom strongly influenced by contextual factors during ADLs. Using synchronized egocentric video, wearable IMUs, and expert-annotated FOG labels collected from 13 PD participants in their homes, we evaluate frozen representations from pretrained ego-video and time-series foundation models, alongside an IMU-based TCN trained from scratch, under leave-one-subject-out evaluation. The IMU-based TCN achieved the strongest event-detection performance, reaching 42.3 F1 and 83.0 AUROC, compared with 32.6 F1 and 77.2 AUROC for V-JEPA2 ego-video features. Although ego-video alone did not outperform IMU-based sensing, it showed above-chance discrimination, and qualitative analyses suggest that egocentric vision may capture FOG-relevant information independent of IMUs. Together, these results support the use of pretrained ego-video representations to add contextual information to wearable-sensor-based clinical motion understanding in daily living.

0.20

GaitFM: A Cross-Modal Foundation Model for Wearable Gait Assessment

cs.CV ·2026-11-01 ·Yize Cai, Zhiqing Hong
similarity 0.69
Abstract · 18 pages, 4 figures

Gait assessment supports the diagnosis, monitoring, and prognosis of diverse movement-related conditions, such as neurological and musculoskeletal disorders. Recent advances in deep learning using everyday wearable inertial measurement units (IMUs) offer an accessible and low-cost approach to automated gait assessment. However, IMU measurements vary substantially across individuals, body locations, and device configurations, while existing deep learning models are typically developed using small, homogeneous datasets for specific clinical tasks, limiting their robustness and generalizability in diverse real-world contexts. Here we present GaitFM, a cross-modal foundation model for gait assessment using IMU sensors. GaitFM improves the generalizability of gait assessment by transferring knowledge from large vision models and pretraining on unlabeled IMU sensor data. We pretrain GaitFM on a large curated dataset comprising about 4,000 participants and 1.23 million gait windows across diverse clinical and daily conditions, sensor modalities, and body locations. GaitFM demonstrates strong performance across 18 gait-related tasks, including 3 gait parameter estimation tasks, 14 disease and gait abnormality detection tasks, and 1 clinical outcome prediction task. Furthermore, GaitFM generalizes across different sensor configurations and body locations, including those commonly used in clinical settings and everyday wearable devices. These findings demonstrate the effectiveness of transferring visual knowledge in improving sensor-based gait assessment performance, offering a new promising pathway for addressing data scarcity in the sensor and medical domains.

0.13

From Visual Attribution to Clinical Reasoning: Explainable Parkinson's Disease Screening from Hand-Drawn Patterns

cs.CV cs.AI ·2026-09-13 ·Aritra Dey, Utsav Kumar Nareti, Chandranath Adak, Soumi Chattopadhyay, Krishna Gopal Sasmal +1 more
similarity 0.62
Abstract

Parkinson's disease (PD) manifests early neuromotor impairments that become observable in controlled hand-drawn patterns such as spirals and meanders, where tremor-induced oscillations, stroke irregularity, and curvature instability reflect underlying motor degradation. In this work, we present an explainable framework for PD screening from offline hand-drawn patterns that integrates discriminative visual modeling with clinically grounded reasoning. The predictive model captures distributed structural distortions and fine-grained texture variations. It is evaluated under subject-disjoint protocols to ensure reliable generalization. To move beyond black-box classification, we introduce a multi-stage explainability pipeline that combines visual attribution with structured symptom abstraction. Salient regions are identified using attention- and gradient-based localization, followed by extraction of clinically meaningful motor descriptors quantifying contour roughness, curvature irregularity, stroke variability, and tremor-frequency energy. These descriptors are subsequently translated into coherent clinical rationales through a language-based reasoning module, linking model evidence to established PD symptomatology. By bridging visual attribution and clinical interpretation, the proposed framework advances interpretable document intelligence for neurological screening using hand-drawn patterns. Experimental results on publicly available Parkinson's disease handwriting datasets demonstrate competitive predictive performance and clinically consistent explanations.

0.33

MMGait: Benchmarking and Unifying Gait Recognition across Heterogeneous Modalities

cs.CV ·2026-09-10 ·Saihui Hou, Chenye Wang, Qingyuan Cai, Aoqi Li, Yongzhen Huang
benchmark similarity 0.62
Abstract · 21 pages, 6 figures

Gait recognition is commonly studied using RGB videos or their derived silhouettes and poses. Yet human walking produces heterogeneous photometric, geometric, and motion cues that cannot be systematically examined with RGB-centered benchmarks. We present MMGait, a large-scale multi-sensor benchmark that brings visible, infrared, depth, LiDAR, and radar observations into sequence-level correspondence. It provides diverse modalities spanning appearance, contours, geometry, motion, and body structure. Under a shared impostor-augmented protocol, we evaluate single-modal recognition, cross-modal recognition via directed retrieval, and multi-modal recognition using task-specific experts. Across settings, modality rankings vary with probe conditions, cross-modal alignment remains difficult, and fusion often provides complementary gains. This analysis exposes a scalability problem: individual modalities, modality pairs, and fusion configurations are typically handled by separately trained experts. We formulate Omni-Modal Gait Recognition, which unifies single-modal, cross-modal, and multi-modal recognition within a shared identity space. OmniGait++ uses modality-specific front ends followed by a shared identity encoder to preserve modality-dependent cues while learning comparable identity descriptors. An anchor-guided fusion module aggregates modality subsets of varying size without frame-level synchronization. A jointly trained checkpoint covers all three recognition settings and accommodates modality subsets of different compositions and cardinalities. Experiments show OmniGait++ remains competitive with task-specific experts in many shared settings and extends to higher-cardinality fusion unavailable to fixed-pair models. The results establish MMGait as a common testbed for heterogeneous gait sensing and demonstrate the feasibility of unified recognition under varying modality availability.

0.30

SynthGait-19K: A Physically Grounded Synthetic Video Dataset for Gait Parameter Estimation

cs.CV ·2026-09-08 ·Soroush Mehraban, Xin Lei Lin, Vida Adeli, Majid Mirmehdi, Amirhossein Dadashzadeh +3 more
benchmarkcode/data similarity 0.61
Abstract · Project Page: https://soroushmehraban.github.io/SynthGait-19k/

Accurate estimation of clinically meaningful gait parameters from monocular video is important for scalable mobility assessment, yet progress is limited by the small scale, restricted viewpoints, and limited visual diversity of existing datasets. We introduce SynthGait-19k, a physically grounded synthetic video dataset containing 19,272 walking videos derived from 6,427 MoCap sequences across 437 subjects, with paired SMPL motion and annotations for six gait parameters. To construct the dataset, we develop Gait2Vid, which unifies heterogeneous MoCap recordings through SMPL and synthesizes diverse RGB walking videos under controllable viewpoints and scene appearances. We assess the generated videos for consistency with their conditioning gait kinematics and validate extracted gait events against force-platform measurements. Using SynthGait-19K, we benchmark direct RGB, pose-based, biomechanical, and human-mesh-recovery approaches and analyze viewpoint, training-data scale, and synthetic-to-real domain shift. We also introduce GaitXFormer as a direct RGB reference model for estimating gait parameters. Synthetic supervision transfers effectively to real videos across both GaitXFormer and a pose-based architecture, demonstrating utility across different representations. We further find that spatial gait parameters are more sensitive to visual domain shift and that improved HMR reconstruction alone does not necessarily translate to improved downstream gait estimation.

0.34

WaveFoG: Wavelet Gated Transformer for Parkinson's Freezing of Gait Detection Using Wearable Accelerometer Signals

eess.SP ·2026-08-26 ·Md Shihabul Islam Shovo, Nishi Kanta Paul, Adrita Rahman, Israt Jerin Esha
code/data similarity 0.61
Abstract

Freezing of gait (FoG) is one of the most debilitating episodic motor symptoms of Parkinson's disease (PD). FoG detection from wrist worn inertial signals poses challenging problems such as extreme data class imbalance (only 12-18% of signal windows are FoG), low frequency postural instabilities alongside diagnostically crucial 3-8 Hz rhythmic tremors, and the need for subject independent generalisation. To address these, this paper presents WaveFoG, a novel wavelet gated Transformer architecture that combines multi scale representations from discrete wavelet transform (DWT) with a dual branch encoder. A 1D CNN captures local gait texture, and a Transformer captures global temporal context. They are fused via a sigmoid gating fusion layer that modulates the encoder output based on the DWT sub-band descriptors. Trained with focal loss and evaluated on the public Kaggle TLVMC FoG Prediction dataset under subject independent grouped ten fold cross validation (SI-CV), WaveFoG achieves a mean F1-score of 0.875 +/- 0.017 and AUPRC of 0.833 +/- 0.020, exceeding single branch baselines (1D CNN, Bi-LSTM, and vanilla Transformer) by up to 5.4 pp on F1. Temporal saliency maps highlight temporal patterns that are consistent with reported characteristics of FoG onset. The source code of this project is available at: https://github.com/Shihabul-Shuvo/WaveFoG

0.33

IMU-Bench: Benchmarking Wrist IMU Representations Across Kinematic, Semantic and Clinical Tasks

cs.RO cs.AI ·2026-10-14 ·Yiquan Li, Aidan Bradshaw, Chenfeng Gao, Karan Ahuja
benchmarkcode/data similarity 0.61
Abstract

Wrist-worn inertial measurement units (IMUs) provide a common sensing modality for tasks ranging from activity and gesture recognition to pose estimation, inertial odometry and clinical motor assessment. Existing wearable benchmarks evaluate performance across multiple tasks, but leave open how supervision for one task affects transfer to others and whether these tasks benefit from a shared representation. We present IMU-Bench, a benchmark of ten tasks from eight datasets for studying IMU multi-task performance and cross-task transfer. We compare six self-supervised methods pretrained on unlabeled recorded and simulated wrist motion, evaluating classification, pose and velocity estimation, and IMU-language retrieval. We find that our masked autoencoder, InrtlWrist, achieves the best or joint-best frozen-probe performance on nine of ten tasks, and fine-tuning improves on training from scratch on nine of ten tasks, including full-body pose, odometry and retrieval. Across transfer and joint-training experiments, supervision that benefits one target can reduce performance on others, with supervised sources transferring within kinematic estimation while, under frozen probing, none approaches the pretrained encoder on language retrieval, and joint-training gains that come at a cost to the paired task. These findings distinguish broad downstream usefulness from complementary task supervision. Code, task cards and a leaderboard are publicly available.

0.17