CS Paper Radar

ArmPoser: Real-Time, Calibration-Free Arm Pose Estimation from Smartwatch IMU

Bishnu Dev, Vasco Xu, Xi-Aan Loh, Chenfeng Gao, Henry Hoffmann, Karan Ahuja

cs.CV cs.HC ·submitted 2026-09-08 14:33 UTC · v1
published

Abstract

Arm pose estimation enables applications in fitness, extended reality input, rehabilitation, and life logging. Prior smartwatch-based approaches rely on calibration poses and preprocessing pipelines that transform raw IMU measurements into standardized training formats. These steps hinder deployment in everyday settings and introduce errors due to imperfect calibration and sensor drift. We present ArmPoser, a calibration-free arm pose estimation system using a single smartwatch IMU. Our central contribution is training models directly in the reference frame native to consumer smartwatches, aligning learning with how IMU data is produced by deployed devices. By operating on device-native axes, ArmPoser removes the need for coordinate transformations, explicit alignment, and bone-offset calibration used in prior work. We further augment training with physically grounded variations in watch placement and arm morphology to account for user-specific variability. ArmPoser also includes a wear-configuration module that infers anterior or posterior forearm placement and crown orientation. We evaluate pose estimation on public benchmarks and on a 10-participant, 30-activity study using watchOS and Android smartwatches, where ArmPoser matches or exceeds calibrated baselines without any user calibration.

Closest papers

Nearest neighbours by meaning across the whole index.

CoWear: Towards Collaborative Wearable Inertial Localization with Multiple Smart Devices

cs.RO ·2026-10-14 ·Zhuoyuan She, Zhanpeng Zhang, Yapeng Chen, Jiale Wang, Xin Xie +1 more
dataset similarity 0.70
Abstract · 8 pages, 5 figures

Consumer smart devices observe the same human motion from different body locations, yet existing inertial-localization benchmarks rarely provide synchronized heterogeneous wearables with device-specific motion ground truth. We present CoWear, a dataset and reference framework for collaborative inertial localization using a smartphone, smartwatch, and smart glasses. The dataset contains over 30 participants, over 300 walking sessions, and more than 30 hours and 100 km of aggregate device trajectories, with each device individually tracked by motion capture. To enable collaboration across distinct body motions, we train role-specific estimators to predict the displacement of a common reference device over synchronized temporal windows. Their predictions are transformed to the world frame using gyroscope integration and fused according to learned uncertainty. Experiments using phone, glasses, and watch as reference targets show that multi-device fusion consistently improves both displacement and trajectory rollout accuracy over individual-device estimation. CoWear provides a dataset and reproducible baseline for studying collaborative inertial localization across heterogeneous consumer wearables.

0.26

Wear and Go: Calibration Free Sparse Inertial Motion Capture via Online Sensor Setup Estimation

cs.CV ·2026-11-17 ·Yuanshuo Tan, Licheng Zhong
similarity 0.67
Abstract

Sparse inertial motion capture typically assumes that each sensor's anatomical identity and mounting orientation are known. We present MOSAIC (Motion-based Sensor Assignment and Inertial Calibration), a framework that recovers this setup from ordinary motion without a calibration pose. Our central insight is that body pose changes continuously while the sensor setup persists, allowing movements that are individually ambiguous to provide complementary evidence over time. A setup network processes unordered sensor streams with temporal and cross-sensor attention, then accumulates assignment and rotation evidence into a session-level estimate. The recovered setup canonicalizes measurements for frozen pretrained pose estimators, separating sensor configuration from motion reconstruction. For sensors worn at prescribed body sites, evaluation on real inertial recordings with randomized setups shows near-calibrated pose accuracy for bounded offsets and substantial benefits from accumulation under arbitrary orientations. Transfer across pose networks supports setup recovery as a reusable interface between interchangeable sensors and pretrained pose models.

0.18

Rotation-Invariant Multi-IMU Activity Recognition under Independent Per-Location Orientation Shifts

cs.AI cs.LG ·2026-08-16 ·Seungyeol Baek, Yoonbyung Chai, Yonghyeon Lee, Sungjoon Choi, Sungho Suh
published similarity 0.67
Abstract · 6 pages, 2 figures. ACM International Symposium on Wearable Computing (ISWC) 2026

Human Activity Recognition (HAR) with self-administered wearables, such as at-home rehabilitation and exercise monitoring, often requires reattaching inertial measurement units (IMUs) across sessions. In multi-IMU settings, this can induce independent orientation offsets across body locations, a deployment shift that conventional scalar HAR models do not structurally handle. Existing remedies rely on rotation augmentation, whose robustness depends on sampled transformations, or calibration and orientationnormalization pipelines requiring additional reference-frame assumptions or explicit procedures. We present Truly Rotation-Invariant HAR (TRI-HAR), a rotation-invariant framework that makes robustness to independent per-location IMU orientation offsets a structural model property. TRI-HAR reshapes accelerometer and gyroscope streams into triaxial vectors, applies a shared SO(3)-equivariant backbone and invariant projection to each IMU location, and fuses the resulting invariant features for activity classification. Across four multi-IMU benchmarks, TRI-HAR preserves macro-F1 under fixed independent per-location SO(3) rotations and outperforms rotation-augmented baselines under this target shift without requiring rotational augmentation.

0.58

REFIT: Recognize, Fix, and Test Wearable Sensor Placement Shifts without Labels

cs.LG cs.HC ·2026-10-06 ·Bangxun Tang
similarity 0.66
Abstract · 25 pages, 5 figures, 14 tables. Under review

We present REFIT, an input calibration for frozen activity-recognition models whose inertial sensors are worn differently at deployment than in training. When users move a watch to the other wrist or put a strap sensor back on turned, the model sees the same motion on changed axes. REFIT undoes such shifts without labels or retraining. It describes them by families of axis transforms, such as reflections and rotations, and fits each family to the user's data so that simple statistics match those of the training data. The family that removes most of the mismatch names the shift. REFIT fixes the shift by applying the best member of that family before the frozen model and re-estimating its normalization statistics. It tests the fixed model with a label-free accuracy estimate and asks the user to re-wear the sensor when it is low. Experiments on real left/right sensor pairs and on real and simulated re-attachment show that REFIT outperforms label-free test-time adaptation methods on every dataset and restores most of the accuracy lost to re-attachment. It names injected shifts far more reliably than a confidence-based selector. After a correction over all signed permutations of the axes, the estimate separates successful from failed corrections.

0.24

IMU-Bench: Benchmarking Wrist IMU Representations Across Kinematic, Semantic and Clinical Tasks

cs.RO cs.AI ·2026-10-14 ·Yiquan Li, Aidan Bradshaw, Chenfeng Gao, Karan Ahuja
benchmarkcode/data similarity 0.66
Abstract

Wrist-worn inertial measurement units (IMUs) provide a common sensing modality for tasks ranging from activity and gesture recognition to pose estimation, inertial odometry and clinical motor assessment. Existing wearable benchmarks evaluate performance across multiple tasks, but leave open how supervision for one task affects transfer to others and whether these tasks benefit from a shared representation. We present IMU-Bench, a benchmark of ten tasks from eight datasets for studying IMU multi-task performance and cross-task transfer. We compare six self-supervised methods pretrained on unlabeled recorded and simulated wrist motion, evaluating classification, pose and velocity estimation, and IMU-language retrieval. We find that our masked autoencoder, InrtlWrist, achieves the best or joint-best frozen-probe performance on nine of ten tasks, and fine-tuning improves on training from scratch on nine of ten tasks, including full-body pose, odometry and retrieval. Across transfer and joint-training experiments, supervision that benefits one target can reduce performance on others, with supervised sources transferring within kinematic estimation while, under frozen probing, none approaches the pretrained encoder on language retrieval, and joint-training gains that come at a cost to the paired task. These findings distinguish broad downstream usefulness from complementary task supervision. Code, task cards and a leaderboard are publicly available.

0.17

TransfHAR: Self-Supervised Wrist Representations for On-Demand Activity Recognition

cs.LG ·2026-08-16 ·Aidan Bradshaw, Riku Arakawa, Xin Liu, Karan Ahuja
top venue similarity 0.64
Abstract · Accepted to the ACM Symposium on User Interface Software and Technology (UIST '26)

Fine-grained wrist activity recognition can support applications such as procedural step guidance and context-aware assistance, yet acquiring labeled data for every new task, user, and activity granularity remains a bottleneck. We present TransfHAR, a self-supervised wrist IMU framework for on-demand, fine-grained activity recognition by learning transferable motion priors from global, unlabeled activities. We show that self-supervised pretraining on coarse wrist IMU activities (e.g., sitting, walking, exercise) learns motion structure rich enough to transfer to fine-grained manipulative, gestural, and procedural activities (e.g., snapping, stirring, waving) that are absent from pretraining. We implement TransfHAR as a real-time smartwatch application that lets users define and expand their own activity set for personalized recognition from only a few demonstrations. Across three offline cross-dataset evaluations, TransfHAR matches or exceeds fully supervised baselines that use complete label sets with equal or additional sensor channels, by 6.2 balanced-accuracy points on average. In an in-lab study with 10 participants each performing seven novel wrist activities, TransfHAR reaches 86.7% balanced accuracy across participants with five examples per class and 90.4% when updated from a single one-minute recording per class. These results indicate that broad self-supervised wrist pretraining provides an effective foundation for on-demand fine-grained activity recognition.

0.64

HeadIO: Body-Frame Heading Decomposition for Neural Inertial Odometry on Head-Mounted Devices

cs.CV cs.RO ·2026-11-11 ·Taeyoung Yeon, Yiquan Li, Chenfeng Gao, Vasco Xu, Karan Ahuja
benchmark similarity 0.64
Abstract · Submitted to ICRA 2027

Inertial odometry (IO) using only Inertial Measurement Units (IMUs) enables camera-free head tracking in head-mounted devices such as VR headsets, AR glasses, and earbuds. Prior IO methods operate in a gravity-aligned world frame, which is not yaw-equivariant. Yaw augmentation improves robustness but does not guarantee equivariance; learned equivariant layers achieve it, but EqNIO's full pipeline costs nearly 3$\times$ the FLOPs of ours. While raw body-frame IMU inputs are inherently yaw-invariant, head tilt redistributes both gravity and motion signals across device axes, making the representation tilt-variant. We propose HeadIO, which combines a body-frame heading decomposition with a bidirectional Mamba backbone for head-mounted inertial odometry. Our decomposition achieves tilt invariance by separating gravity-aligned and horizontal components using only the tilt component of device orientation. We evaluate on four head-mounted datasets and find that HeadIO improves RTE-5s by 8.1% (0.260 m $\to$ 0.239 m) over world-frame methods with learned equivariant layers on Nymeria, the largest benchmark dataset, at nearly 3$\times$ lower FLOPs and 8$\times$ fewer parameters. HeadIO achieves state-of-the-art performance on our primary metric, RTE-5s, on all Aria-glasses datasets, which natively provide body-frame semantics for our decomposition method. We further evaluate the IMU-only deployment scenario on Nymeria, where HeadIO remains state-of-the-art. We show that our method relies mainly on tilt accuracy, which can be obtained from the gravity signal; for yaw, a single-coefficient gyroscope correction removes 11.6% of the yaw error and improves RTE-5s by 2.5%, and sparse external yaw corrections (for example, one every 30 s) can improve RTE-5s by 12.0%. Our decomposition is lightweight, composes with any backbone, and is suitable for real-time deployment on head-mounted devices.

0.18

WearableBridge: Task-Guided Latent Transport for Free-Living Activity Recognition

cs.LG ·2026-11-06 ·Jingjing Zou, Yixin Wang
code/data similarity 0.64
Abstract · 26 pages, 5 figures, 11 tables. Code and interactive demo: https://github.com/jingjingz/WearableBridge

Wrist-worn accelerometers make activity monitoring feasible at scale, but incidental arm motion, device orientation, and shifting context blur the signal that separates activities. Calibration studies supply higher-quality references such as concurrent thigh recordings or supervised laboratory wrist sessions, but they are expensive, cover few participants, and are rarely available at deployment. How can limited reference data improve recognition for participants who wear only a wrist device? We propose WearableBridge, which holds the reference encoder and classifier fixed and learns a task-guided transport map, the bridge, that carries each wrist feature into the region the classifier associates with its activity. Reference features or the classifier's predictions indicate the destination, and a classification loss penalizes wrist features that land in the region of another activity. We instantiate WearableBridge with three bridges: Mapper regresses onto reference features, Distillation matches the classifier's class probabilities, and MeanFlow learns a one-step flow from wrist toward reference features. On 40 held-out free-living PAAWS participants, all three bridges improve macro-F1 in every training run, and the strongest, MeanFlow, raises mean macro-F1 from 35.1% to 56.0% with concurrent thigh references (Thigh2Wrist), and from 54.0% to 63.5% with group-matched laboratory wrist references (Lab2Freeliving). Fitted only on PAAWS, MeanFlow also raises macro-F1 on the CAPTURE-24 cohort from 42.8% to 54.7%. The gains replicate on selfBACK, hold across four reference encoders, and grow with temporal context. A study can therefore collect reference recordings for a calibration subset and improve wrist-only recognition for every other participant, training only a bridge of 0.13 to 2.44 million parameters while the reference encoder and classifier stay unchanged.

0.29