CS Paper Radar

AnomaSense: Anomaly-based Sensor Activation for Fine-Grained Human Activity Recognition

Xue Wang, Yang Zhang

cs.HC cs.SD ·submitted 2026-09-24 02:38 UTC · v1

Abstract

Audio carries rich cues about human activities, and microphones are already built into most wearable devices. However, microphones also capture speech, and this privacy risk limits their use in Human Activity Recognition (HAR). We present AnomaSense, a sensor activation approach for wrist wearables that keeps the microphone off by default and turns it on for at most one second when an unsupervised anomaly detector flags an IMU segment that is likely to produce sound. The captured audio is further masked before it reaches the recognition model. We study 20 activities from 15 participants, organized into five groups in which activities share similar wrist motion but differ in the object or material involved. With IMU data alone, our recognition model reaches 78.98% accuracy in leave-one-participant-out validation. With the short, masked audio windows added, accuracy reaches 96.89% with no masking and stays above 86% when 90% of each one-second audio window is removed. On the same data, the anomaly detector triggers the microphone with 86.46% precision and 74.28% recall relative to sound events. We also report a small preliminary check of automatic speech recognition on masked speech, which shows that contiguous masking degrades recognition far more than point-wise masking at the same masking ratio. Our evaluation is a controlled, offline feasibility study. We describe the threat model, what the approach does and does not protect, and the steps needed before deployment.

Comment
12 pages, 6 figures

Closest papers

Nearest neighbours by meaning across the whole index.

Removing Speech, Keeping Activities: Speech Removal for Privacy-Preserving Acoustic Sensing in Assisted Living

cs.SD cs.CRcs.HC ·2026-09-02 ·Pavlos Nicolaou, Christos Efstratiou
similarity 0.62
Abstract · 44 pages, 2 figures, 6 tables. Submitted to Engineering Applications of Artificial Intelligence

Acoustic sensing can monitor the daily activities of older adults without cameras or wearables, yet the possibility of capturing private conversations remains a barrier to its deployment. The engineering application addressed in this paper is privacy-preserving acoustic monitoring of daily activities in assisted living. Our contribution in artificial intelligence is a speech-removal front end: a U-Net that masks the short-time Fourier transform of ambient audio and returns a waveform in which speech is suppressed and the sounds needed for activity recognition are kept. The model is trained only on synthetic mixtures of public sounds and speech, so no speech has to be recorded in the target homes. We evaluate it with group-disjoint cross-validation on environmental and domestic sound datasets mixed with reverberant speech, on speech from unseen corpora and on real home recordings, and measure privacy with three voice activity detectors, Whisper speech recognition and a speaker-verification attack. With speech 5 dB above the background, the macro-averaged F1 score of activity recognition is 0.574 and 0.661 after removal on the two datasets, compared with 0.233 and 0.178 without processing, and higher than that of speech enhancement, speech separation and detector-gated muting at every speech level. Whisper recovers 5.2% of the spoken words, compared with 89.0% without processing and a chance floor of 2.2%. On the real recordings, the speech proportion estimated by a voice activity detector falls from 8.5% to 2.7%. The method reduces, but does not eliminate, spoken content and speaker identity.

0.60

Foundation models for movement data: Are they ready for prime-time?

eess.SP cs.LG ·2026-08-13 ·Alexander Bräuer, Benjamin Cauchi, Nils Strodthoff
code/data similarity 0.60
Abstract · 14 pages, 6 figures, 8 tables, code is available at https://github.com/AI4HealthUOL/movement-fm-benchmarking

Foundation models (FMs) trained on large-scale accelerometer data have been proposed as general-purpose feature extractors for health monitoring, but systematic evidence of their advantages is lacking. We present the first comprehensive evaluation of four open-source accelerometer FMs against supervised baselines covering 19 tasks across the domains of activity recognition including activities of daily living, clinical monitoring, and physiological inference. We find task-dependent performance results: supervised models remain competitive with FMs on human action recognition (HAR), with no consistent advantage for either, while selected FMs lead on fall and stress detection and are the most robust to sensor-placement variation. As frozen feature extractors, FMs are strongest for demographic inference, whereas sleep staging performance remains near chance level for all models. The internal FM representations show strong similarity across layers, highlighting potential for future FM improvements. Linear and frozen probing reveals that UniMTS provides the strongest representations and is the only FM that surpasses the supervised baselines without finetuning. Concept discovery analysis shows all models capture high-intensity activities clearly but struggle with sedentary, complex or ambiguous activities. We provide scenario-based deployment recommendations. Furthermore, we identify FM-derived activity profile inference-moving beyond fixed category classification-as a promising research direction.

0.30

When Temporal Perturbations Act Like Sensor Biases: Label-Free Auditing of Wearable Activity Recognizers

cs.LG cs.AIcs.CR ·2026-09-24 ·Qingyu Wu, Yuan Wei, Renju Liu, Hua Cheng
similarity 0.60
Abstract

Wearable human-activity recognition (HAR) models operate across sensors, subjects, and backbones, yet a smooth waveform may appear temporal while exploiting a persistent sensor offset primarily. We introduce SpectrumAudit, a label-sealed audit that fits a phase-randomized full-window stimulus on calibration windows from subjects held out from training and testing. After selection, it replays its exact DC projection and budget-constrained zero-mean residual on the same frozen victim without refitting. Across 27 victims from three datasets and three backbones, the selected waveforms cause 2.87-40.83-point three-phase robust accuracy losses. Under this replay budget, DC is more damaging than AC on 24/27 victims and recovers at least 90% of the full drop on 22/27; all 5 failures occur on WISDM. In a held-out UTD-MHAD check, the selected waveform causes 13.49-pp accuracy and 11.68-pp macro-F1 losses, versus -0.66 pp for matched random changes. The audit diagnoses offset versus zero-mean variation under a common peak-budget cap. The code will be released upon acceptance.

0.34

Coverage-Aware Virtual IMU Augmentation for Low-Resource Human Activity Recognition

cs.AI ·2026-09-15 ·Jiayuan Gao, Yingwei Zhang, Ziyao Tang, Yuejia Ma, Yuanzhe Chen +2 more
similarity 0.59
Abstract

IMU-based human activity recognition (HAR) enables continuous, privacy-friendly monitoring of daily activities using wearable sensors. However, building reliable HAR models that generalize across diverse users and real-world conditions requires large amounts of labeled IMU data, which are expensive and difficult to collect. Existing approaches mainly rely on augmentation or synthesis to expand available data, but indiscriminately adding virtual samples may provide little new coverage and introduce unreliable supervision. To overcome these challenges, we propose a novel coverage-aware virtual IMU augmentation framework that decides where to supplement real data, how to generate and select virtual candidates, and how strongly to weight them during training. Specifically, we select diversity and scarcity anchors in a learned sensor embedding space, convert anchor dynamics into prompts, and generate virtual IMU candidates for each anchor. We then rank candidates by a selection cost combining anchor proximity and label consistency, and incorporate the selected candidates into HAR training with reliability-based weights. Experiments on public HAR benchmarks show that our method consistently improves recognition performance over competitive baselines, and ablation studies confirm the effectiveness of the proposed framework design.

0.18

MacJEPA: Missingness-Robust Audio-Visual Recognition from Untrimmed Egocentric Videos

cs.CV ·2026-10-06 ·Souptik Sen, Zahra Ahmadi
similarity 0.59
Abstract

Audio-visual models improve egocentric action recognition by exploiting complementary cues, yet typically assume that both streams remain available at inference. Existing missing-modality methods operate on trimmed, single-event clips in which a stream is entirely present or absent, whereas real sensors fail and recover within long, untrimmed observations. We redefine egocentric modality missingness as temporally localized sensor outages within untrimmed, multi-event observations, with whole-clip absence as the limiting case. We introduce \textbf{MacJEPA}, a missing-modality-robust \textbf{Ma}sked-\textbf{c}ontext query \textbf{JEPA} that recognizes visual actions and acoustic events from supplied interval queries over audio-visual context. Window-local modality dropout simulates these sensor outages during training. MacJEPA further repurposes masking in JEPA from a self-supervised pretext into a supervised robustness objective, aligning masked and clean latent representations of both multimodal content tokens and the task-conditioned queries. All objectives are optimized jointly with recognition in a single stage, requiring no test-time adaptation. Across Epic-Kitchens-100 and Epic-Sounds, a single checkpoint remains competitive under complete input and consistently surpasses published missing-modality baselines when either the dominant or auxiliary stream is removed. MacJEPA thus unifies strong full-input recognition with temporal missing-modality robustness in a single model operating on untrimmed multi-event videos.

0.22

TransfHAR: Self-Supervised Wrist Representations for On-Demand Activity Recognition

cs.LG ·2026-08-16 ·Aidan Bradshaw, Riku Arakawa, Xin Liu, Karan Ahuja
top venue similarity 0.59
Abstract · Accepted to the ACM Symposium on User Interface Software and Technology (UIST '26)

Fine-grained wrist activity recognition can support applications such as procedural step guidance and context-aware assistance, yet acquiring labeled data for every new task, user, and activity granularity remains a bottleneck. We present TransfHAR, a self-supervised wrist IMU framework for on-demand, fine-grained activity recognition by learning transferable motion priors from global, unlabeled activities. We show that self-supervised pretraining on coarse wrist IMU activities (e.g., sitting, walking, exercise) learns motion structure rich enough to transfer to fine-grained manipulative, gestural, and procedural activities (e.g., snapping, stirring, waving) that are absent from pretraining. We implement TransfHAR as a real-time smartwatch application that lets users define and expand their own activity set for personalized recognition from only a few demonstrations. Across three offline cross-dataset evaluations, TransfHAR matches or exceeds fully supervised baselines that use complete label sets with equal or additional sensor channels, by 6.2 balanced-accuracy points on average. In an in-lab study with 10 participants each performing seven novel wrist activities, TransfHAR reaches 86.7% balanced accuracy across participants with five examples per class and 90.4% when updated from a single one-minute recording per class. These results indicate that broad self-supervised wrist pretraining provides an effective foundation for on-demand fine-grained activity recognition.

0.64

Rotation-Invariant Multi-IMU Activity Recognition under Independent Per-Location Orientation Shifts

cs.AI cs.LG ·2026-08-16 ·Seungyeol Baek, Yoonbyung Chai, Yonghyeon Lee, Sungjoon Choi, Sungho Suh
published similarity 0.58
Abstract · 6 pages, 2 figures. ACM International Symposium on Wearable Computing (ISWC) 2026

Human Activity Recognition (HAR) with self-administered wearables, such as at-home rehabilitation and exercise monitoring, often requires reattaching inertial measurement units (IMUs) across sessions. In multi-IMU settings, this can induce independent orientation offsets across body locations, a deployment shift that conventional scalar HAR models do not structurally handle. Existing remedies rely on rotation augmentation, whose robustness depends on sampled transformations, or calibration and orientationnormalization pipelines requiring additional reference-frame assumptions or explicit procedures. We present Truly Rotation-Invariant HAR (TRI-HAR), a rotation-invariant framework that makes robustness to independent per-location IMU orientation offsets a structural model property. TRI-HAR reshapes accelerometer and gyroscope streams into triaxial vectors, applies a shared SO(3)-equivariant backbone and invariant projection to each IMU location, and fuses the resulting invariant features for activity classification. Across four multi-IMU benchmarks, TRI-HAR preserves macro-F1 under fixed independent per-location SO(3) rotations and outperforms rotation-augmented baselines under this target shift without requiring rotational augmentation.

0.58

Why Personalization Matters: Cross-Subject Challenges in EMG-IMU-based HRI Activity Recognition

cs.RO ·2026-08-21 ·Ruan Rithelle Chagas de Faria Carminati, Giovanni Braglia, Luigi Biagiotti, Ronnier Frates Rohrich, Andre Schneider de Oliveira +2 more
publishedcode/data similarity 0.58
Abstract · Data collection was approved by the Federal University of Technology-Paraná Ethics Committee (CAAE 91430125.0.0000.0177). The MAGIC-HRI (Mul…

This paper investigates wearable-based recognition of human activities and gestures to support Human-Robot Interaction (HRI) in object-handover and assembly-like scenarios. Electromyography (EMG) and Inertial Measurement Unit (IMU) signals were collected using a Myo armband, culminating in a novel dataset introduced as MAGIC-HRI (Multimodal Activity, Gesture and Intention Collection) with a large taxonomy of 53 movement classes, including Brazilian Sign Language (LIBRAS) numbers (0-9), hand gestures, object/tool handover actions (pick up/give/hold), tool-manipulation tasks, and generic assembly/idle motions, collected from 11 participants with 10 samples per class (530 samples per participant). Signals are segmented by detecting muscle activation via an EMG energy envelope, then processed using sliding windows; time- and frequency-domain features are extracted. Multiple classical classifiers are tuned via cross-validated grid search, with Random Forest as the strongest baseline. A Leave-One-Subject-Out (LOSO) protocol reveals a large generalization gap, indicating substantial subject dependence. A personalized adaptation experiment suggests that injecting a small number of samples from a new user can markedly improve recognition. Overall, the study contributes a broad, HRI-driven multimodal dataset, a rigorous evaluation emphasizing generalization, and practical evidence that personalization is likely required for robust deployment in practical HRI.

0.33