CS Paper Radar

KOALA: Koopman Operator Learning for WiFi-Based Anticipatory Hum

Quang-Anh N. D., Duc Pham Minh, Thao Phuong Pham, Minh Anh Nguyen, Huan X. Nguyen, Tuan Dang

cs.LG ·submitted 2026-08-16 15:43 UTC · v1
top venue

Abstract

WiFi Channel State Information (CSI) has emerged as a privacy-preserving alternative to cameras for human pose estimation. However, existing approaches treat pose inference as an instantaneous regression problem and do not model temporal dynamics, making future motion prediction infeasible. Naively applying vision-based prediction methods compounds the estimation noise already present in CSI-derived poses, as autoregressive rollouts amplify errors at every step. We propose KOALA, the framework for human motion prediction directly from WiFi CSI, by lifting noisy CSI-derived pose sequences into a learned Koopman latent space where nonlinear dynamics become linear, enabling multi-horizon prediction via simple matrix-vector products without autoregressive iteration or error accumulation. A residual CSI-conditioned operator resolves the identity attractor problem inherent from Koopman formulations, and an anchor-delta prediction head eliminates the degenerate shortcut of copying the current pose across all horizons. To regularise the lifting and operator jointly, we introduce a Koopman Anchored Latent (KAL) loss that operates in the temporal-encoder feature space, enforcing dynamical consistency across prediction horizons without requiring contrastive, spectral, or auxiliary losses. Experiments on MM-Fi and WiPose show that KOALA achieves robust, consistent performance across both short- and long-term prediction horizons, outperforming all baselines by a substantial margin.

Comment
27 pages, 3 figures
Journal ref
Transactions on Machine Learning Research (TMLR 2026)

Closest papers

Nearest neighbours by meaning across the whole index.

WiSPER: Pose-Supervised Predictive and Residual Flow Refinement For Multi-Person 3D Pose Estimation With WiFi CSI

cs.CV ·2026-10-04 ·Gabriel Lee Jun Rong, Shanhong Liu, Pai Chet Ng, Konstantinos N. Plataniotis, Jamal Seyedmohammadi +1 more
similarity 0.72
Abstract

Multi-person 3D pose estimation with WiFi channel state information (CSI) is challenging because reflections from different people overlap without directly identifying individual joints. Existing masked embedding objectives capture wireless relationships without explicit pose supervision, while structured decoders can retain coordinate errors. We propose WiSPER, a two-stage framework combining pose-aware predictive pretraining with conditional residual flow refinement. Pose-Aware Masked Embedding Learning (PAMEL) couples masked latent prediction with auxiliary pose-set supervision on the same CSI context, guiding the encoder toward joint localization from partial observations. Residual Flow refinement with Transformer (ReFT) generates a set of pose candidates to accommodate a variable number of people and refines each candidate through a conditional flow guided by its coarse coordinates and per-joint decoder features. Both stages use paired CSI and pose annotations during training, while inference requires only CSI. Experiments on the PiW3D dataset show that WiSPER achieves an overall mean per-joint position error of 63.72 mm, a 40.0% reduction relative to WiFi-JEPA. For experiments with two and three people, WiSPER reduces MPJPE by 42.1% and 38.1%, respectively. Pose-supervised pretraining configurations obtain lower errors than CSI-only JEPA, and enabling the trained residual refiner reduces overall MPJPE by 13.8-15.6% across the evaluated configurations.

0.17

Physics-Informed WiFi Sensing for Robust 3D Human Pose Estimation in Mobile and Cross-Environment Settings

eess.SP ·2026-08-25 ·Kaixuan Huang, Yuanbo Chen, Guangjin Pan, Shiyi Mu, Tao Yu +2 more
similarity 0.67
Abstract · 12pages 6figures

Device-free human pose estimation using commodity WiFi signals has emerged as a promising paradigm for pervasive sensing in mobile computing systems. However, existing approaches often suffer from severe performance degradation when deployed across heterogeneous environments due to complex multipath propagation and domain shifts in wireless signals. In this paper, we present a physics-informed WiFi sensing framework for robust 3D human pose estimation under mobile and cross-environment settings. Our approach explicitly models wireless signal propagation characteristics and incorporates multipath-aware attention to capture environment-dependent signal variations. To further improve generalization, we introduce a disentangled representation learning scheme that separates pose-related features from environment-specific factors, enabling effective cross-domain adaptation without requiring extensive retraining. We implement our system using commodity WiFi devices and evaluate it on multiple public benchmarks, including Person-in-WiFi-3D and MM-Fi, as well as real-world deployments across diverse indoor environments. Experimental results demonstrate that our framework significantly improves robustness and generalization performance compared to state-of-the-art methods, particularly under cross-environment scenarios. These results highlight the potential of physics-informed wireless sensing for enabling reliable, scalable, and infrastructure-free human-centric applications in mobile computing systems.

0.12

STEP: Score-Based Temporal Energy for Human Pose Video Anomaly Detection

cs.CV ·2026-08-20 ·Jakub Micorek, Mateusz Koziński, Horst Possegger
top venuebenchmarkcode/data similarity 0.65
Abstract · Accepted to ECCV 2026. Project page: https://jakubmicorek.github.io/STEP-demo | Code: https://github.com/jakubmicorek/STEP

Skeleton-based Video Anomaly Detection (VAD) offers a robust, privacy-preserving solution for identifying abnormal behaviors. To model the distribution of normal static and moving poses, recent methods train Energy-Based Models (EBMs) via Denoising Score Matching (DSM). However, directly injecting noise, required for training, into raw joint coordinates creates physically impossible poses, and this structural collapse severely worsens as the temporal window expands. To address this, we introduce STEP, a simple framework that utilizes Principal Component Analysis (PCA) to project pose sequences into a compact, whitened PC-space. Learning the data density within this well-behaved PC-space ensures that the injected noise translates into physically plausible variations, which allows the model to process longer video sequences without the performance collapse of raw coordinate baselines. Additionally, to mitigate inherent pose estimation inaccuracies arising from occlusions or motion blur, we integrate a sequence-level weighting mechanism based on the estimator's confidence scores. Operating at real-time computational efficiency, our simple and lightweight framework outperforms the previous skeleton-based state-of-the-art by 12.2% (90.1% AUROC) on the challenging UBnormal dataset and achieves highly competitive results by improving on the ShanghaiTech benchmark.

0.57

Real-Time Human Trajectory Prediction from RGB-D for Mobile Robots Using Bi-Mamba

cs.RO cs.LG ·2026-10-24 ·Basheer Al-Tawil, Magnus Jung, Ayoub Al-Hamadi
publishedbenchmark similarity 0.64
Abstract · Accepted at the 14th International Conference on Control, Mechatronics and Automation (ICCMA 2026), Bari, Italy, November 3-5, 2026. 7 pages…

Predicting the short-term future trajectory of a person from RGB-D cameras is a key capability for mobile robots navigating in dynamic and human populated environment. The key challenge to deploying learned predictors on real robots is the train-deploy pipeline gap. Training data are often prepared with an idealized, smoothed ground truth, whereas at inference, the model is fed noisy estimates from pose detectors and depth sensors. The model is therefore evaluated on an input distribution it never encountered during training. Most existing models rely on recurrent neural networks (RNNs) or transformers, which either struggle to capture short-term temporal dependencies or suffer from computational cost. We present TrajMamba, a lightweight, bidirectional state space model for short-term human trajectory prediction using an RGB-D camera. The model uses a single shared processing path for both training and deployment, aligning human pose extraction, depth back projection, and smoothing processes. It encodes a short window of persons's hip center velocities with stacked bidirectional Mamba blocks and decodes future velocities with a GRU head. On the ETH/UCY benchmark, TrajMamba reaches an average ADE of 0.29 m FDE of 0.51 m. With 262K parameters, it runs in real time on modest hardware and is deployed on a TIAGo robot with an onboard NVIDIA Jetson Orin.

0.31

Latent Dataset Distillation for Human Motion Prediction

cs.CV ·2026-09-22 ·Ge Tian, Guang Li, Takahiro Ogawa, Miki Haseyama
similarity 0.63
Abstract

Dataset distillation (DD) compresses a large training set into a compact synthetic set while preserving downstream training utility. Although DD has been widely studied for images and recently extended to time-series forecasting, its application to human motion prediction remains largely unexplored. Human motion is high-dimensional and structurally coupled, and gradient matching (GM) in the original motion space optimizes many correlated variables without a prior on pose plausibility or temporal dynamics, which frequently yields implausible and unstable synthetic motions. To address this limitation, we propose a latent DD framework that regularizes distillation with a learned motion prior. Motions are first compressed by a residual-quantized variational autoencoder (RVQ-VAE), and distillation then updates only a learnable latent bank through the frozen quantizer and decoder. The pretrained decoder restricts synthetic motions to its output space, while residual quantization progressively refines the latent approximation across multiple codebooks and alleviates the representational bottleneck of single-stage vector quantization. Experiments on Human3.6M, CMU, and 3DPW with two prediction backbones show that the proposed framework outperforms direct GM in 27 of 30 evaluated settings and random subsets in every setting, and produces visibly more plausible synthetic motions in qualitative comparisons.

0.21

Vision as Proprioception: Estimating Humanoid Robot Pose from Video

cs.RO cs.AIcs.CV ·2026-10-18 ·Jiaxi Jiang, Matthias Heyrman, René Zurbrügg, Marco Hutter, Christian Holz
code/data similarity 0.63
Abstract · Code: https://github.com/eth-siplab/HumanoidPose

Estimating a humanoid robot's internal state from video has a wide range of real-world applications but remains unexplored. During interactions with a robot, its onboard state information may be unavailable due to privacy, ownership, or communication constraints. Meanwhile, online and AI-generated robot videos provide rich sources of data but typically lack corresponding joint configurations, limiting their direct use in robot learning. To address these issues, we present HumanoidPose, a new framework that treats vision as proprioception to recover full-body joint configurations in world space from video alone. Specifically, we combine pretrained human-pose features with robot kinematics in a unified model that predicts each embodiment's joint configurations. We retarget human motions to sixteen different humanoid embodiments, enabling scalable learning entirely from synthetic data. To improve real-world generalization, we further leverage unlabeled online videos through self-training to develop a 2D robot-pose detector. In our evaluation, HumanoidPose improves joint-angle and joint-position accuracy over the evaluated baselines and reduces joint-angle error by 18-32% relative to the best-performing baseline on each of the three real humanoid platforms. We further show improved global localization and generalization to unseen embodiments. HumanoidPose enables visual understanding of other robots during interaction and extraction of robot motion from online and generated videos.

0.38

Principled Koopman Representations with Kalman Inference for Efficient Time-Series Prediction

cs.LG cs.AI ·2026-09-15 ·Ruiquan Li, Yuheng Bu
similarity 0.63
Abstract

The Koopman operator has been widely used for time-series prediction in dynamical systems. However, prior work that learns latent ``Koopman spaces'' using neural networks often did not construct a valid Koopman space for forecasting, as these representations may be mathematically inconsistent with the operator-theoretic formulation and fail to capture the intrinsic low-rank structure of system dynamics. To address this issue, we introduce K$^2$SVD, a method that explicitly learns the leading singular functions of the Koopman operator by optimizing a Hilbert-Schmidt objective. This yields a well-defined low-rank approximation of the Koopman operator with an interpretable linear combination, featuring a compact latent space with less than $10\%$ of the dimensions used in previous work. In the learned Koopman space, K$^2$SVD further captures temporal evolution with a linear Gaussian state-space model and performs inference via Kalman filtering, mitigating noise accumulation during multi-step prediction. Empirical results show that K$^2$SVD outperforms state-of-the-art methods across multiple datasets, with significantly faster prediction speeds and lower computational cost than previous efficiency-focused models. This highlights the benefits of principled low-rank Koopman representations and opens up broader potential for applications.

0.26

CRB-Guided Sensing and Resource Allocation for Human Pose Prediction in Integrated Sensing, Communication, and Computation Systems

cs.IT cs.NI ·2026-09-01 ·Zhonghao Liu, Yahao Ding, Jiaxiang Wang, Zhaohui Yang, Abdol Hamid Aghvami +1 more
published similarity 0.63
Abstract

Integrated sensing, communication, and computation (ISCC) provides a promising framework for indoor human-centric applications. In these applications, short-term human pose prediction facilitates continuous human pose tracking and proactive resource allocation. This paper proposes a Cramer-Rao bound (CRB)-guided sensing framework and investigates a problem of minimizing prediction error in resource-constrained ISCC systems. Specifically, a pose prediction model (ET-Mamba) is first developed to predict human joint positions for continuous tracking. To account for computation-resource limitations, lightweight prediction heads are attached to different inference layers, enabling adaptive-depth pose prediction. A CRB-guided perturbation strategy is then introduced to translate sensing uncertainty at different sensing SNR levels into point-cloud perturbations. Based on that, an empirical relationship among pose prediction error, sensing SNR, and model inference depth is established. Furthermore, to improve prediction accuracy under limited resources, this paper formulates a resource allocation optimization problem that minimizes the pose prediction error by jointly optimizing the beamforming matrix, model inference depth, and computation frequency. To solve this mixed-integer non-convex optimization problem, we propose an alternating optimization (AO)-based algorithm, where closed-form updates and semidefinite programming (SDP) are integrated into the iterative solution process. Simulation results show that the proposed method effectively improves pose prediction performance by up to 35 percent under resource constraints, verifying the effectiveness of conducting joint sensing, communication, and computation design in ISCC systems.

0.42