CS Paper Radar

PT$^3$: Prototype-Guided Self-Supervised Pretraining for Generalizable mmWave Human Point Cloud Representation

Ziyang Liu, Kieren Yu, Xiao Yang, Xin Yang, Kaishun Wu

cs.AI ·submitted 2026-10-28 09:49 UTC · v1

Abstract

Multi-source millimeter-wave (mmWave) observations offer an opportunity to learn human representations reusable across sensing tasks. However, sparse detections and multipath effects make these observations unstable, challenging existing point cloud pretraining methods. We propose PT$^3$, a prototype-guided self-supervised pretraining framework for mmWave human point clouds. PT$^3$ represents local states in spatial regions as sparse compositions of shared prototypes and learns their temporal evolution through autoregressive prediction. Observation consistency further stabilizes representations under radar perturbations. After pretraining on unlabeled sequences from multiple sources, the same frozen encoder supports action classification, dense point generation, joint pose estimation, and future motion prediction on datasets not used for pretraining. Across these tasks, PT$^3$ outperforms the compared pretraining baselines and achieves competitive or superior performance to task-specific models.

Comment
30 pages

Closest papers

Nearest neighbours by meaning across the whole index.

RaStream: Edge-Deployable Streaming Human Mesh Recovery from mmWave Radar

eess.SP ·2026-08-12 ·Jiazhen Dong, Lei Liu
benchmark similarity 0.67
Abstract

Millimeter-wave (mmWave) radar provides a privacy-preserving sensing modality for human motion analysis, yet continuous SMPL-X recovery on edge devices remains challenging because accurate reconstruction benefits from dense volumetric radar observations that preserve weak articulated reflections, while repeatedly processing such high-dimensional data over time incurs substantial computational cost. Temporal context is also necessary to resolve frame-local ambiguity, but SMPL-X attributes evolve at different rates, with body morphology remaining relatively stable and articulated pose, root orientation, and global translation changing rapidly. We present RaStream, a causal streaming framework that separates rich volumetric spatial perception from lightweight temporal reasoning. For each radar observation, a localization-conditioned spatial encoder exploits global scene context to estimate a coarse body location and focuses detailed reconstruction on a center-guided region of interest while preserving native 3D radar structure. Global and local features are then fused into a compact representation, allowing volumetric evidence to be extracted once per observation and temporal history to be propagated through compact representations and fixed-size persistent states. RaStream further maintains a slow morphology state for shape-related information and a fast motion state for pose, orientation, and translation. On the 661K-frame M4Human benchmark, RaStream reduces MVE from 90.90 mm for RT-Mesh to 84.27 mm with spatial-only inference and to 72.05 mm with causal temporal refinement. The Base configuration requires 26.93 ms per FP32 invocation on a Jetson Orin Nano. These results demonstrate practical edge-streaming radar mesh recovery through one-time volumetric evidence extraction and compact causal temporal propagation.

0.04

Beyond Gait: Person Identification from Millimeter-Wave Point Clouds Across Activities of Daily Living

cs.CV ·2026-09-08 ·Xilai Wang, Zixiong Han, Saad Rhanmouni, Chenzhe Zhao, Yunze Lu +1 more
benchmark similarity 0.66
Abstract · v2: Revised the activity-identity learning framework, updated experimental comparisons, and improved manuscript presentation

Person identification from millimeter-wave (mmWave) radar point clouds has primarily focused on gait. However, since indoor walking is typically brief and fragmented, gait-based identification is largely constrained to designated passages that support continuous walking. Meanwhile, non-gait activities of daily living (ADLs) are pervasive and also encode discriminative personal signatures. In this study, we evaluate the feasibility of ADL-based identification from sparse mmWave point clouds using mm-ADL, a dataset comprising seven core ADLs collected from 11 subjects under controlled protocols. Moving beyond uniform gait introduces significant activity-dependent variations that confound identity representations. To address this challenge, we systematically benchmark three activity-identity learning paradigms: monolithic learning, joint multi-task learning, and two-stage activity-conditioned routing, under both closed-set identification (ID) and subject-disjoint re-identification (ReID) biometric settings. In parallel, we propose Dual-Stream Static-Dynamic PointNet (DS-SDPNet), an architecture tailored to radar point clouds that captures complementary spatial morphology and motion dynamics. Across seven ADLs, DS-SDPNet achieves 70.2% mean closed-set identification accuracy and outperforms existing baselines on both mm-ADL and mmGait. Moreover, benchmarking the three paradigms without ground-truth activity labels during inference yields up to 68.0% closed-set accuracy and 75.4% ReID mAP, providing a systematic baseline and confirming the feasibility of point-cloud-based biometric identification across diverse daily activities.

0.33

Radar-Based 3D Human Pose Estimation via Motion-Aware Pose Initialization and Observation-Guided Refinement

cs.CV cs.AI ·2026-10-20 ·Woo-Jin Jung, Chan-Bin Lim, Seung-Hyun Kong
similarity 0.65
Abstract · 15 pages, 5 figures, 8 tables

3D Human Pose Estimation (3D HPE) is used in indoor environments, such as smart homes, to analyze human movement and monitor human states. Millimeter-wave (mmWave) radar is suitable for this task because it is robust to lighting variations and does not capture images of human appearance. However, radar observations often provide insufficient information about certain body parts or contain overlapping reflections from multiple body parts, limiting accurate 3D joint localization. To address these limitations, we propose the Initialization-to-Refinement Pose Framework (I2R-Pose), comprising two stages: Motion-Aware Pose Initialization and Observation-Guided Refinement. In the first stage, joint-wise features are learned by incorporating temporal relationships across consecutive radar observations and human skeletal structure to compensate for insufficient joint information and estimate an initial 3D pose sequence. In the second stage, separately extracted radar observation features predict joint-wise positional corrections that refine the initial pose and generate the final pose sequence. The framework supports individual and fused inputs of time-Doppler/time-range (TD/TR) maps and radar point clouds (PCs). To evaluate performance across indoor activities, we construct the Radar Motion and Spatial Information for 3D Human Pose Estimation (RaMoS-Pose) dataset, providing synchronized TD/TR maps, radar PCs, and 3D joint ground truth (GT). Evaluations on RaMoS-Pose and the publicly available MVDoppler-Pose dataset show that I2R-Pose reduces mean per-joint position error (MPJPE) by 39.50% on MVDoppler-Pose and by 15.13% on RaMoS-Pose with fused inputs, compared with the best-performing existing methods evaluated on each dataset.

0.25

Soft Temporal Alignment Distillation for Camera-to-mmWave Radar Human Activity Recognition

eess.SP ·2026-10-27 ·Jiuqi Yan, Chengbo Zhu, Chengdong Xu, Qisong Wu
similarity 0.64
Abstract · 13 pages, 7 figures

Millimeter-wave (mmWave) radar provides privacy?preserving human activity recognition (HAR), but sparse point clouds limit its representation of body structure and motion. We propose a cross-modal knowledge distillation framework in which a teacher trained on camera-derived two-dimensional (2D) human keypoints supervises a radar-only student. The stu?dent combines PointNet-based point-set encoding, bidirectional temporal modeling, segment-level aggregation, and lightweight shared latent-channel scaling. The teacher retains contextualized states at every time step, whereas the student summarizes the radar sequence into a small number of segments, making fixed timestamp correspondence unsuitable for feature transfer. Soft Temporal Alignment Distillation lets each student segment adaptively retrieve and aggregate teacher temporal evidence into its distillation target. Alignment denotes student-conditioned representation-space matching rather than chronological or monotonic registration. On the Multi-Modal Non-Intrusive 4D Human Dataset (MM-Fi), the method improves radar-only ac?curacy from 87.04% to 89.17%, achieving the highest mean accuracy among the evaluated established and recent distillation baselines. It also obtains higher mean accuracy and macro?averaged F1 score (macro-F1) than the radar student under most evaluated point- and frame-dropout settings. Adaptation to our independently collected millimeter-wave Office Human Ac?tivity Recognition Dataset (mmO-HAR) improves accuracy from 80.33% to 87.75%, supporting the transferability of the distilled radar representation. The teacher and alignment branch are removed after training, enabling privacy-preserving radar-only inference without additional sensing or computational branches.

0.33

A Universal Raw-IQ Radar Encoder for Cross-Sensor Human Sensing

cs.CV cs.LG ·2026-11-03 ·Jonas Leo Mueller, Dario Zanca, Sebastian Hoefler, Jan Petermann, Thomas Altstidl +3 more
publishedbenchmarkpositioncode/data similarity 0.63
Abstract · Accepted at the 2027 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)

Radar-based human sensing typically relies on end-to-end task-specific models trained on hand-chosen signal projections, which generalize poorly across unseen users, tasks, and hardware. We argue that the dominant signal processing choice, the micro-Doppler spectrogram, is a poor starting point for transfer because it keeps the Doppler axis, which carrier frequency rescales, and discards the angular axis, which carrier frequency leaves unchanged. We instead pretrain a convolutional beta-VAE without labels on clutter-removed raw radar measurements from four platforms spanning 60 and 77 GHz and 4 to 12 virtual antennas, aligned by a unified spectral interpolation grid. For every downstream benchmark the encoder stays frozen, and only a small supervised temporal classifier is trained on its embeddings. We compare against supervised models trained end to end on each benchmark and a pretrained radar foundation model across ten settings covering activity, gesture, sign-language, and gait recognition. When the test users and the radar are seen during training, our encoder is best on all five benchmarks. For users never seen in training, it leads on three of four benchmarks, by 4.8 to 23.2 points. On subject-disjoint gait recognition, it is best for each of the four held-out subjects and 8.3 points above the strongest baseline on average, even though its pretraining excluded every recording of this radar. Excluding the test radar from pretraining costs at most 5.3 points on any benchmark. Code is available at https://github.com/jonasmueler/UniversalRadarEncoder_WACV.

0.50

Phenomenon-Graph JEPA: Label-Efficient Representation Learning for Contactless Cardiorespiratory Sensing

cs.LG cs.ET ·2026-09-26 ·Constantino Álvarez Casado, Nhi Nguyen, Mohammad Rakibur Rahman, Le Nguyen, Manuel Lage Cañellas +2 more
similarity 0.63
Abstract · 9 pages, 4 figures, 7 tables, 24 numbered equations. Extended version of a conference submission, with additional methodological details, im…

Millimeter-wave (mmWave) radar and RGB-D cameras can record cardiac and respiratory waveforms continuously and without contact, but labeled recordings remain scarce because every label requires a supervised acquisition session. Self-supervised pretraining can exploit the unlabeled signals, yet contrastive methods depend on signal transformations and negative pairs whose validity is uncertain for cardiorespiratory data, where time warping changes breathing rate and distant windows can share the same physiological state. We present Phenomenon-Graph JEPA, a joint-embedding predictive architecture that learns from four processed one-dimensional streams without negative pairs or synthetic augmentation in its base configuration. Each stream is encoded by a temporal convolutional branch and a band-limited spectral branch. During pretraining, the model predicts stopped target embeddings along typed edges, which connect streams assigned to the same physiological phenomenon, and forward in time within a state episode. We treat this physiological typing as a testable hypothesis and compare it with wrong-edge and all-pairs prediction graphs. In the OMuSense-23 dataset, pretraining improves label-efficiency area over matched supervised training by 3.91 percentage points (95% interval 2.08 to 5.80, Holm-adjusted p = 0.006), and by 3.74 points under a second configuration evaluated on the same test participants. However, the wrong-edge and all-pairs controls do not establish a benefit from physiological typing. Optional Takens-inspired delay coordinates improve a validation comparison with learned history, whereas two wrist-only WESAD protocols do not establish a pretraining advantage. The study therefore separates the measured benefit of predictive representations from the physiological prior used to organize their training.

0.20

RadPose: ISAC-based RF 3D Human Pose Estimation

cs.CV cs.NI ·2026-11-01 ·Geo Jie Zhou, Kunzhe Song, Ruxin Lin, Huacheng Zeng
publishedcode/data similarity 0.62
Abstract · Accepted to WACV 2027

Human pose estimation under adverse lighting and occlusion conditions remains challenging for camera- and LiDAR-based systems. Although mmWave radars provide robustness to illumination and occlusion, they rely on dedicated hardware and licensed spectrum, limiting scalability. In this paper, we present RadPose, a learning-based monostatic integrated sensing and communication (ISAC) framework that enables mmWave communication devices to perform 3D human pose estimation. Unlike specialized radars, RadPose leverages monostatic full-duplex operation to extract channel impulse responses (CIRs) directly from standard OFDM communication signals, generating 3D radio point clouds without additional sensing hardware. To address the sparsity and noise of RF observations, RadPose employs a cross-frame transformer-based pose inference network with semantic-guided spatial attention and target-driven temporal attention for robust spatio-temporal aggregation. It further adopts a joint-aware inverse kinematics refinement module based on graph convolution to enforce skeletal topology, bone-length consistency, and anatomical plausibility. Extensive experiments demonstrate RadPose achieves competitive accuracy compared to state-of-the-art radar-based approaches while maintaining robust performance in unseen and occluded scenarios. The dataset and source code are available at https://github.com/Geo-Chou/RadPose.

0.46

DiFF: Doppler-informed Flow Matching for Human Motion Flow

cs.RO cs.CVcs.HC ·2026-09-30 ·Kai Wang, Mingle Zhao
publishedbenchmarkcode/data similarity 0.61
Abstract · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2026. Code: https://github.com/keroseus/DiFF

Perceiving human motion via privacy-preserving 4D millimeter-wave (mmWave) radar is critical for next-generation human-robot interaction (HRI), where point cloud scene flow serves as a foundational motion representation. Yet the extreme sparsity and noise of 4D radar point clouds make non-rigid motion flow estimation severely ill-posed--a challenge that existing rigid-centric methods and prior works fail to adequately address, largely because they neglect the rich Doppler velocity cues inherent in 4D radar. We propose DiFF, a generative framework that marries Doppler-informed motion priors with a Kolmogorov-Arnold Network (KAN)-based conditional flow matching model. At its core, a KAN-attention mechanism enables expressive feature extraction, while a prior-guided generative process harnesses Doppler cues to regularize the ill-posed solution space. Extensive experiments show that DiFF achieves state-of-the-art (SOTA) performance across diverse real-world datasets, reducing 3D endpoint error to the millimeter scale on the mmBody benchmark.

0.34