CS Paper Radar

A Universal Raw-IQ Radar Encoder for Cross-Sensor Human Sensing

Jonas Leo Mueller, Dario Zanca, Sebastian Hoefler, Jan Petermann, Thomas Altstidl, Naga Venkata Sai Jitin Jami, Emmanuelle Salin, Bjoern M. Eskofier

cs.CV cs.LG ·submitted 2026-11-03 09:03 UTC · v1
publishedbenchmarkpositioncode/data

Abstract

Radar-based human sensing typically relies on end-to-end task-specific models trained on hand-chosen signal projections, which generalize poorly across unseen users, tasks, and hardware. We argue that the dominant signal processing choice, the micro-Doppler spectrogram, is a poor starting point for transfer because it keeps the Doppler axis, which carrier frequency rescales, and discards the angular axis, which carrier frequency leaves unchanged. We instead pretrain a convolutional beta-VAE without labels on clutter-removed raw radar measurements from four platforms spanning 60 and 77 GHz and 4 to 12 virtual antennas, aligned by a unified spectral interpolation grid. For every downstream benchmark the encoder stays frozen, and only a small supervised temporal classifier is trained on its embeddings. We compare against supervised models trained end to end on each benchmark and a pretrained radar foundation model across ten settings covering activity, gesture, sign-language, and gait recognition. When the test users and the radar are seen during training, our encoder is best on all five benchmarks. For users never seen in training, it leads on three of four benchmarks, by 4.8 to 23.2 points. On subject-disjoint gait recognition, it is best for each of the four held-out subjects and 8.3 points above the strongest baseline on average, even though its pretraining excluded every recording of this radar. Excluding the test radar from pretraining costs at most 5.3 points on any benchmark. Code is available at https://github.com/jonasmueler/UniversalRadarEncoder_WACV.

Comment
Accepted at the 2027 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)

Closest papers

Nearest neighbours by meaning across the whole index.

Robust Continuous Human Activity Recognition Using Deep Learning and Distributed Radar Sensors

eess.SP ·2026-09-09 ·Sohani Munteha Hiam, Mohammad Soleymani
similarity 0.66
Abstract

Continuous human activity recognition (HAR) with distributed radar sensor networks is challenging because Doppler signatures depend strongly on aspect angle, the informativeness of individual radar views varies with motion direction, and activity transitions in uninterrupted sequences are often ambiguous. This paper proposes a ConvNeXt-MHSA-BiGRU framework for frame-wise continuous HAR from Doppler-time spectrograms acquired by five spatially distributed radar nodes. A shared ConvNeXt-inspired encoder extracts time-frequency representations from each radar stream, while radar-wise multi-head self attention (MHSA) models inter-node dependencies and adaptively fuses complementary views. RadarDropout and SpecAugment-style time-frequency masking regularize the network against unreliable radar views and local spectrogram perturbations, whereas stacked bidirectional gated recurrent units (BiGRUs) exploit temporal context for frame-wise classification across activity transitions. The framework is evaluated on nine activities performed by 14 participants using leave-one-person-out (L1PO) subject-independent testing with fivefold cross-validation. It achieves a mean L1PO test accuracy of 87.56% across the 14 held-out subjects and improves recognition performance over the published CNN-RNN baseline. The results demonstrate that combining modern spectrogram encoding, adaptive radar view fusion, and bidirectional temporal modeling provides an effective framework for continuous and distributed radar-based HAR.

0.24

RFShape: An IQ-Pretrained Wireless Foundation Model for Heterogeneous IQ & CIR Representations

eess.SP ·2026-11-02 ·Mohammad Cheraghinia, Sebbe Van Der Smissen, Jaron Fontaine, Delia Rico, Eli De Poorter +1 more
similarity 0.64
Abstract · 17 pages, 12 figures, 8 tables

In wireless systems, foundation models have been proposed as alternative artificial intelligence models to replace task-specific models with a shared pretrained model that can be reused across many downstream applications (source separation, interference detection, localization, etc.). These wireless foundation models promise to reuse learned representations across diverse tasks, but choosing a foundation model also means choosing its input representation. For example, a model pretrained on spectrograms expects spectrograms, while one pretrained on in-phase/quadrature (IQ) samples expects IQ, limiting which pretrained model can be used for a given downstream problem. To remedy this, we present RFShape, a 7.4 M-parameter foundation model that breaks this coupling by adapting how a signal is presented to a shared pretrained encoder. RFShape is pretrained solely on synthetic IQ waveforms, without real captures, labels, or channel impulse responses (CIRs), and uses deterministic representation operators to support heterogeneous downstream inputs. Across ten tasks and nine datasets unseen during pretraining, the same encoder supports classification, ranging-error regression, and two-dimensional positioning. Representation selection alone increases 23-class radar recognition from 23.0% to 69.5% and short-range technology recognition from 70.8% to 97.1%. RFShape also outperforms CIR-pretrained ultra-wideband baselines by up to 12.7 percentage points for (N)LOS classification despite never observing CIRs during pretraining. These results show that foundation-model reuse need not be limited by the representation or modality seen during pretraining, and adapting how a signal is presented can extend a single pretrained encoder across heterogeneous wireless tasks.

0.30

Learning to Assess Heartbeat Observability for mmWave Heart-Rate Sensing

cs.AI ·2026-10-02 ·Yuxuan Hu, Shilin Shan, Jianfei Yang, Feng Xu
code/data similarity 0.64
Abstract · 19 pages, 11 figures. Project page: https://yuxuanhu9.github.io/HEAR/

Contactless heart-rate sensing with millimeter-wave (mmWave) radar requires assessing whether individual measurements support reliable estimation. We study learning to assess heartbeat observability, defined as the readability of the heartbeat component in an acquired phase spectrum, for selective heart-rate estimation. Coherent superposition of scatterer returns can suppress this component even under similar macroscopic observation geometry, motivating assessment directly from acquired measurements. To obtain training supervision across different observability conditions, we develop a controllable multi-scatterer frequency-modulated continuous-wave (FMCW) simulator. Agreement between the dominant heartbeat-band peak and the known heart rate provides an automatic observability label for each simulated measurement. We propose HEAR (Heartbeat Estimation with Assessed Reliability), a compact dual-task Transformer that jointly predicts an observability score and heart rate. Its input combines spectral magnitudes with frequencies relative to the respiration fundamental, providing context for respiratory harmonics. Trained solely on simulated observations, HEAR transfers zero-shot to two public real-world datasets collected at 60 and 120 GHz from 134 subjects. The same learned score supports selective prediction with both HEAR's own heart-rate head and multiple existing estimators. On the 120 GHz dataset, score-based selection reduces the HR head's mean absolute error from 17.9 BPM at full coverage to 1.6 BPM at 50% coverage. The complete pipeline achieves an end-to-end processing latency of 50.8 ms on an edge device. Project page: https://yuxuanhu9.github.io/HEAR/.

0.38

Soft Temporal Alignment Distillation for Camera-to-mmWave Radar Human Activity Recognition

eess.SP ·2026-10-27 ·Jiuqi Yan, Chengbo Zhu, Chengdong Xu, Qisong Wu
similarity 0.63
Abstract · 13 pages, 7 figures

Millimeter-wave (mmWave) radar provides privacy?preserving human activity recognition (HAR), but sparse point clouds limit its representation of body structure and motion. We propose a cross-modal knowledge distillation framework in which a teacher trained on camera-derived two-dimensional (2D) human keypoints supervises a radar-only student. The stu?dent combines PointNet-based point-set encoding, bidirectional temporal modeling, segment-level aggregation, and lightweight shared latent-channel scaling. The teacher retains contextualized states at every time step, whereas the student summarizes the radar sequence into a small number of segments, making fixed timestamp correspondence unsuitable for feature transfer. Soft Temporal Alignment Distillation lets each student segment adaptively retrieve and aggregate teacher temporal evidence into its distillation target. Alignment denotes student-conditioned representation-space matching rather than chronological or monotonic registration. On the Multi-Modal Non-Intrusive 4D Human Dataset (MM-Fi), the method improves radar-only ac?curacy from 87.04% to 89.17%, achieving the highest mean accuracy among the evaluated established and recent distillation baselines. It also obtains higher mean accuracy and macro?averaged F1 score (macro-F1) than the radar student under most evaluated point- and frame-dropout settings. Adaptation to our independently collected millimeter-wave Office Human Ac?tivity Recognition Dataset (mmO-HAR) improves accuracy from 80.33% to 87.75%, supporting the transferability of the distilled radar representation. The teacher and alignment branch are removed after training, enabling privacy-preserving radar-only inference without additional sensing or computational branches.

0.33

PT$^3$: Prototype-Guided Self-Supervised Pretraining for Generalizable mmWave Human Point Cloud Representation

cs.AI ·2026-10-28 ·Ziyang Liu, Kieren Yu, Xiao Yang, Xin Yang, Kaishun Wu
similarity 0.63
Abstract · 30 pages

Multi-source millimeter-wave (mmWave) observations offer an opportunity to learn human representations reusable across sensing tasks. However, sparse detections and multipath effects make these observations unstable, challenging existing point cloud pretraining methods. We propose PT$^3$, a prototype-guided self-supervised pretraining framework for mmWave human point clouds. PT$^3$ represents local states in spatial regions as sparse compositions of shared prototypes and learns their temporal evolution through autoregressive prediction. Observation consistency further stabilizes representations under radar perturbations. After pretraining on unlabeled sequences from multiple sources, the same frozen encoder supports action classification, dense point generation, joint pose estimation, and future motion prediction on datasets not used for pretraining. Across these tasks, PT$^3$ outperforms the compared pretraining baselines and achieves competitive or superior performance to task-specific models.

0.30

STAR: Scene- and Task-Aware 4D Radar Preprocessing Towards End-to-End Cognitive Radar

cs.CV cs.AI ·2026-09-21 ·Seung-Hyun Song, Dong-Hee Paek, Seung-Hyun Kong
benchmark similarity 0.62
Abstract

Four-dimensional (4D) Radar has emerged as a key sensor for environmental perception, providing range, azimuth, elevation, and Doppler measurements while remaining robust to illumination changes and adverse weather conditions. However, conventional Radar preprocessing methods, such as constant false alarm rate (CFAR) detection, select measurements primarily based on signal-level criteria and may therefore discard information valuable for downstream perception during point cloud generation. In addition, existing 4D Radar perception pipelines typically optimize Radar data processing and downstream perception independently, preventing task objectives from directly guiding the preprocessing stage. To address these limitations, we propose a Scene- and Task-Aware Radar (STAR) Preprocessor together with an end-to-end training framework. The STAR Preprocessor incorporates scene context and downstream task objectives to generate task-relevant Radar points, enabling the Radar representation to be optimized directly for perception. On the K-Radar benchmark, the proposed method achieves 74.3 AP, outperforming the previous state of the art by 5.6 AP points. Furthermore, applying the task-relevant points generated by STAR to various existing 3D detectors improves detection performance in most evaluation settings and yields an overall positive average gain over point clouds produced by conventional preprocessing.

0.28

Beyond Gait: Person Identification from Millimeter-Wave Point Clouds Across Activities of Daily Living

cs.CV ·2026-09-08 ·Xilai Wang, Zixiong Han, Saad Rhanmouni, Chenzhe Zhao, Yunze Lu +1 more
benchmark similarity 0.61
Abstract · v2: Revised the activity-identity learning framework, updated experimental comparisons, and improved manuscript presentation

Person identification from millimeter-wave (mmWave) radar point clouds has primarily focused on gait. However, since indoor walking is typically brief and fragmented, gait-based identification is largely constrained to designated passages that support continuous walking. Meanwhile, non-gait activities of daily living (ADLs) are pervasive and also encode discriminative personal signatures. In this study, we evaluate the feasibility of ADL-based identification from sparse mmWave point clouds using mm-ADL, a dataset comprising seven core ADLs collected from 11 subjects under controlled protocols. Moving beyond uniform gait introduces significant activity-dependent variations that confound identity representations. To address this challenge, we systematically benchmark three activity-identity learning paradigms: monolithic learning, joint multi-task learning, and two-stage activity-conditioned routing, under both closed-set identification (ID) and subject-disjoint re-identification (ReID) biometric settings. In parallel, we propose Dual-Stream Static-Dynamic PointNet (DS-SDPNet), an architecture tailored to radar point clouds that captures complementary spatial morphology and motion dynamics. Across seven ADLs, DS-SDPNet achieves 70.2% mean closed-set identification accuracy and outperforms existing baselines on both mm-ADL and mmGait. Moreover, benchmarking the three paradigms without ground-truth activity labels during inference yields up to 68.0% closed-set accuracy and 75.4% ReID mAP, providing a systematic baseline and confirming the feasibility of point-cloud-based biometric identification across diverse daily activities.

0.33

Over-the-Scene Computation: Illumination-Programmed Functional Sensing of Unresolvable Populations

cs.IT cs.ET ·2026-11-05 ·Seyed Mohammad Azimi-Abarghouyi, Christopher G. Brinton
similarity 0.61
Abstract

Decisions about dense populations depend on how much will cross a boundary by a deadline and when the first member arrives. Conventional radar forms a range-Doppler map, whose cells add noise and whose detections merge unresolved members. We introduce over-the-scene computation, where illumination makes the scene compute such functionals without resolving members. With random echo phases, a correlator's mean output energy sums member reflectivities weighted by a kernel set by the waveform and template. The waveform's ambiguity function is this computation's transfer function, determining computable functionals and their noise cost. One readout aggregates members over a small delay-Doppler area but along a line segment growing with duration, longest for the chirp of matching slope among constant-envelope waveforms. Members predicted to share a range lie on such a line, so a chirp's mean beat spectrum is the predicted range profile at a look-ahead horizon set by its rate. For unbiased estimation, a Cramer-Rao bound over all equal-energy waveforms favors resolving members above unit signal-to-noise ratio (SNR) and grouping weaker ones into unit-SNR sets. At 0.65 members per cell, -10 dB member SNR, and one sweep, the chirp's bin sum inside the boundary comes within 16% of this bound in root-mean-square error and, with 77 times fewer samples per horizon, outperforms every unbiased estimator from a coherent processing interval of equal dwell, bandwidth, and energy. No finite set of waveform-template probes determines an unconstrained Rayleigh-fluctuating population's first arrival, but bisection localizes it if first arrivals outweigh later arrivals' sidelobes.

0.34