CS Paper Radar

DiFF: Doppler-informed Flow Matching for Human Motion Flow

Kai Wang, Mingle Zhao

cs.RO cs.CVcs.HC ·submitted 2026-09-30 06:36 UTC · v1
publishedbenchmarkcode/data

Abstract

Perceiving human motion via privacy-preserving 4D millimeter-wave (mmWave) radar is critical for next-generation human-robot interaction (HRI), where point cloud scene flow serves as a foundational motion representation. Yet the extreme sparsity and noise of 4D radar point clouds make non-rigid motion flow estimation severely ill-posed--a challenge that existing rigid-centric methods and prior works fail to adequately address, largely because they neglect the rich Doppler velocity cues inherent in 4D radar. We propose DiFF, a generative framework that marries Doppler-informed motion priors with a Kolmogorov-Arnold Network (KAN)-based conditional flow matching model. At its core, a KAN-attention mechanism enables expressive feature extraction, while a prior-guided generative process harnesses Doppler cues to regularize the ill-posed solution space. Extensive experiments show that DiFF achieves state-of-the-art (SOTA) performance across diverse real-world datasets, reducing 3D endpoint error to the millimeter scale on the mmBody benchmark.

Comment
IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2026. Code: https://github.com/keroseus/DiFF

Closest papers

Nearest neighbours by meaning across the whole index.

You Only Flow Once: Calibrated and Real-Time Radar Pose Estimation with Multi-Hypothesis Normalizing Flows

cs.CV ·2026-08-10 ·Jonas Leo Mueller, Sebastian Hoefler, Dario Zanca, Naga Venkata Sai Jitin Jami, Thomas Altstidl +1 more
publishedcode/data similarity 0.67
Abstract · Accepted at the Winter Conference on Applications of Computer Vision (WACV) 2027

Sparse and noisy millimeter-wave radar point cloud observations often correspond to multiple plausible human poses, making deterministic pose estimation fundamentally ill-posed. Yet existing radar methods remain deterministic, collapsing this ambiguity into a single estimate. Diffusion-based alternatives can model multi-hypothesis distributions but require costly sequential denoising for each sample and lack calibrated uncertainty. We propose Multi-Hypothesis Normalizing Flow Pose Generator (MH-NFPG), which models pose distributions from radar point clouds using a conditional normalizing flow. Specifically, we combine a spatiotemporal transformer backbone with a normalizing flow that transforms a Laplace base distribution into an expressive posterior, generated in parallel through a single forward pass. Leveraging this efficiency, we outperform diffusion-based alternatives in calibration across three radar benchmarks (MM-Fi, mmRadPose, mRI) at comparable pose accuracy, while achieving up to 22x faster inference and reducing calibration error by up to 84%. We find that calibration degrades substantially for diffusion models, whereas our flow-based approach maintains reliable coverage, also across environments. These results demonstrate normalizing flows as a practical alternative to diffusion models for real-time, uncertainty-aware radar pose estimation. Our code is available at https://github.com/jonasmueler/YouOnlyFlowOnce_WACV.

0.77

RaStream: Edge-Deployable Streaming Human Mesh Recovery from mmWave Radar

eess.SP ·2026-08-12 ·Jiazhen Dong, Lei Liu
benchmark similarity 0.67
Abstract

Millimeter-wave (mmWave) radar provides a privacy-preserving sensing modality for human motion analysis, yet continuous SMPL-X recovery on edge devices remains challenging because accurate reconstruction benefits from dense volumetric radar observations that preserve weak articulated reflections, while repeatedly processing such high-dimensional data over time incurs substantial computational cost. Temporal context is also necessary to resolve frame-local ambiguity, but SMPL-X attributes evolve at different rates, with body morphology remaining relatively stable and articulated pose, root orientation, and global translation changing rapidly. We present RaStream, a causal streaming framework that separates rich volumetric spatial perception from lightweight temporal reasoning. For each radar observation, a localization-conditioned spatial encoder exploits global scene context to estimate a coarse body location and focuses detailed reconstruction on a center-guided region of interest while preserving native 3D radar structure. Global and local features are then fused into a compact representation, allowing volumetric evidence to be extracted once per observation and temporal history to be propagated through compact representations and fixed-size persistent states. RaStream further maintains a slow morphology state for shape-related information and a fast motion state for pose, orientation, and translation. On the 661K-frame M4Human benchmark, RaStream reduces MVE from 90.90 mm for RT-Mesh to 84.27 mm with spatial-only inference and to 72.05 mm with causal temporal refinement. The Base configuration requires 26.93 ms per FP32 invocation on a Jetson Orin Nano. These results demonstrate practical edge-streaming radar mesh recovery through one-time volumetric evidence extraction and compact causal temporal propagation.

0.04

4D Radar Perception Algorithms for Autonomous Driving: A Review

cs.RO cs.CVeess.SP ·2026-09-16 ·Xumin Wu, Jun Zhou, Jilin Mei, Chen Min, Yu Hu
similarity 0.65
Abstract · 12 pages, 9 figures, 5 tables. Submitted to IEEE Sensors Journal

Research on 4D millimeter-wave radar perception algorithms has flourished in recent years, extending from signal processing and object detection to semantic segmentation, motion estimation, occupancy prediction, and dynamic scene reconstruction. This review organizes the field according to the evolution of perception tasks and algorithms. It first introduces radar fundamentals, data representations, and quality-enhancement methods, and then reviews object-level perception, motion and localization, local and dense spatial perception, and dynamic scene understanding. Across these directions, we compare radar-only learning, multimodal fusion, and cross-modal supervision and knowledge distillation. Particular attention is paid to how elevation, Doppler measurements, and radar physical priors are exploited across tasks. We further summarize the task coverage, input data, annotations, and evaluation protocols of existing datasets, clarifying the empirical support for different research directions. Finally, we discuss the common challenges and future directions of 4D radar perception for autonomous driving. This review provides a task-oriented perspective on the transition from sparse object perception to dynamic spatial understanding.

0.18

mmHRI: Towards Privacy-Preserving Human-Robot Interaction with Millimeter-Wave Radar

cs.RO cs.CV ·2026-09-28 ·Junqiao Fan, Yuxuan Hu, Bofan Lyu, Yanshuo Lu, Pengfei Liu +5 more
similarity 0.65
Abstract

Assistive robots increasingly operate in many human-centered environments and perform various human-robot interaction (HRI) tasks, such as object delivery. However, most existing HRI systems rely on RGB cameras that continuously observe humans to respond to non-verbal commands, such as hand gestures. This raises privacy concerns in privacy- critical environments, such as hospital wards or restaurants, where direct camera observation of humans is restricted. To develop privacy-preserving HRI, we leverage millimeter-wave (mmWave) radar, which can sense human motion through privacy barriers without identifiable imagery. We propose mmHRI, the first multi-modal robot manipulation framework that achieves mmWave radar-guided privacy-preserving HRI. mmHRI introduces two key designs to mitigate the sparsity and temporal inconsistency of radar data in cluttered robot manipulation environments. First, we propose a dual-stream architecture that jointly learns from unfiltered raw radar tensors and radar point clouds to estimate both human actions and 3D poses. To mitigate signal inconsistency, mmHRI further incorporates a memory-based state-space model (MSSM) that retains historical radar features to reduce abrupt changes in pose/action. These estimated human states are then converted into structured textual robot instructions, which control a vision-language-action (VLA) policy for closed-loop robot manipulation and human-aware reactions. Our evaluation covers human action recognition and closed-loop delivery and retrieval. In the privacy-preserving curtain setting, mmHRI achieves 85.09% action-recognition accuracy, outperforming existing radar-based alternatives. Robot trials further demonstrate successful delivery and retrieval under visual occlusion, with stable task performance across unseen subjects, clutter configurations, and environments.

0.56

Image Frame Dynamic Object Segmentation and Ego Motion Estimation using Radar Image Fusion

cs.CV cs.RO ·2026-09-19 ·Astik Srivastava, Suhani Grover, Avinash Sharma, Madhava Krishna
published similarity 0.64
Abstract · Accepted for presentation at IEEE IROS 2026 Workshop LTP

Dynamic object segmentation and ego-motion estimation are closely coupled problems in autonomous driving, as accurate ego-motion estimation typically requires static scene observations, while identifying static observations requires knowledge of the ego motion. We present Radar-Dot, a radar--RGB framework that exploits radar Doppler measurements to address this coupling. Radar returns are first used to estimate ego velocity through a linear Doppler constraint, with residual-based static/dynamic segmentation and robust estimation used to reduce the influence of moving objects. The estimated motion is then combined with metric depth and dense optical flow to identify image regions whose observed motion is inconsistent with the rigid scene motion. Experiments on 10 nuScenes scenes (part of nuscenes-mini) demonstrate that the resulting geometric pipeline achieves 20.24% dynamic IoU and 33.67% F1-score over 394 frame pairs, while radar-based static-point filtering improves ego-velocity estimation compared with using all radar returns. These results demonstrate the potential of radar as a modality for jointly improving ego-motion estimation and dynamic object segmentation.

0.31

UFMF: Radar-Guided Kinematic Priors for Real-Time 4D Occupancy Forecasting

cs.CV ·2026-10-29 ·Zeping Zhang, Zhuoya Zhao, Robert Laganière
similarity 0.63
Abstract

Real-time 4D occupancy forecasting requires reliably extrapolating both ego motion and surrounding actor motion under strict latency budgets. Recent occupancy world models improve long-horizon fidelity through generative decoders and 3D volumetric modeling, yet their computational cost makes real-time deployment challenging. Light-weight BEV forecasters meet latency requirements but often blur or suppress fast-moving actors at longer horizons. We present UFMF (Ultra-Fast Multi-modal Forecaster), a real-time forecaster that keeps an attention-free diagonal state-space temporal core while injecting targeted kinematic and geometric structure where lightweight models typically fail. A radar-guided velocity-aware dynamic prior (VADP) decomposes future displacement into a linear extrapolation plus a learned horizon-dependent residual. The resulting per-cell flow fields transport a compact dynamic latent subspace during rollout. Two auxiliary training constraints complement this branch: world-frame dynamic supervision enforces inter-horizon trajectory coherence, and class-conditioned vertical-profile supervision regularizes BEV-to-3D lifting. UFMF operates at 122 FPS on a single RTX 4090 and reaches 0.45 fast-displacement mIoU on nuScenes, outperforming recent occupancy world models at a fraction of their computational cost.

0.15

4D Point Splatting for mmWave Radar Micro-Doppler Differentiable Rendering

cs.CV cs.GRcs.LGeess.SP ·2026-11-05 ·Adnan Armouti, Guanyue Qian, Rajalakshmi Nandakumar
similarity 0.62
Abstract · Under Review

Millimeter-wave radar resolves the micrometer displacements of a moving body as micro-Doppler, enabling gait, gesture, activity, pose and vital-sign sensing. Each task relies on a downstream model trained on real data, yet radar datasets lag orders of magnitude behind vision and language. Brute-force collection will not close that gap, and existing synthesis methods (forward-only simulators, learned generators and static-scene inverse renderers) cannot fit a real capture of a moving body, leaving a sim2real gap. We present 4D Point Splatting (4DPS), a physics-based differentiable renderer that fits an explicit moving scene to a real capture. Using the synchronized camera to initialize the room and body, we represent the scene as oriented points carrying material, normal and velocity, and splat each point into range and Doppler bins. Our output is the raw complex return a real radar records, so we apply the same signal processing on the rendered data to obtain any representation for training any SOTA downstream model. We fit 235 sequences on HuPR and 100 on RT-Pose, two datasets with different radars. Our reconstructions set the state of the art, with median correlations of 0.940 and 0.830 on range-azimuth, 0.829 and 0.695 on range-Doppler, and 0.988 and 0.966 on micro-Doppler. Our renders also set the state of the art in downstream sim2real gap. On RT-Pose, a pose model trained on them reaches 21.9 cm MPJPE on real data, against 16.7 cm for real training and 127.6 cm for the RF-Genesis baseline, where both the rendered- and real-trained models learn nearly the same features. Our custom CUDA kernels fit a sequence in 5.7 minutes on HuPR and 12.9 on RT-Pose on one H100, making dataset-scale inverse rendering of mmWave radar possible for the first time. This work can enable synthetic data generation for any human-centered downstream task and model, and scene editing for data augmentation.

0.33

Soft Temporal Alignment Distillation for Camera-to-mmWave Radar Human Activity Recognition

eess.SP ·2026-10-27 ·Jiuqi Yan, Chengbo Zhu, Chengdong Xu, Qisong Wu
similarity 0.62
Abstract · 13 pages, 7 figures

Millimeter-wave (mmWave) radar provides privacy?preserving human activity recognition (HAR), but sparse point clouds limit its representation of body structure and motion. We propose a cross-modal knowledge distillation framework in which a teacher trained on camera-derived two-dimensional (2D) human keypoints supervises a radar-only student. The stu?dent combines PointNet-based point-set encoding, bidirectional temporal modeling, segment-level aggregation, and lightweight shared latent-channel scaling. The teacher retains contextualized states at every time step, whereas the student summarizes the radar sequence into a small number of segments, making fixed timestamp correspondence unsuitable for feature transfer. Soft Temporal Alignment Distillation lets each student segment adaptively retrieve and aggregate teacher temporal evidence into its distillation target. Alignment denotes student-conditioned representation-space matching rather than chronological or monotonic registration. On the Multi-Modal Non-Intrusive 4D Human Dataset (MM-Fi), the method improves radar-only ac?curacy from 87.04% to 89.17%, achieving the highest mean accuracy among the evaluated established and recent distillation baselines. It also obtains higher mean accuracy and macro?averaged F1 score (macro-F1) than the radar student under most evaluated point- and frame-dropout settings. Adaptation to our independently collected millimeter-wave Office Human Ac?tivity Recognition Dataset (mmO-HAR) improves accuracy from 80.33% to 87.75%, supporting the transferability of the distilled radar representation. The teacher and alignment branch are removed after training, enabling privacy-preserving radar-only inference without additional sensing or computational branches.

0.33