CS Paper Radar

mmIR: Frequency-Space Inverse Rendering for 3D Millimeter-Wave Radar ADC Synthesis

Adnan Armouti, Yixuan Gao, Rajalakshmi Nandakumar

cs.CV cs.GRcs.LGeess.SP ·submitted 2026-08-28 22:21 UTC · v1
top venuecode/data

Abstract

High-resolution 3D radar data is scarce. Commodity mmWave sensors use small antenna arrays that limit angular resolution to several degrees, and existing datasets provide only 2D range-azimuth maps or sparse point clouds rather than raw analog-to-digital converter (ADC) signals. Hardware scaling is expensive, synthetic-aperture scanning is impractical at fleet scale, and learned synthesis methods are bottlenecked by the very data shortage they aim to address. We present mmIR, an open-source differentiable frequency-modulated continuous-wave (FMCW) radar inverse renderer that fits a physics-based forward model to real captures and re-renders from dense virtual apertures to synthesize high-resolution 3D radar data. Because radar resolution is too coarse to recover geometry directly, mmIR performs LiDAR-assisted inverse rendering: using LiDAR-derived meshes as a geometric scaffold, mmIR optimizes per-vertex International Telecommunication Union (ITU) physics materials, vertex normals, and antenna beam patterns through end-to-end automatic differentiation of a phase-coherent multiple-input multiple-output (MIMO) forward model with multi-bounce propagation, polarization, and free-space diffraction. On seven outdoor and six indoor ColoRadar scenes, mmIR achieves 0.914 mean Pearson correlation on range-azimuth maps versus 0.307 for Sionna-RT. Scenes trained on a cascaded imaging radar transfer to a co-located single-chip radar without re-training (0.554 correlation), and dense virtual arrays (100x100 elements) produce single-frame 3D occupancy validated against LiDAR. Project page: https://mmwave-inverse-rendering.github.io/

Comment
Accepted to the European Conference on Computer Vision (ECCV) 2026. Project page: https://mmwave-inverse-rendering.github.io/

Closest papers

Nearest neighbours by meaning across the whole index.

SPIRIT: Sparse Physics-Informed Inverse Rendering for SAR Imaging

cs.CV cs.AI ·2026-10-01 ·Yimeng Liu, Maolin Gan, Jingkai Lin, Jialuo Du, Mi Zhang +1 more
similarity 0.69
Abstract

Radio-frequency (RF) and wave-based imaging systems play a critical role in sensing under adverse conditions, but are fundamentally constrained by a trade-off between collection speed and spatial resolution. Reducing measurements to enable rapid collection of fragment geometric information across sensing dimensions, leading to highly non-injective inverse problems that destabilize conventional image-domain reconstruction and super-resolution methods. Synthetic aperture radar (SAR) exemplifies this challenge in sparse-aperture sampling. We introduce SPIRIT, a physics-informed inverse rendering framework for sparse-aperture SAR. Rather than enhancing formed images, SPIRIT reconstructs an explicit three-dimensional surface. By adopting an optimization strategy and explicitly modeling visibility, phase coherence, and wave propagation, SPIRIT aggregates distributed geometric features across sparse aperture views into a coherent reconstruction. Extensive experiments on synthetic and real-world data demonstrate that SPIRIT recovers accurate geometry and signal under extreme sparsity, reducing data collection by over 90% while achieving low geometric and signal error and consistently outperforming baselines.

0.17

3D Point Splatting for mmWave Radar Novel View Synthesis

cs.CV cs.GRcs.LGeess.SP ·2026-09-10 ·Adnan Armouti, Yixuan Gao, Rajalakshmi Nandakumar
top venuecode/data similarity 0.69
Abstract · Accepted to the Conference on Neural Information Processing Systems (NeurIPS) 2026. Project page: https://3d-point-splatting.github.io/

Solving novel view synthesis (NVS) for millimeter-wave (mmWave) radar requires a renderer that is physically faithful, complex-valued, and multi-viewpoint-tractable. No prior method achieves these three properties simultaneously. Differentiable Monte Carlo (MC) ray tracers implement the radar forward model directly with explicit material modeling and complex outputs, but a joint fit across views takes over an hour per scene. Optical-NVS ports of NeRF, hash grids, and 3D Gaussians train fast but discard phase and replace explicit material modeling with opaque learned features, restricting them to power-only range-azimuth (RA) magnitudes. We propose 3D Point Splatting (3DPS), the first differentiable point splatting renderer for radar, derived directly from the standard solid-angle form of the radar equation. Each oriented 3D point carries a material model built on ITU-R P.2040 permittivity and slab reflection, evaluated in closed form, with the resulting complex phasor splatted into range bins through a precomputed point spread function (PSF). The complex-valued output makes the renderer product-agnostic. The same optimized scene yields analog-to-digital converter (ADC), complex range profile (CRP), and RA outputs through standard fast Fourier transform (FFT) pipelines without retraining for each format. On six outdoor ColoRadar scenes, 3DPS reaches 0.587 mean Pearson correlation on held-out RA images. This is between 1.7x and 5.2x the three optical-NVS baselines (RadarSplat, Radar Fields, DART) and 1.3x the mmIR ray tracer at 1/25 of its wall-clock time. Training takes approximately 3 minutes per scene on a single RTX 4090. Project page: https://3d-point-splatting.github.io/

0.47

AuRaSim: A Ray-Tracing Based Automotive FMCW Radar Simulator for Dynamic Scenarios

cs.RO eess.SP ·2026-10-27 ·Zhaoze Wang, Changxu Zhang, Tai Fei, Christopher Grimm, Yi Jin +2 more
similarity 0.65
Abstract

Modern simulators can generate synthetic camera and LiDAR data with accurate annotations for downstream tasks. In contrast, automotive radar simulation remains less mature because it requires not only a 3D engine to model dynamic objects in the scenes but also electromagnetic material properties, antenna configurations, wave propagation mechanisms, and coherent signal synthesis. Existing frameworks are either proprietary or overly simplified, limiting the scalability of radar raw measurements. This paper presents AuRaSim, a ray-tracing-based automotive frequency-modulated continuous wave (FMCW) radar simulation framework for dynamic scenarios. Rather than implementing all components from scratch, AuRaSim builds on Blender for efficient dynamic-scene construction and Sionna RT for graphics processing unit (GPU) accelerated electromagnetic ray tracing, while introducing radar-specific adaptations for path-resolved propagation with complex coefficients, motion-aware Doppler modeling, and subsequently coherent multi-channel analog-to-digital converter (ADC) cube synthesis based on waveform configurations. Quantitative comparison between simulated and measured range-Doppler maps in digital-twin scenarios achieves a root-mean-square error (RMSE) of 0.147 and a peak signal-to-noise ratio (PSNR) of 16.93 dB, demonstrating the accuracy and fidelity of the synthetic signal. AuRaSim therefore provides a complete and extensible platform for reproducible automotive radar simulation and research. Project homepage: https://wangzhaoze.github.io/aurasim.

0.31

RadPose: ISAC-based RF 3D Human Pose Estimation

cs.CV cs.NI ·2026-11-01 ·Geo Jie Zhou, Kunzhe Song, Ruxin Lin, Huacheng Zeng
publishedcode/data similarity 0.60
Abstract · Accepted to WACV 2027

Human pose estimation under adverse lighting and occlusion conditions remains challenging for camera- and LiDAR-based systems. Although mmWave radars provide robustness to illumination and occlusion, they rely on dedicated hardware and licensed spectrum, limiting scalability. In this paper, we present RadPose, a learning-based monostatic integrated sensing and communication (ISAC) framework that enables mmWave communication devices to perform 3D human pose estimation. Unlike specialized radars, RadPose leverages monostatic full-duplex operation to extract channel impulse responses (CIRs) directly from standard OFDM communication signals, generating 3D radio point clouds without additional sensing hardware. To address the sparsity and noise of RF observations, RadPose employs a cross-frame transformer-based pose inference network with semantic-guided spatial attention and target-driven temporal attention for robust spatio-temporal aggregation. It further adopts a joint-aware inverse kinematics refinement module based on graph convolution to enforce skeletal topology, bone-length consistency, and anatomical plausibility. Extensive experiments demonstrate RadPose achieves competitive accuracy compared to state-of-the-art radar-based approaches while maintaining robust performance in unseen and occluded scenarios. The dataset and source code are available at https://github.com/Geo-Chou/RadPose.

0.46

3D Radar Imaging from the UAV Nadir

eess.SP ·2026-09-04 ·S. Hamed Javadi, Hichem Sahli, André Bourdoux
top venue similarity 0.60
Abstract · Accepted for publication in IEEE Transactions on Radar Systems

Radars improve the sensing robustness of UAVs by operating under poor lighting and weather conditions and seeing through occlusions such as vegetation. However, they suffer from poor angular resolution, which can be addressed using synthetic aperture radar (SAR) algorithms. State-of-the-art UAV SAR methods operate at a depression angle and are not suitable for sensor fusion applications where the data are collected from areas directly below the UAV (i.e., the UAV nadir). In this paper, we present an interferometric SAR (InSAR) framework for reconstructing 3D images from the UAV nadir using a low-cost multi-input-multi-output (MIMO) mm-wave radar. Additionally, an effective method based on the phase gradient autofocus (PGA) is presented for compensating the phase error across the virtual receive antennas. We demonstrate the effectiveness of our 3D imaging algorithm in both simulation and experimental scenarios.

0.35

DyRAD: Radar Novel View Synthesis for Dynamic Driving Scenes

cs.CV ·2026-09-30 ·Merav Keidar, Tomer Borreda, Rajalakshmi Nandakumar, Or Litany
benchmarkcode/data similarity 0.60
Abstract · Project page: https://dyrad-nvs.github.io/. Code: https://github.com/Dyrad-NVS/DyRAD

Reconstructing dynamic driving scenes from recorded sensor data supports closed-loop evaluation of autonomous driving systems by synthesizing observations beyond the original trajectory. Unlike cameras and LiDAR, radar measures radial velocity directly through Doppler. Yet existing radar novel-view synthesis fails to exploit this capability: methods addressing dynamic scenes reconstruct only range-azimuth tensors, while methods that render Doppler assume static scenes. Moreover, because radar processing spreads each reflection across multiple bins, existing representations absorb this spread into scene geometry, causing it to render incorrectly when the viewpoint moves. We present DyRAD, which models dynamic driving scenes using static background reflectors and motion-tracked dynamic point reflectors to render complete range-azimuth-Doppler (RAD) tensors. Reflector velocities are derived from object tracks and projected onto the line of sight, making Doppler both a rendered output and supervision for those tracks. Crucially, we render reflectors through a fixed analytic point-spread function (PSF) derived from the radar's signal-processing chain, preventing sensor-induced spread from being baked into the scene representation. Beyond improving scene reconstruction, this separation also enables zero-shot sensor-configuration transfer, allowing the same reconstructed scene to be rendered under different radar specifications without refitting. We evaluate DyRAD on RADIal, Boreas, and a synthetic benchmark across both on-path poses and displaced viewpoints untested by prior work. On RADIal, DyRAD recovers radar detections in 90.7% of reference-detected objects, compared with 26.9% for the strongest baseline.

0.27

4D Point Splatting for mmWave Radar Micro-Doppler Differentiable Rendering

cs.CV cs.GRcs.LGeess.SP ·2026-11-05 ·Adnan Armouti, Guanyue Qian, Rajalakshmi Nandakumar
similarity 0.59
Abstract · Under Review

Millimeter-wave radar resolves the micrometer displacements of a moving body as micro-Doppler, enabling gait, gesture, activity, pose and vital-sign sensing. Each task relies on a downstream model trained on real data, yet radar datasets lag orders of magnitude behind vision and language. Brute-force collection will not close that gap, and existing synthesis methods (forward-only simulators, learned generators and static-scene inverse renderers) cannot fit a real capture of a moving body, leaving a sim2real gap. We present 4D Point Splatting (4DPS), a physics-based differentiable renderer that fits an explicit moving scene to a real capture. Using the synchronized camera to initialize the room and body, we represent the scene as oriented points carrying material, normal and velocity, and splat each point into range and Doppler bins. Our output is the raw complex return a real radar records, so we apply the same signal processing on the rendered data to obtain any representation for training any SOTA downstream model. We fit 235 sequences on HuPR and 100 on RT-Pose, two datasets with different radars. Our reconstructions set the state of the art, with median correlations of 0.940 and 0.830 on range-azimuth, 0.829 and 0.695 on range-Doppler, and 0.988 and 0.966 on micro-Doppler. Our renders also set the state of the art in downstream sim2real gap. On RT-Pose, a pose model trained on them reaches 21.9 cm MPJPE on real data, against 16.7 cm for real training and 127.6 cm for the RF-Genesis baseline, where both the rendered- and real-trained models learn nearly the same features. Our custom CUDA kernels fit a sequence in 5.7 minutes on HuPR and 12.9 on RT-Pose on one H100, making dataset-scale inverse rendering of mmWave radar possible for the first time. This work can enable synthetic data generation for any human-centered downstream task and model, and scene editing for data augmentation.

0.33

RadarDINO: Distilling a Visual Foundation Model into mmWave Radar for Human Mesh Reconstruction

cs.CV ·2026-11-06 ·Furkan Mert Algan, Zhifan Ni, Eckehard Steinbach
similarity 0.57
Abstract

Millimeter-wave (mmWave) radar offers a privacy-preserving and illumination-robust alternative for human mesh reconstruction (HMR). However, high-fidelity radar HMR remains challenging. Sparse point cloud-based methods suffer from information loss and missed detections, while raw radar tensor methods like RT-Mesh lack fine-grained semantics and explicit local body modeling. In this paper, we propose RadarDINO, a radar HMR framework that distills visual semantics from a DINOv3 teacher into 3D radar representations and introduces explicit geometric supervision. DINOv3 descriptors are transferred to radar voxels via calibrated RGB-radar geometry, providing voxel- and body-part-level supervision for part-aware radar representations. We further incorporate vertex- and joint-losses to directly constrain reconstructed geometry. On the M4Human dataset, our method outperforms the SOTA baseline across all three evaluation splits.

0.19