CS Paper Radar

Demo: Vision-Language Model-Guided Online Calibration of an Electromagnetic Digital Twin

Zerui Kang, Yishen Lim, Zhouyou Gu, Seungnyun Kim, Seung-Woo Ko, Tony Q. S. Quek, Jihong Park

cs.RO cs.AI ·submitted 2026-10-05 11:15 UTC · v2
published

Abstract

An electromagnetic (EM) digital twin gives mobile robots wireless situational awareness but depends on material conductivities that change with the environment. Online calibration faces initialization sensitivity and measurement travel costs. We demonstrate a vision-language model (VLM)-guided framework using a Unitree G1 robot and NVIDIA Sionna, with two VLM calls: material classification maps visible materials through ITU-R P.2040 to conductivity priors for Sionna's gradient descent on accumulated received signal strength (RSS) measurements; waypoint planning selects the next measurement location online using residual RSS calibration error and image coverage. In a real indoor scenario, the framework achieves a normalized mean absolute conductivity error of $1.74\times10^{-4}$ within 20 m of travel; random initialization never converges, while random waypoints require over twice the travel.

DOI
10.1145/3842721.3850098

Closest papers

Nearest neighbours by meaning across the whole index.

Turning Multipath into Positioning Information: Persistent Electromagnetic-Twin DoA Localization

eess.SP ·2026-10-29 ·Tuo Wu, Zhentian Zhang, Ye Tian, Hing Cheung So
similarity 0.60
Abstract · 13 pages, 11 figures

Reflected paths can support localization when line-of-sight (LoS) links are blocked, but their usefulness depends on how accurately the stored model predicts their directions. This paper develops electromagnetic-twin (ET) direction-of-arrival (DoA) localization with a persistent, physically calibrated path-angle field. A nominal floor plan supplies virtual-path geometry, and sparse reference probes at known positions learn a compact directional correction. Field order and regularization are selected from calibration data alone. For each user, unordered DoAs are processed by joint position estimation and one-to-one path association, using assignment-aware multistart search and robust Gauss--Newton refinement. New trusted probes update the stored correction through recursive least squares (RLS), without reprocessing all previous measurements. The analysis establishes calibration rank conditions, objective descent, and conditional information gains from reflected paths, while also distinguishing the decay of initialization bias from the convergence of localization error. Synthetic experiments use 500 queries per operating point. At 50% LoS blockage and $1.5^\circ$ DoA error, the calibrated ET achieves 0.39 m root-mean-square error, versus 0.55 m for nominal geometry. After a wall change, its final error is 0.43 m, versus 0.63 m for a frozen calibrated map. Equal-evidence comparisons show competitive calibrated alternatives and faster abrupt-change recovery with sliding-window updates than with the configured RLS. The results support reusable directional calibration, not a universally superior map or update rule, and validation is limited to synthetic first-order reflections with trusted calibration labels.

0.39

Navigation with RF Cues: Embodied Perception Action under Multipath Uncertainty

cs.RO ·2026-10-06 ·Wenlihan Lu, Tianshun Li, Liuqing Yang, Shijian Gao
benchmark similarity 0.56
Abstract

Smart factory inspection requires robots to reach connected equipment without a prior map or known target coordinates. Radio frequency (RF) signals from the target can provide directional cues to complement visual observations when occlusion or poor lighting limits target detection. However, multipath propagation can distort these cues, making it difficult to infer the target's true direction from instantaneous RF measurements. To enable navigation research under these conditions, we first construct a Habitat Sionna RT benchmark that uses detailed scene geometry and assigned material properties to generate aligned visual and RF observations in response to robot actions. Building on this benchmark, we propose an uncertainty aware multimodal navigation framework that jointly estimates target direction and its uncertainty from a history of RF, visual, and pose observations. These estimates inform action selection alongside visual context. Experiments in unseen scenes show relative improvements of 18.2% in success rate (SR) and 11.5% in success weighted by path length (SPL) over the strongest evaluated baseline.

0.56

Physical AI for Intralogistics: Benchmarking the NVIDIA Sim-to-Real Ecosystem for Cosmos-Enhanced COMPASS Navigation on Industrial Mobile Robots

cs.RO ·2026-10-20 ·Charbel Abi Hana, Rayan Zaitouni, Kameel Amareen, Mohamad Mostafa, Hasan Sayour +4 more
benchmark similarity 0.56
Abstract

Classical robot navigation stacks are reliable but often require re-engineering for each new robot type and site. Learning-based navigation shifts this effort from hand-tuned parameters to data generation and training, where a policy adapts to each new embodiment or facility by fine-tuning in simulation. Whether policies adapted this way stay reliable on real industrial robots has seen little systematic testing. We study this for NVIDIA's X-Mobility and COMPASS workflow for training learned navigation policies on Idealworks robots. In simulation, we extend the COMPASS benchmark with two embodiments, the differential-drive iw$\mkern-1.5mu.\mkern-1.5mu$hub and an omnidirectional Humanoid Mobile Platform (HMP), and with a digital twin of the Idealworks R&D center. For the real world, we fine-tune X-Mobility on data from this digital twin, with and without sim-to-real variants generated by NVIDIA Cosmos, train COMPASS on top of it from scratch in the digital twin, deploy the resulting policies on the iw$\mkern-1.5mu.\mkern-1.5mu$hub's onboard Jetson AGX Xavier, and benchmark them in the Idealworks R&D center against COMPASS trained only on NVIDIA's environments and the existing Idealworks navigation stack on task success, efficiency, safety, smoothness and onboard compute. Running every static scenario both in a digital-twin replica and in the real facility measures the sim-to-real gap and how far Cosmos-augmented data closes it. Training in the digital twin with Cosmos sim-to-real augmentation raises the real-world success rate of learned navigation from 65.3% to 91.7% across four scenarios and nearly halves its weighted travel time compared with simulation-only training, bringing it within 8.3 points of the production navigation stack, fully onboard; with an integrated global planner, it also reaches far, occluded goals in every trial.

0.13

TRACE: Learning to Self-Calibrate Wireless Digital Twins from ISAC Measurements

cs.AI eess.SP ·2026-09-26 ·Saad Masrur, Saeed R. Khosravirad, Ismail Guvenc
similarity 0.55
Abstract · Under Review

Wireless digital twins (DTs) rely on 3D environment models to predict radio propagation and support wireless-network decisions, yet these models are often initialized from imperfect 3D maps. Errors in building position, height, footprint, and orientation can therefore cause a high-fidelity propagation engine to simulate the wrong physical environment. In this paper, we study how a deployed wireless network can repair an existing DT using its own radio frequency (RF) measurements. In particular, we introduce Twin Residual Alignment and Calibration Engine (TRACE), a physics-grounded learning-based self-calibration framework that treats twin maintenance as residual alignment between the physical world and the current DT. Using the same sensing configuration as the physical measurements, TRACE ray-traces the current DT, coherently backprojects the measured and simulated RF onto a common world grid, and extracts the same local region around each building's current DT position. A multi-view corrector then fuses evidence across sensing nodes and neighboring buildings to predict a gated six-parameter correction per building, without relying on absolute layout or sensor ordering, and supports iterative correction through re-rendering. On 5,400 held-out samples from unseen simulated scenes at 28 GHz, TRACE reduces 3D position RMSE from 2.202 m to 0.302 m and yaw RMSE from 4.978° to 0.894°, outperforming ViT and U-Net baselines under changes in layout, building count, sensing-node count, and SNR. On measured 28 GHz RF data from the NIST outdoor courtyard, a model trained only on synthetic RF reduces mean planar wall-position error from 1.00 m to 7.8 cm, without measured-data fine-tuning or geometric labels. These results show that the discrepancy between measured and twin-rendered RF can serve as a learning signal for repairing a wireless DT.

0.12

Electromagnetic Twin: From Sparse Measurements to Persistent Wireless Intelligence

eess.SP ·2026-08-25 ·Tuo Wu, Lifeng Mai, Jie Tang, Shihang Lu, Kangda Zhi +5 more
similarity 0.55
Abstract

Radio maps and channel knowledge maps provide reusable propagation knowledge, but a stored map can become locally outdated after persistent changes to doors, partitions, furniture, large equipment, or infrastructure. Motivated by digital-twin state synchronization, we develop an electromagnetic twin that recurrently converts sparse channel measurements into a persistent wireless state, exposes that state to communication queries, and uses its uncertainty to request subsequent measurements. The twin tracks persistent or semi-persistent propagation changes rather than transient human motion or fast fading. Each update is a constrained inverse problem that combines the previous map, a scene graph, and an imperfect physics prior. Measurement consistency with a confidence-calibrated radius and physical gain bounds enforce feasibility, while scene-aware spatial regularization and selective temporal memory preserve propagation boundaries and unchanged regions. Successive convex approximation (SCA), majorization--minimization alternating direction method of multipliers (MM-ADMM), and a low-complexity primal--dual hybrid gradient (LC-PDHG) mode realize the framework at different computational scales, and a local perturbation analysis gives an explicit multi-update tracking recursion. The updated state supports access-point association and codebook beam selection, while change-weighted A-optimal design closes the measurement--update--query loop. Over 500 realizations, the recurrent update remains stable through eight persistent scene events and raises best-beam accuracy from $76.64\%$ to $90.83\%$ as measured-location density grows from $2\%$ to $16\%$.

0.27

Adaptive Calibration of RF Digital Twins via Budget-Constrained Radio Measurement Selection

cs.NI eess.SP ·2026-10-26 ·Yalin E. Sagduyu, Tugba Erpek, Hossein Jafari
published similarity 0.54
Abstract

Radio-frequency (RF) digital twins (DTs) enable site-specific wireless propagation prediction by explicitly reconstructing the physical environment and modeling electromagnetic interactions. Although physics-based DTs reduce reliance on exhaustive measurements, systematic bias relative to real radio data arises from environmental uncertainty, material inaccuracies, and geometric abstractions. Calibration using field measurements is therefore necessary but costly under limited sampling budgets. This paper studies active RF DT calibration, in which measurement locations are selected sequentially and in an online manner to estimate a calibration offset under strict measurement budget constraints. A physics-based RF DT pipeline is developed, integrating semantic geospatial data from OpenStreetMap, 3D modeling in Blender, and deterministic ray tracing in Sionna to capture dominant building and foliage effects. This RF DT is calibrated using path-loss measurements collected with software-defined radios. Multiple online calibration strategies that select radio measurement locations are compared, including random sampling, rank-structured sampling with balanced coverage, quasi-Monte Carlo ranked selection, progressive structure-aware allocation, and fully adaptive uncertainty-driven sampling with estimator stabilization. Results demonstrate that adaptive and structure-aware schemes significantly improve both mean and worst-case calibration accuracy, particularly in the low-budget regime for radio measurements.

0.33

MAPLE-RF: Efficient Probabilistic RF Source Localization in Partially Explored Environments

cs.RO eess.SP ·2026-09-17 ·Haozhe Lei, Sundeep Rangan
similarity 0.54
Abstract

Localizing a radio-frequency (RF) transmitter from received signals often requires a model of the environment to predict how obstacles block and reflect the signal. In many robotic applications, however, only a partial map is available, particularly when a robot localizes the source while exploring with simultaneous localization and mapping (SLAM). We study single-snapshot transmitter localization on such partially explored maps and compare two approaches that output a posterior over transmitter locations. The first extends a digital-twin method, which ray-traces every candidate location, to partial maps by treating unexplored space as free and training on mixed map coverage. The second, MAPLE-RF, encodes estimated path angles of arrival and signal-to-noise ratios as grid channels aligned with map knownness, occupancy, and line-of-sight visibility, and a U-Net scores all candidate positions in one pass without simulating propagation at inference. Ray-tracing simulations of indoor environments indicate that training on mixed map coverage is essential for both approaches. The digital-twin approach is more accurate on most single-snapshot metrics, while MAPLE-RF comes close at a query cost that does not depend on the propagation model and is more than two orders of magnitude below a fresh full-grid query with general-purpose ray tracing. Both outperform Gaussian and Gaussian-mixture baselines, and on exploration routes guided by its own estimates, fused MAPLE-RF posteriors place more probability near the source than the compared methods. Code and data will be released.

0.29

Bridging the Data Gap: Digital Twin as a New Paradigm for AI-based Radio Sensing

cs.LG eess.SP ·2026-08-11 ·Éloi Sainte-Beuve, Guillaume Larue, Louis-Adrien Dufrène, Quentin Lampin, Ali Al Khansa
published similarity 0.54
Abstract · Accepted at EUSIPCO 2026

We present a methodology that places a 3D digital twin (DT) of the environment as the main enabler behind the development of radio sensing at scale. The DT acts as a world model, providing geometry, materials, and transmitter/receiver placements to a ray-tracing engine that generates time-indexed channel impulse responses (CIRs) for large numbers of plausible scenes (moving people and objects, layout variants, seasonal/weather conditions, etc). From these synthetic sequences, we train a sequential neural network that maps CIR time series to spatial occupancy estimates, enabling device-free localization (DFL) without instrumented targets. We posit that sensing is best approached as an environment-conditioned learning problem: rather than seeking a single global model, we advocate training or fine-tuning local models specialized to a site-specific DT. As a first experiment, we introduce a novel State Space Model architecture, trained and evaluated across multiple room geometries. The localization performances obtained demonstrate the potential of the approach.

0.19