CS Paper Radar

SiGMA: Site-Grounded Multi-Modal ISAC Data Generation for Environment-Aware Communications

Yijie Bian, Wei Guo, Kai Zhang, Xiao Ye, Shurui Xu, Shenghui Song, Jun Zhang, Khaled B. Letaief

eess.SP cs.DB ·submitted 2026-11-16 07:27 UTC · v1
code/data

Abstract

Learning-based multi-modal integrated sensing and communication (ISAC) requires data that associate environmental observations with the wireless response generated by the same site and dynamic scene state. Acquiring such data at new sites requires sensor calibration, coordinated measurements, and cross-modal association. We present site-grounded multi-modal ISAC data generation (SiGMA), which combines OpenStreetMap (OSM), Overture Maps, and procedural three-dimensional (3D) reconstruction to create georeferenced urban scenes. A shared scene geometry and time-indexed actor state link synchronized red-green-blue (RGB) images, light detection and ranging (LiDAR) point clouds, positioning records, frequency-modulated continuous-wave (FMCW) Radar in-phase and quadrature (IQ) samples, and ray-traced wireless channels. For configurable reflected propagation under direct-link blockage, SiGMA generates optional reconfigurable intelligent surface (RIS) component channels for phase-configuration evaluation across shared scene states. The evaluation results demonstrate that the framework improves zero-shot beam prediction performance on real DeepSense 6G measurements compared with an OSM-derived synthetic-data baseline. A camera-assisted RIS case study further demonstrates the generation of sensing-associated supervision for configurable reflected propagation. The dataset generation pipeline is available at https://github.com/Trans-cending/SiGMA-main.

Closest papers

Nearest neighbours by meaning across the whole index.

Robust Beam Prediction for V2X Networks with Multi-Modal Sensing

cs.LG ·2026-09-09 ·Chen Shang, Dinh Thai Hoang, Diep N. Nguyen, Jiadong Yu
published similarity 0.67
Abstract · 6 pages, 3 figures

Integrated sensing and communication (ISAC) provides a promising foundation for beam prediction in future vehicle-to-everything (V2X) networks. However, existing sensing-assisted beamforming methods still rely heavily on radio-frequency sensing, which may become unreliable in complex vehicular environments. Meanwhile, the growing availability of heterogeneous sensors, such as cameras and LiDAR, offers new opportunities to improve beam prediction through richer environmental perception. Motivated by this, this paper proposes a multi-modal beam prediction framework for V2X networks. Specifically, we develop BeamTransFuser, a hierarchical Transformer-based architecture that progressively fuses camera, LiDAR, radar, and GPS observations for robust beam prediction. In addition, to handle possible missing modalities in practical deployment, we introduce a generative module that reconstructs missing modality features from the available observations. Experimental results on a real-world multi-modal V2X dataset show that the proposed framework consistently outperforms representative baselines, while the generative module further improves robustness under incomplete sensing conditions.

0.32

AIMS: An Agentic AI Framework for Sim-to-Real Multi-Modal ISAC

cs.AI cs.MAeess.SP ·2026-09-30 ·Yijie Bian, Kai Zhang, Wei Guo, Zixin Wang, Shenghui Song +2 more
benchmark similarity 0.66
Abstract

Multi-modal integrated sensing and communication (ISAC) enables environmental perception and reliable connectivity for intelligent wireless networks. Data-driven multi-modal ISAC models depend heavily on annotated real-world data to learn relationships across sensing and wireless observations, thereby constraining scalable deployment. Although synthetic data generation reduces the burden, adapting existing simulation pipelines to a target deployment requires consistent scene, sensing, wireless, and learning configurations, while mismatches among these coupled components impair sim-to-real transferability. To address the challenge, we propose an agentic artificial intelligence (AI) framework for sim-to-real multi-modal ISAC, named AIMS. Given a natural-language deployment request specifying the target task, deployment conditions, and real-data budget, AIMS derives a deployment-specific sim-to-real configuration and coordinates its execution to produce a deployment-specific task model. A two-agent architecture coordinates scene construction with task learning. A scene construction agent generates geographically grounded, synchronized sensing and wireless records from shared physical states, while a scene understanding agent configures task-relevant modalities and mixture-of-experts (MoE) learning for zero-shot inference or few-shot adaptation. Structured domain knowledge guides dependency-aware planning, while validation evidence supports feedback-driven revision of affected decisions. Experiments on the real-world DeepSense 6G dataset demonstrate improved vehicle detection and beam prediction over the considered simulation and fusion baselines. A separate orchestration benchmark evaluates task interpretation, dependency reasoning, and feedback-driven replanning across diverse deployment requests, showing improved plan correctness with structured domain knowledge and validation feedback.

0.19

VisionSIM: Beyond Idealized Sensors - A Multi-Modal Simulator for Robot Perception

cs.RO cs.AIcs.CV ·2026-11-05 ·Sacha Jungerman, Kaustubh Sadekar, Avery Gump, Sriram Narayanan, Shantanu Gupta +4 more
dataset similarity 0.62
Abstract

Robotic perception increasingly relies on heterogeneous sensor suites combining cameras with emerging modalities (event, SPAD, iToF/dToF, and thermal). Yet existing simulators idealize these sensors, hiding critical modality-specific failure modes that limit real-world deployment. We present VisionSIM, a unified, physically grounded simulation platform that elevates sensor realism to a first-class concern by modeling the raw physical measurement process for diverse conventional and emerging modalities. We expose fine-grained control over physical sensor parameters, reproducing failure modes overlooked by general-purpose simulators: pileup and glare for LiDAR; phase wrapping, saturation, and motion artifacts for iToF; refractory drops for event cameras; and history-dependent emission for thermal cameras. This enables principled cross-modality comparison on identical scenes and trajectories. We release the platform and a large indoor dataset with synchronized raw measurements and rich ground-truth annotations.

0.27

Cross-View Vision-Aided Proactive BS Selection and Beam Prediction for mmWave V2I Communications

eess.SP ·2026-09-01 ·Zijiao Hu, Haiyao Yu, Gaoyang Pang, Guangchen Wang, Litianyi Zhang +4 more
code/data similarity 0.61
Abstract

This paper investigates environmental-sensing-aided proactive base station (BS) selection and beam prediction for millimeter-wave (mmWave) vehicle-to-infrastructure (V2I) wireless systems. We exploit onboard panoramic street-view images and a preloaded satellite map to predict communication-relevant environmental information around the vehicle, including nearby building footprints and heights. The predicted height map provides a compact environmental prior and is combined with historical mobility information to jointly predict the next-slot line-of-sight (LoS) state, transmission rate, and transmit and receive beam selections. On our dataset covering different real-world regions across New South Wales, Australia, the proposed framework achieves 91.4% LoS classification accuracy, 0.638 bps/Hz mean absolute error of data rate prediction, and more than 40% higher transmission rate than the conventional reactive baseline in geographically unseen regions, outperforming all evaluated deployable learning-based baselines. The dataset and code will be released at https://github.com/Huzijiao/Cross-view_V2I

0.25

Bridging the Data Gap: Digital Twin as a New Paradigm for AI-based Radio Sensing

cs.LG eess.SP ·2026-08-11 ·Éloi Sainte-Beuve, Guillaume Larue, Louis-Adrien Dufrène, Quentin Lampin, Ali Al Khansa
published similarity 0.61
Abstract · Accepted at EUSIPCO 2026

We present a methodology that places a 3D digital twin (DT) of the environment as the main enabler behind the development of radio sensing at scale. The DT acts as a world model, providing geometry, materials, and transmitter/receiver placements to a ray-tracing engine that generates time-indexed channel impulse responses (CIRs) for large numbers of plausible scenes (moving people and objects, layout variants, seasonal/weather conditions, etc). From these synthetic sequences, we train a sequential neural network that maps CIR time series to spatial occupancy estimates, enabling device-free localization (DFL) without instrumented targets. We posit that sensing is best approached as an environment-conditioned learning problem: rather than seeking a single global model, we advocate training or fine-tuning local models specialized to a site-specific DT. As a first experiment, we introduce a novel State Space Model architecture, trained and evaluated across multiple room geometries. The localization performances obtained demonstrate the potential of the approach.

0.19

WorkIng: A Real-World and Synthetic Highway Work-Zone Benchmark Dataset for Autonomous Navigation

cs.CV ·2026-10-30 ·Xujun Xie, Dominik Rößle, Adithya Mohan, Daniel Cremers, Torsten Schön
publishedbenchmarkcode/data similarity 0.60
Abstract · Accepted to WACV 2027. 19 pages including supplementary material. (c) 2027 IEEE. Personal use of this material is permitted

Highway work zones are critical edge cases for autonomous vehicles (AVs), combining atypical geometries and jurisdiction-specific regulations that existing benchmarks fail to capture: real-world datasets lack 360° multi-modal, full-scene 3D work zone annotation, and no single recorded site can span the layouts and regulatory variants needed for generalizable models. We present WorkIng, a dual-domain benchmark and training resource. WorkIng-Highway provides a 360° multi-modal work zone benchmark with synchronized 128-channel LiDAR, six RGB cameras, and RTK-GPS across 32 km, yielding 12,000+ annotated frames and 227,399 tracked 3D bounding boxes. WorkIng-CARLA complements this with procedural simulation spanning 24 regulation-aware scenarios across six canonical topologies and six international traffic standards, populated by 100+ custom 3D assets. Benchmarking representative 3D detection and end-to-end planning architectures shows that real-world data remains the primary performance driver, and that scenario-stratified synthetic augmentation improves architectures across 2D recognition, 3D detection, and trajectory planning. Resources: https://github.com/cvims/WorkIng

0.42

PALIN: Pixel-Aligned LiDAR-NIR Multimodal 3D Object Detection with Benchmark

cs.CV ·2026-11-14 ·Yanan Zhang, Amit Vishnia, Chengbo Ai, Eleni Christofa, Runan Cheng +4 more
benchmark similarity 0.60
Abstract · 17 pages, including references and appendices

3D object detection is critical for intelligent transportation systems. LiDAR has been widely adopted for accurate geometric measurements; however, purely geometric representations remain ambiguous under sparse observations and occlusions, while cross-sensor multimodal fusion suffers from calibration and synchronization errors. To overcome these challenges, we leverage an Ouster LiDAR to construct PALIN (Pixel-Aligned LiDAR-NIR), a large-scale intrinsically pixel-aligned multimodal dataset jointly capturing geometry and NIR intensity, comprising approximately 8,000 annotated frames from static urban intersections and diverse mobile suburban scenes. Building upon this dataset, we propose CMambaFusion, a novel fusion framework consisting of a four-directional Cross-Mamba module for bidirectional cross-modal interaction and a Gated Concat-Mamba module for joint feature refinement. Extensive experiments demonstrate that PALIN enables effective multimodal learning, and CMambaFusion achieves superior accuracy-efficiency trade-offs over both LiDAR-only and representative fusion baselines.

0.18

GLocFM: A Geometry-Aware Foundation Model for 3D Indoor Wireless Localization

eess.SP cs.AI ·2026-08-10 ·Chenghong Bian, Chaozheng Wen, Hongze Chen, Jun Zhang
similarity 0.60
Abstract · 13 pages, single column

Learning-based wireless localizers often fail to utilize geometric information about the propagation environment, limiting their ability to exploit non-line-of-sight (NLoS) propagation and generalize across scenes. To bridge this gap, we propose GLocFM, a Geometry-aware Localization Foundation Model, which jointly exploits WiFi measurements and scene geometry represented as a 3D point cloud. We formulate localization as a maximum-likelihood (ML) estimation problem, where the goal is to find a transmitter position that maximizes the likelihood of the wireless observations conditioned on the scene geometry. The likelihood of a candidate transmitter position is calculated by a learned scoring function that matches the observed delay--angle-of-arrival (AoA) spectrum against the spectrum predicted for that candidate. A hierarchical scene encoder extracts propagation-relevant features to produce geometric priors for LoS and one-bounce reflection paths. For scenarios with imperfect synchronization, we further introduce a time-of-flight (ToF)-robust GLocFM model to handle unknown ToF offsets. GLocFM is trained on a multi-modal synthetic indoor localization dataset comprising 221 diverse scenes whose associated wireless signals are generated using Sionna RT. On both synthetic and the NeRF$^{2}$ dataset based on real measurements, GLocFM reduces mean 3D localization error relative to one of the state-of-the-art localization baselines by 49.5\% and 48.8\%, respectively. Ablations across different number of receiver, bandwidths, and array sizes further demonstrate the effectiveness and robustness of the proposed framework.

0.40