CS Paper Radar

An Overview on Multimodal Wireless and Visual Sensing for Environment Reconstruction: Opportunities and Challenges

Guanlin Wu, Chao Hu, Yilong Chen, Yudong Guo, Rongfei Fan, Juyong Zhang, Han Hu, Jie Xu

eess.SP ·submitted 2026-11-07 06:28 UTC · v1
survey

Abstract

Physical-scene reconstruction through visual sensing and channel knowledge map (CKM) reconstruction through wireless sensing have attracted growing research interest but have largely evolved independently within the computer vision and wireless communications communities, respectively. When applied to the same physical environment, the two sensing modalities provide complementary observations. Visual sensing captures detailed scene geometry and appearance but is constrained by occlusion and poor lighting. Wireless sensing can provide information beyond visual line of sight regardless of lighting conditions, but its measurements are often sparse and difficult to associate with scene structures. These complementary strengths motivate a unified multimodal approach to physical-scene and CKM reconstruction. This paper provides an overview of such unified environment reconstruction through wireless and visual sensing. We first compare the capabilities and limitations of the two modalities and discuss potential applications that benefit from jointly reconstructing both targets. We then review two complementary research directions, namely wireless-assisted physical-scene reconstruction and vision-assisted CKM reconstruction, highlighting how each modality supports the reconstruction task associated with the other. Building on these perspectives, we discuss design principles for a joint representation that supports both targets and identify mutual refinement, in which each modality helps improve the reconstruction associated with the other. Finally, we discuss open challenges and future research directions, offering insights into developing site-specific environment models for sixth-generation (6G) and beyond networks.

Closest papers

Nearest neighbours by meaning across the whole index.

Scene Reconstruction Using Ubiquitous Wireless Signals via Diffusion Posterior Inference Over physics-Grounded Representations

cs.LG ·2026-10-15 ·Vahid Yazdnian, Yasaman Ghasempour
similarity 0.65
Abstract

Knowing the layout of the physical world matters for many applications, yet the sensors we rely on for it (e.g., cameras and LiDAR) can fail in darkness, in adverse weather, or behind walls. Wireless signals do not face these limitations: every link between a base station and a user is shaped by the buildings it passes, reflects from, or is blocked by, leaving information about the surrounding scene embedded in the wireless channel measured at the endpoints. However, mapping channel measurements to the environment is a severely ill-posed problem: a measurement records only the structures that interact with the transmitted electromagnetic waves along its propagation paths, and even these observations can often be explained by multiple plausible scenes. We therefore formulate wireless scene reconstruction as posterior inference and develop a hierarchical framework with two complementary stages to approach this problem. A wireless foundation model, pretrained to respect propagation physics, transforms each channel measurement into a representation of the scene structure that shaped the observation. A diffusion model then jointly integrates representations from multiple measurements to model and sample the posterior over building occupancy, with consensus over samples producing the final reconstruction. We demonstrate that the framework can accurately recover building footprints from wireless channel measurements alone and show that its performance depends critically on both design components: physics-grounded representations expose scene-relevant information, while joint posterior inference coherently integrates complementary observations.

0.17

Electromagnetic World Model for 6G: A Unified Framework for Joint Environment Reconstruction and Channel Prediction

eess.SP ·2026-08-18 ·Yizhu Zhao, Li Yu, Jianhua Zhang, Yuxiang Zhang, Zhen Zhang +1 more
published similarity 0.62
Abstract

The integration of sensing, communication, and intelligence is becoming a key enabler for sixth generation (6G) wireless systems, where intelligent terminals are expected to simultaneously support efficient link establishment and reliable environmental sensing. However, existing studies mainly exploit sensing information or communication information to address a single task, such as channel prediction or environment reconstruction. Motivated by the shared dependence of optical and radio-frequency signals on the surrounding environment, we propose the electromagnetic world model (EMWM), the first unified framework for joint environment reconstruction and channel prediction. EMWM learns a common electromagnetic representation with the potential to provide a modeling foundation for 6G tasks. Specifically, partial channel state information (CSI) and multi-view red-green-blue (RGB) images are encoded into CSI and visual tokens and jointly processed by a hierarchical world-model backbone with local and global aggregation. Based on the learned representation, a mixture-of-experts (MoE)-based CSI prediction head reconstructs the complete CSI, while a depth prediction head estimates multi-view depth maps that are further converted into three-dimensional (3D) point clouds. Moreover, a large-scale multi-modal dataset is constructed based on a campus digital twin. Experimental results show that EMWM outperforms conventional neural network and large language model (LLM) baselines in both CSI prediction and environment reconstruction, achieving a squared generalized cosine similarity (SGCS) of 0.9699 for CSI prediction while demonstrating robustness across different signal-to-noise ratio (SNR) conditions and zero-shot generalization at 28 GHz.

0.36

Synergizing CKM and ISAC for Environment-Aware Wireless Networks

eess.SP ·2026-11-08 ·Juncong Zhou, Yilong Chen, Yuan Guo, Zixiang Ren, Han Hu +2 more
survey similarity 0.61
Abstract

Channel knowledge map (CKM) and integrated sensing and communications (ISAC) have emerged as two key technologies for sixth-generation (6G) and beyond wireless networks, enabling a new paradigm toward environment-aware system design. CKMs provide static or stochastic wireless environment information a priori by learning location-dependent mappings from transceiver positions to channel knowledge of various types, whereas ISAC exploits wireless signals to sense environmental objects in real time. CKM and ISAC have primarily been studied separately in prior works, while their integration offers a promising approach for enhancing the environment awareness of 6G networks and further improving both sensing and communication capabilities. On the one hand, sensing can support CKM construction by providing location-tagged measurements and richer environmental observations, while also enabling real-time CKM refinement in dynamic environments. On the other hand, CKM can enhance sensing by offering prior environmental information for clutter mitigation, and CKM and sensing information can be jointly leveraged to improve communications. This article provides an overview of the complementary interactions between CKMs and ISAC, with particular emphasis on three design approaches: sensing-assisted CKM construction and refinement, CKM-assisted sensing, and joint CKM- and sensing-enhanced communications. We also discuss future research directions for achieving environment-aware wireless networks.

0.26

A Material-Aware Channel Model for Efficient CKM Generation via Environment Reconstruction

eess.SP ·2026-09-08 ·Xuancheng Zhu, Zhenjun Dong, Yong Zeng, Cheng-Xiang Wang
similarity 0.59
Abstract

Channel knowledge map (CKM) is a promising technology for environment-aware wireless communication, sensing, and localization in 6G networks. Accurate CKM generation requires precise reconstruction of the environment, including 3D geometries and scatterer materials, typically from multi-modal sensory observations such as LiDAR point clouds and sparse channel measurements. While the former is relatively easy to acquire, materials remain difficult to obtain directly from sparse channel measurements due to the lack of an explicit channel model linking them. To fill this gap, this paper proposes a material-aware channel model that explicitly characterizes the influence of scatterer materials on the wireless channel. Based on this model, an iterative gradient descent based material reconstruction algorithm is proposed. Full wave simulation results validate the developed model and the proposed algorithm, demonstrating their potentials for efficient CKM generation via environment reconstruction.

0.18

SiGMA: Site-Grounded Multi-Modal ISAC Data Generation for Environment-Aware Communications

eess.SP cs.DB ·2026-11-16 ·Yijie Bian, Wei Guo, Kai Zhang, Xiao Ye, Shurui Xu +3 more
code/data similarity 0.57
Abstract

Learning-based multi-modal integrated sensing and communication (ISAC) requires data that associate environmental observations with the wireless response generated by the same site and dynamic scene state. Acquiring such data at new sites requires sensor calibration, coordinated measurements, and cross-modal association. We present site-grounded multi-modal ISAC data generation (SiGMA), which combines OpenStreetMap (OSM), Overture Maps, and procedural three-dimensional (3D) reconstruction to create georeferenced urban scenes. A shared scene geometry and time-indexed actor state link synchronized red-green-blue (RGB) images, light detection and ranging (LiDAR) point clouds, positioning records, frequency-modulated continuous-wave (FMCW) Radar in-phase and quadrature (IQ) samples, and ray-traced wireless channels. For configurable reflected propagation under direct-link blockage, SiGMA generates optional reconfigurable intelligent surface (RIS) component channels for phase-configuration evaluation across shared scene states. The evaluation results demonstrate that the framework improves zero-shot beam prediction performance on real DeepSense 6G measurements compared with an OSM-derived synthetic-data baseline. A camera-assisted RIS case study further demonstrates the generation of sensing-associated supervision for configurable reflected propagation. The dataset generation pipeline is available at https://github.com/Trans-cending/SiGMA-main.

0.39

Electromagnetic Twin: Completing the Wireless World from Sparse Channel Evidence

eess.SP ·2026-08-21 ·Tuo Wu, Jie Tang, Kangda Zhi, Junteng Yao, Maged Elkashlan +3 more
similarity 0.57
Abstract

Acquiring dense channel information over many locations and beams incurs considerable pilot and processing overhead. Radio maps and channel knowledge maps (CKMs) reduce this overhead by reusing site-specific channel information, but their contents must be refreshed when new measurements or environmental observations become available. This paper introduces an \emph{electromagnetic twin} as an updatable digital representation that uses sparse channel evidence to reconstruct the wireless state requested by communication queries. Rather than replacing radio maps or CKMs, the twin uses a CKM as channel memory, combines it with registered scene information, and regenerates its outputs after each evidence update. We instantiate this idea by completing a two-dimensional channel-gain field from sparse samples and an incomplete floor plan. A learned RF completion backbone recovers the main propagation structure, and a lightweight residual adapter tests whether frozen CLIP features provide useful side information. With $4\%$ measured locations and $55\%$ missing semantic objects, the RF backbone attains $4.44$ dB RMSE, compared with $8.29$ dB for CKM interpolation and $8.38$ dB for an incomplete physics prior. Residual adaptation reduces RMSE by a paired mean of $0.135$ dB (95\% confidence interval: $0.100$--$0.169$ dB), but a same-capacity random-feature control is statistically indistinguishable from the CLIP-conditioned adapter. The results therefore support the measurement--update--query loop and lightweight residual correction, while avoiding an unsupported attribution of the correction to visual semantics.

0.45

Physics-Informed WiFi Sensing for Robust 3D Human Pose Estimation in Mobile and Cross-Environment Settings

eess.SP ·2026-08-25 ·Kaixuan Huang, Yuanbo Chen, Guangjin Pan, Shiyi Mu, Tao Yu +2 more
similarity 0.56
Abstract · 12pages 6figures

Device-free human pose estimation using commodity WiFi signals has emerged as a promising paradigm for pervasive sensing in mobile computing systems. However, existing approaches often suffer from severe performance degradation when deployed across heterogeneous environments due to complex multipath propagation and domain shifts in wireless signals. In this paper, we present a physics-informed WiFi sensing framework for robust 3D human pose estimation under mobile and cross-environment settings. Our approach explicitly models wireless signal propagation characteristics and incorporates multipath-aware attention to capture environment-dependent signal variations. To further improve generalization, we introduce a disentangled representation learning scheme that separates pose-related features from environment-specific factors, enabling effective cross-domain adaptation without requiring extensive retraining. We implement our system using commodity WiFi devices and evaluate it on multiple public benchmarks, including Person-in-WiFi-3D and MM-Fi, as well as real-world deployments across diverse indoor environments. Experimental results demonstrate that our framework significantly improves robustness and generalization performance compared to state-of-the-art methods, particularly under cross-environment scenarios. These results highlight the potential of physics-informed wireless sensing for enabling reliable, scalable, and infrastructure-free human-centric applications in mobile computing systems.

0.12

Measurement-Adaptive Light-Cone Transform for Robust Non-Line-of-Sight Imaging

eess.IV cs.CV ·2026-10-16 ·Lianfang Wang, Xueying Liu, Junnan Dong, Qinghua Zhang, Jun Liu +2 more
published similarity 0.55
Abstract

Practical non-line-of-sight (NLOS) measurements are affected by noise and deviations from idealized light transport, which can degrade reconstruction with conventional light-cone transform (LCT) methods. We propose the Measurement-Adaptive Light-Cone Transform (MALCT), a physics-guided framework that retains the efficient Fourier-domain inversion structure of LCT. {Within a unified variational formulation, MALCT combines the analytical response with a shared learned response through measurement-conditioned weighting and predicts a measurement-conditioned, frequency-dependent regularization map.} An attention-enhanced spatio-temporal feature extraction module captures long-range temporal dependencies and local spatial correlations to support reconstruction under low-signal conditions. Experiments on synthetic data and real measurements from different NLOS systems demonstrate improved reconstruction quality. Controlled evaluations further assess robustness to temporal-response broadening and reduced acquisition time. MALCT also preserves scene structures with spatial sampling grids down to $32\times32$, supporting its potential for NLOS imaging under limited acquisition budgets.

0.35