CS Paper Radar

Vision-Wireless Fusion for Multi-User Localization: A Cross-Modal Transformer Approach

Can Zheng, Jiguang He, Guofa Cai, Henk Wymeersch, Merouane Debbah

cs.CV eess.SP ·submitted 2026-09-20 05:46 UTC · v1

Abstract

Accurate multi-user localization is challenging in complex urban environments, where wireless measurements can become ambiguous under noise, blockage, and multipath, while visual observations provide complementary spatial context. This paper presents a vision-wireless fusion framework for multi-user localization using pilot-indexed channel state information (CSI). Orthogonal pilot indices preserve the identities of the communicating UEs in the CSI token sequence and localization outputs. The model encodes each pilot-indexed CSI observation as a query token and uses cross-attention to retrieve user-specific information from spatial visual memory. Self-attention among CSI tokens further captures inter-user interactions, while the resulting multimodal representations are used for user-wise localization. Experiments on different datasets show consistent improvements over model-based, CSI-only, and multimodal-fusion baselines. Further experiments evaluate the model under different wireless and visual conditions.

Comment
12 pages, 8 figures, 7 tables

Closest papers

Nearest neighbours by meaning across the whole index.

Foundation Models for Wireless Localization: Pretraining, Adaptation, and Utilization

eess.SP ·2026-08-31 ·Guangjin Pan, Jiajia Guo, Zheng Xing, Hui Chen, Chao-Kai Wen +2 more
similarity 0.64
Abstract · 4 pages, 1 table. This work has been submitted to the IEEE for possible publication

Accurate wireless localization is a key enabler for 6G networks, yet remains challenging under diverse and rapidly changing propagation conditions. Model-based methods degrade when multipath channels are non-resolvable and model mismatches occur, while supervised deep learning demands large labeled datasets and generalizes poorly to new deployments. Inspired by foundation models (FMs) in language and vision, this article presents a unified framework for FM-based wireless localization that learns transferable channel representations from large-scale unlabeled channel state information and adapts to new environments with minimal or even no supervision. We review the fundamentals of FMs, compare the FM paradigm with existing localization approaches, and introduce a three-stage framework spanning large-scale pretraining, localization-oriented fine-tuning, and context-augmented inference, together with the location-aware applications it enables. Ray-tracing-based case studies show improved positioning accuracy and cross-environment generalization. Finally, we present an outlook on key research directions toward AI-native networks for wireless localization.

0.21

WiFi based Multi user Activity Recognition via User Conditioned Spatial Attention

eess.SP ·2026-09-18 ·Chenhan Yuan, Ruijing Liu, Cunhua Pan
benchmark similarity 0.62
Abstract

WiFi-based human activity recognition has achieved high accuracy under the single-user scene. Recognizing activities performed by multiple concurrent users remains challenging, because their body-reflected propagation paths superimpose in the channel state information (CSI) measurement. Existing multi-user methods either decompose the signal before classification, suffering from error propagation, or predict activities jointly without reliably associating each prediction with the correct user. Moreover, attention mechanisms that have proven effective for single-user sensing compute a single user-agnostic attention map, blending patterns from different users and providing no mechanism to associate features with specific user slots. This paper proposes a user-conditioned spatial attention (UCSA) module that generates per-user spatial attention maps by conditioning multi-scale spatial attention on learnable user-ID embeddings via feature-wise linear modulation (FiLM). UCSA creates a deep coupling between spatial feature enhancement and user differentiation: the attention module itself becomes user-aware, producing distinct attention maps for each user slot rather than relying on a late-fusion embedding addition. Combined with a shareable multi-semantic spatial attention (SMSA) module for multi-scale spatial enhancement, the proposed framework addresses both feature quality and user differentiation in a unified architecture. Experiments on the WiMANS benchmark across three indoor environments and up to five concurrent users show that the proposed method outperforms nine baselines in all settings, achieving an average accuracy of over 93\% across three environments even with five users. Ablation studies confirm that both SMSA and UCSA contribute substantially to recognition accuracy, with UCSA providing the critical link between shared features and per-user predictions.

0.28

MambaLoc: Lightweight Indoor Localization Using Cross-Modal Knowledge Distillation

cs.AI cs.NI ·2026-11-07 ·Mohab Bahnassy, Omar Saqr, Hamada Rizk, Moustafa Youssef
published similarity 0.62
Abstract

Accurate indoor localization on resource-constrained edge devices remains challenging due to the tension between modeling fidelity and computational efficiency. Wi-Fi Channel State Information (CSI) enables low-cost deployment using existing Wi-Fi infrastructure but is heavily distorted by multipath propagation. Transformer-based solutions mitigate these effects but their quadratic complexity limits real-time edge deployment. Ultra-Wideband (UWB) provides geometrically precise ranging yet requires dedicated infrastructure. We present MambaLoc, a cross-modal distillation framework that improves the accuracy-efficiency balance in CSI-based localization. MambaLoc uses a linear-time Selective State Space Model (Mamba) as a lightweight CSI student, guided during training by a UWB-Transformer teacher. A Feature Alignment Module bridges the representational gap between heterogeneous RF modalities, allowing the CSI model to inherit UWB-derived geometric structure without UWB hardware at inference. A probabilistic Gaussian Mixture regression head models multi-modal spatial ambiguity from multipath effects, avoiding the physically implausible averaging of deterministic regression. Evaluations on a real-world dataset show MambaLoc achieves a median localization error of 0.230 m, reducing error by 22.2% over a standalone Mamba baseline, while requiring 65.1% fewer FLOPs and 58% lower inference latency than Transformer-based approaches.

0.39

Radio Echo Representation Learning for Semantic Sensing in Multi-Modal ISAC Systems

eess.SP ·2026-10-11 ·Guanping Shang, Anzheng Tang, Kwok Kit Chan, Shenghui Song
similarity 0.60
Abstract · This paper has been submitted to the IEEE trans journals

Semantic sensing extends integrated sensing and communication (ISAC) beyond physical parameter estimation by exploiting task-oriented representations from radio echoes. Nevertheless, existing multimodal sensing remains largely limited to vision-assisted radio physical parameter estimation or typically processes the visual and radio modalities separately, leaving radio representations underutilized for visual task enhancement. Feature-level fusion enables closer cross-modal interaction, while echo interference complicates the learning of user-specific radio representations and its semantic consistency with the visual entities. To address these challenges, this paper proposes a multimodal semantic sensing framework for orthogonal frequency-division multiplexing (OFDM)-ISAC systems. We propose a radio encoder to learn radio representations oriented toward visual localization from superimposed pilot echoes, together with Variational Radio-Visual Grounding (VRVG) to promote semantic consistency between the learned radio representations and visual spatial features. The radio representations are then formulated as semantic probes to highlight the corresponding entities in the visual scene through cross-modal fusion. Experiments using real-world RCooper dataset show that the resulting synergistic radio-visual embeddings enable accurate radio-visual user matching and significantly improve both user-specific visual localization and physical parameter estimation performance. The results substantiate the benefits of radio representation learning and semantically consistent fusion for multimodal semantic sensing.

0.30

Masked Latent Prediction of CSI for Indoor Localization in Integrated Sensing and Communication Systems

eess.SP ·2026-09-08 ·Ibtissam Labriji, Mahdi Maleki, Pavan Koteshwar Srinath
similarity 0.59
Abstract · Submitted to IEEE ISAC 2026. 4 figures

Channel State Information (CSI)-based fingerprinting can enable accurate indoor localization but suffers from domain shift, limited labeled data, and degraded performance in multipath-rich environments. To address these challenges, we propose a self-supervised localization framework built on a Joint Embedding Predictive Architecture (JEPA). Each CSI time snapshot is treated as a token, and the encoder is pre-trained by predicting the latent embeddings of masked snapshots from the visible ones, learning robust channel representations without any labels. The encoder is then frozen, and only a lightweight regression head is trained on a small set of labeled positions to estimate the user location. We evaluate the framework on the measured DICHASUS-005x dataset, a single-antenna transmitter received by a 32-antenna array in a multipath-rich indoor environment. With a 50% masking ratio, the proposed method reduces the mean localization error from 0.90 m to 0.42 m (a 53% reduction, or 0.48 m) relative to a supervised Convolutional Neural Network (CNN) baseline trained on raw CSI. Owing to its label-efficient, frozen-encoder design, the framework aligns with integrated sensing and communication (ISAC) objectives, enabling reliable sensing with minimal labeling and compute overhead.

0.13

AWM: An All-in-One Wireless Foundation Model for Multi-User Channel Acquisition and Localization Without Downstream Tuning

eess.SP ·2026-10-16 ·Zirui Chen, Ziqing Xing, Zhaoyang Zhang, Hongning Ruan, Zili Wang +1 more
similarity 0.59
Abstract

Wireless foundation models (WFMs) seek to unify diverse wireless tasks and scenarios, yet existing methods still rely on task- or scenario-specific components or deployment-time fine-tuning. This paper presents an all-in-one wireless model (AWM), a WFM that unifies multi-user channel acquisition and localization, despite their distinct data modalities and functional objectives. We reveal that user location and channel state information (CSI) at different frequencies can be viewed as attribute-specific measurements of the same wireless link. By introducing in-scenario same-attribute reference pairs as prompts, AWM converts heterogeneous attributes into alignable relativity representations, formulating intra-band channel interpolation, inter-band channel prediction, and wireless localization as unified wireless-state transformations. Further, a multi-task MateFormer is developed to perform source-attribute relativity extraction, cross-attribute relativity alignment, and target-attribute synthesis within a unified architecture, avoiding task-specific output heads or downstream fine-tuning. Comprehensive experiments demonstrate the effectiveness of model unification, including substantial gains from cross-task knowledge sharing, seamless generalization to unseen scenarios, and an efficient closed-loop service for mobile users.

0.19

SymNetPro: LOS-Aware Directional Multi-Transmitter Localization from Sparse Radio Observations

cs.IT cs.CV ·2026-09-27 ·Lyuzhou Ye, Heng Fan, Yan Huang
code/data similarity 0.59
Abstract · Code, datasets, and model checkpoints are available at:https://github.com/LyuzhouYe98/SymNet--a-multi-task-network-for-joint-radio-map-recon…

Directional multi-transmitter localization from sparse received-power observations is difficult because the receiver observes only the source-unresolved aggregate field: multiple directional sources superpose, building blockage fragments their visible regions, and stronger sources can mask weaker ones. We present SymNetPro, which retains the dual-task radio-map reconstruction and localization backbone of SymNet and adds two targeted components. First, a sparse line-of-sight (LOS)-aware attention bias injects obstruction-aware spatial relations into selected token interactions. Second, transmitter-drop augmentation recomposes training scenes after removing one sample-supported transmitter, exposing the model to controlled source-cardinality variation. Experiments on directional ray-traced urban environments show substantially lower OSPA than representative localization baselines under extreme sparse sampling, with consistent gains under measurement noise and increasing transmitter count. A transmitter-specific evidence analysis further shows that remaining misses concentrate in regimes where the target contributes little distinguishable power to the aggregate observation.

0.31

LocAttMamba: A Low-Complexity Mamba Framework with Attention-Based Multi-AP Fusion for Indoor Localization

cs.NI ·2026-10-05 ·MohammadMahdi Ghadaksaz, Mohammad Farzanullah, Melike Erol-Kantarci
benchmark similarity 0.59
Abstract

Accurate and low-complexity indoor localization is important for location-based services in fifth generation (5G) and sixth generation (6G) networks, where positioning devices operate under limited computational budgets and non conditions. Indoor localization has been studied widely using traditional signal-level localization approaches. However, these techniques often show degraded performance in on-line-of-sight (NLoS) scenarios. Recently, artificial intelligence (AI)-based techniques, including transformers, have been applied to address these challenges. While transformer-based architectures can capture the dependencies within the measurements collected from distributed access points (APs), their high computational complexity results in a large number of multiply-accumulate operations and long inference time. In contrast, lightweight recurrent and convolutional models trade this cost for degraded accuracy. In this paper, we propose LocAttMamba, a low-complexity localization framework in which the channel impulse response (CIR) and time-based features of each AP are processed by a separate Mamba encoder with near-linear complexity, and the resulting per-AP embeddings are fused through a multi-head attention layer that weights each AP according to its importance at every time step. The framework jointly predicts the user location and its per-axis uncertainty, which is refined through a post-hoc calibration step. We evaluate the proposed framework using two real-world 5G and ultra-wideband (UWB) measurement datasets. Our numerical results reveal that LocAttMamba obtains a mean two-dimensional (2-D) positioning error of 1.004 m and 0.599 m, respectively, on 5G and UWB datasets, outperforming the second-best benchmark by 9.79% and 5.82%, while requiring the fewest multiply-accumulate operations among all evaluated models and being 4.4-16 times faster than the transformer-based benchmarks.

0.19