CS Paper Radar

Predictive Beamforming via Echo-Beam Modality Separation with Cross-Attention for ISAC Systems

Seokjun Park, Jiwhan Jang, Jinseok Choi

eess.SP ·submitted 2026-10-07 04:33 UTC · v1
published

Abstract

In this paper, we propose a deep learning-based predictive beamforming algorithm for integrated sensing and communications in vehicle-to-infrastructure networks, where echo signals are leveraged to enhance communication performance. We formulate a sum spectral efficiency maximization problem under a sensing signal-to-interference-plus-noise ratio (SINR) constraint and construct a penalty-based unconstrained surrogate objective for network training. Based on this formulation, the beamforming matrix is predicted using sensing echoes and historical beamformers from previous time slots. To this end, we develop a predictive beamforming network that processes sensing echoes and beamforming inputs through separate encoders to preserve their modality-specific characteristics. This separate design enables each modality to learn rich and complementary representations which are more effectively exploited through a cross-attention mechanism compared to naive input fusion. The resulting features are fused via cross-attention, while gated recurrent units capture temporal dependencies across time slots. Simulation results validate the performance of the proposed method.

Closest papers

Nearest neighbours by meaning across the whole index.

Direct Covariance-to-Beamformer Mapping for Low-Latency Hybrid ISAC Beamforming with Subsequent DoA Estimation

cs.LG eess.SY ·2026-11-01 ·Anjali K. S., Manoj Kumar Panda, Chandramani Singh
similarity 0.72
Abstract

Integrated sensing and communication (ISAC) systems typically employ a two-stage beamforming pipeline, in which target directions are first estimated from the received covariance matrix and then used to compute the beamforming weights by solving non-convex optimization problems. Although effective, this approach introduces processing latency that can limit beam adaptation in dynamic environments. This letter proposes a beamforming deep neural network (BFDNN) for direct covariance-to-beamformer learning in ISAC systems. BFDNN is a compact regression network that maps covariance-matrix features directly to the hybrid beamforming weights produced by a hybrid ISAC beamformer, eliminating the direction-of-arrival (DoA) estimation stage during inference. Compared with MUSIC, classification-DNN, and regression-DNN based beamforming pipelines, BFDNN generates comparable beam patterns and higher effective spectral efficiencies while reducing the beamforming prediction latency by 25-30 times on the evaluated platform, demonstrating the potential of direct covariance-to-beamformer learning for low-latency ISAC beam updates. Moreover, the target directions can be inferred indirectly from the generated radiation pattern, yielding DoA accuracy competitive with that of dedicated learning-based methods in the evaluated scenario.

0.16

Robust Resource Allocation for STAR-RIS-Assisted ISAC Networks Using Deep Learning

cs.PF ·2026-10-19 ·Yuwen Cao, Jiajun Wu, Howard H. Yang, Tomoaki Ohtsuki, Tony Q. S. Quek
theory similarity 0.69
Abstract

In this paper, we focus on investigating the resource allocation in an integrated sensing and communication (ISAC) system assisted by a simultaneously transmitting and reflecting reconfigurable intelligent surface (STAR-RIS). Due to the coupled non-convex nature of the optimization problem, solving the above joint resource optimization problem while guaranteeing superior sensing performance is challenging, especially in dynamic environment. To conquer such challenge, prior iterative optimization algorithms have been presented. However, practical implementations of these prior algorithms face notable computational challenges, e.g., the reliance on accurate global channel information and the assumption of ideal phase shifts. In this context, we propose a model-driven deep learning framework for jointly optimizing the transmit beamforming and waveform in ISAC networks with low-overhead. In addition, we adopt the Cramer-Rao lower bound (CRLB) as the sensing performance metric, and the multi-user interference (MUI) energy as the communication performance metric for performance evaluation. Besides, our framework jointly optimizes the transmit beamforming, waveform and the STAR-RIS reflection/transmission coefficient matrices by minimizing the weighted sum of MUI energy and CRLB. Our study also considers the impact of imperfect channel information, making the research scenario more aligned with real-world applications. Simulation results validate that the proposed framework not only achieves higher sum-rate performance comparable to traditional iterative algorithms but also significantly reduces computational complexity in terms of algorithm execution time, as well as exhibits inherent robustness.

0.11

Robust Beam Prediction for V2X Networks with Multi-Modal Sensing

cs.LG ·2026-09-09 ·Chen Shang, Dinh Thai Hoang, Diep N. Nguyen, Jiadong Yu
published similarity 0.65
Abstract · 6 pages, 3 figures

Integrated sensing and communication (ISAC) provides a promising foundation for beam prediction in future vehicle-to-everything (V2X) networks. However, existing sensing-assisted beamforming methods still rely heavily on radio-frequency sensing, which may become unreliable in complex vehicular environments. Meanwhile, the growing availability of heterogeneous sensors, such as cameras and LiDAR, offers new opportunities to improve beam prediction through richer environmental perception. Motivated by this, this paper proposes a multi-modal beam prediction framework for V2X networks. Specifically, we develop BeamTransFuser, a hierarchical Transformer-based architecture that progressively fuses camera, LiDAR, radar, and GPS observations for robust beam prediction. In addition, to handle possible missing modalities in practical deployment, we introduce a generative module that reconstructs missing modality features from the available observations. Experimental results on a real-world multi-modal V2X dataset show that the proposed framework consistently outperforms representative baselines, while the generative module further improves robustness under incomplete sensing conditions.

0.32

Comparative Evaluation of Deep-Learning based DoA Estimators for Real-Time ISAC Beamforming

cs.LG ·2026-10-14 ·Anjali K. S., Manoj Kumar Panda, Chandramani Singh
similarity 0.65
Abstract

Integrated Sensing and Communication (ISAC) systems rely on accurate Direction-of-Arrival (DoA) estimation for effective beamforming and spectral-efficiency enhancement. While numerous classical and deep-learning-based DoA estimation methods have been reported in the literature, their suitability for real-time ISAC operation remains insufficiently understood because practical deployment requires consideration of both estimation accuracy and computational latency. This paper presents a unified evaluation framework for systematically comparing classical and learning-based DoA estimators within an ISAC beamforming pipeline. Representative classical and learning-based DoA estimators, together with an ensemble architecture, are integrated with Singular Value Decomposition (SVD)-based beamforming and evaluated under identical operating conditions. The impact of DoA estimation accuracy, inference latency, beamforming-weight generation time, and resulting effective spectral efficiency is investigated. Results show that different estimators occupy distinct operating points in the accuracy-latency-performance design space. Although learning-based approaches substantially reduce estimation time compared with MUSIC, the overall processing latency increasingly becomes dominated by the computational cost of SVD-based beamforming. Among the evaluated methods, the Regression Convolutional Neural Network (RCNN) exhibits a favorable balance between estimation accuracy, computational efficiency, and beamforming performance, making it a promising candidate for real-time ISAC deployment. The findings further suggest that future low-latency ISAC systems may benefit from beamforming strategies that reduce or avoid computationally intensive SVD-based beamforming-weight generation.

0.20

Few-Shot Beam Selection for Embodied AI via In-Context Learning and Conformal Prediction

eess.SP ·2026-11-05 ·Weicao Deng, Ruihan Li, Feihang Xu, Min Li
similarity 0.61
Abstract · 6 pages, 4 figures. Accepted by the 2026 IEEE International Symposium on Information Theory Workshop (ISIT 2026 Workshop) on Reliable Machin…

Millimeter-wave (mmWave) communications are essential for supporting massive multimodal sensory data exchange in Embodied Artificial Intelligence (EAI) systems. However, fast and reliable beam selection for mmWave communication remains a critical bottleneck due to the prohibitive overhead associated with conventional beam training. Although location-aided beam prediction offers a promising alternative, its performance is often limited by localization errors and the need for extensive training data. To address these challenges, we propose a novel framework termed In-Context Learning driven Cross-Validation Conformal Prediction (ICL+CVCP) for beam prediction. The proposed framework utilizes the training-free adaptation capability of ICL, enabling effective operation with only a small number of demonstration samples and eliminating the need for large-scale data collection and model fine-tuning. In addition, the CVCP module constructs compact, distribution-free candidate beam sets that provably contain the optimal beam with a user-defined probability. Experimental results confirm that ICL+CVCP consistently satisfies the desired empirical coverage while significantly reducing pilot overhead and computational latency, thereby meeting the stringent reliability and efficiency requirements of EAI applications.

0.28

Radio Echo Representation Learning for Semantic Sensing in Multi-Modal ISAC Systems

eess.SP ·2026-10-11 ·Guanping Shang, Anzheng Tang, Kwok Kit Chan, Shenghui Song
similarity 0.61
Abstract · This paper has been submitted to the IEEE trans journals

Semantic sensing extends integrated sensing and communication (ISAC) beyond physical parameter estimation by exploiting task-oriented representations from radio echoes. Nevertheless, existing multimodal sensing remains largely limited to vision-assisted radio physical parameter estimation or typically processes the visual and radio modalities separately, leaving radio representations underutilized for visual task enhancement. Feature-level fusion enables closer cross-modal interaction, while echo interference complicates the learning of user-specific radio representations and its semantic consistency with the visual entities. To address these challenges, this paper proposes a multimodal semantic sensing framework for orthogonal frequency-division multiplexing (OFDM)-ISAC systems. We propose a radio encoder to learn radio representations oriented toward visual localization from superimposed pilot echoes, together with Variational Radio-Visual Grounding (VRVG) to promote semantic consistency between the learned radio representations and visual spatial features. The radio representations are then formulated as semantic probes to highlight the corresponding entities in the visual scene through cross-modal fusion. Experiments using real-world RCooper dataset show that the resulting synergistic radio-visual embeddings enable accurate radio-visual user matching and significantly improve both user-specific visual localization and physical parameter estimation performance. The results substantiate the benefits of radio representation learning and semantically consistent fusion for multimodal semantic sensing.

0.30

Link Both Views: Cross-Image Routing Attention for Dual-Endpoint V2I Beam-Pair Power Prediction

eess.SP cs.LG ·2026-10-13 ·Shiliang Lu, Lu Bai, Ziwei Huang, Xiang Cheng
similarity 0.59
Abstract

For sensing-aided communications, single-endpoint sensing offers only a partial view of propagation, motivating link-centric sensing across both endpoints. We propose cross-image routing attention (CIRA) for sparse association and mutual enhancement of propagation-relevant features from dual-endpoint RGB images. Building on CIRA, CiFormer predicts millimeter-wave (mmWave) beam-pair power matrices for vehicle-to-infrastructure (V2I) communications. Experiments show that CiFormer reduces mean absolute error and improves top-1 beam-pair selection accuracy by 3.15 percentage points over the strongest baseline. Visualizations and ablation studies support dual-endpoint sensing and selective cross-view interaction in capturing propagation-relevant dependencies, suggesting broader applications in wireless tasks beyond beam prediction.

0.20

Radar-Only Long-Term Beam Prediction in mmWave Vehicular Networks: An Empirical Study of Doppler Input and Cross-Modal Distillation

eess.SP ·2026-10-14 ·Jeongwan Kang, Miguel Bordallo López, Nhan Thanh Nguyen, Junhyeong Kim, Sunwoo Kim
similarity 0.59
Abstract · 13 pages, 6 figures, 3 tables; supplementary material (11 pages) as an ancillary file. This work has been submitted to the IEEE for possible…

Sensing-aided beam prediction could cut the beam-training overhead of millimeter-wave vehicular links, and two design choices could strengthen a radar-only predictor: a Doppler-resolved input and knowledge distilled from a camera during training. We measure what each is worth in received power rather than in top-1 accuracy, whose differences of a few points amount to only tenths of a decibel. We use real DeepSense 6G measurements, mainly at one roadside site, and one convolutional-recurrent predictor family evaluated open-loop up to 0.67 s ahead. Adding a range-Doppler map to the range-angle map lowers the mean beam-selection loss by 0.28 dB (95% bootstrap interval [0.14, 0.41]). The gain comes from the Doppler content, not from the extra input, and replicates, as pre-specified, at a second base station, although holding the last measured beam beats both predictors there. Cross-modal knowledge distillation from a motion-mask vision-radar teacher, by contrast, changes received power by -0.01 dB ([-0.07, +0.06]). The tested teacher, student, fusion and distillation variants show the same null in the metric each was scored on. Where distillation helps, in an exploratory run with few training passes, a radar-only teacher trained on all training passes reproduces most of the gain. For this predictor family and the full training set, the measurable gain at the link comes from Doppler-resolved radar content; no tested route for transferring training-only visual information adds a resolved gain.

0.56