CS Paper Radar

Deep Reinforcement Learning for Blind Passive Beamforming in RIS-Assisted MIMO Systems

Oumaïma Bounhar, Thomas Chêne, Ghaya Rekaya-Ben Othman, Oussama Damen

cs.IT stat.ML ·submitted 2026-10-29 11:09 UTC · v1
published

Abstract

Reconfigurable Intelligent Surfaces (RIS) are a promising solution for improving millimeter-wave (mmWave) communications by controlling signal propagation. Yet, selecting the optimal RIS configuration remains challenging due to the high-dimensional search space and signaling overhead. We propose a Deep Reinforcement Learning (DRL) framework for blind passive beamforming, where a Deep Q-Learning (DQL) agent learns an adaptive policy to select near-optimal RIS configurations while sounding only few codewords, without channel state information. Results show that our approach outperforms Q-Learning (QL) and conventional baselines in convergence, performance, and overhead reduction.

Comment
Accepted at AIxNET 2026. 6 pages. Copyright may be transferred to IEEE after publication, after which this version may no longer be accessible

Closest papers

Nearest neighbours by meaning across the whole index.

Multi-Agent Off-Policy Deep Reinforcement Learning for Smart Campus Coverage

cs.LG eess.SP ·2026-08-19 ·Omar Rady, Mohamed Ayman, Ali Arafa, Mohamed Shalma
benchmark similarity 0.66
Abstract

Deep reinforcement learning (DRL) has recently gained a great attention due to its real-time adaptation and effectiveness in complex optimization problems. This paper investigates the optimal deployment of millimeter-wave (mmWave) base stations (BSs) in a realistic, non-convex campus topology. The optimization problem is NP-hard, due to the non-convex, non-smooth nature of the max-min fairness objective. To overcome these constraints, we formulate the BS placement as a Markov Decision Process (MDP) and systematically benchmark four DRL schemes: a discrete single-agent Deep Q-Network (DQN), a spatially partitioned Multi-Agent DQN, a continuous single-agent Deep Deterministic Policy Gradient (DDPG), and a geographically partitioned multi-agent DDPG framework. Numerical evaluations reveal that the multi-agent DDPG approach substantially outperforms single-agent in dense scenarios. Additionally full coverage is achieved, and a fairness Jain's index of 0.94 is obtained. Finally, the multi-agent demonstrates highly efficient computational convergence of dense scenarios with $400$ users.

0.11

A 3GPP-Compliant Benchmark Dataset for RIS-Aided Beyond 5G Networks

cs.DB cs.AI ·2026-09-30 ·Pujitha Mamillapalli, Pankaj Singh Rathour, Abhinav Kumar
benchmarkcode/data similarity 0.65
Abstract

Reconfigurable Intelligent Surfaces (RIS) are emerging as a key technology for programmable wireless environments in the beyond the fifth generation (B5G) networks. However, data-driven RIS research remains bottleneck by the lack of standardized, high-fidelity and open-source datasets. In this paper, we introduce a large-scale 3GPP TR 38.901-compliant dataset for RIS-aided millimeter wave (mmWave) networks, that considers severe path loss, blockage sensitivity, and spatial channel sparsity make the RIS assistance more impactful. The dataset spans various canonical 3GPP deployment scenarios across 20 controlled variants, capturing diverse user densities, fading conditions, and blockage regimes. Uniquely, every sample includes oracle RIS phase configurations obtained via a globally optimal brute-force codebook search, providing gold-standard supervision labels that are absent from any existing public dataset. Rich multi-task annotations comprising full channel state information (CSI), per-link channel decomposition, optimal phase matrices, and channel quality index (CQI) labels support a broad range of machine learning paradigms and downstream tasks, including phase optimization, channel estimation, and interference management. As the primary benchmark task, we introduce a novel CSI-to-CQI mapping that frames RIS-aided link-quality prediction as a scalable scalar classification problem, thereby avoiding the exponential output complexity of the direct phase vector prediction. We have evaluated this mapping against state-of-the-art architectures under in-distribution, out-of-distribution, and real-world hardware measurement conditions. Our dataset provides a reproducible, extensible, and community-ready foundation to accelerate data-driven research in RIS-aided B5G networks.

0.27

World Model for Energy Efficiency Maximization in Dynamic Metasurface Antennas

cs.IT eess.SY ·2026-10-26 ·Abdolrasoul Sakhaei Gharagezlou, Mehdi Monemi, Matti Latva-aho, Mehdi Rasti
similarity 0.61
Abstract

Dynamic metasurface antennas (DMAs) have emerged as a promising technology for realizing large-scale antenna arrays in next-generation wireless networks owing to their ability to enable dense and energy-efficient antenna architectures. However, jointly optimizing digital beamforming and DMA element weights under time-varying channel conditions remains challenging due to their coupled effects on the received signal and interference powers. We formulate this problem to maximize time-averaged energy efficiency (EE) under time-varying channel conditions, subject to transmit power constraints, Lorentzian phase limitations, and minimum rate requirements. To enable long-term planning and improve sample efficiency compared with model-free reinforcement learning, we develop a world model framework that learns latent system dynamics from interactions with the environment and jointly optimizes digital beamforming and DMA element weights through imagined trajectories. Furthermore, we incorporate feasibility-preserving action mappings and safety cost signals to ensure constraint satisfaction during policy learning. Simulation results demonstrate that the proposed framework improves the time-averaged EE by approximately 20% compared with the deep Q-network baseline, while achieving higher sample efficiency and more reliable constraint satisfaction.

0.32

Adaptive Hybrid Optical RF Transmission Based on Deep Reinforcement Learning for Ultra-Reliable Low Latency Communication

eess.SP ·2026-10-03 ·Stella N. Arinze, Isidore U. Uche, Augustine O. Nwajana
similarity 0.61
Abstract · 7 pages, 9 figures, 2 tables

Ultra Reliable Low-Latency Communication (URLLC) demands submillisecond latency and reliability above 99.999%, which are difficult to guarantee under dynamic wireless and atmospheric conditions. Although free space optical (FSO) links provide high-capacity transmission, their performance degrades under turbulence and fog, while radiofrequency (RF) links offer robustness but limited bandwidth. This paper presents a reinforcement learning-based hybrid optical RF framework that adaptively selects transmission modes to satisfy stringent URLLC requirements. The hybrid decision process is modeled as a Markov Decision Process and optimized using a Deep Q Network (DQN) controller. Extensive simulations under varying attenuation and fading scenarios demonstrate up to 35% latency reduction and 25 to 30 % throughput improvement compared to RF-only and static hybrid schemes, while maintaining reliability above 99.999%. A laboratory-scale testbed operating at 1550 nm and 28 GHz further validates the approach, showing less than 6% deviation between simulated and measured results. The proposed framework demonstrates the practical feasibility of intelligent hybrid transmission for mission-critical next generation wireless networks.

0.37

Blind Beamforming for Intelligent Reflecting Surfaces: A Gradient Bandit Approach

eess.SP ·2026-10-05 ·Wenhai Lai, Mingxiao Li, Kaiming Shen
similarity 0.60
Abstract

The beamforming problem of intelligent reflecting surface (IRS) has been extensively considered from an optimization perspective assuming that channel state information (CSI) is available. However, the reality is that the existing prototypes seldom follow this model-based approach because channel estimation is technically difficult and costly for the network protocols and hardware to date. A recent trend is to perform beamforming blindly without channel knowledge. This work looks at blind beamforming from a reinforcement learning point of view. We first show that the existing blind beamforming method boils down to a special case of the greedy algorithm in the reinforcement learning context. We analyze the resulting cumulative regret, and further propose an upper approximation to facilitate the optimization of the exploration probability. Moreover, we show that a gradient sampling scheme can improve the efficiency of reinforcement learning as compared to the uniform sampling scheme adopted in the existing blind beamforming method. We further verify the convergence of the proposed gradient sampling scheme by showing that it is a stochastic approximation to gradient ascent. Finally, we physically implement the proposed method in a real-world prototype system. Our field test results show that, as compared to the existing blind beamforming method, the proposed gradient sampling boosts the average signal-to-noise ratio (SNR) by more than 5 dB with 1000 samples.

0.33

Adaptive RIS-aided Communications through ML-based Generation of Phase Masks

eess.SY cs.NI ·2026-08-28 ·Corwin Carpenter, Thomas Daltzis, George C. Trichopoulos, Jacek Kibilda, Joao F. Santos
similarity 0.60
Abstract

Reconfigurable Intelligent Surfaces (RISs) are an attractive technology for Millimeter Wave (mmWave) communications due to their ability to passively reflect incident signals. However, current implementations of RIS rely on performing computationally-intensive algorithms offline to generate phase masks, which are stored as a codebook on the embedded microcontroller on the RIS. The codebook size is restricted by the embedded microcontroller's storage capacity, which limits the ability of the RIS to adapt to evolving channel conditions and deployment scenarios. In this demo, we showcase an Machine Learning (ML)-based solution for dynamically generating new phase masks during runtime. Our approach leverages a ML model deployed on the microcontroller for approximating the output of a phase mask generation algorithm, responding to new inputs while remaining smaller than a codebook.

0.35

Digital Twin-Assisted Fast Beam Alignment for mmWave MIMO Networks

cs.IT cs.LGcs.NI ·2026-10-31 ·Muhammad Kamran Saeed, Ashfaq Khokhar
hot direction similarity 0.59
Abstract

Millimeter-wave (mmWave) communication relies on directional beam sweeping for initial access, which incurs substantial latency and signaling overhead as antenna arrays grow large and deployments become denser. Machine-learning-based beam prediction can reduce this overhead, but its effectiveness depends on training data that accurately reflects the deployment environment, a requirement that is difficult to meet in practice. This paper presents a latency-aware beam management architecture that integrates digital-twin-assisted prediction, regionbased localization, weighted ensemble learning, and explainability-driven beam pruning into a unified framework. A digital twin (DT) generates site-specific ray-traced channel knowledge offline, enabling environment-aware beam prediction without exhaustive real-world measurements and supporting both initial access and beam tracking. The proposed ensemble learning scheme combines a base classifier and a specialist classifier through reliability-weighted fusion to improve prediction accuracy and robustness, while explainable artificial intelligence (XAI)-based attribution pruning reduces the beam codebook to a small subset of the most informative beams for beam allocation. To capture the distributional mismatch between the DT and real-world environments: the network was trained on imperfect digital-twin data but evaluated on real-world surrobase data. Results show that the proposed method achieves 96.80% top-1 and 98.36% top-3 beam prediction accuracy, while reducing over-the-air sounding overhead by nearly an order of magnitude relative to exhaustive sweeping, outperforming convolutional and deep neural network baselines from prior work.

0.34

Practical Deep Learning-Based Block Demapping for mmWave MIMO Systems

cs.AR ·2026-11-08 ·Syed Asrar ul haq, Jesus O. Lacruz, Joerg Widmer, Sumit J. Darak
published similarity 0.59
Abstract · Submitted for publication to IEEE Transactions on Wireless Communications

Deep learning is expected to play a central role in 6G systems, by enabling data-driven physical layer designs that can adapt to practical wireless impairments. This is particularly important for millimetre-wave (mmWave) multiple-input multiple-output (MIMO) systems, which are strongly affected by residual channel distortions, interference, and hardware non-idealities that are difficult to capture with analytical models. In this paper, we propose a deep learning-based soft demapper that performs block-wise processing of received symbols instead of conventional symbol-wise detection. The demapper employs a one-dimensional convolutional neural network to learn temporal structure induced by practical propagation and hardware effects, while producing soft bit outputs compatible with soft-decision decoding stages. The proposed approach is trained and evaluated for a 4$\times$4 wideband mmWave MIMO system at 60~GHz, using an over-the-air dataset collected across multiple days and settings. Experimental results demonstrate consistent improvements over a conventional soft-decision demapper, including up to a 2.6$\times$ reduction in bit error rate for 16QAM and up to a 17.6$\times$ reduction for 64QAM under real over-the-air mmWave MIMO conditions. The proposed demapper is further implemented on a Zynq RFSoC platform using a reconfigurable convolution engine and fixed-point arithmetic, achieving an approximately 34.3$\times$ speedup compared to a processor-based implementation, and demonstrating that learning-based demapping can be deployed efficiently on resource-constrained edge hardware.

0.32