CS Paper Radar

Multi-Agent Off-Policy Deep Reinforcement Learning for Smart Campus Coverage

Omar Rady, Mohamed Ayman, Ali Arafa, Mohamed Shalma

cs.LG eess.SP ·submitted 2026-08-19 15:41 UTC · v1
benchmark

Abstract

Deep reinforcement learning (DRL) has recently gained a great attention due to its real-time adaptation and effectiveness in complex optimization problems. This paper investigates the optimal deployment of millimeter-wave (mmWave) base stations (BSs) in a realistic, non-convex campus topology. The optimization problem is NP-hard, due to the non-convex, non-smooth nature of the max-min fairness objective. To overcome these constraints, we formulate the BS placement as a Markov Decision Process (MDP) and systematically benchmark four DRL schemes: a discrete single-agent Deep Q-Network (DQN), a spatially partitioned Multi-Agent DQN, a continuous single-agent Deep Deterministic Policy Gradient (DDPG), and a geographically partitioned multi-agent DDPG framework. Numerical evaluations reveal that the multi-agent DDPG approach substantially outperforms single-agent in dense scenarios. Additionally full coverage is achieved, and a fairness Jain's index of 0.94 is obtained. Finally, the multi-agent demonstrates highly efficient computational convergence of dense scenarios with $400$ users.

Closest papers

Nearest neighbours by meaning across the whole index.

Mission-critical spectrum sharing with decentralized Multi-Agent Reinforcement Learning

eess.SY cs.ITmath.OC ·2026-10-06 ·Dimitrios Pylorof, Imtiaz Nasim, Humberto E. Garcia, Vivek Agarwal, Jasni A. Mannil +1 more
published similarity 0.69
Abstract · Accepted for publication in the proceedings of IEEE CCNC 2027

Motivated by emerging mission-critical applications and an increasingly congested spectrum, we develop a decentralized multi-agent reinforcement learning (MARL) model for dynamic spectrum access. The model enables secondary users to learn effective transmission strategies across shared frequency bands while minimizing collisions with high-priority primary users and among themselves. We design the agent-level learners following a Markov potential game approach, connecting independent local updates to system-level improvement. We instantiate this design using lightweight linear actor-critic learners suitable for resource-constrained edge devices, rather than computationally intensive centralized or deep multi-agent architectures. Across spectrum environments with different incumbent activities, the learned policies adapt their transmission policy and waiting behavior to preserve throughput while greatly reducing transmission collisions relative to random and forecast-aware heuristic baselines. The results establish the value of decentralized MARL and shows up to 96.8% reduction in overall collisions.

0.32

Resilient Integrated Access and Backhaul Network Planning via Graph Neural Network Curriculum Learning

cs.NI ·2026-11-10 ·Jie Zhang, Haochen Sun, Mingzhang Zhou, Paul D. Mitchell, Hamed Ahmadi +1 more
published similarity 0.68
Abstract

Integrated Access and Backhaul (IAB) is a key enabler for cost-effective millimeter-wave (mmWave) densification, yet practical deployments are strongly constrained by the spatial availability of fiber-connected IAB donors. Variations in donor topologies reshape both the initial coverage footprint and the feasible multi-hop backhaul expansion patterns, causing planning policies trained on a single topology to generalize poorly. This paper studies cross-topology generalization for resilience-aware IAB deployment under coverage and transport feasibility requirements. We formulate IAB deployment as a constrained Markov decision process that couples access coverage satisfaction with multi-hop transport feasibility, including hop-by-hop rate-budget degradation and a minimum parent-connectivity requirement for fault tolerance. To solve the resulting large discrete planning problem, we propose a graph-based actor-critic approach that uses a Graph Attention Network v2 (GATv2) encoder with Proximal Policy Optimization (PPO), and we develop a multi-stage curriculum learning strategy that progressively increases constraint difficulty and scenario diversity for stable training and transferable policies. In a 60 GHz urban simulation over a 1000 m x 1000 m area with multiple donor layouts, the trained policy satisfies the minimum coverage requirement of 98% with fewer deployed IAB nodes than learning and heuristic baselines, and maintains higher coverage under progressive backhaul failures. The results demonstrate that curriculum-trained graph-based reinforcement learning can improve deployment efficiency and resilience while transferring across heterogeneous IAB donor topologies without retraining.

0.30

Graph Neural Network-Driven Deep Reinforcement Learning for Scalable RIS Allocation

cs.NI ·2026-10-05 ·Martin Mark Zan, Stefan Schwarz
hot direction similarity 0.66
Abstract

Reconfigurable Intelligent Surfaces (RISs) offer a promising paradigm to mitigate blockage and extend millimeter-wave coverage in 6G multi-cell networks. However, dynamically allocating shared RIS infrastructure across competing base stations is an NP-hard problem posing severe scalability bottlenecks. In this paper, we propose a scalable framework combining Graph Neural Networks (GNNs) with Deep Reinforcement Learning (DRL) for dynamic shared RIS orchestration. By formulating allocation as a Markov Decision Process, we introduce a physical topology sparsification strategy that prunes dense channel matrices into a sparse tripartite graph. This pruning reduces edge density by 77% and removes representation noise, thereby improving global coverage probability while reducing computational complexity. Our relational message-passing architecture naturally generalizes to arbitrary network dimensions without model retraining. Furthermore, structural ablation studies reveal that physical path loss localizes surface dependencies, enabling a highly efficient localized graph design with linear computational scaling. Extensive simulations in dense urban environments demonstrate that under strict infrastructure budget constraints, the proposed GNN-DRL framework consistently outperforms greedy heuristic baselines by up to ~12% in coverage while delivering faster inference speed via GPU acceleration.

0.29

Deep Reinforcement Learning Orchestration of Game-Theoretic User Association and Resource Allocation in HetNets

eess.SP ·2026-08-17 ·Sotiris Kopsinos, Alexandros I. Papadopoulos, Antonios Lalas, Konstantinos Votis, Christos Liaskos
similarity 0.66
Abstract

Managing dynamic User Association and Resource Allocation (UARA) in modern Heterogeneous Cellular Networks (HetNets) remains a critical open challenge. Existing mathematical optimization and Reinforcement Learning approaches face limitations in handling low-latency decision-making under dynamic traffic conditions. This paper introduces a novel orchestration scheme for game-theoretic UARA in HetNets. The proposed bilevel framework distributes UARA decisions to User Equipment through a multi-objective non-cooperative game. Overlaying the distributed game, a centralized Deep Reinforcement Learning controller orchestrates network performance by dynamically configuring the game's utility parameters, enabling transitions between power awareness, coverage enhancement, and balanced operation. Evaluated on urban HetNet topologies with 3GPP TR 38.901-compliant channel modeling, the proposed framework closely approximates the optimal policy for the considered operational objectives, while delivering higher network throughput than conventional association methods. Furthermore, it incurs low computational overhead and maintains stable performance across the evaluated traffic densities without retraining.

0.12

Deep Reinforcement Learning for Blind Passive Beamforming in RIS-Assisted MIMO Systems

cs.IT stat.ML ·2026-10-29 ·Oumaïma Bounhar, Thomas Chêne, Ghaya Rekaya-Ben Othman, Oussama Damen
published similarity 0.66
Abstract · Accepted at AIxNET 2026. 6 pages. Copyright may be transferred to IEEE after publication, after which this version may no longer be accessib…

Reconfigurable Intelligent Surfaces (RIS) are a promising solution for improving millimeter-wave (mmWave) communications by controlling signal propagation. Yet, selecting the optimal RIS configuration remains challenging due to the high-dimensional search space and signaling overhead. We propose a Deep Reinforcement Learning (DRL) framework for blind passive beamforming, where a Deep Q-Learning (DQL) agent learns an adaptive policy to select near-optimal RIS configurations while sounding only few codewords, without channel state information. Results show that our approach outperforms Q-Learning (QL) and conventional baselines in convergence, performance, and overhead reduction.

0.34

SliceWise: Traffic-Aware Multi-Agent RL with Temporal Encoding for Joint Slice Admission and Resource Orchestration in O-RAN

cs.NI ·2026-11-14 ·Xingqi Wu, Yuhui Wang, Junaid Farooq, Hakim Ghazzai, Gianluca Setti
publishedhighly cited similarity 0.64
Abstract

Ultra-reliable low-latency communication (URLLC) services in mission-critical applications such as industrial automation, remote healthcare, and autonomous systems demand stringent quality-of-service (QoS) guarantees under dynamic, resource-constrained wireless conditions. Open radio access networks (O-RAN) offer a software-defined and programmable foundation for embedding AI-driven control. This letter proposes SliceWise, a traffic-aware multi-agent deep reinforcement learning (MARL) framework for joint admission control, user-slice association, and resource orchestration in O-RAN. The control problem is modeled as a cooperative multi-agent Markov decision process, with long short-term memory (LSTM) networks capturing temporal traffic dynamics and dueling double deep Q-networks (D3QN) ensuring stable learning. Simulation results show that SliceWise substantially reduces delay violations and packet drops while improving slice-level reliability and resource efficiency, offering a scalable orchestration solution for AI-native O-RAN deployments.

0.51

Cooperative Dueling DQN SAC Learning for Energy Efficiency in Dynamic OWC Networks

eess.SY ·2026-10-07 ·Walter Zibusiso Ncube, Ahmad Adnan Qidan, Taisir El-Gorashi, Jaafar M. H. Elmirghani
hot direction similarity 0.64
Abstract

Growing wireless traffic is increasing pressure on the congested radio-frequency spectrum. Optical wireless communication (OWC) provides a complementary solution by using the abundant unlicensed optical spectrum. However, indoor OWC networks are dynamic: users move, enter or leave the network, and subscribe to different services. Poorly coordinated resource allocation can consequently waste subcarriers, require excessive transmission power and cause frequent AP reassignments. Energy efficiency (EE), defined as the total delivered data rate divided by the total network power consumption, therefore requires the serving AP, number of allocated subcarriers, and transmission power to be jointly adapted while maintaining QoS. Optimising these in a dynamic time series, multi-service OWC environment produces a complex sequential EE optimisation problem. To address this problem, this work proposes Dual-Agent Resource Allocation using Deep Reinforcement Learning (DARA-DRL). DARA-DRL combines a branching duelling deep Q-network for association and subcarrier allocation with a conditional soft actor-critic agent for continuous power control. The agents are coupled through a common reward and a cooperative value update that evaluates each discrete allocation together with its corresponding power decision. Simulation results show that DARA-DRL remains within 5\% of the optimal solution and, compared with state-of-the-art benchmarks, it improves EE by 28.8\% and QoS satisfaction by 10.5\%, while reducing online decision time by 14.5\%. Results demonstrate that agent specialisation simplifies mixed-action learning, and cooperation outperforms independently trained agents.

0.29

A Mobile Agent-Based Hierarchical Reinforcement Learning Framework for Energy-Balanced Data Collection and Wireless Charging in WSN

cs.NI ·2026-09-24 ·Ali Heidaripour, Nastooh Taheri Javan
published similarity 0.62
Abstract

Energy imbalance remains a key challenge in Wireless Sensor Networks (WSNs), as nodes near the base station deplete their energy faster due to heavy forwarding loads. While mobile agents (MAs) have been employed for either data collection or sensor charging, existing approaches lack adaptability and fail to integrate both functions under realistic hardware constraints. This paper introduces a unified mobile agent framework that performs both data collection and wireless charging sequentially under single-antenna limitations. The agent's decision-making is formulated as a two-layer Hierarchical Reinforcement Learning (HRL) problem, where the upper layer optimizes movement planning and the lower layer determines the appropriate service based on real-time network states. This hierarchical structure enables the agent to learn adaptive task scheduling policies without predefined rules. Extensive simulations demonstrate that the proposed method achieves up to 15% longer network lifetime and more balanced energy distribution compared with state-of-the-art mobile agent and deep RL approaches.

0.21