CS Paper Radar

Cooperative Dueling DQN SAC Learning for Energy Efficiency in Dynamic OWC Networks

Walter Zibusiso Ncube, Ahmad Adnan Qidan, Taisir El-Gorashi, Jaafar M. H. Elmirghani

eess.SY ·submitted 2026-10-07 01:01 UTC · v1
hot direction

Abstract

Growing wireless traffic is increasing pressure on the congested radio-frequency spectrum. Optical wireless communication (OWC) provides a complementary solution by using the abundant unlicensed optical spectrum. However, indoor OWC networks are dynamic: users move, enter or leave the network, and subscribe to different services. Poorly coordinated resource allocation can consequently waste subcarriers, require excessive transmission power and cause frequent AP reassignments. Energy efficiency (EE), defined as the total delivered data rate divided by the total network power consumption, therefore requires the serving AP, number of allocated subcarriers, and transmission power to be jointly adapted while maintaining QoS. Optimising these in a dynamic time series, multi-service OWC environment produces a complex sequential EE optimisation problem. To address this problem, this work proposes Dual-Agent Resource Allocation using Deep Reinforcement Learning (DARA-DRL). DARA-DRL combines a branching duelling deep Q-network for association and subcarrier allocation with a conditional soft actor-critic agent for continuous power control. The agents are coupled through a common reward and a cooperative value update that evaluates each discrete allocation together with its corresponding power decision. Simulation results show that DARA-DRL remains within 5\% of the optimal solution and, compared with state-of-the-art benchmarks, it improves EE by 28.8\% and QoS satisfaction by 10.5\%, while reducing online decision time by 14.5\%. Results demonstrate that agent specialisation simplifies mixed-action learning, and cooperation outperforms independently trained agents.

Closest papers

Nearest neighbours by meaning across the whole index.

Energy Saving in 5G and Beyond Networks: A Quantum Reinforcement Learning Approach

cs.NI cs.LG ·2026-10-01 ·Muhammad Usman, Nguyen Van Huynh, Marianna Lezzi, Mariangela Lazoi
similarity 0.71
Abstract

Energy saving has become a critical challenge in 5G and beyond networks. The rapid growth of connected devices has increased the overall network energy demand, driving operational expenditure to unsustainable heights. The Base Station (BS) accounts for the largest share of energy usage, typically consuming around 60-70\% of the Radio Access Network (RAN)'s total energy. Therefore, to address this issue, this article optimizes the BS's energy usage while accounting for the dynamic behavior of User Equipment (UE). Deep Reinforcement Learning (DRL) is a natural candidate for determining effective energy saving policies, such as automatically switching BSs on or off when user density is low or adjusting transmission power to balance energy efficiency and Quality of Service (QoS). However, its heavy training burden and the exponential growth of state and action spaces in dense 5G environments make exploration increasingly difficult. To overcome these limitations, we introduce a novel Quantum Reinforcement Learning (QRL) algorithm that leverages quantum principles, including superposition and entanglement, through parameterized quantum circuits, enabling significantly faster convergence than DRL, which relies on conventional deep neural networks. Extensive simulations demonstrate that the proposed QRL can substantially reduce energy consumption while maintaining QoS, even when UEs are highly dynamic and frequently switch their association with BS antennas. Additionally, QRL consistently outperforms DRL and Q-Learning in both convergence speed and learning complexity.

0.08

A Multiarmed Bandit Approach for LTE-U/Wi-Fi Coexistence in a Multicell Scenario

cs.NI ·2026-10-14 ·Iago Diógenes do Rego, José M. de Castro Neto, Sildolfo F. G. Neto, Pedro M. de Santana, Vicente A. de Sousa +2 more
publishedhighly cited similarity 0.69
Abstract

Recent studies and literature reviews have shown promising results for 3GPP system solutions in unlicensed bands when coexisting with Wi-Fi, either by using the duty cycle (DC) approach or licensed-assisted access (LAA). However, it is widely known that general performance in these coexistence scenarios is dependent on traffic and how the duty cycle is adjusted. Most DC solutions configure their parameters statically, which can result in performance losses when the scenario experiences changes on the offered data. In our previous works, we demonstrated that reinforcement learning (RL) techniques can be used to adjust DC parameters. We showed that a Q-learning (QL) solution that adapts the LTE DC ratio to the transmitted data rate can maximize the Wi-Fi/LTE-Unlicensed (LTE-U) aggregated throughput. In this paper, we extend our previous solution by implementing a simpler and more efficient algorithm based on multiarmed bandit (MAB) theory. We evaluate its performance and compare it with the previous one in different traffic scenarios. The results demonstrate that our new solution offers improved balance in throughput, providing similar results for LTE and Wi-Fi, while still showing a substantial system gain. Moreover, in one of the scenarios, our solution outperforms the previous approach by 6% in system throughput. In terms of user throughput, it achieves more than 100% gain for the users at the 10th percentile of performance, while the old solution only achieves a 10% gain.

0.60

Adaptive Hybrid Optical RF Transmission Based on Deep Reinforcement Learning for Ultra-Reliable Low Latency Communication

eess.SP ·2026-10-03 ·Stella N. Arinze, Isidore U. Uche, Augustine O. Nwajana
similarity 0.69
Abstract · 7 pages, 9 figures, 2 tables

Ultra Reliable Low-Latency Communication (URLLC) demands submillisecond latency and reliability above 99.999%, which are difficult to guarantee under dynamic wireless and atmospheric conditions. Although free space optical (FSO) links provide high-capacity transmission, their performance degrades under turbulence and fog, while radiofrequency (RF) links offer robustness but limited bandwidth. This paper presents a reinforcement learning-based hybrid optical RF framework that adaptively selects transmission modes to satisfy stringent URLLC requirements. The hybrid decision process is modeled as a Markov Decision Process and optimized using a Deep Q Network (DQN) controller. Extensive simulations under varying attenuation and fading scenarios demonstrate up to 35% latency reduction and 25 to 30 % throughput improvement compared to RF-only and static hybrid schemes, while maintaining reliability above 99.999%. A laboratory-scale testbed operating at 1550 nm and 28 GHz further validates the approach, showing less than 6% deviation between simulated and measured results. The proposed framework demonstrates the practical feasibility of intelligent hybrid transmission for mission-critical next generation wireless networks.

0.37

Deep Reinforcement Learning Orchestration of Game-Theoretic User Association and Resource Allocation in HetNets

eess.SP ·2026-08-17 ·Sotiris Kopsinos, Alexandros I. Papadopoulos, Antonios Lalas, Konstantinos Votis, Christos Liaskos
similarity 0.68
Abstract

Managing dynamic User Association and Resource Allocation (UARA) in modern Heterogeneous Cellular Networks (HetNets) remains a critical open challenge. Existing mathematical optimization and Reinforcement Learning approaches face limitations in handling low-latency decision-making under dynamic traffic conditions. This paper introduces a novel orchestration scheme for game-theoretic UARA in HetNets. The proposed bilevel framework distributes UARA decisions to User Equipment through a multi-objective non-cooperative game. Overlaying the distributed game, a centralized Deep Reinforcement Learning controller orchestrates network performance by dynamically configuring the game's utility parameters, enabling transitions between power awareness, coverage enhancement, and balanced operation. Evaluated on urban HetNet topologies with 3GPP TR 38.901-compliant channel modeling, the proposed framework closely approximates the optimal policy for the considered operational objectives, while delivering higher network throughput than conventional association methods. Furthermore, it incurs low computational overhead and maintains stable performance across the evaluated traffic densities without retraining.

0.12

A Mobile Agent-Based Hierarchical Reinforcement Learning Framework for Energy-Balanced Data Collection and Wireless Charging in WSN

cs.NI ·2026-09-24 ·Ali Heidaripour, Nastooh Taheri Javan
published similarity 0.66
Abstract

Energy imbalance remains a key challenge in Wireless Sensor Networks (WSNs), as nodes near the base station deplete their energy faster due to heavy forwarding loads. While mobile agents (MAs) have been employed for either data collection or sensor charging, existing approaches lack adaptability and fail to integrate both functions under realistic hardware constraints. This paper introduces a unified mobile agent framework that performs both data collection and wireless charging sequentially under single-antenna limitations. The agent's decision-making is formulated as a two-layer Hierarchical Reinforcement Learning (HRL) problem, where the upper layer optimizes movement planning and the lower layer determines the appropriate service based on real-time network states. This hierarchical structure enables the agent to learn adaptive task scheduling policies without predefined rules. Extensive simulations demonstrate that the proposed method achieves up to 15% longer network lifetime and more balanced energy distribution compared with state-of-the-art mobile agent and deep RL approaches.

0.21

Network Adaptation in IRS-Aided Hybrid RF/VLC Systems Using Cooperative Multi-Agent DRL

eess.SY ·2026-10-02 ·Ahrar N. Hamad, Ahmad Adnan Qidan, Taisir E. H. El-Gorashi, Jaafar M. H. Elmirghani
similarity 0.66
Abstract · 6 pages, 4 figures

Hybrid radio frequency (RF) and visible light communication (VLC) networks have emerged as a promising solution for high-capacity indoor wireless connectivity in sixth-generation (6G) systems. However, the limited optical coverage and vulnerability of VLC links to line-of-sight (LoS) blockage under user mobility remain fundamental challenges. In this work, a mirror-based intelligent reflecting surface (IRS) is deployed to assist a dynamic indoor hybrid RF/VLC network, where each mobile user is exclusively assigned to either the IRS-enhanced VLC subnetwork or the RF subnetwork through a binary selection decision. A joint optimization problem is then formulated to maximize proportional fairness by jointly optimizing the RF/VLC technology selection, power allocation, and IRS mirror roll and yaw orientation angles. To enable real-time adaptability, the problem is reformulated as a Markov decision process (MDP) and solved using a cooperative multi-agent deep reinforcement learning (DRL) algorithm based on centralized training with decentralized execution. Simulation results demonstrate the superior performance of the optimized hybrid network compared with optimized standalone VLC and RF networks. The results further validate the practicality and effectiveness of the proposed DRL framework compared to widely adopted DRL algorithms as well as conventional model-based optimization approaches.

0.28

Mission-critical spectrum sharing with decentralized Multi-Agent Reinforcement Learning

eess.SY cs.ITmath.OC ·2026-10-06 ·Dimitrios Pylorof, Imtiaz Nasim, Humberto E. Garcia, Vivek Agarwal, Jasni A. Mannil +1 more
published similarity 0.66
Abstract · Accepted for publication in the proceedings of IEEE CCNC 2027

Motivated by emerging mission-critical applications and an increasingly congested spectrum, we develop a decentralized multi-agent reinforcement learning (MARL) model for dynamic spectrum access. The model enables secondary users to learn effective transmission strategies across shared frequency bands while minimizing collisions with high-priority primary users and among themselves. We design the agent-level learners following a Markov potential game approach, connecting independent local updates to system-level improvement. We instantiate this design using lightweight linear actor-critic learners suitable for resource-constrained edge devices, rather than computationally intensive centralized or deep multi-agent architectures. Across spectrum environments with different incumbent activities, the learned policies adapt their transmission policy and waiting behavior to preserve throughput while greatly reducing transmission collisions relative to random and forecast-aware heuristic baselines. The results establish the value of decentralized MARL and shows up to 96.8% reduction in overall collisions.

0.32

Movable-Element Multi-Functional RIS-Assisted Hybrid RSMA-NOMA: A Constrained Hybrid Reinforcement Learning Approach

cs.AI eess.SP ·2026-10-26 ·Li-Hsiang Shen, Chi-Te Kuo
top venue similarity 0.65
Abstract · Accepted by IEEE TCOM

In this work, we propose a movable-element (ME) multi-functional reconfigurable intelligent surface (MF-RIS) architecture, termed MM-RIS, which integrates element mobility with adaptive signal reflection, transmission, amplification, and energy harvesting (EH) to enhance communication flexibility and energy self-sustainability. A hybrid rate-splitting multiple access (RSMA) and non-orthogonal multiple access (NOMA) transmission scheme is further designed to improve spectral efficiency. We formulate an energy efficiency (EE) maximization problem by jointly optimizing transmit beamforming, power and rate allocation, MM-RIS configurations, panel deployment, and ME positions. To address the resulting non-convex mixed continuous-discrete optimization problem, we propose constrained hybrid deep reinforcement learning with federated transformer and parametrized sharing (CRAFTS). CRAFTS integrates multi-agent proximal policy optimization (MAPPO) and multi-agent deep Q-networks (MADQN) through bi-directional parametrized sharing to handle continuous and discrete actions, respectively. Moreover, a transformer-based architecture and federated learning (FL) are incorporated to enhance state representation and distributed knowledge sharing, whilst a Lagrangian-based adaptive penalty mechanism facilitates the constraint satisfaction. Simulation results have demonstrated that the proposed MM-RIS architecture and CRAFTS framework substantially improve EE and constraint satisfaction compared to the benchmarks in the open literature. Moreover, the results validate the benefits of spatially reconfigurable MM-RIS, hybrid RSMA-NOMA transmissions, and intelligent collaborative learning, highlighting the potential of the proposed framework for energy-efficient and adaptive next-generation wireless networks.

0.28