CS Paper Radar

LLM-based Multi-Criteria Deep Reinforcement Learning for Dynamic Efficient 5G UPF Selection

Abdelrahman Soliman, Ahmed Refaey Hussein, Lei Lei

cs.LG cs.AIcs.CLcs.NI ·submitted 2026-11-08 13:30 UTC · v1
published

Abstract

User plane function (UPF) selection in 5G networks is an active area of research, with various limited approaches proposed to enhance network efficiency and manage resources effectively. In this paper, we address this gap in the related literature by proposing a Deep Reinforcement Learning (DRL) UPF selection method that utilizes a dynamic, multi-objective reward function and validating it on a real-world 5G emulation platform. The proposed method is based on the large language models' (LLMs) translation of network operator intents and the network data analytics function (NWDAF) abnormality alerts. The experimental results validated in the real-world experimental setup demonstrate that our dynamic DRL solution achieves an effective balance between latency, power consumption, and traffic distribution, outperforming the heuristic baselines and fixed DRL by 16.89%. These findings highlight the real-world applicability of our approach for dynamic, intent-driven UPF selection.

Comment
Accepted at IEEE Globecom 2026 CSM

Closest papers

Nearest neighbours by meaning across the whole index.

Deep Reinforcement Learning Orchestration of Game-Theoretic User Association and Resource Allocation in HetNets

eess.SP ·2026-08-17 ·Sotiris Kopsinos, Alexandros I. Papadopoulos, Antonios Lalas, Konstantinos Votis, Christos Liaskos
similarity 0.69
Abstract

Managing dynamic User Association and Resource Allocation (UARA) in modern Heterogeneous Cellular Networks (HetNets) remains a critical open challenge. Existing mathematical optimization and Reinforcement Learning approaches face limitations in handling low-latency decision-making under dynamic traffic conditions. This paper introduces a novel orchestration scheme for game-theoretic UARA in HetNets. The proposed bilevel framework distributes UARA decisions to User Equipment through a multi-objective non-cooperative game. Overlaying the distributed game, a centralized Deep Reinforcement Learning controller orchestrates network performance by dynamically configuring the game's utility parameters, enabling transitions between power awareness, coverage enhancement, and balanced operation. Evaluated on urban HetNet topologies with 3GPP TR 38.901-compliant channel modeling, the proposed framework closely approximates the optimal policy for the considered operational objectives, while delivering higher network throughput than conventional association methods. Furthermore, it incurs low computational overhead and maintains stable performance across the evaluated traffic densities without retraining.

0.12

DRL-driven RAN Slicing Management: A V2X-oriented Approach In Multi-service Scenarios

cs.NI ·2026-10-01 ·Daniel E. Garcia-Fernandez, Pablo Vera-Soto, Sergio Fortes, M. Martinez, I. de-la-Bandera +4 more
similarity 0.67
Abstract

The integration of Vehicle-to-Everything (V2X) communications is driving a profound transformation in vehicular connectivity, expected to significantly enhance traffic efficiency and safety. However, the stringent requirements of V2X services, particularly ultra-low latency and high reliability, present significant technical challenges. 5G's Network Slicing emerges as a key enabler by providing tailored virtual networks that ensure isolation and adaptability for heterogeneous services. This work proposes an intelligent Radio Access Network (RAN) slicing management framework specifically designed for scenarios where safety-critical V2X and high-capacity eMBB slices coexist. In such complex environments, harmonizing conflicting traffic requirements demands continuous, data-driven optimization. To achieve this, the proposed framework leverages an advanced Deep Reinforcement Learning (DRL) approach which dynamically optimizes resource allocation in real time. The framework is empirically validated on a real 5G Standalone (SA) network, where experimental results demonstrate that the DRL-driven approach successfully balances both objectives, outperforming traditional static and proportional allocation strategies by minimizing SLA violations while ensuring high resource utilization for eMBB slices.

0.03

Deep Reinforcement Learning for 6G AI-RAN: A Comprehensive Survey

cs.NI eess.SP ·2026-08-14 ·Jie Lu, Peihao Yan, Qijun Wang, Ruxin Lin, Huacheng Zeng
survey similarity 0.66
Abstract

The evolution toward sixth-generation (6G) networks is transforming the radio access network (RAN) into a programmable and intelligent control platform that must continuously adapt to heterogeneous services, dynamic environments, and competing performance objectives. Open Radio Access Network (O-RAN) provides the open interfaces, disaggregated architecture, and multi-timescale control loops needed to support this transformation, while deep reinforcement learning (DRL) offers a natural framework for optimizing sequential decisions under uncertainty. However, existing surveys either address artificial intelligence (AI) and machine learning (ML) in O-RAN broadly or focus on isolated DRL use cases, leaving a gap in the systematic connection between DRL methodology, O-RAN architecture, and operational deployment. To the best of our knowledge, this article presents the first dedicated and comprehensive survey of DRL for Open AI-RAN. We review the foundations of model-free, model-based, offline, safe, multi-agent, federated, and transfer learning, and provide an O-RAN-aware framework for formulating RAN control problems through states, observations, actions, rewards, constraints, and temporal structure. We classify DRL applications across radio resource management, mobility management, interference control, traffic steering, energy efficiency, network slicing, integrated sensing and communication, security, and massive MIMO. We further examine multi-agent and federated coordination, foundation models and agentic AI, trustworthy DRL, sim-to-real transfer, continual adaptation, resource-efficient inference, and reinforcement learning operations. Finally, we review experimental platforms, benchmarks, standards, and industry activities, and identify research directions toward sample-efficient, safe, scalable, interoperable, and deployable DRL control for 6G Open AI-RAN.

0.04

Energy Saving in 5G and Beyond Networks: A Quantum Reinforcement Learning Approach

cs.NI cs.LG ·2026-10-01 ·Muhammad Usman, Nguyen Van Huynh, Marianna Lezzi, Mariangela Lazoi
similarity 0.65
Abstract

Energy saving has become a critical challenge in 5G and beyond networks. The rapid growth of connected devices has increased the overall network energy demand, driving operational expenditure to unsustainable heights. The Base Station (BS) accounts for the largest share of energy usage, typically consuming around 60-70\% of the Radio Access Network (RAN)'s total energy. Therefore, to address this issue, this article optimizes the BS's energy usage while accounting for the dynamic behavior of User Equipment (UE). Deep Reinforcement Learning (DRL) is a natural candidate for determining effective energy saving policies, such as automatically switching BSs on or off when user density is low or adjusting transmission power to balance energy efficiency and Quality of Service (QoS). However, its heavy training burden and the exponential growth of state and action spaces in dense 5G environments make exploration increasingly difficult. To overcome these limitations, we introduce a novel Quantum Reinforcement Learning (QRL) algorithm that leverages quantum principles, including superposition and entanglement, through parameterized quantum circuits, enabling significantly faster convergence than DRL, which relies on conventional deep neural networks. Extensive simulations demonstrate that the proposed QRL can substantially reduce energy consumption while maintaining QoS, even when UEs are highly dynamic and frequently switch their association with BS antennas. Additionally, QRL consistently outperforms DRL and Q-Learning in both convergence speed and learning complexity.

0.08

WiSDoM: Wireless Sparse Decision Transformer with Mixture-of-Experts for Multi-Task Mobile Network Optimization

cs.NI cs.AIcs.LG ·2026-08-31 ·Fatih Temiz, Shavbo Salehi, Melike Erol-Kantarci
similarity 0.65
Abstract · 13 pages, 11 figures, submitted to IEEE for possible publication

Emerging 6G wireless networks are expected to operate across diverse deployment scenarios, where variations in network topology, user mobility, traffic demand, and radio conditions challenge the scalability of conventional radio resource management (RRM). While offline reinforcement learning (RL) methods have demonstrated strong decision-making capabilities, learning a single policy that performs consistently across heterogeneous wireless environments remains difficult due to conflicting optimization objectives and limited model specialization. These challenges become particularly pronounced in coordinated multipoint (CoMP) transmission, where selecting the optimal serving-cell combination requires sequential decision-making under evolving network conditions. This paper presents the Wireless Sparse Decision Transformer with Mixture of Experts (WiSDoM), a sparse multi-task offline RL framework for adaptive multi-cell selection. WiSDoM combines Decision Transformers (DTs) with a Mixture-of-Experts (MoE) architecture that dynamically activates specialized experts according to task characteristics. This MoE mechanism improves model capacity without proportionally increasing inference cost, mitigates negative transfer, and enables expert specialization across tasks. WiSDoM is trained jointly on diverse network configurations spanning multiple base station and user equipment densities, mobility levels, and scheduler policies. Experimental results show that WiSDoM consistently outperforms heuristic methods, single-task models, and conventional multi-task DTs, improving quality of experience (QoE) by up to 55% while activating approximately one-third of the parameters of its dense counterpart during inference. Furthermore, WiSDoM exhibits strong task generalization and efficiently adapts to unseen wireless scenarios through few-shot prompting without retraining or fine-tuning.

0.34

NetRL: Powering Network Decisions via NetGRPO-Driven Reinforcement Learning

cs.NI cs.LG ·2026-10-12 ·Lingfeng Peng, Xiaohui Xie, Yang Bao, Yong Cui
code/data similarity 0.64
Abstract · 10 pages

Network decision-making tasks, including adaptive bitrate streaming, load balancing, and congestion control, require policies to process heterogeneous states and actions while balancing near- and long-term consequences. Existing solutions generalize poorly across network tasks, require substantial task-specific redesign, and lack reliable evaluation of long- horizon action consequences. Methods that learn through online interaction additionally face unstable optimization. We present NetRL, a two-stage LLM-based reinforcement learning framework that combines supervised fine-tuning on demonstration trajectories with online policy optimization. A universal mixed-embedding interface combines textual task descriptions with precise numerical states, while a network action head produces valid decisions. At its core, NetGRPO evaluates candidate actions over multiple future trajectories, directs deeper search toward uncertain alternatives through confidence-aware budgeting, and performs group-relative, clipped, and KL-regularized updates. NetRL achieves 23.6% higher QoE than NetLLM, 12.2% lower job completion time than Genet, and 24.1% higher network utility than Orca, yielding state-of-the-art performance among the evaluated methods. Confidence-aware budgeting reduces end-to-end training time by 27.1% without reducing validation QoE. NetRL also converges stably and generalizes to unseen network traces without retraining. Code and model weights are available at the repository linked in this paper.

0.42

Deep reinforcement learning-based optimization of multi-RAT handovers

cs.NI ·2026-11-05 ·Pavan Kumar Mangipudi, Satya Krishna Pothapragada, Alina Zare, Janise McNair
similarity 0.63
Abstract · 16 pages, 9 figures, 3 tables

A major evolutionary goal and challenge for next-generation wireless systems is seamless multiple radio access technology (multi-RAT) mobility. In a multi-RAT environment, handover decisions must account for both user quality of service and network priorities, including signal quality, resource availability, energy consumption, latency, and security. Although WiFi/cellular mobility is well explored in research and standardization, many conventional handover mechanisms continue to rely primarily on received signal strength. Such decisions may overlook link delay, congestion, and the distinct operating characteristics of heterogeneous RATs. In this work, we present the Reinforced Flexible Multi-RAT Selection framework (ReFlexMRS), a configurable Deep Reinforcement Learning (DRL)-based approach for multi-criteria handover target selection. ReFlexMRS is evaluated using an SDN-enabled multi-RAT testbed. The decision problem is formulated as a Markov Decision Process with a finite action space and is addressed using a Deep Q-Network (DQN) with convolutional layers that extract relationships among network measurement features. The agent is trained offline from collected network observations using reward guided Q-value updates, then used to select handover targets during inference. A 5G-WiFi case study demonstrates that ReFlexMRS can trade a small amount of throughput for substantial improvements in link delay, handover stability, and WiFi offloading. Compared with RSSI-based selection, the proposed formulations retain approximately 90-95% of the baseline throughput while reducing total handovers by up to 18% and WiFi-to-5G handovers by up to 13%, while maintaining the required RSSI levels.

0.19

SliceWise: Traffic-Aware Multi-Agent RL with Temporal Encoding for Joint Slice Admission and Resource Orchestration in O-RAN

cs.NI ·2026-11-14 ·Xingqi Wu, Yuhui Wang, Junaid Farooq, Hakim Ghazzai, Gianluca Setti
publishedhighly cited similarity 0.62
Abstract

Ultra-reliable low-latency communication (URLLC) services in mission-critical applications such as industrial automation, remote healthcare, and autonomous systems demand stringent quality-of-service (QoS) guarantees under dynamic, resource-constrained wireless conditions. Open radio access networks (O-RAN) offer a software-defined and programmable foundation for embedding AI-driven control. This letter proposes SliceWise, a traffic-aware multi-agent deep reinforcement learning (MARL) framework for joint admission control, user-slice association, and resource orchestration in O-RAN. The control problem is modeled as a cooperative multi-agent Markov decision process, with long short-term memory (LSTM) networks capturing temporal traffic dynamics and dueling double deep Q-networks (D3QN) ensuring stable learning. Simulation results show that SliceWise substantially reduces delay violations and packet drops while improving slice-level reliability and resource efficiency, offering a scalable orchestration solution for AI-native O-RAN deployments.

0.57