CS Paper Radar

Deep Reinforcement Learning Assistance for Resilient Haptic Teleoperation over a 5G Network

Georgios Kokkinis, Aitor Hernandez Herranz, Qi Zhang

eess.SP ·submitted 2026-10-21 10:01 UTC · v1

Abstract

Haptic teleoperation requires low-latency and reliable communication to preserve stable and responsive interaction between a human operator and a remote environment. However, network congestion can strongly degrade teleoperation performance, particularly when haptic traffic competes with other data flows for limited radio resources. This work investigates a deep reinforcement learning (DRL)-assisted teleoperation framework over an Ericsson 5G standalone (SA) testbed under different congestion and quality-of-service (QoS) configurations. In the evaluated task, a human operator uses a haptic device to steer a spaceship through a remote virtual game. The DRL agent runs inside the remote environment, hosted at the network edge, and assists the operator by blending its own actions with the operator's commands received over the 5G network. This placement gives the agent direct access to the current state of the environment, unaffected by network delay. Experimental results show that DRL assistance improves task performance under all network conditions, with the largest benefit under congestion, where it reduces mean task completion time by 20.6%. Under congestion, assisted operation even outperforms unassisted operation with QoS prioritisation, which restores latency close to the uncongested baseline.

Comment
12 pages, 8 figures, 6 tables. Submitted to IEEE Transactions on Mobile Computing

Closest papers

Nearest neighbours by meaning across the whole index.

SliceWise: Traffic-Aware Multi-Agent RL with Temporal Encoding for Joint Slice Admission and Resource Orchestration in O-RAN

cs.NI ·2026-11-14 ·Xingqi Wu, Yuhui Wang, Junaid Farooq, Hakim Ghazzai, Gianluca Setti
publishedhighly cited similarity 0.66
Abstract

Ultra-reliable low-latency communication (URLLC) services in mission-critical applications such as industrial automation, remote healthcare, and autonomous systems demand stringent quality-of-service (QoS) guarantees under dynamic, resource-constrained wireless conditions. Open radio access networks (O-RAN) offer a software-defined and programmable foundation for embedding AI-driven control. This letter proposes SliceWise, a traffic-aware multi-agent deep reinforcement learning (MARL) framework for joint admission control, user-slice association, and resource orchestration in O-RAN. The control problem is modeled as a cooperative multi-agent Markov decision process, with long short-term memory (LSTM) networks capturing temporal traffic dynamics and dueling double deep Q-networks (D3QN) ensuring stable learning. Simulation results show that SliceWise substantially reduces delay violations and packet drops while improving slice-level reliability and resource efficiency, offering a scalable orchestration solution for AI-native O-RAN deployments.

0.57

From Prior-Guided Heuristics to Deployable Agents: Accelerating Demonstration-Driven Reinforcement Learning for Deadline-Constrained Network Control

cs.NI cs.AI ·2026-09-03 ·Vincenzo Norman Vitale, Mohammad Solki, Antonia Maria Tulino, Andreas F. Molisch, Jaime Llorca
similarity 0.63
Abstract

Timely delivery of delay-sensitive information over dynamic, heterogeneous networks is essential for NextG interactive applications, yet providing strict End-to-End (E2E) peak latency guarantees remains an open challenge. Two obstacles limit the adoption of learning-based network control in this setting: traditional volume-based routing metrics, while highly effective for general traffic management, are not designed to capture traffic urgency; and Deep Reinforcement Learning (DRL) controllers trained from scratch suffer from sample inefficiency, long training times, and early-stage exploration volatility. This paper introduces a deployment-focused network control framework that addresses both obstacles. First, we present Effective Congestion (EC), a deadline-aware metric family that quantifies interface congestion by packet urgency and proactively filters non-viable traffic, coupled with a Uniform Path Grouping (UPG) distribution heuristic promoting robust load-balancing; the resulting policies are embedded into Multi-Agent Deep Reinforcement Learning Effective Congestion ($p^*$) (MADRL EC ($p^*$)), a hybrid architecture combining a distributed scheduler with a centralized RL-based router. Second, we introduce a unified training objective that generalizes existing policy-learning paradigms---behavioral cloning, offline Reinforcement Learning (RL), online RL, and offline-to-online schemes---as special cases, combining a live-reward term, a pre-collected-reward term, and a policy-imitation term. From this objective, we derive the Model-Guided Annealed Reinforcement Learning (MGA-RL) protocol, instantiated on a Deep Deterministic Policy Gradient (DDPG) backbone: a deployment-oriented, demonstration-driven training approach that generalizes conventional Offline-to-Online (O2O) schemes, in which trajectories from a lightweight [...]

0.10

AoI-Driven Hierarchical Learning for Cooperative Resource Sharing in Multi-Operator UAV Networks

eess.SY ·2026-09-19 ·Atefeh Hajijamali Arani, Mahyar Shirvanimoghaddam, Abolfazl Mehbodniya, Halim Yanikomeroglu, Fumiyuki Adachi
similarity 0.63
Abstract

Uncrewed aerial vehicle (UAV)-assisted networks provide a versatile paradigm for on-demand connectivity. However, in multi-operator aerial networks (MOANs), the joint optimization of cooperative resource sharing and 3D trajectory control to maintain information freshness is a complex combinatorial problem, which can be shown to be NP-hard. To address this computational complexity, we propose an age of information (AoI)-driven hierarchical deep reinforcement learning (DRL) framework. Specifically, a Dueling Double Deep Q-Network (D3QN) architecture is deployed at both the operator and UAV decision layers to mitigate overestimation bias and enhance stability in high-dimensional state spaces. To improve system resilience, we introduce an AoI- and load-aware outage compensation mechanism that prioritizes users based on instantaneous transmission demands and temporal freshness. Furthermore, a normalized load exchange balance metric is incorporated to regulate cooperative behavior and ensure resource fairness across operators. Simulation results demonstrate that the proposed hierarchical D3QN significantly outperforms conventional DRL, non-cooperative, and cooperative benchmarks, reducing the average AoI by up to 56.1% under severe congestion while ensuring superior inter-operator fairness and outage mitigation.

0.11

Deep Reinforcement Learning for 6G AI-RAN: A Comprehensive Survey

cs.NI eess.SP ·2026-08-14 ·Jie Lu, Peihao Yan, Qijun Wang, Ruxin Lin, Huacheng Zeng
survey similarity 0.63
Abstract

The evolution toward sixth-generation (6G) networks is transforming the radio access network (RAN) into a programmable and intelligent control platform that must continuously adapt to heterogeneous services, dynamic environments, and competing performance objectives. Open Radio Access Network (O-RAN) provides the open interfaces, disaggregated architecture, and multi-timescale control loops needed to support this transformation, while deep reinforcement learning (DRL) offers a natural framework for optimizing sequential decisions under uncertainty. However, existing surveys either address artificial intelligence (AI) and machine learning (ML) in O-RAN broadly or focus on isolated DRL use cases, leaving a gap in the systematic connection between DRL methodology, O-RAN architecture, and operational deployment. To the best of our knowledge, this article presents the first dedicated and comprehensive survey of DRL for Open AI-RAN. We review the foundations of model-free, model-based, offline, safe, multi-agent, federated, and transfer learning, and provide an O-RAN-aware framework for formulating RAN control problems through states, observations, actions, rewards, constraints, and temporal structure. We classify DRL applications across radio resource management, mobility management, interference control, traffic steering, energy efficiency, network slicing, integrated sensing and communication, security, and massive MIMO. We further examine multi-agent and federated coordination, foundation models and agentic AI, trustworthy DRL, sim-to-real transfer, continual adaptation, resource-efficient inference, and reinforcement learning operations. Finally, we review experimental platforms, benchmarks, standards, and industry activities, and identify research directions toward sample-efficient, safe, scalable, interoperable, and deployable DRL control for 6G Open AI-RAN.

0.04

DRL-driven RAN Slicing Management: A V2X-oriented Approach In Multi-service Scenarios

cs.NI ·2026-10-01 ·Daniel E. Garcia-Fernandez, Pablo Vera-Soto, Sergio Fortes, M. Martinez, I. de-la-Bandera +4 more
similarity 0.62
Abstract

The integration of Vehicle-to-Everything (V2X) communications is driving a profound transformation in vehicular connectivity, expected to significantly enhance traffic efficiency and safety. However, the stringent requirements of V2X services, particularly ultra-low latency and high reliability, present significant technical challenges. 5G's Network Slicing emerges as a key enabler by providing tailored virtual networks that ensure isolation and adaptability for heterogeneous services. This work proposes an intelligent Radio Access Network (RAN) slicing management framework specifically designed for scenarios where safety-critical V2X and high-capacity eMBB slices coexist. In such complex environments, harmonizing conflicting traffic requirements demands continuous, data-driven optimization. To achieve this, the proposed framework leverages an advanced Deep Reinforcement Learning (DRL) approach which dynamically optimizes resource allocation in real time. The framework is empirically validated on a real 5G Standalone (SA) network, where experimental results demonstrate that the DRL-driven approach successfully balances both objectives, outperforming traditional static and proportional allocation strategies by minimizing SLA violations while ensuring high resource utilization for eMBB slices.

0.03

Mission-critical spectrum sharing with decentralized Multi-Agent Reinforcement Learning

eess.SY cs.ITmath.OC ·2026-10-06 ·Dimitrios Pylorof, Imtiaz Nasim, Humberto E. Garcia, Vivek Agarwal, Jasni A. Mannil +1 more
published similarity 0.60
Abstract · Accepted for publication in the proceedings of IEEE CCNC 2027

Motivated by emerging mission-critical applications and an increasingly congested spectrum, we develop a decentralized multi-agent reinforcement learning (MARL) model for dynamic spectrum access. The model enables secondary users to learn effective transmission strategies across shared frequency bands while minimizing collisions with high-priority primary users and among themselves. We design the agent-level learners following a Markov potential game approach, connecting independent local updates to system-level improvement. We instantiate this design using lightweight linear actor-critic learners suitable for resource-constrained edge devices, rather than computationally intensive centralized or deep multi-agent architectures. Across spectrum environments with different incumbent activities, the learned policies adapt their transmission policy and waiting behavior to preserve throughput while greatly reducing transmission collisions relative to random and forecast-aware heuristic baselines. The results establish the value of decentralized MARL and shows up to 96.8% reduction in overall collisions.

0.32

LLM-based Multi-Criteria Deep Reinforcement Learning for Dynamic Efficient 5G UPF Selection

cs.LG cs.AIcs.CLcs.NI ·2026-11-08 ·Abdelrahman Soliman, Ahmed Refaey Hussein, Lei Lei
published similarity 0.60
Abstract · Accepted at IEEE Globecom 2026 CSM

User plane function (UPF) selection in 5G networks is an active area of research, with various limited approaches proposed to enhance network efficiency and manage resources effectively. In this paper, we address this gap in the related literature by proposing a Deep Reinforcement Learning (DRL) UPF selection method that utilizes a dynamic, multi-objective reward function and validating it on a real-world 5G emulation platform. The proposed method is based on the large language models' (LLMs) translation of network operator intents and the network data analytics function (NWDAF) abnormality alerts. The experimental results validated in the real-world experimental setup demonstrate that our dynamic DRL solution achieves an effective balance between latency, power consumption, and traffic distribution, outperforming the heuristic baselines and fixed DRL by 16.89%. These findings highlight the real-world applicability of our approach for dynamic, intent-driven UPF selection.

0.32

Deadline-Aware Multi-Agent Reinforcement Learning for TSN-Based Vehicular Edge Networks

cs.NI cs.AI ·2026-10-07 ·Bernardo A. C. Pereira, Marcos Carvalho, Fatih Temiz, Shavbo Salehi, Melike Erol-Kantarci +3 more
similarity 0.59
Abstract

Vehicular edge computing (VEC) enables latency-sensitive applications by bringing computing and networking resources closer to vehicles. However, existing approaches often overlook network contention among co-located services with heterogeneous and dynamic latency requirements. While time-sensitive networking (TSN) provides bounded-latency communication, conventional and reinforcement learning-based schedulers struggle to adapt to highly dynamic vehicular environments and inter-queue dependencies. To address these limitations, we propose a multi-agent reinforcement learning (MARL) approach for queue-level scheduling in TSN-enabled VEC. Each TSN queue is assigned an autonomous agent that jointly learns the queue service order and time-slot duration to minimize deadline misses under speed-dependent latency requirements. We employ multi-agent proximal policy optimization (MAPPO) to enable coordinated yet autonomous scheduling decisions. Evaluation against single-agent, multi-agent, and non-learning-based baselines shows that MAPPO provides robust performance across different traffic profiles. Compared with centralized single-agent methods, it reduces service latency by up to 66.2% and improves reliability by up to 271.8%. Furthermore, unlike urgency-based heuristics, MAPPO ensures balanced scheduling while achieving lower inference times compared to other MARL methods.

0.21