CS Paper Radar

Feedback-Aware Residual Multi-Agent Reinforcement Learning for Timely Communication in Cognitive Radio Networks

Baolin Chong, Xiuqi Chen, Jiandong Guo, Fuhui Zhou, Qihui Wu

cs.NI ·submitted 2026-11-14 07:43 UTC · v1

Abstract

Supporting time-sensitive services in cognitive radio networks requires distributed access strategies that translate shared spectrum opportunities into timely packet delivery. Achieving this objective is challenging because reception conditions are only partially observable to individual transmitters, while dynamic competition and access overhead constrain the service that can be completed within packet deadlines. We formulate a distributed access problem that captures the coupling among interference-dependent reception, queue evolution, and protocol durations. To address this problem, we propose feedback-aware residual multi-agent proximal policy optimization (FAR-MAPPO) under centralized training and decentralized execution. The method constructs channel-specific historical feedback representations that complement instantaneous sensing with evidence from completed access attempts. A residual policy network integrates this experience with current observations and packet urgency to guide local access decisions. Numerical results show improved timely delivery over the baseline methods, particularly under heavy traffic and bursty arrivals with mixed packet deadlines.

Closest papers

Nearest neighbours by meaning across the whole index.

Federated Multiagent Soft Actor Critic for IRS Assisted Cognitive Radio Networks under Partial Observability

cs.NI ·2026-10-16 ·Satvika Pullisani, Usama Mir, Ubaid Abbasi
similarity 0.73
Abstract

Intelligent reflecting surfaces (IRSs) can improve spectrum efficiency and interference control in cognitive radio networks (CRNs) by reconfiguring the propagation environment. This paper presents FMASAC, a federated multi-agent soft actor-critic framework for IRS-assisted CRNs under partial observability, in which distributed secondary users (SUs) jointly select their transmit power and the phase shifts of an assigned subset of IRS elements using only local observations. Primary user (PU) protection is incorporated through an interference-penalized reward, and SUs coordinate by periodically averaging model parameters at a federated server, without exchanging raw observations, channel state information, or replay data. We formulate the problem as a partially observable Markov decision process, specify the local learning and aggregation procedure, and characterize its per-round communication cost. Illustrative performance trends contrast FMASAC with centralized SAC, independent SAC, IRS-fixed multi-agent reinforcement learning, and a heuristic CRN strategy. We also outline the simulation-based evaluation and the open challenges, namely constraint handling, agent heterogeneity, and scalability, that are needed to validate the framework.

0.13

Mission-critical spectrum sharing with decentralized Multi-Agent Reinforcement Learning

eess.SY cs.ITmath.OC ·2026-10-06 ·Dimitrios Pylorof, Imtiaz Nasim, Humberto E. Garcia, Vivek Agarwal, Jasni A. Mannil +1 more
published similarity 0.71
Abstract · Accepted for publication in the proceedings of IEEE CCNC 2027

Motivated by emerging mission-critical applications and an increasingly congested spectrum, we develop a decentralized multi-agent reinforcement learning (MARL) model for dynamic spectrum access. The model enables secondary users to learn effective transmission strategies across shared frequency bands while minimizing collisions with high-priority primary users and among themselves. We design the agent-level learners following a Markov potential game approach, connecting independent local updates to system-level improvement. We instantiate this design using lightweight linear actor-critic learners suitable for resource-constrained edge devices, rather than computationally intensive centralized or deep multi-agent architectures. Across spectrum environments with different incumbent activities, the learned policies adapt their transmission policy and waiting behavior to preserve throughput while greatly reducing transmission collisions relative to random and forecast-aware heuristic baselines. The results establish the value of decentralized MARL and shows up to 96.8% reduction in overall collisions.

0.32

Foundation Model-Aided Multi-Agent Reinforcement Learning for Wireless Random Access Network Optimization

cs.LG cs.AI ·2026-10-06 ·Myeung Suk Oh, Zhiyao Zhang, Alvaro Velasquez, Nathaniel D. Bastian, Jia Liu
published similarity 0.68
Abstract · This paper has been accepted in ACM International Symposium on Theory, Algorithmic Foundations, and Protocol Design for Mobile Networks and …

Random access (RA) is one of the most foundational medium access control (MAC) layer scheduling schemes for handling unpredictable data traffic from multiple terminals. While multi-agent reinforcement learning (MARL) has been explored to optimize RA-based wireless networks, its reliance on experience-driven, distributed policy learning incurs significant training overhead for each optimization task, limiting its feasibility in real-world applications. In this work, we propose to leverage a foundation model (FM) to improve MARL efficiency across diverse RA network optimization tasks. Specifically, we design an FM-aided actor-critic algorithm within a consensus-based decentralized MARL architecture and provide its convergence analysis under local reward exchanges and nonlinear value function approximations to show that our algorithm achieves the same convergence order as the conventional MARL with critic model exchanges and linear approximations. Our numerical results show that our FM-based approach significantly enhances MARL speed for RA network optimization.

0.39

Federated Agentic AI for Interference-Aware Adaptive Spectrum Coordination

cs.NI ·2026-10-29 ·Tomer Baziza Paz, Yaniv Cohen, Kobi Cohen
benchmark similarity 0.67
Abstract · 13 pages, 6 figures

Emerging intelligent wireless networks increasingly rely on autonomous AI agents to adapt spectrum usage to dynamic and heterogeneous operating conditions. However, collective learning is challenging when agents experience different local interference environments and cannot exchange raw observations due to communication-overhead and privacy constraints. We propose Federated Agentic Spectrum Coordination with Interference Awareness (FASCIA), a federated multi-agent deep reinforcement learning framework for adaptive spectrum coordination. In FASCIA, each network manager acts as an autonomous AI agent that perceives its local interference environment, accumulates experience in a persistent local memory, and learns an adaptive spectrum coordination policy through local interactions. Coordination among agents is achieved through periodic federated aggregation of locally trained models, without exchanging raw sensing data or experience. Since spectrum decisions made by one agent alter the environments observed by others, local learning is inherently heterogeneous and coupled. FASCIA employs proximal regularization to mitigate the resulting model drift and explicitly accounts for inter-carrier interference arising from partially overlapping channels. FASCIA is the first federated agentic AI framework to address this challenging interference-coupled spectrum coordination setting. Extensive simulations demonstrate that FASCIA achieves performance close to the centralized benchmark while avoiding raw-experience exchange and maintaining effectiveness across heterogeneous network configurations.

0.11

Multi-Agent Spectrum Sharing

cs.LG ·2026-10-03 ·Job Elliott, Graduate Student Member, IEEE, Justin G. Metcalf, Golnaz Habibi
benchmark similarity 0.66
Abstract

This project explores how multiple cognitive radars can learn to share limited wireless spectrum with other radio users without interfering with one another. Using machine learning (ML), each device independently decides where and how widely to transmit within a fixed 100 MHz band. The system analyzes real or simulated signal activity to detect which parts of the spectrum are currently in use and which are open. Based on this information, the devices adapt their transmission choices to avoid crowded frequencies while making efficient use of available space. The goal is to develop a flexible, scalable approach to spectrum sharing that could support future wireless communication systems. Experimental results using both over-the-air software-defined radio (SDR) recordings and simulated environments demonstrate that the proposed meta-learning approach consistently balances competing objectives better than conventional reinforcement learning (RL) methods in multi-agent spectrum-sharing scenarios. Across five multi-agent benchmark environments, our proposed method achieved the highest average reward among the primary baseline algorithms while simultaneously maintaining low collision rates and stable transmission behavior.

0.17

CoSlice: Coordinated Multi-Agent Reinforcement Learning for 5G Network Slicing Allocation

cs.NI cs.LGcs.MA ·2026-11-18 ·Pengxin Wang, Nakjung Choi, Jingdi Chen
similarity 0.66
Abstract

Radio Access Network (RAN) slicing is a shared resource allocation problem: each slice seeks to satisfy its Service Level Agreement (SLA) with minimal radio-resource consumption, while all slices compete for a limited resource budget. This creates both an intra-slice trade-off between SLA assurance and resource efficiency and an inter-slice conflict over shared resources, calling for specialized yet coordinated control. In this work, we present CoSlice, a coordinated multi-agent reinforcement learning framework for inter-slice physical resource block allocation in Open Radio Access Network (O-RAN). CoSlice uses preference-conditioned slice agents for fast resource allocation and an LLM-based coordinator to adapt their objectives using global network observations at a slower timescale. We formulate the problem as a multi-objective partially observable stochastic game (mo-POSG), establish the existence of a Preference Correlated Equilibrium, and derive a sublinear regret bound for the learning procedure. We implement the algorithm by using a two-stage training procedure that consists of multi-objective pretraining and joint optimization. We build a network slicing evaluation environment using ORANSlice and its radio-frequency simulator (RFsim), supporting realistic protocol-stack execution and application traffic across multiple slices. Experimental results show that CoSlice reduces both average and tail SLA violations compared with other centralized and multi-agent baselines under diverse slice workloads.

0.24

From Prior-Guided Heuristics to Deployable Agents: Accelerating Demonstration-Driven Reinforcement Learning for Deadline-Constrained Network Control

cs.NI cs.AI ·2026-09-03 ·Vincenzo Norman Vitale, Mohammad Solki, Antonia Maria Tulino, Andreas F. Molisch, Jaime Llorca
similarity 0.62
Abstract

Timely delivery of delay-sensitive information over dynamic, heterogeneous networks is essential for NextG interactive applications, yet providing strict End-to-End (E2E) peak latency guarantees remains an open challenge. Two obstacles limit the adoption of learning-based network control in this setting: traditional volume-based routing metrics, while highly effective for general traffic management, are not designed to capture traffic urgency; and Deep Reinforcement Learning (DRL) controllers trained from scratch suffer from sample inefficiency, long training times, and early-stage exploration volatility. This paper introduces a deployment-focused network control framework that addresses both obstacles. First, we present Effective Congestion (EC), a deadline-aware metric family that quantifies interface congestion by packet urgency and proactively filters non-viable traffic, coupled with a Uniform Path Grouping (UPG) distribution heuristic promoting robust load-balancing; the resulting policies are embedded into Multi-Agent Deep Reinforcement Learning Effective Congestion ($p^*$) (MADRL EC ($p^*$)), a hybrid architecture combining a distributed scheduler with a centralized RL-based router. Second, we introduce a unified training objective that generalizes existing policy-learning paradigms---behavioral cloning, offline Reinforcement Learning (RL), online RL, and offline-to-online schemes---as special cases, combining a live-reward term, a pre-collected-reward term, and a policy-imitation term. From this objective, we derive the Model-Guided Annealed Reinforcement Learning (MGA-RL) protocol, instantiated on a Deep Deterministic Policy Gradient (DDPG) backbone: a deployment-oriented, demonstration-driven training approach that generalizes conventional Offline-to-Online (O2O) schemes, in which trajectories from a lightweight [...]

0.10

Distributed Constrained Resource Management in 6G Networks: A Scalable Hybrid Model-Learning Framework

cs.IT ·2026-10-08 ·Thang X. Vu, Cuong Le, Symeon Chatzinotas, Bjorn Ottersten
published similarity 0.62
Abstract

Radio Resource Management (RRM) is a fundamental challenge in 6G wireless networks, particularly under dynamic user demands, inter-cell interference, and heterogeneous QoS constraints. Centralized optimization solutions are often infeasible in practice due to the lack of system-wide state information, dynamic conditions, and signaling delays, making distributed learning-based approaches attractive. However, conventional multi-agent reinforcement learning (MARL) struggles with scalability and constraint satisfaction in such highly dynamic environments. We introduce a scalable two-phase hybrid learning framework to address the RRM challenges where orthogonal-frequency division multiplexing (OFDM) domain knowledge is explicitly incorporated into the MARL pipeline. In our proposed two-phase learning framework, the first phase allocates a minimum resource to satisfy users' QoS requirements based on channel statistics, avoiding the inefficiencies of training policies under hard QoS constraints. Subsequently, a multiagent system is employed in the second phase to optimally allocate the remaining resources for system throughput maximization. By exploiting the structural interference information, we propose a scalable MARL algorithm which decomposes the original learning problem into smaller subproblems that can be handled independently, thereby avoiding exponential growth of the action space without compromising performance. Extensive simulations in realistic scenarios with 50MHz bandwidth and different numerologies show that our method significantly outperforms existing optimization and learning baselines, offering up to 40% throughput improvement, and 100% constraint satisfaction with minimal resource usage.

0.34