CS Paper Radar

Foundation Model-Aided Multi-Agent Reinforcement Learning for Wireless Random Access Network Optimization

Myeung Suk Oh, Zhiyao Zhang, Alvaro Velasquez, Nathaniel D. Bastian, Jia Liu

cs.LG cs.AI ·submitted 2026-10-06 00:27 UTC · v1
published

Abstract

Random access (RA) is one of the most foundational medium access control (MAC) layer scheduling schemes for handling unpredictable data traffic from multiple terminals. While multi-agent reinforcement learning (MARL) has been explored to optimize RA-based wireless networks, its reliance on experience-driven, distributed policy learning incurs significant training overhead for each optimization task, limiting its feasibility in real-world applications. In this work, we propose to leverage a foundation model (FM) to improve MARL efficiency across diverse RA network optimization tasks. Specifically, we design an FM-aided actor-critic algorithm within a consensus-based decentralized MARL architecture and provide its convergence analysis under local reward exchanges and nonlinear value function approximations to show that our algorithm achieves the same convergence order as the conventional MARL with critic model exchanges and linear approximations. Our numerical results show that our FM-based approach significantly enhances MARL speed for RA network optimization.

Comment
This paper has been accepted in ACM International Symposium on Theory, Algorithmic Foundations, and Protocol Design for Mobile Networks and Mobile Computing (MobiHoc) 2026

Closest papers

Nearest neighbours by meaning across the whole index.

Mission-critical spectrum sharing with decentralized Multi-Agent Reinforcement Learning

eess.SY cs.ITmath.OC ·2026-10-06 ·Dimitrios Pylorof, Imtiaz Nasim, Humberto E. Garcia, Vivek Agarwal, Jasni A. Mannil +1 more
published similarity 0.72
Abstract · Accepted for publication in the proceedings of IEEE CCNC 2027

Motivated by emerging mission-critical applications and an increasingly congested spectrum, we develop a decentralized multi-agent reinforcement learning (MARL) model for dynamic spectrum access. The model enables secondary users to learn effective transmission strategies across shared frequency bands while minimizing collisions with high-priority primary users and among themselves. We design the agent-level learners following a Markov potential game approach, connecting independent local updates to system-level improvement. We instantiate this design using lightweight linear actor-critic learners suitable for resource-constrained edge devices, rather than computationally intensive centralized or deep multi-agent architectures. Across spectrum environments with different incumbent activities, the learned policies adapt their transmission policy and waiting behavior to preserve throughput while greatly reducing transmission collisions relative to random and forecast-aware heuristic baselines. The results establish the value of decentralized MARL and shows up to 96.8% reduction in overall collisions.

0.32

Finite-Sample Convergence in Networked Average Reward MARL: Decentralization Pitfalls and Entropy Remedies

cs.LG cs.MA ·2026-11-02 ·Yizhou Zhang, Yashaswini Murthy, Laixi Shi, Adam Wierman
similarity 0.69
Abstract

We study the problem of average reward Multi-Agent Reinforcement Learning (MARL) where agents interact within a network. Each agent in the network conducts local updates with information within its $k$-hop neighborhood and collaboratively maximizes the overall reward of the entire network. We first provide impossibility results in achieving convergence to the global optimal joint policies in our setting. These impossibility results highlight the challenges of decentralized learning with local information in multi-agent systems, namely the issues of multiagency, where each agent independently optimizes its policy, and partial observability, where agents have limited visibility into the global state. Given these challenges which indicate that decentralized policy optimization can generally be certified only up to stationarity, we provide finite-sample convergence guarantees for a decentralized actor-critic algorithm with linear function approximation that converges to an approximate stationary point with small gradient norm. To overcome such impossibility results, we further show that adding fixed-state entropy regularization reshapes the objective landscape which leads to even stronger global convergence guarantees for our proposed actor-critic algorithm.

0.18

Feedback-Aware Residual Multi-Agent Reinforcement Learning for Timely Communication in Cognitive Radio Networks

cs.NI ·2026-11-14 ·Baolin Chong, Xiuqi Chen, Jiandong Guo, Fuhui Zhou, Qihui Wu
similarity 0.68
Abstract

Supporting time-sensitive services in cognitive radio networks requires distributed access strategies that translate shared spectrum opportunities into timely packet delivery. Achieving this objective is challenging because reception conditions are only partially observable to individual transmitters, while dynamic competition and access overhead constrain the service that can be completed within packet deadlines. We formulate a distributed access problem that captures the coupling among interference-dependent reception, queue evolution, and protocol durations. To address this problem, we propose feedback-aware residual multi-agent proximal policy optimization (FAR-MAPPO) under centralized training and decentralized execution. The method constructs channel-specific historical feedback representations that complement instantaneous sensing with evidence from completed access attempts. A residual policy network integrates this experience with current observations and packet urgency to guide local access decisions. Numerical results show improved timely delivery over the baseline methods, particularly under heavy traffic and bursty arrivals with mixed packet deadlines.

0.18

Hybrid Offline-Online Multi-Agent Decision Transformers for Wireless Resource Management

eess.SY cs.AI ·2026-08-28 ·Yiming Zhang, Kun Yang, Cong Shen, Dongning Guo
similarity 0.66
Abstract · 11 pages, 9 figures, 3 tables. Submitted to IEEE Journal on Selected Areas in Communications in Aug 2026. The offline training part was pres…

This paper develops a hybrid offline-online multi-agent reinforcement learning framework based on decision transformers. The policy is first pretrained offline via supervised sequence modeling of trajectories generated by existing policies, providing a safe and sample-efficient initialization. It is then fine-tuned online using a hybrid objective that incorporates critic-guided gradients, enabling performance improvements beyond the offline policy. To facilitate stable offline-to-online transfer and effective multi-agent coordination, the framework incorporates return-weighted sampling, a critic conditioned on neighbors' actions, and neighborhood-correlated exploration. The approach is fully distributed: both training and execution rely only on local observations and limited information exchange among neighboring agents. Evaluations with dynamic traffic arrivals in two settings: (i) joint scheduling and power allocation and (ii) coordinated beamforming, show that the proposed method achieves quality-of-service (QoS) performance comparable to centralized methods. Moreover, when pretrained on lower-quality datasets, online fine-tuning is also observed to surpass the initial offline policy. These results demonstrate a promising learning-based alternative for wireless resource management.

0.25

Distributed Constrained Resource Management in 6G Networks: A Scalable Hybrid Model-Learning Framework

cs.IT ·2026-10-08 ·Thang X. Vu, Cuong Le, Symeon Chatzinotas, Bjorn Ottersten
published similarity 0.66
Abstract

Radio Resource Management (RRM) is a fundamental challenge in 6G wireless networks, particularly under dynamic user demands, inter-cell interference, and heterogeneous QoS constraints. Centralized optimization solutions are often infeasible in practice due to the lack of system-wide state information, dynamic conditions, and signaling delays, making distributed learning-based approaches attractive. However, conventional multi-agent reinforcement learning (MARL) struggles with scalability and constraint satisfaction in such highly dynamic environments. We introduce a scalable two-phase hybrid learning framework to address the RRM challenges where orthogonal-frequency division multiplexing (OFDM) domain knowledge is explicitly incorporated into the MARL pipeline. In our proposed two-phase learning framework, the first phase allocates a minimum resource to satisfy users' QoS requirements based on channel statistics, avoiding the inefficiencies of training policies under hard QoS constraints. Subsequently, a multiagent system is employed in the second phase to optimally allocate the remaining resources for system throughput maximization. By exploiting the structural interference information, we propose a scalable MARL algorithm which decomposes the original learning problem into smaller subproblems that can be handled independently, thereby avoiding exponential growth of the action space without compromising performance. Extensive simulations in realistic scenarios with 50MHz bandwidth and different numerologies show that our method significantly outperforms existing optimization and learning baselines, offering up to 40% throughput improvement, and 100% constraint satisfaction with minimal resource usage.

0.34

Deep Reinforcement Learning for 6G AI-RAN: A Comprehensive Survey

cs.NI eess.SP ·2026-08-14 ·Jie Lu, Peihao Yan, Qijun Wang, Ruxin Lin, Huacheng Zeng
survey similarity 0.66
Abstract

The evolution toward sixth-generation (6G) networks is transforming the radio access network (RAN) into a programmable and intelligent control platform that must continuously adapt to heterogeneous services, dynamic environments, and competing performance objectives. Open Radio Access Network (O-RAN) provides the open interfaces, disaggregated architecture, and multi-timescale control loops needed to support this transformation, while deep reinforcement learning (DRL) offers a natural framework for optimizing sequential decisions under uncertainty. However, existing surveys either address artificial intelligence (AI) and machine learning (ML) in O-RAN broadly or focus on isolated DRL use cases, leaving a gap in the systematic connection between DRL methodology, O-RAN architecture, and operational deployment. To the best of our knowledge, this article presents the first dedicated and comprehensive survey of DRL for Open AI-RAN. We review the foundations of model-free, model-based, offline, safe, multi-agent, federated, and transfer learning, and provide an O-RAN-aware framework for formulating RAN control problems through states, observations, actions, rewards, constraints, and temporal structure. We classify DRL applications across radio resource management, mobility management, interference control, traffic steering, energy efficiency, network slicing, integrated sensing and communication, security, and massive MIMO. We further examine multi-agent and federated coordination, foundation models and agentic AI, trustworthy DRL, sim-to-real transfer, continual adaptation, resource-efficient inference, and reinforcement learning operations. Finally, we review experimental platforms, benchmarks, standards, and industry activities, and identify research directions toward sample-efficient, safe, scalable, interoperable, and deployable DRL control for 6G Open AI-RAN.

0.04

Federated Multiagent Soft Actor Critic for IRS Assisted Cognitive Radio Networks under Partial Observability

cs.NI ·2026-10-16 ·Satvika Pullisani, Usama Mir, Ubaid Abbasi
similarity 0.66
Abstract

Intelligent reflecting surfaces (IRSs) can improve spectrum efficiency and interference control in cognitive radio networks (CRNs) by reconfiguring the propagation environment. This paper presents FMASAC, a federated multi-agent soft actor-critic framework for IRS-assisted CRNs under partial observability, in which distributed secondary users (SUs) jointly select their transmit power and the phase shifts of an assigned subset of IRS elements using only local observations. Primary user (PU) protection is incorporated through an interference-penalized reward, and SUs coordinate by periodically averaging model parameters at a federated server, without exchanging raw observations, channel state information, or replay data. We formulate the problem as a partially observable Markov decision process, specify the local learning and aggregation procedure, and characterize its per-round communication cost. Illustrative performance trends contrast FMASAC with centralized SAC, independent SAC, IRS-fixed multi-agent reinforcement learning, and a heuristic CRN strategy. We also outline the simulation-based evaluation and the open challenges, namely constraint handling, agent heterogeneity, and scalability, that are needed to validate the framework.

0.13

A Multiarmed Bandit Approach for LTE-U/Wi-Fi Coexistence in a Multicell Scenario

cs.NI ·2026-10-14 ·Iago Diógenes do Rego, José M. de Castro Neto, Sildolfo F. G. Neto, Pedro M. de Santana, Vicente A. de Sousa +2 more
publishedhighly cited similarity 0.66
Abstract

Recent studies and literature reviews have shown promising results for 3GPP system solutions in unlicensed bands when coexisting with Wi-Fi, either by using the duty cycle (DC) approach or licensed-assisted access (LAA). However, it is widely known that general performance in these coexistence scenarios is dependent on traffic and how the duty cycle is adjusted. Most DC solutions configure their parameters statically, which can result in performance losses when the scenario experiences changes on the offered data. In our previous works, we demonstrated that reinforcement learning (RL) techniques can be used to adjust DC parameters. We showed that a Q-learning (QL) solution that adapts the LTE DC ratio to the transmitted data rate can maximize the Wi-Fi/LTE-Unlicensed (LTE-U) aggregated throughput. In this paper, we extend our previous solution by implementing a simpler and more efficient algorithm based on multiarmed bandit (MAB) theory. We evaluate its performance and compare it with the previous one in different traffic scenarios. The results demonstrate that our new solution offers improved balance in throughput, providing similar results for LTE and Wi-Fi, while still showing a substantial system gain. Moreover, in one of the scenarios, our solution outperforms the previous approach by 6% in system throughput. In terms of user throughput, it achieves more than 100% gain for the users at the 10th percentile of performance, while the old solution only achieves a 10% gain.

0.60