CS Paper Radar

Interference-Graph-Aided Multi-Agent Reinforcement Learning for Beam Hopping LEO Satellite Networks

Hao Yuan, Ziyi Yang, Xing Zhang

eess.SP ·submitted 2026-10-20 14:07 UTC · v1

Abstract

Beam hopping technology enables LEO satellites to dynamically switch beam coverage positions, providing flexible resource allocation for non-uniformly distributed traffic demands. However, the severe spatial interference coupling among beams makes coordinated scheduling challenging. To address this problem, in this letter, we propose an interference-graph-aided multi-agent reinforcement learning approach that constructs an interference graph to capture beam coupling relationships and injects this structural prior into agent-side representation learning. Each beam agent aggregates information from interference neighbors via graph convolution before estimating local Q-values, which are then decomposed by the standard mixing network. Simulation results demonstrate that the proposed method achieves 34.4\% higher throughput than QMIX with more stable interference management, under non-uniform traffic and co-channel interference settings.

Closest papers

Nearest neighbours by meaning across the whole index.

Topology-Aware Cooperative Beam-Hopping Scheduling for Efficient Resource Allocation in LEO Satellite Systems

cs.NI ·2026-10-08 ·Shuang Zheng, Xing Zhang, Quan Z. Sheng, Wenbo Wang
published similarity 0.72
Abstract · 5 pages, 5 figures, accepted

The dynamic topology and heterogeneous traffic demands of multi-satellite systems present substantial challenges for beam-hopping (BH) resource management. This letter develops a topology-aware cooperative BH framework for multi-satellite systems under partial observability. A lightweight load-balancing scheme first determines the serving relationships, which are then exploited to construct an association-induced satellite-user graph. A two-phase graph neural network (GNN) extracts episode-level topology-aware structural identities for parameter-sharing agents, while a recurrent multi-agent proximal policy optimization (MAPPO) policy handles slot-level dynamics for joint beam scheduling and onboard power control. Simulation results demonstrate notable gains in energy efficiency, throughput, and fairness over the baselines.

0.27

Mission-critical spectrum sharing with decentralized Multi-Agent Reinforcement Learning

eess.SY cs.ITmath.OC ·2026-10-06 ·Dimitrios Pylorof, Imtiaz Nasim, Humberto E. Garcia, Vivek Agarwal, Jasni A. Mannil +1 more
published similarity 0.70
Abstract · Accepted for publication in the proceedings of IEEE CCNC 2027

Motivated by emerging mission-critical applications and an increasingly congested spectrum, we develop a decentralized multi-agent reinforcement learning (MARL) model for dynamic spectrum access. The model enables secondary users to learn effective transmission strategies across shared frequency bands while minimizing collisions with high-priority primary users and among themselves. We design the agent-level learners following a Markov potential game approach, connecting independent local updates to system-level improvement. We instantiate this design using lightweight linear actor-critic learners suitable for resource-constrained edge devices, rather than computationally intensive centralized or deep multi-agent architectures. Across spectrum environments with different incumbent activities, the learned policies adapt their transmission policy and waiting behavior to preserve throughput while greatly reducing transmission collisions relative to random and forecast-aware heuristic baselines. The results establish the value of decentralized MARL and shows up to 96.8% reduction in overall collisions.

0.32

IRIS: Intelligent Random Access with Integrated Shared Learning for Decentralized LEO Direct-to-Satellite Uplinks

cs.NI ·2026-11-04 ·Abdullah Al Noman, Gaurav Singh, Rameez Ahmed, Debashri Roy
similarity 0.68
Abstract

Grant-free uplink access in low-Earth-orbit (LEO) non-terrestrial networks (NTNs) poses a coordination challenge when multiple UEs share a satellite channel without real-time centralized scheduling under long propagation delays and time-varying channels. Conventional random access does not exploit peer context, leading to collisions and inefficient channel use as contention increases. To address this problem, we propose IRIS, a multi-agent reinforcement learning framework for decentralized grant-free direct-to-satellite (D2S) uplink access, with A2A-IRIS as its sidelink-assisted Agent-to-Agent (A2A) realization. Each UE executes a shared binary transmission policy using local traffic/channel information augmented by compact peer states exchanged over UE-to-UE sidelink, while Vanilla-IRIS serves as a local-only ablation. Evaluations in rural and suburban LEO D2S deployments use TLE-based trajectories propagated using the Simplified General Perturbations 4 (SGP4), with a Walker-Delta constellation as fallback. A2A-IRIS achieves 2.1579 and 5.4478 Mbps, corresponding to 99.9% and 96.1% of the respective scenario-specific upper bounds. Relative to Vanilla-IRIS, throughput improves by 40.8% and 325.0%, with collision rates of 0.0000 and 0.0433, while one- and two-epoch delayed centralized schedulers are outperformed by approximately 51--130%. A2A-IRIS also improves fairness, service coverage, and transmission efficiency with moderate Deep Q-Network (DQN) inference overhead. Active-UE load scalability results further show sustained performance over the evaluated load range. These results demonstrate the effectiveness of sidelink-assisted A2A coordination for decentralized LEO D2S uplink access under propagation delay and delayed channel feedback.

0.08

Multi-Agent Reinforcement Learning for Joint Handover Management and Power Allocation in Multi-Orbit Satellite Networks

eess.SP eess.SY ·2026-08-14 ·Yassine Afif, Ashutosh Balakrishnan, Philippe Martins, Mohammed Almekhlafi, Antoine Lesage-Landry +1 more
similarity 0.66
Abstract

Future sixth-generation non-terrestrial networks are expected to combine low Earth orbit (LEO), medium Earth orbit (MEO), and geostationary Earth orbit (GEO) satellites, whose complementary layers must be coordinated through joint user association, power allocation, and handover management under fast LEO dynamics. This paper studies this problem by formulating it as a mixed-integer nonlinear program and decomposing it into a multi-agent reinforcement learning (MARL) policy that selects the associations and a convex power-allocation subproblem solved exactly at each time slot that defines the reward of the MARL part. The association policy is trained with multi-agent proximal policy optimization (MAPPO) and the targeted multi-agent communication (TarMAC) mechanism, and is made aware of the orbital layer through a state that encodes layer-dependent handover penalties. Evaluated on a realistic multi-constellation scenario built from real two-line element data over Nairobi, Kenya, the proposed policy reaches 92% of the throughput of a greedy signal-to-noise ratio (SNR) maximizing scheme while triggering more than four times fewer handovers, and improves throughput by roughly 14% over a conservative stay heuristic. Compared to an LEO-only learned policy of identical architecture, it attains slightly higher throughput with fewer handovers by offloading a fraction of the users to the MEO and GEO layers, an emergent multi-orbit behavior that drives its favorable throughput and handover trade-off.

0.04

RAMA: Retrieval-Augmented Multi-Agent Routing with Episodic Memory for LEO Satellite Networks

cs.NI cs.LG ·2026-10-30 ·Xin Ming Li, Peng Hu
similarity 0.64
Abstract

Low-Earth orbit (LEO) satellite networks present a challenging routing environment due to rapidly changing topology, traffic, and link conditions. Existing deep reinforcement learning (DRL) routing methods typically rely on current observations and information encoded in network weights, limiting their ability to exploit successful past experiences. We propose RAMA, a retrieval-augmented multi-agent DRL framework with Approximate Information State (AIS), in which each satellite combines its local observation, recent routing history, neighboring state information, and retrieved entries from a per-satellite episodic memory. Retrieved experiences are integrated through a Transformer-based decision network trained with AIS-based auxiliary objectives. With weights frozen in a 140-satellite Kepler constellation, RAMA reduces mean end-to-end latency by 13.4% and 99th-percentile latency by 66.7% relative to the state-of-theart double deep Q-network (DDQN) routing method, mainly by avoiding long detours. Updated online, RAMA remains stable while DDQN degrades. On the 1,584-satellite Starlink shell 1, the reduction in mean latency grows to 93.2%, and updating RAMA online lowers its mean latency by a further 25.5%. These results demonstrate the effectiveness of retrieval-augmented memory for scalable decentralized routing in LEO satellite networks.

0.24

Federated Agentic AI for Interference-Aware Adaptive Spectrum Coordination

cs.NI ·2026-10-29 ·Tomer Baziza Paz, Yaniv Cohen, Kobi Cohen
benchmark similarity 0.63
Abstract · 13 pages, 6 figures

Emerging intelligent wireless networks increasingly rely on autonomous AI agents to adapt spectrum usage to dynamic and heterogeneous operating conditions. However, collective learning is challenging when agents experience different local interference environments and cannot exchange raw observations due to communication-overhead and privacy constraints. We propose Federated Agentic Spectrum Coordination with Interference Awareness (FASCIA), a federated multi-agent deep reinforcement learning framework for adaptive spectrum coordination. In FASCIA, each network manager acts as an autonomous AI agent that perceives its local interference environment, accumulates experience in a persistent local memory, and learns an adaptive spectrum coordination policy through local interactions. Coordination among agents is achieved through periodic federated aggregation of locally trained models, without exchanging raw sensing data or experience. Since spectrum decisions made by one agent alter the environments observed by others, local learning is inherently heterogeneous and coupled. FASCIA employs proximal regularization to mitigate the resulting model drift and explicitly accounts for inter-carrier interference arising from partially overlapping channels. FASCIA is the first federated agentic AI framework to address this challenging interference-coupled spectrum coordination setting. Extensive simulations demonstrate that FASCIA achieves performance close to the centralized benchmark while avoiding raw-experience exchange and maintaining effectiveness across heterogeneous network configurations.

0.11

Distributed Constrained Resource Management in 6G Networks: A Scalable Hybrid Model-Learning Framework

cs.IT ·2026-10-08 ·Thang X. Vu, Cuong Le, Symeon Chatzinotas, Bjorn Ottersten
published similarity 0.62
Abstract

Radio Resource Management (RRM) is a fundamental challenge in 6G wireless networks, particularly under dynamic user demands, inter-cell interference, and heterogeneous QoS constraints. Centralized optimization solutions are often infeasible in practice due to the lack of system-wide state information, dynamic conditions, and signaling delays, making distributed learning-based approaches attractive. However, conventional multi-agent reinforcement learning (MARL) struggles with scalability and constraint satisfaction in such highly dynamic environments. We introduce a scalable two-phase hybrid learning framework to address the RRM challenges where orthogonal-frequency division multiplexing (OFDM) domain knowledge is explicitly incorporated into the MARL pipeline. In our proposed two-phase learning framework, the first phase allocates a minimum resource to satisfy users' QoS requirements based on channel statistics, avoiding the inefficiencies of training policies under hard QoS constraints. Subsequently, a multiagent system is employed in the second phase to optimally allocate the remaining resources for system throughput maximization. By exploiting the structural interference information, we propose a scalable MARL algorithm which decomposes the original learning problem into smaller subproblems that can be handled independently, thereby avoiding exponential growth of the action space without compromising performance. Extensive simulations in realistic scenarios with 50MHz bandwidth and different numerologies show that our method significantly outperforms existing optimization and learning baselines, offering up to 40% throughput improvement, and 100% constraint satisfaction with minimal resource usage.

0.34

Multi-Agent Reinforcement Learning for Movable Antenna-aided Cell-Free Massive MIMO Systems

cs.IT ·2026-10-06 ·Bokai Xu, Jiayi Zhang, Shuaifei Chen, Ziheng Liu, Huahua Xiao +2 more
published similarity 0.62
Abstract

The inherent non-convex minimum-separation constraints introduced by movable antennas present a formidable challenge to the joint optimization of antenna positions and transmission strategies, rendering conventional methods computationally infeasible, particularly in large-scale cell-free massive multiple-input multiple-output (MIMO). In this work, we propose the graph-based learning individual intrinsic reward heterogeneous-agent proximal policy optimization (GLIIR-HAPPO) algorithm, a novel heterogeneous multi-agent reinforcement learning (MARL) framework that fundamentally overcomes this impasse by systematically decomposing the original coupled optimization into coordinated subproblems. To ensure tractability, we embed the non-convex geometric constraints into a penalty-augmented reward structure and develop a specialized geometric solver that enables the positioning agents to efficiently navigate the high-dimensional action space. Specifically, we propose an architecture featuring a dynamic-interaction graph critic for adaptive cross-role coordination, together with role-conditioned federated distillation that synchronizes same-role policies through compact actor-output statistics. Beyond architectural design, we establish a rigorous theoretical analysis that derives monotonic performance improvement bounds and establishes convergence guarantees for the proposed bi-level optimization. Numerical simulations demonstrate that our framework yields significant sum-rate improvements over state-of-the-art MARL schemes. Moreover, the performance of our advanced architecture closely approaches its fully centralized counterpart, while drastically reducing communication overhead.

0.59