CS Paper Radar

Multi-Agent Reinforcement Learning for Joint Handover Management and Power Allocation in Multi-Orbit Satellite Networks

Yassine Afif, Ashutosh Balakrishnan, Philippe Martins, Mohammed Almekhlafi, Antoine Lesage-Landry, Gunes Karabulut Kurt

eess.SP eess.SY ·submitted 2026-08-14 14:22 UTC · v1

Abstract

Future sixth-generation non-terrestrial networks are expected to combine low Earth orbit (LEO), medium Earth orbit (MEO), and geostationary Earth orbit (GEO) satellites, whose complementary layers must be coordinated through joint user association, power allocation, and handover management under fast LEO dynamics. This paper studies this problem by formulating it as a mixed-integer nonlinear program and decomposing it into a multi-agent reinforcement learning (MARL) policy that selects the associations and a convex power-allocation subproblem solved exactly at each time slot that defines the reward of the MARL part. The association policy is trained with multi-agent proximal policy optimization (MAPPO) and the targeted multi-agent communication (TarMAC) mechanism, and is made aware of the orbital layer through a state that encodes layer-dependent handover penalties. Evaluated on a realistic multi-constellation scenario built from real two-line element data over Nairobi, Kenya, the proposed policy reaches 92% of the throughput of a greedy signal-to-noise ratio (SNR) maximizing scheme while triggering more than four times fewer handovers, and improves throughput by roughly 14% over a conservative stay heuristic. Compared to an LEO-only learned policy of identical architecture, it attains slightly higher throughput with fewer handovers by offloading a fraction of the users to the MEO and GEO layers, an emergent multi-orbit behavior that drives its favorable throughput and handover trade-off.

Closest papers

Nearest neighbours by meaning across the whole index.

IRIS: Intelligent Random Access with Integrated Shared Learning for Decentralized LEO Direct-to-Satellite Uplinks

cs.NI ·2026-11-04 ·Abdullah Al Noman, Gaurav Singh, Rameez Ahmed, Debashri Roy
similarity 0.72
Abstract

Grant-free uplink access in low-Earth-orbit (LEO) non-terrestrial networks (NTNs) poses a coordination challenge when multiple UEs share a satellite channel without real-time centralized scheduling under long propagation delays and time-varying channels. Conventional random access does not exploit peer context, leading to collisions and inefficient channel use as contention increases. To address this problem, we propose IRIS, a multi-agent reinforcement learning framework for decentralized grant-free direct-to-satellite (D2S) uplink access, with A2A-IRIS as its sidelink-assisted Agent-to-Agent (A2A) realization. Each UE executes a shared binary transmission policy using local traffic/channel information augmented by compact peer states exchanged over UE-to-UE sidelink, while Vanilla-IRIS serves as a local-only ablation. Evaluations in rural and suburban LEO D2S deployments use TLE-based trajectories propagated using the Simplified General Perturbations 4 (SGP4), with a Walker-Delta constellation as fallback. A2A-IRIS achieves 2.1579 and 5.4478 Mbps, corresponding to 99.9% and 96.1% of the respective scenario-specific upper bounds. Relative to Vanilla-IRIS, throughput improves by 40.8% and 325.0%, with collision rates of 0.0000 and 0.0433, while one- and two-epoch delayed centralized schedulers are outperformed by approximately 51--130%. A2A-IRIS also improves fairness, service coverage, and transmission efficiency with moderate Deep Q-Network (DQN) inference overhead. Active-UE load scalability results further show sustained performance over the evaluated load range. These results demonstrate the effectiveness of sidelink-assisted A2A coordination for decentralized LEO D2S uplink access under propagation delay and delayed channel feedback.

0.08

Topology-Aware Cooperative Beam-Hopping Scheduling for Efficient Resource Allocation in LEO Satellite Systems

cs.NI ·2026-10-08 ·Shuang Zheng, Xing Zhang, Quan Z. Sheng, Wenbo Wang
published similarity 0.72
Abstract · 5 pages, 5 figures, accepted

The dynamic topology and heterogeneous traffic demands of multi-satellite systems present substantial challenges for beam-hopping (BH) resource management. This letter develops a topology-aware cooperative BH framework for multi-satellite systems under partial observability. A lightweight load-balancing scheme first determines the serving relationships, which are then exploited to construct an association-induced satellite-user graph. A two-phase graph neural network (GNN) extracts episode-level topology-aware structural identities for parameter-sharing agents, while a recurrent multi-agent proximal policy optimization (MAPPO) policy handles slot-level dynamics for joint beam scheduling and onboard power control. Simulation results demonstrate notable gains in energy efficiency, throughput, and fairness over the baselines.

0.27

Demand-Aware Cooperative Transmission Design for Energy-Efficient LEO Satellite Networks

cs.IT eess.SP ·2026-08-10 ·Wooseok Cha, Kyeongsoo Kim, Seonghoon Kim, Junil Choi, Jihwan P. Choi
top venuebenchmark similarity 0.68
Abstract · 17 pages, 12 figures, accepted to IEEE Transactions on Wireless Communications

Low Earth orbit (LEO) satellite networks are envisioned as a promising solution for providing ubiquitous connectivity and narrowing the digital divide. The extensive footprint of LEO satellite constellations enables broad coverage, resulting in spatially non-uniform traffic demand across the serviced areas. Meanwhile, stringent on-board power constraints make power-intensive transmission architectures less attractive and motivate energy-efficient transmission strategies that effectively exploit scarce satellite network resources. To this end, this paper proposes a cooperative transmission framework that jointly accounts for non-uniform traffic demand and network-wide power consumption. Each LEO satellite integrates hybrid precoding (HPC), radio frequency (RF) chain activation, and hardware quantization, while user-equipment (UE)-centric satellite clusters are organized using statistical channel state information (sCSI) and traffic demands. A framework for joint optimization of cooperative transmission architecture and resource allocation is designed to maximize demand-aware energy efficiency (EE), resulting in a mixed-integer nonlinear program (MINLP) for which finding a globally optimal solution is generally intractable. Accordingly, a two-stage algorithm is developed under a distributed linear precoding structure, in which a modified cross-entropy (CE) method searches over discrete variables, while fractional programming is employed for transmit power allocation. Numerical results indicate that the proposed framework outperforms benchmark schemes while accounting for traffic demands and EE.

0.35

Adaptive Beam Hopping and Power Control for Dual-Layer Over-the-Air Online Federated Learning in LEO Satellite Networks

cs.IT eess.SP ·2026-09-02 ·Zhendong Li, Shaojie Wang, Zhou Su, Zihao Zhang, Haixia Peng +3 more
benchmark similarity 0.67
Abstract

This paper investigates over-the-air (OTA) computation enabled online federated learning (FL) in low-Earth orbit (LEO) satellite networks. Specifically, we consider a dual-layer OTA aggregation architecture, where ground devices upload analog model updates to serving satellites via uplink OTA aggregation, and satellites forward the aggregated signals to a data processing center through the second round OTA aggregation. Then, we formulate a long-term data-utilization maximization problem in which devices continuously collect new data and untrained samples gradually lose freshness. The problem is subject to the satellite beam budget, transmit-power limit, and global mean squared error (MSE) constraint that governs end-to-end aggregation distortion. This yields a coupled mixed-integer nonlinear programming (MINLP) problem, involving tightly coupled discrete beam-hopping decisions and continuous power control. Due to the combinatorial action space and nonconvex constraints, the problem is NP-hard and computationally intractable. Furthermore, the time-varying satellite topology and dynamic data generation render it a sequential decision-making problem, necessitating adaptive online scheduling. To address these issues, we cast the problem as a Markov decision process and develop a proximal policy optimization (PPO)-based deep reinforcement learning framework that jointly optimizes adaptive beam hopping and power control, using an MSE-aware reward to balance data utilization and aggregation accuracy. Numerical simulation results verify that the proposed algorithm consistently outperforms other benchmark schemes, achieving superior long-term data utilization and faster FL convergence while satisfying the MSE requirement.

0.17

ORCA: Onboard Reflective Collaborative Agents for Reliable Task Offloading in LEO Satellite Networks

eess.SP cs.NI ·2026-10-20 ·Yunxin Geng, Shengxiang Hao, Yichen Wu, Haotian Liu, Weiwei Jiang +4 more
hot direction similarity 0.66
Abstract · 6 pages, 4 figures, IEEE Global Communications Conference (GLOBECOM) 2026 Workshop

Low Earth orbit (LEO) satellites increasingly perform time-critical onboard processing, yet heterogeneous computing capabilities, rapidly changing inter-satellite links (ISLs) and uncertain neighbor behavior make decentralized task offloading difficult under partial observations. This paper presents ORCA, an architecture of Onboard Reflective Collaborative Agents for autonomous onboard task offloading. ORCA organizes offloading decisions into a four-Agent \emph{Perceive}$\rightarrow$\emph{Reason}$\rightarrow$ \emph{Act}$\rightarrow$\emph{Learn} loop. It aims to improve efficiency and reliability in dynamic, heterogeneous, and unreliable LEO satellite networks. The simulation results of a heterogeneous 60-satellite Walker-delta constellation demonstrates that ORCA delivers competitive system efficiency in cooperative settings while achieving superior task-success robustness and offloading reliability under previously unseen malicious node behavior, indicating that outcome-grounded reflection can improve the reliability of decentralized onboard offloading without centralized runtime coordination.

0.36

Interference-Graph-Aided Multi-Agent Reinforcement Learning for Beam Hopping LEO Satellite Networks

eess.SP ·2026-10-20 ·Hao Yuan, Ziyi Yang, Xing Zhang
similarity 0.66
Abstract

Beam hopping technology enables LEO satellites to dynamically switch beam coverage positions, providing flexible resource allocation for non-uniformly distributed traffic demands. However, the severe spatial interference coupling among beams makes coordinated scheduling challenging. To address this problem, in this letter, we propose an interference-graph-aided multi-agent reinforcement learning approach that constructs an interference graph to capture beam coupling relationships and injects this structural prior into agent-side representation learning. Each beam agent aggregates information from interference neighbors via graph convolution before estimating local Q-values, which are then decomposed by the standard mixing network. Simulation results demonstrate that the proposed method achieves 34.4\% higher throughput than QMIX with more stable interference management, under non-uniform traffic and co-channel interference settings.

0.14

ParaBlock++: Communication-Computation Parallel Blockwise Hierarchical Federated Learning

cs.LG ·2026-10-31 ·Kai Wang, Chee Wei Tan
similarity 0.65
Abstract · A journal version extended from the published conference version

In the emerging 6G paradigm, where AI-driven services operate over decentralized and resource-constrained infrastructures, the convergence of distributed learning algorithms remains a significant challenge. This paper introduces $\mathrm{ParaBlock}$++, a low Earth orbit (LEO) satellite-assisted hierarchical federated learning (HFL) framework, enabling resource-limited terrestrial clients to collaboratively engage in federated learning (FL) via ground-satellite links (GSLs) supported by LEO satellite megaconstellations. Within this architecture, LEO satellites serve multiple roles: as edge servers aggregating and broadcasting models, as clients accelerating training, and as high-throughput relays managing inter-satellite routing through inter-satellite links (ISLs). Departing from conventional synchronous Jacobian-type updates, $\mathrm{ParaBlock}$++ employs asynchronous BCD-type updates to enhance convergence speed. We provide theoretical convergence guarantees for convex, strongly convex, and non-convex objectives, and formulate optimization problems to determine optimal cluster resource allocation and model partitioning for minimizing global training latency. Experimental evaluations demonstrate that $\mathrm{ParaBlock}$++ achieves faster initial convergence and comparable final performance for deep neural networks (DNNs) such as ResNet-18, offering promising directions for federated learning across geographically remote areas enabled by 6G technologies, but much less global rounds and communication cost.

0.19

Risk-Aware Handover Management Approach for Non-Terrestrial Networks with the NTN-Connect Simulator

eess.SP ·2026-11-16 ·Mohammed Almekhlafi, Antoine Lesage-Landry, Gunes Karabulut Kurt
similarity 0.65
Abstract

The continuous movement of low Earth orbit (LEO) satellites in non-terrestrial networks (NTNs) leads to frequent handovers, requiring joint optimization of visibility duration, transmit power, and user quality-of-service (QoS). This work address this issue by proposing a risk-aware handover management framework that minimizes transmit power and maximizes admitted users under per-satellite power budgets and user-rate chance constraints, while a preference score term penalizes short duration associations. We decompose the resulting mixed-integer nonlinear program (MINLP) into an inner power allocation subproblem and an outer generalized assignment association subproblem. The inner subproblem reformulates the chance constraint into its deterministic counterpart using the value-at-risk (VaR) closed-form. The obtained transmit power allocation is then used in the association subproblem, which is solved optimally using an off-the-shelf solver. Finally, we employ the polynomial-complexity Hungarian algorithm to solve the relaxed assignment problem and enforce the per-satellite power budget using a knapsack algorithm. Simulations, using the proposed NTN-Connect simulator, on the OneWeb and Starlink constellations show that the proposed low-complexity solution achieves near-optimal performance and better performance compared to the risk-aware maximum stay (RA-MST) and risk-aware maximum received signal received power (RA-MRSRP) baselines. On the Starlink constellation, it increases the inter-handover interval (IHI) from $0.37$ to $1.47$~min, reducing the handover rate by $75\%$. At a maximum-IHI operating point, it reduces the total transmit power by 1.1 dBW (22\%) while increasing the admission rate from 97.4\% to 99.3\%.

0.26