CS Paper Radar

Deep Reinforcement Learning for BD-RIS-Assisted Multi-UAV RSMA Communications

Aamer Mohamed Huroon, Ali A. Nasir, Daniel B. da Costa

cs.IT cs.ET ·submitted 2026-11-02 12:44 UTC · v1
published

Abstract

Beyond-diagonal reconfigurable intelligent surfaces (BD-RISs) extend conventional diagonal RISs by allowing controllable coupling among reflecting elements. This flexibility is attractive for aerial networks, where unmanned aerial vehicles (UAVs) face rapidly varying channels and strict power limits. In this paper, we study a hybrid BD-RIS-assisted multi-UAV rate-splitting multiple access (RSMA) downlink, in which UAVs within favourable BD-RIS coverage use both direct and BD-RIS-aided links, whereas the remaining UAVs are served through direct links only. We formulate a network sum-rate maximization problem that jointly optimizes UAV trajectories, RSMA common-rate allocation, beamforming, BD-RIS scattering matrices, and UAV-BD-RIS association. The resulting mixed-integer non-convex problem is solved by using deep deterministic policy gradient (DDPG) algorithm, equipped with a constraint-preserving action layer that generates reciprocal and lossless BD-RIS responses in closed form and handles the binary association inside the environment transition. Simulation results, averaged over three training seeds under a physically consistent channel model with blocked direct links near the BD-RISs, show that the proposed design attains a sum rate comparable to SAC and higher than twin delayed deep deterministic policy gradient ( TD3), proximal policy optimization (PPO), and deep Q-network (DQN), exceeds per-slot alternating optimization with fixed trajectories at about 1/65 of its per-slot runtime, and benefits mainly from RSMA, while the additional BD-RIS gain is modest.

Comment
6 pages, 5 figures, accepted for presentation at the IEEE GLOBECOM 2026 Workshops

Closest papers

Nearest neighbours by meaning across the whole index.

AoI-Driven Hierarchical Learning for Cooperative Resource Sharing in Multi-Operator UAV Networks

eess.SY ·2026-09-19 ·Atefeh Hajijamali Arani, Mahyar Shirvanimoghaddam, Abolfazl Mehbodniya, Halim Yanikomeroglu, Fumiyuki Adachi
similarity 0.71
Abstract

Uncrewed aerial vehicle (UAV)-assisted networks provide a versatile paradigm for on-demand connectivity. However, in multi-operator aerial networks (MOANs), the joint optimization of cooperative resource sharing and 3D trajectory control to maintain information freshness is a complex combinatorial problem, which can be shown to be NP-hard. To address this computational complexity, we propose an age of information (AoI)-driven hierarchical deep reinforcement learning (DRL) framework. Specifically, a Dueling Double Deep Q-Network (D3QN) architecture is deployed at both the operator and UAV decision layers to mitigate overestimation bias and enhance stability in high-dimensional state spaces. To improve system resilience, we introduce an AoI- and load-aware outage compensation mechanism that prioritizes users based on instantaneous transmission demands and temporal freshness. Furthermore, a normalized load exchange balance metric is incorporated to regulate cooperative behavior and ensure resource fairness across operators. Simulation results demonstrate that the proposed hierarchical D3QN significantly outperforms conventional DRL, non-cooperative, and cooperative benchmarks, reducing the average AoI by up to 56.1% under severe congestion while ensuring superior inter-operator fairness and outage mitigation.

0.11

Heterogeneous Services in Multi-UAV Networks with No-Fly Zone and User Mobility

eess.SY ·2026-10-14 ·Bin Li, Senning Wan, Decheng Zhang, Yonglei Yao, Yu Xue
publishedbenchmark similarity 0.70
Abstract · 12 pages, 10 figures

In this paper, we investigate a wireless communication network supported by multiple uncrewed aerial vehicles (UAVs). To ensure reliable connectivity, UAVs are desired to approach the boundaries of designated no-fly zones (NFZs) without violating them when mobile users (MUs) enter NFZs. To support heterogeneous communication services, MUs are assigned different priority levels based on their individual communication types. Considering the practical constraints arising from user mobility, heterogeneous communication requirements, and NFZs, the objective is to maximize the weighted sum of MU communication rates, where weights correspond to MU priorities. This is achieved by jointly optimizing trajectory planning, UAV transmission power control, and the binary association factor between UAVs and MUs. To address this challenging optimization problem under dynamic constraints, we develop a soft actor-critic algorithm with prioritized experience replay, which learns an adaptive control policy. Numerical results show that the proposed algorithm outperforms benchmark schemes in terms of convergence speed and stability. Moreover, it effectively meets diverse communication requirements, satisfies UAV energy constraints, and ensures reliable operation under NFZ constraints.

0.21

Decision Transformer for UAV-Mounted RIS-Assisted Dynamic D2D Communications

cs.AI cs.RO ·2026-09-09 ·Yaxuan Liu
similarity 0.69
Abstract

This paper studies unmanned aerial vehicle (UAV)-mouted reconfigurable intelligent surface (RIS)-assisted device-to-device (D2D) communication with stochastic link activation. It models UAV motion and attitude, time-varying Rician angles, and angle-dependent RIS reflection. A joint optimization of UAV trajectory, attitude, and RIS phases is formulated to maximize average sum rate under mobility, energy, and hardware constraints. The problem is addressed using deep reinforcement learning and a Decision Transformer trained on expert trajectories from multiple scenarios. Results demonstrate effective cross-scenario generalization, with zero-shot transfer outperforming direct DRL transfer and online fine-tuning achieving competitive performance with fewer interactions.

0.20

Distributed Trajectory Planning and Resource Allocation for Dynamic Multi-UAV Collaborative Computing

cs.NI ·2026-08-24 ·Tiankui Zhang, Wenlong Xu, Tianyi Shi, Xiaoxia Xu, Arumugam Nallanathan
publishedbenchmark similarity 0.68
Abstract

This paper investigates a multiple uncrewed aerial vehicles (UAVs)-enabled distributed mobile edge computing (MEC) framework, where the set of collaborative UAVs dynamically varies over time due to their energy states and service loads. The joint optimization of trajectory planning and resource allocation is formulated as a Stackelberg game, where UAVs and mobile terminals (MTs) are modeled as leaders and followers, respectively. UAVs aim to maximize their benefits by balancing executed workload, energy cost, and resource allocation revenue, while MTs seek to minimize their total overhead, composed of computing delay and resource costs, through offloading and resource-request decisions. A hierarchical joint optimization algorithm is developed within a multi-agent deep reinforcement learning (MADRL) framework to coordinate UAVs and MTs in a distributed manner. At the leader level, UAVs jointly determine their trajectories, task migration ratios, MT-UAV association, and unit computing resource pricing. Each UAV is modeled as an agent in a partially observable Markov decision process, and the agents are jointly trained via multi-agent proximal policy optimization (MAPPO) under the centralized-training-and-decentralized-execution paradigm. At the follower level, MTs determine their optimal task offloading ratios and requested computing resources using a two-stage iterative algorithm. Simulation results demonstrate stable convergence under dynamic UAV participation. Compared to the no-collaboration benchmark, the proposed algorithm improves UAV efficiency by 18.58% through inter-UAV task migration and reduces average MT overhead by 33.77% over the fully offloading scheme. It also outperforms other benchmarks under varying network scales and capabilities by jointly optimizing UAV operations and resource utilization.

0.35

Efficient Joint Trajectory Optimization and Resource Allocation for Secure UAV Networks: A Deep Reinforcement Learning Approach

eess.SY cs.IT ·2026-10-27 ·Abdulmalik Alwarafy, Mohamed Hayajneh, Mustafa Al Juboori, Abdulaziz Almarzooqi, Tahnoon Almazrouei +2 more
similarity 0.68
Abstract · This paper has been submitted for possible publication as a journal paper and is currently under review

Unmanned Aerial Vehicles (UAVs) are increasingly deployed in wireless networks; however, ensuring secure communication and efficient energy usage remains challenging. This paper addresses the joint trajectory optimization and resource allocation problem in UAV networks in the presence of a tethered UAV flying center and a malicious ground eavesdropper. In the proposed model, UAVs are equipped with energy harvesting capabilities and can adaptively switch between transmitting, jamming, and relaying roles to deliver data to a tethered UAV center for optimized network performance. An optimization problem is formulated to maximize a utility function that balances secrecy rate, energy consumption, and interference. To address the high non-convexity of the problem and its real-time decision requirements, a Proximal Policy Optimization (PPO)-based Deep Reinforcement Learning (DRL) algorithm is proposed to solve the problem and learn the network dynamics. Extensive simulation results demonstrate that the proposed approach achieves superior secrecy performance, lower energy consumption, and reduced interference compared to conventional and DRL-based methods.

0.11

Lightweight AI for UAV-Mounted RIS: An Overview

cs.NI ·2026-08-26 ·Sherief Hashima, Kohei Hatano, Eiji Takimoto, Mohamed Rihan, Basem. M. Elhalawany +1 more
survey similarity 0.67
Abstract

Unmanned Aerial Vehicles (UAV)-mounted Reconfigurable Intelligent Surfaces (RIS) have emerged as a promising architecture for enhancing wireless coverage, spectral efficiency, and energy performance in 6G networks. By combining programmable electromagnetic wave manipulation with aerial mobility, UAV-RIS systems enable dynamic blockage mitigation, adaptive beamforming, and flexible deployment across terrestrial, maritime, and satellite-integrated environments. However, joint optimization of UAV trajectory, RIS phase configuration, and resource allocation incurs high computational complexity, which is incompatible with the strict energy and onboard processing constraints of UAV platforms. Lightweight AI techniques offer practical solutions to this challenge. Hence, this paper provides a comprehensive overview of lightweight AI techniques for UAV-mounted RIS systems, including Reinforcement Learning (RL), meta-learning, Federated Learning (FL), Multi-Armed Bandits (MAB), and energy-aware optimization. We present a detailed taxonomy and comparative analysis of existing work, highlight computational-energy trade-offs, and identify open research challenges for scalable, energy-efficient airborne intelligent surfaces. Furthermore, we present a case study demonstrating the effect of MAB schemes on throughput and energy efficiency in UAV-mounted RIS.

0.25

Max-Min Throughput in RSMA-Aided SWIPT for Sustainable Internet of Underground Things: A Deep Reinforcement Learning Approach

eess.SY cs.LGeess.SP ·2026-11-15 ·Kaiqiang Lin, Mohamed-Slim Alouini
similarity 0.66
Abstract · 15 pages, 7 figures, and 3 tables

Unmanned aerial vehicle (UAV)-enabled wireless-powered underground sensor networks (WPUSNs) allow underground devices (UDs) to harvest radio-frequency energy from a UAV via wireless energy transfer (WET) techniques, enabling sustainable Internet of underground things (IoUT) applications. This paper integrates rate-splitting multiple access (RSMA)-aided simultaneous wireless information and power transfer (SWIPT) into UAV-enabled WPUSNs to deliver information and energy to UDs. Within this framework, we investigate a max-min fairness problem that jointly optimizes the UAV transmit power, power allocation and message splitting, power-splitting ratios, and time allocation, aiming to maximize the minimum uplink throughput across UDs subject to per-UD downlink throughput requirements and a UAV energy-consumption constraint. To address this non-convex problem under dynamic underground environments, we develop an off-policy deep reinforcement learning approach based on the primal-dual truncated quantile critics (PD-TQC), where a TQC-based reward critic evaluates the fairness objective, cost critics estimate constraint costs, and an actor learns the resource-control policy with adaptive dual-variable updates. Simulations using in-situ soil volumetric water content measurements from a center-pivot irrigation farm demonstrate that PD-TQC achieves the highest annual average minimum uplink throughput compared with four benchmarks while satisfying the per-frame constraints in over 96% of the evaluated frames, and that this throughput advantage is preserved under different Rician factors, numbers of UDs, and downlink throughput thresholds.

0.30

Two-Timescale Trajectory Control, Task Reassignment and Resource Scheduling for Multi-UAV Cooperative Mobile Edge Computing

eess.SY ·2026-11-07 ·Bin Li, Shiyu Sun, Lei Liu, Zesong Fei
top venue similarity 0.65
Abstract · 16 pages, 10 figures

Uncrewed aerial vehicle (UAV)-assisted mobile edge computing (MEC) is a promising technology, leveraging the advantages of UAVs including flexible deployment, wide coverage, and reliable wireless communication. In this paper, we study the task reassignment and offloading problem in a multi-UAV cooperative operating environment, and consider the real occurrence that the computing tasks arrive more frequently than UAVs make flight decisions. In addition, we also take into account the position offset of the UAV during flight and the unavailability of the UAV due to external interference. We propose a two-timescale optimization framework, where the trajectory of the UAV is carried out on a large timescale, while the task reassignment and computing resource allocation are jointly optimized on a small timescale. The core objective is to maximize the long-term average system gain, which takes both energy consumption and task delay into account. To address the UAV trajectory optimization problem under large timescale constraints, we propose a constrained multi-agent deep reinforcement learning algorithm. For decision-making within small timescale, to jointly optimize task offloading decisions and computing resource allocation, we integrate convex optimization with a heuristic algorithm. Finally, we conduct extensive simulations to evaluate the proposed algorithms against the baseline, achieving a 24% reduction in latency and a 28% reduction in total system cost.

0.33