CS Paper Radar

Max-Min Throughput in RSMA-Aided SWIPT for Sustainable Internet of Underground Things: A Deep Reinforcement Learning Approach

Kaiqiang Lin, Mohamed-Slim Alouini

eess.SY cs.LGeess.SP ·submitted 2026-11-15 08:08 UTC · v1

Abstract

Unmanned aerial vehicle (UAV)-enabled wireless-powered underground sensor networks (WPUSNs) allow underground devices (UDs) to harvest radio-frequency energy from a UAV via wireless energy transfer (WET) techniques, enabling sustainable Internet of underground things (IoUT) applications. This paper integrates rate-splitting multiple access (RSMA)-aided simultaneous wireless information and power transfer (SWIPT) into UAV-enabled WPUSNs to deliver information and energy to UDs. Within this framework, we investigate a max-min fairness problem that jointly optimizes the UAV transmit power, power allocation and message splitting, power-splitting ratios, and time allocation, aiming to maximize the minimum uplink throughput across UDs subject to per-UD downlink throughput requirements and a UAV energy-consumption constraint. To address this non-convex problem under dynamic underground environments, we develop an off-policy deep reinforcement learning approach based on the primal-dual truncated quantile critics (PD-TQC), where a TQC-based reward critic evaluates the fairness objective, cost critics estimate constraint costs, and an actor learns the resource-control policy with adaptive dual-variable updates. Simulations using in-situ soil volumetric water content measurements from a center-pivot irrigation farm demonstrate that PD-TQC achieves the highest annual average minimum uplink throughput compared with four benchmarks while satisfying the per-frame constraints in over 96% of the evaluated frames, and that this throughput advantage is preserved under different Rician factors, numbers of UDs, and downlink throughput thresholds.

Comment
15 pages, 7 figures, and 3 tables

Closest papers

Nearest neighbours by meaning across the whole index.

Efficient Joint Trajectory Optimization and Resource Allocation for Secure UAV Networks: A Deep Reinforcement Learning Approach

eess.SY cs.IT ·2026-10-27 ·Abdulmalik Alwarafy, Mohamed Hayajneh, Mustafa Al Juboori, Abdulaziz Almarzooqi, Tahnoon Almazrouei +2 more
similarity 0.70
Abstract · This paper has been submitted for possible publication as a journal paper and is currently under review

Unmanned Aerial Vehicles (UAVs) are increasingly deployed in wireless networks; however, ensuring secure communication and efficient energy usage remains challenging. This paper addresses the joint trajectory optimization and resource allocation problem in UAV networks in the presence of a tethered UAV flying center and a malicious ground eavesdropper. In the proposed model, UAVs are equipped with energy harvesting capabilities and can adaptively switch between transmitting, jamming, and relaying roles to deliver data to a tethered UAV center for optimized network performance. An optimization problem is formulated to maximize a utility function that balances secrecy rate, energy consumption, and interference. To address the high non-convexity of the problem and its real-time decision requirements, a Proximal Policy Optimization (PPO)-based Deep Reinforcement Learning (DRL) algorithm is proposed to solve the problem and learn the network dynamics. Extensive simulation results demonstrate that the proposed approach achieves superior secrecy performance, lower energy consumption, and reduced interference compared to conventional and DRL-based methods.

0.11

AoI-Driven Hierarchical Learning for Cooperative Resource Sharing in Multi-Operator UAV Networks

eess.SY ·2026-09-19 ·Atefeh Hajijamali Arani, Mahyar Shirvanimoghaddam, Abolfazl Mehbodniya, Halim Yanikomeroglu, Fumiyuki Adachi
similarity 0.66
Abstract

Uncrewed aerial vehicle (UAV)-assisted networks provide a versatile paradigm for on-demand connectivity. However, in multi-operator aerial networks (MOANs), the joint optimization of cooperative resource sharing and 3D trajectory control to maintain information freshness is a complex combinatorial problem, which can be shown to be NP-hard. To address this computational complexity, we propose an age of information (AoI)-driven hierarchical deep reinforcement learning (DRL) framework. Specifically, a Dueling Double Deep Q-Network (D3QN) architecture is deployed at both the operator and UAV decision layers to mitigate overestimation bias and enhance stability in high-dimensional state spaces. To improve system resilience, we introduce an AoI- and load-aware outage compensation mechanism that prioritizes users based on instantaneous transmission demands and temporal freshness. Furthermore, a normalized load exchange balance metric is incorporated to regulate cooperative behavior and ensure resource fairness across operators. Simulation results demonstrate that the proposed hierarchical D3QN significantly outperforms conventional DRL, non-cooperative, and cooperative benchmarks, reducing the average AoI by up to 56.1% under severe congestion while ensuring superior inter-operator fairness and outage mitigation.

0.11

Deep Reinforcement Learning for BD-RIS-Assisted Multi-UAV RSMA Communications

cs.IT cs.ET ·2026-11-02 ·Aamer Mohamed Huroon, Ali A. Nasir, Daniel B. da Costa
published similarity 0.66
Abstract · 6 pages, 5 figures, accepted for presentation at the IEEE GLOBECOM 2026 Workshops

Beyond-diagonal reconfigurable intelligent surfaces (BD-RISs) extend conventional diagonal RISs by allowing controllable coupling among reflecting elements. This flexibility is attractive for aerial networks, where unmanned aerial vehicles (UAVs) face rapidly varying channels and strict power limits. In this paper, we study a hybrid BD-RIS-assisted multi-UAV rate-splitting multiple access (RSMA) downlink, in which UAVs within favourable BD-RIS coverage use both direct and BD-RIS-aided links, whereas the remaining UAVs are served through direct links only. We formulate a network sum-rate maximization problem that jointly optimizes UAV trajectories, RSMA common-rate allocation, beamforming, BD-RIS scattering matrices, and UAV-BD-RIS association. The resulting mixed-integer non-convex problem is solved by using deep deterministic policy gradient (DDPG) algorithm, equipped with a constraint-preserving action layer that generates reciprocal and lossless BD-RIS responses in closed form and handles the binary association inside the environment transition. Simulation results, averaged over three training seeds under a physically consistent channel model with blocked direct links near the BD-RISs, show that the proposed design attains a sum rate comparable to SAC and higher than twin delayed deep deterministic policy gradient ( TD3), proximal policy optimization (PPO), and deep Q-network (DQN), exceeds per-slot alternating optimization with fixed trajectories at about 1/65 of its per-slot runtime, and benefits mainly from RSMA, while the additional BD-RIS gain is modest.

0.31

Hybrid PPO-Based Multi-Objective Optimization for UAV-mounted STAR-RIS with Energy Harvesting

eess.SY ·2026-10-20 ·Anas Hussin, Khaled M. Rabie, Mahmoud M. Salim, Ali H. Muqaibel, Louai A. Al-Awami
similarity 0.65
Abstract

This paper investigates a multi-objective optimiza tion framework for UAV-mounted reconfigurable intelligent sur face (STAR-RIS)-assisted Sixth Generation (6G)-enabled Internet of Things (IoT) systems. The considered framework jointly supports communication enhancement and radio-frequency (RF) energy harvesting (EH) through adaptive STAR-RIS transmission and reflection control. A weighted multi-objective optimization problem is formulated to jointly optimize the base station (BS) beamforming vectors, STAR-RIS transmission and reflection coefficients, RF power-splitting factor, and UAV trajectory under quality-of-service (QoS), power, and mobility constraints. Due to the strong coupling among the communications, EH, and UAV mobility variables, the resulting problem is highly non convex and difficult to solve using conventional optimization methods. To address this challenge, a hybrid proximal policy optimization (H-PPO) framework is proposed. The developed hybrid actor-critic architecture combines a categorical policy for discrete UAV movement decisions with Beta-distributed policies for continuous communication-control variables. In addition, a two-stage reward mechanism is introduced to enforce QoS feasibility while balancing system sum rate, energy efficiency, and harvested energy. Simulation results demonstrate that the proposed H-PPO framework achieves faster convergence and more stable training behavior than proximal policy optimization (PPO) and twin delayed deep deterministic policy gradient (TD3) baselines. In particular, H-PPO achieves approximately 66% and 60% faster convergence than PPO and TD3, respectively, while maintaining more stable training behavior. Furthermore, the proposed framework effectively balances communication performance and EH under different QoS requirements, STAR RIS sizes, transmit power levels, and user densities.

0.17

Movable-Element Multi-Functional RIS-Assisted Hybrid RSMA-NOMA: A Constrained Hybrid Reinforcement Learning Approach

cs.AI eess.SP ·2026-10-26 ·Li-Hsiang Shen, Chi-Te Kuo
top venue similarity 0.65
Abstract · Accepted by IEEE TCOM

In this work, we propose a movable-element (ME) multi-functional reconfigurable intelligent surface (MF-RIS) architecture, termed MM-RIS, which integrates element mobility with adaptive signal reflection, transmission, amplification, and energy harvesting (EH) to enhance communication flexibility and energy self-sustainability. A hybrid rate-splitting multiple access (RSMA) and non-orthogonal multiple access (NOMA) transmission scheme is further designed to improve spectral efficiency. We formulate an energy efficiency (EE) maximization problem by jointly optimizing transmit beamforming, power and rate allocation, MM-RIS configurations, panel deployment, and ME positions. To address the resulting non-convex mixed continuous-discrete optimization problem, we propose constrained hybrid deep reinforcement learning with federated transformer and parametrized sharing (CRAFTS). CRAFTS integrates multi-agent proximal policy optimization (MAPPO) and multi-agent deep Q-networks (MADQN) through bi-directional parametrized sharing to handle continuous and discrete actions, respectively. Moreover, a transformer-based architecture and federated learning (FL) are incorporated to enhance state representation and distributed knowledge sharing, whilst a Lagrangian-based adaptive penalty mechanism facilitates the constraint satisfaction. Simulation results have demonstrated that the proposed MM-RIS architecture and CRAFTS framework substantially improve EE and constraint satisfaction compared to the benchmarks in the open literature. Moreover, the results validate the benefits of spatially reconfigurable MM-RIS, hybrid RSMA-NOMA transmissions, and intelligent collaborative learning, highlighting the potential of the proposed framework for energy-efficient and adaptive next-generation wireless networks.

0.28

Heterogeneous Services in Multi-UAV Networks with No-Fly Zone and User Mobility

eess.SY ·2026-10-14 ·Bin Li, Senning Wan, Decheng Zhang, Yonglei Yao, Yu Xue
publishedbenchmark similarity 0.64
Abstract · 12 pages, 10 figures

In this paper, we investigate a wireless communication network supported by multiple uncrewed aerial vehicles (UAVs). To ensure reliable connectivity, UAVs are desired to approach the boundaries of designated no-fly zones (NFZs) without violating them when mobile users (MUs) enter NFZs. To support heterogeneous communication services, MUs are assigned different priority levels based on their individual communication types. Considering the practical constraints arising from user mobility, heterogeneous communication requirements, and NFZs, the objective is to maximize the weighted sum of MU communication rates, where weights correspond to MU priorities. This is achieved by jointly optimizing trajectory planning, UAV transmission power control, and the binary association factor between UAVs and MUs. To address this challenging optimization problem under dynamic constraints, we develop a soft actor-critic algorithm with prioritized experience replay, which learns an adaptive control policy. Numerical results show that the proposed algorithm outperforms benchmark schemes in terms of convergence speed and stability. Moreover, it effectively meets diverse communication requirements, satisfies UAV energy constraints, and ensures reliable operation under NFZ constraints.

0.21

Cooperative Dueling DQN SAC Learning for Energy Efficiency in Dynamic OWC Networks

eess.SY ·2026-10-07 ·Walter Zibusiso Ncube, Ahmad Adnan Qidan, Taisir El-Gorashi, Jaafar M. H. Elmirghani
hot direction similarity 0.63
Abstract

Growing wireless traffic is increasing pressure on the congested radio-frequency spectrum. Optical wireless communication (OWC) provides a complementary solution by using the abundant unlicensed optical spectrum. However, indoor OWC networks are dynamic: users move, enter or leave the network, and subscribe to different services. Poorly coordinated resource allocation can consequently waste subcarriers, require excessive transmission power and cause frequent AP reassignments. Energy efficiency (EE), defined as the total delivered data rate divided by the total network power consumption, therefore requires the serving AP, number of allocated subcarriers, and transmission power to be jointly adapted while maintaining QoS. Optimising these in a dynamic time series, multi-service OWC environment produces a complex sequential EE optimisation problem. To address this problem, this work proposes Dual-Agent Resource Allocation using Deep Reinforcement Learning (DARA-DRL). DARA-DRL combines a branching duelling deep Q-network for association and subcarrier allocation with a conditional soft actor-critic agent for continuous power control. The agents are coupled through a common reward and a cooperative value update that evaluates each discrete allocation together with its corresponding power decision. Simulation results show that DARA-DRL remains within 5\% of the optimal solution and, compared with state-of-the-art benchmarks, it improves EE by 28.8\% and QoS satisfaction by 10.5\%, while reducing online decision time by 14.5\%. Results demonstrate that agent specialisation simplifies mixed-action learning, and cooperation outperforms independently trained agents.

0.29

A Mobile Agent-Based Hierarchical Reinforcement Learning Framework for Energy-Balanced Data Collection and Wireless Charging in WSN

cs.NI ·2026-09-24 ·Ali Heidaripour, Nastooh Taheri Javan
published similarity 0.63
Abstract

Energy imbalance remains a key challenge in Wireless Sensor Networks (WSNs), as nodes near the base station deplete their energy faster due to heavy forwarding loads. While mobile agents (MAs) have been employed for either data collection or sensor charging, existing approaches lack adaptability and fail to integrate both functions under realistic hardware constraints. This paper introduces a unified mobile agent framework that performs both data collection and wireless charging sequentially under single-antenna limitations. The agent's decision-making is formulated as a two-layer Hierarchical Reinforcement Learning (HRL) problem, where the upper layer optimizes movement planning and the lower layer determines the appropriate service based on real-time network states. This hierarchical structure enables the agent to learn adaptive task scheduling policies without predefined rules. Extensive simulations demonstrate that the proposed method achieves up to 15% longer network lifetime and more balanced energy distribution compared with state-of-the-art mobile agent and deep RL approaches.

0.21