CS Paper Radar

Movable-Element Multi-Functional RIS-Assisted Hybrid RSMA-NOMA: A Constrained Hybrid Reinforcement Learning Approach

Li-Hsiang Shen, Chi-Te Kuo

cs.AI eess.SP ·submitted 2026-10-26 01:46 UTC · v1
top venue

Abstract

In this work, we propose a movable-element (ME) multi-functional reconfigurable intelligent surface (MF-RIS) architecture, termed MM-RIS, which integrates element mobility with adaptive signal reflection, transmission, amplification, and energy harvesting (EH) to enhance communication flexibility and energy self-sustainability. A hybrid rate-splitting multiple access (RSMA) and non-orthogonal multiple access (NOMA) transmission scheme is further designed to improve spectral efficiency. We formulate an energy efficiency (EE) maximization problem by jointly optimizing transmit beamforming, power and rate allocation, MM-RIS configurations, panel deployment, and ME positions. To address the resulting non-convex mixed continuous-discrete optimization problem, we propose constrained hybrid deep reinforcement learning with federated transformer and parametrized sharing (CRAFTS). CRAFTS integrates multi-agent proximal policy optimization (MAPPO) and multi-agent deep Q-networks (MADQN) through bi-directional parametrized sharing to handle continuous and discrete actions, respectively. Moreover, a transformer-based architecture and federated learning (FL) are incorporated to enhance state representation and distributed knowledge sharing, whilst a Lagrangian-based adaptive penalty mechanism facilitates the constraint satisfaction. Simulation results have demonstrated that the proposed MM-RIS architecture and CRAFTS framework substantially improve EE and constraint satisfaction compared to the benchmarks in the open literature. Moreover, the results validate the benefits of spatially reconfigurable MM-RIS, hybrid RSMA-NOMA transmissions, and intelligent collaborative learning, highlighting the potential of the proposed framework for energy-efficient and adaptive next-generation wireless networks.

Comment
Accepted by IEEE TCOM

Closest papers

Nearest neighbours by meaning across the whole index.

Deep Reinforcement Learning for Optimization of STAR-RIS Phase and Energy Splitting Coefficients in OTFS-NOMA Framework

cs.IT eess.SY ·2026-09-03 ·Rais. J. Gachaba, Manobendu Sarker, Anirban Bhowal
published similarity 0.69
Abstract · Accepted in IEEE ANTS 2026

This paper considers a downlink communication framework comprising a simultaneously transmitting and reflecting reconfigurable intelligent surface (STAR-RIS)-aided by orthogonal time frequency space (OTFS) and non-orthogonal multiple access (NOMA) technologies. Further, delay-Doppler mobility in such frameworks renders classical alternating optimization impractical for per-coherence interval reconfiguration. To mitigate such issues, the STAR-RIS phase-shift and energy-splitting design is formulated as a constrained, non-convex sum-rate maximization problem with closed-form maximum ratio transmission beamforming and fixed NOMA power allocation. To circumvent the per-interval re-optimization burden, a deep reinforcement learning (DRL) approach is adopted that maps observed channel realizations to STAR-RIS configurations through a single forward pass. Specifically, Beta-Space Soft Actor-Critic (SAC-BSE), a maximum entropy DRL agent, is proposed. Simulation results, with two NOMA-multiplexed users on each STAR-RIS branch, confirm rapid convergence, limit the sum-rate degradation to roughly 10\% across a 128-fold user-speed range, and yield consistent gains over OTFS-only, NOMA-only, STAR-RIS-only, fixed-split, and mode-switching baselines as transmit power and the number of STAR-RIS elements increase.

0.27

Cooperative Dueling DQN SAC Learning for Energy Efficiency in Dynamic OWC Networks

eess.SY ·2026-10-07 ·Walter Zibusiso Ncube, Ahmad Adnan Qidan, Taisir El-Gorashi, Jaafar M. H. Elmirghani
hot direction similarity 0.65
Abstract

Growing wireless traffic is increasing pressure on the congested radio-frequency spectrum. Optical wireless communication (OWC) provides a complementary solution by using the abundant unlicensed optical spectrum. However, indoor OWC networks are dynamic: users move, enter or leave the network, and subscribe to different services. Poorly coordinated resource allocation can consequently waste subcarriers, require excessive transmission power and cause frequent AP reassignments. Energy efficiency (EE), defined as the total delivered data rate divided by the total network power consumption, therefore requires the serving AP, number of allocated subcarriers, and transmission power to be jointly adapted while maintaining QoS. Optimising these in a dynamic time series, multi-service OWC environment produces a complex sequential EE optimisation problem. To address this problem, this work proposes Dual-Agent Resource Allocation using Deep Reinforcement Learning (DARA-DRL). DARA-DRL combines a branching duelling deep Q-network for association and subcarrier allocation with a conditional soft actor-critic agent for continuous power control. The agents are coupled through a common reward and a cooperative value update that evaluates each discrete allocation together with its corresponding power decision. Simulation results show that DARA-DRL remains within 5\% of the optimal solution and, compared with state-of-the-art benchmarks, it improves EE by 28.8\% and QoS satisfaction by 10.5\%, while reducing online decision time by 14.5\%. Results demonstrate that agent specialisation simplifies mixed-action learning, and cooperation outperforms independently trained agents.

0.29

Max-Min Throughput in RSMA-Aided SWIPT for Sustainable Internet of Underground Things: A Deep Reinforcement Learning Approach

eess.SY cs.LGeess.SP ·2026-11-15 ·Kaiqiang Lin, Mohamed-Slim Alouini
similarity 0.65
Abstract · 15 pages, 7 figures, and 3 tables

Unmanned aerial vehicle (UAV)-enabled wireless-powered underground sensor networks (WPUSNs) allow underground devices (UDs) to harvest radio-frequency energy from a UAV via wireless energy transfer (WET) techniques, enabling sustainable Internet of underground things (IoUT) applications. This paper integrates rate-splitting multiple access (RSMA)-aided simultaneous wireless information and power transfer (SWIPT) into UAV-enabled WPUSNs to deliver information and energy to UDs. Within this framework, we investigate a max-min fairness problem that jointly optimizes the UAV transmit power, power allocation and message splitting, power-splitting ratios, and time allocation, aiming to maximize the minimum uplink throughput across UDs subject to per-UD downlink throughput requirements and a UAV energy-consumption constraint. To address this non-convex problem under dynamic underground environments, we develop an off-policy deep reinforcement learning approach based on the primal-dual truncated quantile critics (PD-TQC), where a TQC-based reward critic evaluates the fairness objective, cost critics estimate constraint costs, and an actor learns the resource-control policy with adaptive dual-variable updates. Simulations using in-situ soil volumetric water content measurements from a center-pivot irrigation farm demonstrate that PD-TQC achieves the highest annual average minimum uplink throughput compared with four benchmarks while satisfying the per-frame constraints in over 96% of the evaluated frames, and that this throughput advantage is preserved under different Rician factors, numbers of UDs, and downlink throughput thresholds.

0.31

WiSDoM: Wireless Sparse Decision Transformer with Mixture-of-Experts for Multi-Task Mobile Network Optimization

cs.NI cs.AIcs.LG ·2026-08-31 ·Fatih Temiz, Shavbo Salehi, Melike Erol-Kantarci
similarity 0.63
Abstract · 13 pages, 11 figures, submitted to IEEE for possible publication

Emerging 6G wireless networks are expected to operate across diverse deployment scenarios, where variations in network topology, user mobility, traffic demand, and radio conditions challenge the scalability of conventional radio resource management (RRM). While offline reinforcement learning (RL) methods have demonstrated strong decision-making capabilities, learning a single policy that performs consistently across heterogeneous wireless environments remains difficult due to conflicting optimization objectives and limited model specialization. These challenges become particularly pronounced in coordinated multipoint (CoMP) transmission, where selecting the optimal serving-cell combination requires sequential decision-making under evolving network conditions. This paper presents the Wireless Sparse Decision Transformer with Mixture of Experts (WiSDoM), a sparse multi-task offline RL framework for adaptive multi-cell selection. WiSDoM combines Decision Transformers (DTs) with a Mixture-of-Experts (MoE) architecture that dynamically activates specialized experts according to task characteristics. This MoE mechanism improves model capacity without proportionally increasing inference cost, mitigates negative transfer, and enables expert specialization across tasks. WiSDoM is trained jointly on diverse network configurations spanning multiple base station and user equipment densities, mobility levels, and scheduler policies. Experimental results show that WiSDoM consistently outperforms heuristic methods, single-task models, and conventional multi-task DTs, improving quality of experience (QoE) by up to 55% while activating approximately one-third of the parameters of its dense counterpart during inference. Furthermore, WiSDoM exhibits strong task generalization and efficiently adapts to unseen wireless scenarios through few-shot prompting without retraining or fine-tuning.

0.34

GTAC: A Graph-Transformer Actor-Critic for Carrier-Aware MU-MIMO User Grouping

eess.SP ·2026-11-02 ·Yan Huang, Charles Santhosam Lourdu Raja, Rajesh Gadiyar
similarity 0.62
Abstract · Submitted to IEEE ICC 2027

Achieving higher spectral efficiency (SE) in 5G and future 6G networks requires efficient massive multi-user multiple-input multiple-output (MU-MIMO) grouping under coupled physical-layer constraints and strict scheduling deadlines. Motivated by industry efforts in AI-native RAN, we propose Graph-Transformer Actor-Critic (GTAC), a deep reinforcement learning (DRL) architecture for carrier-aware MU-MIMO user grouping. The objective is to maximize the long-term proportional-fair (PF) utility of the cell subject to constraints from the practical stack protocol. GTAC comprises a shared actor--critic encoder with spatial-edge-biased attention mechanism, a pointer decoder actor that generates grouping decisions, and a multi-stage transformer critic that estimates the carrier-wide state-value during training. Training begins by cloning a teacher policy that cannot meet real-time deployment deadlines, followed by critic warmup and two-stage reinforcement learning using proximal policy optimization (PPO), first on a narrow, power-matched carrier and then on the full carrier with a Kullback--Leibler (KL) policy anchor. After two-stage PPO, the best GTAC policy outperforms the non-real-time teacher by 11.6% in carrier SE, while exceeding its PF utility and preserving the cell-edge throughput. A CUDA implementation of GTAC achieves a 69.6 us inference latency on an RTX PRO 4500 GPU, below the targeted 100 us inference budget in practical systems.

0.17

Multi-Agent Reinforcement Learning for Movable Antenna-aided Cell-Free Massive MIMO Systems

cs.IT ·2026-10-06 ·Bokai Xu, Jiayi Zhang, Shuaifei Chen, Ziheng Liu, Huahua Xiao +2 more
published similarity 0.62
Abstract

The inherent non-convex minimum-separation constraints introduced by movable antennas present a formidable challenge to the joint optimization of antenna positions and transmission strategies, rendering conventional methods computationally infeasible, particularly in large-scale cell-free massive multiple-input multiple-output (MIMO). In this work, we propose the graph-based learning individual intrinsic reward heterogeneous-agent proximal policy optimization (GLIIR-HAPPO) algorithm, a novel heterogeneous multi-agent reinforcement learning (MARL) framework that fundamentally overcomes this impasse by systematically decomposing the original coupled optimization into coordinated subproblems. To ensure tractability, we embed the non-convex geometric constraints into a penalty-augmented reward structure and develop a specialized geometric solver that enables the positioning agents to efficiently navigate the high-dimensional action space. Specifically, we propose an architecture featuring a dynamic-interaction graph critic for adaptive cross-role coordination, together with role-conditioned federated distillation that synchronizes same-role policies through compact actor-output statistics. Beyond architectural design, we establish a rigorous theoretical analysis that derives monotonic performance improvement bounds and establishes convergence guarantees for the proposed bi-level optimization. Numerical simulations demonstrate that our framework yields significant sum-rate improvements over state-of-the-art MARL schemes. Moreover, the performance of our advanced architecture closely approaches its fully centralized counterpart, while drastically reducing communication overhead.

0.59

Pareto-Aware Hierarchical Reinforcement Learning for Online Resource Allocation in RIS-assisted Large-Scale IoT Systems

cs.NI ·2026-08-13 ·Wenhan Xu, Jiashuo Jiang, Danny H. K. Tsang
similarity 0.62
Abstract

With the rapid evolution of 5G and emerging 6G networks, reconfigurable intelligent surfaces (RIS) have become a critical technology for enhancing wireless communication scenarios. However, optimizing RIS-assisted multi-user systems typically introduces high-dimensional physical layer variables and non-convex Pareto-optimal rate sets, posing severe computational challenges for real-time applications. To address these limitations, this paper proposes a dimension-reduced, hierarchical reinforcement learning (RL) framework, termed Pareto-aware autoencoder-assisted RL (PAAERL), to optimize online resource allocation in RIS-assisted Internet of Things (IoT) networks. Our approach first substitutes high-dimensional continuous RIS beamforming variables with lower-dimensional weight vectors that strictly represent the Pareto-optimal frontier, theoretically avoiding geometric information loss across both convex and non-convex rate regions. To further mitigate the curse of dimensionality in dense networks, an autoencoder architecture is integrated to execute a secondary, data-driven compression phase, mapping the priority space into a highly condensed continuous latent action space. Extensive simulations conducted across practical communication scenarios, including multi-user mobile edge computing (MEC) networks, demonstrate that the proposed PAAERL framework drastically reduces offline training times, accelerates online policy convergence, and significantly decreases overall network costs compared to state-of-the-art benchmarks, underscoring its exceptional scalability and practical viability for next-generation intelligent IoT environments.

0.04

Energy-Neutral Coverage Optimization by Joint Deployment and Scheduling in Ambient IoT Devices with Directional Sensing

eess.SY cs.IT ·2026-08-27 ·David E. Ruíz-Guirola, Samuel Montejo-Sánchez, Richard Demo Souza, Onel L. A. López
similarity 0.61
Abstract

Ambient IoT (A-IoT) devices rely on energy harvesting and duty cycling to sustain operation, thereby fundamentally changing collaborative sensing compared with traditional always-ON sensor networks. In this paper, we study the joint deployment and sensing scheduling of A-IoT devices equipped with directional sensing. We explore four solution strategies: (i) a grid deployment with static duty cycling, (ii) a centralized policy-gradient reinforcement learning (RL) approach that begins with a grid deployment and learns energy-aware device relocation and duty-cycling policies, (iii) a mixed-integer linear programming (LP) approach that couples static deployment design with duty-cycle allocation, and (iv) a hybrid LP+RL that combines optimization-based initialization with learning-based refinement. Using representative A-IoT use cases, we evaluate coverage as a function of device density, field-of-view, and maximum feasible duty cycle, determined by harvested energy and device consumption. Numerical results indicate that the proposed LP+RL and RL policies consistently outperform both the grid baseline and the LP-based method, achieving up to 2x higher mean effective coverage in low and medium energy harvesting (EH) regimes. In contrast, the standalone LP method remains limited by its conservative static duty cycle allocation under tight EH constraints. Moreover, the structured initialization of the LP+RL method substantially accelerates convergence, reducing the total offline optimization time by up to 10x compared to the standalone RL.

0.26