CS Paper Radar

SLA-Aware Task Offloading in the IoT Edge-Cloud Continuum: Class-Weighted Deep Reinforcement Learning Against an Exact Optimum

Shilpa Golla, Tharun Kumar Korine Palli, Urmila Bandaru

cs.AI cs.LGcs.NI ·submitted 2026-11-07 10:26 UTC · v1
benchmark

Abstract

Deep reinforcement learning (RL) is often used to decide where Internet of Things (IoT) tasks with different latency limits run in the edge-cloud continuum, yet some methods give every missed task the same penalty and few compare their agents with an optimal policy. This benchmark study builds four offloading systems small enough to solve exactly by dynamic programming and measures reward and learner against the optimum. A bonus for a met limit makes each miss cost its penalty plus the bonus, which shifts the class weights of published per-class rewards. A class-blind penalty stays 65% to 118% above the optimal cost even with a perfect learner, while a corrected per-class penalty comes within 0.13% of the optimum. With a per-class reward, tuned proximal policy optimization (PPO) agents come within 9% to 11% of the optimum in the smallest system, and the corrected one beats a tuned reservation rule in 9 of 10 seeds. In larger systems the agents stay 41% to 76% above the optimum, and a tuned rule beats their mean. Training ten times longer does not close this gap, and a PPO-Lagrangian agent with reachable targets does no better than the per-class reward.

Closest papers

Nearest neighbours by meaning across the whole index.

Coordinated Offloading: Cooperative Multi-Agent Reinforcement Learning for the Edge-Cloud Continuum

eess.SP ·2026-11-19 ·Muhammad Rafid, Golshan Famitafreshi, Vasilis Avgerinos, Kostas Ramantas, Christos Verikoukis
published similarity 0.71
Abstract · 6 pages, 5 figures, 1 table. Accepted at IEEE GLOBECOM 2026

The increasing demand for intelligent, low-latency services in edge-cloud continuum systems poses new challenges for dynamic and efficient task offloading. We propose a Multi-Agent Reinforcement Learning (MARL) framework for distributed task offloading under partial observability, where each device offloads only a portion of its computation to a target edge server while executing the remaining portion locally (partial offloading). The problem is formulated as a Markov Decision Process (MDP), where each device independently learns an optimal policy under partially observable system states. To address the non-convexity, high-dimensional state space, and continuous action space, we develop a multi-agent Advantage-Weighted Actor-Critic (AWAC)-enhanced Proximal Policy Optimization (PPO)-based computation offloading algorithm. The proposed method combines the stable policy updates of PPO with an advantage-weighting mechanism inspired by AWAC to emphasize higher-quality actions during training. To encourage cooperation without inter-device communication, a shared global reward aligned with system cost is employed. Extensive simulations demonstrate rapid convergence and significant improvement in overall system cost compared with baselines, e.g., Deep Deterministic Policy Gradient (DDPG) and Deep Q-Network (DQN).

0.25

Deep Reinforcement Learning for HPC Job Scheduling: A Statistical Evaluation

cs.AI ·2026-10-14 ·Justin M. Cheney, Clement N. Nyirenda
published similarity 0.64
Abstract · 17 pages, 9 Tables, Accepted for Online Proceedings at SACAIR 2026 Conference

Deep Reinforcement Learning (DRL) offers learned job selection for High-Performance Computing (HPC), but performance against heuristics alone does not establish value beyond action masking. Six DRL treatments are compared: Proximal Policy Optimization (PPO), Advantage Actor-Critic (A2C), and Deep Q-Network (DQN), each with and without masking. Three heuristics and a seed-paired masked uniform random control provide baselines across ten seeds on two real Slurm (Simple Linux Utility for Resource Management) workload traces, one homogeneous and one heterogeneous, under a common non-parametric framework. All masked treatments complete evaluation, whereas unmasked treatments are truncated. On the homogeneous trace, no masked DRL treatment significantly improves average waiting time or average and maximum bounded slowdown over the control. On the heterogeneous trace, MaskablePPO improves all three metrics relative to the control. Its average waiting time is equivalent to that of the three heuristics within a 10% margin, and its maximum bounded slowdown is 22% lower than that of Last-Come, First-Served (LCFS). MaskablePPO has the leading average waiting-time rank among complete treatments on both traces, but Nemenyi comparisons detect no significant difference from MaskableA2C. Among masked variants, MaskablePPO has the lowest recorded mean training time, while MaskableDQN has the lowest mean decision latency. These findings distinguish completion from baseline-specific scheduling gains and support including a masked random control when assessing the value of learned selection.

0.28

FARM: Fundamental Agentic Reward Model For Multi-task Wireless Network Optimization

eess.SY ·2026-10-02 ·Feiran You, Changxu Ni, Haozhe Ma, Jun Li, Hongyang Du
similarity 0.63
Abstract

Future wireless networks require learning agents to adapt across heterogeneous channel conditions, traffic patterns, quality-of-service (QoS) requirements, objectives, and operational constraints. Reusing decision knowledge across such tasks is challenging because conventional multi-task and transfer reinforcement learning methods primarily share or transfer policies, coupling transferable knowledge with task-dependent action mappings. This paper proposes FARM (Fundamental Agentic Reward Model for Multi-task Wireless Network Optimization), a reward-space transfer framework that shifts cross-task knowledge reuse from policy space to trajectory-level decision evaluation. FARM introduces an Agentic Reward Model (ARM) that learns a task-conditioned reward prior from heterogeneous source-task trajectories and provides auxiliary guidance for task-specific policy optimization. In Stage I, ARM jointly models task conditions, temporal trajectory dependencies, and objective-dependent reward structures while each source task retains its own controller. In Stage II, the learned reward prior is frozen and reused to guide the adaptation of a target-specific controller for previously unseen tasks, without transferring source-task policies. Experiments on heterogeneous multi-access edge computing (MEC) tasks show that FARM achieves a mean late-stage gain of 29.8% over Single-task SAC on unseen Rate-Latency targets, compared with 16.1% for CRA Transfer, and reaches a 46.2% gain on the moderate-OOD FAR-M case. Further analysis shows that both Mamba and Transformer trajectory encoders support Reward-Space Transfer, while Mamba provides improved robustness as longer history dependencies are introduced.

0.18

Learning in Strategic Queuing with Geometric Recharge: From Bandits to Systems

cs.GT cs.LGcs.PF ·2026-10-13 ·Cristian Palma Foster, Eva Tardos
theory similarity 0.62
Abstract

We study packet routing in queuing systems where queues act as selfish, myopic agents that use no-regret learning algorithms to select servers with unknown service rates. To model the small buffers used in data centers, we assume that each server has a single-packet buffer. Even such a tiny buffer already captures the features of buffering: routers can exploit multiple servers while accepted packets remain in service, but packets can occupy servers for multiple rounds, complicating future service and learning. We first analyze a single queue routing packets to $m$ buffered servers with service rates $μ_1, \dots, μ_m$. This setting is equivalent to an instance of the recharging bandits problem with geometric recharge. We prove that any no-regret learner achieves a long-run average throughput of at least the Hamacher sum of the server rates, and that this bound is tight. Consequently, a single no-regret queue is stable if and only if its arrival rate is below the Hamacher sum of the server rates. Moreover, comparing against an optimal policy, we show that any no-regret learner is at least $0.77$-competitive, which is also tight. We then study multi-queue systems where $n$ queues with arrival rates $λ_1, \dots, λ_n$, each at most $λ_{\max}$, compete for service from $m$ buffered servers. We show that the smallest overprovisioning factor that guarantees the system to be stable under high-probability no-regret learning is $\frac{1}{1-λ_{\max}}$. Equivalently, in terms of the total arrival and service rates, the condition $\sum_{i=1}^n λ_i < (1-λ_{\max})\sum_{j=1}^m μ_j$ is sufficient, and sometimes necessary, for stability under no-regret learning. In particular, for $λ_{\max} = 1/2$, this improves the upper bound of Abel et al. (EC 2026) from $3$ to $2$, matching their lower bound and resolving their open question.

0.08

RAGAM-AI: A Hybrid Reinforcement Learning based Agentic approach powered by Generative AI for Edge Resource Management

cs.AI cs.LG ·2026-11-04 ·Shreya K. Chari, John S. Vardakas, Kostas Ramantas, Christos Verikoukis
similarity 0.62
Abstract

Resource management for distributed edge networks hosting different services has been handled efficiently using Reinforcement learning (RL). Reward generalization in complex workflows is still an open issue within the RL realm. Therefore, this work proposes RAGAM-AI, a hybrid framework that combines the capabilities of RL algorithms and Generative-AI (Gen-AI) for powering an Agentic AI workflow that optimizes resource allocation at the Multi-access Edge Computing (MEC) units. This framework further resolves the reward generalization issue by utilizing Large Language Model (LLM) for feedback. A Multi- Agent Soft-Actor Critic (MASAC) using MathΣtral as a reward oracle is implemented for managing resource allocations at the MEC. RAGAM-AI interprets system state information and evaluates multi-objective trade-offs guiding the resource orchestration loop over distributed edge server agents. The complexity of the framework is further brought down by incorporating surrogate reward model. The overall performance is evaluated against the single agent SAC, REINFORCE and random policies for benchmarking. On using RAGAM-AI, lower allocation deviations for CPU, memory and bandwidth resources are reported while minimizing the latency and data rate costs. A scalable AI as a feedback (AIAF) solution optimizing network resources with significant performance improvement is thus presented.

0.24

Learning Which Device to Call: Online Tool Grounding for LLM Agents in IoT

cs.AI ·2026-10-18 ·Yichen Liu, Xiaochong Jiang, Haoran Yu, Lifei Liu
similarity 0.61
Abstract

The Internet of Things (IoT) gives large language model (LLM) agents access to physical observations, but one requested capability is usually offered by several devices, and the agent's plan does not say which one to call. Existing work retrieves relevant tools and checks plan feasibility, without learning which eligible device returns a useful observation. Unlike previous works, we study online tool grounding: binding an already specified tool call to one physical endpoint under service cost and device participation constraints. We propose a framework that learns which physical device should execute each agent-issued call. It formulates selection as a contextual bandit problem and combines adaptive context partitions with hierarchical endpoint trees, so that execution feedback is reused across related requests and devices. Selection balances task success against service cost while respecting device capability, authorization, battery reserves, and availability. Gaussian reward aggregation provides an optional mechanism for protecting individual feedback values. Theoretical results establish a finite-horizon regret bound and differential privacy for released reward statistics under fixed, reward-independent assignments. In a simulated loading-dock workload, the proposed router achieves higher step success and better cost-adjusted service, at lower simulated device energy and latency, than nearest-device, advertised-quality, UCB1, and LinUCB selection.

0.23

Personalized Privacy-Aware MARL Offloading in the Edge-Cloud Continuum

eess.SP ·2026-11-19 ·Muhammad Rafid, Golshan Famitafreshi, Kostas Ramantas, Luis Alonso, Christos Verikoukis
published similarity 0.61
Abstract · 6 pages, 2 figures. Accepted for presentation at IEEE CloudNet 2026

Task offloading in End-Edge-Cloud computing enables resource-constrained User Devices (UDs) to execute computation-intensive applications with reduced latency and energy consumption. However, existing task offloading schemes generally overlook the heterogeneous privacy requirements of users, leading to inefficient privacy-aware offloading decisions. In this paper, we propose a personalized privacy-aware task offloading framework, where the privacy requirement of each task is quantified through a User-Centric Privacy (UCP) model that considers task sensitivity, user privacy preferences, and task urgency. The task offloading problem is formulated as a Markov Decision Process (MDP) to jointly optimize the offloading ratio and the target computing server while minimizing the long-term system cost under latency, energy, and privacy constraints. To efficiently solve the resulting high-dimensional continuous optimization problem, we develop an Advantage-Weighted Actor-Critic (AWAC)-enhanced Proximal Policy Optimization (PPO)-based Multi-Agent Reinforcement Learning (MARL) algorithm under the Centralized Training and Decentralized Execution (CTDE) paradigm. Simulation results demonstrate that the proposed framework achieves lower average system cost, lower energy consumption, and higher task success rate than representative baselines under diverse privacy requirements.

0.27

Packet Loss Analysis Under Doppler Effects and Reinforcement-Learning-Based Resource Allocation for Direct-to-Satellite LoRaWAN

eess.SP ·2026-10-16 ·Alireza Maleki, Ebrahim Bedeer, Robert Barton
similarity 0.59
Abstract

Long range wide area network (LoRaWAN) has a long communication range and low power consumption, making it a potential technology for direct-to-satellite (DtS) Internet of Things (IoT) connectivity. However, packet collisions and Doppler impairments brought on by satellite motion can greatly increase the packet loss ratio (PLR) in large-scale DtS-IoT networks. In this paper, we study resource allocation in DtSIoT LoRaWAN by first developing an analytical PLR model that simultaneously takes Doppler impairments and packet collisions in the temporal and frequency domains into account. In order to maximize resource allocation, the generated PLR is then used as the reward function for two reinforcement learning (RL) schemes: deep Q-network (DQN) and proximal policy optimization (PPO). Simulation results validate the analytical PLR model against Monte Carlo simulations and demonstrate significant improvements over random resource allocation. At a target PLR of 3*10^-2, while random allocation supports only 100 end devices (EDs), PPO and DQN support approximately 650 and 690 EDs, respectively, representing nearly a seven-fold increase. Furthermore, for 800 EDs, PPO and DQN increase the total number of transmitted packets by 106.7% and 88.0%, respectively, compared with random allocation.

0.22