CS Paper Radar

Reducing Cold-Start Latency on the WebAssembly Edge via Locality and Network-Aware Scheduling

Michiel Van Kenhove, Emiel Sucaet, José Santos, Filip De Turck, Bruno Volckaert, Merlijn Sebrechts

cs.NI cs.SE ·submitted 2026-11-09 16:02 UTC · v1
published

Abstract

Edge computing places computation closer to users and data sources, but introduces heterogeneous hardware, limited resources, and variable network conditions. WebAssembly (Wasm) is increasingly considered for these environments because modules are compact, portable, and quick to start. However, cold-start latency remains a challenge: when a worker node does not yet hold the required artifact, it must fetch it before execution, adding transfer overhead that directly increases the end-to-end (E2E) latency, especially for large artifacts or slow network links. At the same time, the geographical distance between the requester and the assigned worker independently contributes to latency through network routing overhead. Together, artifact locality and network proximity do not always point to the same placement decision, and naive scheduling strategies fail to account for either. This work extends a Wasm-native edge orchestrator with state-aware scheduling strategies that jointly optimize for both objectives. Evaluation on a Raspberry Pi testbed with emulated regional network latency shows that a hybrid scheduler combining cache-state and network-proximity signals reduces mean E2E latency by up to 40.8% compared to a default round-robin baseline. Artifact locality-aware placement is most effective when artifacts are large or repeatedly requested, while network-aware placement dominates when artifacts are small and routing latency has a larger relative effect, but combining both objectives consistently yields the lowest E2E latency across diverse edge workloads.

Comment
Accepted article in WebAssembly, Cloud and Edge: Shaping the Future of Computing (WACE '26)
DOI
10.1145/3861044.3862582

Closest papers

Nearest neighbours by meaning across the whole index.

Online Adaptive Computation Reuse in Collaborative Edge Computing: A Two-Timescale Approach

cs.NI ·2026-10-08 ·Xingqiu He, Chaoqun You, Peng Wang, Xiaoyu Liu, Tony Q. S. Quek
benchmark similarity 0.72
Abstract

Collaborative Edge Computing (CEC) is an efficient computing paradigm that enables neighboring edge servers to share computational resources with each other. Although CEC can enhance resource utilization, it still suffers from duplicate computations because nearby end-users often offload tasks with similar inputs. To improve system efficiency, the computation results of previously executed tasks can be cached and reused by subsequent tasks. However, time-varying task popularity and arrival rates require caching and scheduling decisions to adapt to demand changes, while frequent cache updates incur additional costs. To address this issue, this paper develops a two-timescale computation reuse algorithm for CEC networks. We formulate an optimization problem that jointly considers weighted response time and cache update cost, with result caching decisions determined at the frame level and workload scheduling, cache searching, and computational resource allocation adjusted at the slot level. Using recent workload observations, we construct a frame-level surrogate problem and decompose it into a caching subproblem and a scheduling subproblem. For the caching subproblem, we introduce marginal storage efficiency and incorporate cache update costs into a bisectionbased algorithm. Under bounded normalized marginal sensitivity, the algorithm achieves a near-optimal objective value for the single-BS caching subproblem when individual result sizes are small relative to the cache capacity and the relaxed solution is sufficiently accurate. For the scheduling subproblem, we utilize projected gradient descent and backtracking with warm starts across consecutive slots. Numerical results demonstrate consistent performance gains over benchmark schemes across diverse network and workload settings.

0.23

End-to-End Latency-Minimizing and Load-Balanced Request Scheduling for Edge LLM Inference in Agentic AI Services

cs.AI ·2026-09-15 ·Zhen Li, Jun Cai, Haoran Gao, An Li, Tan Li
similarity 0.71
Abstract

Large language model (LLM)-powered agentic AI services increasingly demand low-latency inference, motivating the deployment of LLMs across distributed edge servers. However, heterogeneous communication and computing capabilities, together with dynamically evolving inference states, make the edge server selection for each incoming request time-varying and tightly coupled across slots. In this paper, we investigate an online request scheduling framework for edge LLM inference that jointly minimizes long-term average end-to-end latency and regulates workload distribution across heterogeneous edge servers. Two main challenges arise in this context. First, conventional latency models cannot accurately capture the fine-grained dynamics of multi-stage LLM execution. Second, the latency consequence of a scheduling decision is observed only after request completion, making immediate decision evaluation difficult. To address these challenges, we develop a cross-slot inference model that captures transmission, prefill, iteration-level decoding, and key-value (KV) cache evolution for each diverse request, and characterize server workload through a KV cache memory-time consumption metric. We propose the LYREO approach that transforms the long-term load-balancing constraint via Lyapunov optimization and employs reward redistribution with sequencebased return prediction to convert delayed outcomes into timely learning signals for earlier decisions. Simulations under various configurations demonstrate that LYREO consistently achieves lower latency and more balanced load distribution than representative learning-based and heuristic baseline schemes.

0.17

Kirin: Cloud-native WebAssembly Service Orchestration

cs.DC ·2026-09-30 ·Joshua Bauer, Sebastian Werner, Maria C. Borges
published similarity 0.69
Abstract · Accepted at IC2E 2026

Modern cloud computing infrastructure relies heavily on virtualization to provide workload isolation and resource efficiency. While containers have become the dominant deployment primitive due to their fast orchestration compared to traditional virtual machines, they still introduce non-trivial overhead. WebAssembly (Wasm) has emerged as an alternative isolation technology, offering a lightweight execution model. Because of compatibility limitations, WebAssembly has mainly been applied to Function-as-a-Service (FaaS) and Edge Computing scenarios, where it has attracted growing research interest. However, recent advances in the WebAssembly ecosystem have significantly matured the technology, raising the question of whether it can serve as a viable replacement for containers in cloud-native workloads. In this paper, we present Kirin, an orchestrator for cloud-native WebAssembly services. It supports core orchestration responsibilities, including resource scheduling, lifecycle management, scaling, and request routing. We evaluate Kirin against container-based deployments for several service workloads and demonstrate promising results, particularly in service lifecycle management. Simultaneously, we confirm known limitations in CPU-intensive workloads. However, for typical service workloads, Kirin demonstrates competitive performance, suggesting that WebAssembly is a viable candidate for broader adoption as a cloud-native deployment technology.

0.17

Efficient KV Cache Migration for Geo-Distributed LLM Inference in Collaborative Edge Computing

cs.DC ·2026-10-15 ·Mingjin Zhang, Jiannong Cao, Tao Wu, Xiangchun Chen, Ne Wang
top venue similarity 0.67
Abstract · Accepted by ICDCS 2026

Large Language Models (LLMs) have shown exceptional capabilities but rely heavily on centralized cloud GPU clusters, leading to high bandwidth costs, latency, and privacy issues. Edge computing offers a solution by deploying LLMs closer to users on edge servers, yet single edge servers often cannot meet LLMs' heavy computational demands. Collaborative Edge Computing (CEC) addresses this challenge by integrating resources of multiple geo-distributed heterogeneous edge servers, which jointly host small model shards and associated Key-Value (KV) cache for distributed inference. However, edge servers may disconnect or easily become overloaded, making it essential to migrate stateful KV cache to other servers for ensuring continuous service. Existing cloud-based KV cache management methods, such as direct cross-GPUs transmission and recomputation from input tokens are ill-suited for bandwidth-constrained and resource-limited edge environments. In this work, we propose EdgeFlow, a novel hybrid KV cache migration scheme that integrates direct KV cache transmission with activation-based recomputation, leveraging parallel computation and communication to minimize migration time. On this basis, we formulate a multi-node KV migration optimization problem and design a 2-approximation algorithm based on improved linear programming relaxation and bipartite graph techniques. Extensive experiments on a real-world distributed LLM inference platform demonstrate that EdgeFlow reduces migration time by up to 45%, significantly enhancing the robustness and efficiency of distributed LLM inference in collaborative edge computing.

0.53

Ermes: a Stateful Serverless Platform for the Edge-to-Cloud Continuum

cs.DC ·2026-09-16 ·Matteo Cenzato, Dario d'Abate, Arianna Dragoni, Giacomo Orsenigo, Luca Tosetti +2 more
similarity 0.67
Abstract · Under revision at Future Generation Computer Systems (FGCS)

Function-as-a-Service (FaaS) is a widely adopted paradigm to simplify application deployment across the edge-to-cloud continuum. However, its stateless nature forces functions to retrieve their state from external, typically cloud-centric, data stores, reintroducing the very latency that edge computing aims to eliminate. This issue is further exacerbated by location-agnostic schedulers and rigid, one-size-fits-all consistency models that fail to capture the diverse requirements of edge applications. In this paper, we propose Ermes, a distributed platform that natively integrates state management into the FaaS paradigm, enabling the joint distribution of computational workloads and application state across the edge-to-cloud continuum. Ermes organizes application state into logical units, termed collections, and employs a distributed coordination algorithm that jointly maps collections and functions onto the available nodes seeking to minimize the latency perceived by the clients. In addition, it supports fine-grained replication and per-collection consistency levels, ranging from sequential to eventual consistency, leaving developers the choice of how to resolve the trade-off between consistency and performance. The experimental evaluation shows that Ermes quickly turns remote state accesses into local ones and sustains low latency as the workload grows, and as clients move across the edge.

0.14

DySCo: Dynamic Sharding for Collaborative Edge-Cloud LLM Inference with Depth-Synchronized Batching

cs.DC cs.AIcs.LGeess.SY ·2026-10-06 ·Jingpo Xu, Paul Joe Maliakel, Ivona Brandic, Shashikant Ilager
code/data similarity 0.66
Abstract · article under submission

Pervasive intelligent applications are increasingly deployed on mobile and Internet of Things (IoT) edge devices. Consequently, Large Language Models (LLMs) are increasingly used to support these applications. Yet, due to their high resource demands, LLMs are mostly deployed in the cloud. Layer-wise edge-cloud inference lets resource-constrained edge devices contribute computation to LLMs they cannot host in full. However, heterogeneous split points introduce two coupled inefficiencies. First, edge execution and communication create idle gaps between cloud invocations. Second, requests arriving at different model depths cannot be conventionally batched. We present DySCo, a collaborative runtime that keeps KV caches local and introduces dyForward, a model-aware layer-range executor that runs configurable contiguous layer ranges from resident model shards without reloading weights. For multi-edge serving settings, we introduce depth-synchronized batching (DSB), which advances heterogeneous requests to the deepest cut and batches their common suffix. Experiments across heterogeneous devices, two model families, and local and wide-area links show that idle gaps increase the latency of subsequent GPU forward calls even when waiting time is excluded, adding up to 25 ms of additional cloud-side suffix latency per decoding step in our measurements. At an average concurrency of eight, DSB improves throughput by 275% over FIFO, 48% over exact-match batching, and 79% over round-robin interleaving while reducing mean per-session latency. Together, these results show that requests with different edge-cloud splits can reuse resident cloud weights and share batched suffix computation. The artifact repository for this work is publicly available at: https://github.com/Large-scale-Sustainable-Computing-LSC/dysco-artifact

0.26

Delay and Throughput Analysis of Computation Offloading in Mobile Edge Computing: A Queueing Network Approach

cs.NI ·2026-09-05 ·Amirparsa Bahrami, Farid Ashtiani
similarity 0.65
Abstract

Mobile edge computing (MEC) enables mobile devices to offload computation to nearby edge servers and to the cloud in order to reduce end-to-end delay for different applications. In this paper, we study static computation offloading when each task consists of multiple dependent subtasks represented by a rooted directed tree. We develop a product-form queueing-network (PFQN) model with an approximation to capture the computation and communication dynamics of tree-structured task execution in a multi-tier MEC system. Based on this model, we derive closed-form expressions for server utilizations and waiting times, and then construct a recursive algorithm for evaluating the average delay of general tree-structured tasks. We formulate the static offloading design problem as the minimization of the rate-weighted average task delay over the routing probabilities, and solve it through a differentiable optimization framework based on softmax parameterization, log-sum-exp smoothing, and a stability barrier on server utilizations. Numerical results show that the proposed PFQN approximation provides accurate delay estimates and that the delay-optimized static policy consistently outperforms the considered baseline algorithms in terms of both average task delay and maximum stable throughput.

0.19

Task Offloading and Resource Allocation for Deterministic Service Provisioning in the IoT-Edge-Cloud Continuum

cs.NI ·2026-10-17 ·K. Aghababaiyan, B. Coll-Perales, J. Gozalvez
similarity 0.65
Abstract

Future networks will seamlessly integrate computing and communication resources within a unified ecosystem of IoT devices, edge nodes, and cloud servers, forming an IoT-edge-cloud continuum. Within such an ecosystem, tasks can be dynamically offloaded to leverage resources across the continuum and efficiently scale with demand. Task offloading has traditionally focused on minimizing task execution latency or energy consumption. In contrast, we propose a deterministic task offloading and resource allocation scheme for the joint management of communication and computing resources to support services with deterministic service-level requirements. The proposed scheme aims to maximize the number of tasks completed on time, rather than minimizing the latency of individual tasks. To do so, it exploits variable task latency requirements to distribute the workload across the continuum. It can also prioritize high-priority tasks under resource scarcity. Numerical evaluations demonstrate that a deterministic task offloading strategy scales more effectively and can better guarantee timely task completion by balancing resource utilization across the continuum.

0.22