CS Paper Radar

CLASP: Chained-Request-Aware Scaling and Operator Placement for Serverless Stream Processing

Tianyu Qi, Maria A. Rodriguez, Rajkumar Buyya

cs.DC ·submitted 2026-08-29 07:19 UTC · v1

Abstract

Stateful serverless (Function-as-a-Service) environments, whose workers host state servers, are increasingly used for stream processing. A stream application is a pipeline of operators, where each operator forwards intermediate data downstream through a chained request. As input rates fluctuate, the system should adjust operator parallelism and place instances across workers to sustain the incoming rate. Existing approaches do so without fully accounting for chained-request overhead, leading them to misestimate the required number of workers. Too few leave the cluster unable to keep up with the input rate, while too many route a larger fraction of chained requests across worker boundaries, increasing end-to-end latency. We propose CLASP, a scaling and scheduling strategy for stream processing in stateful serverless environments. At runtime, CLASP estimates execution cost and chained-request cost from observed metrics. Under a capacity model that covers the two costs, it adjusts operator parallelism and packs operators onto the fewest workers that can sustain the target input rate. Once a scaling decision is made, CLASP migrates each operator's state together with its instances, thereby minimizing execution pause time. Experiments show that CLASP improves throughput by up to 3.3x and reduces median end-to-end latency by up to 76% compared with state-of-the-art scaling strategies.

Closest papers

Nearest neighbours by meaning across the whole index.

Locality-Aware Scheduling for Stream Processing Applications in Serverless Environments

cs.DC ·2026-11-18 ·Tianyu Qi, Maria A. Rodriguez, Rajkumar Buyya
similarity 0.84
Abstract

Stateful Serverless (Function-as-a-Service) environment are increasingly used for stream processing applications. These applications are often structured as pipelines where data flows through a sequence of distinct processing operators. Many operators are stateful and require access to the processing state during execution. To address this, stateful serverless platforms introduce state servers on worker nodes, co-locating state with computation to dramatically reduce access latency. In serverless, these data pipelines are realized by chaining function invocations, passing the processed data from one function to the next. To reduce network overhead, state-of-the-art scheduling approaches employ a distributed worker-level scheduling model, routing locally generated chained requests directly to destination workers, bypassing centralized schedulers. However, these approaches always co-locate state and execution for the same operator in the same worker, which forces most invocations to the next operator to be transferred across workers, thereby increasing both latency and I/O overhead. As a result, this paper proposes LcSched, a novel distributed scheduling strategy tailored for stream processing applications in stateful serverless environments. LcSched seeks to maximise local execution of chained function invocations while ensuring that each execution accesses its state from the local worker. It also monitors workload at runtime to maintain workload balance between workers. Experimental results show that LcSched improves throughput by up to 3.6x and reduces end-to-end latency by up to 79% compared to the state-of-the-art distributed scheduling strategy for stateful serverless environments.

0.12

A Unified Queueing-Based Framework for Comparative Evaluation of Serverless Autoscaling Strategies under Bursty Workloads

cs.DC ·2026-10-26 ·Zahra Zafarzade, Ehsan Ataie
similarity 0.66
Abstract

Serverless computing platforms must preserve quality of service (QoS) while adapting to highly bursty and unpredictable workloads. To cope with workload fluctuations while satisfying QoS requirements and service-level agreements (SLAs), these platforms rely on automatic resource scaling. Commercial function-as-a-service (FaaS) platforms employ different autoscaling strategies to dynamically adjust the number of active function instances and the available processing capacity. In this work, three representative autoscaling strategies are modeled and compared: scale-per-request (SPR), concurrencylevel scaling (CLS), and metric-based scaling (MBS). To enable systematic evaluation independent of proprietary platform-specific implementations, we develop a unified modeling framework that combines analytical queueing network (QN) modeling with queueing Petri nets (QPN) implemented in Java Modeling Tools (JMT) to evaluate autoscaling behavior under two-phase hyper-exponential (H2) arrivals. Evaluation is conducted using four system-level metrics: throughput, mean response time, system population, and system power. The framework captures dynamic function instance (FI) activation, concurrency constraints, and queue buildup under saturation. Comparative analysis reveals a clear trade-off between peak performance and stability: SPR provides the lowest delay under moderate load but becomes fragile near capacity limits; CLS is governed by concurrency tokens and is highly sensitive to configuration under bursty conditions; whereas MBS provides the most stable QoS across the workload range by keeping utilization below a critical region and preventing explosive queue growth under critical load.

0.24

Stable Token-level Scheduling for Serverless LLM Inference

cs.AI ·2026-10-13 ·Hardi Ashishbhai Desai, Balaji N K, Sourav Kanti Addya
similarity 0.66
Abstract · 10 Pages, 7 figures, Conference

Serverless inference is emerging as an efficient deployment model for Large Language Models (LLMs) due to its ability to elastically scale resources, improve GPU utilization through resource sharing, and reduce infrastructure costs. However, existing approaches primarily optimize time to first token (TTFT) while overlooking inter-token latency (ITL), a key determinant of user experience during response generation. We observe that LLM inference comprises a computation-intensive prefill stage and a bandwidth-bound decode stage, where admitting new requests can disrupt ongoing decoding and increase latency. To address this, we propose Token Window Pacing (TWP), a closed-loop scheduling framework that leverages runtime feedback from ITL, queue depth, and workload characteristics to regulate request admission at the token level instead of the batch level. TWP dynamically adjusts a selfregulating token window and incorporates chunk-based prefill to minimize interference from incoming requests. Experiments on realistic serverless workloads show that TWP delivers stable TTFT, significantly reduces latency spikes, lowers P99 tail latency, minimizes ITL variation, and improves throughput by up to 45% over ServerlessLLM and 58% over conventional schedulers. These results demonstrate that feedback-driven tokenlevel scheduling enables stable, high-throughput serverless LLM inference without modifying the underlying model.

0.18

Priority-Aware Scheduler for Serverless Functions Provision and Load Balancing

cs.DC cs.SE ·2026-10-26 ·Cheryl Lee, Zishan Su, Zhouruixing Zhu, Tianyi Yang, Yuxin Su +2 more
similarity 0.66
Abstract

Serverless computing has emerged as a pivotal cloud deployment paradigm due to its efficient utilization of on-demand resources. However, the persistent cold start problem continues to hinder system performance. Current strategies often employ oversimplified preloading/unloading mechanisms that inadequately exploit invocation patterns, failing to optimize the critical trade-off between startup latency and resource efficiency. Furthermore, existing approaches disregard practical constraints of memory-limited multi-worker environments, exacerbating function eviction and load imbalance issues. To address these limitations, we present SPES-Ext, a rule-based priority-aware scheduler designed for memory-constrained distributed worker nodes. Our methodology stems from two key observations: (1) serverless systems exhibit concentrated invocation patterns enabling behavioral predictability, and (2) these patterns permit function categorization for granular instance management. This insight posits that characteristic-driven prioritization enables optimal scheduling rules, leading to our system design, which integrates three coordinated components: predictive function provisioning, eviction optimization, and dynamic load balancing. Experimental validation demonstrates the superiority of SPES-Ext over state-of-the-art solutions: 80% reduction in $75^{th}$-percentile cold starts, 83.92% decrease in memory waste, and 52.49% improvement in worker load variance.

0.24

Don't let your Memory defy you: Fragmentation-Aware Serverless Allocation with Elastic Memory Locality

cs.DC ·2026-09-22 ·Achilleas Tzenetopoulos, Dimosthenis Masouros, Sotirios Xydis, Francky Catthoor, Dimitrios Soudris
similarity 0.64
Abstract

Serverless platforms commonly rely on bundled, memory-centric configurations, where CPU capacity follows the specified memory size. Resource decoupling reduces this waste, but can create external fragmentation by producing diverse CPU-memory shapes that leave residual capacity stranded across nodes. Memory disaggregation and tiering can turn these resource holes into usable capacity by enabling elastic memory locality, where instances use different mixes of local and remote memory. In this paper, we present Memoryless, a fragmentation-aware resource manager that exposes memory locality as a serverless control-plane primitive. Memoryless jointly selects and places function variants that differ in their compute allocation and memory-locality ratio. This allows the scheduler to match instance shapes to fragmented node capacity while preserving SLOs. We implement Memoryless on top of Knative and evaluate it with trace-driven workloads. Compared to state-of-the-art serverless placement frameworks, Memoryless reduces node usage by up to 40% in steady state and 46% at peak, while keeping SLO violations within 3%. It also reduces CPU stranding by 44%, showing that elastic memory locality can convert fragmented node capacity into usable resources.

0.29

Age of Information in Queueing Systems with Merging Server Streams

cs.NI ·2026-10-07 ·Lucrezia Rossi, Leonardo Badia, Andrea Munari
similarity 0.64
Abstract

We present a novel analytical framework for evaluating the age of information (AoI) in multi-path queueing topologies where output streams from multiple servers converge into a single pipeline. This theoretical scenario finds immediate application in ultra-reliable low-latency networks and edge computing architectures, where multi-path routing and stream merging can be vital strategies for maintaining fresh status updates. Specifically, we investigate two setups: a split-merge network and a duplicate-merge network. The split-merge case has been addressed in prior work, but we show that existing studies are incorrect, and our approach provides instead an exact analytical formulation. Moreover, building on similar reasoning, we provide the first formal analysis of the previously unexplored duplicate-merge scenario. Finally, we show how to leverage our theoretical results to solve key system-level optimization problems, deriving the optimal randomized routing probabilities and traffic injection rates that minimize average AoI.

0.24

Characterizing Serverless Workloads: A Large-Scale Analysis of Function-as-a-Service Platforms

cs.DC cs.PF ·2026-11-03 ·Saman Akbari, Yassine Lazreg, Mohammadreza Malekabbasi, Manfred Hauswirth
code/data similarity 0.64
Abstract · Code available at: https://github.com/saman-akbari/characterizing-serverless-workloads

Serverless computing is a popular approach to application deployment without having to manage the underlying infrastructure. Its most common implementation is Function-as-a-Service (FaaS), in which applications are structured as fine-grained functions. FaaS providers face non-trivial tasks in the management of their platforms, including cold starts, auto-scaling, and scheduling. The design of optimized platforms requires an accurate understanding of realistic serverless workloads. However, existing research and platforms rely on insights from single, often outdated datasets. In this work, we conduct a large-scale, cross-platform comparative analysis of eight publicly available FaaS workload traces from major cloud providers (Alibaba, Azure, Huawei, IBM) and a scientific computing platform (Globus). We provide a comprehensive and up-to-date view of the serverless ecosystem by characterizing key dimensions such as invocation patterns, latency, cold start behavior, and resource utilization. Our analysis shows that although invocation patterns remain universal, execution times have shortened, shifting the performance bottleneck toward platform scheduling and dependency management. We also observe an architectural shift toward "Serverless 2.0" models, where concurrent, long-lived containers are increasingly utilized to bypass cold start penalties. Furthermore, we found resource over-provisioning and extreme skew in user request volumes. These findings provide actionable insights for designing more efficient FaaS platforms, informing improved approaches to targeted cold start mitigation, predictive resource management, and dependency-aware scheduling.

0.18

Ermes: a Stateful Serverless Platform for the Edge-to-Cloud Continuum

cs.DC ·2026-09-16 ·Matteo Cenzato, Dario d'Abate, Arianna Dragoni, Giacomo Orsenigo, Luca Tosetti +2 more
similarity 0.64
Abstract · Under revision at Future Generation Computer Systems (FGCS)

Function-as-a-Service (FaaS) is a widely adopted paradigm to simplify application deployment across the edge-to-cloud continuum. However, its stateless nature forces functions to retrieve their state from external, typically cloud-centric, data stores, reintroducing the very latency that edge computing aims to eliminate. This issue is further exacerbated by location-agnostic schedulers and rigid, one-size-fits-all consistency models that fail to capture the diverse requirements of edge applications. In this paper, we propose Ermes, a distributed platform that natively integrates state management into the FaaS paradigm, enabling the joint distribution of computational workloads and application state across the edge-to-cloud continuum. Ermes organizes application state into logical units, termed collections, and employs a distributed coordination algorithm that jointly maps collections and functions onto the available nodes seeking to minimize the latency perceived by the clients. In addition, it supports fine-grained replication and per-collection consistency levels, ranging from sequential to eventual consistency, leaving developers the choice of how to resolve the trade-off between consistency and performance. The experimental evaluation shows that Ermes quickly turns remote state accesses into local ones and sustains low latency as the workload grows, and as clients move across the edge.

0.14