CS Paper Radar

Fleet-Scale Pod Deployment with VPC-Native Networking in Managed Kubernetes

Sri Saran Balaji Vellore Rajakumar, Jayanth Varavani, Murat Parlakisik, Pavani Panakanti

cs.NI cs.DC ·submitted 2026-08-23 04:26 UTC · v1

Abstract

Managed Kubernetes pod deployment is often presented as a choice between VPC-native and overlay networking. Public EKS documentation describes prefix-mode capacity and configuration, while prior CNI studies primarily measure steady-state throughput and latency. This paper presents an operational measurement study of fleet-scale pod deployment across three VPC-native modes on Amazon EKS: individual secondary-IP allocation, ENI preallocation, and IPv4 prefix delegation, using Cilium and Calico as overlay baselines. We compare representative mode-specific warm-pool settings, so the results describe combined operating points. At 400 nodes, the tested three-address secondary-IP warm pool required 7,403 seconds to place 80,000 pods. With the tested ENI-preallocation and prefix-delegation settings, the same workload completed in approximately 495 seconds, moving address provisioning off the pod-creation critical path. In the reported 400-node runs, prefix delegation achieved approximately 162 pods/s, within the same observed range as the fastest overlay baseline and ENI-preallocation runs, while requiring fewer successful EC2 provisioning calls per pod than individual-IP allocation by allocating addresses in /28 blocks. Because each /28 contains 16 addresses and occupies one IPv4 slot, an interface with 30 slots carries 29 prefixes, representing up to 464 potential pod addresses, compared with approximately 30 IPv4 slots per ENI for individual-IP allocation when kubelet and CNI limits are configured accordingly. Prefix delegation also has lower documented network-address usage and retains direct VPC reachability and flow-log visibility but the trade-off is that /28 allocations require contiguous subnet space.

Closest papers

Nearest neighbours by meaning across the whole index.

ScaleBench: An Extensible and Reproducible Experimental Platform for Kubernetes Autoscaling Research

cs.DC ·2026-11-18 ·Ze Wang, Sajid Alam, Amjad Ullah
similarity 0.61
Abstract · Accepted for publication in the 1st Workshop, titled ContinuumOps: DevOps for the Edge-Cloud Continuum, which was co-located with the 24th I…

Autoscaling enables cloud-native applications to meet their SLOs by dynamically adjusting computational resources at runtime. Recent research has explored a broad range of advanced autoscaling approaches, including rule-based, optimisation-based, and machine learning-based techniques, to address the limitations of traditional resource management strategies in cloud-native environments. Despite significant advances, a considerable gap remains between academic innovations and their practical adoption in real-world environments. For instance, Kubernetes deployments commonly rely on the reactive, CPU-based Horizontal Pod Autoscaler, limiting the adoption of more advanced autoscaling approaches. Recent studies have investigated challenges such as autoscaler configuration and reproducible performance evaluation; however, the practical deployment, integration complexity, and operational overhead of research-based autoscalers in Kubernetes environments have received limited attention. To address this gap, we present ScaleBench, a native Kubernetes experimental platform to enable extensible and reproducible evaluation of autoscaling solutions. Unlike existing approaches, ScaleBench integrates autoscalers as native Kubernetes components, allowing researchers to assess not only scaling performance but also deployment characteristics and integration requirements within realistic cloud-native environments. We demonstrate the effectiveness of ScaleBench by evaluating multiple autoscaling approaches through a case study. The results highlight its capability to support systematic, repeatable, and practical assessment of Kubernetes autoscaling approaches.

0.23

A Controlled Experimental Evaluation of Configuration and Cache Architecture in LLM Serving Engines

cs.DC cs.LGcs.PF ·2026-11-04 ·Muhammad Faizan Anwar, Sina Keshvadi
benchmarkcode/data similarity 0.60
Abstract · 29 pages, 14 figures, 7 tables. Artifact: https://github.com/FaizanAnwar01/llm-serving-config-eval

Multi-agent LLM applications repeat most of their prompt tokens across requests, and serving engines reuse the cached state of shared prefixes. vLLM and SGLang reuse it at different granularities, and benchmarks of their default deployments report large performance gaps, usually credited to the cache. Such comparisons rarely control for other configuration differences. One of them, whether a long prompt is prefilled in chunks, is set by each engine's defaults from the hardware and the release, and it confounds the comparison. We built a reproducible testbed on Kubernetes and a seeded multi-agent workload generator, and compared the two engines under vendor defaults and with chunked prefill matched. SGLang's configuration is the same in both arms, which makes it an internal control across independently provisioned deployments. Under defaults, vLLM's median time to first token is up to 64x SGLang's. Matching one setting removes 78% of the capacity gap where that gap is largest and 30 to 67% of the latency gap. Near saturation the remaining latency gap reflects queue growth. A fluid queue model driven only by a 4.4% throughput shortfall reproduces it, and shows that such a gap scales with the length of the benchmark run. What survives matching is a tail-latency advantage of 2.4 to 6.4x at equal throughput, consistent with cache-aware ordering of waiting requests. We derive six recommendations for comparing serving engines.

0.32

InferScale: Kubernetes-Native Predictive Autoscaling for SLO-Aware AI Inference

cs.DC ·2026-10-25 ·Wen Huang
published similarity 0.60
Abstract · 8 pages. Published in IEEE CLOUD 2026

Kubernetes Horizontal Pod Autoscaler (HPA) is widely used for inference services, but reactive scaling is often too slow when pods take tens of seconds to become ready. The problem gets worse in shared clusters, where independent HPAs compete for the same resource budget and can hurt different tenants from run to run. We present InferScale, a Kubernetes-native autoscaler that combines inference-aware signals, short-horizon demand prediction, SLO-state control, fast-path spike handling, and tenant-priority headroom without replacing the default scheduler. We evaluate InferScale on Azure Kubernetes Service in two settings. On a real ResNet-50 ONNX workload with 21-second cold start, repeated ramp-burst runs improve SLO compliance from 47.8% +/- 1.5% under a stronger external-metric HPA to 65.3% +/- 9.0% under InferScale. In repeated controlled multi-tenant contention experiments, InferScale cuts Gold-tier SLO standard deviation from 16.1 to 1.2 percentage points and makes contention outcomes follow the priority policy more closely. Repeated single-service ablations show 6-8 percentage point degradation, with an exploratory priority ablation showing a 27-point drop under contention. The main takeaway is that, in startup-sensitive inference serving, the practical value of prediction lies less in precise forecasting than in moving the controller early enough to offset cold-start delay.

0.25

A Kubernetes-Native Request Router for Quality-Aware Inference Serving in the Computing Continuum

cs.DC ·2026-09-17 ·Ignjat Karanovic, Pantelis A. Frangoudis, Ivan Čilić, Ivana Podnar Žarko, Schahram Dustdar
similarity 0.60
Abstract

We introduce Adaptive Score-based Routing Balancer (ASRB), a dynamic, score-based request routing mechanism for Kubernetes-based service deployments over the computing continuum. ASRB jointly considers infrastructure-level information, response time measurements, and application-level quality indicators, with a particular focus on serving Machine Learning (ML) workloads. For these workloads, ASRB balances requests over service instances deployed in the continuum, following service provider-defined policies encoded as weighted combinations of QoS criteria to flexibly address latency-accuracy trade-offs. To drive routing decisions and swiftly adapt to changes in the operating environment, ASRB monitors a range of runtime metrics across multiple system layers. To deal with the associated monitoring overhead, particularly important for large-scale deployments, it selectively and adaptively controls monitoring intensity without sacrificing on routing quality. ASRB is implemented without requiring any modifications to Kubernetes, making it straightforward to deploy and operate in existing cluster environments. Our testbed experiments demonstrate the versatility of ASRB: When tuned for latency reduction, it achieves at least 10 ms lower mean response time compared with latency-oriented state-of-the-art routing mechanisms, while it achieves higher accuracy when this is prioritized through specific configurations, thus enabling flexible and operator-controllable trade-offs. At the same time, it attains reduced failure rates, higher responsiveness to changes in the operating environment, and up to ~70% less monitoring cost than relevant state-of-the-art solutions, at the potential expense of only a modest latency penalty in some configurations.

0.28

Pod-Deployability in Kubernetes with Inter-Pod Affinity Constraints is PSPACE-Complete

cs.CC cs.DC ·2026-08-20 ·Saverio Giallorenzo, Jacopo Mauro, Gianluigi Zavattaro
similarity 0.59
Abstract

Kubernetes is the de-facto platform for container orchestration. Its scheduler combines resource capacities with label-based affinity and anti-affinity rules, and the interaction of these features can make the eventual placement of a pod. In this paper, we study the pod-deployability problem: given an initial cluster, a pod type, and a designated node, does some legal sequence of pod deployments and deletions cover the target pair? We give three complexity results. First, when dynamic constraints contain no affinity (anti-affinity is allowed), pod-deployability is decidable in polynomial time. Second, required affinity together with required anti-affinity makes the problem PSPACE-complete. Third, required affinity alone is already enough for PSPACE-completeness on a single node with one scalar capacity. The lower bounds encode, respectively, 1-safe Petri-net coverability and bounded black pebbling. These results isolate two independent sources of state-space complexity in Kubernetes scheduling: logical exclusion and resource-bounded prerequisite management.

0.33

Optimizing API Gateway Placement in Multi-Cloud Kubernetes

cs.DC ·2026-08-27 ·Vinoth Punniyamoorthy, Murali Shankar Dulam, Aswathnarayan Muthukrishnan Kirubakaran, Akshay Deshpande, Nachiappan Chockalingam +3 more
theory similarity 0.58
Abstract

The use of API gateways within geographically distributed multi-cloud Kubernetes clusters poses a tradeoff between infrastructure cost, computational resources, and network latencies. We present an optimization formulation that addresses API gateway placement as a capacitated facility location problem that jointly determines which candidate clusters to activate, how many gateway replicas to deploy, and how regional traffic should be distributed across the selected clusters. The formulation imposes an upper bound on estimated client-to-cluster network round-trip latency, excluding gateway processing, queuing, and backendservice latency, and incorporates a utilization headroom factor for gateway replica capacity. We present both a mixed-integer linear programming (MILP) formulation and a constructive greedy heuristic that ranks candidates according to incremental cost, comprising cluster-activation and marginal replica costs, per unit of assignable capacity while explicitly accounting for already committed load. Both formulations are applied to deterministic, seed-controlled, geography-based synthetic instances. For each problem size, 30 instances are generated with random seeds to analyze their performance. The greedy algorithm achieves an optimality gap of 3.2% to 4.7% to the MILP optimal solution, with a maximum observed gap of 25.0% for one particular instance, and a speedup of approximately 660x to 3,490x for 3 to 12 candidate clusters. In a canonical 10-candidate, 10-demand region instance, MILP-optimal deployment saves 24.2% in terms of monthly cost compared to the full-replication baseline. On the other hand, selecting the single cheapest candidate yields savings of 24.8% compared to the MILP optimum but does not satisfy the latency requirement for 3 out of 10 demand regions.

0.20

Characterizing the eBPF-Based Data Plane for Multi-ClusterKubernetes: A Systematic Evaluation of Cilium Cluster Meshand KVStoreMesh

cs.NI ·2026-09-28 ·Simhadri Podala Narasimha
published similarity 0.56
Abstract · 10 pages,3 figures

Kubernetes deployments increasingly span multiple clusters for reasons of scale, fault isolation, regulatory boundaries, and heterogeneous hardware placement, including GPU-dense clusters dedicated to AI and HPC workloads. Cilium Cluster Mesh, built on extended Berkeley Packet Filter (eBPF) technology, has emerged as a widely deployed mechanism for connecting such clusters into a single logical network without a dedicated multi-cluster gateway. Despite its adoption, the academic literature contains no systematic, reproducible evaluation of the eBPF-based multi-cluster data plane itself: existing work either benchmarks eBPF against iptables within a single cluster, or applies eBPF to cross-cluster monitoring rather than forwarding and identity propagation. The most detailed scale data available - a report of a clustermesh-apiserver failing under load at approximately 45,000 nodes across 256 clusters - comes from an industry engineering blog rather than a peer-reviewed source. This paper proposes a methodology to close that gap. We define five research questions covering latency, throughput, control-plane limits, churn, and GPU/RDMA sensitivity, and describe a testbed and eBPF-level instrumentation design. Lacking a live multi-cluster testbed for this submission, we report results from a calibrated analytical simulation of control-plane architectures fit to the production data point. These results are presented as falsifiable predictions intended for validation once the testbed is complete. This paper provides the motivation, experimental design, and preliminary simulation results to serve as a template for a full empirical study.

0.35

Performance vs Portability in Heterogeneous HPC Environments: Why Pre-execution Benchmarking is Required

cs.DC physics.comp-ph ·2026-09-27 ·Mindaugas Macernis
publishedbenchmark similarity 0.54
Abstract · Main results presented at the EGI2026 Conference, September 21-25, 2026, Ghent, Belgium. 4 pages, 3 figures, 2 tables

Cloud computing and high-performance computing (HPC) typically follow different paradigms: cloud services are often orchestrated using Kubernetes, whereas HPC workloads are managed through batch schedulers such as Slurm. Growing demand for shared computational resources increases the need for interoperability between these environments. This study uses Podman as a user-accessible tool to benchmark quantum chemistry software distributed as precompiled executables. Containerized and on-system execution are compared across CPU-optimized builds and multiple compute nodes, without additional network optimizations. The results demonstrate that performance depends on software compilation, startup overhead, and execution configuration. Container preparation can dominate short-task workflows, while long-running calculations require monitoring of computational progress. These findings highlight the importance of workload-specific benchmarking and user-level monitoring when deploying scientific applications in containers on HPC systems.

0.22