CS Paper Radar

CCAaaS: An Orchestration Platform for High-Velocity Design, Development, and Deployment of Data Center Transports at Meta

Prankur Gupta, Balasubramanian Madhavan, Miao Xu, Prashanth Kannan, Jie Meng, Neil Spring, Rajiv Krishnamurthy, Srikanth Sundaresan, Martin Lau, Theophilus A. Benson

cs.NI ·submitted 2026-10-23 19:35 UTC · v1
published

Abstract

Today, data center networks run multiple congestion control algorithms (CCAs), each tailored to distinct traffic characteristics, from latency-sensitive intra-region RPCs to high-throughput cross-region AI workloads. The rapid evolution of AI training and inference further accelerates this trend by continually introducing new traffic characteristics that demand new CCAs. This paper introduces CCA-as-a-Service (CCAaaS), our eBPF-based ecosystem for the rapid design, deployment, and operation of production-grade CCAs at hyperscale. We present (1) a modular BPF-TCP framework and general lessons for designing modular CCAs; (2) a deployment and orchestration layer that provides disruption-free CCA updates, along with best practices for managing a zoo of CCAs at scale; and (3) a lightweight observability tool for CCAs and a blueprint for scaling CCA observability. CCAaaS manages over 97% of our server-to-server traffic using three active eBPF-based CCAs; two of the five CCAs we have built have since been deprecated. We show that our eBPF-centric approach to CCAs development delivers the same application-level performance (retransmissions, bandwidth, and tail latency) as their native in-kernel counterparts, and that our custom CCAs add negligible CPU and memory cost over their eBPF baselines. In terms of realized benefits, the CCA built with CCAaaS significantly improved network, and application metrics, for example, one of our CCAs reduced P99 latency for cross-region services by up to 20X.

Comment
19 pages, 11 figures, 9 tables. To appear in NSDI '27 (24th USENIX Symposium on Networked Systems Design and Implementation)

Closest papers

Nearest neighbours by meaning across the whole index.

PCM: Programmable Congestion Management for Terabit RDMA NICs

cs.NI ·2026-10-09 ·Mikhail Khalilov, Timo Schneider, Tommaso Bonato, Siyuan Shen, Marcin Chrapek +8 more
similarity 0.69
Abstract

Datacenter congestion control (CC) and load balancing (LB) evolve rapidly as workload demands change, but deployment depends on RDMA hardware and transport interfaces. Emerging programmable RDMA engines tie policy code to vendor APIs. Operators must therefore port each policy for every platform they deploy. We address this with PCM, a programming model that separates portable congestion management from platform integration on datacenter NICs. Developers write event-driven handlers that they can debug in simulation and reuse on real hardware. PCM exposes flow telemetry as signals and uses triggers to support per-event reactions and coarser responses to accumulated feedback. Vendors provide optimized backends that map signals and output controls to native NIC mechanisms with defined feedback and control semantics. We demonstrate PCM's expressivity by implementing eleven well-known datacenter CC and LB algorithms. We deploy PCM across four distinct datapaths: an Ultra Ethernet simulator, CPU-based UCCL, and on-NIC offloads on NVIDIA ConnectX-8 and Microsoft 1P. On CX-8, PCM-Swift achieves processing capacity equivalent to one reaction per three 4 KiB packets at 800 Gbit/s, using one-fifth of the NIC's hardware threads. In UCCL, it matches native Swift's 1.56 Tbit/s goodput with 64 KiB chunk-level control. PCM's design informed ongoing Ultra Ethernet work toward a portable congestion management interface.

0.30

DLB: Distributed Load Balancing at Scale for Generative AI Inference

cs.DC eess.SY ·2026-09-17 ·Santiago R. Balseiro, Bartek Wydrowski, Sameer Agarwal, David Applegate, Aaron Archer +9 more
similarity 0.67
Abstract

The reliance on scarce and expensive accelerators such as GPUs and TPUs in modern datacenters places unprecedented demands on backend infrastructure. For workloads characterized by heterogeneous service times and complex multi-stage processing, such as Generative AI, conventional load balancing techniques are often inadequate, relying heavily on costly overprovisioning to maintain service level objectives. This paper introduces DLB, the Distributed Load Balancer, a novel system designed to minimize end-to-end user latency for large-scale, heterogeneous workloads. DLB employs a scalable, distributed design with peer-to-peer probing to maintain real-time visibility into server capacity across large-scale, geographically distributed infrastructure. The system continuously learns latency models to estimate the latency impact of routing decisions, allowing it to effectively manage heterogeneous hardware and diverse model architectures. We provide a novel theoretical analysis of our routing algorithms that establishes their stability and global performance guarantees over time. We also evaluate DLB through extensive simulations, which show substantial gains compared to state-of-the-art load balancing algorithms. Finally, following a 22-month deployment of DLB at Google, where it facilitates large-scale Generative AI inference for thousands of different machine learning models and millions of requests per second, we detail the design choices and practical experiences gained from the system in production. Analysis of production migrations demonstrates that DLB yields statistically significant latency reductions compared to the legacy baseline, including a 17\% decrease in median latency and a 13\% decrease at the p95 tail.

0.22

Carbon-aware Resource Management for Latency-Sensitive Cloud Computing Environments: A Taxonomy and Future Directions

cs.DC ·2026-09-03 ·Tharindu B. Hewage, Shashikant Ilager, Maria Rodriguez Read, Rajkumar Buyya
survey similarity 0.63
Abstract

The proliferation of cloud-based latency-sensitive workloads, such as autonomous driving, real-time healthcare services, industrial automation, immersive interactive services, and intelligent AI-based applications requires infrastructures tuned to meet stringent latency constraints. Today, these workloads are reshaping the cloud from a general-purpose computing platform into a diverse ecosystem of workload-specific cloud environments. As demand for latency-sensitive workloads grows, cloud providers must scale their infrastructures accordingly, adversely increasing the carbon footprint and challenging net-zero goals. However, the strict performance requirements and rigid deployment patterns used to satisfy latency Service Level Objectives (SLOs) constrains conventional carbon optimization. Therefore, novel carbon-aware techniques are needed to exploit application-specific opportunities for reducing both operational and embodied carbon footprints while preserving latency guarantees. This survey examines carbon-aware resource management for latency-sensitive cloud computing environments. We introduce a holistic taxonomy that organizes existing techniques by carbon management objective, workload management mechanism, control architecture, system tier, latency requirements, and deployment stage. Then, we use this taxonomy to classify and critically analyze existing approaches, revealing the interactions and trade-offs between latency guarantees and operational and embodied carbon reduction. Finally, we identify limitations in the literature and present a structured research agenda for the design of sustainable latency-sensitive cloud systems.

0.09

High Speed Network Processing for Scientific Computing in the Open Cloud Testbed

cs.NI cs.DC ·2026-10-05 ·Miriam Leeser, Michael Zink, Suranga Handagala, Carlos Ruben Dell'Aquila, Verena Martinez Outschoorn +1 more
published similarity 0.63
Abstract · Paper accepted to the Twelfth International Workshop on Heterogeneous High-performance Reconfigurable Computing (H2RC'26) Held in conjunctio…

This paper presents an FPGA-based SmartNIC architecture designed to address critical network challenges in data-intensive scientific computing. As scientific workflows increasingly demand high-bandwidth data movement and real-time monitoring across wide-area networks, the Open Cloud Testbed (OCT) provides an ideal platform for investigating in-network processing solutions at line rate. Our approach focuses on implementing network visibility, optimization, and orchestration capabilities through FPGA acceleration. The proposed system leverages P4-programmable data-plane functions to support packet parsing, marking, telemetry extraction, traffic steering, and machine learning inference, all without requiring host CPU intervention or modifications to application servers. As a concrete example, our framework integrates the SciTags standard, which enables associating network traffic with scientific context through packet marking. We implement a high-performance network bridge that counts packets by their SciTag values, as identified in the IPv6 Flow Label. Experimental evaluation on AMD U280 FPGAs in OCT demonstrates the feasibility of this approach, showing imperceptible processing latency and line-rate packet handling, while accurately monitoring scientific data flows.

0.22

Kirin: Cloud-native WebAssembly Service Orchestration

cs.DC ·2026-09-30 ·Joshua Bauer, Sebastian Werner, Maria C. Borges
published similarity 0.62
Abstract · Accepted at IC2E 2026

Modern cloud computing infrastructure relies heavily on virtualization to provide workload isolation and resource efficiency. While containers have become the dominant deployment primitive due to their fast orchestration compared to traditional virtual machines, they still introduce non-trivial overhead. WebAssembly (Wasm) has emerged as an alternative isolation technology, offering a lightweight execution model. Because of compatibility limitations, WebAssembly has mainly been applied to Function-as-a-Service (FaaS) and Edge Computing scenarios, where it has attracted growing research interest. However, recent advances in the WebAssembly ecosystem have significantly matured the technology, raising the question of whether it can serve as a viable replacement for containers in cloud-native workloads. In this paper, we present Kirin, an orchestrator for cloud-native WebAssembly services. It supports core orchestration responsibilities, including resource scheduling, lifecycle management, scaling, and request routing. We evaluate Kirin against container-based deployments for several service workloads and demonstrate promising results, particularly in service lifecycle management. Simultaneously, we confirm known limitations in CPU-intensive workloads. However, for typical service workloads, Kirin demonstrates competitive performance, suggesting that WebAssembly is a viable candidate for broader adoption as a cloud-native deployment technology.

0.17

Software Design Patterns for Latency Measurements with eBPF in SDN Environments

cs.PF cs.NIcs.SE ·2026-11-03 ·Pedro Lucas M. S. Rosa, Bruno L. Dalmazo, Leticia S. Machado, Jéferson Campos Nobre
published similarity 0.62
Abstract · This manuscript has been accepted for presentation at the IEEE International Symposium on Computers and Communications (ISCC 2026)

This work proposes, implements, and evaluates a modular architecture based on software design patterns applied to latency measurement in Software-Defined Networks (SDN) using the Extended Berkeley Packet Filter (eBPF) and the Express Data Path (XDP). Unlike approaches that focus solely on measurement precision, this study examines how design patterns can structure observability systems in an extensible and maintainable way while preserving data-plane efficiency. The proposed architecture applies the Observer, Strategy, and Factory Method patterns to modularize the metric collection and analysis pipeline, allowing dynamic integration of policies and data sinks. The experimental environment was implemented with Mininet, Python agents, and eBPF/XDP programs, and was evaluated in terms of processing time, overhead, and evolution effort. The results show that the introduction of design patterns shows no clear evidence of measurable performance degradation, maintaining continuous measurements with a minimum of 5% overhead. The findings suggest that software engineering practices may be applied to the construction of eBPF-based monitoring and observability systems.

0.35

InferSlice: A Reference Architecture for SLO-driven LLM Inference over a Telco Wide-Area Network

cs.DC cs.AI ·2026-10-23 ·Juan Parra-Ullauri, Anthony Chiu, Hari Madhukumar, Zhaozhou Wu, Anderson Bravalheri +4 more
similarity 0.62
Abstract

LLM inference is increasingly deployed as a core service for enterprise and interactive applications, but sovereignty, compliance, latency and resource-locality requirements can limit the suitability of centralised cloud deployments. Telco operators offer geographically distributed compute and network infrastructure for addressing these constraints. However, existing LLM serving frameworks are designed for tightly coupled data centres and provide limited support for coordinating geographically distributed and heterogeneous resources. We present InferSlice, a reference architecture for Service-Level Objective (SLO)-driven LLM inference over a telco infrastructure, integrating network slicing, GPU partitioning, elastic serving and observability. Led by BT in collaboration with the University of Bristol and NVIDIA, and realised on the JOINER UK national-scale wide-area network (WAN), InferSlice is evaluated on service differentiation across tiers, elastic scale-out, client-edge bandwidth as a bottleneck, and the reach of tensor, pipeline, data parallelism, and prefill/decode disaggregation over the WAN. Our results show that, on a general-purpose telco network, tiers yield distinct latency-throughput regimes, and weight streaming shortens scale-out time. Replication is placement-insensitive but memory-bound; pipeline parallelism tolerates near- and mid-cross-site hops, while tensor parallelism and disaggregation fail beyond a single subnet hop. We share practical insights from operating SLO-driven inference on a telco network, lessons learned and open questions, and hope to encourage further research on distributed and heterogeneous LLM inference systems.

0.19

Tools-CC-Bench: a Benchmark Suite for Collective Communication with Compression in HPC and AI Workloads

cs.DC ·2026-09-08 ·Haozhe Fan, Wei Wang, Xingchen Liu, Man Liu, Xingjian Tian +10 more
publishedbenchmark similarity 0.62
Abstract · 12 pages, 10 figures, accepted by conference IISWC 2026

Distributed HPC and LLM workloads increasingly require efficient communication for scalability, yet growing data movement has become a major performance bottleneck. Communication compression can reduce this overhead and complement execution-level optimizations, but its benefits remain difficult to assess because existing benchmarks lack support for diverse backends, realistic datasets, application-specific accuracy metrics, and overlap-induced resource contention. We present CC-Bench, a lightweight, extensible, and application-oriented benchmark suite for evaluating communication compression under realistic execution conditions. CC-Bench uses declarative application-environment modeling to decouple profiling logic from communication libraries, datasets, and fidelity metrics, enabling portable cross-library evaluation. It further combines function-level interception and hardware counter monitoring to characterize per-phase latency, hardware utilization, numerical fidelity, and computation interference. With representative datasets from HPC and LLM workloads, CC-Bench evaluates three compression-enabled communication libraries on CPU and GPU clusters, revealing accuracy-performance trade-offs and bottlenecks to guide practical deployment and optimization.

0.54