CS Paper Radar

PCM: Programmable Congestion Management for Terabit RDMA NICs

Mikhail Khalilov, Timo Schneider, Tommaso Bonato, Siyuan Shen, Marcin Chrapek, Yehor Kozyr, Alberto Merino, Rip Sohan, Surendra Anubolu, Mohan Kalkunte, Ahmad Ghalayini, Abdul Kabbani, Torsten Hoefler

cs.NI ·submitted 2026-10-09 07:22 UTC · v1

Abstract

Datacenter congestion control (CC) and load balancing (LB) evolve rapidly as workload demands change, but deployment depends on RDMA hardware and transport interfaces. Emerging programmable RDMA engines tie policy code to vendor APIs. Operators must therefore port each policy for every platform they deploy. We address this with PCM, a programming model that separates portable congestion management from platform integration on datacenter NICs. Developers write event-driven handlers that they can debug in simulation and reuse on real hardware. PCM exposes flow telemetry as signals and uses triggers to support per-event reactions and coarser responses to accumulated feedback. Vendors provide optimized backends that map signals and output controls to native NIC mechanisms with defined feedback and control semantics. We demonstrate PCM's expressivity by implementing eleven well-known datacenter CC and LB algorithms. We deploy PCM across four distinct datapaths: an Ultra Ethernet simulator, CPU-based UCCL, and on-NIC offloads on NVIDIA ConnectX-8 and Microsoft 1P. On CX-8, PCM-Swift achieves processing capacity equivalent to one reaction per three 4 KiB packets at 800 Gbit/s, using one-fifth of the NIC's hardware threads. In UCCL, it matches native Swift's 1.56 Tbit/s goodput with 64 KiB chunk-level control. PCM's design informed ongoing Ultra Ethernet work toward a portable congestion management interface.

Closest papers

Nearest neighbours by meaning across the whole index.

CCAaaS: An Orchestration Platform for High-Velocity Design, Development, and Deployment of Data Center Transports at Meta

cs.NI ·2026-10-23 ·Prankur Gupta, Balasubramanian Madhavan, Miao Xu, Prashanth Kannan, Jie Meng +5 more
published similarity 0.69
Abstract · 19 pages, 11 figures, 9 tables. To appear in NSDI '27 (24th USENIX Symposium on Networked Systems Design and Implementation)

Today, data center networks run multiple congestion control algorithms (CCAs), each tailored to distinct traffic characteristics, from latency-sensitive intra-region RPCs to high-throughput cross-region AI workloads. The rapid evolution of AI training and inference further accelerates this trend by continually introducing new traffic characteristics that demand new CCAs. This paper introduces CCA-as-a-Service (CCAaaS), our eBPF-based ecosystem for the rapid design, deployment, and operation of production-grade CCAs at hyperscale. We present (1) a modular BPF-TCP framework and general lessons for designing modular CCAs; (2) a deployment and orchestration layer that provides disruption-free CCA updates, along with best practices for managing a zoo of CCAs at scale; and (3) a lightweight observability tool for CCAs and a blueprint for scaling CCA observability. CCAaaS manages over 97% of our server-to-server traffic using three active eBPF-based CCAs; two of the five CCAs we have built have since been deprecated. We show that our eBPF-centric approach to CCAs development delivers the same application-level performance (retransmissions, bandwidth, and tail latency) as their native in-kernel counterparts, and that our custom CCAs add negligible CPU and memory cost over their eBPF baselines. In terms of realized benefits, the CCA built with CCAaaS significantly improved network, and application metrics, for example, one of our CCAs reduced P99 latency for cross-region services by up to 20X.

0.56

Deploying RDMA for Distributed Inference and Training At Scale

cs.LG cs.DCcs.NIeess.SY ·2026-10-08 ·Yibo Huang, Yi Cui, Yukai Huang, Huaping Zhou, Jiancong Tong +10 more
similarity 0.66
Abstract · 14 pages, 16 figures

RDMA can reduce communication overhead in distributed machine learning systems, but realizing these benefits in production requires more than replacing TCP. Variable-size messages, concurrent connections, bursty incasts, and service failures introduce challenges that hardware offload alone cannot address. This paper reports our experience deploying ByteExpress, a software communication layer over hardware RDMA, in ByteDance's parameter-server-based inference and training systems. We present three lessons from the deployment: efficient RDMA requires buffer management across application and transport layers; scalable communication requires coordinated management of connections, receive capacity, and outstanding data; and service availability requires recovery above the transport layer. ByteExpress combines an RDMA-aware allocator, zero-copy transfers, QP multiplexing, receive-credit flow control, a software send window, and RPC-assisted recovery. In production inference, ByteExpress reduces mean and P99 model parameter fetch latency by 35.7% and 43.7%, respectively, and total CPU usage by 10.69% compared with kernel TCP. In training, ByteExpress reduces P99 model parameter push latency by up to 74.0% compared with TCP, with no PFC pauses observed at runtime.

0.22

Completion-Path Credits: Multi-Resource Control for Scale-Up Fabrics

cs.NI ·2026-08-18 ·Fan Yang, Jiaqi Liu, Tao Jiang, Zhan Wang
similarity 0.66
Abstract · 10 pages, 5 figures

Scale-up fabrics connecting GPUs and AI accelerators carry tensor transfers together with remote reads, writes, atomics, and notifications over shared target-side receiver resources. Byte-denominated credits protect link buffers and streaming HBM traffic, but poorly represent small operations dominated by Atomic execution or response injection. This paper presents SemaCredit, a receiver controller that admits each remote-memory operation against a vector of target-resource demands and returns each component when its corresponding HBM, Atomic, or response stage completes. In a deterministic event simulator with multipath queues, eight HBM partitions, a serialized Atomic engine, and a response engine, SemaCredit matches a strong per-resource byte baseline on HBM-hotspot traffic while reducing small-operation P99 latency by 52.4% under Atomic contention and 10.2% under response incast. Application-shaped mixes show 57.7% and 14.5% P99 latency improvements for AllReduce-shaped and remote-read-shaped traffic while matching byte credits on HBM-dominated MoE traffic.

0.12

CMM-MP: A Novel Memory Pooling Architecture using Purpose-built Memory Device with CXL and Ethernet

cs.AR ·2026-10-15 ·Michael Wasef, Gongjin Sun, Edmund Au, Ranjitha Gopalakrishna, Elham Movahedian +13 more
similarity 0.65
Abstract · 12 pages. Submitted to IEEE Transactions on Computers. CMM-MP is also evaluated in "LLM KV Cache Storage using CXL Memory," IEEE Computer Ar…

CXL Memory Module - Memory Pooling (CMM-MP) introduces a novel memory pooling architecture to meet the unique requirements of emerging memory-intensive workloads. This industry-first, purpose-built memory-pooling device combines the benefits of Compute Express Link (CXL) with the scalability and cost advantages of Ethernet. Multiple CMM-MP devices together provide a flexible, composable, and disaggregated memory expansion solution. The CMM-MP enables 100s of TB capacity for GPU/CPU applications without modifications. The memory pool is exposed as a large-capacity logical aggregate memory, with all pool discovery, composition, and backend network datapath tasks fully offloaded to the device. CMM-MP supports both memory pooling and sharing usage modes. It provides an application latency of 2.7 us and up to 23 GB/s of memory throughput. Our evaluations demonstrate that CMM-MP significantly outperforms RDMA-based solutions in latency and throughput. We also evaluated real-world use cases of Valkey+YCSB and DiskANN using CMM-MP.

0.39

Building A CSFQ-Inspired Transport for Switched CXL Memory Pooling

cs.NI ·2026-08-22 ·Zerui Guo, Emily Shriver, Ming Liu
published similarity 0.64
Abstract

Emerging switched CXL memory pooling systems, albeit promising, suffer from significant performance interference due to the shared but performance-uncontrolled data path among concurrent memory streams between a host core and a remote DIMM. We systematically characterize a memory pooling appliance based on XConn's Apollo CXL switch and identify three issues: intra-host contention, in-fabric congestion, and unmanaged host-remote DIMM interaction. This paper presents a new transport layer, MemChannel, which provides the mchannel abstraction to manage end-to-end fabric bandwidth among competing memory flows and enable application-specific traffic for switched CXL memory pooling. Our key idea is to build a sender-driven, fabric-informed transport protocol, inspired by Core-Stateless Fair Queueing (CSFQ), that admits just the right amount of CXL requests to each mchannel based on the estimated core-to-CXL-DIMM bandwidth availability. To address CXL-induced idiosyncrasies, MemChannel introduces time-based rate control, host-side admission control, cross-host bookkeeping, new congestion signals, rate estimation based on the fluid model, and delay-based link-capacity adjustment. We build MemChannel from scratch and support unmodified applications. Evaluations over switched memory pooling demonstrate its effectiveness from performance-isolation, scalability, and multi-tenancy perspectives.

0.48

A Fast, Programmable GPU-to-CPU Invocation Layer for GPU-Initiated Communication

cs.DC cs.LG ·2026-11-18 ·Jiakun Yan, Shiksha Patel, David Sidler, Gengyu Rao, Brandon Potter +1 more
positioncode/data similarity 0.64
Abstract · 11 pages, 9 figures, 3 tables

Modern GPU workloads increasingly rely on device-initiated communication to let GPU kernels communicate directly without synchronizing with the host at kernel boundaries. Existing libraries commonly favor placing transport control on GPU threads, an approach known as GPUDirect Async (GDA), and treat reverse offload to host proxy threads as a slower fallback. We argue that reverse offload can outperform GDA and should be a first-class communication path. We further propose exposing reverse offload as a programming interface that lets users define workload-specific communication operations. We present KIWI, a programmable reverse-offload framework that achieves sub-microsecond one-way device-to-host latency and hundreds of millions of operations per second. We address practical challenges of scaling reverse offload, including contention on shared queues, polling overhead, and load imbalance across proxies. KIWI-P2P, a SHMEM-like library built on KIWI, outperforms state-of-the-art GDA implementations in both latency and bandwidth. More importantly, KIWI-EP redesigns low-latency mixture-of-experts dispatch and combine kernels around workload-defined operations, lowering expert-parallel communication latency by up to 2.3x over the best existing backend on AMD GPUs and time per output token in vLLM by up to 29%. The source code of KIWI is available at https://github.com/AMDResearch/kiwi.

0.48

GPU-Initiated Communication: Dissecting Down to the Bone

cs.DC cs.NIcs.PF ·2026-10-01 ·Javid Baydamirli, Ismayil Ismayilov, Kaan Oktay, Didem Unat
code/data similarity 0.63
Abstract · 15 pages, 11 figures, 16 tables. Code and data: https://github.com/ParCoreLab/Dissecting-GPU-Communication-Experiments

GPU-initiated communication lets GPU threads post RDMA operations directly to the NIC. It underpins NVSHMEM, NCCL GIN, and DeepEP, which serve the fine-grained, latency-critical communication of Mixture-of-Experts (MoE) models, yet its performance characteristics and optimizations remain scarcely documented beyond source code, and library comparisons fail to separate the costs of the hardware mechanism from those of the library around it. This paper dissects GPU-initiated communication at the GPU-NIC boundary. We first detail the GPU-side network path: queue placement, work-request construction, doorbell ordering, and completion semantics. We then introduce mini-gda and mini-proxy, minimal transports for the GPU and CPU-proxy submission paths, and measure them alongside NVSHMEM IBGDA, NCCL GIN, DeepEP, UCCL-EP, MSCCL++, and fabric-lib on NVIDIA H100, H200, B200, and GB200 platforms. A minimal GPU path issues an operation in 0.7 $μ$s and completes in 4.0 $μ$s; libraries add up to 4.6 $μ$s of issue time through queue management, memory ordering, and completion scope, and issue time scales with the SM clock. A tuned CPU proxy matches or beats the GPU path at idle, at the cost of a dedicated core whose operating state sets its latency and throughput. On either path, sharing a queue with bulk traffic raises latency by one to three orders of magnitude. Reaching the 260 M msg/s ceiling of our InfiniBand platform requires doorbell batching and queue parallelism, and both have resource costs: communication code can reduce GPU block residency even when unused, and all-to-all traffic loses 59% of its NIC message rate at about 3,000 active connections. The submission path alone therefore does not predict communication performance. Our experiment code and results are available at https://github.com/ParCoreLab/Dissecting-GPU-Communication-Experiments.

0.56

Toward Intelligent Networks via AI-Aware GPU-Native Packet Processing

cs.NI ·2026-10-08 ·Seyed Mohammad Mehdi Mirnajafizadeh, Yiwen Hu, Rhongho Jang
similarity 0.63
Abstract

The integration of inline Artificial Intelligence (AI) models into critical network infrastructure is fundamentally bottlenecked by the high latency and synchronization overhead of CPU-mediated packet processing. While legacy GPU offload and recent CPU-bypass frameworks attempt to bridge this gap, they remain trapped in proprietary ecosystems or still rely on the host CPU and coarse-grained batching to coordinate stateful telemetry and AI pipeline execution. In this paper, we propose AGP, a novel framework that promotes the GPU from a passive accelerator to a primary data-path controller, answering what changes architecturally when the GPU autonomously owns the complete packet-to-inference pipeline. By enabling GPU-native packet processing, contention-free in-GPU stateful aggregation, and a persistent mega-kernel for continuous AI inference, AGP removes the CPU from the critical path. Our evaluation demonstrates that native in-GPU packet processing sustains line-rate throughput while being over 6.1x more power-efficient. Furthermore, our integrated Intrusion Detection System (IDS) stress test eliminates the legacy CPU-GPU synchronization tax, reducing end-to-end latency by 7.9x (up to 35x p99) and accelerating whole-system inference throughput by 9.3x using only ~2% of the GPU's thread capacity.

0.24