CS Paper Radar

RingStitch: Demand-Aware Optical Stitching for Fragmented TPU Clusters

Huiru Ao, Fan Yang, Binglei Wang, Bo Liu, Jialong Li

cs.NI ·submitted 2026-09-29 14:02 UTC · v1
published

Abstract

Large-scale AI training clusters increasingly use optical circuit switching (OCS) to reconfigure rack-level interconnects and create elastic accelerator slices. In multi-tenant TPU-style clusters, however, small and medium jobs often leave partial free capacity stranded inside racks. Although the aggregate free capacity may be sufficient for a new job, it cannot be used by local placement or coarse full-rack stitching. This paper presents RingStitch, an OCS-based defragmentation scheduler that turns fragmented rack capacity into schedulable resources. RingStitch follows a local-first policy, stitches compact cross-rack fragments only when needed, and orders the selected racks into a low-cost logical ring. Simulations on a TPU 8t-like SuperPod model show that RingStitch improves schedulability over local and full-rack baselines.

Comment
Accepted by ACP 2026, Top-Score Papers

Closest papers

Nearest neighbours by meaning across the whole index.

Opportunistic Optical Link Scheduling in the Scalable Matching Model

cs.NI ·2026-10-27 ·Marcos T. Henriques, Olga Goussevskaia
similarity 0.64
Abstract

The Scalable Matching Model (SMM) is a useful abstraction for reconfigurable datacenter networks. In SMM, a set of $n$ (Top-of-Rack) electrical packet switches is connected via a set of optical circuit switches (OCS), each realizing a $p \times p$ matching, where $p$ is the number of ports per switch. In the case when $p \ll n$, bounded-degree networks utilize only a small portion of the OCS infrastructure at each point in time, especially when traffic patterns are skewed and bursty. In this work, we propose the Opportunistic Optical Link Scheduling (OOLS), a set of lightweight OCS link scheduling routines, designed to augment bounded-degree reconfigurable network topologies by leveraging idle OCS resources. We evaluate the performance of OOLS in the SMM through simulations, using synthetic and real-world data-center workloads. Our simulation results show that OOLS substantially improves performance, lowering communication costs and boosting throughput.

0.14

Flux: Optimal Scheduling of Optical Circuit Switches for LLM Training

cs.NI cs.DC ·2026-09-22 ·Arno Troch, Seyyidahmed Lahmer, Abubakr Nada, Jeroen Famaey, Michael Peeters
position similarity 0.64
Abstract

Optical Circuit Switching (OCS) offers high bandwidth density and energy efficiency for LLM training, but incurs a non-negligible reconfiguration delay. Prior work typically schedules optical circuit switches independently of compute, using aggregate traffic demand to determine which circuits to provision and when. We argue that this separation creates a fundamental inefficiency: reconfigurations that ignore the compute timeline can stall communication, resulting in low circuit utilization and large buffer requirements. In this paper, we present Flux, a scheduler that optimally schedules optical circuit switches based on the structure of the entire workload. Flux remains effective across a wide range of switching speeds by reusing circuits and amortizing reconfiguration delay behind compute and communication. We show that Flux reduces training iteration time by up to $10\times$ and peak NIC buffer requirements by more than three orders of magnitude compared to traditional periodic schedulers.

0.38

An Optical KV Cache Network for Disaggregated LLM Serving

cs.NI ·2026-10-13 ·Seongwon Yoon, Shimeng Yu
similarity 0.60
Abstract

In disaggregated LLM serving, each request's KV cache moves from a prefill GPU to a decode GPU before the response begins, and a coding-agent turn often moves the KV cache of its whole long context. Behind an oversubscribed packet fabric, this transfer adds to the time to first token (TTFT). An optical circuit switch (OCS) can give every transfer a full-rate circuit for less than an oversubscribed packet fabric costs, but a mechanical OCS needs tens of milliseconds to reconfigure. We hide this delay inside the prefill. A reservation scheduler commits each circuit while the prefill is still computing, so the circuit is ready when the KV cache is, and each GPU reaches the OCS through its own wavelength-multiplexed (WDM) optics on one fiber pair. In simulations of GPU clusters on real serving traces with prefill times measured on H100 GPUs, this optical KV cache network matches 400G packet fabrics with adaptive routing at 0.55 to 0.78 times the cost of a 2:1 fabric and tolerates the 50 ms reconfiguration delay of a 384-port mechanical OCS.

0.24

QPS-ToR: A Parallel Iterative Switching Algorithm for Reconfigurable Optical Datacenter Switching

cs.NI ·2026-09-08 ·Dongzhao Song, Qianru Yu, Jun Xu
similarity 0.60
Abstract

Reconfigurable optical data center networks (RODCNs) have emerged as a promising solution for scaling DCN capacity, yet their scheduling mechanisms remain a performance bottleneck: traffic-oblivious schemes inherently limit throughput, while the state-of-the-art traffic-aware scheme, NegotiaToR, uses single-iteration iSLIP as its scheduling engine, which limits throughput to around 60% and treats all source-destination pairs with equal priority regardless of queue length. We propose QPS-ToR, which replaces NegotiaToR's scheduling logic with SW-QPS, a sliding-window algorithm originally proposed for crossbar scheduling that achieves around 90% throughput with a single low-complexity iteration. QPS-ToR operates within NegotiaToR's existing workflow, requiring only a revision of the scheduling cycle from three-step Request-Grant-Accept (RGA) to two-step Request-Grant (RG) with a sliding window mechanism. In flow-level simulations with 128 ToRs under realistic datacenter workloads, QPS-ToR achieves up to 36% higher throughput and 82% lower flow completion time (FCT) compared to NegotiaToR, and consistently outperforms RotorNet, a representative traffic-oblivious scheme, on the parallel network topology.

0.18

Workload-Static Optical Circuit Switching For Scalable LLM Inference Decode

cs.DC cs.AIcs.ARcs.NI ·2026-11-03 ·Maximilian Thürmer, Nicolas Scotto di Perto, Eloise Stein, Natanijel Vasic, Mikaeil Bagherli +4 more
similarity 0.60
Abstract · 39 pages, 10 main text, 29 appendix, 28 figures

LLM inference decode executes fine-grained collectives whose latency directly limits per-user interactivity, yet scale-up fabrics are still built from electronic packet switches designed for training. We observe that, for a given model deployment and parallelization, the communication structure of decode is fixed, and propose workload-static optical circuit switching (OCS): circuits are configured once per deployment, co-designed with model parallelism and with topology-aware collective schedules that turn spare accelerator ports into fewer communication phases or higher bandwidth. We evaluate this design with GXNET, a graph-based workload emulator coupled to analytical and flit-level network models grounded in hardware measurements. In a 576-accelerator scale-up domain serving 5T-parameter MoE models with up to 5M tokens of context, all-OCS fabrics improve interactivity by up to 65% and energy efficiency by up to 119% over 4:1 oversubscribed packet-switched fabrics.

0.25

Tetris: Circuit Scheduling for Rearrangeably Non-Blocking Photonic Interconnects

cs.NI ·2026-09-21 ·Eliezer Amponsah, Deeksha P Rao, Vamsi Addanki
similarity 0.59
Abstract

Reconfigurable photonic interconnects are emerging as a promising communication architecture for next-generation distributed computing. Yet, most circuit schedulers are designed around an idealized view of the interconnect as either blocking or strictly non-blocking. Practical scalable designs are often rearrangeably non-blocking (RNB), with connections routed through networks of internal $2\times2$ switches. This changes the scheduling problem fundamentally: establishing a new connection can force existing connections to be rerouted, trigger state changes across multiple internal switches, and impose reconfiguration delay on otherwise unrelated traffic. We present Tetris, a circuit scheduling algorithm for RNB photonic interconnects. Tetris builds on two observations. First, All-to-All demands are often not doubly stochastic, leaving a small number of endpoints as communication bottlenecks. Second, reconfiguration delay can be large enough to change which connection should be scheduled next. Tetris prioritizes bottleneck endpoints using their remaining communication and reconfiguration work, while selecting and routing matchings to preserve ongoing connections whenever possible. Matchings ensure progress, but connections are scheduled independently, allowing completed connections to be replaced without matching-wide barriers and incurring delay only at switches whose states change. Our simulation and hardware-emulation results show that Tetris reduces All-to-All demand completion time by up to $6.6$x over Birkhoff--von Neumann-based scheduling and by $30$% over Sunflow. More broadly, RNB interconnects raise new questions in multi-tenant scheduling and routing for partial reconfiguration, which we discuss at the end of the paper.

0.16

Broadcast, Don't Replicate: Eliminating Weight HBM in Datacenter LLM Serving

cs.AR cs.DCcs.ETcs.NI ·2026-11-12 ·Hannah Atmer, Yuan Yao, Thiemo Voigt, Stefanos Kaxiras
similarity 0.59
Abstract

Large language model (LLM) inference at datacenter scale is fundamentally bottlenecked by memory capacity and bandwidth limits. Because model weights are read-only and identical across instances, conventional architectures that replicate these parameters across the local DRAM of tens of thousands of accelerators incur massive capital expenditures, significant static power dissipation, and redundant memory-fetch energy. A recent proposal to use optical fiber loops as delay-line memory alleviates these issues, but its recirculating loop must hold the whole model: loop length grows with model size, slack is fixed when the ring is built, and the layouts that serve modern Mixture of Experts (MoE) models efficiently do not fit in it. We propose Optical Broadcast Memory (OBM), an open-ended hierarchical Tap-and-Amplify fiber architecture that generalizes and replaces the optical ring paradigm. A centralized memory server continuously broadcasts weights to distributed inference nodes that passively tap the data stream using receive-only co-packaged optics, delivering parameters directly to compute units without intermediate buffering. To overcome the limitations of continuous weight streaming, we use dynamically-sized slack to accommodate large and variable context lengths. We show how removing the restrictions of the optical ring allows OBM to efficiently serve MoE models. Evaluated at a datacenter scale of 80,000 accelerators, OBM eliminates petabytes of HBM while being competitive or better in throughput and energy per token than an HBM-based system.

0.25

Capturing Electrical Windows Through Reconfigurable Optical Networks: Transfer-aware Scheduling of Multi-Region AI Data Centers

cs.DC ·2026-10-09 ·Xi Tong, Zijian Li, Xiaoyi Fan, Cong Zhang, Ershun Du +1 more
benchmark similarity 0.58
Abstract · 10 pages, 12 figures

Artificial-intelligence data centers (AIDCs) can provide spatiotemporal load flexibility by migrating training workloads in response to regional electricity prices and renewable-energy (RE) availability. However, transfer-blind scheduling may overestimate migration benefits by overlooking transfer time or prevent training from resuming within renewable-rich or low-price windows when considering transfer time. This paper proposes a transfer-aware compute-electricity-network scheduling framework that coordinates training placement, checkpoint transmission, and optical circuit switching (OCS) to improve local RE utilization and reduce grid-purchase cost. A compact compute-electricity master problem generates candidate schedules using estimated transfer delays. The framework then schedules optical circuits, verifies checkpoint delivery under shared-link capacity constraints, and revises infeasible schedules using updated delay estimates. It integrates day-ahead planning, event-triggered intraday updates, and real-time dispatch. In a five-AIDC benchmark with equal completed work and network resources, the proposed method reduces the real-time grid purchase cost by 12.50% and increases the local RE utilization by 2.724 percentage points relative to static optical operation. Sensitivity and ablation analyses indicate that OCS is particularly beneficial when renewable-rich or low-price windows shift across sites and network bottlenecks hinder timely checkpoint delivery.

0.20