CS Paper Radar

Cross-Layer Analysis of Thermal Tuning Stalls in Wafer-Scale Optical Interconnects for LLM MoE Training

Seongwon Yoon, Pin-Jun Chen, Shimeng Yu

cs.AR ·submitted 2026-08-25 14:48 UTC · v4
unusual

Abstract

Mixture-of-experts (MoE) training depends heavily on all-to-all communication, which wafer-scale optical interconnects with dense wavelength-division multiplexing can serve. Their microring resonators rely on thermo-optic tuning, while MoE compute bursts swing the photonic-layer temperature by about 10 K within tens of milliseconds. This paper quantifies the communication stall by coupling packet-level network simulation, transient thermal simulation of a 3D-stacked GPU and photonic die, and a ring detuning criterion, and by feeding the stall back into the network timeline until both agree. Including this feedback, a tracking loop at the measured 5 nm/s lengthens the training iteration of Mixtral 8x7B and LLaMA-MoE 6.7B by factors of about 1.16 and 1.25 at full model depth. Measured H100 die temperatures match the modeled swing of 300 ms bursts within 15%. A loop slewing eight times faster, a heater driven at each kernel launch, or a dummy load near 75% of peak power removes the stall. An athermalized lithium-niobate ring with a non-volatile ferroelectric setpoint removes it with no holding power or fast loop, which makes it suitable for wafer-scale optical interconnects.

Closest papers

Nearest neighbours by meaning across the whole index.

Comparative Robustness of Mach-Zehender and Microring Photonic Matrix Multipliers Under Laser RIN and Thermal Drift

cs.AR physics.optics ·2026-10-11 ·Andreas Falkenberg
similarity 0.59
Abstract · 22 pages, 16 figures

Photonic matrix multipliers can accelerate neural-network inference, but their practical utility depends on how optical weight elements respond to source noise, thermal variation, and calibration error. This paper compares Mach-Zehnder modulator (MZM) and microring-resonator (MRR) implementations under laser relative-intensity noise (RIN), column-wise thermal/phase drift, and their joint action. Two CIFAR-10 classifiers - a compact four-layer CNN and ResNet-8 - are mapped to matched photonic multiply-accumulate chains using common source-distribution, detection, numerical, and calibration assumptions. Both architectures reproduce the same zero-perturbation baseline for each workload: 79.5% classification accuracy for the Small CNN and 70.0% for ResNet-8. Robustness is then evaluated using separate RIN-only and drift-only sweeps and a full two-dimensional RIN-drift grid, with dimensionless RMS perturbation levels extending from 0.5% to 20%. The analysis reports classification accuracy, class normalized confusion matrices, rolling-accuracy traces, one-dimensional robustness curves, and multidimensional response surfaces. Device equations and parameter definitions follow the models implemented in the QPI Labs Photonic Inference Simulator, with shared laser fluctuations propagated as correlated multiplicative noise and slow drift represented as a per-output-column perturbation of calibrated weights. All numerical results are generated from recorded experiment configurations and random seeds. The study provides a controlled system-level framework for comparing interferometric and resonant photonic weight elements while limiting conclusions to the implemented device models, perturbation ranges, calibration policy, and neural-network workloads.

0.52

Scaling Inference Prefill with High-Radix Photonic Interconnects

cs.DC cs.AR ·2026-09-01 ·Arulselvan Madhavan, Peter Carson, Taylor Groves, Thomas Graham
published similarity 0.54
Abstract

With the rise of inference as today's dominant AI workload, the industry is transitioning to high-bandwidth photonic interconnects to meet the large scale-up requirements of increasingly complex Mixture-of-Experts (MoE) models. This paper quantifies the benefits of 3D-integrated photonic interconnects for inference prefill by analyzing tradeoffs between high-concurrency throughput for Large Language Model (LLM) chat and the large context windows typically required for reasoning and agentic AI. We simulate three MoE models: short context (1K--8K tokens), medium context (128K tokens), and long context (1M tokens). We evaluate this workload across existing copper-based GPU systems and one with high bandwidth integrated photonics. We show 2.1--3.2x latency improvements in the stressed high-batch regimes and 2.8--5.8x improvements over baselines in communication-limited configurations. 3D photonics enable the 1152-GPU footprint required to lower time-to-first-token, yielding 2.2--4.5x speedups across production-grade platforms when electrical systems cross their inherent scale-up-pod limits.

0.24

OMRC: Multipath Reliability for Direct-Connect Optical Interconnects

cs.NI ·2026-10-22 ·Eric Ding, Keshav Pradeep, Rachee Singh
similarity 0.54
Abstract

Network failures degrade Multipath Reliable Connection (MRC) in direct-connect optical datacenters. We propose Optical MRC (OMRC), which remaps collectives and circuits, preserving 95.7% of training throughput, restoring stalled inference, and achieving 4.7x MRC's simulated bandwidth.

0.27

Workload-Static Optical Circuit Switching For Scalable LLM Inference Decode

cs.DC cs.AIcs.ARcs.NI ·2026-11-03 ·Maximilian Thürmer, Nicolas Scotto di Perto, Eloise Stein, Natanijel Vasic, Mikaeil Bagherli +4 more
similarity 0.54
Abstract · 39 pages, 10 main text, 29 appendix, 28 figures

LLM inference decode executes fine-grained collectives whose latency directly limits per-user interactivity, yet scale-up fabrics are still built from electronic packet switches designed for training. We observe that, for a given model deployment and parallelization, the communication structure of decode is fixed, and propose workload-static optical circuit switching (OCS): circuits are configured once per deployment, co-designed with model parallelism and with topology-aware collective schedules that turn spare accelerator ports into fewer communication phases or higher bandwidth. We evaluate this design with GXNET, a graph-based workload emulator coupled to analytical and flit-level network models grounded in hardware measurements. In a 576-accelerator scale-up domain serving 5T-parameter MoE models with up to 5M tokens of context, all-OCS fabrics improve interactivity by up to 65% and energy efficiency by up to 119% over 4:1 oversubscribed packet-switched fabrics.

0.25

Broadcast, Don't Replicate: Eliminating Weight HBM in Datacenter LLM Serving

cs.AR cs.DCcs.ETcs.NI ·2026-11-12 ·Hannah Atmer, Yuan Yao, Thiemo Voigt, Stefanos Kaxiras
similarity 0.53
Abstract

Large language model (LLM) inference at datacenter scale is fundamentally bottlenecked by memory capacity and bandwidth limits. Because model weights are read-only and identical across instances, conventional architectures that replicate these parameters across the local DRAM of tens of thousands of accelerators incur massive capital expenditures, significant static power dissipation, and redundant memory-fetch energy. A recent proposal to use optical fiber loops as delay-line memory alleviates these issues, but its recirculating loop must hold the whole model: loop length grows with model size, slack is fixed when the ring is built, and the layouts that serve modern Mixture of Experts (MoE) models efficiently do not fit in it. We propose Optical Broadcast Memory (OBM), an open-ended hierarchical Tap-and-Amplify fiber architecture that generalizes and replaces the optical ring paradigm. A centralized memory server continuously broadcasts weights to distributed inference nodes that passively tap the data stream using receive-only co-packaged optics, delivering parameters directly to compute units without intermediate buffering. To overcome the limitations of continuous weight streaming, we use dynamically-sized slack to accommodate large and variable context lengths. We show how removing the restrictions of the optical ring allows OBM to efficiently serve MoE models. Evaluated at a datacenter scale of 80,000 accelerators, OBM eliminates petabytes of HBM while being competitive or better in throughput and energy per token than an HBM-based system.

0.25

Beyond HBM-on-GPU: Thermal Design Envelope for 3D Volumetric DRAM-on-GPU Integration

cs.ET ·2026-09-21 ·Yukai Chen, Melina Lofrano, Khakim Akhunov, Jonas Svedas, Arjun Singh +10 more
published similarity 0.53
Abstract · Presented at the 52nd IEEE European Solid-State Electronics Research Conference (ESSERC 2026), Palma de Mallorca, Spain, September 7-10, 202…

The scaling of GPUs for AI and HPC workloads is increasingly constrained by the capacity, bandwidth, and thermal limits of both 2.5D HBM-GPU and direct-stacked 3D HBM-on-GPU integration. This work establishes the thermal design envelope for 3D volumetric DRAM-on-GPU integration, in which vertically oriented DRAM dies and interleaved cooling cavities reshape heat flow and memory interfacing above the GPU. Using a package-level thermal model anchored to a consistent HBM-on-GPU baseline and driven by a realistic reticle-scale non-uniform GPU power map, we quantify the key parameters governing thermal feasibility. Stack height is the dominant limiter of peak temperature, while cooling-cavity conductivity shifts the feasible region, and mold insertion and stack orientation further modulate thermal behavior. A distributed memory-controller and network-on-chip tier introduces only a moderate thermal penalty. Although die-level parallelism increases bandwidth, the reduction in simulated training time saturates once execution becomes compute-bound. These results define a bounded co-design space across bandwidth, capacity, and thermal constraints for 3D volumetric DRAM-on-GPU integration.

0.38

ScaleWire: An End-to-End Scale-Up Ethernet Simulation Framework for LLM Systems

cs.NI ·2026-11-19 ·Kai Ma, Quanfeng Lv, Bowei Dai, Jingguo Ge, Kefan Ruan +1 more
similarity 0.53
Abstract · 7 pages, 8 figures

LLM training and inference rely on scale-up networks for high-bandwidth, low-latency memory-semantic communication among accelerators (XPUs). Ethernet is emerging as an open interconnect for scale-up systems. In Ethernet systems, network-interface (NI) mechanisms across transport, network, and link layers interact, jointly shaping communication performance and on-chip resource requirements. We present ScaleWire, an end-to-end scale-up Ethernet simulation framework for LLM systems. It couples packet-level simulation of cross-layer NI mechanisms with a resource exploration engine that combines analytical estimates and simulation to size on-chip resources. We validate ScaleWire against hardware measurements and nine analytical checks, and evaluate six NI designs across four LLM communication workloads. For DeepSeek-R1 dispatch, NI design choices lead to a 4.5$\times$ difference in per-XPU on-chip storage budget. In grid-comparison experiments, the exploration engine uses 6-19% of the simulations required by exhaustive search. Using vLLM prefill traces with kernel times measured on GH200 GPUs, ScaleWire further guides resource provisioning under a completion-time constraint for inference execution. ScaleWire thus helps designers select NI mechanisms and provision on-chip resources to balance communication performance and resource cost.

0.25

An Optical KV Cache Network for Disaggregated LLM Serving

cs.NI ·2026-10-13 ·Seongwon Yoon, Shimeng Yu
similarity 0.52
Abstract

In disaggregated LLM serving, each request's KV cache moves from a prefill GPU to a decode GPU before the response begins, and a coding-agent turn often moves the KV cache of its whole long context. Behind an oversubscribed packet fabric, this transfer adds to the time to first token (TTFT). An optical circuit switch (OCS) can give every transfer a full-rate circuit for less than an oversubscribed packet fabric costs, but a mechanical OCS needs tens of milliseconds to reconfigure. We hide this delay inside the prefill. A reservation scheduler commits each circuit while the prefill is still computing, so the circuit is ready when the KV cache is, and each GPU reaches the OCS through its own wavelength-multiplexed (WDM) optics on one fiber pair. In simulations of GPU clusters on real serving traces with prefill times measured on H100 GPUs, this optical KV cache network matches 400G packet fabrics with adaptive routing at 0.55 to 0.78 times the cost of a 2:1 fabric and tolerates the 50 ms reconfiguration delay of a 384-port mechanical OCS.

0.24