CS Paper Radar

AI Inference-Aware Optimal Power Flow: Converting Token Requirements into Grid Flexibility

Qianyi Ma, Dawei Qiu, Xiao Liu

eess.SY ·submitted 2026-11-14 03:24 UTC · v1

Abstract

The rapid growth, temporal variability, and geographic concentration of electricity demand from artificial intelligence (AI) inference can aggravate transmission congestion near data centers (DCs). The spatial and temporal flexibility of inference workloads offers an opportunity to alleviate this congestion: requests can be routed across DCs or deferred within users' quality-of-service (QoS) requirements. However, inference routing primarily considers graphics processing unit (GPU) availability, leaving this flexibility underused in grid operation. Using GPU measurements, this paper models inference latency and DC power consumption to quantify the flexibility available under users' time-to-first-token (TTFT) and inter-token latency (ITL) requirements. These models are then incorporated into an AI inference-aware optimal power flow (OPF) that coordinates inference routing and power system operation among DC operators, the transmission system operator, and inference users. By linking user requests, token generation at DCs, and electricity delivery through the transmission network, the formulation schedules inference workloads subject to both users' QoS requirements and transmission constraints. Simulations on the PJM system show that, compared with computing-based routing and with electricity price-based routing, coordinating inference routing with network operation enables growing DC electricity demand deliverable to be met at lower cost.

Closest papers

Nearest neighbours by meaning across the whole index.

Grid Demand Flexibility Assessment of AI Data Centers via Batch Workload Temporal Shifting

eess.SY ·2026-09-29 ·Suntao Su, Liang Du, Shengyi Wang
similarity 0.75
Abstract · 5 pages, 5 figures, conference

The rapid growth of artificial intelligence (AI) data centers has introduced new challenges to power system operation. As their power demand becomes larger and more variable, quantitatively characterizing their demand flexibility is increasingly important for effective power system coordination. However, heterogeneous workload characteristics and resource requirements make this flexibility difficult to characterize directly. This paper proposes a framework for assessing the grid-compatible demand flexibility of AI data centers via batch workload temporal shifting. An averaging-based resource usage processing method is developed to map fine-resolution CPU, GPU and memory usage into unified time intervals compatible with power system operation. A workload temporal scheduling model is then formulated to shift batch workloads while preserving execution continuity, delay constraints, and server resource capacities, and is coupled with a utilization-dependent server power model to translate workload scheduling decisions into server power demand. Two complementary flexibility metrics are evaluated: short-term peak demand shaving and the maximum duration of sustained power reduction. Numerical results based on real GPU cluster traces demonstrate that workload temporal shifting can provide quantifiable and grid-compatible demand flexibility for AI data centers with limited disruption to computing workloads.

0.13

Spatial LLM Workload Shifting Needs Foresight: Model Commitment for AI Data Center Operation under Power Grid Constraints

eess.SY ·2026-09-09 ·Bojun Du, Hongyang Jia, Tonghui Li, Qingchun Hou, Ze Wang +2 more
similarity 0.69
Abstract · 3 pages, 2 figures

AI data centers may face power supply shortages during certain periods, requiring operators to shift large language model (LLM) inference workloads spatially to maintain service rates. However, existing workload-shifting methods typically assume that any data center with sufficient computing resources can immediately serve shifted requests, which may lead to infeasible transfers and unserved demand. This letter proposes model commitment (MC), a mixed-integer linear programming framework that jointly schedules model deployment and cross-site request routing under power constraints and electricity-price signals. First, MC formulates the intertemporal coupling introduced by model replica loading. Second, it translates prefill and decode latency requirements into the amount of demand that each replica can serve. Case studies based on real-world data show that MC enables AI data center operators to achieve a 100% service rate under time-varying grid conditions and reduce total operating cost by 29.0%.

0.44

Analytical Power-Aware Provisioning for Prefill-Decode Disaggregated AI Inference

cs.DC cs.PFeess.SY ·2026-09-21 ·Mingyuan Yan, Haiyu Wang, Linxuan Biao, H. Jonathan Chao, Sai Qian Zhang +1 more
similarity 0.69
Abstract

Power availability increasingly constrains the operation of AI inference fleets, creating a need for provisioning methods that jointly consider serving capacity and power consumption. Prefill--decode (PD) disaggregation has emerged as a prevalent architecture for large-scale inference serving. However, determining the appropriate numbers of prefill and decode instances is challenging because serving capacity depends jointly on workload characteristics, hardware constraints, queueing, and KV-cache reservations. Existing approaches largely rely on profiling and simulation, providing limited analytical insight into how provisioning decisions shape the tradeoff between serving capacity and power consumption. This paper develops an analytical framework for power-aware provisioning of PD-disaggregated AI inference. Given an inference workload and hardware, the framework models the serving capacity and average power consumption of a provisioned deployment. The serving-capacity model is derived from the joint distribution of input--output lengths and hardware compute and memory limits. In particular, it explicitly captures the coupling between prefill and decode induced by KV-cache reservations, as well as the impact of request queueing. On this basis, the power model determines per-instance power consumption as a function of normalized serving throughput. Together, the models determine the serving capacity--power Pareto front among candidate provisioned deployments, enabling the service provider to choose a provisioned deployment as the workload or available power changes.

0.33

Mitigating the Grid Impact of Gigawatt-Scale AI Data Centers via High-Fidelity Model Flexibility

eess.SY math.OC ·2026-11-18 ·Eihab E. E. Ahmed, Saeed Manshadi
hot direction similarity 0.69
Abstract

Gigawatt-scale artificial intelligence data centers (AIDCs) are being connected at a single point of interconnection (POI) faster than transmission can be reinforced, yet studies of their flexibility rely on DC power flow and simplified load envelopes that omit voltage, cooling dynamics, and workload deadlines. This paper addresses this gap by embedding a high-fidelity AIDC model, with separate inference and training pools, hall thermal mass, PV, a battery energy storage system (BESS) with reactive capability, and a STATCOM, in a five-minute multi-period AC optimal power flow. Simulations on the RTS-GMLC test system show that active-power flexibility mitigates local price spikes and relieves congestion, whereas reactive support from the BESS inverter or a STATCOM is needed to keep the POI voltage away from sagging around its 0.95~pu limit. Sensitivity analyses show that cost-driven charging of large batteries can induce new morning congestion and that storage largely substitutes for wider workload deferral.

0.30

Large-Scale Data Center Integration in Power Systems: Uncertainty and Multi-Scale Variation

eess.SY ·2026-10-09 ·Le Xie, Dongjoo Kim, Qian Zhang, Grant Ruan, Paul L. Joskow
published similarity 0.68
Abstract · Accepted for publication in Proceedings of the IEEE. Copyright 2026 IEEE. Personal use of this material is permitted. Permission from IEEE m…

Hyperscale artificial intelligence (AI) data centers are emerging as a structurally new class of electricity demand. Unlike conventional loads, they materialize as spatially concentrated, transmission-level point loads with rapid deployment timelines and converter-dominated dynamics. Their integration is governed by two defining properties: forecast uncertainty in load realization and multi-scale temporal variation spanning sub-seconds to planning horizons. We develop a unified framework that links reliability and affordability to these dimensions through a four-quadrant structure: (reliability, affordability) $\times$ (uncertainty, variation), which integrates probabilistic resource adequacy, generation and transmission expansion, operational security, and market design. Measured traces from a public generative-AI workload dataset show strongly tonal GPU power spectra rather than smooth broadband behavior: harmonic combs at the training-step cadence, and a distinct sub-second periodicity in inference. Across these traces the dominant variance band shifts with workload type (seconds for training and offline inference; tens of seconds and longer for online inference). We report this as a characterization of the measured workloads rather than a universal signature, and take it to indicate that such loads warrant a spectral description rather than a peak-demand one. These signatures directly affect reserve requirements, ramping products, voltage stability, ride-through performance, and harmonic interactions in converter-rich grids. Taken together, the analytical framework and empirical results establish a system-level basis for assessing the reliability and cost implications of large AI data center integration across planning and operational horizons.

0.27

Optimal Operation Method for Computing Power-Electric Power Coordination Considering End-to-End Completion Latency of Computing Tasks

eess.SY ·2026-09-22 ·Yize Liu, Mingyu Yan, Jianfeng Wen, Meng Song, Qing Yang +1 more
similarity 0.67
Abstract · 10 pages, 7 figures, 4 tables. Submitted to Proceedings of the CSEE. This is a bilingual manuscript. The English version appears first, foll…

With the rapid growth of computing demand and the large-scale integration of renewable energy, how to realize the coordinated optimal operation of computing networks and power networks has become an important issue to be addressed. However, existing studies mainly focus on the impacts of spatiotemporal migration of computing loads on power system operation, while the internal task processing procedures of computing networks are largely neglected. To address this issue, this paper proposes an optimal operation method for computing power-electric power coordination considering the end-to-end completion latency of computing tasks. First, a full-process latency model covering the "transmission-buffering-computation" procedure of computing tasks is established, which uniformly characterizes forwarding waiting, network transmission, queueing, and computation processing. Second, a task-level spatiotemporal scheduling mechanism is developed to jointly optimize the task forwarding time, routing path, and destination computing node. Then, a computing power-electric power coordinated optimization model is formulated to minimize the power supply cost of the power network and the total completion latency of computing tasks. Case studies demonstrate that the proposed method can fully exploit the task-level spatiotemporal scheduling flexibility of computing loads and facilitate the spatiotemporal matching between computing loads and renewable energy, thereby reducing the system power supply cost while ensuring the quality of service for computing tasks.

0.22

Creating Power Flexibility in Datacenters via Query Routing

cs.AI ·2026-11-03 ·Shivam Patel, Bingqing Chen, Jared Fernandez, Guannan Qu, Gauri Joshi
similarity 0.67
Abstract

The growing energy requirements of large language model inference highlight the need for datacenters to support safe grid operation by providing power demand flexibility, while maintaining service for interactive workloads. This work investigates query routing as a mechanism to enable such capability by dynamically allocating queries across different language model instances without deferring requests. We develop a general analytical framework that connects query arrival rates and sequence length distributions to token iteration time, request residence time, serving capacity, and power consumption of model instances. Leveraging this framework, we characterize the feasible query routing allocations and the corresponding power demand flexibilities, enabling the selection of routing strategies to achieve desired power consumption behavior. We validate our framework across four GPU platforms on language models ranging from 1B to 70B parameters, obtaining 2.8% mean absolute percentage error in estimating achievable power flexibility. We evaluate our approach on a real trace-driven simulation of a 10,000 H100 GPUs datacenter, exhibiting flexibility up to 16% of total power consumption, with 90% of the desired power reduction achieved within approximately 10 seconds. These results establish query routing as an effective mechanism for datacenter power demand flexibility and demonstrate the broader applicability of our proposed framework to datacenter level planning and orchestration.

0.12

From Grid to Chip: Power Architecture, Stability, and Flexibility of AI Data Centers

cs.ET cs.AReess.SY ·2026-09-10 ·Yubo Song, Rui Kong, Takuro Umihara, Pooya Davari, Frede Blaabjerg +1 more
publishedhot direction similarity 0.66
Abstract · The manuscript has been submitted for review to IEEE Open Journal of Power Electronics

The rapid growth of artificial intelligence (AI) computing is transforming data centers into large, dynamic electrical loads. Their deployment is primarily constrained by energy availability and grid-connection capacity, which is further aggravated by the ability of power-delivery architectures, control systems, and computing workloads to operate reliably during fast grid disturbances. This article presents a technological perspective on AI data centers as grid-interactive computing systems. First, it reviews grid-integration bottlenecks, evolving connection policies, grid-code requirements, which has fostered new technological trends via spatio-temporal flexibility available through workload orchestration, cooling systems, on-site resources, and energy storage. Second, it maps the evolution of power-delivery architectures from medium-voltage grid interfaces to chip-level, discussing higher-voltage DC distribution, solid-state transformers, wide-bandgap devices, advanced chip-level power delivery, and liquid cooling. Third, it establishes a three-level stability framework spanning rack-level DC-bus dynamics, facility-level converter interactions, and system-level grid-coupled behavior. The framework connects dominant instability mechanisms, including constant power load effects, impedance interactions, forced oscillations, and operating-mode transitions, with suitable modeling, assessment, and mitigation approaches. Synthesizing these topics, this article highlights grid-to-chip co-design as a central requirement for scalable AI infrastructure, linking computing workloads, power-delivery systems, energy buffers, and grid operation.

0.37