CS Paper Radar

Reinforcement Learning-Based Optimization of Workload-Aware Power Delivery Networks

Oran Hayes, Maria Pantazi-Kypraiou, Athanasios Tziouvaras, George Stamoulis, Anuj Pathania, Shreejith Shanker, George Floros

cs.AR cs.LG ·submitted 2026-10-05 11:19 UTC · v1
publishedbenchmark

Abstract

Power Delivery Networks (PDNs) are critical components of modern VLSI chips, providing stable voltage levels while satisfying electromigration (EM) and IR-drop constraints. Conventional PDN design methodologies typically rely on worst-case assumptions, often resulting in over-provisioned networks and inefficient use of resources. This paper presents a reinforcement learning-based framework for the optimization of workload-aware PDNs. The proposed methodology first generates workload-aware PDNs using architectural power traces obtained from system-level simulations. These power traces are mapped to spatial power density distributions, enabling adaptive allocation of PDN resources according to local current demand. A reinforcement learning agent then performs wire-width optimization to minimize PDN area while maintaining EM and voltage integrity constraints. Electrical and reliability metrics are obtained using SPICE-based circuit analysis and EM lifetime estimation. Experimental evaluation is performed on a dataset of workload-aware PDNs generated from 4-, 8-, and 16-core multiprocessor floorplans using PARSEC and SPLASH-2 benchmark workloads. Furthermore, the proposed Deep Q-Network (DQN)-based optimizer reduces the average normalized PDN area by 47\% while satisfying all EM and IR-drop constraints. Compared to simulated annealing, the proposed approach achieves comparable optimization quality while providing approximately 26$\times$ faster optimization.

Comment
Accepted at ICECS 2026

Closest papers

Nearest neighbours by meaning across the whole index.

Beyond Accuracy: Physics-Informed Evaluation of ANN-based Power Flow Solvers under System Shift

eess.SY ·2026-11-10 ·Dong Liu, Jochen Stiasny, Jochen Cremer, Pedro P. Vergara
similarity 0.62
Abstract

Recent advances in artificial deep learning-based power flow (PF) solvers have primarily focused on improving estimation accuracy, while their physical feasibility and operational reliability under realistic system variations remain insufficiently evaluated. This paper proposes a physics-informed evaluation framework for deep learning-based PF solvers under topology and load distribution shifts. Beyond conventional voltage-domain errors, the framework incorporates KCL gaps, branch-flow gaps, KVL residuals, continuous feasibility scores, and strict, practical, and relaxed PF feasibility rates. Four case studies are designed to investigate topology shifts, mixed topology training, load distribution shifts, and combined topology distribution shifts. Eight ANN architectures, including graph input models, load-only baselines, and physics-informed neural networks, are evaluated under consistent settings. The results demonstrate that low estimation errors do not necessarily correspond to physically feasible or operationally reliable PF solutions, particularly under combined system shifts. The proposed framework establishes an initial foundation and analysis for the development and verification of trustworthy deep learning-based PF solvers, providing support for their reliable deployment in future AI-enabled power system analysis.

0.09

Reinforcement learning for hardware acceleration and system-level optimization on FPGA and integrated-circuit platforms: A survey

cs.LG cs.AIcs.ARcs.PF+2 ·2026-10-20 ·Hamza Kheddar
publishedsurveyhighly cited similarity 0.62
Abstract

Reinforcement learning (RL) enables adaptive and autonomous decision-making in embedded, edge, and heterogeneous computing systems. Field-programmable gate arrays (FPGAs) and integrated circuits (ICs) offer low latency, energy efficiency, and architectural flexibility for RL deployment, while introducing challenges in resource constraints, implementation complexity, and hardware-algorithm co-design. This review surveys more than 100 studies on RL implementations across FPGA and IC platforms, emphasizing architectures, implementation strategies, performance evaluation, and applications. The literature is organized by RL family, including model-based Markov decision processes, classical value-based methods, deep RL, actor-critic, policy-gradient, and multi-agent approaches. We further classify studies by abstraction level, hardware realization strategy, and application domain, examining how RL algorithms map onto reconfigurable and fixed hardware. Unlike surveys restricted to particular algorithms or platforms, this work provides a unified analysis of FPGA and IC implementations, covering acceleration, reconfigurable architectures, energy-aware control, and system-level optimization. Comparative assessments consider resource utilization, latency, throughput, power consumption, and reported performance gains. Finally, we identify recurring architectural patterns, practical limitations, open challenges, and research directions for efficient RL-driven hardware systems. The survey offers a consolidated reference for researchers developing and evaluating RL solutions on specialized hardware.

0.47

PowerZooJax: A JAX-based Power System Benchmark for Reinforcement Learning

cs.AI cs.LGeess.SY ·2026-09-28 ·Zhanhua Pan, Xiao Liu, Zhilong Cao, Jianhong Wang, Dawei Qiu
benchmarkcode/data similarity 0.62
Abstract · 42 pages

Power system operation is a safety-critical sequential decision-making problem, making it a natural testbed for reinforcement learning (RL). However, existing RL environments for power systems are often narrow in scope and computationally limited by CPU-based simulation workflows, making large-scale evaluation difficult. We introduce PowerZooJax, a JAX-based benchmark suite for RL in power system operation. It provides five constrained Markov decision process tasks spanning generation, transmission, distribution, distributed energy resources, and data center microgrid. By rewriting power flow, economic dispatch, market clearing, and device dynamics as JAX computation graphs, PowerZooJax keeps the entire training and evaluation loop on the GPU. Experiments show substantial speedups over CPU-based simulations and demonstrate standardized evaluation of policy returns, safety violations, and out-of-distribution stress conditions. Our open-source benchmark is available at: https://github.com/powerzoojax/PowerZooJax.

0.41

Physics-Informed Graph-Based Safe RL with Constraint-Specific Critics for Community Battery Coordination in Distribution Network

cs.AI eess.SYmath.OC ·2026-10-24 ·Chenghao Huang, Yue Yang, Hao Wang
top venue similarity 0.60
Abstract

High penetrations of distributed energy resources (DERs) and fast-varying electric vehicle (EV) charging intensify voltage and line-limit violations in distribution networks (DNs), making safe, low-cost operation increasingly challenging. This paper proposes a physics-informed safe deep reinforcement learning (DRL) framework for coordinating community battery energy storage systems to mitigate constraint violations while minimizing operating cost. To represent nonlocal spatiotemporal interactions, we develop a dual-level physics-informed graph representation in which line impedance and capacity shape spatial information propagation, while power-balance, voltage-profile, and flow-reconstruction losses regularize the latent state toward control-relevant physical consistency. Building on this representation, we develop Lagrangian Dedicated-Critic PPO (LDCPPO), which implements constraint-specific value decomposition: independent reward and cost critics preserve heterogeneous voltage- and line-loading-related learning signals and combine their advantages only during the Lagrangian policy update. Case studies on DNs with high rooftop photovoltaic penetration and EV charging demonstrate that the proposed method reduces the key voltage-violation degree by up to 80.5% compared with the tested baselines, while also achieving the lowest voltage-violation count and line-overload degree with competitive operating cost.

0.31

From Grid to Chip: Power Architecture, Stability, and Flexibility of AI Data Centers

cs.ET cs.AReess.SY ·2026-09-10 ·Yubo Song, Rui Kong, Takuro Umihara, Pooya Davari, Frede Blaabjerg +1 more
hot direction similarity 0.59
Abstract · The manuscript has been submitted for review to IEEE Open Journal of Power Electronics

The rapid growth of artificial intelligence (AI) computing is transforming data centers into large, dynamic electrical loads. Their deployment is primarily constrained by energy availability and grid-connection capacity, which is further aggravated by the ability of power-delivery architectures, control systems, and computing workloads to operate reliably during fast grid disturbances. This article presents a technological perspective on AI data centers as grid-interactive computing systems. First, it reviews grid-integration bottlenecks, evolving connection policies, grid-code requirements, which has fostered new technological trends via spatio-temporal flexibility available through workload orchestration, cooling systems, on-site resources, and energy storage. Second, it maps the evolution of power-delivery architectures from medium-voltage grid interfaces to chip-level, discussing higher-voltage DC distribution, solid-state transformers, wide-bandgap devices, advanced chip-level power delivery, and liquid cooling. Third, it establishes a three-level stability framework spanning rack-level DC-bus dynamics, facility-level converter interactions, and system-level grid-coupled behavior. The framework connects dominant instability mechanisms, including constant power load effects, impedance interactions, forced oscillations, and operating-mode transitions, with suitable modeling, assessment, and mitigation approaches. Synthesizing these topics, this article highlights grid-to-chip co-design as a central requirement for scalable AI infrastructure, linking computing workloads, power-delivery systems, energy buffers, and grid operation.

0.27

Adapting DNNs to Embrace Voltage Scaling Induced Errors via Accumulation-Aware Quantization

cs.AR cs.AI ·2026-10-24 ·Alexis Maras, Panagiotis Chaidos, Georgios Alexandris, Dimitrios Soudris, Sotirios Xydis
published similarity 0.59
Abstract · Accepted for publication at the 45rd International Conference on Computer-Aided Design (ICCAD '26), November 08-12, 2026, San Jose, CA, USA

The pervasive deployment of Deep Neural Networks (DNNs) across the computing spectrum has brought growing attention to energy efficiency, alongside raw performance. While voltage scaling is a potent knob for power reduction, its application in next-generation nodes is hampered by severe throughput degradation and susceptibility to process variation. This work aims to address these energy-reliability trade-offs by introducing a hardware-software co-design methodology, based on the observation that timing-robust bit ranges within the accumulator of a systolic array vary significantly across different voltage-frequency operating points. At the software level, we propose an Accumulation-Aware Training approach that incorporates this information as a first-order optimization constraint during training. By integrating mixed-precision quantization with unstructured pruning, we effectively constrain the model's weight distribution to avoid activating the hardware's sensitized high-rank carry chains. At the architectural level, we complement this with selective MSB-masking and fine-grained clock-gating to minimize unnecessary dynamic switching activity. Furthermore, we employ a layer-wise Dynamic Frequency Scaling strategy to further adapt to the accumulator's reduced dynamic range, based on the numerical sensitivity of each layer. We experimentally evaluate the proposed solution on a deep scaled 2nm technology node and demonstrate that, when operating at a low-voltage frontier, our co-optimized post-layout architecture delivers substantial energy and energy-delay product gains of 39% and 23% respectively, relative to the nominal voltage implementation, while maintaining accuracy degradation within 1%.

0.33

AI Inference-Aware Optimal Power Flow: Converting Token Requirements into Grid Flexibility

eess.SY ·2026-11-14 ·Qianyi Ma, Dawei Qiu, Xiao Liu
similarity 0.59
Abstract

The rapid growth, temporal variability, and geographic concentration of electricity demand from artificial intelligence (AI) inference can aggravate transmission congestion near data centers (DCs). The spatial and temporal flexibility of inference workloads offers an opportunity to alleviate this congestion: requests can be routed across DCs or deferred within users' quality-of-service (QoS) requirements. However, inference routing primarily considers graphics processing unit (GPU) availability, leaving this flexibility underused in grid operation. Using GPU measurements, this paper models inference latency and DC power consumption to quantify the flexibility available under users' time-to-first-token (TTFT) and inter-token latency (ITL) requirements. These models are then incorporated into an AI inference-aware optimal power flow (OPF) that coordinates inference routing and power system operation among DC operators, the transmission system operator, and inference users. By linking user requests, token generation at DCs, and electricity delivery through the transmission network, the formulation schedules inference workloads subject to both users' QoS requirements and transmission constraints. Simulations on the PJM system show that, compared with computing-based routing and with electricity price-based routing, coordinating inference routing with network operation enables growing DC electricity demand deliverable to be met at lower cost.

0.20

Emerging Vertical Power Delivery in AI Data Centers: Challenges and Solutions

eess.SY ·2026-11-16 ·Charles Isaac Zahner, Amirhossein Habibi, Mehdi Ferdowsi
similarity 0.59
Abstract · 29 pages, 24 figures. This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after …

The rapid growth of artificial intelligence (AI) workloads is increasing power demand in AI data centers, and processor current is projected to reach 2000 A and beyond. This trend is exposing the limitations of lateral power delivery (LPD) to processors and has motivated growing interest in vertical power delivery (VPD). This paper reviews the power delivery network (PDN) of AI data centers and the upcoming transition from LPD to VPD. First, the 800 VDC power delivery architecture, the impedance behavior of the PDN, and the limitations of LPD are discussed. Then, VPD implementation approaches, reported converter designs, and packaging technologies are examined, together with integrated voltage regulators (IVRs) and embedded passive components. In addition, power integrity simulations are presented to compare regulator and plane positions, from which a set of PDN design guidelines is derived. The results show that moving a voltage regulator module (VRM) from 50 mm away to directly below the die reduces the loop inductance by nearly half. In a complete PDN, this relocation lowers the low-frequency anti-resonant peak, whereas the mid- and high-frequency impedance depends on the decoupling stages and the position of the power and ground planes. This paper also shows that achieving practical VPD remains closely tied to challenges such as regulator height, passive-component embedding, and thermal management.

0.23