CS Paper Radar

TuneBench: Benchmarking AI Agents for Power System Dynamic Model Calibration

Andrea Pomarico, Salvatore Tessitore, Giorgio Giannuzzi, Alberto Berizzi

eess.SY ·submitted 2026-11-03 10:29 UTC · v1
benchmarkcode/data

Abstract

Large Language Model (LLM)-based agents can automate complex engineering workflows by interacting with tools, interpreting feedback, and making iterative decisions. This paper introduces TuneBench, a benchmark for evaluating Agentic AI in power system dynamic model calibration, where controller parameters are adjusted to reproduce target dynamic responses. The benchmark reflects a common Transmission System Operator (TSO) practice, in which equivalent models used for Dynamic Security Assessment (DSA) are calibrated against measurements from grid-code compliance tests when proprietary models are unavailable. Two cases of increasing complexity are considered: the governor and Automatic Voltage Regulator (AVR) of a synchronous generator and the control system of a Battery Energy Storage System (BESS), both tuned against active-power and voltage step tests within a budget of 15 simulations. LLM-based agents are compared with random search and Genetic Algorithm (GA) baselines over 10 independent runs. Only the most advanced models, Opus 5.5 and GPT-6, accurately reproduce all target trajectories, with median R2 above 0.995 in every test and tracking errors up to two orders of magnitude lower than the baselines. The agent logs show that these agents autonomously extract dynamic performance indices, such as settling time and damping, and relate them to specific control parameters, decomposing the tuning problem into physically interpretable sub-problems. The benchmark is publicly available at https://github.com/Power-Agent/PowerAgentBench/tree/main/benchmarks/dynamic/level3.

Closest papers

Nearest neighbours by meaning across the whole index.

RestoreBench: Can AI Agents Restore Power Flow Convergence?

cs.AI eess.SY ·2026-08-31 ·Riccardo Mansutti, Andrea Pomarico, Robert Jakob, Qian Zhang, Alberto Berizzi +1 more
benchmarkcode/data similarity 0.70
Abstract

Large Language Model (LLM) agents increasingly automate multi-step engineering workflows through tool use, interpretation of intermediate results, and iterative planning. Diagnosing and resolving non-convergent power flow cases is a promising yet largely unexplored application, as it requires engineering judgment, experimentation, and decision-making within constrained action spaces. We introduce a benchmark that evaluates these capabilities across multiple LLMs and three architectures: \emph{chatbot}, \emph{single agent}, and \emph{multi-agent} systems. The evaluation covers two power grids and 46 cases per grid, each requiring one or more corrective actions to restore convergence. The benchmark defines the simulation environment, observation and action spaces, and evaluation metrics, providing a reproducible foundation for developing agentic AI systems for power system planning and operation. The code is available at https://github.com/Mansutti081/RestoreBench

0.41

An Agentic Framework for Automated Power System Studies and Grid Connection Assessment

cs.AI ·2026-11-02 ·MaoXun Huang, Kshitij Kapoor, Eli Louise
benchmark similarity 0.66
Abstract · 8 pages for the main paper with 6 pages for Appendix

Connecting new generation and demand to the grid depends on two scarce skills: checking applications against a large, changing body of regulation, and running the network studies that decide where and how a project connects. We present Grid Agents, an AI system that brings both together, and the methodology we use to evaluate it. The Grid Ontology is its knowledge system. It represents the regulation, standards and engineering knowledge that grid work depends on, makes it searchable, and evolves as the agents learn from use. Every answer drawn from it rests on evidence the agents cite. Grid Studio is an agent system specialised for power-system engineering. It works in its own engineering environment with domain tools and a study framework, draws on the Grid Ontology, and relies on established solvers for every number it reports. In both, engineers make the decisions. We bring eight published power-system benchmark families, previously scattered across separate papers, into one suite of 519 tasks, and we verify not only each answer but the work behind it, by replaying the agent's tool calls. The Grid Ontology cites the right regulatory evidence more often than a general coding agent or retrieval pipelines, leads on LegalBench-RAG and ranks first on the public leaderboards of the Enterprise RAG Challenge and ViDoRe V3. Grid Studio scores highest on the 519 power-system tasks and is the fastest of all the systems we compare, both by a statistically significant margin. With an optimiser in the loop, its reinforcement packages cost 33-67% less on average than those the agent designs alone.

0.11

Integrating Agentic Artificial Intelligence with High-Performance Computing for Grid Planning

eess.SY ·2026-09-03 ·Samim Konjicija, Slaven Peles
published similarity 0.65
Abstract · 6 pages, 3 figures, 3 tables. To appear in proceedings of EnergyCon 2026

We present AgentiGrid, an agentic artificial intelligence (AI) framework that integrates large language models (LLMs) intelligence and high-performance computing (HPC) to streamline and accelerate the multi-scenario power flow studies. AgentiGrid is an autonomous decision-making agent that proposes parameter modifications, invokes analyses through HPC analysis toolkit ExaGO, interprets results, and determines subsequent actions. ExaGO provides multiple power flow applications that can perform deterministic, stochastic and security constrained optimal power flow analyses. AgentiGrid provides backends to multiple LLMs (OpenAI, Anthropic, Ollama, and Ollama cloud) augmented with context specific and task specific prompts. Key features include interactive mid-search steering, goal-type-aware post-search analysis, and concurrent variant exploration for power flow optimization. A Streamlit-based graphical launcher provides real-time visualization of iteration progress and generates reports in natural language. AgentiGrid is capable of autonomously converging transmission constrained alternating current optimal power flow in under 20 iterations, with near-perfect reliability

0.25

Skill-Based AI Agents for Power-System Studies

eess.SY ·2026-09-30 ·Pavel Etingov, Shuchismita Biswas
similarity 0.63
Abstract · submitted to 2027 IEEE PES Grid Edge Technologies Conference & Exposition

This paper describes a skill-based agentic framework for power-system studies using Model Context Protocol (MCP)-connected engineering tools. A custom MCP server was developed to expose Siemens PTI PSSE functions for power-flow analysis, dynamic simulation, result extraction, and model-validation workflows. Two implementation pathways built on a programmable OpenAI Agents software development kit (SDK) and a Claude Code command-line interface (CLI) were evaluated, both using reusable skills, subagents, MCP tools, data-repository connections, and local shell/Python execution. Both frontier-model-based implementations successfully executed representative study tasks. Success was evaluated based on task completion, output accuracy, and the need for human expert interventions. Results based on public datasets show that agentic systems can greatly accelerate power system dynamic simulation process for transmission planning studies leveraging industry-grade simulation platforms. This points toward a shift in transmission planning practice, where agentic systems could handle routine simulation setup and result extraction, allowing engineers to focus expert judgment on scenario design and interpretation rather than tool operation.

0.22

Large Language Model Assisted Operational Monitoring for Battery Energy Storage System Integrated Power Distribution Networks

cs.AI cs.CLeess.SY ·2026-08-15 ·Azmeer Akhtar, Md Fazley Rafy, Anurag K. Srivastava
similarity 0.63
Abstract

Battery energy storage systems (BESS) are increasingly used in distribution networks for voltage regulation and demand response, which increases the volume and complexity of operational telemetry available to grid operators. This paper presents an AI-enabled monitoring framework that connects a large language model (LLM) interface with a structured telemetry database for BESS-integrated distribution system analysis. Operator questions are submitted in natural language and translated into validated SQL queries using predefined database schema information and approved KPI views. Retrieved measurements, including bus voltages, state of charge, active power, and reactive power, are evaluated against engineering constraints for voltage limits, BESS operation, and demand response tracking. The framework is validated using hardware-in-the-loop co-simulation data from a BESS-equipped distribution feeder operating under reactive power-based voltage control and price-driven demand response. Case studies show that the framework generates valid database queries, identifies repeated voltage violations, detects reactive power overshoot, and evaluates active-power tracking performance. The results show that LLM-assisted monitoring can connect structured grid telemetry with automated engineering assessment for BESS operation analysis.

0.04

Local AI Agents in Power Electronics Simulation: Deployment, Measurements and Costs

cs.AI eess.SY ·2026-11-18 ·Yuzhuo Li, Pasan Gunawardena, Yunwei Li
similarity 0.62
Abstract · 27 pages, 23 figures, 12 tables

Agentic artificial intelligence (AI) is emerging in power electronics, where it drives the simulation loop that engineers used to close largely by hand. In this loop, a converter design, or a study of the network it connects to, is built, run, measured and revised until it satisfies its specification. Such an agent can be served from a cloud API or from an inference device on the premises. For organizations whose designs, customer data or export-controlled material cannot leave the site, and whose simulators are licensed there, local deployment is the natural choice. In such a deployment, the agent becomes an electrical load on the organization's own equipment, i.e., every run consumes power and energy on a local device, generates tokens and takes time. However, existing work focuses mainly on what the agent can achieve with cloud-hosted models. What a local deployment costs in power, energy, tokens and time on real simulation tasks, and what its choices trade against one another, has received little quantitative investigation under comparable conditions. To address this gap, a systematic evaluation framework is presented in this paper. It applies locally served models to agentic power electronics tasks, from the converter level to the system level, and measures their power, energy, token and time costs. Based on these measurements, the capability boundary of a local agent on such tasks and the cost per solved task are characterized. The results are expected to provide practical insight for the use of agentic AI in power electronics and the power systems it serves.

0.30

AutoEnergyDesign: An LLM-Driven Agent for Automatic Energy System Design Optimization

cs.AI stat.ML ·2026-10-26 ·Amedeo Buonanno, Salvatore Fabozzi, Maria Valenti
similarity 0.62
Abstract

The design optimization of building energy systems represents a complex, high-dimensional search problem involving discrete technology selections, continuous sizing parameters, and intricate economic trade-offs in which every candidate must be evaluated by a year-long simulation. This paper presents AutoEnergyDesign, in which a Large Language Model (LLM) agent reasons over past simulation results, proposes configuration changes and learns in context from each outcome, while a git-based mechanism reverts changes that do not improve the design. The agent couples a reasoning model with explicit memories, a constrained action interface, and an environment-grounded keep/revert feedback signal. The simulator covers eight technologies and computes the 25-year Net Present Cost (NPC). On a case study with measured building loads, the agent, instantiated with three Claude models (Haiku 5.5, Sonnet 5.5 and Opus 5.5; ten runs of 50 simulations each), is compared with random search, Tree-structured Parzen Estimator (TPE) Bayesian optimization and a (1+1) evolution strategy that shares the agent's acceptance rule (ten seeds of 300 evaluations). With all three models the agent reaches an NPC of 3.53MEUR, 1.20% and 0.23% below the best results of TPE and of the evolution strategy, and comes within 1% of it after a median of 18 to 27.5 simulations; among the baselines, only the evolution strategy does so, after a median of 192 evaluations. Every decision is recorded, making the search auditable.

0.26

Profile Screening and Frequency-Secure Operating Limit Derivation for Data Center Microgrids

eess.SY ·2026-11-17 ·Zishuo Yang, Yu Ma, Jong Ha Woo, Aaditya Pandey, Wenyuan Tang +2 more
similarity 0.61
Abstract

Rapid power variations from artificial intelligence (AI) workloads pose frequency-security challenges for islanded data-center microgrids. This paper proposes a generator step- response-based framework to rapidly screen candidate load pro- files and derive frequency-secure operating limits for IT workload scheduling. Identified generator responses are superimposed to assess the cumulative frequency effects of successive load changes without repeated dynamic simulations. The framework identifies potentially unsafe intervals and determines allowable load magni- tudes, ramp rates, and minimum event spacing, while accounting for prediction uncertainty and governor limits. Validation on a 2.5 MVA islanded microgrid with eight generator configurations and 14 synthetic AI workload profiles shows that 94.75% of 40,208 assessment windows receive decisive screening labels, with no frequency-violating windows classified as secure.

0.38