CS Paper Radar

LLMs are Few-Shot Decision-Makers: Generalized Context-Aware Microgrid Frequency Control through Prompt Decision Transformer

Xu Yang, Chenhui Lin, Haotian Liu, Kaihang Deng, Yunhe Li, Wenchuan Wu

eess.SY cs.AI ·submitted 2026-08-22 09:07 UTC · v1

Abstract

The rapid evolution of energy structures has positioned microgrids as pivotal components of next-generation power systems, offering enhanced resilience and renewable energy integration. However, the inherent low inertia, complex dynamics, and poor model conditions of microgrids necessitate advanced data-driven frequency control strategies. Although reinforcement learning (RL) has demonstrated certain potential and advantages, existing RL methods often struggle with generalization across diverse microgrid configurations and lack adaptability to unseen environments, particularly when explicit system parameters are unavailable. To address these challenges, in this paper, we introduce a novel prompt decision transformer (Prompt-DT) architecture for microgrid frequency control. Unlike traditional approaches that rely on hard-to-obtain environmental characteristic parameters, the proposed method leverages few-shot expert historical trajectories as prompts to guide autonomous perception and adaptive decision-making. In addition, we propose a context-aware training and execution mechanism utilizing self-supervised contrastive learning to enhance environment recognition and prompt utilization efficiency. In addition, a physics-informed prompt design technique that filters prompts based on cumulative reward and frequency volatility is proposed, ensuring high-quality physical guidance during online execution. Finally, to ensure generalization in unseen environments with limited data, we develop a lightweight finetuning approach that achieves performance comparable to full-parameter finetuning with minimal adjustments.

Closest papers

Nearest neighbours by meaning across the whole index.

Converter-Grid Interaction Stability Guaranteed Safe Deep Reinforcement Learning for Energy Storage Systems in Grid Frequency Support

eess.SY ·2026-09-15 ·Fei Liu, Mengfan Zhang, Zhipeng Li, Frede Blaabjerg, Qianwen Xu
similarity 0.61
Abstract · 10 pages, 13 figures. Submitted to IEEE Transactions on Smart Grid

The growing integration of converter interfaced renewable energy resources (RESs) intensifies stability challenges. Energy storage system (ESS) can provide fast and flexible frequency support to mitigate frequency deviations. However, the interface converter of ESS may encounter converter-grid interaction stability issues. This paper proposes a converter-grid interaction stability guaranteed safe DRL (CIS-DRL) method for ESS integrated power systems to achieve frequency regulation. We first obtain a double DNN-based stability region to identify the guaranteed converter-grid interaction stability. Next, a novel converter-grid interaction stability Safe-TD3 (CIS-STD3) algorithm is designed that integrates a stability feasibility projection layer to map unsafe actions into stable action set before execution, enforcing converter-grid interaction stability as a hard constraint throughout learning process. The proposed approach enables ESS for grid frequency support with 100% converter-grid interaction stability without violations. Experimental results show that the proposed CIS-DRL method achieves improved frequency regulation performance while preventing unstable operating points, demonstrating its practical applicability for real time ESS frequency support.

0.27

Value-oriented policy distillation for interpretable wind farm power control

cs.AI eess.SY ·2026-11-05 ·Max Bruninx, Timothy Verstraeten, Jan Decuyper, Ann Nowé, Jan Helsen
similarity 0.61
Abstract

Wind farm power control can help the system maintain balance and avoid blackouts. In Europe, this can be achieved through implicit balancing, where a wind farm adjusts its power setpoint in anticipation of the imbalance price using information published by the Transmission System Operator. Thereby, the wind farm helps restore system balance while also reducing its own imbalance costs resulting from forecast errors. However, implicit balancing is a challenging problem since the settlement price is unknown in advance and can be impacted by the pursued control strategy. Reinforcement learning methods could help to solve this problem, but lack the interpretability needed to be deployed on real wind farms. Therefore, we propose a policy distillation framework using explainable boosting machines (EBMs), a variant of generalized additive models, to obtain a transparent control policy. To increase the value captured by the EBM model, we introduce a value-oriented perspective to policy distillation inspired by research in the field of energy forecasting. Furthermore, we suggest to include expert knowledge during model design, such that the control strategy is also justifiable to stakeholders. Using real data of the Belgian grid and an offshore wind farm, we find that our value-oriented and expert-based controller can reduce imbalance costs by 82%, while outperforming a price threshold controller as well as a policy distillation baseline using decision trees.

0.21

LLM-Based Hierarchical Coordinated Control with Continuation-Aware Policy Learning

cs.AI ·2026-08-15 ·Changhong He, Jinda Gao, Xinkuan Liu, Le Zhang, Xizi Luo +1 more
similarity 0.59
Abstract · 30 pages, 5 figures

Coordinating multiple interacting units in complex engineering systems is challenging when system interactions are difficult to model, operational information is heterogeneous, and low-level actions must satisfy strict constraints. We propose an LLM-based hierarchical framework in which the LLM coordinates interacting units based on heterogeneous operational context, while task-specific controllers or optimizers generate executable and constraint-aware actions. We further introduce Continuation-Aware GRPO to capture the consequences of coordination decisions over subsequent control intervals. Rather than judging a decision only by its immediate outcome, the method also evaluates how the system evolves afterward under the current policy. We validate the framework on multi-ramp traffic control and virtual power plant (VPP) energy management, using simplified system models for training and more realistic simulators for evaluation. Across both tasks, the proposed method consistently outperforms direct task-specific control and optimization, end-to-end reinforcement learning, rule-based and RL-based hierarchical coordination, and prompting-only LLM coordinators, demonstrating the value of heterogeneous-context reasoning, hierarchical execution, and continuation-aware policy learning.

0.10

Headroom-Aware Stochastic Adaptive Model Predictive Control for Load Frequency Control in Microgrids

eess.SY ·2026-09-04 ·Erfan Mehdipour Abadi, Shuo Yuan, Le Yi Wang, Caisheng Wang, Feng Lin
similarity 0.59
Abstract

As the penetration of inverter-based resources (IBRs) increases in microgrids, they are increasingly expected to play a greater role in load frequency control (LFC). Model predictive control (MPC) is attractive for LFC because it incorporates system dynamics and operational constraints. However, most MPC-based LFC formulations rely on fixed reserve headroom based on forecasted renewable availability or storage systems. Under short-term renewable intermittency, IBR headroom is stochastic and time-varying, causing optimal control commands to exceed the physically deliverable regulation capability and cause stochastic saturation. This control-actuator mismatch degrades LFC performance. Accordingly, this study develops two headroom-aware strategies for PV-dominated microgrids. First, stochastic headroom constrained MPC (SHCMPC) incorporates headroom predictions through time-varying input constraints to enforce control feasibility. Second, stochastic adaptive MPC (SAMPC) embeds headroom awareness into the MPC objective function by adaptively penalizing control actions based on predicted headroom, reducing reliance on units with limited headroom without hard time-varying constraints. Simulation results show that stochastic saturation degrades conventional MPC-based LFC, particularly under tight reserve margins. Both strategies improve regulation performance. SHCMPC eliminates saturation events, while SAMPC achieves substantial saturation mitigation with lower computational effort, offering a computationally efficient alternative for real-time LFC under stochastic renewable availability.

0.22

Physics-Informed Graph-Based Safe RL with Constraint-Specific Critics for Community Battery Coordination in Distribution Network

cs.AI eess.SYmath.OC ·2026-10-24 ·Chenghao Huang, Yue Yang, Hao Wang
top venue similarity 0.59
Abstract

High penetrations of distributed energy resources (DERs) and fast-varying electric vehicle (EV) charging intensify voltage and line-limit violations in distribution networks (DNs), making safe, low-cost operation increasingly challenging. This paper proposes a physics-informed safe deep reinforcement learning (DRL) framework for coordinating community battery energy storage systems to mitigate constraint violations while minimizing operating cost. To represent nonlocal spatiotemporal interactions, we develop a dual-level physics-informed graph representation in which line impedance and capacity shape spatial information propagation, while power-balance, voltage-profile, and flow-reconstruction losses regularize the latent state toward control-relevant physical consistency. Building on this representation, we develop Lagrangian Dedicated-Critic PPO (LDCPPO), which implements constraint-specific value decomposition: independent reward and cost critics preserve heterogeneous voltage- and line-loading-related learning signals and combine their advantages only during the Lagrangian policy update. Case studies on DNs with high rooftop photovoltaic penetration and EV charging demonstrate that the proposed method reduces the key voltage-violation degree by up to 80.5% compared with the tested baselines, while also achieving the lowest voltage-violation count and line-overload degree with competitive operating cost.

0.31

Learning Control Policies from Heterogeneous Multi-Horizon Time Series in Battery Energy Management Systems

eess.SY ·2026-09-18 ·Sheng Yin, Vivek Teja Tanjavooru, Holger Hesse, Christoph Goebel
published similarity 0.58
Abstract

This paper introduces Representation-to-Decision (R2D), an end-to-end imitation learning framework that maps heterogeneous multi-horizon time-series inputs directly to battery control decisions through modular Temporal Feature Extractors (TFEs) and a shared latent representation, without an explicit load and PV forecasting step. While accurate forecasting improves prediction quality, optimal control performance remains unguaranteed in prediction-then-optimization pipelines; standard Reinforcement Learning (RL) lacks the long-horizon temporal awareness due to short-window observations, even when forecast signals are available as additional inputs. R2D offers a different perspective: rather than forecasting first and deciding second, it learns to decide directly from raw temporal inputs, with control optimality anchored by an aging-aware Mixed-Integer Linear Programming (MILP) expert through Behavior Cloning (BC). Benchmarked against six controllers on a high-fidelity electro-thermal battery simulation across five industrial sites, R2D achieves 62--77\% of the global clairvoyant optimum, outperforms the tested Model Predictive Control (MPC) and RL benchmarks, and yields battery degradation nearly identical to its MILP teacher across all five sites. Comprehensive ablation studies over temporal backbone, model size, expert formulation, aging-cost weighting, and horizon configuration confirm that an LSTM encoder with a 15-minute single-step control horizon provides the most robust and deployment-ready configuration, and cross-site and single-factor out-of-distribution tests show that generalization to unseen profiles is profile-dependent, with policies trained on high-activity sites transferring most reliably.

0.26

Stability-Constrained Imitation Learning for Performance-Guaranteed Neural Feedback Frequency Control in Microgrids

eess.SY ·2026-11-17 ·Sajid Kamal, Yichen Zhang, Feng Qiu, Tianqiao Zhao
similarity 0.58
Abstract · 10 pages

Guaranteeing performance for frequency control is challenging within traditional control paradigms, as it demands precise disturbance estimation and real-time control signal computation within a timeframe of seconds. This paper addresses this challenge by proposing a stability-certified imitation learning framework for frequency regulation in low-inertia microgrids. Based on the frequency response model of the microgrid, an offline numerical optimal control is employed to generate near-optimal pairs of state and control dataset, which are subsequently used to train a deep neural network (DNN) feedback controller. To ensure the closed-loop stability, the DNN controller is trained with a set of Lyapunov-based stability constraints derived using absolute stability theory. Such an imitation learning paradigm can significantly improve the training efficiency, while the resulting DNN controller acts as an end-to-end function that simultaneously handles disturbance estimation and control optimization. The proposed framework is validated on a modified IEEE 33-node diesel-wind microgrid. The DNN controllers are deployed in wind turbine generators (WTGs) to provide decentralized, intelligent supplementary active power support to always maintain frequency deviations within predefined safety bounds under different disturbances while preserving closed-loop stability.

0.34

Learning-Based Augmentation and Adaptation for Grid Sim-to-Real Model Discrepancy

eess.SY ·2026-09-18 ·Sayak Mukherjee, Kyung-Bin Kwon, Ramij R. Hossain, Marcelo Elizondo
benchmark similarity 0.58
Abstract · 12 pages, 5 figures, 3 tables

Modern power systems can encounter increased discrepancy between the operators' simulation model and the actual true dynamics of the grid, driven by uncertainties caused by integration of new inverter-based resources (IBRs), large loads, unmodeled dynamics, parameter drifts, etc., to name a few. All of these impact the control room operations, where some critical oscillations may not be captured during the transient studies. To circumvent these issues, we propose a learning-augmented hybrid approach where the operator simulation model is supplemented with artificial intelligence (AI)-learned residual models using the phasor measurement unit (PMU)/ point-on-wave (PoW) based sensed trajectory data. The physics-based operator model provides interpretability and structural consistency, while the learned residual captures discrepancies caused by non-idealities. The learned model employs advanced neural architectures and consists of a backbone encoder and multi-head decoder layers for heterogeneous grid channels. Subsequently, we formulated a continual learning-motivated adaptation framework such that the baseline residual AI model can also be updated when the underlying real grid model changes in future conditions. Extensive numerical simulations are performed on the IEEE 68-bus benchmark model with a diverse set of disturbances, and different state-of-the-art predictive architectures involving recurrent learners, latent neural ODEs, and transformers are explored to demonstrate both residual learning and adaptation capabilities.

0.28