CS Paper Radar

DER Allocation without Load Prediction via Reinforcement Learning

Abed AlRahman Al Makdah, Aravind Ramana, Shaofeng Zou, Oliver Kosut, Lalitha Sankar

eess.SY ·submitted 2026-08-17 00:06 UTC · v1
published

Abstract

The growing variability of renewable generation increases the need for fast and flexible grid-balancing mechanisms. Existing frameworks for distributed energy resource aggregations (DERAs) rely on short-term forecasts of net demand, making their performance highly sensitive to prediction errors. In this paper we present a forecast-free reinforcement learning (RL) framework for DERA allocation that learns optimal policies directly from operational data. We model the DERA dynamics as a deterministic linear system and the exogenous net load as a feature-based linear Markov process, capturing short-range temporal dependencies without explicit forecasting. We derive a closed-form expression for the optimal policy, which is learned through a least-squares value iteration (LSVI) algorithm using data collected across episodes. The proposed framework preserves the interpretability and constraint satisfaction of DER model while adapting to stochastic demand variations through data-driven updates. Numerical experiments on real California Independent System Operator (CAISO) net-demand data demonstrate that the learned controller achieves high tracking accuracy and stable regulation across heterogeneous DER aggregators without requiring any demand prediction.

Comment
5 pages. Presented at the 2026 IEEE Power & Energy Society General Meeting (PES GM)

Closest papers

Nearest neighbours by meaning across the whole index.

Comparative Evaluation of MILP, MPC, and Reinforcement Learning for Commercial Battery Dispatch Under Time-of-Use Tariffs

eess.SY ·2026-09-13 ·Hafiz Majid Hussain, Wajiha Samar, Lurian Klein, Pedro Nardelli
benchmark similarity 0.71
Abstract

Battery energy storage systems (BESS) paired with rooftop photovoltaics (PVs) can deliver measurable cost savings under time-of-use (TOU) electricity tariffs; however, the relative performance of model-based and model-free dispatch strategies remains insufficiently benchmarked on full-year, real-world commercial datasets. This paper presents a full-year (2023) comparative evaluation of three BESS dispatch approaches using data from a commercial PV installation operating under a TOU tariff. The examined strategies include: (i) a mixed-integer linear programming (MILP) formulation with perfect foresight, providing an oracle performance benchmark under the assumed model; (ii) a model predictive control (MPC) scheme based on a day-ahead persistence forecast, representing a low-complexity deployable approach; and (iii) a soft actor-critic (SAC) deep reinforcement learning agent trained under causal information constraints. The MILP benchmark achieves an annual cost reduction of 24.6\% relative to a no-storage baseline. The persistence-based MPC approach recovers 99.2\% of this benchmark using only prior-day data. In contrast, the evaluated SAC agent yields an annual cost higher than the no-storage baseline. This outcome is analyzed in the context of known challenges in reinforcement learning for energy systems, including limited observability and reward design. Overall, the results indicate that, for the studied dataset and tariff structure, persistence-based MPC captures nearly all achievable economic benefits under practical deployment constraints, whereas the considered RL configuration does not yield competitive performance under the same information limitations.

0.21

RL-PaO: Prediction as Action in Decision Making under Uncertainty

cs.LG ·2026-09-29 ·Jiahui Feng, Dafang Zhao, Zheng Chen, Zhengmao Li, Lingwei Zhu
similarity 0.69
Abstract

Decision-making under uncertainty often relies on predicted parameters, yet accurate prediction does not necessarily lead to good operational decisions. Aligning prediction with downstream optimization requires learning from the consequences of the decisions those predictions induce. We introduce RL-PaO, a reinforcement learning framework that integrates system formulation, optimization, and decision execution into a single environment. This yields a Markov decision process in which prediction is regarded as action: it shifts the environment to produce subsequent context and reward that explicitly aligns prediction error with realized cost, and learning the optimal policy does not require differentiating through the black-box solver. We evaluate RL-PaO on day-ahead energy scheduling using real historical data. On the test year, RL-PaO achieves the lowest annual cost among the non-oracle baselines, achieving on average $10\%$ cost reduction. Moreover, RL-PaO is capable of further analyses to provide strong interpretability both from the policy evolution perspective and the cost-accuracy trade-off.

0.24

Decision-Focused Learning for Optimal Battery Scheduling Under Load and Photovoltaic Power Uncertainty in Microgrids

eess.SY ·2026-10-10 ·Marios Saleptsis, Antoine Lesage-Landry, Marco Mussetta, Sonia Leva
similarity 0.66
Abstract · 20 pages, 13 figures, submitted to Sustainable Energy, Grids and Networks

The increasing penetration of renewable energy sources in microgrids demands accurate day-ahead forecasting to support optimal battery energy storage system (BESS) scheduling. Yet the conventional predict-then-optimize paradigm trains forecasters on statistical accuracy metrics that are agnostic to the asymmetric and time-varying cost impacts of forecast errors. This work proposes a decision-focused learning (DFL) framework that trains gated recurrent unit forecasters for both load demand and photovoltaic generation directly on the downstream operational cost of a two-stage energy management system, where day-ahead BESS commitments are fixed prior to real-time balancing. Gradients are backpropagated from the real-time operational cost through both optimization stages to the forecaster weights via implicit differentiation, enabling the forecasters to internalize the penalty structure of the scheduling problem. The framework is validated on the MG2Lab experimental microgrid testbed at Politecnico di Milano across two operational scenarios of increasing grid constraint and eight forecasting configurations comprising single-source and combined-forecaster settings. Results illustrate that the DFL-trained pipeline reduces operational cost by 3.78% under normal grid conditions and by 30.60% under grid-constrained conditions relative to the accuracy- optimized baseline, while nearly eliminating load shedding events. Notably, the DFL forecasters exhibit higher statistical errors than their accuracy-trained counterparts yet achieve consistently lower operational costs, confirming that forecast accuracy and decision quality do not share a monotonic relationship in constrained microgrid scheduling.

0.12

Learning Control Policies from Heterogeneous Multi-Horizon Time Series in Battery Energy Management Systems

eess.SY ·2026-09-18 ·Sheng Yin, Vivek Teja Tanjavooru, Holger Hesse, Christoph Goebel
published similarity 0.66
Abstract

This paper introduces Representation-to-Decision (R2D), an end-to-end imitation learning framework that maps heterogeneous multi-horizon time-series inputs directly to battery control decisions through modular Temporal Feature Extractors (TFEs) and a shared latent representation, without an explicit load and PV forecasting step. While accurate forecasting improves prediction quality, optimal control performance remains unguaranteed in prediction-then-optimization pipelines; standard Reinforcement Learning (RL) lacks the long-horizon temporal awareness due to short-window observations, even when forecast signals are available as additional inputs. R2D offers a different perspective: rather than forecasting first and deciding second, it learns to decide directly from raw temporal inputs, with control optimality anchored by an aging-aware Mixed-Integer Linear Programming (MILP) expert through Behavior Cloning (BC). Benchmarked against six controllers on a high-fidelity electro-thermal battery simulation across five industrial sites, R2D achieves 62--77\% of the global clairvoyant optimum, outperforms the tested Model Predictive Control (MPC) and RL benchmarks, and yields battery degradation nearly identical to its MILP teacher across all five sites. Comprehensive ablation studies over temporal backbone, model size, expert formulation, aging-cost weighting, and horizon configuration confirm that an LSTM encoder with a 15-minute single-step control horizon provides the most robust and deployment-ready configuration, and cross-site and single-factor out-of-distribution tests show that generalization to unseen profiles is profile-dependent, with policies trained on high-activity sites transferring most reliably.

0.26

Battery Energy Storage Sizing in PV Ramp-Rate Smoothing using Constraint-Aware Reinforcement Learning-Based Controller

eess.SY ·2026-11-13 ·Arash Omidi, Tanmay Mishra, Mads R. Almassalkhi
benchmark similarity 0.66
Abstract

This paper presents a constraint-aware reinforcement learning (CARL) controller for battery energy storage system (BESS) sizing and photovoltaic (PV) ramp-rate smoothing. Rapid PV power fluctuations caused by cloud transitions can introduce operational and stability challenges in power systems. To mitigate these impacts, grid codes impose ramp-rate limits (RRL) on PV power injected to the grid. BESSs can support such grid codes by absorbing or supplying power during rapid variations. However, the extent to which BESSs can reduce PV-induced variability depends on (i) the BESS power and energy capacities; and (ii) the adopted controller. Thus, BESS sizing and controller design are coupled. Conventional moving-average control can increase battery usage due to filter-induced lag, while ramp-rate control is reactive and does not account for the long-term impact of actions on battery state of charge and future ramp events. Optimization-based methods can reduce BESS requirements by anticipating future PV variations, but their performance depends on accurate PV forecasts. To address these limitations, the CARL controller learns a causal state-feedback policy from historical PV data without requiring forecasts during deployment. Ramp-rate, converter-power, and state-of-charge constraints are incorporated into a causal feasibility interval that guarantees feasible actions by definition. A full-horizon input-convex neural network (ICNN)-based optimization is developed as a noncausal benchmark for the minimum achievable BESS requirements. Evaluation using three real-world PV datasets shows that, at a 10\%/min ramp-rate limit, CARL reduces peak BESS power by 8.7--10.4\% and energy capacity by 47.5--62.6\% relative to conventional ramp-rate control while maintaining the grid-codes compliance.

0.20

Cost-Aware Reinforcement Learning with Action Masking and Projection for Battery Energy Storage Dispatch under Suppressed-Spread Market Shifts

cs.LG cs.AIeess.SY ·2026-09-20 ·Kuanlin Chen, Chen-Wei Kuo, Cheng-En Ou
published similarity 0.66
Abstract · Accepted for publication at IEEE IECON 2026. 6 pages, 3 figures, 3 tables

Battery energy storage system (BESS) dispatch must preserve operational feasibility while declining price spreads reduce the margin available to pay for cycling. We study a proximal policy optimization (PPO) controller whose pre-selection physical action mask and emergency projection are separated from a causal, forecast-informed economic advisory. All forecast-dependent methods receive the same causal 24-step forecast and grid-side settlement. Across five PPO seeds, advice-on net profit is 30.59 and 18.04 USD per 336-hour T1 and T2 window, versus 36.77 and 22.94 USD for proxy-cost MPC; PPO remains below this reference in both periods. Advice raises T2 profit from 16.45 to 18.04 USD while reducing throughput, but is immaterial in T1. On disjoint weekly blocks, PPO is stable under daily, weekly, and blended seasonal forecasts, weakens under persistence, and remains below proxy-cost MPC. Paired diagnostics localize changes to the observed 5-10 USD/MWh regime with mixed SoC-dependent effects. An M0-M6 ablation shows that mask removal sends thousands of infeasible requests to projection, while removing both physical layers exposes ramp violations. The evidence separates economic screening from feasibility enforcement without claiming formal safety, lifecycle-optimal aging, or RL dominance.

0.28

Decision-Focused Learning via Neural Optimization Layers for Battery-Assisted Grid-Import Smoothing in AI Data Centers

eess.SY cs.LGmath.OC ·2026-11-10 ·Xinda Zheng, Canchen Jiang, Frits de Nijs, Edward Lam, Markus Wagner
similarity 0.65
Abstract

Rapid AI data-center power fluctuations challenge grid integration. Sustained grid-import smoothing requires allocating limited battery power and energy under uncertain demand. We formulate risk-aware BESS scheduling that coordinates advance reserve commitments and demand-responsive dispatch, accounting for operating costs, grid-ramp penalties, and their conditional value-at-risk. We develop battery scheduling decision-focused learning (BS-DFL) with a constraint-aware neural optimization layer trained directly from this objective without optimal-solution labels. The trained layer is then frozen to guide demand-predictor learning through scheduling costs and ramping risk, enabling solver-free rolling-horizon scheduling. In a 10,000-minute simulation using loads derived from measured AI workloads, BS-DFL reduces mean absolute minute-to-minute grid-import changes by 71.70% and total scheduling cost by 18.01% relative to an MSE-trained predictor coupled with a mixed-integer linear programming solver, with an average decision time of 18.77 ms.

0.20

Scarcity Pricing for State-of-Charge Management in Real-Time Markets Under Intra-Day Uncertainty

eess.SY ·2026-11-09 ·Shailesh Wasti, Aravind Retna Kumar, Sushant Varghese, Anthony Giacomoni, Parviz Alivand +2 more
similarity 0.65
Abstract

The growth in renewable resources will increase net load variability and forecast error. Energy storage resources can provide flexibility when ramp capability is limited and contribute to load leveling over the day. The challenge for system operators is how to balance these two functions in real time, when updated net load forecasts are available for a limited lookahead horizon. U.S. electricity markets are seeking ways to manage state-of-charge (SoC) within their energy markets to more efficiently utilize these resources. We propose a method to manage energy storage SoC in a real-time market by extending Hogan's (2013) scarcity pricing framework. A day-ahead SoC schedule is modified using existing multi-interval real-time unit commitment engines as net load forecasts change. To address 30-minute-ahead forecast errors, an updated 5-minute SoC target and a dynamically updated shortage price for violating the constraint are provided to each economic dispatch. We simulate PJM's energy markets for 365 days and demonstrate reliability improvements relative to a strategy that enforces the day-ahead schedule. We show cost reductions and improved price formation relative to a strategy that updates target quantities but uses an administratively chosen shortage penalty.

0.24