CS Paper Radar

Battery Energy Storage Sizing in PV Ramp-Rate Smoothing using Constraint-Aware Reinforcement Learning-Based Controller

Arash Omidi, Tanmay Mishra, Mads R. Almassalkhi

eess.SY ·submitted 2026-11-13 00:22 UTC · v1
benchmark

Abstract

This paper presents a constraint-aware reinforcement learning (CARL) controller for battery energy storage system (BESS) sizing and photovoltaic (PV) ramp-rate smoothing. Rapid PV power fluctuations caused by cloud transitions can introduce operational and stability challenges in power systems. To mitigate these impacts, grid codes impose ramp-rate limits (RRL) on PV power injected to the grid. BESSs can support such grid codes by absorbing or supplying power during rapid variations. However, the extent to which BESSs can reduce PV-induced variability depends on (i) the BESS power and energy capacities; and (ii) the adopted controller. Thus, BESS sizing and controller design are coupled. Conventional moving-average control can increase battery usage due to filter-induced lag, while ramp-rate control is reactive and does not account for the long-term impact of actions on battery state of charge and future ramp events. Optimization-based methods can reduce BESS requirements by anticipating future PV variations, but their performance depends on accurate PV forecasts. To address these limitations, the CARL controller learns a causal state-feedback policy from historical PV data without requiring forecasts during deployment. Ramp-rate, converter-power, and state-of-charge constraints are incorporated into a causal feasibility interval that guarantees feasible actions by definition. A full-horizon input-convex neural network (ICNN)-based optimization is developed as a noncausal benchmark for the minimum achievable BESS requirements. Evaluation using three real-world PV datasets shows that, at a 10\%/min ramp-rate limit, CARL reduces peak BESS power by 8.7--10.4\% and energy capacity by 47.5--62.6\% relative to conventional ramp-rate control while maintaining the grid-codes compliance.

Closest papers

Nearest neighbours by meaning across the whole index.

Comparative Evaluation of MILP, MPC, and Reinforcement Learning for Commercial Battery Dispatch Under Time-of-Use Tariffs

eess.SY ·2026-09-13 ·Hafiz Majid Hussain, Wajiha Samar, Lurian Klein, Pedro Nardelli
benchmark similarity 0.73
Abstract

Battery energy storage systems (BESS) paired with rooftop photovoltaics (PVs) can deliver measurable cost savings under time-of-use (TOU) electricity tariffs; however, the relative performance of model-based and model-free dispatch strategies remains insufficiently benchmarked on full-year, real-world commercial datasets. This paper presents a full-year (2023) comparative evaluation of three BESS dispatch approaches using data from a commercial PV installation operating under a TOU tariff. The examined strategies include: (i) a mixed-integer linear programming (MILP) formulation with perfect foresight, providing an oracle performance benchmark under the assumed model; (ii) a model predictive control (MPC) scheme based on a day-ahead persistence forecast, representing a low-complexity deployable approach; and (iii) a soft actor-critic (SAC) deep reinforcement learning agent trained under causal information constraints. The MILP benchmark achieves an annual cost reduction of 24.6\% relative to a no-storage baseline. The persistence-based MPC approach recovers 99.2\% of this benchmark using only prior-day data. In contrast, the evaluated SAC agent yields an annual cost higher than the no-storage baseline. This outcome is analyzed in the context of known challenges in reinforcement learning for energy systems, including limited observability and reward design. Overall, the results indicate that, for the studied dataset and tariff structure, persistence-based MPC captures nearly all achievable economic benefits under practical deployment constraints, whereas the considered RL configuration does not yield competitive performance under the same information limitations.

0.21

Hierarchical Control via MPC-RL for Multi-Timescale Battery Systems

eess.SY ·2026-10-02 ·Rasa Pourjam, Ehecatl Antonio del Río Chanona, Paulina Quintanilla
similarity 0.69
Abstract

Multi-timescale systems present a fundamental challenge, where fast operational decisions must coexist with long-horizon sustainability targets. In this work, we propose a new hierarchical control framework via Model Predictive Control (MPC) and Reinforcement Learning (RL) to separate decision-making on two distinct timescales. The high-level MPC optimizes long-horizon setpoints at the slow dynamic and on a fast timescale, a low-level pretrained RL agent tracks these setpoints in real time to maximize short-term objectives. RL is introduced to learn nonlinear control policies, without relying on model linearizations or requiring the heavy online computation from solving repeated optimal control problems. The framework is applied to a Battery Energy Storage System (BESS) operating in frequency regulation markets to balance fast profit opportunities (seconds) and slow battery degradation (weeks to months). The design employs a degradation-aware RL agent trained offline to generate safe long-horizon setpoints, and a degradation-unaware agent fine-tuned from it for fast runtime setpoint tracking. Compared to MPC baselines, the proposed approach successfully extends battery lifetime by 84% and increases operational profit by 34%.

0.17

Grid-Mode-Aware Model Predictive Control of Hybrid Energy Storage Systems for AI Data Center Power Smoothing

eess.SY ·2026-09-03 ·Xin Chen
similarity 0.69
Abstract

To facilitate the grid-friendly integration of highly variable AI data center loads, this paper proposes a grid-mode-aware model predictive control (G-MPC) framework for managing a hybrid energy storage system (HESS) to smooth grid-side power demand. The framework optimally coordinates a battery energy storage system (BESS) and a supercapacitor (SC) by solving a multi-step optimization problem in a receding-horizon manner. In particular, band-pass filter dynamics are directly embedded in the G-MPC formulation to extract and suppress grid-side power components associated with vulnerable grid oscillatory modes, thus mitigating load-induced grid oscillations. The resulting G-MPC optimization jointly minimizes violations of grid-side power-envelope, ramp-rate, and modal-power requirements and the degradation and power-ramping costs of the BESS and SC, while satisfying power limits, state-of-charge limits, and other operational constraints. To enable real-time implementation, a fix-and-re-optimize algorithm is developed to solve each G-MPC problem efficiently while preventing simultaneous charging and discharging. Extensive simulations demonstrate the effectiveness, flexibility, and computational efficiency of the proposed framework. The results also highlight the importance of explicitly suppressing power components associated with vulnerable grid modes, rather than merely reducing overall load variations, to effectively mitigate grid oscillations.

0.40

Decision-Focused Learning for Optimal Battery Scheduling Under Load and Photovoltaic Power Uncertainty in Microgrids

eess.SY ·2026-10-10 ·Marios Saleptsis, Antoine Lesage-Landry, Marco Mussetta, Sonia Leva
similarity 0.69
Abstract · 20 pages, 13 figures, submitted to Sustainable Energy, Grids and Networks

The increasing penetration of renewable energy sources in microgrids demands accurate day-ahead forecasting to support optimal battery energy storage system (BESS) scheduling. Yet the conventional predict-then-optimize paradigm trains forecasters on statistical accuracy metrics that are agnostic to the asymmetric and time-varying cost impacts of forecast errors. This work proposes a decision-focused learning (DFL) framework that trains gated recurrent unit forecasters for both load demand and photovoltaic generation directly on the downstream operational cost of a two-stage energy management system, where day-ahead BESS commitments are fixed prior to real-time balancing. Gradients are backpropagated from the real-time operational cost through both optimization stages to the forecaster weights via implicit differentiation, enabling the forecasters to internalize the penalty structure of the scheduling problem. The framework is validated on the MG2Lab experimental microgrid testbed at Politecnico di Milano across two operational scenarios of increasing grid constraint and eight forecasting configurations comprising single-source and combined-forecaster settings. Results illustrate that the DFL-trained pipeline reduces operational cost by 3.78% under normal grid conditions and by 30.60% under grid-constrained conditions relative to the accuracy- optimized baseline, while nearly eliminating load shedding events. Notably, the DFL forecasters exhibit higher statistical errors than their accuracy-trained counterparts yet achieve consistently lower operational costs, confirming that forecast accuracy and decision quality do not share a monotonic relationship in constrained microgrid scheduling.

0.12

Electric Vehicle Scheduling and Carbon-Aware Energy Dispatch for Charging Stations

eess.SY ·2026-10-27 ·Fulong Yao, Yiming Xu, Maurizio Albano, Liana Cipcigan, Naeima Hamed +2 more
similarity 0.68
Abstract

Electric vehicle (EV) charging stations integrating photovoltaic (PV) generation, battery storage, and bidirectional grid exchange must coordinate access to limited chargers with energy supply while considering both energy cost and carbon emissions. This paper proposes a solver-free greedy heuristic (SFGH) for dynamic vehicle scheduling (VS) and a twin delayed deep deterministic policy gradient (TD3) controller for carbon-aware energy dispatch (ED). SFGH assigns EVs according to reductions in service delays, and TD3 controls battery operation to balance energy costs and carbon emissions while meeting the resulting charging demand. The model allows both PV and battery exports, accounting for export revenue and carbon credits. Simulations on year-long case studies show that SFGH--TD3 achieves the lowest accumulated service delay, energy cost, and carbon emissions among the evaluated heuristic and multi-agent deep reinforcement learning (DRL) configurations.

0.24

Decision-Focused Learning via Neural Optimization Layers for Battery-Assisted Grid-Import Smoothing in AI Data Centers

eess.SY cs.LGmath.OC ·2026-11-10 ·Xinda Zheng, Canchen Jiang, Frits de Nijs, Edward Lam, Markus Wagner
similarity 0.66
Abstract

Rapid AI data-center power fluctuations challenge grid integration. Sustained grid-import smoothing requires allocating limited battery power and energy under uncertain demand. We formulate risk-aware BESS scheduling that coordinates advance reserve commitments and demand-responsive dispatch, accounting for operating costs, grid-ramp penalties, and their conditional value-at-risk. We develop battery scheduling decision-focused learning (BS-DFL) with a constraint-aware neural optimization layer trained directly from this objective without optimal-solution labels. The trained layer is then frozen to guide demand-predictor learning through scheduling costs and ramping risk, enabling solver-free rolling-horizon scheduling. In a 10,000-minute simulation using loads derived from measured AI workloads, BS-DFL reduces mean absolute minute-to-minute grid-import changes by 71.70% and total scheduling cost by 18.01% relative to an MSE-trained predictor coupled with a mixed-integer linear programming solver, with an average decision time of 18.77 ms.

0.20

DER Allocation without Load Prediction via Reinforcement Learning

eess.SY ·2026-08-17 ·Abed AlRahman Al Makdah, Aravind Ramana, Shaofeng Zou, Oliver Kosut, Lalitha Sankar
published similarity 0.66
Abstract · 5 pages. Presented at the 2026 IEEE Power & Energy Society General Meeting (PES GM)

The growing variability of renewable generation increases the need for fast and flexible grid-balancing mechanisms. Existing frameworks for distributed energy resource aggregations (DERAs) rely on short-term forecasts of net demand, making their performance highly sensitive to prediction errors. In this paper we present a forecast-free reinforcement learning (RL) framework for DERA allocation that learns optimal policies directly from operational data. We model the DERA dynamics as a deterministic linear system and the exogenous net load as a feature-based linear Markov process, capturing short-range temporal dependencies without explicit forecasting. We derive a closed-form expression for the optimal policy, which is learned through a least-squares value iteration (LSVI) algorithm using data collected across episodes. The proposed framework preserves the interpretability and constraint satisfaction of DER model while adapting to stochastic demand variations through data-driven updates. Numerical experiments on real California Independent System Operator (CAISO) net-demand data demonstrate that the learned controller achieves high tracking accuracy and stable regulation across heterogeneous DER aggregators without requiring any demand prediction.

0.37

Frequency-Calibrated Decision-Focused Learning for BESS Dispatch Under Renewable Uncertainty

eess.SY ·2026-10-30 ·Zihan Meng, Chenxi Hu, Xin Cui, Francis Yunhe Hou
hot direction similarity 0.66
Abstract

Battery energy storage system (BESS) flexibility depends on state of charge and intertemporal operating constraints. Renewable forecast errors of similar magnitudes can lead to different BESS dispatch results. An error may alter early charging or discharging decisions and reduce the flexibility available for subsequent regulation, while another may have little effect on the resulting schedule. To account for these unequal consequences, this paper proposes Frequency-Calibrated Decision-Focused Learning (FC-DFL), which trains renewable predictors using the realized operating costs of their induced BESS schedules. Gaussian smoothing and solver-based zeroth-order estimation provide training feedback without requiring differentiating through the non-differentiable mixed-integer BESS dispatch problem. The same decision feedback is further combined with prediction sensitivity to develop a decision-oriented XAI method that identifies the input features and forecast periods most consequential to BESS operation. Case studies using German wind and solar data show that FC-DFL reduces the integrated absolute frequency-deviation metric by 6.05% and decision regret by 23.92% relative to accuracy-based predict-then-optimize method. The XAI analysis further demonstrates that the identified high-impact forecast periods produce 2.84 times larger average absolute changes in realized operating cost than low-impact periods. These findings highlight the importance of incorporating downstream decision-making outcomes into renewable forecasting and demonstrate the value of FC-DFL in improving BESS regulation and providing operationally meaningful forecast explanations.

0.24