CS Paper Radar

Learning Control Policies from Heterogeneous Multi-Horizon Time Series in Battery Energy Management Systems

Sheng Yin, Vivek Teja Tanjavooru, Holger Hesse, Christoph Goebel

eess.SY ·submitted 2026-09-18 20:03 UTC · v1
published

Abstract

This paper introduces Representation-to-Decision (R2D), an end-to-end imitation learning framework that maps heterogeneous multi-horizon time-series inputs directly to battery control decisions through modular Temporal Feature Extractors (TFEs) and a shared latent representation, without an explicit load and PV forecasting step. While accurate forecasting improves prediction quality, optimal control performance remains unguaranteed in prediction-then-optimization pipelines; standard Reinforcement Learning (RL) lacks the long-horizon temporal awareness due to short-window observations, even when forecast signals are available as additional inputs. R2D offers a different perspective: rather than forecasting first and deciding second, it learns to decide directly from raw temporal inputs, with control optimality anchored by an aging-aware Mixed-Integer Linear Programming (MILP) expert through Behavior Cloning (BC). Benchmarked against six controllers on a high-fidelity electro-thermal battery simulation across five industrial sites, R2D achieves 62--77\% of the global clairvoyant optimum, outperforms the tested Model Predictive Control (MPC) and RL benchmarks, and yields battery degradation nearly identical to its MILP teacher across all five sites. Comprehensive ablation studies over temporal backbone, model size, expert formulation, aging-cost weighting, and horizon configuration confirm that an LSTM encoder with a 15-minute single-step control horizon provides the most robust and deployment-ready configuration, and cross-site and single-factor out-of-distribution tests show that generalization to unseen profiles is profile-dependent, with policies trained on high-activity sites transferring most reliably.

Closest papers

Nearest neighbours by meaning across the whole index.

Hierarchical Control via MPC-RL for Multi-Timescale Battery Systems

eess.SY ·2026-10-02 ·Rasa Pourjam, Ehecatl Antonio del Río Chanona, Paulina Quintanilla
similarity 0.76
Abstract

Multi-timescale systems present a fundamental challenge, where fast operational decisions must coexist with long-horizon sustainability targets. In this work, we propose a new hierarchical control framework via Model Predictive Control (MPC) and Reinforcement Learning (RL) to separate decision-making on two distinct timescales. The high-level MPC optimizes long-horizon setpoints at the slow dynamic and on a fast timescale, a low-level pretrained RL agent tracks these setpoints in real time to maximize short-term objectives. RL is introduced to learn nonlinear control policies, without relying on model linearizations or requiring the heavy online computation from solving repeated optimal control problems. The framework is applied to a Battery Energy Storage System (BESS) operating in frequency regulation markets to balance fast profit opportunities (seconds) and slow battery degradation (weeks to months). The design employs a degradation-aware RL agent trained offline to generate safe long-horizon setpoints, and a degradation-unaware agent fine-tuned from it for fast runtime setpoint tracking. Compared to MPC baselines, the proposed approach successfully extends battery lifetime by 84% and increases operational profit by 34%.

0.17

Comparative Evaluation of MILP, MPC, and Reinforcement Learning for Commercial Battery Dispatch Under Time-of-Use Tariffs

eess.SY ·2026-09-13 ·Hafiz Majid Hussain, Wajiha Samar, Lurian Klein, Pedro Nardelli
benchmark similarity 0.73
Abstract

Battery energy storage systems (BESS) paired with rooftop photovoltaics (PVs) can deliver measurable cost savings under time-of-use (TOU) electricity tariffs; however, the relative performance of model-based and model-free dispatch strategies remains insufficiently benchmarked on full-year, real-world commercial datasets. This paper presents a full-year (2023) comparative evaluation of three BESS dispatch approaches using data from a commercial PV installation operating under a TOU tariff. The examined strategies include: (i) a mixed-integer linear programming (MILP) formulation with perfect foresight, providing an oracle performance benchmark under the assumed model; (ii) a model predictive control (MPC) scheme based on a day-ahead persistence forecast, representing a low-complexity deployable approach; and (iii) a soft actor-critic (SAC) deep reinforcement learning agent trained under causal information constraints. The MILP benchmark achieves an annual cost reduction of 24.6\% relative to a no-storage baseline. The persistence-based MPC approach recovers 99.2\% of this benchmark using only prior-day data. In contrast, the evaluated SAC agent yields an annual cost higher than the no-storage baseline. This outcome is analyzed in the context of known challenges in reinforcement learning for energy systems, including limited observability and reward design. Overall, the results indicate that, for the studied dataset and tariff structure, persistence-based MPC captures nearly all achievable economic benefits under practical deployment constraints, whereas the considered RL configuration does not yield competitive performance under the same information limitations.

0.21

Safe Receding Horizon Mixed-Integer Differentiable Predictive Control for Degradation-Aware Battery Dispatch

eess.SY ·2026-09-23 ·Eshagh Safarzadeh Ravajiri, Jan Drgona, Benjamin F. Hobbs
publishedbenchmark similarity 0.72
Abstract · 6 pages, accepted to the 65th IEEE Conference on Decision and Control (CDC 2026)

We present a safe receding-horizon mixed-integer differentiable predictive control methodology for residential battery energy storage dispatch that combines neural-network speed with recursive feasibility guarantees. Unlike open-loop learning-to-optimize methods, it incorporates real-time state-of-charge feedback and sinusoidal time-of-day conditioning, enabling closed-loop re-planning at every timestep without re-solving a mixed-integer program. A differentiable rainflow cycle-counting layer enables self-supervised training of the mixed-integer policy on exact degradation physics. The controller is a hybrid closed-loop system pairing a neural mode-selection and continuous-action policy with a quadratic-programming safety filter that guarantees recursive feasibility independent of network weights or mode optimality. We establish mode-conditioned Lipschitz continuity and a conditional regret decomposition into training-quality, mode-mismatch, and forecast-error terms. On a 7-day net-metering evaluation, the method attains a 6.9% cost gap versus the closed-loop mixed-integer MPC benchmark with a 25x speedup (0.11 s vs 2.7 s per step), while average regret rises by under 4% across 0-30% forecast noise. The learned and benchmark modes agree at every step, so the bound reduces to its training-quality and forecast-error terms, both empirically validated.

0.19

Cost-Aware Reinforcement Learning with Action Masking and Projection for Battery Energy Storage Dispatch under Suppressed-Spread Market Shifts

cs.LG cs.AIeess.SY ·2026-09-20 ·Kuanlin Chen, Chen-Wei Kuo, Cheng-En Ou
published similarity 0.70
Abstract · Accepted for publication at IEEE IECON 2026. 6 pages, 3 figures, 3 tables

Battery energy storage system (BESS) dispatch must preserve operational feasibility while declining price spreads reduce the margin available to pay for cycling. We study a proximal policy optimization (PPO) controller whose pre-selection physical action mask and emergency projection are separated from a causal, forecast-informed economic advisory. All forecast-dependent methods receive the same causal 24-step forecast and grid-side settlement. Across five PPO seeds, advice-on net profit is 30.59 and 18.04 USD per 336-hour T1 and T2 window, versus 36.77 and 22.94 USD for proxy-cost MPC; PPO remains below this reference in both periods. Advice raises T2 profit from 16.45 to 18.04 USD while reducing throughput, but is immaterial in T1. On disjoint weekly blocks, PPO is stable under daily, weekly, and blended seasonal forecasts, weakens under persistence, and remains below proxy-cost MPC. Paired diagnostics localize changes to the observed 5-10 USD/MWh regime with mixed SoC-dependent effects. An M0-M6 ablation shows that mask removal sends thousands of infeasible requests to projection, while removing both physical layers exposes ramp violations. The evidence separates economic screening from feasibility enforcement without claiming formal safety, lifecycle-optimal aging, or RL dominance.

0.28

Decision-Focused Learning for Optimal Battery Scheduling Under Load and Photovoltaic Power Uncertainty in Microgrids

eess.SY ·2026-10-10 ·Marios Saleptsis, Antoine Lesage-Landry, Marco Mussetta, Sonia Leva
similarity 0.68
Abstract · 20 pages, 13 figures, submitted to Sustainable Energy, Grids and Networks

The increasing penetration of renewable energy sources in microgrids demands accurate day-ahead forecasting to support optimal battery energy storage system (BESS) scheduling. Yet the conventional predict-then-optimize paradigm trains forecasters on statistical accuracy metrics that are agnostic to the asymmetric and time-varying cost impacts of forecast errors. This work proposes a decision-focused learning (DFL) framework that trains gated recurrent unit forecasters for both load demand and photovoltaic generation directly on the downstream operational cost of a two-stage energy management system, where day-ahead BESS commitments are fixed prior to real-time balancing. Gradients are backpropagated from the real-time operational cost through both optimization stages to the forecaster weights via implicit differentiation, enabling the forecasters to internalize the penalty structure of the scheduling problem. The framework is validated on the MG2Lab experimental microgrid testbed at Politecnico di Milano across two operational scenarios of increasing grid constraint and eight forecasting configurations comprising single-source and combined-forecaster settings. Results illustrate that the DFL-trained pipeline reduces operational cost by 3.78% under normal grid conditions and by 30.60% under grid-constrained conditions relative to the accuracy- optimized baseline, while nearly eliminating load shedding events. Notably, the DFL forecasters exhibit higher statistical errors than their accuracy-trained counterparts yet achieve consistently lower operational costs, confirming that forecast accuracy and decision quality do not share a monotonic relationship in constrained microgrid scheduling.

0.12

Automated Feature Engineering, AutoML, and Decision-Focused Learning for Improved Energy Consumption Forecasting

cs.AI ·2026-09-28 ·Nasser Alkhulaifi
similarity 0.67
Abstract · PhD thesis, School of Computer Science, University of Nottingha, United Kingdom

The rising cost and demand for energy, together with environmental sustainability goals, create major challenges for energy management. Energy Consumption Forecasting (ECF) supports planning by predicting future consumption, but Machine Learning (ML) models for ECF often depend on expert-driven Feature Engineering (FE). This thesis addresses that dependence through three contributions. First, it establishes and evaluates a comprehensive FE pipeline for ECF and investigates domain-specific features. Second, it introduces AutoEnergy, a domain-tailored automated FE algorithm that generates interpretable features from timestamps and lagged consumption and integrates with AutoML for end-to-end ECF modelling. Across eighteen real-world energy datasets spanning residential, commercial, industrial, renewable, and grid domains, AutoEnergy reduces forecasting error by 19.52%-84.72% relative to baseline AutoML and established automated FE methods, while running 1.31-4.41 times faster, with gains varying by dataset. Third, AutoEnergy is integrated with Decision-Focused Learning (DFL) for a Battery Energy Storage System problem, jointly forecasting electricity prices and demand while optimising charging and discharging decisions. On a real-world UK property dataset, this approach reduces operating costs by 22.9%-56.5% compared with the same DFL models without automated FE. Overall, the results show that domain-specific automated FE can reduce reliance on manual feature design, improve forecasting accuracy, and translate predictive gains into measurable operational benefits in energy management.

0.14

RL-PaO: Prediction as Action in Decision Making under Uncertainty

cs.LG ·2026-09-29 ·Jiahui Feng, Dafang Zhao, Zheng Chen, Zhengmao Li, Lingwei Zhu
similarity 0.67
Abstract

Decision-making under uncertainty often relies on predicted parameters, yet accurate prediction does not necessarily lead to good operational decisions. Aligning prediction with downstream optimization requires learning from the consequences of the decisions those predictions induce. We introduce RL-PaO, a reinforcement learning framework that integrates system formulation, optimization, and decision execution into a single environment. This yields a Markov decision process in which prediction is regarded as action: it shifts the environment to produce subsequent context and reward that explicitly aligns prediction error with realized cost, and learning the optimal policy does not require differentiating through the black-box solver. We evaluate RL-PaO on day-ahead energy scheduling using real historical data. On the test year, RL-PaO achieves the lowest annual cost among the non-oracle baselines, achieving on average $10\%$ cost reduction. Moreover, RL-PaO is capable of further analyses to provide strong interpretability both from the policy evolution perspective and the cost-accuracy trade-off.

0.24

Accelerating Nonlinear MPC by Behavior Cloning with Separated Timescales: Application to Nuclear Reactor Load-Following

cs.AI cs.LGmath.OC ·2026-10-28 ·Perceval Beja-Battais, Alain Grosset{ê}te, Nicolas Vayatis
benchmark similarity 0.66
Abstract

Behavior Cloning (BC) of expert controllers for multiscale dynamical systems is challenging: state variables evolving at widely separated timescales induce highly nonlinear couplings that a shared latent space may fail to adequately represent, degrading closed-loop generalization. We propose Structured Representation Learning for Behavior Cloning (SRLBC), an architecture designed to leverage the physical knowledge of multiscale systems in which slow state variables, fast (stiff) state variables, and predictable exogenous inputs evolve at different timescales. On 500 long-horizon roll-outs of a pressurized water reactor (PWR) load-following benchmark -a high-dimensional, stiff industrial system with challenging safety constraints -and across 20 training seeds, separated embeddings systematically outperform shared embedding baselines on cumulative cost, imitation accuracy and constraint satisfaction. The resulting standalone policies approach NMPC closed-loop performance at a computational cost 15 times lower on average. Since standalone policies still leave a few percent of trajectories infeasible, we embed the learned policies in a safety-augmented approximate NMPC framework, in which the optimizer is only called when the predicted input sequence is not certified as safe. This yields controllers still 6.8 times faster than the NMPC controller on average, with a feasibility (i.e., constraint satisfaction) of at least 99.8% for all architectures, SRLBC retaining the lowest closed-loop cost. Under distribution shift, standalone BC policies degrade sharply: SRLBC degrades the least, and once safety-augmented, it retains a feasibility above 98% with the lowest closed-loop cost. Beyond the quantitative results, Jacobian-based sensitivity analysis shows that the resulting controller relies on a sparse and less redundant set of latent features, each attributable to a single group of physical variables, and aligned with the physical timescale decomposition of the system.

0.06