CS Paper Radar

Hierarchical Control via MPC-RL for Multi-Timescale Battery Systems

Rasa Pourjam, Ehecatl Antonio del Río Chanona, Paulina Quintanilla

eess.SY ·submitted 2026-10-02 16:04 UTC · v1

Abstract

Multi-timescale systems present a fundamental challenge, where fast operational decisions must coexist with long-horizon sustainability targets. In this work, we propose a new hierarchical control framework via Model Predictive Control (MPC) and Reinforcement Learning (RL) to separate decision-making on two distinct timescales. The high-level MPC optimizes long-horizon setpoints at the slow dynamic and on a fast timescale, a low-level pretrained RL agent tracks these setpoints in real time to maximize short-term objectives. RL is introduced to learn nonlinear control policies, without relying on model linearizations or requiring the heavy online computation from solving repeated optimal control problems. The framework is applied to a Battery Energy Storage System (BESS) operating in frequency regulation markets to balance fast profit opportunities (seconds) and slow battery degradation (weeks to months). The design employs a degradation-aware RL agent trained offline to generate safe long-horizon setpoints, and a degradation-unaware agent fine-tuned from it for fast runtime setpoint tracking. Compared to MPC baselines, the proposed approach successfully extends battery lifetime by 84% and increases operational profit by 34%.

Closest papers

Nearest neighbours by meaning across the whole index.

Learning Control Policies from Heterogeneous Multi-Horizon Time Series in Battery Energy Management Systems

eess.SY ·2026-09-18 ·Sheng Yin, Vivek Teja Tanjavooru, Holger Hesse, Christoph Goebel
published similarity 0.76
Abstract

This paper introduces Representation-to-Decision (R2D), an end-to-end imitation learning framework that maps heterogeneous multi-horizon time-series inputs directly to battery control decisions through modular Temporal Feature Extractors (TFEs) and a shared latent representation, without an explicit load and PV forecasting step. While accurate forecasting improves prediction quality, optimal control performance remains unguaranteed in prediction-then-optimization pipelines; standard Reinforcement Learning (RL) lacks the long-horizon temporal awareness due to short-window observations, even when forecast signals are available as additional inputs. R2D offers a different perspective: rather than forecasting first and deciding second, it learns to decide directly from raw temporal inputs, with control optimality anchored by an aging-aware Mixed-Integer Linear Programming (MILP) expert through Behavior Cloning (BC). Benchmarked against six controllers on a high-fidelity electro-thermal battery simulation across five industrial sites, R2D achieves 62--77\% of the global clairvoyant optimum, outperforms the tested Model Predictive Control (MPC) and RL benchmarks, and yields battery degradation nearly identical to its MILP teacher across all five sites. Comprehensive ablation studies over temporal backbone, model size, expert formulation, aging-cost weighting, and horizon configuration confirm that an LSTM encoder with a 15-minute single-step control horizon provides the most robust and deployment-ready configuration, and cross-site and single-factor out-of-distribution tests show that generalization to unseen profiles is profile-dependent, with policies trained on high-activity sites transferring most reliably.

0.26

Safe Receding Horizon Mixed-Integer Differentiable Predictive Control for Degradation-Aware Battery Dispatch

eess.SY ·2026-09-23 ·Eshagh Safarzadeh Ravajiri, Jan Drgona, Benjamin F. Hobbs
publishedbenchmark similarity 0.74
Abstract · 6 pages, accepted to the 65th IEEE Conference on Decision and Control (CDC 2026)

We present a safe receding-horizon mixed-integer differentiable predictive control methodology for residential battery energy storage dispatch that combines neural-network speed with recursive feasibility guarantees. Unlike open-loop learning-to-optimize methods, it incorporates real-time state-of-charge feedback and sinusoidal time-of-day conditioning, enabling closed-loop re-planning at every timestep without re-solving a mixed-integer program. A differentiable rainflow cycle-counting layer enables self-supervised training of the mixed-integer policy on exact degradation physics. The controller is a hybrid closed-loop system pairing a neural mode-selection and continuous-action policy with a quadratic-programming safety filter that guarantees recursive feasibility independent of network weights or mode optimality. We establish mode-conditioned Lipschitz continuity and a conditional regret decomposition into training-quality, mode-mismatch, and forecast-error terms. On a 7-day net-metering evaluation, the method attains a 6.9% cost gap versus the closed-loop mixed-integer MPC benchmark with a 25x speedup (0.11 s vs 2.7 s per step), while average regret rises by under 4% across 0-30% forecast noise. The learned and benchmark modes agree at every step, so the bound reduces to its training-quality and forecast-error terms, both empirically validated.

0.19

Comparative Evaluation of MILP, MPC, and Reinforcement Learning for Commercial Battery Dispatch Under Time-of-Use Tariffs

eess.SY ·2026-09-13 ·Hafiz Majid Hussain, Wajiha Samar, Lurian Klein, Pedro Nardelli
benchmark similarity 0.72
Abstract

Battery energy storage systems (BESS) paired with rooftop photovoltaics (PVs) can deliver measurable cost savings under time-of-use (TOU) electricity tariffs; however, the relative performance of model-based and model-free dispatch strategies remains insufficiently benchmarked on full-year, real-world commercial datasets. This paper presents a full-year (2023) comparative evaluation of three BESS dispatch approaches using data from a commercial PV installation operating under a TOU tariff. The examined strategies include: (i) a mixed-integer linear programming (MILP) formulation with perfect foresight, providing an oracle performance benchmark under the assumed model; (ii) a model predictive control (MPC) scheme based on a day-ahead persistence forecast, representing a low-complexity deployable approach; and (iii) a soft actor-critic (SAC) deep reinforcement learning agent trained under causal information constraints. The MILP benchmark achieves an annual cost reduction of 24.6\% relative to a no-storage baseline. The persistence-based MPC approach recovers 99.2\% of this benchmark using only prior-day data. In contrast, the evaluated SAC agent yields an annual cost higher than the no-storage baseline. This outcome is analyzed in the context of known challenges in reinforcement learning for energy systems, including limited observability and reward design. Overall, the results indicate that, for the studied dataset and tariff structure, persistence-based MPC captures nearly all achievable economic benefits under practical deployment constraints, whereas the considered RL configuration does not yield competitive performance under the same information limitations.

0.21

Large Language Model-Empowered Intelligent Agent for Adaptive Fast-Charging Control of Lithium-Ion Batteries

cs.AI ·2026-11-13 ·Joey Chan, Haoyu Pan, Zhen Chen, ErShun Pan
similarity 0.69
Abstract · 9 pages, 5 figures, 6 tables

Fast charging of lithium-ion batteries is a multi-objective control problem that requires reducing charging time and sustaining high energy throughput while suppressing accelerated degradation and safety-critical electrochemical reactions. Conventional protocols are easy to implement but have limited ability to adapt to variations in battery chemistry, operating conditions, and aging state. Model-based optimization and conventional data-driven control often require accurate models, costly iterative search, or task-specific policies. We develop a large language model (LLM)-empowered intelligent agent for adaptive battery fast-charging control. In this closed-loop framework, the agent interprets structured battery states, generates executable charging actions, observes the resulting battery response, and continuously adjusts subsequent decisions. We construct a large-scale simulation corpus with PyBaMM across five battery chemistries and 11 parameter sets and convert the resulting state-action-response trajectories into structured process data for agent training. Our two-stage strategy combines supervised fine-tuning (SFT) and Group Relative Policy Optimization (GRPO). SFT establishes battery-control semantics and feasible decision priors from simulation-derived demonstrations. GRPO aligns the agent with a multi-objective control objective covering energy throughput, degradation mitigation, and operational constraints. Closed-loop simulations show that, under matched energy-throughput requirements, the agent achieves longer cycling life than conventional charging baselines and optimization-based strategies across heterogeneous battery configurations. This study provides a simulation-grounded proof of concept for using LLM agents as sequential decision makers in battery management and takes a step toward language-model-enabled control of physical energy systems.

0.31

Battery Energy Storage Sizing in PV Ramp-Rate Smoothing using Constraint-Aware Reinforcement Learning-Based Controller

eess.SY ·2026-11-13 ·Arash Omidi, Tanmay Mishra, Mads R. Almassalkhi
benchmark similarity 0.69
Abstract

This paper presents a constraint-aware reinforcement learning (CARL) controller for battery energy storage system (BESS) sizing and photovoltaic (PV) ramp-rate smoothing. Rapid PV power fluctuations caused by cloud transitions can introduce operational and stability challenges in power systems. To mitigate these impacts, grid codes impose ramp-rate limits (RRL) on PV power injected to the grid. BESSs can support such grid codes by absorbing or supplying power during rapid variations. However, the extent to which BESSs can reduce PV-induced variability depends on (i) the BESS power and energy capacities; and (ii) the adopted controller. Thus, BESS sizing and controller design are coupled. Conventional moving-average control can increase battery usage due to filter-induced lag, while ramp-rate control is reactive and does not account for the long-term impact of actions on battery state of charge and future ramp events. Optimization-based methods can reduce BESS requirements by anticipating future PV variations, but their performance depends on accurate PV forecasts. To address these limitations, the CARL controller learns a causal state-feedback policy from historical PV data without requiring forecasts during deployment. Ramp-rate, converter-power, and state-of-charge constraints are incorporated into a causal feasibility interval that guarantees feasible actions by definition. A full-horizon input-convex neural network (ICNN)-based optimization is developed as a noncausal benchmark for the minimum achievable BESS requirements. Evaluation using three real-world PV datasets shows that, at a 10\%/min ramp-rate limit, CARL reduces peak BESS power by 8.7--10.4\% and energy capacity by 47.5--62.6\% relative to conventional ramp-rate control while maintaining the grid-codes compliance.

0.26

End-to-End Battery Dispatch with Exact Rainflow Degradation via Mixed-Integer Differentiable Predictive Control

eess.SY ·2026-09-11 ·Eshagh Safarzadeh Ravajiri, Jan Drgona, Mahdi Mehrtash, Benjamin F. Hobbs
top venue similarity 0.69
Abstract · 12 pages. Submitted to IEEE Transactions on Smart Grid

Optimal dispatch of battery energy storage systems requires balancing energy arbitrage against cycle-induced degradation, which is accurately quantified through rainflow cycle counting. However, rainflow's combinatorial, nondifferentiable algorithm is incompatible with both convex optimization and gradient-based neural network training. We present a self-supervised mixed-integer differentiable predictive control framework that trains neural policies directly on exact rainflow degradation through a novel differentiable rainflow layer combining exact gradients at state-of-charge extrema with dense proxy gradients on incremental changes, enabling stable end-to-end training while preserving true degradation physics. A mixed-integer differentiable architecture enforces power balance, dynamics, and mode exclusivity, with a safety filter guaranteeing constraint satisfaction. We evaluate the framework on 3,650 real battery-day scenarios (a 10-battery fleet over 365 days) spanning three utility regions (SDG&E California, Xcel Energy Colorado, APS Arizona) with diverse time-of-use pricing structures. A single-battery trained policy achieves a 0.35% performance gap on its training distribution with over 200x speedup, while fleet-wide training generalizes across all households and utility regions, achieving a 3.33% performance gap with 564x computational speedup (62 seconds vs. 9.7 hours) and 100% feasibility. The millisecond-scale inference reduces computation by over two orders of magnitude compared to mixed-integer solvers, enabling practical deployment at scale.

0.71

Grid-Mode-Aware Model Predictive Control of Hybrid Energy Storage Systems for AI Data Center Power Smoothing

eess.SY ·2026-09-03 ·Xin Chen
similarity 0.69
Abstract

To facilitate the grid-friendly integration of highly variable AI data center loads, this paper proposes a grid-mode-aware model predictive control (G-MPC) framework for managing a hybrid energy storage system (HESS) to smooth grid-side power demand. The framework optimally coordinates a battery energy storage system (BESS) and a supercapacitor (SC) by solving a multi-step optimization problem in a receding-horizon manner. In particular, band-pass filter dynamics are directly embedded in the G-MPC formulation to extract and suppress grid-side power components associated with vulnerable grid oscillatory modes, thus mitigating load-induced grid oscillations. The resulting G-MPC optimization jointly minimizes violations of grid-side power-envelope, ramp-rate, and modal-power requirements and the degradation and power-ramping costs of the BESS and SC, while satisfying power limits, state-of-charge limits, and other operational constraints. To enable real-time implementation, a fix-and-re-optimize algorithm is developed to solve each G-MPC problem efficiently while preventing simultaneous charging and discharging. Extensive simulations demonstrate the effectiveness, flexibility, and computational efficiency of the proposed framework. The results also highlight the importance of explicitly suppressing power components associated with vulnerable grid modes, rather than merely reducing overall load variations, to effectively mitigate grid oscillations.

0.40

Building Seasonal Highways for Residential Energy Hubs: Sizing, planning and operating thermal energy storage

eess.SY math.OC ·2026-10-01 ·Dario Slaifstein, Mohammad Khosravi, Gautham Ram Chandra Mouli, Laura Ramirez-Elizondo, Pavol Bauer
similarity 0.68
Abstract

The operation of residential energy hubs with multiple energy carriers (electricity, heat, mobility) poses a significant challenge due to the energy storage differences in time-constants, round-trip efficiencies and self-discharge rates. Usually, thermal storage exhibits flexibility in yearly planning optimizations or long-term scenarios. However, as optimization horizons shrink (1-48hs) so does their supplied value due to the lower round-trip efficiencies. To avoid this early depletion during operation this paper proposes a data-driven highway to steer the short-term daily control towards long-term optimality. The proposed methodology also presents how to optimally size the thermal storage and avoid yearly simulations and how all of this is related to nonlinearities in the daily operation. The presented framework links seasonal and daily optimizations through dynamic terminal sets and value functions. The seasonally-aware nonlinear economic model predictive controller achieves the most balanced performance, with the second best mean grid cost of all MPCs at -\texteuro 209. It also achieves better battery degradation control than its linear counterparts (between 26-34%) and the best thermal comfort of the nonlinear benchmarks. Nevertheless, the data-driven seasonal highway restrains the ability to control battery degradation and slightly increases computational time.

0.14