CS Paper Radar

Distributionally Robust Chance-Constrained Reinforcement Learning for Home Energy Management under Electric Vehicle Uncertainty

Meng Yuan, Ye Wang, Yi Mei, Ramesh Rayudu, Alan Brent, Xinghuo Yu, Mengjie Zhang

eess.SY ·submitted 2026-10-18 10:07 UTC · v1

Abstract

The electrification of residential energy systems and adoption of electric vehicles (EVs) create demand-side flexibility. However, random driving behaviour makes it difficult to ensure EV readiness at departure and maintain thermal comfort. To address this, we formulate residential scheduling as a distributionally robust (DR) chance-constrained Markov decision process, whose 2-Wasserstein ambiguity set guards against tail risk and distribution shift in EV behaviour. The intractable DR chance constraint is then replaced by a sufficient tightened expectation constraint under the nominal distribution. The resulting problem is solved by a Lagrangian soft actor--critic with distributional cost critics, and an online boundary guard enforces EV reachability and thermal comfort within the actuator limits at every step. Five controllers are compared over one year in Wellington, New Zealand, under a degraded forecast. The proposed controller is the cheapest controller that meets the EV target on every day and both chance constraints under forecast error and random driving behaviour. Compared with the rule-based baseline, the only other controller to do so, it lowers the annual cost by 8.6% and the comfort violation hours by 69%. The ablation study confirms that the boundary guard ensures pointwise feasibility within the actuator limits, and the DR chance constraint reduces tail risk.

Closest papers

Nearest neighbours by meaning across the whole index.

Reinforcement Learning-Based Pricing and Operation for Hybrid Electric and Hydrogen Vehicle Stations

cs.CE ·2026-10-16 ·Lowell Welburn, Yang Chen, Xudong Wang, Abhilasha Saroj, Shaked Regev
similarity 0.64
Abstract

Electric vehicles (EVs) and hydrogen fuel cell vehi- cles (HFCVs) are promising technologies in the efforts to reduce carbon emissions. Challenges such as limited infrastructure, con- sumer dissatisfaction, and electrical grid strain inhibit progress in the adoption of these technologies. A zero emission vehicle (ZEV) station, capable of handling EVs and HFCVs, addresses these challenges by focusing on maximizing station profit and consumer utility, while minimizing grid impacts. The purpose of this research is to develop a realistic simulation of a ZEV station that is capable of handling real-time pricing decisions enabled by reinforcement learning (RL) and to compare the performance of the RL agent to static pricing methods. After training, the RL agent gains 10.83% greater station profit than the baseline manual parameter search. This work serves as a useful tool for ZEV station owners when handling complex pricing and purchasing decisions.

0.26

Electric Vehicle Scheduling and Carbon-Aware Energy Dispatch for Charging Stations

eess.SY ·2026-10-27 ·Fulong Yao, Yiming Xu, Maurizio Albano, Liana Cipcigan, Naeima Hamed +2 more
similarity 0.62
Abstract

Electric vehicle (EV) charging stations integrating photovoltaic (PV) generation, battery storage, and bidirectional grid exchange must coordinate access to limited chargers with energy supply while considering both energy cost and carbon emissions. This paper proposes a solver-free greedy heuristic (SFGH) for dynamic vehicle scheduling (VS) and a twin delayed deep deterministic policy gradient (TD3) controller for carbon-aware energy dispatch (ED). SFGH assigns EVs according to reductions in service delays, and TD3 controls battery operation to balance energy costs and carbon emissions while meeting the resulting charging demand. The model allows both PV and battery exports, accounting for export revenue and carbon credits. Simulations on year-long case studies show that SFGH--TD3 achieves the lowest accumulated service delay, energy cost, and carbon emissions among the evaluated heuristic and multi-agent deep reinforcement learning (DRL) configurations.

0.24

RL-PaO: Prediction as Action in Decision Making under Uncertainty

cs.LG ·2026-09-29 ·Jiahui Feng, Dafang Zhao, Zheng Chen, Zhengmao Li, Lingwei Zhu
similarity 0.62
Abstract

Decision-making under uncertainty often relies on predicted parameters, yet accurate prediction does not necessarily lead to good operational decisions. Aligning prediction with downstream optimization requires learning from the consequences of the decisions those predictions induce. We introduce RL-PaO, a reinforcement learning framework that integrates system formulation, optimization, and decision execution into a single environment. This yields a Markov decision process in which prediction is regarded as action: it shifts the environment to produce subsequent context and reward that explicitly aligns prediction error with realized cost, and learning the optimal policy does not require differentiating through the black-box solver. We evaluate RL-PaO on day-ahead energy scheduling using real historical data. On the test year, RL-PaO achieves the lowest annual cost among the non-oracle baselines, achieving on average $10\%$ cost reduction. Moreover, RL-PaO is capable of further analyses to provide strong interpretability both from the policy evolution perspective and the cost-accuracy trade-off.

0.24

Hours-of-service-aware siting of charging and battery-swapping stations for long-haul electric trucks under adoption uncertainty

cs.CE ·2026-10-06 ·Elnaz Irannezhad, Jia Guo
similarity 0.62
Abstract · 43 pages, 6 figures, 10 tables

Planning en-route charging and battery-swapping infrastructure for long-haul battery-electric trucks (BETs) requires models that reflect how trucks actually operate. This paper develops a mixed-integer programming framework that jointly sites charging or swapping stations and schedules each truck's charging, swapping and mandatory driver rest, so that charging time overlaps with regulated rest instead of being added to it. Energy use is derived segment by segment from road terrain with a tractive-force model, and the truck battery is modelled as a set of independently swappable packs. Staged investment under uncertain BET adoption is formulated as a multistage stochastic program with Markovian demand and solved by stochastic dual dynamic integer programming (SDDiP) with Lagrangian cuts; we show that the Lagrangian multipliers can be bounded by each station's annualised cost without weakening the cuts. Applied to twelve freight corridors on Australia's East Coast, the algorithm jointly optimises charging and battery-swapping events as well as mandatory break events. A +/- 30\% change in adoption alters the final charge-only network by only -12\% to +13\% of stations, with over 95\% of stations built by the second stage. Hedging against uncertainty mainly changes which sites are chosen (71\% overlap with a deterministic rolling-horizon model), not when they are built.

0.14

Every minute counts: Wind farm power control during imbalance settlement using distributional soft actor-critic

eess.SY ·2026-10-19 ·Max Bruninx, Seyed Soroush Karimi Madahi, Timothy Verstraeten, Jan Decuyper, Chris Develder +2 more
similarity 0.62
Abstract

Wind farms can provide flexibility by curtailing their output during periods of excess generation in the power grid. This can be done through implicit balancing, where the power setpoints of the farm are adapted to anticipate the imbalance price. By doing so, the wind farm helps reduce the resulting system imbalance while also lowering its own costs. However, implicit balancing is challenging since the imbalance prices are unknown in advance and hard to predict. Furthermore, the control strategy of the wind farm can affect the imbalance price formation. To address both challenges, we develop a control policy with 1-minute setpoints using distributional soft actor-critic. Evaluated on real data of the Belgian grid and an offshore wind farm, we show that our method outperforms a price threshold controller, often used in industry, by 88% and performs on par with soft actor-critic. Furthermore, unlike soft actor-critic, our approach can learn a risk-averse policy by optimizing a risk measure, such as conditional value at risk. The risk-averse policy was found to better account for its own impact on the price, showing a reduction in overshoots, while achieving a similar degree of imbalance cost savings. Last, we find that using 1-minute instead of 15-minute setpoints further reduces imbalance costs for all evaluated control policies.

0.32

Probabilistic Power Estimation and MPC-based Scheduling of Construction Electric Vehicles Supported by Mobile Charging Stations

eess.SY ·2026-11-16 ·Shubhan Mital, Avik Ghosh, Daniela Rojas, Sujit Dey, Yuanyuan Shi
code/data similarity 0.61
Abstract · 5 pages, 2 tables; Submitted to IEEE PESGM 2027

The adoption of construction electric vehicles (CEVs) is held back by sparse on-site charging infrastructure, limited mobility, and insufficient knowledge of their power consumption. Mobile charging stations (MCSs) towed between grid connection points and construction sites address the first barrier, yet scheduling them optimally requires an accurate estimate of the CEV power consumption. In this paper, we address these gaps, first, by estimating the power of each subactivity of a representative CEV (a compact electric excavator) based on real-world construction demonstration data. We use Bayesian linear regression to obtain the subactivity power distributions, accounting for their operational uncertainty. Second, we formulate a mixed-integer program that jointly schedules CEV work and charging together with MCS routing, charging, and discharging, optimizing electricity costs, carbon emissions, work completion, and MCS travel. We embed the optimizer in a shrinking-horizon model predictive control (MPC) loop to perform real-time control on the CEV-MCS system. Two MPC variants, one a certainty-equivalent MPC (CE-MPC) controller, and the other a scenario-based MPC (SB-MPC) controller, are compared in closed-loop against an open-loop day-ahead schedule. Realistic simulations show that SB-MPC performs the best, lowering the operating costs by $60.2\%$ relative to the open-loop baseline, and by $23.3\%$ relative to CE-MPC, demonstrating the efficacy of incorporating predictive control with feedback. The code is available at https://github.com/Shubhanflash22/CEV-MCS-MPC-Joint-Scheduling

0.31

DER Allocation without Load Prediction via Reinforcement Learning

eess.SY ·2026-08-17 ·Abed AlRahman Al Makdah, Aravind Ramana, Shaofeng Zou, Oliver Kosut, Lalitha Sankar
published similarity 0.61
Abstract · 5 pages. Presented at the 2026 IEEE Power & Energy Society General Meeting (PES GM)

The growing variability of renewable generation increases the need for fast and flexible grid-balancing mechanisms. Existing frameworks for distributed energy resource aggregations (DERAs) rely on short-term forecasts of net demand, making their performance highly sensitive to prediction errors. In this paper we present a forecast-free reinforcement learning (RL) framework for DERA allocation that learns optimal policies directly from operational data. We model the DERA dynamics as a deterministic linear system and the exogenous net load as a feature-based linear Markov process, capturing short-range temporal dependencies without explicit forecasting. We derive a closed-form expression for the optimal policy, which is learned through a least-squares value iteration (LSVI) algorithm using data collected across episodes. The proposed framework preserves the interpretability and constraint satisfaction of DER model while adapting to stochastic demand variations through data-driven updates. Numerical experiments on real California Independent System Operator (CAISO) net-demand data demonstrate that the learned controller achieves high tracking accuracy and stable regulation across heterogeneous DER aggregators without requiring any demand prediction.

0.37

Scarcity Pricing for State-of-Charge Management in Real-Time Markets Under Intra-Day Uncertainty

eess.SY ·2026-11-09 ·Shailesh Wasti, Aravind Retna Kumar, Sushant Varghese, Anthony Giacomoni, Parviz Alivand +2 more
similarity 0.60
Abstract

The growth in renewable resources will increase net load variability and forecast error. Energy storage resources can provide flexibility when ramp capability is limited and contribute to load leveling over the day. The challenge for system operators is how to balance these two functions in real time, when updated net load forecasts are available for a limited lookahead horizon. U.S. electricity markets are seeking ways to manage state-of-charge (SoC) within their energy markets to more efficiently utilize these resources. We propose a method to manage energy storage SoC in a real-time market by extending Hogan's (2013) scarcity pricing framework. A day-ahead SoC schedule is modified using existing multi-interval real-time unit commitment engines as net load forecasts change. To address 30-minute-ahead forecast errors, an updated 5-minute SoC target and a dynamically updated shortage price for violating the constraint are provided to each economic dispatch. We simulate PJM's energy markets for 365 days and demonstrate reliability improvements relative to a strategy that enforces the day-ahead schedule. We show cost reductions and improved price formation relative to a strategy that updates target quantities but uses an administratively chosen shortage penalty.

0.30