CS Paper Radar

DER Aggregators in Wholesale Electricity Markets: A Hybrid Mean-Field Framework with Endogenous Price Feedback

Jun He, Andrew L. Liu

eess.SY ·submitted 2026-11-12 03:21 UTC · v1
theory

Abstract

The rapid growth of distributed energy resources (DERs), including rooftop solar and energy storage, is transforming the grid edge, where customer-side assets increasingly interact with the broader power grid. DER aggregators, entities that coordinate the actions of many small-scale DERs, play a key role in this transformation. This paper presents a hybrid Mean-Field Control (MFC) and Mean-Field Game (MFG) framework for integrating DER aggregators into wholesale electricity markets. Unlike traditional approaches that treat market prices as exogenous, our model captures the feedback between aggregators' collective actions and electricity prices. The MFC component optimizes DER operations within each aggregator, while the MFG models strategic interactions among multiple aggregators. To account for various uncertainties, we incorporate reinforcement learning, which allows aggregators to learn optimal bidding strategies in dynamic market conditions. We prove the existence and uniqueness of a mean-field equilibrium and validate the framework through a case study of the Oahu Island power system. Results show that our approach reduces price volatility and improves market efficiency, offering a scalable and decentralized solution for DER integration in wholesale markets.

Closest papers

Nearest neighbours by meaning across the whole index.

Strategic Arbitrage in Electricity Markets: A Multi-Agent Policy Optimization Framework

eess.SY ·2026-10-21 ·Eli Brock, Srikar Babu Gadipudi, Somayeh Sojoudi, Javad Lavaei
publishedbenchmark similarity 0.68
Abstract · Accepted to the 65th IEEE Conference on Decision and Control (CDC 2026). 8 pages, 6 figures

Grid-scale storage and load aggregators are growing quickly in wholesale electricity markets. If highly consolidated, these new participants may distort the competitive market equilibrium through strategic withholding behavior. Traditional market power mitigation techniques are ill-suited to arbitrage-based participants. We provide a multi-agent policy optimization model to study game-theoretic equilibria for real-time markets with consolidated storage and/or aggregators. The model is designed to facilitate natural first-order equilibrium-seeking update rules. We analyze the theoretical properties of different supply functions and their implications on the convergence of the gradient dynamics. Finally, we train equilibrium policies for a modified version of the IEEE RTS-GMLC benchmark across different levels of storage fleet consolidation, to demonstrate the potential of the framework as a tool to characterize and quantify strategic arbitrage.

0.33

Learning to Price Electricity for Optimal Demand Response

cs.LG stat.MLstat.AP ·2026-09-30 ·Jing Shang, Mohammad Mehrabi, Xinyang Zhou, Mahmoud Saleh, Andrey Bernstein +1 more
similarity 0.63
Abstract

There is considerable interest in using time-varying electricity prices to shape consumer demand response, and better align energy demand with renewable production. However, optimal prices generally vary over time in response to complex signals such as weather forecasts, sunrise/sunset times, and day-of-week patterns; and existing methods are not able to make efficient use of such rich contextual information. Here, we propose a neural-network-based algorithm for contextual energy pricing, modeling pricing as a Stackelberg game and leveraging a mean-field solution representation from Mehrabi et al.(2024). The approach learns constrained mappings from contextual features to feasible price signals. We validate our approach by simulating the energy grid in several US cities, and show that incorporating contextual information can considerably increase the value of the demand response programs.

0.23

DER Allocation without Load Prediction via Reinforcement Learning

eess.SY ·2026-08-17 ·Abed AlRahman Al Makdah, Aravind Ramana, Shaofeng Zou, Oliver Kosut, Lalitha Sankar
published similarity 0.62
Abstract · 5 pages. Presented at the 2026 IEEE Power & Energy Society General Meeting (PES GM)

The growing variability of renewable generation increases the need for fast and flexible grid-balancing mechanisms. Existing frameworks for distributed energy resource aggregations (DERAs) rely on short-term forecasts of net demand, making their performance highly sensitive to prediction errors. In this paper we present a forecast-free reinforcement learning (RL) framework for DERA allocation that learns optimal policies directly from operational data. We model the DERA dynamics as a deterministic linear system and the exogenous net load as a feature-based linear Markov process, capturing short-range temporal dependencies without explicit forecasting. We derive a closed-form expression for the optimal policy, which is learned through a least-squares value iteration (LSVI) algorithm using data collected across episodes. The proposed framework preserves the interpretability and constraint satisfaction of DER model while adapting to stochastic demand variations through data-driven updates. Numerical experiments on real California Independent System Operator (CAISO) net-demand data demonstrate that the learned controller achieves high tracking accuracy and stable regulation across heterogeneous DER aggregators without requiring any demand prediction.

0.37

Reinforcement Learning and Rule-Based Peer-to-Peer Pricing in Residential PV-BES Communities

cs.LG cs.CY ·2026-09-01 ·Pablo Benalcazar, Maciej Kalka, Wilian Guamán, Jacek Kamiński
similarity 0.62
Abstract · 22 pages, 2 figures, submitted to ARTIIS 2026 (Conference on Advanced Research in Technologies, Information, Innovation and Sustainability) …

This paper compares rule-based and learning-based pricing mechanisms for peer-to-peer (P2P) electricity trading in residential photovoltaic communities. The rule-based benchmarks comprise bill-sharing as an ex post allocation mechanism, the mid-market rate, and supply-demand-ratio pricing. The reinforcement-learning (RL) formulation is implemented through a Deep Q-Network and evaluated under multiplier-based and learnable SDR-shaped pricing, with a fixed-parameter SDR variant as a non-learning control. Performance is assessed through community savings together with complementary financial and operational indicators. In the base PV-only configuration, the rule-based benchmarks outperform the best RL policy. With battery energy storage, evaluated for the RL policies only, community savings under the best RL policy increase from EUR 734.23 to EUR 978.52. Across the learning-based modes and in both configurations, SDR-shaped pricing outperforms the multiplier-based parameterization considered. The results indicate that rule-based pricing remains highly competitive wherever the two families are compared directly, and that storage substantially improves the learning-based outcomes under this accounting, while the distribution of benefits remains heterogeneous across households.

0.32

Extended Version: Storage-Based Strategic Manipulation of Constraint-Binding Patterns in Power Networks

eess.SY econ.GN ·2026-09-14 ·Mehdi Davoudi, Minghao Mou, Junjie Qin
similarity 0.61
Abstract · Extended version of our paper submitted to IEEE Transactions on Power Systems

This paper studies the strategic market participation of a monopolistic energy storage aggregator (ESA) in a day-ahead electricity market. The ESA coordinates geographically distributed storage units, submits a coordinated bid for its portfolio, and may hold financial transmission rights (FTRs). The system operator clears the market through a network-constrained, multi-period economic dispatch, determining generation and load schedules, nodal prices, energy-market payments, and FTR payoffs. We formulate the ESA--system-operator interaction as a Stackelberg game and characterize its equilibrium through a constraint-binding-pattern decomposition of the market-clearing problem. Beyond enabling equilibrium computation, the framework reveals how the ESA can increase its profit by strategically inducing or avoiding particular constraint-binding patterns. It also establishes a novel welfare result: although strategic storage without FTRs is known to weakly improve social welfare relative to the no-storage case, certain FTR positions can overturn this guarantee by strengthening the ESA's incentive to induce particular patterns, causing social welfare to fall below the no-storage level. Motivated by these findings, we develop two system-operator mechanisms for limiting undesirable ESA behavior and its adverse effects on market outcomes and social welfare. Finally, a three-bus study illustrates the theoretical findings, while IEEE test systems demonstrate the scalability of the proposed method.

0.18

PowerMarketJax: A JAX Benchmark Suite for Multi-Agent Reinforcement Learning in Power Markets

cs.MA cs.AIeess.SY ·2026-09-29 ·Zhanhua Pan, Xin Qin, Xiao Liu, Zhilong Cao, Jianhong Wang +1 more
benchmarkcode/data similarity 0.58
Abstract · 69 pages

Power markets are a natural testbed for multi-agent reinforcement learning (MARL), where multiple self-interested participants repeatedly submit bids. A market-clearing mechanism then determines dispatch and prices subject to power grid constraints and market settlement rules. However, existing MARL environments typically focus on a single market setting, implement simplified clearing mechanisms, or rely on CPU-based optimization solvers that slow large-scale training and limit the systematic study of bidding strategies and market behavior. We introduce PowerMarketJax, a benchmark suite for MARL across five power markets: day-ahead wholesale, real-time balancing, ancillary services, peer-to-peer double auctions, and local flexibility. Each environment implements its own clearing, pricing, and settlement rules while providing a common framework for learning and evaluation. We find that learned bidding behavior depends strongly on the market design: independent learners can miss better strategies when gains require many agents to change together, when more profitable strategies lie beyond a region of lower profit, or when profits disappear as more agents adopt the same strategy. PowerMarketJax implements both market simulation and policy training in JAX, allowing the entire pipeline to run on the GPU with 1,024 X 1,200 parallelisms across both environments and market participants, achieving up to 33X speedup over CPU-based baselines. Our open-source benchmark is available at: https://github.com/powermarketjax/PowerMarketJax.

0.23

Co-optimizing Bidding and Power Allocation of an EV Aggregator Providing Real-time Frequency Regulation Service

eess.SY ·2026-08-25 ·Ruike Lyu, Hongye Guo, Kedi Zheng, Mingyang Sun, Qixin Chen
top venuehighly cited similarity 0.58
Abstract · Published in: IEEE Transactions on Smart Grid ( Volume: 14, Issue: 6, November 2023)

The rapidly expanding scale of electric vehicle (EV) fleets and continuously decreasing battery costs are making vehicle-to-grid services a reality. In this paper, we study the interaction between the problems of an EV aggregator's bidding in the regulation market and power allocation (i.e., determining the (dis)charging powers of the EVs in regulation deployment). Although the two problems are coupled, they are often regarded as decoupled and optimized separately for complexity issues. However, failing to consider the coupling of bidding and power allocation can lead to a decline in the profit of the EV aggregator (EVA). In this paper, we propose a framework for co-optimizing EVA bidding and power allocation in the regulation market. The bidding model is formulated as a stochastic programming problem with embedded power allocation in discretized regulation signal scenarios. To meet the solution time requirement for regulation deployment, we further propose a power allocation model that can be solved online. It utilizes the Lagrange multipliers from the bidding problem to ensure that the allocation results correspond to the optimal solution of the bidding problem. The effect of the proposed framework on improving EVA profits and reducing degradation costs is verified in the case study.

0.67

Reinforcement Learning-Based Pricing and Operation for Hybrid Electric and Hydrogen Vehicle Stations

cs.CE ·2026-10-16 ·Lowell Welburn, Yang Chen, Xudong Wang, Abhilasha Saroj, Shaked Regev
similarity 0.57
Abstract

Electric vehicles (EVs) and hydrogen fuel cell vehi- cles (HFCVs) are promising technologies in the efforts to reduce carbon emissions. Challenges such as limited infrastructure, con- sumer dissatisfaction, and electrical grid strain inhibit progress in the adoption of these technologies. A zero emission vehicle (ZEV) station, capable of handling EVs and HFCVs, addresses these challenges by focusing on maximizing station profit and consumer utility, while minimizing grid impacts. The purpose of this research is to develop a realistic simulation of a ZEV station that is capable of handling real-time pricing decisions enabled by reinforcement learning (RL) and to compare the performance of the RL agent to static pricing methods. After training, the RL agent gains 10.83% greater station profit than the baseline manual parameter search. This work serves as a useful tool for ZEV station owners when handling complex pricing and purchasing decisions.

0.26