CS Paper Radar

Large Language Models as Strategic Bidding Agents in P2P Energy Trading Markets

Ismail Lotfi, Ali Ghrayeb, Haitham Abu-Rub

cs.GT ·submitted 2026-08-12 14:49 UTC · v1
published

Abstract

Peer-to-peer (P2P) energy trading markets rely on double auction mechanisms to match prosumers and consumers in smart grid distribution networks. However, the strategic behavior of bidding agents in such markets remains not fully explored, particularly in repeated settings with bounded rationality. This paper proposes a novel framework that integrates large language models (LLMs) as reasoning-driven bidding agents in repeated P2P energy double auctions. We compare the performance of three bidding strategies: random bidding, an $\varepsilon$-greedy multi-armed bandit (MAB) approach, and an LLM-based strategy. Simulation results show that the LLM-based strategy achieves superior cleared trading volume over the first episodes compared to $\varepsilon$-greedy MAB and random bidding baselines, eliminating the exploration burn-in period that statistical learning algorithms inherently require before converging to productive price arms. Importantly, when tested in an environment different from the one used during learning, the performance of the $\varepsilon$-greedy strategy drops significantly, while that of the LLM-based bidding strategy continue to achieve higher surplus and successful trades. Nevertheless, the LLM's advanced contextual reasoning also gives rise to an important market dynamic. In a homogeneous population of LLM agents, sellers increasingly exploit buyers' rational outside options to drive clearing prices above the Nash equilibrium, resulting in a progressively more asymmetric allocation of surplus in favor of sellers that does not converge within the observed time horizon.

Comment
Accepted at IEEE IECON 2026

Closest papers

Nearest neighbours by meaning across the whole index.

PowerMarketJax: A JAX Benchmark Suite for Multi-Agent Reinforcement Learning in Power Markets

cs.MA cs.AIeess.SY ·2026-09-29 ·Zhanhua Pan, Xin Qin, Xiao Liu, Zhilong Cao, Jianhong Wang +1 more
benchmarkcode/data similarity 0.70
Abstract · 69 pages

Power markets are a natural testbed for multi-agent reinforcement learning (MARL), where multiple self-interested participants repeatedly submit bids. A market-clearing mechanism then determines dispatch and prices subject to power grid constraints and market settlement rules. However, existing MARL environments typically focus on a single market setting, implement simplified clearing mechanisms, or rely on CPU-based optimization solvers that slow large-scale training and limit the systematic study of bidding strategies and market behavior. We introduce PowerMarketJax, a benchmark suite for MARL across five power markets: day-ahead wholesale, real-time balancing, ancillary services, peer-to-peer double auctions, and local flexibility. Each environment implements its own clearing, pricing, and settlement rules while providing a common framework for learning and evaluation. We find that learned bidding behavior depends strongly on the market design: independent learners can miss better strategies when gains require many agents to change together, when more profitable strategies lie beyond a region of lower profit, or when profits disappear as more agents adopt the same strategy. PowerMarketJax implements both market simulation and policy training in JAX, allowing the entire pipeline to run on the GPU with 1,024 X 1,200 parallelisms across both environments and market participants, achieving up to 33X speedup over CPU-based baselines. Our open-source benchmark is available at: https://github.com/powermarketjax/PowerMarketJax.

0.23

How language model agents reshape price competition

cs.GT cs.LGcs.MAecon.GN ·2026-10-15 ·Martin Bichler, Philip Hierhager, Kassian Köck, Fabian Pieroth
benchmark similarity 0.66
Abstract

Repeated Bertrand competition is a stringent benchmark for algorithmic pricing: in homogeneous-goods markets, standard learning procedures typically select the stage-game Nash outcome with prices near marginal cost. We show that LLM pricing agents can escape this benchmark, depending on deployment decisions. LLMs outperform standard online learners through faster adaptation, but this advantage largely disappears against full-information learners. An extracted surrogate algorithm illustrates how LLMs in specific deployments profit from regularly undercutting slower learners. Agents that post one price every period from raw market histories that include rival prices can sustain supracompetitive regimes in self-play. Agents that periodically emit probability distributions from summarized histories of their own profits instead move toward the competitive region in self-play. Markets mixing these interfaces can generate dynamic regimes in which LLMs establish a high-price anchor and regularly undercut it to capture demand. These regimes can keep both competitors' prices above marginal cost. Deployment choices shape these outcomes, shifting the focus from stationary equilibria to dynamic pricing regimes.

0.19

AI agents in Algorithmic Electricity Markets: On the Emergence of Tacit Collusion

cs.AI cs.GTcs.MAeess.SY ·2026-08-27 ·Jakub Seredyński, Georgios Tsaousoglou
similarity 0.65
Abstract · 10 pages

As electricity market participants increasingly adopt learning-based agents for their bidding strategies, electricity markets are becoming algorithmic. Evidence from algorithmic markets in other domains shows that tacit collusion can arise purely through independent learning. Moreover, electricity markets are typically oligopolistic and feature repeated interaction among a small number of participants, making them structurally susceptible to non-competitive behavior. In the face of these observations, this paper investigates the hypothesis that tacit collusion may emerge in electricity markets where participants' actions are controlled by autonomous learning-based algorithms. We model strategic bidding as a repeated game with imperfect public monitoring, and model the participants' emergent behavior using multi-agent reinforcement learning. We propose a multi-dimensional set of criteria (going beyond profit comparisons against Nash equilibria) to assess whether the resulting behavior constitutes tacit collusion. Our experimental results showcase that such a danger is realistic for electricity markets: there are cases where agents do learn to sustain supra-competitive outcomes that are supportive of tacit collusion indicators, even though the agents were never instructed to collude.

0.38

Peak-Aware One-Way Trading: Competitive and Learning-Augmented Algorithms for Energy Procurement

cs.DS ·2026-10-22 ·Jackson Bibbens, Adam Lechowicz, Nicolas Christianson, Bo Sun, Mohammad Hajiesmaili
similarity 0.62
Abstract · 27 pages, 2 figures

We introduce peak-aware one-way trading ($\texttt{PA-OWT}$), an online problem in which a player must purchase a fixed quantity of an asset by a deadline, minimizing its total cost plus a charge proportional to the largest single purchase at any time. The model is motivated by electricity procurement, where the cost of energy rewards buying at low prices, but peak demand charges discourage concentrating purchases in short periods. Balancing these incentives is challenging when future prices are unknown: a large purchase can establish a peak that later decisions cannot undo. We develop a peak-aware online algorithm that balances price and peak control with deadline feasibility and achieves the order-optimal competitive ratio. We further extend our results to a learning-augmented algorithm that uses predictions of optimal purchasing decisions to achieve an order-optimal consistency-robustness tradeoff, exploiting accurate advice while retaining worst-case guarantees in the face of arbitrary prediction errors. Finally, we evaluate our algorithms in a case study on electricity procurement using historical electricity price traces to understand the benefits of peak-aware purchasing.

0.19

Fine-Tuning Autobidders with Group Relative Policy Optimization

cs.GT ·2026-08-28 ·Anton Safin, Alexandra Khirianova, Andrey Pudovikov, Aleksandr Katrutsa, Egor Samosvat
similarity 0.61
Abstract

Automated bidding (autobidding) is a core component of modern online advertising systems. Within this component, advertisers delegate sequential bid decisions to algorithms that must maximize campaign value while adhering to constraints such as a limited budget and a target cost-per-click (CPC). One of the approaches to resolve the autobidding problem is to formulate it as a Markov decision process and use reinforcement learning (RL) to train a bid generation function. The standard RL framework is actor-critic, which consists of an actor network that generates actions and a critic network that estimates the value of those actions. In our setting, the action is typically a bid or related pacing multiplier, and the value is the expected return from the auction given the bid. However, the alternating training of actor-critic RL models leads to instability and reduced robustness to noise. To address these issues, we adapt the Group Relative Policy Optimization (GRPO) framework to the autobidding setting. This framework is a \emph{critic-free} policy-gradient method originally developed for large language model post-training, where the ground-truth target is unknown. The autobidding setting shares this property, since the optimal bid is unknown in advance. Moreover, GRPO in the LLM domain is used to fine-tune the pre-trained model, and we use the same technique to enhance the performance of the strong heuristic baseline. We empirically compare Autobidding GRPO with actor-critic models, simple heuristics, and controller-based methods on the BAT, iPinYou, and AuctionNet benchmarks. Extensive experiments show that Autobidding GRPO consistently outperforms baselines in clicks and is the best or second-best method in conversion volume.

0.20

Learning Stability of Replay-Based Co-Optimization for Transmission Expansion under Strategic Bidding

eess.SY ·2026-10-07 ·Tomonari Kanazawa, Hikaru Hoshino, Eiko Furutani
similarity 0.60
Abstract

This paper investigates the behavior of learning-based co-optimization for transmission expansion under strategic bidding in electricity markets. In this framework, transmission capacities are updated while market participants simultaneously learn their bidding strategies through deep reinforcement learning, resulting in coupled and non-stationary learning dynamics. We show that transient policy degradation of bidding agents can generate inconsistent cost-capacity samples, which bias the transmission-capacity update and prevent the co-optimization process from converging to the desired solution. To mitigate this, replay-based capacity updates with nearest-neighbor filtering are introduced to exclude inconsistent samples from the update data. We then analyze a new oscillatory behavior that appears when the replay memory is enlarged. Numerical results on the IEEE 30-bus system demonstrate that enlarged replay memories improve robustness against transient policy degradation but can introduce a temporal lag, leading to oscillations unless the capacity-update learning rate is appropriately reduced. These results reveal the trade-off between replay-memory size and learning rate and provide practical guidelines for stable co-optimization.

0.22

LangBP: Language-Guided Reasoning and Acting for Joint Bidding and Pricing

cs.GT ·2026-08-31 ·Jiaqi Ding, Chuan Yang, Linghui Meng, Shengsheng Niu, Jie He +3 more
similarity 0.60
Abstract · 12 pages,6 figures

Auto-bidding is a long-horizon sequential decision problem for maximizing conversion value under budget and key performance indicator (KPI) constraints. Recent work extends this task from bidding alone to joint bidding and pricing, where a policy controls bidding decisions and pricing corrections. Existing methods mainly rely on numerical trajectory modeling, which offers limited support for interpreting campaign context and expressing high-level strategies. Large language models (LLMs) can complement this paradigm with their reasoning capabilities. However, existing language-guided methods have two limitations. First, they condition actions on language strategies without modeling the corresponding state changes, making it difficult to distinguish errors in strategy understanding from errors in action generation. Second, different instructions can produce similar execution effects, leading to imbalanced policy updates across effects. We propose LangBP, a hierarchical framework for language-guided joint bidding and pricing. LangBP's Semantic Decision Transformer (S-DT) predicts target states from the instruction and the trajectory history, then recovers the joint action via inverse dynamics. We further propose Execution-Grouped Policy Optimization (EGPO), which scores candidate effects with a Context--Effect Verifier (CEV) and balances policy updates across effect groups. Experiments on AuctionNet show that LangBP outperforms strong baselines, and online A/B tests further demonstrate business gains in real-world deployment on a large-scale e-commerce platform.

0.26

Decentralized LLM Workflows via Sequential Auctions

cs.GT cs.AIcs.MA ·2026-10-15 ·Ian Gemp, Shubhendu Trivedi, Adam Fisch, Fantine Huot, Michael Kaisers +5 more
similarity 0.59
Abstract · 28 pages

We present a scalable framework for solving multi-step tasks through a decentralized market of self-interested agents. Prior theoretical work demonstrated that agents organized through sequential auctions can converge to a globally optimal policy in a Markov decision process (MDP), but relied on the restrictive assumption of the existence of perfect clones as a model of perfect competition. In reality, exact clones drive agent profit to zero, violating the natural risk-reward incentives of participants. Conversely, we empirically demonstrate that the opposite extreme, absence of competition, also leads to market failure and value extraction collapse. Between these two extremes, we investigate how heterogeneous agents can survive in markets that span the critical middle ground of imperfect competition. To be profitable, agents learn to strategically shade bids to secure positive individual profit margins, while still ensuring that the marketplace functions effectively enough so that collaborative tasks posed by market consumers can still be solved. We formulate and evaluate four bidding algorithms (Mirror Descent, Thompson Sampling, Bayes-Optimal, and PPO) across three auction protocols (1st-Price, 2nd-Price, and Credit-Conserving Vickrey). Across a diverse set of domains, we demonstrate that imperfect competition successfully sustains necessary agent profit margins while maintaining high task success rates at low costs.

0.28