CS Paper Radar

Offline Reinforcement Learning for Wind Farm Control: A Wind Tunnel Study under Dynamic Wind Directions

Yuhan Su, Hongyang Dong, Simone Tamaro, Filippo Campagnolo, Carlo L. Bottasso, Xiaowei Zhao

cs.LG ·submitted 2026-09-11 14:31 UTC · v1
benchmark

Abstract

This paper addresses the wind farm power maximization problem in the presence of wind direction changes. Specifically, a model-free Modified Twin Delayed Deep Deterministic Policy Gradient with Behavior Cloning (MTD3-BC) algorithm is proposed to tackle this task through yaw control under varying wind direction conditions. MTD3-BC is an offline reinforcement learning (RL) algorithm that aims to infer good behavior from only a precollected offline dataset. Additionally, to ensure smooth and moderate yaw adjustments, a new action consistency term is introduced into the policy optimization objective. Unlike online RL methods, MTD3-BC does not require extensive interactions with a wind farm simulator during training, significantly reducing computational costs and training time. A wind tunnel experiment is conducted to validate the effectiveness of the algorithm under varying wind directions. The results demonstrate that MTD3-BC successfully mitigates wake effects, delivering farm-level power gains of approximately 10\% over the baseline greedy strategy and performance on par with a data-calibrated model-based wake-steering benchmark, while requiring no wake model and only a small fraction of the training cost of online RL. To our knowledge, this is the first time an offline RL wind farm control policy has been validated and demonstrated experimentally.

Closest papers

Nearest neighbours by meaning across the whole index.

Noise Steering: Exploiting Wake Steering and Acoustic Directivity for Noise-Constrained Wind Farm Control via Reinforcement Learning

cs.AI eess.SYphysics.ao-phphysics.flu-dyn ·2026-10-27 ·Martín de Frutos, Baptiste Corban, Claire Bizon Monroc, Ana Bušić, Donatien Dubuc +2 more
similarity 0.71
Abstract · 17 pages, 11 figures

Existing noise mitigation strategies for wind farms usually consist in reducing turbines power output, without accounting for the wake interactions between them and the noise emission directivity. This leads to over-conservative policies and unexploited wind power. Yet, turbine yaw can simultaneously affect wake interactions and the direction of noise emission, offering an additional degree of freedom for farm-level noise control. We therefore propose Noise Steering, a model-free reinforcement learning (RL) strategy that jointly controls turbine yaw angles and power setpoints to maximise farm power while satisfying a local noise constraint in a defined region. To enable training at a reduced computational cost, a low-fidelity acoustic model is coupled with the steady-state FLORIS simulator. The RL agent effectively learns how to redirect wakes and noise directivity away from the protected region while redistributing power across the farm. On a six-turbine test farm, the policy satisfies the 40 dB noise constraint throughout the full range of turbulent winds, with an average power deficit of only 2% relative to greedy unimpeded control, and up to 7.7% at high wind speeds. In comparison, the industry-standard noise-reduced operation (NRO) baseline here generates an average power loss of 17%. When deployed in the higher-fidelity FASTFarm simulator, the policy largely retains its performance under steady and turbulent inflow, despite differences in simulation fidelity. These results demonstrate the potential of combining yaw control and power derating through RL to improve the power-noise trade-off in wind farm operation.

0.20

Distributed Model-Free Turbine Repositioning for Wake Overlap Minimization in Floating Offshore Wind Farms

eess.SY math.DS ·2026-10-06 ·Zekai Chen, Ryozo Nagamune
similarity 0.63
Abstract · This paper has been submitted to the American Control Conference (ACC) 2027

This paper presents a model-free method for turbine repositioning control in floating offshore wind farms. The method uses a distributed online optimization framework to mitigate the wake effect. Conventional approaches rely on centralized, model-based control architectures, which suffer from poor practical feasibility. To overcome this limitation and address nonconvexity and inaccessible gradient information, we incorporate physics-informed knowledge of wake dynamics into the control system design. We demonstrate the effectiveness of the proposed method by comparing it with a centralized, model-based baseline in mid-fidelity simulations. Results indicate that the proposed method achieves slightly better performance while significantly reducing computational cost.

0.23

Value-oriented policy distillation for interpretable wind farm power control

cs.AI eess.SY ·2026-11-05 ·Max Bruninx, Timothy Verstraeten, Jan Decuyper, Ann Nowé, Jan Helsen
similarity 0.62
Abstract

Wind farm power control can help the system maintain balance and avoid blackouts. In Europe, this can be achieved through implicit balancing, where a wind farm adjusts its power setpoint in anticipation of the imbalance price using information published by the Transmission System Operator. Thereby, the wind farm helps restore system balance while also reducing its own imbalance costs resulting from forecast errors. However, implicit balancing is a challenging problem since the settlement price is unknown in advance and can be impacted by the pursued control strategy. Reinforcement learning methods could help to solve this problem, but lack the interpretability needed to be deployed on real wind farms. Therefore, we propose a policy distillation framework using explainable boosting machines (EBMs), a variant of generalized additive models, to obtain a transparent control policy. To increase the value captured by the EBM model, we introduce a value-oriented perspective to policy distillation inspired by research in the field of energy forecasting. Furthermore, we suggest to include expert knowledge during model design, such that the control strategy is also justifiable to stakeholders. Using real data of the Belgian grid and an offshore wind farm, we find that our value-oriented and expert-based controller can reduce imbalance costs by 82%, while outperforming a price threshold controller as well as a policy distillation baseline using decision trees.

0.21

Stochastic Sequential Feedback Optimization for Wind Farm Control under Time-Varying Inflow

eess.SY math.OC ·2026-10-02 ·Shijie Huang, Sergio Grammatico
benchmark similarity 0.61
Abstract

We consider steady-state optimization of nonlinear dynamical systems under time-varying stochastic operating conditions, where the objective is an expected-value function over the current operating-condition distribution, so that changes in the distribution induce a moving stochastic optimum. To address this problem, we propose stochastic sequential feedback optimization (S2FO), which combines projected feedback-gradient updates with sequential sensitivity estimates obtained by linearizing the plant at realized stochastic operating conditions. We analyze the closed-loop performance of S2FO through expected dynamic regret, instantaneous tracking in expectation, and almost-sure convergence under diminishing step sizes. We then apply the method to wind farm power maximization under time-varying stochastic inflow. Simulations on a benchmark layout and on a realistic offshore wind farm show that S2FO improves average power generation when persistent wake interactions are present, and that its advantage decreases as wind-direction variability increases.

0.15

Control of hybrid wind-wave energy systems using reinforcement learning

eess.SY ·2026-08-11 ·Zechuan Lin, Kemeng Chen, Maosen Fan, Xiaofan Li, Xi Xiao +1 more
similarity 0.61
Abstract

Integrating wave energy converters (WECs) with floating offshore wind turbines (FOWTs), to form hybrid wind-wave energy (HWWE) systems, is a promising approach to achieve further cost reduction for offshore renewable energy. In such systems, the control of the integrated WECs plays an important role, with the potential to generate additional wave energy while simultaneously suppressing floating platform motion. However, HWWE systems are characterized by complex dynamics, making accurate modelling only viable through numerical simulation, and posing significant challenges for control design. This paper proposes a reinforcement learning (RL) control framework for HWWE systems, in which the real-time control policy is learned directly through interactions with high-fidelity simulation. A numerical model is established for a HWWE system consisting of an IEA 15 MW wind turbine, a VolturnUS semi-submersible platform, and three torus-type WECs, which is then employed as the RL training environment. Control performance is evaluated in terms of both wave energy generation and platform motion reduction, two competing objectives, from a Pareto perspective. It is shown that the proposed RL controller achieves substantial Pareto improvements over conventional control strategies, e.g., over 75\% higher wave energy capture at the same platform motion level, or nearly 50\% lower motion at the same energy capture level, thereby significantly extending the attainable performance boundary of HWWE systems.

0.04

LLM-Enhanced Deep Reinforcement Learning with Human-in-the-Loop Calibration for Real-Time Optimal Power Flow

eess.SY cs.AI ·2026-11-11 ·Jiayang Tu, Bozhen Jiang, Bojun Du, Yunxiao Ling, Ershun Du +1 more
similarity 0.60
Abstract

Real-time optimal power flow (RT-OPF) requires fast and reliable decision-making under time-varying operating conditions. However, many practical operating preferences and safety related requirements are difficult to encode using conventional numerical rewards alone, which limits the effectiveness and controllability of deep reinforcement learning (DRL) for RT-OPF. To address this issue, this paper proposes an LLM-enhanced DRL framework with human-in-the-loop (HITL) for RT-OPF. The proposed method augments the physics-based reward with sparse language evaluative feedback during offline training, so that semantic operational requirements can be incorporated into policy learning. To improve reliability in RT-OPF, a selective HITL calibration mechanism is further introduced to inspect and correct questionable LLM judgments before they continuously shape the learning process. Case studies on the IEEE 30-bus and IEEE 118-bus systems show that LLM feedback improves policy safety over vanilla TD3, while further HITL calibration achieves the best overall performance and more stable training behavior. These results demonstrate the effectiveness of combining language guidance and expert calibration for RT-OPF.

0.14

EndoWake: Modeling Wind by Linear Programming for Wind Farm Layout Optimization

cs.CE math.OC ·2026-09-30 ·Martina Fischetti, Matteo Fischetti
similarity 0.60
Abstract · 30 pages + 12 pages of supplementary material (appended after the references). Replication package: https://doi.org/10.5281/zenodo.22017424

The Wind Farm Layout Optimization problem consists of placing a given number of turbines within a given area so as to maximize energy production. One of its main challenges is the wake effect between turbines, which can strongly affect production and is a cumbersome function to evaluate and to embed in an optimization framework. In this paper we introduce a new endogenous wake model (EndoWake), based on the idea of treating the wind speed at every grid point as a decision variable, linked to its upwind neighbors through linear propagation constraints. This makes it possible to incorporate the wake equations directly into a Mixed-Integer Linear Programming (MILP) model. We then develop such a MILP model and, in particular, introduce a new class of valid "band" inequalities, showing that they substantially strengthen the root bound. Building on this, we design a branch-and-check scheme: the master problem retains only the site variables together with one hypograph variable per cell and scenario, integer solutions are evaluated by an exact linear-time oracle, and the optimality cuts are problem-specific "upwind cuts". Computational results are reported on a ladder of square grids under a four-sector axial wind rose, demonstrating the effectiveness of the proposed improvements.

0.58

Data-Driven Surrogate-Model for Real-Time Loads Prediction in Floating Wind Farms

eess.SY ·2026-10-13 ·Federico Bellini, Majid Ebrahimi, Alessandro Fontanella, Marcus Becker, Jan-Willem Van Wingerden +2 more
similarity 0.59
Abstract

This work presents a data-driven load-surrogate modelling framework for real-time control of floating offshore wind farms. The proposed methodology uses mid-fidelity simulations performed with OpenFAST and FAST.Farm together with MLife fatigue analysis to generate Damage Equivalent Load (DEL) datasets of floating wind turbines under wake steering conditions. A two-stage surrogate modelling approach is adopted: first, a Feed-Forward Neural Network (FFNN) is trained using data from simulations of a single floating wind turbine, while a second stage corrects the residual error between the FFNN predictions and FAST.Farm simulations using either Gaussian Process Regression (GPR) or an additional FFNN. The results show that the proposed framework can estimate DELs with errors generally below 15% for tower base bending moment, blade root bending moment, yaw bearing moment, and fairlead tension, while maintaining computational times compatible with real-time applications. The study also highlights the challenges related to generalization of the load-surrogate model under varying environmental and control conditions, especially for large floating wind farms. Overall, the proposed methodology demonstrates the potential of surrogate-based DEL estimation for load-aware floating wind farm control applications.

0.19