CS Paper Radar

Noise Steering: Exploiting Wake Steering and Acoustic Directivity for Noise-Constrained Wind Farm Control via Reinforcement Learning

Martín de Frutos, Baptiste Corban, Claire Bizon Monroc, Ana Bušić, Donatien Dubuc, Jiamin Zhud, Esteban Ferrer

cs.AI eess.SYphysics.ao-phphysics.flu-dyn ·submitted 2026-10-27 09:23 UTC · v2

Abstract

Existing noise mitigation strategies for wind farms usually consist in reducing turbines power output, without accounting for the wake interactions between them and the noise emission directivity. This leads to over-conservative policies and unexploited wind power. Yet, turbine yaw can simultaneously affect wake interactions and the direction of noise emission, offering an additional degree of freedom for farm-level noise control. We therefore propose Noise Steering, a model-free reinforcement learning (RL) strategy that jointly controls turbine yaw angles and power setpoints to maximise farm power while satisfying a local noise constraint in a defined region. To enable training at a reduced computational cost, a low-fidelity acoustic model is coupled with the steady-state FLORIS simulator. The RL agent effectively learns how to redirect wakes and noise directivity away from the protected region while redistributing power across the farm. On a six-turbine test farm, the policy satisfies the 40 dB noise constraint throughout the full range of turbulent winds, with an average power deficit of only 2% relative to greedy unimpeded control, and up to 7.7% at high wind speeds. In comparison, the industry-standard noise-reduced operation (NRO) baseline here generates an average power loss of 17%. When deployed in the higher-fidelity FASTFarm simulator, the policy largely retains its performance under steady and turbulent inflow, despite differences in simulation fidelity. These results demonstrate the potential of combining yaw control and power derating through RL to improve the power-noise trade-off in wind farm operation.

Comment
17 pages, 11 figures

Closest papers

Nearest neighbours by meaning across the whole index.

Offline Reinforcement Learning for Wind Farm Control: A Wind Tunnel Study under Dynamic Wind Directions

cs.LG ·2026-09-11 ·Yuhan Su, Hongyang Dong, Simone Tamaro, Filippo Campagnolo, Carlo L. Bottasso +1 more
benchmark similarity 0.71
Abstract

This paper addresses the wind farm power maximization problem in the presence of wind direction changes. Specifically, a model-free Modified Twin Delayed Deep Deterministic Policy Gradient with Behavior Cloning (MTD3-BC) algorithm is proposed to tackle this task through yaw control under varying wind direction conditions. MTD3-BC is an offline reinforcement learning (RL) algorithm that aims to infer good behavior from only a precollected offline dataset. Additionally, to ensure smooth and moderate yaw adjustments, a new action consistency term is introduced into the policy optimization objective. Unlike online RL methods, MTD3-BC does not require extensive interactions with a wind farm simulator during training, significantly reducing computational costs and training time. A wind tunnel experiment is conducted to validate the effectiveness of the algorithm under varying wind directions. The results demonstrate that MTD3-BC successfully mitigates wake effects, delivering farm-level power gains of approximately 10\% over the baseline greedy strategy and performance on par with a data-calibrated model-based wake-steering benchmark, while requiring no wake model and only a small fraction of the training cost of online RL. To our knowledge, this is the first time an offline RL wind farm control policy has been validated and demonstrated experimentally.

0.30

Distributed Model-Free Turbine Repositioning for Wake Overlap Minimization in Floating Offshore Wind Farms

eess.SY math.DS ·2026-10-06 ·Zekai Chen, Ryozo Nagamune
similarity 0.63
Abstract · This paper has been submitted to the American Control Conference (ACC) 2027

This paper presents a model-free method for turbine repositioning control in floating offshore wind farms. The method uses a distributed online optimization framework to mitigate the wake effect. Conventional approaches rely on centralized, model-based control architectures, which suffer from poor practical feasibility. To overcome this limitation and address nonconvexity and inaccessible gradient information, we incorporate physics-informed knowledge of wake dynamics into the control system design. We demonstrate the effectiveness of the proposed method by comparing it with a centralized, model-based baseline in mid-fidelity simulations. Results indicate that the proposed method achieves slightly better performance while significantly reducing computational cost.

0.23

Control of hybrid wind-wave energy systems using reinforcement learning

eess.SY ·2026-08-11 ·Zechuan Lin, Kemeng Chen, Maosen Fan, Xiaofan Li, Xi Xiao +1 more
similarity 0.62
Abstract

Integrating wave energy converters (WECs) with floating offshore wind turbines (FOWTs), to form hybrid wind-wave energy (HWWE) systems, is a promising approach to achieve further cost reduction for offshore renewable energy. In such systems, the control of the integrated WECs plays an important role, with the potential to generate additional wave energy while simultaneously suppressing floating platform motion. However, HWWE systems are characterized by complex dynamics, making accurate modelling only viable through numerical simulation, and posing significant challenges for control design. This paper proposes a reinforcement learning (RL) control framework for HWWE systems, in which the real-time control policy is learned directly through interactions with high-fidelity simulation. A numerical model is established for a HWWE system consisting of an IEA 15 MW wind turbine, a VolturnUS semi-submersible platform, and three torus-type WECs, which is then employed as the RL training environment. Control performance is evaluated in terms of both wave energy generation and platform motion reduction, two competing objectives, from a Pareto perspective. It is shown that the proposed RL controller achieves substantial Pareto improvements over conventional control strategies, e.g., over 75\% higher wave energy capture at the same platform motion level, or nearly 50\% lower motion at the same energy capture level, thereby significantly extending the attainable performance boundary of HWWE systems.

0.04

Stochastic Sequential Feedback Optimization for Wind Farm Control under Time-Varying Inflow

eess.SY math.OC ·2026-10-02 ·Shijie Huang, Sergio Grammatico
benchmark similarity 0.61
Abstract

We consider steady-state optimization of nonlinear dynamical systems under time-varying stochastic operating conditions, where the objective is an expected-value function over the current operating-condition distribution, so that changes in the distribution induce a moving stochastic optimum. To address this problem, we propose stochastic sequential feedback optimization (S2FO), which combines projected feedback-gradient updates with sequential sensitivity estimates obtained by linearizing the plant at realized stochastic operating conditions. We analyze the closed-loop performance of S2FO through expected dynamic regret, instantaneous tracking in expectation, and almost-sure convergence under diminishing step sizes. We then apply the method to wind farm power maximization under time-varying stochastic inflow. Simulations on a benchmark layout and on a realistic offshore wind farm show that S2FO improves average power generation when persistent wake interactions are present, and that its advantage decreases as wind-direction variability increases.

0.15

Value-oriented policy distillation for interpretable wind farm power control

cs.AI eess.SY ·2026-11-05 ·Max Bruninx, Timothy Verstraeten, Jan Decuyper, Ann Nowé, Jan Helsen
similarity 0.57
Abstract

Wind farm power control can help the system maintain balance and avoid blackouts. In Europe, this can be achieved through implicit balancing, where a wind farm adjusts its power setpoint in anticipation of the imbalance price using information published by the Transmission System Operator. Thereby, the wind farm helps restore system balance while also reducing its own imbalance costs resulting from forecast errors. However, implicit balancing is a challenging problem since the settlement price is unknown in advance and can be impacted by the pursued control strategy. Reinforcement learning methods could help to solve this problem, but lack the interpretability needed to be deployed on real wind farms. Therefore, we propose a policy distillation framework using explainable boosting machines (EBMs), a variant of generalized additive models, to obtain a transparent control policy. To increase the value captured by the EBM model, we introduce a value-oriented perspective to policy distillation inspired by research in the field of energy forecasting. Furthermore, we suggest to include expert knowledge during model design, such that the control strategy is also justifiable to stakeholders. Using real data of the Belgian grid and an offshore wind farm, we find that our value-oriented and expert-based controller can reduce imbalance costs by 82%, while outperforming a price threshold controller as well as a policy distillation baseline using decision trees.

0.21

Refinement-Based Flow Policy Optimization

cs.LG cs.AI ·2026-09-14 ·Bumgeun Park, Hyukjun Yang, Donghwan Lee
similarity 0.56
Abstract

Flow-based policies offer an expressive representation for online reinforcement learning, but conventional flow matching requires samples drawn from the distribution to be modeled. This poses a challenge when the desired action distribution is defined only implicitly by a Q-function, since directly sampling actions from the resulting distribution is generally intractable. We propose Refinement-Based Flow Policy Optimization (RFPO), a novel framework for training a flow policy in online reinforcement learning by alternating between Q-guided sample refinement and self-target flow matching. RFPO first generates actions from Gaussian noise using the current flow policy and then uses a finite-step stochastic refinement procedure to move them toward an energy-based distribution induced by the Q-function. Each refined action is then paired with its corresponding initial noise sample and used as a fixed target for flow-matching training. By repeatedly refining its own outputs and learning from the resulting targets, RFPO incorporates Q-guidance into the policy without requiring direct samples from the target distribution, while retaining the capacity to represent multiple action modes. We further provide a theoretical analysis of the distributional dynamics induced by RFPO. Across six continuous-control tasks, RFPO matches or outperforms a standard Gaussian-policy baseline on almost every task. Experiments on six synthetic two-dimensional target distributions with diverse geometries demonstrate that RFPO captures complex multimodal structure without mode collapse.

0.46

Every minute counts: Wind farm power control during imbalance settlement using distributional soft actor-critic

eess.SY ·2026-10-19 ·Max Bruninx, Seyed Soroush Karimi Madahi, Timothy Verstraeten, Jan Decuyper, Chris Develder +2 more
similarity 0.55
Abstract

Wind farms can provide flexibility by curtailing their output during periods of excess generation in the power grid. This can be done through implicit balancing, where the power setpoints of the farm are adapted to anticipate the imbalance price. By doing so, the wind farm helps reduce the resulting system imbalance while also lowering its own costs. However, implicit balancing is challenging since the imbalance prices are unknown in advance and hard to predict. Furthermore, the control strategy of the wind farm can affect the imbalance price formation. To address both challenges, we develop a control policy with 1-minute setpoints using distributional soft actor-critic. Evaluated on real data of the Belgian grid and an offshore wind farm, we show that our method outperforms a price threshold controller, often used in industry, by 88% and performs on par with soft actor-critic. Furthermore, unlike soft actor-critic, our approach can learn a risk-averse policy by optimizing a risk measure, such as conditional value at risk. The risk-averse policy was found to better account for its own impact on the price, showing a reduction in overshoots, while achieving a similar degree of imbalance cost savings. Last, we find that using 1-minute instead of 15-minute setpoints further reduces imbalance costs for all evaluated control policies.

0.32

EndoWake: Modeling Wind by Linear Programming for Wind Farm Layout Optimization

cs.CE math.OC ·2026-09-30 ·Martina Fischetti, Matteo Fischetti
similarity 0.55
Abstract · 30 pages + 12 pages of supplementary material (appended after the references). Replication package: https://doi.org/10.5281/zenodo.22017424

The Wind Farm Layout Optimization problem consists of placing a given number of turbines within a given area so as to maximize energy production. One of its main challenges is the wake effect between turbines, which can strongly affect production and is a cumbersome function to evaluate and to embed in an optimization framework. In this paper we introduce a new endogenous wake model (EndoWake), based on the idea of treating the wind speed at every grid point as a decision variable, linked to its upwind neighbors through linear propagation constraints. This makes it possible to incorporate the wake equations directly into a Mixed-Integer Linear Programming (MILP) model. We then develop such a MILP model and, in particular, introduce a new class of valid "band" inequalities, showing that they substantially strengthen the root bound. Building on this, we design a branch-and-check scheme: the master problem retains only the site variables together with one hypograph variable per cell and scenario, integer solutions are evaluated by an exact linear-time oracle, and the optimality cuts are problem-specific "upwind cuts". Computational results are reported on a ladder of square grids under a four-sector axial wind rose, demonstrating the effectiveness of the proposed improvements.

0.58