CS Paper Radar

The Cost of Coverage: Conformalising Native Forecast Uncertainty for GB Wind Power

Carl Somers

cs.LG stat.AP ·submitted 2026-11-06 18:02 UTC · v1

Abstract

Uncertainty bands on wind forecasts help inform reserve sizing and commitments, but often fail to deliver their stated coverage. We apply conformal prediction to native bands to see what it adds and what it costs, comparing six forecasters, from ARIMA to a tabular foundation model, at two wind farms in Great Britain (GB) under operational conditions. We show that calibration repairs empirical marginal coverage with little or no Winkler Score penalty, but this apparent success conceals under-coverage in high winds. Calibrating by forecast wind regime turns marginal coverage into regime-level coverage, cutting day-ahead out-of-band volume in high winds by over 40% relative to pooled calibration while narrowing bands overall.

Closest papers

Nearest neighbours by meaning across the whole index.

Distribution-Free Uncertainty Quantification for Weather Forecasting: Conformal Prediction Under Spatiotemporal Dependence

cs.LG ·2026-11-18 ·Mariya
benchmarkcode/data similarity 0.64
Abstract · 27 pages, 5 figures. Code: https://github.com/Mariyaaleem/conformal-wx

Conformal prediction gives finite-sample, distribution-free marginal coverage and has begun to be applied to gridded weather forecasts. Marginal coverage concerns the long-run fraction of covered points at a location, not how coverage behaves jointly across space and time on a given day. I benchmark eight interval-construction procedures (Gaussian, pooled and per-cell split conformal, per-season, adaptive conformal inference and rolling windows) for gridded 2 m temperature forecasts at eight lead times from 6 hours to 10 days, using ERA5 at 5.625 degree resolution, a deliberately modest convolutional forecaster and ECMWF's operational HRES, with calibration in 2017-2018 and testing in 2019-2021. Alongside coverage, width and interval score, I report dependence diagnostics: a spatial overdispersion index, quantiles of daily spatial coverage and miss persistence relative to a shuffled-days null. Per-cell methods reach near-nominal marginal coverage, yet the day-to-day variance of the fraction of cells covered is roughly 3 to 10 times what independent cells would give, and misses cluster in time; both effects grow with lead time for the CNN, also appear for HRES, and are shared by all methods. An oracle that divides each day's normalised errors by that day's mean removes 81 to 92 percent of the excess dispersion for the CNN and 93 to 98 percent for HRES, so days of large errors across much of the grid account for most of it. A day-scale predicted from seasonal terms and already-verified errors lowers the overdispersion index without widening intervals but improves interval score by at most 0.6 percent; on HRES from 12 hours to 3 days it has the lowest overdispersion index while ACI or season-conditional calibration has the best interval score, so no method is best on both. The study covers one variable, one coarse grid, a reanalysis target and two forecasters.

0.52

Reconstructing Boundary Layer Wind Profiles with Machine Learning: A Comparative Study on IGRA Radiosonde Data

cs.LG cs.NEphysics.ao-ph ·2026-11-09 ·Fedor Buzaev, Dmitry Efremenko, Daria Pugacheva, Fedor Ratnikov, Egor Bugaev +2 more
publishedbenchmark similarity 0.60
Abstract · 10 pages, 8 figures. Accepted at the 2nd Workshop on AI-Ready Data for Science Discovery (ADSD), IEEE ICDM 2026

Wind profiles in the atmospheric boundary layer (ABL) drive short-range forecasting and wind-energy siting, yet sparse radiosonde coverage forces practitioners to interpolate them from a handful of physical predictors. We benchmark modern tabular learners against analytical parameterizations on 158,097 IGRA soundings. CatBoost, TabM, and FT-Transformer reduce RMSE to 1.3-1.6 m/s, against approximately 3 m/s for power-law and Monin-Obukhov fits; once the predictor set is fixed the three architectures differ by less than 0.05 m/s, so accuracy is driven by predictors rather than architecture. A cohort analysis of within-bin profile dispersion places an empirical floor of approximately 1.77 m/s on any point-wise predictor that uses standard radiosonde features, consistent with prior eddy-viscosity inversions. A one-dimensional Fourier Neural Operator applied as a post-hoc corrector to frozen tabular predictions reduces near-surface RMSE on U.S. stations from 1.5 to 1.2 m/s, while leaving upper-layer error unchanged. Further gains should come from new predictors rather than larger models.

0.28

Guarded Agent-in-the-Loop Residual Adaptation for Evolving Wind Power Regimes

cs.MA ·2026-10-26 ·Yuhang Dong, Min Xia, Jinghong Li, Yue Dai, Xinyue Wang +1 more
similarity 0.60
Abstract

Wind power forecasting models are often deployed as fixed temporal backbones, but the errors they make can vary across meteorological regimes such as cold waves, heat-and-calm periods, productive typhoon, and destructive storms. We present a guarded Agent-in-the-loop residual adaptation framework that leaves the temporal backbone unchanged and adapts only a small set of scene-conditioned residual experts. An Agent layer interprets external scene information, identifies the current regime, selects residual experts, and applies a correction only when recent observed errors support it. New experts enter the pool after they outperform their initializations on a validation set, keeping model growth under explicit control. Backbone-only and Agent-enhanced forecasts are compared at lead times of 4, 8, 12, 24, and 48 across nine time-series backbones. Across these runs, the guarded framework lowers both MAE and RMSE for all 45 backbone-horizon combinations, with average MAE reductions ranging from 11.21% to 20.26% across backbones. Online-selection baselines, component ablations, and routing analyses identify the roles of residual experts, scene-aware routing, and the online acceptance rule. Candidate-scene runs evaluate the promotion path under controlled conditions; long-term unattended deployment was not evaluated.

0.13

Aries: A Proprietary Medium-Range Weather Prediction Model for the Energy Industry

cs.LG physics.ao-ph ·2026-09-09 ·Lukas Hedegaard Morsing, Arian Bakhtiarnia, Jonas Lynge Olesen, Tómas Bragi Björnsson Leth, Christian Gøbel Bach
similarity 0.59
Abstract

Medium-range weather forecasting underpins operational and planning decisions across the energy industry. Developing competitive weather models was once the domain of national meteorological centers, but recent advances in machine-learned weather prediction (MLWP) have opened the field to industry. We present Aries, a SwinTransformer-based MLWP model developed at InCommodities. Aries is trained on ERA5 reanalysis data at 0.25\textdegree{} resolution, predicting 74 prognostic and 11 diagnostic atmospheric variables. We evaluate the model on 2025 ECMWF Analysis initializations, ensuring a recent and strictly out-of-sample test period for all models compared. On 10-metre wind speed, Aries outperforms both ECMWF HRES and AIFS in terms of RMSE for lead times up to four days, while on 2-metre temperature it achieves RMSE on par with AIFS operational. These results demonstrate that proprietary development of competitive weather models is technically viable, supporting a broader set of forecasts available for operational and planning applications in the energy industry.

0.10

Adaptive Calibration of Day-Ahead Flexibility Reserve Requirements Under Distribution Shift

eess.SY ·2026-10-12 ·Napoleon Costilla-Enriquez, Yang Weng, Miguel A. Ortega-Vazquez
theory similarity 0.59
Abstract

The increasing integration of variable renewable generation introduces weather-driven variability and uncertainty into power-system operations. System operators manage the resulting deviations between day-ahead forecasts and real-time net load by scheduling upward and downward flexibility reserves. These requirements have traditionally been set using simple deterministic rules. Newer data-driven methods tailor reserves to forecasts and operating conditions. However, their reliability may deteriorate when conditions depart from the historical data. We address this limitation with a sequential framework that updates day-ahead reserve requirements as operating outcomes become available. Within this framework, Adaptive Conformal Inference targets the frequency of insufficient reserve, whereas Rolling Risk Control targets expected uncovered deviation, the mean residual beyond the scheduled reserve. We prove finite-horizon bounds on realized miscoverage and capped uncovered-deviation loss. We test the framework on real ERCOT data in a chronological deployment simulation that respects operational timing: outcomes from one day first affect the reserve schedule two days later. Training and validation use 2022--2023 data; the uninterrupted test covers 2024--2025. Coverage calibration achieves approximately 90% coverage, with upward and downward miscoverage near 5%. Risk calibration brings expected uncovered deviations close to their budgets while using 5.3% less reserve than the uncalibrated requirements. The framework thus produces adaptive reserve requirements that provably control the selected reliability metrics over finite horizons without assuming stationarity or exchangeability.

0.22

WPBench: A Comprehensive Benchmark for Wind Power Forecasting

cs.LG cs.AI ·2026-09-21 ·Yuhan Zhu, Jilin Hu, Xinying Cai, Yingshan Li, Li Ma +5 more
top venuebenchmark similarity 0.57
Abstract · Accepted by ICDE 2027

Accurate, reliable, and deployable wind power forecasting is critical for power system dispatch, renewable energy integration, and electricity market operations. Progress in this field hinges on the ability to empirically and comprehensively benchmark forecasting methods. Yet existing benchmarks fall short of supporting systematic evaluation in four key aspects: 1) limited coverage of wind power scenarios across turbine scale, variable composition, and spatial structure; 2) incomplete coverage of forecasting model families; 3) evaluation metrics misaligned with wind power requirements; and 4) limited structure-aware diagnostics beyond individual temporal patterns. To address these limitations, we propose WPBench, a comprehensive, fair, and extensible benchmark for wind power forecasting. WPBench integrates 26 public datasets organized by turbine scale and variable composition, spanning single-turbine, multi-turbine, univariate, and multivariate settings. Under unified processing, training, and evaluation protocols, it benchmarks 19 representative models covering traditional methods, deep temporal models, spatio-temporal models, and foundation models. Beyond point-wise errors, WPBench assesses forecast-curve fidelity and computational efficiency, and delivers structure-aware diagnostics across temporal, variable-dependency, and spatial-dependency perspectives. Together, these capabilities enable systematic model comparison across diverse wind scenarios and provide a reusable platform for future research.

0.56

Uncertainty-Aware End-to-End AI Weather Forecasting: Disentangling Observation and Model Contributions

cs.LG physics.ao-ph ·2026-08-31 ·Rodrigo Almeida, Noelia Otero, Jost Arndt, Simon Baur, Wojciech Samek +1 more
similarity 0.57
Abstract · 25 pages, 16 figures

End-to-end weather forecasting systems produce skillful global gridded and station forecasts directly from raw Earth observations, replacing the numerical weather prediction pipeline, including data assimilation, at a fraction of its cost. These systems are deterministic and issue no uncertainty. Here we render the Aardvark Weather model probabilistic by attaching one stochastic mechanism to each component: learned, input-dependent noise at the observation encoder, aimed at the uncertainty inherited from the observing system, and Monte Carlo dropout in the processor, aimed at the uncertainty of the learned dynamics. The resulting nested ensemble attributes forecast spread to the two sources through a law-of-total-variance decomposition, cross-checked by withholding observation streams. Probabilistic finetuning significantly improves the mean forecast, by 11% on average across variables and lead times. The ensemble is close to calibrated against ERA5 through the medium range, is at least as accurate as the deterministic model at weather stations while beating it in relative economic value at every lead time, and remains considerably behind the operational ECMWF IFS ensemble. The encoder branch behaves as observation-driven uncertainty. Component-attributed uncertainty makes end-to-end forecasts more transparent, a step toward observation-driven digital twins of the atmosphere.

0.22

StreamLTT: Risk Certification for Policy Selection under Stream Dependence

cs.LG math.STstat.ML ·2026-11-02 ·Jinwen Xu, Qin Lu
similarity 0.56
Abstract

Learn-then-Test (LTT) provides a general framework for certifying deployment policies against a prescribed risk target, but its standard formulation targets a fixed population risk estimated from calibration losses. In sequential deployment, losses may instead be dependent and nonstationary, so the risk of a fixed policy can evolve with the observed history. We introduce StreamLTT, which replaces this fixed population risk with average predictable risk, defined by conditioning each decision's loss on the information available before its outcome is revealed. Conditional centering turns empirical deviations from this target into a martingale sum, enabling both a test that adapts to predictable variance and a finite-sample Hoeffding alternative. Combined with family-wise error rate control, StreamLTT yields a simultaneously certified policy region that supports subsequent deployment selection. Across seven forecasting datasets and seven deployment risks, StreamLTT places the certified frontier substantially closer to the prescribed risk target than IID, robust-variance, block-bootstrap, and concentration-based alternatives, while empirical family-wise error closely follows the prescribed level. A finite-sample ablation further shows that the predictable-risk formulation drives the tighter risk frontier, while adaptation to predictable variance makes more effective use of the available family-wise error budget. StreamLTT therefore extends risk-certified policy selection to dependent deployment streams by addressing both what risk should be certified and how it should be tested.

0.29