CS Paper Radar

Can AI Forecast the Extremes? Evaluating Probabilistic Weather Models on Tropical Cyclones

Nathan Chalumeau, Xavier Renard, Remi Meynadier, Hugo Rakotoarimanga, Marcin Detyniecki, Emmanuel de Bezenac

cs.LG physics.ao-ph ·submitted 2026-10-15 15:35 UTC · v1
benchmark

Abstract

Weather prediction is undergoing an important shift, with data-driven AI models outperforming leading numerical weather prediction systems on average at a fraction of the computational cost. Yet, whether this success extends to extreme events remains contested. Earlier evaluations have exposed important shortcomings, but have largely focused on deterministic models and metrics, leaving the capabilities of the latest probabilistic models less understood. We conduct an extensive benchmark of tropical cyclone forecasting, comparing GenCast, FourCastNet 3, and FGN with ECMWF's operational physics-based ensemble, IFS-ENS. We develop a suite of probabilistic metrics and diagnostic analyses to assess cyclone tracking, intensity biases, uncertainty, and the wind-pressure relationship that characterizes storm structure. We find that, despite remaining limitations in intensity and ensemble spread, leading AI models substantially outperform IFS-ENS in cyclone tracking: FGN achieves better probabilistic location scores for 90% of evaluated storms in 2025. Further analyses reveal model-specific failure modes and suggest a lingering influence of pretraining data, pointing to opportunities for targeted improvement. Nonetheless, these results suggest that AI models are capable of capturing physical relationships that remain predictive well beyond typical weather conditions, motivating further investigation into how these capabilities emerge and how to strengthen them.

Closest papers

Nearest neighbours by meaning across the whole index.

Quantifying the added value of human forecaster discussions for tropical cyclone intensity prediction

cs.LG cs.CL ·2026-11-16 ·Ilia Azizi, Frederick Iat-Hin Tam, Marc-Olivier Boldi, Valérie Chavez-Demoulin, Tom Beucler
hot direction similarity 0.76
Abstract · 74 pages, 24 figures, 35 tables

The National Hurricane Center routinely releases Tropical Cyclone Discussions that offer insights into its tropical cyclone forecasts. This study examines the added value of these discussions for intensity forecasting beyond a set of numerical predictors. For 637 Atlantic and eastern North Pacific tropical cyclones from the 2002-2023 hurricane seasons, we analyze the discussion text and its combination with 13 hurricane environmental predictors from SHIPS and ERA5 10-meter wind fields, for forecasting intensity changes at lead times of 24 to 120 hours using three modeling methods. In one approach, an end-to-end deep learning framework for the text shows that the discussions alone offer comparable value for 24-hour intensity forecasts to the SHIPS predictors. Adding discussion information to the SHIPS predictors in an ensemble of five such models reduces the mean absolute error of 24-hour maximum sustained wind forecasts for 2022-2023 cyclones from 9.89 to 8.90 kt, with a confidence interval that excludes zero when whole storms are resampled. The ERA5 wind fields have a minor effect on mean absolute error in wind predictions, and model rankings differ across predicted variables, lead times and model types. We provide prediction intervals for these forecasts, whose coverage falls below the 90% target during rapid intensification. Analysis of the text shows that words related to intensity and intensity change (such as intensify, weaken, eyewall and category) are important for the model predictions. This retrospective study shows the added value of the forecasters' discussions and suggests that multi-modal learning can help understand the predictive contribution of expert human judgment. The reanalysis data differ from real-time analyses, and the discussions are issued at least three hours after the valid time of the other inputs, so these results should not be interpreted operationally.

0.26

FAST-ML: A Hybrid Physics-Machine Learning Framework for Tropical Cyclone Intensity Forecasting

cs.LG physics.ao-ph ·2026-09-22 ·Shijie Xiao, Jonathan Lin, Thomas Ehrmann, Ali Sarhadi
publishedcode/data similarity 0.75
Abstract · Code and models available at https://github.com/Shijie-Xiao/FAST-ML

Rapid intensification (RI) remains one of the most consequential and difficult aspects of tropical cyclone (TC) forecasting. Although full-physics numerical weather prediction models can represent the processes governing RI, resolving storm-environment interactions remains computationally expensive, while purely data-driven approaches often lack physical interpretability. We present FAST-ML, a hybrid framework that bridges data-driven efficiency with physical constraints. A physically informed dual-stream neural parameterization ingests 3D ERA5 fields to diagnose ventilation controls---environmental wind shear and mid-level entropy deficit. By optimizing these parameters end-to-end through a differentiable FAST intensity model, this architecture establishes a robust new paradigm for observation-driven parameter optimization, ensuring storm evolution remains strictly governed by thermodynamic principles. By better capturing the storm's continuous intensity evolution, FAST-ML improves upon its physical baseline, reducing ensemble CRPS across forecast lead times, with a reduction of approximately 31% at 60 h and nearly halving the RI false alarm ratio without sacrificing detection skill. In a 100-member ensemble configuration, FAST-ML produces intensity forecasts comparable to FNV3 for selected storms under the evaluated input configurations. Furthermore, zero-shot tests on selected Eastern Pacific storms provide encouraging evidence of cross-basin transferability. FAST-ML provides a modular intensity forecasting framework that can be coupled with externally supplied storm tracks and environmental fields. It demonstrates that observation-driven parameter learning within physically constrained dynamics simultaneously enhances accuracy, interpretability, and computational efficiency.

0.32

Tianmu-TC: Physics-constraints Generative Artificial Intelligence for Global Tropical Cyclone Forecasting

cs.LG ·2026-08-19 ·Shiqi Zhang, Pan Mu, Cheng Huang, Hanting Yan, Yuchao Zhu +4 more
similarity 0.74
Abstract

Tropical cyclones (TCs) pose severe risks from strong winds and heavy rainfall. However, forecasting their track and intensity remains challenging due to chaotic atmosphere and the rapid amplification of initial condition errors, leading to growing forecast uncertainty. While numerical weather prediction (NWP) and deep learning models have made progress, they remain computationally demanding and often fail under complex meteorological scenarios. Here, we present Tianmu-TC, a physics-constraints generative framework for global TC forecasting. Trained on Western North Pacific data, Tianmu-TC leverages physics-constraints to generate controllable outputs with reduced uncertainty thus improving forecast reliability. Experiments show Tianmu-TC outperforms deterministic and ensemble meteorological artificial intelligence models and authoritative NWP systems such as ECMWF in global ocean basins, with significantly lower computational cost. We further show Tianmu-TC performs well in challenging scenarios such as data sparsity, anomaly tracks, rapid intensification and weakening. These findings suggest physics-constraints generative AI offers a promising approach for reliable, efficient global TC forecasting.

0.25

TyphoonWorld: A Residual Token World Model for Probabilistic Forecasting of Typhoon Imagery

cs.CV cs.LG ·2026-11-11 ·Xiaoyu Chen, Tianyi Li, Xuanyi Zhang
similarity 0.73
Abstract · 15 pages, 3 figures, 7 tables

A satellite forecast of a typhoon should give a low-error picture of the next two days, together with an ensemble whose members are coherent fields and whose spread responds to the storm at hand. A small direct regression network already gives the lowest squared error on held-out storms, so a generative model has to earn its cost through the part of the future that the regression smooths away. We present TyphoonWorld, a discrete-token world model that generates only that part. Conditioned on tokens of the regression forecast, the latest frames and the best-track state at forecast time, it samples residual fields lead by lead, and members add these residuals to the regression center. We compare it under one protocol with per-pixel Gaussian networks, a diffusion model of the same residuals, residuals from other storms and an unconditional token world model, on 276 forecast windows from 46 held-out storms of Digital Typhoon. TyphoonWorld matches the squared error of the regression, lowers the CRPS of the area colder than 220 K, a direct measure of deep convection, by 14 to 39% against every other ensemble, and ties its spread to forecast error as closely as the Gaussian network, with a correlation of 0.78 across windows. It uses 56% of the parameters of the unconditional world model.

0.12

Beyond the record: evaluating probabilistic AI weather forecasts deep in the tail

cs.LG ·2026-10-18 ·Bastiaan Van Velthoven, Cas Decancq, Diego G. Miralles, Stijn Luca, Thomas Mortier
similarity 0.73
Abstract

Can probabilistic artificial intelligence (AI) weather forecasts retain skill beyond their training domain? We compare four state-of-the-art AI models (ArchesWeatherGen, U-Cast, WeatherNext2 and GenCast-mini) with the physics-based IFS-ENS over land in 2022, at 1.5° resolution and lead times of 2--10 days. Monthly ERA5 records from 1979--2021 define common historical boundaries, and we characterize new records set in 2022 by their depth, i.e., the exceedance margin relative to local historical variability. Two proper scores, record-threshold weighted CRPS and Brier score, assess the magnitude and probability of record exceedances across all eligible cases, retaining both missed extremes and false alarms. Verified against ERA5, AI models outperform IFS-ENS for heat and cold records at almost every lead and depth. For heat, the advantage strengthens with depth, partly because IFS-ENS over-forecasts deep records, and largely persists for deeper records under HRES-fc0 verification. For wind, gains erode for deeper records and longer leads. Better tail scores can nevertheless coexist with aggregate probability deficits or surpluses, and some improvements arise chiefly on non-exceedances. Increasing underprediction on progressively more extreme observed subsets can also arise through outcome selection, even for an ideal conditional-mean forecast. These findings establish that native probabilistic AI ensembles retain, and for heat increase, their advantage over IFS-ENS as thresholds move beyond local records, although all advantages narrow with lead time. They caution against generalizing the extrapolation failure reported for deterministic AI forecasts to probabilistic systems, and motivate common verification conventions that separate comparative tail skill, performance on realized extremes, and probabilistic calibration.

0.09

TC-Next: Zero-Shot Multimodal Cyclone Forecasting

cs.LG cs.AIcs.CV ·2026-09-02 ·Zhe Wang, Sijie Chen, Yiming Luo, Daehyun Kim, Chien-Yi Chang
published similarity 0.72
Abstract · 4 pages; Accepted to NeurIPS 2026 Tackling Climate Change with Machine Learning Workshop

We present TropicalCycloneNext (TC-Next), a multimodal deep learning model that forecasts tropical cyclone track and intensity at $6-24$h leads by leveraging a foundation model's forecast of atmospheric kinematic and thermodynamic fields, along with GridSat infrared satellite imagery. Trained only on GraphCast forecasts over the Western Pacific (WP), yet reliant only on generic atmospheric variables, TC-Next on GraphCast lowers track error by $15-44\%$ and intensity error by a factor of $3-6$ relative to the rule-based tracker TempestExtremes; applied without retraining to Pangu-Weather and IFS HRES forecasts, it stays ahead on both. Applied zero-shot to the generic weather fields of WeatherNext Cyclones on the 2025 WP season, it attains lower average intensity error at every lead time, and lower or comparable average track error, than that model's specialized direct tracker in a deterministic comparison. Ablations show that the satellite branch lowers track error at every lead time and intensity error at longer leads. TC-Next thus gives a common, independently verifiable basis for comparing the AI forecasts that inform cyclone-warning and climate-adaptation decisions as best available science.

0.35

Uncertainty-Aware End-to-End AI Weather Forecasting: Disentangling Observation and Model Contributions

cs.LG physics.ao-ph ·2026-08-31 ·Rodrigo Almeida, Noelia Otero, Jost Arndt, Simon Baur, Wojciech Samek +1 more
similarity 0.71
Abstract · 25 pages, 16 figures

End-to-end weather forecasting systems produce skillful global gridded and station forecasts directly from raw Earth observations, replacing the numerical weather prediction pipeline, including data assimilation, at a fraction of its cost. These systems are deterministic and issue no uncertainty. Here we render the Aardvark Weather model probabilistic by attaching one stochastic mechanism to each component: learned, input-dependent noise at the observation encoder, aimed at the uncertainty inherited from the observing system, and Monte Carlo dropout in the processor, aimed at the uncertainty of the learned dynamics. The resulting nested ensemble attributes forecast spread to the two sources through a law-of-total-variance decomposition, cross-checked by withholding observation streams. Probabilistic finetuning significantly improves the mean forecast, by 11% on average across variables and lead times. The ensemble is close to calibrated against ERA5 through the medium range, is at least as accurate as the deterministic model at weather stations while beating it in relative economic value at every lead time, and remains considerably behind the operational ECMWF IFS ensemble. The encoder branch behaves as observation-driven uncertainty. Component-attributed uncertainty makes end-to-end forecasts more transparent, a step toward observation-driven digital twins of the atmosphere.

0.22

Improving global precipitation forecasts with an AI weather model trained on satellite observations

cs.LG physics.ao-ph ·2026-09-02 ·Julian F. Schmitt, Bertrand Delorme, Robert C. King, Yashica Patodia, Tapio Schneider +2 more
code/data similarity 0.71
Abstract · 16 pages, 4 figures. Submitted

Precipitation forecasts shape decision-making across the global economy, particularly in sectors such as agriculture. However, unlike variables such as temperature, precipitation is highly intermittent and localized, making it difficult to forecast. While recent advances in AI weather prediction systems have enabled them to surpass physical models on globally averaged metrics, improvements in mean error rarely translate to actionable forecasts of severe flooding or dry crop fields. Furthermore, most of these models are trained and evaluated against a reanalysis data product, ERA5, which has well-known biases. Here we retrain AIFS, ECMWF's widely-used, open-source operational 0.25° probabilistic graph-transformer weather model, on satellite-based precipitation observations. Our model, Laxmi, improves global probabilistic accuracy by 19% and resolves systematic distributional biases in ERA5. Specifically, Laxmi reduces drizzle overprediction by 33% for amounts less than 3 mm per day. It also mitigates extreme rainfall underprediction, improving the global 95th percentile Brier skill score by 57%. Across a case study of 10 Indian tropical storms, Laxmi delivered the most accurate forecast of 150 mm event-total precipitation in 7 events, compared to 1 for AIFS and 2 for the leading physical model, IFS. Our results demonstrate that incorporating observation-based precipitation data directly into training can substantially improve forecasts.

0.35