CS Paper Radar

Sensitivity Fidelity Reveals Physical Dependencies Beyond Forecast Accuracy in Deep-Learning Earth-System Models

Qi Chen, Peng Zhan, Hongxing Cui, Wenqing Cui, Ibrahim Hoteit

cs.LG physics.ao-phphysics.geo-ph ·submitted 2026-10-11 11:29 UTC · v1
benchmark

Abstract

Deep-learning models increasingly rival numerical simulation in Earth-system forecasting, yet forecast error alone cannot determine whether their learned input-output dependencies are physically consistent. Here we introduce sensitivity fidelity as an additional criterion: alongside prediction error, model responses to input perturbations should be examined against a trusted reference or established physical knowledge. In an idealized Rayleigh-Bénard convection benchmark, four surrogate architectures achieved low forecast errors yet encoded markedly different sensitivities. Quantitative comparison with a matched adjoint distinguished the models far more sharply than forecast accuracy and showed that predictive skill and sensitivity fidelity need not coincide, while finite-perturbation experiments further linked these differences to the propagation of initial-condition uncertainty. We then applied this strategy to El Niño-Southern Oscillation prediction, where no adjoint reference exists and learned sensitivities showed event-wise robustness, validity under finite perturbations, and consistency with established precursors. These results identify gradient-based evaluation as a discriminating complement to forecast-error metrics, providing a practical test of whether data-driven Earth-system models learn physically credible dependencies.

Closest papers

Nearest neighbours by meaning across the whole index.

Evaluating Dynamical Fidelity through Predictive Structure in Physical Representations

cs.LG ·2026-09-28 ·Oskar Bohn Lassen, Joao Paulo de Souza Boger, Simon Driscoll, Stephen I. Thomson, Sebastian Schemm +2 more
similarity 0.72
Abstract

Machine-learning models for physical systems are currently evaluated primarily through errors between predicted and reference states and, increasingly, through tests of physical consistency. These metrics assess whether predictions are accurate and satisfy selected physical requirements, but provide limited insight into whether learned trajectories reproduce the underlying dynamics. Domain experts examine such relationships through physical representations that expose relevant processes, interactions, and responses, but these analyses are often separated from typical machine-learning evaluation. We introduce a practical framework for evaluating dynamical fidelity through predictive structure in physical representation spaces. Experts define the representations, while reference trajectories determine which relationships are predictive and retained as evaluation tests. We demonstrate the approach in atmospheric forecasting using ERA5 representations of planetary-wave activity and Northern Annular Mode evolution, and evaluate Pangu-Weather, GraphCast, and FengWu. The models exhibit distinct departures from reference predictive structure that are not reflected by conventional forecast errors. The framework thereby turns domain-expert representations into systematic tests of learned physical dynamics without prescribing the relationships in advance.

0.26

Evaluating Predictor-Corrector Dependence in Hard-Constrained Climate Emulators

cs.CY cs.CE ·2026-10-16 ·Tianyu Zhang, Xinyu Wang, Zhenghan Tai, Xiao-Wen Chang, Yoshua Bengio
similarity 0.69
Abstract

Climate emulators often enforce physical constraints by correcting each state predicted by a neural network, but evaluating only the corrected forecast conflates predictor skill with the effect of the correction. We introduce an evaluation framework that separates these contributions by comparing corrected forecasts, pre-correction predictions along corrected trajectories, and autonomous rollouts with correction disabled. We further cross correction during training and inference to distinguish its immediate effect from how it shapes the learned predictor, and use matched nonphysical controls to test whether improvements are specific to the physical correction. In ACE2, correction substantially improves physical consistency. In a three-seed ACE2 pilot, models trained with correction produce less accurate uncorrected precipitation forecasts at short horizons than models trained without correction, but more accurate forecasts at 10 and 30 days. This contradicts our preregistered hypothesis that the deficit would grow with forecast horizon. In a separate nine-seed experiment, directly supervising the uncorrected output reverses its precipitation forecast deficit at 72 hours while maintaining the accuracy of corrected forecasts at 24 hours. In ConCerNet's heat-equation system, projection-trained models have higher uncorrected trajectory-based vector-field RMSE than models trained without projection, while restoring the physical projection recovers accuracy far more effectively than a matched nonphysical control. Together, these results show that hard corrections can reshape the predictors trained alongside them, creating dependencies that are not visible from corrected forecasts alone. Hard-constrained emulators should therefore be evaluated by separately measuring predictor accuracy, correction effects, and their interaction during training.

0.18

Strategic Governance of AI Models in Earth Science

cs.AI physics.soc-phcs.LGphysics.ao-ph ·2026-10-01 ·Makoto Kelp, Amirhossein Arzani, Patricia Castellanos, Paul Griffiths, Ivan Higuera-Mendieta +4 more
benchmark similarity 0.68
Abstract

AI foundation models pretrained on weather and climate data are increasingly fine-tuned to Earth science tasks well beyond weather forecasting. Their development and adoption are outpacing the scientific community's ability to evaluate them. These models are judged almost entirely by benchmark skill metrics, which measure how closely a forecast reproduces a reference product but not whether a model represents the physical processes governing the system it predicts. Forecast skill and physical reliability are therefore distinct properties. The distinction is most consequential under the nonstationary conditions of a changing climate for which these models were never trained. We identify five priorities for the physical evaluation of AI models in Earth science from task-specific emulators to foundation models, spanning training data, fine-tuning, behavioral testing, mechanistic interpretability, and output validation. We recommend three activities for the coming decade: 1) open AI-ready evaluation datasets, 2) a shared reporting standard for physics-based evaluation, and 3) a dedicated research program on the safety of these models.

0.38

Structured Neural Modeling of Daily Arctic Sea-Ice Concentration Evolution: Physical-Trajectory-Driven Learning and Forecast-Domain Adaptation

cs.LG physics.ao-ph ·2026-08-22 ·Maqun Zhang, Feng Gao, Wankun Chen, Hui Yu, Yanhai Gan +1 more
code/data similarity 0.66
Abstract

Accurate modeling of the daily evolution of sea ice concentration (SIC) is central to improving the credibility and operational forecasting capability of deep learning-based sea ice prediction. However, existing deep learning methods often couple the underlying sea ice evolution relationships and data errors within high-dimensional nonlinear mappings, making it difficult to construct a stable and verifiable evolution core and apply it reliably to practical forecasting. To address this issue, this study proposes a reanalysis-forecast dual-domain decoupled framework for learning sea ice evolution operators. The framework builds upon a lightweight physical baseline to generate daily evolution trajectories, employs a temporally constrained joint multi-lead compensation network to com?pensate for unresolved processes, and introduces an ice-mass?aware transport mechanism to suppress numerical dissipation. In the forecasting stage, the parameters of the base evolution core are fixed, while a lightweight variable-semantic adaptation mechanism calibrates inter-domain distributions and evolution responses, thereby separating forecast-domain errors from base evolution errors. Experiments show that the constructed base evolution core can accurately and stably simulate daily sea ice evolution at both short-term and annual scales under reanal?ysis forcing, and can be effectively transferred to the forecast domain through lightweight adaptation, achieving stable prac?tical forecasting capability while preserving the base evolution structure. The source code will be made publicly available at https://github.com/zhangmaqun65535/SNM upon acceptance of this manuscript.

0.36

Artificial intelligence pathways from weather to climate

cs.AI physics.ao-phcs.LG ·2026-10-07 ·Tom Beucler, J. David Neelin, Hui Su, Shivanshi Asthana, Chris Bretherton +8 more
similarity 0.65
Abstract · 33 pages, 10 figures. Submitted to "Science Advances"

Deep learning has made rapid advances in weather forecasting: autoregressive models trained on atmospheric reanalyses now rival dynamical models across nowcasting, medium-range, and subseasonal-to-seasonal lead times, producing well-calibrated ensemble forecasts at reduced cost. We review these advances and consider their extension to climate horizons, where the challenge shifts from initial-condition skill to producing reliable statistical responses under altered forcings. AI-powered climate prediction systems must produce credible forced responses to drivers (e.g., greenhouse gases, land-use change) typically outside the observed record. We propose two minimum requirements for AI in climate modeling: (i) external forcing agents must enter explicitly enough to support interventions in which they vary independently; and (ii) robustness must be stress-tested in out-of-distribution regimes, including extremes and counterfactual trajectories. Using leading AI autoregressive emulators and hybrid physics-AI models, we identify development and coupling challenges. Comparing the reported throughput of these models with that of GPU-ported dynamical models highlights how AI can reduce time-to-solution by advancing only the target variables at the required resolution and using longer time steps, rather than integrating a full high-frequency, multivariate state. Diverse AI downscaling strategies can partially substitute for explicit fine-scale resolution, paving the way toward inexpensive local hazard assessment across prediction horizons.

0.10

Learning Prognostic Variables for AI Convective Parameterizations via Symbolic Distillation

cs.LG physics.ao-ph ·2026-09-21 ·Jurij Schönfeld, Tom Beucler, Julien Savre, Steven Sherwood, Veronika Eyring
similarity 0.65
Abstract

Hybrid AI-physics climate modeling aims to improve coarse (~100km-resolution) Earth system models by learning to parameterize subgrid processes from high-fidelity data. However, this so far mostly involves local-in-time, diagnostic parameterizations, in which the subgrid state depends only on the current coarse state with no memory of previous states, which is unrealistic for processes such as convection that have intrinsic persistence. To address this, we enhance local-in-time parameterizations by learning prognostic variables that compactly carry important, additional past information where no explicit sub-grid information is available. First we compress past information into a low-dimensional latent space using an autoencoder, which then informs a neural network trained to parameterize targeted subgrid-scale processes. We then replace the autoencoder with symbolic equations that govern the time evolution of the latent variables, yielding additional prognostic memory variables that can be integrated alongside the resolved atmospheric state. We evaluate this approach on two systems: the Lorenz-96 model (online) and surface precipitation from high-resolution atmospheric simulations (offline). A forced multivariate linear ordinary differential equation recovers most of the added value achieved by the autoencoder-based approach in both experiments. Benchmarked against diagnostic parameterizations without memory, our memory-informed approach improves climate statistics and temporal structure, including a realistic diurnal cycle of tropical land precipitation.

0.21

Tianmu-TC: Physics-constraints Generative Artificial Intelligence for Global Tropical Cyclone Forecasting

cs.LG ·2026-08-19 ·Shiqi Zhang, Pan Mu, Cheng Huang, Hanting Yan, Yuchao Zhu +4 more
similarity 0.65
Abstract

Tropical cyclones (TCs) pose severe risks from strong winds and heavy rainfall. However, forecasting their track and intensity remains challenging due to chaotic atmosphere and the rapid amplification of initial condition errors, leading to growing forecast uncertainty. While numerical weather prediction (NWP) and deep learning models have made progress, they remain computationally demanding and often fail under complex meteorological scenarios. Here, we present Tianmu-TC, a physics-constraints generative framework for global TC forecasting. Trained on Western North Pacific data, Tianmu-TC leverages physics-constraints to generate controllable outputs with reduced uncertainty thus improving forecast reliability. Experiments show Tianmu-TC outperforms deterministic and ensemble meteorological artificial intelligence models and authoritative NWP systems such as ECMWF in global ocean basins, with significantly lower computational cost. We further show Tianmu-TC performs well in challenging scenarios such as data sparsity, anomaly tracks, rapid intensification and weakening. These findings suggest physics-constraints generative AI offers a promising approach for reliable, efficient global TC forecasting.

0.25

WeatherBridge: Forecast-Aware Multi-Source Input Adaptation for Pretrained Weather Forecasting Models

cs.AI ·2026-10-10 ·Mutian Xi, Xinyu Su, Xiaohui Zhong, Hao Li
similarity 0.64
Abstract

Data-driven weather forecasting models are commonly trained on ERA5, yet operational deployment must use timely initial states from physics-based numerical weather prediction (NWP) systems whose distributions differ substantially from ERA5. Direct initialization with these states can degrade autoregressive forecasts, motivating input adaptation that keeps the pretrained forecasting model unchanged. Here, we introduce WeatherBridge, a unified, source-conditioned adapter that maps heterogeneous operational NWP initial states to ERA5 reference states without updating the pretrained forecasting model. WeatherBridge predicts the structured residual corrections with a Swin-UNet and injects source embeddings across multiple spatial scales, allowing the model to capture shared atmospheric structure and source-specific discrepancies. Its training objective combines latitude-weighted ERA5 reconstruction with One-step Rollout Alignment (ORA), which backpropagates one-step forecast error through a frozen FuXi model to optimize the adapted states for both initial state fidelity and downstream forecast compatibility. We evaluate WeatherBridge using daily GFS, HRES, and CMA initializations from 1 January to 22 November 2025 across six variables and lead times up to 240 hours. Relative to raw source initialization of FuXi, WeatherBridge reduces the mean RMSE of FuXi over the 6- 240-hour rollout for 17 of 18 source-variable combinations, with a median reduction of 10.2% and a maximum reduction of 46.3%. The adapter trained with frozen FuXi also improves FengWu forecasts on GFS inputs without retraining either the adapter or FengWu. These results demonstrate that forecast-aware input adaptation can improve forecasts from heterogeneous operational NWP inputs and transfer across pretrained ERA5-based forecasting models.

0.18