CS Paper Radar

Seasonality-Aware Hybrid Convolutional Transformer for Antarctic Sea Ice Concentration Forecasting

Danyang Li, John Taylor, Thang Bui, Quanling Deng

cs.LG ·submitted 2026-08-31 11:54 UTC · v2

Abstract

Antarctic sea ice concentration (SIC) forecasting is an important yet challenging task due to the coexistence of complex spatial structure, long-range temporal dependencies, and strong seasonal variability. Conventional convolution-based models are effective at capturing local spatial patterns, but often have limited ability to model long-term temporal evolution. To address these challenges, we build on a hybrid convolutional transformer forecasting framework for monthly Antarctic SIC forecasting. This framework combines convolutional encoding for spatial feature extraction with space-time factorised self-attention for SIC modelling. We further introduce two season-based mechanisms: a month-aware positional encoding that injects calendar-month information into the token representation, and a seasonal temporal bias that encourages attention to periodically related historical states. Experimental results show that the proposed framework achieves better performance than convolutional and recurrent baselines as well as ECMWF's physics-based dynamical model SEAS5 under both classification and regression metrics. Ablation studies further indicate that the seasonality-aware components provide consistent additional gains in both short- and long-horizon prediction. These results demonstrate the value of combining convolutional structures, attention mechanisms, and periodic prior information for Antarctic SIC forecasting.

Closest papers

Nearest neighbours by meaning across the whole index.

SETTer: Sparse-Encoder Transformer for Long-term Multivariate Time Series Forecasting

cs.LG ·2026-09-17 ·Abraham Ezema, Chijioke Eze, Ferdinanda Ponci, Antonello Monti
benchmarkhot direction similarity 0.69
Abstract

Long-term multivariate time series plays a significant role in many application areas such as power systems, trading, etc. However, their accurate prediction is quite difficult for conventional forecasting methods as they often exhibit high dimensionality and complex relationships. Recent works show that transformer-based approaches are quite effective for long-term forecasting thanks to their attention mechanism. However, in the presence of complex high-dimensional inputs, they show evidence of oversmoothing, limited capacity, and opacity. To this end, this paper introduces SETTer, a transformer-based model that addresses these challenges by incorporating novel techniques for decoupled self-attention and hybrid masking. The proposed techniques enable SETTer to effectively capture the dominant short- and long-term patterns across the temporal and channel dimensions. In addition, we enrich the model layers with simple explainable structures that indicate the discriminative pattern of SETTer. We show that with a single-layer transformer architecture, SETTer can effectively model long-term dependencies in the presence of varying data complexities. Extensive experiments on real-word benchmark datasets for long-term multivariate time series forecasting demonstrate that SETTer outperforms state-of-the-art models in 88% of the scenarios.

0.26

Skillful Data-Driven Subseasonal Soil Moisture Forecasting: Prospects and Limits for Flash Drought Prediction

cs.LG physics.ao-ph ·2026-10-05 ·Noelia Otero, Atahan Özer, Miguel-Ángel Fernández-Torres, Jackie Ma
published similarity 0.67
Abstract · 27 pages, 8 figures, 5 tables. Accepted for publication in npj Hydrosphere. Supplementary information available with the published version

Despite substantial progress in short-to-medium-range weather forecasting, predicting high-impact events such as flash droughts remains a key challenge for both early warning operations and physically-based subseasonal-to-seasonal (S2S) prediction systems. Here we demonstrate that, for S2S soil-moisture forecasting over Europe, forecast skill depends as much on how the prediction problem is formulated as on the forecasting model itself. Using a Vision Transformer-based architecture with dual-pathway temporal and spatial attention, we show that residual learning is essential to outperform persistence. This advantage is realized only when forecasting root-zone soil moisture in physical units rather than standardized anomalies, revealing that the target representation itself constrains predictability. A probabilistic extension via quantile-head fine-tuning further provides well-calibrated predictive distributions. Benchmarked against deep-learning and operational ECMWF S2S baselines over 2021-2022, our model achieves the highest deterministic and probabilistic skill at all lead times and reliably detects anomalously dry root-zone states (below the 20th percentile). Yet flash drought onset, defined by multi-pentad intensification criteria, remains a fundamental challenge shared across all current S2S systems. These findings advance data-driven S2S soil-moisture forecasting while highlighting the remaining challenge of predicting rapid drought development.

0.19

Neptune: An AI model for Global Ocean Subseasonal Prediction

cs.AI physics.ao-ph ·2026-09-08 ·Davide Donno, Italo Epicoco, Massimo Cafaro, Gabriele Accarino, Mohammad M. Amirian +6 more
similarity 0.67
Abstract

Subseasonal-to-seasonal (S2S) forecasting is societally critical, supporting decision-making in sectors ranging from water and agricultural management to disaster risk reduction, energy planning, and insurance. Achieving reliable predictions at these timescales requires representing the ocean and its dynamics, but traditional physics-based Ocean General Circulation Models (OGCMs), are computationally expensive and difficult to develop and improve because of the code complexity. In this work, we propose Neptune, an end-to-end data-driven framework for global ocean and sea-ice components emulation tailored for S2S timescales, up to 60 days. Neptune combines Convolutional Neural Networks (CNNs) and Spherical Fourier Neural Operators (SFNOs) to effectively capture local features and global cross-scale interactions, thereby obtaining a coherent representation of the ocean state. Forced by prescribed daily atmospheric fields, Neptune emulates ocean state variables, from temperature and salinity, to zonal and meridional currents, from sea surface height to sea ice thickness and concentration, with daily outputs at the ocean surface and through the water column. Specifically, we propose two variants of Neptune, Neptune-1 and Neptune-025, capable of emulating the ocean state at 1° and 0.25° resolution, respectively. Evaluated against a suite of metrics, including statistics (RMSE, CRPS and ACC), physical coherency (Ocean Heat Content, Eddy Kinetic Energy and Ice Brier Score) and climate indices (ENSO and Z20 metric, IOD), Neptune successfully reproduces the spatio-temporal evolution of the oceanic fields up to 60 days, and is stable over long timescales. Neptune provides compelling evidence that end-to-end data-driven ocean emulators can become a powerful component of next-generation S2S forecasting systems, emulating ocean state at high spatio-temporal resolution.

0.17

Climate Temporal Temperature Prediction via an Adaptive Stacked Bidirectional Residual Mamba Network

cs.LG ·2026-11-12 ·Yongxiang Lei
similarity 0.66
Abstract

Accurate temperatures forecasting relies on traditional meteorological parameters that are essential for monitoring weather informatics and guiding forecasting efforts. This study presents Deep DualMamba, a novel deep learning framework for temporal temperature prediction, built upon an enhanced variant of the recently proposed Mamba network. Unlike conventional Transformer-based architectures, which dominate foundation models but suffer from computational inefficiencies on long sequences, Deep DualMamba leverages selective structured state space models (SSMs) to achieve both scalability and performance. We identify a core limitation of prior subquadratic-time models-namely, their inability to perform long-term and robust forecasting and introduce two key innovations. First, we propose an adaptive bidirectional Mamba as encoders to learn complex and efficient features. Second, we develop a hardware-aware, parallel recurrent algorithm to compensate for the loss of convolutional efficiency, ensuring practical viability at scale. Last, the Deep DualMamba's approximation ability and convergence is theoretical analyzed. These design choices culminate in a simplified, attention- and MLP-free neural architecture that exhibits linear complexity with sequence length and significantly improved inference throughput. To demonstrate its applicability in climate informatics, we implement a full forecasting pipeline using historical meteorological data from UK monitoring stations.

0.15

TSAE: Structured Sparse Autoencoders for Interpreting Time-Series Forecasting Models

cs.LG cs.AI ·2026-10-04 ·Baoxi Liu, Yi Xie
similarity 0.65
Abstract

Time-series forecasting informs critical decisions in energy dispatch, industrial operations, and environmental monitoring; understanding the patterns models rely on is essential for assessing reliability and identifying failures. Input attribution identifies important variables and time segments but offers limited insight into internal features, while standard sparse autoencoder (SAE) objectives do not directly constrain cross-variable structure or temporal continuity. We introduce TSAE, a structured sparse autoencoder for forecasting representations that decomposes hidden states into individually inspectable features. TSAE organizes cross-variable structure through shared and variable-routed private dictionaries, separates feature detection from magnitude estimation with gated encoding, and constrains neighboring sparse-code changes according to raw-segment similarity. These mechanisms support analysis of variable context, activation strength, and temporal evolution. Forecast-consistency fine-tuning further improves preservation of the frozen forecaster's outputs. The accompanying TSEVAL protocol separately audits fidelity, feature coherence, and physical calibration to ground feature interpretation. In three-seed experiments with frozen PatchTST on ETTh1, ETTh2, and ETTm1, TSAE achieves the lowest hidden-state reconstruction error, normalized forecast-reconstruction error (NFRE), and feature transition rate among five SAEs at comparable per-token activity. NFRE decreases by 4.3-26.4% relative to the next-best mean. Dataset-dependent tradeoffs in selectivity, physical correlation, and calibration show that fidelity and temporal-stability gains require independent semantic validation to support feature interpretation.

0.26

TempTPI: Informer-Based trajectory prediction for maritime vessels

cs.LG ·2026-09-09 ·Kevin Ferneding, Veronika Lietavcova, Aleksandra M. Blachowiak, Peder Heiselberg
published similarity 0.65
Abstract · Accepted to IEEE International Geoscience and Remote Sensing Symposium (IGARSS) 2026. \c{opyright} 2026 IEEE. Personal use of this material …

Accurate long-term trajectory prediction for maritime vessels is essential for safety and logistical efficiency. While deep learning models, particularly Transformers, have shown promise in processing Automatic Identification System (AIS) data, they often struggle with the quadratic computational complexity of self-attention and the loss of accuracy over extended forecasting horizons. This study proposes TempTPI, a novel prediction framework that integrates an Informer-based encoder with a multi-channel temporal encoding mechanism. The Informer architecture leverages a ProbSparse self-attention mechanism to reduce computational overhead and focus on the most significant dependencies, while the temporal encoder utilizes Fourier-like frequency expansions to capture cyclic patterns (hourly, daily, and seasonal) in vessel behavior. We evaluate our model against the state-of-the-art TPTrans architecture using AIS data from Danish waters. Experimental results demonstrate that TempTPI consistently outperforms existing methods across prediction windows of 1 to 5 hours. Notably, at a 5-hour horizon, the proposed model achieves a 55% improvement in Mean Squared Error (MSE), offering a robust solution for long-range maritime situational awareness.

0.28

Structured Neural Modeling of Daily Arctic Sea-Ice Concentration Evolution: Physical-Trajectory-Driven Learning and Forecast-Domain Adaptation

cs.LG physics.ao-ph ·2026-08-22 ·Maqun Zhang, Feng Gao, Wankun Chen, Hui Yu, Yanhai Gan +1 more
code/data similarity 0.64
Abstract

Accurate modeling of the daily evolution of sea ice concentration (SIC) is central to improving the credibility and operational forecasting capability of deep learning-based sea ice prediction. However, existing deep learning methods often couple the underlying sea ice evolution relationships and data errors within high-dimensional nonlinear mappings, making it difficult to construct a stable and verifiable evolution core and apply it reliably to practical forecasting. To address this issue, this study proposes a reanalysis-forecast dual-domain decoupled framework for learning sea ice evolution operators. The framework builds upon a lightweight physical baseline to generate daily evolution trajectories, employs a temporally constrained joint multi-lead compensation network to com?pensate for unresolved processes, and introduces an ice-mass?aware transport mechanism to suppress numerical dissipation. In the forecasting stage, the parameters of the base evolution core are fixed, while a lightweight variable-semantic adaptation mechanism calibrates inter-domain distributions and evolution responses, thereby separating forecast-domain errors from base evolution errors. Experiments show that the constructed base evolution core can accurately and stably simulate daily sea ice evolution at both short-term and annual scales under reanal?ysis forcing, and can be effectively transferred to the forecast domain through lightweight adaptation, achieving stable prac?tical forecasting capability while preserving the base evolution structure. The source code will be made publicly available at https://github.com/zhangmaqun65535/SNM upon acceptance of this manuscript.

0.36

From Next Day Wildfire to Next Days Wildfire: A New Multi-Continent Dataset of Active-Fire Spread for Sequence-to-Sequence Forecasting

cs.CV ·2026-10-28 ·Jose Guillen, Renzo Zanca, Valentin Barriere
publisheddataset similarity 0.64
Abstract · Accepted at WACV 2027. 19 pages including supplementary material, 12 figures, 13 tables

Benchmarks such as WildfireSpreadTS (WSTS) have enabled data-driven forecasting of wildfire spread from multimodal satellite time series, but they suffer from limitations: size restricts them to large fires that burn for weeks, and they are from similar environments. We introduce AFSeq, a multimodal dataset of 1,194 recent fire events (2023 to 2026) from 8 different countries across three regions, dominated by the short, small fires that existing benchmarks underrepresent. Our dataset is the largest and most diverse to date for time-series-based wildfire spread prediction. We also introduce UTAE-AR, an extension of a classical U-Net with Temporal Auto Encoder, enhanced with recurrent modules before the decoder. This allows the model to not only predict the next day but the whole sequence in an autoregressive way. Moreover, our model can feed its own predicted fire mask back one day at a time, producing the day-by-day forecasts that single-step models cannot. Our results show that UTAE-AR surpasses state-of-the-art on both WSTS, WSTS+ and AFSeq for next-day accuracy while having 21.5% of their parameters, and being able to predict confidently at a longer horizon.

0.28