CS Paper Radar

STeP-TS: Shared Temporal Filters with Pruning for Time Series

Hanne Raum, Jasper Hoffmann, Joschka Boedecker

cs.LG ·submitted 2026-10-26 10:02 UTC · v2
published

Abstract

Multivariate time series forecasting requires capturing temporal dependencies across multiple scales while accounting for both shared and variable-specific dynamics. Compact forecasting models have emerged as an attractive alternative to increasingly large architectures, but preserving strong predictive performance within tight computational and memory budgets remains challenging. We propose a compact one-layer convolutional framework that learns a shared bank of multi-scale temporal filters of each variable. Lightweight variable-specific scalar weights determine the contribution of each temporal component, enabling the model to capture shared temporal structure while adapting to variable-specific dynamics. The learned weights and filter-output energy further provide a natural signal for structured pruning of convolutional filters, reducing computational complexity with minimal degradation in forecasting performance. We additionally introduce an amplitude-adaptive normalization mechanism that preserves variable-dependent temporal amplitudes. Experiments on multiple datasets demonstrate forecasting performance competitive with state-of-the-art methods while requiring substantially fewer parameters and inference FLOPs and achieving low per-sample latency.

Comment
Accepted at the Workshop TS-LIMITS at NeurIPS 2026

Closest papers

Nearest neighbours by meaning across the whole index.

CTLite: Lightweight Irregular Multivariate Time Series Forecasting with Reusable Continuous-Time Summaries

cs.LG ·2026-10-19 ·Shusen Ma, Xudong Zuo, Yangchun Xie, Yun-Bo Zhao
code/data similarity 0.73
Abstract

Irregular multivariate time series forecasting (IMTSF) requires capturing temporal dynamics and cross-variable dependencies from sparse, asynchronous observations. Existing methods often rely on learned temporal encoders, coupling historical representation construction with model optimisation and repeatedly processing the same observations. We propose CTLite, a lightweight forecasting framework that decouples historical summarisation from learned prediction. Specifically, a parameter-free continuous-time moment encoder summarises local levels and trends at multiple temporal scales while suppressing poorly supported slope estimates. A compact readout combines damped local extrapolation with a low-rank cross-variable correction that accounts for observation freshness, target availability, and forecast horizon. Since the historical summaries are independent of trainable parameters and forecast queries, they can be reused across optimisation steps and repeated predictions for an unchanged history without altering predictions or parameter gradients. Experiments on four real-world benchmarks show that CTLite achieves the best or second-best mean MSE and MAE among the evaluated methods with only 88-2,537 trainable parameters. Under matched GPU workloads at batch size 32, CTLite reduces inference latency by 18.6-87.3% compared with the latest models, including the cost of historical encoding. Reusing summaries further reduces total runtime by approximately 52% over 16 forecast calls on the same history compared with recomputing the encoder for every call, including the initial encoding cost. The code is at: https://github.com/SSMa913/NICLab-CTLite.

0.19

Timer-M1: A Multivariate Time Series Foundation Model via Learning Primitives

cs.LG cs.AI ·2026-10-08 ·Haoran Zhang, Haixuan Liu, Xingjian Su, Yong Liu, Zhi Chen +3 more
similarity 0.72
Abstract

We introduce Timer-M1, a pretrained multivariate time series foundation model that learns with primitives for zero-shot forecasting. Across domains, time series share elementary temporal and relational patterns, termed primitives, yet differ in how these primitives manifest and evolve across different contexts. Despite progress in zero-shot and task-general forecasting, existing foundation models may still struggle to generalize to complex real-world scenarios. To this end, we develop a primitive-based data synthesis and pretraining pipeline. The synthesis pipeline generates series with temporal primitives shared across domains and then assembles real and generated series into multivariate samples using relational primitives. Afterwards, samples are organized into episodes by assigning distinct channel roles as target variates, past-only covariates, and known-future covariates, ensuring that the model is optimized on predictable variates using available exogenous information. Technically, Timer-M1 further adapts gated two-dimensional Transformer blocks that dynamically allocate cross-variate attention across layers. Across three large-scale forecasting benchmarks, Timer-M1 ranks first on both FEV and TIME and second on GIFT-Eval among most recent time series foundation models. These results support effective primitive-based pretraining as a route to robust general forecasting technique across domains and task settings.

0.13

Channel-Dependent State Space Model for Multivariate Time Series Forecasting

cs.LG cs.AI ·2026-09-29 ·Yu-Cheng Wu, Fan-Keng Sun, Li-Chun Lu, Duane S. Boning
benchmark similarity 0.72
Abstract

Multivariate time series forecasting (MTSF) is critical across many real-world domains. Existing deep learning approaches fall into two paradigms with distinct limitations: channel-independent (CI) methods unconditionally ignore cross-variable dependencies and model only temporal dynamics, while channel-dependent (CD) methods consider both but typically rely on architectural compromises to mitigate overfitting and computational overhead. We therefore propose Chameleon, a specialized CD state space model (SSM) that enables data-dependent, fine-grained interactions across variables while scaling linearly with their number. By connecting selective SSMs with the Kalman filter, we leverage the missing measurement update in the former for cross-variable modeling while preserving the SSM backbone for robust temporal modeling. We further identify favorable inductive biases of GatedDeltaNet for time series, adapt it as our backbone, and improve generalization through additional techniques, including a previously unexplored stochastic perturbation of reversible instance normalization. On strongly dependent ODE and PEMS datasets, Chameleon achieves the best MSE and MAE across all settings, while its CI ablation and prior CD methods incur 61-178% higher MSE on average. Across 28 standard benchmark settings, Chameleon also achieves better MSE and MAE than each baseline in at least 27 and 22 cases, respectively. Training-time and peak-memory analyses on Traffic and ETT further demonstrate competitive efficiency and favorable memory scalability across different variable counts.

0.29

MSAR: Next-Scale Autoregressive Forecasting for Time Series via Modular Multi-Scale Decoupling

cs.LG ·2026-11-16 ·Fanda Fan, Kuiye Ding, Liming Mao, Bingsong Wang, Yao Wang +7 more
top venuecode/data similarity 0.71
Abstract · Accepted by NeurIPS 2026 main track. 26 pages, 7 figures. Code available at https://github.com/Oryxo/MSAR

Time series forecasting underpins critical applications in finance, energy, healthcare, and transportation. Although deep models have achieved strong results, most adopt single-scale modeling or restrict multiscale processing to the input side, causing a misalignment between multiscale inputs and single-scale outputs and limiting predictive power. We introduce the Modular Scale-wise Autoregressive Framework (MSAR), a model-agnostic design that forecasts progressively across multiple temporal resolutions. MSAR offers three advantages: (1) scale-wise aligned modeling, which disentangles heterogeneous temporal patterns by aligning inputs and outputs at each scale; (2) scale-wise autoregression, where coarse-scale predictions guide finer-scale forecasting through hierarchical information flow; and (3) a modular architecture, enabling seamless integration with diverse backbones such as CNNs, MLPs, and Transformers. Extensive experiments across a broad set of datasets and forecasting models demonstrate that MSAR achieves consistent improvements in both accuracy and inference efficiency, validating the effectiveness of scale-aligned autoregression for multiscale time series forecasting. The code is available at: https://github.com/Oryxo/MSAR.

0.51

Recovering Lost Details: Multi-Scale Frequency Compensation for Long-Term Time Series Forecasting

cs.AI ·2026-09-21 ·Runmin Zou, Siyi Xie, Yaohui Huang, Yun Wang
top venue similarity 0.69
Abstract · 11 pages. Accepted at the 35th ACM International Conference on Information and Knowledge Management (CIKM 2026)

Long-term time series forecasting has made significant progress by leveraging multi-scale information to capture hierarchical temporal patterns and model long-range dependencies. However, temporal downsampling in existing multi-scale methods inevitably smooths detailed temporal fluctuations, and this information loss is further aggravated by their emphasis on dominant trends across scales, resulting in insufficiently expressive representations. To address this, we propose a Multi-Scale Wavelet Mixing (MWMixer) model, which incorporates a Bidirectional Frequency-Bands Mixing strategy to recover lost temporal details across scales, enabling complementary cross-scale information interactions. Then, a Dynamic Scale-Adaptive Fusion module learns time-varying weights for each scale to fuse multi-scale forecasts into the final prediction, enhancing the flexibility of multi-scale aggregation. In addition, a cross-scale consistency loss aligns each coarse-scale prediction with the interval-averaged fine-scale outputs, while a multi-scale supervision loss enforces prediction accuracy at each scale, promoting consistent learning across scales. Extensive experiments on seven real-world datasets demonstrate that MWMixer achieves competitive performance in long-term forecasting.

0.43

Time Flies When You Parallelise: Pareto-efficient Time Series Foundation Models with Linear Recurrences

cs.LG ·2026-11-12 ·Torben Berndt, Lingyi Yang, Federico Nardi, Yi Heng Lim, AJ Shears +1 more
similarity 0.68
Abstract

Time series foundation models have demonstrated strong zero-shot forecasting performance, but computational efficiency remains a central architectural challenge. Temporal self-attention incurs quadratic cost in the number of temporal tokens, while nonlinear recurrent state transitions restrict parallel computation along the temporal dimension. We introduce ci-proto, a family of time series foundation models based on Mamba-3, whose linear state recurrences support chunk-wise parallel-in-time computation during training, and inference costs that scale linearly with sequence length. The architecture alternates recurrent temporal processing with cross-variate attention, supporting multivariate probabilistic forecasting with past-only and future-known covariates. Across fev-bench's 100 forecasting tasks, ci-proto XL sets a new state of the art among time series foundation models. The five ci-proto variants also make up the entire Pareto frontier for mean scaled quantile loss and mean absolute scaled error against both parameter count and median runtime.

0.18

Aurora-X: Built for Extreme Time Series Forecasting

cs.LG ·2026-09-25 ·Xingjian Wu, Chenjuan Guo, Xiangfei Qiu, Zhigang Hu, Hanyin Cheng +4 more
similarity 0.68
Abstract

Time series foundation models (TSFMs) enable cross-domain forecasting, but their development as general-purpose forecasters remains constrained by underexplored training potential and limited architectural versatility. To address these challenges, we introduce Aurora-X, a billion-scale TSFM with a progressive curriculum and a unified architecture. We first use channel-independent pretraining to learn temporal patterns, then introduce cross-variable dependencies, varied context and horizon lengths, and future covariates if available during midtraining. Variable-resolution post-training further enables an adjustable temporal span per token at inference. With fixed model weights, this supports longer histories under a fixed token budget or fewer tokens for the same history, enabling test-time scaling. With a versatile architecture, Aurora-X supports cross-variable modeling, covariate conditioning, and parallel decoding of future patches for probabilistic forecasting. These are supported by a novel pattern-guided mixture-of-experts that expands model capacity through sparse activation and uses shallow patch similarities to constrain deep-layer routing, guiding expert specialization across heterogeneous time series. Furthermore, we propose an implicit quantile network head that predicts arbitrary quantiles to characterize predictive distributions, enhancing probabilistic forecasting flexibility. Comprehensive experiments on GIFT-Eval, TIME, FEV-Bench, TFB, and DAG-Bench demonstrate state-of-the-art forecasting performance against pretrained TSFMs and task-specific supervised models.

0.12

CVTA: Cross-Variable Temporal Attention for Multivariate Irregular Time Series Prediction

cs.LG ·2026-10-18 ·Ankitkumar Joshi, Milos Hauskrecht
top venue similarity 0.68
Abstract · Accepted at the Fortieth Annual Conference on Neural Information Processing Systems (NeurIPS 2026), Main Track

Irregularly sampled multivariate time series arise in many real-world domains such as healthcare, climate science, and large-scale monitoring systems. Characterized by asynchronous observations and heterogeneity across variables, such time series pose significant challenges for forecasting. In this work, we introduce Cross-Variable Temporal Attention (CVTA), a short-term, continuous-time forecasting framework that learns prediction functions from its compact, interpretable state summarizing recent temporal dynamics across variables. CVTA uses Variable-Conditioned Embeddings to project heterogeneous variables into variable-specific latent spaces, and applies temporal attention to capture cross-variable dependencies from this state. Furthermore, we demonstrate that CVTA can be integrated as an auxiliary module into existing state-of-the-art prediction models through a dynamic meta-decision model. Extensive experiments on multiple real-world benchmarks show that CVTA achieves strong short-term predictive accuracy, supports efficient online inference, improves existing prediction models when used as an auxiliary module, and learns attention patterns that align well with domain knowledge.

0.22