CS Paper Radar

Dynamic Windowing in Transformers via Regime Incorporation for Financial Time Series

Praveen, Prince Chouhan, Keshav Maheshwari, Aman Verma

cs.CE ·submitted 2026-08-11 04:57 UTC · v1

Abstract

Financial time series exhibit non-stationary behavior, where the strength and extent of temporal dependencies vary across market regimes. Trending, low-volatility phases typically require long-range contextual information, whereas mean-reverting, high-volatility periods rely more heavily on short-term dynamics. Standard Transformer architectures, with fixed attention windows and static positional encodings, are therefore unable to adapt to such variations. In this work, we propose a regime-aware dynamic windowing framework that incorporates market regime information directly into the Transformer. We construct four generic regime signals from price series: volatility ratio, trend strength, local predictability ratio (LPR), and rolling autocorrelation. We incorporate these signals into the model through two mechanisms: (i) regime-augmented inputs to a standard Transformer architecture, and (ii) a modified attention layer that modulates attention weights using regime embeddings. Experiments on five S&P 500 stocks show consistent improvements across five evaluation metrics, demonstrating that regime-aware dynamic windowing enhances both interpretability and predictive performance in financial forecasting tasks.

Closest papers

Nearest neighbours by meaning across the whole index.

HAN-Mamba: Hierarchical Selective State Space Networks for Multi-Scale Financial Volatility Forecasting

cs.LG cs.CE ·2026-10-07 ·Mihai Bogdan Deaconu, Ioan Daniel Pop
top venuebenchmark similarity 0.74
Abstract · 16 pages. Accepted for publication in Springer Lecture Notes in Artificial Intelligence (ICAART 2026 Revised Selected Papers). Extended vers…

Short-horizon realized volatility forecasting requires the integration of market information that evolves at incompatible temporal resolutions, from second-level order book dynamics to weekly regime drift. Our conference work introduced HAN-T, a hierarchical architecture in which scale-specific Transformer encoders process short, mid, and long-horizon streams and a learned attention fuser weighs their contributions. This article replaces the quadratic attention encoders with selective state space (Mamba) encoders while retaining attention only in the fuser, where the input is a three-token set rather than a long sequence. The resulting hybrid, HAN-Mamba, summarizes each stream through a recurrent state whose input-dependent gating matches two structural properties of volatility: persistent but decaying memory and abrupt regime shifts. On the Optiver Realized Volatility Prediction benchmark under time-aware five-fold cross-validation, HAN-Mamba improves mean RMSPE over HAN-T (0.1942 vs. 0.1965) with 33% fewer parameters. Its linear-time encoders further allow the high-frequency context to be extended from 60 to 240 buckets, reducing error to 0.1927 where the attention variant saturates, and support constant-time streaming updates at inference. Ablations attribute the gains to the encoder swap, confirm that the hierarchical prior transfers across sequence-model families, and show that the permutation-invariant attention fuser remains the correct mechanism for cross-scale integration.

0.33

Forecasting Global Equity Volatility: Econometric Anchoring and Cross-Market Attention

cs.CE ·2026-08-26 ·Xinlin Zhao, Haotian Qiao
similarity 0.68
Abstract

International volatility forecasting requires models to combine persistent local dynamics with information observed across different trading calendars. We propose PGA-Trans-HAR, an attention model that learns a bounded correction to a frozen heterogeneous autoregressive (HAR) forecast. A rolling predictive-connectedness graph guides spatial attention, while source masks accommodate market closures on the union calendar. We evaluate daily range-based volatility for eight equity indices from March 2013 to September 2026 at leads of 1, 5, and 22 union-calendar dates. Among seven primary forecasting models, PGA-Trans-HAR achieves the lowest equal-market average MSE at the 1- and 22-date leads and the lowest average QLIKE at all three leads. Its MSE reductions relative to HAR are 1.70% and 1.13% at the two endpoints. At the 1-date lead, the standard deviation of MSE across five fitted seeds is 0.00057, compared with 0.01260 for DCRNN-HAR and 0.03727 for GNN-HAR. The model is retained in 47 of 48 market-horizon MSE cells in the 90% Model Confidence Sets across two evaluation panels. Under historical-tail calibration, joint VaR/ES scores improve on HAR at the 1- and 5-date leads. A common intraday exposure rule yields annual CER gains of 7.89 to 9.28 basis points relative to HAR across the specified cost scenarios. These results support bounded correction of a local econometric forecast as a design for accurate and stable cross-market volatility prediction.

0.06

KiT: A Foundation Model for Financial Time-Series Forecasting using DiffusionTransformers

cs.LG cs.CV ·2026-09-28 ·Boyu Zhang, Haorui Li
code/data similarity 0.68
Abstract

Financial candlestick forecasting is fundamental to quantitative investment, yet it remains exceptionally challenging due to extremely low signal-to-noise ratios and vast heterogeneity across markets and instruments. Existing approaches have largely attempted to introduce deep learning to capture hidden temporal features, but most adopt an auto-regressive formulation, which leads to error accumulation during inference. Meanwhile, general-purpose time-series foundation models are not tailored to the unique structure of k-line data and yield unsatisfactory performance on downstream candlestick forecasting tasks. To tackle these problems, we introduce KiT, a K-line Diffusion Transformer foundation model, and reformulate future prediction as conditional path generation via flow matching: given a historical context window, the model generates an ensemble of plausible future OHLCV trajectories. We pre-train KiT at multiple parameter scales on billions of candlestick bars spanning multiple markets and timescales. Across three markets and seven resolutions, KiT attains a mean return RankIC of 0.057 and a mean volatility RankIC of 0.66, leading at every timescale and outperforming both task-specific financial forecasters and general time-series foundation models. Code will be available at: https://github.com/Luciferbobo/KiT.

0.37

TailDART: Compute-Efficient Foundation Models for Financial Time-Series Forecasting

cs.AI cs.CE ·2026-11-06 ·Chiang Qin Zhi, Ranjan Satapathy, Erik Cambria, Joyjit Chattoraj
similarity 0.66
Abstract

Although time-series foundation models (TSFMs) achieve strong zero-shot transferability across high-information-density physical systems, applying overparameterized TSFMs to quantitative finance is fundamentally constrained by low signal-to-noise ratios, non-stationarity, and prohibitive autoregressive decoding latency. To reconcile representation learning with institutional operational constraints, this study presents TailDART, a dense 2.68 million-parameter foundation model engineered for financial time-series forecasting. By integrating grouped-query attention into a lightweight backbone, the architecture drastically reduces Key-Value cache memory bandwidth overhead, achieving major inference acceleration (up to 48.1%) over prevailing state-of-the-art TSFMs to enable low-latency, high-throughput streaming execution. Furthermore, multi-asset benchmarks across varying temporal resolutions show that a compact model can remain competitive for market forecasting: TailDART matches the predictive fidelity of substantially larger baselines while demonstrating superior error resilience in high-volatility regimes.

0.23

Hierarchical Multi-Task Learning with Liquidity-Aware Signals for Stock Forecasting

cs.CE q-fin.CP ·2026-09-22 ·Hengyi Yang, Sida Lin, Yiyan Qi, Yankai Chen, Haohan Zhang +2 more
similarity 0.66
Abstract

Stock price forecasting is a long-standing challenge in computational finance, driven by the inherent randomness of markets and complex temporal patterns. While recent deep-learning models have raised forecasting accuracy by jointly modeling inter-stock and temporal price dynamics, they conflate inter-stock relationships with intra-stock temporal dependencies and focus solely on the univariate objective of price movement. To address these limitations, we propose LiMT, a Hierarchical Multi-Task Learning framework that integrates liquidity-aware signals for stock price forecasting. LiMT employs a Market Regime Encoder (MRE) module that first extracts contemporaneous cross-stock dependencies, then models each stock's temporal dynamics, yielding a unified latent state. Building on this latent state, we introduce a Liquidity-Driven Learning (LDL) module, a mixture-of-experts architecture that features cross-task gating mechanisms to jointly predict price movement, volatility, and trading volume. We further design an Adaptive Portfolio Optimization (APO) mechanism that converts multi-task forecasts into executable portfolio weights under transaction-cost and liquidity constraints. Extensive experiments on the CSI300 and CSI500 benchmarks show that LiMT performs best among strong neural and tree-based baselines across the reported metrics. In realistic CSI300 backtests, APO improves annualized return from 3.99% to 10.01% and Sharpe ratio from 1.22 to 1.86 over equal weighting, showing that the multi-task forecasts translate into deployable portfolio gains.

0.21

SETTer: Sparse-Encoder Transformer for Long-term Multivariate Time Series Forecasting

cs.LG ·2026-09-17 ·Abraham Ezema, Chijioke Eze, Ferdinanda Ponci, Antonello Monti
benchmarkhot direction similarity 0.65
Abstract

Long-term multivariate time series plays a significant role in many application areas such as power systems, trading, etc. However, their accurate prediction is quite difficult for conventional forecasting methods as they often exhibit high dimensionality and complex relationships. Recent works show that transformer-based approaches are quite effective for long-term forecasting thanks to their attention mechanism. However, in the presence of complex high-dimensional inputs, they show evidence of oversmoothing, limited capacity, and opacity. To this end, this paper introduces SETTer, a transformer-based model that addresses these challenges by incorporating novel techniques for decoupled self-attention and hybrid masking. The proposed techniques enable SETTer to effectively capture the dominant short- and long-term patterns across the temporal and channel dimensions. In addition, we enrich the model layers with simple explainable structures that indicate the discriminative pattern of SETTer. We show that with a single-layer transformer architecture, SETTer can effectively model long-term dependencies in the presence of varying data complexities. Extensive experiments on real-word benchmark datasets for long-term multivariate time series forecasting demonstrate that SETTer outperforms state-of-the-art models in 88% of the scenarios.

0.26

Beyond Static Positional Encoding: Bayesian Time-aware Attention for Non-Stationary Long-term Time Series Forecasting

cs.LG cs.AI ·2026-10-14 ·Xinjie Lan, Yunbo Mei, Leheng Zhou, Hao Qi
theory similarity 0.65
Abstract · 44 pages, 12 figures, including appendices

Transformers have demonstrated strong performance in long-term time series forecasting (LTSF), yet modeling non-stationary temporal dependencies remains challenging. In this paper, we interpret self-attention as a Bayesian posterior over attended positions, combining a content-based prior induced by scaled dot-product similarity with a temporal-evidence likelihood induced by positional encoding. This formulation provides a probabilistic interpretation of how positional encoding shapes the temporal dependencies captured by self-attention. Building on this perspective, we derive a population-level evidence lower bound (ELBO) that characterizes the role of positional encoding in forecasting weakly stationary time series. Our analysis further highlights a limitation of static positional encoding: when temporal dependence structures vary across input windows, a fixed temporal bias must accommodate heterogeneous dynamics and can therefore be suboptimal for non-stationary forecasting. Motivated by this analysis, we propose Bayesian Time-aware Attention (BTA), which employs a mixture-of-experts (MoE) router to adaptively combine temporal kernels and generate input-conditioned, head-specific positional biases. Extensive experiments on standard LTSF benchmarks demonstrate that BTA consistently improves forecasting performance across multiple Transformer backbones and achieves competitive overall performance with modest computational overhead and stable training.

0.21

Agent Memory with Episodic Retrieval for Financial Decision-Making

cs.AI ·2026-09-23 ·Nuoyue Xu, Jiang Liu, Wenxuan Huang, Xiang Zhang, Juntai Cao +1 more
publishedcode/data similarity 0.64
Abstract · The paper has been accepted for AACL-IJCNLP 2026 findings

Large language models (LLMs) have demonstrated strong capabilities in financial analysis and reasoning, inspiring recent advances in agent-based trading frameworks. While these systems show promise, prior approaches either emphasize long-horizon forecasting or operate as stateless analyzers, limiting their applicability to the demands of trading in complicated settings. To address these gaps, we introduce META (Memory Enhanced Trading Agent), the first RAG-like episodic-memory-augmented multi-agent framework for financial decision making. META integrates a family of specialized indicator agents (e.g., Trend, MACD, Stochastic, RSI, SMA, AVWAP, Heikin-Ashi) with a Decision Agent that fuses their reports, and a Memory module that retrieves and updates past trading episodes encoded as market state embeddings with outcomes and reflections. By recalling relevant experiences and adaptively reweighting signals under similar market regimes, META achieves improved directional accuracy and robustness under short-horizon evaluation. Our results demonstrate that episodic memory provides a powerful mechanism for regime-aware, interpretable, and low-latency decision-making in trading and decision making. The code of this project is released on GitHub.

0.41