CS Paper Radar

Target alignment, dilution and forecast selection when cross-sectional forecasts share a common target

Masoud Soleimani

cs.LG econ.EMq-fin.STstat.ME ·submitted 2026-09-22 12:12 UTC · v1

Abstract

Forecasters often score the same units per date against one standardized realized outcome. We show that every standardized forecast splits exactly into a component aligned with this common target and a component uncorrelated with it. Three consequences follow: forecast-error correlation largely mirrors forecast correlation and is therefore a poor measure of diversity; an equally weighted combination beats a no-information forecast only when average alignment is large relative to the combination's dispersion; and the gain from adding a forecaster separates into genuine improvement and mere dilution, which equal-weight admission can mistakenly reward. We develop a cautious selection rule, study it in simulations, and apply it to language-model forecasts of US equity rankings and mechanical signals ranking exchange-traded funds. Selection removes most dilution losses, but no combination beats the no-information forecast.

Comment
35 pages, 5 figures, 13 tables

Closest papers

Nearest neighbours by meaning across the whole index.

Forecast Collapse in Time-Series Foundation Models

cs.LG cs.AIcs.CEstat.AP+1 ·2026-08-14 ·Shu Wan, Miles Ma, Hank Zhu, Guangqi Liu, Stephen Wang +2 more
benchmarkcode/data similarity 0.66
Abstract · 27 pages, 3 figures, 5 tables. Dataset: https://huggingface.co/datasets/abel-lab/finance1k

When forecasting hourly returns for 1,000 US equities, we observe an unexpected phenomenon: predictions become nearly flat and show poor stock ranking, as measured by cross-sectional correlation. We call this forecast collapse. Surprisingly, the phenomenon largely disappears when forecasting trading volume under the same setting. We investigate forecast collapse across time-series foundation models (TSFMs), twelve deep-learning forecasting models, and 97 public benchmark configurations, and find that it is closely tied to target predictability. We identify two distinct reasons behind it: low predictability limits the amplitude of calibrated point forecasts, while per-series objectives leave cross-series structure unidentified. These findings reveal a calibration-ranking tradeoff: optimizing squared error leads to flat predictions, whereas directly optimizing cross-sectional correlation improves ranking but can inflate forecast amplitude by more than an order of magnitude. To address this tradeoff, we introduce CalibRank, a simple objective that balances calibration and ranking. On Finance1K, CalibRank nearly triples cross-sectional correlation while keeping amplitude close to the target, and improves correlation on all tested models. Our results reveal a blind spot in conventional time-series evaluation: per-series metrics can hide failures in cross-series structure needed by downstream decisions.

0.64

Loss Choice or Model Choice? The Role of Forecast Level in Cryptocurrency Volatility Forecasting

cs.AI q-fin.CPcs.CEq-fin.RM ·2026-09-22 ·Andrzej Tokajuk, Jarosław A. Chudziak
published similarity 0.61
Abstract · Accepted for publication in the proceedings of ADMA 2026

Volatility forecasts play a central role in financial risk management because their overall level and day-to-day movements affect downstream decisions. Most studies compare forecasting models while keeping the training loss fixed. Yet losses emphasise different errors and can target different properties of future volatility, so raw comparisons may combine persistent forecast-level differences with differences in daily forecast movements. This leaves unresolved whether the importance of loss choice comes mainly from the forecast level it targets or from differences that remain after level adjustment. We address this gap through a comparison of seven losses and five models across major cryptocurrencies. Validation-based alignment adjusts the forecast level before the raw and aligned forecasts are evaluated using statistical scores and one-day Value-at-Risk. Before alignment, marginal score variation is greater across losses. After alignment, model choice becomes the larger source of variation in the full five-model comparison, while cross-loss differences in VaR breach rates narrow substantially. Our contribution is a comprehensive evaluation of loss and model choice that shows why losses can appear so influential in raw comparisons and how this interpretation changes when forecast level and downstream risk are considered explicitly.

0.32

Evaluation Choices Decide the Forecasting Leaderboard: Evidence from a Production Marketplace Panel

cs.LG stat.APstat.ME ·2026-08-20 ·Md Rezwanul Islam, Wael Mohammed
benchmark similarity 0.60
Abstract · 18 pages, 8 tables, 1 figure. Evaluation protocol and audit scripts included as ancillary files

A forecasting benchmark reports which method won. We show that the answer is set by the evaluator's choices before any model is fitted. We benchmark 24 forecasting methods and one textbook reference, including six 2025-era time series foundation models, on a production marketplace panel of 1,887 business customers over 67 months. We hold the data, the horizon and the period fixed, and vary only the evaluation design. Three choices each reverse or dissolve a headline conclusion. Changing the unit of analysis from the market total to the individual customer moves our production baseline from second of nineteen, beaten by nothing, to twenty-third of twenty-five. Nineteen of its twenty-four challengers beat it there. Changing how much error is pooled decides whether a Diebold-Mariano test finds anything at all. Scoring prediction intervals rather than point forecasts reorders the field almost completely, with a rank correlation of 0.02 on intermittent demand. We then measure what the deployed system gets from this. Its selection rule captures 55% of the distance between doing nothing and choosing with hindsight. The reversal is not a quirk of our data. We ran the released protocol, unchanged, on the public M5 retail panel. The same baseline shape places first at the market total and last per series, beaten by everything, and a replayed selection rule closes 64.7% of the same floor-to-ceiling distance there. Adding five zero-shot foundation models to that roster changes who wins at the total, not the shape. The bands' blind spot travels too: conformal bands under-cover most on the spikiest items. Splitting our own panel into ever smaller groups turns the contrast into a curve: the baseline's rank worsens at every level of disaggregation. We release the evaluation protocol and report an error of our own that inverted a result before we caught it.

0.26

Two Heads Are Better Than One: Aggregating Weaker LLMs for Better Forecasts

cs.LG ·2026-09-27 ·Cheng Peng, Ruixi Luo, Zhi Chen, Wei Tang
similarity 0.60
Abstract

Large language models (LLMs) are increasingly used to forecast real-world events, but access to the strongest individual forecaster may be costly or otherwise constrained. We study weak-to-strong forecast aggregation: can individually weaker LLM forecasters be aggregated to outperform a stronger forecaster? Using ForecastBench (Karger et al., 2025), we evaluate 70 LLM forecasters across 16 comparison groups, each with more than 1,000 shared subquestions, yielding 1,121 weaker-model pairs. Within each group, we identify the strongest individual by test Brier score and evaluate aggregates composed exclusively of weaker forecasters, with aggregation weights learned on separate training data. We find substantial evidence of weak-to-strong improvement. Learned linear pooling identifies a weaker pair that matches or outperforms the strongest individual in 11 of 16 groups and comes within 5% of its Brier score in all 16 groups. We also find that these improvements do not rely on having a near-best constituent and are generally accompanied by good calibration. Additional analyses show that adding more models does not consistently improve performance, and competitive weaker-model aggregates also remain available under practical constraints.

0.41

Generalized Gibbs Ensemble Weighting for Forecast Combination

cs.LG stat.ML ·2026-08-28 ·Prasen R. Nuthanakaluva, Nava K. Gaddam
benchmarkcode/data similarity 0.59
Abstract · 15 pages, 6 tables, preprint

Forecast combination is a reliable way to improve predictive performance when several forecasting models are available. Simple aggregation rules such as the mean, median, trimmed mean, inverse-loss weighting, and exponential weighting are often strong baselines, but their relative performance can vary across datasets, forecast horizons, deployment settings, and levels of disagreement among base forecasters. We develop Generalized Gibbs Ensemble Weighting (GGEW), a probabilistic framework that treats forecasting models as experts and assigns ensemble weights using a Gibbs-style exponential transformation of normalized predictive loss. The framework extends this basic weighting rule through numerical stabilization, diversity-aware score corrections, and online hyperparameter adaptation. GGEW produces a family of related methods, including Stable Gibbs weighting, Directional Gibbs-NCL, and Symmetric Gibbs-NCL. These variants share one core algorithm and differ only in the score used inside the exponential weighting rule. For sequential deployment, we adopt a UCB-style bandit mechanism, called online Local-UCB, to adapt the learning rate, diversity strength, and Gibbs variant without evaluating the full hyperparameter grid at every prediction step. We evaluate GGEW on official M4 competition forecast submissions and external rolling-origin deployment experiments using Monash Traffic Hourly, Electricity Hourly, and Solar Weekly datasets. Results suggest that Gibbs-style adaptive weighting is a useful and competitive tool across several benchmark settings, although its relative performance varies across datasets, forecast horizons, deployment protocols, and forecast disagreement groups. The contribution is not a universal dominance claim, but a framework and empirical study motivating further investigation of when adaptive Gibbs-style forecast combination is useful.

0.43

When AI Advisors Agree: Distinguishing Forecast Risk from Forecast Reliability

cs.MA ·2026-10-09 ·Russlan Ramdowar
code/data similarity 0.59
Abstract · Version 2: 24 pages, 11 figures, 12 tables. Expanded aggregation, dependence and reproducibility diagnostics; unchanged outcome cutoff and c…

If seven AI advisors agree, what have we learned: that their forecast is reliable, or that the asset is easier to forecast? We examine 928 completed first-quarter outcomes for 232 assets across four historical iPulse AI batches. Ten Gemini configurations form seven equally weighted investment frameworks; no forecasts are regenerated. The council has mean absolute return error of 15.15 percentage points (pp), versus 14.99 for no change, with an unresolved paired difference of +0.16 pp. Error rises from 10.31 to 21.05 pp across disagreement quartiles, but the no-change baseline rises almost identically, from 9.89 to 21.15. In the pooled archive, selecting the lowest-volatility quarter produces lower error than selecting the least-disagreeing quarter: 7.97 versus 10.31 pp. Directional agreement introduces a second trap: its apparent value depends on the aggregation rule. When at least six of seven frameworks share a direction, the sign of their mean return is correct in 53.21% of cases, versus 44.11% with lower agreement. Scoring the modal direction instead changes the lower-agreement accuracy to 54.27%, removing that positive contrast. Recomputed-rank bootstraps, scale-aware losses, and period sensitivities qualify these descriptive results; four overlapping periods cannot establish future-regime performance. Agreement, visible asset risk, and aggregation-specific skill should be evaluated separately before consensus is presented as confidence.

0.60

Which Past Matters? Training History and Multi-Forecast Horizon in Equity Transformers

cs.LG q-fin.STq-fin.CP ·2026-11-17 ·Chariton Chalvatzis
similarity 0.59
Abstract

More historical data can improve estimation while introducing relationships that transfer poorly to current market conditions. We therefore examine how training quantity and historical selection interact with forecast horizon in intraday equity prediction. We deploy twelve configurations which combine four sampling and validation policies with 6, 12 and 24 month training quantities under a fixed cross-sectional Transformer and feature set. A monthly walk-forward experiment produces 264 refits and eight cumulative return forecasts from 5 to 120 minutes. Evaluation covers January 2025 through 20 October 2026 and uses cross-sectional Spearman information coefficients against future five-factor residual returns. Six-month recency sampling leads individual models at 5-15 minutes, while multi recency configurations lead at 20-30 minutes and uniform configurations at 60-120 minutes. Increasing training quantity does not uniformly improve accuracy while an equal-weight ensemble of all twelve models exceeds every individual mean at 15, 20 and 25 minutes, but not at the remaining horizons. These findings support evaluating historical selection jointly with forecast horizon without identifying a universal policy or a causal benefit from recency.

0.15

QUAL: Quantile Alignment Improves Zero-Shot Time-Series Foundation Models

cs.AI ·2026-10-30 ·Junhyeok Kang, Sangjun Han, Jun Seo, Seunghan Lee, Jaehoon Lee +7 more
publishedcode/data similarity 0.59
Abstract · Accepted at the NeurIPS 2026 Workshop on Foundation Models for Temporal Systems: From Forecasting to World Modeling

Time-series foundation models (TSFMs) provide zero-shot forecasts for unseen series, but their accuracy varies across datasets. Combining several models can reduce this variability, yet zero-shot deployment provides no target labels for selecting or weighting them. We find that a model pool is useful not only as a final ensemble, but also as a probability reference for transforming each member forecast. Aligning quantiles through this pool-relative reference improves every member forecast in aggregate. Motivated by this finding, we propose QUAL (Quantile alignment), a simple label-free framework for improving and combining diverse TSFMs. QUAL first produces aligned member forecasts, then applies Huber aggregation and rearrangement to obtain a robust combined forecast without training or model modification. On GIFT-Eval, quantile alignment reduces average member MASE and CRPS by 7.4% and 9.8%. The full framework remains robust when weak forecasters enter a candidate pool whose quality cannot be verified in advance. Code is available at https://github.com/junhyeokkang/QUAL.

0.31