CS Paper Radar

From SEC Filings to Financial Signals: Evaluating LLMs on Stock Market Reactions and Analyst Forecasts

Shengkun Wang, Taoran Ji, Linhan Wang, Sattar Mansi, Chang-Tien Lu, Yinlin Chen

cs.AI ·submitted 2026-10-16 21:54 UTC · v1
hot direction

Abstract

Form 10-Q MD&A sections combine long-form management narrative with accounting and forward-looking information. Benchmarks for financial filings commonly evaluate question answering or numerical reasoning, leaving unclear whether outputs from large language models (LLMs) align with market and analyst outcomes. We evaluate prompted, zero-shot outputs from 13 general-purpose and finance-specialized LLMs on 45,964 Form 10-Q filing events from 2001 to 2024, linking MD&A text to cross-sectional fundamental ranks, pre-filing analyst forecasts, and realized abnormal returns. The evaluation covers return ranking under five controlled information views and next-quarter EPS forecasting under direct and consensus-anchored interfaces. Relative to disclosure alone, adding cross-sectional fundamental ranks raises mean effective-date rank correlation (RankIC) by 0.0089 for effective-event-date returns and 0.0098 for following-day returns, whereas analyst-summary context provides no comparable average gain. Across the four disclosure-containing views, finance-specialized models show an effective-event-date group advantage but no statistically reliable following-day advantage, while general-purpose Qwen models produce the strongest individual return-ranking results. Under the full-information view, direct LLM forecasts average the 43.4th percentile of the analyst distribution and do not consistently beat individual analysts. Consensus anchoring raises the mean inside-range share from 41.1% to 59.5% but increases reported-EPS error.

Closest papers

Nearest neighbours by meaning across the whole index.

Momentum, Commentary Tone, and Earnings Surprise in LLM Financial Forecasts

cs.LG cs.AIcs.CLq-fin.GN ·2026-10-15 ·Alina Khaybullina
benchmark similarity 0.65
Abstract

Which inputs drive a language model's financial forecast? We construct synthetic earnings announcements that independently reverse 20-day momentum, non-informational commentary tone, and whether fixed actual EPS beats a manipulated consensus. Six instruction/chat checkpoint conversions from four families (0.49-7.61B base parameters) receive 2,016 prompts each under a direct-answer Q8_0 protocol. The design includes 24 numerical blocks, six commentary pairs, profits and losses, rising and falling annual EPS, and two surprise sizes. Both A/B answer assignments and U/D cover all 384 primary scenarios per conversion. We estimate matched changes in conditional answer-prefix scores, not calibrated market probabilities. Across these checkpoints, the primary EPS contrast ranges from -0.19 to 86.80 percentage points. The magnitude of the primary signed tone contrast exceeds that of EPS in 5 of six models, and momentum exceeds it in 5. Natural-word numerical diagnostic accuracy ranges from 51.0 to 95.8 percent. Answer-encoding, wording, context, and conversion controls qualify these results. The experiment measures observable input reliance and numerical recognition; it establishes neither an internal reasoning mechanism nor realized return predictability. Public benchmark prompts, recorded scores, and an interactive recorded-results explorer accompany the study.

0.30

Measuring What is Repeated and Novel in Earnings Disclosures Using Optimal Transport

cs.CE ·2026-10-02 ·Yuntao Wu, Lynn Tao, Charles Martineau, Vincent Grégoire, Andreas Veneris
similarity 0.64
Abstract · 9 pages, 4 tables, 4 figures

We introduce an optimal transport framework to decompose the textual content of earnings press releases and conference calls into aligned (shared) and unaligned (unique) components, and use these as predictors of stock returns around earnings announcements. Using over 105,000 document pairs from 2008 to 2023, we find that the aligned portions of both disclosures explain announcement-day returns with high statistical significance. The unaligned component of conference calls, which captures forward-looking content absent from press releases, provides substantial incremental explanatory power, shedding light on why conference calls are central to market reactions to earnings news. Extending to post-announcement returns, investors appear to overreact to shared press release content while underreacting to the unique information conveyed in conference calls.

0.30

Tabular Foundation Models in Financial Predictions

cs.LG q-fin.CPq-fin.ST ·2026-11-10 ·Ed deHaan, Allen H. Huang
benchmark similarity 0.62
Abstract

This technical note introduces tabular foundation models (TFMs) to accounting research and evaluates their performance in one common benchmark task: predicting firms' next-year earnings changes. TFMs produce numeric predictions and learn in context, without post-release training or tuning, making them natural and easy to use for forecasting. The TFMs we study are pretrained on synthetic tabular data, so they avoid look-ahead bias from memorizing firms' outcomes. We compare Google Research's TabFM with a fine-tuned open-weight large language model developed in recent accounting research and with several tree-based models. TabFM is more accurate than all comparison models. It is also faster and less computationally intensive than the large language model. Several tree-based specifications produce predictions more quickly than TabFM, so choosing between them involves a trade-off between accuracy and speed. We conclude that TFMs have promise as a new tool for forecasting in accounting and finance. However, our evaluation is based on a single benchmark task, and no class of models is likely to dominate in all settings.

0.20

FinForesight: A Diagnostic Backtesting Benchmark for Deep Research Systems in Financial Forecasting

cs.CL cs.LG ·2026-11-01 ·Fengyuan Liu, Mukai Li, Jieshun You, Yilin Zheng, Chenjia Min +3 more
benchmarkcode/data similarity 0.62
Abstract

Anticipating financial outcomes requires synthesizing evolving evidence and reasoning under uncertainty, making deep research systems promising tools for forecasting. Yet evaluating forecasting ability is challenging because historical backtests may reward memorized outcomes or information retrieved after the forecast date. Live forecasting avoids direct access to realized outcomes, but feedback arrives only after events resolve. To obtain timely feedback and assess historical scores, we introduce FinForesight, a diagnostic backtesting benchmark for financial deep research systems. Across the Chinese, U.S., and Hong Kong markets, the benchmark evaluates four forecasting tasks: valuation, return ranking, event direction, and future financial fundamentals. Two capability diagnostic tasks assess counterfactual reasoning and financial calculation, scored separately from forecasting performance. Our harness-compatible framework restricts retrieved evidence to information available at the forecast date. We compare pretraining-era, resolved post-cutoff, and live evaluations to test whether controlled backtests approximate future performance. Pretraining-era scores exceed post-cutoff scores in most temporal comparisons, and stronger retrieval controls narrow the gap between post-cutoff backtests and live results. Forecasting attribution and robustness analyses reveal how scores respond to retrieval policy and how predictions respond to prompt rewrites and numerical perturbations. The code and benchmark data are publicly available.

0.30

XForecast: The KDD 2026 Finance Day Challenge on News-Grounded Financial Forecasting

cs.LG ·2026-10-12 ·Jaehoon Lee, Jun Seo, Seunghan Lee, Tae Yoon Lim, Dongwan Kang +8 more
benchmark similarity 0.62
Abstract · 6 pages

Financial forecasting has traditionally relied on numerical time-series (TS) signals, while textual information such as news remains largely underutilized. We introduce XForecast, a benchmark and competition for news-grounded financial forecasting, where the task is to predict stock price movements four weeks ahead using both historical price data and paired news signals. XForecast combines stock price data with curated financial text generated using FinTexTS, a multimodal dataset generation method. It covers 100 large-cap stocks from 2019--2023, with training on 2019--2022 and evaluation on 2023. To better evaluate models that leverage textual signals, we adopt a weighted hit rate metric that places greater emphasis on instances where news information is expected to have stronger influence. The XForecast Challenge, hosted on Kaggle as part of KDD 2026 Finance Day, aims to benchmark methods for integrating financial TS and textual information.

0.09

FinRiskAtlas: Decision-Aligned Evaluation of Large Language Models for Financial Risk Review

cs.AI cs.CL ·2026-08-26 ·Suyang Zhong, Jingzhe Zhu, Qi Xu, Liyao Sun, Yin Wang +3 more
benchmark similarity 0.62
Abstract

Deploying large language models for professional financial review requires more than measuring general financial competence: models must perform the specific review operation required by a workflow and determine whether available evidence is sufficient for a defensible decision. Existing financial benchmarks cover knowledge, reasoning, compliance, and professional tasks, but their evaluation units are often organized around datasets or task formulations rather than the decisions that deployed systems support. We introduce FinRiskAtlas, a Chinese-language benchmark that evaluates financial LLMs along two complementary dimensions: operation execution under fixed evidence states and evidence-state control under evolving review conditions. The static benchmark contains 9,742 instances across 53 task families, including 42 Domain Knowledge families and eleven downstream review operations defined by explicit evaluation contracts. FinRisk-Ask extends this framework through offline replay of 680 pre-action states from 104 de-identified professional trajectories, withholding future evidence during inference and using it only to construct expert-verified evidence targets. Across 33 model configurations, operation-level evaluation yields non-redundant rankings (mean pairwise Spearman correlation 0.42 across downstream operations), and knowledge-based shortlisting can incur up to 18.01 points of regret on individual operations. FinRisk-Ask further shows that entering the Ask branch more frequently does not necessarily improve request targeting or end-to-end evidence acquisition. These results show that broad financial capability scores do not fully capture where models are reliable in professional workflows, motivating evaluation units aligned with the decisions and evidence states that deployed systems must support.

0.45

IGT @ FinMMEval 2026 Task 2: Question-Type Prompting with Targeted Extraction for Multilingual Financial QA

cs.CL cs.AI ·2026-09-08 ·Yuwen Chiu
published similarity 0.61
Abstract · Accepted at CLEF 2026 FinMMEval Workshop (Working Notes). 12 pages, 4 figures

We present the IGT system for PolyFiQA Task 2 of the FinMMEval Lab at CLEF 2026, a multilingual financial question answering task over English SEC filings and multilingual news articles (English, Chinese, Japanese, Spanish, Greek) for four companies. Our central observation is that the 344 development questions divide into two families requiring fundamentally different approaches: structured numeric types (R&D ratio, cash flow, capital expenditure) are best answered by direct keyword extraction on filing text, while synthesis types (investment strategy, capital allocation, top-three revenue focuses) require rule-based multilingual news passage selection. A dataset analysis reveals that 17-18 of 19 ground-truth reference answers per synthesis type share an exact evidence label prefix, whose unigram tokens contribute directly to ROUGE-1 overlap. The final system achieves development ROUGE-1 approximately 0.395, a 60% relative improvement over a generic RAG baseline (approximately 0.247), and ranks 3rd of 12 teams on the official test set with ROUGE-1 = 0.3071, Precision = 0.2821, and Recall = 0.4044.

0.45

EPSBench: Benchmarking LLM Agents for Earnings Forecasting

cs.CL cs.AI ·2026-10-28 ·Yanbin Yin, Chuyu Xu, Liqin Ye, Jing Peng, Chao Zhang
benchmarktheory similarity 0.61
Abstract

Quarterly earnings forecasting requires an agent to connect financial evidence to the income-statement line items that produce a company's reported earnings. We introduce EPSBench, a benchmark for studying this ability across 149 firm-quarters from 86 US companies. Agents read a fixed collection of filings, earnings-call transcripts, and news, then estimate company-specific income-statement drivers with cited evidence. A shared accounting calculator converts these estimates into GAAP diluted earnings per share (EPS), providing a common basis for comparing forecast assumptions. We evaluate six models under a common harness, assessing point forecast error relative to date-matched analyst consensus and checking whether their prediction intervals achieve the stated coverage. Three models show no detectable paired difference in point forecast error from consensus, and three have higher error. Across all six models, nominal 80% intervals contain reported EPS in only 37--70% of runs that return an interval, and most misses fall above the upper bound because the forecasts run low. The driver records trace most of the error gap between models to operating-income estimates.

0.39