CS Paper Radar

Crowds Believe in You: Online Prediction Markets Overestimate How Often People Achieve Their Own Goals

Zihan Chen, Lei Nico Zheng, Di Zhu

cs.HC cs.CYcs.SIecon.GN ·submitted 2026-10-29 15:55 UTC · v1

Abstract

People increasingly share personal goals online and invite others to forecast them. Research on the planning fallacy finds that observers predict a person's behavior less optimistically than the person does, and prediction markets aggregate such judgments, harnessing the wisdom of crowds. We study 6,150 prediction markets on Manifold in which 1,660 users asked about their own futures ("Will I ...?"), reconstruct every order and execution, and compare these self-markets with markets about external events. Self-markets resolved YES 4.4 percentage points less often than their final prices implied, whereas external markets were nearly calibrated (0.9 points); the difference was 2.2 points at the same price level and 1.9 points within creators who made both kinds of market. Absent when markets opened, the overstatement emerged within a day of trading, even where the person never traded, and peaked in the week before close. In a market's first day, observers moved external-market prices down toward the base rate, but moved self-market prices 3.6 points further toward YES, except in markets naming an outcome the person would not want, which were not overpriced. The correction came late and largely from the person, whose orders supplied 38% of the eventual movement toward the outcome in self-markets, against 9% in external markets. Because self-market prices mainly failed to separate successes from failures, a recalibration learned from earlier markets improved later forecasts by only 1.5%. Crowds believe in the futures people pose for themselves, and the information that would correct them lies with the person.

Comment
23 pages, 2 figures, 14 tables; working paper

Closest papers

Nearest neighbours by meaning across the whole index.

The Conversation Turns First: Crowd Discussion and Price Reversals in Prediction Markets

cs.SI ·2026-09-24 ·Omneya Sultan, Fred Morstatter
similarity 0.61
Abstract · 8 pages, 1 figure

Prediction markets combine trading with public discussion of the same events. We examine whether comment-derived signals predict subsequent activity, buying direction, and changes in the leading outcome. A correlation sweep across 79 non-political Polymarket markets guides six classification experiments comparing comment features, trading features, and their combinations. On live blocks containing comments, attention nearly matches trading history in predicting heavy trading within 18 hours (PR-AUC 0.786 versus 0.790, against prevalence 0.606), with its relative advantage concentrated in short markets. Comment content carries directional information: toxicity ranks future buying direction above chance in 43 of 53 scored markets, while adding attention, sentiment, and stance to flow history increases ROC-AUC from 0.780 to 0.788. Leadership changes are predicted primarily by market state. Stance shifts against the leader before reversals in 23 of 27 evaluable markets, but adds no clear improvement in individual-block forecasting. These results distinguish attention from directional support and price uncertainty. They establish predictive associations consistent with discussion and trading responding to shared information, without identifying a causal effect of comments on markets.

0.27

Prediction market visualizations, betting, and uncertainty: A study of Reddit Posts and Comments

cs.HC ·2026-08-17 ·Subham Sah, Alireza Karduni, Douglas Markant, Wenwen Dou
similarity 0.58
Abstract · 5 pages, 2 figures, 1 table. IEEE VIS 2026

Prediction market platforms present contracts about future events through visualizations that show probabilities, prices, trends, odds, and payout information. Although these visualizations often appear precise, they do not always show uncertainty directly. As a result, users infer uncertainty from market movement, visualization cues, and contextual information. In this paper, we examine how users interpret prediction market visualizations through a qualitative analysis of posts and comments from the Reddit community r/Kalshi. From an initial corpus of approximately 12,000 posts and 96,000 comments, we identified 360 posts containing prediction market visualizations and conducted a thematic analysis of annotated posts and related discussions. Our findings show that users infer uncertainty through several forms of interpretation: they interpret chart values, struggle with probability information displayed, bring in external knowledge, question credibility and liquidity, critique visualization design, and connecting visualized information to betting decisions.

0.26

You Know What You Will Like: People Predict Their Own Ratings Better Than Recommender Systems

cs.HC econ.GN ·2026-11-04 ·Zihan Chen, Lei Nico Zheng, Di Zhu
similarity 0.56
Abstract · Working paper. 23 pages, 3 figures, 10 tables

Recommender systems usually learn what people want from what they have done. Platforms could also ask people directly, but asking is costly: 98.8% of the prompts in our data went unanswered. Theories of statistical prediction imply that a person's own forecast adds little to a recommender algorithm built from many people's behavior. Theories of dispersed knowledge imply that people hold information about their own taste that the data lack. We test the two with the MovieLens Beliefs dataset. In 2023-2024, a homepage prompt asked members how they expected to rate movies they said they had not seen. Of 26,020 such expectations, 551 from 277 members were followed by a rating at least a day later. We compare each expectation with four benchmarks: the platform's own predicted rating, two collaborative-filtering models (matrix factorization and an item neighborhood), and a stack trained on other members' outcomes. The trained models use each member's latest ratings. People predict their own ratings more accurately than all four benchmarks (correlation 0.69, against 0.34 for MovieLens and 0.54 to 0.59 for the trained recommenders). Added to all the machine forecasts and features, the expectation lowers out-of-sample mean squared error by 26%. The gain holds when models are trained on earlier expectations and tested on later ones. A member's earlier expectations about other movies do not replace the current one. In ordering a member's own movies, the trained recommenders are statistically indistinguishable from the person. On old, heavily rated movies, the machines do as well or better. The results reconcile the two theories: the crowd's data suffice where they are rich, and the person adds information about the specific item. We derive four design principles for preference elicitation and recommender evaluation.

0.26

Whether Is Not When: Separating Occurrence from Timing in Polymarket Prices

cs.CY ·2026-10-27 ·Sizheng Fan, Yinghe Sun, Rong Zhao
similarity 0.56
Abstract · 11 pages, 6 figures

Prices formed by Web users on Polymarket, a blockchain-based prediction market, are now cited in the news and in a U.S. Army intelligence bulletin as early warnings of geopolitical events. However, it remains unclear whether these prices can serve as such warnings. This cannot be judged from accuracy, because most markets resolve No and accuracy can remain high even when many of the events that did happen are missed. In this paper, we study this question from two aspects, using all 1.24 billion on-chain trades in 2.37 million Polymarket markets through September 2026. First, from 3,686 markets on geopolitical conflicts, we derive 208 unscheduled events that did happen and measure how far in advance prices move before the first Web report of each event. Second, we separate the information in prices about whether an event happens from the information about when it happens. To do so, we group 1,551 contracts into 338 deadline ladders and read each ladder as one forecast of the event date. We then split its log score, and the profit of 9.3 million trades by 491,000 accounts, exactly into these two parts. We find that one day before they were reported, prices correctly predicted only 56% of the geopolitical events but all 51 scheduled central-bank decisions. Separating the two parts shows that only about one eighth of the information in prices concerns timing. A small group of large accounts earns timing profits, but beyond current prices, no group of accounts helps predict an event within days. A high price is therefore a sign that an event is likely rather than imminent. Near-term warnings read from the whole ladder are more often right than those read from a single contract. Platforms should report recall alongside accuracy, and show near-term probabilities read from the whole ladder.

0.34

Price Dislocations, News Citations, and Epistemic Leverage on Polymarket

cs.CY physics.soc-phcs.SI ·2026-09-05 ·Hazem Ibrahim, Yasir Zaki
similarity 0.56
Abstract · 17 pages, 3 figures

Prediction-market probabilities increasingly appear in news coverage, yet little is known about which market movements become news or how much trading money sits behind the numbers journalists quote. Unlike a poll, a market price can be moved by anyone willing to trade, so the cost of manufacturing a number that circulates as news bears directly on the information environment. We link 173.7 million signed Polymarket trades to news coverage from 2024-2025. From 6,990 articles mentioning prediction-market venues, an LLM-based, human-validated matcher extracts 1,582 sentences quoting market odds and attributes 918 to the specific market whose price they cite. We then detect 44,976 price dislocations, movements of at least five percentage points backed by concentrated one-sided trading, and ask whether a market is cited more often afterward. In the days after a dislocation, a market's citation rate is about 33% higher than its matched baseline (log citation-rate ratio $τ_{\mathrm{cite}}=0.283$, permutation $p=0.001$), robust to binary and Poisson count outcomes. Yet move size is not the strongest predictor of citation: prominence dominates (standardized $β=0.610$ vs. $β=0.159$ for move size). Finally, we combine the dollar flow behind a given price change with observed citation rates into a metric we call epistemic leverage, the dollars needed to move a market five points and have the move cited. It stays near \$0.7-1.0 million across prominence quintiles, because cheaper-to-move markets are proportionally less likely to be cited. The implied threat model centers not on the long tail of cheaply moved markets but on the few prominent markets newsrooms treat as informational infrastructure, where a seven-figure price of influence sits within the budgets of actors with a large stake in the quoted number. We release aggregate event-study data and validation materials.

0.44

Approximate Recall, Approximate Forecasts: Recall as a Diagnostic for LLM Forecasting Errors

cs.AI cs.CEcs.LG ·2026-11-07 ·Prashanth Bhaskara, Shubhaankar Gupta, Seojoon Yeon
published similarity 0.56
Abstract · 16 pages, 7 figures (including appendix). Accepted as a Spotlight at the ICML 2026 Workshop on Forecasting as a New Frontier of Intelligence

Large Language Models (LLMs) exhibit approximate recall: they retain coarse associations about past events while losing fidelity on specific details. Past research on LLMs has quantified different models' forecasting ability in real-world events using prediction markets, but has not studied how losing said fidelity impacts their forecasting abilities. For example: while an LLM correctly recalls that the answer to: ''Did Anna Sawai win the Best Drama Actress in the 76th Emmy Awards?'' is ''Yes,'' it incorrectly states the event date as September 22 rather than September 15. In this paper, we introduce a novel metric called the Recall Fidelity Score to study how imprecision in internalized knowledge propagates to forecasting capability on future events. On resolved historical markets, we find that higher local recall fidelity around semantically similar past events predicts lower forecasting error on resolved markets, and the same signal predicts smaller distance from open markets prices.

0.23

Evaluation Choices Decide the Forecasting Leaderboard: Evidence from a Production Marketplace Panel

cs.LG stat.APstat.ME ·2026-08-20 ·Md Rezwanul Islam, Wael Mohammed
benchmark similarity 0.55
Abstract · 18 pages, 8 tables, 1 figure. Evaluation protocol and audit scripts included as ancillary files

A forecasting benchmark reports which method won. We show that the answer is set by the evaluator's choices before any model is fitted. We benchmark 24 forecasting methods and one textbook reference, including six 2025-era time series foundation models, on a production marketplace panel of 1,887 business customers over 67 months. We hold the data, the horizon and the period fixed, and vary only the evaluation design. Three choices each reverse or dissolve a headline conclusion. Changing the unit of analysis from the market total to the individual customer moves our production baseline from second of nineteen, beaten by nothing, to twenty-third of twenty-five. Nineteen of its twenty-four challengers beat it there. Changing how much error is pooled decides whether a Diebold-Mariano test finds anything at all. Scoring prediction intervals rather than point forecasts reorders the field almost completely, with a rank correlation of 0.02 on intermittent demand. We then measure what the deployed system gets from this. Its selection rule captures 55% of the distance between doing nothing and choosing with hindsight. The reversal is not a quirk of our data. We ran the released protocol, unchanged, on the public M5 retail panel. The same baseline shape places first at the market total and last per series, beaten by everything, and a replayed selection rule closes 64.7% of the same floor-to-ceiling distance there. Adding five zero-shot foundation models to that roster changes who wins at the total, not the shape. The bands' blind spot travels too: conformal bands under-cover most on the spikiest items. Splitting our own panel into ever smaller groups turns the contrast into a curve: the baseline's rank worsens at every level of disaggregation. We release the evaluation protocol and report an error of our own that inverted a result before we caught it.

0.26

Time-Varying Favorite-Longshot Bias in Polymarket Weather Markets

cs.DB ·2026-10-09 ·Shouchen Qiu, Huaijun Qiu
top venuecode/data similarity 0.55
Abstract · 5 pages, 4 figures, 3 tables. Accepted at the IEEE ICDM 2026 Teen Research Symposium (Teen Track), Shenyang, China, November 2026. Code avai…

Prediction-market prices combine traders' judgments about future outcomes, but their errors may be concentrated in particular price ranges and change as settlement approaches. Prior cross-category evidence shows that Polymarket weather markets have larger pricing bias than other major categories, without separating low-price longshots from high-price favorites. This study compares the two ranges at different horizons in daily maximum-temperature markets. Event-level probability scores, cubic B-spline recalibration, and Buy-NO tests are used to examine the form and effect of the bias. Longshot overpricing mainly occurs at distant horizons, whereas favorite overpricing becomes much stronger during the final 1-12 hours and is nearly one order of magnitude larger. Event-level scores improve toward the daily temperature peak, while simple recalibration yields no statistically significant gains. The Buy-NO results in the corresponding time windows also show that the main pricing error is concentrated at the favorite end.

0.57