CS Paper Radar

From Repetition to Recognition: Inductive Discovery of Disinformation Narratives

Max Upravitelev, Veronika Solopova, Jing Yang, Charlott Jakob, Alexandra Tsiakalou, Neda Foroutan, Vera Schmitt

cs.CL ·submitted 2026-09-10 06:17 UTC · v1
top venue

Abstract

In disinformation datasets, narratives are often understood as recurring interpretive patterns that group texts under narrative labels. Recent work formalized narrative mining as inductively inferring narrative labels from corpora, but its evaluation stays tied to predefined taxonomies, a closed-world setting that cannot capture narratives absent from the reference labels. We introduce a three-tier evaluation framework for unsupervised narrative label generation: recovery (against a corpus's own taxonomy), mining (against external label sets), and discovery (without predefined labels). Applying it, we compare clustering-based and graph-community-based pipelines across seven disinformation datasets, with human validation of discovery on two. The two families are complementary under automated metrics, but in a corpus with two prominent topics, clustering can reduce one topic to 2% of generated labels while graph-based pipelines stay balanced. Discovery validation also reveals many singletons (narrative labels derived from single claims, 30-62% of graph outputs), which clustering cannot produce. Annotators confirm many as recognizable disinformation narratives, suggesting that in open-world discovery the repetition assumed by narrative mining may be recognized outside the corpus, not within it. We release human-validated narrative candidate labels for the Climate Obstruction and PolyNarrative datasets to support taxonomy development and dataset extension.

Closest papers

Nearest neighbours by meaning across the whole index.

Molecules of a Story: Community Detection in PMI-weighted Narrative Networks

cs.CL ·2026-10-05 ·Kasper Fyhn, Rebekah Baglini
similarity 0.67
Abstract

Automatically extracted narrative networks -- graphs with entities as nodes and their relations as edges -- have proven useful for revealing central narrative structures through salient entities and their connections (Tangherlini et al. 2020; Labatut and Bost 2019). But a narrative is more than those central structures that everything else revolves around. This work is concerned with the everything else: brief sub-plots, small clusters of descriptions, or associations between minor characters that go under the radar at the macro-level. We present an approach to unearth such peripheral structures. They involve rare entities with limited textual presence, overshadowed by dominant entities and lost among each other in the long tail of many but rare entities (Baayen 2001). We leverage the known tendency of pointwise mutual information (PMI, Church and Hanks 1990) to inflate for rare events, turning its weakness into a strength by weighting edges with PMI to foreground peripheral entity configurations. Communities extracted from the resulting network are structural traces of underlying narrative elements. We demonstrate the approach on The Lord of the Rings. From measures of how concentrated or dispersed a community's activations are across the text, a typology emerges that reveals that peripheral structures form more than a single class: episodic passages, echoing long-distance connections, and recurring threads each surface as distinct configurations. The approach is conceptually simple and surfaces fine-grained narrative details that are lost in abundance, though its deliberate amplification of weak signals comes with inherent sensitivity -- best understood as a lens for exploration rather than a robust extraction pipeline.

0.19

Birds of a Feather Flock Together: Network-Based Detection of Coordinated Disinformation Campaigns on Telegram

cs.SI cs.CY ·2026-09-22 ·Panteleimon Tsagkarakis, Emmanouil Papadogiannakis, Evangelos Markatos
published similarity 0.63
Abstract

In recent years, disinformation has increasingly proliferated across social networks. The firing of fact-checkers from Meta and the disbanding of Twitter's Trust and Safety Council suggest that this trend will continue to escalate. While disinformation sources (e.g., social network accounts) can sometimes be identified, new accounts emerge daily, making tracking a moving target. In this work, we propose a graph-based methodology to discover previously unknown Telegram accounts that spread disinformation. Starting from a set of verified disinformation groups, we examine their interconnections and identify new accounts that contribute to the dissemination of false narratives. Our approach is language-agnostic, as it relies solely on structural relationships between accounts rather than analyzing their message content. Using this approach, we identify 37 previously unknown disinformation channels (a threefold increase). We demonstrate that Telegram channels display extremely dogmatic behavior with up to 80% of messages being labeled as propaganda. Our findings reveal that misinformation on Telegram spreads within tightly interconnected clusters, in some cases, with over 86K identical messages being shared to multiple channels, suggesting coordinated disinformation campaigns.

0.38

Look What You Made Us Cluster: Hate Narrative Extraction from Reddit Discourse

cs.CL cs.SI ·2026-09-29 ·Annabelle K. L. Chua, Forster J. Khoo, Joel C. R. Tan, Huey Ting Ang, Kheng Hwee Tan +6 more
published similarity 0.61
Abstract · Accepted to IDeaS Conference 2026

Narrative extraction allows us to identify online hate narratives, supporting the construction of rigorous detection systems. Existing computational approaches, however, are limited in precision as they rely on semantic representations, which tend to capture only surface-level meaning. To detect more precise and interpretable narratives, we present an extraction pipeline that represents narratives as entity-evaluation pairs. Narratives are extracted using a Large Language Model (LLM) reasoning process that extends Aspect-Based Sentiment Analysis, identifying the aspect, classifying its judgement type as the basis for evaluation, and deriving the evaluation accordingly. Extracted narratives are then clustered using Leiden, following which clusters are resolved to an intended level of granularity through an LLM-guided refinement process. We illustrate this narrative pipeline with English Reddit comments from 2024 that criticize Taylor Swift, analyzing a representative cluster that exhibits hate speech patterns to demonstrate its interpretive value.

0.25

Critic's Eye: Review-Grounded Graph Reasoning for Long-Form Motif Discovery

cs.AI ·2026-10-26 ·Xinyu Sun, Zhangyue Yin, Xinyi Ju, Xin Wang
top venuebenchmark similarity 0.60
Abstract · 17 pages, 8 figures, 6 tables. EMNLP 2026

Deep narrative understanding requires distinguishing what happens from how it is presented, moving beyond the chronological fabula to decode the sjuzet, the discursive organization through which themes are staged. We study this through the recurrence of literary motifs (blood in Macbeth, for instance), where symbolic significance emerges not from local cues, but through repetition-with-variation across the whole work. We operationalize this setting as Computational Motif Discovery, a transductive task that predicts missing line-motif links using the full narrative structure of a play. We propose Critic's Eye, which models the work as a heterogeneous Narrative Topology Graph and performs discriminative inference over global structural evidence. Critic's Eye achieves 84.8% Hit@5 on our benchmark, nearly quadrupling state-of-the-art proprietary foundation models (Gemini-3-pro-preview, ~22.6%). Our analysis reveals that despite massive scaling, sequence-based models hit a "semantic ceiling", struggling to resolve dispersed dependencies. These findings suggest that explicit structural priors are a far more effective inductive bias than parameter scaling for decoding the complex architecture of long-form literature.

0.38

WORTH Benchmark: Evidence Discovery under Not-Fully-Cooperative Information -- A Two-Episode Pilot

cs.CL cs.AI ·2026-10-20 ·Bo Zou, Chao Xu
benchmark similarity 0.60
Abstract · 31 pages. Pilot study, two episodes

Existing reasoning evaluations typically provide task-relevant information to models as available premises, rarely examining concurrently how models discover evidence autonomously and how information presentation within narratives guides or obscures judgment. This paper proposes the WORTH Benchmark: models read original mystery narratives without clue annotations, autonomously select key clue sentences, identify case participants, and explain their reasoning process. The narrative materials are designed by professional authors such that evidence facilitating reasoning coexists with narrative arrangements that delay the revelation of truth. We report a pilot experiment under full-story conditions across two episodes, conducting a total of 70 formal runs on models, of which 67 runs are content-scorable. Scoring separately records hits on author-designated key clue sentences and participant selections; natural language explanations are used for case diagnosis and are not scored independently. The results show that clue hits and participant judgments are not synchronized; across 37 content-scorable runs in Episode 02, 31 runs simultaneously selected the same set of incorrect participants, while adjacent suspicion-oriented and exoneration-oriented sentences were selected 25 times and 0 times, respectively. These phenomena illustrate the diagnostic value of component-wise analysis and suggest that not-fully-cooperative informational relationships may influence evidence selection and judgment. This study only implements full-story conditions for two episodes, and does not establish stable model rankings or infer general mechanisms on this basis; human player records have not yet been cleaned, and this version does not report human-model comparisons.

0.24

Evaluating Whether LLMs Can Reliably Connect the DOTs?

cs.CL ·2026-09-29 ·Eftekhar Hossain, John Salvador, Santu Karmaker
benchmarkcode/data similarity 0.60
Abstract

Access to real-world information is often noisy and fragmented. Constructing a coherent narrative from such fragments requires models to reconstruct missing spans within a broader storyline, commonly referred to as text infilling, while preserving consistency with both the local context and the global storyline. Despite using text infilling as a pre-training objective in many Large Language Models (LLMs), their actual performance on real-world narrative infilling remains underexplored. In this paper, we address this gap by introducing a multi-domain benchmark of ~9.2K instances for narrative infilling, constructed by masking one to three sentences across four narrative types: encyclopedic text, commonsense stories, news articles, and visual narratives. Using this benchmark, we evaluate 20 instruction-tuned open-source LLMs ranging from 1.5B to 70B parameters across varying levels of instruction specificity and reasoning guidance. Outputs are assessed using standard automatic metrics and a qualitative framework covering five narrative dimensions. Results show that model scale does not reliably predict infilling quality: Gemma-2-2B achieves the highest qualitative score (4.02/5), outperforming models over ten times larger, including DeepSeek-Qwen-32B (3.77/5, 6.6%) and LLaMA-3.3-70B (3.71/5, 8.3%). We further find that explicit reasoning offers limited benefits as chain-of-thought reasoning yields only a marginal improvement (+0.6%). Additionally, short narratives and domain characteristics emerge as stronger predictors of task difficulty than infill position alone for narrative infilling in current LLMs.

0.27

Investigating Forecasting Capabilities of Narrative Networks on Emerging Stories

cs.SI ·2026-10-26 ·Ravi Teja Karumuri, Steven Corman, Huan Liu, Joshua Garland
similarity 0.58
Abstract

Networks have been used to model the interactions among the elements that underlie a narrative. In this paper, we investigate whether the structure of these networks can predict which links will emerge as a storyline evolves. We construct these networks from news articles using a Semantic Edge Parser (SEP) built on a Large Language Model (LLM), and evaluate the networks on three disparate real-world storylines, the 2022 Canadian Freedom Convoy, Hurricane Melissa (2025), and the 2025-26 Iranian protests. To test the narrative network's predictive capability, we construct a rolling window of the narrative and predict the emergent links between existing narrative elements on the day following the rolling window period. We identify candidate links with seven standard link predictors, including the Adamic-Adar index, Common Neighbors, and Preferential Attachment. Forecasting next-day links across the three storylines, we find Preferential Attachment to be the strongest standard link predictor, achieving a mean ROC-AUC of 0.76-0.79. It exceeds EdgeBank, a memorization baseline, by 11.3-14.5 points on every event. To summarize, narrative-network structure ranks next-day emergent relations above chance, though this advantage is concentrated among the most frequently mentioned, highly connected entities.

0.21

LEDGER: LLM-Evidence-Driven Graph-Encoded Clustering-Friendly Representations for Short Texts

cs.SI ·2026-11-13 ·Jie Wang, Kuang Zhou
similarity 0.58
Abstract

Short-text clustering is important for analyzing user-generated content in social networks and online services, supporting applications such as topic discovery and user intent analysis. However, the desired grouping depends on an application-specific clustering criterion that general-purpose text embeddings may not capture. Large language models (LLMs) can apply such a criterion to a pair of texts without human annotation, and existing methods use their responses as generated descriptions, binary decisions or selected neighbours. These discrete outputs discard how strongly the LLM supports a pair and how many judgments stand behind it. Each judgment is also used alone or within the neighbourhood of a single text, so the relations that link judgments across the corpus remain unused. In this paper, we propose LEDGER, an LLM-guided graph representation learning framework for short-text clustering. LEDGER aggregates the repeated numerical scores of each text pair into an accumulated score and an observation count, which define an evidence target for every queried pair. The pairs whose scores reach a threshold form a relation graph, over which a graph encoder propagates text features with a propagation range learned jointly with the encoder. A pairwise predictor trained to match the evidence targets shapes the learned representation, and a distilled student encoder extends it to unqueried texts without further LLM requests. Extensive experiments on seven datasets with four frozen text embeddings show that LEDGER achieves the highest mean clustering accuracy and NMI against five comparison methods, and ablations show that the graph, the learned propagation range and the evidence target each improve performance.

0.10