CS Paper Radar

Mapping the Afterlife of Future Work Statements in NLP Research

Claudiu Creanga, Ioana-Roxana Boriceanu, Ana Sabina Uban, Liviu P Dinu

cs.CL ·submitted 2026-11-13 19:04 UTC · v1
dataset

Abstract

Future work statements are one of the few places in scientific papers where authors explicitly describe what they think should happen next. We study what happens to such statements after publication. We construct a dataset of 72,070 future work items extracted from 23,113 ACL Anthology papers published between 1979 and 2025, categorize them by type, retrieve candidate follow-up papers through citation-based and content-based search, and use LLM-based judgments to assess whether later papers pursue the proposed directions. Using this resource, we analyze patterns across time, venues, categories, themes, and authors. We find that many items are later addressed, often within a few years, with clear differences across types and themes and a largely community-driven pattern. Our results suggest that future work statements provide useful evidence for studying how research directions develop in NLP. To support further research, we release our dataset at: https://doi.org/10.5281/zenodo.20385239.

Closest papers

Nearest neighbours by meaning across the whole index.

FutureLens: Longitudinal Topic Patterns in Scholarly Text and Future-Work References

cs.DL cs.AIcs.IR ·2026-10-09 ·Ibrahim Al Azher, Hamed Alhoori
similarity 0.69
Abstract · 10 pages, Accepted at AI4SciSci 2026 (3rd International Workshop on Artificial Intelligence for the Science of Science), co-located with ACM…

Future-work statements offer a record of the questions researchers choose to leave open. Using them to describe a field, however, requires separating thematic change from changes in document coverage, topic assignment, and scholarly writing. FutureLens examines these measurement issues in a recovered snapshot of 5,655 ACL papers published during 2012--2022, including 4,005 with supplied future-work reference text. We compare non-negative matrix factorization (NMF) and latent Dirichlet allocation (LDA), aggregate topic presence at the paper level, and examine normalized prevalence, exploratory trend statistics, assignment stability, and agreement between input-derived text and references. Across available seeds, NMF has higher mean normalized pointwise mutual information (0.1387 versus 0.1280) and higher pairwise assignment stability, while LDA has greater keyword diversity (0.8900 versus 0.7750). Seed-zero results show increasing prevalence of broad model-training and question-related vocabulary, alongside substantial citation and document-structure content. Mean input/reference topic-set overlap is 0.5191 for NMF and 0.4924 for LDA; temporal rank agreement varies widely and does not imply agreement in prevalence levels. These findings support a descriptive account of the available corpus, with explicit qualifications for missing references, incomplete runs, and unverified reference provenance. The contribution is an empirical measurement study of longitudinal scholarly signals, not a new topic-modeling algorithm or a forecasting system.

0.41

Citing Less Critically: LLMs Reshape the Rhetoric and Reach of Scientific Citation

cs.DL cs.CLcs.CYcs.SI ·2026-09-01 ·Yixuan Liu, Lin Chen, Zhuoqi Liu, Jianglin Lu, Dakota Murray
top venue similarity 0.65
Abstract · Accepted at the EMNLP 2026 main conference

Scientific citations carry rhetorical intent. Scholars may cite prior work positively (supporting), negatively (contrasting), or neutrally (mentioning). As large language models (LLMs) increasingly assist scientific writing, whether they reproduce citations with the same rhetorical intent as humans remains unclear. We introduce a masked-citation task to compare human and LLM-generated citation behavior. For each citation context, an LLM generates a replacement citation sentence, producing a counterfactual corpus directly comparable to human citation. We analyze what, whom, and how models cite, using an LLM-as-a-judge to classify citation intent and a 20-million-edge coauthorship network to measure social distance between cited authors. Across six popular LLMs and 1,746 top NLP conference papers (63k+ contexts, 132k+ citations), three patterns emerge: (1) Compared with human citation, LLMs cite significantly less critically; (2) LLMs over-cite popular and older papers, a tendency amplified for contrasting citations where human writing more often draws on recent, niche work; (3) Whereas humans often cite within their close social network, especially for supporting citations, LLMs tend to draw on more socially distant authors. Together, these differences are double-edged: LLM citation reaches beyond a scholar's close collaborators while being less critical and amplifying visibility bias, reshaping the rhetoric and reach of scientific citation.

0.34

Drift Inspector: Exploring and Measuring Scientific Drift with Atomic Contribution Claims

cs.CL cs.DL ·2026-09-30 ·Vsevolod Karimov, Stepan Ostarkov, Anastasia Poroshina, Anatoly Frolov, Alexander Panchenko
top venuecode/data similarity 0.65
Abstract · Accepted to EMNLP 2026 System Demonstrations. 11 pages. Live demo, code and data: https://hamyrappy.github.io/drift-inspector

Scientific abstracts mix contributions with background, motivation, and meta-language, so tools that read them as-is cannot separate what a field produces from what it discusses. We present Drift Inspector, an open-source system for measuring and exploring how a research field changes over time at the level of Atomic Contribution Claims (ACCs): decontextualized, contribution-bearing propositions an LLM extracts from each abstract before analysis. The system clusters these claims across years into an interactive map where every trend traces back to the claims and papers behind it. Applied to six years of EMNLP, it shows the field shifting away from classic NLP tasks toward LLM-era capabilities such as reasoning and multimodality -- a movement that keyword or whole-abstract counts blur. The released data extend beyond EMNLP: the same pipeline has processed the full ACL Anthology (346k claims, 80k abstracts, 423 venues). Extraction is human-validated and clustering checked against an external manually constructed taxonomy.

0.51

InsightRAG: Aggregating Distributed Scholarly Judgments for Scientific Survey Generation

cs.AI cs.DL ·2026-10-20 ·Yuquan Lin, Wenshuo Zhang, Yilin Ye, Shuchang Xu, Zhao Chen +2 more
similarity 0.64
Abstract

The rapid growth of scientific literature makes survey writing increasingly challenging, motivating automatic survey generation. Yet, existing systems must search an open space of papers and an open space of concepts with no trace of how the community itself has navigated them, leading to incomplete scope, weak taxonomies, and unreliable citations. Our core insight is that author-written Related Work sections contain the human judgments needed to address both limitations: authors curate the citations relevant to their contributions and explain how the cited works relate, yielding paths for scope reconstruction and relational evidence for organization. We propose InsightRAG. Its insight-guided retrieval follows author-supported citation relations to reconstruct a verified research scope and structure the resulting relational evidence. It aggregates these judgments under a minimum-conflict objective into a taxonomy that reflects recurring author-written perspectives in the retrieved literature, guides generation, and bounds each section's evidence. Results on ten topics show gains in scope reconstruction, taxonomy quality, citation support, and reference reliability.

0.21

Idea Traces: Threading Accretive Content across Scientific Papers

cs.HC ·2026-10-28 ·Luke S. Snyder, Jeffrey Heer
similarity 0.64
Abstract · 15 pages, 9 figures, 1 table

We introduce idea traces, a novel thread-based representation of accretive content across scientific papers. Rather than capturing fixed themes from papers, idea traces surface similarities and deltas of faceted content that enable inspection of how ideas relate and evolve across the citation network. To extract traces, we contribute an LLM pipeline that deconstructs salient transitions between pairs of papers and collates the output into an idea dependency graph. We leverage idea traces in a visual system that supports idea-driven navigation starting from a seed paper, lifting the "plane" of synthesis from papers to ideas. To evaluate the usefulness and quality of generated ideas, we provide a case study that compares our approach with an LLM Q&A workflow, and interview 10 researchers who used our system and annotated 110 ideas. Experts found that our approach helped extract a useful space of connections that would have been difficult to make otherwise. Drawing on their feedback, we outline several promising directions for future research.

0.13

More of the same? Are scientific papers losing originality?

cs.DL ·2026-09-26 ·Ilan Doron-Arad, Elchanan Mossel
similarity 0.62
Abstract

Scientific output is growing rapidly, but it is unclear whether the expanding literature remains original or is increasingly repeating itself. Originality has many dimensions. One dimension can now be measured directly: how textually distinct a paper is from the work that came before it. Using semantic language models, we screened more than 26,000 full-text papers across eight fields in the CORE database (2015--2025) for passages resembling recent literature. We compared every paper against a fixed-size sample of earlier work, so increases in resemblance are not a by-product of literature growth. We find that the share of a paper's text resembling recent work was stable before 2021, then rose sharply, more than tripling by 2025. The increase came not from a few papers borrowing more heavily but from resemblance spreading across the literature: more papers now contain passages that echo recent work, and almost none of this growth is explained by cited work. We also found an analogous increase in passages expressing the same scientific idea. The pattern held across robustness checks and replicated in a second, independent database. Along this one measurable axis of originality, scientific writing has become less distinct from the work preceding it.

0.26

Citations Are Late: Reading epistemic instability from what papers believe, years before the citation graph catches up

cs.DL ·2026-10-05 ·Andrey Ustyuzhanin, Ekaterina Trofimova, Denis Zuenko
hot direction similarity 0.62
Abstract

Paradigm shifts in science are visible in what researchers assert and contest before they are visible in the citation graph. We ask whether a cheap, content-level signal of epistemic instability, derived from the changing distribution of stated modelling beliefs in paper abstracts, can anticipate a paradigm shift earlier than the dominant citation-based disruption index (CD5). On the displacement of recurrent networks by Transformers in NLP, a pre-registered content signal crosses its detection threshold in 2016-Q1, whereas a real-time CD5 monitor cannot even observe the 2017 breakthrough until 2022-Q2, since CD5 needs a five-year forward-citation window: a lead of about 25 quarters. The flat citation baseline is not an artifact of one index, as our CD5, a reference-normalised variant, and an authoritative precomputed index all sit near zero across the shift. The lead is also not a faster proxy: at equal latency the signal beats the content competitors tested, including a learned CD-from-text model and an embedding disruption measure, and the decomposition sees what a keyword cannot (keyword-blind AUC of about 0.9, reproduced on human labels). The lead over CD5 is an observability lead. Which signal carries it depends on the shift: belief adoption in NLP, contestation and applicability stress in vision. Measured against the breakthroughs themselves, the signal leads by five quarters in NLP and is contemporaneous in computer vision.

0.28

KnowMe Paper Radar: Lightweight Discovery of Promising Papers in the Age of Autoresearch

cs.AI cs.IR ·2026-11-18 ·Zhuo Chen, Jian Wang, Jinjie Gu, Lijun Wu
code/data similarity 0.61
Abstract · Ongoing work. 67 pages with 34 figures and 37 tables. Project website: https://hackerchenzhuo.github.io/knowme-research-ai/en/ GitHub: https…

The growing volume of topically similar papers makes it difficult to identify new work that can offer useful research insights or methodological guidance. We present KnowMe Paper Radar, a lightweight discovery tool that combines multiple sources of evidence into personalized reading priorities. Users specify topics, priorities, exclusions and dates; the system brings together topical relevance, recency, author research histories, collaboration paths and direction-specific institutional context. By exposing each score component and its supporting sources, KnowMe helps readers select promising papers for their current questions and discover relevant work beyond familiar authors. Automatic acquisition, version merging and graph-cache reuse keep this workflow convenient without large language model (LLM) inference during ordinary use. Experiments on 174,640 acquired records show more than threefold faster repeated context preparation, while cached abstracts improve normalized discounted cumulative gain at rank 10 (NDCG@10) from 0.412 to 0.598 on expert-judged retrieval tasks. Complementary analyses examine how the same accumulated records support semantic retrieval, research-history modeling and literature-grounded planning. A browser interface and a structured command-line interface (CLI) make these capabilities available to both researchers and research agents, positioning KnowMe as a practical upstream tool for literature-informed recursive self-improvement (RSI) workflows.

0.41