CS Paper Radar

Rethinking Inline Citation Verification in Scholarly Communication

Xinrui Fang, Reese Fairchild, Nasi Wang, Anran Xu, Simo Hosio, Sylvain Malacria, Koji Yatani

cs.HC ·submitted 2026-10-04 16:23 UTC · v1
survey

Abstract

Inline citations are central to scholarly communication, yet verifying their use is becoming increasingly challenging because of growing review pressures. We conducted a mixed-methods exploratory study to investigate how reviewers verify inline citations, the factors shaping their verification practices, and how they envision GenAI supporting this process. Across interviews (n=12) and a survey (n=203) of reviewers from HCI and AI venues, we found that reviewers' perceptions of citation importance, verification practices, and desired AI autonomy varied across citation types, reviewer characteristics, and research backgrounds. These findings highlight the need for adaptive support tailored to different citation types and reviewer practices, while revealing diverse preferences regarding the use of GenAI for this process. Moreover, effective citation verification should involve collaboration among reviewers, authors, and the broader research community, rather than relying solely on individual reviewers.

Closest papers

Nearest neighbours by meaning across the whole index.

Best practices in software citation

cs.DL astro-ph.IM ·2026-09-30 ·Phil R. Van-Lane, Floor S. Broekgaarden, Daniel S. Katz, Bhavesh Patel, Pengyin Shan +9 more
similarity 0.66
Abstract · 17 pages, 1 figure (including appendices)

Software is both a foundational tool and a primary output of modern computational research, yet citation practices for software remain inconsistent, incomplete, and rarely machine-actionable. Existing infrastructure designed for paper and data citation does not adequately serve the distinct needs of software citation, leaving a gap that impedes reproducibility, misattributes scholarly credit, and obscures the labor embedded in research pipelines. Drawing on a NASA-funded community workshop held in April 2026, we present an analysis of four interconnected themes: (I)~the cultural barriers to consistent citation practice; (II)~the need for clearer community norms and conventions; (III)~gaps in existing technical infrastructure and workflow; and (IV)~the emerging challenges posed by AI-assisted research. For each theme we identify targeted interventions and assign responsibility across stakeholder groups. We conclude that meaningful progress requires simultaneous action on technical and cultural fronts. Journal editors and publishers represent the single highest-leverage point for accelerating this change, and correct citation must become the path of least resistance within researchers' existing workflows.

0.13

Citing Less Critically: LLMs Reshape the Rhetoric and Reach of Scientific Citation

cs.DL cs.CLcs.CYcs.SI ·2026-09-01 ·Yixuan Liu, Lin Chen, Zhuoqi Liu, Jianglin Lu, Dakota Murray
top venue similarity 0.65
Abstract · Accepted at the EMNLP 2026 main conference

Scientific citations carry rhetorical intent. Scholars may cite prior work positively (supporting), negatively (contrasting), or neutrally (mentioning). As large language models (LLMs) increasingly assist scientific writing, whether they reproduce citations with the same rhetorical intent as humans remains unclear. We introduce a masked-citation task to compare human and LLM-generated citation behavior. For each citation context, an LLM generates a replacement citation sentence, producing a counterfactual corpus directly comparable to human citation. We analyze what, whom, and how models cite, using an LLM-as-a-judge to classify citation intent and a 20-million-edge coauthorship network to measure social distance between cited authors. Across six popular LLMs and 1,746 top NLP conference papers (63k+ contexts, 132k+ citations), three patterns emerge: (1) Compared with human citation, LLMs cite significantly less critically; (2) LLMs over-cite popular and older papers, a tendency amplified for contrasting citations where human writing more often draws on recent, niche work; (3) Whereas humans often cite within their close social network, especially for supporting citations, LLMs tend to draw on more socially distant authors. Together, these differences are double-edged: LLM citation reaches beyond a scholar's close collaborators while being less critical and amplifying visibility bias, reshaping the rhetoric and reach of scientific citation.

0.34

CITECHOICE: A Causal Audit of How Document Presentation Redistributes Citation Credit in Agentic Search

cs.CL ·2026-09-14 ·Sriram Selvam, Anneswa Ghosh
similarity 0.64
Abstract

When several retrieved sources support the same claim, an answer engine cites some but not others. We call this decision citation allocation and introduce CITECHOICE, a causal audit of authentic multi-turn agentic search. From 129 everyday-query transcripts, CITECHOICE selects 113 same-call document pairs with independently verified support for the same pre-specified fact, without observing ranks or answer outcomes; blinded human review confirms 103. It runs a hash-verified 2-by-2 replay crossing pair order with jointly generated, fidelity-checked structured and prose renderings of one target while the rest of the transcript remains fixed. Three results emerge. First, and most importantly, structured rendering concentrates citation credit rather than clearly increasing source admission. It raises target citation count by +0.50 citations per answer (95 percent CI [+0.20, +0.84]; Holm-adjusted p=.033), without increasing total citations or reducing competitor credit. The pre-specified incidence effect (whether the target is cited at all) is +4.5 percentage points and inconclusive (95 percent CI [-1.4, +10.4]; p=.168). Second, observational position differences exceed controlled reordering effects: the citation-rate gap between rank 1 and rank 5 is 42.3 percentage points, compared with +7.9 percentage points in the main replay and 0.0 percentage points held out. Third, citation evaluation has a measurable noise floor. Although the aggregate count effect repeats under fresh decoding of 30 frozen families, 15 percent of binary decisions change and decoding accounts for an estimated 45 percent of single-generation family-effect variance. Together, these findings isolate what survives control: presentation can causally redistribute visible citation credit within frozen transcripts. They do not establish reliable source admission, a pure formatting mechanism, or a general rank advantage.

0.18

ATIBA: Grounded Integrity and Quality Checking for Research Papers

cs.SE ·2026-09-03 ·Veli Karakaya, Semih Çağlar, Yusuf Yiğit Korkmaz, Eray Tüzün
published similarity 0.63
Abstract · Accepted at The First International Workshop on Automated Techniques for Integrity and Quality in Software-Engineering Research (ATIQSER 202…

Checking a manuscript's reference integrity, its compliance with a target venue's specific submission rules, and its adherence to community reporting standards is manual, repetitive, and different for every venue so in practice it is done inconsistently or skipped. We present ATIBA, a tool that runs five grounded integrity and quality checks on a manuscript: a reference-integrity check that verifies each citation against bibliographic sources and flags retracted or unfindable references; a venue/track compliance check that derives submission criteria directly from a venue's own call-for-papers page and evaluates the manuscript against them, each verdict anchored to a verbatim quote from that page; an empirical-standards compliance check against the ACM SIGSOFT Empirical Standards, with a hallucination defence that discards any evidence quote it cannot locate verbatim in the manuscript; a multi-mode AI review (venue-specific, formal, and page-anchored annotation) powered by GPT-5.4 through Azure OpenAI; and a citation-suggestion feature that proposes candidate references for a manuscript and verifies each against bibliographic sources before it is shown to the user. All five checks are designed around the same principle: an LLM is only trusted to judge, never to invent the evidence it judges against. We evaluated ATIBA through a moderated user study with 13 non-author participants. Agreement across the six survey items ranged from 69% to 92%, with a mean of 85%, providing initial evidence of positive perceived usefulness across the evaluated workflows. These findings establish perceived usefulness; objective accuracy remains to be measured.

0.40

What Drives Citations in Production Large Language Models? An Observational Multi-Method Study of Two Million AI Citations Across Ten Thousand Web Pages

cs.AI ·2026-09-28 ·Ben Moore, Liam Dunne
similarity 0.62
Abstract

Production large language models retrieve and cite web pages alongside generated answers, yet the page-level features that predict citation frequency remain poorly characterised. We present an observational study of approximately 2 million LLM citations from four commercial engines (ChatGPT, Claude, Google AI, Gemini) over six months, joined to 10,000 crawled pages from nineteen B2B SaaS workspaces. Sixty-plus features are tested using a nine-method consensus framework combining mixed-effects regression with domain fixed effects, FDR correction, stability-selection Lasso, double machine learning, generalised additive models, and temporal hold-out replication. Four findings survive all checks. First, prompt-content alignment (Jaccard overlap between page tokens and the full workspace prompt corpus, including non-citing prompts) is the dominant page-level predictor (beta = +0.37, 95% CI [+0.33, +0.41], q ~ 10^-73). Second, the standard AEO checklist (FAQ blocks, structured data, Core Web Vitals) shows positive effects in pooled data that reverse or collapse to zero once domain fixed effects are applied: Simpson's paradox with practical consequences for the AEO literature. Third, domain-level AI authority exceeds the strongest non-alignment page-level feature by a factor of six in mean absolute SHAP value. We release the analytic pipeline as a methodological contribution.

0.10

Debiasing as a Measurement Intervention: Calibrated Ties and Resolution Loss in LLM-as-a-Judge Evaluation

cs.DL cs.AI ·2026-09-11 ·Liang Zhao, Yong Wang, Jiangzhe Chen
top venuebenchmark similarity 0.62
Abstract · Accepted at Emnlp 2026

LLM-as-a-judge protocols are commonly debiased by instructing judges to ignore presentation cues such as citation formatting, source labels, and evidence-display style. We show that this intervention can suppress bias while damaging the resolution of the measurement instrument. We introduce TraceJudgeBench, a diagnostic benchmark for auditing citation-like artifacts in RAG and agent-workflow evaluation, covering content-equivalent pairs, citation ablations, correctness conflicts, human-validated soft and moderate quality gaps, prompt-strength ladders, decoupled judging, and a controlled workflow-ranking probe. Across GPT-5.5, Claude Sonnet 4.6, and DeepSeek V4-Flash, stronger anti-citation prompts reduce worse-cited wins from up to 50.5% to 0%; yet some operating points already convert validated moderate-gap decisions into Tie before the strict stress-test endpoint, while correctness-conflict accuracy remains at or above 93.0%. A second, 50-pair FinQA moderate-gap construction reproduces the qualitative frontier, and open-weight Qwen2.5-14B-Instruct-AWQ and Gemma-3-12B-IT runs reproduce the central HotpotQA frontier. TRACE-style decoupling recovers 96.5-100.0% better-plain resolution across the reported settings. Human validation separates three meanings of Tie: correct equivalence Tie, calibrated soft-boundary Tie, and resolution-destroying Tie on validated quality gaps. We frame debiasing as a measurement intervention whose bias suppression, resolution retention, Tie cost, and protocol cost must be reported jointly. The supplementary artifact contains benchmark splits, prompts, raw judge outputs, validation summaries, and analysis.

0.37

From Public Posts to AI-Search Citations: Measuring the Fragility of AI Search

cs.CR ·2026-10-08 ·Qi Liu, Geng Hong, Xinyang Zhang, Pei Chen, Yutong Li +1 more
similarity 0.61
Abstract

As more users ask AI systems for information, AI-search platforms are becoming a common gateway to web information. Unlike traditional search, which maps keywords to ranked pages, AI search retrieves pages, filters sources, selects citations, and generates answers before users see sources. This selection layer may amplify source bias and turn source choice into a security question. If a platform repeatedly cites domains where new users can publish posts easily, ordinary publication on those domains can become an indirect path into AI-search citations and answer text. Measuring this path is hard: platforms reveal little about citation selection, citations change over time, and the web contains so much background content that later answer changes are hard to attribute to our posts. We present a measurement framework for identifying and measuring this low-barrier publication path, combining cross-platform citation mapping, publication-barrier testing, and marker-controlled publication experiments. Across 10 AI-search platforms, we analyze 17,211 citation instances over 6,356 unique source domains and find: (1) citations concentrate in platform-specific sources, with top-20 domains capturing 20.5--70.8% of per-platform citations, and 15 of 22 tested publication platforms tied to cited source domains had low or medium barriers for both account setup and posting; (2) in our experiments, ordinary publication on preferred platforms changed what entered AI-search outputs: 8 of 10 platforms cited a fabricated concept within seven days, and one high-preference-platform article had greater citation impact than over 20 matched low-preference posts; and (3) this path is commercially available: a $14 GEO purchase produced 13 public posts, and one AI-search platform cited GEO-posted content with our designed markers within one hour.

0.20

Who is the Agent to Blame? Localizing Faithfulness and Citation Mistakes in Agentic Deep Research

cs.CL ·2026-08-25 ·Eran Hirsch, David Wan, Han Wang, Elias Stengel-Eskin, Mohit Bansal +1 more
top venuecode/data similarity 0.61
Abstract · Accepted to EMNLP 2026 (Main Conference). Code: https://github.com/eranhirs/who-is-the-agent-to-blame

Deep research (DR) systems produce long-form cited reports by orchestrating multiple agents that search and synthesize information from the web. Citations are the primary mechanism for evaluating the faithfulness of these reports, yet current DR systems exhibit poor citation recall. Moreover, improving citation recall is challenging because DR systems are complex multi-agent architectures where information passes through agents like a telephone game, and both content and citations can get corrupted along the way. We propose an evaluation method that pinpoints which agent introduced each error by locally testing agent invocations for faithfulness and verifiability relative to their own inputs. Furthermore, we propose a four-type taxonomy to categorize the discovered errors: hallucination, uncited input reliance, uncited output, or insufficient citations. Applying our method to three top-ranked open-source DR systems, we obtain actionable diagnostics. Almost every agent makes a lot of mistakes with the exception being those that summarize a single document. We find that the dominant error type varies systematically across agents, where the orchestrator mistakes are mostly citation-related. We find that 84.7% of final-report errors in AI-Q originate at the orchestrator, roughly 31% of them hallucinations and the rest citation mistakes. Guided by these insights, we demonstrate that two simple interventions raise citation recall by 5% without degrading output quality.

0.64