CS Paper Radar

Who Benefits from Wikipedia's Growth Mentorship? The Role of Newcomer Information Needs

Yubo Zhou, Pengyu Chen, Ceren Budak

cs.SI cs.HC ·submitted 2026-11-08 02:53 UTC · v2
published

Abstract

Evaluations of Wikipedia's programs for new editors usually ask whether a program works on average; we ask which newcomers benefit when their question to a mentor gets a reply. English Wikipedia's Growth mentorship system assigns every new account a volunteer mentor, and about a quarter of the questions newcomers send never get a reply. Using the first questions of 34,073 newcomers (2021 to 2026), we compare newcomers who received a reply with newcomers who did not, within groups that are similar on 164 characteristics recorded before the question (propensity score stratification). (1) Does a reply help newcomers keep editing? Yes, in the short term: in the 14 days after the reply, newcomers who received one were 1.8 percentage points (pp) more likely to edit an article (22% vs. an estimated 20% without a reply) and 1.8pp more likely to edit on two or more days. Later in the first year, the only difference that remains significant after correction for multiple testing is in the share who edit an article at least once between six months and one year after the reply (+1.5pp). (2) Does the effect depend on what the newcomer asked? Yes. We sort questions by Morrison's (1993) information-seeking types, which earlier studies measured mostly through self-reports, using a language model checked against human labels. A reply has significant effects for questions about rules and norms (+3.4pp), questions that show the newcomer's earlier editing (+3.4pp) and specific how-to questions (+2.4pp). For newcomers who do not know what to work on, a reply has no detectable effect, and the effect is 4.2pp smaller than for newcomers with other questions. (3) Which replies are followed by more editing? Among newcomers who received a reply, those whose reply came sooner or stayed closer to the topic of their question edited more.

Comment
25 pages, 10 figures. Accepted at ICWSM 2027

Closest papers

Nearest neighbours by meaning across the whole index.

The Evolution of Coordination in a Collective Intelligence System: 25 Years of English Wikipedia and the Emergence of Generative AI

cs.HC cs.AIcs.SI ·2026-09-15 ·Neal Reeves, Maja Świeczkowska, Amy Rechkemmer, Elena Simperl
similarity 0.53
Abstract · 10 figures, 16 pages

English Wikipedia is one of the largest examples of collective intelligence on the Web, sustained not only by article production but also by volunteer coordination and governance. While prior research has examined coordination work in Wikipedia, less attention has been paid to how participation in these spaces has evolved over time. Drawing on a longitudinal analysis spanning nearly 25 years of English Wikipedia, we examine editing patterns across five namespaces covering content, discussion, and governance. We find that participation in coordination spaces has declined relative to content production, particularly in governance areas, with a shrinking core of editors performing an increasing share of this work. Using Markov-based session metrics, we also find that editing has become more specialised, with editors moving less frequently between namespaces. Motivated by recent governance debates around generative AI, we conclude by investigating whether the availability of LLMs has altered these long-term trends. While short-term changes are visible, we find little evidence that generative AI fundamentally changed existing trajectories of coordination and participation.

0.23

When the Commons Appropriates a Large Language Model: How WikiVault Reshaped Korean Wikipedia

cs.HC ·2026-10-06 ·Inhwa Song, Sohyeon Hwang, Ted Yoo, Manoel Horta Ribeiro
similarity 0.52
Abstract

Large Language Models (LLMs) have disrupted the balance between content production and quality assurance that sustains knowledge commons, leading many to prohibit or restrict their use. But what happens when a community instead appropriates an LLM-powered tool for its own needs? We investigate this question through WikiVault, an LLM-powered editing tool developed within the Korean Wikipedia community and used primarily for translation. Combining ten interviews, platform-scale analyses, and matched quasi-experimental comparisons, we examine how WikiVault reshaped knowledge production on Korean Wikipedia. We find the tool 1) drastically amplified the production capacity of a small group of experienced editors, producing longer and more widely viewed articles; 2) shifted work toward reviewing articles and importing content; 3) imported not only content but also editorial judgments from English Wikipedia. Our findings show how LLM adoption can rebalance the interdependent work that sustains knowledge commons, while illustrating how communities can learn from emerging technologies through use and adapt their governance accordingly.

0.21

Collection-Date Assignment Redirects Answers in Temporal QA

cs.CL ·2026-10-23 ·Minsu Oh, Seunghui Jwa, Soo Kyun Kim, Yeo-Chan Yoon
similarity 0.49
Abstract

A collection date records when a source revision was acquired, not when its contents changed. Can assigning this metadata redirect temporal question answering despite unchanged evidence? Swapping the question day and following day between two revisions lowers later-reference exact match (EM) by 37.6-51.4 percentage points across five open-weight LLMs on 595 Wikipedia pages, replicated on 128 disjoint pages. Previous-day swaps retain 29.0-39.6-point losses on 497 pages. Seven prompt variants, including explicit ignore-collection instructions and archived-revision provenance, retain the effect. Fictional answers and English month-name question dates show that remembered answers and identical date strings are unnecessary. Matched thinking reduces Qwen3.5's penalty from 30.1 to 11.1 points, while native reasoners vary. Two API LLMs show small Wikipedia swap effects and rarely shift to earlier answers on fictional records. Large swap-induced losses concentrate around the question date; swaps between distant dates have much smaller effects. Adding fields containing one observed acquisition date lowers EM relative to omission in three of five LLMs after correction. The diagnostic isolates metadata effects from changing evidence or missing the relevant revision, and provides a practical audit for temporal QA inputs.

0.19

When Successful Knowledge Graph Edits Displace Correct Answers: Rank-Level Locality beyond Parameter Support

cs.AI ·2026-09-10 ·Yi-Cheng Lai, Jerry Wang, Hsin-Ling Hsu, Li-Chu Chi, Ya-Wen Teng +1 more
similarity 0.48
Abstract

Editing a knowledge graph embedding (KGE) model to promote a desired answer can displace correct answers from the returned list. Locality tests based only on facts that reuse the edited parameter can miss this ranking effect. We introduce a common rank-displacement audit at three scopes: facts supported by the edited parameter, other correct answers to the target query, and correct answers across queries with the same relation. We also derive dimensional and geometric conditions for an update to improve the target while exactly preserving selected scores. On FB15k-237 with DistMult and ComplEx, direct promotion always moves the target into the top ten, but does so without damage in only 23.0--23.2\% of edits. Strict preservation causes no measured damage, yet succeeds in only 1.3--1.4\%. Support-regularized entity editing gives the highest joint success, 36.3--37.7\%, while rank-truncated preservation reaches 32.8--34.7\% and reduces the mean number of displaced answers from about 14 to 1.2. Experiments across dimensions, scorers, ranking conventions, and a learned editor show that locality depends on both the protected scope and the editing mechanism. KGE editing should therefore report correction success together with the incidence and severity of rank displacement.

0.12

Return or Revise? Learning When Revision Helps Retrieval-Augmented QA

cs.CL cs.IRcs.LG ·2026-09-24 ·Nicholas Kashani Motlagh, Tim Anderson, Jeremy Gwinnup, Grant Erdmann
similarity 0.47
Abstract · 25 pages, 4 figures

We consider the decision of whether to return an existing draft answer or revise it using retrieved evidence, as in answer-revision systems. Draft confidence estimates whether the current answer is correct, but the decision requires estimating the effect of a specified revision. For offline training and evaluation, we grade both the returned draft and its candidate revision under the same correctness judge, which makes repair, harm, and the gap to an oracle observable. We call this paired effect its recoverability, and we train policies to predict it before revision. On 25,870 held-out open-domain questions across three revision setups, a scorer trained on the paired outcome has greater area under the accuracy--revision-rate curve than a matched draft-correctness scorer in all nine Llama setup--seed fits, and gains 0.23--0.68 accuracy points on average at development-selected thresholds, a difference significant across training runs only for dense retrieval. The resulting policy improves on always revising and on average closes more than a third of the oracle gap, although it still applies 38--46% of the harmful revisions. When a draft-free standard-RAG answer is also available, however, choosing between the draft and that answer is stronger by about two points for Llama and four for OLMo, and adding candidate revision as a third option yields no significant gain. Recoverability describes one revision; its value as an available action also depends on the alternatives.

0.49

Context or Knowledge: How LLMs Attribute Their Own Answers

cs.CL ·2026-10-23 ·Kirill Afendulev, Alexey Dontsov, Alla Chepurova, Elena Tutubalina, Anton Korznikov
similarity 0.46
Abstract · 25 pages, 7 figures, 23 tables, including appendices

Language models can answer a question from a supplied document or from their parametric knowledge. When later asked where the answer came from, what determines their source report? We study this by intervening on the model's key-value (KV) cache after the answer is generated and before the source question is asked. On a controlled factual task with four models (7B-32B), changing or removing the document's cache never changes the report. In contrast, transplanting the question-and-answer (Q+A) cache between runs with identical question and answer tokens transfers it: every eligible report follows the donor, while the recipient's document is unchanged. On the two smaller models, steering Q+A values along directions fitted before any source question reverses 98.9-100% of reports across four document conditions at a fixed strength, while norm-matched random directions flip at most 1.1%. The components of keys and values along fitted directions carry the transplant effect: copying them alone nearly reproduces the report change; holding them fixed nearly prevents it. Applied during answer generation, the same value directions also change which source supplies the answer, though more weakly and depending on the prompt. Source reporting in this setting thus depends causally on cached state produced while answering. We do not establish general reporting accuracy or a complete interpretation of this state.

0.20

Do Reviewers Still Reward Lexical Complexity? A Frozen-Rater Study of Preference Drift in 124K ICLR Reviews

cs.CL cs.DLcs.LG ·2026-09-08 ·Jiabin Zheng
code/data similarity 0.46
Abstract · 23 pages, 8 figures, 11 tables. Code and the machine-readable records behind every number: https://github.com/Biajin-PKU/frozen-rater-drift

Large language models have collapsed the cost of producing lexically elaborate prose, and whether peer reviewers still reward it is a question about the evaluator, not about the text. When the association between a writing cue and review scores moves across years, the reviewers may have changed, the submissions may have changed, or both, and a regression of scores on text cannot say which. We separate the two with a frozen rater: 81,850 machine reviews of ICLR submissions from 2018 to 2025, all generated in one February-April 2025 window with one model family and one prompt, so that its year-to-year coefficients track submission composition alone and the human-minus-frozen trend difference identifies reviewer preference drift. On 32,638 submissions with 124,615 human reviews, the human coefficient on non-domain lexical complexity falls from +0.142 to -0.015 while the frozen rater moves from +0.080 to +0.082; the three-way difference-in-differences is -0.0100 (q=0.013), and forty random-wordlist placebos through the same specification centre on zero. Humans still reward sentence-length variability, which the frozen rater never registers, while the frozen rater still pays for lexical complexity at its earlier rate. Every claim is held to a double gate of false-discovery control and interval exclusion, and the findings that failed adversarial re-testing are reported. Reviewers discounted a cue whose production cost collapsed, as models of manipulable signals prescribe; an LLM judge calibrated to historical human preferences inherits the earlier schedule and drifts out of alignment while its agreement with humans on totals stays ordinary.

0.43

Where Language-Model Vocabulary Rose on Stack Exchange

cs.CL cs.CYcs.SI ·2026-11-11 ·Fengming Liu
similarity 0.46
Abstract · 8 pages, 4 figures, 3 tables (one in the appendix)

Stack Overflow banned answers generated by ChatGPT five days after the model's release, and many other Stack Exchange sites announced similar rules. I measure the vocabulary of the answers posted afterwards, 3.8 million answers on 170 sites from 2019 to 2024. A fixed list of 56 words that language models overuse rose about 14 percent above its pre-2023 trend, while the highest of 1,000 frequency-matched lists of ordinary words rose about 7 percent. In a screen of 15 site traits measured before the release, the strongest predictor of where the rise happened was the share of answers that received a body edit within 30 days of posting. On the least edited third of sites the list rose about 29 percent relative to matched controls, and on the most edited third it did not change. The gradient is also present among answers that were never edited. It follows the share of answers edited by their own authors, and the share edited by others adds little once that share is held fixed. Use of the model is unobserved, so these results cannot separate lower use on sites with a high self-edit share from any effect of editing on what the vocabulary records. Any vocabulary-based estimate for the network as a whole therefore pools sites where the marker vocabulary rose sharply with sites where it did not change.

0.28