CS Paper Radar

Earlier citation practices are linked to signs of artificial intelligence use in scientific writing

Hongkan Chen, Yongyuan He, Qingshan Zhou, Yi Bu

cs.DL ·submitted 2026-11-12 14:06 UTC · v1

Abstract

As artificial intelligence (AI) becomes more common in scientific writing, some authors show fewer signs of AI assistance than expected across successive periods. We analysed OpenAlex publication histories for 71,133 authors with at least one eligible first-authored English-language publication each year from 2023 through September 2026. Using abstract-based writing scores, we defined resistance as the predicted minus observed share of abstracts classified as AI-assisted. Positive scores indicate fewer such abstracts than expected. Early resistance was positively associated with later resistance, although the share with positive scores fell. Environmental characteristics contributed more than individual characteristics to predicting later scores, although overall predictive power was low. Having a collaborator with a non-positive early score was associated with lower subsequent resistance. Authors with positive mean scores in both 2023-2024 and 2025-2026 cited older work, drew on fewer subfields and more often cited sources they had used before than authors with non-positive mean scores in both periods. These differences were evident before 2023. The adjusted gap in reference reuse rose from 3.24 percentage points before 2023 to 7.95 in 2025-2026. The findings link variation in signs of AI assistance to earlier differences in how scientists used prior work.

Closest papers

Nearest neighbours by meaning across the whole index.

Accelerated Drift of Scientific Prose in the LLM Era: Evidence from an Interrupted Time-Series Analysis

cs.CL cs.AI ·2026-10-19 ·Hanna Hubarava, Benjamin V. Ineichen
top venue similarity 0.63
Abstract · Accepted at EMNLP 2026. 22 pages, including references and appendices

Every scientific text is embedded in the writing norms of its time: it reflects earlier publications while also shaping the texts that follow. The growing use of LLM-based writing tools introduces a new factor in this evolving system, but the linguistic changes coinciding with their emergence remain insufficiently understood. We investigate whether the trajectories of scientific-preprint writing changed in rate or direction after conversational LLMs became publicly accessible, using the release of ChatGPT in November 2022 as the observational breakpoint. We compare pre- and post-2022 trajectories across vocabulary, punctuation, readability, and syntax in abstracts from arXiv's artificial intelligence, statistics, and quantitative biology categories and from medRxiv. Using interrupted time-series regression, we find substantial post-2022 changes across multiple feature families and research domains, including increases in average word length, reduced formula-based readability, and declines in maximum dependency-tree depth and sentence-level depth variation. Individual features follow different dynamics: some lexical markers accelerate persistently, while others rise and then recede. Our findings show that isolated marker frequencies capture only part of the shift: changes in the rate, direction, and structure of linguistic trajectories provide complementary evidence of broader post-2022 change in scientific-preprint writing. More broadly, these findings motivate monitoring scientific writing as a multidimensional, evolving system rather than through a fixed set of LLM-associated markers.

0.47

Style as a Confound: False Positives in AI Detection of Non-Native Academic Writing

cs.AI ·2026-08-27 ·Hyeonchu Park, Gahye Jeong, Bugeun Kim
top venue similarity 0.63
Abstract · Presented in EMNLP 2026

AI text detectors are increasingly employed in academic settings, but it remains unclear whether their outputs reflect AI authorship itself or broader linguistic features associated with polished academic English. Previous studies have reported high false-positive rates (FPRs) for non-native English writing, but population-level comparisons confound authorship with differences in topic, domain, and writing style. Professional editing provides a useful setting for examining this issue because it changes the linguistic form of manuscripts while preserving authorship and content. We examined 135,389 document pairs from a professional English editing service (2018-2025), comprising non-native manuscripts and their native-edited versions, to assess how editing affects detector responses controlling for content and authorship. For the 13 AI text detectors, FPRs for human-written texts varied widely, from 0.0% to 100.0%. Responses varied across detectors: the same edits increased AI scores in some detectors but decreased them in others. Notably, score changes correlated with the extent of editing. The findings identify professional editing style as a key confounding variable in AI detector outputs, rather than establishing a full separation of text origin from linguistic style, raising concerns about fairness and reliability in academic settings.

0.63

What Changed in Engineering Writing After Generative AI Chat Assistants: Evidence from 1.8 Million Articles

cs.DL cs.CL ·2026-11-08 ·Taner Cokyasar
code/data similarity 0.60
Abstract · 26 pages, 5 figures, 7 tables. Code and data: https://doi.org/10.5281/zenodo.24028663

This study asks what changed in engineering articles after conversational language models became publicly available on November 30, 2022, and develops methods for measuring such change in a field without a uniform abstract index. It assembled 1,839,962 articles published from 2015 to 2025 in 256 engineering journals, tracked twelve measures of wording, style, and article structure, and compared each with the projection of its own pre-2023 trend. Building the monthly series showed that records dated only by year fall on January 1 and create false January peaks unless removed. In abstracts, 31 words associated with language model output appeared 82% more often than projected (55.4 against 27.0 per 10,000 words in 2025), while four control words of the same formal register fell 16%. Words that soften a claim, such as may, fell 13%, words that strengthen one, such as clearly, rose 20%, and long words rose 11%. Author counts, reference lists, page counts, and publication volume stayed within 4% of trend. The change appears in all fourteen subject categories, and false release dates, dropping any one journal or word, and word lists chosen from the data support it. A random sample of 12,866 full texts from the same corpus, including subscription articles and weighted back to it, shows the same words rising 99% in article bodies. Statements disclosing AI writing assistance, measured apart from mentions of language models, went from none to 3.7% in 2025, four in five of them at one publisher. The design is descriptive, not causal.

0.34

Does AI Assistance Leave a Temporal Fingerprint? Detecting Overreliance in AI-Assisted Writing and Programming

cs.HC cs.AI ·2026-09-15 ·Eduardo Davalos, Yike Zhang
similarity 0.60
Abstract · 10 pages, 4 figures, 2 tables, submitted to AHFE 2026 Hawaii Edition

The rapid adoption of generative AI has made final artifacts unreliable evidence of student learning, and AI detectors that examine only the finished product are inaccurate and ethically contentious. Process data offers an alternative, but prior work covers only English essay writing. We ask whether AI assistance carries a temporal signature, whether it generalizes from writing to programming, and whether it distinguishes ordinary collaboration from wholesale delegation. We analyze three public corpora: CoAuthor (1,447 keystroke-level co-writing sessions), RealHumanEval (editor telemetry from 243 programmer records), and a pre-LLM CS1 corpus (5.1 million keystrokes) as a human-only baseline, comparing minimal-AI work, collaborative AI use, and simulated wholesale delegation. Three findings emerge. First, the signature generalizes: AI contributions arrive in bursts far outside the author's own baseline in both mediums (paired d_z = 1.13 and 3.54). Second, engagement diverges by medium: 93% of AI-inserted characters survived to writers' final documents, while only 14% of accepted code suggestions survived intact. Third, classifiers using only observable temporal features separate simulated delegation from authentic work nearly perfectly (F1 $\geq$ 0.997; at most 0.5% of real work misclassified), while ordinary collaboration remains hard to distinguish from unassisted work. Temporal evidence flags wholesale delegation rather than assistance, positioning process visibility as a candidate evidentiary basis for academic integrity, pending validation in authentic coursework.

0.15

Who Can You Hide Behind? Author-Conditioned Style Transfer and the Evasion of a Commercial AI-Text Detector

cs.CL ·2026-11-15 ·Mohd Ruhul Ameen, M. Saifuzzaman Rafat, Akif Islam
similarity 0.59
Abstract · 6 pages, 5 figures, 4 tables, Preprint

AI-text detectors can be sensitive to changes in writing style. We examined this sensitivity by rewriting 991 AI-generated answers with Gemini, using prompts that included a writing sample from one of 55 named authors. We scored each original and its rewrite with the QuillBot AI detector. The median score fell from 100% to 0%, and the mean fell from 67.9% to 11.2%. Of the rewrites, 66% received a score of exactly 0%. Among the 699 originals scored at least 50%, 71% of rewrites scored below 10% (paired Wilcoxon signed-rank test, p-value below 1e-100). Scores varied across the authors used in the prompts. All rewrites associated with 10 authors scored 0%, whereas the mean score remained above 50% for six authors (Kruskal-Wallis test, p-value below 1e-80). Greater reuse of five-word sequences from the author sample was associated with lower scores, although rewrites with less than 5% reuse still averaged 15.2%. These results describe one detector and one rewriting system, not detectors in general. The study lacks a generic-rewrite baseline, did not measure preservation of meaning, and produced rewrites that were substantially longer than the originals.

0.19

IdeaTaste: Do LLMs Share the Research Taste of Human Researchers?

cs.CL ·2026-11-02 ·Shanshan Zhong, Chen Xu, Jingjie Ning, Zhengzhong Liu, Aviral Kumar +1 more
benchmarkcommunity pick similarity 0.59
Abstract

Research taste, the judgment of what to read, which problems to pursue, and which methods to try, increasingly shapes the work of AI scientists, yet it remains far less understood than their ability to execute research. We introduce IdeaTaste, a benchmark that scores an LLM by how closely the research proposals it writes from a literature snapshot frozen at 2025 match the work the same communities published in 2026. Our experiments span six contemporary language models and four research areas from established to emerging fields, covering more than 90{,}000 proposals in total. LLM agents fall well short of human communities in both the literature they cite and the ideas they state, reaching about 26\% of the human level on citation matching and only 18\% on idea matching. Their populations can cite as diversely as a community and take up its tasks and methods, yet rarely arrive at the concrete approaches and research plans it pursues. Neither richer field context nor larger populations substantially close this gap. In blinded expert review, AI proposals trail the 2026 papers least in presentation and most in expected success, echoing their gap in concrete approaches, and those whose ideas no human paper shares fare worst, suggesting weaker directions rather than overlooked opportunities. These results indicate that the main bottleneck in AI research taste lies less in finding the directions a community works on than in turning them into the concrete approaches it goes on to pursue. We release IdeaTaste as an open, continuously updated leaderboard (https://cxcscmu.github.io/IdeaTaste) that benchmarks contemporary AI proposals against future publication cycles.

0.28

Editorial routing shapes how computational results are qualified in AI-assisted scientific writing

cs.CL ·2026-09-13 ·Jihan Kim
similarity 0.58
Abstract

Large language models increasingly analyze computational results and draft manuscripts, making reliable communication as important as correct analysis. Using fixed computational evidence, we tested whether assigning comparisons across modeling choices elsewhere in a research workflow changes manuscript reporting. In constrained sentence-writing tasks, Anthropic's Claude Sonnet 5 often omitted numerical qualifications when detailed comparisons were assigned to a group repository, but retained them more often when the same comparison was assigned to Supporting Information or its own working notes; Claude Opus 5 was less sensitive. These effects did not follow a simple accessibility ordering. A targeted placement rule largely restored sentence-level qualification, whereas a generic accuracy reminder did not. Longer contributions retained numerical qualifications, although some summaries across computational settings were still redirected to the repository. Thus, documenting context within an AI workflow does not ensure its communication where readers encounter the result.

0.24

Interpreting Paper-to-Patent Citations in AI-Related PMC Articles

cs.DL ·2026-10-13 ·Haowei Li, Yi Bu
similarity 0.58
Abstract · 28 pages, 3 figures, 6 tables

Paper-to-patent citations connect scientific and technological documents, but their interpretability depends on how patents are used in the citing text. This study examines citation contexts and paper-level outcome associations in AI-related articles in PubMed Central (PMC). Screening 189,073 articles identified 746 with explicitly tagged patent references (0.395%), including 713 with 1,544 identifiable in-text patent-citation instances. Three large language models classified these instances by represented content, motivation, textual position, and evaluative stance. Regression models examined associations among these dimensions, and propensity-score-matched regressions compared patent-citing papers with XML-screened controls. Background-oriented citations were predominantly neutral and introductory, whereas adoption-oriented citations were more closely associated with methodological positions and supportive evaluations. These patterns suggest heuristic configurations of contextualization, operationalization, and problematization, although classification overlap and exclusion of rare categories limit their interpretation. In matched samples, the patent-citing indicator was associated with lower scholarly-citation measures and more citations received from patents. These associations are observational, not causal. Together, these findings show that paper-to-patent citations encode heterogeneous forms of engagement with patented technology and should be interpreted as selective bibliographic signals rather than direct indicators of technology-to-science knowledge transfer or research value.

0.23