CS Paper Radar

An LLM-Associated Register Shift in Korean Journal Abstracts: A Morphology-Aware Excess-Vocabulary Study, 2018-2026

Aron Lee

cs.CL cs.DL ·submitted 2026-09-07 12:56 UTC · v1
theory

Abstract

Excess vocabulary, a word's frequency above its pre-2023 trend, is how the change in scholarly English after 2022 has been measured. We adapt it to Korean with morphological units on 398,296 KCI abstracts (2018-August 2026), with 47,165 Vietnamese abstracts for comparison. Placebo floors are 0.1-2.2 points for the single-word statistic and at most 2.9 for the re-selected split-half set statistic. Korean abstracts show nothing in 2023, onset in late 2024, a rise through 2025 flattening in mid-2026: sisahada "suggest" appears in 21.4% of 2026 abstracts against 5.3% expected; plain verbs like araboda "look into" fall to a quarter of trend. Under stated assumptions the single-word conditional lower bound on LLM-processed abstracts is 3.5%, 10.5% and 16.1% for 2024-2026 and a split-half set bound 7.8%, 20.6% and 33.0%. Holzwarth et al.'s estimator under the same discipline gives 41.9% and 72.1% for 2025-2026. Subject-matter controls reduce but do not remove it: restricting the set to lemmas three language-model annotators all call style leaves 14.7 of the 33.0 points, and pairing each 2026 abstract with its journal's closest base-period abstract leaves 34.1. Tested translation routes do not explain it: the surface marks of translated Korean fall as the markers rise. In the same articles' English abstracts the excess appears a year earlier; where the English side carries none, the Korean shift persists at 30 to 66% of the rate where it does. Control abstracts from three providers reproduce the rising words, with marker turnover consistent with model generations; implied prevalences are scenario-dependent.

Comment
30 pages, 8 figures, 31 tables. Working paper. Data, code and this version are deposited at Zenodo: doi:10.5281/zenodo.22303588

Closest papers

Nearest neighbours by meaning across the whole index.

KoNeoBench: A Curated Evaluation Dataset for LLM Understanding of Korean Neologisms

cs.CL cs.AI ·2026-09-17 ·Soha Lee, Soojin Lee, Heesung Yang, Hyunju Song, Hyunji Lee +6 more
top venuebenchmarkcode/data similarity 0.64
Abstract · Accepted to Findings of EMNLP 2026. Code and data are available at the project repository

Large language models (LLMs) are typically evaluated on static benchmarks, even though natural language constantly evolves through newly emerging words and meanings. Existing Korean benchmarks are centered on established vocabulary and therefore provide limited coverage of such recent lexical change, and their English-oriented design makes it difficult to assess the typological properties of Korean, in which content words combine productively with functional morphemes. In this paper, we introduce KoNeoBench, a benchmark for evaluating LLMs' understanding of Korean neologisms. KoNeoBench is built on 1,785 Korean neologisms attested in online news since 2020 and curated through expert lexicographic review. Each entry provides usage examples, word-formation analyses, and dictionary-style definitions. Based on this resource, we define four tasks and report results on recent models, together with a human baseline. Our experiments show that current LLMs exhibit clear limitations in recovering source components, distinguishing semantic categories, and generating accurate definitions. These results reveal specific aspects of recent Korean lexical change that remain challenging for current LLMs. KoNeoBench is available at https://github.com/bcmilab/ko-neobench/ .

0.55

How Did Writing Change At CHI? Analyzing 44 Years of CHI Writing Before and After the Introduction of Large Language Models

cs.HC ·2026-09-19 ·Thomas Kosch, Robin Welsch, Michael Hedderich, Christopher Katins
similarity 0.64
Abstract

The availability of Large Language Models (LLMs) reshaped scientific discourse at a linguistic level. LLMs are assumed to homogenize academic writing, flattening it into a single generic lexical register. To understand how CHI writing has changed since the public release of LLMs, we analyzed full texts of 14,262 archival papers across all 44 CHI proceedings from 1982 to 2026, measuring readability, register, lexical diversity, and marker words typically produced by LLMs. We find that prose did not homogenize, while vocabulary grew more varied, and sentence rhythm remained irregular. CHI prose changed more between 2016 and 2026 than in other decades toward greater density, and reading ease has declined since 2022. The word-level shift began before any author used LLMs, so LLMs did not start the change but accelerated it. Reflecting on the history of CHI papers, we discuss what may have caused changes in prose and how LLMs accelerated them.

0.16

Accelerated Drift of Scientific Prose in the LLM Era: Evidence from an Interrupted Time-Series Analysis

cs.CL cs.AI ·2026-10-19 ·Hanna Hubarava, Benjamin V. Ineichen
top venue similarity 0.64
Abstract · Accepted at EMNLP 2026. 22 pages, including references and appendices

Every scientific text is embedded in the writing norms of its time: it reflects earlier publications while also shaping the texts that follow. The growing use of LLM-based writing tools introduces a new factor in this evolving system, but the linguistic changes coinciding with their emergence remain insufficiently understood. We investigate whether the trajectories of scientific-preprint writing changed in rate or direction after conversational LLMs became publicly accessible, using the release of ChatGPT in November 2022 as the observational breakpoint. We compare pre- and post-2022 trajectories across vocabulary, punctuation, readability, and syntax in abstracts from arXiv's artificial intelligence, statistics, and quantitative biology categories and from medRxiv. Using interrupted time-series regression, we find substantial post-2022 changes across multiple feature families and research domains, including increases in average word length, reduced formula-based readability, and declines in maximum dependency-tree depth and sentence-level depth variation. Individual features follow different dynamics: some lexical markers accelerate persistently, while others rise and then recede. Our findings show that isolated marker frequencies capture only part of the shift: changes in the rate, direction, and structure of linguistic trajectories provide complementary evidence of broader post-2022 change in scientific-preprint writing. More broadly, these findings motivate monitoring scientific writing as a multidimensional, evolving system rather than through a fixed set of LLM-associated markers.

0.47

Most biomedical publications show signs of LLM-assisted writing

cs.CL cs.AIcs.CYcs.DL+1 ·2026-08-11 ·Lena Holzwarth, Rita González-Márquez, Dmitry Kobak
similarity 0.61
Abstract

Over the past several years, LLM-powered chatbots and agents have become widely used as a tool for academic writing. LLM-assisted writing can be valuable by removing language barriers but at the same time causes concerns about misconduct and fraud. To inform policy decisions, it is necessary to monitor the prevalence of LLM-altered texts in scholarly publications. Despite some recent progress in this direction, no existing method can produce reliable estimates. Here we suggest and validate a new unbiased approach to estimate LLM usage in a corpus of texts based on changing word frequencies. We apply our method to the full texts of open-access biomedical papers from Pubmed Central, and show that by the end of 2025, 89% of papers show excess of LLM-associated vocabulary. We also find that LLMs are twice as likely to be used when writing a paragraph in the Discussion section (68%) compared to a paragraph in the Methods section (32%), but even inside the Methods section, the overall prevalence of LLM usage is over 50%. We believe that our estimates are crucial to shape future guidelines and policies.

0.41

KoAGE: Do Large Language Models Know, Learn, and Coin Korean Neologisms?

cs.CL cs.AI ·2026-10-14 ·SungHo Kim, Eojin Jeon, SangKeun Lee
benchmark similarity 0.61
Abstract

Korean neologisms arise through diverse mechanisms, a demanding test of language models beyond lexical memorization. To enable systematic evaluation, we introduce NeoTrace, an automated, rerunnable framework that collects and dates neologisms across genres. Its 4,271 entries support KoAGE, a cutoff-aware benchmark with three acquisition and two generation tasks---the latter, to our knowledge, the first to compare model coinages with attested neologisms under their original materials and community context. Across 21 models, lexical recall diverges from contextual inference: in a five-model temporal analysis, models define only 13-30% of cutoff-year words, yet can infer meaning from one usage. However, structural understanding remains limited: even correct definitions barely improve formation analysis of pre-cutoff neologisms. Given word-building materials, single models often follow human formation mechanisms, whereas multi-agent coinage favors abbreviation and depends on interaction framing. KoAGE thus separates what models know, learn from use, and coin.

0.07

What Changed in Engineering Writing After Generative AI Chat Assistants: Evidence from 1.8 Million Articles

cs.DL cs.CL ·2026-11-08 ·Taner Cokyasar
code/data similarity 0.59
Abstract · 26 pages, 5 figures, 7 tables. Code and data: https://doi.org/10.5281/zenodo.24028663

This study asks what changed in engineering articles after conversational language models became publicly available on November 30, 2022, and develops methods for measuring such change in a field without a uniform abstract index. It assembled 1,839,962 articles published from 2015 to 2025 in 256 engineering journals, tracked twelve measures of wording, style, and article structure, and compared each with the projection of its own pre-2023 trend. Building the monthly series showed that records dated only by year fall on January 1 and create false January peaks unless removed. In abstracts, 31 words associated with language model output appeared 82% more often than projected (55.4 against 27.0 per 10,000 words in 2025), while four control words of the same formal register fell 16%. Words that soften a claim, such as may, fell 13%, words that strengthen one, such as clearly, rose 20%, and long words rose 11%. Author counts, reference lists, page counts, and publication volume stayed within 4% of trend. The change appears in all fourteen subject categories, and false release dates, dropping any one journal or word, and word lists chosen from the data support it. A random sample of 12,866 full texts from the same corpus, including subscription articles and weighted back to it, shows the same words rising 99% in article bodies. Statements disclosing AI writing assistance, measured apart from mentions of language models, went from none to 3.7% in 2025, four in five of them at one publisher. The design is descriptive, not causal.

0.34

Large Language Models Have Not Yet Transformed Science

cs.CL cs.AIcs.CYcs.DL ·2026-11-17 ·Ron Bekkerman, Nichole Li, Bahareh Kargar, Mark Somers
similarity 0.58
Abstract

The emergence of large language models (LLMs) in late 2022 has raised questions about the future of science, but assessing their impact requires understanding how scientific practices were already evolving. We analyze 2.5 million full-text papers across five disciplines from 2015 to 2025, tracing trends over a broad spectrum of lexical, structural, epistemological, and social dimensions. Aside from frequency shifts in some words, most pre-2023 trends continued with limited detectable breaks. Among the notable exceptions, growth in computer science coauthor teams accelerated significantly in Asia, while apparent declines in biomedical topic diversity largely reflected standardization in article formats. Our study establishes an empirical foundation for monitoring the health of global science as LLM adoption deepens.

0.22

More than half of recent astronomy papers are written with language-model assistance

cs.CL astro-ph.IMcs.DL ·2026-09-09 ·Serat M. Saad, Yuan-Sen Ting
similarity 0.56
Abstract · 14 pages, 6 figures, 2 tables

Language models leave a distinctive vocabulary in the prose they help write, and we measure how much of the astronomy literature now carries it. From the full text of 207,111 astro-ph papers spanning 2015 to mid-2026, we count those words in each paper and model the counts, in proportion to paper length, as a mixture of assisted and unassisted writing in a hierarchical Bayesian model. Papers from before 2020 calibrate the unassisted rate, and the 392 papers that disclose model use calibrate the assisted one. Our answer depends on how often these words would appear today if nobody used a model, a rate that must be modeled rather than observed, so we extend it past 2020 under three assumptions and report all three. For 2025 that gives $54^{+8}_{-8}\,(\mathrm{stat},\,95\%)\,^{+26}_{-0}\,(\mathrm{sys,\ background})$% of papers, the second error being the spread across the three. The estimate stays at or above 36% when we vary that choice, the calibration, and the requirement that adoption only rises. A word list built from the astro-ph corpus, keeping only words that rose across every subfield, leaves 2025 in the same range. Assisted writing is also getting harder to see, since authors adapt to the words that reveal it and the marker excess more than halves between 2023 and 2026. Our model allows for that fading, so it can separate a fainter trace from reduced use. More than half of recent astro-ph papers therefore carry a language-model trace, while only 0.81% of 2025 papers disclose it, one declaration for every $\sim$66 papers with a trace.

0.39