CS Paper Radar

Assessing the Reliability of Longitudinal Bibliometric Rankings under Incomplete Citation Histories

João Paulo Papa

cs.DL ·submitted 2026-11-12 18:23 UTC · v1

Abstract

Longitudinal bibliometric analyses rely on annual citation histories, yet missing annual observations are frequently difficult to distinguish from true zero-citation years. We present a reproducible case-study framework for auditing citation-history gaps, tracing retrieval provenance, and quantifying the sensitivity of longitudinal indicators and rankings to explicit missing-data assumptions. The case study comprises 595 consolidated publications derived from 616 Google Scholar article records, and examines the completed calendar years 2024-2025. Three scenarios are compared: strictly observed annual counts (A), hypothetical zeros for internal gaps only (B), and hypothetical zeros for all missing years within a valid publication interval (C). The number of publications with computable growth and acceleration indicators increases from 196 (A) to 257 (B) and 490 (C); the corresponding counts for a revival indicator are 131, 152, and 177. Although the ordering of publications eligible under both compared scenarios is unchanged (Spearman and Kendall correlations equal to 1.0), complete-ranking positions and Top-K membership may change markedly. In particular, acceleration exhibits a Jaccard overlap of 0.563 at K=50 between A and C, while revival exhibits 0.667 at K=10. These findings demonstrate that unchanged indicator values and perfect correlations on a common subset do not guarantee stable rankings when eligibility changes. The zero assignments are hypothetical sensitivity assumptions, not evidence of provider semantics. The results are descriptive and based on a single researcher profile. A reusable Google Scholar Tracker software workflow supports the audit and sensitivity analysis; version 1.8.5 is publicly available on GitHub and archived on Zenodo (DOI: 10.5281/zenodo.24170285).

Comment
20 pages, 2 figures. Software available at https://doi.org/10.5281/zenodo.24170285

Closest papers

Nearest neighbours by meaning across the whole index.

Life after delisting: tracking the publication output and citation impact of Scopus-discontinued journals with OpenAlex

cs.DL ·2026-10-08 ·Álvaro Cabezas-Clavijo, Fernando Sánchez-Pita
similarity 0.66
Abstract

Selective databases such as Scopus periodically re-evaluate the titles they index and remove those that no longer meet their quality criteria. Once a journal is removed, its subsequent output is no longer recorded, and the consequences of this process have therefore received little attention. This study analyzes the publication output and citation impact of 514 sources delisted from Scopus between 2018 and 2024, based on 678,162 documents retrieved from OpenAlex over a window of three years before and after delisting. Differences between periods are tested with the Wilcoxon signed-rank test for paired samples, globally and by geographic region, field of knowledge and SJR quartile. Output grows until the year of delisting and falls sharply afterwards. The median annual output per journal drops from 35 to 22 documents (-37.1%; p < 0.001; r = 0.30), 61.7% of journals publish less, and 84 (16.3%) have no documents recorded in OpenAlex after delisting. Citations per document, measured over a three-year window, barely change when each journal is compared with itself (-2.4%; p = 0.079), and journals are almost evenly split between those whose impact falls and those whose impact holds steady or rises. The results indicate that delisting makes these journals less attractive as publication venues but does not penalize the citation of the work they publish to the same extent, which raises concerns from a research integrity perspective.

0.28

Mentorship resources and citation-elite journal publication trajectories after training: Evidence from bioscience mentor-mentee networks

cs.DL cs.SI ·2026-09-11 ·Hao Zeng, Yi Bu, Qingshan Zhou, Hongkan Chen
similarity 0.61
Abstract

Mentorship provides mentees with knowledge, collaborators, and reputation, but when do these resources support scholarship beyond training? Using 239,931 bioscience mentor-mentee pairs from the Academic Family Tree (1800-2020) linked to OpenAlex metadata, we examine this question through mentees' publication in citation-elite journals, identified by three source-level citation indicators. We distinguish training-period mentor and peer resources from post-graduation topic and coauthorship network continuity. Analyses combine trajectory comparisons, three propensity-score matching designs, PPML, two-part model and non-overlapping landmarks relating continuity in post-graduation years 1-3 to publication outcomes in years 4-10 and 6-20. Citation-elite journal publishing during mentorship is positively associated with later publishing, but correlates differ by career stage. Close mentor collaboration and larger peer group size characterize training-period publishing, whereas both are negatively associated with post-graduation publication probability and intensity. Early topic and network continuity are positively associated with later output in landmark analyses; across extended conventional windows, topic continuity becomes less favorable, consistent with later agenda broadening. Network continuity shows the most stable positive association with publication intensity. The findings suggest a stage-dependent resource-conversion process: training resources may facilitate entry into citation-elite journal publishing, whereas sustained output is associated with how intellectual and relational ties are retained or reconfigured.

0.22

Large scientific teams are more, not less, disruptive

cs.DL physics.soc-ph ·2026-10-05 ·Junming Huang, Yu Xie
similarity 0.57
Abstract · 34 pages, 14 figures

Modern science has moved decisively toward larger and more collaborative research teams. Yet an influential 2019 study reported that small teams are more disruptive than large teams, a conclusion difficult to reconcile with both the well-known trend toward increasing collaboration and the disproportionately large teams behind widely recognized disruptive research, such as work that has been awarded Nobel Prizes. Here we show that the reported disruptive advantage of small teams is largely an artifact of measurement rather than a fact about teams. Analyzing about 60 million scientific publications, we find that larger teams cite more references and that the standard disruption index declines mechanically as reference lists lengthen. Equalizing reference-count distributions across team sizes reverses the negative gradient within the index's own framework. Evaluating each paper against one reference at a time yields a reference-robust measure that distinguishes recognized breakthroughs from ordinary papers more sharply and more consistently than the standard disruption index across six independently curated benchmarks, including the Nobel Prize. Under this validated measurement, scientific disruption increases with team size across four scientific domains and in every decade from the 1970s to the 2010s. For scientific publications, we conclude that larger teams are more, not less, disruptive.

0.42

More Papers, Not More Output: Collaboration and Publication Trends among Early-Career Researchers

cs.DL ·2026-11-09 ·Christian Leibel
similarity 0.57
Abstract · 29 pages, 6 figures, 6 tables

Concerns about publication pressure often assume that early-career researchers now produce more papers than comparable researchers did in the past. However, publication counts may give a misleading impression of individual publication output because research has become increasingly collaborative. Researchers can list more papers on their CVs not only by producing more work themselves, but also by working with more coauthors and sharing credit for the same papers. Whether the collaboration-adjusted output of early-career researchers has increased over time has rarely been examined empirically. This study addresses this gap by analysing publication records from more than 200,000 early-career researchers at Western research institutions across five major science, engineering, and health fields between 1996 and 2024. Recent cohorts appear on more papers than earlier cohorts, but they also work in much larger research teams. Once publication credit is divided among coauthors, collaboration-adjusted output remains stable or declines across all fields. These findings show that research evaluation metrics that ignore collaboration are increasingly poorly aligned with how scientific work is produced. Rising publication counts alone should therefore not be taken as evidence that early-career researchers are producing more individual publication output.

0.35

Citations Are Late: Reading epistemic instability from what papers believe, years before the citation graph catches up

cs.DL ·2026-10-05 ·Andrey Ustyuzhanin, Ekaterina Trofimova, Denis Zuenko
hot direction similarity 0.57
Abstract

Paradigm shifts in science are visible in what researchers assert and contest before they are visible in the citation graph. We ask whether a cheap, content-level signal of epistemic instability, derived from the changing distribution of stated modelling beliefs in paper abstracts, can anticipate a paradigm shift earlier than the dominant citation-based disruption index (CD5). On the displacement of recurrent networks by Transformers in NLP, a pre-registered content signal crosses its detection threshold in 2016-Q1, whereas a real-time CD5 monitor cannot even observe the 2017 breakthrough until 2022-Q2, since CD5 needs a five-year forward-citation window: a lead of about 25 quarters. The flat citation baseline is not an artifact of one index, as our CD5, a reference-normalised variant, and an authoritative precomputed index all sit near zero across the shift. The lead is also not a faster proxy: at equal latency the signal beats the content competitors tested, including a learned CD-from-text model and an embedding disruption measure, and the decomposition sees what a keyword cannot (keyword-blind AUC of about 0.9, reproduced on human labels). The lead over CD5 is an observability lead. Which signal carries it depends on the shift: belief adoption in NLP, contestation and applicability stress in vision. Measured against the breakthroughs themselves, the signal leads by five quarters in NLP and is contemporaneous in computer vision.

0.28

The conservative turn in science: The changing character of knowledge recombination

cs.SI cs.DL ·2026-09-08 ·Cheng Zhang, Xinkun Liang, Yi Bu, Hongkan Chen
similarity 0.57
Abstract

This study examines how the dominant mode of cross-disciplinary knowledge combination has changed over time. It addresses a limitation of existing studies that document declining disruptiveness at the level of individual works but do not reveal whether the mechanisms of cross-disciplinary knowledge flow have themselves shifted. Drawing on 56 million publications and 816 million citations from OpenAlex (1960-2025), the study classifies cross-disciplinary citations into five types based on the set relationship between the subfield portfolios of citing and cited publications and introduces a semantic-distance measure derived from SPECTER-v2 embeddings of publication abstracts. It subsequently demonstrates that cross-disciplinary citations have shifted markedly away from configurations with no shared disciplinary ground toward types built on partially overlapping knowledge bases, a reorientation further reflected in a steady decline in average semantic distance between citing and cited fields. This shift co-occurs with a diffusion of brokerage positions across the network and a greater reliance on older literature, consistent with a system that has become broader and more decentralized in its interdisciplinary reach. Finally, this study discusses the implications of these findings for science policy and for the design of cross-disciplinary funding mechanisms.

0.08

A Graph Approach to the Academic Publishing Network: A Heterogeneous Model and Structural Screening over OpenAlex Open Data

cs.SI cs.DLstat.AP ·2026-08-11 ·Robert Šamárek, Radek Martinek
code/data similarity 0.56
Abstract

The academic publishing ecosystem is a vast, heterogeneous network of works, authors, institutions, journals, and topics. Traditional scientometrics reduces it to isolated tabular indicators (h-index, Impact Factor) that ignore topological context and are not designed to capture coordinated illegitimate practices. Building on our companion review, which proposed graph analysis of publishing integrity, this paper implements that approach. We define a heterogeneous multivariate graph model over OpenAlex open data (seven node types, seven edge types) and a methodology based on projections (citation and co-authorship networks), interpretable structural metrics, community detection, and three screening detectors of anomalous publishing patterns. We deliberately avoid binary classification: detectors return ranked candidates with explicit structural evidence for human assessment. On the institutional corpus of VSB - Technical University of Ostrava (2020-2025) with its one-hop citation neighbourhood, community detection recovers real research groups, centralities identify cross-disciplinary bridges, and the screenings flag dense co-authorship cliques, locally closed citation loops, and thematically isolated venues. On a second, venue-centric corpus with external ground truth (journals delisted by Scopus and DOAJ) and size-matched controls, a naive case-control design yields seemingly strong but spurious detectors (a prominence confound), whereas after matching the only robust signal is the breadth of disciplinary scope (AUC 0.70); an open graph-based prestige measure (PageRank over the journal citation network) tracks a JIF proxy while being an order of magnitude more resistant to citation gaming than count-based indicators. We release the method as the open-source library apnet with a reproducible CLI workflow and a web interface; the analysis runs on commodity hardware in minutes.

0.20

Earlier citation practices are linked to signs of artificial intelligence use in scientific writing

cs.DL ·2026-11-12 ·Hongkan Chen, Yongyuan He, Qingshan Zhou, Yi Bu
similarity 0.56
Abstract

As artificial intelligence (AI) becomes more common in scientific writing, some authors show fewer signs of AI assistance than expected across successive periods. We analysed OpenAlex publication histories for 71,133 authors with at least one eligible first-authored English-language publication each year from 2023 through September 2026. Using abstract-based writing scores, we defined resistance as the predicted minus observed share of abstracts classified as AI-assisted. Positive scores indicate fewer such abstracts than expected. Early resistance was positively associated with later resistance, although the share with positive scores fell. Environmental characteristics contributed more than individual characteristics to predicting later scores, although overall predictive power was low. Having a collaborator with a non-positive early score was associated with lower subsequent resistance. Authors with positive mean scores in both 2023-2024 and 2025-2026 cited older work, drew on fewer subfields and more often cited sources they had used before than authors with non-positive mean scores in both periods. These differences were evident before 2023. The adjusted gap in reference reuse rose from 3.24 percentage points before 2023 to 7.95 in 2025-2026. The findings link variation in signs of AI assistance to earlier differences in how scientists used prior work.

0.42