CS Paper Radar

A precision-driven evaluation protocol for author name disambiguation

Francesca Cappelli, Giovanni Colavizza, Silvio Peroni

cs.DL ·submitted 2026-10-09 11:23 UTC · v1
code/data

Abstract

Author Name Disambiguation systems are commonly evaluated using pairwise and B$^3$ metrics, which summarize clustering quality through aggregate precision and recall but do not explicitly distinguish between lumping (merging publications from different authors) and splitting (fragmenting the publications of a single author). Because these errors can have asymmetric consequences in bibliographic applications, we propose a precision-driven evaluation framework that decomposes clustering quality into complementary instance-level and cluster-level signals. The framework introduces cluster-level Lumping Error and Splitting Error to separately quantify contamination and fragmentation, combines them with standard pairwise and B$^3$ metrics through a Composite Score, and uses the resulting measures for model selection. We characterize the diagnostic behavior of the framework through a controlled synthetic case study and assess the sensitivity of the Composite Score to its weighting scheme. The results show that the proposed structural measures distinguish error patterns that aggregate metrics can mask, while the sensitivity analysis identifies a stable region for the adopted weighting configuration. The framework is released as an open-source Python pipeline.

Closest papers

Nearest neighbours by meaning across the whole index.

Retrieve-then-Disambiguate: Scalable Reviewer Discovery

cs.IR cs.DL ·2026-10-29 ·Vicente Amado Olivo, Joe Petrecca, Wolfgang E. Kerzendorf
code/data similarity 0.65
Abstract

The growing number of scientific submissions is straining the peer review process, while a large pool of relevant researchers remains unidentified and is therefore not considered as potential reviewers. We formulate the discovery of these researchers as a retrieval problem: given a submission, we retrieve and rank researchers based on domain similarity. We present retrieve-then-disambiguate, an open-source algorithm that identifies domain relevant researchers for a query submission by retrieving similar publications from the scientific literature and resolving ambiguous author names into unique researcher identities. We evaluate the algorithm on 500 astronomy and physics publications. We use ORCID-linked first authors, coauthors, and cited researchers as proxy labels to evaluate the recovery of researchers with known connections to each query publication. When retrieving the top 500 most similar papers, the algorithm recovers roughly 68\% of first authors, 60\% of coauthors, and 45\% of cited researchers. Our code is available at https://github.com/deepthought-initiative/Retrieve-then-Disambiguate.

0.39

OC-AND: A Citation-Enriched Benchmark for Author Name Disambiguation

cs.DL ·2026-10-09 ·Francesca Cappelli, Giovanni Colavizza, Silvio Peroni
benchmark similarity 0.65
Abstract

Author Name Disambiguation benchmarks are predominantly derived from closed infrastructures that may not reflect conditions in open bibliographic environments. We present OC-AND, an open citation-enriched benchmark for AND constructed by remapping the WhoIsWho dataset with bibliographic and citation metadata from OpenCitations. The dataset is produced through a reproducible five-stage pipeline involving DOI verification, metadata validation, OpenCitations enrichment, and author-centric transformation. OC-AND retains the WhoIsWho ground-truth files and partition structure unchanged, while its publication files are restricted to publications that could be matched to a validated DOI and an OpenCitations record. It introduces realistic characteristics of open environments: heterogeneous metadata completeness, absent abstracts and affiliations, asymmetric citation coverage, and sparse graph connectivity. The dataset contains author records with associated citations, enabling evaluation of AND methods in settings closer to real-world open scholarly infrastructure scenarios. OC-AND is released as open data with complete provenance documentation and pipeline code, supporting reproducible research and the development of citation-aware disambiguation approaches.

0.25

GRAND-HC: Graph-Refined Author Name Disambiguation

cs.IR ·2026-08-24 ·Yuanhao Sun, Zhouyang Jin, Yi Xu, Luoyi Fu, Jiaxin Ding +3 more
publishedcode/datahot direction similarity 0.61
Abstract

From-Scratch Name Disambiguation (SND) groups papers sharing an ambiguous name into clusters of distinct real-world authors. Existing methods suffer from two critical limitations: (1) inherent long-tailed author distribution biases representation learning, causing over-merging of tail authors; (2) existing cluster number estimation methods are unreliable for long paper sequences, hindering large-scale deployment. We propose \textbf{GRAND-HC}, a complete end-to-end SND framework. We construct a heterogeneous paper graph via co-author, co-organization, and co-venue relations, using a graph attention network as the embedding backbone. \textbf{Harmony Contrastive Learning (HCL)} dynamically reweights training loss to suppress overfitting to prolific authors, learning discriminative embeddings. A \textbf{Graph-Refined Distance Matrix (GRDM)} leverages graph topology to optimize pairwise distances, further preventing tail author over-merging. Meanwhile, a lightweight \textbf{Paper Compression Module (PCM)} achieves accurate cluster number estimation across varying scales. Finally, Hierarchical Agglomerative Clustering outputs the final clusters. Extensive experiments demonstrate state-of-the-art macro F1 performance. GRAND-HC has been deployed in a billion-scale academic database. Source code: https://github.com/baokou-fw2/GRAND-HC.

0.62

Understanding before verifying: Claim normalization for automated citation verification

cs.IR ·2026-08-31 ·Yifan He, Mengjia Wu, Siming Deng, Yi Zhang
similarity 0.59
Abstract

Citation accuracy has been studied for decades because of its importance to research reliability. Content-level citation verification assesses the reliability of scholarly claims. Recent work adopts a two-stage retrieval-classification framework inherited from fact-checking. However, this design overlooks the complexity of the raw citing claim and introduces three issues into the verification system, namely scope mismatch, perspective mismatch, and proposition entanglement. These issues increase the difficulty of retrieval and classification, thereby limiting model performance. Motivated by this gap, we propose claim normalization, which applies three rewriting strategies to the raw citing claim before retrieval and classification, allowing each downstream model to perform a single, well-defined task. Building on this method, we develop Claim-Normalized Citation Verification (CNCV), a new three-stage framework consisting of claim normalization, evidence retrieval with grounding, and citation classification. We evaluate CNCV across 18 classifiers using a factorial experiment on human-annotated citation instances. Compared with the prior two-stage framework, CNCV improves macro F1 by an average of 12% for encoders and 10% for generative LLMs, driven by improved evidence quality, the dominant factor identified in our experiments. Evidence retrieved from automatically normalized claims yields downstream classification performance statistically equivalent to that obtained with manually annotated evidence.

0.25

Multi-Functional Embedding Models for Funder Name Disambiguation in Scientific Publication Records

cs.CL ·2026-09-09 ·Kanyao Han, Zhiwen You, Jinseok Kim, Jana Diesner
published similarity 0.58
Abstract

Understanding the historical allocation and distribution of research funding advances our knowledge of how scientific research is supported across fields, institutions, and regions. However, large-scale analyses are hindered by the lack of comprehensive funder name disambiguation solutions, as funder names often exhibit spelling variations, translations, abbreviations, and inconsistent levels of granularity. In this paper, we present a framework for developing multilingual, multi-functional funder name disambiguation models and demonstrate its application to research publications in biodiversity conservation. To construct a training dataset, we integrated the Research Organization Registry (ROR), which provides unique identifiers for research organizations, with two publication datasets: the Web of Science (WoS) and the Crossref Open Funder Registry (OFR). We used multi-task learning with Contrastive Loss and Multiple Negatives Ranking Loss to fine-tune three open-weight embedding models from the Sentence Transformer, Gemma, and Qwen3 families. The best-performing models achieved accuracy above 0.90 when matching WoS funder names to ROR identifiers, outperforming general-purpose LLMs, including GPT-5.2, Claude-Sonnet-4.6, and Gemini-2.5-Flash, by more than 0.1. For funder names not indexed in ROR, we constructed a similarity network among funder names and identified clusters within it. Finally, we analyzed the disambiguation results and highlighted challenges arising from limited knowledge of smaller funders and funders from non-English-speaking countries. This work provides a reusable framework for funder name disambiguation with potential applicability across different model architectures and datasets, featuring cost-effective training data creation and multi-task learning and disambiguation.

0.30

CiteAnchor: Source-Aware Grounding for Hallucinated Citation Detection and Resolution

cs.CL cs.DL ·2026-10-21 ·Chenyi Tong, Ping Nie, Yu Zhang
benchmarkcode/data similarity 0.57
Abstract · 18 pages, 4 figures, 9 tables

Hallucinated citations make it harder to trace sources and verify the evidence behind scientific claims. Detecting them is difficult because bibliographic records can be incomplete or differ across source types and publication versions. A plausible match alone neither validates the citation nor establishes its underlying source. We introduce CiteResolve, a benchmark of 423 real-world hallucinated citations annotated for source recoverability. We further propose CiteAnchor, a source-aware grounding framework that adapts retrieval and field-level verification to each source type and separately determines citation validity and whether a unique source can be recovered. On an existing detection benchmark, CiteAnchor achieves 0.841 F1 with GPT-5.6 Luna and 0.784 with an open-weight backbone, outperforming the baseline with the same backbones by 7.9 and 36.8 points. On CiteResolve, it achieves 74.7% and 70.7% end-to-end task success, outperforming the corresponding web search baselines by 16.3 and 65.3 percentage points. Our code and data are available at https://github.com/oldwizard7/CiteANCHOR_code.

0.36

Bounds, Decompositions and Null Behaviour of KRATOS: A Mathematical Specification of a Recognition-Comparability Diagnostic

cs.AI ·2026-10-03 ·Maria Dolores Gonzalez, Alberto Barbado
unusual similarity 0.57
Abstract · 10 pages

KRATOS is a group-structured bibliometric diagnostic that compares the distribution of documents (participation) with the distribution of citation weight (recognition) over a fixed, finite universe of analytical groups. This note gives its complete mathematical specification and derives the properties that govern its interpretation. Beyond bounds and equality conditions for each component, we show that the recognition-alignment score depends on citation data only through ratios of group mean citation rates to the corpus mean; that the composition ceiling of the participation--recognition factor is an affine function of the total variation distance to the uniform reference; and that the factor itself satisfies Fréchet-type bounds in terms of this ceiling and a participation-weighted recognition score. We establish an exact logarithmic decomposition of the composite index, an ordering between the primary and a reciprocal-symmetric recognition score, invariance properties, and closed-form first and second moments of the recognition ratios under global and stratified permutation nulls. These results explain why composite orderings can be sensitive to small groups, heavy-tailed citation distributions and metadata reassignment. All results are verified numerically with an accompanying script. The specification concerns measurement structure only; it does not define a measure of epistemic change or justice.

0.28

MATCH: Multi-stage Article Tagging to Concept Hierarchies

cs.DL ·2026-10-19 ·Daniele Raimondi, Feichi Lu, Oliver Grun, Mariia Eremina, Andrea Perlato
similarity 0.57
Abstract

The increasing specialization of scientific research calls for fine-grained representations of publication content that can be applied at scale. SCALE extends the OpenAlex taxonomy with approximately 114,000 Scientific Concepts derived from author keywords, but assigning these Concepts to individual publications remains challenging. We introduce MATCH (Multi-stage Article Tagging to Concept Hierarchies), a framework that assigns SCALE Concepts using article titles, abstracts, and author keywords. MATCH combines multi-view embedding retrieval, deterministic taxonomy-aware filtering, and language-model reranking to select relevant Concepts while preserving their hierarchical paths. Its modular design supports both API-based and self-hosted inference. In a blind evaluation involving 22 domain experts across 11 fields and 110 articles, the self-hosted and API-based configurations achieved Precision of 81.5% and 83.3%, respectively, and Precision@5 of approximately 89% for both, with no statistically significant difference detected. These findings support the practical viability of the evaluated self-hosted configuration for large-scale Concept assignment without per-token API charges. Deployed on over two million MDPI publications, MATCH assigns a median of five Concepts per article, covers approximately 94% of the SCALE Concepts, and supports daily indexing of newly available publications. By operationalizing the SCALE taxonomy at publication level, MATCH provides a foundation for fine-grained scientometric analysis and monitoring of scientific developments.

0.21