CS Paper Radar

Quantifying research uniqueness with Unbalanced Optimal Transport: Case studies of ten leading universities in neuroscience

Yu Ota, Takayuki Ogi, Yo Nakawake, Daichi Mochihashi, Amane Koizumi

cs.DL cs.SI ·submitted 2026-10-13 08:58 UTC · v1

Abstract

Research evaluation has developed around quantity, citation impact and, more recently, the accumulation of research excellence, but these dimensions remain largely dependent on citations, which describe how research is received rather than what it contains. We propose uniqueness as a complementary dimension and measure it with the Research Uniqueness Index (RUI), which asks how far a university's research sits from what is typical in its field, using no citations but the content of the research itself. Each researcher is represented by an embedding of their abstracts, and an unbalanced version of Optimal Transport returns the share of that work which cannot be matched into a random sample of same-field researchers, the standard population; that unmatched share is the score. Because every researcher is scored, a university is a distribution rather than one number, and we treat that distribution as the indicator: it shows whether uniqueness is spread across an institution or concentrated in a few groups. Quantiles summarise it and, being quantiles, do not reward size. Since uniqueness is defined only relative to a field, we demonstrate the index within one, neuroscience (Scopus, 2021-2025; 9,861 reference researchers). All ten universities studied sit well above the standard (RUI_median 0.40-0.53 versus 0.27) and five specialised institutes higher still (0.57-0.73), the most unique work often confined to a narrow niche that the same scores name. RUI is uncorrelated with citation impact, and a controlled test confirms that it tracks displacement from the field rather than topical concentration; results are robust to an alternative embedding and replicate in computer science and economics. The index is not intended as a league table: the gap above the standard and the researcher-level niche structure are stable, whereas the order among adjacent universities is not.

Comment
22 pages, 6 figures, 2 tables. Supplementary material included as an ancillary file

Closest papers

Nearest neighbours by meaning across the whole index.

Where Do Highly Cited Scientists Stand? A Multi-Dimensional Analysis of Positional Distribution in the Top 2% Ranking

cs.DL ·2026-11-17 ·Yuehai Li, Zuguang Gu
similarity 0.62
Abstract

The Top 2% Scientists list is widely used. Yet current applications either only count the number of scientists in different units or criticize the list's validity as a measure of individual impact. We introduce positional enrichment, a scale-free framework that compares observed to expected counts across percentile bins on the list, revealing where a unit's scientists tend to sit along the list rather than how many it contributes. Applying positional enrichment to the 2026 single-year list, we show that countries, institutions, and research subfields each fall into a consistent pattern of three groups with Head-, Middle-, and Tail-enriched profiles. Subfield profiles are globally stable, whereas countries and institutions are heterogeneous across subfields. National and institutional differences are therefore compositional. Therefore, the reason of a country or institution sitting near the head of the list is unlikely its scientists performing better within the same subfields, while it holding more of the subfields that sit near the head. Academic age adds an independent dimension, with stronger enrichment toward the head after 25 years and exceptions in fast-moving subfields such as artificial intelligence. We show the ranking is not a flat ordering of individual influence but a subfield-organized stratification.

0.24

The illusion of neutrality in metric-based research evaluation

cs.DL ·2026-09-21 ·Fengyuan Liu, Hazem Ibrahim, Yasir Zaki, Talal Rahwan
similarity 0.58
Abstract

Venue prestige and citation counts are two widely used, albeit imperfect, signals of research quality. When the two signals conflict, evaluators must decide how much weight to assign each. Yet, it remains unknown how researchers across disciplines trade off these two signals when evaluating research outcome. To fill this gap, we surveyed 869 researchers using paired choices between hypothetical departmental hiring rules that assigned different weights to venue prestige and citation counts, asking which would produce better science. Among 795 respondents whose choices were largely internally consistent, choices placed nearly equal aggregate weight on the two signals. This apparent balance concealed substantial individual heterogeneity: nearly two in five respondents occupied the most venue-heavy or citation-heavy intervals. Moreover, respondents on average chose more citation-heavy rules than they believed their departments used in hiring. By revealing the subjective judgments that arise when research indicators conflict, our findings reinforce calls for greater caution when quantitative indicators are used to evaluate research.

0.27

Has Scientific Talent Shifted from Depth to Breadth?Evidence across Papers, Knowledge Inputs, Careers, and Teams

cs.DL cs.AIcs.HC ·2026-09-13 ·Xiaoshn Nee, Haobo Zhong, Xiaomin Ni
similarity 0.54
Abstract

Generative artificial intelligence raises a central question for scientific training and organization. Is research shifting from deep specialization toward broad individual knowledge? We examine this proposition across papers, cited knowledge, contributor histories, and teams using 47,959 articles from six fields over 2010-2025, 51,736 resolved cited works, and chronologically reconstructed prior publication histories for 1,754 randomly selected index contributors. From 2010 to 2022, team size increased by an estimated 37.3% (95% confidence interval [34.4%, 40.3%]), while paper topic breadth declined by 0.0144 on a 0-1 hierarchical distance scale. Cited knowledge was stable to modestly broader, revealing a divergence between focused outputs and the reach of knowledge inputs. Established contributors' prior breadth increased by 0.0190 [-0.0078, 0.0459] by 2019-2022, within a +/-0.05 equivalence bound assessed in sensitivity analysis. In mature citation windows, one standard deviation of focal depth was associated with 8.2% higher 1 + FWCI [1.9%, 14.9%]; average breadth and interaction associations were smaller under the specified equivalence bounds. Post-2022 deviations from earlier trends were not systematic, and recent changes did not vary clearly with baseline AI intensity across 83 subfields. The findings support a differentiated structure of scientific expertise in which focused individual accumulation coexists with expanding collaboration and sustained access to diverse knowledge inputs.

0.27

Foreign-trained faculty and the collaborative organization of high-impact U.S. science

cs.DL ·2026-09-30 ·Erjia Yan, Chaoqun Ni, Xiang Zheng, Weiye Gu
similarity 0.54
Abstract

Internationally mobile scientists are central to national research and innovation systems. We link faculty rosters from the Academic Analytics Research Center to OpenAlex publication records for 2011-2020, yielding more than 12 million faculty-publication observations for 236,394 tenure-system faculty at more than 300 major U.S. universities. Foreign-trained faculty, defined by a terminal degree awarded outside the United States, constitute about 11% of the observed faculty workforce but account for 13-14% of publications and 14-16% of top-1% cited elite output. We find that this elevated representation in elite output is closely related to their organizational embeddedness: when faculty are compared within the same institution, scientific domain, rank, and year, the difference in elite output narrow substantially while overall productivity and collaboration differences persist within those settings. In addition, foreign-trained faculty enter each publication year with larger and broader prior collaboration networks, and collaborator reach is more strongly associated with subsequent elite output. By contrast, although raw topic breadth is greater among foreign-trained faculty, after accounting for prior publication volume, we find that their topic breadth is slightly narrower and more cognitively concentrated. These findings recast international training in the lens of scientific capacity and organizational integration in that internationally accumulated scientific capabilities become embedded in institutions and relationships through which research is organized and produced.

0.18

Large scientific teams are more, not less, disruptive

cs.DL physics.soc-ph ·2026-10-05 ·Junming Huang, Yu Xie
similarity 0.54
Abstract · 34 pages, 14 figures

Modern science has moved decisively toward larger and more collaborative research teams. Yet an influential 2019 study reported that small teams are more disruptive than large teams, a conclusion difficult to reconcile with both the well-known trend toward increasing collaboration and the disproportionately large teams behind widely recognized disruptive research, such as work that has been awarded Nobel Prizes. Here we show that the reported disruptive advantage of small teams is largely an artifact of measurement rather than a fact about teams. Analyzing about 60 million scientific publications, we find that larger teams cite more references and that the standard disruption index declines mechanically as reference lists lengthen. Equalizing reference-count distributions across team sizes reverses the negative gradient within the index's own framework. Evaluating each paper against one reference at a time yields a reference-robust measure that distinguishes recognized breakthroughs from ordinary papers more sharply and more consistently than the standard disruption index across six independently curated benchmarks, including the Nobel Prize. Under this validated measurement, scientific disruption increases with team size across four scientific domains and in every decade from the 1970s to the 2010s. For scientific publications, we conclude that larger teams are more, not less, disruptive.

0.43

Kernel-based External Validity of Experimental Studies

cs.LG stat.MEstat.ML ·2026-10-09 ·Federico Matteucci, Gerhard Neumann
unusual similarity 0.53
Abstract

The external validity of a study is the extent to which its findings generalize to unseen experimental conditions, such as datasets. We formalize the results of a study together with its research question, and define external validity as the probability that two independent and same-sized replications of the study produce results whose maximum mean discrepancy (MMD) is below a given tolerance. This approach, centered on the choice of a space of results and of an appropriate reproducing kernel for the MMD, is flexible enough to be deployed on multiple different types of results and research questions. Additionally, we establish an asymptotic power-law relation between the size of a study and the quantiles of the MMD, which we use to estimate the desired size of an externally valid study from a small-scale preliminary study. We deploy our framework to rankings with ties and discuss kernels suited to different research questions. Experiments on existing case studies and on a simulated newly designed study illustrate the applicability of the framework. We release our code as part of the genexpy Python module.

0.25

From Citations to Contributions: LLM-Assisted Credit Scoring of Research Articles

cs.DL cs.AIcs.CLcs.IR ·2026-09-07 ·Sana Ebrahimi, Suraj Shetiya, Abolfazl Asudeh
top venuecode/data similarity 0.52
Abstract

Citation-based measures of scientific influence typically treat citations as uniform signals, ignoring the different roles that cited works play in a paper's contribution. We introduce contribution-based credit scoring for research articles: a structured citation analysis that decomposes a paper's credit between its own original contribution and the prior work it builds on. Motivated by a cooperative-game view of scientific credit, we propose the contribution tree, a hierarchical framework that conserves importance across the document structure and separates original from citation-derived contribution. To make this framework scalable, we use LLMs as noisy comparative estimators of local importance. We further extend the model to article collections by propagating contributions through weighted citation graphs, yielding corpus-level contributions and normalized influence scores. Our experiments suggest that our framework captures contribution signals beyond surface-level heuristics. Our code is available at https://github.com/sanaebrahimi/Importance_Scoring/

0.38

The Scientific Research Taste Benchmark (SciTaste-Bench): A Multi-Disciplinary Benchmark for Assessing Large Language Models' Judgment of Research Taste in Scientific Papers

cs.DL ·2026-10-09 ·Xinyi Gao, Xiaogang Li, Jialu Nie, Yuting Fu, Jingye Xu +7 more
benchmarkcode/data similarity 0.52
Abstract

Whether large language models can assess the innovativeness of academic publications matters for literature reviews, research funding screening, and academic recommendations. We introduce the Scientific Research Taste Benchmark (SciTaste), a gold-standard benchmark covering six disciplines: physics, mathematics, chemistry, biology, medicine, and finance. SciTaste measures research taste, that is, the ability to judge a paper's scientific value and potential, of which innovativeness is one of seven dimensions. It integrates metadata from NASA ADS, OpenAlex, and arXiv, aligns papers to field-specific classification systems, and scores each paper with the seven-dimensional Academic Taste Index. Dimension weights come from a bi-objective machine-learning approach (five regressors, including random forests) and are shared across all six disciplines. The benchmark includes 300 selected papers (50 per discipline) and a backup pool of 5,700 papers (950 per field) covering 1996-2023. SciTaste supports four evaluation modes: pure_qa, native_search, our structured tool-calling agent DiM (9 bibliometric tools in ADS/OpenAlex), and dimagent_v3_native. Outputs are evaluated by two complementary schemes: correlation with dimension-level accuracy, and WAPE relative error. Across ten state-of-the-art LLMs, tool-based structured evaluation generally outperformed toolless and raw web search modes, though gains were model-dependent. Physics is easiest; chemistry, biology, and medicine remain more challenging. SciTaste supplies the value-judgment substrate auto-research systems need to screen, rank, and prioritize candidate ideas. Dataset, harness, and scoring code are available on Hugging Face at https://huggingface.co/datasets/skylenage/SciTaste-benchmark.

0.38