CS Paper Radar

How much does the map of science depend on the embedding model?

Alejandro Treny Ortega

cs.DL ·submitted 2026-10-12 12:00 UTC · v1

Abstract

Science maps describe relationships between papers. Text embedding models build these relationships by turning a paper's text into a list of numbers that gives it a position. Would another model give the same picture? I compare ten models using titles and abstracts of 500,000 publications from the OpenAlex bibliographic catalogue, published in 2000-2024 across 26 disciplines, including Medicine and Physics. Representing each discipline by its papers' average position produces similar arrangements across models, including when other papers are selected. Yet within a discipline and period, two models share, on average, only eight of the 25 papers closest to a given paper. Supplying titles alone also changes much of that list, with effects that vary between models. Models can also disagree about which discipline's papers are more spread out. One measure of spread compares angles between lines drawn from a common point to each paper's position. Moving that point can reverse which discipline appears more spread out, even when selecting other papers leaves the answer unchanged. The broad arrangement can remain similar while answers about nearby papers and differences between disciplines depend on the model, the text it receives and how its output is processed.

Comment
24 pages, 4 figures; supplementary material: 39 pages, 11 figures

Closest papers

Nearest neighbours by meaning across the whole index.

More of the same? Are scientific papers losing originality?

cs.DL ·2026-09-26 ·Ilan Doron-Arad, Elchanan Mossel
similarity 0.55
Abstract

Scientific output is growing rapidly, but it is unclear whether the expanding literature remains original or is increasingly repeating itself. Originality has many dimensions. One dimension can now be measured directly: how textually distinct a paper is from the work that came before it. Using semantic language models, we screened more than 26,000 full-text papers across eight fields in the CORE database (2015--2025) for passages resembling recent literature. We compared every paper against a fixed-size sample of earlier work, so increases in resemblance are not a by-product of literature growth. We find that the share of a paper's text resembling recent work was stable before 2021, then rose sharply, more than tripling by 2025. The increase came not from a few papers borrowing more heavily but from resemblance spreading across the literature: more papers now contain passages that echo recent work, and almost none of this growth is explained by cited work. We also found an analogous increase in passages expressing the same scientific idea. The pattern held across robustness checks and replicated in a second, independent database. Along this one measurable axis of originality, scientific writing has become less distinct from the work preceding it.

0.26

Large Language Models Extend Known Research But Do Not Create New Research Directions

cs.AI ·2026-10-12 ·Zhihua Liu, Kurt Butler, Damian Machlanski, Danilo Bueno, Sotirios A. Tsaftaris
similarity 0.55
Abstract

The success of autonomous research systems relies on the quality of the scientific hypotheses they generate. Recent studies report that human reviewers often rate LLM-generated ideas as more novel than those from human experts, raising hope that AI can accelerate scientific discovery. Yet it is unclear whether these models can do more than extend existing knowledge. Can LLMs drive genuine innovation by revealing new research directions, or are they limited to repeating and recombining what science already knows? This unresolved question lies at the heart of discussions about AI and the future of science. To address this, we distinguish two types of novelty: distance, which measures how far a hypothesis lies from existing work, and direction, which captures whether the hypothesis pursues new lines of inquiry beyond those already present. These two measures diverge sharply for LLM-generated work. We represent scientific papers in a shared conceptual space using a pretrained scientific-paper embedding model. Within this space, we embed 26.9 million human-authored papers across 1,325 research topics in six natural-science fields and 524,308 LLM-generated hypotheses from eight models. Distance is measured using existing metrics. To quantify direction, we introduce the orthogonal novelty score (ONS). LLMs consistently generate ideas more distant from existing work than human-authored papers, up to 16\% further when the LLM is given the papers farthest from the centre of the topic. However, LLM hypotheses have a 15--23\% lower median ONS than published papers from the same topic. Thus, LLMs move further from what is already known, but remain within established directions. This pattern persists across changes in various configurations. As a result, LLMs excel at extending the frontier but the creation of new directions is still led by human scientists.

0.29

Geometric Signatures of Conceptual Reorganization: A Counterfactual Embedding Framework for Detecting Scientific Revolutions

cs.DL cs.CLcs.LGphysics.hist-ph+1 ·2026-09-14 ·Dimitris Ntounis, Ariel Schwartzman, Chris Chafe, Thomas A. Ryckman
similarity 0.54
Abstract · 22 pages, 13 figures, 5 tables

We introduce document embedding geometry as a quantitative observable of conceptual reorganization and develop a counterfactual ablation framework for measuring how individual concepts influence the organization of scientific knowledge, providing a quantitative framework for detecting scientific revolutions. The observable is defined by the geometric perturbation induced when removing documents associated with a candidate concept from the embedding space before and after its historical emergence. Statistical validation is performed using five historical case studies spanning physics, mathematics, and machine learning: special relativity, Gödel's incompleteness theorems, the Higgs mechanism, deep learning, and the attention mechanism underlying transformer architectures. Across the historical case studies, the framework identifies measurable geometric signatures associated with conceptual reorganization, while the validation studies expose important limitations arising from document assignment and sparse historical data. These results establish embedding geometry as a medium for quantifying conceptual reorganization, providing a new approach for studying how scientific fields restructure over time.

0.31

Building Weighted PubMed Citation Graphs at Scale: A Pipeline and an Audit of Edge-Similarity Features

cs.SI ·2026-10-31 ·Hossein Mohasel Arjomandi
code/data similarity 0.54
Abstract · 6 pages, 6 tables. Submitted to the BTSD workshop at IEEE BigData 2026

Studies of scientific literature often combine citation links with article metadata, but the data preparation needed to do so at the scale of millions of papers is laborious and error-prone. This paper describes a pipeline that turns PubMed metadata and a citation network into a graph whose edges carry seven similarity features: bibliographic coupling and co-citation (each as a count and as a Jaccard coefficient), publication-year proximity, MeSH-term similarity, and similarity of MedCPT title-abstract embeddings. Applied to a citation network derived from the cell biology research literature, the pipeline yields 7,445,898 nodes and 57,512,086 edges with complete year, MeSH, title, and abstract metadata. The paper reports how nodes are matched across sources, how duplicate identifiers are resolved, and which intermediate tables grew to hundreds of gigabytes. An audit of the full feature table shows that 47.2% of the edges share neither references nor citing documents, so their topological features are zero, and that the MeSH feature is nonzero on only 0.28% of edges. Where both kinds of signal exist, they agree: edges with higher coupling also have more similar abstracts and closer publication years. In a pilot, replacing the median by a mean or best-match aggregation makes the MeSH feature nonzero on nearly all article pairs, and near-identical texts point to possible duplicate records that identifier checks miss. The parsing, graph-construction, and feature code are public; the paper states which data are released and which are not. A companion paper uses the resulting weighted graph for clustering.

0.59

The conservative turn in science: The changing character of knowledge recombination

cs.SI cs.DL ·2026-09-08 ·Cheng Zhang, Xinkun Liang, Yi Bu, Hongkan Chen
similarity 0.54
Abstract

This study examines how the dominant mode of cross-disciplinary knowledge combination has changed over time. It addresses a limitation of existing studies that document declining disruptiveness at the level of individual works but do not reveal whether the mechanisms of cross-disciplinary knowledge flow have themselves shifted. Drawing on 56 million publications and 816 million citations from OpenAlex (1960-2025), the study classifies cross-disciplinary citations into five types based on the set relationship between the subfield portfolios of citing and cited publications and introduces a semantic-distance measure derived from SPECTER-v2 embeddings of publication abstracts. It subsequently demonstrates that cross-disciplinary citations have shifted markedly away from configurations with no shared disciplinary ground toward types built on partially overlapping knowledge bases, a reorientation further reflected in a steady decline in average semantic distance between citing and cited fields. This shift co-occurs with a diffusion of brokerage positions across the network and a greater reliance on older literature, consistent with a system that has become broader and more decentralized in its interdisciplinary reach. Finally, this study discusses the implications of these findings for science policy and for the design of cross-disciplinary funding mechanisms.

0.08

Embedding Models Measure in Peculiar Ways

cs.CL cs.LG ·2026-09-17 ·Juri Opitz, Andrianos Michail
similarity 0.53
Abstract

Embedding spaces define notions of semantic similarity and distance. We study whether those embeddings reflect physical measurements of mass, distance, time and volume, which admit a unique, objective notion of semantic equivalence and distance. We find that physical measurement is only weakly modeled in the embedding space, and that instead quite peculiar measurement patterns can be observed. Further analysis indicates that embedding representations of physical measurements are strongly influenced by superficial string similarity, and recalibration of similarity does not substantially improve the alignment.

0.29

Multiplex coupling: Connecting the intellectual base and the topics of a research field

cs.DL stat.ME ·2026-10-15 ·Massimo Aria, Corrado Cuccurullo, Maria Spano
similarity 0.53
Abstract · 41 pages, 9 figures, 6 tables, plus 24 pages of Supplementary Material. Submitted to Quantitative Science Studies. Data and code: https://do…

Two documents can be close because they cite the same works or because they write about the same things, and the two kinds of proximity often disagree. Science mapping reads them in separate maps or merges them, as hybrid clustering does; either way the disagreement is lost. We introduce multiplex coupling, a framework that couples the same documents by their references and by their terms, keeps the two layers separate and analyses their relation. Similarities are standardised against a null model, and the agreement between the layers is measured on random pairs of documents. Consensus clustering partitions each layer into roots, groups sharing an intellectual base, and themes, groups sharing a topic. A test on the roots $\times$ themes table identifies branching roots and convergent themes, and pairs of roots are followed over time. Every structure is reported with its persistence across numbers of neighbours, which on synthetic data identifies most false branching. Applications to Web of Science, Scopus and OpenAlex collections show that the two layers carry largely complementary information, and reveal substantive and methodological bases that have grown into several topics. The framework is implemented in bibliometrix.

0.35

Weakly Supervised Adaptation of Scientific Embeddings for Exploratory Literature Mapping

cs.DL cs.IR ·2026-10-18 ·Roman Neruda, Petra Vidnerová, Jiří Šejnoha, Jakub Mihálik, Tomáš Marvan
hot direction similarity 0.53
Abstract

Scientific document embeddings such as SPECTER2 are optimized for retrieval over large, heterogeneous corpora. On a small specialized corpus their similarity distributions can become highly compressed, which limits their usefulness for exploratory literature mapping. We study whether weakly supervised in-domain adaptation, using only metadata already available in open citation databases, changes this geometry. On ArtCon, a curated corpus of 704 publications on artificial and machine consciousness, we adapt the SPECTER2 proximity model with a lightweight LoRA adapter trained on positive pairs derived from bibliographic coupling in OpenAlex, using no manually annotated similarity labels. We compare the adapted embeddings with lexical (TF-IDF), citation-based (bibliographic coupling) and generic semantic (SPECTER2) representations under thresholded, fixed-density and symmetric k-nearest-neighbour graph constructions. Adaptation substantially decompresses the similarity distribution (mean pairwise cosine similarity 0.9104 to 0.8663, standard deviation 0.0259 to 0.0594) and yields graphs with markedly higher Louvain modularity and better connectivity than the base encoder under matched edge and neighbourhood budgets. The effect is stable across all three graph constructions and is therefore not an artefact of a single sparsification choice. We report this as a case study on one corpus, and we are explicit about what it does not establish: the clustering indices used here are computed against community labels derived from each representation's own similarity graph, so they measure internal geometric self-consistency rather than agreement with an external topical ground truth, and adaptation and evaluation use the same documents. Evaluation against independent labels, on held-out documents, and against geometric controls is ongoing work.

0.21