CS Paper Radar

Geometric Signatures of Conceptual Reorganization: A Counterfactual Embedding Framework for Detecting Scientific Revolutions

Dimitris Ntounis, Ariel Schwartzman, Chris Chafe, Thomas A. Ryckman

cs.DL cs.CLcs.LGphysics.hist-phphysics.soc-ph ·submitted 2026-09-14 02:08 UTC · v1

Abstract

We introduce document embedding geometry as a quantitative observable of conceptual reorganization and develop a counterfactual ablation framework for measuring how individual concepts influence the organization of scientific knowledge, providing a quantitative framework for detecting scientific revolutions. The observable is defined by the geometric perturbation induced when removing documents associated with a candidate concept from the embedding space before and after its historical emergence. Statistical validation is performed using five historical case studies spanning physics, mathematics, and machine learning: special relativity, Gödel's incompleteness theorems, the Higgs mechanism, deep learning, and the attention mechanism underlying transformer architectures. Across the historical case studies, the framework identifies measurable geometric signatures associated with conceptual reorganization, while the validation studies expose important limitations arising from document assignment and sparse historical data. These results establish embedding geometry as a medium for quantifying conceptual reorganization, providing a new approach for studying how scientific fields restructure over time.

Comment
22 pages, 13 figures, 5 tables

Closest papers

Nearest neighbours by meaning across the whole index.

Tracing individual knowledge trajectories in a changing field: the case of general relativity and gravitation

cs.CL physics.hist-ph ·2026-09-16 ·Raphael Schlattmann, Malte Vogl
similarity 0.58
Abstract · 43 pages including Supplementary Material (11+1 figures, 4+6 tables). Submitted to Frontiers in Complex Systems

Historians have reconstructed the twentieth-century transformation of general relativity and gravitation (GRG) at the field level and through individual careers, but connecting these scales requires a way to compare researchers with the changing field over time. We develop such a comparison, setting a researcher's publications and references against GRG field literature from the same, earlier, and later two-year periods. Building on Own Vocabulary and Embedding Density Estimation from our earlier two-case study (arXiv:2501.00391), we extend the analysis to the fifty most-published authors in a NASA/ADS corpus of about 180,000 GRG records (1911 to 2000) and add two citation-based measures, Referenced Vocabulary and Citation Identity. The four measures compare an author's written language, cited literature, semantic neighbourhood, and cited-authority configuration with the surrounding field. The earlier cases suggested that closer field-vocabulary alignment accompanies a denser semantic neighbourhood. Across the fifty authors this holds only partially. Written and cited vocabularies tend to move together, usually resembling later GRG literature as the field turned towards astrophysical and cosmological research. Semantic neighbourhoods more often lie where the field's publications were concentrated in earlier periods, while co-citation patterns follow no single temporal direction, and the two citation measures frequently place the same researcher differently despite drawing on identical reference lists. Individual trajectories can thus combine vocabulary tied to later field states with older semantic or citation structures, and these divergent cases mark patterns for closer historical investigation. The approach transfers to other fields with defensible corpus boundaries and adequate coverage of texts, references, and disambiguated author identities.

0.38

More of the same? Are scientific papers losing originality?

cs.DL ·2026-09-26 ·Ilan Doron-Arad, Elchanan Mossel
similarity 0.58
Abstract

Scientific output is growing rapidly, but it is unclear whether the expanding literature remains original or is increasingly repeating itself. Originality has many dimensions. One dimension can now be measured directly: how textually distinct a paper is from the work that came before it. Using semantic language models, we screened more than 26,000 full-text papers across eight fields in the CORE database (2015--2025) for passages resembling recent literature. We compared every paper against a fixed-size sample of earlier work, so increases in resemblance are not a by-product of literature growth. We find that the share of a paper's text resembling recent work was stable before 2021, then rose sharply, more than tripling by 2025. The increase came not from a few papers borrowing more heavily but from resemblance spreading across the literature: more papers now contain passages that echo recent work, and almost none of this growth is explained by cited work. We also found an analogous increase in passages expressing the same scientific idea. The pattern held across robustness checks and replicated in a second, independent database. Along this one measurable axis of originality, scientific writing has become less distinct from the work preceding it.

0.26

Efficient Context-Limited Telescope Bibliography Classification for the WASP-2025 Shared Task Using SciBERT

cs.LG astro-ph.IM ·2026-08-29 ·Madhusudhana Naidu
publishedbenchmarkcode/data similarity 0.57
Abstract · 3 pages, 2 tables. 1st place system description for the TRACS shared task at WASP 2025 (Third Workshop for Artificial Intelligence for Scien…

The creation of telescope bibliographies is a crucial part of assessing the scientific impact of observatories and ensuring reproducibility in astronomy. This task involves identifying, categorizing, and linking scientific publications that reference or use specific telescopes. However, this process remains largely manual and resource intensive. In this work, we present an efficient SciBERT-based approach for automatic classification of scientific papers into four categories - science, instrumentation, mention, and not telescope. Despite strict context-length constraints (maximum 512 tokens) and limited compute resources, our approach achieved a macro F1 score of 0.89, ranking at the top of the WASP-2025 leaderboard. We analyze the effect of truncation and show that even with half the samples exceeding the token limit, SciBERT's domain alignment enables robust classification. We discuss trade-offs between truncation, chunking, and long-context models, providing insights into the efficiency frontier for scientific text curation.

0.45

ManiEdit: Sequential Unstructured Knowledge Editing for Language Models from a Manifold Perspective

cs.CL ·2026-09-27 ·Rui Liu, Chenheng Zhang, Haoxuan Li, Zhouchen Lin
code/data similarity 0.56
Abstract · 31 pages, 8 figures

Large language models (LLMs) inevitably generate some incorrect or outdated content, necessitating efficient and precise mechanisms for continual knowledge updates. However, existing model editing methods struggle to sequentially edit unstructured long-form knowledge, suffering from severe edit forgetting and degradation of general capabilities. To address these challenges, we reframe knowledge editing from a manifold perspective, viewing it as a localized displacement of an edit sub-manifold within the global knowledge manifold. Under this formulation, the problem can be decomposed into two key questions: (i) how to identify representative edit points that effectively anchor the edit sub-manifold, and (ii) how to preserve the remaining manifold structure during the sub-manifold displacement process. Based on this perspective, we propose ManiEdit, a novel manifold-aware autoregressive editing framework consisting of two core components. Pivot Localization addresses the mediocre-point dilemma by identifying high-leverage pivots to anchor the edit sub-manifold. Manifold-Aware Preservation preserves different knowledge types through an energy-weighted penalty combined with recursive null-space alignment. Experiments on two base LLMs and four unstructured editing benchmarks demonstrate that ManiEdit achieves state-of-the-art performance, outperforming the strongest baseline by up to +27.81 BERTScore and +8.50 ROUGE-L, while maintaining near-original general capabilities across six representative downstream tasks. Our code is available at: https://github.com/Areyliu/ManiEdit

0.35

Doc2LoRA Provides Decodable Representations of Scientific Ideas

cs.CL cs.DLcs.IRcs.LG+1 ·2026-09-29 ·Chand Sahil Mansuri, Joel Zachariah, Sadamori Kojaku
code/data similarity 0.55
Abstract · 32 pages, 4 figures, 12 tables. Code: https://github.com/skojaku/doc2lora-embedding

Representing scientific papers as points in a space lets us search for similar papers and inquire about how fields relate to one another and drive innovation. Beyond search, the vector space of papers invites generation: mixing papers through simple vector operations creates new points, mirroring combinatorial novelty, the recombination of existing ideas into new ones. However, a mixed point often represents an idea no paper has yet realized, with no papers nearby to identify the idea. We propose representing each paper by a LoRA adapter generated by the Doc-to-LoRA hypernetwork. Every point in the space, including mixtures, thus represents a large language model (LLM) open to questions and instructions in natural language. On papers from the American Physical Society (APS), we instruct the LLM at the average of each subfield to name the field in a few words and obtain labels closer to the official names than the labels of five baselines, as judged by word overlap and a panel of five LLM judges. We also ask the LLMs at points between two APS papers to write an abstract and obtain descriptions shifting from one paper to the other in step with the mixing weight. While Doc-to-LoRA is trained for generation, a small invertible transform makes the embeddings competitive for search, on par with SPECTER2 and EmbeddingGemma and close to SBERT. Because the transform is invertible, every point in the transformed space still maps back to an LLM. The embeddings thus serve both search and generation, enabling researchers to question the idea at any point in the space as a starting point for generating new ideas.

0.45

Filling holes in science draws collective attention, but most higher-order holes remain unexplored

cs.CY cs.DLcs.SI ·2026-08-28 ·Jiajie Luo, James A. Evans
similarity 0.55
Abstract · 26 pages of main text, 155 with all the supplemental information. 4 main figures, many more in the supplemental information

Most scientific discoveries justify their significance by filling holes between existing ideas and arguments. Representing knowledge as high-dimensional concept embeddings, we use persistent homology to detect scientific holes of increasing order, from lower-order gaps between disconnected ideas to higher-order cavities within unevenly connected ones; then we identify the research that fills them. Researchers who close holes draw outsized collective attention, staging novelty that the citing literature rewards, most strongly in discovery-oriented fields and least in design-oriented fields. Closing a hole is followed by persistent work on the same combinations of concepts at many times the rate of comparable holes left open. Yet as knowledge grows, higher-order holes explode while the fraction of research that fills them collapses, leaving most higher-order combinations unexplored. These results call for a richer science of holes and mark a frontier where contemporary AI might help fill the gaps human science opens.

0.15

Hyperspherical Semantic Trajectory Analysis: Mapping Technological Diffusion across Academic Preprints, Patent Signals, and Compute Scaling

cs.CL cs.LG ·2026-09-25 ·Muhammad Sukri Bin Ramli
similarity 0.55
Abstract

Macroeconomic productivity metrics, such as Total Factor Productivity, register technological breakthroughs with multi-year reporting lags due to administrative survey intervals and national accounting conventions. This paper introduces Hyperspherical Semantic Trajectory Analysis (HSTA), an unsupervised quantitative methodology that tracks technology diffusion directly from unstructured scientific and commercial text streams. We analyze 30,000 filtered document records spanning academic preprints from arXiv and patent application records from the USPTO. By projecting high-dimensional Transformer sentence embeddings onto unit hyperspheres using Spherical K-Means clustering across eight primary sub-topics and UMAP manifold reductions, HSTA formalizes two quantitative metrics: (1) Semantic Centroid Vector Drift, which tracks vocabulary shifts between temporal sub-corpora to identify structural paradigm transformations; and (2) Commercialization Offset, which evaluates cross-corpus peak density alignments between scientific discovery and intellectual property filings. Linking quarterly topic volume velocity with physical hardware metrics from the Epoch AI database, Vector Autoregressive F-tests demonstrate that quarterly paper volume velocity alone does not Granger-cause frontier compute allocation surges at conventional statistical significance levels, highlighting the necessity of conditioning textual signals on physical capital constraints. Empirical results reveal that sub-topics covering Large Language Models (with a drift metric of 0.332) and Artificial Intelligence Systems (with a drift metric of 0.234) undergo the highest rate of semantic evolution, offering an objective, real-time mechanism to complement traditional economic statistics.

0.25

Predicting Emerging Topics from Outliers: A Prospective Study of Weak Signals in Embedding Space

cs.CL ·2026-09-24 ·Evangelia Zve, Gauvain Bourgne, Jean-Gabriel Ganascia
published similarity 0.55
Abstract · Accepted to Findings of AACL-IJCNLP 2026

Some documents that embedding-based topic models initially classify as noise later become founding members of emerging topics. At publication time, however, they appear as scattered points in embedding space and are difficult to distinguish from ordinary noise without the benefit of hindsight. We study whether such anticipatory outliers can be predicted prospectively, using only information available when a document first appears. We derive labels from the subsequent trajectories of outlier documents, distinguishing those that anticipate new topics from those that reinforce existing topics or remain isolated, and estimate label confidence through agreement across multiple embedding models. On two French news corpora, anticipatory outliers prove predictable at publication time. Under cross-validation, $F_1$ rises from about 0.77 over the full eligible population to above 0.90 on high-consensus subsets, and remains at 0.76-0.80 under a strictly chronological evaluation. Predictive performance is driven mainly by geometric features capturing each outlier's position in embedding space.

0.46