CS Paper Radar

Stronger artificial intelligence breakthroughs hasten their own exhaustion

Beichen Lu, Honglin Bao, Qi Huang, Qingguo Meng, James A. Evans

cs.AI cs.CYecon.GN ·submitted 2026-10-19 21:12 UTC · v1
hot direction

Abstract

Progress in artificial intelligence is widely assumed to scale with effort: more researchers, papers and investment should accelerate discovery. Theories of technological growth instead locate the difficulty of discovery in the space of ideas. Here we show that neither scale nor space is enough: breakthroughs depend on diverse attention, which collective concentration can deplete. Across 36 years of computer science, we analyze 7.2 million papers, 73,089 peer reviews and 3.7 million researchers and their topic shifts. The field expanded while its attention narrowed sharply after the Transformer. Over half of topic-shifting researchers moved into AI, especially mid-career ones, and reviewer confidence fell by a full point on a 5-point scale. Each successive breakthrough, from AlexNet to large language models, raised the field's carrying capacity for ideas but was followed by steeper exhaustion of those ideas. Bigger breakthroughs capture attention more completely, depleting the diversity their successors may draw upon. Structural biology shows the same cycle after cryo-EM and AlphaFold. Simulations show how well-intentioned and widely used policies such as reciprocal reviewing empower prolific authors and further narrow what the field accepts. We call for institutions that protect diverse attention in artificial intelligence rather than reward concentration on the winning paradigm.

Closest papers

Nearest neighbours by meaning across the whole index.

IdeaTaste: Do LLMs Share the Research Taste of Human Researchers?

cs.CL ·2026-11-02 ·Shanshan Zhong, Chen Xu, Jingjie Ning, Zhengzhong Liu, Aviral Kumar +1 more
benchmarkcommunity pick similarity 0.69
Abstract

Research taste, the judgment of what to read, which problems to pursue, and which methods to try, increasingly shapes the work of AI scientists, yet it remains far less understood than their ability to execute research. We introduce IdeaTaste, a benchmark that scores an LLM by how closely the research proposals it writes from a literature snapshot frozen at 2025 match the work the same communities published in 2026. Our experiments span six contemporary language models and four research areas from established to emerging fields, covering more than 90{,}000 proposals in total. LLM agents fall well short of human communities in both the literature they cite and the ideas they state, reaching about 26\% of the human level on citation matching and only 18\% on idea matching. Their populations can cite as diversely as a community and take up its tasks and methods, yet rarely arrive at the concrete approaches and research plans it pursues. Neither richer field context nor larger populations substantially close this gap. In blinded expert review, AI proposals trail the 2026 papers least in presentation and most in expected success, echoing their gap in concrete approaches, and those whose ideas no human paper shares fare worst, suggesting weaker directions rather than overlooked opportunities. These results indicate that the main bottleneck in AI research taste lies less in finding the directions a community works on than in turning them into the concrete approaches it goes on to pursue. We release IdeaTaste as an open, continuously updated leaderboard (https://cxcscmu.github.io/IdeaTaste) that benchmarks contemporary AI proposals against future publication cycles.

0.28

Large Language Models Extend Known Research But Do Not Create New Research Directions

cs.AI ·2026-10-12 ·Zhihua Liu, Kurt Butler, Damian Machlanski, Danilo Bueno, Sotirios A. Tsaftaris
similarity 0.69
Abstract

The success of autonomous research systems relies on the quality of the scientific hypotheses they generate. Recent studies report that human reviewers often rate LLM-generated ideas as more novel than those from human experts, raising hope that AI can accelerate scientific discovery. Yet it is unclear whether these models can do more than extend existing knowledge. Can LLMs drive genuine innovation by revealing new research directions, or are they limited to repeating and recombining what science already knows? This unresolved question lies at the heart of discussions about AI and the future of science. To address this, we distinguish two types of novelty: distance, which measures how far a hypothesis lies from existing work, and direction, which captures whether the hypothesis pursues new lines of inquiry beyond those already present. These two measures diverge sharply for LLM-generated work. We represent scientific papers in a shared conceptual space using a pretrained scientific-paper embedding model. Within this space, we embed 26.9 million human-authored papers across 1,325 research topics in six natural-science fields and 524,308 LLM-generated hypotheses from eight models. Distance is measured using existing metrics. To quantify direction, we introduce the orthogonal novelty score (ONS). LLMs consistently generate ideas more distant from existing work than human-authored papers, up to 16\% further when the LLM is given the papers farthest from the centre of the topic. However, LLM hypotheses have a 15--23\% lower median ONS than published papers from the same topic. Thus, LLMs move further from what is already known, but remain within established directions. This pattern persists across changes in various configurations. As a result, LLMs excel at extending the frontier but the creation of new directions is still led by human scientists.

0.09

Dataset repurposing and disruptive AI research

cs.CY ·2026-09-15 ·Yulin Yu, Yong-Yeol Ahn, Daniel M. Romero
similarity 0.66
Abstract

Technological advancements are enabling increasingly systematic and large-scale data collection across all areas of science, driving scientific innovation. In particular, AI research exemplifies this trend, having advanced rapidly through the assembly of massive datasets used to train and evaluate machine learning models. However, the escalating demand for data, the difficulty of creating high-quality datasets, and the exhaustion of easily accessible data sources in AI research raise important questions about how to maximize the value of existing datasets through recombination and repurposing. Here, we draw on two theoretical frameworks---recombinational novelty and transformational creativity---to examine the practice of data repurposing and its scientific impact. Focusing on AI, we analyze scientific outcomes associated with data repurposing across more than 10,000 machine learning papers. First, we find that although most repurposed datasets do not achieve broad visibility in the short term, data repurposing is associated with greater disruption. Second, when repurposed data is adopted by subsequent research, the repurposing paper is associated with higher disruption and increased citation impact. Third, repurposing teams tend to be more experienced, more institutionally prestigious, and involve academic--industry collaboration. However, team characteristics poorly predict which repurposed datasets will be adopted by the community. These findings suggest that data repurposing may be an important approach to scientific discovery, and that its successful adoption is more common among larger teams and collaborations spanning academia and industry.

0.23

Are We Just Applying LLMs? What SE Research Studies, Underexplores, and Could Do Next in the LLM Era

cs.SE ·2026-11-03 ·Zeinadsadat Saghi, Sadra Sabouri, Souti Chattopadhyay
similarity 0.66
Abstract

Is AI taking over software engineering research? By 2025, AI appeared in more than half of the research-track papers at ICSE, FSE, and ASE. Yet ubiquity alone cannot show whether AI expanded the publication portfolio, displaced other work, or redirected the intellectual agenda. We investigate this shift through a census of 4,790 research papers (three SE conferences; 2018--2026) and free-form responses from 244 SE researchers discussing the future of the SE research community (ICSE 2026 FoSE survey). We found that while AI is taking over, it is not simply replacing non-AI research; it is unevenly reorganizing SE research around its capabilities. AI accounted for 93.1% of publication growth, but expansion in some venues and problem areas concealed contraction in others. The LLM wave also changed the field's tools more than its problem boundaries: 92.6% of LLM-wave AI papers revisit established SE problems, while most new problems arise from the introduction of LLMs in the SE process themselves. Engagement with AI remains predominantly downstream: 95.7% of AI papers apply AI or evaluate and support its use, while only 4.3% interrogate the paradigm or develop alternatives. The FoSE survey reveals a similar imbalance between recognition and response. AI-driven change was the second-most frequent concern about the field's future 20.2%, yet only 7.0% of responses mentioned AI when researchers were asked to identify one change the community should make. Together, our findings reveal a field reorganizing an inherited agenda around AI faster than it is questioning that reorganization. We discuss how SE can respond by building cumulatively across technological waves, strengthening evaluation and assurance, developing SE-native foundations, and studying software engineering in an AI-transformed world.

0.32

Scientist World: Grow a Research World for AI-Scientists

cs.AI cs.CLcs.LG ·2026-11-05 ·Taiming Lu, Sachin Konan, Zhuang Liu
similarity 0.64
Abstract

AI-Scientists arrived, but a world for them to grow in did not. Human scientists take theirs for granted $-$ centuries of open problems and shared ideas, waiting in the scientific literature. To AI-Scientists, that world is readable but not runnable, so we build Scientist World for them: every research paper becomes an executable environment. Our autonomous pipeline extracts each paper's open-ended research question and reproduces its proposed method as the baseline; an AI-Scientist entering the environment is evaluated against what the human authors achieved. We release the first 1,000 verified environments. With the world as an evaluation, we found that frontier AI-Scientists can already surpass the human researchers 74% of the time. As a record of how AI does science, it contains 6,400 hours of research traces, with every idea, experiment, failure, and success. As a training ground, we are able to raise (+158%) a small model to match frontier models in research ideation. At last, AI-Scientists' own world has arrived, grown from and growing with science itself.

0.10

Science Utopia? Closed-Loop LLM Simulation of Academic Research Ecosystems

cs.CL cs.AIcs.CY ·2026-10-01 ·Yiqiao Jin, Yiyang Wang, Lucheng Fu, Bing He, Siheng Xiong +6 more
code/data similarity 0.64
Abstract · https://ahren09.github.io/ScienceUtopia/

Scientific progress emerges from a longitudinal ecosystem in which researchers, institutions, funding agencies, collaboration networks, and the scientific literature co-evolve. As AI becomes increasingly involved throughout the scientific research cycle, understanding these interconnected and evolving processes becomes increasingly important. We introduce SciUtopia, a persistent, closed-loop LLM-agent simulation framework for studying academic research ecosystems. SciUtopia models interconnected scientific processes such as research-direction choice, collaboration, submission, peer review, resubmission, citation, funding, and researcher attrition, while maintaining evolving states across simulated years. Its configurable institutional mechanisms and information channels provide a controlled testbed for matched counterfactual experiments and targeted interventions. Across 61 simulation worlds, SciUtopia simulates over 40,000 researchers from 8,000 institutions, producing around 400,000 publication decisions and 1.2 million LLM-generated peer reviews. Using these longitudinal simulations, we find that rejection-driven resubmission substantially amplifies reviewer burden beyond population growth alone, cautious exploration balances citation impact with career success and long-term topic diversity, and resource inequality can emerge even without detectable cumulative advantage from narrowly winning early funding. Code is available at https://github.com/Ahren09/ScienceUtopia.

0.27

Filling holes in science draws collective attention, but most higher-order holes remain unexplored

cs.CY cs.DLcs.SI ·2026-08-28 ·Jiajie Luo, James A. Evans
similarity 0.63
Abstract · 26 pages of main text, 155 with all the supplemental information. 4 main figures, many more in the supplemental information

Most scientific discoveries justify their significance by filling holes between existing ideas and arguments. Representing knowledge as high-dimensional concept embeddings, we use persistent homology to detect scientific holes of increasing order, from lower-order gaps between disconnected ideas to higher-order cavities within unevenly connected ones; then we identify the research that fills them. Researchers who close holes draw outsized collective attention, staging novelty that the citing literature rewards, most strongly in discovery-oriented fields and least in design-oriented fields. Closing a hole is followed by persistent work on the same combinations of concepts at many times the rate of comparable holes left open. Yet as knowledge grows, higher-order holes explode while the fraction of research that fills them collapses, leaving most higher-order combinations unexplored. These results call for a richer science of holes and mark a frontier where contemporary AI might help fill the gaps human science opens.

0.15

The Emerging AI Paper-Review Arms Race: Adversarial Co-Evolution in Scholarly Publishing

cs.AI cs.CL ·2026-09-07 ·Chenguang Wang, Ming Li, Adebayo Braimah, Chenrui Fan, Tuo Wang +4 more
positionhighly cited similarity 0.63
Abstract

Generative and agentic AI are reshaping both the production and evaluation of scientific research. These developments are often studied separately, as questions of how AI can produce research and how AI can review it. We argue that this separation misses an increasingly important feature of scholarly publishing: changes on one side alter the incentives, constraints, and behavior of the other. We synthesize 230 scholarly publications and institutional records using a taxonomy of six connected dynamics: production scaling, evaluation automation, evaluation manipulation, defense mechanisms and policy responses, evasion and side effects, and long-horizon ecosystem feedback. The literature shows an emerging progression in which cheaper and faster research production increases pressure on evaluation, AI-mediated evaluation becomes more scalable and repeatable, participants can exploit evaluator regularities, and institutions respond with technical safeguards and policy controls. These responses can in turn induce evasion, redistribute errors and workload, and shape the scholarly records reused by future research and evaluation systems. Evidence is strongest for production and evaluation at scale, reproducible manipulation, and institutional response, while post-policy adaptation and artifact-level long-horizon feedback remain less directly observed. This systems view shifts attention from isolated AI capabilities toward how scholarly actors and AI systems adapt to one another over time.

0.52