CS Paper Radar

Science Utopia? Closed-Loop LLM Simulation of Academic Research Ecosystems

Yiqiao Jin, Yiyang Wang, Lucheng Fu, Bing He, Siheng Xiong, Yijia Xiao, B. Aditya Prakash, Josiah Hester, Srijan Kumar, James Evans, Jindong Wang

cs.CL cs.AIcs.CY ·submitted 2026-10-01 07:51 UTC · v1
code/data

Abstract

Scientific progress emerges from a longitudinal ecosystem in which researchers, institutions, funding agencies, collaboration networks, and the scientific literature co-evolve. As AI becomes increasingly involved throughout the scientific research cycle, understanding these interconnected and evolving processes becomes increasingly important. We introduce SciUtopia, a persistent, closed-loop LLM-agent simulation framework for studying academic research ecosystems. SciUtopia models interconnected scientific processes such as research-direction choice, collaboration, submission, peer review, resubmission, citation, funding, and researcher attrition, while maintaining evolving states across simulated years. Its configurable institutional mechanisms and information channels provide a controlled testbed for matched counterfactual experiments and targeted interventions. Across 61 simulation worlds, SciUtopia simulates over 40,000 researchers from 8,000 institutions, producing around 400,000 publication decisions and 1.2 million LLM-generated peer reviews. Using these longitudinal simulations, we find that rejection-driven resubmission substantially amplifies reviewer burden beyond population growth alone, cautious exploration balances citation impact with career success and long-term topic diversity, and resource inequality can emerge even without detectable cumulative advantage from narrowly winning early funding. Code is available at https://github.com/Ahren09/ScienceUtopia.

Comment
https://ahren09.github.io/ScienceUtopia/

Closest papers

Nearest neighbours by meaning across the whole index.

The Emerging AI Paper-Review Arms Race: Adversarial Co-Evolution in Scholarly Publishing

cs.AI cs.CL ·2026-09-07 ·Chenguang Wang, Ming Li, Adebayo Braimah, Chenrui Fan, Tuo Wang +4 more
positionhighly cited similarity 0.70
Abstract

Generative and agentic AI are reshaping both the production and evaluation of scientific research. These developments are often studied separately, as questions of how AI can produce research and how AI can review it. We argue that this separation misses an increasingly important feature of scholarly publishing: changes on one side alter the incentives, constraints, and behavior of the other. We synthesize 230 scholarly publications and institutional records using a taxonomy of six connected dynamics: production scaling, evaluation automation, evaluation manipulation, defense mechanisms and policy responses, evasion and side effects, and long-horizon ecosystem feedback. The literature shows an emerging progression in which cheaper and faster research production increases pressure on evaluation, AI-mediated evaluation becomes more scalable and repeatable, participants can exploit evaluator regularities, and institutions respond with technical safeguards and policy controls. These responses can in turn induce evasion, redistribute errors and workload, and shape the scholarly records reused by future research and evaluation systems. Evidence is strongest for production and evaluation at scale, reproducible manipulation, and institutional response, while post-policy adaptation and artifact-level long-horizon feedback remain less directly observed. This systems view shifts attention from isolated AI capabilities toward how scholarly actors and AI systems adapt to one another over time.

0.52

Winning Reviews, Losing Discovery? The Co-Evolutionary Game of AI Research and Peer Review

cs.GT econ.GN ·2026-11-07 ·Hongze Liu, Mengru Wang, Zhiyuan Zhang, Yao Liu
similarity 0.68
Abstract

Autonomous research systems can produce convincing manuscripts whose scientific value is difficult to verify, while automated reviewers can reward the appearance of completeness. We examine how this gap can become a self-reinforcing threat to the research community. In a two-population evolutionary game, authors choose between acceptance-oriented and exploratory research, and reviewers choose between proxy evaluation and careful assessment. Cheaper acceptance-oriented production can increase review pressure, making proxy assessment more attractive; proxy assessment can in turn strengthen the incentive to produce reviewer-compatible work. We derive conditions under which this feedback sustains a stable regime that rewards publication success while reducing the scientific value of accepted work. An analytical basin boundary explains why the same institutional rules can support either this regime or exploratory research with careful review. Numerical illustrations show how publication counts can conceal scientific loss, how faster adjustment can reinforce existing practices, and why partial reform may fail once the cycle is established. Careful audits and reviewer rewards can reverse the feedback, while temporary support can produce lasting recovery if it moves the community across the basin boundary. The model identifies a warning mechanism rather than forecasting a particular conference: a review system can remain operational and privately rewarding while becoming less effective at supporting discovery. Its institutional implication is to reward independently validated contributions and consequential checking, rather than accelerate paper production and scoring alone.

0.06

Scientific Liquidity Agents: An Agent-Based Model of Knowledge Arbitrage in Scientific Communities

cs.MA cs.AIcs.SI ·2026-11-02 ·Koutian Wu, You Liu, Yue Liu, Zhiping Li, Ergan Shang
code/data similarity 0.66
Abstract · First version on Zenodo, December 8, 2025: https://zenodo.org/records/17861999

Scientific knowledge gains new uses when ideas, methods, and results move between research contexts. Understanding this process requires a model of both specialized knowledge production and the intermediary activities that connect it. We introduce Scientific Liquidity Agents (SLA), an agent-based model of knowledge arbitrage in scientific communities. SLA represents domain specialists and epistemic arbitrageurs within a shared environment of research problems, researcher interactions, and scientific artifacts. Individual decisions connect to collective state through a belief-desire-intention architecture, a prerequisite-based problem graph, a dynamic interaction network, and a persistent publication registry. The framework makes researcher roles and knowledge-transfer frictions explicit, providing a process-oriented account of scientific production and exchange. Two exploratory simulations with 10 and 18 agents produce 68 and 274 registered papers. In the smaller run, arbitrageurs take part in 16 of the 26 contact pairs that join this community into one connected component, while topical similarity alone splits it into seven. The same run yields three similarity edges, 26 contact pairs, and two coauthorship pairs, so measured community structure depends on what counts as a link. Paper production is abundant, but no recorded replication succeeds, which places the bottleneck in verification rather than production. Solved-problem sets correspond to the prerequisite reachability of the disciplines represented in each community. SLA makes knowledge liquidity, the movement and reuse of artifacts across domains, a quantity that agentic science can simulate and measure.

0.27

Modeling Authentic Research Trajectories: A Graph View of Research Ideation

cs.AI ·2026-10-19 ·Ruochen Li, Zhiyu Chen
similarity 0.65
Abstract · A research progress graph

The research idea in a published paper is rarely the project's initial idea. Between the two lies a closed-loop process in which researchers run experiments, derive insights, identify challenges, explore the literature, and revise the idea, with each step motivated by evidence and reasoning they can articulate. We call the path one idea takes through this process, with the evidence and reasoning behind each step, its authentic research trajectory. Although it underlies every scientific contribution, it is rarely recorded. Existing LLM-based research agents generate an idea once, refine it against reviews, literature, or their own experimental results, or search over candidate ideas and experiments. In all of them the text of the idea evolves, while the evidence and reasoning that connected one version to the next are discarded. We propose the Research Progress Graph (RPG), a representation of authentic and complete research trajectories. An RPG records the evolution of an idea as typed research states connected by cognitive transitions, each annotated with a temporal index and the reasoning behind it, which makes this evolution an explicit object of study. We ground the representation in authentic trajectories whose steps their authors justify in annotations and papers. Comparing the representations of thirty-five research agents with the RPG, we find that each is an incomplete RPG; none keeps the state that changed, the evidence that changed it, and the reasoning connecting them as one evolving record. Modeling ideation with the RPG changes how research agents should be designed, evaluated, and trained. Agents predict the next research state instead of a complete idea, ideas are evaluated as trajectories instead of final texts, and authentic trajectories serve as training data. These directions shift research agents from generating static ideas to modeling how ideas evolve.

0.14

Eliciting Research Taste in LLMs through Future Research Direction Choice

cs.LG cs.AIcs.CL ·2026-11-15 ·Dhruv Trehan, Paras Chopra
code/data similarity 0.65
Abstract · Accepted at the LM4Sci Workshop at COLM 2026. 23 pages, 7 figures, 18 tables. Code and data: https://github.com/Lossfunk/research-taste-eval…

Research taste is increasingly discussed in the context of AI scientists, yet remains little studied or measured in LLMs. This judgment matters especially in open-ended research, where a direction must be chosen before a verifiable objective is available. We operationalize research taste through one task of the scientific workflow: choosing which future research direction to pursue given a research goal and current results. Rather than predicting downstream scientometric metrics or treating author choices as objectively correct, we evaluate models against human-authored future-work directions and report author alignment, stable divergence, and unstable choice as behavioral measures of research-direction preference. We construct an order-controlled evaluation of 80 pairwise comparisons from 19 ML papers published between December 2025 and February 2026 across four subdomains. Each query presents a research goal, hypotheses tested with key results, and two candidate future directions: one author-recommended direction and one plausible distractor sampled from a similar paper. Across eight LLMs, author alignment ranges from 45.0% to 67.5%, while stable divergence ranges from 15.0% to 36.2%. These divergences largely persist under similarity controls, prompt ablations, and an author-preference framing check, and inter-model agreement suggests that some non-author preferences are shared across models. We characterize what models diverge toward using transition-category labels and selected reasoning examples, finding recurring movement toward scaling/generalization and trade-offs between methodological and infrastructure/data extensions. We view this as an early descriptive evaluation for eliciting research taste profiles in LLMs, and as a step toward understanding and steering their research preferences as they take on more autonomous roles in novel open-ended scientific discovery.

0.36

SocioVerse2: A Longitudinal Dynamic Social Simulation Framework under a Human-AI Co-evolutionary Paradigm

cs.CL cs.CY ·2026-09-21 ·Xinnong Zhang, Jiayu Lin, Jia Wang, Yixu Huang, Xinyi Mou +17 more
code/data similarity 0.65
Abstract · Project page: https://socioverse.fudan-disc.com/

Social simulation offers the social sciences an experimental instrument that the real world cannot supply, and generative agents have transformed it by acting as silicon samples that unite agent-based modeling with real behavioral data. Existing platforms verify collective behavior, align simulated populations with real societies in cross-sections, and employ autonomous agents for the research process. However, two social science requirements remain without systematic support: intervention in the content of a simulation and the researcher's control over the process that produces it. We present SocioVerse2, which extends SocioVerse 1.0 into a human-AI co-evolutionary paradigm built from two loops and one infrastructure. The longitudinal simulation loop simulates the target population with evolving environments and forks counterfactual branches via interventions. The controllable research loop takes the study itself as an editable state and updates state versions via controllable editing. The social science agentic infrastructure carries both loops through composable skills with researcher checkpoints, a population service over five persona pools, and an environment service over 21 real-world signal sources with point-in-time guarantees. We validate SocioVerse2 across three case families and seven case studies, from reproducing canonical agent-based models to modeling policy processes on real records and nowcasting macro-economic indices beyond the response model's knowledge cutoff. With the human-AI co-evolutionary paradigm, these cases go beyond system demonstrations to become substantive studies that investigate frontier questions in their respective disciplines. Code, data services, and a workbench are released as open-source resources.

0.27

Stronger artificial intelligence breakthroughs hasten their own exhaustion

cs.AI cs.CYecon.GN ·2026-10-19 ·Beichen Lu, Honglin Bao, Qi Huang, Qingguo Meng, James A. Evans
hot direction similarity 0.64
Abstract

Progress in artificial intelligence is widely assumed to scale with effort: more researchers, papers and investment should accelerate discovery. Theories of technological growth instead locate the difficulty of discovery in the space of ideas. Here we show that neither scale nor space is enough: breakthroughs depend on diverse attention, which collective concentration can deplete. Across 36 years of computer science, we analyze 7.2 million papers, 73,089 peer reviews and 3.7 million researchers and their topic shifts. The field expanded while its attention narrowed sharply after the Transformer. Over half of topic-shifting researchers moved into AI, especially mid-career ones, and reviewer confidence fell by a full point on a 5-point scale. Each successive breakthrough, from AlexNet to large language models, raised the field's carrying capacity for ideas but was followed by steeper exhaustion of those ideas. Bigger breakthroughs capture attention more completely, depleting the diversity their successors may draw upon. Structural biology shows the same cycle after cryo-EM and AlphaFold. Simulations show how well-intentioned and widely used policies such as reciprocal reviewing empower prolific authors and further narrow what the field accepts. We call for institutions that protect diverse attention in artificial intelligence rather than reward concentration on the winning paradigm.

0.27

Inspire: Benchmarking Scientific Literature Search for Open Research Problems

cs.IR cs.AI ·2026-09-27 ·Jianrong Ding, Zhengyan Shi, Jianyuan Zhong, Kai Qiu, Qi Dai +3 more
benchmark similarity 0.63
Abstract

Scientific literature search often begins with an open research problem rather than a known target paper or a fixed candidate set. We introduce INSPIRE, a benchmark for evaluating agents that search prior literature to make progress on solution-redacted research problems. Each instance pairs a research brief with a target-specific cutoff three months before a later paper and evaluates ranked outputs against graded cited antecedents from that paper's realized research lineage. Search proceeds over an open corpus, while the identity of the target paper and membership of its cited antecedents remain hidden from the agent. Beyond end-to-end retrieval quality, INSPIRE uses logged search trajectories to distinguish three coupled stages: resource exposure, whether useful antecedents are surfaced during search; selection, whether exposed antecedents are retained; and ranking, how effectively retained papers are ordered. Across 476 computer-science targets under a shared search interface and budget, the strongest evaluated agent achieves 0.284 nDCG@10. Results show that current agents more readily recover an isolated antecedent than assemble a broader portfolio of relevant prior work. The stagewise analysis identifies resource exposure as the largest observed bottleneck, with further losses in selection and ranking. We additionally construct replay-valid hindsight demonstrations and show that they improve held-out search without changing test-time information, establishing that the benchmark provides an actionable learning signal. INSPIRE therefore enables both end-to-end comparison and stage-resolved diagnosis in a setting where the agent must construct its own working criterion of relevance.

0.12