CS Paper Radar

Modeling Authentic Research Trajectories: A Graph View of Research Ideation

Ruochen Li, Zhiyu Chen

cs.AI ·submitted 2026-10-19 22:00 UTC · v1

Abstract

The research idea in a published paper is rarely the project's initial idea. Between the two lies a closed-loop process in which researchers run experiments, derive insights, identify challenges, explore the literature, and revise the idea, with each step motivated by evidence and reasoning they can articulate. We call the path one idea takes through this process, with the evidence and reasoning behind each step, its authentic research trajectory. Although it underlies every scientific contribution, it is rarely recorded. Existing LLM-based research agents generate an idea once, refine it against reviews, literature, or their own experimental results, or search over candidate ideas and experiments. In all of them the text of the idea evolves, while the evidence and reasoning that connected one version to the next are discarded. We propose the Research Progress Graph (RPG), a representation of authentic and complete research trajectories. An RPG records the evolution of an idea as typed research states connected by cognitive transitions, each annotated with a temporal index and the reasoning behind it, which makes this evolution an explicit object of study. We ground the representation in authentic trajectories whose steps their authors justify in annotations and papers. Comparing the representations of thirty-five research agents with the RPG, we find that each is an incomplete RPG; none keeps the state that changed, the evidence that changed it, and the reasoning connecting them as one evolving record. Modeling ideation with the RPG changes how research agents should be designed, evaluated, and trained. Agents predict the next research state instead of a complete idea, ideas are evaluated as trajectories instead of final texts, and authentic trajectories serve as training data. These directions shift research agents from generating static ideas to modeling how ideas evolve.

Comment
A research progress graph

Closest papers

Nearest neighbours by meaning across the whole index.

Learning Scientific Exploration from Human Research Decision Trajectories

cs.LG cs.CL ·2026-10-05 ·Xuchen Gong, Shane Gu, Haokun Liu, Dixi Yao, Chenhao Tan +1 more
dataset similarity 0.74
Abstract

A key challenge in building AI systems for scientific research is enabling $\textit{scientific exploration}$: the systematic process of investigating unknown phenomena or ideas to gain new knowledge through sequences of research decisions and actions. Yet this process is largely missing from existing scientific corpora; for example, research papers primarily record final outcomes rather than the trajectories that produced them. In this work, we introduce $\textbf{ResearchTrails}$, a dataset of $\textbf{human research trajectories constructed from Git repositories}$, where $\textbf{commit histories}$ serve as proxies for research exploration. We develop an automated and scalable pipeline that extracts structured research trajectories from repository commits, capturing successive changes to methods, experiments, and ablations. We characterize the resulting dataset and show that these trajectories contain meaningful signals about intermediate research decisions beyond what final papers reveal. We further demonstrate utilities of ResearchTrails in multiple use cases, including retrieving human research experience as external skills at test time and training models on research trajectories to improve generalization to new research decisions. Our results suggest a path toward AI systems that learn not only from the products of science, but from the evolving process of discovery itself.

0.32

Tree-of-Ideas: Automated Research Ideation via Cross-Trajectory Reasoning over Scholarly Evolution

cs.AI ·2026-08-11 ·Xun Li, Yiying Yang, Pengtao Li, Xiao Yao, Suyu Liu +6 more
similarity 0.72
Abstract

Effective research ideation requires moving beyond a static understanding of prior work to trace how research problems and solutions evolve across the literature. Existing methods either treat papers as unstructured context or model scholarly evolution as isolated citation chains, overlooking interactions among research trajectories. We propose Tree-of-Ideas (ToI), a two-stage framework. EvoTrace reconstructs branching scholarly trajectories from citations, tracking evolving methods, resolved problems, and gaps. EvoAgent then reasons across trajectories to identify convergent problems and complementary solutions, generating grounded research ideas. Across six AI research topics, ToI achieves the highest score among automatic methods (6.27 vs. 5.36 for the strongest baseline on a 10-point scale), with strong Novelty (6.36) and Groundedness (7.00). Also, its score approaches that of human-paper references (6.29), demonstrating the value of cross-path evolutionary reasoning.

0.34

IdeaScientist: Orchestrating Agents for Grounded Scientific Ideation

cs.CL ·2026-10-02 ·Jiarui Liu, Renjie Tao, Yiwei Liao, Chuanyang Jin, Kai Sun +13 more
code/data similarity 0.71
Abstract

Despite rapid progress in automating scientific research, generating promising and well grounded research solutions remains a central challenge. We isolate research ideation as a standalone task and build our solution on the intuition that a challenge in one field can often be addressed by a mechanism that solved an analogous challenge in another. Accordingly, we introduce IdeaScientist, which decomposes ideation into gap finding, innovation, and report writing, and trains each role with reinforcement learning. These roles identify limitations in related work, draw solution intuitions from analogous problem settings, and develop those intuitions into complete research proposals. To facilitate discovery of insights across domains, we construct the Svalbard Idea Vault, a corpus of 2.77M decomposed research ideas for retrieval, training, and temporally controlled evaluation. Our evaluation restricts access to literature available before a cutoff date and assesses how closely proposed directions align with those later explored in 15K papers authored by human researchers. On Qwen3.6-27B, IdeaScientist outperforms the strongest open-source autoresearch baseline by 14.0%, driven mainly by gains in novelty. On this 27B open backbone, IdeaScientist even outperforms Claude Code SDK with Claude-4.8-Opus and Codex SDK with GPT-5.4, by up to 5.9%.

0.52

Idea Traces: Threading Accretive Content across Scientific Papers

cs.HC ·2026-10-28 ·Luke S. Snyder, Jeffrey Heer
similarity 0.69
Abstract · 15 pages, 9 figures, 1 table

We introduce idea traces, a novel thread-based representation of accretive content across scientific papers. Rather than capturing fixed themes from papers, idea traces surface similarities and deltas of faceted content that enable inspection of how ideas relate and evolve across the citation network. To extract traces, we contribute an LLM pipeline that deconstructs salient transitions between pairs of papers and collates the output into an idea dependency graph. We leverage idea traces in a visual system that supports idea-driven navigation starting from a seed paper, lifting the "plane" of synthesis from papers to ideas. To evaluate the usefulness and quality of generated ideas, we provide a case study that compares our approach with an LLM Q&A workflow, and interview 10 researchers who used our system and annotated 110 ideas. Experts found that our approach helped extract a useful space of connections that would have been difficult to make otherwise. Drawing on their feedback, we outline several promising directions for future research.

0.13

Can Frontier Auto Researchers Reconstruct Human Research Decisions in AI?

cs.CL ·2026-11-09 ·Shengyun Si, Junjie Chu, Ye Leng, Yang Zhang
similarity 0.67
Abstract · 14 pages, 18 figures

Scientific research has long been driven by substantial human effort, from reviewing prior work and identifying research gaps to designing specific solutions. Recent advances in harness-based agent systems have given rise to increasingly capable Auto Researchers, which can autonomously carry out these tasks and, increasingly, entire end-to-end research workflows. Yet despite their rapidly growing capabilities, how closely the research decisions and outputs produced by auto researchers align with those of human experts remains underexplored. We investigate this gap through the lens of research reconstruction and study: Given comparable research context, how similar are the research decisions and outputs produced by Auto Researchers to those of human researchers in AI research? To answer this question, we introduce AutoResearcher-Human Research Similarity (AutoHRS), a stage-wise evaluation framework for assessing how faithfully auto researchers reconstruct human research artifacts across different stages of the scientific workflow. Using 152 ICLR oral papers as our corpus, we generate 456 research trajectories spanning different stages of the research process. Applying AutoHRS to these trajectories, we find that current Auto Researchers frequently fail to recover key research decisions and their resulting artifacts across multiple stages of the research process.

0.21

Distill, Debate, Discover: Grounding Multi-Agent Research Ideation in Graph-Distilled Insights

cs.CL ·2026-10-23 ·Qianran Ma, Muxin Liu, Grace Kim, Xiao Liang, Chris Callison-Burch
similarity 0.65
Abstract

Large language models (LLMs) are increasingly used to support research idea generation and scientific discovery. Multi-agent debate offers a collaborative approach to this task by bringing together diverse perspectives to develop research ideas. However, a key challenge is how to equip agents with distinct and meaningful expertise to guide their contributions. Inspired by how researchers draw on accumulated research experience during collaboration, we introduce InsightAgents, a framework that constructs literature-grounded agent expertise by distilling insights from graphs of related papers. Guided by insights that trace evolving research approaches, reveal underlying assumptions, and identify unresolved challenges, within a line of research, agents jointly identify a research problem and propose, challenge, and refine potential approaches through debate to develop a research proposal. We evaluate InsightAgents against three research idea generation frameworks on 100 NLP research topics through pairwise comparisons by two LLM judges. In both online and offline knowledge settings, InsightAgents achieves the highest average score across the five evaluation dimensions under both judges.

0.17

Science Utopia? Closed-Loop LLM Simulation of Academic Research Ecosystems

cs.CL cs.AIcs.CY ·2026-10-01 ·Yiqiao Jin, Yiyang Wang, Lucheng Fu, Bing He, Siheng Xiong +6 more
code/data similarity 0.65
Abstract · https://ahren09.github.io/ScienceUtopia/

Scientific progress emerges from a longitudinal ecosystem in which researchers, institutions, funding agencies, collaboration networks, and the scientific literature co-evolve. As AI becomes increasingly involved throughout the scientific research cycle, understanding these interconnected and evolving processes becomes increasingly important. We introduce SciUtopia, a persistent, closed-loop LLM-agent simulation framework for studying academic research ecosystems. SciUtopia models interconnected scientific processes such as research-direction choice, collaboration, submission, peer review, resubmission, citation, funding, and researcher attrition, while maintaining evolving states across simulated years. Its configurable institutional mechanisms and information channels provide a controlled testbed for matched counterfactual experiments and targeted interventions. Across 61 simulation worlds, SciUtopia simulates over 40,000 researchers from 8,000 institutions, producing around 400,000 publication decisions and 1.2 million LLM-generated peer reviews. Using these longitudinal simulations, we find that rejection-driven resubmission substantially amplifies reviewer burden beyond population growth alone, cautious exploration balances citation impact with career success and long-term topic diversity, and resource inequality can emerge even without detectable cumulative advantage from narrowly winning early funding. Code is available at https://github.com/Ahren09/ScienceUtopia.

0.27

IdeaAnchor: Teaching LLMs to Turn Literature into Research Ideas

cs.CL cs.AI ·2026-10-06 ·Ziyu Chen, Yilun Zhao, Jiashuo Sun, Yiling Ma, Manasi Patwardhan +1 more
similarity 0.65
Abstract

Scientific research often begins by synthesizing ideas from a set of related papers to identify gaps and formulate new directions. However, training language models to perform this form of literature-grounded ideation remains challenging, as existing approaches based on prompting or feedback lack structured supervision for how papers should be synthesized. We introduce IdeaAnchor, a paradigm for training LLMs to perform research ideation using structured specifications as privileged signals. Each IdeaAnchor instance encodes how each input paper should be synthesized into a successful idea, including their functional roles, relationships, and target synthesis criteria. We build this paradigm by mining instances from published papers, capturing how real ideas emerge from prior literature. We then train models via demonstration, self-distillation, and reinforcement learning, and further enhance generation with retrieval at inference time. Experiments show consistent improvements in ideation quality. Our analysis reveals a functional decomposition: anchor-based training strengthens creative synthesis, retrieval enhances detail elaboration, and combining both yields the best performance.

0.13