CS Paper Radar

Matched Starts, Divergent Objects: Research-Object Formation in Human-AI Scholarly Work

Mehmed Zahid Çögenli

cs.HC ·submitted 2026-09-03 23:00 UTC · v2

Abstract

Human-artificial intelligence interaction can shape what scholarly inquiry becomes about. This study examines research-object formation in two isolated scholarly trajectories developed by the same expert researcher with Claude and ChatGPT under matched starting conditions. Both used the same five-session, 843-turn project corpus, frozen research problem, initial prompt, publication objective, and conduct rules. The subsequent scholarly-development conversations were reconstructed ex post through consequential-event analysis, a Socioduality relational-process overlay, and tracing into completed manuscripts. Both trajectories shifted the continuity problem from recall toward usability, then developed different research objects: continuity labour and distributed, evolving project state. Failed analytical units, rejected explanations, counterexamples, and changes in analytical scale shaped these objects and propagated into research questions, methods, findings, evidence logic, and theoretical contributions. Consequential scholarly change, reciprocal relational continuity, and local substantive re-formation had distinct boundaries; continued interaction did not itself establish epistemic endorsement. The findings demonstrate empirically constrained research-object formation under matched scholarly starts. For systems supporting sustained scholarly knowledge work, they motivate recoverable analytical provenance, inspectable proposal status, and source-faithful contribution records.

Comment
v2: Title and abstract revised; human-AI interaction framing, methodological reporting, and discussion clarified; references corrected. Core study design and substantive findings unchanged. 75 pages, including three supplementary documents

Closest papers

Nearest neighbours by meaning across the whole index.

A Stateful Research Information Infrastructure for Scientific Discovery

cs.DL cs.IR ·2026-10-10 ·Lingyuan Kong, Zhengran Wu
similarity 0.63
Abstract · Project Website: https://gewu-insight.com

Artificial intelligence (AI) is being applied to an expanding range of scientific activities, including literature analysis, hypothesis generation, software development and automated experimentation. Yet these capabilities rely on a more fundamental condition: scientific information must remain identifiable and traceable across systems and over time. In practice, different versions of a work and their associated records and files are often scattered across services and local directories. This fragmentation forces researchers to identify and contextualize the same work repeatedly, disrupting continuity from discovery to interpretation and reuse. Here we present Gewu Insight, a research information infrastructure that addresses this discontinuity by organizing scientific works as persistent, stateful research objects. It unifies venue-, topic- and author-based discovery records through a shared paper identity. Within a local library, metadata, versioned resources and research context can then accumulate around the same work. These capabilities are integrated into an iterative workflow for scientist-led, AI-assisted research, allowing earlier interpretations to be revisited and refined. A bounded evaluation showed high candidate precision across the three discovery modes, with paper identity, resource distinctions and research state preserved in controlled workflows. By making continuity an explicit property of research infrastructure, Gewu Insight can serve as an inspectable scientific memory layer shared by researchers and AI assistants, supporting human judgment and providing a foundation for future, more epistemically grounded scientific AI.

0.15

Modeling Authentic Research Trajectories: A Graph View of Research Ideation

cs.AI ·2026-10-19 ·Ruochen Li, Zhiyu Chen
similarity 0.62
Abstract · A research progress graph

The research idea in a published paper is rarely the project's initial idea. Between the two lies a closed-loop process in which researchers run experiments, derive insights, identify challenges, explore the literature, and revise the idea, with each step motivated by evidence and reasoning they can articulate. We call the path one idea takes through this process, with the evidence and reasoning behind each step, its authentic research trajectory. Although it underlies every scientific contribution, it is rarely recorded. Existing LLM-based research agents generate an idea once, refine it against reviews, literature, or their own experimental results, or search over candidate ideas and experiments. In all of them the text of the idea evolves, while the evidence and reasoning that connected one version to the next are discarded. We propose the Research Progress Graph (RPG), a representation of authentic and complete research trajectories. An RPG records the evolution of an idea as typed research states connected by cognitive transitions, each annotated with a temporal index and the reasoning behind it, which makes this evolution an explicit object of study. We ground the representation in authentic trajectories whose steps their authors justify in annotations and papers. Comparing the representations of thirty-five research agents with the RPG, we find that each is an incomplete RPG; none keeps the state that changed, the evidence that changed it, and the reasoning connecting them as one evolving record. Modeling ideation with the RPG changes how research agents should be designed, evaluated, and trained. Agents predict the next research state instead of a complete idea, ideas are evaluated as trajectories instead of final texts, and authentic trajectories serve as training data. These directions shift research agents from generating static ideas to modeling how ideas evolve.

0.14

Tree-of-Ideas: Automated Research Ideation via Cross-Trajectory Reasoning over Scholarly Evolution

cs.AI ·2026-08-11 ·Xun Li, Yiying Yang, Pengtao Li, Xiao Yao, Suyu Liu +6 more
similarity 0.62
Abstract

Effective research ideation requires moving beyond a static understanding of prior work to trace how research problems and solutions evolve across the literature. Existing methods either treat papers as unstructured context or model scholarly evolution as isolated citation chains, overlooking interactions among research trajectories. We propose Tree-of-Ideas (ToI), a two-stage framework. EvoTrace reconstructs branching scholarly trajectories from citations, tracking evolving methods, resolved problems, and gaps. EvoAgent then reasons across trajectories to identify convergent problems and complementary solutions, generating grounded research ideas. Across six AI research topics, ToI achieves the highest score among automatic methods (6.27 vs. 5.36 for the strongest baseline on a 10-point scale), with strong Novelty (6.36) and Groundedness (7.00). Also, its score approaches that of human-paper references (6.29), demonstrating the value of cross-path evolutionary reasoning.

0.34

Can Frontier Auto Researchers Reconstruct Human Research Decisions in AI?

cs.CL ·2026-11-09 ·Shengyun Si, Junjie Chu, Ye Leng, Yang Zhang
similarity 0.61
Abstract · 14 pages, 18 figures

Scientific research has long been driven by substantial human effort, from reviewing prior work and identifying research gaps to designing specific solutions. Recent advances in harness-based agent systems have given rise to increasingly capable Auto Researchers, which can autonomously carry out these tasks and, increasingly, entire end-to-end research workflows. Yet despite their rapidly growing capabilities, how closely the research decisions and outputs produced by auto researchers align with those of human experts remains underexplored. We investigate this gap through the lens of research reconstruction and study: Given comparable research context, how similar are the research decisions and outputs produced by Auto Researchers to those of human researchers in AI research? To answer this question, we introduce AutoResearcher-Human Research Similarity (AutoHRS), a stage-wise evaluation framework for assessing how faithfully auto researchers reconstruct human research artifacts across different stages of the scientific workflow. Using 152 ICLR oral papers as our corpus, we generate 456 research trajectories spanning different stages of the research process. Applying AutoHRS to these trajectories, we find that current Auto Researchers frequently fail to recover key research decisions and their resulting artifacts across multiple stages of the research process.

0.20

From Research Gaps to Theoretical Opportunities: Theory-Oriented GenAI for Research Opportunity Evaluation

cs.CE ·2026-10-05 ·Jujun Huang, Shun Cao
similarity 0.60
Abstract

Generative AI (GenAI) can explore large bodies of literature and generate plausible research ideas, but identifying what is missing, understudied, contradictory, or potentially connected does not by itself reveal where theory should advance. We develop a theory-oriented agentic AI system that helps researchers identify potential theorizing opportunities by incorporating established theorizing approaches into literature exploration and evaluation. The system operates through three stages. Stage 1 expands the theoretical search space and constructs a provisional Candidate Knowledge Graph. Stage 2 independently reconstructs what the literature supports through source grounded evidence extraction and theory-state reconstruction. Stage 3 evaluates the reconstructed knowledge state to determine whether an unresolved configuration warrants theory development or another research action and, when theory development is warranted, which theorizing approach is appropriate. We demonstrate the system through an end-to-end analysis of human oversight of agentic AI systems in organizations. The analysis shows that literature gaps alone are insufficient for identifying theoretical opportunities. For example, "transparency to trust" is routed to mechanism-based theorizing because the relationship is repeatedly documented in prior studies, while the generative mechanism explaining how transparency shapes trust remains insufficiently specified. By combining large-scale literature processing, structured knowledge representation, and theorizing-guided diagnosis, the system serves as a theory-oriented research assistant that supports researchers in identifying theoretically meaningful directions for subsequent research.

0.17

HiPaper: A Human-AI Collaborative System for Academic Literature Triage and Scholarly Sensemaking

cs.HC cs.AIcs.IR ·2026-10-31 ·Ye Jia, Pangjing Wu, Hayley Y. W. Gu, Zhuohang Jiang, Xu Yuan +4 more
similarity 0.60
Abstract

Scholarly sensemaking requires researchers to iteratively retrieve, collect, and synthesize literature as their understanding evolves. However, existing scholarly tools provide only partial support for this process. Recent artificial intelligence (AI)-based systems generally rely on opaque submit-and-wait pipelines that it users' ability to inspect, revise, and validate intermediate outputs. In this study, we present HiPaper, a human-AI collaborative literature review system informed by a formative study with academic experts. HiPaper organizes scholarly sensemaking as a three-stage interactive workflow spanning Retrieval, Collection, andSynthesis, centralized on the human-in-the-loop paradigm that exposes intermediate AI-generated content for user inspection and intervention through a persistent workspace called Knowledge Pocket. Through this workflow, users can iteratively refine search intent, curate and organize papers with AI-enriched metadata and clustering, and conduct evidence-traceable synthesis over selected literature. Our user study with 14 Ph.D. students showed that HiPaper efficiently supports scholarly sensemaking by enabling human-AI collaboration in both familiar and unfamiliar domains. Furthermore, the findings highlight how staged interactions with intermediate AI-generated content successfully scaffold user interactions throughout the sensemaking process.

0.05

Gricea: An Open Science Platform for Conversational AI Research

cs.HC cs.AI ·2026-09-18 ·Nikhil Sharma, Yunlin Gong, Xinyang Cheng, Ziang Xiao
similarity 0.60
Abstract · 19 pages, 3 figures, 4 tables. Pre-print

We need studies on conversational AI (CAI) at scale to understand human behavior and shape CAI design. However, fragmented reporting of systems and study configurations hinders replication, extension, and knowledge accumulation. We present Gricea, an open-science platform representing studies as configurable, deployable research artifacts that researchers can run, inspect, share, and reuse. Informed by a formative analysis of prior CAI research, Gricea couples study procedures, participant-facing systems, and conversational task behavior in. In a replication study using Gricea, we replicated configurations 93% of eligible CUI 2026 papers; while also flagging missing information in 96% of papers that hinder faithful replication --- further motivating Gricea's need. In a user study, researchers and practitioners from diverse backgrounds successfully constructed runnable studies addressing various open-ended research questions. Together, these findings demonstrate Gricea's support for constructing, reproducing, and extending CAI studies through shared research artifacts, enabling cumulative knowledge building through open science.

0.27

IdeaTaste: Do LLMs Share the Research Taste of Human Researchers?

cs.CL ·2026-11-02 ·Shanshan Zhong, Chen Xu, Jingjie Ning, Zhengzhong Liu, Aviral Kumar +1 more
benchmarkcommunity pick similarity 0.59
Abstract

Research taste, the judgment of what to read, which problems to pursue, and which methods to try, increasingly shapes the work of AI scientists, yet it remains far less understood than their ability to execute research. We introduce IdeaTaste, a benchmark that scores an LLM by how closely the research proposals it writes from a literature snapshot frozen at 2025 match the work the same communities published in 2026. Our experiments span six contemporary language models and four research areas from established to emerging fields, covering more than 90{,}000 proposals in total. LLM agents fall well short of human communities in both the literature they cite and the ideas they state, reaching about 26\% of the human level on citation matching and only 18\% on idea matching. Their populations can cite as diversely as a community and take up its tasks and methods, yet rarely arrive at the concrete approaches and research plans it pursues. Neither richer field context nor larger populations substantially close this gap. In blinded expert review, AI proposals trail the 2026 papers least in presentation and most in expected success, echoing their gap in concrete approaches, and those whose ideas no human paper shares fare worst, suggesting weaker directions rather than overlooked opportunities. These results indicate that the main bottleneck in AI research taste lies less in finding the directions a community works on than in turning them into the concrete approaches it goes on to pursue. We release IdeaTaste as an open, continuously updated leaderboard (https://cxcscmu.github.io/IdeaTaste) that benchmarks contemporary AI proposals against future publication cycles.

0.28