CS Paper Radar

False-science induction in autonomous scientific discovery

Hanbing Liang, Fujun Liu

cs.LG cs.AIphysics.comp-ph ·submitted 2026-08-20 18:06 UTC · v1

Abstract

Closed-loop discovery systems increasingly execute experiments and update decisions autonomously, turning record integrity into part of the experimental apparatus. We show that false-science induction arises when legitimate physical objects and measurements are paired incorrectly, driving neural surrogates to faithfully learn record-induced associations that do not correspond to the true object-outcome relationship while marginal data distributions remain unchanged. Across green fluorescent protein fitness and materials band-gap prediction loops, coherent paired misbinding systematically redirects experimental budgets toward low-performing basins, whereas same-volume random swaps have negligible effects. These observations identify error coherence, rather than raw error frequency, as the primary variable controlling this budget misallocation in the tested loops. The resulting binding identifiability boundary supports monitored-axis quarantines and feedback-conflict triage, which intercept over-concentrated proposals before execution and isolate the corrupted hypothesis axis.

Closest papers

Nearest neighbours by meaning across the whole index.

Learning to Outgrow a Theory: Experimental Discovery Beyond the Initial Hypothesis Space

cs.AI ·2026-10-06 ·SiYuan Ma, Albert Gao, Chunzheng Zhu, Xin Yan, Wenlong Zhang +4 more
similarity 0.64
Abstract

Scientific discovery systems typically optimize experiments within a fixed hypothesis space. This creates a failure mode when all available candidates omit the same missing mechanism: candidate disagreement can collapse even while the model class is systematically wrong. We formulate experimental model-class revision, in which a discovery policy jointly proposes a structural edit and a diagnostic experiment that tests whether that edit is necessary. The method couples a class-level distinguishability objective, in which one shared parameterization must explain all selected experiments, with anytime-valid sequential evidence that triggers structural revision only after the current class is rejected. On 400 held-out controlled dynamical environments, the joint policy reaches 89.5% exact recovery with a budget of 32 real experiments, improving the strongest matched baseline by 10.0 percentage points while requiring fewer executed experiments and candidate fits. The learned revision-experiment pairing transfers across unseen mechanism combinations, held-out but expressible primitives, parameter extrapolation, and shifted experiment costs; when the true mechanism is outside the edit grammar, it detects library insufficiency in 88% of cases with a 5.5% false-support rate. Revision gains also transfer to ODEBench and ODEBase model-library tasks, as well as DiscoverPhysics worlds. These results support a view of scientific discovery in which deciding what mechanisms a theory should make expressible and where to collect evidence are treated as a single sequential decision problem.

0.46

Scaling Science: Toward a Chain Reaction of Discovery

cs.AI cs.LG ·2026-10-31 ·Yuheng Ding
similarity 0.63
Abstract · 32 pages, 1 figure

Scientific discovery advances when evidence from different investigations is brought into contact. Yet paper-centric communication leaves experimental results fragmented, delayed, or unreported, creating a "scientific memory wall" between growing computational capabilities and the evidence accessible to AI systems. Breaking this wall requires a paradigm shift in AI for Science: moving from passively mining questions out of existing datasets to actively generating and recombining experimental data around unresolved scientific questions. In a network of intelligent laboratories, a failed or overlooked experiment in one domain could trigger a hypothesis in another, whose testing generates fresh evidence that stimulates further investigations. To enable these evidence-driven chain reactions, we propose Evolving Scientific Intelligence (ESI), an architectural blueprint that connects shared scientific memory with distributed experimental loops. ESI preserves experimental records -- including negative and anomalous results -- in computable representations, allowing models to recombine evidence across domains into testable hypotheses. Rather than following fixed screening pipelines, the system jointly selects candidates and evaluation methods according to scientific value, reliability, and cost, directing distributed laboratories to produce the exact evidence needed next. Their physical feedback revises both hypotheses and evaluators, while slower meta-control reassesses research objectives and decision criteria to guard against self-reinforcing errors and misguided optimization. We call for an Information Manhattan Project to build this infrastructure for scaling science through experimentally grounded chains of discovery.

0.12

CADS: Measuring the Incremental Contribution of LLM Proposers in Sequential Biological Screens

cs.AI q-bio.QM ·2026-10-17 ·Guanxing Chen, Nihui Shao, Haohuai He, Zhenchao Tang, Pengwei Hu +1 more
hot direction similarity 0.63
Abstract · 32 pages, 5 figures; supplementary material included

Large language models are increasingly used to guide sequential biological screens, yet standard evaluations rely on raw hit counts that conflate a proposer's value with the discovery capacity of the underlying algorithmic harness. Adaptive sampling and corrupted experimental readouts also leave standard feedback comparisons without a false-discovery guarantee. We present Certified Agentic Discovery Screens (CADS), an error-controlled measurement framework that quantifies a proposer's incremental value against an identical LLM-free loop. By integrating capped likelihood-ratio bounds, e-processes, and complete-family e-BH, CADS controls the terminal false discovery rate under the stated null and corruption conditions despite arbitrary dependence, adaptive sampling, and optional stopping. In DepMap-backed simulations, CADS exposes two evaluation artifacts: the LLM-free harness alone clears the feedback gate, and raw origin counts overstate LLM contributions because LLM picks displace fallback discoveries. On the initial 20 seeds, supplying tested-candidate history to Qwen2.5-7B shifts its net contribution from +0.35, with a 95% interval covering zero, to +2.90, with the interval above zero. This initial net effect did not persist in a 50-seed matched-quota replication. These results support net gain over an LLM-free loop, measured between separately certified sets, as the appropriate measure of proposer contribution.

0.32

Asking the right question is key to AI discovery in a chain of hypotheses

cs.AI physics.optics ·2026-11-15 ·Hu Cang
similarity 0.61
Abstract

Artificial intelligence (AI) scientists pursue discovery through a chain of hypotheses, fitting candidate models to existing observations and often choosing experiments where those models disagree most. Yet when early data miss a narrow feature, model selection favors simple curves that agree across the unmea- sured gap, so disagreement-guided search has no reason to look where the new phenomenon lies. When we tested two frontier AI scientists, Claude Sonnet 5.5 and GPT-6.1-Sol, across 32 closed-loop trials with a narrow peak omitted from the starting data, both sampled inside the peak region less often than random search. Surprisingly, the agent that followed model disagreement more closely missed the peak in all 32 trials, whereas discovery succeeded when experiments probed the unmeasured gap early and followed up across the anomaly. Asking the right question, rather than testing only where current models disagree, is therefore key to AI discovery in a chain of hypotheses.

0.28

Domain Bounds as a Silent-Fault Detector for AI-Ready Scientific Data

cs.CE ·2026-09-24 ·Kathryn Knight, Patrick Widener, Heather Bort
similarity 0.60
Abstract

We address an incipient type of data-based faults, driven by increasing amounts of data reuse in AI-enhanced computational science workflows. A dataset is created under specific conditions; its categories, measures, and labels depend on that context, but that context is usually unavailable to downstream users and is completely abandoned when datasets are used to train models. We propose the encoding and association of this context, called domain bounds, with the data to which it applies. By specifying the conditions for valid reuse, a domain bound flags apparently valid data from being reused in an incompatible context, even when every value falls within its expected range. We present cases from computational science and biomedicine where missing domain bounds allow silent reuse errors, and show why description and provenance approaches do not detect them. Our detector uses standard data container technologies, catches faults drawn from a taxonomy of published cases, irrespective of dataset size. Against injected faults it caught 17 of 18 domain-bound reuses that lineage and descriptive records accepted.

0.22

Steering Automated Scientific Discovery via Audited Epistemic Exploration

cs.AI cs.CL ·2026-11-18 ·Fangxu Yu, Tao Feng, Xuying Ning, Haolin Yang, Jiaxuan You +1 more
similarity 0.60
Abstract · Technical Report

Scientific discovery requires revisiting the hypotheses in light of new evidence and identifying promising directions beyond those already explored. However, research agents often make claims that are not supported by their observations, explore a narrow hypothesis space, and retain beliefs despite recognizing conflicting evidence. These failures motivate the search for an explicit epistemic state that captures both beliefs about existing hypotheses and directions for further exploration. To this end, we propose Epistemic Compass (EpiCompass), a plug-and-play scaffold that maintains this state through an epistemic tree linking hypotheses, evidence, beliefs, and revisions. As an investigation progresses, EpiCompass reassesses the hypotheses explored against accumulated evidence and uses unresolved conflicts and unexplored directions to guide hypothesis generation. It selects candidates by anticipated epistemic gain, accounting for both refinement of existing hypotheses and expansion of the hypothesis space, and verifies the final conclusion against the accumulated evidence. Experimental results across three scientific discovery benchmarks and three LLM backbones demonstrate that EpiCompass consistently outperforms baselines, with further analyses showing reductions in errors of unsupported claims and failures to revise beliefs.

0.07

Beyond Execution: Auditing Experimental Fidelity in LLM-Driven Scientific Research

cs.SE cs.AI ·2026-08-27 ·Lezhi Yu, Xiaogang Xu, Yuhua Zhou, Shuibing He, Aimin Pan
code/data similarity 0.60
Abstract · 20pages, 5 figures, code link: https://github.com/Flavorfish/AutoRepro

LLM agents used for scientific experimentation must do more than generate executable code: they must implement the reference method faithfully, design experiments that test the paper's claims, and provide evidence supporting those claims. We show that agents often produce methodological hallucinations: silently reducing datasets or training budgets, replacing failed learning or generative components with lookup or oracle functions, or drawing conclusions from resource-limited settings where a method's claimed advantage disappears. To detect these failures, we introduce ABE-Ralph, a reference-anchored auditing framework that represents claims, protocols, required components, baselines, and metrics as structured experimental constraints, guides implementation through an 8-step workflow, and performs quantitative, qualitative, and code-level verification. Across 30 long-horizon reproduction runs covering 12 machine learning domains, ABE-Ralph achieves a 93% robust execution rate and identifies five scientific failure modes. In 23 NatureBench discovery tasks, ABE-Ralph matches or exceeds state-of-the-art performance on 5 tasks. These results show that reliable evaluation of AI scientists must assess whether the experimental design faithfully tests the intended claim and whether the resulting evidence supports it, rather than treating code execution or plausible metrics as evidence of scientific success.

0.33

Beyond FAIR Data: Instrument Traces for Active and Autonomous Scientific Experimentation

cs.DL ·2026-08-24 ·Sergei V. Kalinin, Boris N. Slautin, Yu Liu, Charles Cao
position similarity 0.59
Abstract

Artificial intelligence is turning scientific instruments into active systems in which observations can determine what is measured next. We argue that this creates an additional scientific record, the experimental trajectory, complementing sample provenance, acquired data and metadata, and analysis workflows. Instrument Traces should ultimately be synchronized with Sample Traces describing specimen evolution and Decision Traces recording human or algorithmic choices. We reconstruct an Instrument Trace retrospectively from a longitudinal AFM/PFM archive containing 118,000 timestamped events from 2023-2026. Conventional saved files reveal material campaigns, latent probe and calibration states, session-level complexity, experimental decision grammar, and composite tuning actions. They also expose what is missing, including unsaved tuning and failures, explicit sample/probe identities, complete timing, exogenous state, and decision rationale. We therefore propose a prospective trace architecture that records synchronized sample, instrument, and decision histories, enabling reproducible autonomy, predictive maintenance, counterfactual analysis, operator training, and transfer across facilities.

0.43