CS Paper Radar

From topology-optimization brief to solver-checked release: separating intent, admissibility and acceptance knowledge for language agents

Shaoliang Yang, Jun Wang

cs.CE ·submitted 2026-11-02 23:40 UTC · v1
code/data

Abstract

An engineering solver can converge even when a language agent has misread the user's problem. AgenticTO makes this distinction inspectable by keeping three records separate: the proposed problem specification, deterministic formulation checks, and numerical acceptance of the solved design. We evaluated the workflow on 50 author-built topology-optimization briefs, injected faults, five numerical problems, and complete agent sessions. The tests used linear-elastic models with one load case. Geometry tools raised the primary provider's correct specifications and refusals from 37 to 44 of 50, while also increasing wrong specifications delivered as finished. Input-format feedback alone recovered most of the accuracy gain. The wrong specifications passed the formulation rules, which also missed most injected faults that changed the intended problem. Clarification corrected the remaining errors when an oracle supplied reference facts, but also prompted questions on fully specified briefs. Perfect accuracy under that oracle therefore does not establish performance with engineers. Numerical trials reached 73.7 million elements, although released designs used finer grids than requested. The records expose how a proposed problem becomes an accepted numerical result; agreement with the brief still needs a separate judgment.

Comment
60 pages, 13 figures, 20 tables, including the supplementary appendix. Machine-readable supporting records are ancillary files. Code and data: https://github.com/nbbllxx0/From-brief-design-to-verified-release

Closest papers

Nearest neighbours by meaning across the whole index.

From 1D Benchmarks to Complex 3D Geometries: Can Agents Reliably Specify Problems for Automated PDE Simulation?

cs.AI cs.CE ·2026-10-30 ·Akshat Jani, Sakhinana Sagar Srinivas, Venkataramana Runkana
similarity 0.64
Abstract

Turning an engineering request into a simulation requires consistent geometry, physics, boundary conditions, and numerical settings. Recent agent-assisted systems automate simulation setup, but successful execution alone does not establish whether the specification preserves the intended problem or correctly handles missing information. We study whether agents can reliably construct complete executable JSON specifications for automated geometry creation, meshing, PDE solving, and result export. Our workflow combines model extraction with deterministic scientific and capability checks, explicit clarification, and validated local patches that preserve unrelated accepted fields. We evaluate specification correctness, execution completion, and numerical verification separately. On 120 template-related requests, the full workflow achieves 98.3% correct first-turn specification and decision outcomes, compared with 92.5% for schema-assisted single shot and 82.5% for model-only: gains of 5.8 and 15.8 percentage points, respectively. It makes no false-ready decisions among the 42 requests requiring clarification or rejection. Component ablations identify contributions from scientific checks and conditional repair. Separate downstream studies cover 1D, 2D, and 3D steady and unsteady, linear and nonlinear problems, including flow and elasticity with complex three-dimensional wing geometries. A 30-case gallery demonstrates automated execution from retained JSON specifications, while 9 of 12 selected live user-to-results tests achieve joint specification, execution, export, and numerical-reference success (75.0%). These results support traceable, engineer-reviewed simulation assistance within the tested scope, while fresh-response regressions identify consistency as a priority for further improvement.

0.11

Agent4FreeFem: A Verification-Guided Framework for Autonomous Topology Optimization

cs.AI math.OC ·2026-10-31 ·Zhipu Cui, Helin Gong, Shengfeng Zhu
similarity 0.63
Abstract · 23 pages, 10 figures, 8 tables

Topology optimization has become an indispensable design tool in the aerospace, automotive, and mechanical engineering industries. Yet bringing a new problem from its description to an optimized result remains slow, expert-dependent, and unreliable: the workflow spans many manually connected stages, each new problem requires solver code to be written and debugged manually, and such code often fails or yields incorrect results. We present Agent4FreeFem, a pipeline driven by a large language model that turns a natural-language problem description into a set of verified optimization results without manual solver-code development or debugging. The pipeline keeps the problem definition unmodified, localizes errors using independent sources of evidence, and completes constrained repairs automatically, thereby providing efficient end-to-end automation, autonomous execution, and verifiable results. We evaluated Agent4FreeFem on a suite of single-physics and coupled multiphysics problems. Every case passed runtime verification and finished within the repair budget of each stage.

0.31

Consequence-guided clarification: releasing minimum-compliance topology-optimized designs from incomplete specifications

cs.CE ·2026-10-26 ·Shaoliang Yang, Jun Wang
code/data similarity 0.62
Abstract · 24 pages, 9 figures, 7 tables. Supplementary Information (40 pages) and the two author adjudication records are ancillary files. Code and da…

A converged topology optimization can solve the wrong problem when loads, supports, or the domain are underspecified. Assuming defaults, or asking the problem's author about every detail, ignores how much each answer changes the design. We propose consequence-guided clarification (CGC), which bases acceptance on computed mechanical consequences. CGC computes a reference design for each of a finite list of complete specifications, called readings. A design is accepted only if its compliance is at most 5% above the reference at every reading consistent with the answers. Otherwise, CGC asks the question whose worst answer leaves the smallest worst-case excess, and withholds acceptance at its question limit. To save optimizations, the screening variant CGC-S postpones some references until a candidate must be checked. We developed and evaluated both on 36 arXiv-derived tasks, treating each of 134 readings in turn as intended and simulating correct answers. In 26 tasks, the default-reading design failed another reading. Apart from always-asked questions on load direction, sign, and material budget, asking everything took 188 questions; CGC and CGC-S each took 105. The acceptance test supplied most of this saving: applying it to the default design alone took 117. Relative to asking everything, CGC-S requested 4.6 extra reference designs per avoided question, each from two optimization starts, counting optimization time only. References are not certified optima: better ones pushed three accepted cases above tolerance, and finer meshes changed some decisions. Within these limits, the test shows which readings a design covers and which details block acceptance.

0.49

VERAS: Verification-Guided Adaptive Search for Reliable Operations Research

cs.SE cs.AI ·2026-10-16 ·Minbo Ma, Honglin Zhang, Bangtao Zhou, Chunguang Yu, Dasheng Ju +3 more
code/data similarity 0.62
Abstract

Advances in reasoning, coding, and tool use enable LLM agents to solve operations research (OR) problems through iterative problem modeling, program construction, and verification. Yet passing self-authored verification does not ensure that the solutions satisfy task constraints and meet optimization quality. This gap can arise from correlated specification errors, where the generated program and its checks share omissions or misinterpretations of task requirements, alongside incomplete verification coverage and ineffective optimization strategies. We propose VERAS, a verification-guided adaptive search harness for reliable OR that uses verification evidence to prioritize feasibility repair, constraint-coverage review, and solution-quality improvement. VERAS combines task-specific solution checkers with a separate verification agent to examine candidate solutions against the task requirements. It uses verification feedback to guide feasibility repair, additional constraint checking, and objective improvement, while candidate acceptance criteria screen revisions for deterioration in output validity or constraint satisfaction. On ORAgentBench, VERAS achieves a 42.99% pass rate, exceeding the reported 35.51% of Codex using the same LLM. Across two industrial-scale OR problems, VERAS reduces inter-datacenter peak bandwidth by 10.34% relative to native Codex and lowers annualized distribution network planning costs by 6.28% compared with expert-designed solutions. Code and evidence are available at https://github.com/holimario/VERAS.

0.37

FLARE: Verifying MILP Reformulations with LLM-Based Theorem Proving

cs.AI cs.LOmath.OC ·2026-08-25 ·Henry Robbins, Connor Lawless, Madeleine Udell, Ellen Vitercik
datasettheoryhighly cited similarity 0.61
Abstract

Mixed-Integer Linear Programming (MILP) is a fundamental tool for combinatorial optimization with extensive real-world applications. A central challenge is designing computationally efficient MILP formulations. Large Language Models (LLMs) offer new opportunities to automate the modeling process, from deriving formulations to strengthening them. Reliable automation requires robust methods for verifying that proposed formulations preserve the underlying optimization problem. However, existing approaches evaluate formulations numerically and fail to reason about general problem instances. We resolve this limitation by introducing a constructive definition of MILP reformulation that can be formalized in Lean and machine-checked. We develop FLARE (Formulation-Level Automated Reformulation Evaluation), a method that uses an LLM-based agent and the Lean proof assistant to verify proposed reformulations against a reference formulation. To evaluate our approach, we introduce FormulationBench, a challenging dataset of 20 problems and 109 formulations. FLARE outperforms existing methods, with 100% accuracy on the NP-hard subset of FormulationBench. Furthermore, FLARE produces a machine-checkable certificate for every reformulation it accepts. For cases where formal guarantees are not necessary, we introduce FLARE-NL, a fast and cheap LLM proxy that matches FLARE's accuracy but produces no certificate. These methods enable reliable verification in automated optimization modeling.

0.38

Scientific Coding with Large Language Models: Evidence from Finite Element Analysis

cs.SE cs.AIcs.CE ·2026-11-16 ·Erfan Hamdi, Joel Shor, Emma Lejeune
similarity 0.60
Abstract · 28 pages, 7 figures, 14 tables

Large language models (LLMs) offer a potential route to automating scientific programming, but their usefulness depends on consistently producing numerically correct code. We extend FEM-Bench to 17 finite element tasks involving nonlinear and history-dependent mechanics and structural dynamics, and evaluate 15 models on implementation and test generation without execution feedback. The best model succeeds in 87.1% of single-pass implementation trials, but rankings differ between tracks. The leading implementer achieves 49.9% joint test success, while the leading tester reaches 82.6%. For six models, we compare single-pass generation with two agentic harnesses that allow code execution and revision. Agentic execution solves some tasks that never succeeded in single-pass trials and improves consistency on others, but gains vary across models and harnesses, and regressions also occur. Claude Opus 5 with SimpleAgent reaches 96.5% Pass@3, the probability of at least one success in three attempts, but 77.1% Pass$^3$, the probability that all three succeed. No model-harness combination exceeds 78.8% Pass$^3$. To understand why agents submit incorrect code after passing their own checks, we analyze 249 failed agentic trajectories and identify six categories of self-verification failure. The most common are inadequate test coverage (73.1%) and reliance on generic consistency checks (70.3%), and in 28.5% of cases the tests share an error with the implementation. Motivated by these findings, an exploratory configuration with separate coder and tester agents achieves the highest observed scores, with 97.6% Pass@3 and 88.8% Pass$^3$. These results identify consistency across attempts and independent verification as the main obstacles to trusting LLM-generated scientific code.

0.39

Done Is Not Correct: Measuring Silent Failures and Self-Verification Calibration When LLM Agents Take CAD Actions

cs.SE cs.LG ·2026-11-06 ·Carson Rodrigues, Clive Rodrigues, Aravind Reddy G
benchmarkcode/data similarity 0.60
Abstract · 18 pages, 3 figures, 2 tables. Benchmark, harness, per-attempt records and a 3D viewer: https://github.com/rodriguescarson/cad-silent-failur…

When a large language model (LLM) agent writes parametric CAD code from a natural-language specification, syntactic success decouples from semantic correctness. We release a benchmark and harness that measure this gap directly: 13 specification tasks whose every requirement maps to machine-checkable properties of the produced B-rep, measured by the kernel against published tolerance bands, with reference solutions and the full graded attempt corpus. The exact, self-audited oracle isolates two quantities no prior CAD benchmark reports: the silent-failure rate (a valid, renderable solid, an explicit completion claim, and at least one failed semantic check) and completion-claim calibration. Across four models from four vendors (156 attempts per architecture), single-shot agents claimed completion on 98% of attempts and were silently wrong on 14.1%; a tool-use loop that enforces measure-then-claim cut silent failures to 3.2% and raised claim precision from 64% to 96%, and a four-model ablation attributes the change to the claim contract rather than the feedback. A planner-coder-checker pipeline matched that precision at lower coverage with no success gain. Stated confidence was miscalibrated single-shot (ECE 0.32) and became informative under verification for three of four models. Hardening our own oracle flipped 18 of 519 regraded verdicts from pass to fail and none in reverse; a blind expert audit overturned 3 of the 18 silent failures outside the one ambiguously specified task; and fourfold scaling of every tolerance band leaves the contrast intact.

0.27

When Tools Lie: Reliability of Mathematical Agents Under Corrupted Tool Feedback

cs.CR cs.AIcs.SE ·2026-10-06 ·Kavienan Jegatheesan, Gayathri Lihinikaduarachchi
published similarity 0.60
Abstract · 8 pages, 2 figures, 3 tables. Accepted to the 6th Workshop on Mathematical Reasoning and AI (MathAI) at NeurIPS 2026

Mathematical problem solving often requires deterministic computational steps that agents delegate to tools and implicitly trust. Yet tools can fail silently, returning plausible but incorrect results. How well can agents detect and correct corrupted tool call outputs? We study this through a controlled corruption framework where a hidden interceptor replaces tool call results with plausible incorrect information on targeted problems. We evaluate agents across 31 problems under four verification designs including no verification (baseline), mandatory same-context reflection, optional fresh-context verification, and optional structural verification. Without verification, corruption causes dramatic accuracy loss, from 100% down to 72.4%. Mandatory reflection fully recovers this performance to 100%. Optional verification improves accuracy only when models actively invoke it. Our results show that checking frequency is strongly associated with robustness differences, while unequal invocation prevents a controlled comparison of verifier quality. A supporting recovery experiment shows that full problem restart succeeds in 100% of cases after explicit detection. These findings demonstrate that verifier availability and verification policy are separate components of mathematical-agent reliability. Mandatory policies enforce verification while optional policies depend on the model's own choice to invoke it.

0.35