Poison Valleys: Diagnosing Path-Dependent Failure in LLM Code Repair
Abstract
When code-repair agents fail, final tests reveal that a patch is wrong but not which earlier commitment caused it or which software obligation failed. Poison Valleys traces this path in 1,844 model requests on two synthetic Python cases: PersistQueue adds atomic batch claiming to a disk-backed queue, while Dijkstra adds source-sensitive filtering to shortest-path search. We define plausible but incorrect software mechanisms, place each in a model's hypothesis, implementation plan, or both, and evaluate the generated code with an Oracle and targeted diagnostic coordinates. Moving the misconception from hypothesis to plan increased failure by 32.2% in PersistQueue and 28.9% in Dijkstra after matching the experimental mix. Different misconceptions produced distinct, model-dependent patterns of violated obligations, although several could converge on the same behavior. This creates a measured connection from upstream misconception to code mechanism and failure form. Code, data, and demo: https://poisonvalleys.org.
- Comment
- Accepted to Interpreting Agent Behavior (IAB): The First Workshop on Human-Centered Interpretation for Understanding Agents, Humans, and Interaction (NeurIPS 2026 workshop); 22 pages; code, data, and demo: https://poisonvalleys.org