CS Paper Radar

Built to Remember, Shipped to Forget: Operational Contracts, Failures, and Repairs in Containerized LLM Agent Deployments

Meriem Mastouri, Emna Ksontini, Amine Barrak

cs.SE ·submitted 2026-10-25 22:36 UTC · v1
code/data

Abstract

LLM agent systems reach production as container images run by platforms such as Docker, Compose, and Kubernetes. Framework and platform document opposite expectations: to the platform a container is disposable, while the framework persists conversations, pauses runs for human input, and resumes interrupted work from saved state; each side specifies only its own behavior, so the obligations their composition creates are stated nowhere. We present the first empirical study of this boundary, stating these obligations as operational contracts: guarantees a deployment must provide to preserve an agent system's required behavior when a deployment-platform event occurs. From the first-party documentation of LangGraph and the three platforms we derive 14 contracts and evaluate them with three independent sources of evidence: 433 deployed open-source agent systems, seven events induced on 12 of them, and 122 developer repairs mined from 39,261 commits, pull requests, and issues. At least one contract applies to 75.5% of the systems, yet its guarantee is routinely missing: of systems that must keep state beyond the process, 37.4% lose it to a restart and 41.5% to a container replacement, and every assessable deployment stops the framework's production server mid-drain. Induced events turn 14 of 16 predicted violations into runtime failures and all seven predicted holds pass. The failures are born in deployment but repaired in code: developer repairs independently cover twelve of the fourteen contracts, 78% change application code, and only two in three fully restore the violated guarantee. The 14-contract catalogue, detection rules, fault-injection harness, and 35-pattern repair taxonomy provide a basis for checking and strengthening agent deployments.

Closest papers

Nearest neighbours by meaning across the whole index.

Agent Mesh: Reliability Primitives for Non-Idempotent Agent Delegation - Identity Adequacy and Evidence Adequacy

cs.AI cs.DCcs.MAcs.SE ·2026-08-26 ·Mazhar Shaikh, Anurag Rajkumar Bombarde, Harshal Pathak
similarity 0.66
Abstract · 14 pages, 9 figures, 6 tables; 5-page ancillary supplement

Autonomous agents increasingly perform bounded software tasks under an orchestrator that retries, resumes, and budgets them. The machinery such orchestrators reach for is the service mesh's: retry, timeout, and error-rate circuit breaking. We report a failure study of a production agentic software-delivery platform over 147 numbered incidents spanning 81 runs, each with a measured cost and, in most cases, a mutation proof reproducing the failure. All three assumptions those primitives rest on are violated in practice, and we quantify the consequences: a loop of fifty-four consecutive successful tool calls no error-rate breaker could see; a progress signal constant by construction, guaranteeing a false trip on the third repair round and driving one run from six of six components to three; twenty-one events accumulated across six invocations of one delegation, making a correct, idempotent component unwinnable; a misrouted failure that woke five components for a two-component fault, leaving three bystanders regressing working code; and twelve incidents in which the enforcement layer blocked correct work, the most expensive costing 107 agent turns and zero accepted writes. We find one cross-cutting cause and its dual. Identity adequacy: in five separate subsystems an identity that failed to discriminate produced a confident wrong answer, and two of them derived the corrective rule independently. Evidence adequacy: a reliability decision may be taken only on evidence capable of moving, attributable to what it measures, and deterministic under identical conditions. From the findings we derive seven reliability primitives whose enforcement unit is the delegation rather than the message, and specify the controlled evaluation the study motivates but does not constitute.

0.19

Where Does Exactly-Once Live? Model, Harness, and Tool-Contract Effects on Duplicate Side Effects in LLM Agents

cs.LG cs.AIcs.SE ·2026-09-24 ·Jiapeng Li
theoryhighly cited similarity 0.66
Abstract · 23 pages, 6 figures, 13 tables

When a tool-using agent's write times out or returns a server error, the action may already have taken effect. Retrying blindly duplicates it -- a second charge, a second announcement, a second deployment -- while giving up skips required work. We ask where exactly-once behaviour should be enforced: in the model, in the agent harness, or in the tool contract. We introduce LIMBO, a deterministic sandbox of six services with realistic contracts (optional idempotency keys, eventually consistent and missing read paths) and twelve fault modes injected at the service boundary, including late commits, redelivery and partial batches; every episode is graded against a ledger of committed effects. Across 25,930 episodes spanning nine recent models, three production agent harnesses, two contract variants and fifteen recovery conditions, the answer depends on the fault. When an immediate read-back can reveal what happened, the model decides: frontier models instructed to act exactly once almost never duplicate a write whose acknowledgement was lost (0.5%), weaker models often do, and the model explains 53% of the explained variance. When it cannot -- the request is still in flight, or the transport delivered it twice -- the same frontier models duplicate in 56% and 74% of episodes, and the contract explains 81%. We prove that no verification-only policy is exactly-once under late commits without a bound on in-flight time. Waiting works when such a bound is short and known, but with heavy-tailed in-flight delays even an hour of waiting per episode falls short of offering an idempotency key on every write, which lowers the duplicate rate from 28% to 4% because agents use keys when they exist. The harness barely matters, a guard that attaches keys transfers across harnesses unchanged, and agents reported success in 90% of the episodes in which they had duplicated an effect.

0.54

Relic: From Multi-Agent Collaboration to Persistent Organizational Capability

cs.AI cs.SE ·2026-09-26 ·Hongyi Du, Tianyi Zhang, Weijia Zhang, Yi Yang, Haofei Yu +7 more
benchmark similarity 0.62
Abstract · 83 pages, 8 figures. Preprint

Multiple agents may often conflict in an organization: for example, one coding agent changes an interface in a repository, but another continues to develop on the old version where existing tests become stale. A conversation can resolve the episode, but when the participants change, what makes the lesson continue to govern the team? We introduce Relic, which turns recurring collaboration failures into organization-owned, executable protocols. Members reflect on visible work, propose rules, and govern their adoption. Adopted protocols bind triggers, responsibilities, required evidence, and execution consequences to the runtime, while remaining open to revision and retirement. In one traced case, repeated integration friction produces an interface-review rule that governs later pull requests and is revised as work continues. Across 360 controlled runs over ten software workloads and three models, Relic raises complete-contract delivery from 14.06% to 19.76% (+5.71 percentage points) over a matched structured team without the protocol lifecycle, improving all four verified production endpoints in every model stratum. Under fresh-member transfer, behavioral correctness is 25.4% with no inherited protocol, 34.6% with the same rules provided as readable text, and 41.2% with executable bindings, a +6.5-point advantage over text alone. On the full CooperBench benchmark, after excluding 183 broken benchmark pairs, Relic achieves 371/469 (79.1%), establishing the best reported result among peer-structured systems. On the 47-pair same-model subset, Relic also exceeds Solo (28/47 vs. 26/47), reversing the coordination loss exhibited by the official peer baseline. Together, these results show how collaboration experience can become persistent organizational state that remains useful beyond the members who created it.

0.36

A Controlled Experimental Evaluation of Configuration and Cache Architecture in LLM Serving Engines

cs.DC cs.LGcs.PF ·2026-11-04 ·Muhammad Faizan Anwar, Sina Keshvadi
benchmarkcode/data similarity 0.62
Abstract · 29 pages, 14 figures, 7 tables. Artifact: https://github.com/FaizanAnwar01/llm-serving-config-eval

Multi-agent LLM applications repeat most of their prompt tokens across requests, and serving engines reuse the cached state of shared prefixes. vLLM and SGLang reuse it at different granularities, and benchmarks of their default deployments report large performance gaps, usually credited to the cache. Such comparisons rarely control for other configuration differences. One of them, whether a long prompt is prefilled in chunks, is set by each engine's defaults from the hardware and the release, and it confounds the comparison. We built a reproducible testbed on Kubernetes and a seeded multi-agent workload generator, and compared the two engines under vendor defaults and with chunked prefill matched. SGLang's configuration is the same in both arms, which makes it an internal control across independently provisioned deployments. Under defaults, vLLM's median time to first token is up to 64x SGLang's. Matching one setting removes 78% of the capacity gap where that gap is largest and 30 to 67% of the latency gap. Near saturation the remaining latency gap reflects queue growth. A fluid queue model driven only by a 4.4% throughput shortfall reproduces it, and shows that such a gap scales with the length of the benchmark run. What survives matching is a tail-latency advantage of 2.4 to 6.4x at equal throughput, consistent with cache-aware ordering of waiting requests. We derive six recommendations for comparing serving engines.

0.32

Evidence-Governed Deployment Readiness for Networked Agents on Industrial OT Infrastructure

eess.SY ·2026-10-13 ·Dibyajyoti Nath, Nitya Sathyavageeswaran, Ziran Min, Pavan Athreya Narasimha Murthy
similarity 0.61
Abstract

Deploying a workload on a cyber-secure industrial operational-technology (OT) network requires many cross-layer prerequisites to hold at once, including persistent addressing, name resolution, time synchronization, certificate trust, proxy traversal, and reachable industrial-protocol endpoints. These are owned by different teams, yet current tools treat pre-deployment checking, fault attribution, remediation, and ongoing compliance as separate problems, so evidence is rarely reused across stages. As edge platforms come to host self-adapting agents on shared infrastructure, this fragmentation becomes a governance problem. All four stages ask one question: do a workload's requirements hold on a given network? We present an evidence-governed framework that answers it through a single reference, a versioned network-intent artifact specifying a workload's cross-layer requirements for a specific release. A coverage-driven test loop attaches tested evidence to each requirement before deployment. At a site, readiness is reported as ready, not ready, or undecidable, separating untested requirements from genuine failures; a repair step proposes fixes that preserve co-located workloads; and a drift-watch component re-evaluates the site on change, extending assurance across the workload's lifetime. A multi-workload OPC UA commissioning case illustrates all four stages. By detecting runtime deviations and suspending an agent's guarantees until a refreshed artifact is registered, the framework grounds auditable governance for adapting agent collectives on shared industrial infrastructure.

0.12

Ring-0 by Default: Agents as Operating Systems (AaOS)

cs.CR cs.OS ·2026-10-24 ·Xiang Li, Aichun Huang, Zuyao Xu, Fasheng Miao, Zheli Liu
position similarity 0.61
Abstract · 30 pages, 5 figures

LLM agents now read and write files, start processes and subagents, fetch network content, load third-party extensions, and save and resume state, which are the responsibilities of an operating system. They gained these functions within about three years, but not the protections that operating systems built over five decades: whatever the model outputs, whether it follows the user, a tool result, or a downloaded skill, runs with the full authority of the host user, in effect at ring 0. Prior work builds an operating system for agents (OS4A); we argue that a deployed agent already is like an operating system (AaOS) and analyze it as one. Taking the mechanisms of standard OS textbooks as a checklist, we map 116 mechanisms in 19 subsystems to their agent counterparts, from product documentation, source code, and the literature, and label each as isomorphic, degraded (function present, protection dependent on the model), missing, mutated, or novel. Only 11 are isomorphic and 65 are missing, most of them protections: privilege separation, resource quotas, transactions, and tamper-proof audit. Existing defenses confine the whole agent or review its extensions and leave this interior unprotected, and by Anderson's reference-monitor criteria (non-bypassable, tamper-proof, verifiable), rules written in the prompt satisfy none. We test the map with thirteen experiments on Claude Code, Codex CLI, and Gemini CLI, covering 25 mechanisms whose enforcement documentation cannot settle: 16 match the analysis, 7 gain a more precise description, and 2 change verdict. Checks sit in single tools, so the shell bypasses them; all three CLIs load forged transcript records on resume; and task control, sub-agent permissions, and model-based monitors are weak. We identify five problems for which no OS defense transfers and derive a prioritized list of open problems from the gaps of the map.

0.28

Toward Bounding Coding-Agent Autonomy in Software Maintenance: A Pilot Study of Boundary Contracts

cs.SE cs.AI ·2026-10-31 ·Heeyong Eun, Seungyeon Chung, Hyejee Park, Youngeun Lee, Sukhyun Hong +1 more
published similarity 0.61
Abstract · Accepted at the 2nd Joint Workshop on AI for Software Engineering (JAWS 2), co-located with ASE 2026, Munich, Germany

As coding agents gain greater autonomy to modify software repositories, relevant context alone does not define their operational authority. An artifact may be relevant to a maintenance task while being inappropriate to edit; conversely, another artifact may be required even when it is not explicitly mentioned in the request. We explore Boundary Contracts, a machine-readable abstraction that makes task-specific artifact roles (editable, inspect-only, required, and protected) explicit before repository modification begins. We conduct a pilot study on 40 maintenance tasks from a real enterprise repository, comparing the same Codex-based coding agent with and without Boundary Contract information. The boundary-informed condition improves macro edit-selection F1 from 0.440 to 0.682, required-behavior coverage from 0.429 to 0.673, and the number of strictly acceptable patches from 9 of 40 to 18 of 40. To probe whether these gains are explained only by improved file localization, we additionally analyze 21 tasks in which both conditions modify at least one identical file. Differences remain in edit selectivity and patch outcomes, providing initial evidence that explicit artifact roles may influence maintenance behavior beyond localization alone. This pilot does not establish the general effectiveness of Boundary Contracts. Rather, it provides initial evidence for a broader research question: how should the operational boundaries of coding agents be generated, validated, revised, and ultimately enforced as repository-level maintenance becomes increasingly autonomous?

0.34

Tool-Calling LLM Agents Under Realistic MCP Faults: Scale Does Not Purchase Fault Tolerance

cs.SE cs.AI ·2026-10-23 ·Antonio Ken Iannillo, Joshua Segun Owotogbe, Roberto Natella, Francesco Avallone, Kristina Kudryavtseva +1 more
publishedcode/data similarity 0.61
Abstract · Accepted at NeurIPS 2026 (poster). 10 pages of main text plus appendices and the NeurIPS checklist. Code and data: https://gitlab.com/unilux…

LLM agents increasingly rely on external tools exposed via the Model Context Protocol (MCP), yet current benchmarks assume tools respond correctly and on time, leaving agent resilience to realistic tool failures untested. We introduce MCP-Injector, a drop-in, protocol-aware fault-injection middleware that interposes on the MCP tool-call path with controlled persistence (permanent, transient, intermittent) and seedable schedules. Its fault model is calibrated to the observed incident distribution (failure mode x persistence) of the public MCP ecosystem, mined from 57,240 issue and pull-request threads across 1,774 open-source MCP repositories. In a 1,350-cell experiment crossing two backbone sizes (20B, 120B), three agent stacks (custom multi-round plan-and-execute, OpenAI Agents SDK, LangChain), and three fault injection conditions, we find that faults drawn from this permanent-heavy stress profile cause severe quality degradation (task-completion score falls by 60%, from 8.08 to 3.23 out of 10), and that scale does not purchase fault tolerance for the two gpt-oss backbones we test: the 120B retains no larger share of its fault-free score under faults (resilience-ratio model, $p = 0.96$), and its completion rate under faults is, if anything, lower (91.1% vs. 95.6%; difference 4.4 points, 95% CI $[-0.2, 9.3]$). Agent stacks differ in how well they absorb faults, which suggests that framework-level error handling matters more than model scale. These results demonstrate that capability benchmarks alone can mask operational brittleness, motivating protocol-aware fault injection as a complementary evaluation dimension for tool-calling agents.

0.55