CS Paper Radar

CATP: Design and Evaluation of Local Agent Authorization and Audit Evidence

Linfeng Zhou

cs.CR ·submitted 2026-09-28 04:06 UTC · v1
code/data

Abstract

A signed authorization record authenticates what a signer asserted, but does not by itself establish that the assertion agrees with the policy and action used for a runtime decision. CATP specifies the bindings needed to carry a local pre-execution decision into offline-verifiable evidence. Its hook commits to the enforcement-time policy and adapter-normalized action and durably records the decision before returning permission. A later receipt binds the record to a self-contained audit export. An implementation for Claude Code and Codex supports an empirical examination of these bindings. Controlled, validly signed counterexamples expose five missing consistency checks; adding them changes acceptance on the same inputs, with all 13 expected outcomes satisfied. A separate probe finds that distinct argument vectors can become the same normalized action before hashing, limiting what even a consistent receipt can establish. Eight scripted runtime cases verify hook delivery and allow/deny behavior; failure tests cover tampering, substitution, replay, concurrency and persistence errors. Median paired hook overhead is approximately 85 ms, and offline verification takes approximately 174 ms end to end on the testbed. The evidence remains conditional on a trusted runtime, monitor, host and signing key. It authenticates a decision about the normalized action, without proving execution truth, complete disclosure or autonomous task utility. The optional Groth16 study covers a narrower circuit predicate; the evaluation does not establish superiority over other systems.

Comment
13 pages, 1 figure. Code: https://github.com/lfzkoala/catp

Closest papers

Nearest neighbours by meaning across the whole index.

AgentTriage: Replayable Security Diagnosis Across Heterogeneous Agent Runtimes

cs.AI cs.CR ·2026-10-22 ·Jie Ying, Xutao Zhang, Ningyuan Chen, Mengce Zheng, Jun Li +2 more
code/data similarity 0.68
Abstract

When tool-use agents write files, execute code, or send data, their final responses do not explain the resulting security events, and heterogeneous runtimes expose the relevant evidence through incompatible interfaces. We present AgentTriage, an executable contract for replayable security diagnosis across heterogeneous runtimes. The contract admits conclusions whose claims cite canonical runtime events and requires abstention when mandatory evidence is missing. Runtime overlays normalize native observations, while a deterministic pipeline joins invocation and result lifecycles, constructs a typed provenance graph, selects a policy-decisive slice, and binds claims to recorded evidence. Given the same normalized trace, overlay version, and pre-run policy oracle, the diagnosis can be recomputed and its claims re-checked without language-model judgment. Across five runtimes, one contract captures all 159 applicable executions, with an audit of native and synthesized identifiers. On 120 adjudicated cases, AgentTriage achieves 0.858 grounded joint diagnosis, 0.966 required-evidence coverage, and a 0.036 unsupported-claim rate, improving joint diagnosis by 0.175 (95% CI [0.092, 0.258]) over the strongest same-schema baseline. These results quantify the precision of replayable diagnosis and the observation boundaries that require abstention. We release an open-source implementation and selected sanitized evidence at https://github.com/AsnGs/agenttriage-open-source.

0.28

Tracekit: Tamper-Evident Intent-Reasoning-Action Auditing for AI Agents

cs.CE cs.CR ·2026-09-28 ·Bravish Ghosh
code/data similarity 0.67
Abstract · 40 pages, 24 figures, 34 tables. v2: rewritten for Tracekit 1.4.1 (signer service, evidence format v2, witnesses, intent-reasoning-action ch…

AI agents run shell commands, edit repositories, send email and move money with their operators' credentials, yet the record of what they did is usually a log the agent, or whoever runs it, can edit. We present Tracekit (open source, MIT, version 1.4.1), which turns each agent run into evidence a third party can check offline. A signer that the agent cannot reach outside dev mode decides every tool call against its own policy before it runs, and signs the call, decision and result as Ed25519 records in a per-run hash chain inside an RFC 6962 Merkle tree whose checkpoints independent witnesses can cosign. A run exports as one .tkb bundle that two independent verifiers (Python, and TypeScript, also as a browser page) check against keys the verifier pins, reporting integrity and assurance separately. Disagreements between capture layers become signed records. Tracekit also records the person's request and what the model said before each call, holds calls that break a limit the request stated, compares what was asked, said and done, and finds which input caused an action by replay. Of 21 experiments: a signed log alone detects 6 of 8 tamper classes and a witness the other two; every insider attack in two suites is caught; the signer round trip is 0.45 ms median; no acknowledged record was lost in 60 kill -9 rounds; reconciliation flags 12 of 12 seeded discrepancies; the verifiers reject all 33 forgeries. Policy rules are a tripwire, not the boundary: they stop 570 of 570 harmful calls in our corpus but 211 of 420 (50.2%) written blind to them; the protection is a signer and credentials the agent cannot reach. On a held-out pool, a failed test claimed as passing is found in 86% of sessions with 5 false findings in 1,812 honest calls, and replay names the planted cause in 6 of 6 confirmed harmful runs with a 7B model.

0.57

Actions with Receipts: Jointly Binding Claims, Evidence, and Execution for Replayable Tool-Agent Auditing

cs.CR cs.AIcs.CLcs.LG ·2026-09-29 ·Miaobo Hu, Shuhao Hu, Xiaobo Guo, Xin Wang, Bokun Wang +2 more
similarity 0.66
Abstract · 35 pages, 8 figures

Tool-using agents can expose citations and execution logs while leaving a critical association unaudited: whether the claim shown to a user is the claim emitted by the committed execution and supported by the cited source. A valid citation and a valid trace can therefore remain individually well formed while being transplanted across claims, actions, runs, or source versions. We introduce a claim-anchored execution contract that jointly binds the emitted claim, its exact source span, the ordered execution prefix that produced it, and the source version and access state observed by that execution. Each receipt contains an emission anchor that deterministically locates the claim inside a committed answer or claim-bearing action, together with source identifiers, offsets, hashes, quotes, and a domain-separated execution commitment. A deterministic integrity verifier reconstructs these bindings before semantic or task labels are joined. We separate this integrity plane from a pluggable support plane, so structural validity is not used as a proxy for entailment. The contract exposes seven independently testable properties: claim-emission binding, source binding, ordered-execution binding, oracle separation, persisted-object replay, execution-rerun consistency, and version/access binding. Across 1,280 cross-object attacks, the joint contract detects 1,275 substitutions (0.9961). Removing a targeted property reduces its attack-detection rate to 0.0156-0.0625. On an independently adjudicated 384-pair split, the conflict-aware support guard reaches F1 0.8865 and false acceptance 0.0729; on unseen failure families, these rates are 0.8679 and 0.0938.

0.24

ClaimReceipt: Verifying Evidence Sufficiency and Coverage in Agent Evaluations

cs.AI cs.CRcs.MA ·2026-09-02 ·Peiying Zhu, Sidi Chang
similarity 0.66
Abstract · Submitted to Who Verifies the Agents? Toward Reliable Agent Development (NeurIPS 2026 workshop). 8 pages, 1 figure, 7 tables

Agent evaluations face two distinct evidentiary questions: whether a reported claim is recomputable from retained evidence (sufficiency), and whether the retained records cover the committed experiment set (coverage). Generic logs and hash-linked transcripts answer neither reliably. We introduce ClaimReceipt, a claim-relative receipt specification and selective verifier that binds typed transaction evidence to a signed experiment manifest and returns PASS, INVALID, or INCONCLUSIVE per claim. We freeze the specification before implementation (SHA-256 18d109...b81). On 1,392 historical buyer--seller records, a CR-2 verifier reproduces all five manually labeled audit verdicts, exactly replays 600 deterministic and 792 post-generation records, makes every one of 13 declared field groups non-redundant under tested ablations, and returns the expected result on 11/11 semantic faults with 0/8 false positives. We then run a separate prospective CR-3 epoch: 30 assignments are committed before inference, terminal receipts are signed and chained, and private evidence is encrypted for an auditor. Complete evidence yields coverage and accounting PASS; withholding one terminal receipt returns INCONCLUSIVE_COVERAGE, while withholding all private openings preserves coverage and protocol verification but makes economic claims inconclusive, exactly matching a preregistered prediction. Receipt instrumentation adds 0.021% of model-inference time and 9.9 KB per transaction. A specification-legibility probe indicates that our own frozen specification is not yet unambiguous to an independent reader. Claim verification therefore requires both claim-sufficient evidence and a committed universe against which omissions become visible.

0.37

Agent Flight Recorder: Tamper-Evident Audit Trails with On-Chain Anchoring for Long-Horizon Tool-Using Agents

cs.CR cs.MA ·2026-09-01 ·Laurent Bindschaedler, Quentin Botha, Christoph Siebenbrunner
publishedhighly cited similarity 0.66
Abstract · 9 pages, 1 figure, 4 tables. Accepted at BCCA 2026 (IEEE International Conference on Blockchain Computing and Applications)

Long-horizon agents execute thousands of actions, resulting in sequential failures rather than isolated errors. When a coding agent deletes a production database or a prompt injection spreads across agents, the incident raises questions of causality, authority, and non-repudiable third-party verification. The Agent Flight Recorder captures each agent action as a structured, canonically serialized event binding eight semantic fields from intent through execution to provenance. Hash chaining and Merkle batching provide tamper evidence and compact inclusion proofs. For cross-organizational disputes where no party's infrastructure qualifies as neutral ground, periodic on-chain anchoring of epoch roots lets any verifier with the disclosed payload and Merkle proof check the record independently, without pre-agreeing on a trusted intermediary. The on-chain footprint is minimal: each anchor stores a 32-byte epoch root and a back-pointer, and no event content touches the chain. We evaluate the system across five cumulative ablation configurations on synthetic agent workloads. The full system adds ~48 microseconds median per-event latency and 512 bytes per event. L2 anchoring costs $2.30 per 100K events at 100-event epochs. The full integrity stack detects edit, delete, reorder, and fork tampering at 100% with zero false positives. Structured forensic queries achieve 1.0 precision on guardrail and delegation lookups where unstructured text search yields 0.013 and 0.077 respectively.

0.50

NovaFabric: Tamper-Evident, Replayable Evidence for Autonomous AI Agent Runs

cs.CR cs.DC ·2026-09-11 ·Mohsen Seyedkazemi Ardebili
similarity 0.65
Abstract · 44 pages, 7 figures, 8 tables, 2 listings. Preprint

When an autonomous AI agent does something consequential, what can be proven about what it did? Agent-observability platforms capture traces, but a trace is mutable: alterable undetected, with no recipe for re-executing it, silent on whether captured secrets were removed. Regulation (EU AI Act, ISO 42001, NIST AI RMF) presumes records an independent party can check. We present NovaFabric, producing audit-grade execution evidence: provider-neutral, tamper-evident, replayable, shareable. It records an agent run, without modifying agent logic, into a portable Run Capsule (fifteen-entity schema), sealed with a holistic DSSE signature, RFC 3161 timestamp, Merkle log and redaction attestation. Sealed runs are re-executable under a four-mode replay protocol and exportable as an Evidence Bundle for third-party verification with stock tooling (specified, not evaluated). The contribution is integration, not new cryptography: OpenTelemetry, DSSE/in-toto and W3C PROV. We evaluate eight research questions at measured scope. Mocked replay serves every model response from the capsule (no live model call, 10/10) but is offline w.r.t. models, not the network; only 2/10 tool-using workloads completed; the gap is missing tool-response substitution. Tampering is rejected across three tested classes. Declared-stream completeness is 0.652 (95% CI +/-0.064, ten scenarios). A repaired rule pack redacts 14/14 credential types, preserving 9/9 decoys; diff localises 140/140 mutations. Blast-radius queries: 45.5ms p99 over 10M edges (3.3x faster than a columnar baseline), 167.9ms over 100M (1 client, n=30). A 314-machine, ten-region run finds capsule REST ingest lossless but capped at 61.6 req/s (p99 26.8s) by per-worker serialisation. Six defects found in NovaFabric and its evaluation corpus: four fixed, one withdrawn, one open. Verification is conditional on a stated trusted computing base.

0.43

FCP: Formal Context Protocol for Verifying LLM-Generated Artifacts

cs.AR cs.AI ·2026-11-15 ·Shashank
code/data similarity 0.64
Abstract · 5 figures, 13 tables. Code and data: https://github.com/shankch/FCP_oss

When a language model writes both an artifact and the tests meant to check it, a passing test is weak evidence: one misreading of the request yields a faulty artifact and a check that accepts it. We present the Formal Context Protocol (FCP), which checks AI-generated artifacts against the request itself. FCP separates conformance (does the artifact satisfy a formal reading of a requirement?), which model checkers and SMT solvers decide, from fidelity (does the reading mean what the prose means?), which has no decision procedure but is refutable by a concrete witness. Eight ways a proof can falsely certify a requirement each become a mechanical check that returns a witness, and what no machine can settle becomes concrete questions for the request's author. Adapters place hardware, text, agents and code on the same engines. On four hardware designs, trusting AI-written assertions certified 23% of violated requirements, and FCP certified none. On ten code tasks, code from three language models passed its own tests and violated the request every time. Trusting the tests certified all 33 violations, and FCP certified three. Agreement among models, readings or tests did not imply correctness.

0.38

AgentEval: Evidence-Bounded Assessment of Stateful Agents

cs.AI ·2026-10-15 ·Ruiqi Gao
benchmark similarity 0.64
Abstract · 23 pages, 6 figures

Task completion alone does not determine whether a stateful agent satisfied its obligations, and a recorded judgment does not guarantee that its supporting evidence remains valid. We present AgentEval, a system that carries a contract-bound assessment record through tool execution, state and event capture, report qualification, and case maintenance. It separates what the agent did, what the evidence establishes, and whether the conclusion is supported for present use. On 32 tool-world administrations, independent checks agree on all 30 complete episodes, distinguishing satisfied, violated, and unresolved obligations; two interrupted administrations are retained. Controlled interventions on 48 evidence views expose 36 binding faults missed by hash-only checks without changing agent behavior. Scenario Studio connects reviewed cases to 12 two-turn reports with 24 independently checked answers. A separate public-benchmark study on 192 held-out executions reveals ten completed return subgoals within broader task failures and isolates the effect of missing evidence. These studies show how an executable assessment record makes scope, evidential support, and report reuse inspectable across an agent evaluation workflow.

0.17