CS Paper Radar

From Prompt to Harness: Coderlet from Scratch

Mengfan Li

cs.AI ·submitted 2026-08-10 11:47 UTC · v1
code/data

Abstract

A model alone does not determine how a programming agent acts. What the model sees, how actions enter the environment, how feedback returns, and how one run affects the next all depend on how the harness is organized. Minimal examples usually show only the basic interaction between a model and tools, while production systems spread these relationships across complex components and dependencies. This paper studies a compact harness design by following a single request through context formation, model decision, environmental action, observation return, and state continuation. Three boundaries---model, execution, and state---connect the model service, tool environment, and persistent state, while the request lifecycle determines the order in which these transitions occur. Together, they show the harness's core role: turning model generations into environmental actions, carrying runtime feedback into later decisions, and allowing state to continue across requests. On top of this runtime structure, a harness can also be gradually refined across runs through continued bootstrapping. The design is realized in the executable artifact https://github.com/lilinxi/Coderlet.

Closest papers

Nearest neighbours by meaning across the whole index.

HarnessMM: Recursive Self-Improvement for Harness Token Efficiency

cs.AI ·2026-10-23 ·Yuhua Jiang, Chang Xu, Feifei Gao, Li Zhao
code/data similarity 0.71
Abstract

HarnessMinusMinus (HarnessMM) is a research project on improving the efficiency of agent harnesses, the software coordinating model reasoning, tool execution, and feedback. This technical report consolidates its shared framework, implementations, and empirical studies across coding, tool-based, and visual-agent settings. Agent harnesses often invoke reasoning after each tool result, even when earlier calls have already determined subsequent actions. HarnessMM addresses this overhead through guarded decision reuse and recursive self-improvement (RSI). RSI refines author prompts, context settings, and adapter rules using training feedback, with validation selecting deployment configurations. At inference time, a controller reuses resolved decisions while keeping actor weights fixed. Executable Decision Graphs encode actions, arguments, dependencies, and guards. Guarded Decision Reuse activates eligible decisions from observed feedback, while Certified Callback Substitution executes them through native tool interfaces, replacing redundant reasoning callbacks. The original actor resumes when new reasoning is required or reuse conditions are unmet. On HumanEval, HarnessMM preserves Codex success at 90.82\% while reducing tokens per task from 66.72k to 47.02k (29.54\% saving). For DeepSeek Harness, success improves from 71.43\% to 85.71\% while tokens decrease from 96.28k to 11.62k (87.93\% saving). An MBPP ablation shows 26.91\% lower cost than single-use graphs and 19.52\% lower cost than retained callbacks. The report also summarizes the HarnessMM-V visual-agent study and documents optimization overhead and execution time where measured. Together, these studies investigate guarded decision reuse across diverse agent harness settings. Code is available at https://github.com/YuhuaJiang2002/HarnessMM.

0.27

Harness Self-Evolution for Long-Horizon Terminal Agents

cs.AI cs.CL ·2026-10-20 ·Ruhan Wang, Yucheng Shi, Zongxia Li, Junyao Yang, Zhongzhi Li +4 more
similarity 0.70
Abstract · 38 pages, 7 figures, 7 tables, including appendices

Long-horizon terminal agents may solve a task in one run yet fail across repeated runs. Stochastic errors can propagate through actions and context, creating a gap between capability and reliable execution. We present RiseHarness, a framework for iteratively improving agent harnesses with model weights fixed. It focuses on flipping tasks with both successful and failed runs under the same configuration, using successful runs as references for failure analysis. RiseHarness combines agent trajectories with internal harness traces from a runtime monitor to examine how model actions interact with command execution, feedback, context management, and completion decisions. A Harness Handbook links this evidence to harness behaviors and source code, helping identify recurring problems in mechanisms shared across tasks. RiseHarness then searches other harness frameworks for relevant implementations and adapts them as design references to the current model and execution protocol. Candidate changes undergo behavioral checks and repeated task evaluation; versions with measured gains seed the next iteration. This loop integrates suitable mechanisms from different frameworks into a single harness and uses new execution evidence to guide further changes. We evaluate task success and reliability across models, ablate harness execution evidence and external research, and test transfer to unseen benchmarks with the same model. RiseHarness treats a specific model as the unit of harness adaptation, studying how changes to shared interaction mechanisms can make existing capabilities more reliable across tasks.

0.31

Harness as a Language: A Minimalist Agent Framework With Maximal Expressivity

cs.AI ·2026-09-22 ·Zhening Li, Joshua Liu, Mateja Vukelic, Nicole Shen, Supriya Lall +5 more
published similarity 0.67
Abstract · 26 pages, 3 figures; accepted to NeurIPS 2026 TTCL Workshop

Modern language-model agents are built around the agent loop: the LLM is placed in an environment exposing a set of tools, and the LLM has full control over the workflow by alternating between tool calls and observing their output. However, certain capabilities such as long-term memory and self-improvement currently require specialized systems beyond the agent loop itself. We built an LLM agent framework, JAZ, to explore the extent to which a minimal harness that is little more than the agent loop itself can accomplish tasks these specialized systems are built for. JAZ exposes a single LLM-based primitive `invoke` and provides a set of built-in hooks that allow the programmer to apply constraints and perform monitoring. Generalizing existing code-mode agent loops, `invoke` is the simplest loop that satisfies two defining properties: (1) the LLM can write arbitrary executable code that can include recursive `invoke`; (2) everything visible to the LLM - all inputs to `invoke` as well as its interaction history with the code environment - are variables in the code environment. We motivate our design from first principles, viewing `invoke` as a language primitive representing a function whose implementation is provided at runtime by an LLM every time it is called. To validate the design of our core `invoke` primitive, we evaluate `invoke` - with only prompting, no manually designed tools, harness, or external systems (e.g., memory or the file system) - on workflows traditionally implemented through specialized harnesses. On long-horizon workflows requiring recall far beyond the context window, JAZ `invoke` outperforms Letta (MemGPT) by 8% at half its cost on the recall-heavy portion of StuLife. On continual self-improvement, JAZ `invoke` outperforms ACE by 4% at a lower cost on AppWorld.

0.52

HarnessNet: Feedback-driven Hierarchical Joint Harness Evolution for Multi-agent Systems

cs.AI ·2026-10-10 ·Zhaoyuan Qiu, Sachith Seneviratne, Yu Xia, Saman Halgamuge
similarity 0.66
Abstract · A novel method for multi-agent system harness joint evolution

AI agents can evolve by using environment feedback to revise their harnesses, the software managing model inputs, tool use, memory, and execution, while keeping foundation models frozen. Existing studies focus on single agents, leaving joint harness evolution in multi-agent systems (MAS) underexplored. A single global optimizer faces growing context and workload as the system scales, while access restrictions may prevent sharing agent internals or system-wide feedback. We propose HarnessNet, a framework for hierarchical harness evolution in MAS. Its basic unit, Agentron, comprises an evolvable forward harness, a foundation model, and a local optimizer that can observe and modify only that harness. Agentrons form a layered graph through which environment feedback propagates backward, with upstream optimizers receiving node-specific feedback from the downstream layer. This decomposes joint harness evolution into smaller, coordinated tasks with separate contexts, reducing individual optimizers' context and workload while keeping access local. Optimizers need not access all agent trajectories or raw environment feedback. Experiments on AgentBench with multiple foundation models demonstrate the feasibility of training HarnessNet. Compared with a single global optimizer, HarnessNet achieves comparable performance gains while respecting local privacy and requiring fewer tokens and less time per optimizer call. A migration study shows that the evolved harnesses remain jointly effective after replacing the foundation models, suggesting transferability across models.

0.18

Harnesses That Improve Themselves: A Unifying Framework for Self-Improving Agent Harnesses

cs.AI ·2026-11-17 ·Deep Shah, Nehal Kathrotia
benchmark similarity 0.65
Abstract

Whether a foundation model succeeds as an autonomous agent depends as much on its runtime software as on its weights. This runtime, the agent harness, manages context and memory, steers multi-turn control, executes tool actions in sandboxes, and verifies work before stopping. Because models differ in how they use tools and how they fail, a static harness tuned for one model or task often stalls, wastes tokens, or regresses on another. This bottleneck has driven a shift from prompt tuning to self-improving agent harnesses: systems that inspect their own execution traces, separate runtime bugs from model limits, and rewrite their own code. We present a unifying architectural and design framework for self-improving agent harnesses. The framework pairs four editable runtime components (context and state, agent control, action interfaces, and verification and recovery) with a four-stage loop that diagnoses failures, proposes code patches, validates candidates, and deploys updates. Using this framework, we show how optimizers trace errors across long trajectories and discover reusable runtime mechanisms such as pre-stop verifiers, loop breakers, and persistent memory. We further show why full-harness rewrites overfit or break across modules, and how modular updates with validation gates generalize across models while bounding token cost. To keep self-modification safe, we establish practical defenses against recurring failure modes including prompt bloat, benchmark gaming, and evaluator tampering. Finally, experiments on SWE-bench Lite validate this framework, showing that gated modular evolution outperforms prompt reflection and full-harness rewrites across models and repositories at a fraction of the token cost.

0.07

Beyond the Agent Harness: Boundary Criteria for Software-Factory Orchestration - A Single-Operator Case Study

cs.SE ·2026-11-03 ·Lei Hua
similarity 0.65
Abstract · 37 pages, 3 figures. Ancillary files include E1 data, coding manuals, coded units, and analysis scripts

Several recent systems add a layer above the coding-agent harness that organises many agent runs into continuous delivery. The existence of such a layer is not our contribution. What is missing is a rule for deciding which layer a mechanism or a failure belongs to. We propose boundary criteria based on an operational definition of a run. A mechanism confined to one run belongs to the agent harness. One that coordinates runs, hands state from one run to a later one, or is updated from run outcomes belongs to a factory harness. One that must change with the application domain belongs to the domain layer. Two residual tests assign infrastructure and model causes. For failures, we code where the defect originated, which control failed to stop it, and where it was fixed as three separate fields, and we require positive evidence before attributing a failure to the environment. We developed the criteria on documented failures from a single operator's pipeline serving six projects and on 26 components of three published systems. We then froze them and had two model coders of the same family apply them to the same descriptions of 46 held-out units. They agreed on all 26 mechanism layers and on 18-19 of 20 incident fields (kappa 0.86-0.93), but both coders recorded the incident as undetermined or unrecorded in 8-13 of the 20 cases per field, and each flagged half of all units as ambiguous, mostly for missing information. The check exposed conflicts in the rules; a revision that separates mechanism classification from incident attribution resolves them but, in a sensitivity re-coding, agreed less (kappa 0.78-0.93). Agreement here is neither accuracy nor usefulness, and human coders have not yet been tested. Case evidence and seven hypotheses are reported as preliminary.

0.18

Doc2Harness: Benchmarking LLMs on Long-Horizon Harness Engineering of Coding Agents

cs.SE cs.AI ·2026-10-14 ·Yuyao Ge, Shenghua Liu, Peizhong Gao, Chendi Ge, Chenxu Liu +6 more
benchmarkcommunity pick similarity 0.64
Abstract · 17 pages, 5 figures, 4 tables

Agents driven by large language models are increasingly deployed on long-horizon tasks, where they use tools and modify real workspaces. How well they perform depends as much on the harness around the model as on the model itself. Existing benchmarks measure downstream task success under a fixed harness, leaving harness engineering itself unexplored. We propose Doc2Harness, a two-stage benchmark that evaluates long-horizon harness engineering directly. In Stage 1, a build model operating inside a fixed builder harness receives an interface-only contract skeleton and a development document that consolidates behavior across twelve frontier harnesses, and then constructs a complete candidate harness in an isolated container. In Stage 2, the frozen candidate harness runs in containers separate from the construction environment, one per test case, and is driven end to end by a scripted mock model. We score each test case using four observable channels, namely the requests sent to the model, the files on disk, the process exit code, and the command output; positive controls rule out vacuous passes. Across nineteen frontier models, each evaluated with three independent builds on 328 test cases, GPT-6 Astra achieves the highest pass rate at 53.05%, while the weakest model reaches 27.44%, a gap of roughly twenty-six percentage points. Even the strongest model fails nearly half of the test cases, and failures cluster around how cross-cutting functionality is split across modules. To our knowledge, Doc2Harness provides the first deterministic benchmark for measuring long-horizon harness engineering.

0.29

Object-Oriented Harness: Evolving Agent Harnesses through Object-Oriented Design

cs.CL cs.SE ·2026-10-20 ·Gaotang Li, Bhavana Dalvi Mishra, Yifei Ming, Yanfei Chen, Zhongying CuiZhu +7 more
similarity 0.64
Abstract · 20 pages, 10 figures

Agent harnesses are executable code that specifies the prompts, tools, and control flow around a language model. A single global harness can struggle with heterogeneous tasks that require different execution strategies. In this work, we introduce Object-Oriented Harness (OO-Harness), a harness optimization framework that evolves a hierarchy of executable harness classes using object-oriented design. During training, an evolution agent uses execution records to create a family of specialized harnesses, each implemented as a subclass of the base or an existing specialized harness via inheritance. At test time, a harness router uses routing lessons accumulated during training to select a specific harness for each task. Across three long-horizon benchmarks spanning business workflows, workspace tasks, and tool use, OO-Harness achieves the best performance among the compared methods, with an absolute improvement of up to 15% over the base. Extensive ablation studies examine the contributions of its core components.

0.25