AI Agent Engineering Series
- AI Agents Part 1: Architecture
- AI Agents Part 2: Harness and CLI
- AI Agents Part 3: Tools and APIs
- AI Agents Part 4: MCP
- AI Agents Part 5: Context and Memory
- AI Agents Part 6: Agentic RAG
- AI Agents Part 7: Evals and Observability
- AI Agents Part 8: Security and Deployment
An AI agent harness is the application code that runs the agent loop from the first request to a final status. The model proposes one action per step. The harness checks the budget, validates the proposal, runs or refuses it, records the result, and decides when the run stops. This part explains five ideas that make a harness predictable. They are a fixed step order, enforced budgets, state as an event log, a small status machine, and a CLI with replay. Every later part of this series plugs into this loop.
What the harness owns
Part 1 drew the boundary. The model returns a typed decision, code validates it, and a task contract lists what the run may read and write. The harness is the code that calls the model again, decides how many times, and acts between the calls.
Anthropic describes agents as LLMs "using tools based on environmental feedback in a loop". It names a maximum number of iterations as a common stopping condition (Building Effective Agents). The ReAct paper made this loop common in research. In ReAct, a model alternates between reasoning steps and actions that collect outside information (Yao et al., 2022).
The model's job in the loop is to choose a plausible next action from unclear input. The harness owns every other concern in the table.
| Concern | The model | The harness |
|---|---|---|
| Next action | Proposes a tool call, a final answer, or a handoff | Validates the proposal against the contract |
| Tool execution | Cannot call anything | Calls only allowed tools and records each result |
| Limits | Can be asked to be brief | Counts steps and tokens and checks the deadline |
| State | Sees the messages that it receives | Stores every change as an event |
| Stopping | Can say "done" | Sets the final status |
| Writes | Can request one | Pauses the run until an outside party approves |
The harness column holds the authority. A concern with no entry in that column has no enforcement.
One step, in a fixed order
A harness repeats one step until the run leaves the running status. The order of checks inside the step is the main design decision. Each check that can stop the run comes before the expensive or risky work. A run over budget never pays for another model call. A model reply that does not parse never reaches a tool.
What each branch does
The step refuses to start unless the run status is running.
- Invalid output. A reply that does not parse becomes an observation with the error text. The model sees the error in its next input and can correct it. Each attempt costs one step, so a model that keeps failing hits the step limit. The OpenNash post on output contracts explains why a typed retry at this boundary costs less than any later fix.
- A tool outside the contract. The harness records a refusal as an observation, and the tool never runs. The observation is marked as harness-generated, so replay can tell refusals from tool results.
- A tool error. An exception in a read becomes an observation. The model can try a different argument. A wrong invoice ID, for example, returns "not found", and the next step asks for the right one.
- A write request. The harness asks the tool layer for a proposal, which must have no side effect. Then it pauses the run and stores the request as data. Part 3 defines previews and receipts, and Part 8 defines the approval records.
- A model client error. Network, quota, and authentication errors end the run as failed. The harness does not retry inside the step, because the correct retry depends on the error type.
The harness never calls a tool function directly. It calls a tool executor with four operations: list the tool specs, run a read, propose a write, and commit an approved write. Part 3 supplies a real executor, and Part 8 puts a policy gateway in front of the same four operations. The loop code stays the same in every part.
Two smaller rules keep the step exact. First, read the clock once per step and use that value for the budget check and every event in the step. A replay can then compare deadlines against the same recorded time. Second, the model never sets a status. It returns a decision, and the harness maps the decision to one of a few legal transitions.
Budgets that force a stop
A step limit is the only limit that proves the loop ends. Each step either moves the run out of the running status or records one model call. The budget check stops the run when the call count reaches the maximum. So the loop runs at most max_steps + 1 times. No model output can change that, because model output never reaches the counter.
Two more limits protect cost and time:
| Limit | What it caps | Detail that matters |
|---|---|---|
| Steps | Model calls per run | The only limit that proves the loop ends |
| Tokens | Total tokens across calls | Checked before each call, so the last call can go over. To bound the overrun, count the input first and cap the call's output at the tokens left |
| Deadline | Wall-clock time | Store it as an absolute time. A paused run keeps the same deadline, and replay can compare it with the recorded clock |
A budget check between steps cannot stop a call that is already running. The model client and each tool also need their own timeouts.
Set limits per task. A classification task can allow one step and no tools. A research task can allow 20 steps and only read tools. Each extra step adds cost and another chance of error, as the OpenNash analysis of compound error and cost shows. Keep the step limit low until your evals in Part 7 justify a higher one.
Run state as an event log
Martin Fowler defines event sourcing as a way to "capture all changes to an application state as a sequence of events". You can then rebuild the state by applying the events again to an empty application (Event Sourcing). An agent run fits this pattern well. Its history is short and its order matters. After a failure, you need to know what the model saw, what it requested, and what each tool returned.
The harness keeps the log append-only and defines one fold function. The fold takes the current state and one event and returns the next state. It is the only code that changes run state.
The fold gives three guarantees that a mutable state object does not:
- One code path. The live loop, crash recovery, and replay all call the fold. A run rebuilt after a crash has the same state as the run before the crash.
- Validate before write. The harness applies a new event to a copy of the state first. If the fold rejects it, nothing reaches the log.
- Ordered writes. Each event carries a sequence number, and the state carries a version (the count of events applied). The fold rejects an event that does not follow the current version, and the store rejects a sequence number that another writer took. Two workers that advance one run get a conflict error instead of a forked history.
A chat transcript does not replace this log. A transcript does not record which tool results were real, which requests the harness refused, or which state version each step saw. The OpenNash post on durable agents makes the same point for long tasks: checkpoint the task state, not the transcript.
The log also feeds the model. A simple harness builds each model input from the goal, the inputs, every model reply, and every observation. That works for short runs. Part 5 replaces it with a context builder that selects and compacts per step, and records user turns and compaction in the same log.
A small status machine
The model returns decisions. The harness maps them to status changes, and a single table lists every legal change. A status with no entry in the table is terminal. A completed run then cannot restart, even if a later event asks for it.
Seven statuses cover most agents. Keep the set small. A rejected write, an expired approval, or a denied tool call returns to the run as an observation. None of them needs a new status. Each new status needs a row in the transition table, a CLI exit code, and a test.
completed means that the loop ended normally. It says nothing about task success. Part 7 adds graders that check the result and the final environment state. Part 3 counts a write as done only with a receipt from the system of record.
A paused write resumes with a reviewer decision. The harness records the decision as a status change. A rejection returns to the model as an observation. An approval calls the commit operation and records the receipt like any other tool result. The pause and the receipt are both in the log, so an audit can show who approved what and when.
The CLI as a contract
A command-line interface gives every caller one entry point: developers at a terminal, test suites, schedulers, and CI. The Command Line Interface Guidelines state the rules that matter for automation. Primary output goes to stdout, messages and errors go to stderr, and the program returns zero on success and non-zero on failure.
An agent CLI needs two commands at minimum. One starts a run from a task file that holds the contract and the budget. The other replays a recorded run by its ID. Progress lines go to stderr. The final run summary goes to stdout as JSON: run ID, status, steps, tokens, version, result, and any pending request.
Map each final status to one exit code, so a shell script can branch without parsing JSON:
| Exit code | Status | What the caller does next |
|---|---|---|
0 |
completed |
Read the result from stdout |
2 |
usage error | Fix the command |
10 |
needs_input or needs_approval |
Show the result or the pending request to a person |
11 |
budget_exhausted |
Raise the budget or change the task |
12 |
failed |
Read the error and check the model client |
130 |
cancelled |
Nothing, because the caller pressed Ctrl+C |
Two codes follow shell conventions. Bash builtins return 2 for incorrect usage, and a process killed by signal N exits with 128 plus N (Bash manual, Exit Status). SIGINT is signal 2, so Ctrl+C gives 130. When the user presses Ctrl+C, the CLI records a cancelled status before it exits, so a log never ends in an unknown state. The values 10 to 12 are a local choice. Pick any free codes and document them.
Treat the stdout JSON as a public contract. Add fields when you need them, but do not rename or remove fields that scripts already read. Keep secrets, raw prompts, and private tool results out of the default output.
Replay: three operations with one name
The word "replay" covers three operations. Two of them make no live calls. The third can repeat a production write.
| Operation | What it does | Calls the model | Calls live tools | Use it for |
|---|---|---|---|---|
| Reconstruction | Applies the recorded events to rebuild state | No | No | Debugging and audit |
| Test replay | Feeds recorded model and tool outputs through the current code | No | No | Regression tests in CI |
| Re-execution | Runs the same task again | Yes | Yes | A new run, with new data and new side effects |
Re-execution can repeat a refund or an email. Never use it to debug a production write.
Test replay is the main payoff of the event-log design. The replay feeds the recorded model replies and tool results, in order, to the current harness. It writes to an in-memory log and uses the recorded timestamps as its clock. Then it compares the new events with the recorded ones and reports the first position where they differ.
In the example above, a developer edits the step and breaks the write check. The recorded run paused for approval at event 6. The changed code sends the write request down the read path, so the replay produces an observation at event 6. The replay exits non-zero, and the change fails in CI before it reaches a user. Run replay over a folder of recorded runs on every harness change. The OpenNash post on agent traces and tool calls lists what else to capture when traces feed failure analysis.
Replay has two limits. A cancelled run diverges at the cancel point, because the cancel signal is not a model or tool output. Replay also tests only the harness. It cannot tell you whether a different model or prompt makes a better decision. That is the job of evals in Part 7.
Test the harness itself with a scripted fake model, as Part 1 recommends. Write one test per status and one per branch of the step. A model that requests the same read 50 times must hit the step limit. An unknown action must become an observation. A write request must pause without running. A terminal run must reject a restart. A second writer must get a conflict. These tests check what the harness does with any model output. Measure the model's own decisions with live-model evals in Part 7.
Gaps between a prototype and production
A single-process harness with a file log is correct for development. Close these gaps before real users depend on it:
| Gap | Risk | Production fix |
|---|---|---|
| Concurrency | Two processes race between reading the log and appending | Store events in a table with a unique key on run ID and sequence. Take a lease on the run before the model call, so two workers do not both pay for it. See Two Agents, One Record |
| Durability | A flushed write can still be lost in a power failure | Sync to disk or commit in a database transaction |
| Model retries | Blind retries waste time on errors that cannot recover | Retry rate limits and server errors once after a wait. End the run on authentication or quota errors with a clear message |
| Unknown write outcomes | A timed-out write may have succeeded | Idempotency keys and receipts from Part 3, so the harness can ask the target service first |
| Sensitive data | The log stores model output and tool results | Redact secrets before writing, and apply access control and retention to the log |
| Long logs | A rebuild from event 0 is slow for thousands of events | Store a snapshot from time to time and replay only later events, as Fowler describes |
| Long waits | A run waits days for a person, or must survive a deploy mid-step | Run the harness on a durable workflow engine or a job queue |
Anthropic's article on harnesses for long-running agents reaches a related conclusion from the other direction. Each new session starts with no memory of the previous one. So the harness must leave a progress record and a clean state for the next session. For a single run, the event log is that record.
Where the harness runs
You need a harness only when the model chooses the steps. Part 1 shows when a single model call or a fixed workflow is the better design. Once you need the loop, the open choice is where it runs.
| Situation | Use |
|---|---|
| Runs finish in minutes and one process can own each run | A harness in your service, as described here |
| Runs wait hours or days for people, or must survive deploys | The same harness on a durable workflow engine or a job queue |
| You need many integrations quickly and accept a vendor's limits | A framework or platform, after you test it against the checks above |
If you choose a framework, read its loop code before you trust it. Find where it counts steps, how it stores state, and what it does with a reply that does not parse.
How OpenNash helps
OpenNash puts forward-deployed engineers on your team to build agent run loops for Fortune 2000 companies. We define the task contract and run statuses for one workflow with your team. Then we build the harness, the event log, and the replay tests in your codebase. We connect real models and tools only after replay passes in CI. You own the code, the log schema, and the tests.
Book a call with an OpenNash forward-deployed engineer or email us at hello@opennash.com. In the first session, we review your candidate workflows and show which ones deliver the highest business value and the fastest ROI.
Previous: AI Agents Part 1: Architecture
Next: AI Agents Part 3: Tools and APIs explains how the harness connects an agent to external systems.