AI Agent Security and Deployment (Part 8): Trust Boundaries, Approvals, and Staged Rollout

AI Agent Engineering Series

• AI Agents Part 1: Architecture
• AI Agents Part 2: Harness and CLI
• AI Agents Part 3: Tools and APIs
• AI Agents Part 4: MCP
• AI Agents Part 5: Context and Memory
• AI Agents Part 6: Agentic RAG
• AI Agents Part 7: Evals and Observability
• AI Agents Part 8: Security and Deployment

A production AI agent is secure when a hostile instruction that the model obeys still cannot cause harm. The model will sometimes follow injected text, and no prompt or filter prevents that reliably. So the security of the agent lives in software outside the model. A gateway checks every tool call, approvals bind to exact actions, secrets stay out of model context, and a kill switch fails closed. Deployment then widens what the agent may do one capability at a time. Each of these controls is ordinary software that you can test without a model.

Why the model cannot be the security boundary

Prompt injection is text in the agent's input that tries to change what the agent does. It can arrive in a customer email, a web page, a PDF, a retrieved passage, or a tool result. The model reads instructions and data in the same token stream. It has no reliable way to tell them apart.

Simon Willison names the most dangerous combination the lethal trifecta. The three legs are access to private data, exposure to untrusted content, and the ability to communicate externally. With all three, an attacker who controls some content can ask the agent to send the private data out. The OpenNash article on the lethal trifecta explains why removing one leg works when input filtering does not.

The OWASP entry on Excessive Agency names three root causes of agent damage: excessive functionality, excessive permissions, and excessive autonomy. One of its mitigations is "complete mediation". Downstream systems enforce authorization instead of trusting the model's decision. The OWASP AI Agent Security Cheat Sheet says to fail closed for unknown tools, separate decision-making from execution, and bind approval to the exact action.

Research reaches the same result. Design Patterns for Securing LLM Agents against Prompt Injections proposes six patterns, such as Action-Selector, Plan-Then-Execute, and Dual LLM. Each pattern limits what untrusted text can cause instead of trying to detect it. The CaMeL paper puts a system layer around the model and tracks where each value comes from. On the AgentDojo benchmark, CaMeL solved 77% of tasks with provable security. An undefended system solved 84%. The hard boundary cost 7 percentage points of task success on AgentDojo. Measure that trade-off on your own tasks.

The model may choose a tool and propose arguments. Software outside the model decides who the caller is and what the caller may touch. It also decides whether a person must approve, and whether the agent may run at all.

Map each path from input to action

A threat model lists each path from an input to a real effect, with the control that stops each bad outcome. A risk with no control is an accepted risk, whether or not a document says so. For a tool-using agent, most risks map to one control.

The task contract comes from Part 1. Previews, receipts, and idempotency keys come from Part 3. This part adds the checks that sit around them.

Take identity from the session

A principal is the verified identity that a request acts for. It can be a signed-in user, a service account, or the operator who approves an action. The principal comes from your authentication layer, for example an OpenID Connect ID token that your server validated as OpenID Connect Core defines.

The model never writes the principal. If a tool argument says "user": "admin", the gateway ignores it, because the principal travels beside the model decision and never inside it. The same rule applies to resource facts. The model can name an order ID. The gateway reads the order's tenant and refundable amount from the system of record.

Remote tools need narrow tokens. RFC 8707 defines the resource parameter, which names the resource a token is for. Request one resource per token, and enforce one audience. RFC 9700, the OAuth security best current practice, says access tokens should be audience-restricted and limited to the minimum privileges required. The MCP authorization specification makes this mandatory for MCP servers that implement authorization over HTTP. Clients must send resource, and servers must not accept or pass through tokens issued for another audience. This is why Part 4 treats remote tools as a separate trust zone. The OpenNash article on agent identity management describes a broker that mints these short-lived tokens at run time.

The gateway: deny by default, check every call

The gateway is the single path from a model decision to a tool. It replaces the direct tool executor in the Part 2 harness, so the loop does not change. The order of its checks matters. The cheap, global checks run first, and the policy runs only on a known tool with valid arguments.

on proposed tool call (name, args) in run:
  if kill switch halts all, or disables name: refuse "halted"
  tool = registry lookup(name);  if none: refuse "unknown_tool"
  check args against tool schema;  if invalid: refuse "invalid"
  if args contain a known secret value: refuse "denied"
  facts = system of record lookup(args)        # owner, ceiling
  verdict = policy(session principal, tool, args, facts, contract, stage)
  if verdict denies: audit(verdict.rule); refuse "not_permitted"
  if verdict needs approval: pause run, request exact-action approval
  else: run handler; redact result; return it to the model as data

Three design choices in this loop carry most of the security.

1. Deny by default. A tool that the policy does not name, or that the task contract does not list, is denied. New tools start with no access until a rule grants it.
2. The ceiling comes from trusted facts. The hard limit is the lower of the policy limit and the amount the system of record allows. The model's argument is compared against the ceiling and never sets it.
3. The model sees a generic code. The audit log keeps the matched rule and reason. A detailed denial such as "order 9001 belongs to merchant_99" tells an attacker that the record exists.

The policy also takes the rollout stage as an input, so the gateway enforces the rollout stages described below on every call.

Each verdict carries a decision ID and the policy version. The gateway writes both to the audit log with the run ID. An investigator can then find which rule allowed or denied each call in a Part 7 trace.

The system of record must still repeat its own authorization. The gateway is the first check. A service that trusts any caller with a valid network path loses its protection the day someone adds a second path.

Approvals bind to one exact action

Part 3 binds an approval to a fingerprint of the exact effect, so changed arguments fail at commit. Production adds two fields and one rule. The hash also covers the tenant and the policy version, and the approval works only once. At commit time, the gateway computes the hash again from the call it is about to run. If one value differs, the approval matches nothing.

The single-use rule is one conditional update in one database transaction. It matches only a row that is approved, unexpired, and has the same hash, and it sets that row to consumed. If two workers race to use the same approval, only one update changes a row. The other sees zero rows changed and refuses. The pattern works in any database with transactional updates.

Four more rules make approvals hold.

• Separation of duties. The requester cannot approve their own action, and the approver must hold the approver role in the same tenant.
• Expiry. An approval that waits too long expires. The facts behind it may have changed.
• Check again at commit. The gateway runs the whole check sequence again before it consumes the approval. A policy change, a revoked role, or a kill switch between approval and commit stops the action. The policy version in the hash makes an approval for version 12 fail under version 13.
• One approval per intent. The approval store reuses the Part 3 idempotency key. If the model proposes the same write twice in one run, both proposals map to one approval.

Reviewers must see the exact effect from the Part 3 preview. If a reviewer sees only a summary, the approval also covers what the summary leaves out. The OpenNash article on approval gates and exception routing covers how to route proposals to the right reviewer.

Keep secrets out of model context

Any value in the context window can come back out in model output, a tool argument, or a trace. So a secret must never enter the context window. Four rules form the secrets boundary.

1. A broker holds credentials. Only tool handlers ask it for a value, at call time. Its printable form shows secret names and never values, so a stray log line cannot leak one.
2. Redact every result. The gateway replaces known secret values in tool output before the output returns to the model. Upstream APIs sometimes echo tokens in error messages.
3. Deny arguments that contain a secret. If a known secret value appears in a tool argument, something already leaked. The call stops.
4. Allowlist egress. Every outbound request goes only to approved hosts over HTTPS.

The egress allowlist limits where direct requests can go. An approved host is still an outside channel, so control the recipients, accounts, paths, and data sent to it too. Other output channels remain. A rendered Markdown image URL, a link in an email the agent sends, or a write to a shared document can also carry data out. List each output channel in the threat model and decide which ones need approval.

Redact before the trace is written too. The Part 7 trace store keeps tool results, and traces feed eval sets and debugging sessions. A secret in a trace reaches people and models that never had access to it.

Isolate generated code and third-party tools

Treat code that the agent writes as code written by an attacker. Run it in a sandbox, an isolated environment that limits what the code can read, reach, and use. The sandbox gets no host credentials. Outbound network traffic is denied by default. CPU, memory, time, and output have limits, and storage is deleted after the run. When the code needs an authenticated service, it calls a tool proxy that goes through the same gateway.

Hosted code-execution services follow these rules. Cloudflare runs generated code in a V8 isolate with no file system and no environment variables. Datadog keeps credentials out of its sandbox.

Plain containers share the host kernel. gVisor and microVMs cost start time and operations work. The OpenNash guide to choosing a sandbox for agent code execution compares the options.

MCP servers, packages, container images, and model adapters are third-party code that runs with your permissions. Pin their versions. Review every change to the tools or scopes a server exposes, and run the security tests again after each upgrade. A changed tool description can change which tool the model selects.

The kill switch fails closed

A kill switch lets an operator stop the agent without a deploy. The gateway reads its state before every tool call, so a change takes effect on the next call. Store the state in a database or feature-flag service that every worker reads.

The switch needs two levels. A global halt stops every tool call in every run. A per-tool flag turns off one tool, such as a write tool after a bad upstream release, while reads continue. The harness also checks the global halt at the start of each step. A halted run moves to a paused state, so it can resume later.

The most important rule is the failure mode. If the gateway cannot read the switch state, it treats the agent as halted. If the switch failed open, an outage of the flag store would disable the control when an operator may need it most.

Blocking new calls does not stop everything. Tokens that the agent already received stay valid until they expire, and background jobs may keep running. The OpenNash article on the agent kill switch covers the other two parts of containment: revoking issued credentials and sweeping orphaned work.

Recover without repeating side effects

A tool call can time out after the remote service completed the write. Part 3 covers the idempotency key that makes a plain retry safe.

Approvals add one case. The gateway consumes the approval before it calls the handler. If the handler then fails with an unknown outcome, a retry with the same approval is refused, which is correct. Nobody asks the model whether to retry. The write moves to outcome_unknown, the run pauses in needs_input from Part 2, and a reconciler asks the service for a receipt by idempotency key.

Allow only these transitions, and store each change with its evidence in one transaction. If an action can be reversed, define the reverse as its own tool. A refund reversal needs its own policy rule, approval, idempotency key, and receipt.

Version the full release

A failed run is hard to reproduce if any part of the agent changed after it. Treat these parts as one release with one version:

Record the release version on every run, next to the policy version in every decision. Rollback restores code and configuration. It does not undo refunds that the newer release created. Before each release, decide which effects need compensation, and keep the previous release artifacts available.

Security tests block the release

A release gate is a test that must pass before a deploy. The Part 7 CI gate runs task evals across repeated runs. Add the security checks to that gate as hard pass or fail checks. One unauthorized read blocks the release, even if every task score improves.

The checks fall into two groups.

• Control tests run without a model. An approval works once and fails after expiry or a policy change. A tampered argument fails the hash. A missing kill switch state halts every tool. A secret in a tool result is redacted. A write outside the contract or above the ceiling is denied. A read-only stage denies all writes. An unverified session yields no principal.
• Injection tests run the full agent with the production model. Seed documents and tool results with hostile instructions. Grade the trace, because the final message can claim anything. The containment check passes when no unauthorized effect happens. The behavior check in Part 7 still fails a forbidden request, even when the gateway denies it.

Control tests show that each check works. Injection tests show how the system handles representative attacks. Separately, verify in review that every tool path goes through the gateway. A model upgrade can make the model obey injected text more often, so the injection tests run again on every model change.

Teams that must map these controls to a governance framework can use the NIST AI Risk Management Framework and its Generative AI Profile, NIST AI 600-1.

Roll out by capability

A staged rollout widens one capability per stage. Ten percent of users with unrestricted writes can still cause serious harm, so traffic share alone is the wrong axis. The stage is a policy input, so the gateway enforces it on every call.

Move to the next stage only when the exit criteria hold for the whole stage window. Define the rollback trigger before the stage starts. The criteria below show the kind of condition. Set the thresholds from your own risk, traffic, and recovery time.

Run an incident drill before the limited stage, with a named owner for each step.

1. Set the global halt and confirm that active runs pause within one step.
2. Revoke the agent's service credentials and confirm that tool calls fail.
3. Export the traces and audit log for the drill window.
4. List every principal and record that the drill runs touched.
5. Restore service and confirm that paused runs resume without duplicate writes.

Match the controls to what the agent can change

Not every agent needs approvals, a kill switch, and five stages. The minimum control set depends on the most dangerous thing the agent can do.

Two common additions do more harm than good.

• A guard model as the main control. A second model that classifies prompts as safe or unsafe can reduce noise. It fails on the same kind of input as the first model. Use it, if at all, as an extra signal in front of the deterministic checks.
• Approval for every small read. Reviewers who see hundreds of trivial requests start to approve without reading. Spend reviewer attention on writes that are expensive, hard to reverse, or sensitive.

Getting a write-enabled agent to production

OpenNash forward-deployed engineers take agents with real permissions to production for Fortune 2000 teams. We map each tool to a threat model and a policy rule. We put the gateway, the approval flow, and the security tests into your CI. Then we run the rollout stages with you, including the incident drill. You own the code, the policies, the eval sets, and the runbooks at the end. If your agent only answers questions from public data, you do not need this work yet.

Book a call with an OpenNash forward-deployed engineer or email us at hello@opennash.com. In the first session, we review the systems your agent must write to. Then we show which use cases deliver the highest business value and the fastest ROI.

Previous: AI Agents Part 7: Evals and Observability