One production agent: tools, state and execution limits

An address-change request times out after an agent sends it to an order API. The customer presses the button again. Your application now needs to establish whether the first request succeeded before deciding what to do next.

This hypothetical incident is a useful design exercise. It exposes the responsibilities of the agent runtime: the application code that controls tool execution, persists progress and reports an outcome the user can rely on. A model response alone cannot establish what an external service changed.

Give the model a defined decision

Begin with the business process. A fixed sequence of form validation, API lookup and display can run as ordinary application logic. A model may still help interpret free text within that sequence.

An agent loop becomes useful when intermediate results determine the next step. An unusual request might need a document search, another system lookup or clarification from the user. Anthropic's workflow and agent distinction describes this difference between predefined paths and model-directed execution. The original article dates from 2024; the architectural distinction remains useful without adopting its tool examples as current requirements.

Define success narrowly enough to test. Preparing a valid address change for approval has a clear stopping point. Resolving every aspect of a customer's problem requires additional decisions and authority that should be scoped separately.

Make the tool contract describe the effect

Separate read tools from commands that alter a record. For each command, specify its arguments, resource scope and possible outcomes. A generic success message leaves the runtime with too little information to recover when something goes wrong.

A write result should identify the operation and distinguish confirmed, rejected and pending outcomes. A network error may leave the outcome unknown. That state needs its own handling instead of being presented as a failed business operation.

User identity and tenant membership must come from server-validated session context. Recheck access to the specific resource when the tool executes. An order identifier produced by the model is input to authorisation, not evidence that the requester owns the order.

Persist progress around the external call

An illustrative state model for the address change is:

The runtime receives the request and validates user identity and tenant. It stores the proposal and waits for approval of that version. With approval and current permissions, it executes the tool. A service confirmation allows a confirmed outcome. Without confirmation, the outcome remains unknown and the runtime reconciles it through a status lookup or human review. Rejection and cancellation exits are omitted here and must be defined by the application.

Add cancellation and rejection states for your application. The essential distinction is between a command that has not been sent and one that may already have taken effect.

Bind approval to a specific version of the proposed change. Editing the order or address produces a new proposal. Recheck permissions and business conditions after a long pause, since the world may have changed while the request waited for review.

Persistence systems, including the approach documented by LangGraph, preserve workflow state. The external integration still needs to handle a crash after dispatch but before its result is stored. Saving progress does not make an arbitrary remote write safe to repeat.

Decide how uncertainty ends

Set retry policies per operation. A read may permit bounded retries; a write depends on the receiving API's guarantees. Reuse an idempotency key for retries of one intended operation when the API supports it. Otherwise, use an operation lookup or human reconciliation to establish the outcome. AWS explains caller-provided operation identifiers in its idempotent API guide.

Bound total execution time and tool calls. Exhausting a budget should produce a useful pending or review state. The interface must distinguish an action awaiting confirmation from an action completed successfully.

Stopping rules should use observable events. Repeating the same call without acquiring new information can trigger escalation. A model's self-reported confidence is insufficient authority to continue a risky operation.

Test the application and the model separately

Use ordinary automated tests for argument validation, access checks and state transitions. Integration tests should cover slow APIs, malformed responses and process restarts. Include a duplicate user request while an earlier operation is unresolved.

Evaluate the model's decisions separately: choosing a suitable tool, collecting missing information and declining to act when the task cannot be completed. Include both successful cases and cases that should stop. Schema enforcement helps control output shape, while structured-output documentation notes that the returned content can still be wrong.

Capture the observable execution record: model version, proposed tool call, result and state transition. Prefer references and redacted fields where full payloads are unnecessary. Internal model reasoning is not a dependable logging interface, and unrestricted logs can create another copy of sensitive customer data.

Scope the first implementation

A useful first delivery is one tool operation with permission checks and tested failure recovery. LLM integration work should account for this application logic. An AI technical audit can examine an existing runtime before it receives broader authority.

Describe the tool you want to enable and the records it could change. That gives a concrete basis for deciding which choices belong to the model, which checks belong in code and where a person must approve the result.