An agent's final answer doesn't tell an operator which actions succeeded, which failed, or what will happen if someone presses retry. An agent allowed to change business data needs a record the operator can inspect and use to recover the work. That requirement belongs in the design before the system gets permission to act.
A timeout makes the distinction concrete. A request can reach an outside service even when its response never reaches the agent. Repeating the request might repeat the action. A polished answer at the end of the conversation won't resolve that uncertainty. The operator needs the attempted call, its status and a way to check what happened outside the conversation.
An agent combines a model, a runtime that repeats steps, and tools that can read or change something beyond the model's answer. The model proposes the next action. The runtime decides whether that action is allowed, executes it, and returns an observation. The next step can change because of what came back.
That flexibility is useful when the work can't be specified as one fixed sequence. It also gives the operator more to account for. A usable system makes its actions visible, saves enough state to resume deliberately, and stops when it reaches a limit or needs a person. The first design decision is whether the job requires that flexibility at all.
Choose the job before the agent
A report that always imports the same data, checks the same fields and produces the same output can run as a pipeline. A model might summarize a result at one step without choosing the whole sequence. An agent becomes useful when observations determine what to do next, such as deciding which source to investigate after a search returns incomplete evidence. The number of tools doesn't settle that choice.
Redwood makes scope explicit
I came to agent scoping through unfinished projects. I was building an agentic video-game builder and kept running into the consequences of trying to put the system together without deciding what it needed. That work led to Agent Redwood, a way to make the design choices explicit before building further. It's an assessment of scope, separate from whichever software runs the agent.
Redwood asks about twelve components: personality, planning, mission, constraints, memory, evaluation, tools, awareness, reward model, metadata, strategy and integrations. I rank their importance for the job and decide how deeply to develop each. The list gives a team something specific to disagree about. A research assistant may need careful source evaluation and persistent memory. A bounded data operation may need strict constraints and a small set of integrations, with little investment in personality.
The assessment still leaves ordinary engineering to do. Observability, tests, deployment and version control don't disappear because the team has named the agent's mission. My original scoping problem was deciding what to build deeply enough to finish, and what to leave out.
Choose steps from the job
Start by asking what problem exists independently of the proposed solution. 'We need an agent' names a preferred implementation. A specific manual task, the information available to do it, and the result a person needs give the team something it can design and evaluate. Keep the steps fixed wherever the job permits. Give the model a choice where the next step needs judgment, then define how that choice will be recorded.
Save enough state to recover the loop
An agent run is a job with intermediate results. The operator needs to know where that job stopped and which results remain usable. Saving only the conversation's final answer loses the information needed to make that decision.
The runtime records each transition
The loop has an order even when the model changes its plan:
- The model proposes a plan and selects an action within its allowed tools.
- The runtime validates the proposed call, checks permission and executes the operation.
- The runtime records the result or failure and returns an observation to the model.
- The model uses the observation to continue, revise the plan or finish. The runtime enforces the run's limits.
A checkpoint preserves the state needed at one of these transitions. The saved state includes the current plan, completed operations, pending work and observations the next step depends on. Record the call before executing it, then record the outcome. Otherwise a process failure between the outside action and the saved result can leave an operator unable to tell whether the action ran.
Recovery controls have different effects
A checkpoint gives the operator a recovery point. It doesn't make a model's next answer deterministic, and it doesn't reverse a change in an outside system. A replay of recorded events can show what happened without executing those events again. A retry asks an operation to run again. Those controls need different meanings in the interface.
A request that times out can have an unknown outcome, even when the local call reports failure. An operator may need to query the outside service before choosing a retry. An operation designed to recognize a repeated request can avoid applying the same change twice. That behavior has to exist in the tool or service. A saved conversation can't supply it.
The operator's recovery options should be explicit: resume pending work, retry a known failure, branch from saved state, end the run, or hand it to a person. A branch preserves an alternative route through the work. It leaves earlier outside actions in place. Budget exhaustion and a request for approval also need recorded stopping states, so neither looks like successful completion.
Validate tools and check permission before execution
A tool turns a model's proposed action into an operation against a real system. The runtime has to check that proposal before anything changes. A typed input model describes the arguments an operation accepts; a typed result describes what the caller can expect back.
Typed models describe accepted data
I use Pydantic to make the data passing through a Python system explicit. The same approach gives an agent tool a defined boundary. An email tool can require a recipient and a body. A customer-record tool can require an identifier in a specific format. Invalid arguments should produce an explicit validation error before the operation runs.
Permission limits the action
A valid identifier still doesn't establish permission to change that customer's record. A schema can reject a missing identifier and accept a perfectly formed identifier belonging to someone else. Authorization has to check the caller and the resource. An operation that deletes data or sends a message may also require a person's approval. Restrict the available tools and their credentials to the work the agent is allowed to do.
The distinction affects the tool's design. Reading a record, preparing a proposed change and applying that change can have separate contracts. The model can inspect information or draft an action without automatically receiving permission to execute it. Error results should distinguish bad arguments, denied access and an outside service failure, because each requires a different next step.
Reusable tool functions make those contracts easier to exercise independently of a conversation. A test can check whether invalid inputs are rejected or a repeated request changes a record twice. Hypothesis is one technique I've used to explore input variations around such properties. Passing a tool test establishes only what that test checks. The agent can still select the wrong valid tool or use a valid result to reach the wrong conclusion.
Prompts and tool availability can change an agent's behavior without changing the tool's implementation. Preserve those settings with the run, so a later investigation can establish which instructions and permissions were in effect. The operator also needs to see the action while there's still an opportunity to intervene.
Show actions while the run is happening
An interface can show the work between a request and its answer. A contract-search agent, for example, might search an index, read a returned passage and synthesize an answer. Showing those operations lets a user inspect the selected source and stop an irrelevant line of inquiry before more work follows.
An AG-UI integration should give the application enough execution events to render that sequence. The runtime emits events as operations begin and as results or errors arrive. The application associates each result with the call that produced it, then shows the relevant information in a component suited to the result. A source passage needs a citation a reader can inspect. A failed search needs a visible failure state.
The event contract belongs in the design alongside the tool contract. The application needs stable call identifiers, statuses and result shapes it understands. The transport might use server-sent events or WebSockets. That choice doesn't determine whether the interface explains what happened. Keep secrets and private tool arguments out of the user-facing stream, and restrict detailed diagnostic records to people authorized to inspect them.
A stop control also needs a runtime contract. The interface should say whether it has requested a stop or the run has reached a checkpoint and paused. An outside action already in progress may finish. The runtime must report that outcome before an operator decides what to do next.
Visible tool calls help locate a mistake. They don't establish that a retrieved passage supports a generated answer or that a completed operation was the right one. The user still needs access to the evidence, and the engineer needs the saved event sequence when investigating the result.
Split workflows where the responsibility changes
A larger job can contain both fixed steps and model-selected ones. A workflow makes those boundaries explicit. One step gathers evidence, another checks whether the evidence is sufficient, and a later step produces an output. Some steps may be agents. Others can be ordinary code.
Each step needs a defined handoff
LangGraph is an orchestration tool I've used for custom agent workflows connected to business automation. The architectural question comes before the choice of library: what state does each step receive, what must it return, and which conditions allow the job to move on? A typed handoff gives the next step a defined input and gives the operator a place to inspect a failure.
A research-to-synthesis handoff makes the benefit concrete. The research step should return the evidence it found, its sources and any unresolved gaps. The synthesis step can then work from that record. A polished paragraph with no source trail would leave the next step unable to distinguish evidence from the previous model's interpretation.
That boundary can also contain failure. An engineer can inspect the research result before blaming synthesis, or check the handoff before changing either prompt. The workflow still needs handling for missing outputs, invalid results and steps that never finish. Moving work into separate agents creates those responsibilities. The graph doesn't remove them.
Dependencies determine which branches can run together
Independent branches can run together when neither needs the other's result. Research on separate subjects may qualify, with a later step joining the findings. Steps that depend on an earlier observation have to wait. Concurrent steps that change the same record need coordination even when their prompts look independent.
Split a job when a separate responsibility, permission or recoverable result justifies the handoff. A long tool list alone doesn't justify another agent. Keep a coherent task in one loop when that makes its state and failures easier to understand. Either design still has to demonstrate a useful outcome.
Evaluate the outcome and inspect the failed step
A successful tool call proves that an operation completed under its own contract. The business task may still be wrong. A research agent can retrieve valid documents and leave the question unanswered. An agent can also complete the task while taking unnecessary calls or requiring a person to repair its work. Evaluation needs to keep those outcomes distinguishable.
Outcome checks and step checks answer different questions
End-to-end evaluation asks whether the run delivered the expected result for a task. Per-step evaluation asks whether a particular tool choice, argument or observation was appropriate in the state where it occurred. The final result is what the operator evaluates for acceptance. The intermediate record is how the engineer investigates a failure.
Start with representative tasks and explicit expectations. Keep some cases out of prompt development so repeated tuning doesn't become the only reason the system passes. A case record can live in a CSV with fixtures for expected behavior. Include the input, allowed actions, expected output or stopping condition, actual outcome and a link to the trace. The number of rows should follow the behaviors and risks that need coverage.
Failure cases test the promised recovery
Failure cases belong beside successful ones. A denied operation should remain denied. A missing source should remain a visible gap. A service timeout should produce the recovery state the design promised. A run that reaches its budget should stop and report incomplete work. These cases test what an operator will be left holding when the intended route fails.
Logfire is one tool I've used to record structured operations. Whatever tracing system the team uses, preserve the call sequence, results, errors and relevant configuration so a failing case leads to inspectable evidence. Record cost alongside outcome. A lower token bill on an unfinished task doesn't establish that the agent became more efficient.
Compare those same cases when prompts, tools or routing change, with acceptance checks appropriate to the consequence of the change. A pass-rate dashboard helps locate regressions. The underlying cases show whether the change affected correctness, permission, recovery or cost, and whether the system is ready for the responsibility being proposed.
Ask the team to reopen a run
A vendor or internal team can demonstrate these decisions with an actual session. Ask them to reopen a completed run, show the tools it called and explain what happened at a failed or interrupted step. The record should make the result understandable without asking the model to reconstruct its own history.
Follow an action through its proposed arguments, permission check, execution result and next step. Check what the user could see at the time and what the operator can inspect afterwards. Ask which controls would repeat an outside action, which would only display recorded events, and how the system handles an outcome it can't confirm. Then compare the final result and cost with the task's acceptance criteria.
A team that can show that record gives the operator a basis for deciding what to automate and what still requires a person. A fixed pipeline may meet the need with fewer decisions to manage. A model-selected loop may be worth the added responsibility. The next investment should follow what the team can demonstrate on the job being proposed.
An operator facing a timeout has to account for the uncertain outcome before deciding to run the action again.
Related reading covers retrieval for an agent's evidence, architecture that keeps operations inspectable, and production observability.