Worked example: a support agent
A vendor-neutral architecture review of one production AI agent across all five BROCS obligations.
This example applies BROCS to a fictional customer-support agent that answers product questions, reads account history, updates tickets, and issues refunds within a fixed limit.
Use it to see the level of evidence expected in a BROCS review. It does not prescribe an architecture or recommend a product.
Operating boundary
The agent runs in two regions. It retrieves from a product knowledge base, reads and writes the customer relationship system, creates support tickets, and may issue refunds up to $200. Larger refunds and low-confidence answers go to a human. Customer records must remain in their assigned region.
The operating boundary states where data may move, which actions require approval, and what the organization has promised customers. Those constraints drive the review.
Evidence review
| Letter | Capability that should exist | Evidence to ask for |
|---|---|---|
| Build | A sanctioned path for changing prompts, tools, retrieval logic, and evals | A versioned change, its test result, reviewer, deployment record, and time from approved change to a running environment |
| Run | A durable runtime with regional placement, secrets, state, retries, and a defined provider-failure path | A deployment manifest, region policy, checkpoint or queue behavior, restore result, and a demonstrated failover |
| Observe | One trace across retrieval, model calls, tools, human handoff, quality, latency, and cost | A real production trace, an eval trend, an alert with an owner, and cost per resolved case rather than cost per token alone |
| Control | Versioned configuration, identity, staged rollout, rollback, routing, and action limits | The current configuration, who changed it, canary criteria, a rollback drill, and proof that the refund tool can be disabled without taking down answers |
| Secure | Permissions and policy enforced at retrieval and tool boundaries | A denied cross-customer retrieval, scoped tool credentials, retention behavior, a prompt-injection test, and an incident record that reconstructs what the agent accessed and changed |
The review is complete when each owner can produce the evidence without reconstructing it by hand.
Example scoreline
Suppose the assessment returns B4 R3 O1 C1 S2.
The scoreline describes current coverage. Here, the delivery path is ahead of the operating controls. The team can ship changes faster than it can explain or reverse their effects, so Observe and Control need attention before launch.
Choose the unchecked capability that would make a likely incident shorter or less damaging.
For this system, the next work is an end-to-end action trace and a tested rollback path. The trace connects a bad refund to its retrieved document, model response, and tool call. The rollback path lets the team stop the behavior safely.
Initial work
- Give every case one trace identifier across retrieval, model calls, tool calls, and human handoff.
- Put model, prompt, route, tool permissions, and refund limit in a versioned configuration record.
- Send a small share of traffic to each change and define the signal that stops the rollout.
- Make retrieval enforce the caller's customer and regional permissions before any document reaches the model.
- Exercise one rollback and one provider-outage path in the environment that will carry production traffic.
These changes fit the existing stack and establish basic operating control.
Review questions
Use these before debating suppliers:
- Can someone show one case from user request to every read, model call, action, and final outcome?
- Which changes can ship without code, and are those changes still reviewed and reversible?
- What happens to in-flight work when the model provider, retrieval system, or ticketing tool is unavailable?
- Does the refund tool receive standing authority, or authority scoped to one approved case?
- Can the organization prove that a European customer record stayed in its assigned region?
- Who can stop actions while preserving read-only answers?
If several teams must assemble a special presentation, the system cannot yet produce operational evidence on demand.
Next: score your own coverage or take the buyer questions into the architecture review.