# Cost

> FinOps for AI: who spends what on which model, and why monthly reporting is too slow to be a control.

Source: https://brocs.fyi/secure/cost/
Part of the surface: S · Secure. Prove it is not going off the rails.
Framework: BROCS (Build, Run, Observe, Control, Secure), brocs.fyi. CC BY 4.0, attribute to brocs.fyi.

---

## The Secure surface

- Governance: policy encoded where it is enforced, not a PDF
- Agents: tool permissions, audit trails, human in the loop
- Shadow AI: discovery of what people use when you give them nothing
- Cost: who spends what, on which model, showback and chargeback
- Compliance: HIPAA, SOC 2, PCI, FedRAMP, EU AI Act as platform output


AI spend behaves differently from the spend your finance processes were built for. It is
elastic, it can move an order of magnitude on a prompt change, and agents spend money in a
loop.

## Why the invoice is too late

A monthly invoice tells you what happened four weeks ago, in aggregate, by provider. That
is enough to be alarmed and not enough to act, and by the time it arrives the loop has
been running for a month.

The specific mechanisms that produce surprises:

**Retry loops.** An agent that fails and retries without a bound will spend continuously.
This is the single most common cause of a shocking invoice.

**Context growth.** A prompt that accumulates history gets more expensive with every turn,
and the cost curve is quadratic in a conversation rather than linear.

**A model change.** Moving to a larger model for quality reasons is a defensible decision.
Making it without knowing the multiplier is not.

**Cache invalidation.** If you rely on prompt caching, a small change to a prefix can
silently multiply cost while everything looks identical.

**Batch jobs discovering more data.** A nightly job over a growing corpus grows with it,
and nobody is watching the corpus.

All five are visible within hours in a spend-rate metric and invisible until the invoice
without one. See [metrics](/observe/metrics/).

## Attribution first, allocation second

You cannot control what you cannot attribute. The prerequisite is that every inference
call carries an application, a team, and a user, which is the argument for a
[gateway](/build/keys-and-models/) restated in financial terms.

With attribution you can do the two things that matter:

**Showback.** Tell each team what they spent. This alone changes behaviour more than most
policies, because most overspend is accidental and the person causing it would fix it if
they knew.

**Chargeback.** Move the money. Worth doing when spend is material enough that teams should
be trading it off against other budget, and premature before that.

## Quotas beat approvals, again

A per-team monthly spend ceiling with alerting at 50, 80, and 100 percent gives teams
freedom inside a bound and gives you a conversation at the edge instead of a surprise.

Two refinements. Set a rate limit as well as a total, because a total does not stop a loop
from consuming the month's budget in an afternoon. And make the ceiling raisable in
minutes, or teams will hoard headroom by over-requesting.

## Cost is a design parameter

Once cost is visible per request, it becomes something engineers can optimise, and the
usual wins are large and unglamorous:

- **Right-size the model.** A great deal of production traffic is classification and
  extraction that a smaller model does correctly.
- **Cache aggressively.** Both prompt caching and plain result caching for repeated
  questions.
- **Bound the loop.** A maximum tool-call count per session is a budget control and
  usually also a quality control, since an agent on its fifteenth attempt is not about to
  succeed.
- **Trim context.** Most long prompts contain material that stopped being relevant several
  turns ago.

{{< aside title="How FinOps thinks about this" >}}
The FinOps discipline is built on three phases: inform, optimise, operate. AI programs
usually attempt to operate without informing, which produces blunt controls like a global
spend cap that stops useful work along with wasteful work. Inform first. Attribution is
the whole foundation, and it is cheap to add at the start and expensive to retrofit across
a mature estate.
{{< /aside >}}

{{< checklist >}}
- What did AI cost last month, by team? If the answer is one number by provider, that is
  the finding.
- Do you have a spend-rate alert, or only a total?
- What bounds an agent's retries and tool calls?
- Which team is your largest spender, and do they know?
- What percentage of your traffic goes to your most expensive model, and does it need to?
- If spend tripled tonight, how long until someone noticed?
{{< /checklist >}}