Cost

AI spend behaves differently from the spend your finance processes were built for. It is elastic, it can move an order of magnitude on a prompt change, and agents spend money in a loop.

Why the invoice is too late

A monthly invoice tells you what happened four weeks ago, in aggregate, by provider. That is enough to be alarmed and not enough to act, and by the time it arrives the loop has been running for a month.

The specific mechanisms that produce surprises:

Retry loops. An agent that fails and retries without a bound will spend continuously. This is the single most common cause of a shocking invoice.

Context growth. A prompt that accumulates history gets more expensive with every turn, and the cost curve is quadratic in a conversation rather than linear.

A model change. Moving to a larger model for quality reasons is a defensible decision. Making it without knowing the multiplier is not.

Cache invalidation. If you rely on prompt caching, a small change to a prefix can silently multiply cost while everything looks identical.

Batch jobs discovering more data. A nightly job over a growing corpus grows with it, and nobody is watching the corpus.

All five are visible within hours in a spend-rate metric and invisible until the invoice without one. See metrics.

Attribution first, allocation second

You cannot control what you cannot attribute. The prerequisite is that every inference call carries an application, a team, and a user, which is the argument for a gateway restated in financial terms.

With attribution you can do the two things that matter:

Showback. Tell each team what they spent. This alone changes behaviour more than most policies, because most overspend is accidental and the person causing it would fix it if they knew.

Chargeback. Move the money. Worth doing when spend is material enough that teams should be trading it off against other budget, and premature before that.

Quotas beat approvals, again

A per-team monthly spend ceiling with alerting at 50, 80, and 100 percent gives teams freedom inside a bound and gives you a conversation at the edge instead of a surprise.

Two refinements. Set a rate limit as well as a total, because a total does not stop a loop from consuming the month’s budget in an afternoon. And make the ceiling raisable in minutes, or teams will hoard headroom by over-requesting.

Cost is a design parameter

Once cost is visible per request, it becomes something engineers can optimise, and the usual wins are large and unglamorous:

• Right-size the model. A great deal of production traffic is classification and extraction that a smaller model does correctly.
• Cache aggressively. Both prompt caching and plain result caching for repeated questions.
• Bound the loop. A maximum tool-call count per session is a budget control and usually also a quality control, since an agent on its fifteenth attempt is not about to succeed.
• Trim context. Most long prompts contain material that stopped being relevant several turns ago.