Cost

AI spend can move an order of magnitude after a prompt change. Agents can also spend repeatedly in a loop, which finance processes may not detect promptly.

Invoice lag

A monthly provider invoice reports aggregate spend after the activity occurred. It does not identify a running loop in time to stop it.

Unbounded retries continue spending while an agent fails. If every turn includes the full history, cumulative token use grows quadratically with conversation length. Model upgrades can introduce an unexpected price multiplier, and changes to a cached prefix can reduce the cache hit rate. A batch job over a growing corpus will increase spend with the corpus.

A spend-rate metric makes these changes visible before the next invoice. See metrics.

Attribute spend before allocating it

Every inference call should carry application and team identifiers so spend can be traced to its source. Include user identity where applicable. A gateway can apply that attribution consistently.

Showback tells each team what it spent and gives the owner enough detail to correct accidental overspend. Chargeback assigns the expense to a team when the amount is material enough to affect its budget decisions.

Set quotas and rate limits

A per-team spend ceiling with threshold alerts gives teams a defined budget and warns the owner before it is exhausted.

Set a rate limit as well as a total because a loop can consume a monthly budget quickly. Allow prompt review of ceiling increases so teams do not request excessive headroom.

Cost is a design parameter

Per-request cost data lets engineers make specific design changes:

• Use a smaller model for classification and extraction workloads when evaluation shows it performs correctly.
• Apply prompt caching and result caching to repeated requests.
• Set a maximum tool-call count per session.
• Remove context that is no longer relevant to the request.