Configuration

Configuration is everything that changes system behaviour without changing application code: prompts, model selection, routing rules, tool definitions, retrieval settings, guardrail policies, and feature flags.

In most AI stacks this material lives in three places at once: a database table edited through an admin panel, environment variables set at deploy time, and a few strings in the code. Nobody can say what the current state is, and nobody can put it back.

The prompt is configuration and it is load-bearing

A prompt is the most behaviour-changing artifact in the system and the one most likely to be edited in a text box by whoever is closest to the problem at four in the afternoon.

Treat it like code, because it is:

• In version control, with a diff and an author.
• Reviewed by someone other than the person who changed it, at least for anything customer-facing.
• Tested against the eval suite before it ships.
• Deployed through the same pipeline as everything else, with the same ability to roll back.
• Attributable at runtime: given a response, you can say which prompt version produced it.

The last one is the difference between “we think we changed it last Tuesday” and knowing.

The common objection is that prompts need to be editable by non-engineers, quickly. That is reasonable and it does not require abandoning review. A prompt-editing surface that commits to a repository and opens a pull request satisfies both. What does not satisfy either is a production text box with no history.

Declarative, not imperative

The state of the system should be described somewhere, and something should make reality match the description. The alternative, a sequence of changes applied by hand through a console, has the property that the current state exists only in the system itself and cannot be reproduced.

For AI configuration this matters more than usual because the surface is large and changes frequently. Which model each application uses, which tools each agent can call, which documents each assistant may retrieve, and what the guardrails allow are all security-relevant and all easy to change quietly.

Rollback is the actual requirement

Everything above exists to serve one property: when a change makes things worse, you can put it back quickly and know that you did.

Test it. Pick a prompt change from last month and time how long it takes to revert it in production. If the answer involves finding who made it and asking what it used to say, you do not have rollback, you have an archaeology practice.

Two specific traps:

Reverting the prompt does not revert the index. If a change also rebuilt a retrieval index, putting the prompt back leaves you in a state that never existed before.

Reverting configuration does not revert a model version. If the provider moved the model underneath a floating alias, your rollback restores your side of the change only. See routing.

Environments have to differ, on purpose

Development, staging, and production will use different models, different keys, different data, and often different guardrail settings. That is correct. What is not correct is for those differences to be undocumented, which is how a staging prompt reaches production or a production key ends up in a notebook.

Keep the shape of the configuration identical across environments and let only values differ, so a diff between environments is short and readable.