Storage and backups
Retention, lifecycle, and restore drills for AI data, including the stores nobody put in the catalogue.
Run covers provisioning the stores. Control covers what happens to what is in them: how long it lives, who can reach it, whether you can get it back, and whether you can prove any of that.
Retention has to be a decision
The default retention for most AI data is forever, because nobody chose. Conversation history accumulates, prompt logs accumulate, traces accumulate, and every one of them is a copy of whatever people put in.
Set a period per store, based on what it contains rather than what it is called:
- Conversation history. Often the only record of what your system told a user, which makes it evidence. Long enough for disputes and support, short enough to bound exposure. Somewhere between 30 and 90 days is a common landing spot for internal tools.
- Prompt and completion logs. Assume they contain personal data and credentials regardless of policy. Short by default, longer only where an obligation requires it.
- Traces. Sample the routine, keep the abnormal. See traces.
- Vector indexes. No natural expiry, which is exactly why deletion has to propagate.
- Generated artifacts. Frequently forgotten, frequently in an over-permissive bucket.
The retention period is not a documentation exercise. Something has to enforce it, and that something has to be verified, because a lifecycle policy that was never applied looks identical to one that was.
Deletion has to propagate
When a person exercises a deletion right, or a document is withdrawn, it has to leave the primary store, the object storage, the index, the conversation history, the prompt log, the trace store, and any backup retained long enough to matter.
Write down the propagation path once, as a list of systems and mechanisms. Most organizations discover during that exercise that two or three of the copies have no deletion mechanism at all, which is much better discovered on a Tuesday than during a regulator’s clock.
Backups are a claim until you restore one
The failure is rarely that backups do not run. It is that nobody has proven a restore produces a working system.
For AI workloads, add two specific checks to the drill. Restoring the primary database without the vector index leaves a system that answers from nothing, so the drill has to cover both and the ordering between them. And if the index is treated as rebuildable rather than backed up, time the rebuild. Eleven hours and a large embedding bill is a fact worth knowing before the outage.
Put the drill in the calendar with a name attached, and record how long it took. The number is your actual recovery time objective, whatever the document says.
Access control on the telemetry
The trace store is often the least protected system holding the most sensitive material, because it was built by the platform team for debugging and read access was granted generously.
Apply the same standard you apply to the source data. If only four people can read the claims table, then only four people should be able to read traces containing claims data.
Questions to ask your vendor, or your own team
- What is the retention period for conversation history, and what enforces it?
- List every store that holds a copy of a customer document. Which have a deletion path?
- When did you last restore an AI application from backup, end to end, as a drill?
- How long does a full vector index rebuild take, and what does it cost?
- Who can read the trace store today? Compare that to who can read the source data.
- Is there a bucket of generated artifacts with public or broad access?