Data access

Data access is where most AI projects actually stall. Not the model, not the framework: getting the right person or the right agent to the right data, with a boundary that survives review.

Two different problems that get confused

Builder access is a human question. An analyst needs to look at claims data to build something useful. That is a normal data governance problem with a normal answer: role, scope, purpose, approval, audit. Nothing about AI changes it except the volume of requests.

Runtime access is a machine question. The thing you built needs to read data while it runs, on behalf of whoever is using it. That one is different, and it is where the mistakes happen.

The classic error is solving the second by copying the first: give the application a service account with broad read access, and let the prompt decide what to show. That works right up until the model is asked politely for something else, or reads a document containing instructions written by someone who is not your user.

The property you want is that the data boundary is enforced outside the model. The application asks for data as the user, the data layer applies the user’s permissions, and the model only ever sees what that user could have seen anyway. Then a prompt injection gets you a rude answer rather than a breach.

Masking, scoping, and synthetic data

Not every use needs real data. Ranked by preference:

1. Synthetic or generated data for anything where the shape matters more than the contents. Most development and demos.
2. Masked or tokenized production data where realistic distributions matter. Mask at the source, not in the application, or you will find the unmasked copy in a log.
3. Scoped real data, narrowed to the rows and columns the job needs, time-limited.
4. Full production access, by exception, with an approval and an expiry.

The organizations that move fastest are usually the ones with a decent answer at level one and two, because most requests never have to climb to three.

The retrieval plumbing problem

Retrieval augmented generation is presented as a model technique. In practice it is a data engineering job with a model at the end, and the hard parts are the familiar ones:

Freshness. Your index is a copy. Copies go stale, and a confidently wrong answer from last quarter’s policy document is worse than no answer.

Permissions. If the index is built once with broad credentials and queried by everyone, you have built a system that launders access control. Permissions have to be carried into the index and applied at query time, or the index has to be partitioned by audience.

Provenance. When the answer is wrong, you need to know which document produced it. Without a citation trail, debugging is guesswork and correcting the source has no verifiable effect.

Deletion. When a record is deleted from the source, it has to leave the index too. This one is a legal requirement in several jurisdictions and it is routinely missed because the index is treated as a cache rather than as a copy of personal data.

Prompt logs are a data store

Every prompt and completion your organization sends is a record, and by default those records are retained by someone. They frequently contain exactly the data the rest of this page is about, pasted in by a person trying to get an answer.

Treat the prompt log as what it is: a data store with a classification, a retention period, an owner, and an access control list. Then decide deliberately whether it lives with your provider, in your account, or nowhere.