# Runtime

> Where AI apps and agents actually execute. Containers, functions, and jobs, and why the interesting workload is the long-running one.

Source: https://brocs.fyi/run/runtime/
Part of the surface: R · Run. Give the work somewhere to live.
Framework: BROCS (Build, Run, Observe, Control, Secure), brocs.fyi. CC BY 4.0, attribute to brocs.fyi.

---

## The Run surface

- Runtime: where apps and agents actually execute
- Data and state: provisioned, owned, backed up
- Secrets: injected at runtime, never in code or chat
- Provisioning: self-service, provision-build-deploy as one motion
- Portability: the runtime goes where the data goes


Runtime is the answer to &#34;where does this thing execute&#34;. For a chat interface that
question is boring. For an agent that runs for forty minutes, calls tools, writes files,
and needs a credential the whole time, it is the question that decides whether the
project is real.

## Three shapes, and only one of them is new

**Request-response services.** A user asks, the app calls a model, the app answers. This
is a normal web service and your existing platform already knows how to run it. Nothing
here is special except token latency, which mostly means longer timeouts and streaming.

**Batch and scheduled jobs.** Classification over a corpus, nightly summarisation,
enrichment pipelines. Also solved: this is a job runner, and the AI part is a step inside
it.

**Long-running agents.** This is the one that breaks existing platforms. An agent session
is stateful, lives for minutes or hours, holds credentials for that whole period,
executes code, writes to a filesystem, and can be interrupted and resumed. It is closer
to a developer workstation than to a web request, and platforms built around stateless
twelve-factor services have nowhere natural to put it.

Most organizations discover this by trying to run an agent on their existing serverless
platform and hitting a timeout limit, then quietly moving it to somebody&#39;s VM. That VM is
now production infrastructure with no owner.

## What an agent runtime actually needs

Working through the requirements honestly gives a short and demanding list:

- **A filesystem** that survives the length of a session and is destroyed after it.
- **Execution isolation** strong enough that generated code cannot reach the host or
  other tenants. The threat model here is not a malicious insider; it is a model that
  read a web page containing instructions.
- **Egress control.** An agent with unrestricted outbound network access can exfiltrate
  anything it can read, and it does not have to be compromised to do it, only confused.
- **Credential injection at runtime**, scoped to the session, expiring with it. See
  [secrets](/run/secrets/).
- **Interruptibility.** A human needs to be able to stop it, and stopping it has to
  actually stop it rather than orphan the work.
- **A record of what it did**, emitted as it goes. See [traces](/observe/traces/).

Notice how many of these are boundaries rather than capabilities. The runtime&#39;s job is
mostly to be a box with well-understood walls.

## Blast radius is a runtime property

The most consequential runtime decision is what the executing code can reach. It is worth
writing down explicitly for each workload, because the default answer is almost always
&#34;more than you think&#34;: the network the container sits on, the cloud role attached to it,
the databases reachable from that network, and any credential in its environment.

An agent that can write to production because it happens to run on a host with a
production role is a design decision nobody made.

{{&lt; failure id=&#34;replit-prod-db&#34; &gt;}}

## Internal alternatives to the platforms people actually like

There is a reason developers reach for Railway, Render, Heroku, or a cloud&#39;s app service:
those products answer &#34;where does it run&#34; with one command and no meeting. If your
internal answer is a Kubernetes namespace request form, people will keep reaching.

The realistic goal is not to build a better cloud. It is to present the same small
interface over whatever you already run: push, get a URL, get logs, get a database, get a
secret. Underneath it can be Kubernetes, ECS, VMs, or all three. The interface is the
product; the substrate is an implementation detail, and treating it that way is what
makes [portability](/run/portability/) possible later.

{{&lt; aside title=&#34;How SRE thinks about this&#34; &gt;}}
An SRE will ask what happens when the model provider is slow, not when it is down.
Degraded is the common case: latency doubles, a fraction of requests fail, and retries
turn a provider hiccup into a self-inflicted load spike. Timeouts, budgets, circuit
breakers, and bounded retries are the same tools as always, and they matter more here
because the dependency is slower and less predictable than a database.
{{&lt; /aside &gt;}}

## Multi-tenancy is not optional

Even inside one company, AI workloads are multi-tenant: different teams, different data
classifications, different model access. A runtime that assumes everything on it is
equally trusted will eventually put a finance agent and a public-facing chatbot on the
same host with the same egress.

Decide early which boundary carries the isolation: namespace, node, account, or cluster.
Changing that decision later is a migration.

{{&lt; checklist &gt;}}
- Where does a long-running agent execute today? Name the machine or the platform.
- What can that runtime reach: which network, which cloud role, which databases?
- If generated code tried to make an outbound connection to an arbitrary host, what stops
  it?
- Can a human stop a running agent, and does stopping it release its credentials?
- What happens to the agent&#39;s filesystem when the session ends?
- Which teams share a runtime boundary, and is that deliberate?
- If a person is working inside a remote workspace, what can that workspace reach on the
  outbound network?
{{&lt; /checklist &gt;}}