Runtime

Runtime is the answer to “where does this thing execute”. For a chat interface that question is boring. For an agent that runs for forty minutes, calls tools, writes files, and needs a credential the whole time, it is the question that decides whether the project is real.

Three shapes, and only one of them is new

Request-response services. A user asks, the app calls a model, the app answers. This is a normal web service and your existing platform already knows how to run it. Nothing here is special except token latency, which mostly means longer timeouts and streaming.

Batch and scheduled jobs. Classification over a corpus, nightly summarisation, enrichment pipelines. Also solved: this is a job runner, and the AI part is a step inside it.

Long-running agents. This is the one that breaks existing platforms. An agent session is stateful, lives for minutes or hours, holds credentials for that whole period, executes code, writes to a filesystem, and can be interrupted and resumed. It is closer to a developer workstation than to a web request, and platforms built around stateless twelve-factor services have nowhere natural to put it.

Most organizations discover this by trying to run an agent on their existing serverless platform and hitting a timeout limit, then quietly moving it to somebody’s VM. That VM is now production infrastructure with no owner.

What an agent runtime actually needs

Working through the requirements honestly gives a short and demanding list:

• A filesystem that survives the length of a session and is destroyed after it.
• Execution isolation strong enough that generated code cannot reach the host or other tenants. The threat model here is not a malicious insider; it is a model that read a web page containing instructions.
• Egress control. An agent with unrestricted outbound network access can exfiltrate anything it can read, and it does not have to be compromised to do it, only confused.
• Credential injection at runtime, scoped to the session, expiring with it. See secrets.
• Interruptibility. A human needs to be able to stop it, and stopping it has to actually stop it rather than orphan the work.
• A record of what it did, emitted as it goes. See traces.

Notice how many of these are boundaries rather than capabilities. The runtime’s job is mostly to be a box with well-understood walls.

Blast radius is a runtime property

The most consequential runtime decision is what the executing code can reach. It is worth writing down explicitly for each workload, because the default answer is almost always “more than you think”: the network the container sits on, the cloud role attached to it, the databases reachable from that network, and any credential in its environment.

An agent that can write to production because it happens to run on a host with a production role is a design decision nobody made.

Internal alternatives to the platforms people actually like

There is a reason developers reach for Railway, Render, Heroku, or a cloud’s app service: those products answer “where does it run” with one command and no meeting. If your internal answer is a Kubernetes namespace request form, people will keep reaching.

The realistic goal is not to build a better cloud. It is to present the same small interface over whatever you already run: push, get a URL, get logs, get a database, get a secret. Underneath it can be Kubernetes, ECS, VMs, or all three. The interface is the product; the substrate is an implementation detail, and treating it that way is what makes portability possible later.

Multi-tenancy is not optional

Even inside one company, AI workloads are multi-tenant: different teams, different data classifications, different model access. A runtime that assumes everything on it is equally trusted will eventually put a finance agent and a public-facing chatbot on the same host with the same egress.

Decide early which boundary carries the isolation: namespace, node, account, or cluster. Changing that decision later is a migration.