# Agent Substrate environment: filesystem and process, not a WorkerPool > Agent Substrate environment is the higher-level API agents actually want: remote files and process lifecycle on top of sandboxed actors, not CRDs. - Canonical URL: https://webofmike.com/agent-substrate-environment/ - Author: Mike Moore (https://webofmike.com/about/) - Published: 2026-11-09 - Last modified: 2026-11-08 - Tags: Kubernetes, AI Agents, Platform Engineering - Cite as: Mike Moore, "Agent Substrate Environment", Web of Mike (webofmike.com), 2026-11-09. https://webofmike.com/agent-substrate-environment/ An Agent Substrate environment is the layer above the runtime that an agent loop actually wants: run this command, read this file, write that file, tell me when it exits. The runtime supplies the sandbox and suspend/resume. The environment is the hands, the model is the brain, and Substrate is the skeleton. Do not hand the brain a femur and call it a platform. I can suspend an actor between tool calls and the human does not have to know a WorkerPool exists. That is the demo that sells the project, and it is not the API I want application developers to write against. Frontier-style agent loops already think in files and processes. If every harness has to learn `ActorTemplate`, snapshot config, and ateom, onboarding stays a platform project. ## Why isn't the WorkerPool API enough for agents? Kubernetes CRDs are a good control plane for platform teams. They are a bad agent tool. Two CRDs in `ate.dev`: `WorkerPool` and `ActorTemplate`. The pool is the hardware. The template is the workload. On a running cluster, `ateapi` holds live assignment, `atenet` routes, `atelet` moves snapshots, `ateom` supervises the sandbox. I mapped that in [Agent Substrate dataplane](/agent-substrate-dataplane/). None of those names belong in a system prompt. kagent already had to paper over this. A `SandboxAgent` is not on the regular A2A endpoint. `POST /api/a2a/kagent/` returns not found even when the agent is Ready. Sandbox agents live on a separate mount, and `message/send` requires a `contextId` or you get `message contextId (session id) is required for substrate sandbox agents`. That session id is the environment, leaking through because the harness still has to know it is talking to a snapshot-backed actor. The same leak shows up in install. `platform: substrate` is required or the API rejects the sandbox spec. The field defaults to `agent-sandbox`. Those are different runtimes that share a kind. [Agent sandbox vs Agent Substrate](/agent-sandbox-vs-agent-substrate/). An environment API is how you hide that fork from the loop without pretending the fork is gone. [What is Agent Substrate](/what-is-agent-substrate/) is the runtime. This post is the interface I would put in front of it so harness authors can ignore the rest. ## What belongs in the Agent Substrate environment API? Keep it small. If I cannot keep it small, I have started to rebuild SSH. - `exec`, with cwd, env, timeout, and a streamed exit - `read` / `write` / `list` on a workspace that survives suspend - `mkdir`, `rm`, nothing cute - session lifecycle: create, suspend, resume, destroy - identity of *this* session on every call, so the gateway can authorize That is it. The workspace *is* the snapshot filesystem. If you add a second storage story "for files," you will desync from checkpoint. A file that "survives suspend" survived because [suspend and resume agents](/suspend-and-resume-agents/) wrote it into object storage. [Agent checkpoint to object storage](/agent-checkpoint-to-object-storage/) is the contract. The environment should not grow a parallel disk. Identity on every call is not optional. The gateway needs to know which session spoke. The worker's ServiceAccount is the bay. [Proof of agent identity](/proof-of-agent-identity/) is how you revoke the agent without revoking the human, and how you avoid stuffing a long-lived key into the snapshot the environment just wrote. This layer is also where [agent action verification](/agent-action-verification/) should sit: exec is a tool call. Check it before it runs. A boolean on the path, not a second large-model prompt that says it is 90 percent sure while the actor holds a slot. ## How should actor lifecycle look without exposing ateom? Create environment, then golden snapshot. First exec may be slow. Say so. In my kagent lab the first reconcile took about a minute per agent. Later restores are better and still not a warm in-memory process. After a model switch, sessions on that laptop cluster went to 1.5 to 2 seconds end to end, restore included. That is a number from that cluster. Tell the harness the first call is a bake. Do not pretend it is `cat`. Exec: restore if parked, run, optionally park again. The harness should not have to call `kubectl scale`. It should not have to know `ateom` talks to gVisor `runsc`. One actor per sandbox, always. Density comes from parking, not from packing two shells into one environment. Destroy: delete snapshot. Or keep it as an artifact. Forensics is a real use. I already wanted freeze-and-keep in [stopping an agent without losing evidence](/stopping-an-agent-without-losing-evidence/). An environment destroy that always wipes the tree makes that impossible. An environment destroy that never wipes the tree is how the bucket fills. Make it a parameter, default delete, keep when the caller is an incident. I would not let the environment API set `sandboxClass` per exec. That is a platform policy on the pool, not a prompt-time flag. Mixing gVisor and a microVM in one pool is a production bug. [Running Agent Substrate in production](/running-agent-substrate-in-production/). Watch the workers list, not the actors list, when you debug an environment that "will not resume." [kubectl is lying to you](/kubectl-is-lying-to-you-watch-agent-substrate/) with pods. The actors list grows without bound. The environment API should talk in session ids. The operator still debugs in worker slots. ## Should you suspend on every tool call? Only if restore is fast enough that the user does not feel it, and the tool call is not holding a socket you still need. **Suspend on every tool call.** Maximum density. Restore has to be fast enough that a `cat` does not feel like a cold start. I would measure before I made it default. Golden snapshots were about a minute. Later restores are better and still not free. This is the policy that makes 18 registered agents on two workers possible, because most of those agents are on the shelf. [How many agents per Kubernetes pod](/how-many-agents-per-kubernetes-pod/). **Long-lived environment, suspend on idle only.** Easier on the network. [Suspend-safe networking](/suspend-safe-networking/) matters less if you do not park mid-command. Suspend cuts TCP in the versions I have run. If the agent is waiting on a slow tool, parking is a failure, not a density win. You hold a slot while the human thinks, which is the cost Substrate exists to avoid. Idle timeout is the compromise. **One environment per user versus per task.** Per user is how you get a home directory. Per task is how you get isolation. I would default per task and opt into a persistent workspace when the product is a coding agent. A persistent per-user tree is a longer-lived snapshot, a larger blast radius, and a stronger argument for minting credentials at resume rather than leaving them in the tree. A worked example from the laptop lab, not a new benchmark: two workers, nine agents, then 18. Simultaneous execs equal two. Everyone else is a snapshot the environment can restore. Worker-0 served 50+ sessions across nine agents in an afternoon. The environment API should make that look like "run this command in session S," not "find a free bay in kagent-default." Keep it modular. If the only way to get a filesystem API is to take the entire control plane, nobody will reuse it. Harness authors should depend on exec/read/write and session lifecycle. Platform teams should keep WorkerPool, snapshot config, and sandbox class. The [field guide](/agent-substrate-field-guide/) starts at the runtime. This post is the interface I would put in front of it. Checklist I would use to call an environment API done: - First create bakes a golden snapshot. The client is told it will be slow. - Exec on a parked session restores, runs, and can park again without kubectl. - A file written before suspend is readable after restore on the other worker. - Mid-command suspend is refused, or documented as a dropped TCP session. - Destroy can keep the snapshot for forensics. - Every call carries session identity. No worker SA in the audit log as the agent. - sandboxClass is not a parameter on exec.