Backend architecture: the three planes

Why three roles

A naïve build would put everything behind one server: route the terminal bytes, check the API key, and call the model, all in one process. cmdop splits these into three roles because they have genuinely different shapes — different lifetimes, different transports, different failure modes — and pretending otherwise makes all three worse.

• A live execution stream is long-lived and stateful. It must not drop when a config reloads or a client disconnects.
• An account and metadata store is transactional and consistent. It is the source of truth, but it is never on the hot path of a keystroke.
• A model gateway is stateless request/response. It is cheap to scale and redeploy, and it is never coupled to the execution stream — a model call never sits on the keystroke path.

Keeping them apart is the point. Below is each role, what it does, and — just as important — what it does not do.

The relay — live execution I/O

The relay is the I/O path. It is the single-homed surface that the execution-state object is reached through.

• The agent dials out. Every machine’s daemon opens an outbound connection to the relay over TLS. There is no inbound port to forward, no listener exposed on your machine. The relay never dials in.
• It fans out. A session is reached by ID, not by IP. The relay routes the live stream to whoever is attached — a terminal, a desktop client, an SDK caller, an AI operator — and broadcasts output to all of them at once. Many operators, one live execution.
• The session survives disconnect. When a client drops, the execution does not die with it. The runtime stays alive through a short grace window and the client reattaches — from the same device or a different one — and resynchronizes against the live state. See Transport-independent sessions and Sessions for the lifecycle.

The relay is what makes execution ownerless: no single connected client is the privileged occupant whose departure ends the session. It carries bytes between authenticated parties; it is not where your account lives and not where your model calls go.

The control plane — auth, keys, and metadata

The control plane is the system of record. It holds the durable facts that outlive any one session:

• Accounts and authentication — who you are, your login, your CLI tokens.
• API keys — issuing, validating, and revoking the keys your agents present.
• Fleets and membership — the boundaries you draw around sets of machines, and who belongs to which.
• Session metadata — the catalogue of sessions, their fleets, and their audit trail. (The metadata, not the live byte stream.)
• Schedules and the audit log — the record of what ran and what is queued to run.

The load-bearing fact: the control plane is not in the live I/O path. A keystroke into a terminal session does not transit the control plane. It does not carry or route the execution stream — the relay does that. The control plane is consulted at the edges (authenticate this connection, validate this key, record that this session exists), and then it gets out of the way. If it were slow or briefly unavailable, live sessions already in flight keep flowing.

This separation is also a security boundary: the place that holds your credentials and account records is a different process from the place that carries your bytes.

The model gateway — LLM access for the agent

The third role is the model gateway — the OpenAI-compatible inference surface through which cmdop agents reach language models on the default path. Unlike a wholly separate service, it rides the relay the agent already connects to.

• It exposes an OpenAI-compatible surface for chat (plain, streaming, and structured), embeddings, and vision / OCR.
• It authenticates with the sign-in your agent already holds — the same bearer it uses to join the agent plane — validated against the control plane’s account store. There is no separate provider key to configure on this path.
• It brokers requests to the major model families using platform-supplied provider keys, picking a concrete model server-side (an alias like auto resolves there). You can also bring your own key via a custom provider, which sends chat straight to your endpoint instead.

The gateway is plumbing, not part of the execution-state mechanism — it brokers model access, it does not carry the live session. The full three-mode model (gateway · custom provider · local), the credential model, the gateway-only vision/OCR path, and the honest maturity picture (metering and billing are planned, not live) are in How model access works. For the user-facing chat experience, see AI chat.

The transport split — designed, not accidental

The clearest sign that these three roles were kept apart on purpose is that they ride different transports, chosen for their workloads:

• The relay carries a long-lived gRPC stream where connection stability is sacred — a dropped stream interrupts a live session. Its edge routing is pinned to a static configuration so that unrelated infrastructure events never trigger a reload that would drop active streams.
• The model gateway is stateless HTTP request/response — cheap to route and to redeploy, precisely because a brief reconnect on a stateless call costs nothing.

Two transport shapes, matched to two workloads. The long-lived stream gets the stability-first treatment; the stateless inference surface rides the convenient default. This is a deliberate engineering decision — not an accident of how the pieces were wired together.

Putting it together

The relay is what the architecture is about: the single-homed, ownerless live execution that operators attach to. The control plane and the model gateway are the supporting roles that keep it accountable and capable without ever sitting on the keystroke path.

Related

• Architecture overview — the category and the spine.
• Execution-state object — the single-homed runtime the relay reaches.
• Sessions — the resumable execution primitive.
• How model access works — the model gateway in detail.
• Self-hosted deployment — run the relay yourself.