Backend architecture: the three planes

Why three roles

A naïve build would put everything behind one server: route the terminal bytes, check the API key, and call the model, all in one process. cmdop splits these into three roles because they have genuinely different shapes — different lifetimes, different transports, different failure modes — and pretending otherwise makes all three worse.

• A live execution stream is long-lived and stateful. It must not drop when a config reloads or a client disconnects.
• An account and metadata store is transactional and consistent. It is the source of truth, but it is never on the hot path of a keystroke.
• A model gateway is stateless request/response. It is cheap to scale and redeploy, and it should never be coupled to the execution stream.

Keeping them apart is the point. Below is each role, what it does, and — just as important — what it does not do.

The relay — live execution I/O

The relay is the I/O path. It is the single-homed surface that the execution-state object is reached through.

• The agent dials out. Every machine’s daemon opens an outbound connection to the relay over TLS. There is no inbound port to forward, no listener exposed on your machine. The relay never dials in.
• It fans out. A session is reached by ID, not by IP. The relay routes the live stream to whoever is attached — a terminal, a desktop client, an SDK caller, an AI operator — and broadcasts output to all of them at once. Many operators, one live execution.
• The session survives disconnect. When a client drops, the execution does not die with it. The runtime stays alive through a short grace window and the client reattaches — from the same device or a different one — and resynchronizes against the live state. See Transport-independent sessions and Sessions for the lifecycle.

The relay is what makes execution ownerless: no single connected client is the privileged occupant whose departure ends the session. It carries bytes between authenticated parties; it is not where your account lives and not where your model calls go.

The control plane — auth, keys, and metadata

The control plane is the system of record. It holds the durable facts that outlive any one session:

• Accounts and authentication — who you are, your login, your CLI tokens.
• API keys — issuing, validating, and revoking the keys your agents present.
• Fleets and membership — the boundaries you draw around sets of machines, and who belongs to which.
• Session metadata — the catalogue of sessions, their fleets, and their audit trail. (The metadata, not the live byte stream.)
• Schedules and the audit log — the record of what ran and what is queued to run.

The load-bearing fact: the control plane is not in the live I/O path. A keystroke into a terminal session does not transit the control plane. It does not arbitrate or route the execution stream — the relay does that. The control plane is consulted at the edges (authenticate this connection, validate this key, record that this session exists), and then it gets out of the way. If it were slow or briefly unavailable, live sessions already in flight keep flowing.

This separation is also a security boundary: the place that holds your credentials and account records is a different process from the place that carries your bytes.

The model router — LLM and web-search access

The third role is the model router — a standalone gateway, served at router.cmdop.com, through which cmdop agents reach language models and web search.

• It exposes an OpenAI-compatible surface for chat, structured output, image generation, and web search.
• It authenticates with your platform API key (sent as an X-API-Key header), validated against the control plane’s account store.
• On the default path it brokers requests to OpenRouter and OpenAI for models, and Serper for web search, using platform-supplied provider keys. You can also bring your own key via a custom provider.

The router is plumbing, not part of the execution-state mechanism — it brokers model access, it does not carry the live session. The full credential model, the default vs bring-your-own-key paths, and the honest maturity picture (metering and billing are planned, not live) are in How model access works. For the user-facing chat experience, see AI chat.

The transport split — designed, not accidental

The clearest sign that these three roles were kept apart on purpose is that they ride different transports, chosen for their workloads:

• The relay carries a long-lived gRPC stream where connection stability is sacred — a dropped stream interrupts a live session. Its edge routing is pinned to a static configuration so that unrelated infrastructure events never trigger a reload that would drop active streams.
• The model router is stateless HTTP request/response — cheap to route dynamically and to redeploy. Its edge routing is label-driven, the convenient default for a stateless service, precisely because a brief reconnect on a stateless call costs nothing.

Two transports, two routing strategies, matched to two workloads. The long-lived stream gets the stability-first treatment; the stateless gateway gets the flexible default. This is a deliberate engineering decision — not an accident of how the pieces were wired together.

Putting it together

The relay is what the architecture is about: the single-homed, ownerless live execution that operators attach to. The control plane and the model router are the supporting roles that keep it accountable and capable without ever sitting on the keystroke path.

Related

• Architecture overview — the category and the spine.
• Execution-state object — the single-homed runtime the relay reaches.
• Sessions — the resumable execution primitive.
• How model access works — the model router in detail.
• Self-hosted deployment — run the relay yourself.

TAGS: backend-architecture, relay, control-plane, model-router, single-homed-execution DEPENDS_ON: [architecture/execution-state-object, concepts/sessions, concepts/model-access]