Architecture

Five layers, in the order they depend on each other: the hosts that exist, the agents that run on them, the runtime that starts an agent, the services that stay up between runs, and the secret store everything authenticates against.

The vocabulary

The fleet's structural choices are architectural patterns — there are 14 of them, and the term is the fleet's own, decided rather than drifted into. Everything below is one of two things:

• a component — something that runs or holds state. The box, the sweeper, the secret store. The fleet keeps a registry of them, one document each, and that registry is what this page describes.
• an architectural pattern — one structural choice, stated so a reader on different substrate can adopt it or reject it. Several of them do not run at all: decision discipline and liveness-vs-existence are policy, not processes.

That second half is why this site does not call them mechanisms, components, Fleet Architecture or Fleet Infrastructure when naming them as a set — each of those fits the runtime half and is wrong for the policy half. The adopt-or-reject test is the one a name here has to pass, and only pattern passes it. Elsewhere on these pages "mechanism" keeps its ordinary English meaning; the rule binds the set.

The fleet's canonical inventory of the 14 lives in its knowledge base, at world-infrastructure/reference/architectural-patterns.md. The pages here are the exposition of them, and /feedback walks that same list — one pattern at a time, each explained inside the question before it asks you whether you would adopt it.

Nothing on this page is a health reading. Whether a host is online, whether an agent can run today, and how full the disk is are all volatile and belong on /live. Read that page for them — and note what it does with the ones a public page cannot reach: it prints the exact command and the box that answers it, rather than a number it did not measure.

Hosts

novaserver — the fleet's box

One machine does essentially everything. It is an HP EliteDesk 805 G6 running Ubuntu 24.04, reachable on the tailnet at 100.71.25.43 and on the LAN at 10.0.0.231.

The bare alias does not resolve. Writing ssh with just nova as the whole host — no user, no IP — exits with Name or service not known, because there is no ~/.ssh/config on the Architect box to expand it. The working address is always the user at the tailnet IP, which is exactly what the dispatch script itself uses (NOVA="${DISPATCH_NOVA_IP:-100.71.25.43}"). This is the single most common error in the fleet's internal docs — thirteen files still write the bare alias, one of them inside a copy-pasteable runbook block, so a reader following them fails on their first command.

ETH — the Architect's box

A WSL2 Ubuntu 24.04 guest on the founder's Lenovo, tailnet 100.103.62.104, unix user kxdev. WSL is capped by .wslconfig at 12 GB of memory, 4 GB of swap and 6 processors; inside the guest that reads as roughly 11 GiB of RAM. This box plans, files issues, and dispatches. It does not host any service the fleet depends on.

utopiamodels — a host that is not a machine

100.92.117.107 is an application that joined the tailnet using tsnet — an LLM gateway, not a computer. It appears in the node list beside real hardware, serves on port 80, and has no shell. Worth knowing before you try to SSH to it.

Agents

Who the agents are is generated, not written. The table below is read out of the fleet registry on every request. It used to be typed here, and it was wrong in the way that matters most: it called the DMV-portfolio master dmv, a retired alias for the agent the registry records as dust. That name has never been a unix account on any box — id -u dust returns 1001, and id -u dmv still returns no such user — so every sudo -u dmv command this site published failed for every reader who copied one, and no page can catch that about itself.

Each agent has its own credentials, its own ~/.claude brain directory (each one a version-controlled repo of its own), and its own GitHub identity. They are peers. There is no supervisor process.

The binary is deliberately not in that table. /usr/bin/claude is where it resolves on novaserver and nowhere else — on the Architect's WSL box it resolves elsewhere — so "every agent runs the same binary at the same path" is a claim about one box wearing the clothes of a claim about the fleet. The command that settles it for a given account is sudo -u <account> -H command -v claude, run on the box that account lives on.

Existence is not liveness

The registry's own check for an agent is getent passwd <name>, and that check passes on an agent that cannot execute a single turn. This is the most important distinction on this page.

A master can hold a valid unix account, a present credential file, a clean systemd state, working gh auth, and still be unable to run — because entitlement lives with the account at the provider, not on the box. When that happens the agent looks busier than a healthy one: transient units keep starting, keep dying in seconds, and keep announcing themselves.

Measured 2026-08-16: three of the four masters could actually run, and the fourth reported {"exhausted": true, "resetsAt": null} in its own state file. A resetsAt of null is the important half — it means the provider named no recovery time, so no timer brings that agent back. The dispatch scheduler correctly excluded it and said so in its own log line every five minutes.

The current per-agent answer is on /live. It is derived from each agent's own measured entitlement — a recorded refusal, never getent — and that page names the claude -p command, and the box, that settles it definitively.

There is one backend, and choosing between backends was deleted

dispatch.sh passes a literal backend name into every run — --setenv=TASK_BACKEND=claude. There is no lookup, no registry read and no per-master resolution: the function that once chose between providers was deleted along with the alternate provider it chose between.

This page said the opposite for a while, and the wrong version was the more flattering one — a fleet that resolves each agent's provider at runtime sounds more capable than one that hardcodes a single name. It is worth stating the plain version instead, because the plain version is what a reader would find in the script.

The variable itself survives for a different job. It is the fleet's "am I inside a dispatched run" signal, which is what every board-keyed gate exempts on. A flag whose original purpose was deleted can be worth keeping for the second job it accidentally does — provided somebody writes down which job it is now doing.

The dispatch runtime

Two scripts and systemd. That is the entire agent runtime — no scheduler daemon, no queue, no watchdog. The overview is below; the guards, their exit codes and the race that only a file lock can close are on The dispatch runtime.

What one dispatch actually does, in order:

1. Launch under the one backend the fleet has — a literal, not a lookup (see above).
2. agent runs locally; the novaserver masters are reached with one ssh nova@100.71.25.43 plus sudo -u <master>.
3. Pre-flight refuses a quota-dark master. A dispatch into an agent that is already dark burns the run and charges the issue an attempt for it. The check fails open — an unreadable state file, a missing file, or a reset time that has already passed all mean "not dark", because a false block stops the fleet and that is worse than a wasted run.
4. Pre-flight refuses a second run for an issue that is already running. One issue, one run. The guard lives in the entry point both the scheduler and a hand-typed dispatch share, because a guard in only one consumer cannot see the other path.
5. Pre-flight syncs the master's clones: fetch, stash anything dirty, checkout main, hard-reset to origin/main for every repo in the run's repo list.
6. Launch a detached transient unit — systemd-run --user --unit=task-<issue>-<n> --collect — so the run survives the SSH connection dropping and the laptop going to sleep.
7. task-runner.sh self-updates once, re-checks that the issue is still work, posts ▶ started before anything that can fail, runs claude -p under a turn cap, and on exit its trap posts ✔ / ✘ / ⚠ / ⛔ plus the pull-request link.

The cap in that last step is the kind of number that goes stale the moment someone tunes it, so it is read rather than written:

MAXT="${TASK_MAX_TURNS:-200}"

The last-moment disposal re-check

An issue can stop being work while its run is starting, and every gate that asks about it runs too early. Measured on one real case: the pre-flight read the issue at 14:36:01 and it was open and correct; the issue was labelled a throwaway at 14:36:03 and closed at 14:36:05; systemd started the unit at 14:36:12; ▶ was posted at 14:36:15. A master then spent a full session on board residue.

The fix asks the same two questions from inside the run, immediately before ▶ — the genuine point of no return. Only two signals count: the issue is CLOSED, or it carries the throwaway-fixture label. Anything else is a judgement a human may legitimately be overriding at the command line.

The sweeper

The thing that carries a chain of work forward is not a service and not a cron job, which means every probe that goes looking for a sweeper.service falsely reports it missing. It rides as the second ExecStart line of a systemd drop-in on one master's reconcile-board.service, fired by reconcile-board.timer every five minutes:

/home/nova/.config/systemd/user/reconcile-board.service.d/10-sweep-dispatch.conf
  ExecStart=           (clear)
  -/usr/bin/bash %h/.claude/ops/reconcile-board-status.sh --apply
  -/usr/bin/bash %h/dev/utopia-models/knowledge/_infra/scripts/sweep-dispatch.sh --apply
  -/usr/bin/bash %h/.claude/ops/mute-agent-notifications.sh --apply

Its own log line is the best one-line health summary the fleet produces. One real line, read out of ~nova/.claude/logs/sweep-dispatch.log and emitted 2026-09-04T08:48:34Z — quoted as a sample of the shape, not as a current reading:

sweep: 39 open · 0 dispatchable · in-flight 3 (win=2 nova=0 cloud=1 dust=0 )

A refused master adds a trailing segment naming who and why — · rate-limited: (exhausted nova) — which is absent above because on that pass every master was clear.

Read that carefully, because it is a lesson in reading your own instrumentation. A large open count against zero dispatchable looks like a capacity failure and is not one — it is a routing failure. Issues with no lane, or on a lane no sweeper reaches, are structurally undispatchable no matter how idle the fleet is. The sweeper had been printing the cause every five minutes while investigations looked for a broken mechanism.

Services on the box

The first two groups below run on novaserver and stay up between agent runs; the third is state the box depends on and does not host. The dispatch runtime is the part with no resident process; the box has plenty.

System units: tailscaled · cloudflared (the tunnel that publishes selected local services to the public web) · ttyd plus one read-only browser viewport per master on ports 7810–7813 · ollama on 11434, serving small local models · occ-redirect on 3939, a small Python responder that 301s a legacy hostname to the apex · grafana-server on 3000 · prometheus on 9090 with node-exporter on 9100 · two self-hosted GitHub Actions runners, a full lane and a light lane.

Docker: Infisical — backend, database and redis — which is the fleet's only secret store, and a full SigNoz observability stack (ClickHouse, a Postgres metastore, keeper, ingester).

Managed Postgres, off the box: two hosted databases hold state that nothing here can reconstruct, so they belong in this list even though no process for them runs on novaserver. Neon — the utopia-auth project, Postgres 17 in aws-us-east-1, holding the product's auth and user rows; a nightly pg_dump -Fc | gzip through the direct unpooled endpoint copies it to cold object storage at 35-day retention, because Neon's free point-in-time recovery reaches back only about six hours. The same Neon organisation holds three other projects that are not fleet infrastructure, and conflating them with this one is the easy mistake. Supabase — the peptide-shop project in us-east-1, which holds the database, auth, and the order and payment records behind dmvresearch.shop; those rows are the irreplaceable asset, and they are dumped nightly to the same bucket by two independent jobs, one a pg_dump and one a row-level export through the REST API.

Scheduling is single-homed. One master's user manager carries essentially every scheduled timer in the fleet (measured 2026-09-04 with systemctl --user list-timers --all under each master account: 37 timers on nova against 4 on win and 3 each on cloud and dust). The consequential ones publish the command center's feeds, re-render it and check it, sync creative assets, watch token health, refresh the tailnet snapshot, guard the disk hourly, and self-heal the default route every sixty seconds.

Cron is a second, parallel scheduling layer, and it is easy to miss. Masters carry crontab entries for monitors, self-reports and usage feeds that have no systemd timer at all. An audit that reads only systemctl --user list-timers under-reports every master's real cadence.

A dead timer never fails, so systemctl --failed reports clean. The box has held an enabled timer whose service unit was not-found, and dormant timers that had not fired in a week, while every failure query returned empty. Absence of failure is not presence of function.

Secrets

Infisical is the only secret store, self-hosted in Docker on novaserver. There is no local vault, no .env checked in anywhere, and no second store.

Projects are scoped by domain: fleet infrastructure, the product, per-portfolio research, and engineering. Writing a value as a command-line argument is rejected by a guard, because an argv secret lands in shell history and in the process table.

A second file — a credential registry — carries each credential's meaning: which account it belongs to, what scopes it has, what it can do and what it explicitly cannot. Reading that registry before declaring a credential missing is a standing rule, because the fleet's recurring failure mode is not a missing credential but a grep for the wrong key name.

Two absolute rules:

• Never print a secret, or the presence of one, to any display sink. Capture into a variable and check its length.
• Never write a secret into a tracked file. This is the fleet's single hard stop — the one thing that is not fix-forward, not advisory, and not overridable. A secret scanner runs on every pull request; it is the only check whose result is treated as binding.

Publishing the mechanics of all of this, as this page does, is in scope and deliberate. Publishing a value never is.