Architecture

PROMPT 1 of 2 — learn what the Architecture module teaches. You read and explain. You write one checklist file and nothing else.

I run my own agents with Claude Code. I am a senior technical person: do not explain what an agent or a server is. Explain what THIS page claims, and where it is right or wrong about a setup like mine.

FIRST — WRITE YOUR CHECKLIST TO A FILE

Long sessions compress their own history. Write this to ~/ARCHITECTURE-LEARN-CHECKLIST.md and work from the file:

  # Architecture — learn
  - [ ] 1. Fetch the page
  - [ ] 2. Name the five layers in dependency order
  - [ ] 3. Explain existence versus liveness
  - [ ] 4. Explain the one-backend literal and the flag that outlived it
  - [ ] 5. Read the sweeper line correctly
  - [ ] 6. Quiz me, then tell me to paste prompt 2

  ## Rules
  - Read only. The checklist is the one file I write.
  - Every claim I make about the page quotes the sentence it came from.

1 — FETCH

    curl -s https://docs.utopiamodels.ai/docs/structure/architecture.md

If that fails, say what came back and stop. Everything below is grounded in it, not in what fleets usually look like.

2 — THE FIVE LAYERS

Name them in the order they depend on each other — hosts, agents, runtime, services, secret store — and for each give one sentence on what breaks if it is missing. Then state the vocabulary rule: a COMPONENT runs or holds state; an ARCHITECTURAL PATTERN is a choice a reader can adopt or reject, and some patterns never run at all. Say why the page refuses the word "mechanisms" for the set.

3 — EXISTENCE IS NOT LIVENESS

This is the most important idea on the page. Explain:
  - why `getent passwd <name>` passes on an agent that cannot execute a single turn
  - why a dark agent looks BUSIER than a healthy one (transient units starting and dying in seconds)
  - what `{"exhausted": true, "resetsAt": null}` means, and why the null is the dangerous half
  - what the page says actually settles it: a real `claude -p` turn, run as that agent, on its box

4 — ONE BACKEND

Explain why `--setenv=TASK_BACKEND=claude` is a literal rather than a lookup, and the second job that variable still does (the "am I inside a dispatched run" signal every board-keyed gate exempts on). The lesson is about a flag outliving its purpose — say it in one line.

5 — READ THE SWEEPER LINE

The page quotes: `sweep: 39 open · 0 dispatchable · in-flight 3 (...)`. Explain why a large open count against zero dispatchable is a ROUTING failure and not a capacity failure, and why the log had been printing the cause while people looked for a broken mechanism.

Also cover the two traps the page names: the sweeper is a second `ExecStart` in a drop-in, so any probe looking for `sweeper.service` reports it missing; and a timer whose service unit is not-found never fails, so `systemctl --failed` reads clean.

6 — QUIZ AND FINISH

Ask me three questions with the `AskUserQuestion` tool, one at a time, each with four options and exactly one correct. Draw them from sections 3–5. After each answer, say which sentence on the page settles it.

If `AskUserQuestion` is unavailable in this session, ask the same questions as plain numbered text in one message instead.

Then say exactly this:

  "Architecture learned. Paste prompt 2 and I will map these five layers onto your own setup."

Then stop. Do not survey my machine — that is prompt 2.

PROMPT 2 of 2 — map my own setup onto the five layers from the Architecture page, find the places where I check EXISTENCE and believe I have checked LIVENESS, and propose fixes. Survey is read-only; any write waits for my explicit yes.

HOW LONG THIS TAKES: a multi-step audit (5 steps, about 5 questions, writes only what you approve) — expect roughly 10–25 minutes across several turns. Running headless with a timeout? Budget for that, or start with a shorter prompt first. Cut off partway? Paste this prompt again — it resumes from its checklist rather than starting over.

    curl -s https://docs.utopiamodels.ai/docs/structure/architecture.md

Read it first if prompt 1 did not run in this session.

ASK EVERY QUESTION WITH the `AskUserQuestion` tool, each with a RECOMMENDED option first and one line of why. A question asked in prose has no default, and I want to be able to accept every default and still end up with something that works.

FIRST — WRITE YOUR CHECKLIST TO A FILE

RESUMING? Before you write anything, check whether ~/ARCHITECTURE-INTEGRATE-CHECKLIST.md already exists. If it does, this prompt was interrupted: read it, tell me which steps are already ticked, and continue from the first unchecked step. Do not overwrite it, do not redo a ticked step, and do not re-ask a question whose answer it already records — the scope answer under ## Rules included, if it's there. If ## Rules has no Scope field, this checklist predates the scope question: say so plainly in your first output line and ask it now, before anything else. Only if the file does not exist, write it fresh as below.

Write ~/ARCHITECTURE-INTEGRATE-CHECKLIST.md and work from it:

  # Architecture — integrate
  - [ ] 1. Survey the five layers on this machine
  - [ ] 2. Find existence checks posing as liveness checks
  - [ ] 3. Find schedulers the obvious probe misses
  - [ ] 4. Write the layer map and the findings
  - [ ] 5. Propose changes; write only what I approve

  ## Rules
  - Steps 1–3 are read only. I start, stop, enable and edit nothing.
  - I never print a secret value. Whether I read a credential file at all is my scope answer below; either way I say that one exists and where.
  - Every finding carries the command I ran and its raw output.
  - An unreadable thing is reported as UNREADABLE, never as absent.

SCOPE — ASK ME BEFORE YOU SURVEY ANYTHING OF MINE

Before step 1, ask me this with the `AskUserQuestion` tool, with both options exactly as written:

  "Should I treat any path as off-limits, or do you want me to have full access to survey and propose changes anywhere in this environment?"

  A — Exclude .env*, credentials and CI/deploy config (.github/workflows/, deploy keys) from anything I read or propose touching.   [RECOMMENDED for most readers]
  B — No exclusions. Full access to survey and propose anywhere, including secrets and CI files.

This is my preference, not a policy: some people want their agent to reach everything, some do not, and neither answer is the correct one. Never assume it and never pick for me. My answer governs every read and every proposal for the rest of this prompt. Write my answer under ## Rules in the checklist as "Scope: A" or "Scope: B", so it survives compaction. It governs the SURVEY and PROPOSE steps only — nothing gets written until step 4 (the map, where I choose) and step 5 (only the change I approve).

Before you ask, decide whether an answer can reach me at all. It cannot in a one-shot non-interactive run — `claude -p`, `codex exec`, `gemini -p`, a scripted or CI invocation — and you can tell without guessing: the `AskUserQuestion` tool is missing from your tool list there, while an interactive session has it. It also cannot if the question comes back immediately with no real answer. In either case do not ask, do not wait and do not re-ask: a question asked as plain text in a one-shot run ends the run with nothing done. This is the one case where you pick for me: treat A as my answer, record and hold it exactly as above, and make the first line of your output exactly: "No interactive scope answer available — defaulted to A (excluded .env*, credentials and CI/deploy config). To choose B, re-run this prompt in an interactive session and answer the scope question."

Only if the `AskUserQuestion` tool is in your tool list but fails when you call it, ask the same question as plain numbered text in one message instead.

1 — SURVEY, ONE LAYER AT A TIME

  HOSTS      every machine involved, how each is reached, and whether that address resolves
             from THIS box. Check ~/.ssh/config — the page's most common documentation error
             is a bare host alias that only resolves on the machine it was written on.
  AGENTS     every identity an agent runs as: unix users, API accounts, GitHub identities.
             Where does `claude` resolve for each (`command -v claude` as that user)?
  RUNTIME    what starts an agent run — a hand-typed command, cron, systemd, a CI job, a queue.
  SERVICES   what stays up between runs. Read ALL layers, and say which do not apply:
               systemctl list-units --type=service --state=running --no-pager
               systemctl --user list-timers --all --no-pager
               crontab -l
               docker ps --format '{{.Names}}\t{{.Status}}' 2>/dev/null
  SECRETS    where secrets live. Count the stores. The page's answer is exactly one, with a
             separate registry of what each credential MEANS.

2 — EXISTENCE VERSUS LIVENESS

For every health check, status script or dashboard I have, classify what it proves:
  EXISTS     the account / file / unit / credential is present
  LIVE       the thing actually did its job recently, proved by an artifact of that job

The page's case: a valid account, credential file, clean systemd state and working auth — and still unable to run a single turn, because entitlement lives at the provider. Find my equivalent. For each agent identity, what would a REAL turn be, and when did one last succeed?

3 — THE PROBES THAT LIE

  - a timer whose service unit is not-found: list every timer and confirm its unit exists
  - a component riding as a second ExecStart in a drop-in, so a probe by unit name misses it
  - cron entries that no `systemctl` listing shows

4 — WRITE THE MAP

Ask with the `AskUserQuestion` tool where it goes. RECOMMENDED: a new folder beside where I keep notes, e.g. <notes>/architecture/.

Write LAYER-MAP.md there: the five layers as headings, each with what I have, the command that shows it, and whether it is checked for existence or liveness. Then a FINDINGS section ranked by how likely each item is to be silently dead right now.

5 — PROPOSE, THEN WRITE ONLY WHAT I APPROVE

For the top three findings, propose one concrete change each — typically a liveness probe that reads the artifact of real work rather than the presence of a thing. Show me the exact file or command you would add.

Ask with the `AskUserQuestion` tool: apply all three / apply one (RECOMMENDED: the finding most likely dead right now) / none, just keep the map.

Write only what I choose. After writing, run the new probe and paste its real output. If it reads green on something you just argued might be dead, say so — that is a probe checking existence again.

When the writes are done, tell me in one line, filled in with the real paths:

  "To undo, delete `~/ARCHITECTURE-INTEGRATE-CHECKLIST.md`, `<notes>/architecture/LAYER-MAP.md` and <each probe you added, by path> — nothing else on your system was touched."

If I approved an edit to a file you did not create, name that file in the same line with the command that reverts it (`git checkout -- <path>` if uncommitted) instead of claiming nothing else was touched.

Finish with: the map's path, the findings, what you changed, and the one layer where my setup is weakest.

The fleet has five layers, listed in the order they depend on each other. The hosts exist, the agents run on them, the runtime starts an agent, the services stay up between runs, and the secret store is what everything authenticates against.

The vocabulary

The fleet calls its structural choices architectural patterns. There are 14 of them, and the fleet chose that term on purpose rather than drifting into it. Everything below is one of two things.

• A component is something that runs or holds state, such as the box, the sweeper or the secret store. The fleet keeps a registry of components, one document each, and this page describes that registry.
• An architectural pattern is one structural choice, written so that a reader on a different base can adopt it or reject it. Several patterns do not run at all. Decision discipline and liveness-vs-existence are policies, not processes.

That second point is why this site does not call the set mechanisms, components, Fleet Architecture or Fleet Infrastructure. Each of those names fits the half that runs and is wrong for the policy half. A name for the set has to pass the adopt-or-reject test, and only pattern passes it. Elsewhere on these pages "mechanism" keeps its ordinary English meaning. The rule applies only to the name of the set.

The fleet's canonical list of the 14 patterns is in its knowledge base, at wi/work/reference/architectural-patterns.md. The pages on this site explain them.

Nothing on this page is a health reading. Whether a host is online, whether an agent can run today, and how full the disk is all change often, and a written page that stated them would be wrong within days. Where this page needs one of these values, it gives the exact command and the box that answers it instead of a number. The fleet's cockpit at utopiamodels.ai shows the current state.

Hosts

novaserver, the fleet's box

One machine does almost everything. It is an HP EliteDesk 805 G6 running Ubuntu 24.04. You can reach it on its tailnet address (written 100.x.x.x on this site) and on the LAN.

The bare alias does not resolve. If you run ssh with just nova as the host, with no user and no IP, it exits with Name or service not known. The Architect box has no ~/.ssh/config to expand the alias. The working address is always the user at the tailnet IP, which is what the dispatch script itself uses (NOVA="${DISPATCH_NOVA_IP:-100.x.x.x}"). This is the most common error in the fleet's internal docs. Thirteen files still write the bare alias, and one of them puts it inside a runbook block meant for copy and paste, so a reader who follows it fails on the first command.

ETH, the Architect's box

ETH is a WSL2 Ubuntu 24.04 guest on the founder's Lenovo, at tailnet 100.x.x.x, with unix user kxdev. .wslconfig caps WSL at 12 GB of memory, 4 GB of swap and 6 processors. Inside the guest that shows as about 11 GiB of RAM. This box plans, files issues and dispatches. It hosts no service the fleet depends on.

utopiamodels, a host that is not a machine

This tailnet node is an application that joined the tailnet with tsnet. It is an LLM gateway, not a computer. It appears in the node list next to real hardware, serves on port 80, and has no shell. Know this before you try to SSH to it.

Agents

The site generates the list of agents from the registry instead of writing it by hand. The table below is read from the fleet registry on every request. It used to be typed here, and it was wrong in the way that matters most. It called the DMV-portfolio master dmv, a retired alias for the agent the registry records as dust. That alias has never been a unix account on any box. id -u dust returns 1001, and id -u dmv still returns no such user. Every sudo -u dmv command this site published failed for every reader who copied one, and no page can catch that error in itself.

Each agent has its own credentials, its own ~/.claude brain directory (each one is its own version-controlled repo), and its own GitHub identity. The agents are peers. No supervisor process exists.

The table deliberately leaves out the binary. /usr/bin/claude is where it resolves on novaserver and nowhere else. On the Architect's WSL box it resolves to a different path. So "every agent runs the same binary at the same path" is a claim about one box presented as a claim about the fleet. To settle it for a given account, run sudo -u <account> -H command -v claude on the box where that account lives.

Existence is not liveness

The registry checks an agent with getent passwd <name>, and that check passes on an agent that cannot execute a single turn. This is the most important distinction on this page.

A master can have a valid unix account, a credential file, a clean systemd state and working gh auth, and still be unable to run. Entitlement belongs to the account at the provider, not to anything on the box. When an agent has no entitlement, it looks busier than a healthy one. Transient units keep starting, keep dying within seconds, and keep posting their start messages.

On 2026-08-16, a measurement found three of the four masters able to run. The fourth reported {"exhausted": true, "resetsAt": null} in its own state file. The null in resetsAt is the important part. It means the provider gave no recovery time, so no timer brings that agent back. The dispatch scheduler correctly left it out and said so in its log line every five minutes.

The current answer for each agent comes from that agent's own measured entitlement, meaning a recorded refusal and never getent. A real claude -p turn, run as that agent on its box, settles it for certain.

There is one backend, and the code that chose between backends was deleted

dispatch.sh passes a literal backend name into every run, --setenv=TASK_BACKEND=claude. It does no lookup, reads no registry and resolves nothing per master. The function that once chose between providers was deleted along with the second provider.

This page said the opposite for a while, and the wrong version sounded better. A fleet that resolves each agent's provider at runtime sounds more capable than one that hardcodes a single name. The plain version is stated here because it is what a reader would find in the script.

The variable still exists for a different job. It is the fleet's "am I inside a dispatched run" signal, and every board-keyed gate uses it to decide when to exempt a run. A flag whose first purpose was deleted can be worth keeping for a second job it took on by accident, as long as someone writes down what that job is.

The dispatch runtime

The whole agent runtime is two scripts and systemd. There is no scheduler daemon, no queue and no watchdog. This section is the overview. The guards, their exit codes and the race that only a file lock can close are on The dispatch runtime.

One dispatch does these steps, in order.

1. It launches under the fleet's one backend, which is a literal and not a lookup (see above).
2. agent runs locally. Dispatch reaches the novaserver masters with one ssh nova@<tailnet-ip> plus sudo -u <master>.
3. Pre-flight refuses a master that is out of quota. A dispatch to an agent that cannot run wastes the run and charges the issue an attempt for it. The check fails open. An unreadable state file, a missing file, or a reset time that has already passed all count as "can run". A false block stops the fleet, and that is worse than a wasted run.
4. Pre-flight refuses a second run for an issue that is already running. Each issue gets one run. The guard is in the entry point that both the scheduler and a hand-typed dispatch use, because a guard in only one of them cannot see the other path.
5. Pre-flight syncs the master's clones. For every repo in the run's repo list it fetches, stashes anything uncommitted, checks out main and hard-resets to origin/main.
6. It launches a detached transient unit with systemd-run --user --unit=task-<issue>-<n> --collect, so the run keeps going if the SSH connection drops or the laptop goes to sleep.
7. task-runner.sh updates itself once and checks again that the issue is still work. It posts ▶ started before anything that can fail, then runs claude -p under a turn cap. On exit, its trap posts ✔ / ✘ / ⚠ / ⛔ and the pull-request link.

Someone may tune the cap in that last step at any time, so a written number would go stale. The page reads it from the source instead:

MAXT="${TASK_MAX_TURNS:-200}"

The last-moment disposal re-check

An issue can stop being work while its run is starting, and every gate that asks about it runs too early. One real case shows the timing. The pre-flight read the issue at 14:36:01, and it was open and correct. Someone labelled the issue a throwaway at 14:36:03 and closed it at 14:36:05. systemd started the unit at 14:36:12, and the runner posted ▶ at 14:36:15. A master then spent a full session on leftover board items.

The fix asks the same two questions from inside the run, right before ▶, which is the real point of no return. Only two signals count. Either the issue is CLOSED, or it has the throwaway-fixture label. Any other signal is a judgement that a human may be overriding at the command line on purpose.

The sweeper

The sweeper moves a chain of work forward. It is not a service and not a cron job, so every probe that looks for a sweeper.service wrongly reports it missing. It runs as the second ExecStart line of a systemd drop-in on one master's reconcile-board.service, which reconcile-board.timer fires every five minutes:

/home/nova/.config/systemd/user/reconcile-board.service.d/10-sweep-dispatch.conf
  ExecStart=           (clear)
  -/usr/bin/bash %h/.claude/ops/reconcile-board-status.sh --apply
  -/usr/bin/bash %h/dev/utopia-models/knowledge/_infra/scripts/sweep-dispatch.sh --apply
  -/usr/bin/bash %h/.claude/ops/mute-agent-notifications.sh --apply

The sweeper's log line is the best one-line health summary the fleet produces. Here is one real line, read from ~nova/.claude/logs/sweep-dispatch.log and written at 2026-09-04T08:48:34Z. It is a sample of the format, not a current reading.

sweep: 39 open · 0 dispatchable · in-flight 3 (win=2 nova=0 cloud=1 dust=0 )

When the sweeper refuses a master, it adds a trailing segment that names the master and the reason, such as · rate-limited: (exhausted nova). The sample has no such segment because every master was clear on that pass.

This line teaches you to read your own logs carefully. A large open count against zero dispatchable looks like a capacity failure, but it is a routing failure. Issues with no lane, or on a lane that no sweeper reaches, can never be dispatched, however idle the fleet is. The sweeper printed the cause every five minutes while people investigated a mechanism they assumed was broken.

Services on the box

The first two groups below run on novaserver and stay up between agent runs. The third group is state the box depends on but does not host. The dispatch runtime has no resident process, but the box has plenty.

System units. tailscaled. cloudflared, the tunnel that publishes selected local services to the public web. ttyd, plus one read-only browser viewport per master on ports 7810 to 7813. ollama on 11434, serving small local models. occ-redirect on 3939, a small Python responder that sends a 301 from a legacy hostname to the apex. grafana-server on 3000. prometheus on 9090, with node-exporter on 9100. Two self-hosted GitHub Actions runners, one for a full lane and one for a light lane.

Docker. Infisical runs here (backend, database and redis), and it is the fleet's only secret store. A full SigNoz observability stack runs here too, with ClickHouse, a Postgres metastore, keeper and ingester.

Managed Postgres, off the box. Two hosted databases hold state that nothing here can rebuild, so they belong in this list even though no process for them runs on novaserver.

The first is Neon, the utopia-auth project, Postgres 17 in aws-us-east-1. It holds the product's auth and user rows. Every night a pg_dump -Fc | gzip job copies it through the direct unpooled endpoint to cold object storage, with 35-day retention. The nightly copy exists because Neon's free point-in-time recovery only reaches back about six hours. The same Neon organisation holds three other projects that are not fleet infrastructure, and it is easy to confuse them with this one.

The second is Supabase, the peptide-shop project in us-east-1. It holds the database, auth, and the order and payment records behind dmvresearch.shop. Those rows cannot be replaced. Two independent jobs dump them nightly to the same bucket, one with pg_dump and one with a row-level export through the REST API.

Scheduling runs on one master. One master's user manager carries almost every scheduled timer in the fleet. A count on 2026-09-04 with systemctl --user list-timers --all under each master account found 37 timers on nova, 4 on win and 3 each on cloud and dust. The important ones publish the command center's feeds, re-render it and check it, sync creative assets, watch token health, refresh the tailnet snapshot, guard the disk hourly, and repair the default route every sixty seconds.

Cron is a second scheduling layer that runs in parallel, and it is easy to miss. Masters have crontab entries for monitors, self-reports and usage feeds that have no systemd timer at all. An audit that reads only systemctl --user list-timers undercounts how often every master really runs jobs.

A dead timer never fails, so systemctl --failed reports clean. The box has had an enabled timer whose service unit was not-found, and dormant timers that had not fired in a week, while every failure query came back empty. An absence of failures does not mean the thing works.

Secrets

Infisical is the only secret store. It is self-hosted in Docker on novaserver. There is no local vault, no .env checked in anywhere and no second store.

Projects are split by domain into fleet infrastructure, the product, per-portfolio research and engineering. A guard rejects any value passed as a command-line argument, because a secret in argv lands in shell history and in the process table.

A second file, the credential registry, records what each credential means. It lists which account the credential belongs to, what scopes it has, what it can do and what it explicitly cannot. A standing rule says to read that registry before you call a credential missing. The rule exists because the fleet's usual failure is not a missing credential. It is a grep for the wrong key name.

Two rules have no exceptions.

• Never print a secret, or the fact that one exists, to anything that displays output. Capture it into a variable and check its length.
• Never write a secret into a tracked file. This is the fleet's one hard stop. It is the only rule you cannot fix forward, treat as advisory or override. A secret scanner runs on every pull request, and it is the only check whose result the fleet treats as binding.

This page publishes the mechanics of all of this on purpose, and that is in scope. It never publishes a value.