Hosted hub
pnpm dev runs one backend on 127.0.0.1 for one local user — the loopback bind
is its only access control. The hosted hub (backend/hub/) lets many people
use one deployment from a browser, each with their own dashboard.
browser ──► hub (public): sign-in, sessions, reverse proxy for /api and /ws
│ per-user network, X-Horrible-Instance-Token
▼
one instance per user = the normal backend, HORRIBLE_PROFILE=hosted,
its own /data volume, cpu/mem/pid limits, no published port
Sign-in
Accounts are the game server's accounts (email + password, GitHub, Google). The
hub relays the same flows a node uses (backend/modules/games/server_auth.py)
under /hub/auth/*, but turns the minted JWT into a browser session: an
opaque hd_session cookie (HttpOnly, SameSite=Lax, Secure by default). The JWT
never reaches the browser; the hub hands it to the user's instance so games work
there without a second sign-in.
The frontend calls probeHub() (packages/core/src/hosted.ts) before any /api
request. Signed out it renders HubLogin — the shared SignInCard routed to the
hub via setSignInViaHub. Signed in it waits for the instance (waitForInstance)
before loading settings. Both run as boot steps ("Checking sign-in", "Starting your
workspace") on the already-mounted boot splash, so a waking instance shows a ticking
step with a hint rather than a blank window. Without a hub, /hub/me 404s (or returns Vite's HTML) and
nothing changes.
Instances
backend/hosted.py turns the backend into an instance:
HORRIBLE_INSTANCE_TOKEN— every HTTP request and WebSocket upgrade must carry it, or is refused before any route runs. Unset → no-op.HORRIBLE_PROFILE=hosted— drops the Clubhouse modules, exposesGET /api/hostand the hub-onlyPUT /api/hosted/games-session(the proxy refuses/api/hosted/from browsers).
Image: deploy/dashboard-node/Dockerfile. The docker spawner
(backend/hub/spawner/docker.py, uv sync --extra hub) creates one container,
volume and bridge network per user. HUB_SPAWNER=static points every user at one
running backend — development only, no isolation.
Idle instances
An instance runs only while its person is using it, and an open tab is not use:
an unattended tab polls /api/health every 10 s, and an open notebook pane every
2 s. So the hub ignores traffic and sockets, and counts input instead:
- The page reports keystrokes, clicks, scrolls and touches to
POST /hub/activity, at most once a minute (startActivityReporterinhosted.ts). Loading the page and signing in also count. - After
HUB_IDLE_MINUTESwith no input, the reaper asks the instanceGET /api/hosted/busy. That reports in-flight work: agent turns, executing kernels, training runs and sweeps, eval sweeps, docs crawls, research runs, and background tasks touched in the last hour. If anything is running the instance is kept, for at mostHUB_BUSY_MAX_HOURS(default 12). Otherwise it is stopped. - A stopped instance is parked: the tab's polls get 503
parkedand its socket is closed with 4408, and neither wakes it. The next input does, and the page shows a "Resuming" toast while it starts. - When the hub restarts, every user who has an instance is put on the reaper's watch list without counting as active. A machine nobody returns to is stopped one idle window later, and a parked one is not woken by its tab's polls.
Isolation strength: plain Docker is fine for invite-only use. For open signup
use a sandboxing runtime (HUB_RUNTIME=runsc) or microVMs (Fly Machines, planned).
What actually separates two users' instances is the instance token, not the
network. Each instance answers only its own token (checked on 2026-10-06: user A's
container calling user B's API gets 401). The per-user bridge networks were not
isolated from each other on Docker Desktop — A's container could open a TCP
connection to B's port. Linux Docker normally isolates bridges with its
DOCKER-ISOLATION rules; confirm it on your host before relying on it, and treat
anything else listening inside an instance as reachable by other instances.
Configuration
| Variable | Meaning |
|---|---|
GAMES_SERVER_URL | Account server (also passed to instances) |
HUB_DB_PATH | Hub SQLite (sessions, users, instance tokens) |
HUB_WEB_DIST | Built apps/web/dist served at /; absent → API only |
HUB_FRONTEND_URL | Frontend hosted elsewhere: page requests redirect there and HUB_WEB_DIST is not served (live: the Vercel URL) |
HUB_SPAWNER | docker (default) or static |
HUB_INSTANCE_IMAGE | Instance image tag |
HUB_DOCKER_REACH | network (hub is a container, HUB_SELF_CONTAINER) or published (loopback port) |
HUB_RUNTIME, HUB_MEM, HUB_CPUS, HUB_PIDS | Instance runtime and limits |
HUB_FLY_APP, HUB_FLY_REGION, HUB_FLY_VOLUME_GB | fly spawner: instances app, region, per-user volume |
HUB_FLY_TOKEN | Machines API token scoped to the instances app (a Fly secret) |
HUB_IDLE_MINUTES | Stop an instance after this long with no input (default 30; live: 20) |
HUB_BUSY_MAX_HOURS | Longest an idle but busy instance is kept running (default 12) |
HUB_ALLOWED_ORIGINS | Extra origins allowed to open /ws (CORS with credentials) |
HUB_ALLOWED_ACCOUNTS | Invite-only allowlist of ids or handles, checked on every request (live: one id) |
HUB_COOKIE_SECURE | 0 only for plain-http localhost |
Run: uv run uvicorn backend.hub.app:app --host 0.0.0.0 --port 8080.
Frontend on Vercel
Vercel can host the static frontend, not the backend. Rewrite /api/* and
/hub/* to the hub so the cookie stays first-party, and build with
VITE_WS_ORIGIN set to the hub: rewrites cannot carry WebSockets, so /ws dials
the hub directly with a single-use ticket from POST /hub/ws-ticket
(socketUrl in hosted.ts). Add the Vercel origin to HUB_ALLOWED_ORIGINS.
Deploying
Live deployment (2026-10-06): frontend https://horrible-dashboard.vercel.app, the only front door — page requests to the hub redirect there (HUB_FRONTEND_URL), so a UI fix ships once
(Vercel project horrible-dashboard), hub https://horrible-hub.fly.dev (Fly app
horrible-hub), user machines in Fly app horrible-hub-instances. Both Fly apps are
on their own private network, horrible-hub-net.
-
Fly (
HUB_SPAWNER=fly):backend/hub/spawner/fly.pycreates one Fly Machine (a Firecracker microVM) and one encrypted volume per user, with no services, so a machine is reachable only at<id>.vm.horrible-hub-instances.internal. Setup and release commands are in the header offly.hub.toml; the instance image is pushed byfly.hub-instances.toml, and bumping its label in both files rolls every user's machine onto the new build the next time the hub starts it. A machine that has been running throughout keeps its old image until then. To roll one immediately, runfly machine update <id> -a horrible-hub-instances --image registry.fly.io/horrible-hub-instances:<label>. This restarts it, keeps its volume, and the hub treats the resulting:<label>@sha256:…pin as current. -
Vercel: the project is connected to the GitHub repo, so every push to
maindeploys the frontend, and other branches get preview deployments. Two project settings make that build correct, and both are needed:- Root Directory
apps/web, with files outside it included. Vite writes toapps/web/dist, and only from there does Vercel readapps/web/vercel.json, whose rewrites send/apiand/hubto the hub. Built from the repo root, it failed with "No Output Directory named dist". Fixing only the output path would have shipped a site with no rewrites. - Env var
VITE_WS_ORIGIN=https://horrible-hub.fly.dev(Production and Preview). Rewrites cannot carry a WebSocket, so the bundle must know to dial the hub directly.
A manual deploy (
vercel build --prod, thenvercel deploy --prebuilt --prod, fromapps/web) still works, but is no longer the release path. A new frontend origin must be added toHUB_ALLOWED_ORIGINS(fly.hub.toml), or its sockets are refused. Preview deployments' own URLs are not on that list, so their live socket is refused by design. - Root Directory
-
Bind address trap: asyncio makes a
::socket IPv6-only. Fly's proxy and health checks reach the hub over IPv4, so the hub binds0.0.0.0; Fly's private network is IPv6-only, so the Fly spawner starts instances withHORRIBLE_BIND_HOST=::. Docker bridges use the instance's IPv4 default. -
One Docker host (VPS):
deploy/hub/docker-compose.yml— Caddy (TLS) → hub, with the instance image built fromdeploy/dashboard-node/Dockerfilevia--profile build. Buildapps/webfirst; the hub image copies itsdist. -
Local test without the compose file: run the hub natively with
HUB_SPAWNER=docker HUB_DOCKER_REACH=published HUB_COOKIE_SECURE=0, and a local game server (uv run uvicorn backend.games_server.app:app --port 9090) withGAMES_SERVER_URL=ws://host.docker.internal:9090so the instances reach it too. Signed in, the command palette has Account: Sign out of this dashboard (hosted.signOut, thehostedmodule, registered only behind a hub).
Not built yet
Quotas and a signup cap, volume backups, and image upgrades for the Docker spawner (an existing user container keeps its old image until removed; the Fly spawner rolls them).