Module: network
Its surfaces are sections of the People pane
(people.home). Everything below still describes the services, settings and
components; only the "where do I click" answer changed. Old names keep resolving —
show("Peers"), show("social.friends") and the rest land on the right section.
Distributed multi-user collaboration: connect this node to other users' nodes, share panes live, and let your agent talk to theirs. The frontend is a Peers widget plus settings; the heavy lifting is the backend peer fabric documented in Distributed peer fabric.
Status: implemented (groundwork). Frontend in
packages/core/src/modules/network/, backend in backend/modules/network/.
Contributions to the layout shell
No panes, and no widgets. This module contributes services and settings only.
It used to own five widgets — Peers, Peer Chat, Peer Monitor, Lobby and Agent Relay, grouped as a "Peer Fabric" panel group. Only one of them was something a person wanted to look at, and it is now the Messages section of the People pane. The other four were infrastructure readouts that became destinations because a pane was the only way to show anything — "Peers" in particular answered which machines are connected when the question people actually have is which of my friends is around, which People answers by person.
Nothing was lost. The transports, trust store and rendezvous are still here as
services (ws.ts, lobby.ts, peerchat.ts, collab.ts), the knobs are on the
settings page, and the traffic is in observability. What went
away is the idea that each of them deserved somewhere to live.
- Commands: none of its own; People's commands reach every surface.
- Settings: below, split basic / advanced.
Network-aware panes (the collab contract)
A pane declares it participates in the peer fabric through the collab field
on its PanelDecl/WidgetDecl (the CollabDecl shape in
@horribledashboard/sdk):
collab?: {
room: 'shared' | 'instance'; // one well-known room, or one room per pane instance
key?: string; // key suffix; defaults to the view id
autoShare?: boolean; // start shared on open (default false — opt-in)
}
This makes networking a declared, first-class property of a pane rather than
something each pane hand-wires. Panes consume it through the useCollab(paneKey)
host hook (exported from the network module), which owns joining/leaving the room,
tracking the authoritative revision, rebasing on a rejected (stale) op, and
surfacing a live presence count — returning { text, setText, shared, setShared, members }. The scratch panel is the reference consumer.
The agent-to-agent tools list_peers and agent.ask_peer are backend-static
(resolved in the orchestrator against the PeerHub), so unlike other modules they
need no frontend agentTools handler — there's nothing for a browser to execute.
They're documented in Agent chat.
Backend surface
REST — /api/network
| Method & path | Purpose |
|---|---|
GET /identity | This node's NodeIdentity (node_id, public_key, node_name, capabilities). |
GET /peers | { self, peers[] } snapshot. |
POST /invite | Mint a single-use invite { invite, token, expires }. |
POST /pair | Redeem an invite (decode → dial → handshake with the token). |
POST /connect | Dial an address directly { address, transport }. |
DELETE /peers/{node_id} | Disconnect a peer. |
POST /ask-peer | Ask a peer's agent { peer_id, prompt } → { ok, answer?, error? } (REST entry to agent.ask_peer; the remote turn is gated read-only). |
POST /bench | Time a link { node_id?, mode: echo|sweep|local, count }. 409 while one is already running — two benches each measure the other's traffic. |
GET /leases | Every live lease in both directions, plus this node's lending stance. |
DELETE /leases/{lease_id} | End a lease: give back one borrowed, or reclaim one lent. |
WebSocket endpoints
-
/peer-ws— inbound peer connections (the signedPeerEnvelopeprotocol between nodes; not the browser channel protocol). Distinct from/ws. -
/wsnetworkchannel — live presence + control for the browser:Direction event data client→server list_peers{}client→server list_metrics{}(request an immediate Peer Monitor snapshot)client→server connect{ address?, nodeId?, transport }client→server disconnect{ nodeId }client→server pair_redeem{ invite }server→client peers{ self, peers[] }server→client peer_update{ peer }server→client peer_metrics{ metrics[] }(periodic + on request — see Peer Monitor)server→client pair_result{ ok, peer?, error? }server→client error{ message } -
/wscollabchannel — shared-pane sync (see below). -
/wspeerchatchannel — direct 1:1 peer messaging (see Peer Chat). -
/wslobbychannel — discovery + rooms for the browser:Direction event data client→server state{}(request a snapshot)client→server connect{ url? }(connect to a lobby server)client→server list_rooms/create_room/join_room/leave_room{ … }server→client state{ connected, url, rooms[], directory[], self }server→client rooms{ rooms[] }server→client directory{ directory[] }server→client room_created/joined/peer_joining/error{ … }
The lobby server itself is a standalone app (lobby_server.py) exposing
/lobby-ws (directory + rooms + signaling) and the bundled /relay-ws for the
fallback path; run it separately:
uv run uvicorn backend.modules.network.lobby_server:app --port 9000. A node opts in
via network.lobbyUrl. See
the lobby system.
One relay connection per data directory. Every backend that shares a data
directory is the same node (one identity), and the relay broker keeps a single socket
per node id, closing the older one when the same id registers again. So when two
backends ran at once — a dev backend and the desktop one, or a standalone module — they
knocked each other off the relay every few seconds (measured: 38 re-registrations in
six minutes). build_transports() now takes the relay under the bus claim
network.relay (<data>/bus/claim-network.relay.json, created O_EXCL; a dead
holder's claim is reclaimed). The second process logs relay: dashboard (pid N) already holds this node's relay connection and runs without the relay; it does not
take over if the holder stops, so restart it if it needs the relay.
Collaborative shared panes
A CollabRoom (collab.py), keyed by an opaque paneKey, holds authoritative
text + a monotonic rev. Members sync with a rev check — last-writer-wins,
but no lost updates: an op with a stale baseRev is rejected and the writer
rebases onto the authoritative state. Inbound peer ops are adopted as authoritative
by rev and rebroadcast to local members (not re-forwarded, so no loops). This is
groundwork, not a CRDT — a clean seam for one later.
A share is addressed to a person
You share a pane with Andrew, and the fabric picks whichever of his machines is online. Asking someone which of their own computers to send a pane to is a question with no good answer, and it is the same reason the roster is keyed by person.
Each room tracks the nodes it is actually shared with (room.peers). A node
lands there two ways: you shared the pane with its owner, or an op for that pane
arrived from it — which only happens if they shared with you, and is how the far
side learns to route edits back.
_forward_to_peers used to iterate peer_hub.peers — every node with a live
session — and send the pane's full text to each. Anyone you were connected to
received the contents of every shared pane, whether or not they had any part in
it. "Sharing" was a global broadcast wearing the word "room".
Inbound ops are now also gated on session.info.trusted. Knowing a pane key is not
permission: ops are last-writer-wins, so an untrusted peer that guessed or overheard
a key could both read the pane (we would start forwarding to them) and rewrite it.
| Direction | event | data |
|---|---|---|
| client→server | join | { paneKey } |
| client→server | leave | { paneKey } |
| client→server | op | { paneKey, baseRev, text } |
| client→server | share | { paneKey, personId } |
| client→server | unshare | { paneKey, personId } |
| server→client | state | { paneKey, rev, text, members } |
| server→client | op | { paneKey, rev, text, from } |
| server→client | presence | { paneKey, members } (occupancy changed) |
| server→client | shared | { paneKey, people: [{ personId, name }] } |
| server→client | rejected | { paneKey, rev, text } |
| server→client | error | { paneKey, message } (e.g. nobody of theirs is online) |
shared resolves nodes back to people on the backend, because the node holds
the device→person map and a pane listing node ids would be exactly the
machine-shaped UI the roster exists to replace.
The reference consumer is the scratch panel's "Share with…" picker, which
declares a collab room and drives the
useCollab hook
(modules/scratch/index.tsx + modules/network/useCollab.ts). See
Scratch.
Peer Monitor
A backend PeerMonitor (monitor.py, started by the app lifespan) heartbeats
every connected peer on an interval: it pings each (the hub's ping/pong
round-trip) to measure RTT and samples the per-session byte/message counters the
hub maintains. Each tick is broadcast as a peer_metrics event on the network
channel; a freshly opened monitor requests an immediate snapshot with
list_metrics. The network.monitor widget renders the table. The monitor is
read-only — it never mutates peer state beyond stamping each session's rtt_ms.
Each PeerMetrics row: { node_id, node_name, transport, status, rtt_ms, bytes_in, bytes_out, msgs_in, msgs_out, last_seen }.
Note that rtt_ms is not a wire latency. ping is answered inside
_dispatch, so a ping queued behind a slow handler measures the network plus
the remote pump's head-of-line delay. That makes it a useful blocked-pump
detector and a poor network measurement — which is what the bench exists for.
Bench
bench.py measures the fabric itself. PeerMonitor keeps one latest RTT sample
per peer, which cannot answer the question that actually matters — what did a
20 Hz stream's p99 do while something bulky ran — because that distribution is
bimodal. Every number the bench reports is a percentile; none is a mean.
Four modes:
| mode | what it answers |
|---|---|
local | the CPU floor: construct, sign, serialize, deserialize, verify, with no link at all |
echo | round-trip latency against a peer's bench_echo handler, one message at a time |
sweep | how that round trip grows with payload size |
sustained | what bulk traffic does to a concurrent 20 Hz victim stream |
local rather than loopback is the zero-wire baseline: LoopbackLink.send does
decode(encode(env)) inline on the sender, so it charges deserialization to the
wrong side and adds a queue and a task switch.
Results carry the bench's own transport label, never PeerInfo.transport —
LoopbackLink.transport_name is "direct", so reading the peer's field would
file every loopback run under the wrong transport.
Cross-node wire time is not measurable without clock sync, and the bench does not
pretend otherwise: what remains after subtracting the local phases from the round
trip is reported as wire_residual_ms, named for the weaker claim it can support.
Observation must not be what it observes
The hub hook is a module-level _probe read behind a single
if _probe is not None, installed by hub.set_probe() only while a run is in
flight. Not a decorator, not a context manager, not logging — at 20 Hz across
several peers each of those costs enough to appear in its own measurement.
Samples land in preallocated array('d') rings (Ring), so recording one
allocates nothing; percentiles are computed at snapshot time.
Capabilities
A node advertises what it offers during the handshake, and refreshes it live via
presence. Modules contribute through a registry (capabilities.py) rather than
a hardcoded list, so adding a capability never edits the fabric core.
Two fields, and why they can never be merged
| field | shape | who reads it |
|---|---|---|
capabilities | list[str] | everything that exists today, incl. the Android client |
caps | list[PeerCapability] | anything wanting live detail |
capabilities is frozen forever. commons.build_profile feeds it into
CommonsProfile.agent_capabilities, which is Ed25519-signed over
canonical_profile_bytes and re-served by a federated index — changing the
element type invalidates every profile already published. The lobby also puts it
on the rendezvous wire typed string[], and the Kotlin client sends exactly this
shape.
So caps is additive. A peer that sends only capabilities — any Android
build, any node older than this — has its caps synthesized from it, and every
existing consumer ('agent' in p.capabilities in the relay panel, hassault's
fabric and routes, mobile_tools) is untouched. PeerHub.capabilities() still
returns exactly ["agent", "collab", "hassault", "share"], pinned by a test.
PeerCapability
{ id, version, attrs }. attrs is deliberately untyped — each capability's
owner defines its own keys, and a schema here would mean editing the fabric core
to add a feature. It must be JSON-serializable: these bytes are signed, so a
value json.dumps chokes on would fail the handshake rather than merely being
dropped. snapshot() therefore discards an unserializable attr and logs it,
rather than letting a module make its node unreachable by putting a datetime in
its attrs.
Providers are called at advertisement time, not registration, so attrs are
current rather than whatever was true at boot. Returning None withdraws the
capability — "not offering this right now" has to be expressible, or a node with
no models would advertise inference and fail every request against it. A provider
that raises is logged and skipped: one module's bad probe costs that module's
capability, never the node's ability to connect.
Built-ins register at import, not in start_network, because a bare
PeerHub() must report them.
What each module contributes
hassault—openMatches,matches,maxPlayers. This is what letsbrowse_peersskip a friend hosting nothing instead of fanning out to everyone and waiting on the deadline.inference(llamacpp/capability.py) —accelerator,vramMb,ramMb,models,modelCount, andserving. Three rules it follows: it reportscertain: falserather than claiming there is no accelerator when the probe could not ask (thehardwaremodule's three-state rule, which would become a fiction acted on by a peer if flattened here); it caches, because a model list is a filesystem walk and presence can fire every few seconds; and it sends model names, never paths, since a path leaks the machine's directory layout and username for no benefit.
serving is the field the compute-lease policy turns on — granting against the
already-loaded model needs no eviction — so capability.changed() invalidates the
cache and announces whenever a model is loaded or stopped.
Presence is debounced
announce_presence() enforces a 5-second minimum gap with a trailing edge. A
filling hassault match would otherwise broadcast to every peer on every join; and
dropping the last call of a burst rather than deferring it would leave peers
believing a match is still open after it filled.
Dispatch policy
Every inbound message type declares how its handler runs relative to the
session's receive pump, via register_handler(..., mode=…):
| mode | behaviour | when |
|---|---|---|
inline (default) | awaited on the pump | the handler is a dict write and a send |
detach | its own task | required if the handler calls hub.request |
serial | per-(session, type) FIFO worker | slow and order-sensitive |
detach is not a preference for anything that awaits a reply, it is a
correctness requirement: a reply arrives on the very pump an inline handler is
occupying, so such a handler deadlocks the session against itself.
serial exists because detach cannot promise order — detached tasks race for
the send lock, and two collab ops applied backwards is a silently corrupted
document. Current assignments: collab_op is serial; remote_command and
hassault_browse are detach; everything else is inline.
agent_request stays inline despite running for minutes, because the handler
only spawns the turn and returns — it has to keep a cancellable handle in
_active_remote_turns so agent_cancel can find one turn by request id, which a
generic detach cannot offer.
A serial queue that overflows (256 deep) closes the session rather than
dropping the envelope. An ordered stream with a hole in it has no correct
continuation the transport can invent, so closing forces the feature layer's
existing reconnect-and-resync path — a defined recovery, where a silent drop is
not. Only that one peer is affected, and only by its own flooding.
Failures in a detached or queued handler no longer unwind into _pump, so
_invoke catches and logs them. Without it they would vanish entirely, which is
a worse failure than the blocking that moving them off the pump cures.
Compute lending
Borrowing a friend's machine: their loaded model, their GPU, or an optional extra
they have installed. Two pieces — a lease (lease.py) that says who may use
what until when, and a tunnel (tunnel.py) that makes their loopback service
reachable as a local port.
Consent is three gates
| gate | default | effect |
|---|---|---|
network.allowComputeLending | off | nothing is lent at all |
network.computeLeasePolicy | ask | ask / trusted / off |
session.info.trusted | — | checked unconditionally |
They are independent, and each alone is sufficient to refuse. Trust is checked first because friendship grants reachability, not authority — accepting a friend is not agreeing to run their workloads, and knowing a lease id is not friendship.
computeLeasePolicy fails closed on any unrecognised value, the way
remoteAgentMode falls back to read-only plan: a typo or a downgrade from a
newer version must narrow access, never widen it. ask currently denies,
because there is no approval UI yet — silently treating "ask" as "yes" would grant
access the user believes they still have to approve.
Only what is already loaded
A lease naming a model is granted only if that model is already loaded
(network.computeAllowModelSwap opts out). Evicting somebody's own chat model
because a friend asked should not be reachable by default, and the refusal names
what is loaded ("this node is serving gemma; ask for that") so the borrower
can simply ask again. This is what makes the serving attr on the inference
capability load-bearing: a borrower reads it first and the eviction problem
disappears for the common case.
Authorization is re-checked on every connection, not once at grant time — a lease can expire or be revoked between opening two sockets.
The tunnel is a raw byte pipe
Not an HTTP proxy. For rpc-server there is no choice; it is raw TCP. For the
chat endpoint a proxy is tempting — per-request accounting, clean cancellation —
but it would mean reimplementing chunked transfer and, worse, preserving SSE chunk
boundaries, when _openai_chat_stream parses data: lines straight off httpx's
byte stream. A raw tunnel gets SSE for free, because SSE is bytes on a socket:
httpx against http://127.0.0.1:<tunnel_port> is byte-identical to httpx
against a local server. That is precisely what lets a borrowed model be an
openai-dialect provider with no new provider code.
Three limits, and the third exists because of a measurement:
- 64 KB frame cap — a 1 MB frame occupies the receiving pump for 19 ms, uninterruptibly, against a 50 ms budget at 20 Hz.
- Credit-based flow control, returned only after bytes are drained to the local socket. Returning credit on receipt would let a sender outrun a slow service and rebuild the unbounded buffer this exists to prevent — and the producer is a remote peer, so that buffer is a remote memory-exhaustion lever.
- A rate limit below saturation (16 MiB/s default). This is the one that is easy to leave out. Saturating the fabric moves ~57 MiB/s but costs a concurrent 20 Hz stream 6× its latency, and because the cost is signing the bytes rather than scheduling them, neither smaller frames nor a dispatch mode fixes it. A tunnel that may share a link with a game has to stay below saturation.
One TunnelManager per node. Stream ids are unique per node, not globally, so
two hubs sharing a manager would have one end of a stream overwrite the other
under the same id — impossible across real processes, but the default in an
in-process test, where it silently proxies a stream back to itself.
Borrowing installed extras, not just a GPU
A lease also carries an optional extra a peer has installed and this node does
not — voice (torch), llamacpp (the tracer, needing cmake and a compiler),
browser-engine (Chromium), clip (ONNX weights). No GPU is involved; the
scarce thing is the install.
borrow.route(extra) decides where work runs: local if the extra is here, a peer
if it is certainly absent and a friend advertises it, otherwise the install
hint the feature always gave. An uncertain local answer — installed but failing
to load — never routes to a peer, because that would paper over a broken local
install by silently shipping the user's data elsewhere.
See optional extras for the probe convention and the full rule.
ggml --rpc offload: available, but not through this pipe
--rpc distributes a single model's layers across machines, and the build this
node downloads turns out to have it (probed, not assumed — see
speculative decoding; an earlier reading of
upstream's docs said release binaries lacked it, and that is no longer true).
Tunnelling rpc-server over the fabric would also be the right answer to
upstream's warning never to expose it to an untrusted network.
It is still not wired up, and the reason is bandwidth rather than availability. ggml RPC moves tensors — gigabytes — and the fabric signs every byte it carries: measured, Ed25519 signing alone caps the link near 57 MiB/s, and this tunnel deliberately runs below that. Distributed inference through it would be slower than staying on the CPU. Getting there needs a binary side-channel authenticated once at open rather than per frame, which is a different and larger piece of work.
Borrowing, and losing a lease mid-turn
The peer provider has dialect="openai" — not a new dialect, because what is on
the far end is a llama-server. _endpoint_for resolves it from the live lease
and does so before the saved-config check, for the same reason the llamacpp
branch does: the tunnel port is ephemeral and chosen at lease time, so a saved
endpoint for this provider can only be stale. Setting agent.coder.provider to
peer runs one agent on a friend's GPU while the orchestrator stays local.
peer is listed in GET /api/agent/status only while a lease is held. It is
the one provider nobody can install or fix, so a permanently-unreachable row would
be noise in a list whose purpose is telling the user what they could switch to.
Revocation closes the lease's streams immediately, which is what makes it visible: the borrower's socket closes, so httpx raises inside an already-200 response rather than the answer silently truncating.
Byte accounting
Per-session bytes_in/bytes_out come from the transport, which already holds
the encoded frame (PeerLink.last_sent_bytes / last_recv_bytes). The hub used
to serialize every envelope a second time in each direction purely to length it.
Two notes. The counters are bytes, not characters — the old len(str)
under-counted every non-ASCII payload, so an accented node name or a message in
any non-Latin script made the observability panel quietly wrong. And send still
takes an envelope rather than a pre-encoded string, because RelayLink.send
legitimately rewrites the dst routing header (excluded from the signature)
before encoding; a pre-encoded frame could only be patched by editing JSON text.
What the first run found
Measured on loopback, so these are a CPU floor rather than a wire result:
-
Sustained 64 KB bulk moves ~57 MiB/s, and a competing 20 Hz stream degrades from 0.29 ms to 1.80 ms p50 (6.1×).
This one is worth reading carefully, because the dispatch policy did not fix it — the number is unchanged after that work. Two different problems were hiding under "head-of-line blocking": a slow handler occupying the pump, which
detach/serialcure (a 500 ms handler went from stalling an unrelated echo for >300 ms to under 100 ms), and a bulk payload occupying the pump with signing and verification, which they do not, because the handler is trivial and the cost is the bytes. The second needs rate limiting, not a dispatch mode: bulk traffic sharing a link with a game must stay below saturation, not merely be split into smaller frames. -
The throughput ceiling is signing, not serialization. At 256 KB,
sign_envelopecosts 0.74 ms of which the raw Ed25519 sign is 0.49 ms — Ed25519 hashes the whole message, so signing is linear in payload at roughly 525 MB/s per core. Serialization changes cannot get past that; only not signing bulk bytes per frame can. -
A single 1 MB frame occupies the pump for 19 ms, against a 50 ms budget at 20 Hz. Bulk traffic on the fabric needs a frame cap well below that.
What the panes show
Three readouts, folded into Me → Connection diagnostics / Link health / Agent relay rather than given panes of their own — they answer questions, they are not destinations.
Peer Monitor gained a capability chip per peer (inference · 12.0 GB · serving gemma4:e2b, extras · voice, clip, hassault · 1 open) and a Leases table
covering both directions, with Revoke on one lent and Release on one borrowed.
Lease rows arrive on the lease_update push rather than a poll, because the
transitions that matter start on the other node — a peer revoking mid-turn, a
lease expiring — and a view that refreshed only on its own actions would keep
showing a lease already taken away. Revoke re-reads rather than removing the row
optimistically: a row that vanished locally while the lease survived would be a
lie in the one view whose job is saying what is currently granted.
Link health runs the bench and shows the phase split as
p50/p90/p99/max. local mode needs no peer, and that is the point — it is the
only way to tell "that peer is far away" from "this machine is slow".
Agent relay shows a borrowed provider when one is live. It is the pane about routing turns off-node, so it is where "this answer came from somebody else's GPU" belongs; nothing else in the UI says so.
A borrowed lease is reached through a 127.0.0.1 tunnel port, so anything
derived from an endpoint reports "local" for a remote run. Every honest signal is
therefore recorded explicitly rather than inferred:
- an eval run stores
node(see evals), - a peer-embedded batch returns
method: "peer:<node>/<model>"(see library), - a remote research subagent stores
ran_on(see research).
Each renders only when it is set, because "ran on this node" on every row is noise and its absence already means local.
Embedding offload is opt-in
database.embedOnPeer (default off) routes whole embedding batches through a
lease. A batch is the one embedding shape worth sending off-machine — a 200 ms
round trip is nothing against a multi-second batch — but shipping every index
build to a friend's machine should be a choice, not a default.
The lender's own /api/database/embeddings/batch calls get_embeddings with
allow_peer=False. Without that, a lender whose offload setting is also on would
answer a borrowed batch by borrowing it again, and a two-node pair would hand one
batch back and forth.
Peer Chat
The transport half of direct messaging: ChatManager (chat.py) relays a message
over the signed peer wire (a peer_chat envelope) and mirrors it to this node's
own tabs. Where collab syncs an editable document, peerchat is an
append-only message log.
Conversations are keyed by person and persisted in social_messages, not held
in memory per node — see Messages for the channel contract,
the unread counts, and why an inbound message nobody is reading raises a
notification.
Agent-to-agent relay
The network.relay widget (Agent Relay) is the UI over agent-to-agent: pick an
agent-capable peer, ask a question, and read the answer. It calls POST /api/network/ask-peer, a thin wrapper over the same agent_bridge.ask_peer the
agent.ask_peer tool uses, so the remote turn runs gated and read-only-by-default
on the peer's node (which must have network.allowRemoteAgent enabled). See
Agent chat.
Agent tools
network/agent_tools.py is the agent's surface over everything on this page:
capability advertisement, the bench, leases, the tunnel and borrowable extras were
all machinery the agent could not reach.
| tool | what it answers |
|---|---|
network.survey | who is reachable, what each offers, at what latency, which leases exist, and whether this node lends |
network.measure_peer | link latency as p50/p90/p99 per phase (echo, sweep, or local for this machine's own crypto floor) |
network.find_peers | peers offering a capability, filtered on its live attributes |
network.request_compute | ask a peer for a lease and open its tunnel |
network.renew_lease / network.release_lease | extend one, or end one in either direction |
One find, not four
"Find a friend", "find an open game", "find a peer with a GPU" and "who can
transcribe this?" were three separate mechanisms before
capability v2. They are now one query over PeerCapability.attrs:
find_peers(capability="inference", attr="vramMb", atLeast=8000) → a free GPU
find_peers(capability="extras", attr="installed", contains="voice") → a transcriber
find_peers(capability="hassault", attr="openMatches", atLeast=1) → an open game
The tool count therefore does not grow when the next capability is added — and for
the same reason the three lease tools take the generic service string
(llama, embed, voice, clip, trace, browser) rather than one tool per
borrowable thing.
Not "matches as zero", and not "matches because we could not tell". Answering
"who has 8 GB of VRAM?" with a node that never reported its VRAM is worse than an
empty answer — the same three-state discipline the hardware
probe exists to enforce. find_peers also reports how many peers
it searched, so an empty list is not read as "nobody has one" when the real
answer is "nobody is connected".
All six are loadable, and that is the point
They register with group="network", so they cost nothing on a turn that is not
about the fabric. An ungrouped backend tool is in every turn's schema, and the
always-on core was cut from 34 tools to 11 against a reasoning cliff that appears
around 40 — six new always-on tools would quietly undo a chunk of that. The two
peer verbs that stay always-on are unchanged: list_peers (the prerequisite
agent.ask_peer documents) and agent.ask_peer itself, the one peer verb a user
names mid-conversation. See Agent chat for the group mechanism.
survey reports why this node will or will not lend as a sentence, not as two
flags. The commonest confusion here is a user who has turned
network.allowComputeLending on and still sees every request refused, because
network.computeLeasePolicy defaults to ask and there is no approval UI yet.
Remote control (remote_command)
A trusted peer can drive this node: open a pane, play media, raise a notification,
ask for an agent-generated briefing, or cancel a turn it started. Handled by
remote_control.py, which drops any command from an untrusted peer; the reverse
direction is the mobile.* agent tools in mobile_tools.py. The Android
companion app is the primary consumer of both — it pairs by
QR and appears on the fabric as an ordinary peer carrying the mobile capability.
That page holds the full command table.
A remote agent turn (agent_request) is run on a detached task tracked in
agent_bridge._active_remote_turns, so a follow-up agent_cancel can stop it.
Handlers are awaited inline by the session pump, so a handler that awaited a long
turn would wedge the receive loop for its duration — see
Cancelling a remote agent turn.
Every agent_request is built by agent_bridge.request_peer_agent (the chat's
agent.ask_peer and research's peer sub-agents alike). Besides request_id, prompt
and the origin_chain loop guard, it carries optional trace fields, all ignored by an
older peer:
| Field | On | |
|---|---|---|
traceparent | agent_request | W3C trace context; the callee's turn continues the caller's trace |
want_spans | agent_request | ask for that turn's spans back |
trace_sinks | agent_request | fingerprints of the caller's Opik destinations (at most 4 are read) |
trace | agent_result | the turn's spans: enc, body (OTLP protobuf, gzip, base64, ≤ 256 KiB), truncated, exported_to |
trace is sent only when the callee has network.returnTraces on (default off), and
never carries prompts or tool data. See Peers return their
spans.
Screen watching (view_request / view_frame) is scaffolded but not
functional — the desktop never emits a frame. See
Remote view.
Settings
Split basic and advanced on the settings page. The line between them: a basic setting is one a working install might legitimately change (what am I called, do I accept connections, who may pair). Advanced is NAT-traversal plumbing — TURN credentials, STUN hosts, relay and signalling URLs — which is either irrelevant or copied verbatim from whoever runs your infrastructure. Fourteen knobs at equal weight taught people to skim past the six that matter.
Basic
| Key | Type | Default | Meaning |
|---|---|---|---|
network.nodeName | string | hostname | Display name advertised to peers. |
network.enableDirect | boolean | true | Accept/dial direct WebSocket peers. Restart. |
network.enableLanDiscovery | boolean | false | Advertise/discover via mDNS. Restart. |
network.trustMode | enum | manual | manual (an invite you generated) or open-lan. |
network.lobbyUrl | string | "" | Lobby server ws://…/lobby-ws for discovery + rooms (blank = off). |
network.allowRemoteAgent | boolean | false | Let a trusted peer's agent ask yours. |
network.returnTraces | boolean | false | Send the asking peer the spans of its answer — shape, timings, models, tool names; never prompts or tool data. |
network.allowComputeLending | boolean | false | Let a trusted friend borrow this machine (loaded model, GPU, installed extras). Off means nothing is lent whatever the policy says. |
network.computeLeasePolicy | enum | ask | Who may hold a lease: ask (approval required — currently denies, no UI yet), trusted, off. Unrecognised values read as off. |
Advanced
| Key | Type | Default | Meaning |
|---|---|---|---|
network.advertisedAddress | string | "" (auto) | URL peers dial to reach this node, encoded into invites. Blank auto-detects this machine's LAN IPv4 — localhost would make every invite undialable from another device — and the port this backend is actually serving on (see The advertised port). Set it for a public hostname or a forwarded port. |
network.relayUrl | string | "" (the game server's relay) | Relay broker URL, which is how friends on other networks reach this node. Blank uses wss://<game server>/relay-ws; off disables relaying, leaving direct and LAN only. HORRIBLE_RELAY_URL overrides it. See social. Restart. |
network.iceEnabled | boolean | false | Gather a STUN server-reflexive candidate (public IP) alongside host/LAN candidates. |
network.stunServer | string | stun.l.google.com:19302 | host:port for STUN public-IP discovery (when ICE is on); also the STUN ICE server for the WebRTC transport. |
network.enableWebRtc | boolean | false | Connect over a WebRTC data channel, SDP signaled via the lobby. Needs uv sync --extra webrtc; the relay stays the fallback. Restart. |
network.turnUrl | string | "" | Optional TURN relay for WebRTC (turn:host:3478) for NATs STUN can't punch (blank = STUN only). |
network.turnUsername | string | "" | TURN username credential (if required). |
network.turnCredential | string | "" | TURN password/credential (if required). |
network.computeAllowModelSwap | boolean | false | Let a borrower load a different model. Off means a lease is granted only against the model already loaded, so a friend's request can never evict your own chat model. |
network.remoteAgentMode | enum | plan | Permission mode for a remote turn (plan = read-only, then acceptEdits, autonomous). |
Restart marks the four settings build_transports() reads once at startup. A
transport owns a bound socket or a multicast thread, so swapping one under live
sessions is a different job from reading a flag; before they said so, they looked
like live switches that silently did nothing.
Strangers are admitted, not rejected
trust.admit replaces the handshake's accept-or-reject. A node that proves its
identity but is not trusted gets a stranger session: it can carry only the types
opened with hub.allow_from_strangers, is invisible to peers and list_peers(),
and is closed after two minutes unless it becomes trusted. Rejecting it used to make
friending anyone not already paired by invite impossible. The full rules are in
social. An older node still rejects strangers, and the
dialer's AUTH_RESULT handling reads a missing trusted field as trusted, since
such a node admitted only peers it trusted.
hub.connect is bounded by HANDSHAKE_TIMEOUT_S (20 s). A relay dial to an offline
node, or a direct dial into a router that drops the packet, used to wait forever.
The advertised port
Every address this node hands out — invites, ICE host candidates, the
presence record, the tunnel target
for lent extras, the OAuth redirect back to this backend — takes its port from
backend/server_port.py, the one authority for it. It used to be
HORRIBLE_DEV_BACKEND_PORT with a default of 8000, and only scripts/dev.mjs sets
that variable. So a node started any other way — the documented bare
uv run uvicorn backend.app:app --port 8100, a systemd unit — advertised 8000 wherever
it listened. Found live: a friend's Linux node published ws://10.0.0.142:8000/peer-ws
while answering on 8100, and every reconnect through the directory dialed a closed
port, silently. A repo .env that sets the variable has the mirror-image effect: any
node on that machine on a different port advertised the .env one.
The port cannot simply be read off the socket at startup, because uvicorn runs lifespan startup before it binds — and presence is published from lifespan. So the answer is resolved in order of authority:
| Source | When it is known |
|---|---|
scope["server"] of a real request | from the first HTTP or WebSocket request on. It is getsockname() of the socket the request arrived on — behind the Vite proxy still the backend's own |
--port / --bind host:port on the command line, or UVICORN_PORT | at startup. Under --reload the worker is a spawned child whose sys.argv is the parent's, so this holds there too (checked on Windows) |
HORRIBLE_DEV_BACKEND_PORT | fallback only |
8000 | nothing else is known |
network.advertisedAddress still overrides all of it. When the first request shows a
port different from the one startup assumed — a server started from Python with
uvicorn.run(...), whose command line names no port — the presence record is
republished, since the one already out carries a dead address, and the log says
serving on port N, not the port assumed at startup. The OAuth redirect additionally
honours an explicit HORRIBLE_BACKEND_PORT, because a registered redirect URI names a
port and whoever pinned it meant that one.
Two settings that were removed
network.directoryUrlhad zero readers repo-wide. Finding people by name is the Atlas presence directory and the game server's username directory, neither of which is configured here.trustMode: 'directory'was never implemented, sotrust.evaluaterejected every peer — selecting it made the node silently unpairable.trust_mode()now reads a storeddirectoryasmanual, so an existing config recovers rather than staying broken.
remoteAgentMode: 'ask' went the same way: ASK prompts a human, and the human it
would prompt is not the one who started the turn. It was mapped to default, so
choosing "ask" silently granted a higher permission level than its name promises.
An unrecognized mode now falls back to plan — read-only — so a stored ask fails
closed.
Pairing is mutual
handshake_dial persists a known_peers record for the peer it dialled, the same
way trust.evaluate does for one it accepted. It used to mark trust in memory
only, so after A redeemed B's invite only B remembered — and the next time A
was the one dialled, B's node had never heard of it and rejected the handshake.
Both ends have verified an identity cryptographically during the handshake; both
should write it down.
Server-side state under $HORRIBLE_DATA_DIR: network-identity.key,
network-peers.json, network-invites.json.
Browser vs desktop
| Browser | Desktop | |
|---|---|---|
Peers widget, REST, /ws channels | ✅ | ✅ |
| Direct / relay transports | ✅ (backend) | ✅ (backend) |
| LAN discovery (mDNS) | depends on the host network allowing multicast | same |
Everything runs in the backend, so both layouts behave identically; LAN discovery is the only capability that depends on the host network (and degrades gracefully when multicast is unavailable).