Skip to main content

Agent tools & permissions

Status: implemented. This page specifies the generalized agent tool surface of the agent orchestrator (per-widget agentTools, agent-exposed commands, pull-based getAgentContext()) and the permission system that gates side effects. It is the shared foundation the file explorer, editor, and terminal modules are built on; the surface was built first, validated against a stub, then those modules were layered onto it. Slice 1 (the static LAYOUT_TOOLS catalog) is in agent-chat.mdx.

Landed: the SDK declaration contract (AgentToolDecl, CommandDecl.agent, getAgentContext/AgentContextSnapshot); the capability-manifest protocol (below); the per-instance getAgentContext registry with the list_open_panes / get_pane_context read tools; the permission engine core (backend/modules/agent/permissions.py — rule parsing, deny → ask → allow precedence, the four modes, circuit breakers) including shell-aware specifier matching (shell.py); the gate + approval round-trip in the orchestrator (_gate, approval_request/approval_response, rule persistence via permission_store); and the approval UI (ApprovalPrompts + useApprovals) with the settings-page permissions section (mode picker + rule lists). The whole permission system (Epic A) is in place, the stub validation ran green across all four modes, and all three real modules now declare their tools on the surface: the file explorer (files.list/read reads, gated files.create/write/rename/delete), the editor (getAgentContext buffer reads, gated editor.proposeEdit [accept/decline diff] / editor.applyEdit/save), and the terminal (terminal.list/read reads, gated shell-matched terminal.exec). Content widgets add their own: observability (getAgentContext I/O snapshot + gated observability.clear) and clubhouse (clubhouse.status/listRooms reads, gated clubhouse.disconnect) — illustrating that any widget gets agent tools just by declaring agentTools on its decl. The chat widget's /tools command lists the full catalog via a list_tools→tools WS round-trip, labeled by group. The model itself doesn't see the whole catalog at once: tools are grouped by prefix and disclosed progressively — a small core plus list_tool_groups/load_tools meta-tools, with each turn's list recomputed and groups injected on demand (see Hierarchical tools & progressive disclosure). Remaining work is incremental per-module polish (e.g. the editor's desktop-only osfile: source), not foundational.

A tool's group is its name prefix​

_group_of(name) takes the namespace before the first dot — github.searchCode is in group github; a dotless name (the layout verbs) is in the always-present layout core.

AgentTool.group does not name the group. It's only a flag: provider_tools(grouped=…) checks whether it is set, to split always-core tools from progressively-disclosed ones. So a tool named game.getObservation that declares group="games" really lands in a group called game — and if nothing describes game, it shows up in the catalog as a bare "game tools" fallback with no blurb and no guide. That exact mismatch existed for the games tools until the connectors work added an invariant test (test_connectors.py::test_agent_tool_names_match_their_declared_group).

Keep the name prefix and the declared group identical. For a connector, both must also equal the connector's id — that's what lets one definition feed the home tile, the group blurb, and the guide.

Forgetting group= is worse than getting it wrong

A mismatched group costs a blurb. An omitted one costs a slot on every turn of every agent: provider_tools(grouped=False) treats a tool with group=None as always-on core, so its schema is charged to each round whether or not the turn has anything to do with it — and the _GROUP_DESCRIPTIONS entry naming it can never fire, because _group_catalog only walks the dynamic pool.

That is exactly what research.* (5 tools) and arxiv.* (3) did: app.py claimed they were "grouped under research/arxiv", _GROUP_DESCRIPTIONS had entries for both, and researcher's preload_groups=[…, "research", "arxiv"] was partly a no-op — while all 8 sat permanently in the main orchestrator's core. Fixing it took the main agent's always-on core from 34 tools to 26, nearly doubling the room left under TOOL_BUDGET for capability groups.

test_connectors.py::test_every_namespaced_agent_tool_declares_a_group now fails on a dotted tool with no group=, alongside the older mismatch test.

Two tiers of disclosure​

list_tool_groups shows each group's one-line blurb (_GROUP_DESCRIPTIONS, falling back to a connector's blurb); load_tools delivers the group's full guide — a SKILL.md-style markdown doc. The blurb answers "would this help?"; the guide answers "how do I use it without getting it wrong?". Guides ride the tool-result message, so they need no orchestrator plumbing of their own.

Any group can ship a guide, not just connectors. _group_guide resolves backend/modules/agent/guides/<group>.md first (agent/guides.py, lru_cache'd), then a connector's resolve_guide(), then the MCP bridge. For a long time only the latter two existed, so every built-in group — layout, files, editor, database, games — had a one-line blurb and nothing else; detail that belonged in a guide had nowhere to go but the system prompt, where it was charged to every round of every turn.

Guides are plain markdown so they review and diff like code. Put anything in one that a model gets wrong repeatedly and that would be too expensive to say on every turn: which argument is an instanceId rather than a view id, which combinations are useless, what the module calls a thing.

show: the intent verb​

Reaching a surface used to cost three rounds — list_available_panes (a row per registered view), open_pane, then get_pane_context. Three chances to emit malformed JSON and a large tool result, for a model whose whole window may be 8k.

show(target, where?) collapses that to one call. target is loose text — a pane title, a view id, a section label, a side-strip name, a workspace name — and the host does the matching (layout/show.ts, pure and unit-tested; showTarget in layout/controller.ts acts on the result). It opens or focuses (never a second copy of something already visible), switches to the named section, reveals a region inside its host, or switches workspace, and returns the resulting pane's context snapshot inline, so no get_pane_context follow-up is needed.

Because panes are being merged rather than deleted, show resolves a section label directly: the user says "friends", not "the People pane, Friends tab", and so should the model. An embedded view resolves to where it lives — its host pane and section — because it has no standalone home to open.

Its argument is deliberately one string, not a discriminated union: _coerce_args turns an unparseable payload into an error, and a wide schema is more of them. The way to shrink a tool list is to move tools out of the always-on core, not to widen the ones that remain.

VIEW_ALIASES (in layout/controller.ts) maps retired ids and titles to where their content lives now. It is deliberately separate from serialize.ts's RENAMED_VIEWS, which repairs saved layouts: a stored arrangement is disposable and reseeds from its preset, but a name the agent has ever used should never stop resolving. Add an entry whenever a pane is merged away.

The model: one tool path, one gate​

Every frontend-declared agent tool call flows through the frontend relay over the shared /ws agent channel — there is no relay bypass, even for tools whose resource (a PTY, the filesystem) lives in the backend. Those tools' handlers run in the frontend and call the backend's existing HTTP/WS API, so the agent has exactly one tool model to reason about.

Backend-registered tools are the exception

Tools contributed via backend/sdk (host.add_agent_tool) run their handler in the backend and never touch the relay — used where a tab may not exist at all (a cron run, the dash REPL, agent.ask_peer) or where the resource is a server-held secret the browser must never see. Today: games, training, and the connectors. They pass through the same permission gate, so the sentence that matters is unaffected: one gate, not one path.

The consequence that shapes everything: because all calls pass through the orchestrator loop, the permission gate sits in the backend orchestrator, before the relay. Permissions are enforced by the orchestrator, not by the model — the system prompt shapes what the agent tries, but the engine decides what runs. This mirrors Claude Code, where rules are enforced by the harness rather than the LLM.

A read-only tool never reaches the engine — the same read-only tier Claude Code uses for file reads and grep.

Declaring agent tools​

Two surfaces, by design (the "hybrid" decision in [[agent-orchestrator-architecture]]):

  1. App-wide verbs are commands. CommandDecl gains an optional agent field so a command opts into being model-callable:

    export interface CommandDecl {
    id: string;
    title: string;
    run: () => void | Promise<void>;
    agent?: { description: string; params?: JSONSchema; sideEffect?: boolean };
    }

    Layout/navigation verbs (open pane, switch workspace) are exposed this way and are read-only or harmless, so they stay ungated.

  2. Widget/panel-specific actions are agentTools. WidgetDecl and PanelDecl gain an optional agentTools?: AgentToolDecl[] (MCP-style):

    export interface AgentToolDecl {
    name: string; // namespaced like a command id, e.g. "terminal.exec"
    description: string; // natural language, for the model
    params?: JSONSchema; // JSON-schema object for the arguments
    sideEffect?: boolean; // falsy = read-only, never gated
    /**
    * Template rendered into the permission *specifier* the engine matches
    * rule specifiers against. `{name}` placeholders are filled from args; the
    * tool name is implicit. e.g. "{command}" → specifier `npm run build`,
    * matched by rule `terminal.exec(npm run *)`. Declarative so the backend
    * builds the specifier itself, keeping the gate fully server-side.
    */
    specifierTemplate?: string;
    /** Decide the gate per call; see "Per-call gates" below. */
    gate?: (args: Record<string, unknown>) => AgentToolGate | Promise<AgentToolGate>;
    handler: (args: Record<string, unknown>) => unknown | Promise<unknown>;
    }

    handler and the live arguments stay frontend-owned; only the serialized schema and specifierTemplate reach the backend.

Per-call gates​

Some tools can't be gated from their arguments. browser.click{ref: 12} is a link or a form's submit button depending on the page, and only the page knows. Such a tool declares gate(args), which returns {sideEffect, specifier?, rule?} for this call:

  1. The manifest marks the tool gated: true.
  2. Before relaying a call, the backend gate (_gate in orchestrator.py) sends a tool_call named __gate__ with {tool, args}. The relay answers it from the tool's gate (gateDynamicTool in manifest.ts), without running the tool.
  3. The answer replaces sideEffect and the rendered specifier. rule is the always-allow rule the approval card offers, which may be broader than the call (browser.click(submit:github.com *)).
  4. A harmless answer (sideEffect: false) still meets explicit deny and ask rules, so a user can tighten what a tool calls harmless.

The probe fails closed. If it errors, times out (15s) or answers something malformed, the tool's static fields apply, so a gated tool keeps sideEffect: true. A simulated run (fixtures) and a remote turn skip the probe. The browser's use of it is in the browser module.

Giving the agent a new tool (recipe)​

There are three ways a tool reaches the model. Pick by what the tool is, not by where it's convenient to write:

If the tool is…Declare it as…Lives inGated?
An app-level layout/navigation verb (open/split a pane)a static entry in LAYOUT_TOOLSbackend/modules/agent/orchestrator.pyno (layout-only)
An app-wide command the user could also run from the paletteCommandDecl.agent on the commandthe module's manifest (frontend)optional (sideEffect)
A widget/panel-specific action tied to a pane instancean AgentToolDecl in the decl's agentToolsthe widget/panel decl (frontend)optional (sideEffect)

Only LAYOUT_TOOLS is authored on the backend; it is the one catalog the backend owns because layout verbs have no frontend handler to serialize — they execute directly in tool-exec.ts's executeTool switch. Everything else is frontend-owned and pushed to the backend in the capability manifest, so a runtime-loaded plugin's tools appear with no backend change.

Steps for a frontend tool (command or agentTools)​

  1. Declare it. Add agent: { description, params?, sideEffect? } to a CommandDecl, or push an AgentToolDecl onto a WidgetDecl/PanelDecl's agentTools. The description is the model's only guidance — write it like an API doc, name the args, and say when not to call it (see the editor's proposeEdit-vs-applyEdit note in SYSTEM_PROMPT).
  2. Write a precise params JSON-schema. Local models follow enums and required far better than prose. Keep ids out of enums — the model discovers live ids with the read tools, so the catalog stays frontend-owned.
  3. Mark side effects. A tool that mutates anything sets sideEffect: true and passes through the gate; a read-only tool omits it and is relayed without a prompt. For a gated tool, add a specifierTemplate (e.g. "{command}") so the engine can match scoped rules like terminal.exec(npm run *).
  4. Implement the handler (frontend) — or the executeTool case (layout verb). The handler runs in the browser and may call backend HTTP/WS APIs; it never crosses the wire. Unknown relayed names fall through executeTool's default to executeDynamicTool, which dispatches to the registered handler.
  5. It's already wired. initAgentManifestSync pushes the manifest on connect, reconnect, and every registry change, and the backend merges it into the model's tool list each turn (_tools_for). No registration step beyond the decl.
  6. Verify. Open the chat widget and run /tools — the live catalog (exactly what _tools_for feeds the model) should list your tool with the right source. Then ask the agent to use it and watch the per-turn action log.

Backend layout verbs​

A new LAYOUT_TOOLS entry needs three edits kept in lockstep: the _tool(...) schema in orchestrator.py, a case in executeTool (tool-exec.ts), and the verb list in tool-names.ts that feeds "did you mean". The case calls a controller export (layout/controller.ts) rather than dispatching itself, so the agent, the keyboard and the pointer share one rule. Argument validation belongs at the tool-exec.ts boundary, not in the controller: an unknown place_pane target or arrange style is refused there with the valid options, so the model can correct itself in one round.

Reading widget state: getAgentContext()​

State the agent needs to read — the active buffer's text, the tree selection, a terminal's recent output — lives in a live pane instance, not in a static declaration. It is read on a pull basis (not a push bus):

  • A pane component registers a snapshot provider via the useAgentContext(() => snapshot) hook (packages/core/src/agent-context.ts). The provider registry lives in core so feature modules (which live in core) can import the hook without a core→ui cycle; packages/ui's PaneHost supplies the live instance id through PaneInstanceContext and, around a section body it renders itself, the section id through SectionInstanceContext. Providers are keyed by the pair.
  • Two built-in read tools (ungated) expose it: list_open_panes() (the listOpenPanes seam, which now reports each pane's instanceId and a hasContext flag) and get_pane_context(instanceId, section?), which invokes the registered provider and returns its JSON snapshot.

For a pane with sections, the active section id is stamped onto every snapshot by readPaneAgentContext — centrally, so a module can't forget it and hand back a snapshot that is ambiguous about which tab it describes. Passing section reads a different one, which switches the pane to it first: only the visible section is mounted, so there is no way to read a background section without bringing it forward, and pretending otherwise would return the wrong section's data as if it were the one asked for. A pane-level provider and the mounted section's provider merge, section last — see Embedded views and sections.

A section with no provider is reported as hasPayload: false rather than as an empty object. That distinction matters more than it looks: a pane that returns nothing does not make a model answer "I don't know", it makes it answer from whatever else is in the context window.

This is how the agent "reads widget A to act on widget B": pull A's context with a read tool, then call a gated action tool on B. It forces per-pane-instance state (a known windowing gap — instances currently share a store), which these modules need anyway.

Beyond the read/write modules above, the content widgets expose snapshots too, so the agent can reason about what's on screen even where it has no write tools: the backend-status widget reports reachability and version; the observability widget and panel report the data-flow summary (total calls, errors) and the most recent calls; the Clubhouse account widget reports the connection state and the live-rooms panel reports the rooms currently listed. These are read-only getAgentContext providers — no agentTools.

Capability manifest protocol​

The frontend owns the tool catalog and pushes it to the backend so the manifest reflects runtime-loaded plugins and the current registry. New agent-channel events extend the slice-1 protocol:

Directioneventdata
client→servermanifest{tools: SerializedTool[]} — on connect and on registry change
server→clientapproval_request{turnId, callId, tool, specifier, mode, rule}
client→serverapproval_response{callId, decision: 'allow_once'|'allow_always'|'deny', rule?}

SerializedTool = {name, description, params, sideEffect, specifierTemplate, gated?, kind} where kind is 'agentTool' | 'command' | 'layout'. Each turn the backend merges the pushed manifest with the static LAYOUT_TOOLS into the model's tool list. The handler is never serialized; the backend learns which tools to gate from sideEffect and renders the rule string from specifierTemplate + the call args before consulting the engine.

The permission system​

A direct port of Claude Code's model (backend/modules/agent/permissions.py), scoped to this app's tools.

Rules​

Three lists — allow, ask, deny — of Tool or Tool(specifier) strings, with * glob support. Precedence is deny → ask → allow; first match wins, and specificity does not reorder. A matching ask prompts even when a more specific allow also matches. A bare tool name (terminal.exec) matches all uses; a scoped rule (terminal.exec(rm -rf *)) matches only matching calls.

Modes​

A defaultMode setting (overridable per session), mirroring Claude Code:

ModeBehavior
defaultPrompt (ask) on every side-effecting tool not already covered by an allow rule.
planRead-only: every sideEffect tool is auto-denied; the agent can explore but not mutate.
acceptEditsAuto-allow editor saves and safe FS creation (files.create, mkdir-equivalents); still ask on files.delete/files.rename/terminal.exec.
autonomousAllow everything except explicit ask/deny rules — the bypassPermissions analog.

The chat pane exposes plan / default / autonomous as Plan / Agent / Auto, chosen per chat and sent with each ask; in plan the side-effecting tools and the DESKTOP_TOOLS verbs are also left out of the offered set, not just denied. See activity timeline & modes.

Circuit breakers apply even in autonomous: destructive roots (files.delete at a workspace root, terminal.exec(rm -rf /*) and equivalents) always prompt, as a guard against model error.

Terminal & filesystem specifiers​

The terminal carries most of the engine's complexity, because matching a shell command safely is the hard part. Implemented in backend/modules/agent/shell.py (tools listed in permissions.SHELL_TOOLS), porting Claude Code's Bash/PowerShell rule logic:

  • * globs with word-boundary semantics (terminal.exec(npm run *) matches npm run build, not npm runner);
  • compound-command splitting on && || ; | & — for an allow every subcommand must be covered; a deny/ask fires if any subcommand matches (so a dangerous part can't hide behind a safe one);
  • wrapper stripping (sudo, timeout N, nice -n N, env VAR=…, …) before matching;
  • a built-in read-only command allowlist (ls, cat, pwd, …) that runs without a prompt in every mode (a command with an output redirect is not treated as read-only);
  • shell circuit breakers (mkfs, dd of=/dev/…, fork bombs, rm -rf of a root) that always prompt, even in autonomous.

Filesystem rules use gitignore-style path anchors against the workspace roots, matching Claude Code's Read/Edit patterns (//abs, ~/home, /projectRoot, ./cwd), so a rule like deny files.delete(**) or allow files.write(/src/**) behaves as expected. See file-explorer.mdx.

Where rules live​

Rules and the default mode are persisted server-side in the existing settings store ($HORRIBLE_DATA_DIR) under agent.permissions.{mode,allow,ask,deny}, read through permission_store and surfaced in the settings module. Precedence, lightest to heaviest: shipped defaults → user settings → session override. "Always allow" from an approval prompt appends a rule to agent.permissions.allow.

Out of scope for v1​

OS-level sandboxing (Claude Code's complementary enforcement layer) is not part of v1 — permissions are the only boundary. As the terminal page notes, exposing the backend beyond localhost exposes shell execution on that host; that is an auth problem at the backend boundary, not something the permission engine solves.

Approval UI​

approval_request renders a global surface (packages/ui/ApprovalPrompts.tsx, fed by the useApprovals store in packages/core/.../agent/approvals.ts) — and, on desktop, an OS notification via notifications.system — showing the tool, the rendered specifier, and Allow once / Always allow / Deny. "Always allow" sends an approval_response carrying an editable rule that the backend persists. It is prefilled with the request's rule: the exact call, or the broader rule a per-call gate suggested, and can be edited to e.g. terminal.exec(npm run *).

The mode and rule lists are editable on the settings page: the agent module contributes a settingsSections entry (PermissionsSettings) — a registry seam for configuration too rich for the declarative SettingDecl controls — that reads/writes agent.permissions.{mode,allow,ask,deny} through /api/settings.

Build order​

  1. Extend the SDK decl types (CommandDecl.agent, AgentToolDecl, getAgentContext host hook) and the manifest protocol. ✅
  2. Build the permission engine and the approval UI. ✅
  3. Validate end to end against a stub module — one read tool, one gated tool — exercising every mode before any real module exists. ✅
  4. Layer the file explorer, editor, and terminal onto the surface. ✅ — all three ship with their agentTools/getAgentContext on the gate. Incremental per-module follow-ups remain (e.g. the editor's desktop osfile: source, live files.* watch events), but the foundation is fully in use.

Validation stub (retired)​

A dev-only stub module once exercised the whole surface — a read tool, a gated tool with a specifierTemplate, and a getAgentContext snapshot — and validated all four permission modes live. It was removed once every real module carried agentTools of its own: it appeared in the dev build's launcher as "Agent stub", which is a pane nobody wanted, and any shipped module is now the shape to copy.

load_tools reads whatever the model spelled​

An array parameter comes back from small models as a string more often than not. Measured against llama-3.2-3b: {"groups": "[\"github\"]"} and {"groups": "['files', 'editor']"}, the second not valid JSON at all.

The dispatcher used to do if isinstance(requested, str): requested = [requested], which turned '["github"]' into a group named ["github"], matched nothing, and answered loaded: []. The model had asked for exactly the right group; the capability simply never arrived, and it read as a model that could not work progressive disclosure.

_coerce_group_list accepts a real list, a JSON array in a string, a Python-style list, a bracketed comma list, or a bare name — and deliberately does not run a bare name through a list parser, because github is a group name and must stay one.

_coerce_args handles the same class of problem one level up, for the arguments payload as a whole. This one slipped past it because the payload was valid JSON; it just had a string where the schema said array.