Agent tools & permissions
Status: implemented. This page specifies the generalized agent
tool surface of the agent orchestrator (per-widget
agentTools, agent-exposed commands, pull-based getAgentContext()) and the
permission system that gates side effects. It is the shared foundation the
file explorer, editor,
and terminal modules are built on; the surface was built
first, validated against a stub, then those modules were layered onto it. Slice 1
(the static LAYOUT_TOOLS catalog) is in agent-chat.mdx.
Landed: the SDK declaration contract (AgentToolDecl, CommandDecl.agent,
getAgentContext/AgentContextSnapshot); the capability-manifest protocol
(below); the per-instance getAgentContext registry with the
list_open_panes / get_pane_context read tools; the permission engine
core (backend/modules/agent/permissions.py — rule parsing, deny → ask → allow precedence, the four modes, circuit breakers) including shell-aware
specifier matching (shell.py); the gate + approval round-trip in the
orchestrator (_gate, approval_request/approval_response, rule persistence via
permission_store); and the approval UI (ApprovalPrompts + useApprovals)
with the settings-page permissions section (mode picker + rule lists). The
whole permission system (Epic A) is in place, the stub validation ran green
across all four modes, and all three real modules now declare their tools on the
surface: the file explorer (files.list/read reads, gated
files.create/write/rename/delete), the editor (getAgentContext
buffer reads, gated editor.proposeEdit [accept/decline diff] /
editor.applyEdit/save), and the terminal
(terminal.list/read reads, gated shell-matched terminal.exec). Content widgets
add their own: observability (getAgentContext I/O snapshot + gated
observability.clear) and clubhouse (clubhouse.status/listRooms reads, gated
clubhouse.disconnect) — illustrating that any widget gets agent tools just by
declaring agentTools on its decl. The chat widget's /tools command lists the
full catalog via a list_tools→tools WS round-trip, labeled by group. The model
itself doesn't see the whole catalog at once: tools are grouped by prefix and
disclosed progressively — a small core plus list_tool_groups/load_tools
meta-tools, with each turn's list recomputed and groups injected on demand (see
Hierarchical tools & progressive disclosure).
Remaining work is incremental per-module polish (e.g. the editor's desktop-only
osfile: source), not foundational.
A tool's group is its name prefix
_group_of(name) takes the namespace before the first dot — github.searchCode
is in group github; a dotless name (the layout verbs) is in the always-present
layout core.
AgentTool.group does not name the group. It's only a flag: provider_tools(grouped=…)
checks whether it is set, to split always-core tools from progressively-disclosed
ones. So a tool named game.getObservation that declares group="games" really lands
in a group called game — and if nothing describes game, it shows up in the catalog
as a bare "game tools" fallback with no blurb and no guide. That exact mismatch
existed for the games tools until the connectors work added an invariant test
(test_connectors.py::test_agent_tool_names_match_their_declared_group).
Keep the name prefix and the declared group identical. For a
connector, both must also equal the connector's id —
that's what lets one definition feed the home tile, the group blurb, and the guide.
group= is worse than getting it wrongA mismatched group costs a blurb. An omitted one costs a slot on every turn of
every agent: provider_tools(grouped=False) treats a tool with group=None as
always-on core, so its schema is charged to each round whether or not the turn has
anything to do with it — and the _GROUP_DESCRIPTIONS entry naming it can never fire,
because _group_catalog only walks the dynamic pool.
That is exactly what research.* (5 tools) and arxiv.* (3) did: app.py claimed
they were "grouped under research/arxiv", _GROUP_DESCRIPTIONS had entries for both,
and researcher's preload_groups=[…, "research", "arxiv"] was partly a no-op —
while all 8 sat permanently in the main orchestrator's core. Fixing it took the main
agent's always-on core from 34 tools to 26, nearly doubling the room left under
TOOL_BUDGET for capability groups.
test_connectors.py::test_every_namespaced_agent_tool_declares_a_group now fails on a
dotted tool with no group=, alongside the older mismatch test.
Two tiers of disclosure
list_tool_groups shows each group's one-line blurb (_GROUP_DESCRIPTIONS, falling
back to a connector's blurb); load_tools delivers the group's full guide — a
SKILL.md-style markdown doc. The blurb answers "would this help?"; the guide answers
"how do I use it without getting it wrong?". Guides ride the tool-result message, so
they need no orchestrator plumbing of their own.
Any group can ship a guide, not just connectors. _group_guide resolves
backend/modules/agent/guides/<group>.md first (agent/guides.py, lru_cache'd),
then a connector's resolve_guide(), then the MCP bridge. For a long time only the
latter two existed, so every built-in group — layout, files, editor,
database, games — had a one-line blurb and nothing else; detail that belonged in a
guide had nowhere to go but the system prompt, where it was charged to every round of
every turn.
Guides are plain markdown so they review and diff like code. Put anything in one that a model gets wrong repeatedly and that would be too expensive to say on every turn: which argument is an instanceId rather than a view id, which combinations are useless, what the module calls a thing.
show: the intent verb
Reaching a surface used to cost three rounds — list_available_panes (a row per
registered view), open_pane, then get_pane_context. Three chances to emit
malformed JSON and a large tool result, for a model whose whole window may be 8k.
show(target, where?) collapses that to one call. target is loose text — a pane
title, a view id, a section label, a side-strip name, a workspace name — and the
host does the matching (layout/show.ts, pure and unit-tested; showTarget in
layout/controller.ts acts on the result). It opens or focuses (never a second copy of
something already visible), switches to the named section, reveals a region inside its
host, or switches workspace, and returns the resulting pane's context snapshot inline,
so no get_pane_context follow-up is needed.
Because panes are being merged rather than deleted, show resolves a section
label directly: the user says "friends", not "the People pane, Friends tab", and so
should the model. An embedded view resolves to where it lives — its host pane and
section — because it has no standalone home to open.
Its argument is deliberately one string, not a discriminated union: _coerce_args
turns an unparseable payload into an error, and a wide schema is more of them. The way
to shrink a tool list is to move tools out of the always-on core, not to widen the ones
that remain.
VIEW_ALIASES (in layout/controller.ts) maps retired ids and titles to where their
content lives now. It is deliberately separate from serialize.ts's
RENAMED_VIEWS, which repairs saved layouts: a stored arrangement is disposable and
reseeds from its preset, but a name the agent has ever used should never stop
resolving. Add an entry whenever a pane is merged away.
The model: one tool path, one gate
Every frontend-declared agent tool call flows through the frontend relay over
the shared /ws agent channel — there is no relay bypass, even for tools whose
resource (a PTY, the filesystem) lives in the backend. Those tools' handlers run in
the frontend and call the backend's existing HTTP/WS API, so the agent has exactly
one tool model to reason about.
Tools contributed via backend/sdk (host.add_agent_tool) run their handler in the
backend and never touch the relay — used where a tab may not exist at all (a cron run,
the dash REPL, agent.ask_peer) or where the resource is a server-held secret the
browser must never see. Today: games, training, and the
connectors. They pass through the same permission gate,
so the sentence that matters is unaffected: one gate, not one path.
The consequence that shapes everything: because all calls pass through the orchestrator loop, the permission gate sits in the backend orchestrator, before the relay. Permissions are enforced by the orchestrator, not by the model — the system prompt shapes what the agent tries, but the engine decides what runs. This mirrors Claude Code, where rules are enforced by the harness rather than the LLM.
A read-only tool never reaches the engine — the same read-only tier Claude Code
uses for file reads and grep.
Declaring agent tools
Two surfaces, by design (the "hybrid" decision in [[agent-orchestrator-architecture]]):
-
App-wide verbs are commands.
CommandDeclgains an optionalagentfield so a command opts into being model-callable:export interface CommandDecl {id: string;title: string;run: () => void | Promise<void>;agent?: { description: string; params?: JSONSchema; sideEffect?: boolean };}Layout/navigation verbs (open pane, switch workspace) are exposed this way and are read-only or harmless, so they stay ungated.
-
Widget/panel-specific actions are
agentTools.WidgetDeclandPanelDeclgain an optionalagentTools?: AgentToolDecl[](MCP-style):export interface AgentToolDecl {name: string; // namespaced like a command id, e.g. "terminal.exec"description: string; // natural language, for the modelparams?: JSONSchema; // JSON-schema object for the argumentssideEffect?: boolean; // falsy = read-only, never gated/*** Template rendered into the permission *specifier* the engine matches* rule specifiers against. `{name}` placeholders are filled from args; the* tool name is implicit. e.g. "{command}" → specifier `npm run build`,* matched by rule `terminal.exec(npm run *)`. Declarative so the backend* builds the specifier itself, keeping the gate fully server-side.*/specifierTemplate?: string;/** Decide the gate per call; see "Per-call gates" below. */gate?: (args: Record<string, unknown>) => AgentToolGate | Promise<AgentToolGate>;handler: (args: Record<string, unknown>) => unknown | Promise<unknown>;}handlerand the live arguments stay frontend-owned; only the serialized schema andspecifierTemplatereach the backend.
Per-call gates
Some tools can't be gated from their arguments. browser.click{ref: 12} is a
link or a form's submit button depending on the page, and only the page knows.
Such a tool declares gate(args), which returns
{sideEffect, specifier?, rule?} for this call:
- The manifest marks the tool
gated: true. - Before relaying a call, the backend gate (
_gateinorchestrator.py) sends atool_callnamed__gate__with{tool, args}. The relay answers it from the tool'sgate(gateDynamicToolinmanifest.ts), without running the tool. - The answer replaces
sideEffectand the rendered specifier.ruleis the always-allow rule the approval card offers, which may be broader than the call (browser.click(submit:github.com *)). - A harmless answer (
sideEffect: false) still meets explicitdenyandaskrules, so a user can tighten what a tool calls harmless.
The probe fails closed. If it errors, times out (15s) or answers something
malformed, the tool's static fields apply, so a gated tool keeps
sideEffect: true. A simulated run (fixtures) and a remote turn skip the probe.
The browser's use of it is in
the browser module.
Giving the agent a new tool (recipe)
There are three ways a tool reaches the model. Pick by what the tool is, not by where it's convenient to write:
| If the tool is… | Declare it as… | Lives in | Gated? |
|---|---|---|---|
| An app-level layout/navigation verb (open/split a pane) | a static entry in LAYOUT_TOOLS | backend/modules/agent/orchestrator.py | no (layout-only) |
| An app-wide command the user could also run from the palette | CommandDecl.agent on the command | the module's manifest (frontend) | optional (sideEffect) |
| A widget/panel-specific action tied to a pane instance | an AgentToolDecl in the decl's agentTools | the widget/panel decl (frontend) | optional (sideEffect) |
Only LAYOUT_TOOLS is authored on the backend; it is the one catalog the backend
owns because layout verbs have no frontend handler to serialize — they execute
directly in tool-exec.ts's
executeTool switch. Everything else is frontend-owned and pushed to the
backend in the capability manifest, so a runtime-loaded plugin's tools appear with
no backend change.
Steps for a frontend tool (command or agentTools)
- Declare it. Add
agent: { description, params?, sideEffect? }to aCommandDecl, or push anAgentToolDeclonto aWidgetDecl/PanelDecl'sagentTools. Thedescriptionis the model's only guidance — write it like an API doc, name the args, and say when not to call it (see the editor'sproposeEdit-vs-applyEditnote inSYSTEM_PROMPT). - Write a precise
paramsJSON-schema. Local models followenums andrequiredfar better than prose. Keep ids out of enums — the model discovers live ids with the read tools, so the catalog stays frontend-owned. - Mark side effects. A tool that mutates anything sets
sideEffect: trueand passes through the gate; a read-only tool omits it and is relayed without a prompt. For a gated tool, add aspecifierTemplate(e.g."{command}") so the engine can match scoped rules liketerminal.exec(npm run *). - Implement the
handler(frontend) — or theexecuteToolcase (layout verb). The handler runs in the browser and may call backend HTTP/WS APIs; it never crosses the wire. Unknown relayed names fall throughexecuteTool'sdefaulttoexecuteDynamicTool, which dispatches to the registered handler. - It's already wired.
initAgentManifestSyncpushes the manifest on connect, reconnect, and every registry change, and the backend merges it into the model's tool list each turn (_tools_for). No registration step beyond the decl. - Verify. Open the chat widget and run
/tools— the live catalog (exactly what_tools_forfeeds the model) should list your tool with the rightsource. Then ask the agent to use it and watch the per-turn action log.
Backend layout verbs
A new LAYOUT_TOOLS entry needs three edits kept in lockstep: the _tool(...)
schema in orchestrator.py, a case in executeTool (tool-exec.ts), and the
verb list in tool-names.ts that feeds "did you mean". The case calls a
controller export (layout/controller.ts) rather than dispatching itself, so the
agent, the keyboard and the pointer share one rule. Argument validation belongs
at the tool-exec.ts boundary, not in the controller: an unknown place_pane
target or arrange style is refused there with the valid options, so the model can correct
itself in one round.
Reading widget state: getAgentContext()
State the agent needs to read — the active buffer's text, the tree selection, a terminal's recent output — lives in a live pane instance, not in a static declaration. It is read on a pull basis (not a push bus):
- A pane component registers a snapshot provider via the
useAgentContext(() => snapshot)hook (packages/core/src/agent-context.ts). The provider registry lives in core so feature modules (which live in core) can import the hook without a core→ui cycle;packages/ui'sPaneHostsupplies the live instance id throughPaneInstanceContextand, around a section body it renders itself, the section id throughSectionInstanceContext. Providers are keyed by the pair. - Two built-in read tools (ungated) expose it:
list_open_panes()(thelistOpenPanesseam, which now reports each pane'sinstanceIdand ahasContextflag) andget_pane_context(instanceId, section?), which invokes the registered provider and returns its JSON snapshot.
For a pane with sections, the active section id is stamped onto every snapshot by
readPaneAgentContext — centrally, so a module can't forget it and hand back a
snapshot that is ambiguous about which tab it describes. Passing section reads a
different one, which switches the pane to it first: only the visible section is
mounted, so there is no way to read a background section without bringing it forward,
and pretending otherwise would return the wrong section's data as if it were the one
asked for. A pane-level provider and the mounted section's provider merge,
section last — see
Embedded views and sections.
A section with no provider is reported as hasPayload: false rather than as an
empty object. That distinction matters more than it looks: a pane that returns
nothing does not make a model answer "I don't know", it makes it answer from
whatever else is in the context window.
This is how the agent "reads widget A to act on widget B": pull A's context with a read tool, then call a gated action tool on B. It forces per-pane-instance state (a known windowing gap — instances currently share a store), which these modules need anyway.
Beyond the read/write modules above, the content widgets expose snapshots too,
so the agent can reason about what's on screen even where it has no write tools:
the backend-status widget reports reachability and version; the
observability widget and panel report the data-flow summary (total calls,
errors) and the most recent calls; the Clubhouse account widget reports the
connection state and the live-rooms panel reports the rooms currently listed.
These are read-only getAgentContext providers — no agentTools.
Capability manifest protocol
The frontend owns the tool catalog and pushes it to the backend so the
manifest reflects runtime-loaded plugins and the current registry. New
agent-channel events extend the
slice-1 protocol:
| Direction | event | data |
|---|---|---|
| client→server | manifest | {tools: SerializedTool[]} — on connect and on registry change |
| server→client | approval_request | {turnId, callId, tool, specifier, mode, rule} |
| client→server | approval_response | {callId, decision: 'allow_once'|'allow_always'|'deny', rule?} |
SerializedTool = {name, description, params, sideEffect, specifierTemplate, gated?, kind}
where kind is 'agentTool' | 'command' | 'layout'. Each turn the backend merges
the pushed manifest with the static LAYOUT_TOOLS into the model's tool list. The
handler is never serialized; the backend learns which tools to gate from
sideEffect and renders the rule string from specifierTemplate + the call args
before consulting the engine.
The permission system
A direct port of Claude Code's model
(backend/modules/agent/permissions.py), scoped to this app's tools.
Rules
Three lists — allow, ask, deny — of Tool or Tool(specifier) strings,
with * glob support. Precedence is deny → ask → allow; first match wins, and
specificity does not reorder. A matching ask prompts even when a more specific
allow also matches. A bare tool name (terminal.exec) matches all uses; a scoped
rule (terminal.exec(rm -rf *)) matches only matching calls.
Modes
A defaultMode setting (overridable per session), mirroring Claude Code:
| Mode | Behavior |
|---|---|
default | Prompt (ask) on every side-effecting tool not already covered by an allow rule. |
plan | Read-only: every sideEffect tool is auto-denied; the agent can explore but not mutate. |
acceptEdits | Auto-allow editor saves and safe FS creation (files.create, mkdir-equivalents); still ask on files.delete/files.rename/terminal.exec. |
autonomous | Allow everything except explicit ask/deny rules — the bypassPermissions analog. |
The chat pane exposes plan / default / autonomous as Plan / Agent / Auto,
chosen per chat and sent with each ask; in plan the side-effecting tools and
the DESKTOP_TOOLS verbs are also left out of the offered set, not just denied.
See activity timeline & modes.
Circuit breakers apply even in autonomous: destructive roots
(files.delete at a workspace root, terminal.exec(rm -rf /*) and equivalents)
always prompt, as a guard against model error.
Terminal & filesystem specifiers
The terminal carries most of the engine's complexity, because matching a shell
command safely is the hard part. Implemented in backend/modules/agent/shell.py
(tools listed in permissions.SHELL_TOOLS), porting Claude Code's Bash/PowerShell
rule logic:
*globs with word-boundary semantics (terminal.exec(npm run *)matchesnpm run build, notnpm runner);- compound-command splitting on
&& || ; | &— for an allow every subcommand must be covered; a deny/ask fires if any subcommand matches (so a dangerous part can't hide behind a safe one); - wrapper stripping (
sudo,timeout N,nice -n N,env VAR=…, …) before matching; - a built-in read-only command allowlist (
ls,cat,pwd, …) that runs without a prompt in every mode (a command with an output redirect is not treated as read-only); - shell circuit breakers (
mkfs,dd of=/dev/…, fork bombs,rm -rfof a root) that always prompt, even inautonomous.
Filesystem rules use gitignore-style path anchors against the workspace roots,
matching Claude Code's Read/Edit patterns (//abs, ~/home, /projectRoot,
./cwd), so a rule like deny files.delete(**) or allow files.write(/src/**)
behaves as expected. See file-explorer.mdx.
Where rules live
Rules and the default mode are persisted server-side in the existing settings
store ($HORRIBLE_DATA_DIR) under agent.permissions.{mode,allow,ask,deny}, read
through permission_store and surfaced in the
settings module. Precedence, lightest to heaviest:
shipped defaults → user settings → session override. "Always allow" from an
approval prompt appends a rule to agent.permissions.allow.
Out of scope for v1
OS-level sandboxing (Claude Code's complementary enforcement layer) is not part of v1 — permissions are the only boundary. As the terminal page notes, exposing the backend beyond localhost exposes shell execution on that host; that is an auth problem at the backend boundary, not something the permission engine solves.
Approval UI
approval_request renders a global surface (packages/ui/ApprovalPrompts.tsx,
fed by the useApprovals store in packages/core/.../agent/approvals.ts) — and,
on desktop, an OS notification via notifications.system — showing the tool, the
rendered specifier, and Allow once / Always allow / Deny. "Always allow" sends
an approval_response carrying an editable rule that the backend persists. It is
prefilled with the request's rule: the exact call, or the broader rule a per-call
gate suggested, and can be edited to e.g. terminal.exec(npm run *).
The mode and rule lists are editable on the settings page: the agent module
contributes a settingsSections entry (PermissionsSettings) — a registry seam
for configuration too rich for the declarative SettingDecl controls — that
reads/writes agent.permissions.{mode,allow,ask,deny} through /api/settings.
Build order
- Extend the SDK decl types (
CommandDecl.agent,AgentToolDecl,getAgentContexthost hook) and the manifest protocol. ✅ - Build the permission engine and the approval UI. ✅
- Validate end to end against a stub module — one read tool, one gated tool — exercising every mode before any real module exists. ✅
- Layer the file explorer, editor, and terminal onto the surface. ✅ — all three
ship with their
agentTools/getAgentContexton the gate. Incremental per-module follow-ups remain (e.g. the editor's desktoposfile:source, livefiles.*watch events), but the foundation is fully in use.
Validation stub (retired)
A dev-only stub module once exercised the whole surface — a read tool, a gated
tool with a specifierTemplate, and a getAgentContext snapshot — and validated
all four permission modes live. It was removed once every real module carried
agentTools of its own: it appeared in the dev build's launcher as "Agent stub",
which is a pane nobody wanted, and any shipped module is now the shape to copy.
load_tools reads whatever the model spelled
An array parameter comes back from small models as a string more often than not.
Measured against llama-3.2-3b: {"groups": "[\"github\"]"} and
{"groups": "['files', 'editor']"}, the second not valid JSON at all.
The dispatcher used to do if isinstance(requested, str): requested = [requested],
which turned '["github"]' into a group named ["github"], matched nothing, and
answered loaded: []. The model had asked for exactly the right group; the
capability simply never arrived, and it read as a model that could not work
progressive disclosure.
_coerce_group_list accepts a real list, a JSON array in a string, a Python-style
list, a bracketed comma list, or a bare name — and deliberately does not run a
bare name through a list parser, because github is a group name and must stay one.
_coerce_args handles the same class of problem one level up, for the arguments
payload as a whole. This one slipped past it because the payload was valid JSON; it
just had a string where the schema said array.