The model graph: a design, its shapes, and its PyTorch
The interpretability pane's Design mode edits a
DesignGraph and generates a nn.Module subclass from it. This page is the contract
between the two: what the IR holds, what the generator promises, and the rules that
are silent failures if they are broken.
Backend: backend/modules/interpretability/graph/. Frontend:
packages/core/src/modules/interpretability/designer/.
Three things it is not
- Not
ModelArchitecture(../modules/interpretability.mdx#model-explorer), which is a description normalised out of GGUF metadata. - Not
ModelGraph(packages/core/src/modules/training/client.ts), which is atorch.fxtrace of a module that ran. - Not a runtime. The generated file imports nothing from this app. You can move it into a Kaggle notebook or a colleague's repo and it runs.
Those first two both describe a model that exists. This one describes one that does not yet — which is why it is a separate type with a separate name.
The IR
class GraphNode: id, type, params, muted, name
class GraphEdge: source, sourceHandle, target, targetHandle
class SubGraph: id, name, nodes, edges # one generated nn.Module subclass
class DesignGraph: name, config, nodes, edges, groups
Two rules the shape enforces:
No cosmetics in the IR. Positions, frames, collapse state and labels live in a
separate layout sidecar (<name>.graph.json), because a .py file has nowhere sane
to put an x/y coordinate. That separation is what lets the generated source be the
structural source of truth without a round trip scattering your canvas.
A param is a literal or a reference, never ambiguously both. "$d_model" reads
DesignGraph.config; anything else is the value itself. Config entries become the
generated class's __init__ keyword arguments, so one graph describes a family of
models rather than a single frozen one — RMSNorm(d_model), not RMSNorm(512).
What each node knows
One entry per type in spec.py, carrying everything about it: sockets, editable
params (in the RecipeField shape the training pane already renders), a shape_fn, a
params_fn, and the two lines of Python it becomes. Adding a node type is adding an
entry there and nothing else — the palette, the inspector and the generator all read
it, and the catalog is served (GET /api/interpretability/graph/specs) rather than
restated in TypeScript, the same reason hassault serves its weapon numbers.
Notable members of the vocabulary:
op.addis the residual connection. Not a flag on a block, not an implicit convention: a wire you can see, reroute, mute, or get wrong.attn.mhais MHA, GQA and MQA, distinguished by the head counts exactly asAttentionSpec.kindderives them from a GGUF's metadata.groupwithcount > 1is the ×N stack — the direct descendant of the model explorer's "drawn once with a multiplier".custom.moduleis the escape hatch, and where the round-trip parser puts source it cannot map: preserved verbatim, still emitted into the generated file so it keeps running, and named to the reader — never dropped.
Codegen
A topological walk in SSA form. Three structures do the work: a group emits a class, a
repeated group emits a ModuleList and a loop, and config values stay variables.
Primitives torch.nn does not ship (RMSNorm, MultiHeadAttention, SwiGLU, MoE,
RotaryEmbedding) are emitted into the file, only the ones used, in
dependency-safe order.
class DecoderBlock(nn.Module):
def __init__(self, d_model: int = 512, n_heads: int = 8, n_kv_heads: int = 2, ...):
super().__init__()
self.norm_1 = RMSNorm(d_model, eps=1e-06) # horrible:node=blk_n1
...
def forward(self, x):
x_1 = self.norm_1(x) # horrible:node=blk_n1
x_2 = self.attn_1(x_1) # horrible:node=blk_attn
x_3 = x + x_2 # horrible:node=blk_res1 ← the residual
Every emitted line carries # horrible:node=<id> on the line ast reports for the
statement. Re-parsing the file recovers node identity, and with it the layout sidecar.
Without it, a code edit would generate fresh ids and re-lay-out the canvas on every
save — the single most annoying thing a node editor can do.
Mute is ablation. A muted node emits nothing, its input passes through, and it
stops being counted — and it stops dragging its primitive class into the file. A
generated module that still defines SwiGLU after you ablated the only SwiGLU makes
the ablation look like it did not take.
Determinism is required, not incidental. The same graph must produce the same
bytes: golden fixtures and round-tripping both depend on it, so topo_order is Kahn's
algorithm in declaration order rather than anything hash-ordered.
Two tiers of validation, and why they stay separate
Tier 1 — shapes.py | Tier 2 — a probe | |
|---|---|---|
| Runs | in the backend, per keystroke | in a training project's venv |
| Needs torch | no | yes |
| Answers | our symbolic arithmetic | what torch actually does |
| Shown as | estimated | measured |
The backend has no torch and must not grow one;
heavy dependencies live in per-project uv envs. So shape inference is pure Python, and
batch and sequence stay symbolic (B, T) — a symbolic dimension compared against a
concrete one is an unknown, not a conflict, and reporting unknowns as errors would
paint every honest graph red.
Tier 2 (probe.py) follows recipes._probe exactly, subprocess.run on the caller's
thread included (asyncio subprocess spawn is broken under uvicorn --reload on
Windows). It writes the generated module to a temp file, imports it under the project's
python, instantiates it, and runs a (2, 8) batch of token ids through it — two rows so
a bug that only appears with a batch dimension cannot hide.
It reports three states, never two: it ran / it raised, with the traceback verbatim / we could not ask. When the two tiers disagree, tier 1 is wrong and the disagreement is surfaced rather than reconciled — the estimate is what the cost overlay shows on every keystroke, so a silent divergence means the pane has been lying.
The probe script is self-contained and does not import horrible_train. A project's
venv is not guaranteed to have it (an evals-owned project carries only its benchmark's
requirements), and a probe failing on a technicality about our own helper package would
be read as the model being broken.
A node failing inside a group must not also make the group claim its output is
unwired. It is wired; the chain feeding it broke. Two explanations, one of them false,
sends the reader after the wrong one — so _group distinguishes "nothing is connected"
from "the upstream value never arrived" and stays quiet about the second.
Groups
A group is one generated nn.Module subclass, which is the whole reason Blender's
metaphor fits a neural network. It lives in DesignGraph.groups (flat, addressed by
id — nesting is a matter of who instantiates whom, not of where the record is stored),
and the canvas edits one level at a time: the root, or the inside of one group. The
list of group ids you have stepped into is the context path, Blender's breadcrumb,
and every rule about it is a pure function in
packages/core/src/modules/interpretability/designer/scope.ts.
Four decisions the implementation is built on:
- A path segment naming a group that no longer exists truncates. Deleting a group you are standing in drops you to the level above; resolving to nothing would leave a canvas that shows nothing and silently swallows every edit.
- A group has exactly one input and one output, because a
forward(self, x)that takes one tensor and returns one is what the ×N stack is built on. So folding a selection into a group (Blender's Ctrl-G) is refused, with the count, when the selection would need two of either — wiring the second one to whichever socket came first is precisely the silent reinterpretation this pane exists to refuse. Widening the selection is the fix, and saying so is more use than a generic complaint. - Two groups compiling to the same class name are refused at codegen.
class_namestrips punctuation, soBlock 1andBlock-1are one class, and a second definition of it silently wins — every instance of the first would then run the second's code, with no error anywhere. The same check covers a group named after the model itself. - A group nothing instantiates is still inferred, so its wires carry shapes while you build it, and its errors are still reported — codegen emits every group, so a broken one really does break the file. Its parameter counts are reported per node but kept out of the total: a class the model never instantiates holds no weights.
Identity, in two directions
Codegen emits two maps, and they answer different questions about the same file:
markers— line → node. How re-parsing recovers which node a line came from, so your canvas layout survives a round trip.attrs/attrClasses— class →self.<attr>→ node, and which class a group attribute holds. How a runtime module path finds its way back to a box: a measureddecoderblocks_1.0.norm_1is resolved by walking those tables fromrootClass.
Both are built as the code is emitted, never recovered afterwards. A second pass that re-derives attribute names would have to reimplement the naming rules, and the day the two disagree nothing says so — a measured parameter count simply lands on the wrong node.
Two rules make the per-node arithmetic come out right. The deepest mapped node
wins: attn_1.q_proj is inside a primitive emitted whole, so its weights belong to
the attention node rather than to some node of their own, and once the walk enters a
class with no map everything below is charged to the last node it recognised. And
numeric segments are skipped and accumulate: a ×N stack is one node drawn once, so
all N copies of norm_1 add up to that node's cost — exactly what shapes.py does
when it multiplies an inner node by the count.
Agent tools
Six verbs under the model group: list_designs, inspect_design, import_model,
set_config, generate_code, to_training_project. The shape of the set is a
decision rather than an oversight, and the omission is the interesting half.
The obvious design — add_node, connect, delete_node — gives an agent that
cannot see the canvas a way to build a graph one wire at a time, which is both the
slowest way to describe a transformer and the easiest way to produce a disconnected
one. What an agent is genuinely good at here is the parametric question: "what would
this be at half the width", "how much of Llama 3.2 is the embedding", "import what I am
looking at and tell me whether the counts agree". So node surgery stays on the canvas,
where the person doing it can see the wires.
Two rules the handlers follow, both for the same reason a wrong answer here is expensive:
set_configrefuses a key no node reads. Adding one would report success and change the model not at all.- A config that breaks the graph is not written. Saving also rewrites the
.pybeside it, so ann_headsthat no longer divides the KV heads would take the generated module down with it.
Every result labels our arithmetic estimated. The only thing that turns an estimate
into a measurement is probe.py, in a venv that has torch.
What a shape error must carry
A message with no nodeId cannot become a red socket, which is the whole point of
inferring shapes. ShapeIssue carries nodeId/handle, and inference accumulates
issues instead of raising on the first: a graph mid-edit usually has several, and
fixing them one round-trip at a time is worse than seeing them all.
HTTP surface
| Method | Path | Purpose |
|---|---|---|
| GET | /api/interpretability/graph/specs | the node catalog + templates |
| POST | /api/interpretability/graph/code | graph → Python (stateless) |
| POST | /api/interpretability/graph/validate | tier-1 shape inference (stateless) |
| POST | /api/interpretability/graph/parse | source → graph (stateless; an unreadable file returns error, not a 400) |
| POST | /api/interpretability/graph/probe | tier-2: build and run it in a project venv |
| GET | /api/interpretability/graph | saved designs |
| GET/PUT/DELETE | /api/interpretability/graph/{name} | one design |
| GET | /api/interpretability/graph/template/{id} | a starting graph |
| POST | /api/interpretability/graph/from-model | the inspected model → a design (stateless) |
| POST | /api/interpretability/graph/from-traced | a real nn.Module, traced by torch.fx in a project venv |
| POST | /api/interpretability/graph/handoff | write a design into a training project as model.py + notebook cells |
/from-model re-reads the architecture through the same function the inspect route
serves rather than accepting one the caller posts. The two tabs must never be able to
disagree about what the model is, and a caller that can supply the architecture is a
caller that can supply a different one.
/code and /validate are stateless on purpose: the canvas calls them on every
edit, long before a design has a name, and they must not touch the disk.
A save that cannot generate code is still written, with the reason in codeError.
Refusing the write would lose an afternoon over one unfinished wire, and a canvas is
unfinished nearly all the time.
Reading the file back (parse.py)
ast walks the module: __init__ assignments become nodes, forward statements
become edges, and the __init__ signature's defaults become the model's config — a
Name argument like RMSNorm(d_model) is read back as the reference "$d_model",
never as the 512 it happens to resolve to, because collapsing it would quietly turn a
family of models into one.
The correctness argument is emit(parse(emit(g))) == emit(g), byte for byte, over
every template. That is deliberately a stronger and more useful promise than IR
equality: it is exactly what a user experiences when they edit the code pane and save.
Two details fall out of insisting on it — attribute names are only recorded when they
are not what codegen would have derived (otherwise every generated name gets frozen
as a user override), and the x_n locals must renumber identically, which is why
topo_order is deterministic.
Three shapes need explicit handling because they are one node written as several statements, or several nodes written as one:
| Source | Read as |
|---|---|
x_2 = x_1 followed by for _layer in self.blocks_1: | the loop attaches the stack; the alias above it carries no structure. Treating the alias as the connection leaves the whole block stack unwired |
x + self.pos_1(torch.arange(...)) | one learned-positional node, not an Add of two — it adds itself |
x * 2.0 | a Scale, not a Multiply with a missing input, which would draw a broken wire |
CONSTRUCTORS mirrors init_fn, and mirrors driftinit_fn builds a string and a string cannot be run backwards, so parse.py carries an
explicit constructor table. A node type added to the catalog but missing from that table
does not crash — it silently becomes an opaque custom.module on the way back in. A
test asserts every spec that emits a constructor appears in the table, which is the only
thing standing between that mirror and quiet rot.