Skip to main content

Module: notebook

A domain-neutral reactive notebook — JupyterLab-style .ipynb cells plus a marimo-style reactive dataflow mode, with interactive widgets and animations. Where the training module is notebooks bound to ML projects and venvs, the notebook module is a general computational surface: open any .ipynb under a root folder, run it on a shared managed kernel, toggle between reactive (edit or run a cell and every downstream cell re-runs automatically) and classic (linear Jupyter) execution.

Status: Phases 0–3 implemented — the generic notebook (file catalog, create, open, cell execution, markdown rendering), the marimo-style reactive dataflow engine (dependency DAG, cascade re-execution, stale-variable deletion, diagnostics), interactive ipywidgets (the Jupyter comm protocol; core widgets render as native controls), custom anywidget views (_esm/_css front-ends executed in the browser), reattach-resync (a pane that reopens a running kernel re-hydrates its live widgets from a backend comm snapshot), and agent tools (cell CRUD/run/mode) over the shared kernel core. Frontend in packages/core/src/modules/notebook/ (+ the shared kit packages/core/src/notebook/), backend in backend/modules/notebook/ (+ the shared engine backend/notebook_core/, whose reactive.py holds the dataflow analysis and session.py the comm handling). The one remaining Phase 3 tail — an opt-in bridge from widget values into reactive re-runs — is deferred.

Shared kernel core​

The notebook and training modules share one kernel engine so there is no duplicated Jupyter plumbing:

notebook_core is parametrized by a SessionConfig(python_executable, cwd, notebook_abs_path, rel_path, channel) rather than any domain model; each module fills it and overrides the open seams. The kernel spawns via jupyter_client's sync API on daemon threads (event-loop-agnostic — works under uvicorn --reload on Windows), the in-memory nbformat doc is authoritative and flushed atomically to disk, and sessions are process-global (closing a pane leaves the kernel running; reopening reattaches). The frontend counterpart lives in packages/core/src/notebook/ — CellEditor (CodeMirror), OutputRenderer (nbformat MIME + ANSI), the generic SessionStore (useSyncExternalStore), and the channel-parametrized kernelClient.

Notebooks on disk​

Notebooks are plain nbformat v4.5 .ipynb files under the notebook.root setting (default ~/horrible/notebooks); paths are escape-guarded against traversal outside the root. The execution mode is persisted in metadata.horrible.execution_mode (reactive | classic).

Kernel interpreter​

Cells run on a single managed uv venv under the data dir ($HORRIBLE_DATA_DIR/notebook-venv), bootstrapped on first open with ipykernel + ipywidgets (blocking uv via Popen on a thread — the Windows-safe pattern shared with LSP/terminal/training). The notebook.python setting overrides it with an existing interpreter (which must have ipykernel + ipywidgets); notebook.python.version pins the managed venv's Python.

AI libraries by default​

The managed venv also carries the common AI stack — numpy pandas scipy matplotlib scikit-learn tqdm torch transformers datasets accelerate huggingface_hub safetensors sentencepiece — configurable as notebook.python.packages. Two stages, because their costs differ by three orders of magnitude:

  • Kernel packages install blocking on first open, as before: nothing runs without them, and they take seconds.
  • Libraries install in the background (env.start_library_install, started on every kernel open) while the kernel is already usable. The pane polls GET /notebook/env and shows a banner with the live installer line, a Retry on failure (POST /notebook/env/install), and — once, to whoever watched it finish — "restart the kernel if an import still fails". Blocking the kernel on a multi-gigabyte PyTorch download would make first open look hung for minutes.

Three rules that are quiet if wrong:

  • A stamp file records what is installed (notebook-venv/.horrible-packages.json). The venv used to bootstrap once and never be revisited, so a package added to the defaults simply never reached an existing install — the ModuleNotFoundError: transformers this section exists because of. Missing specs are diffed against the stamp on every open; uv skips anything already satisfied, so a mostly-present list costs seconds.
  • PyTorch installs first, from its own index. PyPI's Windows torch wheels are CPU-only, so Windows with an NVIDIA card (per the hardware probe) uses download.pytorch.org/whl/cu128; Linux PyPI wheels already bundle CUDA, and macOS wheels use Metal. The order matters as much as the index: transformers and accelerate only require some torch, and run first they would pull the CPU wheel from PyPI. A torch spec also counts as missing when the index it was installed from is not the one that would be chosen now. notebook.python.torchIndex overrides the choice (pypi, or an index URL).
  • An override interpreter is never installed into. notebook.python points at the user's own Python; its library status is unmanaged, and managing someone else's environment behind their back is not this module's call.

The stamp is written after each stage, so a failure installing transformers leaves torch recorded and a retry does not download it again.

Installing packages​

!pip install regex, %pip install regex and %uv pip install regex all install into the interpreter the kernel is running on, through uv. A cell is handed to ipykernel verbatim, so IPython syntax always ran. It just did the wrong thing:

  • !pip shells out to whichever pip is first on PATH. The kernel inherits the backend's environment, so that is the backend's venv or a system pip, and a uv venv has no pip at all. At best it failed; at worst it installed into a different Python and reported success.
  • %pip runs sys.executable -m pip, which fails with No module named pip for the same reason.

The fix is an IPython extension, backend/notebook_core/kernel_ext/horrible_pip.py, loaded into every kernel by KernelSession (--IPKernelApp.extensions=horrible_pip in the kernelspec, with its directory prepended to the kernel's PYTHONPATH). Training notebooks share that engine, so they get it too. It replaces %pip with uv pip <cmd> --python <sys.executable>, and adds an input transformer that rewrites !pip …, !pip3 …, !python -m pip … and !uv pip … into %pip …, so there is one code path. Any other ! command is still a shell command.

  • uv is resolved by the backend and passed as HORRIBLE_UV, because a packaged app's PATH is not a terminal's. Without uv the magic falls back to python -m pip and says so, which is right for a notebook.python override that has real pip.
  • Only install, uninstall, list, show and freeze pass through. pip flags with a different uv spelling are translated: uninstall -y is dropped (uv never prompts, and rejects the flag) and --no-cache-dir becomes --no-cache.
  • A failed install raises UsageError, so the cell fails and Run all stops.
  • After an install, importlib.invalidate_caches() runs, so a new package imports in the next cell with no restart. An already-imported package keeps its old code in sys.modules; the magic names it and tells you to restart the kernel rather than restarting behind your back.
  • -r requirements.txt resolves against the kernel's cwd, which is the notebook's folder.

The cell language server still underlines ! lines; that is cosmetic.

Execution modes​

  • Classic — run cells manually in any order on a shared namespace (standard Jupyter).
  • Reactive — each code cell is statically analyzed (via stdlib symtable, so scoping, comprehensions, global and closures resolve correctly) for the module-level names it defines and the free names it reads. A dependency DAG (reactive.py) drives automatic re-execution of a cell's transitive dependents in topological order when it is run; deleting or redefining a cell drops the now-undefined names from the kernel namespace so dependents raise NameError instead of reading stale values. Single-definition (multiple_defs) and cycle violations surface as per-cell diagnostics. Toggled per notebook from the pane header (⚡/↳); persisted in .ipynb metadata.horrible.execution_mode. IPython syntax is analyzed, not rejected: once the raw cell fails to parse, !cmd and %magic lines count as pass and name = !cmd still defines name, so !pip install regex above import regex provides regex downstream. A %% cell magic's body is analyzed as Python when it parses (%%time) and is otherwise opaque (%%bash): no defs and no syntax diagnostic. Stripping only after a failed parse means a ! inside a triple-quoted string is never rewritten.

Interactive widgets (ipywidgets)​

Standard ipywidgets code works unchanged — widgets.IntSlider(), Button, Checkbox, Text, Dropdown, etc. The kernel speaks the Jupyter comm protocol: a widget's comm_open carries its initial model state (with a application/vnd.jupyter.widget-view+json display output whose model_id === the comm id), and comm_msg carries incremental state updates. The backend forwards these over the notebook channel; the frontend's per-session WidgetManager tracks model state and WidgetView renders the common core widgets as native controls (a real <input type=range> for a slider, etc.) — no heavy @jupyter-widgets view stack. Changing a control sends a comm_msg back to the kernel, so in-kernel observe/on_click callbacks fire and linked widgets update.

The shell-socket threading rule: the sync jupyter_client shell socket is not thread-safe and the worker thread already reads it, so browser→kernel comm_msgs are queued (comm_q) and sent by the worker thread — drained both when idle and mid-execution (inside _await_reply), so widgets stay live even while a long cell runs. Binary comm buffers are base64-encoded across the wire.

Kernel lifecycle: two rules teardown gets wrong​

The same threading rule that governs comm_msg governs closing a channel, and that half was missing. stop_channels() closes the very sockets the two pump threads (_iopub_loop, _worker_loop) are blocked reading — and it ran on whichever thread served the restart/shutdown event. Closing a zmq socket under a concurrent recv is undefined behaviour in libzmq, which is why the symptoms looked unrelated to each other: an occasional ZMQError: not a socket, an occasional teardown that never returned, and an occasional segfault. Restart was the worst case — it tore the channels down and immediately built new ones with both pumps still live.

So _stop_pumps (notebook_core/session.py) is now the only way in: it sets closing, posts the worker's poison pill, and joins both threads before any channel is touched. Every blocking read a pump makes is capped at POLL_INTERVAL_S, so this returns in a fraction of a second; the join deadline is a deadlock bound, not a schedule. That poll interval is what restart/shutdown latency is made of — hence 0.25s rather than the 1s the sockets were originally read with. If a pump refuses to leave, teardown proceeds anyway — an abandoned kernel process is worse than the risk — but it is logged, because that is the one remaining path that can corrupt a socket. restart and shutdown also take a per-session lock: they both rebuild km/kc, and nothing else orders them.

The second rule is where a blocking kernel call runs. Not asyncio.to_thread: that uses the loop's default executor, which is joined unconditionally by asyncio.run's teardown and by concurrent.futures' interpreter-exit hook, neither with a timeout. A call that never returns therefore stopped being a stuck kernel and became a process that would not exit, with a traceback pointing at loop teardown. It could not be defended against from the caller either — asyncio.wait_for cancels the await, not the thread, so the orphaned worker sits in the executor and the join finds it regardless. Kernel calls go through notebook_core/detach.py instead (run_detached to await one, fire_and_forget for restart/shutdown), which is the same daemon-thread contract on a thread nobody joins. A private ThreadPoolExecutor would not be enough — the interpreter-exit hook joins those too.

shutdown_all shuts every session down in parallel with a deadline, since it runs inside the backend's lifespan shutdown, where blocking blocks the process.

Custom widgets (anywidget)​

Widgets whose model ships its own front-end — an anywidget _esm ES-module string (exporting render({ model, el }), optionally initialize) and optional _css — render via AnyWidgetView. It executes the _esm in the browser (a Blob + dynamic import()) and hands render a Backbone-flavoured model shim backed by the same WidgetManager: model.get/set/save_changes ride the comm (set stages locally, save_changes flushes a comm_msg update), model.on('change:…' | 'change' | 'msg:custom', …) fans kernel updates and custom messages out, and model.send posts a custom message back. anywidget is installed into the managed venv. Same trusted-local posture as HTML output — the _esm is the user's own kernel code, run unsandboxed. Classic third-party widgets that ship AMD view bundles (@jupyter-widgets/…) are not loaded; those unknown models show a compact fallback.

Reattach-resync​

Kernels are process-global, and the frame unmounts inactive tabs — so a pane can reopen a notebook whose kernel already holds live widgets. The backend is the source of truth for comm state (it sees every comm_open and folds each update patch into a per-comm snapshot), so the opened payload carries a comms snapshot and the frontend seeds its WidgetManager from it — no comm_info round-trip to the kernel. Widgets rehydrate to their current values on reattach (and after a full page reload) as long as the kernel is alive; widget outputs from a dead prior kernel can't be resynced and show the fallback.

ws protocol (notebook channel)​

Client → server: open {path}, run_cell {sessionKey, cellId}, run_all {sessionKey}, cells {sessionKey, ops} (insert/edit/delete/move; insert may carry a client-assigned cellId), set_mode {sessionKey, mode}, comm_msg {sessionKey, commId, data} (widget update), interrupt, restart, shutdown. Server → client: opened {sessionKey, path, notebook, kernel, mode}, kernel_status, execution_state {cellId, state, execCount?}, output {cellId, output} (raw nbformat dict), output_updated {cellId, index, output} (a display_id output rewritten in place — see below), cells_changed {notebook}, mode {mode}, graph {edges, defs, diagnostics} (the reactive DAG), comm_open/comm_msg/comm_close {comm, buffers} (ipywidgets), error {sessionKey?, message}.

update_display_data is honoured. display(obj, display_id=True) followed by handle.update(...) rewrites the original output instead of appending a new one — it is how the Hugging Face Trainer draws its progress bar and loss table. It used to be dropped, so a fine-tune's cell sat on its first frame ("2/50", an empty loss table) for the whole of a run that was progressing. The session holds each display's output by identity, not by index: a re-run clears the cell's outputs, and an index kept from before would rewrite whatever output landed in that slot next.

Backend surface​

/api/notebook/* (pydantic models in backend/modules/notebook/models.py):

Method + pathPurpose
GET /notebook/fileslist .ipynb under the root
GET /notebook/doc?path=load a document (NotebookModel)
POST /notebookcreate a notebook (starter cells + mode)
PUT /notebook/modeset the execution mode
GET /notebook/envwhether the kernel interpreter is ready, and the library install state
POST /notebook/env/installstart or retry installing notebook.python.packages
GET /notebook/publish?path=where this notebook is published, and which targets are usable now
POST /notebook/publish/preflightwhat would leave the machine: secrets, home paths, tracebacks, widgets
POST /notebook/publishpublish to gist, pages, kaggle or colab (see Publishing)
DELETE /notebook/publish?path=&target=delete the remote copy and the record; &forget=true drops only the record
GET /notebook/publish/html · POST /notebook/publish/htmldownload a standalone HTML page, or save it beside the notebook

Rendering a cell​

OutputRenderer takes the raw nbformat data dict — the backend forwards it untouched, so disk and wire agree — and picks the richest representation it can draw:

MIMERendered as
application/vnd.jupyter.widget-view+jsonlive WidgetView (see above)
text/latexKaTeX, display mode
image/svg+xmlinlined SVG
image/png, image/jpeg, image/gif, image/webp<img> from a data URI
text/htmlinjected (trusted-local posture, same as plugins)
text/markdown, text/plainmonospace, ANSI-parsed

Four things about that order and its edges:

  • text/latex outranks the images. SymPy and IPython.display.Math emit TeX plus a text/plain repr, and taking the repr shows <IPython.core.display.Latex object> where the user asked for an equation.
  • The delimiters are stripped first. display(Latex(r'$$x$$')) arrives wrapped and Math('x') does not; handed $$x$$, KaTeX typesets the dollar signs as literal characters.
  • A bundle with no drawable MIME says so. It used to render nothing, which looks exactly like a cell that produced no output — so the reader debugs their code instead of reading "this needs a renderer".
  • SVG is inlined rather than data-URI'd. matplotlib's svg backend and graphviz both emit text, and an <img> would lose the selectable labels that are half the reason to pick a vector backend.

ANSI is hand-rolled (no dep) and covers 16-colour SGR, the xterm 256 cube and greyscale ramp, and truecolor. The extended forms (38;5;n, 38;2;r;g;b) consume the arguments that follow them — read as plain SGR codes those arguments are interpreted as colours in their own right, which is why the loop advances rather than iterating. Swallowing them, as the first version did, produced tracebacks that were colourful in a terminal and half plain here: a rendering bug, not a missing feature.

Images are capped at 60vh with their own scrollbar. One 8-inch figure at 100 dpi otherwise pushes every later cell off the pane, and scrolling past it is the only way to reach them.

Math in markdown cells​

markdown.ts stays hand-rolled — it covers headings, emphasis, code, links, lists, blockquotes, rules, pipe tables and paragraphs — but math is lifted out before anything else touches the source and put back at the very end. Every other rule in the file corrupts TeX in a way that still renders: escapeHtml turns a raw < into an entity, the italic rule eats a lone *, and the backtick rule claims anything between two of them. The placeholder is a private-use codepoint, which survives escaping, matches none of the inline rules, and is not something a person can type.

An inline span must open on a non-space, close on a non-space, stay on one line, and not be followed by a digit. All four exist to keep it costs $5 and then $10 prose — those two dollars are a perfectly plausible inline pair, and typesetting the words between them is the classic false positive. A pipe table is likewise identified by its |---|---| separator, not by its pipes: choose a | b is prose.

KaTeX runs with throwOnError: false, so a malformed expression renders in KaTeX's own error colour with the source visible. The alternative is an exception inside a cell's render and a blank output, where the reader can see neither the result nor what they typed.

Cell chrome​

Cell styling lives in notebook/notebook.css rather than in inline style objects, so the markdown body's generated <h1>/<pre>/<table> have scoped rules instead of inheriting whatever is global, and KaTeX's stylesheet has a home. A cell carries its state as a class (nb-cell--running, --error, --markdown) and a 2px left accent rail; the run-state badges are stroke SVG inheriting currentColor, because an emoji is a font — full-colour in a monochrome gutter and impossible to tint by state, which is the one thing those glyphs exist to signal.

Type is on the ramp, everywhere in the module — notebook.css, the cell editor's CodeMirror theme, the output renderer, the browser and the widget views. DESIGN.md names six steps by job and themes.css declares them as --fs-*, so the step is chosen by what a piece of text is, not by which pixel value is nearest:

TextStepWhy
Execution counts, diagnostics, banners, a root path--fs-metaTelemetry: "every id, count, duration, path"
The md kind badge, kernel status--fs-micro"chips and badges only"
Pane headers, header buttons, table headers--fs-labelA row's identity
Cell source, output, rendered prose, h3--fs-body"prose in a pane"
h2--fs-leadHeadline
h1--fs-displayThe largest identity type

Tracking and weight come from the same place (--tracking-display, --tracking-badge, --fw-bold); a size on the ramp with hand-picked tracking is only half on it. Two rules decided the awkward cases:

  • A widget's caption is telemetry, not identity. It shows the model's variable name, and DESIGN.md's Two Jobs Rule is explicit that identity is uppercase tracked sans while telemetry is muted mono, never blended — an identifier like n uppercased and tracked stops being the name of the thing.
  • An em value is a ratio, not a step. Inline code inside prose stays 0.85em so it tracks whatever it sits in, at any step.

design-tokens.test.ts fails on a literal font-size anywhere in the module, in CSS and in React style objects alike. It is scoped to this module because this is the one converted end to end — widening it is the goal, but widening it before the files are clean is how a rule gets switched off.

Scroll position and the markdown cell being edited are kept in the pane's view-state bag (see windowing.mdx), so a workspace switch does not scroll the notebook back to the top or close a cell mid-edit. Pending debounced edits are flushed on unmount rather than cleared: an edit typed within EDIT_SYNC_MS of a tab switch otherwise never reached the backend document, and the store's optimistic copy hid that until the notebook was reloaded from disk.

training.notebook consumes the same kit, so it inherits all of this.

Publishing​

Publish in the notebook header opens a panel that puts the notebook somewhere other people can read it, on free services:

TargetWhat you getVisibilityRemoving it
GitHub GistAn nbviewer render of a gist, plus the gist pagesecret (unlisted) or publicUnpublish deletes the gist
GitHub Pageshttps://<you>.github.io/notebooks/<slug>/ as static HTML, plus an nbviewer linkpublic onlyUnpublish deletes the files
KaggleA Kaggle notebook, which Kaggle runs onceprivate or publicForget only — Kaggle has no delete API
Google ColabA Drive file that opens in Colabonly you, or anyone with the linkUnpublish deletes the Drive file
Save as HTML<name>.html beside the notebook——

The pipeline is always prepare → preflight → publish → record, in that order:

  • Prepare a copy. The user's file is never modified: Strip outputs and the removal of saved widget state happen to a deep copy. The copy comes from the live kernel session's document when one is open, because that is what the person is looking at; the file on disk otherwise.
  • Preflight the copy — after preparation, so stripping outputs really clears the findings that lived in outputs. It looks for credential shapes (GitHub, sk- keys, Hugging Face, AWS, Google, Slack, private keys, and quoted TOKEN = "…" assignments), home-directory paths that name the account, tracebacks, and widgets (which render as nothing without a kernel). Base64 image payloads are skipped: random base64 matches short key prefixes often enough to train people to click through.
  • A secret blocks. Publishing waits for an explicit Publish anyway, and the server enforces it before any network call, so a target never receives content that was flagged and not confirmed. Everything else is a warning.
  • A finding shows only a secret's first four characters: the list goes to the browser, and finding lists get pasted.
  • The panel says "found these", never "clean". A pattern list does not know every credential format, and a green tick over a notebook holding an unrecognised key would be the most harmful thing it could draw.

There is no agent tool that publishes. Putting a file on the internet from a sentence is the same problem share.grant refuses.

Where publications are recorded​

notebook_publications in app.db, keyed by the path relative to the notebook root plus the target, so publishing again updates the same gist, page, kernel or Drive file instead of scattering copies. Deliberately not the .ipynb metadata: publishing must never rewrite the file, and a record inside a notebook would itself get published. The cost, stated plainly: renaming or moving a notebook forgets where it was published, and the next publish makes a new copy.

Target rules that are quiet if wrong​

  • A secret gist is unlisted, not private. Anyone holding the link can read it, and the panel says so under the choice. GitHub cannot change a gist's visibility, so switching creates a new gist and deletes the old one — and the result says the old link stopped working rather than leaving it to be discovered.
  • The link handed out is nbviewer, not the gist page: GitHub's own renderer gives up on large notebooks.
  • gist is its own GitHub scope, added after the first sign-ins, so an older token simply lacks it. github.granted_scopes() reads what the stored token actually carries, and the panel says Reconnect GitHub instead of relaying GitHub's bare 404.
  • Pages is public-only. Pages on a private repository needs a paid plan, so a private repository is refused up front rather than producing a link that 404s. The repository (notebook.publish.pagesRepo, default notebooks on your account) is created public when missing; .nojekyll goes in with every push so Jekyll does not drop _-prefixed paths; the page, the .ipynb and any stale-file removals land as one commit (Git Data API, via the shared backend/publishing/ core); and the slug is remembered, so renaming a notebook does not move its public URL. Pages builds asynchronously and the result says the first publish takes about a minute — a fresh link that 404s looks exactly like a broken one.
  • Kaggle and Colab reuse the training pushers, handed a real ProjectModel whose root is $HORRIBLE_DATA_DIR/notebook-publish/<notebook>/ with the copy staged as main.ipynb, so push bookkeeping (kernel-metadata.json, .push-colab.json) never lands in the notebook folder. Kaggle is pushed with the GPU off — a publish is for reading, not for spending quota — and keeps the title and slug of its first push, because Kaggle checks that a title resolves to the kernel id. Credentials stay where training keeps them.

Contributions to the Layout Shell​

  • Panels — notebook.browser ("Notebooks", singleton, left: file catalog + create) and notebook.editor ("Notebook Editor", non-singleton, center: the cell UI, params {path}). The editor is titled Notebook Editor rather than plain "Notebook" because training contributes a second, genuinely different notebook pane (training.notebook, "Training Notebook", running on a project's own venv). Sharing one title made two identical Open: Notebook palette rows, two tabs a person could not tell apart, and a show "notebook" that resolved by module registration order; registry.register now warns when a pane title is already taken.
  • Opening a notebook — always through openNotebook(path) (modules/notebook/open.ts), never a bare openPanel. It calls openDocument with the instance id notebook.editor:<path>, so picking a notebook from the browser focuses the pane already showing it, or takes over an open notebook pane in place when that pane has no unsaved edits — rather than splitting the area and leaving two notebook documents open. Each pane is also labelled with its file name (a title param), because the view's own title is the same string for every notebook. Opened cold with no path, the pane renders the browser inline instead of the old "open one from the Notebooks pane" — which named a view that is embedded and so cannot be opened — and picking there retargets that same instance. Notebook cells sync to the kernel session as you type, so a clean pane holds nothing to lose; the reuse predicate is therefore "any clean notebook pane". To get two notebooks side by side, split the area first and open the second from the empty area's view picker.
  • Completion in a cell — code cells get the editor's full completion stack, whether or not a language server is running, plus the Tab keymap and the ctrl+shift+space toggle. See Notebook cells get the same stack for what was missing and why the two stacks are mutually exclusive.
  • Commands — notebook.open (open the catalog).
  • Regions — notebook.editor hosts the notebook browser as a left region strip (k). The browser is no longer a dock tool of its own: it is the Notebooks section of Explorer (embedded), which is exactly the doubling embedded exists to remove.
  • Agent vocabulary — show "notebook" is pinned by a VIEW_ALIASES entry to the editor pane: two panes answer to the bare word, so it resolves through nothing on its own.
  • Settings — notebook.root, notebook.python, notebook.python.version, notebook.python.packages, notebook.python.torchIndex, notebook.publish.pagesRepo.

Roadmap​

  • Phase 1 (done) — reactive dataflow engine (notebook_core/reactive.py): static analysis, the cell DAG, cascade re-run of downstream cells, stale-variable deletion, multiple_defs/cycle diagnostics.
  • Phase 2 (done) — ipywidgets Comm protocol (sliders, buttons, checkboxes, text, dropdowns) via a lightweight WidgetManager + native WidgetView; worker-owned shell sends keep widgets live during execution.
  • Phase 3 (done, minus the reactive bridge) — custom anywidget (_esm/_css) views via AnyWidgetView, comm_info reattach-resync (backend comm snapshot → frontend seed), and agent tools (notebook.list_cells/read_cell/insert_cell/edit_cell/ delete_cell/run_cell/run_all/set_mode/kernel_status). Classic AMD-bundle @jupyter-widgets views remain a fallback. Deferred: an opt-in bridge from widget values into reactive re-runs.

Documentation in cells​

Cells had no hover, no completion, and no docs at all — CellEditor wired only history, a keymap, and a language. They now have the documentation popup: hover, plus Shift+Tab into an inline panel under the cell.

A cell asks kernel first and gets an answer about the object that actually exists in its namespace, which is strictly better than anything a language server could say.

Cells now also carry a language server — completion, hover and diagnostics, with names resolving across cells, described in Notebook cells get the same language server. The two do not compete: when a cell has a server attached, its docs chain narrows to kernel alone, because the LSP extension runs its own hover with the same index/web fallback behind it and keeping those here too would render two tooltips saying the same thing. lsp is never in the cell's own chain either way — that branch belongs to the extension, and re-entering it would ask the same server the same question twice.

The pane hands the coordinator an absolute path, joining the notebook root from GET /api/notebook/files to the pane's relative path. The language server resolves its interpreter and project root by walking up from the file, so a relative path would silently resolve some other directory's environment.

The kernel side is KernelSession.inspect, and the care it needs (worker-thread affinity, routing shell replies by parent msg_id so an execute reply and an inspect reply cannot eat each other) is described in Documentation popup.

Browser vs desktop​

CapabilityBrowserDesktop
Cell UI, run, outputs✅✅
Managed venv + kernel subprocess✅ (backend-side)✅ (backend-side)
Reactive dataflow mode✅✅
Interactive widgets (core ipywidgets)✅✅
Custom anywidget (_esm/_css) views✅✅
Widget reattach-resync (live kernel)✅✅
Classic AMD @jupyter-widgets views⏳ (fallback)⏳ (fallback)
Publish (Gist, Pages, Kaggle, Colab, HTML file)✅✅