# nbio
## Reading a notebook
A notebook is just a json file.
Exported source
``` python
def _read_json(self, encoding=None, errors=None):
return loads(Path(self).read_text(encoding=encoding, errors=errors))
```
``` python
minimal_fn = Path('../tests/minimal.ipynb')
minimal_txt = AttrDict(_read_json(minimal_fn))
```
It contains two sections, the `metadata`…:
``` python
minimal_txt.metadata
```
{'solveit_dialog_mode': 'learning', 'solveit_ver': 2}
…and, more importantly, the `cells`:
``` python
minimal_txt.cells
```
[{'cell_type': 'markdown',
'id': '801558df',
'metadata': {},
'source': ['## A minimal notebook']},
{'cell_type': 'code',
'execution_count': None,
'id': 'e2147a69',
'metadata': {'time_run': '2026-01-04T20:52:49.901559+00:00'},
'outputs': [{'data': {'text/plain': ['2']},
'execution_count': 0,
'metadata': {},
'output_type': 'execute_result'}],
'source': ['# Do some arithmetic\n', '1+1']}]
The second cell here is a `code` cell, however it contains no outputs,
because it hasn’t been executed yet. To execute a notebook, we first
need to convert it into a format suitable for `nbclient` (which expects
some `dict` keys to be available as attrs, and some available as regular
`dict` keys). Normally, `nbformat` is used for this step, but it’s
rather slow and inflexible, so we’ll write our own function based on
`fastcore`’s handy
[`dict2obj`](https://fastcore.fast.ai/xtras.html#dict2obj), which makes
all keys available as both attrs *and* keys.
------------------------------------------------------------------------
source
### nb_lang
``` python
def nb_lang(
nb
):
```
*Call self as a function.*
Each notebook language has its own comment character(s), taken from
Quarto’s table; [`nb_lang`](https://fastcore.fast.ai/nbio.html#nb_lang)
reads the notebook’s language from its kernelspec, defaulting to Python.
Cells record their language in `lang_` (a per-cell `metadata.language`
overrides the notebook default), which the directive functions below use
to recognize comments.
------------------------------------------------------------------------
source
### NbCell
``` python
def NbCell(
idx, cell, lang:str='python'
):
```
*`dict` subclass that also provides access to keys as attrs, and has a
pretty markdown repr*
We use an [`AttrDict`](https://fastcore.fast.ai/basics.html#attrdict)
subclass which has some basic functionality for accessing notebook
cells.
Two cells are equal if they have the same `id`, even when their content
differs – `id` is what identifies *which* cell this is, so this is what
lets `list.index`/`list.remove` (used by
[`Notebook`](https://fastcore.fast.ai/nbio.html#notebook)’s cell-moving
methods) find the right cell even when another cell elsewhere has
identical source. Cells lacking an `id` fall back to comparing
`source`/`cell_type`.
``` python
c1 = NbCell(0, dict(cell_type='code', source='a', id='x'))
c2 = NbCell(0, dict(cell_type='code', source='b', id='x'))
c3 = NbCell(0, dict(cell_type='code', source='a', id='y'))
test_eq(c1, c2) # same id, different content -- still equal
assert c1 != c3 # same content, different id -- not equal
cells = [c1, c3] # duplicate content ('a'), different ids
test_eq(cells.index(c3), 1) # finds the actual cell, not the first with matching content
```
------------------------------------------------------------------------
source
### dict2nb
``` python
def dict2nb(
js:NoneType=None, **kwargs
):
```
*Convert dict `js` to an
[`AttrDict`](https://fastcore.fast.ai/basics.html#attrdict),*
We can now convert our JSON into this `nbclient`-compatible format,
which pretty prints the source code of cells in notebooks.
``` python
minimal = dict2nb(minimal_txt)
cell = minimal.cells[1]
cell
```
``` python
{ 'cell_type': 'code',
'execution_count': None,
'id': 'e2147a69',
'idx_': 1,
'lang_': 'python',
'metadata': {'time_run': '2026-01-04T20:52:49.901559+00:00'},
'outputs': [ { 'data': {'text/plain': '2'},
'execution_count': 0,
'metadata': {},
'output_type': 'execute_result'}],
'source': '# Do some arithmetic\n1+1'}
```
The abstract syntax tree of source code cells is available in the
`parsed_` property:
``` python
cell.parsed_(), cell.parsed_()[0].value.op
```
([], )
------------------------------------------------------------------------
source
### read_nb
``` python
def read_nb(
path
):
```
*Return notebook at `path` (expands `~`)*
This reads the JSON for the file at `path` and converts it with
[`dict2nb`](https://fastcore.fast.ai/nbio.html#dict2nb). For instance:
``` python
minimal = read_nb(minimal_fn)
str(minimal.cells[0])
```
"{'cell_type': 'markdown', 'id': '801558df', 'metadata': {}, 'source': '## A minimal notebook', 'idx_': 0, 'lang_': 'python'}"
The file name read is stored in `path_`:
``` python
minimal.path_
```
'../tests/minimal.ipynb'
## Creating a notebook
------------------------------------------------------------------------
source
### mk_cell
``` python
def mk_cell(
text, # `source` attr in cell
cell_type:str='code', # `cell_type` attr in cell
**kwargs
):
```
*Create an [`NbCell`](https://fastcore.fast.ai/nbio.html#nbcell)
containing `text`*
``` python
mk_cell('print(1)', execution_count=0)
```
``` python
{ 'cell_type': 'code',
'directives_': {},
'execution_count': 0,
'id': '3b0a226b',
'idx_': 0,
'lang_': 'python',
'metadata': {},
'outputs': [],
'source': 'print(1)'}
```
------------------------------------------------------------------------
source
### new_nb
``` python
def new_nb(
cells:NoneType=None, meta:NoneType=None, nbformat:int=4, nbformat_minor:int=5
):
```
*Returns an empty new notebook*
[`nb_frontmatter`](https://fastcore.fast.ai/nbio.html#nb_frontmatter)
merges a notebook’s frontmatter from two sources: its first raw cell
anywhere, and its first markdown cell — a cell that is *entirely* a
literal `---` block parses as YAML (so `---`-prefixed prose never
half-parses), and a markdown cell that isn’t one contributes `# title`,
`> description`, and `- key: value` lines instead. Raw keys win. It
takes anything whose `cells` items carry `cell_type` and `source`, which
includes aidialog/solveit dialogs (their messages duck-type as cells);
`strvals=True` keeps every YAML scalar a string, and malformed YAML in a
literal block raises.
[`cell_frontmatter`](https://fastcore.fast.ai/nbio.html#cell_frontmatter)
and
[`md_frontmatter`](https://fastcore.fast.ai/nbio.html#md_frontmatter)
are the per-cell pieces, shared with nbdev’s `FrontmatterProc`.
------------------------------------------------------------------------
source
### nb_frontmatter
``` python
def nb_frontmatter(
nb, strvals:bool=False
):
```
*Frontmatter from `nb`: its first raw cell plus first markdown cell
(literal `---` block, or `# title` synthesis), raw keys winning*
------------------------------------------------------------------------
source
### md_frontmatter
``` python
def md_frontmatter(
s:str
):
```
*Frontmatter synthesized from an H1-formatted markdown cell: `# title`,
`> description`, and `- key: value` lines*
------------------------------------------------------------------------
source
### cell_frontmatter
``` python
def cell_frontmatter(
s:str, strvals:bool=False
):
```
*Frontmatter mapping from a cell source that is entirely a literal `---`
block, else {}*
``` python
fm_nb = new_nb([mk_cell('---\ntitle: T\nformdata:\n who: Sam\n n: 1000\n---', 'raw'), mk_cell('Body', 'markdown')])
test_eq(nb_frontmatter(fm_nb), {'title':'T', 'formdata':{'who':'Sam', 'n':1000}})
test_eq(nb_frontmatter(fm_nb, strvals=True)['formdata'], {'who':'Sam', 'n':'1000'})
test_eq(nb_frontmatter(new_nb([])), {})
h1 = mk_cell('# My title\n\n> A subtitle\n\n- author: Zoë\n\nProse.', 'markdown')
test_eq(nb_frontmatter(new_nb([h1])), {'title':'My title', 'description':'A subtitle', 'author':'Zoë'})
both = new_nb([h1, mk_cell('code'), mk_cell('---\ntitle: Raw wins\n---', 'raw')])
test_eq(nb_frontmatter(both)['title'], 'Raw wins') # first raw anywhere; raw beats markdown
test_eq(nb_frontmatter(both)['description'], 'A subtitle') # markdown keys merge beneath
test_eq(nb_frontmatter(new_nb([mk_cell('---\nt: v\n---\ntrailing', 'raw')])), {}) # whole-cell anchoring: a body disqualifies
test_eq(nb_frontmatter(new_nb([mk_cell('---\nt: v\n---', 'markdown')])), {'t':'v'}) # a markdown cell that IS a literal block parses
test_fail(lambda: nb_frontmatter(new_nb([mk_cell('---\nbad: "unclosed\n---', 'raw')])))
```
Use this function when creating a new notebook. Useful for when you
don’t want to create a notebook on disk first and then read it.
``` python
test_eq(new_nb().cells, [])
```
## Directives
nbdev and Quarto put *directives* in comments at the top of a cell:
lines like `#| export` or `#| eval: false`, ending at the first
non-comment line. All spellings are equivalent: `#| foo: bar`,
`#| foo:bar`, and `#| foo bar` parse to the same thing, and a
directive’s value is the raw text after its name, so nothing is lost to
tokenization. A value of
[`true`](https://fastcore.fast.ai/basics.html#true) means the same as no
value at all: `#| hide`, `#| hide: true`, and `#| hide:` are one
directive (which also means no directive can take the literal word
[`true`](https://fastcore.fast.ai/basics.html#true) as its value).
Directives can also be stored in cell metadata, as a dict under an
`nbdev` key, with the same value domain: every value is a str,
`\"true\"` meaning bare (non-str values raise, so a JSON `false` fails
loudly rather than silently differing from `\"false\"`). When the same
name appears in both places, the comment wins.
------------------------------------------------------------------------
source
### first_code_ln
``` python
def first_code_ln(
code_list, re_pattern:NoneType=None, lang:str='python'
):
```
*get first line number where code occurs, where `code_list` is a list of
code*
``` python
_tst = """
#| default_exp
#| export
#| hide_input
foo
"""
test_eq(first_code_ln(_tst.splitlines(True)), 4)
```
[`_directive`](https://fastcore.fast.ai/nbio.html#_directive) parses one
directive line into its name and value. The value is the raw text after
the name (and optional colon), byte-exact apart from one leading space
and trailing whitespace, so both spellings parse identically; a value of
[`true`](https://fastcore.fast.ai/basics.html#true) collapses to `''`,
the same as a bare directive. Non-directive lines (including cell
magics) parse to `None`.
``` python
test_eq(_directive('#| export: utils'), ('export','utils'))
test_eq(_directive('#| export utils'), ('export','utils'))
test_eq(_directive('#| export:utils'), ('export','utils'))
test_eq(_directive('#| hide'), ('hide',''))
test_eq(_directive('#| hide:'), ('hide',''))
test_eq(_directive('#| hide: true'), ('hide',''))
test_eq(_directive(' # | woo:baz'), ('woo','baz'))
test_eq(_directive('#| fig-cap: "two spaces: kept"'), ('fig-cap','"two spaces: kept"'))
test_eq(_directive('#| filter_stream secret apikey'), ('filter_stream','secret apikey'))
test_eq(_directive('%%timeit'), None)
test_eq(_directive('# plain comment'), None)
```
[`dir_tag`](https://fastcore.fast.ai/nbio.html#dir_tag) renders
meta-form directives as the compact bracket tag that summary rows splice
after the type character (`c[export]:...`).
[`CellRow`](https://fastcore.fast.ai/nbio.html#cellrow) uses it below,
and aidialog’s message previews share the same form.
------------------------------------------------------------------------
source
### dir_tag
``` python
def dir_tag(
meta
):
```
*Meta-form nbdev directives in `meta` as a compact `[k k=v]` bracket
tag, or `''` if none*
``` python
test_eq(dir_tag({}), '')
test_eq(dir_tag({'nbdev': {'export': 'utils', 'hide': 'true'}}), '[export=utils hide]')
dir_tag({'nbdev': {'export': 'true'}})
```
'[export]'
`directives` reads as a plain dict of name to value string:
`{'export': 'utils', 'hide': ''}` (bare directives have value `''`). The
getter returns a copy, so to edit, modify the copy and assign it back;
the setter regenerates the comment block in canonical colon form (cell
magics stay first, untouched). Directives that came from cell metadata
are tracked by name and written back to metadata rather than to
comments; names added through the setter become comments.
``` python
c = mk_cell('#| export: utils\n#| hide\ndef f(): pass')
test_eq(c.directives, {'export':'utils', 'hide':''})
d = c.directives
d.pop('hide')
d['eval'] = 'false'
c.directives = d
test_eq(c.source, '#| export: utils\n#| eval: false\ndef f(): pass')
```
Metadata directives merge in (comments win on conflict), and edits route
back to where each directive came from. The setter writes bare
directives to metadata as `"true"`:
``` python
c = mk_cell('#| export: utils\n1', metadata=dict(nbdev=dict(export='other', eval='false')))
test_eq(c.directives, {'export':'utils', 'eval':'false'})
d = c.directives
d['eval'] = ''
d['hide'] = ''
c.directives = d
test_eq(c.metadata['nbdev'], {'eval': 'true'})
test_eq(c.source, '#| export: utils\n#| hide\n1')
test_eq(c.directives, {'export':'utils', 'hide':'', 'eval':''})
with expect_fail(TypeError, 'must be str'): mk_cell('1', metadata=dict(nbdev=dict(eval=False))).directives
```
------------------------------------------------------------------------
source
### NbCell.remove_directives
``` python
def remove_directives(
quarto:bool=False
):
```
*Strip directives from source, keeping cell magics; with `quarto`,
instead materialize every directive (metadata included) as a Quarto
option line*
------------------------------------------------------------------------
source
### NbCell.has_directive
``` python
def has_directive(
name
):
```
*Call self as a function.*
------------------------------------------------------------------------
source
### NbCell.directive
``` python
def directive(
name, default:NoneType=None
):
```
*Value of directive `name` (`''` if bare), or `default` if absent*
`directive` answers “what value?” and `has_directive` answers “is it
there?”: a bare directive’s value is `''`, which is falsy, so presence
tests need `has_directive`. `remove_directives` serves the two pipeline
consumers: the default strips every directive line (module export needs
clean source), while `quarto=True` instead rewrites the cell with every
directive (comments and metadata alike) as a Quarto option line, bare
ones as `name: true`; Quarto consumes the options it knows and quietly
turns unknown ones into `data-*` attributes.
``` python
c = mk_cell('%%time\n#| exports: utils\n#| hide\n#| code-fold: show\nslow()', metadata=dict(nbdev=dict(echo='false')))
test_eq(c.directive('exports'), 'utils')
assert c.has_directive('hide') and not c.has_directive('eval')
c.remove_directives(quarto=True)
test_eq(c.source, '%%time\n#| exports: utils\n#| hide: true\n#| code-fold: show\n#| echo: false\nslow()')
c = mk_cell('%%time\n#| exports: utils\n#| hide\n#| code-fold: show\nslow()', metadata=dict(nbdev=dict(echo='false')))
c.remove_directives()
test_eq(c.source, '%%time\nslow()')
```
## Writing a notebook
------------------------------------------------------------------------
source
### nb2dict
``` python
def nb2dict(
d, k:NoneType=None
):
```
*Convert parsed notebook to `dict`*
This returns the exact same dict as is read from the notebook JSON.
``` python
minimal_fn = Path('../tests/minimal.ipynb')
minimal = read_nb(minimal_fn)
minimal_dict = _read_json(minimal_fn)
assert minimal_dict==nb2dict(minimal)
```
------------------------------------------------------------------------
source
### nb2str
``` python
def nb2str(
nb
):
```
*Convert `nb` to a `str`*
To save a notebook we first need to convert it to a `str`:
``` python
print(nb2str(minimal)[:45])
```
{
"cells": [
{
"cell_type": "markdown",
------------------------------------------------------------------------
source
### write_nb
``` python
def write_nb(
nb, path
):
```
*Write `nb` to `path` (expands `~`)*
This returns the exact same string as saved by Jupyter.
``` python
tmp = Path('tmp.ipynb')
try:
minimal_txt = minimal_fn.read_text()
write_nb(minimal, tmp)
test_eq(minimal_txt, tmp.read_text())
finally: tmp.unlink()
```
## Cell tools
Cell tools apply `fastcore.tools`’ string editing primitives to one
notebook cell’s source, addressed by path and cell id, mirroring that
module’s file tools: the same operations and parameters, with
`path, cell_id` in place of `path`. Each editor (including the
structural `cell_ast_replace`) returns a diff of the change, and
[`view_cell`](https://fastcore.fast.ai/nbio.html#view_cell) shows a
cell’s source with optional line numbers or exhash addresses.
Naming and parameter conventions shared across the editing toolkit are
documented in `fastcore.editskill`, which also re-exports this module’s
editing tools.
Everything in this family addresses its unit by id, with one contract:
exact match, or any unique prefix, and a loud `KeyError` naming the
problem otherwise.
[`find_id`](https://fastcore.fast.ai/nbio.html#find_id) is that contract
as a function, shared by the notebook tools here and the dialog tools
built on them:
------------------------------------------------------------------------
source
### find_id
``` python
def find_id(
items, # Items carrying `.id` attrs
k:str, # Id to find: exact, or unique prefix
unit:str='item', # Unit name for error messages
):
```
*The item with id `k`; a missing or ambiguous id raises `KeyError`
naming the problem*
``` python
_its = [AttrDict(id='aa12'), AttrDict(id='ab34')]
test_eq(find_id(_its, 'ab'), _its[1])
with expect_fail(KeyError, 'ambiguous item id'): find_id(_its, 'a')
with expect_fail(KeyError, 'no thing id'): find_id(_its, 'zz', 'thing')
```
------------------------------------------------------------------------
source
### cell_edit
``` python
def cell_edit(
f, name:NoneType=None
):
```
*Wrap text editor `f` as a cell editing function: `path, cell_id`
addressing, diff-or-error return*
The spliced editors inherit `fastcore.tools`’ signatures, so the
toolkit-wide `replace_params` group collapses the same six options in
`doc()` overviews here:
------------------------------------------------------------------------
source
### view_cell
``` python
def view_cell(
path:str, # Notebook file to read (expands `~`)
cell_id:str, # Id of the cell to view (exact, or unique prefix)
start_line:int=1, # Starting line to view
end_line:int=None, # End line (defaults to last line if None; may be past EOF, which clamps to the last line)
nums:bool=True, # Show line numbers?
lnhashs:bool=False, # Show exhash `lineno|hash|` addresses instead of line numbers?
incl_out:bool=False, # Append the cell's outputs in an `` block?
trunc_out:bool=True, # Truncate included outputs to ~512 chars?
):
```
*View a cell’s source, optionally limited to 1-based line range*
A scratch notebook gives the editors something to work on.
[`view_cell`](https://fastcore.fast.ai/nbio.html#view_cell) reads one
cell’s source by id: plain line numbers by default, a line range, or
lnhash addresses ready for `cell_exhash`:
``` python
tmp_nb = Path('tmp_cells.ipynb')
write_nb(new_nb([mk_cell('a=1\nprint(a)'), mk_cell('# title', 'markdown')]), tmp_nb)
cid = read_nb(tmp_nb).cells[0].id
test_eq(str(view_cell(tmp_nb, cid)), '1: a=1\n2: print(a)')
test_eq(str(view_cell(tmp_nb, cid, start_line=2, nums=False)), 'print(a)')
test_eq(str(view_cell(tmp_nb, cid, lnhashs=True)).splitlines()[0], lnhash(1,'a=1')+'a=1')
```
Each editor is a transaction: it applies the edit, writes the file, and
returns a diff – the diff is the verification, no second read needed:
``` python
res = cell_str_replace(tmp_nb, cid, 'a=1', 'a=2')
assert '-a=1' in str(res) and '+a=2' in str(res)
test_eq(read_nb(tmp_nb).cells[0].source, 'a=2\nprint(a)')
```
The line-addressed ops keep the text primitives’ conventions:
[`insert_line`](https://fastcore.fast.ai/tools.html#insert_line) at 0
prepends, deleting that line undoes it, and `cell_replace_lines` with no
range replaces the whole source – yielding a line block, so the result
carries a trailing newline:
``` python
cell_insert_line(tmp_nb, cid, 0, 'import sys')
cell_del_lines(tmp_nb, cid, 1, 1)
test_eq(read_nb(tmp_nb).cells[0].source, 'a=2\nprint(a)')
cell_replace_lines(tmp_nb, cid, new_content='b=3')
test_eq(read_nb(tmp_nb).cells[0].source, 'b=3\n')
```
Failures are loud: an unmatched
[`str_replace`](https://fastcore.fast.ai/tools.html#str_replace) reports
an error instead of writing, and a missing cell id raises:
``` python
assert str(cell_str_replace(tmp_nb, cid, 'q', 'r')).startswith('error:')
with expect_fail(KeyError, 'no cell id'): view_cell(tmp_nb, 'zzzz')
tmp_nb.unlink()
```
Here’s how to put all the pieces of `fastcore.nbio` together:
``` python
nb = new_nb([mk_cell('print(1)')])
path = Path('test.ipynb')
write_nb(nb, path)
nb2 = read_nb(path)
print(nb2.cells)
path.unlink()
```
[{'cell_type': 'code', 'execution_count': None, 'id': 'd0c314de', 'metadata': {}, 'outputs': [], 'source': 'print(1)', 'idx_': 0, 'lang_': 'python'}]
Notebook files on disk store multiline text as lists of lines
(nbformat’s `split_lines`), for source, stream `text`, textual mime
`data`, and attachments. Like nbformat’s own reader, we join these to
plain strings on read, and split them back on write, so in-memory code
always sees strings while files round-trip byte-identically with
Jupyter’s. JSON mimes and error tracebacks are genuinely structured, so
they pass through untouched.
``` python
disk = dict(nbformat=4, nbformat_minor=5, metadata={}, cells=[
dict(cell_type='code', id='c0', metadata={}, execution_count=1, source=['a=1\n','a'],
outputs=[dict(output_type='execute_result', metadata={}, execution_count=1,
data={'text/plain':['hi\n','there'], 'text/markdown':['single'],
'application/json':{'a':[1,2]}, 'image/png':'iVBOR\nw0KG=='}),
dict(output_type='stream', name='stdout', text=['s1\n','s2\n']),
dict(output_type='error', ename='E', evalue='e', traceback=['t1','t2'])]),
dict(cell_type='markdown', id='m0', metadata={}, source=['# t\n','x'],
attachments={'im.txt':{'text/plain':['a\n','b']}, 'im.png':{'image/png':'aGk='}})])
tmp = Path('tmp.ipynb')
try:
tmp.write_text(dumps(disk))
nb = read_nb(tmp)
res,strm,err = nb.cells[0].outputs
test_eq(res['data']['text/plain'], 'hi\nthere') # textual data joined on read
test_eq(res['data']['text/markdown'], 'single') # one-element lists too
test_eq(res['data']['application/json'], {'a':[1,2]}) # JSON mimes untouched
test_eq(res['data']['image/png'], 'iVBOR\nw0KG==')
test_eq(strm['text'], 's1\ns2\n') # stream text joined
test_eq(err['traceback'], ['t1','t2']) # tracebacks stay lists
test_eq(nb.cells[1].attachments['im.txt']['text/plain'], 'a\nb')
test_eq(nb2dict(nb), disk) # splitting on save restores the disk form exactly
finally: tmp.unlink()
```
## Validation
Direct edits to cell dicts can produce notebooks that Jupyter and other
tools reject: an `outputs` key on a markdown cell, a code cell missing
`execution_count`, duplicate ids.
[`validate_cell`](https://fastcore.fast.ai/nbio.html#validate_cell) and
[`validate_nb`](https://fastcore.fast.ai/nbio.html#validate_nb) are
cheap structural checks for exactly those mistakes. They raise
`ValueError` naming the offending cell, and never repair: fixing is the
caller’s decision. This is not the full nbformat schema, just the rules
whose violation breaks notebooks in practice.
------------------------------------------------------------------------
source
### validate_nb
``` python
def validate_nb(
nb
):
```
*Raise `ValueError` for structural problems in notebook `nb`; returns it
unchanged if fine*
------------------------------------------------------------------------
source
### validate_cell
``` python
def validate_cell(
cell, idx:NoneType=None
):
```
*Raise `ValueError` for structural problems in notebook cell dict
`cell`; returns it unchanged if fine*
A valid notebook passes through unchanged, and each rule fails loudly,
naming the cell (the markdown-with-outputs case is a real one: stray
keys from hand-edited files):
``` python
vnb = new_nb([mk_cell('1+1'), mk_cell('a note', 'markdown')])
test_eq(validate_nb(vnb), vnb)
vnb.cells[1]['outputs'] = []
with expect_fail(ValueError, 'not allowed in a markdown cell'): validate_nb(vnb)
del vnb.cells[1]['outputs'], vnb.cells[0]['execution_count']
with expect_fail(ValueError, 'requires execution_count'): validate_nb(vnb)
```
``` python
dup = new_nb([mk_cell('a', id='x1'), mk_cell('b', id='x1')])
with expect_fail(ValueError, 'duplicate cell id'): validate_nb(dup)
with expect_fail(ValueError, 'unknown cell_type'): validate_cell(dict(cell_type='wat', source=''))
with expect_fail(ValueError, 'str or list of str'): validate_cell(dict(cell_type='raw', source=[1,2]))
```
------------------------------------------------------------------------
source
### repair_nb
``` python
def repair_nb(
nb
):
```
*Fix deterministic structural problems in `nb`, returning a list of
repairs made*
------------------------------------------------------------------------
source
### repair_cell
``` python
def repair_cell(
cell, idx:NoneType=None
):
```
*Fix deterministic structural problems in `cell`, returning a list of
repairs made*
Each validation rule has a deterministic repair, applied by
[`repair_nb`](https://fastcore.fast.ai/nbio.html#repair_nb): non-code
cells lose stray `outputs`/`execution_count`, code cells gain missing
ones, broken `source`/`metadata` are coerced, missing notebook fields
are added, and duplicate cell ids are regenerated (the first occurrence
keeps the id). Unknown `cell_type`s are left alone, since no repair can
know the intent. The returned list says what was done, so callers can
report or count repairs; a valid notebook returns `[]` and is untouched.
``` python
bad = dict2nb(dict(cells=[
dict(cell_type='markdown', source='hi', outputs=[], execution_count=1, id='x1'),
dict(cell_type='code', source='1+1', id='x1'),
], metadata={}, nbformat=4, nbformat_minor=5))
with expect_fail(ValueError): validate_nb(bad)
repairs = repair_nb(bad)
validate_nb(bad)
test_eq(len(repairs), 5)
assert bad.cells[0].id != bad.cells[1].id
test_eq(repair_nb(bad), [])
```
Code outputs have required structure too. Each entry must be a dict with
one of the four standard `output_type`s, carrying its required fields:
`stream` a `name` and `text`, the two display types a `data` dict and
`metadata`, `execute_result` also an `execution_count`, and `error` an
`ename`, `evalue`, and `traceback`. Repair fills the deterministic
omissions (`metadata`, `execution_count`) and removes entries too
malformed to keep, reporting each removal with its reason.
``` python
onb = new_nb([mk_cell('1+1')])
onb.cells[0].outputs = [dict(output_type='execute_result', data={'text/plain':['2']}, metadata={}, execution_count=1)]
test_eq(validate_nb(onb), onb)
onb.cells[0].outputs = [dict(output_type='wat')]
with expect_fail(ValueError, 'unknown output_type'): validate_nb(onb)
onb.cells[0].outputs = [dict(output_type='stream', name='stdout')]
with expect_fail(ValueError, 'text'): validate_nb(onb)
onb.cells[0].outputs = [dict(output_type='execute_result', data={})]
with expect_fail(ValueError, 'metadata'): validate_nb(onb)
onb.cells[0].outputs = [dict(output_type='error', ename='E', evalue='boom', traceback='not a list')]
with expect_fail(ValueError, 'traceback'): validate_nb(onb)
```
``` python
rnb = new_nb([mk_cell('1+1')])
rnb.cells[0].outputs = [
dict(output_type='execute_result', data={'text/plain':'2'}),
dict(output_type='stream', name='stdout', text='hi'),
dict(output_type='error', ename='E', evalue='boom'),
'junk']
repairs = repair_nb(rnb)
validate_nb(rnb)
test_eq(len(rnb.cells[0].outputs), 2)
test_eq(rnb.cells[0].outputs[0]['metadata'], {})
test_eq(rnb.cells[0].outputs[0]['execution_count'], None)
test_eq(repair_nb(rnb), [])
repairs
```
['cell 58d10327 output 0: set metadata',
'cell 58d10327 output 0: set execution_count',
'cell 58d10327 output 2: removed output (error requires a traceback list of str)',
'cell 58d10327 output 3: removed output (output must be a dict)']
## Output rendering
------------------------------------------------------------------------
source
### preferred_out
``` python
def preferred_out(
data, html1st:bool=True, include_imgs:bool=False
):
```
*Call self as a function.*
[`preferred_out`](https://fastcore.fast.ai/nbio.html#preferred_out)
selects the best MIME type from an output’s `data` dict, preferring HTML
by default:
``` python
data = dict(text_plain=['42'], **{'text/html': ['42'], 'text/plain': ['42']})
wdata = {'image/webp': 'AAAA', 'text/plain': ['im']}
test_eq(preferred_out(wdata, include_imgs=True)[0], 'image/webp')
test_eq(preferred_out(wdata)[0], 'text/plain')
preferred_out(data), preferred_out(data, html1st=False)
```
(('text/html', ['42']), ('text/html', ['42']))
------------------------------------------------------------------------
source
### join_out
``` python
def join_out(
d
):
```
*Join Jupyter’s list-of-lines output data into one string*
------------------------------------------------------------------------
source
### mk_error
``` python
def mk_error(
traceback, ename:str='', evalue:str=''
):
```
*Helper to create an error output dict*
------------------------------------------------------------------------
source
### mk_display
``` python
def mk_display(
metadata:NoneType=None, **data
):
```
*Helper to create a display_data output dict*
------------------------------------------------------------------------
source
### mk_result
``` python
def mk_result(
metadata:NoneType=None, **data
):
```
*Helper to create an execute_result output dict*
------------------------------------------------------------------------
source
### mk_stream
``` python
def mk_stream(
name, text
):
```
*Helper to create an output stream dict*
------------------------------------------------------------------------
source
### concat_streams
``` python
def concat_streams(
outputs
):
```
*Concatenate stream outputs by name (stdout/stderr), preserving
execute_result at end*
[`concat_streams`](https://fastcore.fast.ai/nbio.html#concat_streams)
merges consecutive stream outputs by name and moves execute_results to
the end, like standard jupyter output rendering:
``` python
outs = [mk_result(text_plain=['42']),
mk_stream('stdout', 'hello '), mk_stream('stdout', 'world\n'), mk_stream('stderr', 'warn\n')]
outs
```
[{'output_type': 'execute_result',
'data': {'text/plain': ['42']},
'metadata': {}},
{'output_type': 'stream', 'name': 'stdout', 'text': 'hello '},
{'output_type': 'stream', 'name': 'stdout', 'text': 'world\n'},
{'output_type': 'stream', 'name': 'stderr', 'text': 'warn\n'}]
``` python
concat_streams(outs)
```
[{'output_type': 'stream', 'name': 'stdout', 'text': 'hello world\n'},
{'output_type': 'stream', 'name': 'stderr', 'text': 'warn\n'},
{'output_type': 'execute_result',
'data': {'text/plain': ['42']},
'metadata': {}}]
Carriage-return overwrites apply across chunk boundaries, as they would
on a live terminal — a progress line ending in `\r` is overwritten by
the next chunk, but a final `\r` (nothing follows it) leaves the text
visible:
``` python
prog = [mk_stream('stdout', 'step 1\r'), mk_stream('stdout', 'step 2\r'), mk_stream('stdout', 'done\n')]
test_eq(concat_streams(prog), [mk_stream('stdout', 'done\n')])
test_eq(concat_streams([mk_stream('stdout', 'working\r')]), [mk_stream('stdout', 'working')])
```
------------------------------------------------------------------------
source
### preferred_msg_out
``` python
def preferred_msg_out(
out, html1st:bool=True, include_imgs:bool=False
):
```
*Preferred mime type and content for any Jupyter output dict (stream,
error, or data-bearing)*
[`preferred_msg_out`](https://fastcore.fast.ai/nbio.html#preferred_msg_out)
extends
[`preferred_out`](https://fastcore.fast.ai/nbio.html#preferred_out) to
any output dict: streams and errors are always plain text, while
data-bearing outputs go through mime preference:
``` python
test_eq(preferred_msg_out(mk_stream('stdout', ['a\n','b\n'])), ('text/plain', 'a\nb\n'))
test_eq(preferred_msg_out(mk_result(text_html=['4'], text_plain=['4'])), ('text/html', ['4']))
test_eq(preferred_msg_out(mk_result(text_markdown=['*4*'], text_html=['4']), html1st=False)[0], 'text/markdown')
```
------------------------------------------------------------------------
source
### render_output
``` python
def render_output(
out
):
```
*Convert a single output dict to an HTML string*
``` python
print(render_output(mk_result(text_plain=['42'])))
```
42
------------------------------------------------------------------------
source
### render_outputs
``` python
def render_outputs(
outputs
):
```
*Render a full list of outputs, concatenating streams first.*
``` python
print(render_outputs(outs))
```
hello world
warn
42
------------------------------------------------------------------------
source
### render_text
``` python
def render_text(
outputs, html1st:bool=False, tb_maxlen:NoneType=None
):
```
*Render notebook outputs to concise ANSI-stripped text, using XML-ish
tags when multiple outputs are present; `tb_maxlen` caps over-long
error-traceback lines*
A single output renders as plain text directly:…
``` python
print(render_text([outs[0]]))
```
42
…but multiple outputs get wrapped in XML-ish tags so they stay
distinguishable. For rich outputs we prefer markdown by default; pass
`html1st=True` to prefer HTML instead. Outputs without a text
representation (e.g. images alone) render as empty.
``` python
print(render_text(outs))
```
hello world
warn
42
Errors arrive ANSI-colored from IPython, so
[`render_text`](https://fastcore.fast.ai/nbio.html#render_text) always
strips escape sequences. A traceback line can also be enormous - a cell
magic’s transformed source echoes its whole payload on one line - so
`tb_maxlen` caps over-long traceback lines, keeping `File`/`Cell`
locations and the final chunk (the exception message) whole, and
dropping caret-decoration rows that mean nothing once cut.
``` python
tb = ['\x1b[31mFile /a/b.py:1\x1b[0m\n' + 'x'*200 + '\n~~~~^^^' + '~'*200, 'ValueError: ' + 'y'*200]
t = render_text([mk_error(tb, 'ValueError', 'boom')])
test_eq('\x1b' in t, False)
assert 'x'*200 in t
t2 = render_text([mk_error(tb, 'ValueError', 'boom')], tb_maxlen=120)
assert 'x'*200 not in t2 and 'File /a/b.py:1' in t2 and '~~~~^^^' not in t2 and 'y'*200 in t2
print(t2)
```
[`view_cell`](https://fastcore.fast.ai/nbio.html#view_cell) composes
with output rendering: `incl_out=True` appends the rendered outputs in
an `` block (`trunc_out=False` lifts the ~512-char cap).
``` python
vnb = Path('tmp_out.ipynb')
vouts = [dict(output_type='execute_result', metadata={}, data={'text/plain': ['1']}, execution_count=1)]
write_nb(new_nb([mk_cell('a=1\nprint(a)', outputs=vouts)]), vnb)
vcid = read_nb(vnb).cells[0].id
test_eq(str(view_cell(vnb, vcid, incl_out=True)), '1: a=1\n2: print(a)\n\n1\n')
assert '' not in str(view_cell(vnb, vcid))
vnb.unlink()
```
``` python
disp = [mk_display(text_html=['42'], text_markdown=['**42**'], text_plain=['42'])]
test_eq(render_text(disp), '**42**')
test_eq(render_text(disp, html1st=True), '42')
test_eq(render_text([mk_display(image_png='abc')]), '')
test_eq(render_text([mk_error('oops')]), 'oops')
```
``` python
nb.cells[0].outputs = [mk_stream('stdout', '1\n')]
```
## Notebook class
------------------------------------------------------------------------
source
### item2xml
``` python
def item2xml(
typ, # Tag name: the cell or message type, e.g. 'code', 'markdown', 'raw', 'prompt'
content:str='', # The item's source text
out:str='', # Rendered output text
id:NoneType=None, # Optional id attribute
meta:NoneType=None, # Cell/message metadata: directives in its `nbdev` dict render as attrs, bare ones as bare attrs
**attrs
):
```
*A notebook cell or dialog message as concise XML: content, then an
`` section when `out` is non-empty*
[`item2xml`](https://fastcore.fast.ai/nbio.html#item2xml) renders
notebook cells and dialog messages to LLM-friendly XML
([`cell2xml`](https://fastcore.fast.ai/nbio.html#cell2xml) below and
`aidialog`’s message renderers build on it). The content sits directly
inside the type tag, with no wrapper of its own, and an `` section
marks where output begins - so an item without output carries no extra
tags at all. Passing `meta` renders the metadata’s nbdev directives as
attributes, so every projection built on
[`item2xml`](https://fastcore.fast.ai/nbio.html#item2xml) shows them the
same way.
``` python
test_eq(to_xml(item2xml('code', 'x*2', '42', id='ab')), 'x*242')
to_xml(item2xml('markdown', '# hi', id='cd'))
```
'# hi'
Falsy attrs are dropped, literal True attrs have no value:
``` python
test_eq(to_xml(item2xml('code', 'x', time='', kind='system', foo=True)), 'x')
test_eq(to_xml(item2xml('code', 'x', meta=dict(nbdev=dict(hide='true', eval='false')))), 'x')
```
------------------------------------------------------------------------
source
### cells2xml
``` python
def cells2xml(
cells, wrap:partial=functools.partial(, 'nb'), ids:bool=True, incl_out:bool=True,
**kw
):
```
*Convert notebook cells to XML format*
------------------------------------------------------------------------
source
### cell2xml
``` python
def cell2xml(
cell, ids:bool=True, incl_out:bool=True
):
```
*Convert NbCell to concise XML format*
We can view any notebook as concise XML. For instance, our minimal
notebook:
``` python
print(cells2xml(nb.cells, incl_out=False))
```
a=1
a# t
x
``` python
repr(cell2xml(nb.cells[0], incl_out=False))
```
'a=1\na'
``` python
repr(cell2xml(nb.cells[0], incl_out=True))
```
'a=1\na1\n'
``` python
# Metadata directives render as attrs: bare as bare, others verbatim (names unhyphenated)
c = mk_cell('1+1', metadata=dict(nbdev=dict(hide='true', default_exp='core')))
test_eq(repr(cell2xml(c, incl_out=False)), f'1+1')
```
------------------------------------------------------------------------
source
### Notebook
``` python
def Notebook(
nb, path:NoneType=None
):
```
*Read, query, and edit Jupyter notebooks*
We can now open a notebook and access its metadata and cells:
``` python
nbo = Notebook.open(minimal_fn)
list(nbo.meta), len(nbo.cells), len(nbo)
```
(['solveit_dialog_mode', 'solveit_ver'], 2, 2)
``` python
nbo.path.name
```
'minimal.ipynb'
``` python
[o.id for o in nbo]
```
['801558df', 'e2147a69']
``` python
'e2147a69' in nbo, 'nonexistent' in nbo
```
(True, False)
Notebooks’ repr is their xml:
``` python
nbo
```
## A minimal notebook# Do some arithmetic
1+12
You can also get a more concise version that doesn’t include outputs or
the full path:
``` python
print(nbo.concise)
```
## A minimal notebook# Do some arithmetic
1+1
Cells can be accessed by integer index or by their string id – exact, or
any unique prefix. A missing or ambiguous id raises a `KeyError` naming
the problem:
``` python
nbo[0].source
```
'## A minimal notebook'
``` python
nbo['e2147a69'].source
```
'# Do some arithmetic\n1+1'
``` python
test_eq(nbo['e214'].source, nbo['e2147a69'].source)
with expect_fail(KeyError, 'no cell id'): nbo['nope']
```
You can directly set a cell’s source by id or index:
``` python
nbo['e2147a69'] = '2+2'
nbo['e2147a69'].source
```
'2+2'
Cells also carry the shared edit family as methods - the same operations
as the `cell_*` functions, minus the address arguments, since the cell
in hand *is* the carrier. These are in-memory edits: the diff comes back
for verification, and nothing touches disk until the notebook is saved.
``` python
d = nbo['e2147a69'].str_replace('2+2', '3+3')
test_eq(nbo['e2147a69'].source, '3+3')
print(d)
```
@@ -1 +1 @@
-2+2
+3+3
You can also update outputs and metadata directly on a cell:
``` python
nbo['e2147a69'].outputs = [{'output_type': 'execute_result', 'data': {'text/plain': ['4']}}]
nbo['e2147a69'].outputs
```
[{'output_type': 'execute_result', 'data': {'text/plain': ['4']}}]
``` python
nbo['e2147a69'].metadata['custom'] = True
nbo['e2147a69'].metadata
```
``` python
{'custom': True, 'time_run': '2026-01-04T20:52:49.901559+00:00'}
```
The `add` method inserts a new cell at a given position (defaulting to
the end):
------------------------------------------------------------------------
source
### Notebook.add
``` python
def add(
source, cell_type:str='code', idx:NoneType=None, after:NoneType=None, before:NoneType=None, **kwargs
):
```
*Add a new cell with `source` at `idx` (default: end), or
`after`/`before` a cell id*
``` python
nbo.add('print("hello")')
nbo.add('# A heading', cell_type='markdown', idx=0)
len(nbo), nbo[0].source
```
(4, '## A minimal notebook')
Cells can also be inserted relative to an existing cell by id:
``` python
cid = nbo[0].id
nbo.add('# After first', cell_type='markdown', after=cid)
nbo.add('# Before first', cell_type='markdown', before=cid)
[c.source for c in nbo[:3]]
```
['# Before first', '## A minimal notebook', '# After first']
------------------------------------------------------------------------
source
### Notebook.md
``` python
def md(
source, idx:NoneType=None, after:NoneType=None, before:NoneType=None, **kwargs
):
```
*Add a new cell with `source` at `idx` (default: end), or
`after`/`before` a cell id*
`md` is a shortcut to `add(..., cell_type='markdown')`
``` python
nbo.md('A note')
len(nbo), nbo[-1].cell_type
```
(7, 'markdown')
You can delete by id or index:
``` python
prev_len = len(nbo)
del nbo[0]
len(nbo) == prev_len - 1
```
True
------------------------------------------------------------------------
source
### Notebook.move
``` python
def move(
src_ids, after:NoneType=None, before:NoneType=None
):
```
*Move cells with `src_ids` after/before a cell id, or to end*
Cells can be moved by id, either relative to another cell or to the end:
``` python
nbo = Notebook.open(minimal_fn)
c0,c1 = nbo[0].id,nbo[1].id
nbo.move(c1, before=c0)
[c.id for c in nbo] == [c1, c0]
```
True
Use `save` to write to disk:
``` py
nbo.save('path.ipynb')
```
If no path is passed, the path used in `open()` will be re-used.
------------------------------------------------------------------------
source
### Notebook.view_cell
``` python
def view_cell(
id, nums:bool=True, incl_out:bool=False, trunc_out:bool=True
):
```
*Show cell source with optional line numbers*
The [`view_cell`](https://fastcore.fast.ai/nbio.html#view_cell) method
displays a cell’s source with optional line numbers:
``` python
print(nbo.view_cell('e2147a69', incl_out=True))
assert nbo.view_cell('e2147a69', incl_out=True).endswith('\n2\n')
assert '' not in nbo.view_cell('e2147a69')
```
1 │ # Do some arithmetic
2 │ 1+1
2
The path-taking twins of the query methods return *snapshots*, not live
cells: [`CellRow`](https://fastcore.fast.ai/nbio.html#cellrow) records
id, type, source, and the cell meta, and its summary line
`id:t[directives]:source` (t: c=code m=markdown r=raw) shows any nbdev
directives, so export state is visible at a glance. Rows are data to
read and addresses to act on - edits then go through the `cell_*`
functions or `cell_exhash`. The correspondence is mechanical: the
function is the method with a `path` argument standing where the held
notebook was.
------------------------------------------------------------------------
source
### CellRows
``` python
def CellRows(
*args, **kwargs
):
```
*Built-in mutable sequence.*
If no argument is given, the constructor creates a new empty list. The
argument must be an iterable if specified.
------------------------------------------------------------------------
source
### CellRow
``` python
def CellRow(
c, maxlen:int=120, kind:str='match'
):
```
*Snapshot of one cell, shown as `id:t[directives]:source` (t: c=code
m=markdown r=raw); a context row from a find shows `-` in place of its
final `:`*
------------------------------------------------------------------------
source
### Notebook.to_dict
``` python
def to_dict():
```
*The plain dict form of the held notebook
([`nb2dict`](https://fastcore.fast.ai/nbio.html#nb2dict)): the
representation layer*
------------------------------------------------------------------------
source
### summary_nb
``` python
def summary_nb(
path, # Notebook file to read
maxlen:int=120, # Maximum source characters per line
):
```
*One snapshot line per cell of the notebook at `path`*
------------------------------------------------------------------------
source
### Notebook.summary
``` python
def summary(
maxlen:int=120
):
```
*One [`CellRow`](https://fastcore.fast.ai/nbio.html#cellrow) line per
cell*
``` python
test_eq(type(Notebook.open(minimal_fn).to_dict()), dict)
summary_nb(minimal_fn)
```
801558df:m:## A minimal notebook
e2147a69:c:# Do some arithmetic\n1+1
[`find_cells`](https://fastcore.fast.ai/nbio.html#find_cells) searches
cell sources by regex, and by default includes one neighbouring cell of
context around each match, because in a documented notebook the
neighbouring markdown usually explains the match. Results are
[`FoundCells`](https://fastcore.fast.ai/nbio.html#foundcells) – the
[`Found`](https://fastcore.fast.ai/nbio.html#found) mixin over a plain
list, so sibling toolkits (aidialog’s message finds) share the identical
contract: index them by cell id (exact or unique prefix), never by
position – a context row can sit at any index, so `fc[0]` may be a
neighbour of the match, and integer indexing raises an error saying so.
In the display, a context row shows `-` in place of its final `:`.
------------------------------------------------------------------------
source
### Found
``` python
def Found(
items:NoneType=None, matched:NoneType=None
):
```
*Mixin for find-result containers: `matched` ids, id-only indexing, and
match/context kinds*
------------------------------------------------------------------------
source
### FoundCells
``` python
def FoundCells(
items:NoneType=None, matched:NoneType=None
):
```
*Find results: cells indexed by id (exact or unique prefix), shown as
[`CellRow`](https://fastcore.fast.ai/nbio.html#cellrow) lines with
context rows marked*
------------------------------------------------------------------------
source
### Notebook.find_cells
``` python
def find_cells(
pat:str='', # Regex over cell source
cell_type:str=None, # Optional limit by type ('code', 'markdown', or 'raw')
ids:str='', # Optional limit by cell ids (comma-separated str, or list); exact or unique prefixes
context:int=None, # Cells of context around matches (default 1)
):
```
*Find live cells matching all the given criteria, plus `context`
neighbouring cells*
``` python
nbo = Notebook.open(minimal_fn)
fc = nbo.find_cells(r'\d\+\d')
fc
```
The match is the code cell; its markdown neighbour rides along as
context, marked with `-`. Positional indexing fails with an error
teaching the id idiom, and `context=0` returns only matches:
``` python
with expect_fail(TypeError, 'context rows'): fc[0]
test_eq(fc['e214'].source, '# Do some arithmetic\n1+1')
test_eq([c.id for c in nbo.find_cells(r'\d\+\d', context=0)], ['e2147a69'])
test_eq(fc.matched, {'e2147a69'})
```
The path-taking twin returns
[`CellRow`](https://fastcore.fast.ai/nbio.html#cellrow) snapshots in a
[`FoundCells`](https://fastcore.fast.ai/nbio.html#foundcells), with the
same indexing contract:
------------------------------------------------------------------------
source
### find_cells
``` python
def find_cells(
path, # Notebook file to search
pat:str='', # Regex over cell source
cell_type:str=None, # Optional limit by type ('code', 'markdown', or 'raw')
ids:str='', # Optional limit by cell ids (comma-separated str, or list); exact or unique prefixes
context:int=None, # Cells of context around matches (default 1)
):
```
*Snapshot [`FoundCells`](https://fastcore.fast.ai/nbio.html#foundcells)
for matching cells in the notebook at `path`*
``` python
rows = find_cells(minimal_fn, r'\d\+\d')
test_eq(type(rows['e2147a69']), CellRow)
test_eq(rows.matched, {'e2147a69'})
rows
```
[`update_cell`](https://fastcore.fast.ai/nbio.html#update_cell) is the
transaction for a cell’s *attributes* rather than its source’s lines:
plain keywords assign (`cell_type=`, `metadata=` replacing wholesale),
`mergemeta` deep-merges into the metadata, and `export` sets the nbdev
`export` directive. There is no negative export directive – a cell is
exported by having it or not by lacking it – so `export=False` means
removal, in whichever form (comment or metadata) the cell uses, and
`export=True` adds it.
[`deep_merge`](https://fastcore.fast.ai/nbio.html#deep_merge) treats a
`None` value as deletion:
------------------------------------------------------------------------
source
### deep_merge
``` python
def deep_merge(
d:dict, # Base dict
u:dict, # Updates: nested dicts merge recursively, and a `None` value deletes its key
):
```
*Copy of `d` updated by `u`*
``` python
test_eq(deep_merge(dict(a=1,b=dict(c=2,d=3)), dict(b=dict(c=None,e=4), a=None)), dict(b=dict(d=3,e=4)))
```
------------------------------------------------------------------------
source
### update_cell
``` python
def update_cell(
path:str, # Notebook file to modify
cell_id:str, # Id of the cell to update (exact, or unique prefix)
mergemeta:dict=None, # `deep_merge` into cell metadata; a `None` value deletes its key
export:bool=None, # Add (True) or remove (False) the nbdev export directive, in whichever form the cell uses
**kwargs
):
```
*Update a cell’s attributes, metadata, or export directive*
The diff covers the cell’s concise XML and its metadata, so every change
this function can make is visible in its return value. Removing the
directive from a comment-form cell edits its source; adding one to a
bare cell writes the comment form too, keeping the cell’s existing
style:
``` python
up = Path('tmp_upd.ipynb')
write_nb(new_nb([mk_cell('#| export\ndef f(): pass', id='aa11'), mk_cell('x=1', id='bb22')]), up)
print(update_cell(up, 'aa11', export=False))
print(update_cell(up, 'bb22', mergemeta=dict(tags=['demo'])))
```
``` python
test_eq(read_nb(up).cells[0].directives, {})
test_eq(read_nb(up).cells[1].metadata['tags'], ['demo'])
print(update_cell(up, 'bb22', metadata={}))
up.unlink()
```
## Selecting cells to run
Kernels and notebook tools (execnb’s `CaptureShell.run_all`, aidialog’s
`%nbrun` magic) need to pick code cells from a notebook by cell id.
[`select_cells`](https://fastcore.fast.ai/nbio.html#select_cells)
matches an id or unique prefix, optionally including cells above/below
the match or all code cells, and optionally filtering to nbdev-exported
cells or dropping cells that `nbdev-test` would skip:
------------------------------------------------------------------------
source
### select_cells
``` python
def select_cells(
nb, # A notebook read with `read_nb`
*msgids:str, # Cell ids, or unique prefixes, to match
above:bool=False, # Include each matched cell and all cells above it?
below:bool=False, # Include each matched cell and all cells below it?
all:bool=False, # Include all code cells (ignores `msgids`)?
exported:bool=False, # Only cells with `#| export` or `#| exports`?
skip_noeval:bool=False, # Skip `#| eval: false` and `nbdev_export` cells (like `nbdev-test`)?
):
```
*Select code cells from `nb` by cell id or unique prefix, in the order
given*
``` python
snb = new_nb([mk_cell('# intro', 'markdown')] + [mk_cell(f'x{i} = {i}', id=f'aa{i}{i}0000') for i in range(4)])
codes = [c for c in snb.cells if c.cell_type=='code']
c1 = codes[1]
test_eq(select_cells(snb, 'aa11'), [c1])
test_eq(select_cells(snb, 'aa110000'), [c1])
test_eq(select_cells(snb, 'aa11', above=True), codes[:2])
test_eq(select_cells(snb, 'aa11', below=True), codes[1:])
test_eq(select_cells(snb, all=True), codes)
test_eq(select_cells(snb, 'aa33', 'aa00'), [codes[3], codes[0]]) # several ids, in the order given
c1.source = '#| export\n'+c1.source
test_eq(select_cells(snb, all=True, exported=True), [c1])
c1.source = '#| eval: false\n'+c1.source
test_eq(select_cells(snb, all=True, skip_noeval=True), [c for c in codes if c is not c1])
codes[2].source = 'nbdev_export'+'()'
test_eq(select_cells(snb, all=True, skip_noeval=True), [codes[0], codes[3]])
with expect_fail(Exception, 'aa'): select_cells(snb, 'aa')
with expect_fail(Exception, 'msgids'): select_cells(snb)
select_cells(snb, 'aa11')
```
[{'cell_type': 'code',
'source': '#| eval: false\n#| export\nx1 = 1',
'directives_': {},
'id': 'aa110000',
'metadata': {},
'outputs': [],
'execution_count': None,
'idx_': 2,
'lang_': 'python',
'_meta_names_': set(),
'_directives_': {'eval': 'false', 'export': ''}}]
Directive lookalikes inside strings are not directives, while
metadata-form directives count like comment-form ones:
``` python
sc = mk_cell('x = """\n#| eval: false\n"""')
assert sc in select_cells(new_nb([sc]), all=True, skip_noeval=True)
mc = mk_cell('y = 1', metadata=dict(nbdev=dict(eval='false')))
assert mc not in select_cells(new_nb([mc]), all=True, skip_noeval=True)
```
## Running a cell
------------------------------------------------------------------------
source
### run_cell
``` python
async def run_cell(
shell, # An `InteractiveShell`-compatible object: `transform_cell`, `run_cell_async`, `events`
raw_cell:str, # Python/IPython source for one cell: magics, `!` commands, and top-level `await` all work
store_history:bool=False, # Store the cell in the shell's history? (enables native `;` suppression and execution counts)
silent:bool=False, # Suppress displayhook and `post_run_cell` event? (the result value stays unset)
shell_futures:bool=True, # Share `__future__` imports with the shell?
cell_id:NoneType=None, # Optional cell id, passed through to `run_cell_async`
):
```
*Run one cell on [`shell`](https://fastcore.fast.ai/xtras.html#shell):
transform, await `run_cell_async` on the calling loop, and fire the post
events; returns the `ExecutionResult`*
[`run_cell`](https://fastcore.fast.ai/nbio.html#run_cell) is the shared
execution core. It transforms the source itself, since IPython no longer
auto-transforms, then awaits the shell’s own `run_cell_async`, and fires
the post events that `run_cell_async` omits. It is duck-typed over the
shell, needing only `transform_cell`, `run_cell_async`, and `events`, so
any `InteractiveShell`-compatible object works and fastcore needs no
IPython import. What happens to displays, streams, and tracebacks is
decided by the shell you pass and the `silent` flag, never here. A
capturing shell records, a kernel shell publishes, and `silent=True`
suppresses the displayhook and `post_run_cell` while streams still flow,
which also leaves the result value unset. With `store_history=True` the
shell’s native trailing-`;` suppression and execution counts work
exactly as in a live kernel.
One run returns the value and fires `post_run_cell`. A failing run
records its exception in the result rather than raising. A `silent` run
fires neither the displayhook nor `post_run_cell`, so the value goes
unrecorded. A trailing `;` suppresses as in a notebook:
``` python
sh = InteractiveShell()
events = []
sh.events.register('post_run_cell', lambda result: events.append(result))
r = await run_cell(sh, 'y = 6\ny*7', store_history=True)
test_eq((r.result, r.error_in_exec, len(events)), (42, None, 1))
r2 = await run_cell(sh, '1/0', store_history=True)
test_eq((type(r2.error_in_exec), len(events)), (ZeroDivisionError, 2))
rs = await run_cell(sh, '9*9', silent=True, store_history=True)
test_eq((rs.result, len(events)), (None, 2))
r3 = await run_cell(sh, 'y*7;', store_history=True)
test_eq(r3.result, None)
r
```
A fresh `InteractiveShell` shows the contract: source in,
`ExecutionResult` out, state kept in the shell’s namespace. With
`store_history=True` the shell’s native trailing-`;` suppression
applies, and magics, `!` commands, and top-level `await` all work.
``` python
sh = InteractiveShell()
r = await run_cell(sh, 'x = 6\nx*7', store_history=True)
test_eq((sh.user_ns['x'], r.result), (6, 42))
test_eq((await run_cell(sh, 'x;', store_history=True)).result, None)
test_eq((await run_cell(sh, '%who_ls int', store_history=True)).result, ['x'])
(await run_cell(sh, 'import asyncio\nawait asyncio.sleep(0.001)\n"slept"', store_history=True)).result
```
Prints and displays go through the ordinary user channels, so an
enclosing capture sees them. On a plain shell the displayhook writes the
result to stdout as well, since displaying is exactly what a non-silent
run does. `silent=True` runs the same code with no display and no
populated result.
``` python
with capture_output() as cap: r = await run_cell(sh, 'print("inner"); 5', store_history=True)
assert cap.stdout.startswith('inner\n') and 'Out' in cap.stdout
test_eq(r.result, 5)
with capture_output() as cap2: rs = await run_cell(sh, 'print("quiet"); 5', silent=True)
test_eq((cap2.stdout, rs.result), ('quiet\n', None))
```
The execution events fire around the cell, so extensions that display
from them work. matplotlib’s inline backend flushes figures on
`post_execute`, and since a bare `InteractiveShell` has no GUI loop,
events are all it needs.
``` python
sh.enable_gui = lambda gui=None: None
await run_cell(sh, '%matplotlib inline')
with capture_output() as cap: await run_cell(sh, 'import matplotlib.pyplot as plt\nplt.plot([1,4,2]);', store_history=True)
test_eq(len(cap.outputs), 1)
cap.outputs[0]
```
## Message outputs
Consumers of kernel messages usually want the iopub outputs in the shape
a notebook file stores (the wire framing itself lives in
[jupywire](https://github.com/AnswerDotAI/jupywire)), so downstream code
(renderers, dialog models) needn’t know about protocol wrapping.
[`msg2out`](https://fastcore.fast.ai/nbio.html#msg2out) converts one
message, with `nbformat.v4.output_from_msg`’s semantics but no nbformat
dependency: fields are selected explicitly, so the protocol-only
`transient` never survives into the output dict, `metadata` tolerates
being absent or `None`, and a non-output message raises `ValueError`.
The `msg_type` may sit at the message top level (as
jupyter_client-shaped dicts carry it) or in the header (as wire messages
do). [`msgs2outs`](https://fastcore.fast.ai/nbio.html#msgs2outs)
converts a whole drained list, skipping non-output chatter such as
`status` and `execute_input`.
------------------------------------------------------------------------
source
### msgs2outs
``` python
def msgs2outs(
msgs
):
```
*nbformat-style output dicts for the output messages in `msgs`, skipping
other message types*
------------------------------------------------------------------------
source
### msg2out
``` python
def msg2out(
msg
):
```
*nbformat-style output dict for Jupyter iopub message dict `msg`*
``` python
def _hdr_msg(typ, **c): return dict(header=dict(msg_type=typ), content=c) # wire style
def _top_msg(typ, **c): return dict(msg_type=typ, content=c) # jupyter_client style
test_eq(msg2out(_hdr_msg('stream', name='stdout', text='hi\n')), dict(output_type='stream', name='stdout', text='hi\n'))
test_eq(msg2out(_top_msg('error', ename='E', evalue='boom', traceback=['t'])),
dict(output_type='error', ename='E', evalue='boom', traceback=['t']))
test_eq(msg2out(_hdr_msg('execute_result', data={'text/plain':'42'}, metadata={}, execution_count=1, transient={'display_id':'x'})),
dict(output_type='execute_result', metadata={}, data={'text/plain':'42'}, execution_count=1))
test_eq(msg2out(_top_msg('display_data', data={'text/plain':'hi'}, metadata=None, transient={})),
dict(output_type='display_data', metadata={}, data={'text/plain':'hi'}))
with expect_fail(ValueError): msg2out(_hdr_msg('status', execution_state='idle'))
msgs = [_hdr_msg('status', execution_state='busy'), _hdr_msg('execute_input', code='1+1'),
_hdr_msg('stream', name='stdout', text='hi\n'), _top_msg('execute_result', data={'text/plain':'2'}, metadata={})]
msgs2outs(msgs)
```
[{'output_type': 'stream', 'name': 'stdout', 'text': 'hi\n'},
{'output_type': 'execute_result',
'metadata': {},
'data': {'text/plain': '2'},
'execution_count': None}]