Script - CLI

Part of fast.ai’s toolkit for delightful developer experiences.

Overview

Creates a CLI from a Python function decorated with call_parse.

The function’s parameters become the script’s arguments, its docstring becomes the program description, and its docments comments become the help for each argument.

Sometimes, you want to create a quick script, either for yourself, or for others. But in Python, that involves a whole lot of boilerplate and ceremony, especially if you want to support command line arguments, provide help, and other niceties. You can use argparse for this purpose, which comes with Python, but it’s complex and verbose.

fastcore.script makes life easier. There are much fancier modules to help you write scripts (we recommend Python Fire, and Click is also popular), but fastcore.script is very fast and very simple. Basically, it’s just a little wrapper around argparse that uses modern Python features and some thoughtful defaults to get rid of the boilerplate.

Example

Here’s a complete example (examples/test_fastcore.py in the fastcore repo):

from fastcore.script import *
@call_parse
def main(
    msg:str, # The message
    upper:bool # Convert to uppercase?
):
    "Print `msg`, optionally converting to uppercase"
    print(msg.upper() if upper else msg)

Copy that into a file and run it, and you get a CLI with help, defaults, and error handling, with no if __name__ == "__main__" or argument parsing code:

$ examples/test_fastcore.py --help
usage: test_fastcore.py [-h] [--upper] msg

Print `msg`, optionally converting to uppercase

positional arguments:
  msg         The message

options:
  -h, --help  show this help message and exit
  --upper     Convert to uppercase? (default: False)

The function is still a plain Python function too, so it can also be called normally, e.g from a REPL such as Jupyter.

Annotated params

For argparse features that docments can’t express, use typing.Annotated type hints instead of plain types:

from fastcore.script import *
from typing import Annotated
@call_parse
def main(msg:Annotated[str, "The message"],
         upper:Annotated[store_true, "Convert to uppercase?"]):
    "Print `msg`, optionally converting to uppercase"
    print(msg.upper() if upper else msg)

The first element of Annotated is the parameter’s type, and the first string in its metadata is its help. A dict in the metadata passes extra arguments to argparse: its keys can be opt, action, nargs, const, choices, required, and version. All but opt go straight to add_argument; opt is a bool that overrides whether the param is a flag or positional, which is normally inferred from whether it has a default.

Short flags

A capital letter in a parameter name declares a short flag: the capitalized letter becomes the short spelling and the lowercased name the long one, so Resume:int=None gets both -r and --resume. The capital can be any letter (sUggest:str=None gives -u/--suggest), only the first capital counts, and names without capitals get a long flag only. Underscores in optional parameter names appear as hyphens (cache_dir becomes --cache-dir), while the Python argument keeps its underscore. Since the flags are lowercased, the parameter’s actual name keeps its capital – so a Python caller writes main(Resume=3), which usefully advertises that it’s invoking a CLI entry point. Positional (default-less) parameters have no flags, so capitals there are left alone.

Positional params

Parameters without a default are positional, and the rest are flags. Pass pos to call_parse (or to anno_parser) to keep named parameters positional even when they have a default, in which case they are optional on the command line and take the default when omitted. Command line order follows the signature, not the order of the names in pos. Naming a bool parameter in pos raises, since a flag takes no value.

setuptools scripts

pip and setuptools can create command-line scripts directly from functions, make them available in the PATH, and even make them cross-platform (e.g. in Windows it creates an exe). To use this with a call_parse function, add it to the [project.scripts] section of your pyproject.toml, of the form script_name = "module:function_name". For instance, fastcore itself has:

[project.scripts]
py2pyi = "fastcore.py2pyi:py2pyi"

After installing the package (pip install -e . for an editable install while developing), py2pyi can be typed at any time, from any directory, and the function runs as a CLI. In an nbdev project, add the same section to your pyproject.toml and nbdev takes care of the rest.

API details

---

source

store_true

def store_true():

Placeholder annotation type for a store_true argparse action

---

source

store_false

def store_false():

Placeholder annotation type for a store_false argparse action

---

source

bool_arg

def bool_arg(
    v
):

Annotation type giving bool behavior for CLI args

Param types

A bool parameter is a store_true flag defaulting to False. If its default is True it becomes a --no- prefixed store_false flag instead, so passing the flag turns it off. Use bool_arg as the type when you want an explicit --flag true|false argument that honors its default.

Each parameter in your function can be annotated Annotated[type, "help"], optionally with a dict of extra argparse arguments as a further metadata element (as described above). You should provide a default (after the =) for any optional parameters. If you don’t provide a default for a parameter, then it will be a positional parameter.

def f(
    required:Annotated[int, "Required param"],
    a:Annotated[bool_arg, "param 1"],
    b:Annotated[str, "param 2"]="test",
):
    "my docs"
    ...

help(f)

Help on function f in module __main__:

f(
    required: Annotated[int, 'Required param'],
    a: Annotated[bool_arg, 'param 1'],
    b: Annotated[str, 'param 2'] = 'test'
)
    my docs

test_eq(_arg_kw('some_flag', int, 'help', 1, {}), ('--some-flag', dict(type=int, default=1, help='help (default: 1)')))
test_eq(_arg_kw('a', int, 'help', inspect.Parameter.empty, {}), ('a', dict(type=int, help='help')))
test_eq(_arg_kw('some_flag', bool, 'enabled?', True, {}), ('--no-some-flag', dict(action='store_false', default=True, help='enabled? (default: True)', dest='some_flag')))
test_eq(_arg_kw('p', str, 'path', inspect.Parameter.empty, dict(opt=False, nargs='?', default='-')),
        ('p', dict(type=str, default='-', help='path (default: -)', nargs='?')))
test_eq(_arg_kw('a', int, 'help', 1, {}, 'int'), ('--a', dict(type=int, default=1, help='help (default: 1)', metavar='(int)')))
test_eq(_arg_kw('a', int, 'help', inspect.Parameter.empty, {}, 'int'), ('a', dict(type=int, help='help')))
test_eq(_arg_kw('f', str, 'flags', '', {}), ('--f', dict(type=str, default='', help="flags (default: '')")))

By default argparse repeats an option’s name, upper-cased, as the placeholder for its value: --path PATH. That repeat says nothing useful, so help shows the annotation instead: --path (str). anno2str gives the display name. The name describes the semantics rather than the mechanics: Path displays as path, and a union names each member once, dropping str when path is present since a path already is a string – which is why Path|str is plain path. Enum params keep argparse’s {choice,...} display, and positional params keep their name.

---

source

anno2str

def anno2str(
    t
):

Display name for CLI annotation t, e.g. for an argparse metavar

test_eq(anno2str(int), 'int')
test_eq(anno2str(int|str), 'int|str')
test_eq(anno2str(int|None), 'int')
test_eq(anno2str(bool_arg), 'bool')
test_eq(anno2str(Path), 'path')
anno2str(Path|str)

'path'

---

source

anno_parser

def anno_parser(
    func, prog:str=None, pos:list=None
):

Look at params (with type/docments/Annotated annotations) in func and return an ArgumentParser

This converts a function with docments and/or Annotated parameter annotations into an argparse.ArgumentParser object. Function arguments with a default provided are optional, and other arguments are positional.

_en = str_enum('_en', 'aa','bb','cc')
def f(
    required:Annotated[int, "Required param"],
    a:Annotated[bool_arg, "param 1"],
    v:Annotated[str, "Print version", dict(action='version', version='%(prog)s 2.0.0')],
    b:Annotated[str, "param 2"]="test",
    c:Annotated[_en, "param 3"]=_en.aa,
):
    "my docs"
    ...

p = anno_parser(f, 'progname')
p.print_help()

usage: progname [-h] [--v] [--b (str)] [--c {aa,bb,cc}] required a

my docs

positional arguments:
  required        Required param
  a               param 1

options:
  -h, --help      show this help message and exit
  --v             Print version
  --b (str)       param 2 (default: test)
  --c {aa,bb,cc}  param 3 (default: aa)

We can also check the version flag is working.

try: p.parse_args(['--v'])
except: pass

progname 2.0.0

f lives in this notebook, so its help has no version. A function from an installed package gets the package name and version as the help epilog (from the package’s __version__, else its distribution metadata), so -h on any call_parse console script tells the user which release they are running:

test_is(_pkg_version(f), None)
test_eq(anno_parser(docments).epilog, f'fastcore {fastcore.__version__}')
!py2pyi -h

usage: py2pyi [-h] [--package (str)] fname

Convert `fname.py` to `fname.pyi` by removing function bodies and expanding
`delegates` kwargs

positional arguments:
  fname            The file name to convert

options:
  -h, --help       show this help message and exit
  --package (str)  The parent package

fastcore 2.2.15

It also works with type annotations and docments:

def g(
    required:int,  # Required param
    a:bool_arg,    # param 1
    b="test",      # param 2
    c:_en=_en.aa # param 3
):
    "my docs"
    ...

p = anno_parser(g, 'progname')
p.print_help()

usage: progname [-h] [--b (str)] [--c {aa,bb,cc}] required a

my docs

positional arguments:
  required        Required param
  a               param 1

options:
  -h, --help      show this help message and exit
  --b (str)       param 2 (default: test)
  --c {aa,bb,cc}  param 3 (default: aa)

def s(
    Resume:int=None,     # resume session N
    Load:str=None,       # load a file
    Verbose:bool=False,  # say more
    model:str='m1',      # no capital: long flag only
):
    "shorts docs"

p = anno_parser(s, 'prog')
test_eq(p.parse_args(['-r','3']).Resume, 3)            # capital letter -> short flag, dest is the param name
test_eq(p.parse_args(['--resume','4']).Resume, 4)      # lowercased name -> long flag
test_eq(p.parse_args(['-l','x','-v']).Load, 'x')
test_eq(p.parse_args(['-v']).Verbose, True)            # works for store_true bools too
test_eq(p.parse_args(['--model','m2']).model, 'm2')    # capital-free params are unchanged
test_eq(p.parse_args([]).Resume, None)

Union types such as int|str try each type in turn, and enum types such as those from str_enum become argparse choices.

def h(n:int|str, exts:str|list=None):
    "Test union types"

p = anno_parser(h, 'test')
p.print_help()

usage: test [-h] [--exts (str|list)] n

Test union types

positional arguments:
  n

options:
  -h, --help         show this help message and exit
  --exts (str|list)

test_eq(p.parse_args(['42', '--exts', 'py']).n, 42)
test_eq(p.parse_args(['hello']).n, 'hello')

To use argparse features that docments can’t express, add a dict to the Annotated metadata; its keys are passed through, e.g. nargs:

def j(words:Annotated[str, "Words to join", dict(nargs='+')]):
    "Test nargs"

p = anno_parser(j, 'test')
test_eq(p.parse_args(['a','b']).words, ['a','b'])

def k(
    fname:str=None,  # File to read (default: stdin)
    n:int=1,         # How many
    verbose:bool=False,  # Say more
):
    "Test pos"

p = anno_parser(k, 'test', pos=['fname'])
p.print_help()
test_eq(p.parse_args([]).fname, None)              # optional: omitted, so the default
test_eq(p.parse_args(['a.md']).fname, 'a.md')      # positional, no flag needed
test_eq(p.parse_args(['a.md','--n','2']).n, 2)     # other params are flags as usual

with expect_fail(ValueError, "can't be a bool"): anno_parser(k, 'test', pos=['fname','verbose'])

usage: test [-h] [--n (int)] [--verbose] [fname]

Test pos

positional arguments:
  fname       File to read (default: stdin)

options:
  -h, --help  show this help message and exit
  --n (int)   How many (default: 1)
  --verbose   Say more (default: False)

Sometimes it’s convenient to extract arguments from the actual name of the called program. args_from_prog will do this, assuming that names and values of the params are separated by a #. Optionally there can also be a prefix separated by ## (double underscore).

---

source

args_from_prog

def args_from_prog(
    func, prog
):

Extract args from prog

exp = {'a': False, 'b': 'baa'}
test_eq(args_from_prog(f, 'foo##a#0#b#baa'), exp)
test_eq(args_from_prog(f, 'a#0#b#baa'), exp)

---

source

set_ctx

def set_ctx(
    cv, val:bool=True
):

Call self as a function.

def _chk(co_file, name='__main__', file='ascript.py', locs=None):
    g = {'__name__':name, '__file__':file, 'inspect':inspect, '_is_script_run':_is_script_run}
    return eval(compile('_is_script_run(inspect.currentframe())', co_file, 'eval'), g, locs)

test_eq(_chk('ascript.py'), True)               # file run directly
test_eq(_chk('ascript.py', name='amod'), False) # imported module
test_eq(_chk('other.py'), False)                # notebook cell: code isn't __main__'s file
test_eq(_chk('ascript.py', locs={}), False)     # not top-level: function/class body in __main__'s file

---

source

call_parse

def call_parse(
    func:NoneType=None, nested:bool=False, pos:list=None
):

Decorator to create a simple CLI from func using anno_parser

@call_parse
def test_add(
    a:int=0,  # param a
    b:int=0  # param 1
):
    "Add up `a` and `b`"
    return a + b

call_parse decorated functions work as regular functions and also as command-line interface functions.

test_eq(test_add(1,2), 3)

The CLI entry point

The decorated function is always the CLI entry point, and only the entry point gets sys.argv: argv is parsed when the function’s file is run directly (python foo.py, python -m foo, or %run foo.py), or when it’s called with no arguments from the top-level body of a directly-run file (which is how console script wrappers invoke it). Every other call is a plain Python call that ignores argv, whether from a notebook, the REPL, another function, or another call_parse function.

@call_parse
def main(a:int=0): return test_add(a, a)

argv,sys.argv = sys.argv,['prog', '--a', '3']
try: test_eq(main(), 0)  # interactive call: uses defaults, argv untouched
finally: sys.argv = argv

Use the nested keyword argument to create nested parsers, where earlier parsers consume only their known args from sys.argv before later parsers are used. This is useful to create one command line application that executes another. For example:

myrunner --keyword 1 script.py -- <script.py args>

A separating -- after the first application’s args is recommended though not always required, otherwise args may be parsed in unexpected ways. For example:

myrunner script.py -h

would display myrunner’s help and not script.py’s.

call_parse auto-executes the function when its file is run directly, e.g python foo.py or %run foo.py. Importing a module never runs its CLI functions, and neither does defining one in a notebook cell:

ran=[]
@call_parse
def cell_cmd(): ran.append(1)
test_eq(ran, [])

This is the main way to use fastcore.script; decorate your function with call_parse, add type annotations and docments (as shown above), optionally with Annotated metadata for extra argparse features, and it can then be used as a script.

---

source

is_cli

def is_cli(
    func:NoneType=None
):

True if a call_parse CLI run is in progress, optionally checking that func is the function being run

A function usable both from the command line and from Python can call is_cli to tell which way it was invoked, e.g. returning a value to Python callers but printing it (or exiting with an error code) when run as a CLI:

@call_parse
def sum_args(a:int=0, b:int=0):
    "Add `a` and `b`"
    if is_cli(): print(a+b)
    else: return a+b

test_eq(sum_args(1,2), 3)  # Python call: returns the value, prints nothing