Script - CLI

Part of fast.ai’s toolkit for delightful developer experiences.

Overview

Creates a CLI from a Python function decorated with call_parse.

The function’s parameters become the script’s arguments, its docstring becomes the program description, and its docments comments become the help for each argument.

Python includes argparse for command-line arguments and help, but configuring it can take a lot of boilerplate for a quick script.

fastcore.script is a small, fast wrapper around argparse. It uses type annotations and defaults to keep the setup short. We recommend Python Fire for fancier CLIs. Click is also popular.

Example

Here’s a complete example (examples/test_fastcore.py in the fastcore repo):

from fastcore.script import *
@call_parse
def main(
    msg:str, # The message
    upper:bool # Convert to uppercase?
):
    "Print `msg`, optionally converting to uppercase"
    print(msg.upper() if upper else msg)

call_parse provides argument parsing, help, defaults and error handling. Copy the example into a file and run it without adding an if __name__ == "__main__" block:

$ examples/test_fastcore.py --help
usage: test_fastcore.py [-h] [--upper] msg

Print `msg`, optionally converting to uppercase

positional arguments:
  msg         The message

options:
  -h, --help  show this help message and exit
  --upper     Convert to uppercase? (default: False)

You can also call the function normally from Python, including in a Jupyter notebook.

Annotated params

Use typing.Annotated for argparse options that docments can’t express:

from fastcore.script import *
from typing import Annotated
@call_parse
def main(msg:Annotated[str, "The message"],
         upper:Annotated[store_true, "Convert to uppercase?"]):
    "Print `msg`, optionally converting to uppercase"
    print(msg.upper() if upper else msg)

The first element is the parameter’s type. The first string in its metadata provides help text. Add a metadata dictionary to set action, nargs, const, choices, required or version arguments for argparse.add_argument.

The dictionary also accepts opt. Set it to True for a flag or False for a positional parameter. Without opt, parameters with defaults become flags.

Short flags

The first capital letter in an optional parameter’s name declares a short flag:

• Resume:int=None gives -r and --resume.
• sUggest:str=None gives -u and --suggest.
• Names without capitals have only a long flag.

Long flags use lowercase and hyphens, such as --cache-dir for cache_dir. Python calls keep the parameter’s spelling, such as main(Resume=3). Positional parameters have no flags and keep their names unchanged.

Positional params

Parameters without defaults are positional. Other parameters become flags. *args accepts zero or more positional values. Parameters after it are flags. CLI calls ignore **kwargs.

Pass parameter names in pos to call_parse or anno_parser to keep them positional even with defaults. Omitted values use those defaults. Command-line order follows the signature, regardless of the order in pos. A boolean flag cannot appear in pos because it takes no value.

setuptools scripts

pip and setuptools can install a command that calls a Python function. The command is available on PATH, including as an .exe on Windows. For a call_parse function, add script_name = "module:function_name" under [project.scripts] in pyproject.toml. Fastcore uses:

[project.scripts]
py2pyi = "fastcore.py2pyi:py2pyi"

For development, install the package with pip install -e .. You can then run py2pyi from any directory in that environment. The same configuration works in an nbdev project.

API details

---

source

store_true

def store_true():

Placeholder annotation type for a store_true argparse action

---

source

store_false

def store_false():

Placeholder annotation type for a store_false argparse action

---

source

bool_arg

def bool_arg(
    v
):

Annotation type giving bool behavior for CLI args

Param types

A bool parameter is a store_true flag defaulting to False. If its default is True it becomes a --no- prefixed store_false flag instead, so passing the flag turns it off. Use bool_arg as the type when you want an explicit --flag true|false argument that honors its default.

Each parameter in your function can be annotated Annotated[type, "help"], optionally with a dict of extra argparse arguments as a further metadata element (as described above). You should provide a default (after the =) for any optional parameters. If you don’t provide a default for a parameter, then it will be a positional parameter.

def f(
    required:Annotated[int, "Required param"],
    a:Annotated[bool_arg, "param 1"],
    b:Annotated[str, "param 2"]="test",
):
    "my docs"
    ...

help(f)

Help on function f in module __main__:

f(
    required: Annotated[int, 'Required param'],
    a: Annotated[bool_arg, 'param 1'],
    b: Annotated[str, 'param 2'] = 'test'
)
    my docs

test_eq(_arg_kw('some_flag', int, 'help', 1, {}), ('--some-flag', dict(type=int, default=1, help='help (default: 1)')))
test_eq(_arg_kw('a', int, 'help', inspect.Parameter.empty, {}), ('a', dict(type=int, help='help')))
test_eq(_arg_kw('some_flag', bool, 'enabled?', True, {}), ('--no-some-flag', dict(action='store_false', default=True, help='enabled? (default: True)', dest='some_flag')))
test_eq(_arg_kw('p', str, 'path', inspect.Parameter.empty, dict(opt=False, nargs='?', default='-')),
        ('p', dict(type=str, default='-', help='path (default: -)', nargs='?')))
test_eq(_arg_kw('a', int, 'help', 1, {}, 'int'), ('--a', dict(type=int, default=1, help='help (default: 1)', metavar='(int)')))
test_eq(_arg_kw('a', int, 'help', inspect.Parameter.empty, {}, 'int'), ('a', dict(type=int, help='help')))
test_eq(_arg_kw('f', str, 'flags', '', {}), ('--f', dict(type=str, default='', help="flags (default: '')")))

argparse normally repeats an option’s name as its value placeholder, such as --path PATH. Here help shows the type instead, such as --path (str).

anno2str formats the type name. It displays Path as path. Unions list each type once and omit str when path is present. For example, Path|str displays as path.

Enums show their choices as {choice,...}. Positional parameters show their names.

---

source

anno2str

def anno2str(
    t
):

Display name for CLI annotation t, e.g. for an argparse metavar

test_eq(anno2str(int), 'int')
test_eq(anno2str(int|str), 'int|str')
test_eq(anno2str(int|None), 'int')
test_eq(anno2str(bool_arg), 'bool')
test_eq(anno2str(Path), 'path')
anno2str(Path|str)

'path'

---

source

anno_parser

def anno_parser(
    func, prog:str=None, pos:list=None
):

Look at params (with type/docments/Annotated annotations) in func and return an ArgumentParser

This converts a function with docments and/or Annotated parameter annotations into an argparse.ArgumentParser object. Function arguments with a default provided are optional, and other arguments are positional.

_en = str_enum('_en', 'aa','bb','cc')
def f(
    required:Annotated[int, "Required param"],
    a:Annotated[bool_arg, "param 1"],
    v:Annotated[str, "Print version", dict(action='version', version='%(prog)s 2.0.0')],
    b:Annotated[str, "param 2"]="test",
    c:Annotated[_en, "param 3"]=_en.aa,
):
    "my docs"
    ...

p = anno_parser(f, 'progname')
p.print_help()

usage: progname [-h] [--v] [--b (str)] [--c {aa,bb,cc}] required a

my docs

positional arguments:
  required        Required param
  a               param 1

options:
  -h, --help      show this help message and exit
  --v             Print version
  --b (str)       param 2 (default: test)
  --c {aa,bb,cc}  param 3 (default: aa)

We can also check the version flag is working.

try: p.parse_args(['--v'])
except: pass

progname 2.0.0

For functions from installed packages, help includes the package name and version. The version comes from __version__ or distribution metadata. Notebook functions have no package version:

test_is(_pkg_version(f), None)
test_eq(anno_parser(docments).epilog, f'fastcore {fastcore.__version__}')
!py2pyi -h

usage: py2pyi [-h] [--package (str)] fname

Convert `fname.py` to `fname.pyi` by removing function bodies and expanding
`delegates` kwargs

positional arguments:
  fname            The file name to convert

options:
  -h, --help       show this help message and exit
  --package (str)  The parent package

fastcore 2.2.18

It also works with type annotations and docments:

def g(
    required:int,  # Required param
    a:bool_arg,    # param 1
    b="test",      # param 2
    c:_en=_en.aa # param 3
):
    "my docs"
    ...

p = anno_parser(g, 'progname')
p.print_help()

usage: progname [-h] [--b (str)] [--c {aa,bb,cc}] required a

my docs

positional arguments:
  required        Required param
  a               param 1

options:
  -h, --help      show this help message and exit
  --b (str)       param 2 (default: test)
  --c {aa,bb,cc}  param 3 (default: aa)

def s(
    Resume:int=None,     # resume session N
    Load:str=None,       # load a file
    Verbose:bool=False,  # say more
    model:str='m1',      # no capital: long flag only
):
    "shorts docs"

p = anno_parser(s, 'prog')
test_eq(p.parse_args(['-r','3']).Resume, 3)            # capital letter -> short flag, dest is the param name
test_eq(p.parse_args(['--resume','4']).Resume, 4)      # lowercased name -> long flag
test_eq(p.parse_args(['-l','x','-v']).Load, 'x')
test_eq(p.parse_args(['-v']).Verbose, True)            # works for store_true bools too
test_eq(p.parse_args(['--model','m2']).model, 'm2')    # capital-free params are unchanged
test_eq(p.parse_args([]).Resume, None)

Union types such as int|str try each type in turn, and enum types such as those from str_enum become argparse choices.

def h(n:int|str, exts:str|list=None):
    "Test union types"

p = anno_parser(h, 'test')
p.print_help()

usage: test [-h] [--exts (str|list)] n

Test union types

positional arguments:
  n

options:
  -h, --help         show this help message and exit
  --exts (str|list)

test_eq(p.parse_args(['42', '--exts', 'py']).n, 42)
test_eq(p.parse_args(['hello']).n, 'hello')

Add a dictionary to Annotated metadata to pass options to argparse.add_argument. Here nargs='+' accepts one or more words:

def j(words:Annotated[str, "Words to join", dict(nargs='+')]):
    "Test nargs"

p = anno_parser(j, 'test')
test_eq(p.parse_args(['a','b']).words, ['a','b'])

A *args annotation converts each positional value. The following example also has a keyword-only flag:

def m(
    x:int,          # First value
    *ys:int,        # More values
    tot:bool=False, # Show total?
    **kw,           # Ignored: the CLI passes nothing for it
):
    "Test variadic"

p = anno_parser(m, 'test')
p.print_help()
a = p.parse_args(['1','2','3'])
test_eq((a.x, a.ys), (1, [2,3]))
assert 'kw' not in a
test_eq(p.parse_args(['1']).ys, [])

def mb(*b:bool): ...
with expect_fail(ValueError, "can't be a bool"): anno_parser(mb)

usage: test [-h] [--tot] x [ys ...]

Test variadic

positional arguments:
  x           First value
  ys          More values

options:
  -h, --help  show this help message and exit
  --tot       Show total? (default: False)

def k(
    fname:str=None,  # File to read (default: stdin)
    n:int=1,         # How many
    verbose:bool=False,  # Say more
):
    "Test pos"

p = anno_parser(k, 'test', pos=['fname'])
p.print_help()
test_eq(p.parse_args([]).fname, None)              # optional: omitted, so the default
test_eq(p.parse_args(['a.md']).fname, 'a.md')      # positional, no flag needed
test_eq(p.parse_args(['a.md','--n','2']).n, 2)     # other params are flags as usual

with expect_fail(ValueError, "can't be a bool"): anno_parser(k, 'test', pos=['fname','verbose'])

usage: test [-h] [--n (int)] [--verbose] [fname]

Test pos

positional arguments:
  fname       File to read (default: stdin)

options:
  -h, --help  show this help message and exit
  --n (int)   How many (default: 1)
  --verbose   Say more (default: False)

args_from_prog reads arguments encoded in the program name. Use # between parameter names and values. An optional prefix ends with ## (two hash characters).

---

source

args_from_prog

def args_from_prog(
    func, prog
):

Extract args from prog

exp = {'a': False, 'b': 'baa'}
test_eq(args_from_prog(f, 'foo##a#0#b#baa'), exp)
test_eq(args_from_prog(f, 'a#0#b#baa'), exp)

---

source

set_ctx

def set_ctx(
    cv, val:bool=True
):

def _chk(co_file, name='__main__', file='ascript.py', locs=None):
    g = {'__name__':name, '__file__':file, 'inspect':inspect, '_is_script_run':_is_script_run}
    return eval(compile('_is_script_run(inspect.currentframe())', co_file, 'eval'), g, locs)

test_eq(_chk('ascript.py'), True)               # file run directly
test_eq(_chk('ascript.py', name='amod'), False) # imported module
test_eq(_chk('other.py'), False)                # notebook cell: code isn't __main__'s file
test_eq(_chk('ascript.py', locs={}), False)     # not top-level: function/class body in __main__'s file

---

source

call_parse

def call_parse(
    func:NoneType=None, nested:bool=False, pos:list=None
):

Decorator to create a simple CLI from func using anno_parser

@call_parse
def test_add(
    a:int=0,  # param a
    b:int=0  # param 1
):
    "Add up `a` and `b`"
    return a + b

call_parse decorated functions work as regular functions and also as command-line interface functions.

test_eq(test_add(1,2), 3)

The CLI entry point

call_parse parses sys.argv when you run the function’s file directly with python foo.py, python -m foo or %run foo.py. It also parses arguments on a zero-argument call from the top-level body of a directly run file. Console-script wrappers use this form.

Calls from a notebook, REPL or another function are ordinary Python calls. This includes calls from another call_parse function.

CLI calls return integers, including booleans, as exit codes. They discard other return values. Python calls preserve return values. Raise CliError to report an error message to CLI users.

@call_parse
def main(a:int=0): return test_add(a, a)

argv,sys.argv = sys.argv,['prog', '--a', '3']
try: test_eq(main(), 0)  # interactive call: uses defaults, argv untouched
finally: sys.argv = argv

Set nested=True when one CLI launches another. The outer parser removes the arguments it recognizes from sys.argv, leaving the rest for the inner CLI:

myrunner --keyword 1 script.py -- <script.py args>

-- is optional in some invocations. Use it to separate the applications’ arguments and avoid cases such as:

myrunner script.py -h

myrunner handles -h here instead of passing it to script.py.

call_parse auto-executes the function when its file is run directly, e.g python foo.py or %run foo.py. Importing a module never runs its CLI functions, and neither does defining one in a notebook cell:

ran=[]
@call_parse
def cell_cmd(): ran.append(1)
test_eq(ran, [])

---

source

is_cli

def is_cli(
    func:NoneType=None
):

True if a call_parse CLI run is in progress, optionally checking that func is the function being run

Use is_cli to distinguish CLI execution from Python calls. Here the CLI prints a result that Python callers receive as a return value:

@call_parse
def sum_args(a:int=0, b:int=0):
    "Add `a` and `b`"
    if is_cli(): print(a+b)
    else: return a+b

test_eq(sum_args(1,2), 3)  # Python call: returns the value, prints nothing

---

source

CliError

def CliError(
    *args, **kwargs
):

Raised when a command can’t continue: a CLI run exits with the message, a Python call sees the exception

Raise CliError when a command cannot continue. CLI execution prints its message to stderr and exits with status 1. Python callers can catch the exception:

@call_parse
def show(fname:str=None):
    "Print `fname`"
    if fname is None: raise CliError("fname is required")
    print(fname)

with expect_fail(CliError, contains='fname is required'): show()