Script - CLI

A fast way to turn your python function into a script.

Part of fast.ai’s toolkit for delightful developer experiences.

Overview

Sometimes, you want to create a quick script, either for yourself, or for others. But in Python, that involves a whole lot of boilerplate and ceremony, especially if you want to support command line arguments, provide help, and other niceties. You can use argparse for this purpose, which comes with Python, but it’s complex and verbose.

fastcore.script makes life easier. There are much fancier modules to help you write scripts (we recommend Python Fire, and Click is also popular), but fastcore.script is very fast and very simple. In fact, it’s <50 lines of code! Basically, it’s just a little wrapper around argparse that uses modern Python features and some thoughtful defaults to get rid of the boilerplate.

For full details, see the docs for core.

Example

Here’s a complete example (available in examples/test_fastcore.py):

from fastcore.script import *
@call_parse
def main(
    msg:str, # The message
    upper:bool # Convert to uppercase?
):
    "Print `msg`, optionally converting to uppercase"
    print(msg.upper() if upper else msg)

If you copy that info a file and run it, you’ll see:

$ examples/test_fastcore.py --help
usage: test_fastcore.py [-h] [--upper] msg

Print `msg`, optionally converting to uppercase

positional arguments:
  msg          The message

optional arguments:
  -h, --help   show this help message and exit
  --upper      Convert to uppercase? (default: False)

As you see, we didn’t need any if __name__ == "__main__", we didn’t have to parse arguments, we just wrote a function, added a decorator to it, and added some annotations to our function’s parameters. As a bonus, we can also use this function directly from a REPL such as Jupyter Notebook - it’s not just for command line scripts!

You should provide a default (after the =) for any optional parameters. If you don’t provide a default for a parameter, then it will be a positional parameter.

WarningBoolean Arguments Default to False

Arguments of type bool or store_true default to False regardless of whether you provide a default or not. Use bool_arg as the type instead of bool if you want to set a default value of True. For example:

@call_parse
def main(
    msg:str="Hi", # The message
    upper:bool_arg=True # Convert to uppercase?
):

Annotated params

If you want to use the full power of argparse, you can do so by using typing.Annotated type hints instead of plain types, like so:

from fastcore.script import *
from typing import Annotated
@call_parse
def main(msg:Annotated[str, "The message"],
         upper:Annotated[store_true, "Convert to uppercase?"]):
    "Print `msg`, optionally converting to uppercase"
    print(msg.upper() if upper else msg)

The first element of Annotated is the parameter’s type, and the first string in its metadata is its help. To use argparse features that docments can’t express, add a dict to the metadata; its keys can be opt,action,nargs,const,choices,required,version. Except for opt, all of these are just passed directly to argparse, so you have all the power of that module at your disposal. opt is a bool that defines whether a param is optional or required (positional) - but you’ll generally not need to set this manually, because fastcore.script will set it for you automatically based on default values.

setuptools scripts

There’s a really nice feature of pip/setuptools that lets you create commandline scripts directly from functions, makes them available in the PATH, and even makes your scripts cross-platform (e.g. in Windows it creates an exe). fastcore.script supports this feature too. The trick to making a function available as a script is to add a console_scripts section to your setup file, of the form: script_name=module:function_name. E.g. in this case we use: test_fastcore.script=fastcore.script.test_cli:main. With this, you can then just type test_fastcore.script at any time, from any directory, and your script will be called (once it’s installed using one of the methods below).

You don’t actually have to write a setup.py yourself. Instead, just use nbdev. Then modify settings.ini as appropriate for your module/script. To install your script directly, you can type pip install -e .. Your script, when installed this way (it’s called an editable install), will automatically be up to date even if you edit it - there’s no need to reinstall it after editing. With nbdev you can even make your module and script available for installation directly from pip and conda by running make release.

API details

---

source

store_true

def store_true():

Placeholder annotation type for a store_true argparse action

---

source

store_false

def store_false():

Placeholder annotation type for a store_false argparse action

---

source

bool_arg

def bool_arg(
    v
):

Annotation type giving bool behavior for CLI args

Each parameter in your function can be annotated Annotated[type, "help"], optionally with a dict of extra argparse arguments as a further metadata element (as described above). You should provide a default (after the =) for any optional parameters. If you don’t provide a default for a parameter, then it will be a positional parameter.

def f(
    required:Annotated[int, "Required param"],
    a:Annotated[bool_arg, "param 1"],
    b:Annotated[str, "param 2"]="test",
):
    "my docs"
    ...

help(f)

Help on function f in module __main__:

f(
    required: int <Required param>,
    a: bool_arg <param 1>,
    b: str <param 2> = 'test'
)
    my docs

test_eq(_arg_kw('a', int, 'help', 1, {}), ('--a', dict(type=int, default=1, help='help (default: 1)')))
test_eq(_arg_kw('a', int, 'help', inspect.Parameter.empty, {}), ('a', dict(type=int, help='help')))
test_eq(_arg_kw('up', bool, 'upper?', True, {}), ('--no-up', dict(action='store_false', default=True, help='upper? (default: True)', dest='up')))
test_eq(_arg_kw('p', str, 'path', inspect.Parameter.empty, dict(opt=False, nargs='?', default='-')),
        ('p', dict(type=str, default='-', help='path (default: -)', nargs='?')))

---

source

anno_parser

def anno_parser(
    func, prog:str=None
):

Look at params (with type/docments/Annotated annotations) in func and return an ArgumentParser

This converts a function with docments and/or Annotated parameter annotations into an argparse.ArgumentParser object. Function arguments with a default provided are optional, and other arguments are positional.

_en = str_enum('_en', 'aa','bb','cc')
def f(
    required:Annotated[int, "Required param"],
    a:Annotated[bool_arg, "param 1"],
    v:Annotated[str, "Print version", dict(action='version', version='%(prog)s 2.0.0')],
    b:Annotated[str, "param 2"]="test",
    c:Annotated[_en, "param 3"]=_en.aa,
):
    "my docs"
    ...

p = anno_parser(f, 'progname')
p.print_help()

usage: progname [-h] [--v] [--b B] [--c {aa,bb,cc}] required a

my docs

positional arguments:
  required        Required param
  a               param 1

options:
  -h, --help      show this help message and exit
  --v             Print version
  --b B           param 2 (default: test)
  --c {aa,bb,cc}  param 3 (default: aa)

We can also check the version and help flags are working.

try: p.parse_args(['--v'])
except: pass

progname 2.0.0

try: p.parse_args(['-h'])
except: pass

usage: progname [-h] [--v] [--b B] [--c {aa,bb,cc}] required a

my docs

positional arguments:
  required        Required param
  a               param 1

options:
  -h, --help      show this help message and exit
  --v             Print version
  --b B           param 2 (default: test)
  --c {aa,bb,cc}  param 3 (default: aa)

It also works with type annotations and docments:

def g(
    required:int,  # Required param
    a:bool_arg,    # param 1
    b="test",      # param 2
    c:_en=_en.aa # param 3
):
    "my docs"
    ...

p = anno_parser(g, 'progname')
p.print_help()

usage: progname [-h] [--b B] [--c {aa,bb,cc}] required a

my docs

positional arguments:
  required        Required param
  a               param 1

options:
  -h, --help      show this help message and exit
  --b B           param 2 (default: test)
  --c {aa,bb,cc}  param 3 (default: aa)

It also works with Union types:

def h(n:int|str, exts:str|list=None):
    "Test union types"

p = anno_parser(h, 'test')
p.print_help()

usage: test [-h] [--exts EXTS] n

Test union types

positional arguments:
  n

options:
  -h, --help   show this help message and exit
  --exts EXTS

test_eq(p.parse_args(['42', '--exts', 'py']).n, 42)
test_eq(p.parse_args(['hello']).n, 'hello')

To use argparse features that docments can’t express, add a dict to the Annotated metadata; its keys are passed through, e.g. nargs:

def j(words:Annotated[str, "Words to join", dict(nargs='+')]):
    "Test nargs"

p = anno_parser(j, 'test')
test_eq(p.parse_args(['a','b']).words, ['a','b'])

Sometimes it’s convenient to extract arguments from the actual name of the called program. args_from_prog will do this, assuming that names and values of the params are separated by a #. Optionally there can also be a prefix separated by ## (double underscore).

---

source

args_from_prog

def args_from_prog(
    func, prog
):

Extract args from prog

exp = {'a': False, 'b': 'baa'}
test_eq(args_from_prog(f, 'foo##a#0#b#baa'), exp)
test_eq(args_from_prog(f, 'a#0#b#baa'), exp)

---

source

set_ctx

def set_ctx(
    cv, val:bool=True
):

Call self as a function.

def _chk(co_file, name='__main__', file='ascript.py', locs=None):
    g = {'__name__':name, '__file__':file, 'inspect':inspect, '_is_script_run':_is_script_run}
    return eval(compile('_is_script_run(inspect.currentframe())', co_file, 'eval'), g, locs)

test_eq(_chk('ascript.py'), True)               # file run directly
test_eq(_chk('ascript.py', name='amod'), False) # imported module
test_eq(_chk('other.py'), False)                # notebook cell: code isn't __main__'s file
test_eq(_chk('ascript.py', locs={}), False)     # not top-level: function/class body in __main__'s file

---

source

call_parse

def call_parse(
    func:NoneType=None, nested:bool=False
):

Decorator to create a simple CLI from func using anno_parser

@call_parse
def test_add(
    a:int=0,  # param a
    b:int=0  # param 1
):
    "Add up `a` and `b`"
    return a + b

call_parse decorated functions work as regular functions and also as command-line interface functions.

test_eq(test_add(1,2), 3)

The decorated function is always the CLI entry point, and only the entry point gets sys.argv: argv is parsed when the function’s file is run directly, or when it’s called with no arguments from the top-level body of a directly-run file (which is how console script wrappers invoke it). Every other call is a plain Python call that ignores argv, whether from a notebook, the REPL, another function, or another call_parse function:

@call_parse
def main(a:int=0): return test_add(a, a)

argv,sys.argv = sys.argv,['prog', '--a', '3']
try: test_eq(main(), 0)  # interactive call: uses defaults, argv untouched
finally: sys.argv = argv

Use the nested keyword argument to create nested parsers, where earlier parsers consume only their known args from sys.argv before later parsers are used. This is useful to create one command line application that executes another. For example:

myrunner --keyword 1 script.py -- <script.py args>

A separating -- after the first application’s args is recommended though not always required, otherwise args may be parsed in unexpected ways. For example:

myrunner script.py -h

would display myrunner’s help and not script.py’s.

call_parse auto-executes the function when its file is run directly, e.g python foo.py or %run foo.py. Importing a module never runs its CLI functions, and neither does defining one in a notebook cell:

ran=[]
@call_parse
def cell_cmd(): ran.append(1)
test_eq(ran, [])

This is the main way to use fastcore.script; decorate your function with call_parse, add type annotations and docments (as shown above), optionally with Annotated metadata for extra argparse features, and it can then be used as a script.

---

source

is_cli

def is_cli(
    func:NoneType=None
):

True if a call_parse CLI run is in progress, optionally checking that func is the function being run

A function usable both from the command line and from Python can call is_cli to tell which way it was invoked, e.g. returning a value to Python callers but printing it (or exiting with an error code) when run as a CLI:

@call_parse
def sum_args(a:int=0, b:int=0):
    "Add `a` and `b`"
    if is_cli(): print(a+b)
    else: return a+b

test_eq(sum_args(1,2), 3)  # Python call: returns the value, prints nothing