Script - CLI

A fast way to turn your python function into a script.

Part of fast.ai’s toolkit for delightful developer experiences.

Overview

Sometimes, you want to create a quick script, either for yourself, or for others. But in Python, that involves a whole lot of boilerplate and ceremony, especially if you want to support command line arguments, provide help, and other niceties. You can use argparse for this purpose, which comes with Python, but it’s complex and verbose.

fastcore.script makes life easier. There are much fancier modules to help you write scripts (we recommend Python Fire, and Click is also popular), but fastcore.script is very fast and very simple. In fact, it’s <50 lines of code! Basically, it’s just a little wrapper around argparse that uses modern Python features and some thoughtful defaults to get rid of the boilerplate.

For full details, see the docs for core.

Example

Here’s a complete example (available in examples/test_fastcore.py):

from fastcore.script import *
@call_parse
def main(msg:str,     # The message
         upper:bool): # Convert to uppercase?
    "Print `msg`, optionally converting to uppercase"
    print(msg.upper() if upper else msg)

If you copy that info a file and run it, you’ll see:

$ examples/test_fastcore.py --help
usage: test_fastcore.py [-h] [--upper] msg

Print `msg`, optionally converting to uppercase

positional arguments:
  msg          The message

optional arguments:
  -h, --help   show this help message and exit
  --upper      Convert to uppercase? (default: False)

As you see, we didn’t need any if __name__ == "__main__", we didn’t have to parse arguments, we just wrote a function, added a decorator to it, and added some annotations to our function’s parameters. As a bonus, we can also use this function directly from a REPL such as Jupyter Notebook - it’s not just for command line scripts!

You should provide a default (after the =) for any optional parameters. If you don’t provide a default for a parameter, then it will be a positional parameter.

Param annotations

If you want to use the full power of argparse, you can do so by using Param annotations instead of type annotations and docments, like so:

from fastcore.script import *
@call_parse
def main(msg:Param("The message", str),
         upper:Param("Convert to uppercase?", store_true)):
    "Print `msg`, optionally converting to uppercase"
    print(msg.upper() if upper else msg)

If you use this approach, then each parameter in your function should have an annotation Param(...) (as in the example above). You can pass the following when calling Param: help,type,opt,action,nargs,const,choices,required . Except for opt, all of these are just passed directly to argparse, so you have all the power of that module at your disposal. Generally you’ll want to pass at least help (since this is provided as the help string for that parameter) and type (to ensure that you get the type of data you expect). opt is a bool that defines whether a param is optional or required (positional) - but you’ll generally not need to set this manually, because fastcore.script will set it for you automatically based on default values.

setuptools scripts

There’s a really nice feature of pip/setuptools that lets you create commandline scripts directly from functions, makes them available in the PATH, and even makes your scripts cross-platform (e.g. in Windows it creates an exe). fastcore.script supports this feature too. The trick to making a function available as a script is to add a console_scripts section to your setup file, of the form: script_name=module:function_name. E.g. in this case we use: test_fastcore.script=fastcore.script.test_cli:main. With this, you can then just type test_fastcore.script at any time, from any directory, and your script will be called (once it’s installed using one of the methods below).

You don’t actually have to write a setup.py yourself. Instead, just use nbdev. Then modify settings.ini as appropriate for your module/script. To install your script directly, you can type pip install -e .. Your script, when installed this way (it’s called an editable install), will automatically be up to date even if you edit it - there’s no need to reinstall it after editing. With nbdev you can even make your module and script available for installation directly from pip and conda by running make release.

API details

from fastcore.test import *

---

store_true

 store_true ()

Placeholder to pass to Param for store_true action

---

store_false

 store_false ()

Placeholder to pass to Param for store_false action

---

bool_arg

 bool_arg (v)

Use as type for Param to get bool behavior

---

clean_type_str

 clean_type_str (x:str)

class Test: pass

test_eq(clean_type_str(argparse.ArgumentParser), 'argparse.ArgumentParser')
test_eq(clean_type_str(Test), 'Test')
test_eq(clean_type_str(int), 'int')
test_eq(clean_type_str(float), 'float')
test_eq(clean_type_str(store_false), 'store_false')

---

Param

 Param (help=None, type=None, opt=True, action=None, nargs=None,
        const=None, choices=None, required=None, default=None)

A parameter in a function used in anno_parser or call_parse

test_eq(repr(Param("Help goes here")), '<Help goes here>')
test_eq(repr(Param("Help", int)), 'int <Help>')
test_eq(repr(Param(help=None, type=int)), 'int')
test_eq(repr(Param(help=None, type=None)), '')

Each parameter in your function should have an annotation Param(...). You can pass the following when calling Param: help,type,opt,action,nargs,const,choices,required (i.e. it takes the same parameters as argparse.ArgumentParser.add_argument, plus opt). Except for opt, all of these are just passed directly to argparse, so you have all the power of that module at your disposal. Generally you’ll want to pass at least help (since this is provided as the help string for that parameter) and type (to ensure that you get the type of data you expect).

opt is a bool that defines whether a param is optional or required (positional) - but you’ll generally not need to set this manually, because fastcore.script will set it for you automatically based on default values. You should provide a default (after the =) for any optional parameters. If you don’t provide a default for a parameter, then it will be a positional parameter.

Param’s __repr__ also allows for more informative function annotation when looking up the function’s doc using shift+tab. You see the type annotation (if there is one) and the accompanying help documentation with it.

def f(required:Param("Required param", int),
      a:Param("param 1", bool_arg),
      b:Param("param 2", str)="test"):
    "my docs"
    ...

help(f)

Help on function f in module __main__:

f(required: int <Required param>, a: bool_arg <param 1>, b: str <param 2> = 'test')
    my docs

p = Param(help="help", type=int)
p.set_default(1)
test_eq(p.kwargs, {'help': 'help (default: 1)', 'type': int, 'default': 1})

---

anno_parser

 anno_parser (func, prog=None, from_name=False)

Look at params (annotated with Param) in func and return an ArgumentParser

This converts a function with parameter annotations of type Param into an argparse.ArgumentParser object. Function arguments with a default provided are optional, and other arguments are positional.

_en = str_enum('_en', 'aa','bb','cc')
def f(required:Param("Required param", int),
      a:Param("param 1", bool_arg),
      b:Param("param 2", str)="test",
      c:Param("param 3", _en)=_en.aa):
    "my docs"
    ...

p = anno_parser(f, 'progname')
p.print_help()

usage: progname [-h] [--b B] [--c {aa,bb,cc}] required a

my docs

positional arguments:
  required        Required param
  a               param 1

optional arguments:
  -h, --help      show this help message and exit
  --b B           param 2 (default: test)
  --c {aa,bb,cc}  param 3 (default: aa)

It also works with type annotations and docments:

def g(required:int,  # Required param
      a:bool_arg,    # param 1
      b="test",      # param 2
      c:_en=_en.aa): # param 3
    "my docs"
    ...

p = anno_parser(g, 'progname')
p.print_help()

usage: progname [-h] [--b B] [--c {aa,bb,cc}] required a

my docs

positional arguments:
  required        Required param
  a               param 1

optional arguments:
  -h, --help      show this help message and exit
  --b B           param 2 (default: test)
  --c {aa,bb,cc}  param 3 (default: aa)

---

args_from_prog

 args_from_prog (func, prog)

Extract args from prog

Sometimes it’s convenient to extract arguments from the actual name of the called program. args_from_prog will do this, assuming that names and values of the params are separated by a #. Optionally there can also be a prefix separated by ## (double underscore).

exp = {'a': False, 'b': 'baa'}
test_eq(args_from_prog(f, 'foo##a#0#b#baa'), exp)
test_eq(args_from_prog(f, 'a#0#b#baa'), exp)

---

call_parse

 call_parse (func=None, nested=False)

Decorator to create a simple CLI from func using anno_parser

@call_parse
def test_add(
    a:int,  # param a
    b:int  # param 1
): return a + b

call_parse decorated functions work as regular functions and also as command-line interface functions.

test_eq(test_add(1,2), 3)

This is the main way to use fastcore.script; decorate your function with call_parse, add Param annotations (as shown above) or type annotations and docments, and it can then be used as a script.

Use the nested keyword argument to create nested parsers, where earlier parsers consume only their known args from sys.argv before later parsers are used. This is useful to create one command line application that executes another. For example:

myrunner --keyword 1 script.py -- <script.py args>

A separating -- after the first application’s args is recommended though not always required, otherwise args may be parsed in unexpected ways. For example:

myrunner script.py -h

would display myrunner’s help and not script.py’s.