Basic functionality

Basic functionality used in the fastai library

This module has an unusually large API surface, so the symbol listing in doc(fastcore.basics) is suppressed via __pyskill_sigs__: the exported cells below assemble into a curated module docstring instead.

Basics

The staples used throughout fastai code, each replacing a common multi-line pattern with a one-liner: ifnone(a,b) is b if a is None else a (though both args are always evaluated); listify and tuplify convert anything to a list or tuple the way you’d mean it (None becomes [], a str or dict stays a single item, a generator is consumed); and basic_repr('a,b') gives a class a deterministic key=value repr with no memory address, so notebook and git diffs stay clean.

ifnone

def ifnone(
    a, b
):

b if a is None else a

Since b if a is None else a is such a common pattern, we wrap it in a function. However, be careful, because python will evaluate both a and b when calling ifnone (which it doesn’t do if using the if version directly).

test_eq(ifnone(None,1), 1)
test_eq(ifnone(2   ,1), 2)

req

def req(
    x, msg:NoneType=None, exc:type=ValueError
):

x unless it’s None, in which case raise exc(msg)

The complement of ifnone for the cases where None is not an acceptable answer: many lookups signal “not found” with None (first on an empty sequence, dict.get, a C API returning a null handle), and the caller’s next line is often if x is None: raise .... req folds that guard into the lookup expression itself.

test_eq(req(2, 'missing'), 2)
test_eq(req(False, 'missing'), False)
test_fail(lambda: req(None, 'missing'), contains='missing')
test_fail(lambda: req(None, 'gone', exc=KeyError), exc=KeyError)

maybe_attr

def maybe_attr(
    o, attr
):

getattr(o,attr,o)

Return the attribute attr for object o. If the attribute doesn’t exist, then return the object o instead.

class myobj: myattr='foo'

test_eq(maybe_attr(myobj, 'myattr'), 'foo')
test_eq(maybe_attr(myobj, 'another_attr'), myobj)

basic_repr

def basic_repr(
    flds:NoneType=None
):

Minimal __repr__

In types which provide rich display functionality in Jupyter, their __repr__ is also called in order to provide a fallback text representation. Unfortunately, this includes a memory address which changes on every invocation, making it non-deterministic. This causes diffs to get messy and creates conflicts in git. To fix this, put __repr__=basic_repr() inside your class.

class SomeClass: __repr__=basic_repr()
repr(SomeClass())

'SomeClass()'

If you pass a list of attributes (flds) of an object, then this will generate a string with the name of each attribute and its corresponding value. The format of this string is key=value, where key is the name of the attribute, and value is the value of the attribute. For each value, attempt to use the __name__ attribute, otherwise fall back to using the value’s __repr__ when constructing the string.

class SomeClass:
    a=1
    b='foo'
    __repr__=basic_repr('a,b')

repr(SomeClass())

"SomeClass(a=1, b='foo')"

Nested objects work too:

class AnotherClass:
    c=SomeClass()
    d='bar'
    __repr__=basic_repr(['c', 'd'])

repr(AnotherClass())

"AnotherClass(c=SomeClass(a=1, b='foo'), d='bar')"

Instance variables (but not class variables) are shown if basic_repr is called with no arguments:

class SomeClass:
    def __init__(self, a=1, b='foo'): self.a,self.b = a,b
    __repr__=basic_repr()

repr(SomeClass())

"SomeClass(a=1, b='foo')"

BasicRepr

def BasicRepr(
    *args, **kwargs
):

Base class for objects needing a basic __repr__

As a shortcut for creating a __repr__ for instance variables, you can inherit from BasicRepr:

class SomeClass(BasicRepr):
    def __init__(self, a=1, b='foo'): self.a,self.b = a,b

repr(SomeClass())

"SomeClass(a=1, b='foo')"

is_array

def is_array(
    x
):

True if x supports __array__ or iloc

is_array(np.array(1)),is_array([1])

(True, False)

listify

def listify(
    o:NoneType=None, *rest, use_list:bool=False, match:NoneType=None
):

Convert o to a list

Conversion is designed to “do what you mean”, e.g:

test_eq(listify('hi'), ['hi'])
test_eq(listify(b'hi'), [b'hi'])
test_eq(listify(array(1)), [array(1)])
test_eq(listify(1), [1])
test_eq(listify([1,2]), [1,2])
test_eq(listify(range(3)), [0,1,2])
test_eq(listify(None), [])
test_eq(listify(1,2), [1,2])

arr = np.arange(9).reshape(3,3)
listify(arr)

[array([[0, 1, 2],
        [3, 4, 5],
        [6, 7, 8]])]

listify(array([1,2]))

[array([1, 2])]

Generators are turned into lists too:

gen = (o for o in range(3))
test_eq(listify(gen), [0,1,2])

Use match to provide a length to match:

test_eq(listify(1,match=3), [1,1,1])

If match is a sequence, it’s length is used:

test_eq(listify(1,match=range(3)), [1,1,1])

If the listified item is not of length 1, it must be the same length as match:

test_eq(listify([1,1,1],match=3), [1,1,1])
with expect_fail(): listify([1,1],match=3)

tuplify

def tuplify(
    o, use_list:bool=False, match:NoneType=None
):

Make o a tuple

test_eq(tuplify(None),())
test_eq(tuplify([1,2,3]),(1,2,3))
test_eq(tuplify(1,match=[1,2,3]),(1,1,1))

true

def true(
    x
):

Test whether x is truthy; collections with >0 elements are considered True

[(o,true(o)) for o in
 (array(0),array(1),array([0]),array([0,1]),1,0,'',None)]

[(array(0), False),
 (array(1), True),
 (array([0]), True),
 (array([0, 1]), True),
 (1, True),
 (0, False),
 ('', False),
 (None, False)]

NullType

def NullType(
    *args, **kwargs
):

An object that is False and can be called, chained, and indexed

bool(null.hi().there[3])

False

tonull

def tonull(
    x
):

Convert None to null

bool(tonull(None).hi().there[3])

False

get_class

def get_class(
    nm, *fld_names, sup:NoneType=None, doc:NoneType=None, funcs:NoneType=None, anno:NoneType=None, **flds
):

Dynamically create a class, optionally inheriting from sup, containing fld_names

_t = get_class('_t', 'a', b=2, anno={'b':int})
t = _t()
test_eq(t.a, None)
test_eq(t.b, 2)
t = _t(1, b=3)
test_eq(t.a, 1)
test_eq(t.b, 3)
t = _t(1, 3)
test_eq(t.a, 1)
test_eq(t.b, 3)
test_eq(t, pickle.loads(pickle.dumps(t)))
test_eq(_t.__annotations__, {'b':int, 'a':typing.Any})
repr(t)

'_t(a=1, b=3)'

Most often you’ll want to call mk_class, since it adds the class to your module. See mk_class for more details and examples of use (which also apply to get_class).

mk_class

def mk_class(
    nm, *fld_names, sup:NoneType=None, doc:NoneType=None, funcs:NoneType=None, mod:NoneType=None, anno:NoneType=None,
    **flds
):

Create a class using get_class and add to the caller’s module

Any kwargs will be added as class attributes, and sup is an optional (tuple of) base classes.

mk_class('_t', a=1, sup=dict)
t = _t()
test_eq(t.a, 1)
assert(isinstance(t,dict))

A __init__ is provided that sets attrs for any kwargs, and for any args (matching by position to fields), along with a __repr__ which prints all attrs. The docstring is set to doc. You can pass funcs which will be added as attrs with the function names.

def foo(self): return 1
mk_class('_t', 'a', sup=dict, doc='test doc', funcs=foo)

t = _t(3, b=2)
test_eq(t.a, 3)
test_eq(t.b, 2)
test_eq(t.foo(), 1)
test_eq(t.__doc__, 'test doc')
t

{}

wrap_class

def wrap_class(
    nm, *fld_names, sup:NoneType=None, doc:NoneType=None, funcs:NoneType=None, **flds
):

Decorator: makes function a method of a new class nm passing parameters to mk_class

@wrap_class('_t', a=2)
def bar(self,x): return x+1

t = _t()
test_eq(t.a, 2)
test_eq(t.bar(3), 4)

ignore_exceptions

def ignore_exceptions(
    *args, **kwargs
):

Context manager to ignore exceptions

with ignore_exceptions(): 
    # Exception will be ignored
    raise Exception

exec_local

def exec_local(
    code, var_name
):

Call exec on code and return the var var_name

test_eq(exec_local("a=1", "a"), 1)

risinstance

def risinstance(
    types, obj:NoneType=None
):

Curried isinstance but with args reversed

assert risinstance(int, 1)
assert not risinstance(str, 0)
assert risinstance(int)(1)
assert not risinstance(int)(None)

types can also be strings:

assert risinstance(('str','int'), 'a')
assert risinstance('str', 'a')
assert not risinstance('int', 'a')

ver2tuple

def ver2tuple(
    v:str
)->tuple:

Call self as a function.

test_eq(ver2tuple('3.8.1'), (3,8,1))
test_eq(ver2tuple('3.1'), (3,1,0))
test_eq(ver2tuple('3.'), (3,0,0))
test_eq(ver2tuple('3'), (3,0,0))

NoOp

These are used when you need a pass-through function.

noop

def noop(
    x:NoneType=None, *args, **kwargs
):

Do nothing

noop()
test_eq(noop(1),1)

noops

def noops(
    x:NoneType=None, *args, **kwargs
):

Do nothing (method)

class _t: foo=noops
test_eq(_t().foo(1),1)

Infinite Lists

These lists are useful for things like padding an array or adding index column(s) to arrays.

Inf defines the following properties:

count: itertools.count()
zeros: itertools.cycle([0])
ones : itertools.cycle([1])
nones: itertools.cycle([None])

test_eq([o for i,o in zip(range(5), Inf.count)],
        [0, 1, 2, 3, 4])

test_eq([o for i,o in zip(range(5), Inf.zeros)],
        [0]*5)

test_eq([o for i,o in zip(range(5), Inf.ones)],
        [1]*5)

test_eq([o for i,o in zip(range(5), Inf.nones)],
        [None]*5)

Operator Functions

Curried versions of the comparison and arithmetic functions in Python’s operator module: lt gt le ge eq ne add sub mul truediv is_ is_not in_ mod. With two args they work like operator’s versions; with one arg they return a partial that binds it as the second argument, so lt(3) means “is less than 3” and in_(vals) means “is contained in vals”. They read especially well as cmp arguments to fastcore.test.test, e.g. test(x, valid, in_).

in_

def in_(
    x, a
):

True if x in a

# test if element is in another
assert in_('c', ('b', 'c', 'a'))
assert in_(4, [2,3,4,5])
assert in_('t', 'fastai')
assert not in_('h', 'fastai')

# use in_ as a partial
assert in_('fastai')('t')
assert in_([2,3,4,5])(4)
assert not in_('fastai')('h')

In addition to in_, the following functions are provided matching the behavior of the equivalent versions in operator: lt gt le ge eq ne add sub mul truediv is_ is_not mod.

lt(3,5),gt(3,5),is_(None,None),in_(0,[1,2]),mod(3,2)

(True, False, True, False, 1)

Similarly to in_, they also have additional functionality: if you only pass one param, they return a partial function that passes that param as the second positional parameter.

lt(5)(3),gt(5)(3),is_(None)(None),in_([1,2])(0),mod(2)(3)

(True, False, True, False, 1)

ret_true

def ret_true(
    *args, **kwargs
):

Predicate: always True

assert ret_true(1,2,3)
assert ret_true(False)

ret_false

def ret_false(
    *args, **kwargs
):

Predicate: always False

stop

def stop(
    e:type=StopIteration
):

Raises exception e (by default StopIteration)

gen

def gen(
    func, seq, cond:function=ret_true
):

Like (func(o) for o in seq if cond(func(o))) but handles StopIteration

test_eq(gen(noop, Inf.count, lt(5)),
        range(5))
test_eq(gen(operator.neg, Inf.count, gt(-5)),
        [0,-1,-2,-3,-4])
test_eq(gen(lambda o:o if o<5 else stop(), Inf.count),
        range(5))

chunked

def chunked(
    it, chunk_sz:NoneType=None, drop_last:bool=False, n_chunks:NoneType=None, pad:bool=False, pad_val:NoneType=None
):

Return batches from iterator it of size chunk_sz (or return n_chunks total)

Note that you must pass either chunk_sz, or n_chunks, but not both.

t = list(range(10))
test_eq(chunked(t,3),      [[0,1,2], [3,4,5], [6,7,8], [9]])
test_eq(chunked(t,3,True), [[0,1,2], [3,4,5], [6,7,8],    ])

t = map(lambda o:stop() if o==6 else o, Inf.count)
test_eq(chunked(t,3), [[0, 1, 2], [3, 4, 5]])
t = map(lambda o:stop() if o==7 else o, Inf.count)
test_eq(chunked(t,3), [[0, 1, 2], [3, 4, 5], [6]])

t = np.arange(10)
test_eq(chunked(t,3),      [[0,1,2], [3,4,5], [6,7,8], [9]])
test_eq(chunked(t,3,True), [[0,1,2], [3,4,5], [6,7,8],    ])

test_eq(chunked([], 3),          [])
test_eq(chunked([], n_chunks=3), [])

Pass pad=True and an optional pad_val to pad the last chunk:

t = list(range(10))
test_eq(chunked(t,3,pad=True), [[0,1,2], [3,4,5], [6,7,8], [9,None,None]])
test_eq(chunked(t,3,pad=True,pad_val=0), [[0,1,2], [3,4,5], [6,7,8], [9,0,0]])
test_eq(chunked(t,4,pad=True,pad_val=-1), [[0,1,2,3], [4,5,6,7], [8,9,-1,-1]])
test_eq(chunked(range(5),2,pad=True), [[0,1], [2,3], [4,None]])

otherwise

def otherwise(
    x, tst, y
):

y if tst(x) else x

test_eq(otherwise(2+1, gt(3), 4), 3)
test_eq(otherwise(2+1, gt(2), 4), 4)

Attribute Helpers

AttrDict is a dict whose keys are also attributes, so d.foo reads and writes d['foo'] (to convert a whole nested structure at once, see dict2obj in fastcore.xtras); NS is the same idea built on SimpleNamespace, adding indexing and iteration. store_attr(), called inside __init__, stores the function’s arguments as attributes of self in one line:

These functions reduce boilerplate when setting or manipulating attributes or properties of objects.

custom_dir

def custom_dir(
    c, add
):

Implement custom __dir__, adding add to cls

custom_dir allows you extract the __dict__ property of a class and appends the list add to it.

class _T: 
    def f(): pass

s = custom_dir(_T(), add=['foo', 'bar'])
assert {'foo', 'bar', 'f'}.issubset(s)

adict

def adict(
    *args, **kwargs
):

dict subclass that also provides access to keys as attrs

d = adict(a=1,b="two")
test_eq(d.a, 1)
test_eq(d['b'], 'two')
test_eq(d.get('c','nope'), 'nope')
d.b = 2
test_eq(d.b, 2)
test_eq(d['b'], 2)
d['b'] = 3
test_eq(d['b'], 3)
test_eq(d.b, 3)
assert 'a' in dir(d)

AttrDict

def AttrDict(
    *args, **kwargs
):

dict subclass that also provides access to keys as attrs, and has a pretty markdown repr

AttrDict will pretty print in Jupyter Notebooks:

d = AttrDict(a=1,b="two")

_test_dict = {'a':1, 'b': {'c':1, 'd':2}, 'c': {'c':1, 'd':2}, 'd': {'c':1, 'd':2},
              'e': {'c':1, 'd':2}, 'f': {'c':1, 'd':2, 'e': 4, 'f':[1,2,3,4,5]}}
AttrDict(_test_dict)

{ 'a': 1,
  'b': {'c': 1, 'd': 2},
  'c': {'c': 1, 'd': 2},
  'd': {'c': 1, 'd': 2},
  'e': {'c': 1, 'd': 2},
  'f': {'c': 1, 'd': 2, 'e': 4, 'f': [1, 2, 3, 4, 5]}}

AttrDictDefault

def AttrDictDefault(
    *args, default_:NoneType=None, **kwargs
):

AttrDict subclass that returns default_ for missing attrs

d = AttrDictDefault(a=1,b="two", default_='nope')
test_eq(d.a, 1)
test_eq(d['b'], 'two')
test_eq(d.c, 'nope')

NS

def NS(
    *args, **kwargs
):

SimpleNamespace subclass that also adds iter and dict support

This is very similar to AttrDict, but since it starts with SimpleNamespace, it has some differences in behavior. You can use it just like SimpleNamespace:

d = NS(**_test_dict)
d

namespace(a=1,
          b={'c': 1, 'd': 2},
          c={'c': 1, 'd': 2},
          d={'c': 1, 'd': 2},
          e={'c': 1, 'd': 2},
          f={'c': 1, 'd': 2, 'e': 4, 'f': [1, 2, 3, 4, 5]})

…but you can also index it to get/set:

d['a']

…and iterate t:

list(d)

['a', 'b', 'c', 'd', 'e', 'f']

get_annotations_ex

def get_annotations_ex(
    obj, globals:NoneType=None, locals:NoneType=None
):

Backport of py3.10 get_annotations that returns globals/locals

In Python 3.10 inspect.get_annotations was added. However previous versions of Python are unable to evaluate type annotations correctly if from future import __annotations__ is used. Furthermore, all annotations are evaluated, even if only some subset are needed. get_annotations_ex provides the same functionality as inspect.get_annotations, but works on earlier versions of Python, and returns the globals and locals needed to evaluate types.

eval_type

def eval_type(
    t, glb, loc
):

eval a type or collection of types, if needed, for annotations in py3.10+

In py3.10, or if from future import __annotations__ is used, a is a str:

class _T2a: pass
def func(a: _T2a): pass
ann,glb,loc = get_annotations_ex(func)

eval_type(ann['a'], glb, loc)

__main__._T2a

| is supported for defining Union types when using eval_type even for python versions prior to 3.9:

class _T2b: pass
def func(a: _T2a|_T2b): pass
ann,glb,loc = get_annotations_ex(func)

eval_type(ann['a'], glb, loc)

__main__._T2a | __main__._T2b

type_hints

def type_hints(
    f
):

Like typing.get_type_hints but returns {} if not allowed type

def type_hints(f):
    "Like `typing.get_type_hints` but returns `{}` if not allowed type"
    if not isinstance(f, _allowed_types): return {}
    ann,glb,loc = get_annotations_ex(f)
    return {k:_eval_type(v,glb,loc) for k,v in ann.items()}

For example, type func is allowed so type_hints returns the same value as typing.get_hints:

def f(a:int)->bool: ... # a function with type hints (allowed)
exp = {'a':int,'return':bool}
test_eq(type_hints(f), typing.get_type_hints(f))
test_eq(type_hints(f), exp)

However, class is not an allowed type, so type_hints returns {}:

class _T:
    def __init__(self, a:int=0)->bool: ...
assert not type_hints(_T)

annotations

def annotations(
    o
):

Annotations for o, or type(o)

This supports a wider range of situations than type_hints, by checking type() and __init__ for annotations too:

for o in _T,_T(),_T.__init__,f: test_eq(annotations(o), exp)
assert not annotations(int)
assert not annotations(print)

anno_ret

def anno_ret(
    func
):

Get the return annotation of func

def f(x) -> float: return x
test_eq(anno_ret(f), float)

def f(x) -> typing.Tuple[float,float]: return x
assert anno_ret(f)==typing.Tuple[float,float]

If your return annotation is None, anno_ret will return NoneType (and not None):

def f(x) -> None: return x

test_eq(anno_ret(f), NoneType)
assert anno_ret(f) is not None # returns NoneType instead of None

If your function does not have a return type, or if you pass in None instead of a function, then anno_ret returns None:

def f(x): return x

test_eq(anno_ret(f), None)
test_eq(anno_ret(None), None) # instead of passing in a func, pass in None

signature_ex

def signature_ex(
    obj, eval_str:bool=False
):

Like inspect.signature, but a declared Parameter.empty default is kept as a default

inspect uses Parameter.empty both as its “no default” marker and as a value a function may declare as an actual default (a common sentinel idiom, e.g. fasthtml.core.add_sig_param). Plain signature() reads such a declaration back as “no default”, producing a signature that fails validation as soon as it is rebuilt, e.g. by Signature.replace. signature_ex keeps the declared default distinct, so the signature stays valid and renders as the source declared it:

def _f(a, b=1, c=Parameter.empty, *, d=Parameter.empty): ...
with expect_fail(ValueError, contains='non-default'): signature(_f).replace()
sig = signature_ex(_f)
assert sig.parameters['c'].default is not Parameter.empty
assert sig.parameters['d'].default is not Parameter.empty
sig.replace()
sig

union2tuple

def union2tuple(
    t
):

Call self as a function.

test_eq(union2tuple(Union[int,str]), (int,str))
test_eq(union2tuple(int), int)
assert union2tuple(Tuple[int,str])==Tuple[int,str]
test_eq(union2tuple((int,str)), (int,str))
if UnionType: test_eq(union2tuple(int|str), (int,str))

argnames

def argnames(
    f, frame:bool=False
):

Names of arguments to function or frame f

test_eq(argnames(f), ['x'])

with_cast

def with_cast(
    f
):

Decorator which uses any parameter annotations as preprocessing functions

@with_cast
def _f(a, b:Path, c:str='', d=0): return (a,b,c,d)

test_eq(_f(1, '.', 3), (1,Path('.'),'3',0))
test_eq(_f(1, '.'), (1,Path('.'),'',0))

@with_cast
def _g(a:int=0)->str: return a

test_eq(_g(4.0), '4')
test_eq(_g(4.4), '4')
test_eq(_g(2), '2')

store_attr

def store_attr(
    names:NoneType=None, but:str='', cast:bool=False, store_args:NoneType=None, **attrs
):

Store params named in comma-separated names from calling context into attrs in self

class Point:
    def __init__(self, x, y, scale=1): store_attr()
p = Point(1, 2)
test_eq((p.x, p.y, p.scale), (1, 2, 1))

In it’s most basic form, you can use store_attr to shorten code like this:

class T:
    def __init__(self, a,b,c): self.a,self.b,self.c = a,b,c

…to this:

class T:
    def __init__(self, a,b,c): store_attr('a,b,c', self)

This class behaves as if we’d used the first form:

t = T(1,c=2,b=3)
assert t.a==1 and t.b==3 and t.c==2

class T1:
    def __init__(self, a,b,c): store_attr()

In addition, it stores the attrs as a dict in __stored_args__, which you can use for display, logging, and so forth.

test_eq(t.__stored_args__, {'a':1, 'b':3, 'c':2})

Since you normally want to use the first argument (often called self) for storing attributes, it’s optional:

class T:
    def __init__(self, a,b,c:str): store_attr('a,b,c')

t = T(1,c=2,b=3)
assert t.a==1 and t.b==3 and t.c==2

With cast=True any parameter annotations will be used as preprocessing functions for the corresponding arguments:

class T:
    def __init__(self, a:listify, b, c:str): store_attr('a,b,c', cast=True)

t = T(1,c=2,b=3)
assert t.a==[1] and t.b==3 and t.c=='2'

You can inherit from a class using store_attr, and just call it again to add in any new attributes added in the derived class:

class T2(T):
    def __init__(self, d, **kwargs):
        super().__init__(**kwargs)
        store_attr('d')

t = T2(d=1,a=2,b=3,c=4)
assert t.a==2 and t.b==3 and t.c==4 and t.d==1

You can skip passing a list of attrs to store. In this case, all arguments passed to the method are stored:

class T:
    def __init__(self, a,b,c): store_attr()

t = T(1,c=2,b=3)
assert t.a==1 and t.b==3 and t.c==2

class T4(T):
    def __init__(self, d, **kwargs):
        super().__init__(**kwargs)
        store_attr()

t = T4(4, a=1,c=2,b=3)
assert t.a==1 and t.b==3 and t.c==2 and t.d==4

class T4:
    def __init__(self, *, a: int, b: float = 1):
        store_attr()
        
t = T4(a=3)
assert t.a==3 and t.b==1
t = T4(a=3, b=2)
assert t.a==3 and t.b==2

You can skip some attrs by passing but:

class T:
    def __init__(self, a,b,c): store_attr(but='a')

t = T(1,c=2,b=3)
assert t.b==3 and t.c==2
assert not hasattr(t,'a')

You can also pass keywords to store_attr, which is identical to setting the attrs directly, but also stores them in __stored_args__.

class T:
    def __init__(self): store_attr(a=1)

t = T()
assert t.a==1

You can also use store_attr inside functions.

def create_T(a, b):
    t = SimpleNamespace()
    store_attr(self=t)
    return t

t = create_T(a=1, b=2)
assert t.a==1 and t.b==2

attrdict

def attrdict(
    o, *ks, default:NoneType=None
):

Dict from each k in ks to getattr(o,k)

class T:
    def __init__(self, a,b,c): store_attr()

t = T(1,c=2,b=3)
test_eq(attrdict(t,'b','c'), {'b':3, 'c':2})

properties

def properties(
    cls, *ps
):

Change attrs in cls with names in ps to properties

class T:
    def a(self): return 1
    def b(self): return 2
properties(T,'a')

test_eq(T().a,1)
test_eq(T().b(),2)

The case-convention functions below share one splitter. id_words segments an identifier into its words and never changes case: punctuation characters in splits are treated as separators, and the letter c enables splitting at case boundaries, including after acronym runs. Constrain splits when a convention gives characters different meanings, such as CSS pseudo-selectors where : must not split. Separator runs come back as empty words, so names like __init__ survive a round trip.

id_words

def id_words(
    s:str, splits:str='-_ :c'
)->list:

Split identifier s into words: punctuation chars in splits are separators (runs kept as empty words, so they survive a round trip), ‘c’ splits case boundaries

test_eq(id_words('beforeRequest'), ['before','Request'])
test_eq(id_words('validation:validate'), ['validation','validate'])
test_eq(id_words('HTMLElement'), ['HTML','Element'])
test_eq(id_words('HX-Request-Type'), ['HX','Request','Type'])
test_eq(id_words('a__b'), ['a','','b'])
test_eq(id_words('a-b_c', 'c'), ['a-b_c'])
id_words('parseHTMLTree')

Each to_* emitter owns its casing policy: to_camel keeps acronym words intact (so an already-camel name round-trips exactly, which HtmxOn-style consumers rely on), to_pascal Title-cases every word (the historic snake2camel output), and to_kebab/to_snake lowercase throughout.

to_snake

def to_snake(
    s:str, splits:str='-_ :c'
)->str:

snake_case form of s

to_kebab

def to_kebab(
    s:str, splits:str='-_ :c'
)->str:

kebab-case form of s

to_pascal

def to_pascal(
    s:str, splits:str='-_ :c'
)->str:

PascalCase form of s; every word Title-cased (acronyms flatten)

to_camel

def to_camel(
    s:str, splits:str='-_ :c'
)->str:

camelCase form of s; acronym words after the first keep their casing

test_eq(to_camel('before-request'), 'beforeRequest')
test_eq(to_camel('parseHTMLTree'), 'parseHTMLTree')
test_eq(to_pascal('a_b_cc'), 'ABCc')
test_eq(to_pascal('parseHTMLTree'), 'ParseHtmlTree')
test_eq(to_kebab('beforeRequest'), 'before-request')
test_eq(to_kebab('noSSESourceError'), 'no-sse-source-error')
test_eq(to_snake('HX-Request-Type'), 'hx_request_type')
test_eq(to_snake('__init__'), '__init__')
test_eq(to_kebab('a__b'), 'a--b')
to_camel('no-sse-source-error'), to_kebab('validation:validate')

camel2words

def camel2words(
    s, space:str=' '
):

Convert CamelCase to ‘spaced words’

test_eq(camel2words('ClassAreCamel'), 'Class Are Camel')

camel2snake

def camel2snake(
    name
):

Convert CamelCase to snake_case

test_eq(camel2snake('ClassAreCamel'), 'class_are_camel')
test_eq(camel2snake('Already_Snake'), 'already_snake')

snake2camel

def snake2camel(
    s
):

Convert snake_case to CamelCase

test_eq(snake2camel('a_b_cc'), 'ABCc')

humanize

def humanize(
    x
):

Concise human-readable x, e.g. 9200 -> ‘9.2k’

test_eq(humanize(950), '950')
test_eq(humanize(0), '0')
test_eq(humanize(9200), '9.2k')
test_eq(humanize(61234), '61.2k')
test_eq(humanize(1_300_000), '1.3M')
test_eq(humanize(-2500), '-2.5k')
test_eq(humanize(2_000_000_000_000_000), '2000T')

class2attr

def class2attr(
    cls_name
):

Return the snake-cased name of the class; strip ending cls_name if it exists.

class Parent:
    @property
    def name(self): return class2attr(self, 'Parent')

class ChildOfParent(Parent): pass
class ParentChildOf(Parent): pass

p = Parent()
cp = ChildOfParent()
cp2 = ParentChildOf()

test_eq(p.name, 'parent')
test_eq(cp.name, 'child_of')
test_eq(cp2.name, 'parent_child_of')

getcallable

def getcallable(
    o, attr
):

Calls getattr with a default of noop

class Math:
    def addition(self,a,b): return a+b

m = Math()

test_eq(getcallable(m, "addition")(a=1,b=2), 3)
test_eq(getcallable(m, "subtraction")(a=1,b=2), None)

getattrs

def getattrs(
    o, *attrs, default:NoneType=None
):

List of all attrs in o

from fractions import Fraction

getattrs(Fraction(1,2), 'numerator', 'denominator')

[1, 2]

hasattrs

def hasattrs(
    o, attrs
):

Test whether o contains all attrs

assert hasattrs(1,('imag','real'))
assert not hasattrs(1,('imag','foo'))

setattrs

def setattrs(
    dest, flds, src
):

Call self as a function.

d = dict(a=1,bb="2",ignore=3)
o = SimpleNamespace()
setattrs(o, "a,bb", d)
test_eq(o.a, 1)
test_eq(o.bb, "2")

d = SimpleNamespace(a=1,bb="2",ignore=3)
o = SimpleNamespace()
setattrs(o, "a,bb", d)
test_eq(o.a, 1)
test_eq(o.bb, "2")

try_attrs

def try_attrs(
    obj, *attrs
):

Return first attr that exists in obj

test_eq(try_attrs(1, 'real'), 1)
test_eq(try_attrs(1, 'foobar', 'real'), 1)

DepProp

def DepProp(
    fchange, fnorm:NoneType=None
):

Property decorator with dependency update triggering

DepProp is a descriptor that stores a value and calls a “change” function whenever that value changes or is deleted. This is useful for invalidating caches or triggering side effects when a dependency is updated. An optional normalizer can preprocess values before storage.

class Square:
    @DepProp
    def width(self): self._area_cache = None

    @property
    def area(self):
        if self._area_cache is None: self._area_cache = self.width**2
        return self._area_cache

r = Square()
r.width = 3
test_eq(r.area, 9)
r.width = 5  # cache cleared automatically
test_eq(r.area, 25)

Use the .norm decorator to add a normalizer that preprocesses values before storage:

class T:
    @DepProp
    def name(self): self.log = f'changed to {self.name}'

    @name.norm
    def name(self, v): return v.strip().lower()

t = T()
t.name = '  Hello '
test_eq(t.name, 'hello')
test_eq(t.log, 'changed to hello')

Setting the same value again does not trigger the change function:

t.log = 'not called'
t.name = 'hello'  # same value -- no change
test_eq(t.log, 'not called')

Deleting the property removes the backing attribute and calls fchange:

del t.name
test_eq(t.log, 'changed to None')
test_eq(t.name, None)

Attribute Delegation

GetAttrBase

def GetAttrBase(
    *args, **kwargs
):

Basic delegation of __getattr__ and __dir__

GetAttr

def GetAttr(
    *args, **kwargs
):

Inherit from this to have all attr accesses in self._xtra passed down to self.default

Inherit from GetAttr to have attr access passed down to an instance attribute. This makes it easy to create composites that don’t require callers to know about their components. For a more detailed discussion of how this works as well as relevant context, we suggest reading the delegated composition section of this blog article.

You can customise the behaviour of GetAttr in subclasses via; - _default - By default, this is set to 'default', so attr access is passed down to self.default - _default can be set to the name of any instance attribute that does not start with dunder __ - _xtra - By default, this is None, so all attr access is passed down - You can limit which attrs get passed down by setting _xtra to a list of attribute names

To illuminate the utility of GetAttr, suppose we have the following two classes, _WebPage which is a superclass of _ProductPage, which we wish to compose like so:

class _WebPage:
    def __init__(self, title, author="Jeremy"):
        self.title,self.author = title,author

class _ProductPage:
    def __init__(self, page, price): self.page,self.price = page,price
        
page = _WebPage('Soap', author="Sylvain")
p = _ProductPage(page, 15.0)

How do we make it so we can just write p.author, instead of p.page.author to access the author attribute? We can use GetAttr, of course! First, we subclass GetAttr when defining _ProductPage. Next, we set self.default to the object whose attributes we want to be able to access directly, which in this case is the page argument passed on initialization:

class _ProductPage(GetAttr):
    def __init__(self, page, price): self.default,self.price = page,price #self.default allows you to access page directly.

p = _ProductPage(page, 15.0)

Now, we can access the author attribute directly from the instance:

test_eq(p.author, 'Sylvain')

If you wish to store the object you are composing in an attribute other than self.default, you can set the class attribute _data as shown below. This is useful in the case where you might have a name collision with self.default:

class _C(GetAttr):
    _default = '_data' # use different component name; `self._data` rather than `self.default`
    def __init__(self,a): self._data = a
    def foo(self): noop

t = _C('Hi')
test_eq(t._data, 'Hi') 
with expect_fail(): t.default # we no longer have self.default
test_eq(t.lower(), 'hi')
test_eq(t.upper(), 'HI')
assert 'lower' in dir(t)
assert 'upper' in dir(t)

By default, all attributes and methods of the object you are composing are retained. In the below example, we compose a str object with the class _C. This allows us to directly call string methods on instances of class _C, such as str.lower() or str.upper():

class _C(GetAttr):
    # allow all attributes and methods to get passed to `self.default` (by leaving _xtra=None)
    def __init__(self,a): self.default = a
    def foo(self): noop

t = _C('Hi')
test_eq(t.lower(), 'hi')
test_eq(t.upper(), 'HI')
assert 'lower' in dir(t)
assert 'upper' in dir(t)

However, you can choose which attributes or methods to retain by defining a class attribute _xtra, which is a list of allowed attribute and method names to delegate. In the below example, we only delegate the lower method from the composed str object when defining class _C:

class _C(GetAttr):
    _xtra = ['lower'] # specify which attributes get passed to `self.default`
    def __init__(self,a): self.default = a
    def foo(self): noop

t = _C('Hi')
test_eq(t.default, 'Hi')
test_eq(t.lower(), 'hi')
with expect_fail(): t.upper() # upper wasn't in _xtra, so it isn't available to be called
assert 'lower' in dir(t)
assert 'upper' not in dir(t)

You must be careful to properly set an instance attribute in __init__ that corresponds to the class attribute _default. The below example sets the class attribute _default to data, but erroneously fails to define self.data (and instead defines self.default).

Failing to properly set instance attributes leads to errors when you try to access methods directly:

class _C(GetAttr):
    _default = 'data' # use a bad component name; i.e. self.data does not exist
    def __init__(self,a): self.default = a
    def foo(self): noop
        
# TODO: should we raise an error when we create a new instance ...
t = _C('Hi')
test_eq(t.default, 'Hi')
# ... or is it enough for all GetAttr features to raise errors
with expect_fail(): t.data
with expect_fail(): t.lower()
with expect_fail(): t.upper()
with expect_fail(): dir(t)

delegate_attr

def delegate_attr(
    k, to
):

Use in __getattr__ to delegate to attr to without inheriting from GetAttr

delegate_attr is a functional way to delegate attributes, and is an alternative to GetAttr. We recommend reading the documentation of GetAttr for more details around delegation.

You can use achieve delegation when you define __getattr__ by using delegate_attr:

class _C:
    def __init__(self, o): self.o = o # self.o corresponds to the `to` argument in delegate_attr.
    def __getattr__(self, k): return delegate_attr(self, k, to='o')
    

t = _C('HELLO') # delegates to a string
test_eq(t.lower(), 'hello')

t = _C(np.array([5,4,3])) # delegates to a numpy array
test_eq(t.sum(), 12)

t = _C(pd.DataFrame({'a': [1,2], 'b': [3,4]})) # delegates to a pandas.DataFrame
test_eq(t.b.max(), 4)

Extensible Types

ShowPrint is a base class that defines a show method, which is used primarily for callbacks in fastai that expect this method to be defined.

Int, Float, and Str extend int, float and str respectively by adding an additional show method by inheriting from ShowPrint.

The code for Int is shown below:

Examples:

Int(0).show()
Float(2.0).show()
Str('Hello').show()

0
2.0
Hello

Collection functions

Functions that manipulate popular python collections.

partition

def partition(
    coll, f
):

Partition a collection by a predicate

ts,fs = partition(range(10), mod(2))
test_eq(fs, [0,2,4,6,8])
test_eq(ts, [1,3,5,7,9])

partition_dict

def partition_dict(
    d, f
):

Partition a dict by a predicate that takes key/value params

d = {'a':1, 'b':2, 'c':3, 'd':4}
ts,fs = partition_dict(d, lambda k,v: v%2)
test_eq(fs, {'b':2, 'd':4})
test_eq(ts, {'a':1, 'c':3})

ts,fs = partition_dict(d, lambda k,v: k in 'bc')
test_eq(ts, {'b':2, 'c':3})
test_eq(fs, {'a':1, 'd':4})

flatten

def flatten(
    o
):

Concatenate all collections and items as a generator

concat

def concat(
    colls
)->list:

Concatenate all collections and items as a list

concat([(o for o in range(2)),[2,3,4], 5])

[0, 1, 2, 3, 4, 5]

concat([["abc", "xyz"], ["foo", "bar"]])

['abc', 'xyz', 'foo', 'bar']

strcat

def strcat(
    its, sep:str=''
)->str:

Concatenate stringified items its

test_eq(strcat(['a',2]), 'a2')
test_eq(strcat(['a',2], ';'), 'a;2')

detuplify

def detuplify(
    x
):

If x is a tuple with one thing, extract it

test_eq(detuplify(()),None)
test_eq(detuplify([1]),1)
test_eq(detuplify([1,2]), [1,2])
test_eq(detuplify(np.array([[1,2]])), np.array([[1,2]]))

replicate

def replicate(
    item, match
):

Create tuple of item copied len(match) times

t = [1,1]
test_eq(replicate([1,2], t),([1,2],[1,2]))
test_eq(replicate(1, t),(1,1))

setify

def setify(
    o
):

Turn any list like-object into a set.

# test
test_eq(setify(None),set())
test_eq(setify('abc'),{'abc'})
test_eq(setify([1,2,2]),{1,2})
test_eq(setify(range(0,3)),{0,1,2})
test_eq(setify({1,2}),{1,2})

merge

def merge(
    *ds
):

Merge all dictionaries in ds

test_eq(merge(), {})
test_eq(merge(dict(a=1,b=2)), dict(a=1,b=2))
test_eq(merge(dict(a=1,b=2), dict(b=3,c=4), None), dict(a=1, b=3, c=4))

range_of

def range_of(
    x
):

All indices of collection x (i.e. list(range(len(x))))

test_eq(range_of([1,1,1,1]), [0,1,2,3])

groupby

def groupby(
    x, key, val:function=noop
):

Like itertools.groupby but doesn’t need to be sorted, and isn’t lazy, plus some extensions

test_eq(groupby('aa ab bb'.split(), itemgetter(0)), {'a':['aa','ab'], 'b':['bb']})

You can use an int as key or val (which uses itemgetter; passing a str will use attrgetter), eg:

test_eq(groupby('aa ab bb'.split(), 0), {'a':['aa','ab'], 'b':['bb']})

…and you can use a tuple as key or val (which creates a tuple from the provided keys or vals), eg:

test_eq(groupby('aaa abc bba'.split(), 0, (1,2)), {'a':[('a','a'),('b','c')], 'b':[('b','a')]})

Here’s an example of how to invert a grouping, and using a val function:

d = {0: [1, 3, 7], 2: [3], 3: [5], 4: [8], 5: [4], 7: [5]}
groupby(((o,k) for k,v in d.items() for o in v), 0, 1)

{1: [0], 3: [0, 2], 7: [0], 5: [3, 7], 8: [4], 4: [5]}

last_index

def last_index(
    x, o
):

Finds the last index of occurence of x in o (returns -1 if no occurence)

test_eq(last_index(9, [1, 2, 9, 3, 4, 9, 10]), 5)
test_eq(last_index(6, [1, 2, 9, 3, 4, 9, 10]), -1)

filter_dict

def filter_dict(
    d, func
):

Filter a dict using func, applied to keys and values

letters = {o:chr(o) for o in range(65,73)}
letters

{65: 'A', 66: 'B', 67: 'C', 68: 'D', 69: 'E', 70: 'F', 71: 'G', 72: 'H'}

filter_dict(letters, lambda k,v: k<67 or v in 'FG')

{65: 'A', 66: 'B', 70: 'F', 71: 'G'}

filter_keys

def filter_keys(
    d, func
):

Filter a dict using func, applied to keys

filter_keys(letters, lt(67))

{65: 'A', 66: 'B'}

filter_values

def filter_values(
    d, func
):

Filter a dict using func, applied to values

filter_values(letters, in_('FG'))

{70: 'F', 71: 'G'}

cycle

def cycle(
    o
):

Like itertools.cycle except creates list of Nones if o is empty

test_eq(itertools.islice(cycle([1,2,3]),5), [1,2,3,1,2])
test_eq(itertools.islice(cycle([]),3), [None]*3)
test_eq(itertools.islice(cycle(None),3), [None]*3)
test_eq(itertools.islice(cycle(1),3), [1,1,1])

zip_cycle

def zip_cycle(
    x, *args
):

Like itertools.zip_longest but cycles through elements of all but first argument

test_eq(zip_cycle([1,2,3,4],list('abc')), [(1, 'a'), (2, 'b'), (3, 'c'), (4, 'a')])

sorted_ex

def sorted_ex(
    iterable, key:NoneType=None, reverse:bool=False, cmp:NoneType=None, **kwargs
):

Like sorted, but if key is str use attrgetter; if int use itemgetter; use cmp comparator function or key with kwargs

Attributes can be used for sorting by passing their name as a string:

class TestObj:
    def __init__(self, x): self.x = x
objs = [TestObj(i) for i in [3,1,2]]
test_eq([o.x for o in sorted_ex(objs, 'x')], [1,2,3])

Tuple/list items can be sorted by index position:

items = [(1,'c'), (2,'b'), (3,'a')]
test_eq(sorted_ex(items, 1), [(3,'a'), (2,'b'), (1,'c')])

A custom key function transforms values:

test_eq(sorted_ex([3,1,2], lambda x: -x), [3,2,1])

You can use a comparison function (returning -1/1/0):

test_eq(sorted_ex([3,1,2], cmp=lambda a,b: 1 if a>b else -1 if a<b else 0), [1,2,3])

Additional parameters can be passed to key/cmp functions:

def key_with_kwargs(x, offset=0): return x + offset
test_eq(sorted_ex([3,1,2], key=key_with_kwargs, offset=10), [1,2,3])

Reverse sort capability:

test_eq(sorted_ex([1,2,3], reverse=True), [3,2,1])

not_

def not_(
    f
):

Create new function that negates result of f

def f(a): return a>0
test_eq(f(1),True)
test_eq(not_(f)(1),False)
test_eq(not_(f)(a=-1),True)

argwhere

def argwhere(
    iterable, f, negate:bool=False, **kwargs
):

Like filter_ex, but return indices for matching items

filter_ex

def filter_ex(
    iterable, f:function=noop, negate:bool=False, gen:bool=False, **kwargs
):

Like filter, but passing kwargs to f, defaulting f to noop, and adding negate and gen

range_of

def range_of(
    a, b:NoneType=None, step:NoneType=None
):

All indices of collection a, if a is a collection, otherwise range

test_eq(range_of([1,1,1,1]), [0,1,2,3])
test_eq(range_of(4), [0,1,2,3])

renumerate

def renumerate(
    iterable, start:int=0
):

Same as enumerate, but returns index as 2nd element instead of 1st

test_eq(renumerate('abc'), (('a',0),('b',1),('c',2)))

first

def first(
    x, f:NoneType=None, negate:bool=False, **kwargs
):

First element of x, optionally filtered by f, or None if missing

test_eq(first(['a', 'b', 'c', 'd', 'e']), 'a')
test_eq(first([False]), False)
test_eq(first([False], noop), None)

last

def last(
    x, f:NoneType=None, negate:bool=False, **kwargs
):

Last element of x, optionally filtered by f, or None if missing

test_eq(last(['a', 'b', 'c', 'd', 'e']), 'e')
test_eq(last([False]), False)
test_eq(last([False], noop), None)

only

def only(
    o
):

Return the only item of o, raise if o doesn’t have exactly one item

nested_attr

def nested_attr(
    o, attr, default:NoneType=None
):

Same as getattr, but if attr includes a ., then looks inside nested objects

class CustomIndexable:
    def __init__(self): self.data = {'a':1,'b':'v','c':{'d':5}}
    def __getitem__(self, key): return self.data[key]

custom_indexable = CustomIndexable()
test_eq(nested_attr(custom_indexable,'a'),1)
test_eq(nested_attr(custom_indexable,'c.d'),5)
test_eq(nested_attr(custom_indexable,'e'),None)

class TestObj: def init(self): self.nested = {‘key’: [1, 2, {‘inner’: ‘value’}]} test_obj = TestObj()

test_eq(nested_attr(test_obj, ‘nested.key.2.inner’),‘value’) test_eq(nested_attr([1, 2, 3], ‘1’),2)

b = {'a':1,'b':'v','c':{'d':5}}
test_eq(nested_attr(b,'b'),'v')
test_eq(nested_attr(b,'c.d'),5)

a = SimpleNamespace(b=(SimpleNamespace(c=1)))
test_eq(nested_attr(a, 'b.c'), getattr(getattr(a, 'b'), 'c'))
test_eq(nested_attr(a, 'b.d'), None)
test_eq(nested_attr(b, 'a'), 1)

nested_setdefault

def nested_setdefault(
    o, attr, default
):

Same as setdefault, but if attr includes a ., then looks inside nested objects

nested_callable

def nested_callable(
    o, attr
):

Same as nested_attr but if not found will return noop

a = SimpleNamespace(b=(SimpleNamespace(c=1)))
test_eq(nested_callable(a, 'b.c'), getattr(getattr(a, 'b'), 'c'))
test_eq(nested_callable(a, 'b.d'), noop)

nested_idx

def nested_idx(
    coll, *idxs
):

Index into nested collections, dicts, etc, with idxs

a = {'b':[1,{'c':2}]}
test_eq(nested_idx(a, 'nope'), None)
test_eq(nested_idx(a, 'nope', 'nup'), None)
test_eq(nested_idx(a, 'b', 3), None)
test_eq(nested_idx(a), a)
test_eq(nested_idx(a, 'b'), [1,{'c':2}])
test_eq(nested_idx(a, 'b', 1), {'c':2})
test_eq(nested_idx(a, 'b', 1, 'c'), 2)

a = SimpleNamespace(b=[1,{'c':2}])
test_eq(nested_idx(a, 'nope'), None)
test_eq(nested_idx(a, 'nope', 'nup'), None)
test_eq(nested_idx(a, 'b', 3), None)
test_eq(nested_idx(a), a)
test_eq(nested_idx(a, 'b'), [1,{'c':2}])
test_eq(nested_idx(a, 'b', 1), {'c':2})
test_eq(nested_idx(a, 'b', 1, 'c'), 2)

set_nested_idx

def set_nested_idx(
    coll, value, *idxs
):

Set value indexed like `nested_idx

set_nested_idx(a, 3, 'b', 0)
test_eq(nested_idx(a, 'b', 0), 3)

val2idx

def val2idx(
    x
):

Dict from value to index

test_eq(val2idx([1,2,3]), {3:2,1:0,2:1})

uniqueify

def uniqueify(
    x, sort:bool=False, bidir:bool=False, start:NoneType=None
):

Unique elements in x, optional sort, optional return reverse correspondence, optional prepend with elements.

t = [1,1,0,5,0,3]
test_eq(uniqueify(t),[1,0,5,3])
test_eq(uniqueify(t, sort=True),[0,1,3,5])
test_eq(uniqueify(t, start=[7,8,6]), [7,8,6,1,0,5,3])
v,o = uniqueify(t, bidir=True)
test_eq(v,[1,0,5,3])
test_eq(o,{1:0, 0: 1, 5: 2, 3: 3})
v,o = uniqueify(t, sort=True, bidir=True)
test_eq(v,[0,1,3,5])
test_eq(o,{0:0, 1: 1, 3: 2, 5: 3})

loop_first_last

def loop_first_last(
    values
):

Iterate and generate a tuple with a flag for first and last value.

test_eq(loop_first_last(range(3)), [(True,False,0), (False,False,1), (False,True,2)])

loop_first

def loop_first(
    values
):

Iterate and generate a tuple with a flag for first value.

test_eq(loop_first(range(3)), [(True,0), (False,1), (False,2)])

loop_last

def loop_last(
    values
):

Iterate and generate a tuple with a flag for last value.

test_eq(loop_last(range(3)), [(False,0), (False,1), (True,2)])

first_match

def first_match(
    lst, f, default:NoneType=None
):

First element of lst matching predicate f, or default if none

a = [0,2,4,5,6,7,10]
test_eq(first_match(a, lambda o:o%2), 3)

last_match

def last_match(
    lst, f, default:NoneType=None
):

Last element of lst matching predicate f, or default if none

test_eq(last_match(a, lambda o:o%2), 5)

joins

def joins(
    sep, its
):

Sugar for sep.join(map(str, its))

# Gives: `TypeError: sequence item 0: expected str…`:
#   ",".join([1,2,3])
joins(",", [1,2,3])

'1,2,3'

fastuple

A tuple with extended functionality.

fastuple

def fastuple(
    *args, **kwargs
):

A tuple with elementwise ops and more friendly init behavior

Friendly init behavior

Common failure modes when trying to initialize a tuple in python:

tuple(3)
> TypeError: 'int' object is not iterable

or

tuple(3, 4)
> TypeError: tuple expected at most 1 arguments, got 2

However, fastuple allows you to define tuples like this and in the usual way:

test_eq(fastuple(3), (3,))
test_eq(fastuple(3,4), (3, 4))
test_eq(fastuple((3,4)), (3, 4))

Elementwise operations

fastuple.add

def add(
    *args
):

+ is already defined in tuple for concat, so use add instead

test_eq(fastuple.add((1,1),(2,2)), (3,3))
test_eq_type(fastuple(1,1).add(2), fastuple(3,3))
test_eq(fastuple('1','2').add('2'), fastuple('12','22'))

fastuple.mul

def mul(
    *args
):

* is already defined in tuple for replicating, so use mul instead

test_eq_type(fastuple(1,1).mul(2), fastuple(2,2))

Other Elementwise Operations

Additionally, the following elementwise operations are available: - le: less than or equal - eq: equal - gt: greater than - min: minimum of

test_eq(fastuple(3,1).le(1), (False, True))
test_eq(fastuple(3,1).eq(1), (False, True))
test_eq(fastuple(3,1).gt(1), (True, False))
test_eq(fastuple(3,1).min(2), (2,1))

You can also do other elementwise operations like negate a fastuple, or subtract two fastuples:

test_eq(-fastuple(1,2), (-1,-2))
test_eq(~fastuple(1,0,1), (False,True,False))

test_eq(fastuple(1,1)-fastuple(2,2), (-1,-1))

test_eq(type(fastuple(1)), fastuple)
test_eq_type(fastuple(1,2), fastuple(1,2))
test_ne(fastuple(1,2), fastuple(1,3))
test_eq(fastuple(), ())

Functions on Functions

Tools for making and transforming functions: compose(f,g,...) chains functions left to right; bind is partial extended with arg0,arg1,… placeholders for reordering positional arguments; and fail_clean re-raises exceptions with the library’s own traceback frames stripped, for errors that are part of a function’s contract rather than bugs.

Utilities for functional programming or for defining, modifying, or debugging functions.

bind

def bind(
    func, *pargs, **pkwargs
):

Same as partial, except you can use arg0 arg1 etc param placeholders

bind is the same as partial, but also allows you to reorder positional arguments using variable name(s) arg{i} where i refers to the zero-indexed positional argument. bind as implemented currently only supports reordering of up to the first 5 positional arguments.

Consider the function myfunc below, which has 3 positional arguments. These arguments can be referenced as arg0, arg1, and arg1, respectively.

def myfn(a,b,c,d=1,e=2): return(a,b,c,d,e)

In the below example we bind the positional arguments of myfn as follows:

The second input 14, referenced by arg1, is substituted for the first positional argument.
We supply a default value of 17 for the second positional argument.
The first input 19, referenced by arg0, is subsituted for the third positional argument.

test_eq(bind(myfn, arg1, 17, arg0, e=3)(19,14), (14,17,19,1,3))

In this next example:

We set the default value to 17 for the first positional argument.
The first input 19 refrenced by arg0, becomes the second positional argument.
The second input 14 becomes the third positional argument.
We override the default the value for named argument e to 3.

test_eq(bind(myfn, 17, arg0, e=3)(19,14), (17,19,14,1,3))

This is an example of using bind like partial and do not reorder any arguments:

test_eq(bind(myfn)(17,19,14), (17,19,14,1,2))

bind can also be used to change default values. In the below example, we use the first input 3 to override the default value of the named argument e, and supply default values for the first three positional arguments:

test_eq(bind(myfn, 17,19,14,e=arg0)(3), (17,19,14,1,3))

Every arg{i} placeholder refers to the i-th call-time argument, in positional and keyword bindings alike; arguments past the highest placeholder index are appended:

test_eq(bind(myfn, arg1, d=arg0)(19,14,8,5), (14,8,5,19,2))
test_eq(bind(myfn, c=arg0, d=arg1)(19,14,3,7), (3,7,19,14,2))
test_eq(bind(myfn, arg0, arg0)(3,5), (3,3,5,1,2))

mapt

def mapt(
    func, *iterables
):

Tuplified map

t = [0,1,2,3]
test_eq(mapt(operator.neg, t), (0,-1,-2,-3))

map_ex

def map_ex(
    iterable, f, *args, gen:bool=False, **kwargs
):

Like map, but use bind, and supports str and indexing

test_eq(map_ex(t,operator.neg), [0,-1,-2,-3])

If f is a string then it is treated as a format string to create the mapping:

test_eq(map_ex(t, '#{}#'), ['#0#','#1#','#2#','#3#'])

If f is a dictionary (or anything supporting __getitem__) then it is indexed to create the mapping:

test_eq(map_ex(t, list('abcd')), list('abcd'))

You can also pass the same arg params that bind accepts:

def f(a=None,b=None): return b
test_eq(map_ex(t, f, b=arg0), range(4))

compose

def compose(
    *funcs, order:NoneType=None
):

Create a function that composes all functions in funcs, passing along remaining *args and **kwargs to all

f1 = lambda o,p=0: (o*2)+p
f2 = lambda o,p=1: (o+1)/p
test_eq(f2(f1(3)), compose(f1,f2)(3))
test_eq(f2(f1(3,p=3),p=3), compose(f1,f2)(3,p=3))
test_eq(f2(f1(3,  3),  3), compose(f1,f2)(3,  3))

f1.order = 1
test_eq(f1(f2(3)), compose(f1,f2, order="order")(3))

maps

def maps(
    *args, retain:function=noop
):

Like map, except funcs are composed first

test_eq(maps([1]), [1])
test_eq(maps(operator.neg, [1,2]), [-1,-2])
test_eq(maps(operator.neg, operator.neg, [1,2]), [1,2])

partialler

def partialler(
    f, *args, order:NoneType=None, **kwargs
):

Like functools.partial but also copies over docstring

def _f(x,a=1):
    "test func"
    return x-a
_f.order=1

f = partialler(_f, 2)
test_eq(f.order, 1)
test_eq(f(3), -1)
f = partialler(_f, a=2, order=3)
test_eq(f.__doc__, "test func")
test_eq(f.order, 3)
test_eq(f(3), _f(3,2))

class partial0:
    "Like `partialler`, but args passed to callable are inserted at started, instead of at end"
    def __init__(self, f, *args, order=None, **kwargs):
        self.f,self.args,self.kwargs = f,args,kwargs
        self.order = ifnone(order, getattr(f,'order',None))
        self.__doc__ = f.__doc__

    def __call__(self, *args, **kwargs): return self.f(*args, *self.args, **kwargs, **self.kwargs)

f = partial0(_f, 2)
test_eq(f.order, 1)
test_eq(f(3), 1) # NB: different to `partialler` example

instantiate

def instantiate(
    t
):

Instantiate t if it’s a type, otherwise do nothing

test_eq_type(instantiate(int), 0)
test_eq_type(instantiate(1), 1)

using_attr

def using_attr(
    f, attr
):

Construct a function which applies f to the argument’s attribute attr

t = Path('/a/b.txt')
f = using_attr(str.upper, 'name')
test_eq(f(t), 'B.TXT')

negate

def negate(
    f
):

Returns the negation of f

def true():
    'Returns True'
    return True

false = negate(true)
print(false.__doc__)
test_eq(false(), not true())

Returns `not true(...)`

Original: Returns True

fail_clean

def fail_clean(
    *excs
):

Re-raise excs (default: Exception) without internal traceback frames

fail_clean marks a boundary where errors are part of a function’s contract rather than something to debug: exceptions of the listed types (any Exception by default) are re-raised with the internal traceback frames removed, so the caller sees the message and their own call site instead of the library’s plumbing. The original exception object is preserved, including its attributes. Exceptions not listed keep their full stack for debugging. Use @fail_clean bare, or pass exception types like @fail_clean(ValueError):

import traceback

def _boom(): raise ValueError('boom')
def _mid(): return _boom()

@fail_clean
def g(): return _mid()

try: g()
except ValueError as e:
    test_eq(str(e), 'boom')
    test_eq(e.__cause__, None)
    test_eq(len(traceback.extract_tb(e.__traceback__)), 2)  # caller + wrapper; internals gone

@fail_clean(KeyError)
def h(): return _mid()
try: h()
except ValueError as e: assert len(traceback.extract_tb(e.__traceback__)) > 2  # unlisted types keep their stack

dstar

def dstar(
    f
):

Wrap f to accept a single dict, unpacking it as keyword args

dstar wraps a function to accept a single dictionary, unpacking it as keyword arguments (the ** counterpart of star in fastcore.foundation). For instance:

def greet(name, greeting='Hello'): return f'{greeting}, {name}!'
configs = [{'name': 'Alice'}, {'name': 'Bob', 'greeting': 'Hi'}]
list(map(dstar(greet), configs))  # ['Hello, Alice!', 'Hi, Bob!']

['Hello, Alice!', 'Hi, Bob!']

A more realistic example showing API request configuration. Each request dictionary may have different keys present, and dspread handles this naturally (missing keys use the function’s defaults):

def api_request(endpoint, method='GET', timeout=30, headers=None):
    return f"{method} {endpoint} (timeout={timeout})"

requests = [
    {'endpoint': '/users', 'method': 'POST', 'timeout': 60},
    {'endpoint': '/data'},
    {'endpoint': '/health', 'method': 'HEAD', 'timeout': 5}
]
list(map(dstar(api_request), requests))

['POST /users (timeout=60)',
 'GET /data (timeout=30)',
 'HEAD /health (timeout=5)']

Self (with an uppercase S)

A Concise Way To Create Lambdas

~Self is a concise alternative to lambda for a function that operates on a single object (note the capitalization!). Write the chain of attribute accesses, method calls, and indexing just as you would after a variable name, with ~Self in its place, and the result is a plain function that runs the chain on its argument:

~Self.sum() is lambda o: o.sum()
~Self.imag is lambda o: o.imag
~Self[1] is lambda o: o[1]
~Self.sum().real is lambda o: o.sum().real
~Self alone is the identity, lambda o: o

Since ., (), and [] bind tighter than ~, the whole chain builds first and ~ then converts it to a function, so the chain never needs its own parentheses: map(~Self.imag, nums) works as written.

The functions ~Self returns are ordinary functions: safe to introspect, and independent of any later chains built from the same prefix. Parens mean what they mean in plain Python: a step written with () is called, so a chain that calls a non-callable attribute raises TypeError rather than quietly fetching it:

test_eq((~Self.imag)(3), 0)
with expect_fail(TypeError, 'not callable'): (~Self.imag())(3)  # `imag` is an attribute; `()` means call, as in plain Python
test_eq(list(map(~Self.real, [3,4])), [3,4])
x = np.array([3.,1])
test_eq((~Self.sum().real)(x), 4.)
test_eq((~Self)(x) is x, True)
test_eq((~Self[1])(x), 1)
test_eq((~Self.strip()[0])(' abc '), 'a')

g1 = Self.strip()
g2 = g1.upper()
test_eq((~g1)('  hi  '), 'hi')
test_eq((~g2)('  hi  '), 'HI')

assert not hasattr(Self.sum(), '__wrapped__')

with expect_fail(TypeError, 'applied'): Self.strip()()
with expect_fail(TypeError, '~Self'): bool(Self.imag)

Calling Self directly adds a call step to the chain, so ~Self(1,b=2) is a function which calls whatever it’s applied to with those arguments:

def f(a, b=3): return a+b+2
def g(a, b=3): return a*b
fg = ~Self(1,b=2)
list(map(fg, [f,g]))

[5, 2]

Patching

@patch adds a function to an existing class as a method, using the function’s self: type annotation to pick the class (a union annotation patches several classes at once); @patch_to(Cls) is the same with the class passed explicitly. Both take as_prop, set_prop, and cls_method. fastai code uses this to build classes incrementally across a notebook, so expect to find a class’s methods defined far from the class itself:

copy_func

def copy_func(
    f
):

Copy a non-builtin function (NB copy.copy does not work for this)

Sometimes it may be desirable to make a copy of a function that doesn’t point to the original object. When you use Python’s built in copy.copy or copy.deepcopy to copy a function, you get a reference to the original object:

import copy as cp

def foo(): pass
a = cp.copy(foo)
b = cp.deepcopy(foo)

a.someattr = 'hello' # since a and b point at the same object, updating a will update b
test_eq(b.someattr, 'hello')

assert a is foo and b is foo

However, with copy_func, you can retrieve a copy of a function without a reference to the original object:

c = copy_func(foo) # c is an indpendent object
assert c is not foo

def g(x, *, y=3): return x+y
test_eq(copy_func(g)(4), 7)

patch_to

def patch_to(
    cls, as_prop:bool=False, cls_method:bool=False, set_prop:bool=False, static_method:bool=False, nm:NoneType=None,
    glb:NoneType=None
):

Decorator: add f to cls

The @patch_to decorator allows you to monkey patch a function into a class as a method:

class _T3(int): pass  

@patch_to(_T3)
def func1(self, a): return self+a

t = _T3(1) # we initialized `t` to a type int = 1
test_eq(t.func1(2), 3) # we add 2 to `t`, so 2 + 1 = 3

if sys.version_info >= (3,11):
    test_eq(_T3.func1.__code__.co_qualname, '_T3.func1')

You can access instance properties in the usual way via self:

class _T4():
    def __init__(self, g): self.g = g
        
@patch_to(_T4)
def greet(self, x): return self.g + x
        
t = _T4('hello ') # this sets self.g = 'hello '
test_eq(t.greet('world'), 'hello world') #t.greet('world') will append 'world' to 'hello '

You can instead specify that the method should be a class method by setting cls_method=True:

class _T5(int): attr = 3 # attr is a class attribute we will access in a later method
    
@patch_to(_T5, cls_method=True)
def func(cls, x): return cls.attr + x # you can access class attributes in the normal way

test_eq(_T5.func(4), 7)

Additionally you can specify that the function you want to patch should be a class attribute with as_prop=True:

@patch_to(_T5, as_prop=True)
def add_ten(self): return self + 10

t = _T5(4)
test_eq(t.add_ten, 14)

Once you have a property, you can assign a setter with set_prop=True:

class _T2():
    def __init__(self, val): self._val = val

@patch_to(_T2, as_prop=True)
def val(self): return self._val

t = _T2(2)
test_eq(t.val, 2)

@patch_to(_T2, set_prop=True)
def val(self, val): self._val = val

t.val = 3
test_eq(t.val, 3)

Instead of passing one class to the @patch_to decorator, you can pass multiple classes in a tuple to simulteanously patch more than one class with the same method:

class _T6(int): pass
class _T7(int): pass

@patch_to((_T6,_T7))
def func_mult(self, a): return self*a

t = _T6(2)
test_eq(t.func_mult(4), 8)
t = _T7(2)
test_eq(t.func_mult(4), 8)

You can also rename the function in the patched class:

class _T8(int): pass  

@patch_to(_T8, nm='add_value')
def func2(self, a): return self+a

t = _T8(1)
test_eq(t.add_value(2), 3)
test_eq(_T8.add_value.__name__, 'add_value')
assert not hasattr(t, 'func2')

patch

def patch(
    f:NoneType=None, as_prop:bool=False, cls_method:bool=False, static_method:bool=False, set_prop:bool=False,
    nm:NoneType=None
):

Decorator: add f to the first parameter’s class (based on f’s type annotations)

@patch
def total(self:Point): return self.x + self.y
test_eq(Point(3, 4).total(), 7)

@patch is an alternative to @patch_to that allows you similarly monkey patch class(es) by using type annotations:

class _T8(int): pass  

@patch
def func(self:_T8, a): return self+a

t = _T8(1)  # we initilized `t` to a type int = 1
test_eq(t.func(3), 4) # we add 3 to `t`, so 3 + 1 = 4
test_eq(t.func.__qualname__, '_T8.func')
if sys.version_info >= (3,11):
    test_eq(_T8.func.__code__.co_qualname, '_T8.func')

class MyMath: pass

@patch_to(MyMath, static_method=True)
def add(a, b): return a + b

@patch(static_method=True)
def mul(a:MyMath, b): return a * b

test_eq(MyMath.add(2, 3), 5)
test_eq(MyMath.mul(2, 3), 6)

Similarly to patch_to, you can supply a union of classes instead of a single class in your type annotations to patch multiple classes:

class _T9(int): pass 

@patch
def func2(x:_T8|_T9, a): return x*a # will patch both _T8 and _T9

t = _T8(2)
test_eq(t.func2(4), 8)
test_eq(t.func2.__qualname__, '_T8.func2')

t = _T9(2)
test_eq(t.func2(4), 8)
test_eq(t.func2.__qualname__, '_T9.func2')

Just like patch_to decorator you can use as_prop, set_prop, and cls_method parameters with patch decorator:

@patch(as_prop=True)
def add_ten(self:_T5): return self + 10

t = _T5(4)
test_eq(t.add_ten, 14)

class _T2():
    def __init__(self, val): self._val = val

@patch(as_prop=True)
def val(self:_T2): return self._val

t = _T2(2)
test_eq(t.val, 2)

@patch(set_prop=True)
def val(self:_T2, val): self._val = val

t.val = 3
test_eq(t.val, 3)

class _T5(int): attr = 3 # attr is a class attribute we will access in a later method
    
@patch(cls_method=True)
def func(cls:_T5, x): return cls.attr + x # you can access class attributes in the normal way

test_eq(_T5.func(4), 7)

class _T8(int): pass  

@patch(nm='add_value')
def func2(self:_T8, a): return self+a

t = _T8(1)
test_eq(t.add_value(2), 3)
test_eq(_T8.add_value.__name__, 'add_value')
assert not hasattr(t, 'func2')

Patching classmethod shouldn’t affect how python’s inheritance works

class FastParent: pass

@patch(cls_method=True)
def type_cls(cls: FastParent): return cls

class FastChild(FastParent): pass

parent = FastParent()
test_eq(parent.type_cls(), FastParent)

child = FastChild()
test_eq(child.type_cls(), FastChild)

It also works on iterable classes like Enums:

class Color(Enum):
    RED = 1
    GREEN = 2

@patch(cls_method=True)
def from_name(cls: Color, s): return cls[s.upper()]

test_eq(Color.from_name.__name__, 'from_name')
test_eq(Color.from_name('red'), Color.RED)
test_eq(Color.from_name('GREEN'), Color.GREEN)

Other Helpers

extend_enum

def extend_enum(
    cls, # Enum class to modify
    n, # Name of the new enum member
    v, # Value of the new enum member
):

Add new member n with value v to enum class cls at runtime

extend_enum mutates an existing enum class by constructing a new member, registering it in the enum’s internal lookup tables, and attaching it as a class attribute, so it behaves like a normal enum member created in the original class definition.

from enum import Enum

class Color(Enum): red = 1; blue = 2

extend_enum(Color, 'green', 3)
Color.green, Color['green'], Color(3)

(<Color.green: 3>, <Color.green: 3>, <Color.green: 3>)

compile_re

def compile_re(
    pat
):

Compile pat if it’s not None

assert compile_re(None) is None
assert compile_re('a').match('ab')

ImportEnum

def ImportEnum(
    *args, **kwds
):

An Enum that can have its values imported

_T = ImportEnum('_T', {'foobar':1, 'goobar':2})
_T.imports()
test_eq(foobar, _T.foobar)
test_eq(goobar, _T.goobar)

StrEnum

def StrEnum(
    *args, **kwds
):

An ImportEnum that behaves like a str

str_enum

def str_enum(
    name, *vals
):

Simplified creation of StrEnum types

ValEnum

def ValEnum(
    *args, **kwds
):

An ImportEnum that stringifies using values

import pickle

_T = str_enum('_T', 'a', 'b')
test_eq(f'{_T.a}', 'a')
test_eq(_T.a, 'a')
test_eq(list(_T.__members__), ['a','b'])
test_eq(_T.__module__, __name__)
test_eq(pickle.loads(pickle.dumps(_T.a)), _T.a)
print(_T.a, _T.a.upper())

a A

Stateful

def Stateful(
    *args, **kwargs
):

A base class/mixin for objects that should not serialize all their state

class _T(Stateful):
    def __init__(self):
        super().__init__()
        self.a=1
        self._state['test']=2

t = _T()
t2 = pickle.loads(pickle.dumps(t))
test_eq(t.a,1)
test_eq(t._state['test'],2)
test_eq(t2.a,1)
test_eq(t2._state,{})

Override _init_state to do any necessary setup steps that are required during __init__ or during deserialization (e.g. pickle.load). Here’s an example of how Stateful simplifies the official Python example for Handling Stateful Objects.

class TextReader(Stateful):
    """Print and number lines in a text file."""
    _stateattrs=('file',)
    def __init__(self, filename):
        self.filename,self.lineno = filename,0
        super().__init__()

    def readline(self):
        self.lineno += 1
        line = self.file.readline()
        if line: return f"{self.lineno}: {line.strip()}"

    def _init_state(self):
        self.file = open(self.filename)
        for _ in range(self.lineno): self.file.readline()

reader = TextReader("00_test.ipynb")
print(reader.readline())
print(reader.readline())

new_reader = pickle.loads(pickle.dumps(reader))
print(reader.readline())

1: {
2: "cells": [
3: {

NotStr

def NotStr(
    s
):

Behaves like a str, but isn’t an instance of one

s = NotStr("hello")
assert not isinstance(s, str)
test_eq(s, 'hello')
test_eq(s*2, 'hellohello')
test_eq(len(s), 5)
test_eq(s[:2], "he")
test_eq(s[2], "l")

PrettyString

def PrettyString(
    *args, **kwargs
):

Little hack to get strings to show properly in Jupyter.

Allow strings with special characters to render properly in Jupyter. Without calling print() strings with special characters are displayed like so:

with_special_chars='a string\nwith\nnew\nlines and\ttabs'
with_special_chars

'a string\nwith\nnew\nlines and\ttabs'

We can correct this with PrettyString:

PrettyString(with_special_chars)

a string
with
new
lines and   tabs

even_mults

def even_mults(
    start, stop, n
):

Build log-stepped array from start to stop in n steps.

test_eq(even_mults(2,8,3), [2,4,8])
test_eq(even_mults(2,32,5), [2,4,8,16,32])
test_eq(even_mults(2,8,1), 8)

num_cpus

def num_cpus():

Get number of cpus

num_cpus()

add_props

def add_props(
    f, g:NoneType=None, n:int=2
):

Create properties passing each of range(n) to f

class _T(): a,b = add_props(lambda i,x:i*2)

t = _T()
test_eq(t.a,0)
test_eq(t.b,2)

class _T(): 
    def __init__(self, v): self.v=v
    def _set(i, self, v): self.v[i] = v
    a,b = add_props(lambda i,x: x.v[i], _set)

t = _T([0,2])
test_eq(t.a,0)
test_eq(t.b,2)
t.a = t.a+1
t.b = 3
test_eq(t.a,1)
test_eq(t.b,3)

str2bool

def str2bool(
    s
):

Case-insensitive convert string s too a bool (y,yes,t,true,on,1->True)

True values are ‘y’, ‘yes’, ‘t’, ‘true’, ‘on’, and ‘1’; false values are ‘n’, ‘no’, ‘f’, ‘false’, ‘off’, and ‘0’. Raises ValueError if ‘val’ is anything else.

for o in "y YES t True on 1".split(): assert str2bool(o)
for o in "n no FALSE off 0".split(): assert not str2bool(o)
for o in 0,None,'',False: assert not str2bool(o)
for o in 1,True: assert str2bool(o)

str2int

def str2int(
    s
)->int:

Convert s to an int

str2float

def str2float(
    s:str
):

Convert s to a float

str2list

def str2list(
    s:str
):

Convert s to a list

str2date

def str2date(
    s:str
)->date:

date.fromisoformat with empty string handling

str2dt

def str2dt(
    s:str
)->datetime:

datetime.fromisoformat with Z suffix and empty string handling

to_date

def to_date(
    arg
):

Call self as a function.

to_list

def to_list(
    arg
):

Call self as a function.

to_float

def to_float(
    arg
):

Call self as a function.

to_int

def to_int(
    arg
):

Call self as a function.

to_bool

def to_bool(
    arg
):

Call self as a function.

typed

def typed(
    _func:NoneType=None, cast:bool=False
):

Decorator to check param and return types at runtime, with optional casting

typed validates argument types at runtime. This is in contrast to MyPy which only offers static type checking.

For example, a TypeError will be raised if we try to pass an integer into the first argument of the below function:

@typed
def discount(price:int, pct:float) -> float:
    return (1-pct) * price

with ExceptionExpected(TypeError): discount(100.0, .1)

You can have automatic casting based on heuristics by specifying typed(cast=True). If casting is not possible, a TypeError is raised.

@typed(cast=True)
def discount(price:int, pct:float) -> float:
    return (1-pct) * price

assert 90.0 == discount(100.5, .1) # will auto cast 100.5 to the int 100
assert 90.0 == discount(' 100 ', .1) # will auto cast the str "100" to the int 100
with ExceptionExpected(TypeError): discount("a", .1)

We can also optionally allow multiple types by enumarating the types in a tuple as illustrated below:

@typed
def discount(price:int|float, pct:float): 
    return (1-pct) * price

assert 90.0 == discount(100.0, .1)

@typed(cast=True)
def discount(price:int|None, pct:float):
    return (1-pct) * price

assert 90.0 == discount(100.0, .1)

We currently do not support union types when casting.

@typed(cast=True)
def discount(price:int|float, pct:float):
    return (1-pct) * price

with ExceptionExpected(AssertionError): assert 90.0 == discount("100.0", .1)

typed works with classes, too:

class Foo:
    @typed
    def __init__(self, a:int, b: int, c:str): pass
    @typed(cast=True)
    def test(cls, d:str): return d

with ExceptionExpected(TypeError): Foo(1, 2, 3) 
assert isinstance(Foo(1,2, 'a string').test(10), str)

It also works with custom types.

@typed
def test_foo(foo: Foo): pass

with ExceptionExpected(TypeError): test_foo(1)
test_foo(Foo(1, 2, 'a string'))

class Bar:
    @typed
    def __init__(self, a:int): self.a = a
@typed(cast=True)
def test_bar(bar: Bar): return bar

assert isinstance(test_bar(1), Bar)
test_eq(test_bar(1).a, 1)
with ExceptionExpected(TypeError): test_bar("foobar")

exec_new

def exec_new(
    code
):

Execute code in a new environment and return it

g = exec_new('a=1')
test_eq(g['a'], 1)

exec_import

def exec_import(
    mod, sym
):

Import sym from mod in a new environment

sig_with_params

def sig_with_params(
    sig, remove:NoneType=None, **updates
):

Copy of sig with removed params dropped and updates added, kind-sorted

kindsort

def kindsort(
    ps
):

Params sorted by Parameter.kind

sig_with_params lets you modify a function signature by adding, replacing, or removing parameters. This is useful when creating wrapper functions or decorators that need to adjust the signature of the wrapped function.

You can remove parameters by name:

def foo(a, b, c=3): pass
sig = signature(foo)

new_sig = sig_with_params(sig, remove=['b'])
test_eq(list(new_sig.parameters.keys()), ['a', 'c'])

You can also add new parameters:

new_param = Parameter('d', Parameter.KEYWORD_ONLY, default=4)
new_sig = sig_with_params(sig, d=new_param)
test_eq(list(new_sig.parameters.keys()), ['a', 'b', 'c', 'd'])

The result is kind-sorted (kindsort: a stable sort on Parameter.kind, Python’s own ordering rule), so composing groups of params can never produce an illegal signature: a collector in sig stays last however many params are added after it:

def bar(a, *, b=2, **kwargs): pass
new_sig = sig_with_params(signature(bar), c=Parameter('c', Parameter.KEYWORD_ONLY, default=3))
test_eq(str(new_sig), '(a, *, b=2, c=3, **kwargs)')
test_eq([p.name for p in kindsort(new_sig.parameters.values())], ['a', 'b', 'c', 'kwargs'])  # already legal: unchanged

fdelegates

def fdelegates(
    to
):

Call self as a function.

This is a simplified version of fastcore.meta.delegates that supports only regular functions.

xdumps

def xdumps(
    o, **kwargs
):

json.dumps, using __json__ methods when available

Python’s json.dumps can handle custom objects through a default callback, but it does not look for serialization methods on the objects themselves. Therefore even a class is not normally serializable:

class Person:
    def __init__(self, name): self.name = name
    def __json__(self): return dict(name=self.name)

with expect_fail(TypeError, 'Person is not JSON serializable'): json.dumps(Person('Alyssa'))

xdumps recognizes __json__ and serializes the value it returns. The protocol is applied recursively, so custom objects can appear inside ordinary lists and dictionaries:

people = dict(team=[Person('Alyssa'), Person('Ben')])
test_eq(json.loads(xdumps(people)), dict(team=[dict(name='Alyssa'), dict(name='Ben')]))

xdumps also supports default like dumps does:

test_eq(json.loads(xdumps(1j, default=str)), '1j')

Some standard library types cannot implement __json__, since builtin types cannot gain methods, so xdumps handles the common ones directly: datetimes, dates and times become ISO8601 strings, UUID and Path become strings, and enums become their values. An object’s own __json__ still comes first, and default remains the escape hatch for anything else:

class Priority(enum.Enum): high = 1
d = dict(t=datetime(2026,8,2,10,30), day=date(2026,8,2), u=UUID(int=18), p=Path('a/b'), pri=Priority.high)
test_eq(json.loads(xdumps(d)), dict(t='2026-08-02T10:30:00', day='2026-08-02', u='00000000-0000-0000-0000-000000000012', p='a/b', pri=1))

revive_dates is the inverse for the datetime case: it walks a decoded JSON tree and converts any string in ISO8601 form (Z suffix included) back into a datetime. Conversion is by shape, so a string that merely looks like a timestamp converts too - apply it to trees where date strings can only be dates, such as message headers, rather than to arbitrary user content:

revive_dates

def revive_dates(
    o
):

Recursively convert ISO8601-formatted strings in a decoded JSON tree into datetime objects

msg = json.loads(xdumps(dict(date=datetime(2026,8,2,10,30), note='sent on 2026-08-02, not a timestamp')))
test_eq(revive_dates(dict(d='2026-08-02T10:30:00Z'))['d'], datetime.fromisoformat('2026-08-02T10:30:00+00:00'))
revive_dates(msg)

Notebook functions

ipython_shell

def ipython_shell():

Same as get_ipython but returns False if not in IPython

in_ipython

def in_ipython():

Check if code is running in some kind of IPython environment

in_colab

def in_colab():

Check if the code is running in Google Colaboratory

in_jupyter

def in_jupyter():

Check if the code is running in a jupyter notebook

in_notebook

def in_notebook():

Check if the code is running in a jupyter notebook

These variables are available as booleans in fastcore.basics as IN_IPYTHON, IN_JUPYTER, IN_COLAB and IN_NOTEBOOK.

IN_IPYTHON, IN_JUPYTER, IN_COLAB, IN_NOTEBOOK

(False, False, False, False)