The `L` class and helpers for it
from fastcore.test import *
from nbdev.showdoc import *
from fastcore.nb_imports import *

Foundational Functions

working_directory[source]

working_directory(path)

Change working directory to path and return to previous on exit.

add_docs[source]

add_docs(cls_doc=None, **docs)

Copy values from docs to cls docstrings, and confirm all public methods are documented

add_docs allows you to add docstrings to a class and its associated methods. This function allows you to group docstrings together seperate from your code, which enables you to define one-line functions as well as organize your code more succintly. We believe this confers a number of benefits which we discuss in our style guide.

Suppose you have the following undocumented class:

class T:
    def foo(self): pass
    def bar(self): pass

You can add documentation to this class like so:

add_docs(T, cls_doc="A docstring for the class.",
            foo="The foo method.",
            bar="The bar method.")

Now, docstrings will appear as expected:

test_eq(T.__doc__, "A docstring for the class.")
test_eq(T.foo.__doc__, "The foo method.")
test_eq(T.bar.__doc__, "The bar method.")

add_docs also validates that all of your public methods contain a docstring. If one of your methods is not documented, it will raise an error:

class T:
    def foo(self): pass
    def bar(self): pass

f=lambda: add_docs(T, "A docstring for the class.", foo="The foo method.")
test_fail(f, contains="Missing docs")

docs[source]

docs()

Decorator version of add_docs, using _docs dict

Instead of using add_docs, you can use the decorator docs as shown below. Note that the docstring for the class can be set with the argument cls_doc:

@docs
class _T:
    def f(self): pass
    def g(cls): pass
    
    _docs = dict(cls_doc="The class docstring", 
                 f="The docstring for method f.",
                 g="A different docstring for method g.")

    
test_eq(_T.__doc__, "The class docstring")
test_eq(_T.f.__doc__, "The docstring for method f.")
test_eq(_T.g.__doc__, "A different docstring for method g.")

For either the docs decorator or the add_docs function, you can still define your docstrings in the normal way. Below we set the docstring for the class as usual, but define the method docstrings through the _docs attribute:

@docs
class _T:
    "The class docstring"
    def f(self): pass
    _docs = dict(f="The docstring for method f.")

    
test_eq(_T.__doc__, "The class docstring")
test_eq(_T.f.__doc__, "The docstring for method f.")

is_iter[source]

is_iter(o)

Test whether o can be used in a for loop

assert is_iter([1])
assert not is_iter(array(1))
assert is_iter(array([1,2]))
assert (o for o in range(3))

coll_repr[source]

coll_repr(c, max_n=10)

String repr of up to max_n items of (possibly lazy) collection c

coll_repr is used to provide a more informative __repr__ about list-like objects. coll_repr and is used by L to build a __repr__ that displays the length of a list in addition to a preview of a list.

Below is an example of the __repr__ string created for a list of 1000 elements:

test_eq(coll_repr(range(1000)),    '(#1000) [0,1,2,3,4,5,6,7,8,9...]')
test_eq(coll_repr(range(1000), 5), '(#1000) [0,1,2,3,4...]')
test_eq(coll_repr(range(10),   5),   '(#10) [0,1,2,3,4...]')
test_eq(coll_repr(range(5),    5),    '(#5) [0,1,2,3,4]')

We can set the option max_n to optionally preview a specified number of items instead of the default:

test_eq(coll_repr(range(1000), max_n=5), '(#1000) [0,1,2,3,4...]')

is_bool[source]

is_bool(x)

Check whether x is a bool or None

mask2idxs[source]

mask2idxs(mask)

Convert bool mask or index list to index L

test_eq(mask2idxs([False,True,False,True]), [1,3])
test_eq(mask2idxs(array([False,True,False,True])), [1,3])
test_eq(mask2idxs(array([1,2,3])), [1,2,3])

cycle[source]

cycle(o)

Like itertools.cycle except creates list of Nones if o is empty

test_eq(itertools.islice(cycle([1,2,3]),5), [1,2,3,1,2])
test_eq(itertools.islice(cycle([]),3), [None]*3)
test_eq(itertools.islice(cycle(None),3), [None]*3)
test_eq(itertools.islice(cycle(1),3), [1,1,1])

zip_cycle[source]

zip_cycle(x, *args)

Like itertools.zip_longest but cycles through elements of all but first argument

test_eq(zip_cycle([1,2,3,4],list('abc')), [(1, 'a'), (2, 'b'), (3, 'c'), (4, 'a')])

is_indexer[source]

is_indexer(idx)

Test whether idx will index a single item in a list

You can, for example index a single item in a list with an integer or a 0-dimensional numpy array:

assert is_indexer(1)
assert is_indexer(np.array(1))

However, you cannot index into single item in a list with another list or a numpy array with ndim > 0.

assert not is_indexer([1, 2])
assert not is_indexer(np.array([[1, 2], [3, 4]]))

L helpers

class CollBase[source]

CollBase(items)

Base class for composing a list of items

ColBase is a base class that emulates the functionality of a python list:

class _T(CollBase): pass
l = _T([1,2,3,4,5])

test_eq(len(l), 5) # __len__
test_eq(l[-1], 5); test_eq(l[0], 1) #__getitem__
l[2] = 100; test_eq(l[2], 100)      # __set_item__
del l[0]; test_eq(len(l), 4)        # __delitem__
test_eq(str(l), '[2, 100, 4, 5]')   # __repr__

class L[source]

L(items=None, *rest, use_list=False, match=None) :: GetAttr

Behaves like a list of items but can also index with list of indices or masks

L is a drop in replacement for a python list. Inspired by NumPy, L, supports advanced indexing and has additional methods (outlined below) that provide additional functionality and encourage simple expressive code. For example, the code below takes a list of pairs, selects the second item of each pair, takes its absolute value, filters items greater than 4, and adds them up:

from fastcore.utils import gt

d = dict(a=1,b=-5,d=6,e=9).items()
test_eq(L(d).itemgot(1).map(abs).filter(gt(4)).sum(), 20) # abs(-5) + abs(6) + abs(9) = 20; 1 was filtered out.

Read this overview section for a quick tutorial of L, as well as background on the name.

You can create an L from an existing iterable (e.g. a list, range, etc) and access or modify it with an int list/tuple index, mask, int, or slice. All list methods can also be used with L.

t = L(range(12))
test_eq(t, list(range(12)))
test_ne(t, list(range(11)))
t.reverse()
test_eq(t[0], 11)
t[3] = "h"
test_eq(t[3], "h")
t[3,5] = ("j","k")
test_eq(t[3,5], ["j","k"])
test_eq(t, L(t))
test_eq(L(L(1,2),[3,4]), ([1,2],[3,4]))
t
(#12) [11,10,9,'j',7,'k',5,4,3,2...]

Any L is a Sequence so you can use it with methods like random.sample:

assert isinstance(t, Sequence)
import random
random.sample(t, 3)
['j', 5, 10]

There are optimized indexers for arrays, tensors, and DataFrames.

arr = np.arange(9).reshape(3,3)
t = L(arr, use_list=None)
test_eq(t[1,2], arr[[1,2]])


import pandas as pd
df = pd.DataFrame({'a':[1,2,3]})
t = L(df, use_list=None)
test_eq(t[1,2], L(pd.DataFrame({'a':[2,3]}, index=[1,2]), use_list=None))

You can also modify an L with append, +, and *.

t = L()
test_eq(t, [])
t.append(1)
test_eq(t, [1])
t += [3,2]
test_eq(t, [1,3,2])
t = t + [4]
test_eq(t, [1,3,2,4])
t = 5 + t
test_eq(t, [5,1,3,2,4])
test_eq(L(1,2,3), [1,2,3])
test_eq(L(1,2,3), L(1,2,3))
t = L(1)*5
t = t.map(operator.neg)
test_eq(t,[-1]*5)
test_eq(~L([True,False,False]), L([False,True,True]))
t = L(range(4))
test_eq(zip(t, L(1).cycle()), zip(range(4),(1,1,1,1)))
t = L.range(100)
test_shuffled(t,t.shuffle())
def _f(x,a=0): return x+a
t = L(1)*5
test_eq(t.map(_f), t)
test_eq(t.map(_f,1), [2]*5)
test_eq(t.map(_f,a=2), [3]*5)

An L can be constructed from anything iterable, although tensors and arrays will not be iterated over on construction, unless you pass use_list to the constructor.

test_eq(L([1,2,3]),[1,2,3])
test_eq(L(L([1,2,3])),[1,2,3])
test_ne(L([1,2,3]),[1,2,])
test_eq(L('abc'),['abc'])
test_eq(L(range(0,3)),[0,1,2])
test_eq(L(o for o in range(0,3)),[0,1,2])
test_eq(L(array(0)),[array(0)])
test_eq(L([array(0),array(1)]),[array(0),array(1)])
test_eq(L(array([0.,1.1]))[0],array([0.,1.1]))
test_eq(L(array([0.,1.1]), use_list=True), [array(0.),array(1.1)])  # `use_list=True` to unwrap arrays/arrays

If match is not None then the created list is same len as match, either by:

  • If len(items)==1 then items is replicated,
  • Otherwise an error is raised if match and items are not already the same size.
test_eq(L(1,match=[1,2,3]),[1,1,1])
test_eq(L([1,2],match=[2,3]),[1,2])
test_fail(lambda: L([1,2],match=[1,2,3]))

If you create an L from an existing L then you'll get back the original object (since L uses the NewChkMeta metaclass).

test_is(L(t), t)

An L is considred equal to a list if they have the same elements. It's never considered equal to a str a set or a dict even if they have the same elements/keys.

test_eq(L(['a', 'b']), ['a', 'b'])
test_ne(L(['a', 'b']), 'ab')
test_ne(L(['a', 'b']), {'a':1, 'b':2})

L Methods

L.__getitem__[source]

L.__getitem__(idx)

Retrieve idx (can be list of indices, or mask, or int) items

t = L(range(12))
test_eq(t[1,2], [1,2])                # implicit tuple
test_eq(t[[1,2]], [1,2])              # list
test_eq(t[:3], [0,1,2])               # slice
test_eq(t[[False]*11 + [True]], [11]) # mask
test_eq(t[array(3)], 3)

L.__setitem__[source]

L.__setitem__(idx, o)

Set idx (can be list of indices, or mask, or int) items to o (which is broadcast if not iterable)

t[4,6] = 0
test_eq(t[4,6], [0,0])
t[4,6] = [1,2]
test_eq(t[4,6], [1,2])

L.unique[source]

L.unique(sort=False, bidir=False, start=None)

Unique items, in stable order

test_eq(L(4,1,2,3,4,4).unique(), [4,1,2,3])

L.val2idx[source]

L.val2idx()

Dict from value to index

test_eq(L(1,2,3).val2idx(), {3:2,1:0,2:1})

L.filter[source]

L.filter(f=noop, negate=False, gen=False, **kwargs)

Create new L filtered by predicate f, passing args and kwargs to f

list(t)
[0, 1, 2, 3, 1, 5, 2, 7, 8, 9, 10, 11]
test_eq(t.filter(lambda o:o<5), [0,1,2,3,1,2])
test_eq(t.filter(lambda o:o<5, negate=True), [5,7,8,9,10,11])

L.argwhere[source]

L.argwhere(f, negate=False, **kwargs)

Like filter, but return indices for matching items

test_eq(t.argwhere(lambda o:o<5), [0,1,2,3,4,6])

L.map[source]

L.map(f, *args, gen=False, **kwargs)

Create new L with f applied to all items, passing args and kwargs to f

test_eq(L.range(4).map(operator.neg), [0,-1,-2,-3])

If f is a string then it is treated as a format string to create the mapping:

test_eq(L.range(4).map('#{}#'), ['#0#','#1#','#2#','#3#'])

If f is a dictionary (or anything supporting __getitem__) then it is indexed to create the mapping:

test_eq(L.range(4).map(list('abcd')), list('abcd'))

You can also pass the same arg params that bind accepts:

def f(a=None,b=None): return b
test_eq(L.range(4).map(f, b=arg0), range(4))

L.map_dict[source]

L.map_dict(f=noop, *args, gen=False, **kwargs)

Like map, but creates a dict from items to function results

test_eq(L(range(1,5)).map_dict(), {1:1, 2:2, 3:3, 4:4})
test_eq(L(range(1,5)).map_dict(operator.neg), {1:-1, 2:-2, 3:-3, 4:-4})

L.zip[source]

L.zip(cycled=False)

Create new L with zip(*items)

t = L([[1,2,3],'abc'])
test_eq(t.zip(), [(1, 'a'),(2, 'b'),(3, 'c')])
t = L([[1,2,3,4],['a','b','c']])
test_eq(t.zip(cycled=True ), [(1, 'a'),(2, 'b'),(3, 'c'),(4, 'a')])
test_eq(t.zip(cycled=False), [(1, 'a'),(2, 'b'),(3, 'c')])

L.map_zip[source]

L.map_zip(f, *args, cycled=False, **kwargs)

Combine zip and starmap

t = L([1,2,3],[2,3,4])
test_eq(t.map_zip(operator.mul), [2,6,12])

L.zipwith[source]

L.zipwith(*rest, cycled=False)

Create new L with self zip with each of *rest

b = [[0],[1],[2,2]]
t = L([1,2,3]).zipwith(b)
test_eq(t, [(1,[0]), (2,[1]), (3,[2,2])])

L.map_zipwith[source]

L.map_zipwith(f, *rest, cycled=False, **kwargs)

Combine zipwith and starmap

test_eq(L(1,2,3).map_zipwith(operator.mul, [2,3,4]), [2,6,12])

L.itemgot[source]

L.itemgot(*idxs)

Create new L with item idx of all items

test_eq(t.itemgot(1), b)

L.attrgot[source]

L.attrgot(k, default=None)

Create new L with attr k (or value k for dicts) of all items.

a = [SimpleNamespace(a=3,b=4),SimpleNamespace(a=1,b=2)]
test_eq(L(a).attrgot('b'), [4,2])

#Example of when items are a dict
b =[{'id': 15, 'name': 'nbdev'}, {'id': 17, 'name': 'fastcore'}]
test_eq(L(b).attrgot('id'), [15, 17])

L.sorted[source]

L.sorted(key=None, reverse=False)

New L sorted by key. If key is str use attrgetter; if int use itemgetter

test_eq(L(a).sorted('a').attrgot('b'), [2,4])

L.split[source]

L.split(s, sep=None, maxsplit=-1)

Class Method: Same as str.split, but returns an L

test_eq(L.split('a b c'), list('abc'))

L.range[source]

L.range(a, b=None, step=None)

Class Method: Same as range, but returns L. Can pass collection for a, to use len(a)

test_eq_type(L.range([1,1,1]), L(range(3)))
test_eq_type(L.range(5,2,2), L(range(5,2,2)))

L.concat[source]

L.concat()

Concatenate all elements of list

test_eq(L([0,1,2,3],4,L(5,6)).concat(), range(7))

L.copy[source]

L.copy()

Same as list.copy, but returns an L

t = L([0,1,2,3],4,L(5,6)).copy()
test_eq(t.concat(), range(7))

L.map_first[source]

L.map_first(f=noop, g=noop, *args, **kwargs)

First element of map_filter

t = L(0,1,2,3)
test_eq(t.map_first(lambda o:o*2 if o>2 else None), 6)

L.setattrs[source]

L.setattrs(attr, val)

Call setattr on all items

t = L(SimpleNamespace(),SimpleNamespace())
t.setattrs('foo', 'bar')
test_eq(t.attrgot('foo'), ['bar','bar'])

save_config_file[source]

save_config_file(file, d, **kwargs)

Write settings dict to a new config file, or overwrite the existing one.

read_config_file[source]

read_config_file(file, **kwargs)

Config files are saved and read using Python's configparser.ConfigParser, inside the DEFAULT section.

_d = dict(user='fastai', lib_name='fastcore', some_path='test')
try:
    save_config_file('tmp.ini', _d)
    res = read_config_file('tmp.ini')
finally: os.unlink('tmp.ini')
test_eq(res['DEFAULT'], _d)

Config[source]

Config(cfg_name='settings.ini')

Reading and writing settings.ini

Config searches parent directories for a config file, and provides direct access to the 'DEFAULT' section. Keys ending in _path are converted to paths in the config file's directory.

try:
    save_config_file('../tmp.ini', _d)
    cfg = Config('tmp.ini')
finally: os.unlink('../tmp.ini')

test_eq(cfg.user,'fastai')
test_eq(cfg.doc_baseurl,'/fastcore/')
test_eq(cfg.get('some_path'), 'test')
test_eq(cfg.path('some_path'), Path('../test').resolve())
test_eq(cfg.get('foo','bar'),'bar')