Foundation

The L class and helpers for it

Foundational Functions

add_docs attaches docstrings to a class and its methods from a dictionary. docs is its decorator form. Both check that every public method has documentation.

coll_repr shows a collection’s length and a preview of its items. flatmap(f, xs) applies f to each item and concatenates the results.

source

working_directory

def working_directory(
    path
):

Change working directory to path and return to previous on exit.

source

add_docs

def add_docs(
    cls, cls_doc:NoneType=None, **docs
):

Copy values from docs to cls docstrings, and confirm all public methods are documented

add_docs groups a class’s docstrings separately from its method definitions. This lets you document one-line methods without expanding their definitions. We discuss this style in our style guide.

Suppose you have the following undocumented class:

class T:
    def foo(self): pass
    def bar(self): pass

You can add documentation to this class like so:

add_docs(T, cls_doc="A docstring for the class.",
            foo="The foo method.",
            bar="The bar method.")

Now, docstrings will appear as expected:

test_eq(T.__doc__, "A docstring for the class.")
test_eq(T.foo.__doc__, "The foo method.")
test_eq(T.bar.__doc__, "The bar method.")

add_docs also validates that all of your public methods contain a docstring. If one of your methods is not documented, it will raise an error:

class T:
    def foo(self): pass
    def bar(self): pass

with expect_fail(Exception, "Missing docs"): add_docs(T, "A docstring for the class.", foo="The foo method.")

source

docs

def docs(
    cls
):

Decorator version of add_docs, using _docs dict

To use the docs decorator, put the docstrings in the class’s _docs dictionary. Use method names as keys and cls_doc for the class docstring:

@docs
class _T:
    def f(self): pass
    def g(cls): pass
    
    _docs = dict(cls_doc="The class docstring", 
                 f="The docstring for method f.",
                 g="A different docstring for method g.")

    
test_eq(_T.__doc__, "The class docstring")
test_eq(_T.f.__doc__, "The docstring for method f.")
test_eq(_T.g.__doc__, "A different docstring for method g.")

For either the docs decorator or the add_docs function, you can still define your docstrings in the normal way. Below we set the docstring for the class as usual, but define the method docstrings through the _docs attribute:

@docs
class _T:
    "The class docstring"
    def f(self): pass
    _docs = dict(f="The docstring for method f.")

    
test_eq(_T.__doc__, "The class docstring")
test_eq(_T.f.__doc__, "The docstring for method f.")

is_iter

def is_iter(
    o
):

Test whether o can be used in a for loop

assert is_iter([1])
assert not is_iter(array(1))
assert is_iter(array([1,2]))
assert (o for o in range(3))

source

coll_repr

def coll_repr(
    c, max_n:int=250
):

String repr of up to max_n items of (possibly lazy) collection c

coll_repr is used to provide a more informative __repr__ about list-like objects. coll_repr and is used by L to build a __repr__ that displays the length of a list in addition to a preview of a list.

test_eq(coll_repr(range(1000),10), '(#1000) [0, 1, 2, 3, 4, 5, 6, 7, 8, 9...]')
test_eq(coll_repr(range(1000), 5), '(#1000) [0, 1, 2, 3, 4...]')
test_eq(coll_repr(range(10),   5), '(#10) [0, 1, 2, 3, 4...]')
test_eq(coll_repr(range(5),    5), '[0, 1, 2, 3, 4]')

source

is_bool

def is_bool(
    x
):

Check whether x is a bool or None

source

mask2idxs

def mask2idxs(
    mask
):

Convert bool mask or index list to index L

test_eq(mask2idxs([False,True,False,True]), [1,3])
test_eq(mask2idxs(array([False,True,False,True])), [1,3])
test_eq(mask2idxs(array([1,2,3])), [1,2,3])

source

cycle

def cycle(
    o
):

Like itertools.cycle except creates list of Nones if o is empty

test_eq(itertools.islice(cycle([1,2,3]),5), [1,2,3,1,2])
test_eq(itertools.islice(cycle([]),3), [None]*3)
test_eq(itertools.islice(cycle(None),3), [None]*3)
test_eq(itertools.islice(cycle(1),3), [1,1,1])

source

zip_cycle

def zip_cycle(
    x, *args
):

Like itertools.zip_longest but cycles through elements of all but first argument

test_eq(zip_cycle([1,2,3,4],list('abc')), [(1, 'a'), (2, 'b'), (3, 'c'), (4, 'a')])

source

is_indexer

def is_indexer(
    idx
):

Test whether idx will index a single item in a list

You can, for example index a single item in a list with an integer or a 0-dimensional numpy array:

assert is_indexer(1)
assert is_indexer(np.array(1))

However, you cannot index into single item in a list with another list or a numpy array with ndim > 0.

assert not is_indexer([1, 2])
assert not is_indexer(np.array([[1, 2], [3, 4]]))

source

product

def product(
    xs
):

The product of elements of xs, with Nones removed

product([None, 3, 4, 5])

product([])

sum([])

flatmap

source

flatmap

def flatmap(
    f, xs, **kwargs
):

Apply f to each element and flatten the results into a single list.

flatmap(f, xs) returns [y for x in xs for y in f(x)]. Each call to f returns an iterable. Its items become part of the resulting list:

flatmap(range, range(4))

[0, 0, 1, 0, 1, 2]

Compare map (which nests results) with flatmap (which flattens them):

list(map(str.split, ["hello world", "foo bar"]))  # nested

[['hello', 'world'], ['foo', 'bar']]

flatmap(str.split, ["hello world", "flatmap rocks"])

['hello', 'world', 'flatmap', 'rocks']

Parse CSV-like lines into all values:

flatmap(~Self.split(','), ["a,b,c", "d,e"])

['a', 'b', 'c', 'd', 'e']

Return [] to skip an item, [x] to keep it, or [x, y, ...] to expand it:

flatmap(lambda x: [x*10] if x else [], [1, 0, 2])  # skips zeros

[10, 20]

dat = [{'emails': ['[email protected]','[email protected]']}, {'emails': []}, {'emails': ['[email protected]']}]
flatmap(~Self['emails'], dat)

['[email protected]', '[email protected]', '[email protected]']

All files in multiple directories:

flatmap(~Self.iterdir(), [Path('files'), Path('images')])

[Path('files/test.txt.bz2'),
 Path('images/mnist3.png'),
 Path('images/att_00000.png'),
 Path('images/att_00005.png'),
 Path('images/att_00007.png'),
 Path('images/att_00006.png'),
 Path('images/puppy.jpg')]

Pair each item with its factors:

def factpairs(n): return [(n,i) for i in range(1,n+1) if n%i==0]
flatmap(factpairs, [6,10])

[(6, 1), (6, 2), (6, 3), (6, 6), (10, 1), (10, 2), (10, 5), (10, 10)]

You can also use kwargs, for instance to apply str.split with a custom separator:

flatmap(str.split, ["a-b-c", "d-e"], sep="-")

['a', 'b', 'c', 'd', 'e']

`L` helpers

source

CollBase

def CollBase(
    items
):

Base class for composing a list of items

ColBase is a base class that emulates the functionality of a python list:

class _T(CollBase): pass
l = _T([1,2,3,4,5])

test_eq(len(l), 5) # __len__
test_eq(l[-1], 5); test_eq(l[0], 1) #__getitem__
l[2] = 100; test_eq(l[2], 100)      # __set_item__
del l[0]; test_eq(len(l), 4)        # __delitem__
test_eq(str(l), '[2, 100, 4, 5]')   # __repr__

source

L

def L(
    items:NoneType=None, *rest, use_list:bool=False, match:NoneType=None
):

Behaves like a list of items but can also index with list of indices or masks

L extends Python’s list operations with indexing inspired by NumPy. You can index with an integer, slice, collection of integers or boolean mask. Methods such as map, filter, sorted, unique, itemgot and attrgot return an L:

Examples and overview

from fastcore.utils import gt

Read this overview section for a quick tutorial of L, as well as background on the name.

You can create an L from an existing iterable (e.g. a list, range, etc) and access or modify it with an int list/tuple index, mask, int, or slice. All list methods can also be used with L.

t = L(range(12))
test_eq(t, list(range(12)))
test_ne(t, list(range(11)))
t[3] = "h"
test_eq(t[3], "h")
t[3,5] = ("j","k")
test_eq(t[3,5], ["j","k"])
test_eq(t, L(t))
test_eq(L(L(1,2),[3,4]), ([1,2],[3,4]))
t[0:3] = [1, 2, 3]
test_eq(t[0:3], [1, 2, 3])
t

[1, 2, 3, 'j', 4, 'k', 6, 7, 8, 9, 10, 11]

Any L is a Sequence so you can use it with methods like random.sample:

assert isinstance(t, Sequence)

import random

random.seed(0)
random.sample(t, 3)

[6, 11, 1]

There are optimized indexers for arrays, tensors, and DataFrames.

import pandas as pd

arr = np.arange(9).reshape(3,3)
t = L(arr, use_list=None)
test_eq(t[1,2], arr[[1,2]])

df = pd.DataFrame({'a':[1,2,3]})
t = L(df, use_list=None)
test_eq(t[1,2], L(pd.DataFrame({'a':[2,3]}, index=[1,2]), use_list=None))

You can also modify an L with append, +, and *.

t = L()
test_eq(t, [])
t.append(1)
test_eq(t, [1])
t += [3,2]
test_eq(t, [1,3,2])
t = t + [4]
test_eq(t, [1,3,2,4])
t = 5 + t
test_eq(t, [5,1,3,2,4])
test_eq(L(1,2,3), [1,2,3])
test_eq(L(1,2,3), L(1,2,3))
t = L(1)*5
test_eq(~L([True,False,False]), L([False,True,True]))

You can construct an L from an iterable. By default, L keeps a tensor or array as one item. Pass use_list=True to collect its elements instead:

test_eq(L([1,2,3]),[1,2,3])
test_eq(L(L([1,2,3])),[1,2,3])
test_ne(L([1,2,3]),[1,2,])
test_eq(L('abc'),['abc'])
test_eq(L(range(0,3)),[0,1,2])
test_eq(L(o for o in range(0,3)),[0,1,2])
test_eq(L(array(0)),[array(0)])
test_eq(L([array(0),array(1)]),[array(0),array(1)])
test_eq(L(array([0.,1.1]))[0],array([0.,1.1]))
test_eq(L(array([0.,1.1]), use_list=True), [array(0.),array(1.1)])  # `use_list=True` to unwrap arrays/arrays

If match is not None then the created list is same len as match, either by:

If len(items)==1 then items is replicated,
Otherwise an error is raised if match and items are not already the same size.

test_eq(L(1,match=[1,2,3]),[1,1,1])
test_eq(L([1,2],match=[2,3]),[1,2])
with expect_fail(): L([1,2],match=[1,2,3])

If you create an L from an existing L then you’ll get back the original object (since L uses the NewChkMeta metaclass).

test_is(L(t), t)

An L is considred equal to a list if they have the same elements. It’s never considered equal to a str a set or a dict even if they have the same elements/keys.

test_eq(L(['a', 'b']), ['a', 'b'])
test_ne(L(['a', 'b']), 'ab')
test_ne(L(['a', 'b']), {'a':1, 'b':2})

xdumps serializes an L as a regular list, including Ls nested inside other objects. L supplies this conversion through its __json__ method.

items = dict(names=L('Alyssa', 'Ben'), scores=L(8, 9))
test_eq(json.loads(xdumps(items)), dict(names=['Alyssa','Ben'], scores=[8,9]))

`L` Methods

source

L.getitem

def __getitem__(
    idx
):

Retrieve idx (can be list of indices, or mask, or int) items

t = L(range(12))
test_eq(t[1,2], [1,2])                # implicit tuple
test_eq(t[[1,2]], [1,2])              # list
test_eq(t[:3], [0,1,2])               # slice
test_eq(t[[False]*11 + [True]], [11]) # mask
test_eq(t[array(3)], 3)

Slicing or selecting from an L subclass returns the same subclass:

class _Rows(L):
    def __repr__(self): return '\n'.join(f'row {o}' for o in self)

r = _Rows(range(5))
for sel in (slice(1,3), [0,2], [True,False,True,False,True], (1,2)): test_eq(type(r[sel]), _Rows)
test_eq(r[1], 1)  # a single item is still the item, not a collection
r[1:3]

row 1
row 2

Override _new to preserve subclass attributes in results such as slices and concatenations. It receives only the resulting items. Here _Titled copies its title when slicing or adding items:

class _Titled(L):
    def __init__(self, items=None, title=None, **kw):
        super().__init__(items, **kw)
        self.title = title
    def _new(self, items, *args, **kw):
        res = super()._new(items, *args, **kw)
        res.title = self.title
        return res

tl = _Titled(range(5), title='counts')
test_eq(tl[1:3].title, 'counts')
test_eq((tl + [5]).title, 'counts')                # one hook covers every derived collection
test_eq(_Titled(range(3))[:2].title, None)

Keep per-item state on the items themselves. _new doesn’t receive the selected indices and cannot select corresponding entries from a separate attribute.

self.items stores the collection’s contents. Use another name for subclass attributes.

_new(items) must preserve the items it receives. Subclasses must also preserve indexing and iteration behavior. For a single index, x[i] must return x.items[i]. list(x) must equal list(x.items). Methods such as map and filter depend on these rules. For a lazy view that transforms items on access, store an L as an attribute instead of subclassing it.

To select raw items into a plain L, use L(x.items, use_list=None)[idx]. use_list=None preserves an array-backed collection. Calling super().__getitem__ can still invoke overridden methods.

source

L.setitem

def __setitem__(
    idx, o
):

Set idx (can be list of indices, or mask, or int) items to o (which is broadcast if not iterable)

t[4,6] = 0
test_eq(t[4,6], [0,0])
t[4,6] = [1,2]
test_eq(t[4,6], [1,2])

source

L.unique

def unique(
    sort:bool=False, bidir:bool=False, start:NoneType=None
):

Unique items, in stable order

test_eq(L(4,1,2,3,4,4).unique(), [4,1,2,3])

source

L.val2idx

def val2idx():

Dict from value to index

test_eq(L(1,2,3).val2idx(), {3:2,1:0,2:1})

source

L.range

def range(
    a, b:NoneType=None, step:NoneType=None
):

Class Method: Same as range, but returns L. Can pass collection for a, to use len(a)

test_eq_type(L.range([1,1,1]), L(range(3)))
test_eq_type(L.range(5,2,2), L(range(5,2,2)))

source

L.enumerate

def enumerate():

Same as enumerate

test_eq(L('a','b','c').enumerate(), [(0,'a'),(1,'b'),(2,'c')])

source

L.renumerate

def renumerate():

Same as renumerate

test_eq(L('a','b','c').renumerate(), [('a', 0), ('b', 1), ('c', 2)])

source

L.split

def split(
    s, sep:NoneType=None, maxsplit:int=-1
):

Class Method: Same as str.split, but returns an L

L.split is a class method that works like str.split, but returns an L instead of a list:

test_eq(L.split('a b c'), ['a','b','c'])
test_eq(L.split('a-b-c', '-'), ['a','b','c'])
test_eq(L.split('a-b-c', '-', maxsplit=1), ['a','b-c'])

source

L.splitlines

def splitlines(
    s, keepends:bool=False
):

Class Method: Same as str.splitlines, but returns an L

L.splitlines is a class method that works like str.splitlines, but returns an L instead of a list:

test_eq(L.splitlines('a\nb\nc'), ['a','b','c'])
test_eq(L.splitlines('a\nb\nc', keepends=True), ['a\n','b\n','c'])

source

curryable

def curryable(
    f
):

Call a curryable method on the class to create a function that accepts an iterable later. For example, you can convert each nested list with a lambda:

L(lines).map(lambda x: L(x).map(int))

Or pass the curried L.map(int):

L(lines).map(L.map(int))

Curried calls also work with adapters such as star. To multiply the pairs inside each sublist:

nested.map(L.map(star(operator.mul)))

map

def map(
    f, *args, **kwargs
):

Create new L with f applied to all items, passing args and kwargs to f

t = L(1,2,3)
t.append(4)
test_eq(t[[0,3]], [1,4])
test_eq(t[[False,True,False,True]], [2,4])
test_eq(t.map(lambda o:o*2), [2,4,6,8])

Call L.map(int) on the class to get a function that converts an iterable’s items to integers. This curried form works with map, filter, groupby, argwhere, argfirst, first, last, sorted, reduce, partition, takewhile, dropwhile and accumulate.

star(f) unpacks the last argument before calling f. In map, that argument is the item. In reduce, it is the item after the accumulator. rstar reverses the unpacked arguments.

test_eq(L.range(4).map(operator.neg), [0,-1,-2,-3])

If f is a string then it is treated as a format string to create the mapping:

test_eq(L.range(4).map('#{}#'), ['#0#','#1#','#2#','#3#'])

For a dictionary or other indexable object f, map looks up f[item] for each item:

test_eq(L.range(4).map(list('abcd')), list('abcd'))

You can also pass the same arg params that bind accepts:

def f(a=None,b=None): return b
test_eq(L.range(4).map(f, b=arg0), range(4))

source

star

def star(
    f
):

Adapt f to unpack its last argument, e.g. for use in map-style functions

star(f) unpacks the last argument before calling f. Use it when each item contains multiple arguments. With reduce, the accumulator stays first and the item’s values follow it:

test_eq(L((1,2),(3,4)).map(star(operator.add)), [3,7])

source

rstar

def rstar(
    f
):

Like star, but unpack the last argument in reverse order

rstar is the same adapter with the unpacked arguments reversed, for when the function expects them in the opposite order to how they’re stored:

test_eq(L((1,2),(4,7)).map(rstar(operator.sub)), [1,3])  # b-a

source

splitter

def splitter(
    sep:NoneType=None, maxsplit:int=-1
):

Create a partial function that splits strings into L

A curried version of L.split, useful for mapping over collections of strings. For instance to split some lines with the same separator:

data = '''1,2,3
4,5,6
7,8,9'''

grid = L.splitlines(data).map(splitter(','))
grid

[['1', '2', '3'], ['4', '5', '6'], ['7', '8', '9']]

Use the curried L.map(int) to convert every row of grid to integers:

intgrid = grid.map(L.map(int))
intgrid

[[1, 2, 3], [4, 5, 6], [7, 8, 9]]

Although in this particular example numpy has a useful shortcut:

np.genfromtxt(data.splitlines(), delimiter=',', dtype=int)

array([[1, 2, 3],
       [4, 5, 6],
       [7, 8, 9]])

source

linesplitter

def linesplitter(
    keepends:bool=False
):

Create a partial function that splits strings by lines into L

A curried version of L.splitlines, useful for splitting multi-line strings into Ls when mapping over a collection.

L(['a\nb\nc', 'd\ne']).map(linesplitter())

[['a', 'b', 'c'], ['d', 'e']]

source

groupby

def groupby(
    key, val:function=noop
):

Same as fastcore.basics.groupby

words = L.split('aaa abc bba')
test_eq(words.groupby(0, (1,2)), {'a':[('a','a'),('b','c')], 'b':[('b','a')]})

Group each inner collection’s strings by their first character:

L([['a1','b2','a3'], ['x1','y2','x3']]).map(L.groupby(0))

[{'a': ['a1', 'a3'], 'b': ['b2']}, {'x': ['x1', 'x3'], 'y': ['y2']}]

source

L.map_dict

def map_dict(
    f:function=noop, *args, **kwargs
):

Like map, but creates a dict from items to function results

test_eq(L(range(1,5)).map_dict(), {1:1, 2:2, 3:3, 4:4})
test_eq(L(range(1,5)).map_dict(operator.neg), {1:-1, 2:-2, 3:-3, 4:-4})

source

L.zip

def zip(
    cycled:bool=False
):

Create new L with zip(*items)

t = L([[1,2,3],'abc'])
test_eq(t.zip(), [(1, 'a'),(2, 'b'),(3, 'c')])

t = L([[1,2,3,4],['a','b','c']])
test_eq(t.zip(cycled=True ), [(1, 'a'),(2, 'b'),(3, 'c'),(4, 'a')])
test_eq(t.zip(cycled=False), [(1, 'a'),(2, 'b'),(3, 'c')])

source

L.map_zip

def map_zip(
    f, *args, cycled:bool=False, **kwargs
):

Apply f to zip of items, unpacking each zipped tuple

t = L([1,2,3],[2,3,4])
test_eq(t.map_zip(operator.mul), [2,6,12])

source

L.zipwith

def zipwith(
    *rest, cycled:bool=False
):

Create new L with self zip with each of *rest

b = [[0],[1],[2,2]]
t = L([1,2,3]).zipwith(b)
test_eq(t, [(1,[0]), (2,[1]), (3,[2,2])])

source

L.map_zipwith

def map_zipwith(
    f, *rest, cycled:bool=False, **kwargs
):

Apply f to zipwith of items, unpacking each zipped tuple

test_eq(L(1,2,3).map_zipwith(operator.mul, [2,3,4]), [2,6,12])

filter

def filter(
    f:function=noop, negate:bool=False, **kwargs
):

Create new L filtered by predicate f, passing args and kwargs to f

t = L(range(12))
test_eq(t.filter(lambda o:o<5), [0,1,2,3,4])
test_eq(t.filter(lambda o:o<5, negate=True), [5,6,7,8,9,10,11])

Filter each row of intgrid with a curried call:

intgrid

[[1, 2, 3], [4, 5, 6], [7, 8, 9]]

intgrid.map(L.filter(ge(5)))

[[], [5, 6], [7, 8, 9]]

source

argwhere

def argwhere(
    f, negate:bool=False, **kwargs
):

Like filter, but return indices for matching items

t = L([0,1,2,3,4,99,0])
test_eq(t.argwhere(lambda o:o<5), [0,1,2,3,4,6])

source

first

def first(
    f, negate:bool=False
):

Return first matching item

test_eq(t.first(lambda o:o>4), 99)
test_eq(t.first(lambda o:o>4,negate=True), 0)

nested = L([[1,2,8,4], [5,9,7], [1,1,1]])
nested.map(L.first(gt(5)))

[8, 9, None]

source

last

def last(
    f, negate:bool=False
):

Return last matching item

test_eq(L(5,4,3,2,99,1).last(lambda o:o>4), 99)
test_eq(L(5,4,3,2,99,1).last(lambda o:o>4,negate=True), 1)

nested = L([[1,2,8,4], [5,9,7], [1,1,1]])
nested.map(L.last(gt(5)))

[8, 7, None]

argfirst

def argfirst(
    f, negate:bool=False
):

Return index of first matching item

test_eq(t.argfirst(lambda o:o>4), 5)
test_eq(t.argfirst(lambda o:o>4,negate=True),0)

Find the index of the first value greater than 5 in each inner collection:

nested = L([[1,2,8,4], [5,9,7], [1,1,1]])
nested.map(L.argfirst(gt(5)))

[2, 1, None]

source

L.itemgot

def itemgot(
    *idxs
):

Create new L with item idx of all items

t = L([['x', [0]], ['y', [1]], ['z', [2,2]]])
test_eq(t.itemgot(1), b)

source

L.attrgot

def attrgot(
    k, default:NoneType=None
):

Create new L with attr k (or value k for dicts) of all items.

# Example when items are not a dict
a = [SimpleNamespace(a=3,b=4),SimpleNamespace(a=1,b=2)]
test_eq(L(a).attrgot('b'), [4,2])

#Example of when items are a dict
b =[{'id': 15, 'name': 'nbdev'}, {'id': 17, 'name': 'fastcore'}]
test_eq(L(b).attrgot('id'), [15, 17])

sorted

def sorted(
    key:NoneType=None, reverse:bool=False, cmp:NoneType=None, **kwargs
):

New L sorted by key, using sort_ex. If key is str use attrgetter; if int use itemgetter

test_eq(L(a).sorted('a').attrgot('b'), [2,4])

Sort each inner collection by the first element of each tuple:

nested = L([[(3,'c'),(1,'a'),(2,'b')], [(6,'f'),(4,'d')]])
nested.map(L.sorted(0))

[[(1, 'a'), (2, 'b'), (3, 'c')], [(4, 'd'), (6, 'f')]]

source

L.concat

def concat():

Concatenate all elements of list

test_eq(L([0,1,2,3],4,L(5,6)).concat(), range(7))

source

L.copy

def copy():

Same as list.copy, but returns an L

source

L.copy

def __copy__():

copy.copy creates a separate items list. Appending to the copy leaves the original unchanged:

a = L(1,2,3)
b = copy(a)
b.append(4)
test_eq(a, [1,2,3])
test_eq(b, [1,2,3,4])

t = L([0,1,2,3],4,L(5,6)).copy()
test_eq(t.concat(), range(7))

source

L.shuffle

def shuffle():

Same as random.shuffle, but not inplace

L.shuffle returns a new shuffled L, leaving the original unchanged:

t = L(1,2,3,4,5)
s = t.shuffle()
test_eq(set(s), set(t))  # same elements
test_eq(t, [1,2,3,4,5])  # original unchanged

reduce

def reduce(
    f, initial:NoneType=None
):

Wrapper for functools.reduce

test_eq(L(1,2,3,4).reduce(operator.add), 10)
test_eq(L(1,2,3,4).reduce(operator.mul, 10), 240)

Sum each inner collection:

nested = L([[1,2,3], [4,5], [6,7,8,9]])
nested.map(L.reduce(operator.add))

[6, 9, 30]

E.g implement a dot product:

def dot(a,b): return a.zipwith(b).reduce(star(lambda acc,a,b: acc+a*b), 0)
dot(L(1,3,5), L(2,4,6))

source

L.sum

def sum():

Sum of the items

test_eq(L(1,2,3,4).sum(), 10)
test_eq(L().sum(), 0)

source

L.product

def product():

Product of the items

test_eq(L(1,2,3,4).product(), 24)
test_eq(L().product(), 1)

source

L.map_first

def map_first(
    f:function=noop, g:function=noop, *args, **kwargs
):

First element of map_filter

t = L(0,1,2,3)
test_eq(t.map_first(lambda o:o*2 if o>2 else None), 6)

source

L.setattrs

def setattrs(
    attr, val
):

Call setattr on all items

t = L(SimpleNamespace(),SimpleNamespace())
t.setattrs('foo', 'bar')
test_eq(t.attrgot('foo'), ['bar','bar'])

source

L.flatmap

def flatmap(
    f, **kwargs
):

Apply f to each element and flatten the results into a single L.

L.flatmap applies flatmap and returns an L:

test_eq(L("a,b,c", "d,e").flatmap(~Self.split(',')), ['a', 'b', 'c', 'd', 'e'])

or alternatively use kwargs:

test_eq(L("a-b-c", "d-e").flatmap(str.split, sep='-'), ['a', 'b', 'c', 'd', 'e'])

As an alternative, you can just chain map and concat:

L("a,b,c", "d,e").map(~Self.split(',')).concat()

['a', 'b', 'c', 'd', 'e']

L("a-b-c", "d-e").map(str.split, sep='-').concat()

['a', 'b', 'c', 'd', 'e']

itertools wrappers

L has methods for itertools operations including cycle, takewhile, dropwhile, accumulate, pairwise, batched, compress, permutations and combinations.

partition returns two collections according to a predicate. flatten recursively flattens nested iterables, leaving strings intact.

source

L.cycle

def cycle():

Same as itertools.cycle

L.cycle returns an infinite iterator that cycles through the elements:

test_eq(list(itertools.islice(L(1,2,3).cycle(), 7)), [1,2,3,1,2,3,1])

takewhile

def takewhile(
    f
):

Same as itertools.takewhile

L.takewhile returns elements from the beginning of the list while the predicate is true:

test_eq(L(1,2,3,4,5,1,2).takewhile(lambda x: x<4), [1,2,3])
test_eq(L(1,2,3,11).takewhile(lt(10)), [1,2,3])

Take values from each inner collection until reaching one that is 5 or greater:

nested = L([[1,2,5,3], [2,3,8,1], [9,1,2]])
nested.map(L.takewhile(lt(5)))

[[1, 2], [2, 3], []]

dropwhile

def dropwhile(
    f
):

Same as itertools.dropwhile

L.dropwhile skips elements from the beginning while the predicate is true, then returns the rest:

test_eq(L(1,2,3,4,5,1,2).dropwhile(lt(4)), [4,5,1,2])
test_eq(L(1,2,3).dropwhile(lt(10)), [])

accumulate

def accumulate(
    f:builtin_function_or_method=add, initial:NoneType=None
):

Same as itertools.accumulate

L.accumulate returns running totals (or running results of any binary function):

test_eq(L(1,2,3,4).accumulate(), [1,3,6,10])
test_eq(L(1,2,3,4).accumulate(operator.mul), [1,2,6,24])
test_eq(L(1,2,3).accumulate(initial=10), [10,11,13,16])

Calculate running products for each inner collection:

nested = L([[1,2,3], [4,5,6], [10,20]])
nested.map(L.accumulate(operator.mul))

[[1, 2, 6], [4, 20, 120], [10, 200]]

source

L.pairwise

def pairwise():

Same as itertools.pairwise

L.pairwise returns consecutive overlapping pairs:

test_eq(L(1,2,3,4).pairwise(), [(1,2),(2,3),(3,4)])
test_eq(L(list('abcd')).pairwise(), [('a','b'),('b','c'),('c','d')])

source

L.batched

def batched(
    n
):

Same as itertools.batched (but also works on older Python versions

L.batched splits into chunks of size n:

test_eq(L(1,2,3,4,5).batched(2), [(1,2),(3,4),(5,)])
test_eq(L(list('abcdefg')).batched(3), [('a','b','c'),('d','e','f'),('g',)])

source

L.compress

def compress(
    selectors
):

Same as itertools.compress

L.compress filters elements using a boolean selector:

test_eq(L(list('abcd')).compress([1,0,1,0]), ['a','c'])
test_eq(L(1,2,3,4,5).compress([True,False,True,False,True]), [1,3,5])

source

L.permutations

def permutations(
    r:NoneType=None
):

Same as itertools.permutations

L.permutations returns all permutations of length r (defaults to full length):

test_eq(L(1,2,3).permutations(), [(1,2,3),(1,3,2),(2,1,3),(2,3,1),(3,1,2),(3,2,1)])
test_eq(L(list('abc')).permutations(2), [('a','b'),('a','c'),('b','a'),('b','c'),('c','a'),('c','b')])

source

L.combinations

def combinations(
    r
):

Same as itertools.combinations

L.combinations returns all combinations of length r:

test_eq(L(1,2,3,4).combinations(2), [(1,2),(1,3),(1,4),(2,3),(2,4),(3,4)])
test_eq(L(list('abcd')).combinations(3), [('a','b','c'),('a','b','d'),('a','c','d'),('b','c','d')])

source

partition

def partition(
    f:function=noop, **kwargs
):

Split into two Ls based on predicate f: (true_items, false_items)

L.partition returns two Ls: items for which f is true, then items for which it is false:

t,f = L(1,2,3,4,5,6).partition(lambda x: x%2==0)
test_eq(t, [2,4,6])
test_eq(f, [1,3,5])

evens,odds = L.range(10).partition(lambda x: x%2==0)
test_eq(evens, [0,2,4,6,8])
test_eq(odds, [1,3,5,7,9])

Split each inner collection into values greater than 5 and the rest:

nested = L([[1,2,3,4,5], [10,15,20,25], [3,6,9]])
nested.map(L.partition(gt(5)))

[([], [1, 2, 3, 4, 5]), ([10, 15, 20, 25], []), ([6, 9], [3])]

source

L.flatten

def flatten():

Recursively flatten nested iterables (except strings)

L.flatten recursively flattens nested iterables into a single L. Strings are treated as atomic (not iterated over):

test_eq(L([[1,2],[3,[4,5]]]).flatten(), [1,2,3,4,5])
test_eq(L([1,[2,[3,[4]]]]).flatten(), [1,2,3,4])
test_eq(L(['a',['b','c'],'d']).flatten(), ['a','b','c','d'])  # strings not flattened
test_eq(L([1,2,3]).flatten(), [1,2,3])  # already flat

Since star and rstar are plain function adapters, they compose with all of L’s methods, including the curried forms:

test_eq(L((1,2),(3,1),(2,3)).filter(star(lt)), [(1,2),(2,3)])
test_eq(L((3,1),(1,2),(2,0)).sorted(star(operator.sub)), [(1,2),(3,1),(2,0)])
test_eq(L((1,2),(3,4),(5,6)).reduce(star(lambda acc,a,b: acc+a*b), 0), 44)
test_eq(L((2,1),(3,2),(1,4)).takewhile(rstar(lt)), [(2,1),(3,2)])
test_eq(L([[(1,2),(3,4)], [(5,6)]]).map(L.map(star(operator.mul))), [[2,12],[30]])

Foundational Functions

working_directory

add_docs

docs

is_iter

coll_repr

is_bool

mask2idxs

cycle

zip_cycle

is_indexer

product

flatmap

flatmap

L helpers

CollBase

L

Examples and overview

L Methods

L.__getitem__

L.__setitem__

L.unique

L.val2idx

L.range

L.enumerate

L.renumerate

L.split

L.splitlines

curryable

map

star

rstar

splitter

linesplitter

groupby

L.map_dict

L.zip

L.map_zip

L.zipwith

L.map_zipwith

filter

argwhere

first

last

argfirst

L.itemgot

L.attrgot

sorted

L.concat

L.copy

L.__copy__

L.shuffle

reduce

L.sum

L.product

L.map_first

L.setattrs

L.flatmap

itertools wrappers

L.cycle

takewhile

dropwhile

accumulate

L.pairwise

L.batched

L.compress

L.permutations

L.combinations

partition

L.flatten

`L` helpers

`L` Methods

L.getitem

L.setitem

L.copy