tools

Text and file editing primitives shared by the fastai editing tools

The text editors take a string and return the edited text. Invalid edits raise ValueError. File editors read and write a path and return a diff. Aidialog provides corresponding message editors.

Read fastcore.editskill for the editing toolkit’s naming, parameter and workflow conventions. It exposes these tools alongside the notebook editors from fastcore.nbio.

---

source

insert_line

def insert_line(
    text:str,
    insert_line:int, # The 1-based line number after which to insert the text (0: before 1st line, 1: after 1st line, 2: after 2nd, etc.)
    new_str:str, # The text to insert
):

Insert new_str at specified line number

test_eq(insert_line('a\nb', 0, 'first'), 'first\na\nb')
test_eq(insert_line('a\nb', 2, 'end'), 'a\nb\nend')
with expect_fail(ValueError, 'Invalid line'): insert_line('a', 5, 'x')

---

source

str_replace

def str_replace(
    text:str, old_str:str, # Text to find and replace
    new_str:str, # Text to replace with
    start_line:int=None, # Optional 1-based start line to limit search
    end_line:int=None, # Optional 1-based end line to limit search
    n_matches:int=None, # Max replacements (None=all)
    re_filter:str=None, # If provided, only process lines matching this regex (like g// in ex)
    invert_filter:bool=False, # Invert the filter (like g!// in ex)
    use_regex:bool=False, # Treat old_str as a regex, and new_str as an `re.sub` template?
):

Replace occurrence(s) of old_str with new_str

Pass use_regex=True to use regex patterns and re.sub replacement templates. Word boundaries avoid renaming parts of longer names:

res = str_replace('use x; only_use x; used x', r'\buse\b', 'usage', use_regex=True)
test_eq(res, 'usage x; only_use x; used x')
res = str_replace('a=1\nb=2', r'(\w+)=(\d+)', r'\2=\1', use_regex=True)
test_eq(res, '1=a\n2=b')
with expect_fail(Exception, 'Failed to find'): str_replace('abc', r'\bxyz\b', 'q', use_regex=True)

The text, file, notebook-cell and message editors share their replacement options. doc() module overviews list these once under replace_params, rather than repeating them for every function.

Use re_filter to restrict replacement to matching lines. invert_filter=True selects the other lines:

res = str_replace('keep q\nfix q\nkeep q', 'q', 'y', re_filter='fix')
test_eq(res, 'keep q\nfix y\nkeep q')
res = str_replace('keep q\nfix q\nkeep q', 'q', 'y', re_filter='fix', invert_filter=True)
test_eq(res, 'keep y\nfix q\nkeep y')
str_replace('a-b\na b', r'(\w+) (\w+)', r'\2 \1', re_filter=' ', use_regex=True)

'a-b\nb a'

start_line and end_line select an inclusive range. Their defaults are the first and last line. A negative end_line counts from the end, with -1 selecting the last line:

test_eq(str_replace('xa\nxb\nxc\n', 'x', 'y', start_line=2, end_line=-1), 'xa\nyb\nyc\n')
test_eq(str_replace('xa\nxb\nxc\n', 'x', 'y', end_line=2), 'ya\nyb\nxc\n')

---

source

strs_replace

def strs_replace(
    text:str, old_strs:list[str], # List of strings to find and replace
    new_strs:list[str], # List of replacement strings (must match length of old_strs)
    start_line:int=None, # Optional 1-based start line to limit search
    end_line:int=None, # Optional 1-based end line to limit search
    n_matches:int=None, # Max replacements per string (None=all)
    re_filter:str=None, # If provided, only process lines matching this regex (like g// in ex)
    invert_filter:bool=False, # Invert the filter (like g!// in ex)
    use_regex:bool=False, # Treat old_strs as regexes, and new_strs as `re.sub` templates?
):

Replace multiple strings simultaneously

res = strs_replace('f(a); g(a)', [r'\bf\b', r'\bg\b'], ['ff','gg'], use_regex=True)
test_eq(res, 'ff(a); gg(a)')

---

source

replace_lines

def replace_lines(
    text:str, start_line:int=None, # Starting line number to replace (1-based); None means line 1
    end_line:int=None, # Ending line number to replace (1-based, inclusive, negative counts from end); None means the last line
    new_content:str='', # New content to replace the specified lines
):

Replace line range with new content; the defaults replace the entire contents

Omit line numbers to replace the entire text, including empty text. Pass only start_line to replace from that line to the end:

test_eq(replace_lines('a\nb\nc\n', new_content='x\ny\n'), 'x\ny\n')
test_eq(replace_lines('', new_content='x\n'), 'x\n')
test_eq(replace_lines('a\nb\nc\n', 2, new_content='x\n'), 'a\nx\n')
test_eq(replace_lines('a\nb\nc\n', end_line=2, new_content='x\n'), 'x\nc\n')

---

source

del_lines

def del_lines(
    text:str, start_line:int, # Starting line number to delete (1-based); required
    end_line:int, # Ending line number to delete (1-based, inclusive, negative counts from end); required
    re_filter:str=None, # If provided, only delete lines matching this regex (like g// in ex)
    invert_filter:bool=False, # Invert the filter (like g!// in ex)
):

Delete line range; deletion is destructive, so both line numbers must be given explicitly (1, -1 for all lines)

del_lines requires both line numbers and rejects None. Use 1, -1 for all lines or 5, 5 for line 5:

test_eq(del_lines('a\nb\nc\n', 2, 2), 'a\nc\n')
test_eq(del_lines('a\nb\nc\n', 1, -1), '')
with expect_fail(Exception, 'explicit'): del_lines('a\nb\n', 1, None)

Line hashes

line_hash, lnhash and lnhash_at create exhash addresses without depending on the exhash package. Addresses use lineno|hash|. The hash is the low 12 bits of CRC32, encoded as two Base64url characters (A–Z, a–z, 0–9, -, _), high six bits first.

---

source

lnhash_at

def lnhash_at(
    s:str | list | tuple, # A document as a str, or its lines
    line:int, # 1-based line number within `s`
)->str:

lineno|hash| exhash address of line line of s

---

source

lnhash

def lnhash(
    lineno:int, # 1-based line number
    line:str, # The line's current text
)->str:

lineno|hash| exhash address for line at lineno

---

source

line_hash

def line_hash(
    line:str, # Text to hash; when hashing a single line, omit its trailing newline
)->str:

2-char Base64url hash of line

test_eq(line_hash('def foo(): pass'), 'Tw')  # same value the exhash Rust impl produces
s = 'a\ndef foo(): pass\nb'
test_eq(lnhash_at(s, 2), '2|Tw|')
test_eq(lnhash_at(s.splitlines(), 2), '2|Tw|')
test_eq(lnhash_at(s, 2), lnhash(2, 'def foo(): pass'))
lnhash_at(s, 2)

'2|Tw|'

File tools

File editors take the path first and return a unified diff. An unchanged result returns none: No changes. Invalid edits return error: ....

view_file('~/a/b.py', 3)
create_file('~/a/b/c.py', 'content here')
file_str_replace('myfile.py', 'old_name', 'new_name')
file_del_lines('myfile.py', 2, 4)
file_replace_lines('myfile.py', new_content=src)   # no line numbers: replace the entire contents

file_str_replace, file_strs_replace and file_del_lines support line ranges, re_filter and invert_filter. These work like ex’s g// and g!// filters.

For Python syntax-tree replacements, use ast_replace or file_ast_replace. They accept ast-grep (pattern, replacement) rules with $VAR metavariables and require the optional remold package.

---

source

view_file

def view_file(
    path:str, # Path to view (expands `~` if needed)
    start_line:int=1, # Starting line to view
    end_line:int=None, # End line (defaults to last line if None; may be past EOF, which clamps to the last line - handy when the file size is unknown)
    nums:bool=True, # Show line numbers?
    lnhashs:bool=False, # Show exhash `lineno|hash|` addresses instead of line numbers?
):

Read file contents, optionally limited to 1-based line range

tmp = TemporaryDirectory()
test_path = f'{tmp.name}/test.txt'
test_content = 'alpha\nbeta\ngamma\ndelta\n'
def test_txt(): return Path(test_path).read_text()
Path(test_path).write_text(test_content)
test_path

'/var/folders/mv/nt5dfl8j0xbg7zkfw_sk_d8m0000gn/T/tmpxaep7ttt/test.txt'

view_file(test_path, 2, 30)

2: beta
3: gamma
4: delta

res = view_file(test_path, 2, 3, lnhashs=True)
test_eq(res.splitlines()[0], f'{lnhash_at(test_content, 2)}beta')
test_eq(view_file(test_path, nums=False), test_content[:-1])
res

2|Rj|beta
3|Bx|gamma

---

source

create_file

def create_file(
    path:str, # Path to create (expands `~` if needed)
    contents:str, # Contents of file to create
    overwrite:bool=False, # Replace the file if it already exists?
):

Create a new file with contents. Error if file exists, unless overwrite.

new_path = Path(tmp.name)/'demo.txt'
print(create_file(new_path, 'one\ntwo\nthree\n'))
assert str(create_file(new_path, 'x')).startswith('error: File exists')
test_eq(create_file(new_path, 'four\n', overwrite=True), 'done')
test_eq(Path(new_path).read_text(), 'four\n')

done

---

source

file_edit

def file_edit(
    f, name:NoneType=None
):

Wrap text editor f as a file editing function: path addressing, diff-or-error return

res = file_insert_line(test_path, 0, 'first')
test_eq(test_txt().splitlines()[0], 'first')
assert repr(res) == str(res)
res

@@ -1 +1,2 @@
+first
 alpha

res = file_str_replace(test_path, 'beta', 'BETA')
assert 'BETA' in test_txt(), f"Expected 'BETA' in file"
res

@@ -2,3 +2,3 @@
 alpha
-beta
+BETA
 gamma

with expect_fail(ValueError, test_path): str_replace(test_path, 'beta', 'x')

res = file_strs_replace(test_path, ['gamma', 'delta'], ['GAMMA', 'DELTA'])
test_eq(test_txt().splitlines()[-2:], ['GAMMA', 'DELTA'])
res

@@ -3,3 +3,3 @@
 BETA
-gamma
-delta
+GAMMA
+DELTA

Use file_replace_lines(path, new_content=src) to replace a whole file. Notebook cells have the equivalent cell_replace_lines(fname, id, new_content=src).

res = file_replace_lines(test_path, 2, 3, 'two\nthree\n')
test_eq(test_txt().splitlines()[1:3], ['two', 'three'])
res

@@ -1,4 +1,4 @@
 first
-alpha
-BETA
+two
+three
 GAMMA

---

source

ast_replace

def ast_replace(
    text:str,
    repls:list, # (pattern, replacement) ast-grep rules; replacement is a `$VAR` template or a callable(match)->str
):

Apply ast-grep structural pattern replacements to python source

ast_replace and file_ast_replace use ast-grep patterns to match Python syntax. $VAR captures an expression. An unknown $VAR in the replacement raises an error.

Formatting and comments outside the matched span stay unchanged. Rules apply in order, each to the previous rule’s output.

res = ast_replace("print('a')\nx = 1  # keep\n", [("print($X)", "log($X)")])
test_eq(res, "log('a')\nx = 1  # keep\n")
with expect_fail(KeyError, '$Y'): ast_replace("print('a')\n", [("print($X)", "log($Y)")])

pyp = Path(tmp.name)/'ast.py'
pyp.write_text("print('a')\nprint(b)\n")
res = file_ast_replace(pyp, [("print($X)", "log($X)")])
test_eq(pyp.read_text(), "log('a')\nlog(b)\n")
res

@@ -1,2 +1,2 @@
-print('a')
-print(b)
+log('a')
+log(b)