tools

Text and file editing primitives shared by the fastai editing tools

The editors here are string-level: each takes text plus edit parameters and returns the new text, raising ValueError when an edit can’t apply. The file tools below wrap them with path I/O and diff reporting; message-level wrappers live in aidialog. (This module previously held experimental LLM path-editing and command tools, superseded by safecmd, rgapi, and the tools here.) Naming, parameter, and workflow conventions for the whole editing toolkit, this module included, are documented in fastcore.editskill, which also re-exports these tools alongside fastcore.nbio’s.

---

source

insert_line

def insert_line(
    text:str,
    insert_line:int, # The 1-based line number after which to insert the text (0: before 1st line, 1: after 1st line, 2: after 2nd, etc.)
    new_str:str, # The text to insert
):

Insert new_str at specified line number

test_eq(insert_line('a\nb', 0, 'first'), 'first\na\nb')
test_eq(insert_line('a\nb', 2, 'end'), 'a\nb\nend')
with expect_fail(ValueError, 'Invalid line'): insert_line('a', 5, 'x')

---

source

str_replace

def str_replace(
    text:str, old_str:str, # Text to find and replace
    new_str:str, # Text to replace with
    start_line:int=None, # Optional 1-based start line to limit search
    end_line:int=None, # Optional 1-based end line to limit search
    n_matches:int=None, # Max replacements (None=all)
    re_filter:str=None, # If provided, only process lines matching this regex (like g// in ex)
    invert_filter:bool=False, # Invert the filter (like g!// in ex)
    use_regex:bool=False, # Treat old_str as a regex, and new_str as an `re.sub` template?
):

Replace occurrence(s) of old_str with new_str

With use_regex=True, old_str is a regex and new_str an re.sub template, so word boundaries and backrefs work. Handy for renames where plain substring matching would hit longer names too:

res = str_replace('use x; only_use x; used x', r'\buse\b', 'usage', use_regex=True)
test_eq(res, 'usage x; only_use x; used x')
res = str_replace('a=1\nb=2', r'(\w+)=(\d+)', r'\2=\1', use_regex=True)
test_eq(res, '1=a\n2=b')
with expect_fail(Exception, 'Failed to find'): str_replace('abc', r'\bxyz\b', 'q', use_regex=True)

The range and filter options recur identically across the replace family here and in its file, cell, and msg carriers, so the toolkit-wide replace_params group (declared alongside in fastcore.editskill, fastcore.nbio, and aidialog.dlgskill) collapses them in doc() module overviews:

res = str_replace('keep q\nfix q\nkeep q', 'q', 'y', re_filter='fix')
test_eq(res, 'keep q\nfix y\nkeep q')
res = str_replace('keep q\nfix q\nkeep q', 'q', 'y', re_filter='fix', invert_filter=True)
test_eq(res, 'keep y\nfix q\nkeep y')
str_replace('a-b\na b', r'(\w+) (\w+)', r'\2 \1', re_filter=' ', use_regex=True)

Line ranges use the same rules as replace_lines: None bounds mean first/last line, and a negative end_line counts inclusively from the end (-1 is the last line):

test_eq(str_replace('xa\nxb\nxc\n', 'x', 'y', start_line=2, end_line=-1), 'xa\nyb\nyc\n')
test_eq(str_replace('xa\nxb\nxc\n', 'x', 'y', end_line=2), 'ya\nyb\nxc\n')

---

source

strs_replace

def strs_replace(
    text:str, old_strs:list, # List of strings to find and replace
    new_strs:list, # List of replacement strings (must match length of old_strs)
    start_line:int=None, # Optional 1-based start line to limit search
    end_line:int=None, # Optional 1-based end line to limit search
    n_matches:int=None, # Max replacements per string (None=all)
    re_filter:str=None, # If provided, only process lines matching this regex (like g// in ex)
    invert_filter:bool=False, # Invert the filter (like g!// in ex)
    use_regex:bool=False, # Treat old_strs as regexes, and new_strs as `re.sub` templates?
):

Replace multiple strings simultaneously

res = strs_replace('f(a); g(a)', [r'\bf\b', r'\bg\b'], ['ff','gg'], use_regex=True)
test_eq(res, 'ff(a); gg(a)')

---

source

replace_lines

def replace_lines(
    text:str, start_line:int=None, # Starting line number to replace (1-based); None means line 1
    end_line:int=None, # Ending line number to replace (1-based, inclusive, negative counts from end); None means the last line
    new_content:str='', # New content to replace the specified lines
):

Replace line range with new content; the defaults replace the entire contents

start_line=None means line 1 and end_line=None means the last line, so with no line numbers replace_lines replaces the entire contents (an empty text included), and a bare start_line runs through to the end. This is the idiomatic whole-file or whole-cell rewrite in the wrapper layers, e.g. pyskills’ file_replace_lines(path, new_content=src).

test_eq(replace_lines('a\nb\nc\n', new_content='x\ny\n'), 'x\ny\n')
test_eq(replace_lines('', new_content='x\n'), 'x\n')
test_eq(replace_lines('a\nb\nc\n', 2, new_content='x\n'), 'a\nx\n')
test_eq(replace_lines('a\nb\nc\n', end_line=2, new_content='x\n'), 'x\nc\n')

---

source

del_lines

def del_lines(
    text:str, start_line:int, # Starting line number to delete (1-based); required
    end_line:int, # Ending line number to delete (1-based, inclusive, negative counts from end); required
    re_filter:str=None, # If provided, only delete lines matching this regex (like g// in ex)
    invert_filter:bool=False, # Invert the filter (like g!// in ex)
):

Delete line range; deletion is destructive, so both line numbers must be given explicitly (1, -1 for all lines)

del_lines is destructive, so it takes no defaults and rejects None: state the range explicitly (1, -1 deletes all lines, 5, 5 just line 5).

test_eq(del_lines('a\nb\nc\n', 2, 2), 'a\nc\n')
test_eq(del_lines('a\nb\nc\n', 1, -1), '')
with expect_fail(Exception, 'explicit'): del_lines('a\nb\n', 1, None)

Line hashes

line_hash, lnhash, and lnhash_at implement the exhash line-address format in pure Python: lineno|hash|, where the hash is 4 hex chars of crc32. They let any tool create lnhash-addressed views of text it holds, without depending on the exhash package.

---

source

lnhash_at

def lnhash_at(
    s:str | list | tuple, # A document as a str, or its lines
    line:int, # 1-based line number within `s`
)->str:

lineno|hash| exhash address of line line of s

---

source

lnhash

def lnhash(
    lineno:int, # 1-based line number
    line:str, # The line's current text
)->str:

lineno|hash| exhash address for line at lineno

---

source

line_hash

def line_hash(
    line:str, # Text to hash; when hashing a single line, omit its trailing newline
)->str:

4-char hex hash of line

test_eq(line_hash('def foo(): pass'), '54f0')  # same value the exhash Rust impl produces
s = 'a\ndef foo(): pass\nb'
test_eq(lnhash_at(s, 2), '2|54f0|')
test_eq(lnhash_at(s.splitlines(), 2), '2|54f0|')
test_eq(lnhash_at(s, 2), lnhash(2, 'def foo(): pass'))
lnhash_at(s, 2)

'2|54f0|'

File tools

File tools wrap the primitives with path I/O, returning unified diffs of what changed (“none: No changes.” / “error: …” otherwise). The path is the first argument, e.g:

view_file('~/a/b.py', 3)
create_file('~/a/b/c.py', 'content here')
file_str_replace('myfile.py', 'old_name', 'new_name')
file_del_lines('myfile.py', 2, 4)
file_replace_lines('myfile.py', new_content=src)   # no line numbers: replace the entire contents

file_str_replace, file_strs_replace, and file_del_lines support re_filter and invert_filter for targeting only lines matching (or not matching) a regex, like ex’s g// and g!//, combined with start_line/end_line to restrict to a region. ast_replace(text, repls) and file_ast_replace(path, repls) apply ast-grep (pattern, replacement) rules with $VAR metavariables (requires the optional remold package).

---

source

view_file

def view_file(
    path:str, # Path to view (expands `~` if needed)
    start_line:int=1, # Starting line to view
    end_line:int=None, # End line (defaults to last line if None; may be past EOF, which clamps to the last line - handy when the file size is unknown)
    nums:bool=True, # Show line numbers?
    lnhashs:bool=False, # Show exhash `lineno|hash|` addresses instead of line numbers?
):

Read file contents, optionally limited to 1-based line range

tmp = TemporaryDirectory()
test_path = f'{tmp.name}/test.txt'
test_content = 'alpha\nbeta\ngamma\ndelta\n'
def test_txt(): return Path(test_path).read_text()
Path(test_path).write_text(test_content)
test_path

'/var/folders/51/b2_szf2945n072c0vj2cyty40000gn/T/tmpaax7zfjh/test.txt'

view_file(test_path, 2, 30)

2: beta
3: gamma
4: delta

res = view_file(test_path, 2, 3, lnhashs=True)
test_eq(res.splitlines()[0], f'{lnhash_at(test_content, 2)}beta')
test_eq(view_file(test_path, nums=False), test_content[:-1])
res

2|0463|beta
3|d071|gamma

---

source

create_file

def create_file(
    path:str, # Path to create (expands `~` if needed)
    contents:str, # Contents of file to create
    overwrite:bool=False, # Replace the file if it already exists?
):

Create a new file with contents. Error if file exists, unless overwrite.

new_path = Path(tmp.name)/'demo.txt'
print(create_file(new_path, 'one\ntwo\nthree\n'))
assert str(create_file(new_path, 'x')).startswith('error: File exists')
test_eq(create_file(new_path, 'four\n', overwrite=True), 'done')
test_eq(Path(new_path).read_text(), 'four\n')

done

---

source

file_edit

def file_edit(
    f, name:NoneType=None
):

Wrap text editor f as a file editing function: path addressing, diff-or-error return

res = file_insert_line(test_path, 0, 'first')
test_eq(test_txt().splitlines()[0], 'first')
assert repr(res) == str(res)
res

@@ -1 +1,2 @@
+first
 alpha

res = file_str_replace(test_path, 'beta', 'BETA')
assert 'BETA' in test_txt(), f"Expected 'BETA' in file"
res

@@ -2,3 +2,3 @@
 alpha
-beta
+BETA
 gamma

with expect_fail(ValueError, test_path): str_replace(test_path, 'beta', 'x')

res = file_strs_replace(test_path, ['gamma', 'delta'], ['GAMMA', 'DELTA'])
test_eq(test_txt().splitlines()[-2:], ['GAMMA', 'DELTA'])
res

@@ -3,3 +3,3 @@
 BETA
-gamma
-delta
+GAMMA
+DELTA

start_line=None means line 1 and end_line=None means the last line, so with no line numbers replace_lines replaces the entire contents (an empty text included), and a bare start_line runs through to the end. This is the idiomatic whole-file or whole-cell rewrite: file_replace_lines(path, new_content=src), cell_replace_lines(fname, id, new_content=src).

res = file_replace_lines(test_path, 2, 3, 'two\nthree\n')
test_eq(test_txt().splitlines()[1:3], ['two', 'three'])
res

@@ -1,4 +1,4 @@
 first
-alpha
-BETA
+two
+three
 GAMMA

---

source

ast_replace

def ast_replace(
    text:str,
    repls:list, # (pattern, replacement) ast-grep rules; replacement is a `$VAR` template or a callable(match)->str
):

Apply ast-grep structural pattern replacements to python source

ast_replace/file_ast_replace do structural find-and-replace using ast-grep patterns, via the optional remold package. $VAR metavariables capture expressions; matching is by syntax tree, so formatting and comments outside the matched span are untouched; an unknown $VAR in a replacement raises rather than splicing garbage. Rules apply in order, reparsing between them, so later rules see earlier rules’ output.

res = ast_replace("print('a')\nx = 1  # keep\n", [("print($X)", "log($X)")])
test_eq(res, "log('a')\nx = 1  # keep\n")
with expect_fail(KeyError, '$Y'): ast_replace("print('a')\n", [("print($X)", "log($Y)")])

pyp = Path(tmp.name)/'ast.py'
pyp.write_text("print('a')\nprint(b)\n")
res = file_ast_replace(pyp, [("print($X)", "log($X)")])
test_eq(pyp.read_text(), "log('a')\nlog(b)\n")
res

@@ -1,2 +1,2 @@
-print('a')
-print(b)
+log('a')
+log(b)