tools

Text and file editing primitives shared by the fastai editing tools

The editors here are string-level: each takes text plus edit parameters and returns the new text, raising ValueError when an edit can’t apply. The file tools below wrap them with path I/O and diff reporting; message-level wrappers live in llmsurgery. (This module previously held experimental LLM path-editing and command tools, superseded by safecmd, rgapi, and the tools here.)

---

source

insert_line

def insert_line(
    text:str,
    insert_line:int, # The 1-based line number after which to insert the text (0: before 1st line, 1: after 1st line, 2: after 2nd, etc.)
    new_str:str, # The text to insert
):

Insert new_str at specified line number

test_eq(insert_line('a\nb', 0, 'first'), 'first\na\nb')
test_eq(insert_line('a\nb', 2, 'end'), 'a\nb\nend')
with expect_fail(ValueError, 'Invalid line'): insert_line('a', 5, 'x')

---

source

str_replace

def str_replace(
    text:str, old_str:str, # Text to find and replace
    new_str:str, # Text to replace with
    start_line:int=None, # Optional 1-based start line to limit search
    end_line:int=None, # Optional 1-based end line to limit search
    n_matches:int=None, # Max replacements (None=all)
    re_filter:str=None, # If provided, only process lines matching this regex (like g// in ex)
    invert_filter:bool=False, # Invert the filter (like g!// in ex)
    use_regex:bool=False, # Treat old_str as a regex, and new_str as an `re.sub` template?
):

Replace occurrence(s) of old_str with new_str

With use_regex=True, old_str is a regex and new_str an re.sub template, so word boundaries and backrefs work. Handy for renames where plain substring matching would hit longer names too:

res = str_replace('use x; only_use x; used x', r'\buse\b', 'usage', use_regex=True)
test_eq(res, 'usage x; only_use x; used x')
res = str_replace('a=1\nb=2', r'(\w+)=(\d+)', r'\2=\1', use_regex=True)
test_eq(res, '1=a\n2=b')
with expect_fail(Exception, 'Failed to find'): str_replace('abc', r'\bxyz\b', 'q', use_regex=True)

Line ranges use the same rules as replace_lines: None bounds mean first/last line, and a negative end_line counts inclusively from the end (-1 is the last line):

test_eq(str_replace('xa\nxb\nxc\n', 'x', 'y', start_line=2, end_line=-1), 'xa\nyb\nyc\n')
test_eq(str_replace('xa\nxb\nxc\n', 'x', 'y', end_line=2), 'ya\nyb\nxc\n')

---

source

strs_replace

def strs_replace(
    text:str, old_strs:list, # List of strings to find and replace
    new_strs:list, # List of replacement strings (must match length of old_strs)
    start_line:int=None, # Optional 1-based start line to limit search
    end_line:int=None, # Optional 1-based end line to limit search
    n_matches:int=None, # Max replacements per string (None=all)
    re_filter:str=None, # If provided, only process lines matching this regex (like g// in ex)
    invert_filter:bool=False, # Invert the filter (like g!// in ex)
    use_regex:bool=False, # Treat old_strs as regexes, and new_strs as `re.sub` templates?
):

Replace multiple strings simultaneously

res = strs_replace('f(a); g(a)', [r'\bf\b', r'\bg\b'], ['ff','gg'], use_regex=True)
test_eq(res, 'ff(a); gg(a)')

---

source

replace_lines

def replace_lines(
    text:str, start_line:int=None, # Starting line number to replace (1-based); None means line 1
    end_line:int=None, # Ending line number to replace (1-based, inclusive, negative counts from end); None means the last line
    new_content:str='', # New content to replace the specified lines
):

Replace line range with new content; the defaults replace the entire contents

start_line=None means line 1 and end_line=None means the last line, so with no line numbers replace_lines replaces the entire contents (an empty text included), and a bare start_line runs through to the end. This is the idiomatic whole-file or whole-cell rewrite in the wrapper layers, e.g. pyskills’ file_replace_lines(path, new_content=src).

test_eq(replace_lines('a\nb\nc\n', new_content='x\ny\n'), 'x\ny\n')
test_eq(replace_lines('', new_content='x\n'), 'x\n')
test_eq(replace_lines('a\nb\nc\n', 2, new_content='x\n'), 'a\nx\n')
test_eq(replace_lines('a\nb\nc\n', end_line=2, new_content='x\n'), 'x\nc\n')

---

source

del_lines

def del_lines(
    text:str, start_line:int, # Starting line number to delete (1-based); required
    end_line:int, # Ending line number to delete (1-based, inclusive, negative counts from end); required
    re_filter:str=None, # If provided, only delete lines matching this regex (like g// in ex)
    invert_filter:bool=False, # Invert the filter (like g!// in ex)
):

Delete line range; deletion is destructive, so both line numbers must be given explicitly (1, -1 for all lines)

del_lines is destructive, so it takes no defaults and rejects None: state the range explicitly (1, -1 deletes all lines, 5, 5 just line 5).

test_eq(del_lines('a\nb\nc\n', 2, 2), 'a\nc\n')
test_eq(del_lines('a\nb\nc\n', 1, -1), '')
with expect_fail(Exception, 'explicit'): del_lines('a\nb\n', 1, None)

Line hashes

line_hash, lnhash, and lnhash_at implement the exhash line-address format in pure Python: lineno|hash|, where the hash is 4 hex chars of crc32. They let any tool create lnhash-addressed views of text it holds, without depending on the exhash package.

---

source

lnhash_at

def lnhash_at(
    s:str | list | tuple, # A document as a str, or its lines
    line:int, # 1-based line number within `s`
)->str:

lineno|hash| exhash address of line line of s

---

source

lnhash

def lnhash(
    lineno:int, # 1-based line number
    line:str, # The line's current text
)->str:

lineno|hash| exhash address for line at lineno

---

source

line_hash

def line_hash(
    line:str, # A single line of text, without trailing newline
)->str:

4-char hex exhash hash of line

test_eq(line_hash('def foo(): pass'), '54f0')  # same value the exhash Rust impl produces
s = 'a\ndef foo(): pass\nb'
test_eq(lnhash_at(s, 2), '2|54f0|')
test_eq(lnhash_at(s.splitlines(), 2), '2|54f0|')
test_eq(lnhash_at(s, 2), lnhash(2, 'def foo(): pass'))
lnhash_at(s, 2)

File tools

File tools wrap the primitives with path I/O, returning unified diffs of what changed (“none: No changes.” / “error: …” otherwise). The path is the first argument (with view_file/create_file named verb-first, since the file is the verb’s object rather than the location of an edit), e.g:

view_file('~/a/b.py', 3)
create_file('~/a/b/c.py', 'content here')
file_str_replace('myfile.py', 'old_name', 'new_name')
file_del_lines('myfile.py', 2, 4)
file_replace_lines('myfile.py', new_content=src)   # no line numbers: replace the entire contents

file_str_replace, file_strs_replace, and file_del_lines support re_filter and invert_filter for targeting only lines matching (or not matching) a regex, like ex’s g// and g!//, combined with start_line/end_line to restrict to a region. ast_replace(text, repls) and file_ast_replace(path, repls) apply ast-grep (pattern, replacement) rules with $VAR metavariables (requires the optional remold package). Where the exhash package is available, prefer it for editing: its hash-verified addressing fails loudly on stale context instead of editing nearby text.

---

source

view_file

def view_file(
    path:str, # Path to view (expands `~` if needed)
    start_line:int=1, # Starting line to view
    end_line:int=None, # End line (defaults to last line if None; may be past EOF, which clamps to the last line - handy when the file size is unknown)
    lnhashs:bool=False, # Prefix `lineno|hash|` exhash addresses instead of `lineno: `
):

Read file contents, optionally limited to 1-based line range

tmp = TemporaryDirectory()
test_path = f'{tmp.name}/test.txt'
test_content = 'alpha\nbeta\ngamma\ndelta\n'
def test_txt(): return Path(test_path).read_text()
Path(test_path).write_text(test_content)
test_path

'/var/folders/51/b2_szf2945n072c0vj2cyty40000gn/T/tmpo1xjivig/test.txt'

view_file(test_path, 2, 30)

2: beta
3: gamma
4: delta

res = view_file(test_path, 2, 3, lnhashs=True)
test_eq(res.splitlines()[0], f'{lnhash_at(test_content, 2)}beta')
res

---

source

create_file

def create_file(
    path:str, # Path to create (expands `~` if needed)
    contents:str, # Contents of file to create
    overwrite:bool=False, # Replace the file if it already exists?
):

Create a new file with contents. Error if file exists, unless overwrite.

new_path = Path(tmp.name)/'demo.txt'
print(create_file(new_path, 'one\ntwo\nthree\n'))
assert str(create_file(new_path, 'x')).startswith('error: File exists')
test_eq(create_file(new_path, 'four\n', overwrite=True), 'done')
test_eq(Path(new_path).read_text(), 'four\n')

done

---

source

file_edit

def file_edit(
    f, name:NoneType=None
):

Call self as a function.

res = file_insert_line(test_path, 0, 'first')
test_eq(test_txt().splitlines()[0], 'first')
assert repr(res) == str(res)
res

@@ -1 +1,2 @@
+first
 alpha

res = file_str_replace(test_path, 'beta', 'BETA')
assert 'BETA' in test_txt(), f"Expected 'BETA' in file"
res

@@ -2,3 +2,3 @@
 alpha
-beta
+BETA
 gamma

with expect_fail(ValueError, test_path): str_replace(test_path, 'beta', 'x')

res = file_strs_replace(test_path, ['gamma', 'delta'], ['GAMMA', 'DELTA'])
test_eq(test_txt().splitlines()[-2:], ['GAMMA', 'DELTA'])
res

@@ -3,3 +3,3 @@
 BETA
-gamma
-delta
+GAMMA
+DELTA

start_line=None means line 1 and end_line=None means the last line, so with no line numbers replace_lines replaces the entire contents (an empty text included), and a bare start_line runs through to the end. This is the idiomatic whole-file or whole-cell rewrite: file_replace_lines(path, new_content=src), cell_replace_lines(fname, id, new_content=src).

res = file_replace_lines(test_path, 2, 3, 'two\nthree\n')
test_eq(test_txt().splitlines()[1:3], ['two', 'three'])
res

@@ -1,4 +1,4 @@
 first
-alpha
-BETA
+two
+three
 GAMMA

---

source

ast_replace

def ast_replace(
    text:str,
    repls:list, # (pattern, replacement) ast-grep rules; replacement is a `$VAR` template or a callable(match)->str
):

Apply ast-grep structural pattern replacements to python source

ast_replace/file_ast_replace do structural find-and-replace using ast-grep patterns, via the optional remold package. $VAR metavariables capture expressions; matching is by syntax tree, so formatting and comments outside the matched span are untouched; an unknown $VAR in a replacement raises rather than splicing garbage. Rules apply in order, reparsing between them, so later rules see earlier rules’ output.

res = ast_replace("print('a')\nx = 1  # keep\n", [("print($X)", "log($X)")])
test_eq(res, "log('a')\nx = 1  # keep\n")
with expect_fail(KeyError, '$Y'): ast_replace("print('a')\n", [("print($X)", "log($Y)")])

pyp = Path(tmp.name)/'ast.py'
pyp.write_text("print('a')\nprint(b)\n")
res = file_ast_replace(pyp, [("print($X)", "log($X)")])
test_eq(pyp.read_text(), "log('a')\nlog(b)\n")
res

@@ -1,2 +1,2 @@
-print('a')
-print(b)
+log('a')
+log(b)