Skip to main content

Text diffs

Line and word differences between plain-text sources.

diff_lines pairs similar changed lines and identifies the words that were removed or inserted. Renderers can use the same result for terminal output or HTML.

from fastcore.test import *
from fastcore.basics import PrettyString

Structured changes


source

diff_lines

def diff_lines(
    old, # Original source string
    new, # New source string
    context:int=3, # Unchanged lines shown either side of a change; None shows the full source
    final_newline:bool=True, # Distinguish a missing final newline (unnecessary for notebook cells)?
    min_ratio:float=0.5, # Minimum shared-word similarity for refining a changed line pair
):

Line diffs with word-level replacements for similar changed lines.

Each row has a marker and a list of spans. The row markers are for unchanged context, - for a deleted line, + for an added line, ! for a refined replacement, and @ for a hunk header. Adding the row marker to its spans reconstructs an ordinary unified-diff line; replacement spans instead combine old and new text.

Spans use for unchanged text, - for removed text and + for inserted text. When a paired line gains or loses its final newline, the change is a separate ('+', '\n') or ('-', '\n') span. The strings contain no display markup. Use final_newline=False to ignore this distinction for notebook cells:

old = 'x=1\ny=2\nz=3\nk=4'
new = 'x = 100\ny=2\nw=5\nq=6'
rows = diff_lines(old, new, context=None, final_newline=False)
test_eq([kind for kind,_ in rows], ['@', '!', ' ', '-', '-', '+', '+'])
mixed = diff_lines('apple\nx = 1', 'pear\nx = 2\nextra', final_newline=False)[1:]
test_eq([kind for kind,_ in mixed], ['-', '+', '!', '+'])
rows
[('@', [(' ', '@ -1,4 +1,4 @@')]),
 ('!', [(' ', 'x'), ('+', ' '), (' ', '='), ('-', '1'), ('+', ' 100')]),
 (' ', [(' ', 'y=2')]),
 ('-', [(' ', 'z=3')]),
 ('-', [(' ', 'k=4')]),
 ('+', [(' ', 'w=5')]),
 ('+', [(' ', 'q=6')])]

A line pair is refined when its shared-word similarity reaches min_ratio. Unrelated lines remain separate deletions and additions. Words, punctuation and whitespace are separate tokens; whitespace does not anchor matches.

Short coincidental matches inside a larger rewrite are included in the replaced phrase. Longer unchanged text still separates independent edits:

before = 'selects the fields, shown by the vectors'
after = 'reverses the coordinates, arranged in two columns'
suffix = '. Dragging either point redraws both points.'
phrase = diff_lines('The model '+before+suffix, 'The model '+after+suffix, final_newline=False)[1]
test_eq(phrase, ('!', [(' ', 'The model '), ('-', before), ('+', after), (' ', suffix)]))
phrase
('!',
 [(' ', 'The model '),
  ('-', 'selects the fields, shown by the vectors'),
  ('+', 'reverses the coordinates, arranged in two columns'),
  (' ', '. Dragging either point redraws both points.')])
shared = ' while both points retain their positions '
separate = diff_lines('We select fields'+shared+'using the old formula.',
                      'We reverse coordinates'+shared+'using the new expression.', final_newline=False)[1]
test_eq(separate, ('!', [(' ', 'We '), ('-', 'select fields'), ('+', 'reverse coordinates'),
                       (' ', shared+'using the '), ('-', 'old formula'), ('+', 'new expression'), (' ', '.')]))
test_eq(diff_lines('hello\n', 'hello')[1:], [('!', [(' ', 'hello'), ('-', '\n')])])
test_eq(diff_lines('hello', 'hello\n')[1:], [('!', [(' ', 'hello'), ('+', '\n')])])
test_eq(diff_lines('hello\n', 'hello', final_newline=False), [])
separate
('!',
 [(' ', 'We '),
  ('-', 'select fields'),
  ('+', 'reverse coordinates'),
  (' ', ' while both points retain their positions using the '),
  ('-', 'old formula'),
  ('+', 'new expression'),
  (' ', '.')])

Unified text


source

source_diff

def source_diff(
    old_source, # Original source string
    new_source, # New source string
    context:int=3, # Unchanged lines shown either side of a change
    final_newline:bool=True, # Distinguish a missing final newline (unnecessary for notebook cells)?
):

Unified source diff, including missing-final-newline markers.

source_diff returns an ordinary unified diff rather than inline word spans. It marks a missing final newline in files; final_newline=False ignores that distinction for notebook cells:

patch = source_diff('hello\n', 'hello')
assert r'\ No newline at end of file' in patch
test_eq(source_diff('hello\n', 'hello', final_newline=False), '')
print(patch)
--- 
+++ 
@@ -1 +1 @@
-hello
+hello
\ No newline at end of file