Whole file
rsennrich/subword-nmt
The author described this change as “fix regression from 7bb1c: don't duplicate empty line”. It counts as a record because the check below fails on the code as it stood at 662efd1b9 and passes on f4f95998c, with nothing else changed between the two runs.
Projectrsennrich/subword-nmt
Fix saved2018-05-01
Sharing licenceMIT · LICENSE
Change size+3 −3
What the code was meant to do, written into the code itself as a save note
fix regression from 7bb1c: don't duplicate empty line
The change
| 65 | 65 | ||
| 66 | 66 | out = "" | |
| 67 | 67 | ||
| 68 | - | leading_whitespace = len(line)-len(line.lstrip()) | |
| 68 | + | leading_whitespace = len(line)-len(line.lstrip('\r\n ')) | |
| 69 | 69 | if leading_whitespace: | |
| 70 | 70 | out += line[:leading_whitespace] | |
| 71 | 71 | ||
| 72 | 72 | out += self.segment(line) | |
| 73 | 73 | ||
| 74 | - | trailing_whitespace = len(line)-len(line.rstrip()) | |
| 75 | - | if trailing_whitespace: | |
| 74 | + | trailing_whitespace = len(line)-len(line.rstrip('\r\n ')) | |
| 75 | + | if trailing_whitespace and trailing_whitespace != len(line): | |
| 76 | 76 | out += line[-trailing_whitespace:] | |
| 77 | 77 | ||
| 78 | 78 | return out |
The check that tells the two apart
fail→pass·test/test_bpe.py::TestBPESegmentMethod::test_empty_line
Check file test/test_bpe.py, taken without changes from the fix and copied onto the older code, so the exact same check runs against both versions.
Origin and history
The code before it662efd1b9d34ae6d3bfdfd5e6dffe2c72f2e7285
Broken version dated2018-05-01
Moduleapply_bpe
Units changedBPE
Fingerprintd934e51bed0f9be0
Checked2026-08-18 by goldset/0.1
Every field above is generated by our program. None of it is written by hand.