One function

next_token in desgeeko/pdfsyntax

The author described this change as Fix #2 with support for balanced parentheses inside literals. It counts as a record because the check below fails on the code as it stood at 28b9d9c9c and passes on 96385155e, with nothing else changed between the two runs.

Fix saved2024-07-20
Sharing licenceMIT · LICENSE
Change size+6 3

What the code was meant to do, written into the code itself as a docstring

Find next token in raw string starting at some index.

The change

5757 elif search == "LSTRING":
5858 if double == b'\\(' or double == b'\\)':
5959 i += 1
60- else:
61- if single in b')':
62- return (h, i + 1, 'STRING')
60+ elif single == b'(':
61+ nested += 1
62+ elif single == b')':
63+ nested -= 1
64+ if nested == 0:
65+ return (h, i + 1, 'STRING')
6366 elif search == "HSTRING":
6467 if double == b'\\<' or double == b'\\>':
6568 i += 1

The check that tells the two apart

failpass·tests/test_tokenization.py::Tokenization::test_literal_string_with_parentheses

Check file tests/test_tokenization.py, taken without changes from the fix and copied onto the older code, so the exact same check runs against both versions.

Origin and history

The code before it28b9d9c9cc74b38c1566ad82bcbd85701f074e3c
Broken version dated2024-06-16
Modulepdfsyntax.objects
Units changednext_token
Fingerprint09410f97ca9161cd
Checked2026-08-18 by goldset/0.1

Every field above is generated by our program. None of it is written by hand.

Other bugs found in desgeeko/pdfsyntax