One function
Transformer in lark-parser/lark
The author described this change as “Fix for issue #1394”. It counts as a record because the check below fails on the code as it stood at 7646fb316 and passes on 544893115, with nothing else changed between the two runs.
Projectlark-parser/lark
Fix saved2024-02-25
Sharing licenceMIT · LICENSE
Change size+5 −1
What the code was meant to do, written into the code itself as a docstring
Transformers work bottom-up (or depth-first), starting with visiting the leaves and working their way up until ending at the root of the tree. For each node visited, the transformer will call the appropriate method (callbacks), according to the node's `data`, and use the returned value to replace the node, thereby creating a new tree structure. Transformers can be used to implement map & reduce patterns. Because nodes are reduced from leaf to root, at any point the callbacks may assume the children have already been transformed (if applicable). If the transformer cannot find a method with the right name, it will instead call `__default__`, which by default creates a copy of the node. To discard a node, return Discard (`lark.visitors.Discard`). `Transformer` can do anything `Visitor` can do, but because it reconstructs the tree, it is slightly less efficient. A transformer without methods essentially performs a non-memoized partial deepcopy. All these classes implement the transformer interface: - `Transformer` - Recursively transforms the tree. This is the one you probably want. - `Transformer_InPlace` - Non-recursive. Changes the tree in-place instead of returning new instances - `Transformer_InPlaceRecursive` - Recursive. Changes the tree in-place instead of returning new instances Parameters: visit_tokens (bool, optional): Should the transformer visit tokens in addition to rules. Setting this to `False` is slightly faster. Defaults to `True`. (For processing ignored tokens, use the `lexer_callbacks` options)
The change
| 85 | 85 | ||
| 86 | 86 | def transform(self, tree: Tree[_Leaf_T]) -> _Return_T: | |
| 87 | 87 | "Transform the given tree, and return the final result" | |
| 88 | - | return self._transform_tree(tree) | |
| 88 | + | res = list(self._transform_children([tree])) | |
| 89 | + | if not res: | |
| 90 | + | return None # type: ignore[return-value] | |
| 91 | + | assert len(res) == 1 | |
| 92 | + | return res[0] | |
| 89 | 93 | ||
| 90 | 94 | def __mul__( | |
| 91 | 95 | self: 'Transformer[_Leaf_T, Tree[_Leaf_U]]', |
The check that tells the two apart
fail→pass·tests/test_trees.py::TestTrees::test_transform_token
Check file tests/test_trees.py, taken without changes from the fix and copied onto the older code, so the exact same check runs against both versions.
Origin and history
The code before it7646fb31609c6c83e0998b121eeffb908f5ec5ba
Broken version dated2024-02-02
Modulelark.visitors
Units changedTransformer
Fingerprintddbb413f71e602a7
Checked2026-08-18 by goldset/0.1
Every field above is generated by our program. None of it is written by hand.
Other bugs found in lark-parser/lark
- 2026-07-08Fix lexer_callbacks silently dropped for keyword terminals
- 2026-06-24Transformer_NonRecursive
- 2026-06-04Raise GrammarError for non-terminal names in %declare
- 2026-06-02Fix empty SPPF node from xearley ignore carry-over (fixes #1598)
- 2026-05-24fix: raise GrammarError for template usage inside terminals
- 2024-10-25Adjust behaviour in #1481 to raise a NotImplementedError