A token yielded by a custom lexer: terminal NAME + matched text, plus any of
the six optional positions (absent/null positions stay null — the engines
never recompute them). Extra fields are preserved: the ORIGINAL yielded
objects become the tree leaves.
A token yielded by a custom lexer: terminal NAME + matched text, plus any of the six optional positions (absent/
nullpositions staynull— the engines never recompute them). Extra fields are preserved: the ORIGINAL yielded objects become the tree leaves.