Module

lexers.diff_sm

Hand-written Diff/Patch lexer optimized for speed.

O(n) guaranteed, zero regex, thread-safe. Uses C-level str.find() for fast line scanning.

Classes

DiffStateMachineLexer 1
Diff/Patch lexer optimized with C-level str.find(). Line-based format - uses find() to scan lines …

Diff/Patch lexer optimized with C-level str.find().

Line-based format - uses find() to scan lines without allocating intermediate lists.

Methods

tokenize 2 Iterator[Token]
def tokenize(self, code: str, config: LexerConfig | None = None) -> Iterator[Token]
Parameters
Name Type Description
code
config Default:None
Returns
Iterator[Token]