Classes
DiffStateMachineLexer
1
▼
Diff/Patch lexer optimized with C-level str.find().
Line-based format - uses find() to scan lines …
DiffStateMachineLexer
1
▼
Diff/Patch lexer optimized with C-level str.find().
Line-based format - uses find() to scan lines without allocating intermediate lists.
Methods
tokenize
2
Iterator[Token]
▼
tokenize
2
Iterator[Token]
▼
def tokenize(self, code: str, config: LexerConfig | None = None) -> Iterator[Token]
Parameters
| Name | Type | Description |
|---|---|---|
code |
— |
|
config |
— |
Default:None
|
Returns
Iterator[Token]