Module

`lexers.toml_sm`

Hand-written TOML lexer using composable scanner mixins.

O(n) guaranteed, zero regex, thread-safe.

Classes

TomlStateMachineLexer 2 ▼

TOML lexer using composable mixins.

tokenize 2 Iterator[Token] ▼

def tokenize(self, code: str, config: LexerConfig | None = None) -> Iterator[Token]

Name	Type	Description
`code`	`—`
`config`	`—`	Default:`None`

Iterator[Token]

Internal Methods 1 ▼

_scan_toml_number 2 tuple[TokenType, int] ▼

Scan TOML number (including dates).

def _scan_toml_number(self, code: str, pos: int) -> tuple[TokenType, int]

Name	Type	Description
`code`	`—`
`pos`	`—`

tuple[TokenType, int]