Module

`lexers.cpp_sm`

Hand-written C++ lexer using composable scanner mixins.

O(n) guaranteed, zero regex, thread-safe.

Classes

CppStateMachineLexer 2 ▼

C++ lexer using composable mixins.

tokenize 2 Iterator[Token] ▼

def tokenize(self, code: str, config: LexerConfig | None = None) -> Iterator[Token]

Name	Type	Description
`code`	`—`
`config`	`—`	Default:`None`

Iterator[Token]

Internal Methods 1 ▼

_scan_cpp_number 2 tuple[TokenType, int] ▼

Scan C++ number with ' separators.

def _scan_cpp_number(self, code: str, pos: int) -> tuple[TokenType, int]

Name	Type	Description
`code`	`—`
`pos`	`—`

tuple[TokenType, int]