Module

lexers.kida_sm

Hand-written Kida lexer using state machine approach.

O(n) guaranteed, zero regex, thread-safe.

Kida is Bengal's native template engine with: {# ... #} - Comments {{ ... }} - Variable expressions (output) {% ... %} - Statements (if, for, let, match, etc.)

Key differences from Jinja2:

  • Unified endings: {% end %} instead of {% endif %}, {% endfor %}, etc.
  • Pipeline operator: |> for left-to-right filter chains
  • Pattern matching: {% match %}...{% case %}...{% end %}
  • Template-scoped variables: {% let %}
  • Functions with lexical scope: {% def %}
  • Fragment caching: {% cache %}
1Class

Classes

KidaStateMachineLexer 6 ▼
Kida template lexer. Tokenizes Kida template syntax including: - Comments: {# ... #} - Expressions…

Kida template lexer.

Tokenizes Kida template syntax including:

  • Comments: {# ... #}
  • Expressions: {{ ... }}
  • Statements: {% ... %}
  • Plain text between template tags
  • Pipeline operator: |>

Methods

tokenize 2 Iterator[Token] ▼
def tokenize(self, code: str, config: LexerConfig | None = None) -> Iterator[Token]
Parameters
Name Type Description
code —
config — Default:None
Returns
Iterator[Token]
Internal Methods 5 ▼
_find_closing_braces 4 tuple[int, int, int] ▼
Find closing }} and return updated position and line info.
def _find_closing_braces(self, code: str, pos: int, line: int, line_start: int) -> tuple[int, int, int]
Parameters
Name Type Description
code —
pos —
line —
line_start —
Returns
tuple[int, int, int]
_find_closing_percent 4 tuple[int, int, int] ▼
Find closing %} and return updated position and line info.
def _find_closing_percent(self, code: str, pos: int, line: int, line_start: int) -> tuple[int, int, int]
Parameters
Name Type Description
code —
pos —
line —
line_start —
Returns
tuple[int, int, int]
_tokenize_expression 4 Iterator[Token] ▼
Tokenize inside {{ ... }}.
def _tokenize_expression(self, code: str, pos: int, line: int, line_start: int) -> Iterator[Token]
Parameters
Name Type Description
code —
pos —
line —
line_start —
Returns
Iterator[Token]
_tokenize_statement 4 Iterator[Token] ▼
Tokenize inside {% ... %}.
def _tokenize_statement(self, code: str, pos: int, line: int, line_start: int) -> Iterator[Token]
Parameters
Name Type Description
code —
pos —
line —
line_start —
Returns
Iterator[Token]
_tokenize_kida_content 5 Iterator[Token] ▼
Tokenize Kida content (expressions or statements).
def _tokenize_kida_content(self, code: str, pos: int, line: int, line_start: int, end_marker: str) -> Iterator[Token]
Parameters
Name Type Description
code —
pos —
line —
line_start —
end_marker —
Returns
Iterator[Token]