Module

parsing.token_nav

Token navigation utilities for Patitas parser.

Provides mixin for token stream navigation and basic parsing operations.

1Class

Classes

TokenNavigationMixin 12 ▼
Mixin providing token stream navigation methods. Required Host Attributes: - _tokens: list[Tok…

Mixin providing token stream navigation methods.

Required Host Attributes:

  • _tokens: list[Token]
  • _tokens_len: int (cached len(_tokens) for hot loops)
  • _pos: int
  • _current: Token | None
  • _source: str
  • _line_starts: list[int] | None (lazy line index for O(log n) boundary lookup)

Attributes

Name Type Description
_tokens list[Token] —
_tokens_len int —
_pos int —
_current Token | None —
_source str —
_line_starts list[int] | None —

Methods

Internal Methods 6 ▼
_at_end 0 bool ▼
Check if at end of token stream.
def _at_end(self) -> bool
Returns
bool
_advance 0 Token | None ▼
Advance to next token and return it.
def _advance(self) -> Token | None
Returns
Token | None
_peek 1 Token | None ▼
Peek at token at offset from current position.
def _peek(self, offset: int = 1) -> Token | None
Parameters
Name Type Description
offset — Default:1
Returns
Token | None
_line_start_for_offset 1 int ▼
Get start offset of line containing offset. O(log n) with lazy line index.
def _line_start_for_offset(self, offset: int) -> int
Parameters
Name Type Description
offset —
Returns
int
_get_line_at 1 str ▼
Get the full line content containing the given source offset.
def _get_line_at(self, offset: int) -> str
Parameters
Name Type Description
offset —
Returns
str
_strip_columns 2 str ▼
Strip up to 'count' columns of whitespace/tabs from start of text.
def _strip_columns(self, text: str, count: int) -> str
Parameters
Name Type Description
text —
count —
Returns
str