Module

parsing.inline.tokens

Typed inline tokens for Patitas parser.

Uses NamedTuples for inline token representation, providing:

  • Immutability by default (required for external match tracking)
  • Tuple unpacking support
  • Lower memory footprint (~80 bytes vs ~200 for dict)
  • Faster attribute access (tuple index vs hash lookup)
  • Full type safety with IDE autocomplete

Thread Safety:

All tokens are immutable and safe to share across threads.

Usage:

from patitas.parsing.inline.tokens import ( DelimiterToken, TextToken, InlineToken, )

token = DelimiterToken(char="*", count=2, can_open=True, can_close=False)

match token: case DelimiterToken(char="*", count=count): print(f"Asterisk delimiter with count {count}")

6Classes

Classes

DelimiterToken 7 ▼
Delimiter token for emphasis/strikethrough processing.

Delimiter token for emphasis/strikethrough processing.

Attributes

Name Type Description
char DelimiterChar

DelimiterChar
run_length int

int
can_open bool

bool
can_close bool

bool
tag int

int = TOKEN_DELIMITER

Methods

type 0 Literal['delimiter'] ▼
Token type identifier for dispatch.
property
def type(self) -> Literal['delimiter']
Returns
Literal['delimiter']
original_count 0 int ▼
Original count (same as run_length for immutable tokens).
property
def original_count(self) -> int
Returns
int
TextToken 3 ▼
Plain text token.

Plain text token.

Attributes

Name Type Description
content str —
tag int —

Methods

type 0 Literal['text'] ▼
Token type identifier for dispatch.
property
def type(self) -> Literal['text']
Returns
Literal['text']
CodeSpanToken 3 ▼
Inline code span token.

Inline code span token.

Attributes

Name Type Description
code str —
tag int —

Methods

type 0 Literal['code_span'] ▼
Token type identifier for dispatch.
property
def type(self) -> Literal['code_span']
Returns
Literal['code_span']
NodeToken 3 ▼
Pre-parsed AST node token (links, images, etc.).

Pre-parsed AST node token (links, images, etc.).

Attributes

Name Type Description
node object —
tag int —

Methods

type 0 Literal['node'] ▼
Token type identifier for dispatch.
property
def type(self) -> Literal['node']
Returns
Literal['node']
HardBreakToken 2 ▼
Hard line break token.

Hard line break token.

Attributes

Name Type Description
tag int —

Methods

type 0 Literal['hard_break'] ▼
Token type identifier for dispatch.
property
def type(self) -> Literal['hard_break']
Returns
Literal['hard_break']
SoftBreakToken 2 ▼
Soft line break token.

Soft line break token.

Attributes

Name Type Description
tag int —

Methods

type 0 Literal['soft_break'] ▼
Token type identifier for dispatch.
property
def type(self) -> Literal['soft_break']
Returns
Literal['soft_break']