Module

parsing.inline.tokens

Typed inline tokens for Patitas parser.

Uses NamedTuples for inline token representation, providing:

  • Immutability by default (required for external match tracking)
  • Tuple unpacking support
  • Lower memory footprint (~80 bytes vs ~200 for dict)
  • Faster attribute access (tuple index vs hash lookup)
  • Full type safety with IDE autocomplete

Thread Safety:

All tokens are immutable and safe to share across threads.

Usage:

from patitas.parsing.inline.tokens import ( DelimiterToken, TextToken, InlineToken, )

token = DelimiterToken(char="*", count=2, can_open=True, can_close=False)

match token: case DelimiterToken(char="*", count=count): print(f"Asterisk delimiter with count {count}")

6Classes

Classes

DelimiterToken 6 ▼
Delimiter token for emphasis/strikethrough processing. Immutable by design — match state tracked e…

Delimiter token for emphasis/strikethrough processing.

Immutable by design — match state tracked externally in MatchRegistry.

NamedTuple chosen over dataclass for:

  • Immutability by default (required for external match tracking)
  • Tuple unpacking support
  • Lower memory footprint (~80 bytes vs ~200 for dict)
  • Faster attribute access (tuple index vs hash lookup)

Attributes

Name Type Description
char DelimiterChar

The delimiter character ("*", "_", or "~").
count int

Number of consecutive delimiter characters.
can_open bool

Whether this delimiter can open emphasis.
can_close bool

Whether this delimiter can close emphasis.

Methods

type 0 Literal['delimiter'] ▼
Token type identifier for dispatch.
property
def type(self) -> Literal['delimiter']
Returns
Literal['delimiter']
original_count 0 int ▼
Original count (same as count for immutable tokens).
property
def original_count(self) -> int
Returns
int
TextToken 2 ▼
Plain text token.

Plain text token.

Attributes

Name Type Description
content str

The text content.

Methods

type 0 Literal['text'] ▼
Token type identifier for dispatch.
property
def type(self) -> Literal['text']
Returns
Literal['text']
CodeSpanToken 2 ▼
Inline code span token.

Inline code span token.

Attributes

Name Type Description
code str

The code content (already processed per CommonMark rules).

Methods

type 0 Literal['code_span'] ▼
Token type identifier for dispatch.
property
def type(self) -> Literal['code_span']
Returns
Literal['code_span']
NodeToken 2 ▼
Pre-parsed AST node token (links, images, etc.). Used when inline content is parsed directly into …

Pre-parsed AST node token (links, images, etc.).

Used when inline content is parsed directly into an AST node (e.g., links, images, autolinks, roles, math).

Attributes

Name Type Description
node object

The pre-parsed inline AST node.

Methods

type 0 Literal['node'] ▼
Token type identifier for dispatch.
property
def type(self) -> Literal['node']
Returns
Literal['node']
HardBreakToken 1 ▼
Hard line break token. Represents a hard line break (backslash + newline or two trailing spaces).

Hard line break token.

Represents a hard line break (backslash + newline or two trailing spaces).

Methods

type 0 Literal['hard_break'] ▼
Token type identifier for dispatch.
property
def type(self) -> Literal['hard_break']
Returns
Literal['hard_break']
SoftBreakToken 1 ▼
Soft line break token. Represents a soft line break (single newline in paragraph). Typically rende…

Soft line break token.

Represents a soft line break (single newline in paragraph). Typically rendered as a space or newline depending on settings.

Methods

type 0 Literal['soft_break'] ▼
Token type identifier for dispatch.
property
def type(self) -> Literal['soft_break']
Returns
Literal['soft_break']