Module

parsing.compiled_dispatch

Compile-time dispatch for pattern-based parsing.

Generates optimized dispatch code at module load time based on CommonMark token patterns. Uses Python 3.10+ match/case for efficient pattern matching.

Architecture:

  1. At import time: Build pattern → parser lookup table
  2. At parse time: O(1) lookup based on token type signature
  3. Fallback: Standard parser for unknown patterns

Performance:

  • O(1) parser selection (dict lookup)
  • No runtime pattern analysis
  • Specialized parsers eliminate branching

Future: Could generate Cython code for even faster dispatch.

1Class3Functions

Classes

PatternDispatcher 4 ▼
Compile-time pattern → parser dispatcher. Built once at module load, used for all parses. Uses fro…

Compile-time pattern → parser dispatcher.

Built once at module load, used for all parses. Uses frozenset of token types as pattern signature.

Methods

register 2 ▼
Register a parser for a token pattern.
def register(self, pattern: frozenset[TokenType], parser: ParserFn) -> None
Parameters
Name Type Description
pattern —
parser —
get_parser 1 ParserFn | None ▼
Get parser for token pattern, or None for fallback. O(1) lookup after O(n) sig…
def get_parser(self, tokens: list[Token]) -> ParserFn | None

Get parser for token pattern, or None for fallback.

O(1) lookup after O(n) signature computation. Signature computation is very cheap (just type checks).

Parameters
Name Type Description
tokens —
Returns
ParserFn | None
get_stats 0 dict ▼
Get dispatch statistics.
def get_stats(self) -> dict
Returns
dict
Internal Methods 1 ▼
__init__ 0 ▼
def __init__(self) -> None

Functions

build_dispatcher 0 PatternDispatcher ▼
Build the pattern dispatcher at module load time. Registers ONLY patterns wher…
def build_dispatcher() -> PatternDispatcher

Build the pattern dispatcher at module load time.

Registers ONLY patterns where token types alone determine parsing.

NOT registered (require content analysis):

  • PARAGRAPH_LINE patterns: Can contain setext underlines (===, ---), tables
  • LIST patterns: Can contain indented code, complex nesting
  • FENCED_CODE: FencedCode node uses source_start/source_end tracking

These are handled by the ultra-fast path (can_use_ultra_fast) or the main parser's existing fast paths.

Returns
PatternDispatcher
get_dispatcher 0 PatternDispatcher ▼
Get the global pattern dispatcher (lazy initialization).
def get_dispatcher() -> PatternDispatcher
Returns
PatternDispatcher
dispatch_parse 3 tuple ▼
Dispatch to optimal parser based on token pattern.
def dispatch_parse(tokens: list[Token], parse_inline_fn: Callable, fallback_fn: Callable) -> tuple
Parameters
Name Type Description
tokens list[Token]

Token list from lexer
parse_inline_fn Callable

Function to parse inline content
fallback_fn Callable

Standard parser for unknown patterns
Returns
tuple