Module

parsing.pattern_parsers

Pattern-specific optimized parsers.

Each parser is tailored for a specific token pattern, eliminating unnecessary branching and checks.

Top 10 patterns cover 79.1% of CommonMark spec:

  1. (PARAGRAPH_LINE,) 45.7% ← ultra_fast.py
  2. (BLANK_LINE, LINK_REFERENCE_DEF, PARAGRAPH_LINE) 10.7% ← parse_linkref_paragraphs
  3. (HTML_BLOCK,) 3.7% ← parse_html_only
  4. (LIST_ITEM_MARKER, PARAGRAPH_LINE) 3.5% ← parse_simple_flat_list
  5. (BLANK_LINE, INDENTED_CODE, LIST_ITEM_MARKER, ...) 3.1% ← (complex)
  6. (BLANK_LINE, PARAGRAPH_LINE) 3.1% ← parse_paragraphs_with_blanks
  7. (FENCED_CODE_CONTENT, FENCED_CODE_END, ...) 2.9% ← parse_fenced_code_only
  8. (BLANK_LINE, LIST_ITEM_MARKER, PARAGRAPH_LINE) 2.3% ← parse_simple_list_with_blanks
  9. (ATX_HEADING,) 2.1% ← parse_atx_only
  10. (INDENTED_CODE,) 2.0% ← parse_indented_only
8Functions

Functions

parse_html_only 2 tuple[Block, ...] ▼
Pattern 3: (HTML_BLOCK,) - 3.7% of examples. Just HTML blocks, no other conten…
def parse_html_only(tokens: list, parse_inline_fn: Callable) -> tuple[Block, ...]

Pattern 3: (HTML_BLOCK,) - 3.7% of examples.

Just HTML blocks, no other content.

Parameters
Name Type Description
tokens list
parse_inline_fn Callable
Returns
tuple[Block, ...]
parse_atx_only 2 tuple[Block, ...] ▼
Pattern 9: (ATX_HEADING,) - 2.1% of examples. Just ATX headings, no other cont…
def parse_atx_only(tokens: list, parse_inline_fn: Callable) -> tuple[Block, ...]

Pattern 9: (ATX_HEADING,) - 2.1% of examples.

Just ATX headings, no other content.

Parameters
Name Type Description
tokens list
parse_inline_fn Callable
Returns
tuple[Block, ...]
parse_indented_only 2 tuple[Block, ...] ▼
Pattern 10: (INDENTED_CODE,) - 2.0% of examples. Just indented code blocks, no…
def parse_indented_only(tokens: list, parse_inline_fn: Callable) -> tuple[Block, ...]

Pattern 10: (INDENTED_CODE,) - 2.0% of examples.

Just indented code blocks, no other content.

Parameters
Name Type Description
tokens list
parse_inline_fn Callable
Returns
tuple[Block, ...]
parse_fenced_code_only 2 tuple[Block, ...] ▼
Pattern 7: (FENCED_CODE_*) - 2.9% of examples. Just fenced code blocks, no oth…
def parse_fenced_code_only(tokens: list, parse_inline_fn: Callable) -> tuple[Block, ...]

Pattern 7: (FENCED_CODE_*) - 2.9% of examples.

Just fenced code blocks, no other content.

Parameters
Name Type Description
tokens list
parse_inline_fn Callable
Returns
tuple[Block, ...]
parse_paragraphs_with_blanks 2 tuple[Block, ...] ▼
Pattern 6: (BLANK_LINE, PARAGRAPH_LINE) - 3.1% of examples. Paragraphs separat…
def parse_paragraphs_with_blanks(tokens: list, parse_inline_fn: Callable) -> tuple[Block, ...]

Pattern 6: (BLANK_LINE, PARAGRAPH_LINE) - 3.1% of examples.

Paragraphs separated by blank lines.

Parameters
Name Type Description
tokens list
parse_inline_fn Callable
Returns
tuple[Block, ...]
parse_simple_flat_list 2 tuple[Block, ...] ▼
Pattern 4: (LIST_ITEM_MARKER, PARAGRAPH_LINE) - 3.5% of examples. Simple flat …
def parse_simple_flat_list(tokens: list, parse_inline_fn: Callable) -> tuple[Block, ...]

Pattern 4: (LIST_ITEM_MARKER, PARAGRAPH_LINE) - 3.5% of examples.

Simple flat list with no nesting, no blank lines.

Parameters
Name Type Description
tokens list
parse_inline_fn Callable
Returns
tuple[Block, ...]
parse_simple_list_with_blanks 2 tuple[Block, ...] ▼
Pattern 8: (BLANK_LINE, LIST_ITEM_MARKER, PARAGRAPH_LINE) - 2.3%. List with bl…
def parse_simple_list_with_blanks(tokens: list, parse_inline_fn: Callable) -> tuple[Block, ...]

Pattern 8: (BLANK_LINE, LIST_ITEM_MARKER, PARAGRAPH_LINE) - 2.3%.

List with blank lines (loose list).

Parameters
Name Type Description
tokens list
parse_inline_fn Callable
Returns
tuple[Block, ...]
get_pattern_parser 1 Callable | None ▼
Get specialized parser for token pattern if available.
def get_pattern_parser(tokens: list) -> Callable | None
Parameters
Name Type Description
tokens list

Token list from lexer
Returns
Callable | None