# tokens

URL: /patitas/api/parsing/inline/tokens/
Section: inline
Description: Typed inline tokens for Patitas parser.

Uses NamedTuples for inline token representation, providing:
- Immutability by default (required for external match tracking)
- Tuple unpacking support
- Lower memory footprint (~80 bytes vs ~200 for dict)
- Faster attribute access (tuple index vs hash lookup)
- Full type safety with IDE autocomplete

Thread Safety:
All tokens are immutable and safe to share across threads.

Usage:
from patitas.parsing.inline.tokens import (
    DelimiterToken,
    TextToken,
    InlineToken,
)

    token = DelimiterToken(char="*", count=2, can_open=True, can_close=False)
match token:
    case DelimiterToken(char="*", count=count):
        print(f"Asterisk delimiter with count {count}")

---

> For a complete page index, fetch /patitas/llms.txt.

Open LLM text
(/patitas/api/parsing/inline/tokens/index.txt)

Share with AI

Ask Claude
(https://claude.ai/new?q=Please%20help%20me%20understand%20this%20documentation%3A%20%2Fpatitas%2Fapi%2Fparsing%2Finline%2Ftokens%2Findex.txt)

Ask ChatGPT
(https://chatgpt.com/?q=Please%20help%20me%20understand%20this%20documentation%3A%20%2Fpatitas%2Fapi%2Fparsing%2Finline%2Ftokens%2Findex.txt)

Ask Gemini
(https://gemini.google.com/app?q=Please%20help%20me%20understand%20this%20documentation%3A%20%2Fpatitas%2Fapi%2Fparsing%2Finline%2Ftokens%2Findex.txt)

Ask Copilot
(https://copilot.microsoft.com/?q=Please%20help%20me%20understand%20this%20documentation%3A%20%2Fpatitas%2Fapi%2Fparsing%2Finline%2Ftokens%2Findex.txt)

Module

#
`parsing.inline.tokens`

Typed inline tokens for Patitas parser.

Uses NamedTuples for inline token representation, providing:

- Immutability by default (required for external match tracking)

- Tuple unpacking support

- Lower memory footprint (~80 bytes vs ~200 for dict)

- Faster attribute access (tuple index vs hash lookup)

- Full type safety with IDE autocomplete

Thread Safety:

All tokens are immutable and safe to share across threads.

Usage:

from patitas.parsing.inline.tokens import (
DelimiterToken,
TextToken,
InlineToken,
)

```
token = DelimiterToken(char="*", count=2, can_open=True, can_close=False)
```

match token:
case DelimiterToken(char="*", count=count):
print(f"Asterisk delimiter with count {count}")

6Classes

## Classes

`DelimiterToken`

7

▼

Delimiter token for emphasis/strikethrough processing.

Bases:

`NamedTuple`

Delimiter token for emphasis/strikethrough processing.

#### Attributes

Name
Type
Description

`char`

`DelimiterChar`

DelimiterChar

`run_length`

`int`

int

`can_open`

`bool`

bool

`can_close`

`bool`

bool

`tag`

`int`

int = TOKEN_DELIMITER

#### Methods

`type`

0

`Literal['delimiter']`

▼

Token type identifier for dispatch.

property

`def type(self) -> Literal['delimiter']`

##### Returns

`Literal['delimiter']`

`original_count`

0

`int`

▼

Original count (same as run_length for immutable tokens).

property

`def original_count(self) -> int`

##### Returns

`int`

`TextToken`

3

▼

Plain text token.

Bases:

`NamedTuple`

Plain text token.

#### Attributes

Name
Type
Description

`content`

`str`

—

`tag`

`int`

—

#### Methods

`type`

0

`Literal['text (/patitas/api/text/)']`

▼

Token type identifier for dispatch.

property

`def type(self) -> Literal['text']`

##### Returns

`Literal['text (/patitas/api/text/)']`

`CodeSpanToken`

3

▼

Inline code span token.

Bases:

`NamedTuple`

Inline code span token.

#### Attributes

Name
Type
Description

`code`

`str`

—

`tag`

`int`

—

#### Methods

`type`

0

`Literal['code_span']`

▼

Token type identifier for dispatch.

property

`def type(self) -> Literal['code_span']`

##### Returns

`Literal['code_span']`

`NodeToken`

3

▼

Pre-parsed AST node token (links, images, etc.).

Bases:

`NamedTuple`

Pre-parsed AST node token (links, images, etc.).

#### Attributes

Name
Type
Description

`node`

`Inline`

—

`tag`

`int`

—

#### Methods

`type`

0

`Literal['node (/patitas/api/parsing/inline/tokens/#NodeToken)']`

▼

Token type identifier for dispatch.

property

`def type(self) -> Literal['node']`

##### Returns

`Literal['node (/patitas/api/parsing/inline/tokens/#NodeToken)']`

`HardBreakToken`

2

▼

Hard line break token.

Bases:

`NamedTuple`

Hard line break token.

#### Attributes

Name
Type
Description

`tag`

`int`

—

#### Methods

`type`

0

`Literal['hard_break']`

▼

Token type identifier for dispatch.

property

`def type(self) -> Literal['hard_break']`

##### Returns

`Literal['hard_break']`

`SoftBreakToken`

2

▼

Soft line break token.

Bases:

`NamedTuple`

Soft line break token.

#### Attributes

Name
Type
Description

`tag`

`int`

—

#### Methods

`type`

0

`Literal['soft_break']`

▼

Token type identifier for dispatch.

property

`def type(self) -> Literal['soft_break']`

##### Returns

`Literal['soft_break']`