Module

formatters.null

Null formatter for Rosettes.

Does nothing but return the raw text. Useful for timing or as a fallback. Thread-safe and optimized for streaming.

Design Philosophy:

The null formatter exists for specific use cases where you need the formatter interface but don't want any transformation.

Use Cases:

  1. Benchmarking: Measure lexer performance without formatter overhead
  2. Testing: Verify tokenization without HTML/ANSI complexity
  3. Fallback: Handle unsupported output formats gracefully
  4. Pipeline Integration: When downstream expects raw text

Performance:

The absolute minimum overhead possible:

  • format(): yieldtoken.value(one attribute access per token)
  • format_fast(): yield value (already unpacked)

~5µs per 100-line file (vs ~50µs for HTML)

Example:

>>> from rosettes import highlight
>>> raw = highlight("def foo(): pass", "python", formatter="null")
>>> raw
'def foo(): pass'

See Also:

  • rosettes.formatters.html: HTML output with CSS classes
  • rosettes.formatters.terminal: ANSI terminal output
1Class

Classes

NullFormatter 5 ▼
Formatter that yields raw token values without any styling. Thread-safe: immutable dataclass with …

Formatter that yields raw token values without any styling.

Thread-safe: immutable dataclass with no shared state.

This is the simplest possible formatter — it just concatenates token values without any transformation.

Example:

>>> from rosettes import get_lexer
>>> from rosettes.formatters import NullFormatter
>>> lexer = get_lexer("python")
>>> formatter = NullFormatter()
>>> output = formatter.format_string(lexer.tokenize("x = 1"))
>>> output
'x = 1'

Use Cases:

  • Benchmarking lexer performance
  • Testing tokenization correctness
  • Fallback when output format is unsupported

Methods

name 0 str ▼
property
def name(self) -> str
Returns
str
format 2 Iterator[str] ▼
Format tokens by yielding their raw values.
def format(self, tokens: Iterator[Token], config: FormatConfig | None = None) -> Iterator[str]
Parameters
Name Type Description
tokens —
config — Default:None
Returns
Iterator[str]
format_fast 2 Iterator[str] ▼
Fast formatting — just yield raw values.
def format_fast(self, tokens: Iterator[tuple[TokenType, str]], config: FormatConfig | None = None) -> Iterator[str]
Parameters
Name Type Description
tokens —
config — Default:None
Returns
Iterator[str]
format_string 2 str ▼
Format tokens and return as a single string.
def format_string(self, tokens: Iterator[Token], config: FormatConfig | None = None) -> str
Parameters
Name Type Description
tokens —
config — Default:None
Returns
str
format_string_fast 2 str ▼
Fast format and return as a single string.
def format_string_fast(self, tokens: Iterator[tuple[TokenType, str]], config: FormatConfig | None = None) -> str
Parameters
Name Type Description
tokens —
config — Default:None
Returns
str