Token Types - Rosettes

Complete reference for theTokenTypeenumeration. Each token type maps to a Pygments-compatible CSS class.

Keywords

TokenType	Pygments Class	Description	Example
`KEYWORD`	`.k`	Generic keyword	`if`,`else`,`for`
`KEYWORD_CONSTANT`	`.kc`	Constant keyword	`True`,`False`,`None`
`KEYWORD_DECLARATION`	`.kd`	Declaration	`let`,`const`,`var`
`KEYWORD_NAMESPACE`	`.kn`	Namespace keyword	`import`,`from`,`as`
`KEYWORD_PSEUDO`	`.kp`	Pseudo keyword	`self`,`cls`
`KEYWORD_RESERVED`	`.kr`	Reserved keyword	`goto`
`KEYWORD_TYPE`	`.kt`	Type keyword	`int`,`str`,`bool`

Names

TokenType	Pygments Class	Description	Example
`NAME`	`.n`	Generic name	`variable`
`NAME_ATTRIBUTE`	`.na`	Attribute	`.attr`
`NAME_BUILTIN`	`.nb`	Built-in name	`print`,`len`
`NAME_BUILTIN_PSEUDO`	`.bp`	Pseudo built-in	`self`,`cls`
`NAME_CLASS`	`.nc`	Class name	`MyClass`
`NAME_CONSTANT`	`.no`	Constant name	`PI`,`MAX_SIZE`
`NAME_DECORATOR`	`.nd`	Decorator	`@property`
`NAME_ENTITY`	`.ni`	Entity name	HTML entities
`NAME_EXCEPTION`	`.ne`	Exception name	`ValueError`
`NAME_FUNCTION`	`.nf`	Function name	`my_func`
`NAME_FUNCTION_MAGIC`	`.fm`	Magic method	`__init__`
`NAME_LABEL`	`.nl`	Label	`goto_label:`
`NAME_NAMESPACE`	`.nn`	Namespace	`package.module`
`NAME_OTHER`	`.nx`	Other name	Misc identifiers
`NAME_PROPERTY`	`.py`	Property	`@property`
`NAME_TAG`	`.nt`	Tag name	HTML/XML tags
`NAME_VARIABLE`	`.nv`	Variable	`$var`
`NAME_VARIABLE_CLASS`	`.vc`	Class variable	`@@var`
`NAME_VARIABLE_GLOBAL`	`.vg`	Global variable	`$GLOBAL`
`NAME_VARIABLE_INSTANCE`	`.vi`	Instance variable	`@var`
`NAME_VARIABLE_MAGIC`	`.vm`	Magic variable	`__name__`

Literals

TokenType	Pygments Class	Description	Example
`LITERAL`	`.l`	Generic literal
`LITERAL_DATE`	`.ld`	Date literal	`2026-01-01`

Strings

TokenType	Pygments Class	Description	Example
`STRING`	`.s`	Generic string	`"hello"`
`STRING_AFFIX`	`.sa`	String affix	`f`,`r`,`b`
`STRING_BACKTICK`	`.sb`	Backtick string	`string`
`STRING_CHAR`	`.sc`	Character	`'c'`
`STRING_DELIMITER`	`.dl`	Delimiter	`"`,`'`
`STRING_DOC`	`.sd`	Docstring	`"""doc"""`
`STRING_DOUBLE`	`.s2`	Double-quoted	`"string"`
`STRING_ESCAPE`	`.se`	Escape sequence	`\n`,`\t`
`STRING_HEREDOC`	`.sh`	Heredoc	`<<EOF`
`STRING_INTERPOL`	`.si`	Interpolation	`{var}`
`STRING_OTHER`	`.sx`	Other string
`STRING_REGEX`	`.sr`	Regex	`/pattern/`
`STRING_SINGLE`	`.s1`	Single-quoted	`'string'`
`STRING_SYMBOL`	`.ss`	Symbol	`:symbol`

Numbers

TokenType	Pygments Class	Description	Example
`NUMBER`	`.m`	Generic number	`42`
`NUMBER_BIN`	`.mb`	Binary	`0b1010`
`NUMBER_FLOAT`	`.mf`	Float	`3.14`
`NUMBER_HEX`	`.mh`	Hexadecimal	`0xff`
`NUMBER_INTEGER`	`.mi`	Integer	`42`
`NUMBER_INTEGER_LONG`	`.il`	Long integer	`42L`
`NUMBER_OCT`	`.mo`	Octal	`0o755`

Operators

TokenType	Pygments Class	Description	Example
`OPERATOR`	`.o`	Operator	`+`,`-`,`=`
`OPERATOR_WORD`	`.ow`	Word operator	`and`,`or`,`not`

Punctuation

TokenType	Pygments Class	Description	Example
`PUNCTUATION`	`.p`	Punctuation	`(`,`)`,`,`
`PUNCTUATION_MARKER`	`.pm`	Marker	`;`,`:`

Comments

TokenType	Pygments Class	Description	Example
`COMMENT`	`.c`	Generic comment
`COMMENT_HASHBANG`	`.ch`	Hashbang	`#!/bin/bash`
`COMMENT_MULTILINE`	`.cm`	Multi-line	`/* ... */`
`COMMENT_PREPROC`	`.cp`	Preprocessor	`#include`
`COMMENT_PREPROCFILE`	`.cpf`	Preprocessor file	`<stdio.h>`
`COMMENT_SINGLE`	`.c1`	Single-line	`// comment`
`COMMENT_SPECIAL`	`.cs`	Special comment	`TODO`,`FIXME`

Generic (Diffs)

TokenType	Pygments Class	Description	Example
`GENERIC`	`.g`	Generic
`GENERIC_DELETED`	`.gd`	Deleted line	`-removed`
`GENERIC_EMPH`	`.ge`	Emphasis	`italic`
`GENERIC_ERROR`	`.gr`	Error
`GENERIC_HEADING`	`.gh`	Heading	`# Heading`
`GENERIC_INSERTED`	`.gi`	Inserted line	`+added`
`GENERIC_OUTPUT`	`.go`	Output
`GENERIC_PROMPT`	`.gp`	Prompt	`>>>`
`GENERIC_STRONG`	`.gs`	Strong	`bold`
`GENERIC_SUBHEADING`	`.gu`	Subheading	`## Sub`
`GENERIC_TRACEBACK`	`.gt`	Traceback

Special

TokenType	Pygments Class	Description	Example
`TEXT`	(empty)	Plain text
`WHITESPACE`	`.w`	Whitespace	spaces, tabs
`ERROR`	`.err`	Error token	Invalid syntax
`OTHER`	`.x`	Other	Unclassified

Usage

from rosettes import tokenize, TokenType

tokens = tokenize("def foo(): pass", "python")

for token in tokens:
    if token.type == TokenType.KEYWORD:
        print(f"Keyword: {token.value}")
    elif token.type == TokenType.NAME_FUNCTION:
        print(f"Function: {token.value}")

Semantic Class Mapping

For semantic CSS classes, each TokenType maps to a readable class name:

TokenType	Semantic Class
`KEYWORD`	`.syntax-keyword`
`NAME_FUNCTION`	`.syntax-function`
`NAME_CLASS`	`.syntax-class`
`STRING`	`.syntax-string`
`NUMBER`	`.syntax-number`
`COMMENT`	`.syntax-comment`

See CSS Classes for the complete mapping.