Module

lexers.typescript_sm

Hand-written TypeScript lexer using composable scanner mixins.

O(n) guaranteed, zero regex, thread-safe.

Language Support:

  • TypeScript 5.x syntax
  • All JavaScript ES2024 features (inherited from JS lexer pattern)
  • Type annotations and generics (<T>,extends, etc.)
  • TypeScript-specific keywords:type,interface,enum,namespace
  • Utility types:Partial,Required,Readonly,Pick,Omit, etc.
  • Decorators (@decorator)
  • satisfiesoperator, const type parameters

Architecture:

Like the JavaScript lexer, uses mixin composition for C-style syntax. TypeScript-specific additions:

  • Type keywords (any,never,unknown, etc.)
  • Decorator handling (@)
  • Extended operator set (!.for non-null assertion)

Performance:

~50µs per 100-line file.

Thread-Safety:

All lookup tables are frozen sets. Scanning methods use local variables only.

See Also:

  • rosettes.lexers.javascript_sm: Base JavaScript pattern
  • rosettes.lexers._scanners: Shared mixin implementations
1Class

Classes

TypeScriptStateMachineLexer 1 ▼
TypeScript lexer using composable mixins. Extends JavaScript syntax with TypeScript-specific featu…

TypeScript lexer using composable mixins.

Extends JavaScript syntax with TypeScript-specific features.

Token Classification:

  • Type keywords: any, boolean, never, number, string, unknown, void
  • Declaration keywords: type, interface, enum, class, function
  • Namespace keywords: import, export, namespace
  • Utility types as builtins: Partial, Required, Pick, Omit, etc.

Special Handling:

  • Decorators: @decorator → NAME_DECORATOR
  • Template literals:string ${expr}→ STRING
  • Non-null assertion: !. operator

Methods

tokenize 2 Iterator[Token] ▼
def tokenize(self, code: str, config: LexerConfig | None = None) -> Iterator[Token]
Parameters
Name Type Description
code —
config — Default:None
Returns
Iterator[Token]