Open LLM text
Share with AI
Ask Claude Ask ChatGPT Ask Gemini Ask Copilot
lbliii 5 min read 988 words Updated 1 month ago

Bengal 0.3.0

Immutable page pipeline, plugin system, structured concurrency, protocol-based architecture

Key additions: Immutable page pipeline with frozen records at every stage, plugin system with 9 extension points, PageProxy deleted (906 lines), structured concurrency via WorkScope, Site mixin hierarchy replaced with protocol types, template dependency tracking for selective rebuilds, free-threading hardening, and excerpt-break directive.

This release includes all changes from the unshipped 0.2.7.

Highlights

Immutable Page Pipeline

The entire page lifecycle now flows through frozen dataclass records. Each build phase produces an immutable snapshot that downstream phases consume — no mutation, no shared mutable state between threads.

  • SourcePage (#199): Captures discovery output (file hash, content hash, frontmatter, lang). Created once during content discovery, never mutated.
  • ParsedPage (#196): Captures parse output (AST, plain text, word count, reading time, links, code blocks). Produced by the Patitas parser, consumed by rendering.
  • RenderedPage (#197): Captures render output (HTML, template used, render duration). Produced by the template engine, consumed by post-processing.
  • SiteSnapshot decomposed (#198): The monolithicSiteSnapshot is split into NavigationPlan, TaxonomyPlan, and RenderSchedule— each frozen, each consumed by exactly one downstream phase.

PageProxy Deleted

The 906-linePageProxy lazy-loading wrapper is gone (#200). Pages are now reconstructed directly from cached PageCore + ParsedPagedata with zero disk I/O for unchanged pages in incremental builds. This eliminates an entire class of threading bugs (lazy-load races) and simplifies the codebase.

Plugin System

Bengal now has a formal plugin framework. Plugins implement aPlugin protocol (name, version, register()) and register extensions through a PluginRegistrythat supports 9 extension points: directives, roles, template functions, template filters, template tests, content sources, health validators, shortcodes, and build phase hooks.

Plugins are auto-discovered via thebengal.pluginsentry point group:

# pyproject.toml of your plugin package
[project.entry-points."bengal.plugins"]
my-plugin = "my_package:MyPlugin"

from bengal.plugins.protocol import Plugin
from bengal.plugins.registry import PluginRegistry

class MyPlugin(Plugin):
    name = "my-plugin"
    version = "1.0.0"

    def register(self, registry: PluginRegistry) -> None:
        registry.add_template_filter("shout", lambda v: v.upper() + "!")
        registry.on_phase("post_render", self.after_render)

The registry uses a builder-to-immutable pattern: mutable during registration, frozen into an immutable FrozenPluginRegistrydataclass before parallel rendering begins. Thread-safe by design.

See Writing Plugins for the full guide.

Structured Concurrency

WorkScope (#189) replaces ad-hoc ThreadPoolExecutorusage across all executor sites. WorkScope provides:

  • Deterministic cleanup: tasks are always joined before scope exits
  • Cancellation propagation: if one task fails, siblings are cancelled
  • Resource accounting: executor lifetime is scoped, not global

All executor call sites have been migrated.

Protocol-Based Architecture

The Site mixin hierarchy (SiteDiscoveryMixin, SiteRenderingMixin, etc.) has been eliminated (#194). Site capabilities are now defined as protocol types — consumers declare what they need, not what class they inherit from. This makes the dependency graph explicit and enables better static analysis.

Template Dependency Tracking

The build cache now records which templates (and their full include/extends chain) each page uses. When a template changes, only the pages that depend on it rebuild — instead of a full site rebuild.

  • First build: dependencies are recorded (no change in behavior)
  • Subsequent builds: template changes trigger selective rebuilds
  • Falls back to full rebuild if no dependency data exists (cache miss or first build)

On a 1,000-page site where you edit a partial template used by 50 pages, this means rebuilding 50 pages instead of 1,000.

Free-Threading Hardening

Three changes prepare Bengal for Python 3.14t (PEP 703) free-threading:

  • EffectTracer: All mutations and reads are now serialized under a lock. Without the GIL, concurrentdefaultdictaccess could observe partial state.
  • get_bengal_dir() cache: Replaced @lru_cache(maxsize=1) with Bengal's LRUCache, which uses an RLock internally. functools.lru_cacherelies on the GIL for thread safety.
  • Kida 0.2.9: Updated the template engine dependency for free-threading support.

Performance

  • O(n^2) elimination (#192): Taxonomy slug normalization was quadratic in page count. Consolidated into a single-pass dictionary lookup. Also fixed O(n^2) in section page sorting.
  • Page walk optimizations: Coalesced 8 redundantsite.pagestraversals into single passes across menu building, finalization, provenance, and rendering. On large sites (1K+ pages), this reduces orchestration phase time by ~8-12%.
  • Stale code audit (#203): ~50 files audited, 205 lines of dead code removed, ty diagnostics reduced from 837 to 715.

Features

Excerpt-Break Directive

New:::{excerpt-break}directive (#202) gives authors explicit control over where page excerpts are cut:

This paragraph appears in the excerpt.

:::{excerpt-break}

This paragraph only appears on the full page.

When absent, the existing max_charsdefault applies unchanged.

Code Simplification

  • Exception hierarchy: Replaced 11 manual__init__ methods in exception subclasses with a _default_build_phase_nameclass variable pattern — less boilerplate, same behavior.
  • Deprecated transforms removed:escape_jinja_blocks(), transform_internal_links(), and normalize_markdown_links() are gone. Their work is handled by HybridHTMLTransformer.
  • Renderer helpers: Extracted_default_pagination() and _coerce_pagination_ints()to eliminate duplicated inline dict construction.

Fixes

  • Render progress bar stuck at 0% during WaveScheduler rendering (#190)
  • Render metrics now accurate with per-page timing and regression detection (#184)
  • xref pipe placeholder surviving Patitasescape_html(#182)
  • CLI maturity audit — fixed bugs, added missing commands, wired sources (#187)
  • Error codes added to all Kida rendering engine exception handlers
  • WriteBehind collector gated on parallel builds to prevent thread deadlocks in sequential mode (#201)
  • CI watchdog replaced SIGALRM (unreliable under free-threading) with daemon thread +os._exit(#199)

Tooling

  • Linting: 8 new ruff rule sets enabled, ty config tightened, migrated to PEP 695 generics (#191)
  • CI: Test pipeline flattened and coverage deduplicated (#185, #186). Restructured for 6 parallel integration shards with signal-based timeouts.
  • Dependencies: Bumped deps, fixed ty 0.0.26 errors, added Renovate config (#183)
  • Test docs: Comprehensivetests/README.mdcovering 4,065+ tests (116 property-based via Hypothesis), 10 test roots, markers, parallel safety, and CI strategy.

Upgrading

uv pip install --upgrade bengal
# or
pip install --upgrade bengal
# or use the self-update command
bengal upgrade

No breaking changes in this release. PageProxy removal is internal — the public API is unchanged.