Bengal 0.1.5

rc1

Directive system v2, media embeds, proactive template validation, build-integrated health checks, zstd cache compression, and 25+ feature modules

Executive Summary

This release candidate delivers a complete directive system overhaul with named closers and contract validation, rich media embedding (YouTube, Vimeo, CodePen, etc.), proactive template validation, build-integrated health checks, and zstd cache compression for 10x faster cache I/O.

Key Highlights:

  • 🎯 Directive System v2: Named closers (:::{/tabs}), typed options, contract validation
  • 🎬 Media Embed Directives: YouTube, Vimeo, Gist, CodePen, CodeSandbox, StackBlitz, Asciinema
  • ✅ Proactive Template Validation: Catch syntax errors before build starts
  • 🏥 Build-Integrated Validation: 95% faster health checks (4.6s → 64ms)
  • 🗜️ Zstd Cache Compression: 92% smaller caches, 10x faster I/O (PEP 784)
  • ⚡ Parallel Health Checks: 50-70% faster validation with auto-scaling workers
  • 👁️ Page Visibility System: Granular control over nav, listings, sitemap, search, RSS

Who should test: All users interested in the new directive syntax, media embeds, or performance improvements.

🎯 Major Features

Directive System v2

Complete architectural overhaul with three major improvements:

1. Named Closers — No more fence-depth counting!

1
2
3
4
5
6
7
8
9
:::{tabs}
:::{tab-item} Python
Code here...
:::{/tab-item}

:::{tab-item} JavaScript
More code...
:::{/tab-item}
:::{/tabs}

2. Contract Validation — Invalid nesting caught at parse time:

1
2
3
4
<!-- This now produces a warning instead of broken HTML -->
:::{step}  <!-- ⚠️ Warning: step requires parent steps -->
Content
:::{/step}

3. BengalDirective Base Class — Typed options with automatic coercion:

  • DirectiveTokendataclass for typed AST tokens
  • DirectiveOptionsbase class with bool/int/str coercion
  • Preset options:StyledOptions,TitledOptions,ContainerOptions
  • Preset contracts:STEPS_CONTRACT,TAB_SET_CONTRACT,CARDS_CONTRACT
Migration Guide

Backward compatible: Standard::::fence counting still works. Named closers are opt-in.

When to use named closers:

  • Complex nesting (3+ levels deep)
  • Directives inside code examples
  • When fence counting becomes error-prone

Syntax:

  • Opening::::{directive-name}(unchanged)
  • Named closer::::{/directive-name}(new!)
  • Standard closer::::(still works)

Media Embed Directives

Rich media embedding with privacy-first defaults:

Video Platforms:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
:::{youtube} dQw4w9WgXcQ
:title: Never Gonna Give You Up
:::

:::{vimeo} 148751763
:color: 00adef
:::

:::{video} /videos/demo.mp4
:autoplay: false
:controls: true
:::

Code Playgrounds:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
:::{gist} octocat/6cad326836d38bd3a7ae
:file: hello.py
:::

:::{codepen} aWJVwy
:theme: dark
:tab: result
:::

:::{codesandbox} new
:view: preview
:::

:::{stackblitz} vitejs-vite
:file: src/App.tsx
:::

Terminal & Figures:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
:::{asciinema} 335480
:autoplay: true
:speed: 2
:::

:::{figure} /images/architecture.png
:alt: System architecture diagram
:caption: Bengal's modular architecture
:::

:::{audio} /sounds/notification.mp3
:controls: true
:::
Security & Accessibility

Privacy-first defaults:

  • YouTube usesyoutube-nocookie.comby default
  • Vimeo uses Do Not Track mode
  • All embeds includeloading="lazy"

Accessibility:

  • ARIA labels on all embeds
  • Descriptive titles required
  • Keyboard navigation support
  • Screen reader compatible

CSP requirements: Add to your Content Security Policy:

frame-src youtube-nocookie.com player.vimeo.com codepen.io codesandbox.io stackblitz.com;

Proactive Template Validation

Catch template syntax errors before content discovery:

1
2
3
# In bengal.toml
[build]
validate_templates = true

What it catches:

  • Undefined variables
  • Syntax errors in Jinja2 templates
  • Missing template files
  • Invalid filter usage

BuildStats categories:

  • syntax_errors: Template syntax issues
  • not_found_errors: Missing template files

Note

Template validation runs before content discovery, so you get immediate feedback on template issues without waiting for the full build.

Build-Integrated Validation

Health checks now run during the build instead of re-reading files afterward:

Before: 4.6 seconds (773 disk reads) After: 64 milliseconds (uses cached content)

Tiered validation:

1
2
3
4
5
# In config/_default/health.yaml
health_check:
  build_validators: [frontmatter, directives]    # Fast, every build
  full_validators: [links, seo, accessibility]   # Comprehensive
  ci_validators: [all]                           # Everything in CI

New CLI command:

1
2
3
4
5
bengal validate                    # Run validation
bengal validate --file docs/       # Validate specific path
bengal validate --changed          # Only changed files
bengal validate --incremental      # Use cached results
bengal validate --watch            # Continuous validation

Zstandard Cache Compression

Python 3.14's newcompression.zstdmodule (PEP 784) for dramatically better cache performance:

Size reduction: 92-93% (12-14x compression) Load time: 10x faster (0.5ms vs 5ms) Save time: 3x faster (1ms vs 3ms)

Cache files now use.json.zstextension. Auto-migration from uncompressed.jsonfiles.

CI/CD impact: 16x smaller cache transfers (1.6MB → 100KB)

Parallel Health Check Validators

Validators now run in parallel with auto-scaling:

  • 50-70% faster health checks
  • Auto-scaling workers (50% of CPU cores, 2-8 range)
  • Error isolation — one validator crash doesn't affect others
  • HealthCheckStats dataclass with speedup/efficiency metrics
bengal health --verbose  # Shows execution mode, worker count, metrics

Page Visibility System

Granular control over page visibility across all outputs:

Quick hide:

1
2
3
4
---
title: Draft Post
hidden: true  # Excludes from nav, listings, sitemap, search, RSS
---

Granular control:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
---
title: Internal Page
visibility:
  menu: false       # Hide from navigation
  listings: false   # Hide from section listings
  sitemap: false    # Exclude from sitemap.xml
  search: false     # Exclude from search index
  rss: false        # Exclude from RSS feeds
  robots: noindex   # Add noindex meta tag
---

Environment-aware rendering:

1
2
3
4
---
visibility:
  render: local  # Only render in local dev, not production
---

🏗️ Architecture Improvements

TemplateEngine Package Decoupling

Refactored 861-line monolithictemplate_engine.pyinto focused package:

  • core.py— Thin facade with mixins
  • environment.py— Jinja environment setup
  • asset_url.py— Asset URL generation
  • menu.py— Menu caching helpers
  • manifest.py— Asset manifest loading
  • url_helpers.py— URL generation utilities

Progressive Enhancements Architecture

New JavaScript architecture for progressive enhancement:

1
2
3
<!-- Declare enhancements via data attributes -->
<div data-bengal="tabs">...</div>
<button data-bengal="theme-toggle">...</button>

Features:

  • bengal-enhance.js(~1.5KB) loader with registry
  • Modular enhancement scripts (theme-toggle, mobile-nav, tabs, toc)
  • Auto-discovery and lazy-loading
  • MutationObserver for dynamic content

Systematic Observability

Newobservability.pymodule for standardized stats collection:

  • ComponentStatsdataclass for counts, cache metrics, timings
  • HasStatsprotocol for components
  • format_summary()for compact CLI output
  • format_phase_stats()for slow phase diagnostics

Virtual Section Page Reference Fix

Fixed critical bug where virtual pages had flat navigation:

  • Added_section_urlfield for URL-based section lookups
  • get_section_by_url()for O(1) virtual section lookups
  • Consolidated_setup_page_referencesinto Site

Centralized Path Resolution

NewPathResolverutility for consistent path handling:

  • resolve(),resolve_many(),resolve_if_exists()
  • Security methods:is_within_base(),relative_to_base()
  • Path traversal protection

📚 Autodoc Enhancements

Incremental Build Support

Autodoc now supports incremental builds:

  • Watch autodoc source directories for changes
  • Selective rebuilds based on source file changes
  • Dependency tracking in cache

Resilience Improvements

Better error handling and observability:

  • AutodocRunResultsummary with success/failure tracking
  • autodoc.strictconfig flag for fail-fast in CI/CD
  • Fallback-rendered pages tagged with_autodoc_fallback_template

Typed Metadata Access

14 new typed metadata helpers with fallback:

1
2
3
4
get_python_class_bases(element)
get_openapi_tags(element)
get_cli_command_name(element)
# ... 11 more

🔒 Security & Robustness

Phase 1: Security Hardening

  • Actual internal link validation with URL resolution
  • Include file size limits (10MB max)
  • Symlink rejection in include directives
  • Inode-based loop detection in discovery

Phase 2: Feature Correctness

  • Cross-platform file locking for concurrent build safety
  • Graph analysis fixes for link extraction

Phase 3: Developer Experience

  • Translation warnings for missing i18n keys
  • Theme not found warnings on stderr
  • Graceful handling of empty feeds

🛠️ CLI Improvements

New Commands

1
2
3
bengal validate                    # Standalone validation
bengal fix                         # Auto-fix common issues
bengal sources list/status/fetch   # Content layer management

UX Polish

  • bengal new sitepreferred overbengal site new(deprecated)
  • Interactive baseurl prompt during site creation
  • Hidden advanced linkcheck options (still functional)

🎨 Theme & CSS Improvements

Typography Refinements

  • Optional heading fonts with--font-heading
  • Display tracking gradient
  • Reordered code font stack for modern developer fonts

Component Updates

  • Dropdown directive with icon, badge, color options
  • Checklist with interactive checkboxes and progress bar
  • Steps directive with:start:,:optional:,:duration:options
  • Example-label directive for lightweight section headers

📦 Content Layer API

New unified content abstraction for remote content sources:

1
2
3
4
5
6
# In bengal.toml
[collections.external-docs]
loader = "github"
repo = "owner/repo"
path = "docs/"
branch = "main"

Supported sources:

  • local— Filesystem content
  • github— GitHub repositories
  • rest— REST APIs with field mapping
  • notion— Notion databases

CLI commands:

1
2
3
4
bengal sources list     # Show configured sources
bengal sources status   # Check source availability
bengal sources fetch    # Fetch remote content
bengal sources clear    # Clear cached content

🧹 Cleanup & Quality

Code Quality Improvements

  • BuildOrchestrator refactored from 894-linebuild()into 20 focused_phase_*methods
  • health/directives.py(1,094 lines) → package with focused modules
  • cli/new.py(1,088 lines) → package structure
  • Removed experimental reactive pipeline (15 files)

Directive Registry

Single source of truth for directive names:

  • DIRECTIVE_NAMESclass attribute on all 27 directives
  • get_known_directive_names()computed from registry
  • Health checks use rendering package as source of truth

Upgrading

uv pip install --upgrade bengal==0.1.5-rc1

pip install --upgrade bengal==0.1.5rc1

After Upgrading

  1. Directive syntax: Named closers are opt-in; existing syntax works unchanged
  2. Media embeds: New directives available immediately
  3. Template validation: Enable withvalidate_templates = truein build config
  4. Cache format: Auto-migrates from.jsonto.json.zst
  5. Health checks: Now integrated into build; standalonebengal validateavailable

Note

This is a Release Candidate. Please test thoroughly and report any issues before the final 0.1.5 release.

Known Issues

  • Line-length warnings in inline JavaScript (cosmetic, doesn't affect functionality)

What's Next for 0.1.5 Final

  • Address any RC feedback
  • Final documentation polish
  • Performance regression testing

Thank You

Thank you to everyone testing this release candidate! Your feedback helps ensure a stable 0.1.5 final release.