Module

cache

Cache subsystem for incremental builds and fast queries.

This package provides caching infrastructure that enables Bengal's incremental build system, achieving 10-100x faster rebuilds by tracking file changes, dependencies, and pre-computed indexes.

Core Components:

BuildCache: Central cache for file fingerprints, dependencies, and build state. Tracks mtime/size/hash for change detection, template dependencies, and taxonomy indexes. Persisted as compressed JSON (92-93% smaller with Zstandard).

CacheStore: Generic type-safe storage for Cacheable types with version management. Handles JSON serialization, compression, and tolerant loading.

EffectTracer: Tracks dependencies during rendering via declarative effects. Enables selective rebuilding when dependencies change. (Located in bengal.effects.tracer.)

QueryIndex: Base class for O(1) page lookups by attribute. Built-in indexes include section, author, category, and date_range.

TaxonomyIndex: Bidirectional tag-to-page mappings for incremental taxonomy updates without full rebuilds.

Caching Strategy:

  • File fingerprints: Fast mtime+size check, SHA256 hash for verification
  • Parsed content: Cached HTML/TOC skips re-parsing when only templates change
  • Rendered output: Cached final HTML skips both parsing and template rendering
  • Query indexes: Pre-computed for O(1) template lookups

Performance Impact:

  • Incremental builds: 10-100x faster than full builds
  • Change detection: <1ms per file (mtime+size fast path)
  • Compression: 92-93% cache size reduction, <1ms overhead

Directory Structure:

.bengal/ ├── cache.json.zst # Main build cache (compressed) ├── page_metadata.json.zst # Page discovery cache (compressed) ├── asset_deps.json.zst # Asset dependency map (compressed) ├── taxonomy_index.json.zst # Tag/category index (compressed) └── indexes/ # Query indexes (section, author, etc.)

Performance Note:

This module uses lazy imports to avoid loading heavy cache infrastructure until it's actually needed. The lightweight BengalPaths and STATE_DIR_NAME are loaded eagerly.

Related Modules:

  • bengal.orchestration.incremental: Build logic using this cache
  • bengal.rendering.pipeline: Rendering with dependency tracking

See Also:

  • architecture/cache.md: Cache architecture documentation
  • plan/active/rfc-incremental-builds.md: Incremental build design