Free-Threading

Bengal is designed for Python 3.14t (free-threaded Python). On free-threaded builds, page rendering achieves true parallelism — no GIL contention. The same code runs on standard Python; it just uses sequential rendering until you switch interpreters.

Run It

uv python install 3.14t
uv run --python=3.14t bengal build

Bengal auto-detects free-threading at runtime and uses ThreadPoolExecutorwhen available.

Key Patterns

Runtime Detection

Bengal checkssys._is_gil_enabled()to decide whether threads provide real parallelism:

def is_free_threaded() -> bool:
    if hasattr(sys, "_is_gil_enabled"):
        try:
            return not sys._is_gil_enabled()
        except (AttributeError, TypeError):
            pass
    # Fallback: sysconfig for Py_GIL_DISABLED
    ...

When True, Bengal spins up a ThreadPoolExecutorfor page rendering.

Immutable Snapshots (Lock-Free)

After content discovery, Bengal freezes the entire site into immutable dataclasses —PageSnapshot, SectionSnapshot, SiteSnapshot. During rendering, workers only read from snapshots. No locks in the hot path.

@dataclass(frozen=True, slots=True)
class PageSnapshot:
    title: str
    href: str
    source_path: Path
    parsed_html: str
    content_hash: str
    section: SectionSnapshot | None = None
    next_page: PageSnapshot | None = None
    prev_page: PageSnapshot | None = None
    ...

Context Propagation

ThreadPoolExecutor.submit() does not inherit ContextVar values. Bengal uses contextvars.copy_context().runso each worker gets the parent's context:

ctx = contextvars.copy_context()
future_to_page = {
    executor.submit(ctx.run, process_page_with_pipeline, page): page
    for page in batch
}

Thread-Local Pipelines

Each worker gets its own rendering pipeline viathreading.local(), avoiding shared mutable state:

def get_or_create_pipeline(current_gen, create_pipeline_fn):
    needs_new = (
        not hasattr(thread_local, "pipeline")
        or getattr(thread_local, "pipeline_generation", -1) != current_gen
    )
    if needs_new:
        thread_local.pipeline = create_pipeline_fn()
        thread_local.pipeline_generation = current_gen
    return thread_local.pipeline

Performance

On free-threaded Python, ~1.5–2x faster than GIL builds at the same worker count. Incremental builds: 35–80 ms for a single-page change on an 800-page site.

Thread-Safe Internals (0.2.7+)

Several internal components were hardened for free-threading in v0.2.7:

EffectTracer Lock Serialization

The build tracer records timing, cache hits, and fingerprint data. Without the GIL, concurrentdefaultdict access can observe partially-constructed entries. All EffectTracer mutations (record, record_batch, update_fingerprint, flush_pending_fingerprints, clear) and reads now acquire a lock:

class EffectTracer:
    def __init__(self):
        self._lock = threading.Lock()
        self._effects: dict[str, list] = defaultdict(list)

    def record(self, phase, event):
        with self._lock:
            self._effects[phase].append(event)

Thread-Safe LRU Cache

functools.lru_cache relies on the GIL for internal dict coherency. Bengal's get_bengal_dir() — called from multiple threads during rendering — now uses LRUCache from bengal.utils.primitives.lru_cache, which protects access with an RLock:

# Before (GIL-dependent):
@lru_cache(maxsize=1)
def get_bengal_dir(): ...

# After (free-threading safe):
_cache = LRUCache(maxsize=1)
def get_bengal_dir():
    return _cache.get_or_compute("bengal_dir", _find_bengal_dir)

Plugin Registry Freeze

The plugin system uses a builder→immutable pattern.PluginRegistry collects extensions during startup, then freeze() produces a FrozenPluginRegistry(frozen dataclass with tuple fields) — no locks needed during rendering because the data is immutable.

Code References

See Also

• Large Site Optimization — Free-threading notes and scaling
• Performance Overview — Incremental and parallel strategies
• Bengal source