# incremental URL: /api/orchestration/incremental/ Section: orchestration -------------------------------------------------------------------------------- incremental - Bengal window.BENGAL_THEME_DEFAULTS = { appearance: 'dark', palette: 'snow-lynx' }; // Progressive Enhancement System Configuration window.Bengal = window.Bengal || {}; window.Bengal.enhanceBaseUrl = '/bengal/assets/js/enhancements'; window.Bengal.watchDom = true; window.Bengal.debug = false; (function () { try { var defaults = window.BENGAL_THEME_DEFAULTS || { appearance: 'system', palette: '' }; var defaultAppearance = defaults.appearance; if (defaultAppearance === 'system') { defaultAppearance = (window.matchMedia && window.matchMedia('(prefers-color-scheme: dark)').matches) ? 'dark' : 'light'; } var storedTheme = localStorage.getItem('bengal-theme'); var storedPalette = localStorage.getItem('bengal-palette'); var theme = storedTheme ? (storedTheme === 'system' ? defaultAppearance : storedTheme) : defaultAppearance; var palette = storedPalette ?? defaults.palette; document.documentElement.setAttribute('data-theme', theme); if (palette) { document.documentElement.setAttribute('data-palette', palette); } } catch (e) { document.documentElement.setAttribute('data-theme', 'light'); } })(); Skip to main content Magnifying Glass ESC Recent Clear Magnifying Glass No results for "" Try different keywords or check your spelling Start typing to search... ↑↓ Navigate ↵ Open ESC Close Powered by Lunr ᓚᘏᗢ Documentation Info About Arrow Clockwise Get Started Note Tutorials File Text Content Palette Theming Settings Building Starburst Extending Bookmark Reference Learning Tracks Releases Dev GitHub API Reference bengal CLI Magnifying Glass Search ⌘K Palette Appearance Chevron Down Mode Monitor System Sun Light Moon Dark Palette Snow Lynx Brown Bengal Silver Bengal Charcoal Bengal Blue Bengal List ᓚᘏᗢ Magnifying Glass Search X Close Documentation Info About Arrow Clockwise Get Started Note Tutorials File Text Content Palette Theming Settings Building Starburst Extending Bookmark Reference Learning Tracks Releases Dev GitHub API Reference bengal CLI Palette Appearance Chevron Down Mode Monitor System Sun Light Moon Dark Palette Snow Lynx Brown Bengal Silver Bengal Charcoal Bengal Blue Bengal API Reference __main__ bengal Caret Right Folder Analysis community_detection graph_analysis graph_reporting graph_visualizer knowledge_graph link_suggestions link_types page_rank path_analysis performance_advisor results Caret Right Folder Assets manifest pipeline Caret Right Folder Autodoc base config docstring_parser utils virtual_orchestrator Caret Right Folder Extractors cli openapi python Caret Right Folder Models cli common openapi python Caret Right Folder Cache asset_dependency_map cache_store cacheable compression dependency_tracker page_discovery_cache query_index query_index_registry taxonomy_index utils Caret Right Folder Build Cache autodoc_tracking core file_tracking fingerprint parsed_content_cache rendered_output_cache taxonomy_index_mixin validation_cache Caret Right Folder Indexes author_index category_index date_range_index section_index Caret Right Folder Cli __main__ base site_templates utils Caret Right Folder Commands assets build clean collections config debug explain fix health init perf project serve site skeleton sources theme utils validate Caret Right Folder Graph __main__ bridges communities orphans pagerank report suggest Caret Right Folder New config presets scaffolds site wizard Caret Right Folder Helpers cli_app_loader cli_output config_validation error_handling menu_config metadata progress site_loader traceback validation Caret Right Folder Skeleton hydrator schema Caret Right Folder Templates base registry Caret Right Folder Blog template Caret Right Folder Changelog template Caret Right Folder Default template Caret Right Folder Docs template Caret Right Folder Landing template Caret Right Folder Portfolio template Caret Right Folder Resume template Caret Right Folder Collections errors loader schemas validator Caret Right Folder Config defaults deprecation directory_loader env_overrides environment feature_mappings hash loader merge origin_tracker validators Caret Right Folder Content Layer entry loaders manager source Caret Right Folder Sources github local notion rest Caret Right Folder Content Types base registry strategies Caret Right Folder Core build_context cascade_engine menu section theme Caret Right Folder Asset asset_core css_transforms Caret Right Folder Page computed content metadata navigation operations page_core proxy relationships utils Caret Right Folder Site core data discovery factories page_caches properties section_registry theme Caret Right Folder Debug base config_inspector content_migrator delta_analyzer dependency_visualizer explainer incremental_debugger models reporter shortcode_sandbox Caret Right Folder Discovery asset_discovery content_discovery Caret Right Folder Fonts downloader generator Caret Right Folder Health autofix base health_check report Caret Right Folder Linkcheck async_checker ignore_policy internal_checker models orchestrator Caret Right Folder Validators anchors assets cache config connectivity cross_ref fonts links menu navigation output performance rendering rss sitemap taxonomy tracks Caret Right Folder Directives analysis checkers constants Caret Right Folder Orchestration asset content full_to_incremental incremental menu postprocess related_posts render section static streaming taxonomy Caret Right Folder Postprocess html_output redirects rss sitemap special_pages Caret Right Folder Output Formats index_generator json_generator llm_generator lunr_index_generator txt_generator utils Caret Right Folder Rendering api_doc_enhancer asset_extractor errors jinja_utils link_transformer link_validator pygments_cache renderer template_context template_profiler validator Caret Right Folder Parsers base factory mistune native_html pygments_patch python_markdown Caret Right Folder Pipeline core output thread_local toc transforms Caret Right Folder Plugins badges cross_references inline_icon term variable_substitution Caret Right Folder Directives _icons admonitions badge base button cache cards checklist code_tabs container contracts data_table dropdown embed errors example_label fenced figure glossary icon include list_table literalinclude marimo navigation options rubric steps tabs target term terminal tokens utils validator video Caret Right Folder Template Engine asset_url core environment manifest menu url_helpers Caret Right Folder Template Functions advanced_collections advanced_strings autodoc collections content crossref data dates debug files get_page i18n icons images math_functions navigation pagination_helpers seo strings tables taxonomies theme urls Caret Right Folder Server build_handler component_preview constants dev_server live_reload pid_manager reload_controller request_handler request_logger resource_manager utils Caret Right Folder Services validation Caret Right Folder Themes config Caret Right Folder Utils atomic_write autodoc build_context build_stats build_summary cli_output css_minifier dates dotdict error_handlers file_io file_lock hashing incremental_constants js_bundler live_progress logger metadata observability page_initializer pagination path_resolver paths performance_collector performance_report profile progress retry rich_console sections swizzle text theme_registry theme_resolution thread_local traceback_config traceback_renderer url_normalization url_strategy API Reference Orchestration ᗢ Caret Down Link Copy URL External Open LLM text Copy Copy LLM text Share with AI Ask Claude Ask ChatGPT Ask Gemini Ask Copilot Module orchestration.incremental Incremental build orchestration for Bengal SSG. Handles cache management, change detection, and determining what needs rebuilding. Uses file hashes, dependency graphs, and taxonomy indexes to identify changed content and minimize rebuild work. Supports both full and incremental builds. Key Concepts: Change detection: File hash comparison for content changes Dependency tracking: Template and data file dependencies Taxonomy invalidation: Tag/category change detection Selective rebuilding: Only rebuild changed pages and dependencies Related Modules: bengal.cache.build_cache: Build cache persistence bengal.cache.dependency_tracker: Dependency graph construction bengal.orchestration.build: Build orchestration entry point See Also: bengal/orchestration/incremental.py: IncrementalOrchestrator for incremental logic plan/active/rfc-incremental-builds.md: Incremental build design View source 1 Class Classes IncrementalOrchestrator Orchestrates incremental build logic for efficient rebuilds. Handles cache initialization, change … 14 Caret Right Orchestrates incremental build logic for efficient rebuilds. Handles cache initialization, change detection, dependency tracking, and selective rebuilding. Uses file hashes, dependency graphs, and taxonomy indexes to minimize rebuild work by only rebuilding changed content. Creation: Direct instantiation: IncrementalOrchestrator(site) Created by BuildOrchestrator when incremental builds enabled Requires Site instance with content populated Attributes Name Type Description site — Site instance for incremental builds cache — BuildCache instance for build state persistence tracker — DependencyTracker instance for dependency graph construction logger — Logger instance for incremental build events Relationships — Uses: BuildCache for build state persistence - Uses: DependencyTracker for dependency graph construction - Used by: BuildOrchestrator for incremental build coordination - Uses: Site for content access and change detection Thread Safety: Not thread-safe. Should be used from single thread during build. Cache and tracker operations are thread-safe internally. Methods 7 initialize Initialize cache and dependency tracker for incremental builds. Sets up BuildC… 1 tuple[BuildCache, D… Caret Right def initialize(self, enabled: bool = False) -> tuple[BuildCache, DependencyTracker] Initialize cache and dependency tracker for incremental builds. Sets up BuildCache and DependencyTracker instances. If enabled, loads existing cache from .bengal/cache.json (migrates from legacy location if needed). If disabled, creates empty cache instances. Parameters 1 enabled bool Whether incremental builds are enabled. If False, creates empty cache instances (full rebuilds always). Returns tuple[BuildCache, DependencyTracker] — Tuple of (BuildCache, DependencyTracker) instances Process: 1. Create .bengal/ directory if enabled 2. Migrate legacy cache from output_dir/.bengal-cache.json if exists 3. Load or create BuildCache instance 4. Create DependencyTracker with cache and site check_config_changed Check if configuration has changed (requires full rebuild). Uses config hash v… 0 bool Caret Right def check_config_changed(self) -> bool Check if configuration has changed (requires full rebuild). Uses config hash validation which captures the effective configuration state: Base config from files (bengal.toml, config/*.yaml) Environment variable overrides (BENGAL_*) Build profile settings (--profile writer) This is more robust than file-based tracking because it detects changes in split config files, env vars, and profiles that wouldn't trigger a file hash change. Returns bool — True if config changed (cache was invalidated) find_work_early Find pages/assets that need rebuilding (early version - before taxonomy generat… 3 tuple[list[Page], l… Caret Right def find_work_early(self, verbose: bool = False, forced_changed_sources: set[Path] | None = None, nav_changed_sources: set[Path] | None = None) -> tuple[list[Page], list[Asset], ChangeSummary] Find pages/assets that need rebuilding (early version - before taxonomy generation). This is called BEFORE taxonomies/menus are generated, so it only checks content/asset changes. Generated pages (tags, etc.) will be determined later based on affected tags. Uses section-level optimization: skips checking individual pages in unchanged sections. Parameters 3 verbose bool Whether to collect detailed change information forced_changed_sources set[Path] | None nav_changed_sources set[Path] | None Returns tuple[list[Page], list[Asset], ChangeSummary] — Tuple of (pages_to_build, assets_to_process, change_summary) where change_summary is a ChangeSummary dataclass (supports dict-like access for compatibility) find_work Find pages/assets that need rebuilding (legacy version - after taxonomy generat… 1 tuple[list[Page], l… Caret Right def find_work(self, verbose: bool = False) -> tuple[list[Page], list[Asset], dict[str, list[Any]]] Find pages/assets that need rebuilding (legacy version - after taxonomy generation). This is the old method that expects generated pages to already exist. Kept for backward compatibility but should be replaced with find_work_early(). Parameters 1 verbose bool Whether to collect detailed change information Returns tuple[list[Page], list[Asset], dict[str, list[Any]]] — Tuple of (pages_to_build, assets_to_process, change_summary) process Bridge-style process for testing incremental invalidation. ⚠️ TEST BRIDGE ONL… 2 None Caret Right def process(self, change_type: str, changed_paths: set[str]) -> None Bridge-style process for testing incremental invalidation. ⚠️ TEST BRIDGE ONLY This method is a lightweight adapter used in tests to simulate an incremental pass without invoking the entire site build orchestrator. Not for production use: Writes placeholder output ("Updated") for verification only Does not perform full rendering or asset processing Skips postprocessing (RSS, sitemap, etc.) Use run() or full_build() for production builds Primarily consumed by: tests/integration/test_full_to_incremental_sequence.py bengal/orchestration/full_to_incremental.py (test bridge helper) Test scenarios validating cache invalidation logic Parameters 2 change_type str One of "content", "template", or "config" changed_paths set[str] Set of paths that changed (ignored for "config") full_rebuild 2 None Caret Right def full_rebuild(self, pages: list[Any], context: BuildContext) -> None Parameters 2 pages list[Any] context BuildContext save_cache Update cache with processed files. 2 None Caret Right def save_cache(self, pages_built: list[Page], assets_processed: list[Asset]) -> None Update cache with processed files. Parameters 2 pages_built list[Page] Pages that were built assets_processed list[Asset] Assets that were processed Internal Methods 7 Caret Right __init__ Initialize incremental orchestrator. 1 None Caret Right def __init__(self, site: Site) Initialize incremental orchestrator. Parameters 1 site Site Site instance for incremental builds _get_changed_sections Identify sections with any changed files (section-level optimization). Uses ma… 1 set[Section] Caret Right def _get_changed_sections(self, sections: list[Section] | None = None) -> set[Section] Identify sections with any changed files (section-level optimization). Uses max mtime of pages in each section to quickly skip entire sections that haven't changed. This is a major optimization for large sites where only a few sections have changes. Parameters 1 sections list[Section] | None List of sections to check. If None, uses site.sections. Returns set[Section] — Set of Section objects that have changed files Performance: O(sections) instead of O(pages) for initial filtering Only checks individual pages in changed sections Uses fast mtime+size check from cache _write_output Write a placeholder output file corresponding to a content path. ⚠️ TEST HELP… 2 None Caret Right def _write_output(self, path: Path, context: BuildContext) -> None Write a placeholder output file corresponding to a content path. ⚠️ TEST HELPER - Used by process() bridge only. For tests that exercise the bridge-only flow, derive the output location from the content path under the site's content dir. Writes diagnostic placeholder content for test verification. Parameters 2 path Path Source content path context BuildContext Build context (not used in this simplified version) _cleanup_deleted_autodoc_sources Clean up autodoc pages when their source files are deleted. Checks tracked aut… 0 None Caret Right def _cleanup_deleted_autodoc_sources(self) -> None Clean up autodoc pages when their source files are deleted. Checks tracked autodoc source files and removes corresponding output when the source no longer exists. This prevents stale autodoc pages from remaining when Python/OpenAPI source files are deleted. _cleanup_deleted_files Clean up output files for deleted source files. Checks cache for source files … 0 None Caret Right def _cleanup_deleted_files(self) -> None Clean up output files for deleted source files. Checks cache for source files that no longer exist and deletes their corresponding output files. This prevents stale content from remaining in the output directory after source deletion. _find_cascade_affected_pages Find all pages affected by a cascade change in a section index. When a section… 1 set[Path] Caret Right def _find_cascade_affected_pages(self, index_page: Page) -> set[Path] Find all pages affected by a cascade change in a section index. When a section's _index.md has cascade metadata and is modified, all descendant pages inherit those values and need to be rebuilt. Parameters 1 index_page Page Section _index.md page with cascade metadata Returns set[Path] — Set of page source paths that should be rebuilt due to cascade _get_theme_templates_dir Get the templates directory for the current theme. 0 Path | None Caret Right def _get_theme_templates_dir(self) -> Path | None Get the templates directory for the current theme. Returns Path | None — Path to theme templates or None if not found ← Previous full_to_incremental Next → menu List © 2025 Bengal ᓚᘏᗢ window.BENGAL_LAZY_ASSETS = { tabulator: '/bengal/assets/js/tabulator.min.js', dataTable: '/bengal/assets/js/data-table.js', mermaidToolbar: '/bengal/assets/js/mermaid-toolbar.9de5abba.js', mermaidTheme: '/bengal/assets/js/mermaid-theme.344822c5.js', graphMinimap: '/bengal/assets/js/graph-minimap.cc7e42e3.js', graphContextual: '/bengal/assets/js/graph-contextual.440e59c6.js' }; window.BENGAL_ICONS = { close: '/bengal/assets/icons/close.911d4fe1.svg', enlarge: '/bengal/assets/icons/enlarge.652035e5.svg', copy: '/bengal/assets/icons/copy.3d56e945.svg', 'download-svg': '/bengal/assets/icons/download.04f07e1b.svg', 'download-png': '/bengal/assets/icons/image.c34dfd40.svg', 'zoom-in': '/bengal/assets/icons/zoom-in.237b4a83.svg', 'zoom-out': '/bengal/assets/icons/zoom-out.38857c77.svg', reset: '/bengal/assets/icons/reset.d26dba29.svg' }; Arrow Up X -------------------------------------------------------------------------------- Metadata: - Author: lbliii - Word Count: 2234 - Reading Time: 11 minutes