Module

content_layer.sources.github

GitHubSource - Content source for GitHub repositories.

Fetches markdown files from GitHub repos, supporting both public and private repositories with token authentication.

Requires: pip install bengal[github] (installs aiohttp)

View source
1 Class

Classes

GitHubSource
Content source for GitHub repositories. Fetches markdown files from a GitHub repo using the GitHub…
6

Content source for GitHub repositories.

Fetches markdown files from a GitHub repo using the GitHub API. Supports both public repos and private repos with token authentication.

Configuration:

repo: str - Repository in "owner/repo" format (required)
branch: str - Branch name (default: "main")
path: str - Directory path within repo (default: "")
token: str - GitHub token (optional, uses GITHUB_TOKEN env var)
glob: str - File pattern to match (default: "*.md")
Inherits from ContentSource

Methods 4

fetch_all async
Fetch all markdown files from the repository.
0 AsyncIterator[Conte… 
async def fetch_all(self) -> AsyncIterator[ContentEntry]

Fetch all markdown files from the repository.

Returns

AsyncIterator[ContentEntry]

fetch_one async
Fetch a single file by path.
1 ContentEntry | None 
async def fetch_one(self, id: str) -> ContentEntry | None

Fetch a single file by path.

Parameters 1
id str

Relative path within the configured path
Returns

ContentEntry | None —

ContentEntry if found, None otherwise

get_last_modified async
Get latest commit time for the configured path.
0 datetime | None 
async def get_last_modified(self) -> datetime | None

Get latest commit time for the configured path.

Returns

datetime | None —

Datetime of most recent commit or None

is_changed async
Check if repo has changed since last fetch. Uses latest commit SHA for the path.
1 bool 
async def is_changed(self, cached_checksum: str | None) -> bool

Check if repo has changed since last fetch.

Uses latest commit SHA for the path.

Parameters 1
cached_checksum str | None

Previous commit SHA
Returns

bool —

True if changed or unknown

Internal Methods 2
__init__
Initialize GitHub source.
2 None 
def __init__(self, name: str, config: dict[str, Any]) -> None

Initialize GitHub source.

Parameters 2
name str

Source name
config dict[str, Any]

Configuration with 'repo' required
_fetch_file async
Fetch a single file from GitHub.
3 ContentEntry | None 
async def _fetch_file(self, session: aiohttp.ClientSession, path: str, sha: str | None) -> ContentEntry | None

Fetch a single file from GitHub.

Parameters 3
session aiohttp.ClientSession

aiohttp session
path str

Full path within repo
sha str | None

Git SHA (optional, for cache key)
Returns

ContentEntry | None —

ContentEntry or None