ai._providers

Page actions AI-ready formats and sharing
Open LLM text
Share with AI
Ask Claude Ask ChatGPT Ask Gemini Ask Copilot
module

LLM provider implementations.

Each provider is a pair of functions: one for complete generation, one for streaming. Both use raw HTTP via httpx — no provider SDKs required.

Supported providers:

  • anthropic—…

LLM provider implementations.

Each provider is a pair of functions: one for complete generation, one for streaming. Both use raw HTTP via httpx — no provider SDKs required.

Supported providers:

  • anthropic— Claude models (Messages API)
  • openai— GPT models (Chat Completions API)
  • azure— Azure OpenAI deployments (OpenAI-compatible API)
  • gemini— Google Gemini (Generative Language API)
  • bedrock — AWS Bedrock Converse API (optional botocorefor SigV4)
  • ollama— Local models via Ollama (OpenAI-compatible API)
  • lmstudio— Local models via LM Studio (OpenAI-compatible API)
  • localai— Local models via LocalAI (OpenAI-compatible API)

Provider string format:provider:model

  • **anthropic**: claude-sonnet-4-20250514
  • **openai**: gpt-4o
  • **azure**: my-gpt4o-deployment (AZURE_OPENAI_ENDPOINT, AZURE_OPENAI_API_KEY)
  • **gemini**: gemini-2.0-flash (GOOGLE_API_KEY or GEMINI_API_KEY)
  • **bedrock**: anthropic.claude-3-sonnet-20240229-v1:0(AWS credentials + region)
  • **ollama**: llama3.2 (uses OLLAMA_BASEenv, default http://localhost:11434)
  • **lmstudio**: model-id (uses LMSTUDIO_BASEenv, default http://localhost:1234)
  • **localai**: model-id (uses LOCALAI_BASEenv, default http://localhost:8080)

ai._providers

Name Type Default Description
type —
qualified_name —
element_type —
description —
source_file —
line_number —
is_autodoc —
autodoc_element —
_autodoc_template —
_autodoc_url_path —
_autodoc_page_type —
title —
doc_content_hash —

Symbols on this page

class ProviderConfig

Parsed provider configuration.

Jump to symbol function parse_provider

Parse aprovider:modelstring into a config.

Jump to symbol function _get_httpx

Import httpx or raise a clear error.

Jump to symbol async function _iter_sse_events

Parse SSE events from response stream. Yields parsed JSON event dicts.

Jump to symbol async function anthropic_generate

Generate a complete response from Anthropic's Messages API.

Jump to symbol async function anthropic_complete

Generate a completion, optionally with tool-use blocks.

Jump to symbol async function anthropic_stream

Stream text tokens from Anthropic's Messages API.

Jump to symbol async function openai_generate

Generate a complete response from an OpenAI-compatible Chat Completions API.

Jump to symbol async function openai_complete

Generate a chat completion, optionally with function tools.

Jump to symbol async function openai_stream

Stream text tokens from an OpenAI-compatible Chat Completions API.

Jump to symbol async function gemini_generate

Generate a complete response from Google Gemini.

Jump to symbol async function gemini_complete

Generate a Gemini completion with optional function declarations.

Jump to symbol async function gemini_stream

Stream text tokens from Google Gemini.

Jump to symbol async function bedrock_generate

Generate a complete response via AWS Bedrock Converse.

Jump to symbol async function bedrock_complete

Generate a Bedrock Converse completion with optional tool config.

Jump to symbol async function bedrock_stream

Stream text tokens via AWS Bedrock Converse Stream.

Jump to symbol
ProviderConfig
class

Parsed provider configuration.

parse_provider
function 
def parse_provider(provider_string: str, /, *, api_key: str | None = None) -> ProviderConfig

Parse aprovider:modelstring into a config.

Parameters

Name Type Default Description
provider_string str —
api_key str | None None
_get_httpx
function 
def _get_httpx() -> Any

Import httpx or raise a clear error.

No parameters.

_iter_sse_events
function async 
async def _iter_sse_events(response: Any) -> AsyncIterator[dict[str, Any]]

Parse SSE events from response stream. Yields parsed JSON event dicts.

Parameters

Name Type Default Description
response Any —
anthropic_generate
function async 
async def anthropic_generate(config: ProviderConfig, messages: list[dict[str, str]], *, max_tokens: int = 4096, temperature: float = 0.0, system: str | None = None, json_schema: dict[str, Any] | None = None) -> str

Generate a complete response from Anthropic's Messages API.

Parameters

Name Type Default Description
config ProviderConfig —
messages list[dict[str, str]] —
max_tokens int 4096
temperature float 0.0
system str | None None
json_schema dict[str, Any] | None None
anthropic_complete
function async 
async def anthropic_complete(config: ProviderConfig, messages: list[dict[str, Any]], *, max_tokens: int = 4096, temperature: float = 0.0, system: str | None = None, tools: list[dict[str, Any]] | None = None) -> ChatCompletion

Generate a completion, optionally with tool-use blocks.

Parameters

Name Type Default Description
config ProviderConfig —
messages list[dict[str, Any]] —
max_tokens int 4096
temperature float 0.0
system str | None None
tools list[dict[str, Any]] | None None
anthropic_stream
function async 
async def anthropic_stream(config: ProviderConfig, messages: list[dict[str, str]], *, max_tokens: int = 4096, temperature: float = 0.0, system: str | None = None) -> AsyncIterator[str]

Stream text tokens from Anthropic's Messages API.

Parameters

Name Type Default Description
config ProviderConfig —
messages list[dict[str, str]] —
max_tokens int 4096
temperature float 0.0
system str | None None
openai_generate
function async 
async def openai_generate(config: ProviderConfig, messages: list[dict[str, str]], *, max_tokens: int = 4096, temperature: float = 0.0, json_schema: dict[str, Any] | None = None) -> str

Generate a complete response from an OpenAI-compatible Chat Completions API.

Parameters

Name Type Default Description
config ProviderConfig —
messages list[dict[str, str]] —
max_tokens int 4096
temperature float 0.0
json_schema dict[str, Any] | None None
openai_complete
function async 
async def openai_complete(config: ProviderConfig, messages: list[dict[str, Any]], *, max_tokens: int = 4096, temperature: float = 0.0, tools: list[dict[str, Any]] | None = None) -> ChatCompletion

Generate a chat completion, optionally with function tools.

Parameters

Name Type Default Description
config ProviderConfig —
messages list[dict[str, Any]] —
max_tokens int 4096
temperature float 0.0
tools list[dict[str, Any]] | None None
openai_stream
function async 
async def openai_stream(config: ProviderConfig, messages: list[dict[str, str]], *, max_tokens: int = 4096, temperature: float = 0.0) -> AsyncIterator[str]

Stream text tokens from an OpenAI-compatible Chat Completions API.

Parameters

Name Type Default Description
config ProviderConfig —
messages list[dict[str, str]] —
max_tokens int 4096
temperature float 0.0
gemini_generate
function async 
async def gemini_generate(config: ProviderConfig, messages: list[dict[str, str]], *, max_tokens: int = 4096, temperature: float = 0.0, system: str | None = None, json_schema: dict[str, Any] | None = None) -> str

Generate a complete response from Google Gemini.

Parameters

Name Type Default Description
config ProviderConfig —
messages list[dict[str, str]] —
max_tokens int 4096
temperature float 0.0
system str | None None
json_schema dict[str, Any] | None None
gemini_complete
function async 
async def gemini_complete(config: ProviderConfig, messages: list[dict[str, Any]], *, max_tokens: int = 4096, temperature: float = 0.0, system: str | None = None, tools: Sequence[McpToolInfo] | None = None) -> ChatCompletion

Generate a Gemini completion with optional function declarations.

Parameters

Name Type Default Description
config ProviderConfig —
messages list[dict[str, Any]] —
max_tokens int 4096
temperature float 0.0
system str | None None
tools Sequence[McpToolInfo] | None None
gemini_stream
function async 
async def gemini_stream(config: ProviderConfig, messages: list[dict[str, str]], *, max_tokens: int = 4096, temperature: float = 0.0, system: str | None = None) -> AsyncIterator[str]

Stream text tokens from Google Gemini.

Parameters

Name Type Default Description
config ProviderConfig —
messages list[dict[str, str]] —
max_tokens int 4096
temperature float 0.0
system str | None None
bedrock_generate
function async 
async def bedrock_generate(config: ProviderConfig, messages: list[dict[str, str]], *, max_tokens: int = 4096, temperature: float = 0.0, system: str | None = None) -> str

Generate a complete response via AWS Bedrock Converse.

Parameters

Name Type Default Description
config ProviderConfig —
messages list[dict[str, str]] —
max_tokens int 4096
temperature float 0.0
system str | None None
bedrock_complete
function async 
async def bedrock_complete(config: ProviderConfig, messages: list[dict[str, Any]], *, max_tokens: int = 4096, temperature: float = 0.0, system: str | None = None, tools: Sequence[McpToolInfo] | None = None) -> ChatCompletion

Generate a Bedrock Converse completion with optional tool config.

Parameters

Name Type Default Description
config ProviderConfig —
messages list[dict[str, Any]] —
max_tokens int 4096
temperature float 0.0
system str | None None
tools Sequence[McpToolInfo] | None None
bedrock_stream
function async 
async def bedrock_stream(config: ProviderConfig, messages: list[dict[str, str]], *, max_tokens: int = 4096, temperature: float = 0.0, system: str | None = None) -> AsyncIterator[str]

Stream text tokens via AWS Bedrock Converse Stream.

Parameters

Name Type Default Description
config ProviderConfig —
messages list[dict[str, str]] —
max_tokens int 4096
temperature float 0.0
system str | None None

View source · /home/runner/work/chirp/chirp/site/../src/chirp/ai/_providers.py:1