Module

supervisor

Supervisor — spawns, monitors, and restarts workers.

The supervisor sits between theServer and the Workerlayer. It detects the GIL state at startup and spawns workers as either threads (on nogil / 3.14t) or processes (on GIL builds). The worker implementation is identical in both modes — only the spawning mechanism differs.

Responsibilities:

  • Spawn N workers with their sockets
  • Monitor worker health (is_alive check on a watchdog loop)
  • Restart crashed workers (up tomax_restartsper window)
  • Coordinate graceful shutdown viathreading.Event
  • Forward SIGINT/SIGTERM to workers

Classes

TCPWorker 4
Structural contract for TCP workers (Worker and SyncWorker). Both async and sync workers implement…
Bases: Protocol

Structural contract for TCP workers (Worker and SyncWorker).

Both async and sync workers implement this interface. The supervisor uses it for lifecycle management: spawning, draining, and idle checks.

Methods

run 0
def run(self) -> None
set_lifespan_state 1
def set_lifespan_state(self, state: dict[str, Any]) -> None
Parameters
Name Type Description
state
start_draining 0
def start_draining(self) -> None
is_idle 0 bool
def is_idle(self) -> bool
Returns
bool
_WorkerHandle 1
Metadata about a running worker (thread or process).

Metadata about a running worker (thread or process).

Methods

Internal Methods 1
__init__ 4
def __init__(self, worker_id: int, target: threading.Thread | multiprocessing.Process, worker: TCPWorker | None, generation: int = 0) -> None
Parameters
Name Type Description
worker_id
target
worker
generation Default:0
_H3WorkerHandle 1
Metadata about a running H3 worker (UDP/QUIC).

Metadata about a running H3 worker (UDP/QUIC).

Methods

Internal Methods 1
__init__ 4
def __init__(self, worker_id: int, target: threading.Thread | multiprocessing.Process, worker: H3Worker | None = None, reload_shutdown_event: threading.Event | None = None) -> None
Parameters
Name Type Description
worker_id
target
worker Default:None
reload_shutdown_event Default:None
Supervisor 29
Spawn and supervise N workers as threads or processes. The supervisor detects the GIL state and pi…

Spawn and supervise N workers as threads or processes.

The supervisor detects the GIL state and picks the appropriate spawning strategy automatically. Workers share the frozen ServerConfig and (in thread mode) the ASGI app reference.

Methods

mode 0 WorkerMode
The active worker mode (``"thread"``, ``"process"``, or ``"subinterpreter"``).
property
def mode(self) -> WorkerMode

The active worker mode ("thread", "process", or "subinterpreter").

Returns
WorkerMode
worker_count 0 int
Number of workers the supervisor manages.
property
def worker_count(self) -> int
Returns
int
set_lifespan_state 1
Set the lifespan state dict to be shared with all workers.
def set_lifespan_state(self, state: dict[str, Any]) -> None
Parameters
Name Type Description
state

The state dict populated during lifespan startup.

run 2
Start all workers and block until shutdown. Installs signal handlers, spawns w…
def run(self, sockets: list[socket.socket], udp_sockets: list[socket.socket] | None = None) -> None

Start all workers and block until shutdown.

Installs signal handlers, spawns workers, runs the health-check loop, then joins all workers on shutdown.

Parameters
Name Type Description
sockets

One TCP socket per worker, created bycreate_listeners().

udp_sockets

Optional UDP sockets for HTTP/3 workers.

Default:None
shutdown 0
Signal all workers to stop (non-blocking).
def shutdown(self) -> None
restart_workers 0
Gracefully restart all workers (for dev reload). Signals all running workers t…
def restart_workers(self) -> None

Gracefully restart all workers (for dev reload).

Signals all running workers to stop, waits for them to drain, clears the shutdown event, and spawns fresh workers.

Serialized with graceful_reload and watch-loop respawns via _lifecycle_lock. Skips if a reload is already in progress.

When anapp_pathwas provided and workers run as threads, the app module is reimported so that code changes on disk take effect. Process-based workers inherit the parent via fork and are fully replaced on restart, so reimport is not needed.

graceful_reload 0
Perform zero-downtime rolling restart of all workers. This method implements a…
def graceful_reload(self) -> None

Perform zero-downtime rolling restart of all workers.

This method implements a rolling restart strategy:

  1. Reimport the app (thread mode only)
  2. Spawn new worker generation
  3. Mark old workers for draining (finish existing, reject new connections)
  4. Wait for old workers to become idle
  5. Shut down old workers

This ensures zero dropped requests during code reload.

Note: Only works in thread mode. In process mode, falls back to restart_workers() which has brief downtime.

Internal Methods 22
__init__ 7
def __init__(self, config: ServerConfig, app: ASGIApp, *, mode: WorkerMode | None = None, ssl_context: ssl.SSLContext | None = None, lifecycle_collector: LifecycleCollector | None = None, app_path: str | None = None, sync_app: SyncApp | None = None) -> None
Parameters
Name Type Description
config
app
mode Default:None
ssl_context Default:None
lifecycle_collector Default:None
app_path Default:None
sync_app Default:None
_await_initial_workers_ready 0
Wait for every initial worker hook and listener to succeed.
def _await_initial_workers_ready(self) -> None
_raise_startup_failure 2 Never
Abort initial startup with the stable operator-facing error.
def _raise_startup_failure(self, message: str, hint: str) -> Never
Parameters
Name Type Description
message
hint
Returns
Never
_wait_for_subinterpreter_generation_ready 1 str | None
Wait for a replacement generation without changing global state. Initial start…
def _wait_for_subinterpreter_generation_ready(self, iic_queues: list[tuple[Any, Any]]) -> str | None

Wait for a replacement generation without changing global state.

Initial startup failures abort the server through _raise_startup_failure(). A reload is different: the old generation is still healthy, so a replacement failure must be reported to the caller without setting the supervisor shutdown event.

Parameters
Name Type Description
iic_queues
Returns
str | None
_signal_workers_start_draining 0
Mark async workers as draining (503 new connections) during shutdown. Thread-m…
def _signal_workers_start_draining(self) -> None

Mark async workers as draining (503 new connections) during shutdown.

Thread-mode workers expose a Worker / SyncWorker instance; process workers do not (handle.worker is None). Subinterpreter workers receive drain commands via IIC queue.

_restart_workers_impl 0
Internal implementation of restart_workers (no lock).
def _restart_workers_impl(self) -> None
_graceful_reload_impl 0
Internal implementation of graceful_reload (no lock).
def _graceful_reload_impl(self) -> None
_uses_sync_infrastructure 0 bool
True when AsyncPool/AcceptDistributor apply (thread mode, sync exec).
def _uses_sync_infrastructure(self) -> bool
Returns
bool
_build_async_pool 0 tuple[AsyncPool | None, …
Build and start a fresh AsyncPool bound to the current ``self._app``. Returns …
def _build_async_pool(self) -> tuple[AsyncPool | None, threading.Thread | None]

Build and start a fresh AsyncPool bound to the currentself._app.

Returns the pool and its thread handle without mutatingselfso the caller can repointself._async_pooland retire the old one in the order it chooses (issue #102).

Returns
tuple[AsyncPool | None, threading.Thread | None]
_build_accept_distributor 0 tuple[queue.Queue[tuple[…
Build and start a fresh AcceptDistributor for shared-socket sync mode. Returns…
def _build_accept_distributor(self) -> tuple[queue.Queue[tuple[socket.socket, object]] | None, threading.Thread | None, threading.Event | None]

Build and start a fresh AcceptDistributor for shared-socket sync mode.

Returns(conn_queue, handle, drain_event) (all Nonewhen a distributor does not apply) without mutatingself.

Returns
tuple[queue.Queue[tuple[socket.socket, object]] | None, threading.Thread | None, threading.Event | None]
_setup_sync_infrastructure 0
Create AsyncPool and AcceptDistributor for sync worker mode.
def _setup_sync_infrastructure(self) -> None
_create_worker 2 Worker | SyncWorker
Create a Worker or SyncWorker based on the execution mode.
def _create_worker(self, worker_id: int, socket_index: int) -> Worker | SyncWorker
Parameters
Name Type Description
worker_id
socket_index
Returns
Worker | SyncWorker
_worker_socket 1 socket.socket
Return a worker-owned socket handle. Thread workers always receive a duplicate…
def _worker_socket(self, socket_index: int) -> socket.socket

Return a worker-owned socket handle.

Thread workers always receive a duplicated handle so one generation can close its asyncio server without invalidating the canonical listener kept inself._socketsfor the next reload generation.

This applies both when workers share one listener object (macOS / Windows fallback) and when each worker has its ownSO_REUSEPORT socket (Linux): without a dup, an old worker closing its asyncio server would close the supervisor-owned socket and leave reload withEBADF.

Parameters
Name Type Description
socket_index
Returns
socket.socket
_spawn_worker 1
Create and start a single worker.
def _spawn_worker(self, worker_id: int) -> None
Parameters
Name Type Description
worker_id
_spawn_subinterpreter_worker 1
Create and start a worker inside a subinterpreter (PEP 734). Each worker runs …
def _spawn_subinterpreter_worker(self, worker_id: int) -> None

Create and start a worker inside a subinterpreter (PEP 734).

Each worker runs in a dedicated thread that hosts a subinterpreter. Communication uses IIC queues (tagged tuples) instead of direct method calls or threading.Events.

The supervisor passes config as JSON, the app as an import path, and the socket as a dup'd file descriptor — all IIC-safe types.

Parameters
Name Type Description
worker_id
_spawn_h3_worker 1
Create and start a single H3 (HTTP/3) worker.
def _spawn_h3_worker(self, worker_id: int) -> None
Parameters
Name Type Description
worker_id
_respawn_worker 1
Restart a crashed worker if within restart budget. Serialized with restart_wor…
def _respawn_worker(self, worker_id: int) -> None

Restart a crashed worker if within restart budget.

Serialized with restart_workers/graceful_reload via _lifecycle_lock. Skips if a reload is in progress (avoids overlapping restarts).

Parameters
Name Type Description
worker_id
_watch 0
Health-check loop — detects crashed workers and restarts them.
def _watch(self) -> None
_drain 0
Wait for all workers to finish draining connections, then clean up. Signals sh…
def _drain(self) -> None

Wait for all workers to finish draining connections, then clean up.

Signals shutdown to all workers, waits for them to finish processing active connections (seeshutdown_timeout on ServerConfig), then force-terminates process workers that haven't stopped.

Workers will reject new connections but finish existing ones for clean shutdown (important for Kubernetes graceful termination).

_reclaim_subinterpreter_fd 1
Close a subinterpreter worker's dup'd listener FD, if still recorded. Subinter…
def _reclaim_subinterpreter_fd(self, handle: _WorkerHandle) -> None

Close a subinterpreter worker's dup'd listener FD, if still recorded.

Subinterpreter workers run as daemon threads the parent cannot force kill (see _force_stop()). When such a worker stops abnormally — a crash/respawn, or a force-stop of an old generation that never drained — its bootstrap finally-block may not have closed the dup'd listener FD. Each abnormal cycle would then leak one FD until the process exhausts its descriptor budget (issue #106). The parent recorded the FD on sock_fd, so it can reclaim it here.

The generated bootstrap wrapper acknowledges its worker-side close with a parent-owned Event before the host thread exits. In that case this method only clears the record: calling os.close on the old numeric value could close an unrelated resource if the OS already reused it. Without an acknowledgement, the parent still owns abnormal cleanup.

Parameters
Name Type Description
handle
_force_stop 2
Force-terminate a worker that did not drain in time. Process workers receive S…
def _force_stop(self, handle: _WorkerHandle, join_timeout: float) -> None

Force-terminate a worker that did not drain in time.

Process workers receive SIGTERM then SIGKILL. Thread workers cannot be terminated from Python; they are daemon threads and may outlive this join.

Parameters
Name Type Description
handle
join_timeout
_install_signals 0
Install SIGINT/SIGTERM handlers to trigger graceful shutdown. Only effective w…
def _install_signals(self) -> None

Install SIGINT/SIGTERM handlers to trigger graceful shutdown.

Only effective when the supervisor runs on the main thread (e.g., direct testing). In production the supervisor runs inside a run_in_executor thread, so signal.signal()will fail silently. Server installs asyncio signal handlers that call supervisor.shutdown()instead.

Functions

_get_fork_context 0 multiprocessing.context.…
Return a ``"fork"`` multiprocessing context. Process workers must use ``fork``…
def _get_fork_context() -> multiprocessing.context.BaseContext

Return a"fork"multiprocessing context.

Process workers must useforkso the ASGI app (which may contain closures from middleware wrappers or framework decorators) is inherited via the forked address space rather than pickled. The default start method on macOS and Windows isspawn, which pickles the target — any non-picklable object (closures, local functions, lambdas) in the app will crash the server at startup.

Returns
multiprocessing.context.BaseContext
_parallel_join_targets 2 None
Join each worker thread/process in parallel with its own timeout. ``shutdown_t…
def _parallel_join_targets(targets: list[threading.Thread | multiprocessing.Process], timeout_per: float) -> None

Join each worker thread/process in parallel with its own timeout.

shutdown_timeoutis applied per worker (not split across N workers). Wall-clock time is roughlytimeout_perwhen all workers finish together, instead of one shared deadline that starved later workers in the join order.

Parameters
Name Type Description
targets list[threading.Thread | multiprocessing.Process]

Threads or processes tojoin.

timeout_per float

Maximum seconds to wait for each target.

_serialize_lifespan_state 1 str
Serialize IIC-safe lifespan state keys to JSON. Non-serializable values (DB po…
def _serialize_lifespan_state(state: dict[str, Any]) -> str

Serialize IIC-safe lifespan state keys to JSON.

Non-serializable values (DB pools, HTTP clients, etc.) are skipped with a debug log. The resulting JSON string is passed to each subinterpreter worker viaprepare_main().

Parameters
Name Type Description
state dict[str, Any]
Returns
str
_try_iic_get 1 tuple[Any, ...] | None
Non-blocking get from an IIC queue. Returns None if empty or unbound.
def _try_iic_get(queue: Any) -> tuple[Any, ...] | None
Parameters
Name Type Description
queue Any
Returns
tuple[Any, ...] | None
_target_id 1 str
Return an identifier string for a thread or process.
def _target_id(target: threading.Thread | multiprocessing.Process) -> str
Parameters
Name Type Description
target threading.Thread | multiprocessing.Process
Returns
str