Classes
TCPWorker
4
▼
Structural contract for TCP workers (Worker and SyncWorker).
Both async and sync workers implement…
TCPWorker
4
▼
Protocol
Structural contract for TCP workers (Worker and SyncWorker).
Both async and sync workers implement this interface. The supervisor uses it for lifecycle management: spawning, draining, and idle checks.
Methods
run
0
▼
run
0
▼
def run(self) -> None
set_lifespan_state
1
▼
set_lifespan_state
1
▼
def set_lifespan_state(self, state: dict[str, Any]) -> None
Parameters
| Name | Type | Description |
|---|---|---|
state |
— |
start_draining
0
▼
start_draining
0
▼
def start_draining(self) -> None
is_idle
0
bool
▼
is_idle
0
bool
▼
def is_idle(self) -> bool
Returns
bool
_WorkerHandle
1
▼
Metadata about a running worker (thread or process).
_WorkerHandle
1
▼
Metadata about a running worker (thread or process).
Methods
Internal Methods 1 ▼
__init__
4
▼
__init__
4
▼
def __init__(self, worker_id: int, target: threading.Thread | multiprocessing.Process, worker: TCPWorker | None, generation: int = 0) -> None
Parameters
| Name | Type | Description |
|---|---|---|
worker_id |
— |
|
target |
— |
|
worker |
— |
|
generation |
— |
Default:0
|
_H3WorkerHandle
1
▼
Metadata about a running H3 worker (UDP/QUIC).
_H3WorkerHandle
1
▼
Metadata about a running H3 worker (UDP/QUIC).
Methods
Internal Methods 1 ▼
__init__
4
▼
__init__
4
▼
def __init__(self, worker_id: int, target: threading.Thread | multiprocessing.Process, worker: H3Worker | None = None, reload_shutdown_event: threading.Event | None = None) -> None
Parameters
| Name | Type | Description |
|---|---|---|
worker_id |
— |
|
target |
— |
|
worker |
— |
Default:None
|
reload_shutdown_event |
— |
Default:None
|
Supervisor
29
▼
Spawn and supervise N workers as threads or processes.
The supervisor detects the GIL state and pi…
Supervisor
29
▼
Spawn and supervise N workers as threads or processes.
The supervisor detects the GIL state and picks the appropriate
spawning strategy automatically. Workers share the frozen
ServerConfig and (in thread mode) the ASGI app reference.
Methods
The active worker mode (``"thread"``, ``"process"``, or ``"subinterpreter"``).
property
def mode(self) -> WorkerMode
The active worker mode ("thread", "process", or "subinterpreter").
Returns
WorkerMode
worker_count
0
int
▼
Number of workers the supervisor manages.
property
worker_count
0
int
▼
def worker_count(self) -> int
Returns
int
set_lifespan_state
1
▼
Set the lifespan state dict to be shared with all workers.
set_lifespan_state
1
▼
def set_lifespan_state(self, state: dict[str, Any]) -> None
Parameters
| Name | Type | Description |
|---|---|---|
state |
— |
The state dict populated during lifespan startup. |
run
2
▼
Start all workers and block until shutdown.
Installs signal handlers, spawns w…
run
2
▼
def run(self, sockets: list[socket.socket], udp_sockets: list[socket.socket] | None = None) -> None
Start all workers and block until shutdown.
Installs signal handlers, spawns workers, runs the health-check loop, then joins all workers on shutdown.
Parameters
| Name | Type | Description |
|---|---|---|
sockets |
— |
One TCP socket per worker, created by |
udp_sockets |
— |
Optional UDP sockets for HTTP/3 workers. Default:None
|
shutdown
0
▼
Signal all workers to stop (non-blocking).
shutdown
0
▼
def shutdown(self) -> None
restart_workers
0
▼
Gracefully restart all workers (for dev reload).
Signals all running workers t…
restart_workers
0
▼
def restart_workers(self) -> None
Gracefully restart all workers (for dev reload).
Signals all running workers to stop, waits for them to drain, clears the shutdown event, and spawns fresh workers.
Serialized with graceful_reload and watch-loop respawns via _lifecycle_lock. Skips if a reload is already in progress.
When anapp_pathwas provided and workers run as threads,
the app module is reimported so that code changes on disk take
effect. Process-based workers inherit the parent via fork and
are fully replaced on restart, so reimport is not needed.
graceful_reload
0
▼
Perform zero-downtime rolling restart of all workers.
This method implements a…
graceful_reload
0
▼
def graceful_reload(self) -> None
Perform zero-downtime rolling restart of all workers.
This method implements a rolling restart strategy:
- Reimport the app (thread mode only)
- Spawn new worker generation
- Mark old workers for draining (finish existing, reject new connections)
- Wait for old workers to become idle
- Shut down old workers
This ensures zero dropped requests during code reload.
Note: Only works in thread mode. In process mode, falls back to restart_workers() which has brief downtime.
Internal Methods 22 ▼
__init__
7
▼
__init__
7
▼
def __init__(self, config: ServerConfig, app: ASGIApp, *, mode: WorkerMode | None = None, ssl_context: ssl.SSLContext | None = None, lifecycle_collector: LifecycleCollector | None = None, app_path: str | None = None, sync_app: SyncApp | None = None) -> None
Parameters
| Name | Type | Description |
|---|---|---|
config |
— |
|
app |
— |
|
mode |
— |
Default:None
|
ssl_context |
— |
Default:None
|
lifecycle_collector |
— |
Default:None
|
app_path |
— |
Default:None
|
sync_app |
— |
Default:None
|
_await_initial_workers_ready
0
▼
Wait for every initial worker hook and listener to succeed.
_await_initial_workers_ready
0
▼
def _await_initial_workers_ready(self) -> None
_raise_startup_failure
2
Never
▼
Abort initial startup with the stable operator-facing error.
_raise_startup_failure
2
Never
▼
def _raise_startup_failure(self, message: str, hint: str) -> Never
Parameters
| Name | Type | Description |
|---|---|---|
message |
— |
|
hint |
— |
Returns
Never
_wait_for_subinterpreter_generation_ready
1
str | None
▼
Wait for a replacement generation without changing global state.
Initial start…
_wait_for_subinterpreter_generation_ready
1
str | None
▼
def _wait_for_subinterpreter_generation_ready(self, iic_queues: list[tuple[Any, Any]]) -> str | None
Wait for a replacement generation without changing global state.
Initial startup failures abort the server through
_raise_startup_failure(). A reload is different: the old
generation is still healthy, so a replacement failure must be
reported to the caller without setting the supervisor shutdown event.
Parameters
| Name | Type | Description |
|---|---|---|
iic_queues |
— |
Returns
str | None
_signal_workers_start_draining
0
▼
Mark async workers as draining (503 new connections) during shutdown.
Thread-m…
_signal_workers_start_draining
0
▼
def _signal_workers_start_draining(self) -> None
Mark async workers as draining (503 new connections) during shutdown.
Thread-mode workers expose a Worker / SyncWorker instance; process
workers do not (handle.worker is None). Subinterpreter workers receive
drain commands via IIC queue.
_restart_workers_impl
0
▼
Internal implementation of restart_workers (no lock).
_restart_workers_impl
0
▼
def _restart_workers_impl(self) -> None
_graceful_reload_impl
0
▼
Internal implementation of graceful_reload (no lock).
_graceful_reload_impl
0
▼
def _graceful_reload_impl(self) -> None
_uses_sync_infrastructure
0
bool
▼
True when AsyncPool/AcceptDistributor apply (thread mode, sync exec).
_uses_sync_infrastructure
0
bool
▼
def _uses_sync_infrastructure(self) -> bool
Returns
bool
Build and start a fresh AsyncPool bound to the current ``self._app``.
Returns …
def _build_async_pool(self) -> tuple[AsyncPool | None, threading.Thread | None]
Build and start a fresh AsyncPool bound to the currentself._app.
Returns the pool and its thread handle without mutatingselfso the
caller can repointself._async_pooland retire the old one in the
order it chooses (issue #102).
Returns
tuple[AsyncPool | None, threading.Thread | None]
_build_accept_distributor
0
tuple[queue.Queue[tuple[…
▼
Build and start a fresh AcceptDistributor for shared-socket sync mode.
Returns…
_build_accept_distributor
0
tuple[queue.Queue[tuple[…
▼
def _build_accept_distributor(self) -> tuple[queue.Queue[tuple[socket.socket, object]] | None, threading.Thread | None, threading.Event | None]
Build and start a fresh AcceptDistributor for shared-socket sync mode.
Returns(conn_queue, handle, drain_event) (all Nonewhen a
distributor does not apply) without mutatingself.
Returns
tuple[queue.Queue[tuple[socket.socket, object]] | None, threading.Thread | None, threading.Event | None]
_setup_sync_infrastructure
0
▼
Create AsyncPool and AcceptDistributor for sync worker mode.
_setup_sync_infrastructure
0
▼
def _setup_sync_infrastructure(self) -> None
Create a Worker or SyncWorker based on the execution mode.
def _create_worker(self, worker_id: int, socket_index: int) -> Worker | SyncWorker
Parameters
| Name | Type | Description |
|---|---|---|
worker_id |
— |
|
socket_index |
— |
Returns
Worker | SyncWorker
_worker_socket
1
socket.socket
▼
Return a worker-owned socket handle.
Thread workers always receive a duplicate…
_worker_socket
1
socket.socket
▼
def _worker_socket(self, socket_index: int) -> socket.socket
Return a worker-owned socket handle.
Thread workers always receive a duplicated handle so one generation
can close its asyncio server without invalidating the canonical
listener kept inself._socketsfor the next reload generation.
This applies both when workers share one listener object (macOS /
Windows fallback) and when each worker has its ownSO_REUSEPORT
socket (Linux): without a dup, an old worker closing its asyncio
server would close the supervisor-owned socket and leave reload
withEBADF.
Parameters
| Name | Type | Description |
|---|---|---|
socket_index |
— |
Returns
socket.socket
_spawn_worker
1
▼
Create and start a single worker.
_spawn_worker
1
▼
def _spawn_worker(self, worker_id: int) -> None
Parameters
| Name | Type | Description |
|---|---|---|
worker_id |
— |
_spawn_subinterpreter_worker
1
▼
Create and start a worker inside a subinterpreter (PEP 734).
Each worker runs …
_spawn_subinterpreter_worker
1
▼
def _spawn_subinterpreter_worker(self, worker_id: int) -> None
Create and start a worker inside a subinterpreter (PEP 734).
Each worker runs in a dedicated thread that hosts a subinterpreter. Communication uses IIC queues (tagged tuples) instead of direct method calls or threading.Events.
The supervisor passes config as JSON, the app as an import path, and the socket as a dup'd file descriptor — all IIC-safe types.
Parameters
| Name | Type | Description |
|---|---|---|
worker_id |
— |
_spawn_h3_worker
1
▼
Create and start a single H3 (HTTP/3) worker.
_spawn_h3_worker
1
▼
def _spawn_h3_worker(self, worker_id: int) -> None
Parameters
| Name | Type | Description |
|---|---|---|
worker_id |
— |
_respawn_worker
1
▼
Restart a crashed worker if within restart budget.
Serialized with restart_wor…
_respawn_worker
1
▼
def _respawn_worker(self, worker_id: int) -> None
Restart a crashed worker if within restart budget.
Serialized with restart_workers/graceful_reload via _lifecycle_lock. Skips if a reload is in progress (avoids overlapping restarts).
Parameters
| Name | Type | Description |
|---|---|---|
worker_id |
— |
_watch
0
▼
Health-check loop — detects crashed workers and restarts them.
_watch
0
▼
def _watch(self) -> None
_drain
0
▼
Wait for all workers to finish draining connections, then clean up.
Signals sh…
_drain
0
▼
def _drain(self) -> None
Wait for all workers to finish draining connections, then clean up.
Signals shutdown to all workers, waits for them to finish processing
active connections (seeshutdown_timeout on ServerConfig), then
force-terminates process workers that haven't stopped.
Workers will reject new connections but finish existing ones for clean shutdown (important for Kubernetes graceful termination).
_reclaim_subinterpreter_fd
1
▼
Close a subinterpreter worker's dup'd listener FD, if still recorded.
Subinter…
_reclaim_subinterpreter_fd
1
▼
def _reclaim_subinterpreter_fd(self, handle: _WorkerHandle) -> None
Close a subinterpreter worker's dup'd listener FD, if still recorded.
Subinterpreter workers run as daemon threads the parent cannot force
kill (see _force_stop()). When such a worker stops abnormally —
a crash/respawn, or a force-stop of an old generation that never drained
— its bootstrap finally-block may not have closed the dup'd listener FD.
Each abnormal cycle would then leak one FD until the process exhausts
its descriptor budget (issue #106). The parent recorded the FD on
sock_fd, so it can reclaim it here.
The generated bootstrap wrapper acknowledges its worker-side close with
a parent-owned Event before the host thread exits. In that case this
method only clears the record: calling os.close on the old numeric
value could close an unrelated resource if the OS already reused it.
Without an acknowledgement, the parent still owns abnormal cleanup.
Parameters
| Name | Type | Description |
|---|---|---|
handle |
— |
_force_stop
2
▼
Force-terminate a worker that did not drain in time.
Process workers receive S…
_force_stop
2
▼
def _force_stop(self, handle: _WorkerHandle, join_timeout: float) -> None
Force-terminate a worker that did not drain in time.
Process workers receive SIGTERM then SIGKILL. Thread workers cannot be terminated from Python; they are daemon threads and may outlive this join.
Parameters
| Name | Type | Description |
|---|---|---|
handle |
— |
|
join_timeout |
— |
_install_signals
0
▼
Install SIGINT/SIGTERM handlers to trigger graceful shutdown.
Only effective w…
_install_signals
0
▼
def _install_signals(self) -> None
Install SIGINT/SIGTERM handlers to trigger graceful shutdown.
Only effective when the supervisor runs on the main thread (e.g.,
direct testing). In production the supervisor runs inside a
run_in_executor thread, so signal.signal()will fail
silently. Server installs asyncio signal handlers that call
supervisor.shutdown()instead.
Functions
_get_fork_context
0
multiprocessing.context.…
▼
Return a ``"fork"`` multiprocessing context.
Process workers must use ``fork``…
_get_fork_context
0
multiprocessing.context.…
▼
def _get_fork_context() -> multiprocessing.context.BaseContext
Return a"fork"multiprocessing context.
Process workers must useforkso the ASGI app (which may contain
closures from middleware wrappers or framework decorators) is inherited
via the forked address space rather than pickled. The default start
method on macOS and Windows isspawn, which pickles the target —
any non-picklable object (closures, local functions, lambdas) in the
app will crash the server at startup.
Returns
multiprocessing.context.BaseContext
_parallel_join_targets
2
None
▼
Join each worker thread/process in parallel with its own timeout.
``shutdown_t…
_parallel_join_targets
2
None
▼
def _parallel_join_targets(targets: list[threading.Thread | multiprocessing.Process], timeout_per: float) -> None
Join each worker thread/process in parallel with its own timeout.
shutdown_timeoutis applied per worker (not split across N workers).
Wall-clock time is roughlytimeout_perwhen all workers finish together,
instead of one shared deadline that starved later workers in the join order.
Parameters
| Name | Type | Description |
|---|---|---|
targets |
list[threading.Thread | multiprocessing.Process] |
Threads or processes to |
timeout_per |
float |
Maximum seconds to wait for each target. |
_serialize_lifespan_state
1
str
▼
Serialize IIC-safe lifespan state keys to JSON.
Non-serializable values (DB po…
_serialize_lifespan_state
1
str
▼
def _serialize_lifespan_state(state: dict[str, Any]) -> str
Serialize IIC-safe lifespan state keys to JSON.
Non-serializable values (DB pools, HTTP clients, etc.) are skipped
with a debug log. The resulting JSON string is passed to each
subinterpreter worker viaprepare_main().
Parameters
| Name | Type | Description |
|---|---|---|
state |
dict[str, Any] |
Returns
str
_try_iic_get
1
tuple[Any, ...] | None
▼
Non-blocking get from an IIC queue. Returns None if empty or unbound.
_try_iic_get
1
tuple[Any, ...] | None
▼
def _try_iic_get(queue: Any) -> tuple[Any, ...] | None
Parameters
| Name | Type | Description |
|---|---|---|
queue |
Any |
Returns
tuple[Any, ...] | None
_target_id
1
str
▼
Return an identifier string for a thread or process.
_target_id
1
str
▼
def _target_id(target: threading.Thread | multiprocessing.Process) -> str
Parameters
| Name | Type | Description |
|---|---|---|
target |
threading.Thread | multiprocessing.Process |
Returns
str