Graceful shutdown

When a service is stopped, for example during a deploy or when encore run restarts it, it receives SIGTERM. Encore then shuts down gracefully: it stops taking new Pub/Sub messages, waits for message handlers in progress, closes database, Redis and object storage connections, and flushes metrics and traces.

Most apps need no code for this. Read on if your app runs its own background work that uses Encore resources, or if you want to control when Encore shuts down.

Default behavior

Importing encore sets up shutdown for you:

• It registers an exit hook that shuts Encore down when the interpreter exits.
• It installs a SIGTERM handler that exits the interpreter, with exit code 0 if shutdown completed within its deadline and 1 otherwise. It does this only when the process has no SIGTERM handler yet at import time, and not under encore test, where the test runner handles the signal.

This fits servers like Uvicorn, which handle SIGTERM themselves: Uvicorn stops accepting connections and drains requests in progress, then restores the previous signal handler and raises the signal again. That hands over to Encore, which drains Pub/Sub and cleans up after your HTTP server has finished. A FastAPI app run with uvicorn.run(app, ...) shuts down gracefully without any shutdown code.

If your server handles SIGTERM without raising it again, Encore still shuts down from its exit hook as the interpreter exits.

How shutdown proceeds

Shutdown moves through these phases, which shutdown.phase() returns as a ShutdownPhase:

Once cleanup has run, Encore's resources can't be used anymore, even through objects you still hold. New operations fail, interrupted object downloads raise an error rather than returning partial data, and Redis subscriptions end. Clients you create yourself from a resource's connection string are yours to close.

Shutting down from your own code

To shut Encore down at a point you choose, call shutdown.begin() and await shutdown.completed(). For example, in a FastAPI lifespan handler, the code after yield runs once Uvicorn has drained HTTP requests:

main.py
import logging
from contextlib import asynccontextmanager

import encore
from encore import shutdown
from fastapi import FastAPI

log = logging.getLogger(__name__)


@asynccontextmanager
async def lifespan(app: FastAPI):
    await encore.start()
    yield
    # HTTP requests have drained: shut Encore down and wait for it.
    shutdown.begin()
    result = await shutdown.completed()
    log.info("encore shut down", extra={"result": result.value})


app = FastAPI(lifespan=lifespan)

begin() returns immediately, and all callers share one shutdown: repeated calls don't restart it or reset its deadline. When the interpreter exits afterwards, Encore's own hooks see that shutdown has already completed.

Cooperative shutdown

Use cooperative shutdown when work your application started, such as background tasks, may still be using Encore resources when shutdown begins. Encore then stops Pub/Sub right away, but doesn't close resources until you call mark_application_drained() or the deadline expires.

For example, a worker process with its own background tasks:

worker.py
import asyncio
import signal

import encore
from encore import shutdown


async def process_jobs(stop: asyncio.Event) -> None:
    while not stop.is_set():
        ...  # take a job and process it, using Encore resources


async def main() -> int:
    await encore.start()

    stop = asyncio.Event()
    asyncio.get_running_loop().add_signal_handler(signal.SIGTERM, stop.set)
    workers = [asyncio.create_task(process_jobs(stop)) for _ in range(4)]

    await stop.wait()
    shutdown.begin(cooperative=True)  # stop Pub/Sub, keep resources open

    await asyncio.gather(*workers)    # let the jobs in progress finish
    shutdown.mark_application_drained()

    result = await shutdown.completed()
    return result.code()


if __name__ == "__main__":
    raise SystemExit(asyncio.run(main()))

Keep these rules in mind:

• Call begin() before awaiting started(), cleanup() or completed(). Awaiting them before begin() raises ShutdownError.
• The first call to begin() chooses the mode. Later calls don't change it.
• In cooperative mode, call mark_application_drained() before awaiting cleanup() or completed(). Otherwise Encore waits until the deadline expires.
• Keep the event loop running until completed() returns. Cancelling a wait doesn't cancel the shutdown itself.

await shutdown.started() and await shutdown.cleanup() wait until shutdown has reached or passed those phases, and return immediately if it already has. You don't need to await every phase in turn.

Deadline and forced cleanup

Shutdown has a deadline, five seconds by default, which Encore sets for each deployment. It starts when shutdown begins and covers both draining and cleanup. When it expires:

1. The result becomes TIMEOUT, and stays that way even if cleanup later succeeds.
2. If Encore is still waiting for draining, it stops waiting and starts cleanup, even without mark_application_drained().
3. Cleanup continues, and completed() returns once it has finished.

The deadline doesn't stop the process, and completed() can return after it. Your host or server decides when to terminate the process.

To start cleanup before the deadline without waiting for draining, call force_cleanup():

shutdown.force_cleanup()            # requires begin() first
await shutdown.cleanup()            # cleanup has started
result = await shutdown.completed() # cleanup has finished

force_cleanup() returns immediately and can be called repeatedly. Forcing cleanup doesn't make the result TIMEOUT.

Neither the deadline nor force_cleanup() stops handlers that are still running. They may continue and fail as the resources they use close, so drain your own work before mark_application_drained() where you can.

Results and errors

completed() returns a ShutdownResult:

If cleanup fails, or shutdown's progress can't be observed, completed() raises ShutdownError. code() gives you a process exit code.