# Workers

> Background processes of a service

A worker is a long-running process of a service that serves no requests. It runs next to the service's main process, from an entrypoint of its own in the service's code, and does whatever that code does: consume Pub/Sub messages, run a Temporal worker, poll a queue, reconcile state on a timer.

A worker *is* its service at runtime. It has the service's secrets, configs, databases and permissions, and it calls other services as the service does. There is nothing to grant or share: code a worker uses is the service's code.

## Enabling workers

Workers are experimental: they work, but how they are declared may still change. An app uses them only by enabling the `workers` feature in its `encore.app.json`:

```json
-- encore.app.json --
{
  "features": ["workers"]
}
```

Without it, Encore refuses to parse an app that declares workers, with an error saying what to add.

## Declaring workers

Workers are declared in the service's `encore.service.json`, by name:

```json
-- orders/encore.service.json --
{
  "name": "orders",
  "lang": "python",
  "api": [
    { "type": "http", "prefix": "/", "name": "orders", "expose": true }
  ],
  "workers": {
    "fulfillment": { "entry": "fulfillment_worker.py" },
    "temporal": { "entry": "-m orders.temporal_worker", "shutdown_timeout": "10m" },
    "reconciler": { "entry": "reconciler.py", "instances": "one" }
  }
}
```

| Field | Required | Description |
|---|---|---|
| `entry` | Yes, unless `run` is given | The worker's entrypoint: a file relative to the service directory, or `-m module` to run a module. |
| `description` | No | A sentence or two about what the worker is for, shown in Encore's tools. |
| `instances` | No | `"any"` (the default), or `"one"` for code that must never run twice at once, such as a leader loop. A deployment then runs a single copy of it, and stops the old one before starting a new one. This is best effort: take a lock where correctness depends on it. |
| `shutdown_timeout` | No | How long the worker may take to stop once told to, as `"30s"` or `"10m"`. Defaults to the deployment's graceful shutdown. |
| `run` | No | How `encore run` starts it, as a service's `run`, instead of from `entry`. |
| `docker` | No | `{ "entrypoint": [...] }`: the command its process runs in a Docker image, instead of one derived from `entry`. |

A worker's name is lowercase letters, digits and underscores, unique within its service. Within the app, Encore calls it `<service>.<worker>`, as in `orders.fulfillment`.

## Writing a worker

A worker's entrypoint is an ordinary Python program that starts Encore and runs until it is told to stop. One that only consumes Pub/Sub messages needs nothing more than `encore.run()` without an app:

```python
-- orders/fulfillment_worker.py --
import encore

import subscriptions  # declares the subscriptions this worker consumes

encore.run()
```

One with work of its own runs it until shutdown begins, then lets it finish. See [Graceful shutdown](/docs/py/develop/shutdown) for how:

```python
-- orders/reconciler.py --
import asyncio
import signal

import encore
from encore import shutdown


async def reconcile(stop: asyncio.Event) -> None:
    while not stop.is_set():
        ...  # compare and fix state, using Encore resources
        try:
            await asyncio.wait_for(stop.wait(), timeout=60)
        except asyncio.TimeoutError:
            pass


async def main() -> int:
    await encore.start()
    stop = asyncio.Event()
    asyncio.get_running_loop().add_signal_handler(signal.SIGTERM, stop.set)
    task = asyncio.create_task(reconcile(stop))

    await stop.wait()
    shutdown.begin(cooperative=True)
    await task
    shutdown.mark_application_drained()
    return (await shutdown.completed()).code()


if __name__ == "__main__":
    raise SystemExit(asyncio.run(main()))
```

`encore.env.app_meta().deploy.worker` is the name of the worker a process runs, and `None` in the service's main process.

## Consuming Pub/Sub messages

A subscription names the worker that consumes it with `worker`:

```python
-- orders/subscriptions.py --
from encore import pubsub

from topics import orders, Order


async def fulfil(message: pubsub.Message[Order]) -> None:
    ...


fulfil_orders = pubsub.Subscription[Order](
    orders, "fulfil-orders", handler=fulfil, worker="fulfillment"
)
```

Only that worker's process takes the subscription's messages. The service's main process, and its other workers, take none, even when their code imports the module that declares it. Without `worker`, the service's main process consumes the subscription, as it always has.

Encore reads `worker` from the source, so write it as a string, and it must name a worker the service declares.

## A service with only workers

A service that serves nothing has no main process: set its `run` to `"none"`.

```json
-- indexer/encore.service.json --
{
  "name": "indexer",
  "lang": "python",
  "run": "none",
  "workers": {
    "index": { "entry": "index_worker.py" }
  }
}
```

Such a service can't declare an `api`, a frontend or a gateway, or cron jobs, which call a path of their service. Each subscription it declares needs a `worker`, since no main process would consume it.

## Cron jobs

A cron job calls an HTTP path of its service, which only the main process serves, so it can't target a worker. Declare it in the service, and have its endpoint publish a message a worker consumes if the work belongs in the worker.

## Running workers locally

Under `encore run`, a service's workers start when the service does, and keep running while anything keeps the service up. Unlike the main process, a worker is never stopped for being idle, since no request would start it again. A message waiting on a subscription a worker consumes starts the worker too.

Each worker shows up in `encore run` and the dashboard as `<service>.<worker>`. Its log records are its service's, with an `encore.worker.name` attribute naming the worker. It counts as running once it has stayed up for 250 milliseconds; one that exits sooner failed to start, and its exit and last lines are shown. A worker that crashes stays down until the next rebuild, as a service does.

When the service's source changes, its workers are rebuilt with it: each is stopped, then started again, so a worker never runs twice.

## Deploying workers

`encore build docker` packages a service's workers in its image, and the image runs them beside the service. A deployment can also run them on their own, in containers that run only workers. Its infra config's `hosted_workers` lists the workers a container runs, by `<service>.<worker>`, and `encore infra config generate` lists each worker where its service runs.

A worker that runs with its service scales with it. On compute that throttles CPU between requests, as Cloud Run does by default, a worker sharing a container with its service needs CPU allocated all the time.
