Workers

A worker is a long-running process of a service that serves no requests. It runs next to the service's main process, from a binary of its own in the service's Cargo package, and does whatever that binary does: consume Pub/Sub messages, run a Temporal worker, poll a queue, reconcile state on a timer.

A worker is its service at runtime. It has the service's secrets, configs, databases and permissions, and it calls other services as the service does. There is nothing to grant or share: code a worker uses is the service's code.

Enabling workers

Workers are experimental: they work, but how they are declared may still change. An app uses them only by enabling the workers feature in its encore.app.json:

encore.app.json
{
  "features": ["workers"]
}

Without it, Encore refuses to parse an app that declares workers, with an error saying what to add.

Declaring workers

Workers are declared in the service's encore.service.json, by name. A Rust worker's entry is a [[bin]] of the service's package:

orders/encore.service.json
{
  "name": "orders",
  "lang": "rust",
  "api": [
    { "type": "http", "prefix": "/", "name": "orders", "expose": true }
  ],
  "workers": {
    "fulfillment": { "entry": "fulfillment-worker" },
    "reconciler": { "entry": "reconciler", "instances": "one", "shutdown_timeout": "2m" }
  }
}

orders/Cargo.toml
[package]
name = "orders"
# ...

[[bin]]
name = "orders"
path = "src/main.rs"

[[bin]]
name = "fulfillment-worker"
path = "src/bin/fulfillment_worker.rs"

[[bin]]
name = "reconciler"
path = "src/bin/reconciler.rs"

The service's own binary is the one left that no worker names.

A worker's name is lowercase letters, digits and underscores, unique within its service. Within the app, Encore calls it <service>.<worker>, as in orders.fulfillment.

Writing a worker

A worker's binary starts Encore and runs until it is told to stop. One that only consumes Pub/Sub messages calls encore::run_blocking(), which starts the registered subscriptions, blocks until shutdown, drains and exits:

orders/src/bin/fulfillment_worker.rs
use orders::subscriptions as _; // registers the subscriptions this worker consumes

fn main() {
    encore::run_blocking();
}

One with work of its own runs it until shutdown begins, then lets it finish. See Graceful shutdown for how:

orders/src/bin/reconciler.rs
#[tokio::main]
async fn main() {
    encore::initialize();
    // At startup, before the signal can arrive.
    let stop = encore::shutdown::signal();
    tokio::pin!(stop);
    loop {
        tokio::select! {
            _ = &mut stop => break,
            _ = tokio::time::sleep(std::time::Duration::from_secs(60)) => {
                // compare and fix state, using Encore resources
            }
        }
    }
    std::process::exit(encore::shutdown::drained().await.code());
}

encore::env::app_meta().deploy.worker is the name of the worker a process runs, and None in the service's main process.

Consuming Pub/Sub messages

A subscription names the worker that consumes it with .worker(...):

orders/src/subscriptions.rs
use encore::pubsub::{Message, Subscription};

use crate::topics::{Order, ORDERS};

#[encore::pubsub::register]
static FULFIL_ORDERS: Subscription<Order> =
    Subscription::new(&ORDERS, "fulfil-orders", fulfil).worker("fulfillment");

async fn fulfil(msg: Message<Order>) -> Result<(), encore::Error> {
    Ok(())
}

Only that worker's process takes the subscription's messages. The service's main process, and its other workers, take none, even when they link the module that declares it. Without .worker(...), the service's main process consumes the subscription, as it always has.

Encore reads the worker's name from the source, so write it as a string, and it must name a worker the service declares.

A service with only workers

A service that serves nothing has no main process: set its run to "none".

indexer/encore.service.json
{
  "name": "indexer",
  "lang": "rust",
  "run": "none",
  "workers": {
    "index": { "entry": "index-worker" }
  }
}

Such a service can't declare an api, a frontend or a gateway, or cron jobs, which call a path of their service. Each subscription it declares needs a .worker(...), since no main process would consume it.

Cron jobs

A cron job calls an HTTP path of its service, which only the main process serves, so it can't target a worker. Declare it in the service, and have its endpoint publish a message a worker consumes if the work belongs in the worker.

Running workers locally

Under encore run, a service's workers start when the service does, and keep running while anything keeps the service up. Unlike the main process, a worker is never stopped for being idle, since no request would start it again. A message waiting on a subscription a worker consumes starts the worker too.

Each worker is built with cargo build --bin <entry> and shows up in encore run and the dashboard as <service>.<worker>. Its log records are its service's, with an encore.worker.name attribute naming the worker. It counts as running once it has stayed up for 250 milliseconds; one that exits sooner failed to start, and its exit and last lines are shown. A worker that crashes stays down until the next rebuild, as a service does.

When the service's source changes, its workers are rebuilt with it: each is stopped, then started again, so a worker never runs twice.

Deploying workers

encore build docker packages a service's workers in its image, each binary at bin/<worker> beside the service's, and the image runs them beside the service. A deployment can also run them on their own, in containers that run only workers. Its infra config's hosted_workers lists the workers a container runs, by <service>.<worker>, and encore infra config generate lists each worker where its service runs.

A worker that runs with its service scales with it. On compute that throttles CPU between requests, as Cloud Run does by default, a worker sharing a container with its service needs CPU allocated all the time.