Nov 20, 20265 min read

Run Delayed and Scheduled Jobs in a Backend Without a Job Server

Send a reminder in 24 hours, expire a trial in 14 days, retry a payment tomorrow. Why queues do not delay messages for days, how a Postgres table plus a cron job handles future work, and a TypeScript example with Encore.

Illustration for Run Delayed and Scheduled Jobs in a Backend Without a Job Server

Most background work should run now, but some of it should run later: the reminder 24 hours before an appointment, the trial that ends in 14 days or the payment retry tomorrow morning. Queues are built for immediate delivery, and delays of days are outside what they offer.

A database table and a cron job cover it without a job server. This guide shows how, with duplicates and cancellation, in a TypeScript service.

Why can you not delay a queue message?

Queues cap delays: SQS delay queues and message timers delay delivery by at most 15 minutes, and Google Cloud Pub/Sub has no per-message delay at all. Holding a message back by extending its visibility timeout is not a scheduler either. For anything longer than minutes, the schedule has to live somewhere durable and queryable.

How does a run_at table work?

Store each future job as a row with a run_at timestamp, a type, a payload and a state. A job that runs every minute selects rows whose run_at has passed and that nobody has claimed, marks them and processes them. The schedule is plain data, so you can list, reschedule and cancel jobs with ordinary queries, and nothing is lost on restart.

Claim rows atomically, so two dispatchers never take the same one. A partial index on unclaimed rows keeps the dispatcher query fast as finished jobs pile up:

UPDATE jobs SET claimed_at = now() WHERE id IN ( SELECT id FROM jobs WHERE claimed_at IS NULL AND run_at <= now() ORDER BY run_at LIMIT 100 FOR UPDATE SKIP LOCKED ) RETURNING id, kind, payload;

SKIP LOCKED, described in the Postgres SELECT documentation, makes concurrent dispatchers pick different rows instead of waiting. The pattern is the same one behind Postgres job queues.

What about duplicates, failures and cancellation?

Dispatch is at least once: a dispatcher can crash after publishing and before the claim commits, or a worker can fail after doing the work. Make handlers idempotent, keyed on the job ID, and record completion in the job row. Cancellation is a DELETE or a state change before run_at, which is a single statement, and a handler should check the job is still wanted when it runs. The dispatcher interval sets the precision, so a one-minute interval means jobs run up to a minute late, which suits reminders and expiries but not sub-second timers.

How do you split dispatch from execution?

Let the dispatcher only claim rows and publish messages, and run the actual work in queue subscribers. That keeps the cron job fast, gives each job type retries and dead-lettering through the queue, and lets execution scale separately from scheduling.

Delayed jobs with Encore

Encore

Encore is a backend framework with built-in infrastructure, and the table, the dispatcher and the queue are declared in one TypeScript service. The cron job is managed by the Encore platform and needs no scheduler of your own in your AWS or GCP account, and the Pub/Sub topic carries due jobs to workers.

Set up the app

Get started
brew install encoredev/tap/encore &&
encore app create
Copy

Run in your terminal to get started locally.

-- scheduler/migrations/1_create_jobs.up.sql CREATE TABLE jobs ( id BIGSERIAL PRIMARY KEY, kind TEXT NOT NULL, payload JSONB NOT NULL DEFAULT '{}', run_at TIMESTAMPTZ NOT NULL, claimed_at TIMESTAMPTZ ); CREATE INDEX jobs_due ON jobs (run_at) WHERE claimed_at IS NULL;
// scheduler/scheduler.ts import { api } from "encore.dev/api"; import { SQLDatabase } from "encore.dev/storage/sqldb"; import { Topic, Subscription } from "encore.dev/pubsub"; import { CronJob } from "encore.dev/cron"; import log from "encore.dev/log"; const db = new SQLDatabase("scheduler", { migrations: "./migrations" }); interface JobDue { id: number; kind: string; payload: unknown; } const jobDue = new Topic<JobDue>("job-due", { deliveryGuarantee: "at-least-once" }); export const schedule = api( { method: "POST", path: "/jobs", expose: false }, async ({ kind, payload, runAt }: { kind: string; payload: object; runAt: string }): Promise<{ id: number }> => { const row = await db.queryRow<{ id: number }>` INSERT INTO jobs (kind, payload, run_at) VALUES (${kind}, ${JSON.stringify(payload)}::jsonb, ${runAt}::timestamptz) RETURNING id`; return { id: row!.id }; }, ); export const cancel = api( { method: "DELETE", path: "/jobs/:id", expose: false }, async ({ id }: { id: number }): Promise<{ cancelled: boolean }> => { const row = await db.queryRow<{ id: number }>` DELETE FROM jobs WHERE id = ${id} AND claimed_at IS NULL RETURNING id`; return { cancelled: row !== null }; }, ); export const dispatch = api({ expose: false }, async (): Promise<{ dispatched: number }> => { const due: JobDue[] = []; for await (const r of db.query<JobDue>` UPDATE jobs SET claimed_at = now() WHERE id IN ( SELECT id FROM jobs WHERE claimed_at IS NULL AND run_at <= now() ORDER BY run_at LIMIT 100 FOR UPDATE SKIP LOCKED) RETURNING id, kind, payload`) { due.push(r); } for (const job of due) await jobDue.publish(job); return { dispatched: due.length }; }); const _dispatcher = new CronJob("dispatch-due-jobs", { title: "Dispatch due jobs", every: "1m", endpoint: dispatch, }); export const _run = new Subscription(jobDue, "run-job", { handler: async (job) => { // Idempotent: key side effects on job.id. log.info("running job", { id: job.id, kind: job.kind }); }, maxConcurrency: 10, retryPolicy: { minBackoff: "30s", maxRetries: 6 }, });

Encore cron jobs run in cloud environments and not under encore run or in preview environments, so call dispatch from the local development dashboard to test due jobs locally. If the process dies after claiming rows and before publishing them, those jobs stay claimed and do not run, so in production add a published_at column and have the dispatcher also pick up rows claimed long ago and never published. Each run of the dispatcher and each job appears in a trace.

A coding agent can add a new job kind with its handler and the code that schedules it in a single change, then trigger the dispatcher locally and watch the subscriber run.

Best fit: TypeScript teams that need reminders, expiries and retries days ahead and want the schedule in their own Postgres database with the cron and queue provisioned in their own cloud account.

Frequently asked questions

How do you schedule a job to run in the future in a backend?

Insert a row with a run_at timestamp, and run a short-interval job that selects rows whose run_at has passed, marks them claimed and processes them. This handles delays of minutes, days or months and survives restarts, since the schedule lives in the database.

Can SQS or Pub/Sub delay messages?

Yes. SQS delays messages by at most 15 minutes, and Pub/Sub has no per-message delay. For longer delays, store the schedule in a database or use a scheduler service, and publish to the queue when the job is due.

How do you prevent a delayed job from running twice?

Claim due rows with an UPDATE that sets a claimed timestamp and returns the rows, using FOR UPDATE SKIP LOCKED so concurrent dispatchers take different rows. Make the job handler idempotent as well, since a crash after the work but before the mark can cause a retry.

How do I run delayed jobs with Encore?

Store jobs in an Encore Postgres table with a run_at column and add a cron job that calls a dispatcher endpoint every minute. The dispatcher claims due rows and publishes them to Pub/Sub for processing. Cron jobs run in cloud environments, not under encore run, so call the dispatcher endpoint by hand when testing locally.

Deploy to your cloud on AWS and Google Cloud
without managing infrastructure

Encore lets developers and agents define application infrastructure directly in code, then automatically provisions it from local development to production in your AWS or GCP account.

~/orders — claude
Claude Code
Claude Code v2.1.180Opus 5.5~/orders