Nov 18, 20269 min read

Cloud Scheduler vs Cloud Tasks: Recurring Jobs vs One-Off Work

When to use Cloud Scheduler for work on a fixed schedule and when to use Cloud Tasks for individual, delayed or rate-limited jobs, and how to combine them.

Illustration for Cloud Scheduler vs Cloud Tasks: Recurring Jobs vs One-Off Work

Cloud Scheduler and Cloud Tasks both make Google Cloud call your code later, and they answer different questions. Cloud Scheduler is managed cron: the same job, on a fixed schedule, for as long as the job exists. Cloud Tasks is a queue of individual tasks: each one has its own payload, can carry a scheduled time, and is dispatched under a rate limit with retries until it succeeds.

The distinction matters because the common mistake runs both ways. A Scheduler job that loops over 50,000 users inside one HTTP request hits the attempt deadline and retries the whole batch. A Cloud Task created for "every night" has to re-enqueue itself and drifts. Most production systems end up using both: a schedule that finds the work, and a queue that does each piece.

Cloud Scheduler vs Cloud Tasks at a glance

Cloud SchedulerCloud Tasks
Unit of workA job that runs on a cron scheduleA task with its own payload
TimingUnix cron with a time zone, at most once a minuteImmediately, or at a scheduleTime up to 30 days ahead
TargetsHTTP, Pub/Sub, App EngineHTTP, App Engine
Rate controlOne run per tickQueue max dispatches per second and max concurrent dispatches
FailureLogged, retried only if you configure itRetried until it succeeds, within the queue's attempt and age limits
IdentitySame job each runEach task has a unique name, usable for deduplication

When is Cloud Scheduler the better fit?

Cloud Scheduler fits anything that is defined by the calendar: nightly invoice runs, an hourly cache warm-up, a weekly report. The schedule is a unix-cron expression with a time zone, so "02:00 in Vienna" stays 02:00 across daylight saving changes, and a target can be an HTTP endpoint with an OIDC token or a Pub/Sub topic.

What it does not do is split work: each run is one request to one target, bounded by the job's attempt deadline, and Google's comparison of the two services notes that a failed run is not repeated before the next tick unless you set retries. A job that processes thousands of items should find them and hand each one to a queue.

When is Cloud Tasks the better fit?

Cloud Tasks fits work that exists per item: a reminder for one user in three days, a webhook delivery to one customer, a thumbnail for one upload. Each task can be scheduled up to 30 days ahead, the queue throttles dispatches so a burst of 100,000 tasks does not overwhelm the target or a third-party API, and failures retry with backoff per task.

The limits show up at the edges: there is no recurring schedule, so a nightly job built on Cloud Tasks has to enqueue its next run itself, delays past 30 days need a database and something that runs periodically to enqueue tasks later, and every task is an HTTP call your service has to authenticate and make idempotent, since a task can be delivered more than once.

How do you schedule work with Encore?

Encore

Encore is a backend framework with built-in infrastructure, and it splits scheduled work the same way. Recurring work is a cron job declared next to the endpoint it calls; cron jobs are Encore Platform managed and need no extra cloud infrastructure. Per-item work with retries is a Pub/Sub subscription, provisioned as Pub/Sub on GCP or SNS and SQS on AWS in your own account. There is no per-message delay, so one-off reminders are rows with a due_at column that the cron job sweeps.

Application code passes through Encore and infrastructure guardrails to resources provisioned in your own AWS or GCP account.

Set up the app

Get started
brew install encoredev/tap/encore &&
encore app create
Copy

Run in your terminal to get started locally.

~
$ encore app create reminders --example=hello-world$ cd reminders && mkdir -p reminders/migrations

The schema

Reminders are rows with a due time. The partial index keeps the sweep cheap no matter how many sent reminders pile up:

CREATE TABLE reminders ( id BIGSERIAL PRIMARY KEY, user_id TEXT NOT NULL, kind TEXT NOT NULL, due_at TIMESTAMPTZ NOT NULL, queued_at TIMESTAMPTZ ); -- The sweep only ever looks at reminders that are due and not yet queued. CREATE INDEX reminders_due_idx ON reminders (due_at) WHERE queued_at IS NULL;

Schedule, sweep and send

Schedule stores a reminder for any time, a week or a year out. QueueDue is the recurring part: every five minutes it claims due rows and publishes one message each. The subscription is the per-item part, with its own backoff and retry limit:

// Service reminders sends reminders at a time chosen when they are created. package reminders import ( "context" "time" "encore.dev/cron" "encore.dev/pubsub" "encore.dev/rlog" "encore.dev/storage/sqldb" ) var db = sqldb.NewDatabase("reminders", sqldb.DatabaseConfig{Migrations: "./migrations"}) type ReminderDue struct { ID int64 `json:"id"` UserID string `json:"user_id"` Kind string `json:"kind"` } var dueTopic = pubsub.NewTopic[*ReminderDue]("reminder-due", pubsub.TopicConfig{ DeliveryGuarantee: pubsub.AtLeastOnce, }) type ScheduleParams struct { UserID string `json:"user_id"` Kind string `json:"kind"` DueAt time.Time `json:"due_at"` } // Schedule stores a one-off reminder, for example "trial ends in 3 days". // //encore:api private method=POST path=/reminders func Schedule(ctx context.Context, p *ScheduleParams) error { _, err := db.Exec(ctx, `INSERT INTO reminders (user_id, kind, due_at) VALUES ($1, $2, $3)`, p.UserID, p.Kind, p.DueAt) return err } // The recurring part: every 5 minutes, queue everything that has come due. var _ = cron.NewJob("queue-due-reminders", cron.JobConfig{ Title: "Queue reminders that are due", Every: 5 * cron.Minute, Endpoint: QueueDue, }) //encore:api private func QueueDue(ctx context.Context) error { tx, err := db.Begin(ctx) if err != nil { return err } defer tx.Rollback() // SKIP LOCKED lets a second, overlapping run take other rows instead of waiting. rows, err := tx.Query(ctx, ` UPDATE reminders SET queued_at = now() WHERE id IN ( SELECT id FROM reminders WHERE queued_at IS NULL AND due_at <= now() ORDER BY due_at LIMIT 500 FOR UPDATE SKIP LOCKED) RETURNING id, user_id, kind`) if err != nil { return err } var due []*ReminderDue for rows.Next() { r := &ReminderDue{} if err := rows.Scan(&r.ID, &r.UserID, &r.Kind); err != nil { rows.Close() return err } due = append(due, r) } rows.Close() if err := rows.Err(); err != nil { return err } // Publish before committing: if a publish fails, the rows stay unqueued // and the next run picks them up. A crash after publishing but before the // commit can send a reminder twice, so the handler must tolerate repeats. for _, r := range due { if _, err := dueTopic.Publish(ctx, r); err != nil { return err } } return tx.Commit() } // The per-item part: each reminder is its own message with its own retries. var _ = pubsub.NewSubscription(dueTopic, "send-reminder", pubsub.SubscriptionConfig[*ReminderDue]{ Handler: send, RetryPolicy: &pubsub.RetryPolicy{ MinBackoff: 30 * time.Second, MaxBackoff: 30 * time.Minute, MaxRetries: 10, }, }) func send(ctx context.Context, r *ReminderDue) error { rlog.Info("sending reminder", "id", r.ID, "user", r.UserID, "kind", r.Kind) return nil // call the email provider here, with r.ID as its idempotency key }

The precision is the cron interval, so a reminder due at 10:02 goes out by about 10:05. The cron docs note that a cron endpoint may be called more than once, which is why the sweep claims rows with FOR UPDATE SKIP LOCKED, and cron jobs do not run under encore run or in Preview Environments, so you call QueueDue from the local development dashboard to test it.

encore check compiled this app. Our build machine has no Docker, which encore run needs for the local Postgres, so we ran the sweep's SQL against Postgres 17 instead: two due reminders and one future one were inserted, the first sweep claimed the two due rows, a concurrent second sweep claimed none, and after the first sweep rolled back (a failed publish) a third sweep claimed both again.

With the schedule, the queue and the table in one file, a coding agent that adds a "card expiring" reminder adds a kind and a handler branch, and the change is reviewable without opening the Cloud Scheduler or Cloud Tasks console.

Best fit: Backends that need both recurring jobs and per-item retries, on AWS or GCP, and want the schedule and the queue declared in code.

Cloud Scheduler and Cloud Tasks by hand

Create the recurring job and the queue with gcloud. Both call your Cloud Run service with an OIDC token from a service account that has roles/run.invoker:

~
# Recurring: Cloud Scheduler calls the endpoint at 02:00 Vienna time every day.gcloud scheduler jobs create http nightly-invoices --location=us-central1 --schedule="0 2 * * *" --time-zone="Europe/Vienna" --uri="https://billing-abc123-uc.a.run.app/invoices/run" --http-method=POST --oidc-service-account-email=scheduler-invoker@my-project.iam.gserviceaccount.com --attempt-deadline=10m --max-retry-attempts=3 # Per item: a Cloud Tasks queue with a rate limit and retry policy.gcloud tasks queues create reminders --location=us-central1 --max-dispatches-per-second=20 --max-concurrent-dispatches=10 --max-attempts=10 --min-backoff=30s --max-backoff=1800s

The service enqueues one task per reminder with the Go client. A deterministic task name turns a retried CreateTask call into an AlreadyExists error instead of a second reminder:

// tasks.go: schedule a one-off reminder as a Cloud Task. package reminders import ( "context" "encoding/json" "fmt" "time" cloudtasks "cloud.google.com/go/cloudtasks/apiv2" "cloud.google.com/go/cloudtasks/apiv2/cloudtaskspb" "google.golang.org/grpc/codes" "google.golang.org/grpc/status" "google.golang.org/protobuf/types/known/timestamppb" ) const queue = "projects/my-project/locations/us-central1/queues/reminders" type Reminder struct { ID int64 `json:"id"` UserID string `json:"user_id"` Kind string `json:"kind"` } func ScheduleReminder(ctx context.Context, c *cloudtasks.Client, r Reminder, at time.Time) error { body, err := json.Marshal(r) if err != nil { return err } _, err = c.CreateTask(ctx, &cloudtaskspb.CreateTaskRequest{ Parent: queue, Task: &cloudtaskspb.Task{ // A named task is rejected if the name was used recently, // which deduplicates retries of this call. Name: fmt.Sprintf("%s/tasks/reminder-%d", queue, r.ID), ScheduleTime: timestamppb.New(at), // up to 30 days ahead MessageType: &cloudtaskspb.Task_HttpRequest{ HttpRequest: &cloudtaskspb.HttpRequest{ HttpMethod: cloudtaskspb.HttpMethod_POST, Url: "https://reminders-abc123-uc.a.run.app/send", Headers: map[string]string{"Content-Type": "application/json"}, Body: body, AuthorizationHeader: &cloudtaskspb.HttpRequest_OidcToken{ OidcToken: &cloudtaskspb.OidcToken{ ServiceAccountEmail: "tasks-invoker@my-project.iam.gserviceaccount.com", }, }, }, }, }, }) if status.Code(err) == codes.AlreadyExists { // The name is reminder-{id}, so this is the same reminder already queued. return nil } return err }

Reminders more than 30 days out still need a table and a Scheduler job that enqueues tasks once they come within range, which is the same sweep as the Encore version with Cloud Tasks in place of Pub/Sub. The job, the queue, both service accounts and their IAM bindings are per environment, and keeping them in step is yours, usually in Terraform.

Best fit: Services already on Cloud Run that need per-task rate limits toward a third-party API, or exact scheduled times within 30 days.

How to choose

Use Cloud Scheduler when the trigger is the calendar and Cloud Tasks when the trigger is an item, and combine them when a schedule finds many items. Reach for Cloud Tasks specifically when you need an exact dispatch time within 30 days or a hard dispatch rate toward a downstream API. Pick Encore when you want the cron job, the queue and the database behind them declared in Go or TypeScript and deployed to your own AWS or GCP account.

Frequently asked questions

What is the difference between Cloud Scheduler and Cloud Tasks?

Cloud Scheduler runs the same job on a fixed cron schedule, at most once a minute, and calls an HTTP endpoint, a Pub/Sub topic or App Engine. Cloud Tasks queues individual tasks, each with its own payload and optional scheduled time, and dispatches them to an HTTP target with a queue-level rate limit and retry policy. Use Scheduler for 'every night at 2:00' and Tasks for 'send this one email in three days'.

Can Cloud Scheduler run a job every 30 seconds?

No. Google's comparison of the two services states that once a minute is the most fine-grained interval Cloud Scheduler supports. For sub-minute work, run a worker that loops, or enqueue Cloud Tasks with scheduled times.

How far ahead can a Cloud Task be scheduled?

Up to 30 days. For longer delays, store the due time in your database and have a recurring job enqueue the task once it is within range.

How do you schedule jobs with Encore?

Recurring work is a cron job declared in code with cron.NewJob, which the Encore Platform runs in deployed environments without extra cloud infrastructure. Per-item work with retries is a Pub/Sub subscription with a retry policy, and one-off delays are a due_at column that the cron job sweeps.

Deploy to your cloud on AWS and Google Cloud
without managing infrastructure

Encore lets developers and agents define application infrastructure directly in code, then automatically provisions it from local development to production in your AWS or GCP account.

~/orders — claude
Claude Code
Claude Code v2.1.180Opus 5.5~/orders