Updated Nov 20, 20267 min read

How to Move Heroku Worker Dynos to Google Cloud Run

A step-by-step guide to moving Heroku worker dynos to Google Cloud: Cloud Run worker pools, Pub/Sub push to a Cloud Run service and Cloud Run jobs, how shutdown and scaling differ, and a Go version where the queue and the worker are declared in the service code.

Illustration for How to Move Heroku Worker Dynos to Google Cloud Run

A Heroku worker dyno is a Procfile process, such as worker: bundle exec sidekiq, that pulls jobs from Heroku Key-Value Store without receiving traffic. You scale it with heroku ps:scale worker=3, and Heroku restarts each dyno about daily with a 30-second SIGTERM window.

Google Cloud has three places for that process, and they treat shutdown and scaling differently. This guide picks one per worker and ends with the worker declared in Go.

How do worker dynos map to Google Cloud?

Heroku worker dynoCloud Run worker poolPub/Sub push to a Cloud Run serviceCloud Run job
ShapeLong-running process, no routingLong-running container, no URL or portHTTP handler per messageContainer that runs to completion
QueueKey-Value Store with Sidekiq, RQ or CeleryMemorystore, or a Pub/Sub pull subscriptionPub/Sub pushes each messageWhatever the job reads at start
Scalingheroku ps:scaleManual, or an autoscaler on an external metricAutomatic, on request loadTask count per execution
ShutdownSIGTERM, 30 seconds to SIGKILLSIGTERM, 10 seconds to SIGKILLRequest finishes or times outTask timeout, up to 168 hours
FitsEvery job typeA worker kept as-isJobs that finish within 10 minutesBatches and backfills

Heroku's dyno behavior is from its dyno docs, checked on November 20, 2026. Worker pools are described in the Cloud Run resource model, the 10-second window in the container runtime contract, and the push acknowledgment limits in the push subscription docs.

Step 1: Inventory the worker

Read the Procfile and heroku ps output for every process type other than web, then list each job class with its queue, its slowest run and how many run at once per dyno. Check the Heroku Scheduler entries for the app too, since teams often enqueue jobs from there, and note which jobs are unsafe to run twice, because every option on GCP redelivers a message whose processing was interrupted.

Step 2: Choose a home for each kind of job

A worker you want to keep unchanged, with the same Sidekiq or Celery code, runs in a Cloud Run worker pool with gcloud run worker-pools deploy, reading from Memorystore for Redis in the same VPC. Worker pools have no automatic scaling, so you set the instance count, or use an autoscaler that reads an external metric such as a Pub/Sub subscription's backlog. Jobs that finish within a few minutes fit better as Pub/Sub push subscriptions to a Cloud Run service, which scales with the delivery rate and bills per request, and where the acknowledgment deadline (up to 600 seconds per the subscription properties) caps each job. Batches, backfills and anything that runs for an hour become Cloud Run jobs.

Step 3: Plan for the shorter shutdown

Cloud Run gives a container 10 seconds between SIGTERM and SIGKILL, a third of Heroku's 30, so long jobs on a worker pool need checkpoints or a design where an interrupted job starts over safely. With push subscriptions the problem shrinks to a single request, since a handler that does not return 2xx in time leads Pub/Sub to resend the message with backoff.

Step 4: Declare the queue and the worker in code

Encore brings infrastructure from code, and a worker becomes a Pub/Sub subscription whose handler is the job, in the same Go service as the endpoint that enqueues it. Per the GCP infrastructure docs, Encore creates the topic, the subscription and a dead-letter topic in your project and runs the service on Cloud Run, where subscriptions on unordered topics are delivered as push requests, so the worker scales with the service instead of needing its own pool.

Application code passes through Encore and infrastructure guardrails to resources provisioned in your own AWS or GCP account.
Get started
brew install encoredev/tap/encore &&
encore app create
Copy

Run in your terminal to get started locally.

A CSV import that a Sidekiq worker used to run, with the file in a bucket and the rows in Postgres:

// imports/imports.go package imports import ( "context" "encoding/csv" "errors" "io" "time" "encore.dev/beta/errs" "encore.dev/pubsub" "encore.dev/storage/objects" "encore.dev/storage/sqldb" ) var db = sqldb.NewDatabase("imports", sqldb.DatabaseConfig{ Migrations: "./migrations", }) // Customers upload CSV files here with a signed upload URL. var Uploads = objects.NewBucket("csv-uploads", objects.BucketConfig{}) type ImportRequested struct { ImportID int64 Key string } var ImportTopic = pubsub.NewTopic[*ImportRequested]("import-requested", pubsub.TopicConfig{ DeliveryGuarantee: pubsub.AtLeastOnce, }) type StartParams struct { Key string } type StartResponse struct { ImportID int64 } // StartImport is what the web dyno did before calling ImportJob.perform_async. // //encore:api public method=POST path=/imports func StartImport(ctx context.Context, p *StartParams) (*StartResponse, error) { if ok, err := Uploads.Exists(ctx, p.Key); err != nil { return nil, err } else if !ok { return nil, &errs.Error{Code: errs.InvalidArgument, Message: "upload not found"} } var id int64 if err := db.QueryRow(ctx, `INSERT INTO imports (object_key) VALUES ($1) RETURNING id`, p.Key).Scan(&id); err != nil { return nil, err } if _, err := ImportTopic.Publish(ctx, &ImportRequested{ImportID: id, Key: p.Key}); err != nil { return nil, err } return &StartResponse{ImportID: id}, nil } // The worker: one message per import, retried with backoff on error. var _ = pubsub.NewSubscription(ImportTopic, "run-import", pubsub.SubscriptionConfig[*ImportRequested]{ Handler: runImport, AckDeadline: 5 * time.Minute, RetryPolicy: &pubsub.RetryPolicy{MinBackoff: 10 * time.Second, MaxRetries: 5}, }) func runImport(ctx context.Context, msg *ImportRequested) error { r := Uploads.Download(ctx, msg.Key) defer r.Close() tx, err := db.Begin(ctx) if err != nil { return err } defer tx.Rollback() rows := csv.NewReader(r) for { rec, err := rows.Read() if errors.Is(err, io.EOF) { break } else if err != nil { return err } if len(rec) < 2 { continue } // The (import_id, email) key makes a redelivered message a no-op. if _, err := tx.Exec(ctx, ` INSERT INTO contacts (import_id, email, name) VALUES ($1, $2, $3) ON CONFLICT (import_id, email) DO NOTHING`, msg.ImportID, rec[0], rec[1]); err != nil { return err } } if _, err := tx.Exec(ctx, `UPDATE imports SET finished_at = now() WHERE id = $1`, msg.ImportID); err != nil { return err } return tx.Commit() }
-- imports/migrations/1_init.up.sql CREATE TABLE imports ( id BIGSERIAL PRIMARY KEY, object_key TEXT NOT NULL, created_at TIMESTAMPTZ NOT NULL DEFAULT now(), finished_at TIMESTAMPTZ ); CREATE TABLE contacts ( import_id BIGINT NOT NULL REFERENCES imports(id), email TEXT NOT NULL, name TEXT NOT NULL, PRIMARY KEY (import_id, email) );

The whole import runs in one transaction, so a message interrupted by a shutdown leaves no partial rows and the redelivery starts clean, and the conflict clause covers the case where the same message arrives twice after a commit. The Pub/Sub docs explain AckDeadline, the retry policy and the dead-letter topic that collects messages after MaxRetries, and the object storage docs cover signed upload URLs for the CSV. With encore run, Postgres starts locally, the bucket is a local directory and the subscription receives messages as soon as StartImport publishes them.

To deploy, connect your GCP project as described in Connect your cloud account and push with git push encore.

An agent adding a new job type writes a topic, a subscription and the handler in one file and runs them with encore run, and the deploy creates the Pub/Sub resources and the push endpoint, so the change never involves the GCP console or a new worker pool.

Best fit: Backends moving from Heroku to their own GCP project that want background jobs, retries and dead letters declared next to the code that enqueues them.

Step 5: Drain the dynos

Switch the web side to publish to the new topic first, and let the Heroku worker work through what is left in Key-Value Store. When the queue has stayed empty for a day, run heroku ps:scale worker=0, and remove the process type from the Procfile once the new subscription has handled a full day of traffic.

Frequently asked questions

What is the Cloud Run equivalent of a Heroku worker dyno?

A Cloud Run worker pool is the closest match, since it runs a container with no URL and no port, the way a worker dyno runs a process with no routing. Worker pools do not scale on their own, so you set the instance count or attach an autoscaler driven by an external metric such as Pub/Sub backlog.

How long does Cloud Run wait after SIGTERM?

Cloud Run sends SIGTERM and then SIGKILL 10 seconds later, compared with 30 seconds on Heroku. A job that cannot finish in 10 seconds has to be safe to interrupt, so that its message is redelivered and processed again.

Can I keep Sidekiq, RQ or Celery on Google Cloud?

Yes. Point the worker at Memorystore for Redis in the same VPC and run it in a worker pool, and the job code stays the same. Moving to Pub/Sub removes the Redis dependency and lets Cloud Run services receive jobs as push requests instead of polling.

How do I run a background worker on Cloud Run without wiring Pub/Sub and IAM by hand?

With Encore you declare a Pub/Sub topic and a subscription in application code, and the subscription handler is the worker. Deploying to GCP creates the topic, the subscription and its dead-letter topic in your project and delivers messages to the service on Cloud Run.

Deploy to your cloud on AWS and Google Cloud
without managing infrastructure

Encore lets developers and agents define application infrastructure directly in code, then automatically provisions it from local development to production in your AWS or GCP account.

~/orders — claude
Claude Code
Claude Code v2.1.180Opus 5.5~/orders