A priority queue lets urgent jobs jump ahead of bulk work. Why SQS and Pub/Sub have no built-in priorities, how separate queues or topics with weighted consumers solve it, how to avoid starvation, and a Go example on Encore with two topics.

A queue treats every message the same, which is fine until a customer waits for a password reset email behind two hundred thousand newsletter sends, because both sit in the same line.
Priorities fix that, but the managed queues in common use do not offer them. This guide covers the pattern that works on SQS and Pub/Sub, how to keep low-priority work moving, and a Go service with an urgent and a bulk lane.
Neither SQS nor Google Cloud Pub/Sub exposes a priority field. They optimize for throughput and availability, and priorities would complicate sharding and delivery. RabbitMQ supports a per-queue priority range, but a managed cloud broker leaves the design to you.
The Azure Architecture Center describes two ways to get priorities: a queue that supports them natively, or one queue per priority with consumers that serve the higher queues first. The second works on any broker.
Publish urgent work to one topic and bulk work to another, and give each its own subscription and workers. A worker pool that reads only the urgent queue is never delayed by bulk work, since the bulk backlog sits somewhere else.
Choose the queue at publish time from a rule in code, such as the user waiting, the job type or the customer tier. Keep the rule in one function so the policy lives in one place and is easy to change.
A strict scheme where workers always drain the urgent queue first can starve bulk work indefinitely under constant urgent load. Dedicated consumers per lane avoid that: the bulk lane always has some capacity, however busy the urgent lane is.
Give the urgent lane high concurrency so bursts clear fast, and the bulk lane a lower, capped concurrency, which also protects shared dependencies such as the database from a backfill that would otherwise use every connection.
Yes. A Postgres job table can order by priority with ORDER BY priority DESC, created_at and claim rows with FOR UPDATE SKIP LOCKED, as covered in Postgres job queue with SKIP LOCKED. It is a good fit at modest volume and gives real priority ordering, at the cost of building retries and dead-lettering yourself.
Encore is a backend framework with built-in infrastructure, so each lane is a Pub/Sub topic declared in code with its own subscription settings. On AWS the topics become SNS with SQS and on GCP they become Pub/Sub, each with a dead-letter queue.
Run in your terminal to get started locally.
// mail/mail.go
package mail
import (
"context"
"time"
"encore.dev/pubsub"
"encore.dev/rlog"
)
type Email struct {
To string `json:"to"`
Subject string `json:"subject"`
}
var Urgent = pubsub.NewTopic[*Email]("email-urgent", pubsub.TopicConfig{
DeliveryGuarantee: pubsub.AtLeastOnce,
})
var Bulk = pubsub.NewTopic[*Email]("email-bulk", pubsub.TopicConfig{
DeliveryGuarantee: pubsub.AtLeastOnce,
})
func send(ctx context.Context, e *Email) error {
rlog.Info("sending email", "to", e.To, "subject", e.Subject)
return nil // call the email provider here, keyed so a retry does not send twice
}
// Fast lane: high concurrency, short retries.
var _ = pubsub.NewSubscription(Urgent, "send-urgent", pubsub.SubscriptionConfig[*Email]{
Handler: func(ctx context.Context, e *Email) error { return send(ctx, e) },
MaxConcurrency: 50,
RetryPolicy: &pubsub.RetryPolicy{MinBackoff: time.Second, MaxBackoff: time.Minute, MaxRetries: 5},
})
// Bulk lane: capped concurrency so a campaign cannot crowd out the database.
var _ = pubsub.NewSubscription(Bulk, "send-bulk", pubsub.SubscriptionConfig[*Email]{
Handler: func(ctx context.Context, e *Email) error { return send(ctx, e) },
MaxConcurrency: 5,
RetryPolicy: &pubsub.RetryPolicy{MinBackoff: 10 * time.Second, MaxBackoff: 10 * time.Minute, MaxRetries: 10},
})
// Enqueue picks the lane from one rule.
func Enqueue(ctx context.Context, e *Email, userWaiting bool) error {
topic := Bulk
if userWaiting {
topic = Urgent
}
_, err := topic.Publish(ctx, e)
return err
}
Call Enqueue from an API endpoint once the request is validated. The two subscriptions scale independently, since every instance of the service consumes from both and MaxConcurrency applies per instance. A backlog in the bulk topic never delays the urgent one, and the traces show how long each lane takes.
A coding agent can add a third lane or change a concurrency cap in the same change as the producer code, and test both lanes locally with encore run.
Best fit: Go teams that run user-facing and bulk jobs on the same infrastructure and want separate lanes with their own concurrency and retries in their own AWS or GCP account.
No. SQS standard and FIFO queues and Google Cloud Pub/Sub deliver messages without a priority field, so a burst of low-priority messages delays everything behind it. The usual solution is a separate queue or topic per priority, each with its own consumers.
Give every priority its own consumers instead of letting high-priority work always drain first. Dedicated capacity for the low-priority queue guarantees it makes progress, and you can give the high-priority queue more concurrency so it still finishes faster.
Two or three. A fast lane for work a user is waiting on and a bulk lane for backfills and reports covers most systems, and more levels make capacity planning harder without improving latency.
Declare one Pub/Sub topic per priority and a subscription on each, then set a higher maxConcurrency on the urgent subscription and a lower one on the bulk subscription. Encore provisions the topics and queues in your AWS or GCP account and runs them locally under encore run.
Deploy to your cloud on AWS and Google Cloud
without managing infrastructure
Encore lets developers and agents define application infrastructure directly in code, then automatically provisions it from local development to production in your AWS or GCP account.