Nov 13, 20268 min read

How to Add Pub/Sub to a chi App

Publish events from go-chi handlers and process them in idempotent subscribers, with Encore or SNS and SQS wired by hand.

Illustration for How to Add Pub/Sub to a chi App

A chi handler that places an order and then sends the confirmation email, notifies the warehouse and updates analytics before it responds gets slower and more fragile with every step it takes on. Moving that work behind a topic keeps the handler to one fast publish call. That code is short, and the remaining work is a broker in every environment, subscriptions with retry and dead-letter settings, IAM for publishing and consuming, and a local setup that behaves like the cloud one.

This guide adds Pub/Sub to a chi orders API: the handler publishes an OrderPlaced event and a subscriber sends the confirmation email, retried with backoff and safe to run twice. It starts with Encore, which declares the topic and subscription in Go and runs them locally and in your own AWS or GCP account, and then shows the same flow with SNS, SQS and aws-sdk-go-v2.

Pub/Sub options for a chi app at a glance

EncoreSNS + SQS by handGoroutine and channel
Topic declared inGo code (pubsub.NewTopic)Terraform or the consoleNot applicable
Local brokerIn-memory, started by encore runLocalStack or a real AWS accountThe process itself
Retries and DLQRetryPolicy on the subscriptionVisibility timeout and redrive policyYour own code
Survives a deploy or crashYesYesNo
Consumer codeA handler functionA polling loop with deletesA goroutine

How do you add Pub/Sub to chi with Encore?

Encore

Encore works alongside chi, so the orders router stays as it is and only gains a topic to publish to. You declare the topic and its subscriptions as Go values in the same package, mount the router in one raw endpoint with a fallback route, and Encore takes care of the broker locally and in the cloud.

Set up the app

Get started
brew install encoredev/tap/encore &&
encore app create
Copy

Run in your terminal to get started locally.

~
$ encore app create orders-api --example=hello-world$ cd orders-api && go get github.com/go-chi/chi/v5@v5.5.0$ mkdir orders

The service publishes from the chi handler and subscribes in the same file. The subscriber claims each order ID in Redis before sending, so a redelivered message finds the claim and returns without a second email, and a failed send releases the claim so the retry can go ahead:

// orders/orders.go package orders import ( "context" "crypto/rand" "encoding/hex" "encoding/json" "errors" "net/http" "time" "encore.dev/pubsub" "encore.dev/rlog" "encore.dev/storage/cache" "github.com/go-chi/chi/v5" "github.com/go-chi/chi/v5/middleware" ) type OrderPlaced struct { OrderID string `json:"order_id"` Email string `json:"email"` TotalCents int64 `json:"total_cents"` } // In-memory locally, SNS + SQS on AWS or Google Pub/Sub on GCP when deployed. var OrderEvents = pubsub.NewTopic[*OrderPlaced]("order-placed", pubsub.TopicConfig{ DeliveryGuarantee: pubsub.AtLeastOnce, }) //encore:service type Service struct { router chi.Router } func initService() (*Service, error) { r := chi.NewRouter() r.Use(middleware.Recoverer) r.Post("/orders", placeOrder) return &Service{router: r}, nil } // Fallback sends every request that no Encore endpoint matches to chi. // //encore:api public raw path=/!fallback func (s *Service) Fallback(w http.ResponseWriter, req *http.Request) { s.router.ServeHTTP(w, req) } func placeOrder(w http.ResponseWriter, r *http.Request) { var in struct { Email string `json:"email"` TotalCents int64 `json:"total_cents"` } if err := json.NewDecoder(r.Body).Decode(&in); err != nil || in.Email == "" || in.TotalCents <= 0 { writeJSON(w, http.StatusBadRequest, map[string]string{"error": "email and total_cents are required"}) return } b := make([]byte, 8) rand.Read(b) ev := &OrderPlaced{OrderID: "ord_" + hex.EncodeToString(b), Email: in.Email, TotalCents: in.TotalCents} // The handler returns as soon as the event is stored; email sending // and anything else downstream happens in subscribers. if _, err := OrderEvents.Publish(r.Context(), ev); err != nil { writeJSON(w, http.StatusInternalServerError, map[string]string{"error": "could not queue order"}) return } writeJSON(w, http.StatusAccepted, map[string]string{"order_id": ev.OrderID}) } func writeJSON(w http.ResponseWriter, status int, v any) { w.Header().Set("Content-Type", "application/json") w.WriteHeader(status) json.NewEncoder(w).Encode(v) } // Delivery is at least once, so the subscriber records which orders already // got their email and skips duplicates. var cluster = cache.NewCluster("orders", cache.ClusterConfig{ EvictionPolicy: cache.NoEviction, }) var confirmationSent = cache.NewStringKeyspace[string](cluster, cache.KeyspaceConfig{ KeyPattern: "confirmation-sent/:key", DefaultExpiry: cache.ExpireIn(7 * 24 * time.Hour), }) var _ = pubsub.NewSubscription(OrderEvents, "send-confirmation", pubsub.SubscriptionConfig[*OrderPlaced]{ Handler: sendConfirmation, RetryPolicy: &pubsub.RetryPolicy{ MinBackoff: 10 * time.Second, MaxBackoff: 10 * time.Minute, MaxRetries: 8, // then the message moves to the dead-letter queue }, }) func sendConfirmation(ctx context.Context, ev *OrderPlaced) error { // Claim the order first. If the key exists, an earlier delivery sent it. err := confirmationSent.SetIfNotExists(ctx, ev.OrderID, "sending") if errors.Is(err, cache.KeyExists) { return nil } else if err != nil { return err } if err := sendEmail(ctx, ev.Email, "Order "+ev.OrderID+" confirmed"); err != nil { // Release the claim so the retry can send it. confirmationSent.Delete(ctx, ev.OrderID) return err } return nil } func sendEmail(ctx context.Context, to, subject string) error { rlog.Info("sending email", "to", to, "subject", subject) return nil // call your email provider here }

Running this with encore run, a POST /orders returns 202 with the order ID right away and the subscriber logs the confirmation a moment later, and the local development dashboard shows the request and the message delivery as traces. A request without an email gets a 400 from the chi handler and publishes nothing. Topics and subscriptions run in memory locally, and the Pub/Sub docs describe the delivery guarantees and ordered topics.

The RetryPolicy controls how often a failing message is retried and when it moves to the dead-letter queue, which Encore sets up for every subscription. On deploy, Encore provisions SNS topics and SQS queues on AWS or Google Pub/Sub on GCP in your own account and gives the service the permissions to publish and consume. If you write the order to a database first, the transactional outbox guide shows how to publish only after the transaction commits.

An agent adding a second consumer, such as a warehouse notification, writes one more pubsub.NewSubscription call next to the first and sees it run locally, and the next deploy creates its queue and permissions.

Best fit: chi services that want to move slow or failure-prone work out of the request path, with retries, dead-lettering and a broker that runs the same locally and in your own AWS or GCP account.

SNS, SQS and aws-sdk-go-v2

On AWS the common setup is an SNS topic per event type and an SQS queue per consumer subscribed to it, with the app publishing through aws-sdk-go-v2 and a goroutine long-polling the queue:

package main import ( "context" "encoding/json" "log" "net/http" "os" "github.com/aws/aws-sdk-go-v2/aws" "github.com/aws/aws-sdk-go-v2/config" "github.com/aws/aws-sdk-go-v2/service/sns" "github.com/aws/aws-sdk-go-v2/service/sqs" "github.com/go-chi/chi/v5" ) type OrderPlaced struct { OrderID string `json:"order_id"` Email string `json:"email"` } func main() { cfg, err := config.LoadDefaultConfig(context.Background()) if err != nil { log.Fatal(err) } snsClient := sns.NewFromConfig(cfg) topicARN := os.Getenv("ORDER_TOPIC_ARN") r := chi.NewRouter() r.Post("/orders", func(w http.ResponseWriter, r *http.Request) { ev := OrderPlaced{OrderID: "ord_123", Email: "a@example.com"} body, _ := json.Marshal(ev) _, err := snsClient.Publish(r.Context(), &sns.PublishInput{ TopicArn: aws.String(topicARN), Message: aws.String(string(body)), }) if err != nil { http.Error(w, "could not queue order", http.StatusInternalServerError) return } w.WriteHeader(http.StatusAccepted) }) go consume(context.Background(), sqs.NewFromConfig(cfg), os.Getenv("CONFIRMATION_QUEUE_URL")) log.Fatal(http.ListenAndServe(":8080", r)) } // consume long-polls the queue subscribed to the topic. With raw message // delivery enabled on the subscription, the body is the event JSON. func consume(ctx context.Context, client *sqs.Client, queueURL string) { for { out, err := client.ReceiveMessage(ctx, &sqs.ReceiveMessageInput{ QueueUrl: aws.String(queueURL), MaxNumberOfMessages: 10, WaitTimeSeconds: 20, }) if err != nil { log.Print(err) continue } for _, m := range out.Messages { var ev OrderPlaced if err := json.Unmarshal([]byte(aws.ToString(m.Body)), &ev); err != nil { continue // left for the redrive policy to move to the DLQ } if err := sendConfirmation(ctx, ev); err != nil { continue // becomes visible again after the visibility timeout } client.DeleteMessage(ctx, &sqs.DeleteMessageInput{ QueueUrl: aws.String(queueURL), ReceiptHandle: m.ReceiptHandle, }) } } } func sendConfirmation(ctx context.Context, ev OrderPlaced) error { return nil }

Around that code you create the topic, the queue, a subscription from the queue to the topic with a queue policy that lets SNS send to it, and raw message delivery so the body is your JSON rather than an SNS envelope. Retries come from the visibility timeout, which has to be longer than the slowest handler, and a redrive policy moves a message to a dead-letter queue after a set number of receives. The task role needs sns:Publish on the topic and receive and delete permissions on the queue, and local development needs either LocalStack or a personal AWS account.

The consumer loop is also yours to harden: shutting it down cleanly on deploy, backing off when SQS returns errors, and running it in a separate process if the API and the worker should scale independently. Idempotency works the same way as in the Encore example, since SQS standard queues also deliver at least once.

Best fit: teams with SNS and SQS already managed in Terraform, or that need SQS features such as FIFO queues configured directly.

A goroutine and a channel

The shortest version is go sendConfirmation(ev) or a buffered channel read by a worker goroutine inside the chi process. It is fine for best-effort work such as warming a cache, and it loses messages on every deploy, crash or scale-in because nothing is stored outside the process. There is no retry, no dead-letter queue and no way for a second service to consume the same event.

Best fit: work you can afford to drop.

How to choose

Pick Encore when you want to keep chi and get a topic and subscriber declared in Go, retries and a dead-letter queue configured on the subscription, a local broker from encore run, and SNS and SQS or Google Pub/Sub provisioned in your own account. Pick SNS and SQS by hand when the messaging setup already lives in Terraform. Use a goroutine only for work where losing a message costs nothing.

Frequently asked questions

How do I publish messages from a go-chi handler?

Validate the request, build the event, publish it with r.Context(), and return 202 Accepted once the publish call succeeds. The work the event triggers, such as sending an email, runs in a subscriber, so the HTTP response no longer waits for it.

How do I add Pub/Sub to a chi app with Encore?

Declare the topic with pubsub.NewTopic[*OrderPlaced]("order-placed", pubsub.TopicConfig{DeliveryGuarantee: pubsub.AtLeastOnce}), call OrderEvents.Publish from the chi handler, and register a handler with pubsub.NewSubscription. The chi router is mounted in a raw endpoint with path=/!fallback, and Encore runs the topic in memory locally and provisions SNS and SQS or Google Pub/Sub in your own cloud account.

Why does my subscriber need to be idempotent?

At-least-once delivery means the same message can arrive twice, for example after a timeout or a crash before the acknowledgement. Recording what you already handled, such as with confirmationSent.SetIfNotExists keyed by order ID, lets the second delivery return nil without sending the email again.

What happens when a subscriber keeps failing?

The message is retried with backoff according to the subscription's RetryPolicy, and after MaxRetries attempts it moves to a dead-letter queue so the rest of the topic keeps flowing. You fix the bug and then release the dead-lettered messages for processing again.

Deploy to your cloud on AWS and Google Cloud
without managing infrastructure

Encore lets developers and agents define application infrastructure directly in code, then automatically provisions it from local development to production in your AWS or GCP account.

~/orders — claude
Claude Code
Claude Code v2.1.180Opus 5.5~/orders