Publish events from a Go Fiber v3 handler and process them in idempotent subscribers with retries, using Encore on SNS and SQS or GCP Pub/Sub, or the AWS SDK by hand.

A Fiber handler should answer quickly, so work like sending emails, revoking sessions or calling a CRM usually moves out of the request and into an event that other code processes. Starting a goroutine looks like the shortcut, but it has no retry and dies with the process on the next deploy, and the durable version on AWS means an SNS topic, an SQS queue per consumer, a dead-letter queue for each, IAM for every publisher and consumer, and a local stand-in for all of it.
This guide adds Pub/Sub to a Fiber v3 API that handles account deletion requests by fanning one event out to two consumers, one that revokes sessions and one that sends a confirmation email exactly once. It starts with Encore, which declares the topic and subscriptions in Go and runs them locally and in your own AWS or GCP account, then shows the SNS and SQS version with the AWS SDK.
Encore works alongside Fiber for this, so the Fiber handler stays the place that accepts the request while the topic and its subscribers are Go declarations in the same package. The app is mounted with Fiber's adaptor.FiberApp in one fallback route, and the Pub/Sub API gives each consumer its own subscription with its own retries.
Run in your terminal to get started locally.
The Fiber handler publishes AccountDeleted and returns 202 with the message ID. revoke-sessions deletes the user's session set, which is safe to repeat, and send-deletion-email claims the send with SetIfNotExists before calling the email provider and releases the claim if the call fails, so a redelivery after a failure sends the email and a redelivery after a success does nothing:
// account/account.go
package account
import (
"context"
"errors"
"net/http"
"strings"
"time"
"encore.dev/pubsub"
"encore.dev/rlog"
"encore.dev/storage/cache"
"github.com/gofiber/fiber/v3"
"github.com/gofiber/fiber/v3/middleware/adaptor"
)
type AccountDeleted struct {
UserID string `json:"user_id"`
Email string `json:"email"`
RequestedAt time.Time `json:"requested_at"`
}
// Every subscription gets its own copy of each event.
var Deletions = pubsub.NewTopic[*AccountDeleted]("account-deleted", pubsub.TopicConfig{
DeliveryGuarantee: pubsub.AtLeastOnce,
})
var cluster = cache.NewCluster("account", cache.ClusterConfig{
EvictionPolicy: cache.NoEviction,
})
// Session tokens per user (written by the login flow, not shown).
var sessions = cache.NewSetKeyspace[string, string](cluster, cache.KeyspaceConfig{
KeyPattern: "sessions/:key",
})
// Marks a confirmation email as sent so a redelivery does not send it twice.
var sent = cache.NewStringKeyspace[string](cluster, cache.KeyspaceConfig{
KeyPattern: "deletion-email/:key",
DefaultExpiry: cache.ExpireIn(7 * 24 * time.Hour),
})
//encore:service
type Service struct {
handler http.HandlerFunc
}
func initService() (*Service, error) {
app := fiber.New()
app.Post("/account/delete", requestDeletion)
return &Service{handler: adaptor.FiberApp(app)}, nil
}
//encore:api public raw path=/!fallback
func (s *Service) Fallback(w http.ResponseWriter, req *http.Request) {
s.handler(w, req)
}
func requestDeletion(c fiber.Ctx) error {
// Stand-in for your auth middleware.
userID := c.Get("X-User-ID")
email := c.Get("X-User-Email")
if userID == "" || !strings.Contains(email, "@") {
return c.Status(fiber.StatusUnauthorized).JSON(fiber.Map{"error": "not signed in"})
}
id, err := Deletions.Publish(c.Context(), &AccountDeleted{
UserID: userID,
Email: email,
RequestedAt: time.Now().UTC(),
})
if err != nil {
return err
}
return c.Status(fiber.StatusAccepted).JSON(fiber.Map{"status": "scheduled", "message_id": id})
}
// Deleting the session set is idempotent, so a redelivery is harmless.
var _ = pubsub.NewSubscription(Deletions, "revoke-sessions",
pubsub.SubscriptionConfig[*AccountDeleted]{
Handler: func(ctx context.Context, ev *AccountDeleted) error {
if _, err := sessions.Delete(ctx, ev.UserID); err != nil {
return err
}
rlog.Info("sessions revoked", "user", ev.UserID)
return nil
},
},
)
var _ = pubsub.NewSubscription(Deletions, "send-deletion-email",
pubsub.SubscriptionConfig[*AccountDeleted]{
Handler: sendDeletionEmail,
AckDeadline: time.Minute,
RetryPolicy: &pubsub.RetryPolicy{
MinBackoff: 30 * time.Second,
MaxBackoff: 10 * time.Minute,
MaxRetries: 10,
},
},
)
func sendDeletionEmail(ctx context.Context, ev *AccountDeleted) error {
// Claim the send first; only one delivery of this event gets past here.
if err := sent.SetIfNotExists(ctx, ev.UserID, ev.RequestedAt.Format(time.RFC3339)); errors.Is(err, cache.KeyExists) {
return nil
} else if err != nil {
return err
}
if err := emailProvider(ctx, ev.Email); err != nil {
// Release the claim so the retry can send it.
_, _ = sent.Delete(ctx, ev.UserID)
rlog.Warn("deletion email failed, will retry", "user", ev.UserID, "err", err)
return err
}
rlog.Info("deletion email sent", "user", ev.UserID)
return nil
}
// emailProvider stands in for your email API client.
func emailProvider(ctx context.Context, to string) error {
if strings.HasSuffix(to, "@bounce.test") {
return errors.New("email provider unavailable")
}
return nil
}
Running this with encore run, a request with the two headers returns 202 and both subscribers log within a second, and a request without them returns 401 before anything is published. For an address on bounce.test the session revocation succeeds while the email subscriber logs the failure and returns an error, and Encore delivers the message to that subscription again while the other subscription is unaffected. After MaxRetries failures the message moves to the subscription's dead-letter queue instead of blocking newer events.
AckDeadline is the time a handler has before the message is considered lost and redelivered, and the handler's ctx is cancelled when it passes. Encore uses it as the SQS visibility timeout on AWS, and it defaults to 30 seconds. On deploy, Encore provisions SNS and SQS on AWS or Pub/Sub on GCP in your own account, with a dead-letter queue per subscription and publish and consume permissions only for the services that use them.
When an agent adds a third consumer, for example one that deletes uploaded files, the change is one more pubsub.NewSubscription in Go, and encore run delivers real events to it locally before the pull request is opened.
Best fit: Fiber APIs that hand work to background consumers and want retries, dead-lettering and fan-out running locally and on SNS/SQS or Pub/Sub in their own cloud.
The hand-built version publishes to an SNS topic from the Fiber handler with aws-sdk-go-v2 and reads one subscription queue in a long-polling loop. A message is only deleted after the work succeeds, so a failure leaves it to reappear when the visibility timeout expires:
package main
import (
"context"
"encoding/json"
"log"
"os"
"time"
"github.com/aws/aws-sdk-go-v2/aws"
"github.com/aws/aws-sdk-go-v2/config"
"github.com/aws/aws-sdk-go-v2/service/sns"
"github.com/aws/aws-sdk-go-v2/service/sqs"
"github.com/gofiber/fiber/v3"
)
type AccountDeleted struct {
UserID string `json:"user_id"`
Email string `json:"email"`
RequestedAt time.Time `json:"requested_at"`
}
func main() {
ctx := context.Background()
cfg, err := config.LoadDefaultConfig(ctx) // task role credentials on ECS
if err != nil {
log.Fatal(err)
}
snsClient, sqsClient := sns.NewFromConfig(cfg), sqs.NewFromConfig(cfg)
go consume(ctx, sqsClient, os.Getenv("EMAIL_QUEUE_URL"))
app := fiber.New()
app.Post("/account/delete", func(c fiber.Ctx) error {
body, _ := json.Marshal(AccountDeleted{UserID: c.Get("X-User-ID"), Email: c.Get("X-User-Email"), RequestedAt: time.Now().UTC()})
_, err := snsClient.Publish(c.Context(), &sns.PublishInput{
TopicArn: aws.String(os.Getenv("ACCOUNT_DELETED_TOPIC_ARN")),
Message: aws.String(string(body)),
})
if err != nil {
return err
}
return c.SendStatus(fiber.StatusAccepted)
})
log.Fatal(app.Listen(":8080"))
}
// consume long-polls one subscription queue (raw message delivery enabled).
func consume(ctx context.Context, client *sqs.Client, queueURL string) {
for {
out, err := client.ReceiveMessage(ctx, &sqs.ReceiveMessageInput{
QueueUrl: aws.String(queueURL),
MaxNumberOfMessages: 10,
WaitTimeSeconds: 20,
VisibilityTimeout: 60,
})
if err != nil {
log.Println("receive:", err)
time.Sleep(time.Second)
continue
}
for _, m := range out.Messages {
var ev AccountDeleted
if err := json.Unmarshal([]byte(aws.ToString(m.Body)), &ev); err != nil {
continue // left for the redrive policy to move to the DLQ
}
if err := sendDeletionEmail(ctx, ev); err != nil {
continue // becomes visible again after the visibility timeout
}
client.DeleteMessage(ctx, &sqs.DeleteMessageInput{QueueUrl: aws.String(queueURL), ReceiptHandle: m.ReceiptHandle})
}
}
}
func sendDeletionEmail(ctx context.Context, ev AccountDeleted) error { return nil }
Each consumer gets its own SQS queue subscribed to the topic with raw message delivery so the body is the event JSON, a queue policy that lets the topic send to it, a dead-letter queue with a maxReceiveCount in the redrive policy, and a task role that can sns:Publish on the topic and receive and delete on its queue. The idempotency claim from the Encore version still applies, since SQS standard queues deliver at least once. Running the consumer inside the API process, as here, couples its scaling to the web tier, and splitting it out means a second ECS service with its own task definition.
Best fit: teams that already manage SNS and SQS in Terraform and want full control over queue settings and the consumer loop.
go sendDeletionEmail(ev) inside the Fiber handler returns fast and needs no infrastructure. The work is lost if the process restarts before it finishes, there is no retry when the email provider fails, and with several replicas there is no record of which deletions were confirmed. It is fine for best-effort work such as warming a cache, and a poor fit for anything a user or a regulation expects to happen.
Best fit: best-effort side work that can be lost without consequences.
Pick Encore when the Fiber app needs durable events with several consumers, retries and dead-letter queues, running locally with encore run and on SNS/SQS or Pub/Sub in your own account. Pick SNS and SQS by hand when the queues are already defined in Terraform and you want every setting under your control. Keep goroutines for work that does not need to survive a restart.
Publish from the Fiber handler to a topic, return 202, and process the event in a separate consumer with retries and a dead-letter queue. On AWS that is usually an SNS topic with one SQS queue per consumer, published with the aws-sdk-go-v2 SNS client and read by a long-polling SQS loop.
Declare a topic with pubsub.NewTopic and one pubsub.NewSubscription per consumer, mount the Fiber app in a raw endpoint with path=/!fallback through adaptor.FiberApp, and call Topic.Publish from the Fiber handler. encore run runs the topic and subscribers locally, and deploys provision SNS and SQS on AWS or Pub/Sub on GCP in your own account.
Only for work you can afford to lose. A goroutine started from a handler dies with the process during a deploy or crash and has no retry, so jobs like emails and data deletion belong on a queue with retries and a dead-letter queue.
Assume every message can arrive more than once. Make the handler's effect naturally repeatable, like deleting a key, or claim the work first with an atomic write such as SET NX keyed by the event, and release the claim if the side effect fails so the retry can run it.
Deploy to your cloud on AWS and Google Cloud
without managing infrastructure
Encore lets developers and agents define application infrastructure directly in code, then automatically provisions it from local development to production in your AWS or GCP account.