SQS Dead-Letter Queues: Redrive Policy, maxReceiveCount and Redrive

An SQS consumer that keeps failing on one message has two bad outcomes without a dead-letter queue: the message is retried until it expires, eating consumer time and log volume for days, or someone deletes it by hand and the failure is never understood. A dead-letter queue fixes that by moving the message aside after a set number of receives, and getting it right involves a few settings that interact in ways the console does not make obvious.

This guide covers how SQS dead-letter queues work, the Terraform for a queue with a DLQ, a redrive allow policy and an alarm, how to move messages back once the bug is fixed, and how Encore configures retries and a dead-letter queue for every subscription from Go code.

SQS dead-letter queue settings at a glance

How does Encore handle dead-letter queues?

Encore brings infrastructure from code to messaging, so retries and dead-lettering are part of the subscription declaration in Go rather than settings spread over two queues. Encore provisions SNS and SQS in your own AWS account, or Google Pub/Sub on GCP, and configures a dead-letter queue for each subscription automatically.

Set up the app

The subscriber below renders invoice PDFs, and a message it can never process, one without an invoice ID, is logged and acknowledged, while a failed render returns an error so it is retried with backoff and dead-lettered after five retries:

// invoices/render.go
package invoices

import (
	"context"
	"errors"
	"fmt"
	"time"

	"encore.dev/pubsub"
	"encore.dev/rlog"
)

type InvoiceRequested struct {
	InvoiceID string `json:"invoice_id"`
	Customer  string `json:"customer"`
}

// SNS + SQS on AWS, Google Pub/Sub on GCP, in memory locally.
var InvoiceRequests = pubsub.NewTopic[*InvoiceRequested]("invoice-requested", pubsub.TopicConfig{
	DeliveryGuarantee: pubsub.AtLeastOnce,
})

var _ = pubsub.NewSubscription(InvoiceRequests, "render-pdf", pubsub.SubscriptionConfig[*InvoiceRequested]{
	Handler: renderPDF,
	// The handler's ctx is cancelled after AckDeadline, and the message is retried.
	AckDeadline: 2 * time.Minute,
	RetryPolicy: &pubsub.RetryPolicy{
		MinBackoff: 30 * time.Second,
		MaxBackoff: 15 * time.Minute,
		MaxRetries: 5, // after 5 retries the message goes to this subscription's DLQ
	},
})

var errTemplateMissing = errors.New("invoice template missing")

func renderPDF(ctx context.Context, req *InvoiceRequested) error {
	if req.InvoiceID == "" {
		// Retrying cannot fix a message without an ID: log it and ack it.
		rlog.Error("dropping invoice request without id", "customer", req.Customer)
		return nil
	}
	if err := render(ctx, req.InvoiceID); err != nil {
		// Returning the error nacks the message: it is retried with backoff,
		// then dead-lettered once MaxRetries is reached.
		return fmt.Errorf("render invoice %s: %w", req.InvoiceID, err)
	}
	return nil
}

func render(ctx context.Context, invoiceID string) error {
	// Call the PDF renderer and store the result under invoiceID, overwriting
	// any earlier attempt so retries stay idempotent.
	if invoiceID == "inv_broken" {
		return errTemplateMissing
	}
	return nil
}

type RequestParams struct {
	InvoiceID string `json:"invoice_id"`
	Customer  string `json:"customer"`
}

//encore:api private method=POST path=/invoices/render
func Request(ctx context.Context, p *RequestParams) error {
	_, err := InvoiceRequests.Publish(ctx, &InvoiceRequested{InvoiceID: p.InvoiceID, Customer: p.Customer})
	return err
}

Running it with encore run, a normal request renders without errors, inv_broken logs a handler error and is redelivered after the backoff, and the request without an ID is dropped with an error log and not retried. Separating permanent from transient failures matters as much as the retry numbers: retrying a message that can never succeed only delays the moment it lands in the DLQ, so permanent failures are better logged and acknowledged, or marked with MaxRetries: pubsub.NoRetries on a subscription where every error is permanent.

The Pub/Sub docs describe the flow after MaxRetries: the message moves to the subscription's dead-letter queue, the subscription keeps processing new events, and once the bug is fixed the dead-lettered messages can be released to the subscriber again. Retry values are read at compile time and clamped to what the cloud supports, so the same declaration works on SQS and Google Pub/Sub. An agent changing a subscriber's retry behavior edits one struct in the same file as the handler, and the change is visible in review next to the code it affects.

Best fit: services that publish and consume events in Go or TypeScript and want retries, backoff and a dead-letter queue for every consumer declared with the subscription.

Configuring an SQS dead-letter queue by hand

A dead-letter queue is an ordinary queue in the same account and Region as the source queue. The source queue's redrive policy names it and sets maxReceiveCount, the number of times a message can be received before SQS moves it. Two details from the AWS docs shape the configuration: for standard queues with a maxReceiveCount above 3, SQS moves a message to the back of the queue after it has been received three times without being deleted, and a message's expiry is based on its original enqueue timestamp, so the DLQ needs a longer retention period than the source queue or messages can expire shortly after they arrive. Here is the setup in Terraform with the AWS provider 6.x, validated with terraform validate:

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 6.0"
    }
  }
}

resource "aws_sqs_queue" "invoices_dlq" {
  name                      = "invoices-dlq"
  message_retention_seconds = 1209600 # 14 days, longer than the source queue
  sqs_managed_sse_enabled   = true
}

resource "aws_sqs_queue" "invoices" {
  name                       = "invoices"
  visibility_timeout_seconds = 60     # longer than the slowest handler
  message_retention_seconds  = 345600 # 4 days
  sqs_managed_sse_enabled    = true

  redrive_policy = jsonencode({
    deadLetterTargetArn = aws_sqs_queue.invoices_dlq.arn
    maxReceiveCount     = 5
  })
}

# Only the invoices queue may use this DLQ.
resource "aws_sqs_queue_redrive_allow_policy" "invoices_dlq" {
  queue_url = aws_sqs_queue.invoices_dlq.id

  redrive_allow_policy = jsonencode({
    redrivePermission = "byQueue"
    sourceQueueArns   = [aws_sqs_queue.invoices.arn]
  })
}

variable "alerts_topic_arn" {
  type = string
}

resource "aws_cloudwatch_metric_alarm" "invoices_dlq_not_empty" {
  alarm_name          = "invoices-dlq-not-empty"
  namespace           = "AWS/SQS"
  metric_name         = "ApproximateNumberOfMessagesVisible"
  dimensions          = { QueueName = aws_sqs_queue.invoices_dlq.name }
  statistic           = "Maximum"
  period              = 300
  evaluation_periods  = 1
  threshold           = 0
  comparison_operator = "GreaterThanThreshold"
  treat_missing_data  = "notBreaching"
  alarm_actions       = [var.alerts_topic_arn]
}

The visibility timeout is the retry delay, and it has to be longer than the slowest successful handler, otherwise a message still being processed becomes visible again and a second consumer picks it up, which also counts toward maxReceiveCount. Unlike Encore's backoff, the delay is the same for every attempt, so five retries with a 60 second timeout give a downstream outage only about five minutes to recover. The redrive allow policy stops other queues from pointing at this DLQ, and the CloudWatch alarm on ApproximateNumberOfMessagesVisible turns a silent failure into a page.

Best fit: queues already managed in Terraform, and consumers such as Lambda event source mappings that rely on queue-level settings.

Moving messages back with redrive

After a fix is deployed, dead-letter queue redrive moves messages from the DLQ back to the source queue, or to another queue, from the console or through the StartMessageMoveTask API with an optional rate limit. Redriving into a consumer that still fails sends the messages straight back to the DLQ, so check a sample of the dead-lettered bodies against the fixed code first. Redriven messages are processed again from the start, which is one more reason for consumers to be idempotent. AWS also advises against a DLQ on FIFO queues when the exact order of operations matters, since moving one message aside lets later messages in the group go ahead.

How to choose

Use Encore when your services publish and consume events from Go or TypeScript and you want retries with backoff, a dead-letter queue per subscription and the SQS setup provisioned in your own AWS account from the subscription declaration. Configure SQS by hand when the queues already live in Terraform or are consumed by tools that read queue settings directly, and in both cases alarm on the dead-letter queue so failures are seen.

Related guides

• What is Amazon SQS
• SQS with Terraform
• SQS with Lambda
• SNS vs SQS
• How to add Pub/Sub to a chi app