Nov 17, 202611 min read

SNS Message Filtering: Subscription Filter Policies by Attribute or Payload

How Amazon SNS subscription filter policies work, attribute-based versus payload-based filtering, every operator with an example, how to publish attributes from Go, how to apply policies with the AWS CLI and Terraform, the limits that trip people up, and how Encore approaches the same routing problem.

Illustration for SNS Message Filtering: Subscription Filter Policies by Attribute or Payload

SNS message filtering lets each subscriber of a topic receive only the messages it cares about. You attach a JSON filter policy to a subscription, and SNS compares it against either the message attributes or the JSON message body before delivering. Messages that do not match are dropped for that subscription only, so the topic stays shared and each queue, function or endpoint gets a narrower stream.

Short answer: set the FilterPolicy attribute on the subscription with a JSON object whose keys are attribute or field names and whose values are arrays of allowed values or operators. Keys are combined with AND, values within one array with OR. Leave FilterPolicyScope at MessageAttributes to match publisher-set attributes, or set it to MessageBody to match fields in a JSON payload.

This guide covers both scopes, every operator with an example, publishing attributes from Go, the CLI and Terraform setup, the limits that cause silent drops, and how Encore handles the same routing problem.

Attribute-based vs payload-based filtering

MessageAttributes (default)MessageBody
Matches againstMessage attributes set by the publisherFields in the JSON message body
Nested keysNot supported, policy must be flatSupported, nest objects to match nested fields
Publisher change neededYesevery message must carry the attributesNothe body already has the fields
Non-JSON messagesFine, the body is ignoredDo not match
Data typesString, String.Array and Number; Binary attributes are ignoredJSON strings, numbers, booleans, arrays
Best forRouting on a few stable keys like type or tenantRouting on fields you do not want to duplicate

The scope is a separate subscription attribute, FilterPolicyScope, described in the filter policy scope docs. Each subscription has one scope, so a single policy cannot mix attribute keys and body keys.

How filter policies match

A filter policy is a JSON object in which every key must find a matching value in the message, and if any key in the policy fails to match, the message is not delivered to that subscription. Within a key, the array lists alternatives:

{ "type": ["order.placed", "order.updated"], "country": ["DE", "AT"] }

This policy delivers messages whose type is order.placed or order.updated, and whose country is DE or AT. A message without a country key does not match. String matching is exact and case-sensitive. When the message value is an array, a String.Array attribute or a JSON array in the body, the message matches if any element matches, as the string matching docs show.

Filter policy operators with examples

Every operator goes inside the value array, and a key can mix plain values and operators.

OperatorPolicy fragmentMatches
Exact string"type": ["order.placed"]order.placed only
anything-but"country": [{"anything-but": ["AT", "CH"]}]Any country except AT and CH
anything-but prefix"type": [{"anything-but": {"prefix": "test."}}]Any type not starting with test.
prefix"sku": [{"prefix": "BOOK-"}]BOOK-123, BOOK-9
suffix"file": [{"suffix": ".csv"}]orders.csv
equals-ignore-case"status": [{"equals-ignore-case": "paid"}]paid, PAID, Paid
numeric range"amount": [{"numeric": [">=", 1000, "<", 5000]}]1000 up to, not including, 5000
numeric exact"priority": [{"numeric": ["=", 1]}]1
exists"coupon": [{"exists": true}]Any message that has the key
not exists"coupon": [{"exists": false}]Messages without the key
cidr"source_ip": [{"cidr": "10.0.0.0/24"}]10.0.0.0 to 10.0.0.255

To combine keys with OR instead of AND, use $or with a list of sub-policies:

{ "type": ["order.placed"], "$or": [ { "amount": [{ "numeric": [">=", 1000] }] }, { "tags": ["first-order"] } ] }

This delivers placed orders that are either 1000 or more, or tagged first-order. Numeric values must fall between -1,000,000,000 and 1,000,000,000 with up to five decimal places, per the filter policy constraints.

Publish messages with attributes from Go

Attribute-based policies only see attributes, so the publisher has to copy each routing field out of the body. Numeric operators need the Number data type, and arrays use String.Array with a JSON-encoded value.

// main.go package main import ( "context" "encoding/json" "log" "os" "strconv" "github.com/aws/aws-sdk-go-v2/aws" "github.com/aws/aws-sdk-go-v2/config" "github.com/aws/aws-sdk-go-v2/service/sns" "github.com/aws/aws-sdk-go-v2/service/sns/types" ) type OrderEvent struct { Type string `json:"type"` // "order.placed", "order.cancelled", ... OrderID string `json:"order_id"` Amount float64 `json:"amount"` Country string `json:"country"` Tags []string `json:"tags"` } func publish(ctx context.Context, client *sns.Client, topicARN string, ev OrderEvent) error { body, err := json.Marshal(ev) if err != nil { return err } tags, err := json.Marshal(ev.Tags) if err != nil { return err } _, err = client.Publish(ctx, &sns.PublishInput{ TopicArn: aws.String(topicARN), Message: aws.String(string(body)), // Attribute-based filter policies only see these, not the body. MessageAttributes: map[string]types.MessageAttributeValue{ "type": {DataType: aws.String("String"), StringValue: aws.String(ev.Type)}, "country": {DataType: aws.String("String"), StringValue: aws.String(ev.Country)}, // Numeric operators need the Number data type. "amount": {DataType: aws.String("Number"), StringValue: aws.String(strconv.FormatFloat(ev.Amount, 'f', -1, 64))}, // A String.Array matches if any element matches. "tags": {DataType: aws.String("String.Array"), StringValue: aws.String(string(tags))}, }, }) return err } func main() { ctx := context.Background() cfg, err := config.LoadDefaultConfig(ctx) if err != nil { log.Fatal(err) } client := sns.NewFromConfig(cfg) err = publish(ctx, client, os.Getenv("TOPIC_ARN"), OrderEvent{ Type: "order.placed", OrderID: "ord_123", Amount: 1250, Country: "DE", Tags: []string{"b2b", "priority"}, }) if err != nil { log.Fatal(err) } }

Keeping attributes and body in sync is the main cost of attribute-based filtering. A publisher that forgets an attribute produces messages that silently skip every subscription filtering on that key. Payload-based filtering avoids the duplication at the price of requiring a JSON body.

Apply a filter policy with the AWS CLI

You can set the policy when subscribing, or add it to an existing subscription:

~
# New subscription with an attribute-based policy$ aws sns subscribe \ --topic-arn arn:aws:sns:us-east-1:123456789012:order-events \ --protocol sqs \ --notification-endpoint arn:aws:sqs:us-east-1:123456789012:fraud-review \ --attributes '{"FilterPolicy":"{\"type\":[\"order.placed\"],\"amount\":[{\"numeric\":[\">=\",1000]}]}"}' # Existing subscription: switch the scope to the body, then set the policy$ aws sns set-subscription-attributes \ --subscription-arn arn:aws:sns:us-east-1:123456789012:order-events:6b0e71bd-7e97-4d97-80ce-4a0994e55286 \ --attribute-name FilterPolicyScope \ --attribute-value MessageBody$ aws sns set-subscription-attributes \ --subscription-arn arn:aws:sns:us-east-1:123456789012:order-events:6b0e71bd-7e97-4d97-80ce-4a0994e55286 \ --attribute-name FilterPolicy \ --attribute-value '{"type":[{"prefix":"order."}],"shipping":{"country":["DE","AT","FR"]}}' # Check what is attached$ aws sns get-subscription-attributes \ --subscription-arn arn:aws:sns:us-east-1:123456789012:order-events:6b0e71bd-7e97-4d97-80ce-4a0994e55286 \ --query 'Attributes.[FilterPolicy,FilterPolicyScope]'

The second policy uses a nested shipping object, which only works with the MessageBody scope, so the scope changes first. To remove filtering from a subscription, set FilterPolicy to an empty object, --attribute-value "{}", as in the AWS CLI examples.

Filter policies in Terraform

The aws_sns_topic_subscription resource has filter_policy and filter_policy_scope arguments, and jsonencode keeps the policy readable:

# sns_filters.tf resource "aws_sns_topic" "orders" { name = "order-events" } resource "aws_sqs_queue" "fraud_review" { name = "fraud-review" } resource "aws_sqs_queue" "eu_fulfillment" { name = "eu-fulfillment" } # Attribute-based: matches on MessageAttributes set by the publisher. resource "aws_sns_topic_subscription" "fraud_review" { topic_arn = aws_sns_topic.orders.arn protocol = "sqs" endpoint = aws_sqs_queue.fraud_review.arn filter_policy_scope = "MessageAttributes" filter_policy = jsonencode({ type = ["order.placed"] amount = [{ numeric = [">=", 1000] }] country = [{ "anything-but" = ["AT"] }] }) } # Payload-based: matches on fields inside the JSON message body. resource "aws_sns_topic_subscription" "eu_fulfillment" { topic_arn = aws_sns_topic.orders.arn protocol = "sqs" endpoint = aws_sqs_queue.eu_fulfillment.arn filter_policy_scope = "MessageBody" filter_policy = jsonencode({ type = [{ prefix = "order." }] shipping = { country = ["DE", "AT", "FR"] } }) }

Each queue also needs a queue policy that lets the topic send to it, and usually a dead-letter queue. The SNS with Terraform guide covers both, along with topic encryption and subscription redrive.

Limits and gotchas

The filter policy constraints are where most surprises come from:

  • Five keys per policy. In a nested body policy, only leaf keys count toward the five.
  • 150 value combinations. SNS multiplies the number of values for each key, so three keys with five, five and six values make 150, the limit, and one more value on any key is rejected. Nested body policies also multiply by the nesting depth.
  • 256 KB maximum policy size, 200 filter policies per topic and 10,000 per account by default. The topic and account quotas can be raised through Service Quotas.
  • Up to 15 minutes to propagate. A policy change does not apply to the next message, so test with a fresh subscription or wait before concluding a policy is wrong.
  • Case-sensitive strings. Use equals-ignore-case when publishers are inconsistent.
  • Attribute types matter. Binary attributes are never compared, and a number published as a String attribute is compared as a string, so numeric operators need DataType Number.
  • Special characters. Quotes and backslashes in attribute-based policy values need double escaping.

When messages go missing, check the CloudWatch metrics for the topic. NumberOfNotificationsFilteredOut counts messages a policy rejected, and the -NoMessageAttributes, -InvalidAttributes and -InvalidMessageBody variants separate "the publisher sent no attributes" and "the body was not valid JSON" from a plain mismatch, as listed in the SNS CloudWatch metrics.

Filter policies, separate topics or consumer-side filtering

Filter policies are one of three ways to route events, and the right one depends on volume and on who owns the rule:

  • Filter policies save the delivery and the queue receive for every rejected message, and they suit high-volume topics where each consumer wants a small slice. The rule lives in subscription configuration, away from the consumer code, and is hard to unit test.
  • Separate topics per event kind make the routing explicit in the publisher and need no policy at all. They work well when event kinds have different schemas or retention needs, and they get awkward when consumers want slices that cut across kinds, such as every event for one tenant.
  • Filtering in the consumer means every subscriber receives everything and returns early on messages it does not want. You pay for every delivery, and in exchange the rule is ordinary code next to the handler, reviewed and tested like the rest of it.

Many systems combine them: a topic per domain, a filter policy where a consumer reads a small fraction of a busy topic, and a guard in the handler for anything more specific. For comparisons with other routing services, see SNS vs EventBridge and SNS vs SQS.

How Encore handles event routing

Encore

Encore gives you infrastructure from code for Pub/Sub: you declare a topic and its subscriptions in your application, and on AWS Encore provisions an SNS topic with an SQS queue and dead-letter queue for each subscription in your own account, as the AWS infrastructure docs describe. Encore subscriptions have no filter policy option, so every subscription receives every message published to its topic. Routing in an Encore app uses the other two approaches: a guard in the handler, or a separate topic for events most consumers ignore.

Set up the app

Get started
brew install encoredev/tap/encore &&
encore app create
Copy

Run in your terminal to get started locally.

~
$ encore app create shop --example=hello-world$ cd shop && mkdir fraud

Declare the topic and filter in the handler

// fraud/fraud.go package fraud import ( "context" "slices" "encore.dev/pubsub" "encore.dev/rlog" ) type OrderEvent struct { Type string `json:"type"` OrderID string `json:"order_id"` Amount float64 `json:"amount"` Country string `json:"country"` Tags []string `json:"tags"` } // One topic for every order event. var OrderEvents = pubsub.NewTopic[*OrderEvent]("order-events", pubsub.TopicConfig{ DeliveryGuarantee: pubsub.AtLeastOnce, }) // The fraud review consumer only cares about large orders outside the home // market. The check lives in the handler, next to the code that depends on it. var _ = pubsub.NewSubscription(OrderEvents, "fraud-review", pubsub.SubscriptionConfig[*OrderEvent]{ Handler: reviewOrder, MaxConcurrency: 20, }, ) func wantsReview(ev *OrderEvent) bool { return ev.Type == "order.placed" && ev.Amount >= 1000 && ev.Country != "AT" && !slices.Contains(ev.Tags, "trusted") } func reviewOrder(ctx context.Context, ev *OrderEvent) error { if !wantsReview(ev) { return nil // acknowledged, nothing to do } rlog.Info("queueing fraud review", "order_id", ev.OrderID, "amount", ev.Amount) return nil }

wantsReview is the same rule as the fraud_review filter policy in the Terraform example, plus a tag check that is awkward in a policy: anything-but on an array value matches as soon as one element differs, so ["trusted", "b2b"] would still pass. Returning nil acknowledges the message, and returning an error makes Encore retry it and eventually move it to the subscription's dead-letter queue, as the Pub/Sub docs explain. The rule is a plain function, so a table test run with encore test covers it, and the local development dashboard shows each delivery in a trace when you run the app with encore run.

The trade-off is cost at volume: every message on order-events is delivered to fraud-review and acknowledged, including the ones it skips. When one event kind dominates a topic and most consumers ignore it, declare it as its own topic with pubsub.NewTopic so those consumers never subscribe to it. When an SNS topic is shared with systems outside Encore that rely on their own subscriptions, you can import the existing SNS topic into the app instead of having Encore provision a new one.

For an AI coding agent, a routing rule in a Go function is something it can read, change and test in the same diff as the handler, instead of editing a JSON policy in another tool and waiting up to 15 minutes to see the effect.

Best fit: teams on AWS whose consumers filter on business rules that change with the code, and who want to test routing with the code.

Frequently asked questions

How do I add a filter policy to an SNS subscription?

Run aws sns set-subscription-attributes with the subscription ARN, --attribute-name FilterPolicy and the policy JSON as --attribute-value, for example {"type":["order.placed"]}. To filter on the message body instead of message attributes, also set the FilterPolicyScope attribute to MessageBody. In Terraform, set filter_policy and filter_policy_scope on aws_sns_topic_subscription.

What is the difference between attribute-based and payload-based SNS filtering?

Attribute-based filtering, the default FilterPolicyScope of MessageAttributes, matches only the message attributes the publisher sets, and the policy must be flat. Payload-based filtering, FilterPolicyScope MessageBody, matches fields inside a JSON message body, including nested objects, and needs a well-formed JSON body to match.

How long does an SNS filter policy change take to apply?

AWS says additions and changes to a subscription filter policy can take up to 15 minutes to fully take effect, so messages published right after a change may still be filtered by the old policy.

Does Encore support SNS filter policies?

Encore Pub/Sub subscriptions have no filter policy option, so every subscription receives every message on its topic. The Encore approach is to filter in the subscription handler, where the rule is ordinary code you can test with encore test, or to split high-volume events into their own topic. Existing SNS topics shared with other systems can be imported into an Encore app.

Deploy to your cloud on AWS and Google Cloud
without managing infrastructure

Encore lets developers and agents define application infrastructure directly in code, then automatically provisions it from local development to production in your AWS or GCP account.

~/orders — claude
Claude Code
Claude Code v2.1.180Opus 5.5~/orders