Build a point-to-point pipe from SQS, Kinesis, DynamoDB Streams or Kafka to a target, with optional filtering and enrichment, and when an Encore subscription is a better fit.

A common AWS glue job is "read this queue or stream and put the payload on that other service," and the default way to do it used to be a small Lambda that polls and forwards.
EventBridge Pipes is the managed version of that glue: one source, an optional filter, an optional enrichment, and one target, with order preserved when the source itself is ordered. You are charged for events that match the filter, not for every record the source emits.
This guide walks through creating a pipe from SQS to a target, then shows the Encore path for events that start in your application code, where a topic and subscription replace the pipe.
Pick the queue, add a filter if you only want some messages, optionally attach an enrichment, and choose a target, as described in Creating a pipe. Before running the command, create or verify the order-paid-pipe role has a trust policy allowing pipes.amazonaws.com to assume it, and least-privilege permissions for sqs:ReceiveMessage, sqs:DeleteMessage and sqs:GetQueueAttributes on the queue plus states:StartExecution on the state machine (and any required KMS permissions for encrypted resources). EventBridge validates the source permissions when the pipe is created. The CLI command below assumes this role already exists. Enrichments and targets can be changed later; the source cannot.
A typical SQS-to-Step-Functions pipe looks like this in the AWS CLI:
Filters use an event pattern: for an SQS message whose body is JSON, match its fields under body, for example { "body": [{ "orderStatus": ["PAID"] }] }, as shown in the pipes filtering documentation. Use data only after an enrichment explicitly produces that field. Enrichment can also drop an event by returning {}, [] or an empty string; return [{}] if the target should be invoked with an empty payload.
Batch size is validated against the target: if the target does not support batches, the pipe cannot batch, and API Gateway or API destination enrichments only support a batch size of 1.
Best fit: a single queue or stream that should feed a single AWS target, with a filter and maybe a small enrichment, and no desire to run a polling function.
Write a Lambda when the mapping needs branches, retries against your own database, or a side effect that a pipe cannot express. Pipes do not replace a workflow engine: long-running, branching processes still belong in Step Functions or in application code. They also do not replace a topic with several independent consumers; use SNS or an event bus when one event must reach many targets.
Encore provisions topics from code: when the event is produced by your API, declare a topic and a subscription in Go. On AWS, a deploy provisions SNS and SQS in your account; on GCP that is Pub/Sub. Encore does not create EventBridge Pipes. Encore provisions an IAM role for the service, or a service account on GCP.
A coding agent can add the topic, the publish call and the subscriber in the same change, then use encore run to publish a message and watch the subscriber run on the laptop.
var Paid = pubsub.NewTopic[*OrderPaid]("order-paid", pubsub.TopicConfig{
DeliveryGuarantee: pubsub.AtLeastOnce,
})
var _ = pubsub.NewSubscription(Paid, "start-fulfillment",
pubsub.SubscriptionConfig[*OrderPaid]{
Handler: fulfillOrder,
RetryPolicy: &pubsub.RetryPolicy{MaxRetries: 8},
},
)
If the messages already live in an SQS queue or a Kinesis stream that Encore does not own, keep a pipe (or a consumer you write) in front of that source, and publish into an Encore topic only once the payload has entered your application.
Best fit: events that start in your API and should be processed by code in the same repository, with local delivery under encore run.
Pick EventBridge Pipes when an existing SQS queue or stream should feed one AWS target without a polling Lambda. Pick a Lambda when the transformation needs application logic. Pick Encore when your service produces the event and you want the subscriber declared next to the publisher in Go.
EventBridge Pipes is a point-to-point integration that reads from a single source, optionally filters and enriches the event, and sends it to a single target. Supported sources include SQS, Kinesis, DynamoDB Streams, Amazon MQ, MSK and self-managed Kafka. You pay for events that match the filter.
Not always. A pipe can read an SQS queue and send matching messages to a target such as Step Functions, Kinesis, a Lambda function or an API destination. Use a Lambda anyway when the transformation needs custom code that a pipe filter and enrichment cannot express.
No. Encore provisions SNS and SQS or GCP Pub/Sub for topics and subscriptions declared in your code, and it does not create pipes. When the event starts in your application, publish to an Encore topic and handle it in a subscription. When the event already lives in SQS or a stream you do not own, a pipe can move it without a polling function.
The pipe can call a Lambda function, an Express Step Functions workflow, API Gateway or an API destination synchronously, then pass the response to the target. Enrichment responses are limited to 6 MB. Returning an empty payload such as {}, [] or an empty string skips the target.
Deploy to your cloud on AWS and Google Cloud
without managing infrastructure
Encore lets developers and agents define application infrastructure directly in code, then automatically provisions it from local development to production in your AWS or GCP account.