Nov 20, 20269 min read

How to Test Event-Driven Systems Locally

Test publishers, subscribers, retries and the full flow of an event-driven backend on your laptop: what each layer proves, where cloud emulators drift from production, and a Go example with a publisher test, an idempotent subscriber test and a local end-to-end run.

Illustration for How to Test Event-Driven Systems Locally

An event-driven backend is easy to draw and harder to test, since the interesting behavior happens between services. The order service publishes, a receipt service picks the message up later, and a failure becomes a retry instead of an error.

Many teams end up testing that in staging, where a missing subscription or a handler that sends two emails on redelivery shows up after merging. This guide splits the testing into layers that run on a laptop.

How do you test an event-driven system locally?

Test it in layers: the subscriber's logic as a plain function call, the publisher by asserting on what it published, and a small number of end-to-end runs through a local broker that check the wiring between them. The table shows what each layer proves and how it runs.

LayerEncoreEmulators and Docker brokers
Publisher testet.Topic(topic).PublishedMessages() in a Go test, isolated per testConsume from a test subscription and wait
Subscriber testCall the handler function directly, with a real local databaseSame, with a database you start yourself
End-to-end flowencore run runs topics and subscriptions locally, visible in local tracesRun the broker or emulator next to every service
Retries and dead lettersDeclared on the subscription and configured on the cloud queues at deployOnly as faithful as the emulator's support
Message orderingOrdering attribute has no effect locallyDepends on the broker

What should a publisher test check?

A publisher test checks that the right message was published, with the right payload, and only when the work it describes happened. The last part is where most bugs hide: an order handler that inserts a row and then publishes can fail between the two steps, and a test that only checks the happy path never sees it. If losing that message is unacceptable, the transactional outbox pattern writes the event in the same database transaction as the order, and the test then asserts on the outbox row.

How do you test a subscriber?

Call the handler directly with a message and assert on what it changed, then call it again with the same message. Brokers deliver at least once, which the SQS standard queue docs and Google Cloud Pub/Sub both state plainly, so the second call is the production scenario and not an edge case. A handler that sends a receipt should send one receipt for two deliveries, and the cleanest way to prove that is a test that counts the sends.

Failure cases belong in the same tests, where you make the side effect fail on the first call and succeed on the second, and check that the handler returns an error the first time, so the broker retries, and finishes the work the second time without repeating what already succeeded.

Should you use a cloud emulator or a local broker?

Both work for wiring and serialization, and both drift from production in ways that matter for event-driven tests. The Google Cloud Pub/Sub emulator, for example, lists known limitations that include no IAM operations, no configurable message retention and no subscription expiration, and messages are retained indefinitely. LocalStack and Docker images of Kafka or RabbitMQ have their own gaps.

Configuration drifts even further, since retry counts, backoff and dead-letter queues usually live in Terraform or a cloud console, so the local emulator runs a queue with different settings than production, and a test that passes locally says little about what happens on the fifth failed delivery in the cloud.

How do you test retries and dead-letter queues locally?

Write a handler test that fails a fixed number of times and assert that the handler is safe to rerun, since that part is your code. The retry schedule and the dead-letter move are broker behavior, configured on SQS through a redrive policy and on Pub/Sub through a dead-letter topic. Test those in a deployed environment with a message that always fails, and keep the settings next to the subscription in code so they are reviewed together with the handler.

How do you test the whole flow end to end?

Start every service and the broker, call the API that kicks off the flow, then poll for the final effect, such as the receipt row, with a timeout. Keep these tests few, since they are slower and fail for more reasons, and make them assert on outcomes rather than intermediate messages. Tracing helps more than logs here, because a trace that connects the publish to the handler that processed it makes a missing subscription obvious instead of leaving you with a timeout.

How does Encore test publishers and subscribers locally?

Encore

Encore brings infrastructure from code, so a topic and its subscriptions are Go declarations that encore run runs on your laptop and that become SNS and SQS on AWS or Google Cloud Pub/Sub on GCP when you deploy. Tests get a special Pub/Sub implementation: each test sees only the messages it published, subscriptions are not triggered by them, and message IDs are deterministic.

Application code passes through Encore and infrastructure guardrails to resources provisioned in your own AWS or GCP account.
Get started
brew install encoredev/tap/encore &&
encore app create
Copy

Run in your terminal to get started locally.

The orders service stores each order and publishes an OrderPlaced event. Its table is created by a migration:

-- orders/migrations/1_create_orders.up.sql CREATE TABLE orders ( id BIGSERIAL PRIMARY KEY, email TEXT NOT NULL, total_cents BIGINT NOT NULL, created_at TIMESTAMPTZ NOT NULL DEFAULT now() );

The service declares the database, the topic and the endpoint that publishes to it:

// orders/orders.go package orders import ( "context" "encore.dev/pubsub" "encore.dev/storage/sqldb" ) var db = sqldb.NewDatabase("orders", sqldb.DatabaseConfig{ Migrations: "./migrations", }) type OrderPlacedEvent struct { OrderID int64 Email string TotalCents int64 } // The topic runs locally under encore run and becomes SNS/SQS on AWS // or Google Cloud Pub/Sub on GCP when deployed. var OrderPlaced = pubsub.NewTopic[*OrderPlacedEvent]("order-placed", pubsub.TopicConfig{ DeliveryGuarantee: pubsub.AtLeastOnce, }) type PlaceParams struct { Email string `json:"email"` TotalCents int64 `json:"totalCents"` } type PlaceResponse struct { OrderID int64 `json:"orderId"` } //encore:api public method=POST path=/orders func Place(ctx context.Context, p *PlaceParams) (*PlaceResponse, error) { var id int64 err := db.QueryRow(ctx, `INSERT INTO orders (email, total_cents) VALUES ($1, $2) RETURNING id`, p.Email, p.TotalCents, ).Scan(&id) if err != nil { return nil, err } if _, err := OrderPlaced.Publish(ctx, &OrderPlacedEvent{ OrderID: id, Email: p.Email, TotalCents: p.TotalCents, }); err != nil { return nil, err } return &PlaceResponse{OrderID: id}, nil }

The publisher test calls the endpoint against the local test database and reads back what was published during this test only:

// orders/orders_test.go package orders import ( "context" "testing" "encore.dev/et" ) func TestPlacePublishesOrderPlaced(t *testing.T) { resp, err := Place(context.Background(), &PlaceParams{ Email: "ada@example.com", TotalCents: 4200, }) if err != nil { t.Fatal(err) } // Only messages published by this test, and no subscriber has run. msgs := et.Topic(OrderPlaced).PublishedMessages() if len(msgs) != 1 { t.Fatalf("want 1 message, got %d", len(msgs)) } if msgs[0].OrderID != resp.OrderID || msgs[0].TotalCents != 4200 { t.Fatalf("unexpected event: %+v", msgs[0]) } }

The email service records each receipt in its own database, keyed by order, so a redelivery can tell the work is already done:

-- email/migrations/1_create_receipts.up.sql CREATE TABLE receipts ( order_id BIGINT PRIMARY KEY, email TEXT NOT NULL, sent_at TIMESTAMPTZ );

The subscription declares its retry policy next to the handler, and the handler only marks a receipt as sent after delivery succeeds:

// email/email.go package email import ( "context" "time" "encore.app/orders" "encore.dev/pubsub" "encore.dev/rlog" "encore.dev/storage/sqldb" ) var db = sqldb.NewDatabase("email", sqldb.DatabaseConfig{ Migrations: "./migrations", }) var _ = pubsub.NewSubscription(orders.OrderPlaced, "send-receipt", pubsub.SubscriptionConfig[*orders.OrderPlacedEvent]{ Handler: SendReceipt, RetryPolicy: &pubsub.RetryPolicy{ MinBackoff: 10 * time.Second, MaxBackoff: 5 * time.Minute, MaxRetries: 5, }, }, ) // deliver is a variable so tests can count calls instead of sending email. var deliver = func(ctx context.Context, e *orders.OrderPlacedEvent) error { rlog.Info("sending receipt", "order", e.OrderID, "to", e.Email) return nil } // SendReceipt is safe to run more than once for the same order. func SendReceipt(ctx context.Context, e *orders.OrderPlacedEvent) error { if _, err := db.Exec(ctx, `INSERT INTO receipts (order_id, email) VALUES ($1, $2) ON CONFLICT (order_id) DO NOTHING`, e.OrderID, e.Email, ); err != nil { return err } var sent bool if err := db.QueryRow(ctx, `SELECT sent_at IS NOT NULL FROM receipts WHERE order_id = $1`, e.OrderID, ).Scan(&sent); err != nil { return err } if sent { return nil // a redelivery of a message we already handled } if err := deliver(ctx, e); err != nil { return err // retried with backoff, then dead-lettered } _, err := db.Exec(ctx, `UPDATE receipts SET sent_at = now() WHERE order_id = $1`, e.OrderID) return err }

The subscriber test calls the handler twice with the same message, the way a redelivery would, and counts the sends:

// email/email_test.go package email import ( "context" "testing" "time" "encore.app/orders" ) func TestSendReceiptHandlesRedelivery(t *testing.T) { calls := 0 orig := deliver deliver = func(context.Context, *orders.OrderPlacedEvent) error { calls++ return nil } t.Cleanup(func() { deliver = orig }) // A unique ID, since tests share one database by default. e := &orders.OrderPlacedEvent{OrderID: time.Now().UnixNano(), Email: "ada@example.com"} for i := 0; i < 2; i++ { if err := SendReceipt(context.Background(), e); err != nil { t.Fatal(err) } } if calls != 1 { t.Fatalf("want 1 delivery for 2 deliveries of the same message, got %d", calls) } }

encore test ./... runs both tests against local Postgres databases that Encore starts for the test run, as described in the testing docs. Two handlers running the same message at the same moment could still both send, so for payments rather than receipts you would lock the row with SELECT ... FOR UPDATE inside a transaction.

For the end-to-end check, encore run starts both services, their databases and the topic locally. A POST /orders from the API explorer in the local development dashboard produces traces for the request with its insert and publish and for the send-receipt handler, so a subscription that never fires shows up as a missing trace instead of a timeout. The OrderingAttribute setting has no effect locally, so ordering is the one Pub/Sub behavior to check in a deployed environment.

When you deploy, the same declarations become SNS topics and SQS queues on AWS or Pub/Sub topics and subscriptions on GCP, with the retry policy and a dead-letter queue configured from the subscription.

The local MCP server exposes those traces to a coding agent, so after adding a subscriber an agent can call the endpoint and check the trace to confirm the handler ran, in the same session where it wrote the code.

Best fit: Backends on AWS or GCP that use Pub/Sub between services and want publisher, subscriber and end-to-end tests running locally against the same topic and subscription definitions that get deployed.

Frequently asked questions

Can you test Pub/Sub locally without a cloud account?

Yes. Publisher and subscriber logic can be tested as plain function calls, and the end-to-end flow can run against a local broker, a cloud emulator or a framework that runs topics locally, such as encore run. Emulators do not support every cloud feature, so check retry and dead-letter behavior against the real service before relying on it.

Should subscriber tests go through the message broker?

No. Most subscriber tests should call the handler directly with a message, including the same message twice, and assert on the database or the side effect. Keep a smaller number of end-to-end tests that publish through the broker to check wiring, serialization and subscriptions.

How do you test that a message handler is idempotent?

Call the handler twice with the same message and assert that the side effect happened once, for example one email sent or one row written. Brokers such as SQS standard queues and Google Cloud Pub/Sub deliver at least once, so a handler that fails this test will eventually double-process in production.

How do I test a Pub/Sub publisher in Go with Encore?

Call the endpoint or function that publishes from a normal Go test, then read et.Topic(OrderPlaced).PublishedMessages() to get the messages published during that test. Run the tests with encore test. Subscriptions are not triggered by messages published in tests, so the publisher is tested on its own.

Deploy to your cloud on AWS and Google Cloud
without managing infrastructure

Encore lets developers and agents define application infrastructure directly in code, then automatically provisions it from local development to production in your AWS or GCP account.

~/orders — claude
Claude Code
Claude Code v2.1.180Opus 5.5~/orders