SQS FIFO vs Standard Queues: Ordering, Duplicates and Throughput

Standard queues are the SQS default, and for background jobs they are the right one: they scale without configuration and only ask that consumers tolerate the occasional duplicate or out-of-order message. That tolerance breaks for things like balance updates, where applying a debit before the credit it depends on produces a wrong number, and the usual reaction is to switch to FIFO queues without checking what they cost in throughput and failure handling.

This guide compares SQS FIFO and standard queues on ordering, duplicates, throughput and dead-letter behavior, and shows how Encore handles per-key ordering for application events from TypeScript code.

SQS FIFO vs standard at a glance

Throughput figures come from the high throughput FIFO docs and Encore's ordered topics section.

How does Encore handle ordered messages?

Encore brings infrastructure from code to messaging: topics and subscriptions are declared in TypeScript or Go, and Encore provisions SNS and SQS in your own AWS account, or Google Pub/Sub on GCP. Ordering is one field on the topic, orderingAttribute, pointing at the attribute that identifies the entity, so the decision lives next to the event type instead of in a queue name ending in .fifo.

Set up the app

The read model stores each account's balance and the last ledger sequence number applied to it:

-- balances/migrations/1_create_balances.up.sql
CREATE TABLE account_balances (
    account_id TEXT PRIMARY KEY,
    balance_cents BIGINT NOT NULL DEFAULT 0,
    last_seq BIGINT NOT NULL DEFAULT 0
);

CREATE TABLE account_owners (
    account_id TEXT NOT NULL REFERENCES account_balances (account_id),
    user_id TEXT NOT NULL,
    PRIMARY KEY (account_id, user_id)
);

A ledger service publishes entries with a per-account sequence number, and the subscriber applies them to the balance table. Ordering by accountId keeps entries for one account in sequence, and the sequence check covers the two things ordering does not, redelivered messages and an earlier entry that has not been applied yet:

// balances/auth.ts
import { APIError, Gateway, Header } from "encore.dev/api";
import { authHandler } from "encore.dev/auth";
import { secret } from "encore.dev/config";
import { createHmac, timingSafeEqual } from "node:crypto";

const authTokenHMAC = secret("AuthTokenHMAC");

interface AuthParams {
  authorization: Header<"Authorization">;
}

export interface AuthData {
  userID: string;
}

export const handler = authHandler<AuthParams, AuthData>(async (params) => {
  const token = params.authorization?.replace("Bearer ", "") ?? "";
  // Tokens are "<id>.<unix-expiry>.<hex hmac>". In production, verify a JWT
  // against your IdP JWKS (signature, issuer, audience, expiry) and return the subject.
  if (!authTokenHMAC()) throw APIError.unauthenticated("invalid token");
  const parts = token.split(".");
  if (parts.length !== 3) throw APIError.unauthenticated("invalid token");
  const [userID, expStr, sig] = parts;
  const exp = Number(expStr);
  if (!userID || userID.includes("/") || userID.includes(" ") || !Number.isFinite(exp) || exp * 1000 <= Date.now()) {
    throw APIError.unauthenticated("invalid token");
  }
  const expected = createHmac("sha256", authTokenHMAC()).update(`${userID}.${expStr}`).digest("hex");
  const a = Buffer.from(sig);
  const b = Buffer.from(expected);
  if (a.length !== b.length || !timingSafeEqual(a, b)) {
    throw APIError.unauthenticated("invalid token");
  }
  return { userID };
});
export const gateway = new Gateway({ authHandler: handler });

The auth handler verifies a signed, expiring access token, and the balance endpoint uses that identity to authorize the requested account before it reads a row:

// balances/balances.ts
import { api, APIError } from "encore.dev/api";
import { Topic, Subscription, Attribute } from "encore.dev/pubsub";
import { SQLDatabase } from "encore.dev/storage/sqldb";
import { getAuthData } from "~encore/auth";

const db = new SQLDatabase("balances", { migrations: "./migrations" });

export interface LedgerEntry {
  accountId: Attribute<string>; // ordering key
  ownerUserId: string; // recorded on the opening entry
  seq: number; // per-account sequence assigned by the ledger
  amountCents: number; // positive for credits, negative for debits
}

// SNS and SQS on AWS, Google Pub/Sub on GCP.
// Entries for the same account reach each subscriber in publish order.
export const ledgerEntries = new Topic<LedgerEntry>("ledger-entries", {
  deliveryGuarantee: "at-least-once",
  orderingAttribute: "accountId",
});

// Keeps a read model of balances. Each entry is applied only if it is the
// next sequence number for its account, so redeliveries are skipped and a
// gap is retried until the missing entry has been applied.
const _ = new Subscription(ledgerEntries, "update-balance", {
  handler: async (entry) => {
    await db.exec`
      INSERT INTO account_balances (account_id) VALUES (${entry.accountId})
      ON CONFLICT (account_id) DO NOTHING`;
    if (entry.seq === 1) {
      await db.exec`
        INSERT INTO account_owners (account_id, user_id)
        VALUES (${entry.accountId}, ${entry.ownerUserId})
        ON CONFLICT DO NOTHING`;
    }
    const applied = await db.queryRow<{ last_seq: number }>`
      UPDATE account_balances
      SET balance_cents = balance_cents + ${entry.amountCents}, last_seq = ${entry.seq}
      WHERE account_id = ${entry.accountId} AND last_seq = ${entry.seq - 1}
      RETURNING last_seq`;
    if (applied) return;

    const current = await db.queryRow<{ last_seq: number }>`
      SELECT last_seq FROM account_balances WHERE account_id = ${entry.accountId}`;
    if (current && current.last_seq >= entry.seq) return; // already applied
    throw new Error(`entry ${entry.seq} for ${entry.accountId} arrived before ${(current?.last_seq ?? 0) + 1}`);
  },
});

export const balance = api(
  { expose: true, auth: true, method: "GET", path: "/balances/:accountId" },
  async ({ accountId }: { accountId: string }): Promise<{ balanceCents: number }> => {
    const { userID } = getAuthData()!;
    const allowed = await db.queryRow`
      SELECT 1 FROM account_owners
      WHERE account_id = ${accountId} AND user_id = ${userID}`;
    if (!allowed) {
      throw APIError.notFound("account not found");
    }
    const row = await db.queryRow<{ balance_cents: number }>`
      SELECT balance_cents FROM account_balances WHERE account_id = ${accountId}`;
    if (!row) {
      throw APIError.notFound("account not found");
    }
    return { balanceCents: row.balance_cents };
  },
);

The opening ledger entry (sequence 1) writes the account_owners row for ownerUserId in the same subscriber as the first balance insert, so a newly created account is readable by its owner. GET /balances/:accountId requires a signed access token and only returns a row the caller owns, so an unknown or foreign account ID is a 404 rather than a zero balance. encore run provides Postgres and an in-memory topic locally, though ordering has no effect locally, which is one more reason for the sequence check: the handler stays correct whether or not messages arrive in order, and ordering on AWS mostly saves retries. Encore's docs list 300 messages per second for an ordered topic on AWS, and a message that keeps failing holds back later messages with the same key until it succeeds or is dead-lettered, so pair ordered topics with alerts on subscription errors. Encore provisions the topic and queues with the permissions the service needs.

Adding a second consumer, such as a fraud check that also needs entries in order, is another Subscription on the same topic. An agent working in this code sees the ordering choice and the idempotency logic in one file and can change them together.

Best fit: application events where order matters per entity, such as accounts, carts or documents, with fan-out to several consumers and retries managed per subscription.

SQS FIFO queues

FIFO queues preserve the order in which messages are sent within each message group, and every SendMessage call carries a MessageGroupId, usually the entity ID, and a MessageDeduplicationId, or content-based deduplication enabled on the queue. SQS drops a message whose deduplication ID it has seen in the last five minutes, which protects against a producer retrying a send, and AWS describes this as exactly-once processing. It does not protect against your consumer crashing after it applied a change and before it deleted the message, so the handler has to be idempotent anyway.

Throughput is limited per partition: 300 send, receive or delete operations per second, or 3,000 messages per second with batching, and high throughput mode spreads load across partitions by message group, so a few very busy groups stay bottlenecked. While a message from a group is in flight, no other message from that group is delivered, so one slow or failing message stalls its whole group. Dead-letter queues must also be FIFO, and moving a message aside lets the rest of its group continue out of order. Fan-out takes an SNS FIFO topic in front of one FIFO queue per consumer.

Best fit: single-consumer workflows where strict order per entity matters more than throughput, such as applying commands to an account.

SQS standard queues

Standard queues support a nearly unlimited number of API calls per second and deliver every message at least once, occasionally more than once and occasionally out of order. For most background work that is fine, because jobs like sending an email or resizing an image do not depend on each other, and an idempotent consumer handles duplicates. Where order does matter, a version or sequence column in the database, like the one in the Encore example, often gives the same correctness on a standard queue without the FIFO limits.

Best fit: background jobs and events where each message stands on its own.

How to choose

Use Encore when you want per-key ordering and fan-out for application events declared next to the event type in TypeScript or Go, with SNS and SQS provisioned in your own AWS account. Use a FIFO queue for a single consumer that must apply commands strictly in order and can live within the per-partition limits. Use standard queues for everything else, with idempotent consumers and a sequence check where order matters.

Related guides

• SQS dead-letter queues
• Kinesis vs SQS
• SNS vs SQS
• Double-entry ledger backend
• What is Amazon SQS