Design the system that sends webhooks to your customers: an event log, per-endpoint signing secrets in the Standard Webhooks format, retries with exponential backoff, dead endpoints, and replay, built by hand on AWS or with a topic and subscription in code.

Sending a webhook looks like one fetch call after a database write, and that version works until a customer's endpoint is down for an hour, takes 40 seconds to respond, or returns 500 for every event during a deploy.
At that point you need to know which events were delivered, retry the rest without flooding the endpoint when it comes back, prove to the customer that a request came from you, and let them replay what they missed.
This guide designs that system: the tables, the signature format, the retry schedule, and what to do with endpoints that never recover. It builds the delivery path by hand on AWS, then as a topic and subscription in TypeScript.
Keep the event separate from its deliveries: the event is what happened (invoice.paid, with a payload); a delivery is that event going to one endpoint; an attempt is one HTTP request. That split lets you show customers a delivery log, retry one endpoint without touching others, and replay events after an outage.
CREATE TABLE webhook_endpoints (
id UUID PRIMARY KEY,
customer_id TEXT NOT NULL,
url TEXT NOT NULL,
secret_ciphertext BYTEA NOT NULL, -- AES-256-GCM under WebhookKEK, shown once then stored encrypted
disabled_at TIMESTAMPTZ
);
CREATE TABLE webhook_events (
id TEXT PRIMARY KEY, -- msg_..., sent as webhook-id
customer_id TEXT NOT NULL,
type TEXT NOT NULL,
payload JSONB NOT NULL,
created_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE TABLE webhook_attempts (
id BIGSERIAL PRIMARY KEY,
event_id TEXT NOT NULL REFERENCES webhook_events(id),
endpoint_id UUID NOT NULL REFERENCES webhook_endpoints(id),
status INT, -- NULL on timeout or connection error
error TEXT,
created_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
Insert the event in the same transaction as the business change that caused it. If you publish to a queue directly from the request instead, a crash between the commit and the publish loses the webhook. The endpoint secret is stored encrypted under a wrapping key held as an Encore secret, so a database reader cannot recover the signing key.
The Standard Webhooks specification gives receivers one way to verify you: three headers, webhook-id, webhook-timestamp (Unix seconds) and webhook-signature, where the signature is v1, followed by the base64 HMAC-SHA256 of id.timestamp.body with the endpoint's secret. Receivers recompute it over the raw body, compare in constant time, and reject timestamps more than a few minutes old. The webhook-id doubles as their idempotency key, which matters because your retries will deliver some events twice.
import { createHmac } from "node:crypto";
export function signWebhook(secret: string, id: string, body: string, now = Date.now()) {
const timestamp = Math.floor(now / 1000).toString();
const signature = createHmac("sha256", Buffer.from(secret, "base64"))
.update(`${id}.${timestamp}.${body}`)
.digest("base64");
return {
"webhook-id": id,
"webhook-timestamp": timestamp,
"webhook-signature": `v1,${signature}`,
};
}
Treat 2xx as delivered, and retry timeouts, connection errors, 429 and 5xx. Do not retry other 4xx responses: the receiver is telling you the request itself is wrong. Use a short request timeout (10 to 15 seconds) so one slow customer cannot tie up your workers, and spread retries with exponential backoff over hours to a few days; Stripe, for example, retries live-mode events for up to three days.
On AWS the hand-built version is an SQS queue with one message per delivery, a worker on ECS or Lambda that sends the request and records the attempt, a backoff implemented by changing the message's visibility timeout on failure, and a dead-letter queue after the last attempt. A separate job disables endpoints whose deliveries have all failed for several days and emails the customer.
Disabling is kinder than retrying forever: the customer gets an email and a dashboard state instead of a silent backlog. Replay then means selecting events for that customer since a timestamp and enqueueing a delivery for each, which is cheap because the event log already holds the payloads. Cap the replay rate so a recovered endpoint is not flattened by a day of events in one minute.
Encore brings infrastructure from code to this TypeScript service, and the delivery queue is a Pub/Sub topic whose subscription carries the retry schedule. A subscription's retryPolicy sets the minimum and maximum backoff and the number of retries before a message moves to the dead-letter queue, and maxConcurrency caps how many deliveries run at once per instance. encore run starts Postgres locally and runs Pub/Sub in memory, so you can point an endpoint at a local test server and watch the retries in the trace. On deploy to your own AWS account Encore provisions RDS, SNS and SQS and provisions an IAM role for the service, or a service account on GCP.
Run in your terminal to get started locally.
// webhooks/deliver.ts
import { createDecipheriv } from "node:crypto";
import { secret } from "encore.dev/config";
import { Topic, Subscription } from "encore.dev/pubsub";
import { SQLDatabase } from "encore.dev/storage/sqldb";
import { signWebhook } from "./sign";
const webhookKek = secret("WebhookKEK"); // 32-byte key, base64
function decryptEndpointSecret(blob: Buffer): string {
const iv = blob.subarray(0, 12);
const tag = blob.subarray(12, 28);
const data = blob.subarray(28);
const decipher = createDecipheriv("aes-256-gcm", Buffer.from(webhookKek(), "base64"), iv);
decipher.setAuthTag(tag);
return Buffer.concat([decipher.update(data), decipher.final()]).toString("utf8");
}
const db = new SQLDatabase("webhooks", { migrations: "./migrations" });
export interface Delivery {
eventId: string;
endpointId: string;
}
export const deliveries = new Topic<Delivery>("webhook-deliveries", {
deliveryGuarantee: "at-least-once",
});
const _ = new Subscription(deliveries, "send", {
maxConcurrency: 20,
ackDeadline: "30s",
retryPolicy: { minBackoff: "30s", maxBackoff: "6h", maxRetries: 25 },
handler: async ({ eventId, endpointId }) => {
const row = await db.queryRow<{ url: string; secret_ciphertext: Buffer; disabled_at: Date | null; payload: unknown }>`
SELECT e.url, e.secret_ciphertext, e.disabled_at, ev.payload
FROM webhook_endpoints e JOIN webhook_events ev ON ev.id = ${eventId}
WHERE e.id = ${endpointId}`;
if (!row || row.disabled_at) return; // nothing to send
const body = JSON.stringify(row.payload);
const endpointSecret = decryptEndpointSecret(row.secret_ciphertext);
let status: number | null = null;
try {
const res = await fetch(row.url, {
method: "POST",
headers: { "content-type": "application/json", ...signWebhook(endpointSecret, eventId, body) },
body,
signal: AbortSignal.timeout(15_000),
});
status = res.status;
} finally {
await db.exec`
INSERT INTO webhook_attempts (event_id, endpoint_id, status)
VALUES (${eventId}, ${endpointId}, ${status})`;
}
if (status === 429 || status === null || status >= 500) {
throw new Error(`retryable status ${status}`); // retried with backoff, then dead-lettered
}
// 2xx is delivered; other 4xx are recorded and not retried.
},
});
The endpoint that records an event inserts it into webhook_events and then publishes one Delivery per active endpoint for that customer. Replay is the same publish loop over older events. Disabling dead endpoints is a periodic job over webhook_attempts; as an Encore cron job it is Encore platform managed and does not run locally or in preview environments, so call its endpoint directly when testing.
A coding agent can change the retry schedule or add a new event type in one diff and exercise the failure path locally by pointing an endpoint at a server that returns 503.
Best fit: TypeScript or Go backends that send webhooks to customers and want retries, backoff and dead-lettering from the queue itself, with events and attempts in their own AWS or GCP account.
Give each customer endpoint its own random secret and sign every delivery with HMAC-SHA256 over the message ID, a timestamp and the raw body. The Standard Webhooks specification puts these in webhook-id, webhook-timestamp and webhook-signature headers, so receivers can verify the signature and reject old timestamps to stop replays.
Retry on timeouts, connection errors, 429 and 5xx responses with exponential backoff, spreading attempts over hours to a few days, and stop retrying 4xx responses other than 429. Record every attempt, disable an endpoint that has failed continuously for days, and give customers a way to replay events once they fix it.
Deploy to your cloud on AWS and Google Cloud
without managing infrastructure
Encore lets developers and agents define application infrastructure directly in code, then automatically provisions it from local development to production in your AWS or GCP account.