Nov 17, 202616 min read

IoT Device Command Backend: Queues, Acknowledgements, Retries and Expiry

Design the backend that sends commands and config to devices: a per-device queue that runs one command at a time, acknowledgements with a deadline, retries for lost acks, commands that expire, superseded config and an append-only history, with what AWS IoT Core already provides and a TypeScript build with Encore.

Illustration for IoT Device Command Backend: Queues, Acknowledgements, Retries and Expiry

Telemetry flows from devices to the cloud and tolerates loss, since the next reading replaces the last one. Commands go the other way and do not: a thermostat setpoint, a door unlock or a pump stop has to reach one device, run once, and report back, and the person who pressed the button wants to know whether it worked.

The ways this goes wrong are specific to devices: a device is offline for an hour and comes back to a queue of stale commands, an ack is lost on a flaky cellular link and the retry runs the command twice, or two config changes arrive in the wrong order and the older one wins. This guide designs a command backend for those cases, compares it with what AWS IoT Core already gives you, and builds it in TypeScript with Encore.

It is the other half of the IoT telemetry ingestion backend, which covers readings coming in.

The command backend at a glance

JobPatternAWS IoT CoreYou build (by hand or with Encore)
Queue per deviceOrdered, one in flightMQTT persistent sessions queue QoS 1 messages for an offline deviceCommand table with status, order and expiry
AcknowledgementDevice reports a result within a deadlineCommands and Jobs track execution statusAck endpoint, deadline per command
Retry and timeoutRe-deliver a lost ack, then give upJobs retry config, Commands TIMED_OUTSweep job with an attempt limit
ExpiryStale commands are never deliveredSession expiry drops queued messages, not by command meaningexpires_at per command
ConfigNewest value winsDevice Shadow desired and reported stateSupersede older queued commands of the same name
Audit trailWho asked for what and what happenedStatus and job execution historyAppend-only events tied to your users

What AWS IoT Core gives you

AWS IoT Core is the default for fleets of constrained devices, and it covers the delivery side of this. Devices connect over MQTT with an X.509 certificate each, and persistent sessions store QoS 1 messages for a disconnected device and send them on reconnect. The broker keeps those sessions for an hour by default, a quota you can raise, and discards the queued messages after that.

On top of the broker there are three features for sending work to devices. Device Shadow holds desired and reported state, so a config change is a new desired value and the device reconciles when it connects. Jobs run remote operations across one or many devices, with rollout, abort, timeout and retry configuration, which is the right tool for firmware updates. Commands send a single instruction and track an execution through CREATED, IN_PROGRESS, SUCCEEDED, FAILED, REJECTED and TIMED_OUT, where a timeout detected by AWS stays open to a later device report.

IoT Core has no idea which of your users may unlock which door, that an unlock request is meaningless after two minutes, that a newer setpoint makes an older queued one wrong, or that support needs to see "Ana asked, device rejected: door ajar" next to the customer's account. Those rules and that history live in your backend whichever transport you use, and devices that speak HTTPS through a gateway, rather than MQTT, need the queue itself as well.

Commands with a status, an expiry and a key

Each command is a row with the device, the name, a JSON payload, who requested it and a status. The caller supplies a request_id, and UNIQUE (device_id, request_id) turns a double tap or a retried API call into the same command instead of two.

Every command has an expires_at, set from a TTL the caller chooses. Ten seconds is right for "unlock now", an hour for "reboot at next contact", and a week for a config value. The poll query never delivers an expired command, and the sweep marks leftovers expired and reports them.

Config commands set supersede, which marks any older queued command with the same name superseded in the same transaction. A device that was offline while someone changed a setpoint three times receives only the latest value, which is the behaviour a Device Shadow gives you for state.

One in flight, acks with a deadline

Commands for a device run in the order they were sent, one at a time. A partial unique index on device_id WHERE status = 'delivered' makes a second in-flight command impossible at the database level, and the poll locks the device row so two concurrent polls from the same device cannot both claim one. A poll that arrives while a command is in flight returns that same command, which covers a response lost on the way to the device.

Delivery sets ack_deadline to the command's ack_timeout, 30 seconds by default. The device acks with succeeded, failed or rejected, and repeating an ack returns the stored status without another event.

Retries, timeouts and duplicate execution

A sweep every minute puts deliveries that missed their deadline back to queued while they are under max_attempts, and marks them timed_out after the last attempt. A late ack is still accepted after a timeout or a re-queue, since the device may have run the command and lost only the reply, and the final status is the one the device reported.

Retries mean a device can receive a command it has already run. The device has to treat the command ID as an idempotency key, keeping the IDs of recent commands it executed and acking a repeat without running it again. A pump that starts twice is usually harmless and a door that toggles twice is not, so this belongs in the device firmware for every command that is not naturally idempotent.

An audit trail per command

command_events gets a row for every transition, queued, superseded, delivered, retry, and the final status, with the actor (a user ID, the device or the sweeper) and detail such as the device's reason for rejecting. A trigger rejects updates and deletes, and each event is written in the same transaction as the status change it records, so the history cannot disagree with the command row.

How do you build it by hand on AWS?

  1. Store commands and events. RDS for PostgreSQL with the schema below, and an API on ECS Fargate or Lambda with a role, security groups and the connection string in Secrets Manager.
  2. Deliver. HTTPS devices poll the API, and for MQTT devices the API publishes to a per-device topic through the IoT data plane, and an IoT rule forwards acks from a response topic to the ack endpoint or a queue.
  3. Sweep. An EventBridge Scheduler rule invokes the sweep every minute.
  4. Report outcomes. Final statuses go to SNS, and your notification service reads them from an SQS queue.

Each of those pieces needs its own IAM policy and deploy step, and the IoT rule, the scheduler and the topic are separate resources to keep in step with the command names in code.

How do you build it with Encore?

Encore

Encore brings infrastructure from code to the command service: the database, the outcomes topic and the one-minute sweep are declared in the same TypeScript file as the endpoints devices call. Deploying to your own AWS account provisions RDS, and SNS with SQS for the topic, as the AWS infrastructure docs list, provisions an IAM role for the service, and the Encore platform calls the sweep on schedule without extra cloud resources for cron. The same code deploys to GCP with Cloud SQL and Pub/Sub, and it sits next to IoT Core rather than replacing it, receiving acks from an IoT rule when devices speak MQTT.

Application code passes through Encore and infrastructure guardrails to resources provisioned in your own AWS or GCP account.
Encore provisions the infrastructure your application needs in your own AWS or GCP account.

A coding agent adding a new command type, say a firmware rollback with a longer ack timeout, works in one migration and one file, and encore run gives it Postgres and local Pub/Sub to poll and ack against before the change is reviewed.

Set up the app

Get started
brew install encoredev/tap/encore &&
encore app create
Copy

Run in your terminal to get started locally.

~
$ encore app create device-commands --example=ts/hello-world$ cd device-commands && mkdir -p commands/migrations

Create commands/encore.service.ts with export default new Service("commands") from encore.dev/service, and put the schema in commands/migrations/1_commands.up.sql:

CREATE TABLE devices ( id TEXT PRIMARY KEY, token_hash TEXT NOT NULL UNIQUE, -- SHA-256 of the device's API token last_seen TIMESTAMPTZ ); CREATE TABLE commands ( id TEXT PRIMARY KEY, device_id TEXT NOT NULL REFERENCES devices (id), request_id TEXT NOT NULL, -- caller's idempotency key name TEXT NOT NULL, -- e.g. 'set_config', 'reboot', 'unlock' payload JSONB NOT NULL, requested_by TEXT NOT NULL, status TEXT NOT NULL DEFAULT 'queued' CHECK (status IN ( 'queued', 'delivered', 'succeeded', 'failed', 'rejected', 'timed_out', 'expired', 'superseded')), attempts INT NOT NULL DEFAULT 0, max_attempts INT NOT NULL DEFAULT 3, ack_timeout INTERVAL NOT NULL DEFAULT '30 seconds', ack_deadline TIMESTAMPTZ, expires_at TIMESTAMPTZ NOT NULL, created_at TIMESTAMPTZ NOT NULL DEFAULT now(), finished_at TIMESTAMPTZ, UNIQUE (device_id, request_id) ); -- At most one command in flight per device, which keeps them in order. CREATE UNIQUE INDEX commands_one_in_flight ON commands (device_id) WHERE status = 'delivered'; CREATE INDEX commands_queue ON commands (device_id, created_at) WHERE status = 'queued'; -- Every state change, append-only. CREATE TABLE command_events ( id BIGSERIAL PRIMARY KEY, command_id TEXT NOT NULL REFERENCES commands (id), at TIMESTAMPTZ NOT NULL DEFAULT now(), status TEXT NOT NULL, actor TEXT NOT NULL, -- a user, the device, or 'sweeper' detail JSONB ); CREATE FUNCTION command_events_append_only() RETURNS trigger LANGUAGE plpgsql AS $$ BEGIN RAISE EXCEPTION 'command_events is append-only'; END $$; CREATE TRIGGER command_events_no_change BEFORE UPDATE OR DELETE ON command_events FOR EACH ROW EXECUTE FUNCTION command_events_append_only(); -- Outcome events to publish, written in the same transaction as the status change. CREATE TABLE command_outbox ( id BIGSERIAL PRIMARY KEY, command_id TEXT NOT NULL REFERENCES commands (id), device_id TEXT NOT NULL, status TEXT NOT NULL );

The commands service

send is an internal endpoint (expose: false) that your user-facing API calls after checking that the user may control the device, as described in the API calls docs. Devices authenticate with a per-device token in X-Device-Token, stored as a hash:

// commands/commands.ts import { api, APIError, Header } from "encore.dev/api"; import { CronJob } from "encore.dev/cron"; import { Topic } from "encore.dev/pubsub"; import { SQLDatabase } from "encore.dev/storage/sqldb"; import { createHash, randomUUID } from "node:crypto"; // Postgres locally with `encore run`; RDS or Cloud SQL in your own account when deployed. const db = new SQLDatabase("commands", { migrations: "./migrations" }); export interface CommandOutcome { commandId: string; deviceId: string; status: "succeeded" | "failed" | "rejected" | "timed_out" | "expired"; } // Other services subscribe to tell the user what happened to their command. export const outcomes = new Topic<CommandOutcome>("command-outcomes", { deliveryGuarantee: "at-least-once", }); const FINAL = ["succeeded", "failed", "rejected"] as const; type Result = (typeof FINAL)[number]; interface Command { id: string; name: string; payload: Record<string, unknown>; status: string; } // Internal: your app's API checks that the user may control the device, // then calls this. A retry with the same requestId returns the same command. export const send = api( { expose: false }, async (p: { deviceId: string; requestId: string; name: string; payload: Record<string, unknown>; requestedBy: string; ttlSeconds: number; supersede: boolean; // replace older queued commands with the same name }): Promise<Command> => { if (p.ttlSeconds < 10 || p.ttlSeconds > 7 * 24 * 3600) { throw APIError.invalidArgument("ttlSeconds must be between 10 seconds and 7 days"); } await using tx = await db.begin(); const created = await tx.queryRow<Command>` INSERT INTO commands (id, device_id, request_id, name, payload, requested_by, expires_at) VALUES (${randomUUID()}, ${p.deviceId}, ${p.requestId}, ${p.name}, ${JSON.stringify(p.payload)}::jsonb, ${p.requestedBy}, now() + make_interval(secs => ${p.ttlSeconds})) ON CONFLICT (device_id, request_id) DO NOTHING RETURNING id, name, payload, status`; if (!created) { const existing = await tx.queryRow<Command>` SELECT id, name, payload, status FROM commands WHERE device_id = ${p.deviceId} AND request_id = ${p.requestId}`; return existing!; } await event(tx, created.id, "queued", p.requestedBy, { name: p.name }); if (p.supersede) { for await (const old of tx.query<{ id: string }>` UPDATE commands SET status = 'superseded', finished_at = now() WHERE device_id = ${p.deviceId} AND name = ${p.name} AND status = 'queued' AND id <> ${created.id} RETURNING id`) { await event(tx, old.id, "superseded", p.requestedBy, { by: created.id }); } } await tx.commit(); return created; }, ); interface DeviceAuth { token: Header<"X-Device-Token">; deviceId: string; } async function checkDevice({ token, deviceId }: DeviceAuth): Promise<void> { const hash = createHash("sha256").update(token).digest("hex"); const row = await db.queryRow` UPDATE devices SET last_seen = now() WHERE id = ${deviceId} AND token_hash = ${hash} RETURNING id`; if (!row) throw APIError.unauthenticated("unknown device or token"); } // The device polls for its next command. It gets one only when nothing else // is in flight, so commands run in the order they were sent. export const next = api( { expose: true, method: "POST", path: "/devices/:deviceId/commands/next" }, async (p: DeviceAuth): Promise<{ command: Command | null; ackWithinSeconds: number }> => { await checkDevice(p); await using tx = await db.begin(); // Lock the device row so two concurrent polls cannot each claim a command. await tx.exec`SELECT 1 FROM devices WHERE id = ${p.deviceId} FOR UPDATE`; const inFlight = await tx.queryRow<Command & { secs: number }>` SELECT id, name, payload, status, extract(epoch FROM ack_deadline - now())::int AS secs FROM commands WHERE device_id = ${p.deviceId} AND status = 'delivered'`; if (inFlight) { // A repeat poll before the ack, e.g. after a dropped response, gets the same command. const { secs, ...command } = inFlight; return { command, ackWithinSeconds: Math.max(secs, 0) }; } const cmd = await tx.queryRow<Command & { secs: number }>` UPDATE commands SET status = 'delivered', attempts = attempts + 1, ack_deadline = now() + ack_timeout WHERE id = ( SELECT id FROM commands WHERE device_id = ${p.deviceId} AND status = 'queued' AND expires_at > now() ORDER BY created_at LIMIT 1 ) RETURNING id, name, payload, status, extract(epoch FROM ack_timeout)::int AS secs`; if (!cmd) return { command: null, ackWithinSeconds: 0 }; await event(tx, cmd.id, "delivered", `device:${p.deviceId}`, null); await tx.commit(); const { secs, ...command } = cmd; return { command, ackWithinSeconds: secs }; }, ); // The device reports the result. Repeating the same ack is a no-op. export const ack = api( { expose: true, method: "POST", path: "/devices/:deviceId/commands/:commandId/ack" }, async (p: DeviceAuth & { commandId: string; result: string; detail?: string }): Promise<{ status: string }> => { if (!(FINAL as readonly string[]).includes(p.result)) { throw APIError.invalidArgument("result must be succeeded, failed or rejected"); } await checkDevice(p); await using tx = await db.begin(); const done = await tx.queryRow<{ status: string }>` UPDATE commands SET status = ${p.result}, finished_at = now() WHERE id = ${p.commandId} AND device_id = ${p.deviceId} -- A late ack still counts: after a timeout, or after the sweep re-queued it. AND (status IN ('delivered', 'timed_out') OR (status = 'queued' AND attempts > 0)) RETURNING status`; if (!done) { const cur = await tx.queryRow<{ status: string }>` SELECT status FROM commands WHERE id = ${p.commandId} AND device_id = ${p.deviceId}`; if (!cur) throw APIError.notFound("no such command for this device"); await tx.commit(); await flushOutcomes(p.commandId); return { status: cur.status }; } await event(tx, p.commandId, p.result, `device:${p.deviceId}`, p.detail ? { detail: p.detail } : null); await tx.exec` INSERT INTO command_outbox (command_id, device_id, status) VALUES (${p.commandId}, ${p.deviceId}, ${p.result})`; await tx.commit(); await flushOutcomes(p.commandId); return { status: done.status }; }, ); // Every minute: retry unacknowledged deliveries, give up after max_attempts, // and expire queued commands that are no longer worth running. export const sweep = api({}, async (): Promise<{ retried: number; finished: number }> => { let retried = 0; const finished: CommandOutcome[] = []; await using tx = await db.begin(); for await (const r of tx.query<{ id: string; device_id: string; status: string }>` UPDATE commands SET status = CASE WHEN status = 'queued' THEN 'expired' WHEN attempts >= max_attempts THEN 'timed_out' WHEN expires_at <= now() THEN 'expired' ELSE 'queued' END, ack_deadline = NULL, finished_at = CASE WHEN status = 'queued' OR attempts >= max_attempts OR expires_at <= now() THEN now() END WHERE (status = 'delivered' AND ack_deadline < now()) OR (status = 'queued' AND expires_at <= now()) RETURNING id, device_id, status`) { await event(tx, r.id, r.status === "queued" ? "retry" : r.status, "sweeper", null); if (r.status === "queued") retried++; else { finished.push({ commandId: r.id, deviceId: r.device_id, status: r.status as CommandOutcome["status"] }); await tx.exec` INSERT INTO command_outbox (command_id, device_id, status) VALUES (${r.id}, ${r.device_id}, ${r.status})`; } } await tx.commit(); await flushOutcomes(); return { retried, finished: finished.length }; }); const _ = new CronJob("command-sweep", { title: "Retry and expire device commands", every: "1m", endpoint: sweep, }); // Full history of one command, for support and audits. export const history = api( { expose: false }, async ({ commandId }: { commandId: string }): Promise<{ events: { at: string; status: string; actor: string }[] }> => { const events: { at: string; status: string; actor: string }[] = []; for await (const e of db.query<{ at: Date; status: string; actor: string }>` SELECT at, status, actor FROM command_events WHERE command_id = ${commandId} ORDER BY id`) { events.push({ at: e.at.toISOString(), status: e.status, actor: e.actor }); } return { events }; }, ); type Tx = Awaited<ReturnType<typeof db.begin>>; async function event(tx: Tx, commandId: string, status: string, actor: string, detail: object | null) { await tx.exec` INSERT INTO command_events (command_id, status, actor, detail) VALUES (${commandId}, ${status}, ${actor}, ${detail ? JSON.stringify(detail) : null}::jsonb)`; } // Publish any committed outcomes that have not yet been sent. A failed publish // leaves the row in place so the next ack or sweep retries it. async function flushOutcomes(commandId?: string) { const pending = commandId ? db.query<{ id: number; command_id: string; device_id: string; status: string }>` SELECT id, command_id, device_id, status FROM command_outbox WHERE command_id = ${commandId} ORDER BY id` : db.query<{ id: number; command_id: string; device_id: string; status: string }>` SELECT id, command_id, device_id, status FROM command_outbox ORDER BY id`; for await (const o of pending) { await outcomes.publish({ commandId: o.command_id, deviceId: o.device_id, status: o.status as CommandOutcome["status"], }); await db.exec`DELETE FROM command_outbox WHERE id = ${o.id}`; } }

The code type-checks against the released encore.dev package and passes encore check, and the SQL was run against Postgres 17: a repeated request_id created nothing, a newer set_config superseded the queued one, a second in-flight command hit the unique index, a delivery past its deadline was re-queued and then accepted a late succeeded ack, a command on its last attempt became timed_out, an expired command was skipped by the poll and marked expired by the sweep, and a DELETE on command_events raised an exception.

The database docs cover transactions, and the cron job docs cover schedules, including that cron jobs do not run locally or in preview environments. Locally, call sweep from the development dashboard to watch a retry happen. outcomes is a Pub/Sub topic with at-least-once delivery, so a subscriber that notifies users should key on the command ID and status, which also handles a timed_out followed by a late succeeded. Each terminal transition gets its own command_outbox row, written in the same transaction as the status change, and flushOutcomes publishes and deletes those rows afterwards. A failed publish is retried on the next ack or sweep, and a late succeeded after a timed_out is a second row rather than a replacement, so both outcomes are delivered.

Deploy to your AWS account

Connect your AWS account, create an environment and run git push encore. Encore builds the image, creates the database and the topic, runs the migrations and starts calling the sweep every minute.

Things to decide early

  • Polling interval and long polling. HTTPS devices that poll every 30 seconds add up to many requests for large fleets. Long polling or MQTT through IoT Core lowers that, with this service still owning the queue rules and the history.
  • Which commands are safe to retry. Set max_attempts to 1 for commands that must never run twice if the device cannot deduplicate, and rely on the user retrying instead.
  • Who may send what. send trusts its caller, so keep the permission check, such as owner, installer or support role, in the user-facing service that calls it, and record the resolved user in requested_by.
  • History retention. command_events grows with every command, so decide how long support and audits need it, and move older rows to cheaper storage on a schedule.

Frequently asked questions

How do you send commands to IoT devices reliably?

Store each command with a status, an expiry and an attempt count, deliver one at a time per device, and require an acknowledgement within a deadline. A scheduled sweep re-queues commands whose ack did not arrive, marks them timed out after the last attempt and expires the ones nobody should run any more, and the device ignores a command ID it has already executed.

What is the difference between AWS IoT Jobs, Commands and Device Shadow?

Device Shadow keeps desired and reported state, which suits configuration that should converge. Jobs run remote operations across one or many devices with rollouts, timeouts and retries, which suits firmware updates. Commands send a single instruction to a device and track its execution status, including a timed-out status when the device does not respond.

Should an IoT command expire?

A command such as unlocking a door or starting a pump should expire, since it is only correct for a short time after a person asked for it, so give each one an expiry and refuse to deliver it afterwards. Configuration that should eventually apply is better modelled as desired state, where a newer value replaces older queued ones.

How do you build an IoT command queue with Encore?

Declare a database for commands and their events, a topic for outcomes and a cron job with every: "1m" in TypeScript. Devices poll an endpoint that claims their next command in a transaction and ack it on another, the sweep retries and expires commands, and deploys provision RDS, SNS and SQS in your own AWS account while the Encore platform runs the schedule.

Deploy to your cloud on AWS and Google Cloud
without managing infrastructure

Encore lets developers and agents define application infrastructure directly in code, then automatically provisions it from local development to production in your AWS or GCP account.

~/orders — claude
Claude Code
Claude Code v2.1.180Opus 5.5~/orders