Nov 20, 20267 min read

How to Handle Third-Party API Rate Limits in a Backend

Why outbound calls to CRMs, payment providers and SaaS APIs hit 429 errors, how to read Retry-After and rate limit headers, retries with backoff and jitter, client-side limiters, a shared budget across replicas, and moving the calls onto a queue so bursts wait instead of failing.

Illustration for How to Handle Third-Party API Rate Limits in a Backend

Your own API has rate limits you choose, but the APIs your backend calls have limits someone else chose, such as a CRM that allows 100 requests per 10 seconds. The first sign is usually a batch job that turns into a wall of 429 Too Many Requests and half-synced data.

This guide covers reading the limit signals, retrying correctly, sharing a budget across replicas, and moving the calls behind a queue.

How Do APIs Signal Rate Limits?

The standard signal is status 429 Too Many Requests, defined in RFC 6585, often with a Retry-After header in seconds or as an HTTP date, as RFC 9110 specifies. Many providers also send headers on every response telling you where you stand before you hit the wall. GitHub, for example, returns x-ratelimit-limit, x-ratelimit-remaining and x-ratelimit-reset, and also uses 403 for secondary rate limits, so check the provider's docs rather than assuming 429 is the only code.

Limits come in a few shapes, and the right handling depends on which one you hit:

  • Per time window, such as 100 requests per 10 seconds per account, which a client-side budget can respect.
  • Concurrency limits, such as a maximum number of in-flight requests, which a cap on parallel workers respects.
  • Cost-based limits, where expensive endpoints use more of the budget, which needs the provider's headers to track.

How Should a Backend Retry Rate-Limited Calls?

Retry only what is safe to repeat: reads are fine, and writes are fine when the provider accepts an idempotency key or the operation is naturally idempotent, such as an upsert by external ID. When you retry, honor Retry-After when it is present, otherwise back off exponentially with jitter:

package crmclient import ( "context" "math/rand/v2" "net/http" "strconv" "time" ) // retryDelay returns how long to wait before attempt n (starting at 0). func retryDelay(resp *http.Response, attempt int) time.Duration { if resp != nil { if s := resp.Header.Get("Retry-After"); s != "" { if secs, err := strconv.Atoi(s); err == nil { return time.Duration(secs) * time.Second } if t, err := http.ParseTime(s); err == nil { return time.Until(t) } } } base := time.Duration(1<<min(attempt, 6)) * 250 * time.Millisecond return base/2 + time.Duration(rand.Int64N(int64(base/2))) } func doWithRetry(ctx context.Context, c *http.Client, newReq func() (*http.Request, error)) (*http.Response, error) { for attempt := 0; ; attempt++ { req, err := newReq() if err != nil { return nil, err } resp, err := c.Do(req.WithContext(ctx)) retryable := err == nil && (resp.StatusCode == http.StatusTooManyRequests || resp.StatusCode >= 500) if !retryable || attempt == 4 { return resp, err } delay := retryDelay(resp, attempt) resp.Body.Close() select { case <-time.After(delay): case <-ctx.Done(): return nil, ctx.Err() } } }

Jitter matters more than it looks: without it, every worker that hit the limit at the same second retries at the same second, and the API sees the same spike again.

Why Is a Per-Process Limiter Not Enough?

A token bucket such as golang.org/x/time/rate is the easiest way to stay under a per-second limit, with limiter.Wait(ctx) before each call. It only knows about its own process, though. With four replicas each allowed 10 requests per second, the account sends 40, and autoscaling changes that number without anyone touching the limiter.

A shared counter, usually in Redis, holds the budget per time window, and every instance increments it before calling and backs off when it is over. The other fix moves the calls onto a queue, where the number of consumers, not the number of web replicas, decides the call rate. The queue also helps with the other half of the problem: a request handler that calls a rate-limited API directly fails the user's request when the limit hits, while a queued call waits.

How Do You Build Rate-Limited Calls with Encore?

Encore

Encore is a backend framework with built-in infrastructure, so the queue, the shared counter and the secret for the third-party API are Go declarations in the service that makes the calls, and on deploy they become Pub/Sub, Redis and a secret in your own AWS or GCP account. The example syncs contacts to a CRM that allows 100 requests per 10 seconds: the signup flow publishes an event, and a subscriber with bounded concurrency spends a shared budget before each call.

Set up the app

Get started
brew install encoredev/tap/encore &&
encore app create
Copy

Run in your terminal to get started locally.

~
$ encore app create crm-sync --example=hello-world$ cd crm-sync && mkdir -p crm

The topic decouples signups from the CRM, and the subscription caps in-flight calls per instance and retries with backoff:

// crm/crm.go package crm import ( "bytes" "context" "encoding/json" "errors" "fmt" "net/http" "time" "encore.dev/pubsub" "encore.dev/rlog" "encore.dev/storage/cache" ) type ContactChanged struct { UserID string `json:"user_id"` Email string `json:"email"` Name string `json:"name"` } var ContactChanges = pubsub.NewTopic[*ContactChanged]("contact-changes", pubsub.TopicConfig{ DeliveryGuarantee: pubsub.AtLeastOnce, }) var secrets struct { CRMAPIKey string } // Shared budget: one counter per 10-second window, visible to every instance. var limits = cache.NewCluster("crm-limits", cache.ClusterConfig{ EvictionPolicy: cache.AllKeysLRU, }) var calls = cache.NewIntKeyspace[int64](limits, cache.KeyspaceConfig{ KeyPattern: "crm-calls/:key", DefaultExpiry: cache.ExpireIn(time.Minute), }) const budgetPer10s = 90 // the CRM allows 100, keep headroom for manual tools var errOverBudget = errors.New("crm budget exhausted for this window") var _ = pubsub.NewSubscription(ContactChanges, "sync-to-crm", pubsub.SubscriptionConfig[*ContactChanged]{ Handler: syncContact, MaxConcurrency: 4, RetryPolicy: &pubsub.RetryPolicy{ MinBackoff: 5 * time.Second, MaxBackoff: 5 * time.Minute, MaxRetries: 20, }, }) func syncContact(ctx context.Context, c *ContactChanged) error { window := time.Now().Unix() / 10 n, err := calls.Increment(ctx, window, 1) if err != nil { return err } if n > budgetPer10s { return errOverBudget // nacked, redelivered after backoff } body, _ := json.Marshal(map[string]string{"external_id": c.UserID, "email": c.Email, "name": c.Name}) req, err := http.NewRequestWithContext(ctx, http.MethodPut, "https://api.crm.example.com/v2/contacts/by-external-id/"+c.UserID, bytes.NewReader(body)) if err != nil { return err } req.Header.Set("Authorization", "Bearer "+secrets.CRMAPIKey) req.Header.Set("Content-Type", "application/json") resp, err := http.DefaultClient.Do(req) if err != nil { return err } defer resp.Body.Close() switch { case resp.StatusCode == http.StatusTooManyRequests: rlog.Warn("crm rate limited", "user_id", c.UserID, "retry_after", resp.Header.Get("Retry-After")) return fmt.Errorf("crm returned 429") case resp.StatusCode >= 500: return fmt.Errorf("crm returned %d", resp.StatusCode) case resp.StatusCode >= 400: rlog.Error("crm rejected contact", "user_id", c.UserID, "status", resp.StatusCode) return nil // not retryable, do not block the queue } return nil }

The call is an upsert by external ID, so a redelivered event writes the same contact again rather than creating a duplicate. Any service publishes with crm.ContactChanges.Publish(ctx, &crm.ContactChanged{...}), and a burst of ten thousand signups becomes a queue that drains at the CRM's pace. A 4xx other than 429 is logged and acknowledged, since retrying a rejected payload would only fill the queue.

encore run starts Pub/Sub and an in-memory cache locally, so you can publish a burst of events and watch the counter push work into retries in the local traces. On deploy to your AWS or GCP account, the topic becomes SNS and SQS or Google Pub/Sub, the cache becomes ElastiCache or Memorystore Redis, and the API key comes from your cloud's secret manager. Messages that exhaust their retries land in a dead-letter queue instead of disappearing.

When the CRM changes its limit, the fix is one constant and maybe MaxConcurrency, which an agent can change and test locally in the same diff that touches the handler, with no separate queue or Redis configuration to update.

Best fit: backends that sync data to CRMs, payment providers or other rate-limited SaaS APIs and want queued, bounded calls with a shared budget running in their own AWS or GCP account.

Third-Party Rate Limit Checklist

  • Read the provider's docs for which status codes and headers signal limits.
  • Honor Retry-After, otherwise back off exponentially with jitter.
  • Retry only idempotent calls, and send idempotency keys where supported.
  • Share the budget across replicas with a counter, not per-process limiters.
  • Move calls that can wait onto a queue with bounded concurrency.
  • Acknowledge non-retryable 4xx errors so they do not block the queue.
  • Leave headroom below the published limit for other clients on the same account.

Frequently asked questions

How do I handle 429 Too Many Requests from an API?

Stop sending for the time the Retry-After header gives, or until the reset time in the provider's rate limit headers, then retry the request with exponential backoff and jitter. Only retry requests that are safe to repeat, and send an idempotency key where the provider supports one.

How do I rate limit outgoing API calls across multiple instances?

A limiter inside each process only knows about its own calls, so with several replicas you need a shared budget, such as a Redis counter per time window that every instance increments before calling. Alternatively, put the calls on a queue and cap how many consumers process it at once.

Should I retry immediately after a rate limit error?

No. Immediate retries add load to an API that has already told you it is overloaded and usually fail again. Wait for Retry-After, or back off exponentially with random jitter so many clients do not retry at the same moment.

What is the best way to call a rate-limited API from a web request?

Avoid it for anything that can happen later. Publish an event from the request handler and make the third-party call from a queue consumer with bounded concurrency and retries, so a burst of user traffic waits in the queue instead of turning into 429 errors and failed requests.

Deploy to your cloud on AWS and Google Cloud
without managing infrastructure

Encore lets developers and agents define application infrastructure directly in code, then automatically provisions it from local development to production in your AWS or GCP account.

~/orders — claude
Claude Code
Claude Code v2.1.180Opus 5.5~/orders