Handle rate limits and retries properly

Back off exponentially on 429 while honouring Retry-After, and treat the rate limit as a design constraint rather than an error to swallow.

The problem

The default failure mode is a burst of parallel calls, a wall of 429s, and a retry loop that makes it worse. Rate limits are per plan and enforced per replica, so the fix is to shape your traffic rather than to hammer harder.

What to send

Retry-After

response header

When present, it is authoritative. Sleep for it rather than guessing.

limit

lower

Fewer, better-filtered calls beat many broad ones — the same tactic that saves tokens saves quota.

The code

A retry that behavespython

import time, random, httpx

def search(params, *, attempts=5):
    for attempt in range(attempts):
        r = httpx.get("https://api.unlob.com/search",
                      headers={"x-api-key": "ulb_..."}, params=params)

        if r.status_code == 429:
            # Honour the server's own figure when it gives one.
            wait = float(r.headers.get("Retry-After", 2 ** attempt))
            # Jitter, so a fleet of workers does not resynchronise into
            # the same burst on every retry.
            time.sleep(wait + random.uniform(0, 0.5))
            continue

        if r.status_code >= 500:
            time.sleep(2 ** attempt + random.uniform(0, 0.5))
            continue

        r.raise_for_status()
        return r.json()

    raise RuntimeError("rate limited after %d attempts" % attempts)

The mistake to avoid

Retrying without jitter. A pool of workers that all back off by exactly 2, 4, 8 seconds stays synchronised and arrives together every time, so the burst that caused the 429 is faithfully reproduced. Add randomness.

Frequently asked questions

Are limits enforced per account or per replica?

Per serving replica, so the effective ceiling in production is generally higher than the published number. Treat the published figure as a guaranteed floor and design to it rather than to what you observe.

Should I retry a 4xx?

Only 429. A 400 or 422 means the request is wrong and will be exactly as wrong the second time; a 401 means the key is bad. Retrying those burns quota to receive the same answer.

Try it against your own queries

10,000 requests a month on the free tier, no card. Enough to run a real evaluation set rather than a demo.

Get an API key All recipes