Rate Limiting

429 and Retry-After, the RateLimit-* headers, token-bucket / sliding-window / fixed-window algorithms, client backoff with jitter, and per-user vs per-IP limits.

Rate limiting caps how many requests a client may send in a given window of time. The goals are to prevent abuse and runaway spend, allocate resources fairly, and contain cascading failures. A well-designed limit isn't just a wall — it's a cooperative contract that tells the client how much is left and when it resets, so the client can pace itself.

429 and Retry-After

A request that exceeds the limit gets a 429 Too Many Requests. The key is the Retry-After header telling the client when to try again. Retry-After can be an integer number of seconds (e.g. Retry-After: 30) or an HTTP date. Clients should honor it and wait that long before retrying; ignoring it and hammering immediately can push the reset out further or extend a block.

HTTP/1.1 429 Too Many Requests
Content-Type: application/json
Retry-After: 30
RateLimit-Limit: 100
RateLimit-Remaining: 0
RateLimit-Reset: 30

{"error":"rate_limited","message":"Too many requests, retry in 30s"}

The RateLimit-* headers

Attaching the remaining budget to every response (success and failure alike) lets clients slow down before they ever hit a 429. The headers, now being standardized, are:

Algorithms: what you count

A limiter's accuracy and smoothness come from its algorithm. A fixed window ('100 per minute') resets a counter on clock boundaries — the simplest scheme, but it suffers 'boundary bursts' where requests clustering just before and after the edge let nearly double the limit through in a brief moment. A sliding window continuously measures the trailing N seconds from now, eliminating the boundary burst at higher compute and storage cost. A token bucket refills tokens at a steady rate and spends one per request, allowing bursts while capping the average rate — the most widely used in practice. The related leaky bucket enforces a constant output rate, smoothing traffic.

Client backoff and jitter

A client that receives 429 or 5xx should retry with exponential backoff, widening the interval each time (1s → 2s → 4s → 8s …). But if many clients hit the limit at once and all back off by the same amount, they retry in the same instant — the 'thundering herd' problem. The remedy is jitter, adding randomness to the wait so each client retries at a slightly different time and load spreads across the time axis.

# exponential backoff + full jitter (pseudocode)
base = 1s, cap = 30s
delay = min(cap, base * 2^attempt)
sleep = random(0, delay)   # random between 0 and delay
# if Retry-After is present, honor it first

Per-user vs per-IP limits

What you key the count on is a major design choice. Per-IP limits help for pre-auth traffic (login, signup, public endpoints), but when many users share one IP behind NAT, a corporate network, or a mobile carrier, innocent users get blocked together — and, conversely, an attacker can rotate IPs to evade it. Per-user (or per-API-key) limits are fair for authenticated requests but unavailable before auth. The realistic design layers both: loose IP+endpoint limits before auth, strict user/key limits after. Adding a separate, lower limit for expensive endpoints protects resources with more precision.