Imagine GitHub's API with no limits. One developer's runaway script hammers every public endpoint. Suddenly GitHub crawls to a halt for everyone on the planet. That's exactly why rate limiting exists — it's the server politely (and firmly) saying: you get X requests per window, that's your share, try again later. Blow past that and you get the infamous HTTP 429 Too Many Requests.
THE REAL REASONS YOUR API NEEDS THIS
DoS protection — one runaway script shouldn't melt your API for every other paying customer
Fair play — the user hammering you at 10,000 req/s doesn't get to ruin the experience for someone making 3
Your AWS bill — a missing await inside a while(true) loop can cost thousands. Rate limits are a circuit breaker.
Brute-force defense — cracking passwords is a lot less fun when you're limited to 5 attempts per minute per IP
Monetization — free tier gets 100/hr, paid gets 10,000/hr. The rate limit is the product differentiation.
What You Actually Get When You're Throttled
HTTP 429 Too Many Requests
HTTP/1.1 429Too Many RequestsContent-Type: application/json
X-RateLimit-Limit:100← total requests allowed per windowX-RateLimit-Remaining:0← requests left in current windowX-RateLimit-Reset:1750001200← Unix timestamp when window resetsRetry-After:47← seconds until client can retry
{
"error": "rate_limit_exceeded",
"message": "Too many requests. Retry after 47 seconds.",
"limit": 100,
"window": "1 minute"
}
Where Rate Limiting Actually Lives
Client SDK — your own SDK self-throttles with backoff. Polite, but you're trusting the client to behave. Good luck with that.
CDN / WAF (Cloudflare, AWS WAF) — sees the request before your servers even wake up. Blocks bad IPs, rate-limits entire ASNs. Most garbage never reaches your origin at all. Cheapest protection per bad request stopped.
API Gateway (Kong, AWS API GW, nginx) — where most teams actually enforce limits. Per-API-key quotas, tiered plans, burst configs. If you only protect one layer, make it this one.
App Middleware (express-rate-limit, Django REST throttle) — where things get smart. Your CDN doesn't know what a phone number is. Your middleware does: "max 3 failed OTP attempts per number per hour, max 5 password resets per email per day."
Database Layer — PgBouncer's max_client_conn. Your last line of defense and, honestly, an admission that every layer above it failed.
02
Algorithm 1: Fixed Window Counter
Start simple. Chop time into equal buckets — say, 1-minute chunks. Every request bumps a counter. Counter hits 100? You're blocked until the next bucket starts. Counter resets, everyone gets a fresh slate. Simple enough to implement in 15 minutes, which also means simple enough to exploit in 15 minutes.
The Redis Implementation
Redis Implementation
-- Atomic INCR + EXPIRE (set TTL only if key is new)local key = "ratelimit:user:u_12345:" .. math.floor(tonumber(redis.call('TIME')[1]) / 60)
local count = redis.call('INCR', key)
if count == 1then
redis.call('EXPIRE', key, 60) -- only set TTL when key is brand newendif count > 100thenreturn0-- rejectedendreturn1-- allowed
THE EXPLOIT BAKED INTO THE DESIGN
Here's the dirty secret. An attacker sends 100 requests at 11:59:59 — all allowed, window isn't full yet. One second later at 12:00:01, window resets, they fire 100 more — also allowed, fresh window. That's 200 requests in 2 seconds against a "100 per minute" limit. Free doubling, every single minute. Any system using naive fixed window is vulnerable to this — forever.
Interactive: Fixed Window Demo
Fill up Window 1, then hit Boundary Burst Attack — watch 20 requests slip through a 10-request limit by straddling the window edge.
FIXED WINDOW COUNTER — limit: 10 requests/window
Window 1 (0–9s) │ Window 2 (10–19s)
Click "Send Request" — each slot represents one request. Limit = 10 per window.
Window 1: 0/10
Pros
So simple a junior dev can ship it before lunch
One Redis key, one INCR — basically free
Works at massive scale with near-zero overhead
Cons
Boundary burst exploit — double your traffic for free, every minute
Users who hit the limit at 11:59:50 wait longer than users who hit it at 11:59:00
Hard reset at window edge feels arbitrary and punishing
03
Algorithm 2: Sliding Window Log
Instead of a hard window, log the exact timestamp of every request. When a new one arrives, look back exactly 60 seconds — count how many timestamps are in that rolling range. Under the limit? Let it through, add the timestamp. Over the limit? 429. No hard reset. No window edge to exploit. Just a clean, rolling 60-second view of what the user has actually been doing.
Redis Sorted Set Implementation
-- Redis sorted set: score = timestamp, member = request_idlocal key = "swlog:user:u_12345"local now = tonumber(redis.call('TIME')[1]) * 1000 + tonumber(redis.call('TIME')[2]) / 1000local window = 60000-- 60 seconds in mslocal cutoff = now - window
-- Remove expired timestamps
redis.call('ZREMRANGEBYSCORE', key, 0, cutoff)
-- Count remaining (all within the window)local count = redis.call('ZCARD', key)
if count >= 100thenreturn0-- rejectedend-- Add current request timestamp
redis.call('ZADD', key, now, now .. ":req")
redis.call('EXPIRE', key, 60)
return1-- allowed
WHY THE BOUNDARY EXPLOIT DIES HERE
The window is always the last 60 seconds relative to right now — not a calendar minute. Send 100 requests at 11:59:59? They're still in the sorted set at 12:00:01. You're blocked until each one ages out. There's no midnight to exploit. No free doubling. The math simply doesn't have a seam to attack.
THE CATCH: MEMORY GETS EXPENSIVE FAST
Here's where reality bites. You're storing every single timestamp — not just a counter. Scale that up: 1M users × 100 requests/min = 100M sorted set entries in Redis. At ~80 bytes each, that's 8GB just for rate limit tracking. Most startups never hit this ceiling. When you do, it's time to switch algorithms.
Pros
Mathematically accurate — boundary exploit is impossible
Fair and smooth — no sudden resets that feel arbitrary
You can tell users exactly when their oldest request ages out
Cons
Memory scales with traffic, not user count — 8GB at 1M users × 100 req/min
ZREMRANGEBYSCORE on huge sets has a real cost
At serious scale, this is the wrong algorithm — use token bucket
04
Algorithm 3: Leaky Bucket
Picture your API as a bucket with a hole in the bottom. Requests pour in from the top at whatever rate they want. The hole drains at a fixed, constant rate — doesn't matter if a tsunami arrives, outflow stays steady. Bucket fills past capacity? New requests immediately overflow with a 429. Queued ones wait their turn at the drain rate.
The superpower here is traffic shaping, not just limiting. A payment processor doesn't care that you had a burst moment — it'll process your requests, just at its own steady pace. Downstream services see perfectly smooth traffic regardless of what hits the top of the bucket.
Parameters
Bucket capacity = max queue size
Leak rate = requests processed/second
Requests queue in bucket (FIFO)
Overflow = immediate 429
Queued requests = delayed, not rejected
Use Cases
Payment processors (smooth throughput)
Video encoding pipelines
Email sending services (SMTP limits)
Anywhere downstream is rate-sensitive
CDN request smoothing
THE DOWNSIDE: LEGIT BURSTS GET PUNISHED TOO
A user opens a dashboard that fires 10 API calls at once. Your leak rate is 2/second. All 10 go into the queue — first 2 come back fast, next 2 in 1 second, last few take 5 seconds. Your server could have handled all 10 in parallel — but leaky bucket didn't let it. The next algorithm fixes exactly this.
Pros
Downstream sees perfectly flat traffic — never a spike
Great for payment processors, email queues, video encoding
Naturally absorbs client-side bursts without crashing backends
Cons
Punishes legitimate bursts — a dashboard load feels slow
Queued requests accumulate latency the user can see
Queue can grow unbounded if the drain rate is too slow
05
Algorithm 4: Token Bucket
This is the one everyone actually uses — AWS, GitHub, Stripe. Think of it as a prepaid card that refills itself. The bucket fills up with tokens at a steady rate. Each request spends one token. Got tokens? You're through. No tokens? 429. The twist that makes it better than everything else: if you've been idle and tokens stacked up, you can spend them all at once. Bursts are the feature, not a bug.
WHY EVERY MAJOR API USES THIS
With capacity=10 and refill_rate=2/second: your dashboard loads and fires 10 API calls at once — all allowed, you had 10 tokens saved up from being idle. Then you click around at human speed (1-2 actions/minute) and tokens slowly refill. Token bucket matches how users actually behave instead of penalizing the natural way apps batch requests.
Real-World Usage
AWS API Gateway
Token bucket per account. Default: 10,000 RPS burst + 5,000 RPS steady state. Burst tokens refill at 5,000/second.
GitHub API
5,000 requests/hour per token. Roughly 1.38 req/s steady state. Burst to 5,000 if bucket is full.
Stripe API
100 requests/second for most endpoints. Payment endpoints lower. Uses sliding window + token bucket hybrid.
Redis + Lua Atomic Implementation
Token Bucket — Redis Lua Script (atomic)
-- Token Bucket in Redis. Atomic via Lua (no race condition)local key = KEYS[1] -- "tb:user:u_12345"local capacity = tonumber(ARGV[1]) -- max tokens (e.g. 100)local refill_rate = tonumber(ARGV[2]) -- tokens/second (e.g. 10)local now = tonumber(ARGV[3]) -- current Unix ms timestamplocal tokens_requested = tonumber(ARGV[4]) -- usually 1-- Get current state: [tokens, last_refill_time]local state = redis.call('HMGET', key, 'tokens', 'ts')
local tokens = tonumber(state[1]) or capacity
local ts = tonumber(state[2]) or now
-- Refill: add tokens based on elapsed timelocal elapsed = math.max(0, now - ts) / 1000-- seconds
tokens = math.min(capacity, tokens + elapsed * refill_rate)
if tokens < tokens_requested then-- Not enough tokens — save updated count and reject
redis.call('HMSET', key, 'tokens', tokens, 'ts', now)
redis.call('EXPIRE', key, math.ceil(capacity / refill_rate) + 1)
return {0, math.floor(tokens)} -- {allowed, tokens_remaining}end-- Consume tokens and allow
tokens = tokens - tokens_requested
redis.call('HMSET', key, 'tokens', tokens, 'ts', now)
redis.call('EXPIRE', key, math.ceil(capacity / refill_rate) + 1)
return {1, math.floor(tokens)} -- allowed
Interactive Token Bucket Demo
Let the bucket fill up, then burn all 10 tokens at once with a burst. Watch tokens slowly refill. Try overwhelming it — see exactly what a real 429 scenario looks like.
Send requests and watch the token level drop — then wait for refill.
Tokens refill automatically every interval. Watch the bucket fill back up!
Pros
Bursts are allowed — dashboards, app loads feel instant
Average rate is still enforced — no free ride
O(1) per request — Lua script, two Redis keys, done
Industry standard: AWS, GitHub, Stripe all use it
Cons
Slightly more code than fixed window (but not much)
Needs atomic execution — Redis Lua or a compare-and-swap loop
Clock drift across distributed nodes can cause subtle inconsistencies
06
Bonus: Sliding Window Counter — The Best of Both
You want sliding window accuracy but you can't afford to store every timestamp. Here's the trick: you don't have to. Cloudflare and most large-scale rate limiters use this hybrid. Keep two counters — current window and previous window. Apply a weighted formula to estimate where you fall in the sliding window. Two Redis keys per user instead of potentially thousands of sorted set entries. Nearly as accurate, fraction of the cost.
This approximation is off by at most ~0.1% on real traffic patterns. In practice, indistinguishable from exact sliding window — but costs 2 Redis keys instead of up to N sorted set entries where N = your request limit.
07
What Do You Actually Limit On?
Key Type
Redis Key Example
Use When
Weakness
Per IP
rl:ip:203.0.113.1:1750001
Unauthenticated endpoints, public APIs
Shared IPs (NAT, corporate), VPNs
Per User
rl:user:u_12345:1750001
Authenticated APIs, fair usage
Doesn't stop multi-account abuse
Per API Key
rl:key:sk_live_abc:1750001
Developer APIs, SaaS tiering
Leaked keys used to hit limits
Per Endpoint
rl:ep:POST:/checkout:1750001
Sensitive endpoints (login, OTP)
Attacker spreads across endpoints
Composite
rl:u_12345:POST:/checkout:1750001
Most accurate, fine-grained
More Redis keys, more memory
08
Distributed Rate Limiting
Here's a nasty surprise waiting for teams that don't think this through. You've got 3 app servers, each doing their own in-memory rate limiting. A smart client discovers the round-robin pattern and distributes requests across all three: 100 to server A, 100 to B, 100 to C. Every one of them allowed. That's 300 requests against your "100 per minute" limit. Per-process rate limiting is theater. You need one central counter that every server talks to.
❌ The Problem
App Server A sees 100 requests → allows all. App Server B sees 100 requests → allows all. App Server C sees 100 requests → allows all. Result: 300 requests through against a 100 limit.
✅ The Solution
All servers share one Redis counter. INCR rl:user:u_12345:1750001 is atomic — no race condition. Every server checks the same counter.
WHY YOU CAN'T SKIP THE LUA SCRIPT
Check-then-increment as two separate Redis commands has a race condition. Server A checks (count=99, allow). Server B checks (count=99, allow). Server A increments to 100. Server B increments to 101. Both let the request through. Classic TOCTOU. Redis Lua scripts are atomic — nothing touches the key while your script runs. This is non-negotiable if you care about correctness under load.