Rate Limiting Visualized: Fixed Window, Sliding Window & Token Bucket

Rate limiting protects finite capacity and keeps one caller from crowding out everyone else. A useful policy answers three questions: who is limited, how much traffic is allowed, and over what interval.

Use the simulator above to send individual requests or generate traffic automatically. Change algorithms and limits, then watch accepted requests, 429 responses, counters, windows, and tokens evolve.

A limit needs a key

“100 requests per minute” is incomplete without a rate-limit key. A public endpoint might key by IP address. An authenticated API often keys by account, user, API key, tenant, or a combination such as tenant plus operation.

Per-IP limits are easy to deploy but can punish many users behind one NAT and can be bypassed by distributed clients. Per-user limits are fairer after authentication but do not protect the unauthenticated edge. Production systems often layer several scopes.

Fixed window

A fixed-window counter groups requests by a wall-clock interval. For a five-request, one-minute policy, the system stores a count for a key such as user:42:12:00. It is compact and fast.

Its boundary is the weakness:

12:00:59  ✓ ✓ ✓ ✓ ✓
12:01:01  ✓ ✓ ✓ ✓ ✓

Ten requests can pass within two seconds while both minute buckets remain valid. The policy still enforces five requests in each named minute, but not every rolling sixty-second span.

Sliding window

An exact sliding log stores request timestamps and counts only those newer than now - window. It removes the hard-boundary burst and gives intuitive fairness, but active keys can consume more memory and every decision must expire old entries.

A sliding-window counter approximation combines the current and previous fixed buckets with a weight based on elapsed time. It uses bounded state and smooths boundaries, at the cost of approximation.

Token bucket

A token bucket has a capacity and a refill rate. Each accepted request consumes a token. Idle time fills the bucket, so a caller may send a controlled burst and then settles to the refill rate.

capacity = 5 tokens
refill   = 1 token / second
request  = consume 1 token or return 429

Token bucket is often a good fit when short bursts are useful but sustained load must remain bounded. A leaky bucket instead drains work at a controlled rate, which is useful for smoothing but can add queueing latency.

429 and Retry-After

When a policy rejects a request, HTTP 429 Too Many Requests communicates that the client exceeded a limit. A Retry-After header can tell a cooperative client when to try again. Clients should apply jittered backoff and avoid synchronizing a new burst exactly at reset time.

Headers describing limit, remaining capacity, and reset time can improve client behavior, but their exact names vary. Treat server time as authoritative and never assume a client-side counter is a security boundary.

Distributed rate limiting

One process can update an in-memory counter atomically. Multiple regions and instances need shared or partitioned state, atomic operations, and an explicit consistency tradeoff. A central store improves coordination but adds latency and a dependency; local budgets are faster but can allow aggregate overshoot.

At high scale, systems may assign regional quotas, use a fast shared data store, batch updates, or deliberately accept small bounded inaccuracies. The best choice depends on whether the limit protects availability, controls cost, enforces a commercial quota, or mitigates abuse.

Fairness and failure policy

No algorithm is universally best. Consider:

Also decide what happens when the limiter's store is unavailable. Fail open protects availability but may expose the dependency. Fail closed protects the resource or quota but can turn a limiter outage into an API outage.

Production connections

Rate limiting is only one layer. Validate and authenticate requests as described in Secure APIs with Cloudflare Workers, Zod & Rate Limiting. Make retries safe with Idempotency and Retry-Safe APIs. Move slow work behind Background Jobs and Message Queues, while keeping authoritative decisions on the server with Server-Authoritative Architecture.

Key takeaways

• Pick a key and scope before picking an algorithm.
• Fixed windows are cheap but allow boundary bursts.
• Sliding windows improve fairness at a state or approximation cost.
• Token buckets make burst tolerance explicit.
• Return 429 with useful retry guidance.
• Distributed enforcement is a consistency and availability design decision.