Caching Visualized: Browser, CDN, API & Database Caches

Why is this API still serving old data? A cache made yesterday's request fast by remembering its answer. After today's write, that same memory can become a distributed consistency problem.

One idea, different constraints

A CPU cache, a mobile in-memory map, a browser cache, a CDN, and an API cache all exploit reuse. They differ in who owns the key, how long data lives, whether copies coordinate, and what stale data costs. CPU Caches & Locality explains the hardware version of the idea.

A hit returns a usable entry. A miss proceeds to the next layer. Misses include keys that were never stored, entries removed by eviction, and entries whose freshness policy expired.

TTL is policy, not invalidation

A time to live bounds how long one cache treats an entry as fresh. It does not remove every copy at the moment the database changes. With a 60-second TTL, a write immediately after cache population can remain invisible for almost 60 seconds.

Choose TTL from the cost of staleness and the cost of origin work. Product descriptions may tolerate minutes; authorization or inventory decisions may require a different design entirely.

const key = "product:" + productId + ":locale:" + locale + ":v3";
const cached = await cache.get(key);
if (cached) return cached;

const product = await database.products.find(productId);
await cache.put(key, product, { ttl: 60 });
return product;

Cache keys are part of correctness

A key must include every input that can change the response: resource identity, tenant, locale, representation version, and sometimes permission scope. Omitting authenticated context can leak one user's response to another. Including unstable values destroys reuse.

CDN cache keys and Vary behavior deserve particular care. Do not cache authenticated or private responses publicly unless the policy and key explicitly make that safe. Normalize untrusted header and query input to reduce cache poisoning risk.

Eviction and capacity

Finite caches evict data even before TTL. Conceptual LRU removes the least recently used entry, but real caches may use approximations, size-aware policies, frequency signals, or provider-specific algorithms. A 99% hit rate can still hide misses concentrated on the most expensive key, so measure miss cost as well as count.

Database changed: which copies are stale?

After an origin write, client memory, persisted client data, browser storage, a CDN point of presence, and an API cache can each hold v1. No single invalidation message is automatically atomic across all layers.

Common strategies are:

• short TTLs when bounded staleness is acceptable;
• versioned keys that make old entries unreachable;
• explicit purge events after a successful commit;
• write-through updates where the cache participates in the write path;
• bypassing shared caches for data whose correctness cannot tolerate staleness.

Invalidating before the database commit risks deleting the cache and then leaving old database state after rollback. Publishing an event after commit can still fail. Transactional outbox patterns help connect durable writes to later invalidation work.

Stale-while-revalidate

Stale-while-revalidate returns an old but acceptable value immediately and refreshes it in the background. It is excellent for read-heavy content where latency matters more than immediate freshness. It is inappropriate when showing the stale answer causes an unsafe decision.

Only one refresher should normally rebuild a hot key. Otherwise expiry can create a cache stampede: hundreds of requests miss together and all execute the expensive query.

const value = await cache.get(key);
if (value?.fresh) return value.data;
if (value?.stale && policy.allowStale) {
  context.waitUntil(singleFlight(key, () => refresh(key)));
  return value.data;
}
return singleFlight(key, () => refresh(key));

Request coalescing, early refresh with jitter, soft and hard expiry, and carefully bounded cache warming reduce stampedes. Warming everything can waste capacity and overload the origin before any user asks for it.

Browser and CDN behavior

HTTP cache behavior comes from request method, status, cache directives, validators, request headers, shared-cache rules, and intermediaries. Cache-Control: no-store differs from forcing revalidation. ETags and If-None-Match can avoid transferring an unchanged body while still contacting the origin.

CDNs can add surrogate controls, purge APIs, shield caches, and stale-on-error policies. Those features vary by provider. The HTTP Request Lifecycle shows where DNS, TLS, edge, API, and database time appears around these decisions.

Database and query caches

Databases already cache pages in their buffer pool and benefit from the operating system's page cache. Application-level query-result caches are a separate choice and must account for every table and predicate that can invalidate a result. Before adding one, verify whether a better database index solves the actual latency.

Production checklist

• Define the maximum acceptable staleness for each resource.
• Design keys with tenant, authorization, locale, and version boundaries.
• Record hit ratio, miss latency, eviction, origin load, and response age.
• Prevent stampedes with coalescing or controlled refresh.
• Treat purge as a fallible distributed operation.
• Never place secrets or private responses into an incorrectly shared cache.

Key takeaways

• Caching reduces repeated work by creating another copy of state.
• TTL limits freshness; it does not synchronize copies.
• A cache key is a security and correctness boundary.
• Stale-while-revalidate improves latency only where stale answers are acceptable.
• Faster reads increase invalidation and consistency complexity.