Caching

What it is

A cache stores copies of frequently read data in faster storage (memory, local SSD, edge) to reduce latency and load on authoritative stores. Write policies and eviction control consistency and memory use.

Write policies

• Write-through: each write updates cache and backing store together. Stronger consistency at cache; higher write latency.
• Write-back (write-behind): writes go to cache first, persistence happens asynchronously. Lower perceived write latency; risk of data loss on crash unless replicated/durable buffer.
• Write-around: writes bypass cache to store; cache fills on read. Avoids polluting cache with one-off writes; next read may miss.

Eviction

• LRU (Least Recently Used): evict the entry unused for the longest time. Good temporal locality; vulnerable to one-off scans flooding working set.
• LFU (Least Frequently Used): evict least popular count. Resists scans better; needs aging to avoid stale hot keys.
• TTL: time-based expiry—simple; combine with LRU/LFU in practice.

Stampede mitigation

When a popular key expires, many requests miss simultaneously and reload the same value from DB.

Mitigations:

• Probabilistic early expiration: refresh before hard TTL with random jitter.
• Single-flight / mutex per key: only one backend computes; others wait or get stale.
• Stale-while-revalidate: return old value while one worker refreshes.
• Request coalescing at cache layer (Redis/Memcached patterns, application locks).

  Without mitigation:  [miss][miss][miss] --> DB spike
  With single-flight:  [miss] --> one rebuild; others get result or stale

When to use

• Read-heavy workloads with repeatable keys (objects, query results, rendered fragments).
• Rate-sensitive downstream services (DB, search); measure with latency-throughput goals.

Alternatives

• No cache: simpler consistency; use scaling and read replicas instead (see replication).
• Materialized views / precomputed tables: heavier but queryable like primary data.
• CDN for HTTP assets (see reverse-proxy-cdn).

Failure modes

• Stale reads: TTL too long or invalidation bugs.
• Thundering herd: see stampede above.
• Memory pressure: wrong eviction → thrashing.
• Cache aside inconsistency: update DB without invalidating cache (classic bug).

Interview talking points

• State consistency model (eventual vs strong) for your write policy.
• Size cache with back-of-envelope: working set, hit ratio, eviction under load.
• Connect to downstream protection: circuit breakers + caching layers.

Related fundamentals

• Back-of-the-envelope estimation
• Latency and throughput