Redis Caching Patterns for Production: Penetration, Breakdown, Avalanche
Caching isn't "add it and it's fast"
I've seen this conversation too often: "The API is slow? Add a Redis cache." Then the cache goes in, the problem isn't solved, and a bunch of weirder bugs appear: data updated but not reflected, caches all expiring at midnight and taking down the DB, or someone dragging the DB down with a non-existent ID.
Caching is an accelerator, not magic. It introduces consistency, invalidation, and avalanche as new problems. This article covers the patterns that actually bite in production, with copy-paste code.
1. Three read/write strategies
Cache Aside — the most common
Read: check cache first; on hit return directly; on miss query DB, write to cache, return. Write: update DB first, then delete the cache (not update it).
// read
async function getUser(id: string) {
const cached = await redis.get(`user:${id}`);
if (cached) return JSON.parse(cached);
const user = await db.users.find(id);
if (user) await redis.set(`user:${id}`, JSON.stringify(user), "EX", 300);
return user;
}
// write: update DB + delete cache
async function updateUser(id: string, data: Partial<User>) {
await db.users.update(id, data);
await redis.del(`user:${id}`); // delete, not set
}
Why "delete" rather than "update" the cache? Under concurrent writes, updating cache then DB can leave a stale value in cache; deleting cache lets the next read go to source and naturally get the latest. The cost is a brief data gap, but it's safer.
Read/Write Through
The app only talks to the cache; the cache syncs with the DB. Reads and writes go through the cache layer, transparent to the caller. Good when treating the cache as "primary storage," but higher implementation complexity.
Write Behind
Writes land only in cache; the cache asynchronously flushes to DB in batches. Best performance, but highest data-loss risk — if the process crashes, unwritten data is gone. Avoid unless consistency requirements are very low.
2. Cache penetration: querying a key that doesn't exist
An attacker or dirty data repeatedly queries id=-1, id=99999999. It's not in cache (DB has nothing either), so every request hits the DB. At volume, that's a DoS.
Fix 1: cache the null value
On a DB miss, also store a short-TTL null marker in cache.
const user = await db.users.find(id);
if (!user) {
await redis.set(`user:${id}`, "NULL", "EX", 60); // cache null for a bit
return null;
}
Keep the null TTL short (e.g. 60s) so a real insert isn't shadowed by a stale "doesn't exist."
Fix 2: bloom filter
Put all valid IDs in a bloom filter; before querying, ask it "might this ID exist?" If not, return immediately — never touch cache or DB.
// pseudo-code
if (!bloom.mightContain(id)) return null; // definitely absent
const cached = await redis.get(`user:${id}`);
// ...
A bloom filter has a small false-positive rate ("might exist" when it doesn't) but never reports an existing item as absent. Enough for penetration defense.
3. Cache breakdown: one hot key expires exactly now
Unlike avalanche, breakdown is a single extremely hot key (e.g. a homepage hero product) expiring at the exact moment, so a flood of requests hits the DB simultaneously.
Fix: mutex lock (singleflight)
The first request that finds the cache empty acquires a lock, queries DB, refills cache; others wait or retry briefly.
async function getHotProduct(id: string) {
const cached = await redis.get(`product:${id}`);
if (cached) return JSON.parse(cached);
const locked = await redis.set(`lock:${id}`, "1", "EX", 5, "NX");
if (!locked) {
await sleep(50); // didn't get the lock, wait and retry
return getHotProduct(id);
}
try {
const product = await db.products.find(id);
await redis.set(`product:${id}`, JSON.stringify(product), "EX", 300);
return product;
} finally {
await redis.del(`lock:${id}`); // release
}
}
This is the singleflight idea — only one request per key goes to source at a time. In Go, golang.org/x/sync/singleflight does exactly this.
Fix 2: logical expiration
Don't give the key a physical TTL; store an expireAt inside the value. On read, if "logically expired," return the old value while asynchronously triggering a refresh. Users never wait for source — best experience, but briefly read stale data.
4. Cache avalanche: mass keys expire at the same moment
If every cache TTL is 300s and they were written together, they all expire together — instantly every request penetrates to the DB, which gets flooded.
Fix 1: randomize TTL
const ttl = 300 + Math.floor(Math.random() * 60); // jitter 300~360s
await redis.set(key, value, "EX", ttl);
One line avoids collective expiry.
Fix 2: multi-level cache
Local cache (LRU Map, Caffeine) + Redis + DB. If Redis dies or all expire, the local layer still absorbs most traffic.
function get(key: string) {
const local = localCache.get(key);
if (local) return local; // layer 1
const redisVal = redis.get(key);
if (redisVal) { localCache.set(key, redisVal); return redisVal; } // layer 2
return dbQueryAndFill(key); // layer 3
}
Fix 3: HA + rate limit + degrade
Make Redis itself master/sentinel/cluster to avoid a single point. Add rate limiting (token bucket) and degradation (return default/static) at the entry to protect the DB under load.
5. Consistency: cache and DB drift apart
Under Cache Aside there's a classic race: request A reads a miss → queries DB → about to write cache; meanwhile request B updates DB and deletes cache; then A writes the old value back. Cache is now stale.
Mitigations:
- Delayed double-delete: after updating DB, delete cache, then delete again a few hundred ms later to clear such "stale refill."
- Subscribe to binlog: use Canal etc. to subscribe to DB binlog and evict cache asynchronously. Cleanest — cache invalidation is decoupled from business.
- Accept eventual consistency: most businesses don't need strong cache/DB consistency; a reasonable TTL lets it expire naturally.
6. Memory and eviction policy
Redis is an in-memory DB with limited memory. Configure maxmemory and maxmemory-policy:
| Policy | Behavior |
|---|---|
noeviction |
error on full write (default, dangerous) |
allkeys-lru |
evict least-recently-used among all keys |
allkeys-lfu |
evict least-frequently-used among all keys (4.0+, more accurate) |
volatile-lru |
evict only among keys with TTL |
allkeys-random |
evict randomly |
Don't ship with default noeviction. For caching, usually allkeys-lru or allkeys-lfu. If some keys must never be lost (hot config), give them no TTL and use a volatile-* policy.
7. Hot keys and big keys
- Hot key: a single key with extreme QPS (e.g. hero product). Risk: one node's CPU pegged. Fix: local cache + key sharding (split one dataset into
product:{id}:1~:Nreplicas to distribute). - Big key: a single value too large (e.g. a giant JSON of the whole product list). Risk: slow serialization, blocking, big network packets. Fix: split, compress, or cache only needed fields. Use
redis-cli --bigkeysfor periodic scans.
8. Pipeline, batching, and Lua atomic ops
Multiple round-trips have network overhead. Batch with Pipeline:
const pipeline = redis.pipeline();
for (const id of ids) pipeline.get(`user:${id}`);
const results = await pipeline.exec();
For atomic "read-modify-write" (e.g. decrement stock), use a Lua script to avoid race conditions:
-- decrement stock: only if > 0
local stock = tonumber(redis.call('GET', KEYS[1]))
if stock > 0 then
redis.call('DECR', KEYS[1])
return 1
end
return 0
9. Walkthrough: product-detail caching architecture
A high-concurrency product page usually stacks like this:
- CDN / static: page skeleton, images.
- Local cache: hot products, millisecond, zero network.
- Redis: product master data (JSON or Hash), TTL with random jitter; stock as a separate string with Lua decrement.
- Database: source of truth.
- Anti-penetration: product IDs in a bloom filter.
- Anti-breakdown: mutex lock or logical expiration for hot products.
- Anti-avalanche: random TTL + local cache fallback + rate limit.
Skeleton:
async function getProductDetail(id: string) {
if (!bloom.mightContain(id)) return null; // anti-penetration
const local = localCache.get(id);
if (local) return local; // layer 1
const redisVal = await redis.get(`product:${id}`);
if (redisVal) {
localCache.set(id, redisVal);
return JSON.parse(redisVal);
}
const product = await loadWithLock(id); // miss → mutex backfill (sec 3)
return product;
}
FAQ
Q1: Why does Cache Aside "delete" rather than "update" the cache?
Under concurrent writes, writing cache then DB can occasionally leave a stale cache value; deleting cache lets the next read naturally fetch the latest. Brief gap, but safer. For stronger consistency, add delayed double-delete or binlog subscription.
Q2: Won't null caching crush the DB?
A null is only written after actually querying DB and finding nothing, with a short TTL. It defends against "repeatedly querying the same non-existent ID." If an attacker uses many different illegal IDs, you still need the bloom filter.
Q3: How long should TTL be?
Depends on change frequency and business tolerance. Product detail 5~10 min, config can be longer, flash-sale stock must be very short or real-time. Core rule: TTL must have random jitter.
Q4: What if Redis goes down?
The local layer in multi-level cache absorbs some; meanwhile rate-limit and degrade, protect DB with connection pooling. Redis itself uses sentinel/cluster for HA. Don't treat "cache miss" as "must hit DB" — return a slightly stale local copy or a default page.
Q5: Cache and DB dual-write — how to not lose data?
Cache isn't durable; don't expect it not to lose. Critical writes must hit the DB; cache only accelerates reads. Use binlog subscription for async invalidation — more reliable than synchronous dual-write in business code.
Recommended tools
When dealing with cache keys and JSON, these ToolsKu tools come in handy:
- Hash — generate stable fingerprints for cache keys, or hash before a bloom filter
- JSON minify — strip whitespace from JSON you stuff into Redis to save memory
- JSON formatter — inspect whether the JSON in cache is actually correct
The core of cache design is admitting "cache and DB will eventually diverge," then choosing a divergence window you can accept. Rather than chase strong consistency, think through invalidation, jitter, and degradation — that's production-grade caching.
Try these browser-local tools — no sign-up required →