How Big Keys Affect Redis Persistence

In Redis, a big key is a key whose value is unusually large in bytes or cardinality. It might be a 5 MB string, a hash with hundreds of thousands of fields, or a set that has quietly grown far beyond what the application intended. The exact threshold depends on workload, but the operational effect is consistent: one logical key now behaves like a large object that is expensive to serialise, copy, transfer, persist, expire, and delete.

That cost shows up clearly when AOF persistence is enabled. Redis handles commands on the main thread. For every write command, it must update in-memory state and append the command to the AOF buffer. The data then reaches the kernel page cache through write(), and fsync() decides when it is forced to durable storage.

AOF appendfsync always

In always mode, Redis calls fsync() after each write command. This gives the strongest durability of the three AOF settings, but it turns large writes into obvious latency spikes. If one command updates a very large value, the main thread does not just spend time mutating memory. It also waits for durable flush on that path.

For a hot system, that can be catastrophic because Redis is single-threaded for command execution. While one client pushes a large write through fsync(), other clients wait.

AOF appendfsync everysec

In everysec mode, Redis usually lets a background thread perform fsync() about once per second. This reduces the direct durability stall on the main thread and is the common production compromise.

However, big keys still hurt. The main thread must still process the command, materialise the change, and append the command bytes to the AOF stream. Large payloads mean bigger memory copies, larger kernel writes, and more pressure during AOF rewrite. So everysec removes the synchronous flush on each operation, but it does not make oversized values free.

AOF appendfsync no

In no mode, Redis leaves flush timing to the operating system. This avoids Redis-managed fsync() calls on the request path, but the server still issues write() calls and still pays the cost of handling large command payloads. Durability is weaker and latency is less predictable because the kernel decides when dirty pages are flushed.

So the statement “big keys do not affect the main thread in no mode” is too optimistic. They affect it less through explicit fsync waits, but still through command processing and memory movement.

Why big keys are a broader problem than AOF mode

Persistence settings are only part of the story. Big keys also make replication bursts larger, increase failover recovery time, stretch fork-based background work such as AOF rewrite and RDB snapshotting, and create painful deletion spikes. Even reading a big key can block the event loop long enough to show up as tail latency.

They also worsen eviction behaviour. If one key occupies a large fraction of memory, eviction cannot free space smoothly. A few oversized values can distort memory usage more than thousands of ordinary keys.

Practical mitigation

The safest fix is usually to redesign the data model before the key becomes huge. Split one giant aggregate into smaller keys, paginate large collections, cap list or set growth, and avoid storing large opaque blobs in Redis when object storage or a database is a better fit.

Operationally, teams should monitor both key size and element count, not just total memory. A 200 KB string and a set with 2 million members are different shapes of risk. If large keys already exist, migrate them gradually, because rewriting them all at once can create exactly the persistence spikes you are trying to remove.

AOF mode changes how Redis absorbs the pain of big keys. It does not remove the pain. The durable fix is to stop treating an in-memory data structure server like a warehouse for oversized objects.