8 Common System Design Problems
Eight system design problems with common patterns for scale, availability, and data flow.
Many production incidents look unique in the postmortem, but the underlying design problems are usually familiar. Systems tend to fail around hotspots, retries, stale state, missing isolation, or unsafe change. Learning the pattern matters because the best solution is often architectural, not heroic.
1. Hot keys
A small number of records can attract a disproportionate amount of traffic. One celebrity profile, one trending product, or one shared configuration key can overload a cache node, shard, or lock. Common mitigations include replication, request coalescing, local caching, and redesigning the key space so extreme popularity does not collapse onto one resource.
2. Thundering herd
When a popular cache entry expires, or a failing dependency starts recovering, many clients can retry at once. The recovery traffic then becomes worse than the original failure. Jittered TTLs, exponential backoff, and single-flight request collapsing help because they spread work over time instead of letting every caller stampede simultaneously.
3. Single points of failure
One primary database, one queue broker, or one load balancer can turn a routine fault into a full outage. Redundancy helps only if failover is real. That means health checks, tested cutover procedures, and clear ownership of which component is authoritative after a failure. An unused standby is not resilience. It is hope stored on another machine.
4. Dual writes
Trouble starts when an application writes to the database and publishes an event as two separate steps. If the first succeeds and the second fails, state diverges. The outbox pattern is a common fix: commit the business change and the intent to publish atomically, then let a separate relay deliver the event. This keeps external side effects aligned with durable state.
5. Unbounded queues
Queues protect interactive latency by absorbing bursts, but they become dangerous when they silently grow forever. Large backlogs hide overload, increase retry age, and can turn transient incidents into day-long recovery exercises. Capacity limits, dead-letter queues, consumer lag alerts, and admission control keep a queue from becoming a storage layer nobody intended to run.
6. Noisy neighbours
Shared workers, thread pools, or multi-tenant nodes let one workload starve another. A cheap batch job can damage a premium customer flow if they compete for the same resource without limits. Isolation, quotas, priority classes, and separate pools for critical paths keep one class of traffic from consuming all available concurrency.
7. Stale caches
Caching improves speed by reusing work, but it also creates a second source of truth for a while. Problems appear when teams add a cache without deciding who invalidates it, how long data can be stale, and whether updates must be visible immediately. Good cache design starts with ownership and freshness rules, not with a default TTL copied from a blog post.
8. Incompatible deploys
Rolling deployments assume old and new versions can coexist briefly. That fails when a schema change is destructive, an event shape changes incompatibly, or a client expects a field that no longer exists. Expand-then-contract migrations, version-tolerant consumers, and additive API changes let deployments happen gradually instead of as all-or-nothing leaps.
The deeper pattern
These problems share a theme: local mistakes become system-wide incidents when there is no containment. Good design adds small protective boundaries such as quotas, retries with backoff, backward-compatible contracts, queue caps, and explicit ownership of freshness. Observability then makes those boundaries visible before users discover them first.
Reliable systems are rarely built from perfect components. They are built from ordinary components surrounded by practical limits, safe defaults, and failure paths that were planned before the incident arrived.