Vertical vs. Horizontal Partitioning

Partitioning means splitting data so the database does not have to treat one large table as a single unit for storage, I/O, or scaling. The two common strategies split along different axes.

Vertical partitioning splits by columns. Horizontal partitioning splits by rows. They solve different problems, and large systems often use both.

With vertical partitioning, each partition keeps the same primary key but stores a different subset of columns. A common example is moving infrequently used or bulky fields, such as profile photos, long text, or compliance metadata, into a separate table. The hot path then reads a narrower row, which means fewer bytes per page, better cache behaviour, and lower I/O for common queries.

This can help even before a system is truly large. If most requests only need user_id, name, and status, it is wasteful to keep scanning rows that also include settings blobs, audit notes, or large JSON documents. Vertical partitioning reduces that waste.

The tradeoff is that reads become join-heavy when the application needs the full object. It can also push complexity into the write path because related columns now live in different places and must stay consistent. Vertical partitioning is therefore good when access patterns are uneven and well understood.

Horizontal partitioning, usually called sharding, keeps the same columns but divides the rows across multiple partitions or servers. This is the strategy used when one machine cannot comfortably handle the dataset size, write rate, or read traffic.

The key design choice is the routing key. Every row must map to a shard deterministically. Common routing schemes include:

• Range based sharding, such as customer IDs 1 to 1,000,000 on shard A and 1,000,001 to 2,000,000 on shard B
• Hash based sharding, where a hash of a key spreads rows more evenly
• Directory based sharding, where a lookup service tells the application which shard holds a tenant or account

Range sharding preserves order and makes range scans easier, but it creates hotspot risk if new keys cluster in one range. Hash sharding spreads load better, but it makes ordered scans and rebalancing harder. Directory schemes are flexible, but the mapping service becomes critical infrastructure.

Horizontal partitioning gives real scaling headroom, but it introduces operational costs that many teams underestimate. Cross-shard joins are expensive. Global secondary indexes are hard to maintain. Transactions that touch multiple shards become slower and more failure-prone because they need distributed coordination or application-level compensation. Resharding is also not free. Moving live data between shards without downtime requires backfills, dual writes, or careful cutover logic.

A useful rule is this: vertical partitioning optimises how a single logical row is stored and fetched, while horizontal partitioning optimises how the whole dataset is spread across machines.

Choose vertical partitioning when the main problem is wide rows, poor locality, or different column access patterns. Choose horizontal partitioning when the main problem is size or throughput beyond one database node. If the workload needs both narrow hot rows and multi-node scale, combine them deliberately. Start from query patterns and failure modes, not from the diagram. Partitioning changes how the system behaves under growth, maintenance, and incidents, so the operational model matters as much as the schema layout.