Distributed ID Generation

Generating unique IDs sounds trivial until the system becomes distributed. A good ID scheme often needs to satisfy several goals at once: uniqueness across many machines, low latency, compact storage, decent index behaviour, and sometimes rough time ordering. Those goals pull in different directions, which is why there is no single best answer.

Start with the properties you need

The first question is whether the ID only needs to be unique, or whether it also needs to be sortable, hard to guess, short enough for URLs, or partition-friendly for storage. A public payment link may care about unpredictability. A database primary key may care more about insert locality. An event stream may want approximate creation order.

If you skip this step, teams often choose a fashionable ID format and only later discover that it fragments indexes, leaks timing information, or requires central coordination they were trying to avoid.

Centralised generators

The simplest design is a central counter, often implemented with a database sequence or a dedicated ID service. This gives short ordered numbers and makes duplicates easy to reason about. It is a very good fit when one primary database already exists and global ordering matters more than independent generation.

The weakness is coordination. Every writer depends on the same authority. If that authority becomes slow, unavailable, or region-bound, the entire write path inherits the bottleneck. Central counters also create awkward failover questions. After a disaster recovery event, the new primary must continue from a safe range or duplicate IDs become possible.

Random and pseudo-random IDs

UUIDs remove most coordination because each node can generate an identifier locally. That is attractive operationally. A service can keep writing even if other regions or control planes are unavailable.

The tradeoff is storage behaviour. Random identifiers scatter inserts across database indexes, which can increase page splits and cache misses. They are also larger than simple integers. For many systems that cost is acceptable, especially when developer simplicity and decentralisation matter more than perfect index locality.

Time-ordered distributed IDs

Snowflake-style schemes try to balance both worlds. A common 64-bit layout combines a timestamp, a worker or shard identifier, and a per-time-unit sequence number. The result is unique without a network round trip and roughly ordered by creation time.

This works well for distributed systems, but it adds operational rules. Clock discipline matters because time moving backwards can produce collisions or non-monotonic IDs. Worker identifiers must remain unique, which means provisioning and failover need care. Burst traffic also matters. If one worker generates more IDs in a millisecond than the sequence field allows, it must block, spin into the next time unit, or widen the bit allocation.

Related formats such as ULID and KSUID aim for sortable identifiers with friendlier text encoding. They solve slightly different ergonomics, but they face the same underlying tradeoff between decentralised generation, ordering, and entropy.

The storage and security angle

ID format affects databases directly. Sequential IDs improve B-tree locality but can create hot spots in some distributed stores. Random IDs spread load but hurt locality. Predictable IDs also leak information. If order numbers increase one by one, outsiders can infer volume or enumerate resources unless access control is perfect.

So a unique ID generator is really a systems design choice. Coordination buys simpler ordering. Randomness buys independence. Time-based layouts buy a useful middle ground but demand stronger operational discipline. The right scheme is the one whose failure mode your system can tolerate, not the one with the most elegant bit diagram.