Design a Unique ID Generator at Scale
Design a service that hands out unique, roughly time-ordered 64-bit IDs to thousands of servers at millions per second, with no single point of failure. Covers Snowflake layouts, machine ID assignment, clock problems and ID blocks.
In a nutshell
- Generate IDs locally in each service with a Snowflake-style layout; no network call per ID.
- The only coordination is assigning each generator a unique machine ID, through a lease in etcd or ZooKeeper.
- Handle clock regression by waiting or failing, and sequence exhaustion by waiting for the next millisecond.
- Alternative: a ticket server handing out blocks of IDs, simpler but with a central dependency.
Requirements
- Functional: return a new ID; IDs are unique forever; roughly sortable by creation time.
- Non-functional: 64-bit numbers (fit a BIGINT column); millions per second across the company; microseconds per ID; no single point of failure.
Estimates
A 41-bit millisecond timestamp lasts about 69 years from the chosen epoch. 10 machine bits allow 1,024 generators; 12 sequence bits allow 4,096 IDs per millisecond each, about 4 million per second per generator. Need more generators? Trade sequence bits for machine bits.
API
A library call, not a network call
long id = ids.next(); // 1823749283749283840
IdParts p = ids.decode(id); // { time: 2026-10-29T10:42:07.311Z, machine: 37, sequence: 12 }Data model
The coordination store holds leases: /id-machines/37 → host-a14, expires in 30 s. Each generator renews its lease while running. Nothing else is stored: IDs are computed, not looked up.
High-level design
Local generators with leased machine IDs
Diagram as text
- Orders service → etcd / ZooKeeper: lease
- Payments service → etcd / ZooKeeper
- Chat service → etcd / ZooKeeper
- Orders service → Databases: rows with IDs
- Payments service → Databases
- Chat service → Databases
The generator code is in the Unique ID generation topic (tested in Java and Python).
Deep dives
Clocks
Run NTP in slewing mode so it adjusts the clock gradually rather than stepping it back. If the clock still goes backwards, refuse to generate until time catches up (for small jumps) or take the instance out of service (for large ones). Persisting the last timestamp on shutdown guards against restarts with a slow clock.
Machine ID safety
Two generators with the same machine ID produce duplicates. Leases with TTLs and fencing prevent it: a generator that cannot renew its lease must stop issuing IDs before the lease expires and another instance could take the same number.
The ticket-server alternative
A central service (or a database sequence) hands each client a block of 1,000 consecutive IDs; the client uses them from memory. Simple and compact, ordered within a block, but the server is a dependency (run two with odd and even ranges for availability) and ordering across clients is loose.
Bottlenecks and trade-offs
| Option | Good | Watch out |
|---|---|---|
| Snowflake-style | No calls per ID, time-ordered, 64 bits | Clock handling, machine ID leases |
| Ticket server with blocks | Simple, compact | Central dependency, loose ordering |
| UUID v7 | Zero coordination | 128 bits |
The last five minutes
Common mistakes
- Making every ID a network call to a central service.
- Hand-assigned machine IDs that collide after scaling events.
- Ignoring clock adjustments.
Interview questions
Try answering out loud before you open each one.
You need more than 1,024 generators. What changes?
Re-balance the bits: for example 12 machine bits and 10 sequence bits gives 4,096 generators with 1,024 IDs per millisecond each. Or include a datacentre field. The total stays 63 usable bits; choose the split from the expected number of generators and per-generator rate.
Check yourself
Related topics
Further reading
- Snowflake (archived source and README) Twitter, on GitHub
- RFC 9562: Universally Unique IDentifiers (UUIDs) IETF
Your notes
Saved in this browser as you type.
Finished this topic?
Mark it read to track your progress, then carry on with the next one.