Ecommet
Case study

Design a Unique ID Generator at Scale

Design a service that hands out unique, roughly time-ordered 64-bit IDs to thousands of servers at millions per second, with no single point of failure. Covers Snowflake layouts, machine ID assignment, clock problems and ID blocks.

Intermediate 4 min read 3 quiz questions

Jump to the quiz

In a nutshell

  • Generate IDs locally in each service with a Snowflake-style layout; no network call per ID.
  • The only coordination is assigning each generator a unique machine ID, through a lease in etcd or ZooKeeper.
  • Handle clock regression by waiting or failing, and sequence exhaustion by waiting for the next millisecond.
  • Alternative: a ticket server handing out blocks of IDs, simpler but with a central dependency.

Requirements

  • Functional: return a new ID; IDs are unique forever; roughly sortable by creation time.
  • Non-functional: 64-bit numbers (fit a BIGINT column); millions per second across the company; microseconds per ID; no single point of failure.

Estimates

A 41-bit millisecond timestamp lasts about 69 years from the chosen epoch. 10 machine bits allow 1,024 generators; 12 sequence bits allow 4,096 IDs per millisecond each, about 4 million per second per generator. Need more generators? Trade sequence bits for machine bits.

API

A library call, not a network call

Example
long id = ids.next();          // 1823749283749283840
IdParts p = ids.decode(id);    // { time: 2026-10-29T10:42:07.311Z, machine: 37, sequence: 12 }

Data model

The coordination store holds leases: /id-machines/37 → host-a14, expires in 30 s. Each generator renews its lease while running. Nothing else is stored: IDs are computed, not looked up.

High-level design

Local generators with leased machine IDs

Local generators with leased machine IDs Orders service → etcd / ZooKeeper: lease Payments service → etcd / ZooKeeper Chat service → etcd / ZooKeeper Orders service → Databases: rows with IDs Payments service → Databases Chat service → Databases etcd / ZooKeepermachine ID leasesOrders servicemachine 12Payments servicemachine 13Chat servicemachine 14Databasesleaserows with IDs
Each service instance leases a machine ID once at start-up, then creates IDs in memory with no further calls.
Diagram as text
  • Orders service → etcd / ZooKeeper: lease
  • Payments service → etcd / ZooKeeper
  • Chat service → etcd / ZooKeeper
  • Orders service → Databases: rows with IDs
  • Payments service → Databases
  • Chat service → Databases

The generator code is in the Unique ID generation topic (tested in Java and Python).

Deep dives

Clocks

Run NTP in slewing mode so it adjusts the clock gradually rather than stepping it back. If the clock still goes backwards, refuse to generate until time catches up (for small jumps) or take the instance out of service (for large ones). Persisting the last timestamp on shutdown guards against restarts with a slow clock.

Machine ID safety

Two generators with the same machine ID produce duplicates. Leases with TTLs and fencing prevent it: a generator that cannot renew its lease must stop issuing IDs before the lease expires and another instance could take the same number.

The ticket-server alternative

A central service (or a database sequence) hands each client a block of 1,000 consecutive IDs; the client uses them from memory. Simple and compact, ordered within a block, but the server is a dependency (run two with odd and even ranges for availability) and ordering across clients is loose.

Bottlenecks and trade-offs

OptionGoodWatch out
Snowflake-styleNo calls per ID, time-ordered, 64 bitsClock handling, machine ID leases
Ticket server with blocksSimple, compactCentral dependency, loose ordering
UUID v7Zero coordination128 bits

The last five minutes

Common mistakes

  • Making every ID a network call to a central service.
  • Hand-assigned machine IDs that collide after scaling events.
  • Ignoring clock adjustments.

Interview questions

Try answering out loud before you open each one.

You need more than 1,024 generators. What changes?

Re-balance the bits: for example 12 machine bits and 10 sequence bits gives 4,096 generators with 1,024 IDs per millisecond each. Or include a datacentre field. The total stays 63 usable bits; choose the split from the expected number of generators and per-generator rate.

Check yourself

1. What is the only coordination a Snowflake-style design needs?
Show the answer

Answer: Unique machine IDs. Everything else is local.

2. About how long does a 41-bit millisecond timestamp last?
Show the answer

Answer: 69 years. 2^41 ms ≈ 69.7 years.

3. A generator cannot renew its machine ID lease. It should…
Show the answer

Answer: Stop issuing IDs before the lease can be reassigned. Otherwise two generators may share an ID.

Further reading

Your notes

Saved in this browser as you type.

Finished this topic?

Mark it read to track your progress, then carry on with the next one.

Next topic