A machine needs to spit out unique numbers, fast, without talking to any other machine. The Snowflake ID guarantees uniqueness by encoding WHO generated it and WHEN into the number itself.
Three layers make this work:
Layer 1: Identity (the "where"). Datacenter ID + Machine ID. Baked in at startup, never changes. If machine 1 and machine 7 both generate an ID at the exact same millisecond, the IDs are still different because the machine component is different. No coordination needed.
Layer 2: Time (the "when"). Timestamp in milliseconds since a custom epoch. Makes IDs sortable. An ID generated now is always a bigger number than one generated five minutes ago.
Layer 3: Collision handling (the "how many so far"). The sequence number. Exists because time only has millisecond resolution. If one machine generates 500 IDs in the same millisecond, timestamp and identity are identical for all 500. The sequence (0, 1, 2 ... up to 4095) distinguishes them.
All three layers get crushed into a single 64-bit integer using bit shifting.
| 1 bit | 41 bits | 5 bits | 5 bits | 12 bits |
| sign | timestamp | datacenter_id | machine_id | sequence_number |
1 + 41 + 5 + 5 + 12 = 64 bits total. The sign bit is always 0 (keeps the number positive).
Why these specific sizes? It's a tradeoff. The total budget is 63 usable bits.
- 41 bits of timestamp = ~69 years of milliseconds. Twitter launched this around 2010, so that carries them to ~2079. They decided 69 years was long enough.
- 5 + 5 for datacenter + machine = 32 datacenters x 32 machines = 1024 total machines. Enough for Twitter's infrastructure.
- 12 for sequence = 4096 IDs per millisecond per machine. Enough throughput.
If you were building for a smaller company, you might do 3+3 for identity (only 64 machines) and give the extra 4 bits to sequence (65,536 IDs/ms). Different needs, different split. Same total budget.
TIMESTAMP_BITS = 41
DATACENTER_BITS = 5
MACHINE_BITS = 5
SEQUENCE_BITS = 12From these, calculate the max value each slot can hold:
MAX_DATACENTER_ID = (1 << 5) - 1 # 31
MAX_MACHINE_ID = (1 << 5) - 1 # 31
MAX_SEQUENCE = (1 << 12) - 1 # 4095(1 << 5) - 1 is a bit trick for "the biggest number that fits in 5 bits." Same idea as 999 being the biggest number that fits in 3 decimal digits.
def __init__(self, datacenter_id: int, machine_id: int):
# Validate inputs are in range (0-31)
self.datacenter_id = datacenter_id
self.machine_id = machine_id
self.sequence = 0
self.last_timestamp = -1
self._lock = threading.Lock()You pass in which datacenter and machine this generator represents. It validates they're in range, stores them, then sets up two pieces of state plus a lock.
What is state? Any value that changes over time as your program runs. datacenter_id and machine_id are NOT state. They're set once and never change. They're configuration. sequence and last_timestamp ARE state. They change every time you generate an ID. The generator "remembers" what millisecond it last used and how many IDs it generated that millisecond. That memory is state. If you unplugged the machine and lost those two values, the generator wouldn't know where it left off.
The _ before lock: Python convention meaning "this is internal, don't touch it from outside." No enforcement, just a signal to other developers. Same reason _current_timestamp has an underscore. It's a helper, not part of the public interface.
self._lock = threading.Lock(): Creates a lock object and stores it on the instance. Doesn't lock anything yet. It's like buying a padlock and hanging it on a hook. Later, with self._lock: in generate() is what actually locks and unlocks it.
def _current_timestamp(self) -> int:
return int(time.time() * 1000) - self.EPOCHtime.time() gives seconds since Jan 1, 1970 as a float. Multiply by 1000 for milliseconds. int() drops the decimal. Subtract your custom epoch so the number is smaller and your 41 bits last longer.
One line. Called every time you generate an ID.
The outer shell:
def generate(self) -> int:
with self._lock:
timestamp = self._current_timestamp()
# ... case logic ...
return snowflake_idEverything inside with self._lock: means only one thread can be in this method at a time. Others wait. First thing: grab the current timestamp.
Then four cases, built by asking "what could go wrong?" repeatedly:
if timestamp < self.last_timestamp:
raise Exception("Clock moved backwards!")If current time is LESS than the last time you generated an ID, the system clock is broken (maybe synced with a server and jumped back). You'd risk duplicating an ID from a millisecond you already used. So refuse. Crash rather than risk duplicates.
Where does last_timestamp come from? Initialized to -1 in __init__. Updated to the current timestamp inside generate() after the case logic, before bit packing: self.last_timestamp = timestamp. So every call saves its timestamp, and the next call compares against it.
if timestamp == self.last_timestamp:
self.sequence = (self.sequence + 1) & self.MAX_SEQUENCETwo calls in the same millisecond = same timestamp. Increment sequence to tell them apart. First call gets 0, next gets 1, etc.
The & self.MAX_SEQUENCE part: MAX_SEQUENCE is 4095, which is 0b111111111111 (twelve 1s). ANDing with it means if sequence goes past 4095, it wraps to 0 instead of going to 4096. For any number below 4095, ANDing with all 1s is a no-op (does nothing). It only matters at the exact moment of overflow.
What is & (bitwise AND)? Compares two numbers bit by bit. If BOTH bits are 1, result is 1. Otherwise 0. Practical use: it's a mask. Wherever your mask has a 1, the original bit survives. Wherever your mask has a 0, the result is forced to 0.
if self.sequence == 0:
timestamp = self._wait_next_millis(self.last_timestamp)When sequence wraps from 4095 to 0, that means you've used all 4096 slots this millisecond. Wait for the next millisecond. This check only runs inside the if timestamp == self.last_timestamp block, so it won't trigger on the first call of a new millisecond (that hits the else branch instead).
else:
self.sequence = 0Different millisecond from last call. Reset sequence and move on. The simplest case.
How an SWE thinks about building this: Start with the happy path (Case 4), then ask "what if two calls land in the same ms?" (Case 2), then "what if I run out of sequence?" (Case 3), then "what if the clock is broken?" (Case 1). Happy path first, edge cases by asking "yeah but what if..." repeatedly.
snowflake_id = (
(timestamp << 22)
| (self.datacenter_id << 17)
| (self.machine_id << 12)
| self.sequence
)Four values, each shifted left by the total number of bits to its right:
timestamp << 22because 5 + 5 + 12 = 22 bits to its rightdatacenter_id << 17because 5 + 12 = 17 bits to its rightmachine_id << 12because 12 bits to its rightsequencedoesn't shift, it's already at the right edge
The | (OR) combines them because their bit positions don't overlap.
Why self. for three variables but not timestamp? self. means it's stored on the instance and lives between method calls. datacenter_id, machine_id, and sequence all persist. timestamp is a local variable created fresh inside generate() every time. Born, used, discarded within that single call.
def _wait_next_millis(self, last_timestamp: int) -> int:
timestamp = self._current_timestamp()
while timestamp <= last_timestamp:
timestamp = self._current_timestamp()
return timestampKeeps asking "what time is it?" until the clock ticks to a new millisecond. Not running in parallel. Only called when generate() needs it. Everything stops until a new millisecond arrives, then returns that new timestamp.
Why not just sleep? time.sleep(0.001) is imprecise, could sleep 1ms or 15ms depending on the OS. The spin-wait resumes at the very first new millisecond.
Why wait at all? The alternatives are worse: return a duplicate (unacceptable) or raise an error (annoying). Waiting ~1ms is the least bad option. And in practice this almost never fires. You'd need 4096 IDs in one millisecond from one machine.
The exact reverse of packing. Two operations per field: right-shift to move target bits to position 0, then AND with a mask to keep only those bits.
sequence = snowflake_id & ((1 << 12) - 1)
machine_id = (snowflake_id >> 12) & ((1 << 5) - 1)
datacenter_id = (snowflake_id >> 17) & ((1 << 5) - 1)
timestamp = (snowflake_id >> 22) & ((1 << 41) - 1)The shift amounts mirror generate(). Shifted left to pack, shift right to unpack. The mask (1 << N) - 1 keeps only N bits.
Then convert timestamp back to a readable date by adding the epoch back:
epoch_ms = timestamp + self.EPOCH
created_at = datetime.fromtimestamp(epoch_ms / 1000, tz=timezone.utc)Returns a dictionary with all extracted fields plus the human-readable time. Just presentation.
This is NOT decimal concatenation. The concatenation happens in binary.
Example from the student ID exercise:
department 9 = 1001 (4 bits)
year 3 = 011 (3 bits)
student 21 = 10101 (5 bits)
Packed together in binary: 100101110101 = 2421 in decimal. The decimal looks nothing like 9, 3, 21. But the binary IS 9, 3, 21 sitting next to each other. The decimal representation is meaningless for reading the fields. The binary is where the structure lives.
Packing: shift each value left by the number of bits to its right, OR them together.
packed = (department << 8) | (year << 5) | student_numberUnpacking: shift right to move target to position 0, AND with a mask.
extracted_student = packed & ((1 << 5) - 1)
extracted_year = (packed >> 5) & ((1 << 3) - 1)
extracted_department = (packed >> 8) & ((1 << 4) - 1)The Snowflake ID does this with four fields instead of three, bigger bit widths, same pattern.
Basic generation test: Generate 10 IDs, check four properties:
len(ids) == len(set(ids))-- a set removes duplicates, same length means all uniqueall(i > 0 for i in ids)--all()is a built-in that returns True if every item passes. Checks no ID is negative.ids == sorted(ids)-- if the list equals its sorted version, IDs came out time-orderedall(i < 2**63 for i in ids)-- fits in a positive 64-bit integer (sign bit stays 0)
Decode round-trip test: Generate an ID with known inputs (datacenter 7, machine 23), decode it, assert the extracted values match. assert means "this must be true, crash if not."
Stress test: Generate 10,000 IDs, measure throughput, verify uniqueness and ordering at scale.
Two-machine test: Two generators with different machine IDs, generate 1000 each, combine, verify all 2000 are unique. Proves the identity layer works: no coordination needed, uniqueness guaranteed by the bit layout.