Context
docs/guides/produce-and-consume.md:55-56 tells users:
Omit the key and records are distributed across ranges, which maximises parallelism and gives you no ordering between them.
The code does the opposite. Routing hashes the key with :erlang.phash2(key, keyspace_size) (lib/malachi/keyspace.ex:48) and nil is just another term:
iex> :erlang.phash2(nil, 256)
252
It is a constant. So every keyless record in a topic hashes to the same position, lands in the same range, and is serialized by that range's single primary. Instead of maximum parallelism, keyless traffic gets the narrowest possible path, and it is fully ordered rather than unordered.
The impact is a write hotspot exactly for the workload the documentation steers users toward when they do not need ordering, plus a promise the system does not keep.
Plan
Two honest ways out; the PR should pick one and say why.
- Make it true: route a record with no key to a range chosen per record (round-robin over the active ranges, or a random position in the keyspace). This delivers the documented parallelism and the documented absence of ordering. It touches
Malachi.Broker's routing (owning_range_id/3 and the grouping around it) and must keep the single-range fast path correct.
- Make the docs true: state that keyless records share one range, are ordered among themselves, and do not scale out, and point users who want spread at supplying a key (even a random one). Zero risk, and it may be the right answer if the model prefers one rule for every record.
Whichever is chosen, docs/guides/produce-and-consume.md and the diagram above that paragraph must end up matching the behaviour.
Risks and open questions
- Option 1 makes routing non-deterministic for keyless records, which is fine for ordering (there is none to preserve) but changes what a reader sees on a replay; the consume side already tolerates it, since a consumer walks ranges rather than keys.
- Option 1 interacts with the split policy: keyless traffic would suddenly spread its load, which changes what a size or rate trigger observes.
- Check whether any test or benchmark relies on keyless records landing together before changing the routing.
Verification
- A test asserting the chosen behaviour explicitly (either records with no key land in more than one range, or they land in exactly one and the docs say so).
- Full suite,
mix credo --strict, mix dialyzer.
- If option 1 is chosen, a loadtest with keyless records to confirm the hotspot is gone.
Context
docs/guides/produce-and-consume.md:55-56tells users:The code does the opposite. Routing hashes the key with
:erlang.phash2(key, keyspace_size)(lib/malachi/keyspace.ex:48) andnilis just another term:It is a constant. So every keyless record in a topic hashes to the same position, lands in the same range, and is serialized by that range's single primary. Instead of maximum parallelism, keyless traffic gets the narrowest possible path, and it is fully ordered rather than unordered.
The impact is a write hotspot exactly for the workload the documentation steers users toward when they do not need ordering, plus a promise the system does not keep.
Plan
Two honest ways out; the PR should pick one and say why.
Malachi.Broker's routing (owning_range_id/3and the grouping around it) and must keep the single-range fast path correct.Whichever is chosen,
docs/guides/produce-and-consume.mdand the diagram above that paragraph must end up matching the behaviour.Risks and open questions
Verification
mix credo --strict,mix dialyzer.