What a whole-shard compaction costs - #1349
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Compaction holds the whole shard, and nothing bounds how long.
compact_shard_pagestakes the shard write lock and relocates EVERY live page of every model ontoa freshly rolled slab. No caller passes a budget and the function does not take one, so the work is
whatever the shard happens to hold. A write cannot proceed while it runs, so its duration is the
stall every reader and writer of that shard sees.
Measured, debug build, so a floor rather than a ceiling:
Twenty seconds at twenty thousand objects, reproduced at 20,162 ms on a second run. The per-ref
cost RISES with the shard -- 2.6x between five and twenty thousand -- so it is worse than linear in
the one thing it is unbounded in. A shard with millions of objects does not have a twenty-second
stall; it has a much longer one.
This adds the measurement, not a fix, and says why the fix is not a one-liner. The counterpart in
the design this follows scans a bounded number of slots per round and drives the rest through
callbacks, which is why its stall does not grow with the store. Simply stopping ours early would
not get there: it rolls a fresh slab at the top of every call, so a bounded round would leave its
own half-filled slab behind and trade one stall for slab proliferation. Bounding it properly means
rolling once and continuing to fill that slab across rounds -- campaign state the function does not
have.
The probe is
#[ignore], so it costs nothing on a normal run and can be asked for by name whensomeone changes this path. Its suite is 41 passed, 0 failed.