0014 — GCCP data-plane facts (SPEC-75, stacked on #17)#19
Draft
rducom wants to merge 10 commits into
Draft
Conversation
…C-CONTRACT.md) Stacked sur feat/seeded-rebuild (#17). 12 incréments, un commit/patch chacun : epochs noncés + truncated-au-resize + close consumed|preserve (cbt), incarnation/-ESTALE + registre d'outcomes CANCELED + get_bdevs enrichi + SB étendu + verify intégrée + enveloppes + epoch-auto + pause×rebuild (raid/nvmf). Conformité : mêmes scénarios que SimDataPlane (spdk-csi W1).
…es (SPEC-75 contract)
0014.1 (RPC-CONTRACT §5): bdev_cbt_epoch_open gains an optional opaque nonce
(CP-generated, <=31 chars) stored on the epoch and echoed in get_bdevs — kills
the epoch-id ABA across maintenance rounds; a generation takeover re-stamps it.
0014.2 (§5, T-D6): a base-bdev RESIZE marks every live epoch truncated (the
live bitmap does not cover the growth zone — the delta is a lie). Never
cleared: survives takeovers, dies with the epoch. Routes the CP to D14 full.
0014.3 (§4): bdev_cbt_epoch_close gains mode preserve|consumed. preserve
(default) keeps the H1 restore-unconsumed-delta discipline; consumed discards
the frozen delta under caller certification and REQUIRES a non-empty
rebuild_token (-EPERM otherwise; registry validation lands with 0014.5).
Unknown mode is -EINVAL, never a silent preserve.
0014.6 (cbt part): get_bdevs driver_specific.cbt.epochs[] = {epoch_id, nonce,
state, generation, truncated} — the EpochObservation source, no dedicated poll.
Validated: -fsyntax-only clean (vendored headers + config stub); standalone
suite 27/27; conformance mirror = SimDataPlane scenarios (spdk-csi W1).
…ome registry (CANCELED contract)
- patch 0014: bdev_raid_create requires incarnation; delete/add/remove/
seeded-rebuild accept expected_incarnation => -ESTALE on mismatch (never
best-effort). Unclaimed SB-reassembled raids refuse any expected_incarnation.
- patch 0015: process-wide rebuild outcome registry (survives the raid,
15 min lazy TTL), tokens per attempt (CP-deterministic or auto), full
rebuilds feed the same registry, bdev_raid_get_rebuild_outcomes {token?},
CANCELED contract (destroy/member-removal with rebuild in flight => no
process write after the RPC returns).
- cbt: epoch_close(mode=consumed) now validates the token against the
registry (succeeded+verified) — unusable by design until 0014.8 (verify).
Contract: spdk-csi/docs/RPC-CONTRACT.md §1/§2/§4 (frozen W0 2026-07-13).
…ation in get_bdevs - patch 0016: SB minor 0->1, per-member content_generation/view_epoch carved from reserved bytes. RecordDivergence: survivors gen++ in the SAME SB transaction as the ejection; rebuild-completion/skip_rebuild promotion adopts the survivors' max in the same transaction as the CONFIGURED flip. Extended reassembly restores both for ALL slots (a lagging generation is the durable staleness proof the cold-recovery pass reads). - patch 0017: get_bdevs base_bdevs_list[] gains state (derived precedence failed > write_only > configured > configuring > absent), since (unix seconds of last state flip — CP anti-flap input), generation facts, and the wrapping cbt's live-epoch facts (epoch_nonce/epoch_state/truncated) via the new vbdev_cbt_query_latest_epoch() cross-module query. - cbt: vbdev_cbt_query.h — minimal raid-facing query surface (no internals). Contract: spdk-csi/docs/RPC-CONTRACT.md §3/§8 (frozen W0 2026-07-13).
…IVERGENT on double mismatch) After the copy completes and BEFORE the process concludes (SB CONFIGURED flip and registry seal both gated): sample 64 uniform-stride windows across the copied extent (seed ranges for seeded rebuilds, whole raid otherwise), compare source leg (arbiter) vs target under quiesce_range. First mismatch: re-copy from the arbiter + re-verify (counts in registry bytes, logged as healed). Second mismatch under the same lock: outcome DIVERGENT (-EILSEQ) — no host write can explain it, the copy process is broken. Success: verified=true, which is exactly what bdev_cbt_epoch_close(mode=consumed) demands (§4). Scope: raid1 plain-data (no DIF); anything else completes unverified. Honest coverage (T-C12): probabilistic process-bug detector (~64 MiB sampled); the exhaustive detector is bdev_raid_verify_ranges (0014.12, §10b). Cancellation mid-verify unwinds like the copy loop (CANCELED seal, no writes after delete returns). Contract: spdk-csi/docs/RPC-CONTRACT.md §10a (frozen W0 2026-07-13).
…, verify_ranges (0014 COMPLETE) - patch 0019 (0014.10): the raid opens an auto epoch on every surviving member's cbt at ejection (vbdev_cbt_auto_epoch_open — never an implicit takeover, -EEXIST when an OPEN round already bounds it); the auto nonce is reported via get_bdevs, which is how the CP adopts the round. Without this, the unplanned-loss path degrades to D14 full rebuilds. - patch 0020 (0014.11, DEC-11): explicit audited force-resume — the fence path resumes whatever barrier stands before preempt-and-abort; the broken barrier's own tokened resume reports it, the snapshot saga replays cleanly. - patch 0021 (0014.9): process-wide envelopes — per-class caps x(nominal, maintenance), regime switch, rebuild concurrency bound. Rebuild cap frozen at process alloc; seeded beyond the bound = -EAGAIN; auto-attach admitted loudly. Default 0 = unlimited: a failed set is a CP incident. - patch 0022 (0014.12): bdev_raid_verify_ranges — exhaustive divergence detector, 1 MiB LBA-locked chunks, verify-envelope paced, reports (<=128 ranges + exact count + truncated flag) and never repairs; registry mirrors progress under the token. All 12 increments of the 0014 GCCP data-plane facts are now implemented. Contract: spdk-csi/docs/RPC-CONTRACT.md (frozen W0 2026-07-13).
When the oldest epoch is stuck OPEN (opened but never frozen/consumed) and wedges new epoch_open with -ENOSPC, emit a specific diagnostic naming the stale_backend_id and pointing at the CP node-id-vs-observer-lvol-id key mismatch (§4.5). Additive log only; no behaviour change.
…e regen The series was format-mixed (format-patch mboxes vs raw diffs), which broke git am and made the documented regen round-trip unable to reproduce the current bytes. patches.sh regen now emits one raw git diff per commit, with the commit subject (NNNN-name) as the filename — a regen of an untouched series is byte-stable. git apply (Dockerfile + check/apply) is the only consumer and is unchanged. patches.sh check: 24/24 green on a pristine pinned clone.
…ability
- 0022 (amended): bdev_raid_verify_ranges becomes bi-mode. raid1 keeps the
copy-compare walk; raid5f gets a SYNDROME mode — XOR of all k+1 members'
strips must be zero per stripe (rotating parity position irrelevant; no
raid5f internals, generic geometry only). All members readable or -EAGAIN
('repair first' — m=1 leaves nothing to check against); ranges must be
full-stripe aligned; per-stripe report, report-only; envelope pacing
charged on the k+1 strips actually read; LBA lock / outcome registry /
incarnation reused. Result gains a 'mode' field.
- 0023 (new): degraded-service counters in get_bdevs driver_specific.raid —
reconstruct_reads_absent / reconstruct_reads_error (0006 fallback) /
degraded_write_stripes (F-e violation gauge) / last_degraded_ts. Relaxed
atomics on the I/O paths; process-rebuild reconstructs deliberately not
counted (repair work, not degraded host service).
- 0024 (new): spdk_dd propagates bdev I/O completion errors — upstream
ignores 'success' in all three bdev callbacks, so dd exited 0 while every
write was rejected (found by vec-smoke); a silently failed READ is worse
(stale buffer written out as data).
- RPC-CONTRACT: new 'Erasure coding — raid5f' section (syndrome contract,
-EAGAIN semantics, verify-at-seal is EXPLICIT: 0018 completes raid5f
processes UNVERIFIED by design, INV-EC-5 seals via syndrome verify).
- Dockerfile: honest module comments (no raid6/raid10 anywhere in SPDK;
raid5f = XOR single parity, not Reed-Solomon) + image label loses the
fictional RDMA claim (built without --with-rdma, SF-11/SF-2).
- scripts/vec-smoke.sh: single-node pre-flight on file-backed AIO (2+1,
strip 64K): full-stripe readback byte-identical; 4K write rejected by
write_unit discipline; pristine syndrome 0-divergent; parity-strip
corruption detected per stripe; rebuild_ranges repairs; degraded refuses
-EAGAIN; 0023 counters visible. ALL GREEN 9/9. Oracle note: spdk_dd exit
code pre-0024 was not trustworthy — readback + bdev error message are.
Validated: patches.sh check 24/24 on pristine clone; docker build (builder
stage) green; vec-smoke ALL GREEN in the built image.
…SPEC-75G G1) raid5f only accepts full-stripe writes (EC-3) and remote initiators are MDTS-bounded (AM-5), so the bench writer must run INSIDE the target pod. Patch 0024 makes spdk_dd's exit code honest, which the bench oracles rely on.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Contrat gelé :
spdk-csi/docs/RPC-CONTRACT.md(2026-07-13). Plan des 12 incréments :docs/0014-GCCP-FACTS.md. Un commit/patch par item ; la suite de conformité SimDataPlane (spdk-csi W1) et ce fork partagent les mêmes scénarios. Le labelbuild-imagessera posé quand les premiers incréments C seront là.