0014 — GCCP data-plane facts (SPEC-75, stacked on #17)

docs/0014-GCCP-FACTS.md

@@ -0,0 +1,30 @@
+# 0014 — GCCP data-plane facts (SPEC-75, contrat gelé W0)
+
+> Contrat normatif : `spdk-csi/docs/RPC-CONTRACT.md` (gelé 2026-07-13). Cette PR (stackée
+> sur #17 seeded-rebuild) livre les faits data-plane que le control-plane GCCP consomme.
+> La suite de conformité de modèle (SimDataPlane, `spdk-csi` W1) et ce fork sont testés par
+> les MÊMES scénarios — une divergence est une CI rouge.
+
+## Incréments (un commit/patch par item, dans cet ordre)
+
+| # | Item | Contrat | Véhicule |
+|---|---|---|---|
+| 0014.1 ✅ | Epochs noncés : `bdev_cbt_epoch_open(nonce, generation)` ; nonce dans get_bdevs | §5 | module/bdev/cbt (arbre) |
+| 0014.2 ✅ | `truncated=true` au resize sous epoch ouvert (phase OPEN incluse) | §5/T-D6 | module/bdev/cbt |
+| 0014.3 ✅ | `bdev_cbt_epoch_close(epoch_id, mode: consumed\|preserve)` — consumed exige un token SUCCEEDED-verified local (-EPERM sinon ; consultation registre effective depuis patch 0015) | §4 | module/bdev/cbt |
+| 0014.4 ✅ (patch 0014) | Incarnation : `bdev_raid_create(..., incarnation)` ; `expected_incarnation` sur tous les RPC engageants ⇒ `-ESTALE` | §1 | patch raid |
+| 0014.5 ✅ (patch 0015) | Registre d'outcomes process-wide (TTL 15 min) : `{token, state RUNNING\|VERIFYING\|SUCCEEDED\|FAILED\|DIVERGENT\|CANCELED, bytes, verified, finished_at}` + `bdev_raid_get_rebuild_outcomes(token?)` ; contrat CANCELED (destroy pendant rebuild ⇒ zéro écriture après retour) ; `epoch_close(consumed)` consulte le registre | §2 | patch raid |
+| 0014.6 ✅ (patch 0017) | get_bdevs enrichi par membre : `{state, since, content_generation, view_epoch, epoch_nonce, epoch_state, truncated}` (pont `vbdev_cbt_query_latest_epoch`) | §3 | patch raid+cbt |
+| 0014.7 ✅ (patch 0016) | SB étendu : `content_generation` + `view_epoch` par membre, MÊME transaction que l'état (éjection = gen++ survivants ; complétion/skip = adoption du max) ; reassembly étendue ; SB minor 0→1 | §8/V-2 | patch raid (0001 discipline) |
+| 0014.8 ✅ (patch 0018) | Verify intégrée au rebuild : K=64 fenêtres stride-uniforme sous quiesce_range (étendue = seed ranges ou raid entier), arbitre = jambe source, re-copie+re-verify avant DIVERGENT (-EILSEQ) ; registre VERIFYING → verified=true (débloque consumed) ; flip CONFIGURED gated | §10a | patch raid |
+| 0014.9 ✅ (patch 0021) | Enveloppes : caps rebuild/verify/relocate ×(nominal, maintenance) + borne de concurrence ; `bdev_raid_set/get_envelopes` ; rebuild cap figé à l'alloc du process ; seeded → -EAGAIN au-delà de la borne, auto-attach admis bruyamment | §6 | patch raid/rpc |
+| 0014.10 ✅ (patch 0019) | Epoch automatique à l'éjection : le raid ouvre un epoch auto (nonce généré, rapporté via get_bdevs) sur le cbt de CHAQUE survivant ; -EEXIST si un OPEN couvre déjà (jamais de takeover implicite) | §12 | patch raid+cbt |
+| 0014.11 ✅ (patch 0020) | Pause H10 × rebuild : jamais d'échec du fait de la pause (structurel : aucun timeout dans le path process, writes parqués côté target) ; force-resume EXPLICITE et audité (`force:true`) au chemin de fence — la barrière percée échoue proprement à SON resume | §13 | patch nvmf (0003) |
+| 0014.12 ✅ (patch 0022) | `bdev_raid_verify_ranges {name, ranges?, token?, expected_incarnation?}` : détecteur exhaustif, chunks 1 MiB sous LBA-lock, cadence = enveloppe verify (figée au dispatch), rapporte {verified/divergent_blocks, divergent_ranges ≤128 + flag truncated} et ne répare JAMAIS (arbitrage = CP R≥3) ; registre alimenté (verifying → succeeded-verified/divergent) ; -EBUSY si process | §10b | patch raid |
+
+Hors périmètre de cette PR : 0014b (enforcement PR) = wrapper C# côté spdk-csi (T-A5 —
+vanilla `bdev_nvme_send_cmd` suffit), livré W3.
+
+## Build
+
+Label CI `build-images` sur la PR ⇒ image `v26.01-prNN-<sha>` à pinner dans spdk-csi.

docs/RPC-CONTRACT.md

@@ -186,11 +186,181 @@ a split-brain across disks.
   after the unquiesce; a crash between the two costs a full rebuild at reboot
   (conservative, safe).
 
+## Member observation & extended superblock (GCCP 0014.6/0014.7, patches 0016/0017)
+
+- **0014.7 — extended superblock (V-2).** SB minor 0→1 (carved from reserved
+  bytes: a minor-0 SB reads back as generation 0 / epoch 0). Per member:
+  `content_generation` — survivors increment it in the SAME SB transaction
+  that records a member's ejection (RecordDivergence); a member completing a
+  rebuild (or skip_rebuild promotion) ADOPTS the survivors' max generation in
+  the same transaction as its CONFIGURED flip. A lagging generation is the
+  durable proof of staleness the cold-recovery pass reads. `view_epoch` is
+  persisted/reassembled here; the view protocol mutates it (W3, 0014b).
+  Reassembly restores both for ALL slots (a FAILED slot keeps its stale
+  generation — that lag is the point).
+- **0014.6 — per-member observation.** `get_bdevs` → `driver_specific.raid.
+  base_bdevs_list[]` gains: `state` (derived, precedence:
+  `failed > write_only > configured > configuring > absent`), `since` (unix
+  seconds of the last observable state flip — the CP's anti-flap input),
+  `content_generation`, `view_epoch`, and — when the member is a cbt bdev
+  tracking a live epoch — `epoch_nonce`/`epoch_state`/`truncated` (the
+  Decider's EpochObservation source; fields absent otherwise). The raid reads
+  the cbt facts via `vbdev_cbt_query_latest_epoch()` (most recently opened
+  live epoch; closed epochs leave the list).
+
+## Envelopes, ejection epochs, fence-resume, verify_ranges (GCCP 0014.9-.12)
+
+- **0014.9 — envelopes (patch 0021).** `bdev_raid_set_envelopes {rebuild|verify|
+  relocate}_{nominal|maintenance}_mb_sec, max_concurrent_rebuilds, regime?` +
+  `bdev_raid_get_envelopes`. CAPS, never shares; data-plane default 0 =
+  UNLIMITED (a failed set is a control-plane INCIDENT, never a silent
+  fallback); unknown regime = -EINVAL. The rebuild cap (under the current
+  regime) is FROZEN at process allocation and takes precedence over the legacy
+  `bdev_raid_set_options` bandwidth. Concurrency: an RPC-driven seeded rebuild
+  beyond the bound is refused -EAGAIN (retryable); the attach-triggered auto
+  rebuild is admitted LOUDLY (refusing would strand an attached member). The
+  relocate cap is stored for the tier module (wiring deferred, documented).
+- **0014.10 — epoch at ejection (patch 0019).** The raid opens an auto epoch
+  (`auto-<ticks>` / nonce `auto<hex>`) on EVERY surviving member's cbt in the
+  ejection path, stale_backend_id = the ejected member. -EEXIST when an OPEN
+  epoch already bounds the round (never an implicit takeover), -ENODEV when
+  the member is not cbt-wrapped — both fine. The auto nonce reaches the CP
+  through get_bdevs (0014.6): that is how the round is adopted.
+- **0014.11 — pause × rebuild (patch 0020).** Structural contract: a paused
+  subsystem QUEUES the rebuild's writes target-side; nothing in the process
+  path times out, so the outcome is never FAILED because of a pause.
+  Force-resume (DÉC-11): `nvmf_subsystem_resume {…, force: true}` breaks any
+  standing barrier — loud (WARNLOG) and audited; the barrier's own tokened
+  resume then reports `resumed:false/barrier_intact:false`, so the
+  group-snapshot saga fails cleanly and replays.
+- **0014.12 — verify_ranges (patch 0022).** `bdev_raid_verify_ranges {name,
+  ranges?:[{start_lba,num_blocks}], token?, expected_incarnation?}` — the
+  EXHAUSTIVE detector (§10a's integrated phase is the probabilistic one).
+  Two modes, selected by the raid LEVEL (reported back as `mode` in the
+  result): **raid1 = copy-compare** (below), **raid5f = syndrome** (see the
+  Erasure-coding section). raid1: first configured leg = implicit arbiter,
+  compared to every other readable leg, 1 MiB chunks each under a
+  channel-owned LBA lock (host writes held + replayed); paced by the verify
+  envelope (frozen at dispatch). REPORTS `{verified_blocks, divergent_blocks,
+  divergent_ranges (≤128, exact count + truncated flag), mode}` and never
+  repairs — arbitration is the CP's (R ≥ 3, T-D2). No ranges = the whole
+  raid; callers drive bounded batches and re-drive, like rebuild_ranges.
+  -EBUSY with a live background process (and aborts cleanly if one appears
+  mid-run); registry mirrors progress under the token (verifying →
+  succeeded+verified on zero divergence / divergent otherwise).
+
+## Erasure coding — raid5f (SPEC-75G F-b/F-d, patches 0022/0023)
+
+The raid5f strate is CP-written only (DÉC-EC-0: no host ever writes a
+raid5f); these contracts serve the `EcImage` lifecycle (seal-verify, periodic
+verify, degraded-service observation).
+
+- **verify_ranges, syndrome mode (patch 0022, F-b).** On a raid5f, the same
+  RPC recomputes the RAID-5 consistency relation per stripe: the XOR of all
+  k+1 members' strips must be zero (the rotating parity position is
+  irrelevant to the check — no raid5f internals involved). Result field
+  `mode: "syndrome"`.
+  - **Preconditions**: ONLINE; no background process (-EBUSY); **every
+    member readable, else -EAGAIN** ("repair first") — with m = 1 a stripe
+    missing one strip has nothing left to check against. A member read
+    error or membership change mid-run also aborts -EAGAIN. Plain-data
+    bdevs only (interleaved md ⇒ -ENOTSUP).
+  - **Ranges** must be full-stripe aligned (`full_stripe_blocks` from
+    get_bdevs, 0008 discipline) — -EINVAL otherwise. No ranges = the whole
+    raid (aligned by construction).
+  - **Reporting** is per STRIPE in raid LBA space; report-only, never
+    repairs. Divergence handling is the CP's: re-fold the stripe from its
+    source, or raise `EcContentDivergence`.
+  - Same LBA-lock, verify-envelope pacing (charged on the k+1 strips
+    actually read), token/outcome-registry and incarnation semantics as the
+    raid1 mode.
+- **Verify-at-seal is EXPLICIT (0018 interplay).** The integrated verify
+  phase (0014.8) is raid1-only and completes a raid5f rebuild process
+  UNVERIFIED (clean no-op, registry `verified=false`). A raid5f is therefore
+  NEVER auto-sealed by a rebuild: the CP must run a syndrome verify_ranges
+  pass to seal (INV-EC-5) — including after `bdev_raid_rebuild_ranges` and
+  after a full rebuild process.
+- **Degraded-service counters (patch 0023, F-d).** get_bdevs
+  `driver_specific.raid` gains `reconstruct_reads_absent` (member missing —
+  vanilla reconstruct-read), `reconstruct_reads_error` (member
+  present-but-erroring — 0006 fallback), `degraded_write_stripes` (stripes
+  written at zero margin; F-e: the CP freezes folds on a degraded EcImage,
+  so nonzero here is a contract-violation gauge), `last_degraded_ts` (unix
+  seconds, 0 = never). Written on the I/O paths with relaxed atomics
+  (single reactor today — D5). These feed `EcRebuildDebt.since` and the
+  "serving degraded since T" incident escalation. The rebuild process's own
+  reconstruct reads are intentionally NOT counted (they are repair work,
+  not degraded host service).
+- **Seeded rebuild / auto-epoch stay raid1-only by design** (0013 refuses
+  -EINVAL; 0019 is gated): EC repair = re-provision the chunk +
+  `bdev_raid_rebuild_ranges` bounded to allocated ranges (thin-preserving,
+  INV-EC-9), or the native full process for a near-full image.
+
+## Incarnation & rebuild outcomes (GCCP 0014.4/0014.5, patches 0014/0015)
+
+- **0014.4 — identity.** `bdev_raid_create {…, incarnation}` (required, ≤63
+  chars): no creation without the creating control-plane incarnation's
+  identity. Engaging RPCs — `bdev_raid_delete`, `bdev_raid_add_base_bdev`,
+  `bdev_raid_remove_base_bdev`, `bdev_raid_start_seeded_rebuild` — accept
+  `expected_incarnation?` and refuse with **-ESTALE** on mismatch (never
+  best-effort execution). Omission is reserved for manual rescue tooling: the
+  control-plane client ALWAYS sends it. Superblock-reassembled raids are
+  *unclaimed* (`incarnation` absent from `get_bdevs`); any engaging RPC that
+  carries `expected_incarnation` against an unclaimed raid is -ESTALE.
+  `get_bdevs` → `driver_specific.raid.incarnation` exposes ownership.
+- **0014.5 — rebuild outcome registry.** Process-wide, survives the raid bdev,
+  terminal entries purged 15 min after finishing (lazy TTL).
+  `bdev_raid_start_seeded_rebuild {…, rebuild_token?}` records the attempt
+  under the CP token (deterministic per attempt — a retry re-opens the same
+  entry); full rebuilds feed the SAME registry under `auto:<raid>:<member>:<ticks>`
+  tokens. `bdev_raid_get_rebuild_outcomes {token?}` returns
+  `[{token, raid_bdev, base_bdev, state, bytes, verified, finished_at}]` with
+  `state ∈ running|verifying|succeeded|failed|divergent|canceled` (lowercase on
+  the wire), `bytes` = bytes actually copied (seeded fast-skip windows do not
+  count), `finished_at` = unix seconds (0 while non-terminal).
+  - **CANCELED contract (T-D5)**: `bdev_raid_delete` (or member removal) with a
+    rebuild in flight ⇒ the process is cancelled, the outcome is `canceled`,
+    and **no process write is emitted after the RPC returns** (the delete reply
+    is gated behind the process fully stopping). The CP deliberately has no
+    `Cancel` action — cleanliness is guaranteed here.
+  - `verified` is set by the integrated verify phase (0014.8, patch 0018):
+    after the copy completes and BEFORE the process concludes (the SB
+    CONFIGURED flip is gated on it), 64 uniform-stride windows across the
+    copied extent (seed ranges for seeded rebuilds, whole raid otherwise) are
+    compared two-legs under `quiesce_range`, arbiter = the source leg. A
+    mismatch is re-copied from the arbiter and re-verified once; a second
+    mismatch under the same lock seals the outcome **DIVERGENT** (-EILSEQ).
+    Success seals `verified=true` — unlocking `epoch_close(consumed)` for the
+    token. Honest scope (T-C12): a probabilistic process-bug detector
+    (~64 MiB sampled); the exhaustive one is `bdev_raid_verify_ranges` (§10b).
+    Raid1 plain-data only; anything else completes unverified. The registry
+    shows `verifying` while the phase runs; re-copied windows count in `bytes`.
+
 ## CBT epochs / rebuild
 
 - `bdev_cbt_epoch_freeze` / `epoch_close` / `epoch_open` (higher generation):
   return **-EBUSY** while a rebuild is RUNNING on the epoch (**CBT-1/2/c5**) —
   the control-plane must `bdev_cbt_cancel_rebuild` first.
+- **0014.1** — `bdev_cbt_epoch_open {…, nonce?}`: opaque CP-generated nonce
+  (≤31 chars), stored on the epoch and echoed in `get_bdevs` — kills the
+  epoch-id ABA across maintenance rounds. A generation takeover re-stamps the
+  nonce of the NEW round.
+- **0014.2** — resize under a live epoch ⇒ `truncated: true` on that epoch
+  (get_bdevs): the live bitmap does not cover the growth zone — the delta is a
+  lie, the control-plane must route to a FULL rebuild (D14). Never cleared:
+  survives generation takeovers, dies with the epoch.
+- **0014.3** — `bdev_cbt_epoch_close {…, mode?: "preserve"|"consumed",
+  rebuild_token?}`: `preserve` (default) restores any unconsumed frozen delta
+  to the live bitmap (H1 discipline); `consumed` deliberately DISCARDS it under
+  caller certification and **requires a rebuild_token naming a LOCAL
+  succeeded+verified outcome-registry entry (-EPERM otherwise)** — validated
+  against the 0014.5 registry; `verified` is produced by the integrated verify
+  phase (0014.8, patch 0018). An unknown `mode` is -EINVAL, never a silent
+  preserve.
+- **0014.6 (cbt part)** — `get_bdevs` exposes `driver_specific.cbt.epochs[]`:
+  `{epoch_id, nonce, state: open|frozen|rebuilding|completed|invalid,
+  generation, truncated}` — the control-plane's `EpochObservation` source (no
+  dedicated poll).
 - `bdev_cbt_partial_rebuild` / `bdev_cbt_start_rebuild`: one rebuild per
   (cbt, epoch); a second returns -EBUSY (interpret as "already running", not a
   failure). **CBT-4**: the rebuild FLUSHes the target before COMPLETED.

images/spdk/Dockerfile

@@ -84,8 +84,10 @@ RUN dnf install -y --setopt=install_weak_deps=False --nodocs liburing-devel liba
 #
 # Modules KEPT (required by project features):
 #   bdev_lvol   — thin provisioning, snapshots, clones
-#   bdev_raid   — RAID-1 (Nexus), RAID-0 (striping), RAID-10
-#   raid5f      — Erasure Coding (RAID-5f / RAID-6)
+#   bdev_raid   — raid0, raid1 (Nexus), concat (real levels; no raid6/raid10
+#                 exists anywhere in SPDK — SPEC-75G)
+#   raid5f      — XOR single-parity full-stripe-write raid (m=1) — the
+#                 EcImage capacity strate (SPEC-75G; NOT Reed-Solomon)
 #   bdev_nvme   — NVMe-oF initiator for remote replicas
 #   bdev_malloc — RAM disks for testing / fast scratch
 #   bdev_uring  — io_uring bdev for modern kernels (>= 5.1), SATA SSDs
@@ -210,6 +212,11 @@ RUN dnf install -y --setopt=install_weak_deps=False --nodocs \
 # Binary only (runtime .so provided by Fedora packages above)
 COPY --from=builder /build/spdk/build/bin/spdk_tgt /usr/local/bin/spdk_tgt
 
+# SPEC-75G G1: spdk_dd is the V-EC bench's full-stripe writer (raid5f only accepts
+# full-stripe writes — EC-3 — and remote initiators are MDTS-bounded, so the writer
+# must run INSIDE the target pod). Patch 0024 makes its exit code honest.
+COPY --from=builder /build/spdk/build/bin/spdk_dd /usr/local/bin/spdk_dd
+
 # SPDK RPC scripts (useful for live debugging: rpc.py, spdkcli, etc.)
 COPY --from=builder /build/spdk/scripts/ /usr/local/share/spdk/scripts/
 
@@ -239,7 +246,7 @@ CMD ["-r", "/var/tmp/spdk.sock", "-m", "0x1", "--wait-for-rpc"]
 FROM scratch
 
 LABEL org.opencontainers.image.title="SPDK Data Engine" \
-      org.opencontainers.image.description="Ultra-minimal hardened SPDK for NVMe-oF TCP/RDMA" \
+      org.opencontainers.image.description="Ultra-minimal hardened SPDK for NVMe-oF TCP (built without RDMA — SPEC-75G SF-11)" \
       org.opencontainers.image.vendor="Evariops" \
       org.opencontainers.image.source="https://github.com/Evariops/containers" \
       org.opencontainers.image.licenses="Apache-2.0" \