corrupt-3.3 is an intermittent divergence and trips the exception ratchet

[timsehn]

corrupt corrupt-3.3 is pinned in test/known_testfixture_divergences.txt, but its outcome varies between runs, so sqlite-regression-storage intermittently fails with:

SQLite regression storage: 1 entry/entries should be updated or removed
  stale/mismatched entry list:
  corrupt corrupt-3.3

Demonstrated on #1958: the bucket failed, and a re-run of the same commit and same job passed. #1954, on the same master base, passed it. Same platform, same TESTFIXTURE_VARIANT: coverage.

The assertion is not a raw byte poke, so 93c481b6c2's reasoning does not directly apply: corrupt-3.2 swaps the rootpage values of t1 and t1i1 through writable_schema, and 3.3 expects INSERT INTO t1 to report "database disk image is malformed". In doltlite rootpage is an iTable number into the prolly catalog, so this is really "is reading a table through an index's tree detected".

Hypothesis for the variability, not yet proven: doltlite_core.c stamps commits with time(0), so the fixture's content hashes differ per run, chunk layout differs with them, and whether the mismatched-tree read trips a validation check follows. If that is right, giving the suites a fixed clock would remove a whole class of content-hash variation rather than just this entry. The gate's current comment ("hexio lands on CTLD bytes") describes its 3.4/3.5 siblings, not 3.3.

Either way the entry cannot be pinned as-is: a flaky assertion trips the ratchet in the "should be removed" direction and produces reds on unrelated PRs, which erodes the property that a red non-perf job is a real signal.

Options: make test commits deterministic so the outcome stabilises; or add an explicit "unstable" disposition that suppresses both ratchet directions for such an entry.

[timsehn]

Hit again on #1967 (a dolt_commit('--amend') change, nothing to do with corrupt.test), same stale/mismatched entry list: corrupt corrupt-3.3 signature.

That is two of the last three PRs red on an unrelated change. The re-run cost is small but the signal cost is not: it trains us to glance past a red sqlite-regression-storage, which is the one bucket where a real storage regression would show up.

[timsehn]

Two hypotheses tested and both falsified, which narrows this usefully.

Not commit timestamps. I suggested doltlite_core.c stamping commits with time(0) as the source of content-hash variation. That does not apply here: corrupt.test's fixture is built by plain SQL before any commit is involved.

Not randstr. The fixture is INSERT INTO t1 VALUES(randstr(100,100)) and friends, which looked like the obvious culprit. It is not — the test harness seeds the PRNG deterministically. Three consecutive local runs of the fixture:

run1: FIXTUREDIGEST 42483ee6c4802f5244796f0261bf1f2e size 4124 FIRSTROW dh<Ihd4S/E?U
run2: FIXTUREDIGEST 42483ee6c4802f5244796f0261bf1f2e size 4124 FIRSTROW dh<Ihd4S/E?U
run3: FIXTUREDIGEST 42483ee6c4802f5244796f0261bf1f2e size 4124 FIRSTROW dh<Ihd4S/E?U

Byte-identical. So the input is deterministic, and the variability is in doltlite's execution, not in the fixture.

What that leaves. For identical input and identical binary to produce different outcomes across runs, the candidates are roughly: a read of uninitialized memory (value depends on heap state), pointer- or address-dependent ordering (ASLR), or something environment-derived leaking into content. corrupt-3.2 swaps t1 and t1i1's rootpage via writable_schema, and in doltlite rootpage is an iTable number into the prolly catalog, so 3.3 asks whether reading a table through an index's tree gets detected — a decision that an uninitialized byte could plausibly tip either way.

Note the sanitizer gap: CI runs asan-ubsan, and ASAN does not detect uninitialized reads — that is MSAN, which is not in the workflow. So an uninitialized-read bug in this path would be invisible to the current sanitizer jobs while producing exactly this symptom.

Revised suggestion. Before adding an unstable disposition, it is worth one MSAN (or valgrind) run over corrupt.test on linux. If it finds an uninitialized read, this stops being a flaky-gate problem and becomes a real bug with a real fix — and possibly explains other content-dependent outcomes in this family, including corrupt4. If it finds nothing, the unstable disposition is the fallback with the search documented.

Caveat: none of this reproduces on darwin, where the gate correctly records that 3.3 passes, so the investigation has to happen in a linux build.

Recurrence count is now three consecutive PRs: #1958, #1967, #1968.