Skip to content

Publish the debt ledgers, cap three debt classes, and unbreak cargo off Linux - #283

Merged
niveousdragon merged 6 commits into
mainfrom
fix/audit-ledgers-and-ratchet
Aug 30, 2026
Merged

Publish the debt ledgers, cap three debt classes, and unbreak cargo off Linux#283
niveousdragon merged 6 commits into
mainfrom
fix/audit-ledgers-and-ratchet

Conversation

@niveousdragon

@niveousdragon niveousdragon commented Aug 30, 2026

Copy link
Copy Markdown
Contributor

Four independent changes, each answering a finding from the architecture-debt audits.

build: cargo could not run on Windows or macOS

.cargo/config.toml set build.rustc-wrapper = "scripts/rustc-wrapper.sh". That script
LD_PRELOADs tools/exdev-shim to emulate cross-directory rename() where the filesystem
returns EXDEV. Cargo has no per-platform form of that key, so every host read it, and off
Linux the shell script cannot be executed at all:

error: could not execute process `scripts/rustc-wrapper.sh` (never executed)
Caused by: %1 is not a valid Win32 application. (os error 193)

Every cargo command, on a clean clone, before any compilation. CI never saw it because its
runners are Linux. The shim is a property of one build environment, so it moves there:
export RUSTC_WRAPPER="$PWD/scripts/rustc-wrapper.sh" — absolute, because cargo resolves a
relative wrapper against the current directory rather than the workspace root. The script
and the shim are unchanged.

docs: the debt ledgers are now in the repository

They lived only in working copies, so their findings could not be read from a clone. The
external reconciliation of audit 2 had to mark nine findings "not independently verifiable
because Audit 2 omits the original finding text" — SD-011, SD-014, SD-020, SD-021, SD-028,
SD-033, SD-035, SD-037, SD-038. Their full text was in docs/architecture-debt.md all
along; the file was simply untracked.

Two corrections travel with the audit-2 ledger and are recorded inside it: its trend row
said new = 11 where the entries run SD-040 through SD-051, which is twelve; and "184
Ok(None)" is a count of lines, while one line reads Ok(None) => Ok(None), so there
are 185 occurrences.

A personal email address quoted in SD-043 was removed. The finding does not need it: what
it rests on is that five commits carry a local git identity rather than a GitHub noreply
one. Note this is a redaction of the ledger text only — the same identity remains visible
in .github/CODEOWNERS and in the authorship of those commits on main.

test: ceilings on three debt classes no lint can see

None can be driven to zero in one change, and none is visible to rustc or clippy: a
pub fn in a library crate is reachable from outside as far as the compiler knows, an
Ok(None) decline is ordinary control flow, and an environment variable is a string.

Enforced in the required python-contract job:

class ceiling
pub fn outside crates/pyxlog with no reference anywhere in workspace Rust source 105
Ok(None) occurrences in executor/wcoj_dispatch.rs 185
distinct XLOG_* names 107

A one-way ratchet, not an amnesty: lowering a ceiling is an ordinary pull request, raising
one has to be argued for in review. ENGINEERING.md rejects deferring cleanup as technical
debt and asks for a scope decision instead of a temporary substitute — a dated, enforced
ceiling is that decision, written where CI reads it.

crates/pyxlog is excluded deliberately: a #[pymethods] pub fn is reached from Python
and never from Rust, so the same measurement would call 21 live entry points dead and
would fail this job on any ordinary binding addition.

Verified both ways: green at these numbers, and an artificial increment of each of the
three fails it (106 > 105, 186 > 185, 108 > 107).

Known limit, stated in the test's own docstring rather than glossed: the measurement is
token-level, so a name merely mentioned in a comment counts as a reference. It bounds a
class; it does not prove each member dead.

docs: ENGINEERING.md now says what it actually enforces

Two sentences claimed more than the tree delivers, which is the cheapest way for a binding
document to stop being treated as one.

"Never silently fall back" is right about degrading a result and wrong about changing
route. The WCOJ dispatcher's two kinds of decline now have different standing in the text:
a real layout or kernel failure goes through wcoj_decline_on_error, which counts, logs
and raises under XLOG_WCOJ_STRICT — observable at runtime. A structural decline returns
Ok(None) and nothing observes it at runtime at all; its only check is the source-level
cap above, which tells an operator nothing during a run. The paragraph says exactly that,
rather than claiming such declines are "counted".

"Do not merge unused public functions" is the goal, not the present state. The standard
now names the measured number, says why pyxlog is excluded, and says the number bounds a
class rather than listing items proven safe to delete.

Note for reviewers

Two tests in the required python-contract job already fail on main when run on Windows:
test_cuda_change_classifier_cli_consumes_null_delimited_git_paths (b'true\r\n' vs
b'true\n') and test_runner_self_test_has_no_benchmark_runtime_dependencies. Verified on
an untouched checkout of 404e9219; pre-existing and platform-specific, not introduced
here and not addressed here. Same class as the cargo fix above.

An independent review of this branch caught the three ceiling defects listed above before
merge; the numbers in the table are the corrected ones.

`build.rustc-wrapper` pointed at `scripts/rustc-wrapper.sh`, which LD_PRELOADs
`tools/exdev-shim` to emulate cross-directory rename() where the filesystem
returns EXDEV. Cargo has no per-platform form of that key, so every host read
it, and on Windows and macOS the shell script cannot be executed at all:

    error: could not execute process `scripts/rustc-wrapper.sh` (never executed)
    Caused by: %1 is not a valid Win32 application. (os error 193)

That is every cargo command, on a clean clone, before any compilation. CI never
saw it because its runners are Linux.

The shim is a property of one build environment, not of the project, so it moves
into that environment: set RUSTC_WRAPPER=scripts/rustc-wrapper.sh where it is
needed. The script and the shim are unchanged.
The three audit ledgers lived only in working copies, so the findings they
record could not be read from a clone. The external reconciliation of audit 2
had to mark nine findings "not independently verifiable because Audit 2 omits
the original finding text" — SD-011, SD-014, SD-020, SD-021, SD-028, SD-033,
SD-035, SD-037 and SD-038. Their full text was in `docs/architecture-debt.md`
the whole time; the file was simply untracked. Publishing it closes that gap.

Two corrections travel with the audit-2 ledger, both recorded in it:

- its trend row said `new = 11` while the entries run SD-040 through SD-051,
  which is twelve. The external reconciliation counted twelve and was right;
- "184 Ok(None)" in the WCOJ dispatcher is a count of lines. One line reads
  `Ok(None) => Ok(None)`, so the number of occurrences is 185. The substance of
  the finding is unchanged; the reproducibility of the number was not.
Each class is real debt in the architecture-debt ledgers, none can be driven to
zero in one change, and none is detectable by rustc or clippy: a `pub fn` in a
library crate is reachable from outside as far as the compiler knows, an
`Ok(None)` decline is ordinary control flow, and an environment variable is a
string.

Record where each count stands and fail when it rises. Measured on this commit:

- 126 `pub fn` under `crates/*/src` whose name appears exactly once in all
  workspace Rust source — the definition and no reference anywhere;
- 185 `Ok(None)` occurrences in the WCOJ dispatcher, of which the structural
  declines are deliberate and documented as such;
- 115 distinct `XLOG_*` names. Reading them was centralised on
  `xlog_core::config_value`, which fixed how they are parsed and not how many
  exist; the count has risen since that work.

This is a one-way ratchet, not an amnesty: lowering a ceiling is an ordinary
pull request, raising one has to be argued for in review. ENGINEERING.md rejects
deferring cleanup as technical debt and asks for a scope decision instead of a
temporary substitute — a dated, enforced ceiling is that decision, written where
CI reads it, and an undated prose exception list is the substitute it rejects.

Wired into the required `python-contract` job.
Two sentences claimed more than the tree delivers, which is the cheapest way for
a binding document to stop being treated as one.

"Never silently fall back" is right about degrading a result and wrong about
changing route. The WCOJ dispatcher declines structurally — gate off, shape
mismatch, missing buffer — and the binary-join tree then runs verbatim over the
same rows; the doc comment on `wcoj_decline_on_error` already says this is
deliberate and separate from the error path it counts. Spell that distinction
out instead of leaving the rule contradicted by design.

"Do not merge unused public functions" is the goal, not the present state: 126
public functions under `crates/*/src` have no reference anywhere, and
`validators.rs` is 709 lines of them. Name the number and point at the ceiling
test that holds it, so a reader learns the real state from the standard rather
than from a grep that contradicts it.
@niveousdragon
niveousdragon requested a review from levi770 as a code owner August 30, 2026 11:15
SD-043 identified the commits that bypassed pull requests by the git identity
that authored them, and the ledger quoted that identity in full. The finding
does not need the address: what it rests on is that those five commits carry a
local identity rather than a GitHub noreply one, and that is now what it says.

This repository is public. The same redaction was applied to the PDF sent
outside and should have been applied here first; a public repository is the more
exposed of the two, not the less.
An independent review of this branch found three defects, all confirmed by
re-measuring here.

The XLOG_* ceiling was wrong. 115 came from a grep over every file under
crates/, scripts/ and docs/; the test scans a narrower set of extensions and
sees 107. Eight free slots is the opposite of a ratchet, and the comment saying
the number was measured on this commit was false for that line. Both ceilings
are now taken from the tests themselves.

The unreferenced-public-function count included crates/pyxlog. A
`#[pymethods] pub fn` is called from Python and never from Rust, so "appears
once in workspace Rust source" says nothing about it: 21 live entry points were
being counted as dead, and adding an ordinary binding would have failed a
required check. pyxlog is now excluded and the ceiling is 105.

ENGINEERING.md repeated the 126 as if it were a list of items to delete, which
is what made the pyxlog error dangerous rather than merely wrong. It now names
105, says why pyxlog is excluded, and says plainly that the number bounds a
class rather than proving each member dead.

The same paragraph claimed a structural WCOJ decline "must be counted". It is
not counted: nothing observes it at runtime, and the only check is the
source-level cap. Saying "counted" would let a future change satisfy the
sentence while leaving an operator with no signal, so the paragraph now
distinguishes the counted error path from the merely bounded structural one.

Also: RUSTC_WRAPPER is resolved against the current directory, not the workspace
root, so the .cargo/config.toml note now gives the absolute form.
@niveousdragon
niveousdragon merged commit 8108c75 into main Aug 30, 2026
15 checks passed
@niveousdragon
niveousdragon deleted the fix/audit-ledgers-and-ratchet branch August 30, 2026 14:24
niveousdragon added a commit that referenced this pull request Aug 31, 2026
…ate (#284)

* docs: remove the debt ledgers and the ceiling test from the repository

The project is in release stage, where technical-debt artifacts do not belong
in the tree. #283 added four audit ledgers, a remediation plan, and
python/tests/test_structural_debt_ceilings.py, whose ceilings of 105
unreferenced public functions, 185 silent WCOJ declines and 107 environment
names record a debt level as acceptable rather than remove it. ENGINEERING.md
gained a paragraph stating the same inside the binding contract.

All of it is reverted. Audit findings are tracked outside the release
repository, one issue per finding, and each is closed by the change that fixes
it: the counts go to zero, and the tests that hold them there assert zero
rather than a ceiling.

The wording about the two kinds of WCOJ decline is reverted with the rest. It
states an observability requirement the dispatcher does not meet today, so it
belongs in the same change as the counter that satisfies it, not ahead of it.

.cargo/config.toml from the same pull request is kept: it repairs a real defect
(cargo unusable outside Linux) rather than recording one.

* ci: run the Rust suite nightly and gate every pull request on the device crates

cargo test --workspace --all-targets --release ran under workflow_dispatch
alone, so it ran when somebody remembered to press the button. A regression
once sat on main until an outside run happened to find it. It now runs on a
nightly schedule against main.

The workspace suite takes up to two hours, which is too long to put on a pull
request, so cuda-slice runs the two crates whose purpose is the device --
including the certification suite in xlog-gpu/tests -- on every pull request
from this repository. It carries no path filter on purpose: a filter deciding
which pull requests get GPU coverage leaves whatever it misses unchecked, which
is the mechanism that produced the regression above.

A scheduled failure opens an issue, or comments on the open one when a previous
night already failed. A nightly run whose failure reaches nobody is a cron job,
not a gate.

Two details the schedule exposed. The change classifier rejected any event it
did not recognise, so a scheduled run would have failed in its first job before
reaching the tests. And the concurrency key grouped by ref alone, so a push to
main would have cancelled a nightly run already an hour in.

python/tests/test_ci_workflow_contract.py asserts the whole shape, including
the absence of a path filter on cuda-slice.

Making cuda-slice and python-wheel-gpu required checks is a ruleset change and
is not part of this commit.

* ci: report a nightly run that never started, not only one that failed

report-scheduled-failure fired on failure(), which excludes a cancellation. A
job that no self-hosted CUDA runner accepts stays queued until GitHub's 24-hour
limit and is then cancelled, so the one outcome meaning "the nightly did not run
at all" was the one that reported nothing.

That is not hypothetical. The CUDA runner has accepted no job since
2026-08-28T10:38Z: the next GPU job after that sat queued for exactly 24 hours
and was cancelled, and both GPU jobs on this pull request have been queued for
hours without starting.

The condition is now "some dependency did not succeed", with always() so the job
still runs after a cancelled dependency, and the issue body carries each
dependency's actual result instead of asserting a test failure.

The contract test asserts the new shape and rejects the old one.

* chore: untrack the agent-tooling telemetry file

artifact_gate.jsonl holds three records of an agent gate -- timestamps and empty
lists -- written by tooling that lives outside this repository. It is internal
state, not part of the product, and a release tree should not carry it.

It also caused a concrete problem twice: the writer appends to it while a
checkout sits idle, so any working tree accumulates an uncommitted change to a
tracked file and ordinary git operations refuse to switch branches.

Untracked rather than deleted, and added to .gitignore: the local file survives
for whatever writes it, and git stops noticing.

* chore: keep internal working artifacts out of the tree by name

The audit ledgers, the remediation plan and the telemetry file were removed
earlier in this pull request, but nothing stopped them coming back: they still
sit untracked in working copies, so an ordinary `git add -A` would recommit
them. The same is true of the analysis scratch directory, generated clippy
reports, handoff notes and worktree directories created inside the checkout.

They are now named in .gitignore. Findings are tracked outside this repository,
one issue per finding, and a release tree carries none of them.

Contributor-facing rules are deliberately untouched: AGENTS.md, CLAUDE.md,
CONTRIBUTING.md, ENGINEERING.md, ROADMAP.md and docs/DOCS-PROTOCOL.md state
what a change must satisfy rather than what our current state happens to be,
which is the line this commit draws.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant