ADR 0011: fleet operations and measurement as sibling repositories - #1
Open
thierrysays wants to merge 7 commits into
Open
ADR 0011: fleet operations and measurement as sibling repositories#1thierrysays wants to merge 7 commits into
thierrysays wants to merge 7 commits into
Conversation
Fleet operations and measurement kept being described as part of this programme while plainly not being part of this repository. Both use the whole rig rather than any one board, and both are hardware-agnostic in a way this package deliberately is not — hal/devices.py names five specific boards because the admission gate has to know what each can enforce. ADR 0011 records the decision and its cost: three repositories to keep coherent instead of one, and a real risk of schema drift across a boundary that nothing mechanically couples. measurement-harness — instrument-agnostic power, latency and thermal measurement. The core knows open/read/close returning volts and amps and nothing else. Built around one rule: a figure that was not measured never leaves labelled as one. provenance.kind is derived from the instrument, not asserted by the caller, and the energy-model export refuses a synthetic report unless asked in as many words, at which point the source string says so permanently. Energy is integrated over actual sample timestamps rather than averaged; every figure carries an error bar from the instrument's own declarations; the throttle verdict says throughput regressed and never claims a cause without a temperature series. 61 tests, mypy --strict clean, 96% coverage, no runtime dependencies. ina219.py is written from the datasheet, has driven nothing, and refuses to return readings. fleet-ops-lab — A/B slots with automatic rollback, digest-bound manifests, SBOM diffing, reproducible-build checking, waved rollouts with a halt rule. Two rules: silence is a rollback (activation is provisional, PENDING reverts on the node's own timer), and the fleet halts itself (no override on the budget). The transfer that succeeds and delivers the wrong bytes is a separate failure code from the transfer that drops, because a did-the-download-work check misses it. No cryptography ships; an absent verifier with a required quorum raises rather than passing. 67 tests, mypy --strict clean, 96% coverage, no runtime dependencies. Nothing changes in src/. Invariant 10 stands: every energy_model here is still labelled estimate. What changes is that the replacement now has a named producer and a named format, and energy_model_source will carry the harness's source string verbatim, digest and all. Both trees are staged under spinoff/ because the GitHub App backing this session cannot create repositories (403 Resource not accessible by integration). Nothing there is imported, packaged, or reached by make qa, which scopes ruff, mypy, bandit and pytest to src and tests. spinoff/README.md carries the transplant commands. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VBPJHknRGLJzEbEvLQLbaR
Both repositories now exist and carry the work: https://github.com/thierrysays/measurement-harness https://github.com/thierrysays/fleet-ops-lab The trees were staged under spinoff/ only because the session could not create repositories. Leaving them here would have quietly undone ADR 0011 — a governance repository that also contains a benchmark suite and a deployment tool is exactly the arrangement that ADR argues against — so the directory goes in the same change that gives it somewhere to go. The build log entries that described the staging are corrected to record where the work actually landed, in both languages, and ADR 0011 now links the two repositories by URL rather than by name alone. Nothing in src/ was touched at any point. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VBPJHknRGLJzEbEvLQLbaR
Seven points, now the standing default across this repository and both siblings: technical documentation, functional documentation, a neophyte path, a bare-metal run, a five-tier test harness, a QA gate, and a threat model whose residual risks are each pinned by a test. The same list is in measurement-harness/CLAUDE.md and fleet-ops-lab/CLAUDE.md, where all seven are met. Here, two are not, and the entry says so rather than leaving it to be discovered. There is no bare-metal guide — nothing here has run on a board, and writing it is part of milestone 1. The suite is one flat tier plus test_adversarial.py, which is the pen-test tier under an older name; there is no smoke tier and the security concerns are spread through the adversarial file rather than separated. Stating a gap is cheaper than finding it. Neither is fixed in this change, because doing so would mean restructuring 113 tests in a change whose subject is a paragraph. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VBPJHknRGLJzEbEvLQLbaR
Every repository carries glossolalie-advisory as a topic — the tag that makes the family findable from one search — alongside its own descriptive topics rather than instead of them. Checked against the account before writing it down: it sits with fifteen to twenty descriptive topics on every substantive repository, so it is the common tag and not a replacement for the rest. A new repository is not finished until it has it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VBPJHknRGLJzEbEvLQLbaR
Running `make qa` leaves .coverage, and on a full run .mypy_cache and .ruff_cache as well. None were ignored, so the gate a contributor is told to run dirties the repository it is checking, and the first thing anyone does with an untracked build artefact is commit it by accident. Found by running the gate here to verify the test count for a handoff document, which is the same way a contributor would find it. The two sibling repositories already carry these entries. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VBPJHknRGLJzEbEvLQLbaR
Counted rather than remembered. The figure was accurate at v0.1.0 and the suite has grown since; both places that quote it as current guidance were still saying 113. Changed: CLAUDE.md make test # 113 tests, ~0.5s -> 132 tests, ~0.5s CONTRIBUTING.md make test # 113 tests -> 132 tests The timing holds: 0.38 to 0.42 seconds across three runs. Left alone, deliberately: docs/BUILD_LOG.en.md and .fr.md record 113 as the state at the end of day one, docs/adr/0010 cites it as the count at v0.1.0, and docs/AUDIT-2026-08-21.md records 113 passing on the day of the audit. All three were true when written. Editing a dated record to match today makes it a worse record, and the build log is the one place in this repository where the reasons behind a choice are still legible. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VBPJHknRGLJzEbEvLQLbaR
The count next to `make test` said 113 while the suite was 132, and nothing in the gate noticed, because nothing in the gate reads the documentation. This adds the step that does. `scripts/check_documented_counts.py` takes the count from pytest's own collection rather than by parsing its output, so a future pytest release that rewords its summary line does not break the check. It compares that against every `# N tests` in CLAUDE.md and CONTRIBUTING.md, prints file:line for each mismatch, and exits non-zero. Two failure paths, both exercised before committing: drift CLAUDE.md:11: documented 131 tests, the suite collects 132 deletion no documented test count found in CLAUDE.md, CONTRIBUTING.md The second matters more than it looks. A check that guards a number passes vacuously once somebody deletes the number, so an empty result is a failure rather than a pass. Scope is deliberately narrow: the files that state a count as current guidance. The dated records are left alone. docs/BUILD_LOG.* records the state at the end of day one, docs/adr/0010 cites the count at v0.1.0, and docs/AUDIT-2026-08-21 records what passed on the day of the audit. A gate that rewrites history to match today produces a worse record than no gate. Wired into `make qa` and into the CI qa job, which runs its steps explicitly rather than through make. Also brought scripts/ inside the tooling it belongs to: ruff now lints it and mypy --strict now checks it, since a gate whose own code is unchecked is making an exception for itself. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VBPJHknRGLJzEbEvLQLbaR
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Spin off fleet operations (OTA, rollback, SBOM, reproducible builds, orchestration) and measurement harness (power, latency, thermal throttling) into separate hardware-agnostic repositories. This clarifies the governance claim of this repository by removing cross-cutting concerns that do not belong in a refusal-with-evidence system.
Changes
ADR 0011 (
docs/adr/0011-cross-cutting-work-lives-in-sibling-repositories.md): Documents the decision to separate fleet operations and measurement intofleet-ops-labandmeasurement-harnessrepositories, with rationale and consequences.Build logs (
docs/BUILD_LOG.en.mdanddocs/BUILD_LOG.fr.md): Record the spinoff on 2026-08-21, describing:measurement-harness: instrument-agnostic power/latency/thermal measurement with strict provenance tracking (provenance.kindderived from instrument, never asserted; synthetic reports explicitly marked; integration not averaging; anti-noise rules for throttle detection)fleet-ops-lab: A/B slots with automatic rollback, digest-bound manifests, SBOM diffing, reproducible-build checking, waved rollouts with halt rule (silence = rollback; fleet halts itself on failure budget exceeded)spinoff/staging directory removed after transplantCLAUDE.md: Updated project conventions to document:
measurement-harness/energy-model/v1export replacesDeviceProfile.energy_model, withsourcestring copied verbatim intoenergy_model_sourcemeasurement-harnessas the producer and specify that itsina219.pyporting note is the specificationRationale
Three reasons for separation:
Governance claim clarity: This repository's claim is that every control either fires or does not, provably. A power meter and rollout controller prove nothing of the kind; folding them in dilutes an auditable claim.
Hardware agnosticism mismatch:
hal/devices.pynames five specific boards (admission gate requirement). A measurement harness or update model that hard-codes five boards is a lab benchmark script, not an instrument. Agnostic tools should not be constrained by this repository's hardware specificity.Dependency budget: This package carries one runtime dependency as a governance property. A measurement harness speaking I²C, SCPI, and USB-HID cannot live under that rule.
Cost
Invariant 10 stands unchanged: every
energy_modelremains labelledestimateuntil measured. The replacement now has a named producer and format, withenergy_model_sourcecarrying the harness'ssourcestring verbatim (including digest and any throughput regression note). Manual file copying replaces mechanical coupling — a real risk of schema drift that is accepted because auditability of the governance claim is the whole product.Repositories
mypy --strictclean, 96% coverage, no runtime dependencies.mypy --strictclean, 96% coverage, no runtime dependencies.Neither depends on this package, and this package depends on neither. The only interface is a file.
https://claude.ai/code/session_01VBPJHknRGLJzEbEvLQLbaR