Repository navigation
docs(release): make docs/releases/ the authoring source, and finish the curation method - #1183
Merged
Merged
Conversation
v1.3.0 was the first release whose notes were curated rather than generated, and the method was rediscovered from scratch. Both inputs mislead in ways worth naming once. operator-caveats-since.sh reports headings that CHANGED, not headings that are NEW. v1.3.0 listed two BREAKING headings and neither was a caveat for an upgrading operator: priorityTiers was already in v1.2.0, and capacityGate.mode's removed values had only ever existed on main. Repeating them unexamined would have sent operators after migrations they did not need. The section gives the one-line git test that tells the two apart. The commit log needs the inverse filter: keep what ships, drop ci/dogfood/test/ docs/build, and say you dropped them so 461 commits do not read as 57 user-facing changes. Keeping the trailing (#NNNN) costs nothing and GitHub auto-links it. Also recorded: promote by danger rather than by label (v1.3.0's most hazardous item carried no BREAKING heading), distinguish a guarded migration from a breaking change, caveat anything the validation run did not exercise, verify anchors by hand because make doc-links cannot see a notes file, and the chart-version form that makes a copy-pasted helm command fail.
…odies The v1.3.0 notes were drafted in a scratch file and edited straight on the GitHub Release. By the time they were right they had been through a wrong count, a dead anchor, 46 forced line breaks, two mismatched PR numbers, and a GHES caveat that never said "GHES" — each caught by hand, none by review, because the text was never in a diff. Move them to docs/releases/v1.3.0.md, byte-identical to the published draft body, and republish the draft from that path. One file per stable tag, holding the release body verbatim; `gh release edit --notes-file` publishes it. These files are written for github.com's comment-flavour renderer, not for MkDocs: their GFM alerts would publish as literal text and their links already point into the versioned site. So docs/releases/ is excluded from every site version including dev, which means all four copies of that exclusion have to agree — mkdocs.yml, two `env:` blocks and one `export` in pages.yml, and docs-preview.sh. Missing the last would make `make docs-build` disagree with CI. Also corrects two claims in the method doc that the shipped notes contradict: they link the versioned docs site, not `blob/vX.Y.Z/` (18 such links, zero blob), and being in-repo does not get them link-checked, since check-doc-links skips external URLs by design and every link in a notes file is absolute. The hard-wrap check the method doc gave was void: `gh release view --json body` returns raw Markdown, which contains no `<br>` however badly wrapped. Replaced with a render through `gh api /markdown -f mode=gfm`, verified to distinguish the two (mode=markdown reports 0 on the same input). Against the real body: 0 `<br>`, 3 alert blocks, 6 folds, no literal `[!`.
check-doc-links builds its scan list from `git ls-files -- '*.md'`, which is tracked-only, while the existence oracle deliberately includes untracked files "so a link to a brand-new file added in the same change resolves before it is staged". The asymmetry means links *to* a new doc resolve and links *from* it are never read. Hit while adding docs/releases/: the gate reported ok with an unchanged file count and one extra link, which reads exactly like a clean pass. CI never sees this because it runs on committed trees; the gap is local, pre-stage, and it makes a green local run mean less than it appears to.
The method doc recorded the first third of that work and stopped. Everything
from "add a feature list" onward — the enumerations, the API surface, the
receipts, the contributor section, the structure they all sit in — existed
only in the session that produced them.
Adds the section skeleton as a table, ordered by what a reader needs first,
plus the rules behind the four sections whose content is not self-evident:
- Deprecations, written even when nothing is removed, and stating whether the
apiserver actually warns. v1.3.0 deprecated v2alpha1 (warns) alongside
v1alpha1 (removal-slated, warns on nothing) — an asymmetry no reader can
discover for themselves.
- API surface, led by any new CRD, because the chart installs chart-root crds/
on a fresh install only; that is why "apply the CRDs" is upgrade step 1.
- Validation as receipts: link the run, quote the counts, quote a value
measured at the layer that matters.
- Project and tooling, last, labelled not-user-facing.
Also records three constraints found by hitting them: GFM alerts render as real
callouts in a release body and carry the danger banner; the Releases index
truncates behind "read more" and a fold is the only lever against it; and a
caveat is read out of its heading's context, so it has to name its own subject
("Untested against a real GHES appliance", not "a real appliance").
Corrects the claim that the shipped notes ran ~525 words with nine links. They
ran 2100 words and 25 links. The brevity rule applies to prose, not to the
enumerations a reader actually searches — as written it prescribed cutting the
notes to a quarter of what shipped.
karlkfi
changed the title
The Upgrading section closed with three unrelated operator-facing changes — quota accounting counting more, an EgressProxy pool losing a label, v2alpha1 starting to warn — run together in one sentence-run directly below its two numbered steps. Three things a reader might act on individually, positioned and formatted as a footnote to the procedure above them. As a bulleted list with a bold lead each, the same words are scannable, and the deprecation line now points at the Deprecations section that owns it rather than restating it. Records the general rule in the method doc: prose is for framing, and anything a reader might need to act on individually gets its own line. Republished the draft body from the file and re-diffed: identical. Rendered through mode=gfm: 0 <br>, 3 alerts, 6 folds, the new list a real <ul>.
The three "will be visible" changes shared one link to the top of the upgrade guide, under a line claiming each had its own section. They do — so each bullet now links its own, and carries the PR that made the change. An operator lands on what to do rather than on a table of contents. Anchors were not hand-slugged. MkDocs emits them with punctuation, backticks and parenthesised clauses folded in ways a re-derived slug gets wrong, so they were read out of the built site, and every site URL in the notes was then resolved against site/ in one pass: 20 resolved, plus a deliberately bogus anchor the same run flagged. Without that planted failure a checker that resolves nothing is indistinguishable from a clean file. Replaces the method doc's hand-slugging recipe with that approach, and records the planted-failure requirement with it.
docs/releases/ is in-repo now, but check-doc-links skips external URLs by design and every link in a notes file is absolute — so the one doc whose links are entirely unverifiable is the one published to every operator. site/ turns out to be a working oracle: resolving the v1.3.0 notes against the built HTML checked all 20 site URLs and flagged a planted bad anchor. Filed rather than built, since a gate needs somewhere to live and doc-links skipping external URLs is deliberate.
…lists Critical review turned up five defects in the notes. rc.4 passed the gate the day before rc.5. The notes claimed no candidate had ever cleared it and that rc.5 was the first to return a verdict; the plan doc records otherwise. The true version is better and checkable: five candidates, three aborted, rc.4 passed without catching a live worker pod, rc.5 caught one. The commit-log scope filter dropped seven shipping fixes and nothing said so — two fix(scalesetlistener) (the pattern had `scaleset`, which does not match a longer scope), a compound fix(agc,gmc), three fix(migrate) for the shipped gag-migrate binary, and one fix(observability). Reconciling the residue found them: 57 kept of 132 feat/fix. Fixes 34 -> 41. "New spec fields (10)" listed thirteen fields in ten bullets, because three bullets group related ones. Every other fold counts the noun in its own label. The commit total was 461 and had already drifted to 462; replaced with a form that cannot go stale. Validation now says what the gate does not assert, rather than leaving the leg list to be read as full coverage. Method doc records each of these as a rule: reconcile what the scope filter drops, count the unit named in the label, and check the validation story against the plan doc rather than memory.
…d path validate-release.sh has zero references to capacityGate, nodeAutoscaling or quota; its legs are deploy, e2e, sizing and CRD smoke. It builds an autoscaling cluster with headroom, so the capacity gate and the namespace-quota rung — the paths that exist precisely for a constrained cluster — are never exercised on real infrastructure by the gate that decides a release ships.
Reconciling every commit since v1.2.0 against the PR refs the notes cite — 462 with a ref, 91 cited — left 371 uncited. Most is docs and harness work, correctly excluded. Three categories were not. Security had no section at all, which reads as an omission to the one reader scanning for it. The release carries a gRPC and x/net security bump (#749) and six routine dependency updates, no advisory, and three fixes that harden credential and trust handling without patching a reported vulnerability. Says so, and names the gates: govulncheck and Trivy on every PR with no path filter, re-run on every push to main. 52 test and 15 refactor commits were unrepresented. Thirteen are now folded in — the ones that changed what the suites can observe (a live cluster-autoscaler matcher, the Karpenter drift gate, day-2 helm upgrade over a live release, migration on GKE, quota-blocked redelivery, cosign's own download path) and the consolidation behind them, including the code-restraint sweep. Kata, DinD, workload identity, backlog tooling and perf were checked and need nothing: their shipping changes are already cited, or they are dogfood and test infrastructure that ships in no image. Method doc gains the Security row in the skeleton, and the rule that the section is written even when the answer is "no advisory".
Measuring collapsed height — each section's bytes with <details> bodies excluded — put the body at 9.5k, with Validation third largest at 1781. Not an obvious candidate by eye: Upgrading and Highlights are both larger, and both have to stay open. Folding it needed care, because the four legs are the receipts and hiding receipts defeats the section. So the summary line carries the finding rather than a label: "73/73 e2e specs on Kata microVM workers, on live GKE — the four legs, and what none of them assert". A reader who never expands it has still seen the number. Validation drops to 902 chars collapsed, the body to 8638. Method doc records both: measure collapsed height to choose the next fold, and when the folded content is evidence, put the evidence in the summary — the exception to the count-in-the-summary convention.
Three findings, all from checking whether Validation and the testing fold should merge. They should not: Validation answers "why believe this build", operator-facing and about the artifact; the testing fold answers "is the project built well", contributor-facing and about the cycle. Merging drags one into the other's audience. What they needed was the cross-reference — Validation names the suites the gate does not exercise, and the fold lists the ones that grew to reach a live cluster. Checking that seam found #988 cited as "Gate upgrades from the last released chart tag" in the tooling fold. #988 is docs(plan): file and scope Q507 — the backlog row, not the gate. #1008 is the gate, already cited in Validation with more context. Removed the duplicate; tooling 26 -> 25. Verifying every shipped-fold entry touches a product path then found #840, listed as a feature, touching only claude-usage/ — the usage-tracking tooling. The `metrics` scope covers both that module and product metrics. It was the only one of 64; features 23 -> 22, shipped total 64 -> 63. Metrics themselves had no surface listing at all, though the condition reasons got one. Sixteen are added since v1.2.0 and none removed, so existing queries keep working; the fold reconciles both ways against a name diff of the two trees. The shipped Grafana dashboards were rebuilt around them and the PrometheusRule gains an egress alert — neither was mentioned. Section renamed to "API and metric surface" to match what it now covers.
Diffing docs/ from v1.2.0 to main found three things the notes had missed. Eight Kubernetes Event reasons are new and none removed — a third enumerable surface alongside the condition reasons and the metrics, and the one an operator meets first, since it is what `kubectl get events` prints. Six already have their own troubleshooting section. WorkerCapacityDeclined is both an event and a condition reason, and says so. Extracting them needed care: the same call takes a reason and an action, so ProvisionWorker and ApplyAGCAutoscaler read as reasons until each call site is checked. Two new operator guides were unlinked. resourcequota-sizing.md is 411 lines turning runner shapes into quota numbers — exactly what an operator needs after the accounting change the notes already flag, so it is linked from that bullet. node-shutdown-budgets.md is 337 lines on terminationGracePeriodSeconds being a request rather than a guarantee, with no feature of its own to hang off. troubleshooting.md gained 36 sections, most of them the remedy for a reason listed in one of the three surface folds. The notes linked the page without signalling that. Method doc records that a table of contents is not merely unnecessary but impossible: release-body headings carry no id — verified against the published v1.2.0 page, where the only ids belong to GitHub's own chrome — so an unlinked list would cost collapsed height to navigate nothing. Ten folds already read as the outline.
Diffing code from v1.2.0 to main found one operator-facing category with no representation at all: configuration. Four chart values (the GMC's managed VPA block, off by default), three environment tunables documented in the operator docs but nowhere else — PROXY_SHUTDOWN_DRAIN_TIMEOUT (45s), PROXY_SHUTDOWN_LINGER (10s), AGC_QUOTA_ADMISSION — and one renamed GMC flag, --priority-class-allowlist-configmap to --priority-class-allowlist-name, which matters only to an install passing GMC arguments directly. Nothing here is required; every item defaults to the previous behaviour. What the diff confirmed rather than changed is worth recording too. The 13 spec and 2 status fields match the CRD schemas at both tags exactly, and nothing was removed — the notes now say so, since "nothing removed" is what an operator is looking for. Both upgrade steps really are guarded: each is a Helm `fail` naming the fix, one for the CRD not being installed and one for priorityClassAllowlist.configMapName still being set. That values key survives as a tripwire, which is why a values diff shows no removal. Two extraction traps are recorded in the method doc, both hit here. A rename reads as a removal when the scope is one directory: env vars appeared to have 17 removals, every one of them code moving out of cmd/, and zero when re-run repo-wide. And adjacent string arguments read as the same thing — recordEvent(obj, type, reason, action) put ProvisionWorker and ApplyAGCAutoscaler through extraction as reasons until each call site was read.
…table Auditing the method doc against what this session actually did found five techniques that caught real defects but were never written down. Two are citation checks. Passing the scope filter does not make a commit shippable — scopes are reused, and feat(metrics) covers both the product's Prometheus metrics and the claude-usage module, which is how a commit touching only claude-usage/ shipped as a feature. And a docs(plan) commit that files a Q-row reads exactly like the fix it describes, which is how #988 came to be cited under #1008's label. One is a caveat check. Understating coverage is as wrong as overstating it and feels safer, so it survives review: the draft said the capacity gate had unit and envtest coverage only, when a live-cluster-autoscaler test and 305 lines of quota-blocked e2e both exist. Grep every higher tier before writing "only tested at X", and prefer the narrow checkable claim. One is a source. docs/ deserves the same tag-to-tag diff as the code — a new operator page is the strongest signal of a capability the notes forgot, and it should be linked from the bullet it serves rather than a documentation inventory. The last is a table collecting every mechanical check in one place, with the two habits that make them mean anything: plant a known failure in whatever reports all-clear, and report the negative when it is true, since "none removed" is what an operator is scanning for. Also records why these files carry no title heading — the Releases page renders the tag name as its own h1, and stripping a heading at publish time would break the byte-identical property the files exist to guarantee.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Supersedes #1178. Moves the curated
v1.3.0release notes into the repo as the authoring source, writes down the method that produced them, and fixes what a critical review of both turned up.What
docs/releases/v1.3.0.md— the release body verbatim, republished from that path so the file is demonstrably the source rather than a copy.docs/releases/README.md— the convention, and the renderer differences that constrain it.docs/operations/release.md§ Writing the curated notes — 36 rules and a pre-publish check table.devincluded. Four places carry that exclusion and all four must agree:mkdocs.yml, twoenv:blocks and oneexportinpages.yml, andscripts/docs/docs-preview.sh. Missing the last would makemake docs-builddisagree with CI.Why in-repo
The notes were drafted in a scratch file and edited straight on the Release. By the time they were right they had been through a wrong count, a dead anchor, 46 forced line breaks, two mismatched PR numbers, and a GHES caveat that never said "GHES" — each caught by hand, none by review, because the text was never in a diff.
They are excluded from the site because they are written for github.com's comment-flavour renderer: GFM alerts have no MkDocs equivalent and would publish as literal text.
What the review found
Reconciling the notes against the repo — rather than re-reading them — turned up nine defects. Every one had survived several careful readings.
Validation was wrong in the section whose job is to be checkable. It claimed no candidate had ever cleared the gate and that rc.5 was the first to return a verdict. The plan doc records rc.4 passing the day before. The true version is better and verifiable: five candidates, three aborted, rc.4 passed without catching a live worker pod, rc.5 caught one.
The commit-log scope filter dropped seven shipping fixes and said nothing — two
fix(scalesetlistener)(the pattern hadscaleset, which does not match a longer scope), a compoundfix(agc,gmc), threefix(migrate)for the shippedgag-migratebinary, onefix(observability). 57 kept of 132. Fixes 34 → 41.A commit that passed the filter still was not shippable.
#840isfeat(metrics)and touches onlyclaude-usage/— that module and the product's Prometheus metrics share a scope. It was listed as a feature. One of 64; found by asserting paths, not by reading subjects.A citation pointed at the planning commit.
#988isdocs(plan): file and scope Q507, cited under the label of#1008, the gate itself.Three enumerable surfaces had no listing at all — 16 metrics, 8 Kubernetes Event reasons, and the configuration surface (4 chart values, 3 env tunables, 1 renamed flag). None is discoverable from the CRDs. Nothing was removed from any of them, which the notes now say, because that is what an operator scans for.
Two new operator guides were unlinked, and
troubleshooting.mdhad grown by 36 sections.resourcequota-sizing.mdis now linked from the quota bullet it serves."New spec fields (10)" listed 13 fields — the only fold counting bullets instead of the noun in its label. Verified against the CRD schemas at both tags: 13 spec, 2 status, none removed.
The commit total (461) had already drifted to 462, replaced with a form that cannot go stale.
I nearly shipped a false caveat, which is in here as a rule: the draft said the capacity gate had "unit and envtest coverage only" when
#929(live cluster-autoscaler) and#1028(305 lines of quota-blocked e2e) both exist. Understating coverage is as wrong as overstating it and feels safer. What shipped is the narrow checkable claim — the release gate does not assert those paths — which is Q637.Testing
make check,make doc-links plan-index-check no-plan-refs-check,make docs-build— all green. CI:shellcheckran (47s) for thedocs-preview.shedit,doc-links,lint-status,plan-hygiene,conflict-markersall pass. The heavy Go gates skip correctly: no Go changes.ls:site-dev/STATUSis published andsite/STATUSis not, proving the probe distinguishes published from excluded;releases/is absent from both scopes.site/, with a deliberately bogus anchor planted and flagged in the same run.make doc-linksskips external URLs by design, so nothing else checks these.gh api /markdown -f mode=gfm— 0<br>, 3 alert blocks, 11 folds, no literal[!. Confirmedmode=gfmdiscriminates before trusting it (mode=markdownreports 0 on hard-wrapped input).pages.ymlenv:values parsed with a real YAML parser; theexportform checked under bash viaod -c.check-doc-linksscans tracked files only, so the first local run reported ok on files it had never opened (245/5261 → 247/5305 once staged). That gap is Q635.