diff --git a/.coderabbit.yaml b/.coderabbit.yaml index 62aab68d..efbccb26 100644 --- a/.coderabbit.yaml +++ b/.coderabbit.yaml @@ -89,6 +89,10 @@ reviews: - "!website/public/**" - "!docs/public/**" - "!CHANGELOG.md" + # Docs archives are frozen copies of older releases, held to their release + # tag by scripts/docs-archive-config-test.sh. Review the provenance file + # (docs/content/archive-provenance.json) and that contract, not the copy. + - "!docs/content/docs/v*/**" path_instructions: - path: "internal/auth/**" diff --git a/.github/workflows/ci-verify.yml b/.github/workflows/ci-verify.yml index 2f07c7d5..1bb35f86 100644 --- a/.github/workflows/ci-verify.yml +++ b/.github/workflows/ci-verify.yml @@ -427,6 +427,12 @@ jobs: - name: Verify the package release contract run: bash scripts/package-release-config-test.sh + # Archives under docs/content/docs/vX.Y/ must still be byte copies of the + # docs at their recorded release tag. Needs the full-depth checkout above + # for the tags; with no archives it only checks the rules that need none. + - name: Verify the docs archive contract + run: bash scripts/docs-archive-config-test.sh + # Drives the Compose containerd check against a fake `go` on PATH and # canned build info, so it needs no toolchain and no network. - name: Verify the Compose containerd check diff --git a/.github/workflows/release-cut.yml b/.github/workflows/release-cut.yml index 11fb0a67..8abc8c2c 100644 --- a/.github/workflows/release-cut.yml +++ b/.github/workflows/release-cut.yml @@ -221,6 +221,11 @@ jobs: - name: Verify the package release contract against the tag run: bash scripts/package-release-config-test.sh + # A cut that moves current/ to a new line must already carry the archive + # of the line it leaves, built from that line's last tag. + - name: Verify the docs archive contract against the tag + run: bash scripts/docs-archive-config-test.sh + - name: Create and push release tag env: RELEASE_TAG: ${{ steps.next.outputs.release_tag }} diff --git a/RELEASING.md b/RELEASING.md index 37e1b323..6454831f 100644 --- a/RELEASING.md +++ b/RELEASING.md @@ -43,7 +43,7 @@ 6. **No source version bump needed** — the binary's version is injected at build time via GoReleaser ldflags (`-X github.com/codeswhat/portwing/internal/protocol.AgentVersion={{.Version}}`). `AgentVersion` in `internal/protocol/version.go` must stay a `var`: `-X` silently does nothing to a `const`. -7. **Lefthook pre-push** — runs automatically on `git push`. Sequence: clean-tree → release contract (`scripts/package-release-config-test.sh`) → goreleaser snapshot → lint → Qlty → test (-race) → govulncheck → fuzz smoke → actionlint → zizmor → web (`npm run check:web`). The push is blocked if any step fails. +7. **Lefthook pre-push** — runs automatically on `git push`. Sequence: clean-tree → release contract (`scripts/package-release-config-test.sh`) and docs archive contract (`scripts/docs-archive-config-test.sh`) → goreleaser snapshot → lint → Qlty → test (-race) → govulncheck → fuzz smoke → actionlint → zizmor → web (`npm run check:web`). The push is blocked if any step fails. --- @@ -87,6 +87,36 @@ sit at "Expected" forever. So the order is: Read the effective ruleset back afterward rather than trusting the PATCH's 200. The script prints it. +## Cutting a docs archive + +Do this in the prep PR for a new minor or major line (v0.9 to v1.0, say), +never for a patch. The old line's docs get frozen as `docs/content/docs/vX.Y/` +and `current/` moves on to the new line. Layout and rules: +`docs/content/README.md`. + +1. Make sure the old line is finished. The archive comes from its **last** tag, + so the final patch has to be tagged and fetched (`git fetch --tags`). +2. Retitle `docs/content/docs/current/meta.json` to the new line. The cut + refuses to archive the line `current/` still names. +3. Cut it from that tag: + + ```sh + npm run docs:archive -- vX.Y.Z # the old line's last tag + ``` + + This reads `docs/content/docs/current` from the tag with git (never your + working tree), writes `docs/content/docs/vX.Y/`, adds the entry to + `docs/content/archive-provenance.json`, and lists `vX.Y` in + `docs/content/docs/meta.json` after `current`. +4. Run `bash scripts/docs-archive-config-test.sh` after committing the new + directory and the provenance entry together. The same contract runs in + `Release Contract`, `release-cut`, and pre-push; once the changelog's + previous release is on an older line than `current/`, it fails until that + line's archive exists. + +Never edit a file under an archived `vX.Y/`. A correction goes in `current/`; +an archive that no longer matches its tag fails the contract. + ## Cutting the tag **Preferred path: use the `release-cut` workflow.** diff --git a/docs/content/README.md b/docs/content/README.md new file mode 100644 index 00000000..83536145 --- /dev/null +++ b/docs/content/README.md @@ -0,0 +1,43 @@ +# Docs content layout + +This directory is not part of the docs site. Fumadocs ingests everything under +`docs/` only, so keep files that must not become pages out of that directory. +`docs/content/docs/` holds the pages; this directory holds the records about them. + +## Versions + +```text +docs/content/docs/ + meta.json lists the version roots: current first, then archives newest first + current/ the live docs, served at /docs/ + vX.Y/ one frozen archive per retired release line, served at /docs/vX.Y/ +docs/content/archive-provenance.json where each archive came from +``` + +`current/meta.json` carries the line it documents as its title. That line has +no archive until `current/` moves on to the next one. + +## Archives are immutable + +An archive is a byte-for-byte copy of `current/` as it was at the last release +tag of its line. Nothing edits it afterwards, because the point of an archive +is to show what that release shipped with. A mistake found later is fixed in +`current/`. + +Each archive has an entry in `archive-provenance.json` with `sourceTag`, +`sourceCommit`, `sourcePath` and `sourceTree`. `scripts/docs-archive-config-test.sh` +checks that the committed archive is exactly that tree, that the tag resolves +to that commit, and that the tag ends its line. Cut one with +`npm run docs:archive -- `; the steps are in `RELEASING.md`. + +## How archives are published + +- Links inside an archive keep the form they had at release (`/authentication`). + The `remarkVersionedLinks` plugin in `docs/source.config.ts` scopes them to + `/vX.Y/...` at build time, so the copy stays identical to the tag. +- Every archived page is `noindex`, shows an "archived, unsupported" notice + linking to the same page in the current docs, and is left out of the sitemap. +- The static export puts an archive's pages at `docs/out/vX.Y/.html` and + its index at `docs/out/vX.Y.html`; `npm run docs:export-contract` checks both. +- Analytics counts archived paths under `/_other`; the allowlist in + `analytics/src/contract.ts` covers current pages only. diff --git a/docs/content/archive-provenance.json b/docs/content/archive-provenance.json new file mode 100644 index 00000000..0967ef42 --- /dev/null +++ b/docs/content/archive-provenance.json @@ -0,0 +1 @@ +{} diff --git a/docs/source.config.ts b/docs/source.config.ts index 89050993..ff7e73db 100644 --- a/docs/source.config.ts +++ b/docs/source.config.ts @@ -1,7 +1,17 @@ import { defineConfig, defineDocs } from "fumadocs-mdx/config"; +import { remarkVersionedLinks } from "./src/lib/remark-versioned-links.ts"; export const docs = defineDocs({ dir: "content/docs", }); -export default defineConfig(); +// Global, not on the collection: a collection's own `mdxOptions` is used as-is, +// without the Fumadocs preset, which silently drops syntax highlighting and the +// table of contents. The global option layers this plugin on top of the preset. +// Archived vX.Y/ pages keep their release-time links; this scopes them to the +// archive at build time and does nothing for current/. +export default defineConfig({ + mdxOptions: { + remarkPlugins: [remarkVersionedLinks], + }, +}); diff --git a/docs/src/app/[[...slug]]/page.tsx b/docs/src/app/[[...slug]]/page.tsx index 89417287..22a4c2fb 100644 --- a/docs/src/app/[[...slug]]/page.tsx +++ b/docs/src/app/[[...slug]]/page.tsx @@ -1,17 +1,46 @@ +import { Callout } from "fumadocs-ui/components/callout"; import defaultMdxComponents from "fumadocs-ui/mdx"; import { DocsBody, DocsDescription, DocsPage, DocsTitle } from "fumadocs-ui/page"; +import type { Metadata } from "next"; +import Link from "next/link"; import { notFound } from "next/navigation"; +import { archiveVersionOf } from "@/lib/docs-versions"; +import { BASE_URL } from "@/lib/site-config"; import { getDocsPage, source } from "@/lib/source"; +// The same page in the current docs, as the slug list the loader knows it by. +// An archived page's slugs lead with the version segment, so dropping it gives +// the slug the page has under current/ (or none, for the version's index). +function currentSlugs(slugs: string[]): string[] { + return slugs.slice(1); +} + +function hrefFor(slugs: string[]): string { + return slugs.length === 0 ? "/" : `/${slugs.join("/")}`; +} + export default async function Page(props: { params: Promise<{ slug?: string[] }> }) { const params = await props.params; const page = getDocsPage(params.slug); if (!page) notFound(); const MDX = page.data.body; + const version = archiveVersionOf(page.path); + const counterpart = version ? currentSlugs(page.slugs) : []; + const hasCounterpart = version !== undefined && getDocsPage(counterpart) !== undefined; return ( + {version && ( + + This page documents Portwing {version}, which is no longer supported. It is kept for + reference and is not updated.{" "} + + {hasCounterpart ? "Read this page in the current docs" : "Go to the current docs"} + + . + + )} {page.data.title} {page.data.description} @@ -25,13 +54,29 @@ export function generateStaticParams() { return source.generateParams(); } -export async function generateMetadata(props: { params: Promise<{ slug?: string[] }> }) { +export async function generateMetadata(props: { + params: Promise<{ slug?: string[] }>; +}): Promise { const params = await props.params; const page = getDocsPage(params.slug); if (!page) notFound(); + const version = archiveVersionOf(page.path); + if (version) { + // Archives are frozen reference copies: out of search results, but crawlers + // may still follow their links on to the current docs. + return { + title: `${page.data.title} (${version})`, + description: page.data.description, + robots: { index: false, follow: true }, + }; + } + return { title: page.data.title, description: page.data.description, + alternates: { + canonical: new URL(`/docs${hrefFor(page.slugs).replace(/^\/$/u, "")}`, BASE_URL).toString(), + }, }; } diff --git a/docs/src/lib/docs-versions.ts b/docs/src/lib/docs-versions.ts index cf29148d..346e4083 100644 --- a/docs/src/lib/docs-versions.ts +++ b/docs/src/lib/docs-versions.ts @@ -6,6 +6,25 @@ import { getSlugs } from "fumadocs-core/source"; // serving at /docs/, so its directory segment never reaches the URL. // Archived versions keep their segment: vX.Y/ serves at /docs/vX.Y/. const CURRENT_DIR = "current"; +const ARCHIVE_DIR = /^v\d+\.\d+$/u; +const CONTENT_MARKER = "/content/docs/"; + +/** + * The archived version a docs source path belongs to ("v0.9"), or undefined + * for `current/` and anything else. This is the one place that decides what + * counts as an archive: the link scoping plugin and the page renderer both ask + * here. Accepts the loader's page path (`v0.9/authentication.mdx`) or an + * absolute file path that contains `content/docs/`. + */ +export function archiveVersionOf(path: string): string | undefined { + const normalized = path.replaceAll("\\", "/"); + const marker = normalized.lastIndexOf(CONTENT_MARKER); + const relative = marker === -1 ? normalized : normalized.slice(marker + CONTENT_MARKER.length); + const separator = relative.indexOf("/"); + if (separator === -1) return undefined; + const dir = relative.slice(0, separator); + return ARCHIVE_DIR.test(dir) ? dir : undefined; +} /** * For the Fumadocs loader's `slugs` option. A page under `current/` gets the diff --git a/docs/src/lib/remark-versioned-links.ts b/docs/src/lib/remark-versioned-links.ts new file mode 100644 index 00000000..b5bb1a66 --- /dev/null +++ b/docs/src/lib/remark-versioned-links.ts @@ -0,0 +1,149 @@ +import { archiveVersionOf } from "./docs-versions.ts"; + +// An archived page is a byte copy of the current docs, so it still says +// [Auth](/authentication). Inside vX.Y/ that has to land on /vX.Y/authentication +// or a reader of old docs is dropped into the current ones. Rewriting at build +// time (not in the copy) is what keeps the archive identical to its release tag. + +interface EstreeNode { + type: string; + value?: unknown; + raw?: string; + expression?: EstreeNode; + body?: EstreeNode[]; + expressions?: EstreeNode[]; + quasis?: { value: { raw: string; cooked?: string | null } }[]; +} + +interface MdastNode { + type: string; + url?: string; + name?: string; + value?: unknown; + data?: { estree?: EstreeNode | null }; + attributes?: MdastNode[]; + children?: MdastNode[]; +} + +// Last path segment with a file extension: a static asset, not a docs page. +const ASSET_PATH = /\.[A-Za-z0-9]+$/u; + +/** + * Scope one root-relative target to a version. Returns the input unchanged for + * anything that is not an unversioned docs page path: external and protocol + * links, anchors-only, relative paths, `//host`, assets, targets already + * under a vX.Y/ segment, and targets that already carry the /docs basePath. + */ +export function scopeLink(url: string, version: string): string { + if (!url.startsWith("/") || url.startsWith("//")) return url; + const end = url.search(/[?#]/u); + const path = end === -1 ? url : url.slice(0, end); + const rest = end === -1 ? "" : url.slice(end); + // Already carries the site basePath: the router would add /docs again. + if (path === "/docs" || path.startsWith("/docs/")) return url; + if (ASSET_PATH.test(path)) return url; + if (/^\/v\d+\.\d+(?:\/|$)/u.test(path)) return url; + return `/${version}${path === "/" ? "" : path}${rest}`; +} + +// The expression of `href={...}` when it is a single string literal or a +// template literal with no interpolation. Anything dynamic returns undefined. +function staticExpression( + attribute: MdastNode, +): { read: string; write: (next: string) => void } | undefined { + const value = attribute.value as { type?: string; value?: string; data?: MdastNode["data"] }; + if (value?.type !== "mdxJsxAttributeValueExpression" || typeof value.value !== "string") { + return undefined; + } + const source = value.value; + const literal = value.data?.estree?.body?.[0]?.expression; + if (literal) { + if (value.data?.estree?.body?.length !== 1) return undefined; + if (literal.type === "Literal" && typeof literal.value === "string") { + return { + read: literal.value, + write(next) { + literal.value = next; + literal.raw = JSON.stringify(next); + value.value = JSON.stringify(next); + }, + }; + } + if ( + literal.type === "TemplateLiteral" && + literal.expressions?.length === 0 && + literal.quasis?.length === 1 && + typeof literal.quasis[0].value.cooked === "string" + ) { + const quasi = literal.quasis[0]; + return { + read: quasi.value.cooked as string, + write(next) { + quasi.value.raw = next; + quasi.value.cooked = next; + value.value = `\`${next}\``; + }, + }; + } + return undefined; + } + // No estree (parser without acorn): accept only the plain literal shapes. + const quoted = /^\s*(["'])([^"'\\\n]*)\1\s*$/u.exec(source); + if (quoted) { + return { + read: quoted[2], + write(next) { + value.value = JSON.stringify(next); + }, + }; + } + const template = /^\s*`([^`\\$]*)`\s*$/u.exec(source); + if (template) { + return { + read: template[1], + write(next) { + value.value = `\`${next}\``; + }, + }; + } + return undefined; +} + +function visit(node: MdastNode, version: string): void { + if ((node.type === "link" || node.type === "definition") && typeof node.url === "string") { + node.url = scopeLink(node.url, version); + } else if ( + (node.type === "mdxJsxFlowElement" || node.type === "mdxJsxTextElement") && + node.attributes && + // A native gets no basePath from the router, so a scoped root-relative + // href would 404 there just as the unscoped one does. Content tests forbid + // them; the plugin leaves them alone. + node.name !== "a" + ) { + for (const attribute of node.attributes) { + if (attribute.type !== "mdxJsxAttribute" || attribute.name !== "href") continue; + if (typeof attribute.value === "string") { + attribute.value = scopeLink(attribute.value, version); + continue; + } + const expression = staticExpression(attribute); + if (expression) { + const scoped = scopeLink(expression.read, version); + if (scoped !== expression.read) expression.write(scoped); + } + } + } + for (const child of node.children ?? []) visit(child, version); +} + +/** + * Remark plugin for docs/source.config.ts. Pages outside a vX.Y/ directory, + * including all of current/, are returned untouched. + */ +export function remarkVersionedLinks() { + return (tree: MdastNode, file: { path?: string }): void => { + const version = file.path ? archiveVersionOf(file.path) : undefined; + if (!version) return; + visit(tree, version); + }; +} diff --git a/docs/tsconfig.json b/docs/tsconfig.json index 8bfce950..b9069c23 100644 --- a/docs/tsconfig.json +++ b/docs/tsconfig.json @@ -6,6 +6,7 @@ "skipLibCheck": true, "strict": true, "noEmit": true, + "allowImportingTsExtensions": true, "esModuleInterop": true, "module": "esnext", "moduleResolution": "bundler", diff --git a/lefthook.yml b/lefthook.yml index 4492e0a6..a4a9074c 100644 --- a/lefthook.yml +++ b/lefthook.yml @@ -3,7 +3,8 @@ # # Pre-push pipeline (piped = sequential, fail-fast): # 1. Clean-tree check (no stray artifacts before push) -# 2. Package release contract (docs/config agreement, sub-second) +# 2. Package release contract (docs/config agreement, sub-second) and the +# docs archive contract # 3. GoReleaser snapshot dry-run (full release pipeline, skip publish) # 4. Pinned golangci-lint through the isolated CI adapter # 5. Qlty full-repository check @@ -65,6 +66,15 @@ pre-push: priority: 1 timeout: 30s + # The docs archive contract: archives under docs/content/docs/vX.Y/ must be + # byte copies of the docs at their recorded release tag. Git plumbing and + # jq only, and a no-op check list while zero archives exist. + docs-archive-contract: + run: bash scripts/docs-archive-config-test.sh + fail_text: "Docs archive contract broken — see the FAIL lines above" + priority: 1 + timeout: 30s + # The ARMv7 Compose containerd check's own test (fake go, canned build # info). Mirrors the "Release Contract" job's second step in ci-verify.yml. compose-containerd-check: diff --git a/package-lock.json b/package-lock.json index bcda785b..27c90916 100644 --- a/package-lock.json +++ b/package-lock.json @@ -15,7 +15,11 @@ "chrome-launcher": "1.2.2", "knip": "6.39.0", "lighthouse": "13.5.0", - "turbo": "2.11.6" + "remark-mdx": "3.1.1", + "remark-parse": "11.0.0", + "remark-stringify": "11.0.0", + "turbo": "2.11.6", + "unified": "11.0.5" }, "engines": { "node": ">=24.0.0" diff --git a/package.json b/package.json index cb72d666..1f9bb2c5 100644 --- a/package.json +++ b/package.json @@ -14,6 +14,7 @@ "format": "biome format --write .", "test:web": "node --test analytics/src/contract.test.ts scripts/*.test.mjs && npm --workspace website run test:headers", "knip": "knip", + "docs:archive": "node scripts/docs-archive-cut.mjs", "docs:export-contract": "node scripts/docs-export-contract.mjs", "budget:web": "node scripts/page-weight-budget.mjs", "lighthouse:marketing": "node scripts/lighthouse-budget.mjs lighthouse/marketing.cjs", @@ -26,7 +27,11 @@ "chrome-launcher": "1.2.2", "knip": "6.39.0", "lighthouse": "13.5.0", - "turbo": "2.11.6" + "remark-mdx": "3.1.1", + "remark-parse": "11.0.0", + "remark-stringify": "11.0.0", + "turbo": "2.11.6", + "unified": "11.0.5" }, "overrides": { "postcss": "^8.5.28", diff --git a/scripts/ci/go-release-check.sh b/scripts/ci/go-release-check.sh index 8e3b9d89..f4e60867 100755 --- a/scripts/ci/go-release-check.sh +++ b/scripts/ci/go-release-check.sh @@ -40,6 +40,8 @@ bash scripts/main-is-released-config-test-test.sh bash scripts/main-is-released-config-test.sh bash scripts/starchart-config-test-test.sh bash scripts/starchart-config-test.sh +bash scripts/docs-archive-config-test-test.sh +bash scripts/docs-archive-config-test.sh bash scripts/drydock-compat-script-test.sh bash scripts/benchstat-gate-script-test.sh bash scripts/benchstat-walk-baselines-test.sh diff --git a/scripts/docs-archive-config-test-test.sh b/scripts/docs-archive-config-test-test.sh new file mode 100755 index 00000000..256baded --- /dev/null +++ b/scripts/docs-archive-config-test-test.sh @@ -0,0 +1,346 @@ +#!/usr/bin/env bash +# Self-tests for scripts/docs-archive-config-test.sh. +# +# Builds a throwaway repository whose history has two release lines, archives +# the older one by hand, and proves the valid state passes. Each case then +# breaks that state in exactly one way, commits it, and asserts the contract +# fails with the matching message. Because the valid state is asserted to pass +# first, every failure below can only come from the mutation, never from a +# fixture that was broken to begin with. +set -euo pipefail + +# A git hook exports GIT_DIR and GIT_INDEX_FILE. Inherited here, they would +# send every fixture commit and tag below into the real repository. +# shellcheck disable=SC2046 +unset $(git rev-parse --local-env-vars) + +contract="$(pwd)/scripts/docs-archive-config-test.sh" +fixture="$(mktemp -d "${TMPDIR:-/tmp}/portwing-docs-archive.XXXXXX")" +trap 'rm -rf "${fixture}"' EXIT + +docs="docs/content/docs" +provenance="docs/content/archive-provenance.json" + +fx() { + git -C "${fixture}" -c user.email=test@example.com -c user.name=test \ + -c commit.gpgsign=false -c tag.gpgsign=false "$@" +} + +write_file() { + mkdir -p "${fixture}/$(dirname "$1")" + printf '%s\n' "$2" >"${fixture}/$1" +} + +commit_all() { + fx add -A + fx commit -q -m "$1" +} + +write_changelog() { + local newest="$1" + local previous="$2" + printf '# Changelog\n\n## [%s] - 2026-10-08\n\n- newest\n\n## [%s] - 2026-10-07\n\n- previous\n' \ + "${newest}" "${previous}" >"${fixture}/CHANGELOG.md" +} + +# --- history --------------------------------------------------------------- +fx init -q -b main +write_file "${docs}/meta.json" '{"pages":["current"]}' +write_file "${docs}/current/meta.json" '{"title":"v0.8","root":true}' +write_file "${docs}/current/index.mdx" 'eight index' +write_file "${docs}/current/setup.mdx" 'eight setup' +write_file "${provenance}" '{}' +write_changelog v0.8.0 v0.7.0 +commit_all "v0.8.0 docs" +fx tag v0.8.0 +write_file "${docs}/current/setup.mdx" 'eight setup, revised' +write_changelog v0.8.1 v0.8.0 +commit_all "v0.8.1 docs" +fx tag v0.8.1 +write_file "${docs}/current/meta.json" '{"title":"v0.9","root":true}' +write_file "${docs}/current/index.mdx" 'nine index' +write_changelog v0.9.0 v0.8.1 +commit_all "v0.9.0 docs" +fx tag v0.9.0 +no_archive_commit="$(fx rev-parse HEAD)" + +# --- zero archives ---------------------------------------------------------- +run_contract() { + (cd "${fixture}" && bash "${contract}" 2>&1) +} + +expect_pass() { + local description="$1" + local output + local status + + set +e + output="$(run_contract)" + status=$? + set -e + if [ "${status}" -ne 0 ]; then + echo "FAIL: ${description}" >&2 + echo "${output}" >&2 + exit 1 + fi +} + +expect_failure() { + local expected="$1" + local description="$2" + local output + local status + + set +e + output="$(run_contract)" + status=$? + set -e + if [ "${status}" -eq 0 ] || ! grep -Fq -- "${expected}" <<<"${output}"; then + echo "FAIL: ${description}" >&2 + echo "expected: ${expected}" >&2 + echo "${output}" >&2 + exit 1 + fi +} + +# v0.9.0 is the newest release and v0.8.1 the previous one, so the line has +# moved on and v0.8 must already be archived. The contract has to say so. +expect_failure "the previous release v0.8.1 is on v0.8 but current/ documents v0.9" \ + "leaving a line without archiving it must fail" + +# Same-line previous release: nothing to archive yet, and zero archives is fine. +write_changelog v0.9.1 v0.9.0 +commit_all "same-line changelog" +expect_pass "zero archives with the previous release on the current line must pass" + +# Today's shape: the line current/ left predates the current/ layout (its last +# tag holds a flat docs tree), so it cannot be archived and must not be demanded. +flat_tree="$(fx hash-object -w -t tree /dev/null)" +flat_commit="$(fx commit-tree "${flat_tree}" -m "flat layout, before current/")" +fx tag v0.7.0 "${flat_commit}" +write_changelog v0.9.1 v0.7.0 +commit_all "previous line predates the current/ layout" +expect_pass "a left-behind line whose last tag has no current/ cannot be archived and must pass" +fx tag -d v0.7.0 >/dev/null +fx reset -q --hard "${no_archive_commit}" + +# --- the valid archive ------------------------------------------------------- +mkdir -p "${fixture}/${docs}/v0.8" +fx archive v0.8.1 "${docs}/current" | tar -x -C "${fixture}/${docs}/v0.8" --strip-components=4 + +v081_commit="$(fx rev-parse v0.8.1)" +v081_tree="$(fx rev-parse "v0.8.1:${docs}/current")" +write_file "${provenance}" "$(printf '{"v0.8":{"sourceTag":"v0.8.1","sourceCommit":"%s","sourcePath":"%s/current","sourceTree":"%s"}}' \ + "${v081_commit}" "${docs}" "${v081_tree}")" +write_file "${docs}/meta.json" '{"pages":["current","v0.8"]}' +commit_all "archive v0.8" +valid_commit="$(fx rev-parse HEAD)" + +expect_pass "a complete archive with matching provenance must pass" + +reset_to_valid() { + fx reset -q --hard "${valid_commit}" + fx clean -fdq + fx tag -l 'v0.8.*' | while read -r name; do + if [ "${name}" != "v0.8.0" ] && [ "${name}" != "v0.8.1" ]; then + fx tag -d "${name}" >/dev/null + fi + done + if ! fx rev-parse -q --verify refs/tags/v0.8.1 >/dev/null || + [ "$(fx rev-parse v0.8.1)" != "${v081_commit}" ]; then + fx tag -f v0.8.1 "${v081_commit}" >/dev/null + fi +} + +# --- (a) bijection ----------------------------------------------------------- +cp -R "${fixture}/${docs}/v0.8" "${fixture}/${docs}/v0.7" +commit_all "archive dir with no provenance" +expect_failure "docs/content/docs/v0.7/ has no entry in ${provenance}" \ + "an archive directory without a provenance entry must fail" +reset_to_valid + +jq '. + {"v0.7": .["v0.8"]}' "${fixture}/${provenance}" >"${fixture}/p.tmp" +mv "${fixture}/p.tmp" "${fixture}/${provenance}" +commit_all "provenance with no archive dir" +expect_failure "lists v0.7 but ${docs}/v0.7/ does not exist" \ + "a provenance entry without an archive directory must fail" +reset_to_valid + +write_file "${docs}/old/index.mdx" 'stray' +commit_all "stray docs dir" +expect_failure "docs/content/docs/old/ is neither current/ nor a vX.Y archive" \ + "a directory that is neither current nor vX.Y must fail" +reset_to_valid + +# --- (b) tag and commit ------------------------------------------------------ +jq --arg c "$(fx rev-parse v0.8.0)" '.["v0.8"].sourceCommit = $c' "${fixture}/${provenance}" >"${fixture}/p.tmp" +mv "${fixture}/p.tmp" "${fixture}/${provenance}" +commit_all "sourceCommit points at another release" +expect_failure "tag v0.8.1 is ${v081_commit}, but" \ + "a sourceTag that resolves to a different commit must fail" +reset_to_valid + +jq '.["v0.8"].sourceCommit = "0000000000000000000000000000000000000000"' "${fixture}/${provenance}" >"${fixture}/p.tmp" +mv "${fixture}/p.tmp" "${fixture}/${provenance}" +commit_all "sourceCommit absent from the repository" +expect_failure "sourceCommit 0000000000000000000000000000000000000000 does not exist in this repository" \ + "a sourceCommit missing from a full clone must fail, not skip" +reset_to_valid + +fx tag -d v0.8.1 >/dev/null +expect_failure "tag v0.8.1 does not exist" \ + "a provenance entry whose tag was deleted must fail" +reset_to_valid + +jq '.["v0.8"].sourceTag = "v0.7.0"' "${fixture}/${provenance}" >"${fixture}/p.tmp" +mv "${fixture}/p.tmp" "${fixture}/${provenance}" +commit_all "sourceTag on another line" +expect_failure "sourceTag v0.7.0 must be a vX.Y.Z tag of the v0.8 line" \ + "a sourceTag from a different line must fail" +reset_to_valid + +jq 'del(.["v0.8"].sourcePath)' "${fixture}/${provenance}" >"${fixture}/p.tmp" +mv "${fixture}/p.tmp" "${fixture}/${provenance}" +commit_all "incomplete provenance entry" +expect_failure "entry v0.8 needs sourceTag, sourceCommit, sourcePath and sourceTree" \ + "an incomplete provenance entry must fail" +reset_to_valid + +# --- (c) tree equality ------------------------------------------------------- +write_file "${docs}/v0.8/setup.mdx" 'hand edited' +commit_all "archive edited after the cut" +expect_failure "an archive must be a byte copy of ${docs}/current at v0.8.1" \ + "an archive edited after the cut must fail" +reset_to_valid + +# Edit the archive and rewrite sourceTree to match it: the first comparison +# passes, so only the check against the tree at sourceCommit can catch this. +write_file "${docs}/v0.8/setup.mdx" 'hand edited' +fx add -A +edited_tree="$(fx rev-parse "$(fx write-tree):${docs}/v0.8")" +jq --arg t "${edited_tree}" '.["v0.8"].sourceTree = $t' "${fixture}/${provenance}" >"${fixture}/p.tmp" +mv "${fixture}/p.tmp" "${fixture}/${provenance}" +commit_all "archive and sourceTree edited together" +expect_failure "at ${v081_commit} is tree '${v081_tree}', not the recorded ${edited_tree}" \ + "a sourceTree rewritten to match an edited archive must fail" +reset_to_valid + +write_file "${docs}/v0.8/setup.mdx" 'uncommitted edit' +expect_failure "docs/content/docs/v0.8/ has uncommitted changes" \ + "an uncommitted edit to an archive must fail" +reset_to_valid + +# --- (d) last tag of the line ------------------------------------------------ +write_file "${docs}/current/index.mdx" 'later' +commit_all "later commit" +fx tag v0.8.2 +expect_failure "v0.8.1 is not the last tag of the line (v0.8.2 is)" \ + "an archive cut from a tag that is not the end of its line must fail" +reset_to_valid + +# --- (e) the current line is not an archive ---------------------------------- +write_file "${docs}/current/meta.json" '{"title":"v0.8","root":true}' +commit_all "current retitled to the archived line" +expect_failure "v0.8 is the line current/ documents, so it must not also be an archive" \ + "archiving the line current/ documents must fail" +reset_to_valid + +write_file "${docs}/current/meta.json" '{"title":"next","root":true}' +commit_all "current without a line title" +expect_failure "current/meta.json title must name a release line like v0.9, found 'next'" \ + "a current/ title that is not a line must fail" +reset_to_valid + +# --- (f) a left-behind line must be archived --------------------------------- +fx rm -rq "${docs}/v0.8" +write_file "${provenance}" '{}' +write_file "${docs}/meta.json" '{"pages":["current"]}' +commit_all "archive removed after the line moved on" +expect_failure "the previous release v0.8.1 is on v0.8 but current/ documents v0.9" \ + "removing the archive of the line the changelog just left must fail" +reset_to_valid + +# At v1.0.1 the previous *heading* is v1.0.0 on the current line, so only the +# CHANGELOG walk to the newest older line (v0.9) can notice the v0.9 archive was +# deleted. Build the valid state first so the failure can only come from the +# deletion. +write_file "${docs}/current/index.mdx" 'nine index, final' +write_changelog v0.9.1 v0.9.0 +commit_all "v0.9.1 docs" +fx tag v0.9.1 +v091_commit="$(fx rev-parse v0.9.1)" +v091_tree="$(fx rev-parse "v0.9.1:${docs}/current")" +mkdir -p "${fixture}/${docs}/v0.9" +fx archive v0.9.1 "${docs}/current" | tar -x -C "${fixture}/${docs}/v0.9" --strip-components=4 +jq --arg c "${v091_commit}" --arg t "${v091_tree}" --arg p "${docs}/current" \ + '. + {"v0.9":{"sourceTag":"v0.9.1","sourceCommit":$c,"sourcePath":$p,"sourceTree":$t}}' \ + "${fixture}/${provenance}" >"${fixture}/p.tmp" +mv "${fixture}/p.tmp" "${fixture}/${provenance}" +write_file "${docs}/meta.json" '{"pages":["current","v0.9","v0.8"]}' +write_file "${docs}/current/meta.json" '{"title":"v1.0","root":true}' +write_file "${docs}/current/index.mdx" 'ten index' +printf '# Changelog\n\n## [v1.0.1] - 2026-10-10\n\n- a\n\n## [v1.0.0] - 2026-10-09\n\n- b\n\n## [v0.9.1] - 2026-10-08\n\n- c\n' >"${fixture}/CHANGELOG.md" +commit_all "v1.0 current with the v0.9 archive" +expect_pass "v1.0.1 with the v0.9 archive present must pass" + +fx rm -rq "${docs}/v0.9" +jq 'del(.["v0.9"])' "${fixture}/${provenance}" >"${fixture}/p.tmp" +mv "${fixture}/p.tmp" "${fixture}/${provenance}" +write_file "${docs}/meta.json" '{"pages":["current","v0.8"]}' +commit_all "v0.9 archive removed at v1.0.1" +expect_failure "the previous release v0.9.1 is on v0.9 but current/ documents v1.0" \ + "deleting the archive of the line before the current one must fail even when the previous heading is on the current line" +fx reset -q --hard "${valid_commit}" +fx clean -fdq +fx tag -d v0.9.1 >/dev/null + +# --- provenance file --------------------------------------------------------- +fx rm -q "${provenance}" +commit_all "provenance removed" +expect_failure "${provenance} is missing" "a missing provenance file must fail" +reset_to_valid + +write_file "${provenance}" '[]' +commit_all "provenance is not an object" +expect_failure "${provenance} must be a JSON object keyed by vX.Y" \ + "a provenance file that is not an object must fail" +reset_to_valid + +# --- shallow clones ---------------------------------------------------------- +# The reusable CI gates run from depth-1 clones that carry neither the release +# commit nor its tags. That shape must skip loudly and pass; a broken archive +# in the same shallow clone must still fail on the checks that need no history. +shallow="$(mktemp -d "${TMPDIR:-/tmp}/portwing-docs-archive-shallow.XXXXXX")" +git clone -q --depth 1 "file://${fixture}" "${shallow}/repo" 2>/dev/null +set +e +shallow_output="$(cd "${shallow}/repo" && bash "${contract}" 2>&1)" +shallow_status=$? +set -e +if [ "${shallow_status}" -ne 0 ] || ! grep -Fq "SKIP: shallow checkout lacks ${v081_commit}" <<<"${shallow_output}"; then + echo "FAIL: a shallow checkout must skip the tag checks with a SKIP line and pass" >&2 + echo "${shallow_output}" >&2 + rm -rf "${shallow}" + exit 1 +fi +rm -rf "${shallow}" + +write_file "${docs}/v0.8/setup.mdx" 'hand edited' +commit_all "archive edited after the cut" +shallow="$(mktemp -d "${TMPDIR:-/tmp}/portwing-docs-archive-shallow.XXXXXX")" +git clone -q --depth 1 "file://${fixture}" "${shallow}/repo" 2>/dev/null +set +e +shallow_output="$(cd "${shallow}/repo" && bash "${contract}" 2>&1)" +shallow_status=$? +set -e +if [ "${shallow_status}" -eq 0 ] || ! grep -Fq "an archive must be a byte copy" <<<"${shallow_output}"; then + echo "FAIL: a shallow checkout must still reject an edited archive" >&2 + echo "${shallow_output}" >&2 + rm -rf "${shallow}" + exit 1 +fi +rm -rf "${shallow}" +reset_to_valid + +expect_pass "the valid archive must pass again after all mutations are reverted" + +echo "Docs archive contract self-tests passed." diff --git a/scripts/docs-archive-config-test.sh b/scripts/docs-archive-config-test.sh new file mode 100755 index 00000000..b0ffa607 --- /dev/null +++ b/scripts/docs-archive-config-test.sh @@ -0,0 +1,213 @@ +#!/usr/bin/env bash +# Docs archive contract. Run from the repository root. +# +# An archive (docs/content/docs/vX.Y/) is a frozen copy of docs/content/docs/ +# current/ as it was at the last release tag of that line, and +# docs/content/archive-provenance.json says which. This holds the two together: +# every archive has an entry, every entry points at a real tag, and the +# committed archive is that tag's tree, so a hand edit to an archive fails here +# instead of passing as "documentation". With no archives it checks only the +# rules that need none, which is why it is safe to run before the first cut. +set -euo pipefail + +failures=0 +docs_root="docs/content/docs" +provenance_file="docs/content/archive-provenance.json" +line_regex='^v[0-9]+\.[0-9]+$' +tag_regex='^v[0-9]+\.[0-9]+\.[0-9]+$' + +fail() { + echo "FAIL: $1" >&2 + failures=$((failures + 1)) +} + +if ! command -v jq >/dev/null 2>&1; then + echo "FAIL: jq is required to read ${provenance_file}" >&2 + exit 1 +fi +if [ ! -f "${provenance_file}" ]; then + echo "FAIL: ${provenance_file} is missing; it holds {} until the first archive is cut" >&2 + exit 1 +fi +if ! jq -e 'type == "object"' "${provenance_file}" >/dev/null 2>&1; then + echo "FAIL: ${provenance_file} must be a JSON object keyed by vX.Y" >&2 + exit 1 +fi + +shallow="false" +if [ "$(git rev-parse --is-shallow-repository 2>/dev/null)" = "true" ]; then + shallow="true" +fi + +# (a) Archive directories and provenance entries are a bijection. +archive_dirs="" +for dir in "${docs_root}"/*/; do + [ -d "${dir}" ] || continue + name="$(basename "${dir}")" + if [ "${name}" = "current" ]; then + continue + fi + if ! grep -Eq "${line_regex}" <<<"${name}"; then + fail "${docs_root}/${name}/ is neither current/ nor a vX.Y archive" + continue + fi + archive_dirs="${archive_dirs} ${name}" +done +provenance_lines="$(jq -r 'keys[]' "${provenance_file}")" + +for name in ${archive_dirs}; do + if ! grep -Fxq -- "${name}" <<<"${provenance_lines}"; then + fail "${docs_root}/${name}/ has no entry in ${provenance_file}; cut archives with npm run docs:archive" + fi +done +for name in ${provenance_lines}; do + if ! grep -Eq "${line_regex}" <<<"${name}"; then + fail "${provenance_file} key ${name} must look like vX.Y" + continue + fi + case " ${archive_dirs} " in + *" ${name} "*) ;; + *) fail "${provenance_file} lists ${name} but ${docs_root}/${name}/ does not exist" ;; + esac +done + +# (e) The line current/ documents is not also archived. +current_line="$(jq -r '.title // empty' "${docs_root}/current/meta.json")" +if ! grep -Eq "${line_regex}" <<<"${current_line}"; then + fail "${docs_root}/current/meta.json title must name a release line like v0.9, found '${current_line}'" + current_line="" +elif [ -d "${docs_root}/${current_line}" ] || grep -Fxq -- "${current_line}" <<<"${provenance_lines}"; then + fail "${current_line} is the line current/ documents, so it must not also be an archive" +fi + +# (b)-(d) Per archive: tag, commit and tree agree, and the tag ends its line. +for name in ${provenance_lines}; do + case " ${archive_dirs} " in + *" ${name} "*) ;; + *) continue ;; + esac + source_tag="$(jq -r --arg v "${name}" '.[$v].sourceTag // empty' "${provenance_file}")" + source_commit="$(jq -r --arg v "${name}" '.[$v].sourceCommit // empty' "${provenance_file}")" + source_path="$(jq -r --arg v "${name}" '.[$v].sourcePath // empty' "${provenance_file}")" + source_tree="$(jq -r --arg v "${name}" '.[$v].sourceTree // empty' "${provenance_file}")" + + if [ -z "${source_tag}" ] || [ -z "${source_commit}" ] || [ -z "${source_path}" ] || [ -z "${source_tree}" ]; then + fail "${provenance_file} entry ${name} needs sourceTag, sourceCommit, sourcePath and sourceTree" + continue + fi + if ! grep -Eq "${tag_regex}" <<<"${source_tag}" || [ "${source_tag%.*}" != "${name}" ]; then + fail "${name}: sourceTag ${source_tag} must be a vX.Y.Z tag of the ${name} line" + continue + fi + + # (c, first half) The committed archive is the recorded tree. HEAD, not the + # working tree: CI and release-cut see only what was committed. + if [ -n "$(git status --porcelain --untracked-files=all -- "${docs_root}/${name}")" ]; then + fail "${docs_root}/${name}/ has uncommitted changes; archives are immutable once committed" + fi + archive_tree="$(git rev-parse --verify --quiet "HEAD:${docs_root}/${name}" || true)" + if [ -z "${archive_tree}" ]; then + fail "${docs_root}/${name}/ is not committed at HEAD" + elif [ "${archive_tree}" != "${source_tree}" ]; then + fail "${docs_root}/${name}/ is tree ${archive_tree} at HEAD but ${provenance_file} records ${source_tree}; an archive must be a byte copy of ${source_path} at ${source_tag}" + fi + + # The rest needs the release commit and its tags. A shallow clone has + # neither, and the org's reusable gates run from one, so say so and skip + # rather than fail every PR or pass silently. A full clone missing them is + # a real failure. + if ! git cat-file -e "${source_commit}^{commit}" 2>/dev/null; then + if [ "${shallow}" = "true" ]; then + echo "SKIP: shallow checkout lacks ${source_commit}, so the ${name} tag and source-tree checks did not run here; they run in Release Contract, release-cut, and pre-push" >&2 + else + fail "${name}: sourceCommit ${source_commit} does not exist in this repository" + fi + continue + fi + + # (b) sourceTag resolves to sourceCommit. + resolved="$(git rev-parse --verify --quiet "refs/tags/${source_tag}^{commit}" || true)" + if [ -z "${resolved}" ]; then + if [ "${shallow}" = "true" ]; then + echo "SKIP: shallow checkout lacks tag ${source_tag}, so its checks did not run here" >&2 + else + fail "${name}: tag ${source_tag} does not exist" + fi + elif [ "${resolved}" != "${source_commit}" ]; then + fail "${name}: tag ${source_tag} is ${resolved}, but ${provenance_file} records ${source_commit}" + fi + + # (c, second half) sourceTree is what sourcePath really held at sourceCommit. + commit_tree="$(git rev-parse --verify --quiet "${source_commit}:${source_path}" || true)" + if [ "${commit_tree}" != "${source_tree}" ]; then + fail "${name}: ${source_path} at ${source_commit} is tree '${commit_tree}', not the recorded ${source_tree}" + fi + + # (d) sourceTag is the last tag of its line. + if [ -n "${resolved}" ]; then + line_tags="$(git tag -l "${name}.*" | grep -E "${tag_regex}" || true)" + last_tag="$(printf '%s\n' "${line_tags}" | sort -t. -k3,3n | tail -n 1)" + if [ "${last_tag}" != "${source_tag}" ]; then + fail "${name}: ${source_tag} is not the last tag of the line (${last_tag} is); archive the docs the line ended with" + fi + fi +done + +# (f) Once the line has moved on, the line it left must already be archived. +# "The line it left" is the newest line in the CHANGELOG that is numerically +# older than current/'s, found by walking release headings (newest first) past +# every patch release on current's own line. Looking only at the previous +# heading would let v1.0.1 (previous release v1.0.0, same line) drop the v0.9 +# archive unnoticed. +# +# Floor: docs/content/docs/current/ did not exist before the docs archive work, +# and the cut script refuses tags without it. A line whose last tag lacks that +# path can never be archived, so the rule does not ask for one. Today +# (current = v0.9) the line it left is v0.8, which predates that layout. +line_key() { + local major="${1#v}" + major="${major%%.*}" + local minor="${1##*.}" + echo $((10#${major} * 100000 + 10#${minor})) +} + +release_versions="$(grep -E '^## \[v[0-9]+\.[0-9]+\.[0-9]+\] - [0-9]{4}-[0-9]{2}-[0-9]{2}$' CHANGELOG.md | + sed -E 's/^## \[(v[0-9]+\.[0-9]+\.[0-9]+)\].*/\1/' || true)" +if [ -z "${release_versions}" ]; then + fail "could not read the previous release from CHANGELOG.md" +elif [ -n "${current_line}" ]; then + current_key="$(line_key "${current_line}")" + previous_version="" + previous_line="" + for version in ${release_versions}; do + if [ "$(line_key "${version%.*}")" -lt "${current_key}" ]; then + previous_version="${version}" + previous_line="${version%.*}" + break + fi + done + if [ -n "${previous_line}" ]; then + case " ${archive_dirs} " in + *" ${previous_line} "*) ;; + *) + previous_last_tag="$(git tag -l "${previous_line}.*" | grep -E "${tag_regex}" | sort -t. -k3,3n | tail -n 1 || true)" + if [ -z "${previous_last_tag}" ]; then + if [ "${shallow}" = "true" ]; then + echo "SKIP: shallow checkout lacks the ${previous_line} tags, so the archive-required check did not run here" >&2 + else + fail "the previous release ${previous_version} is on ${previous_line} but no ${previous_line}.* tag exists to archive from" + fi + elif git rev-parse --verify --quiet "${previous_last_tag}:${docs_root}/current" >/dev/null; then + fail "the previous release ${previous_version} is on ${previous_line} but current/ documents ${current_line}; archive ${previous_line} from its last tag with npm run docs:archive" + fi + ;; + esac + fi +fi + +if [ "${failures}" -ne 0 ]; then + echo "${failures} docs archive contract check(s) failed" >&2 + exit 1 +fi + +echo "Docs archive contract checks passed." diff --git a/scripts/docs-archive-cut.mjs b/scripts/docs-archive-cut.mjs new file mode 100644 index 00000000..0c029aef --- /dev/null +++ b/scripts/docs-archive-cut.mjs @@ -0,0 +1,242 @@ +// Cut a docs archive: copy docs/content/docs/current/ as it was at a release +// tag into docs/content/docs/vX.Y/, record where it came from, and list it in +// the root meta.json. Usage: npm run docs:archive -- vX.Y.Z +// +// The copy comes from the tag's git tree, never the working tree, so the +// archive is the docs that shipped with that release rather than whatever the +// checkout holds today. scripts/docs-archive-config-test.sh holds the result to +// that promise afterwards. +import { execFileSync } from "node:child_process"; +import fs from "node:fs"; +import os from "node:os"; +import path from "node:path"; +import process from "node:process"; +import { fileURLToPath } from "node:url"; + +const SOURCE_PATH = "docs/content/docs/current"; +const DOCS_DIR = "docs/content/docs"; +const PROVENANCE = "docs/content/archive-provenance.json"; +const TAG = /^v(\d+)\.(\d+)\.(\d+)$/u; +const LINE = /^v(\d+)\.(\d+)$/u; + +class CutError extends Error {} + +function fail(message) { + throw new CutError(message); +} + +function parseArgs(argv) { + let root = path.resolve(path.dirname(fileURLToPath(import.meta.url)), ".."); + let tag; + for (let i = 0; i < argv.length; i += 1) { + if (argv[i] === "--root") { + i += 1; + if (!argv[i]) fail("--root needs a directory"); + root = path.resolve(argv[i]); + } else if (argv[i].startsWith("-")) { + fail(`unknown option ${argv[i]}`); + } else if (tag === undefined) { + tag = argv[i]; + } else { + fail(`unexpected argument ${argv[i]}`); + } + } + if (tag === undefined) fail("usage: npm run docs:archive -- vX.Y.Z"); + return { root, tag }; +} + +function readJson(file) { + try { + return JSON.parse(fs.readFileSync(file, "utf8")); + } catch (error) { + fail(`cannot read ${file} as JSON: ${error.message}`); + } +} + +function readJsonObject(file, label) { + const value = readJson(file); + if (value === null || typeof value !== "object" || Array.isArray(value)) { + fail(`${label} must be a JSON object`); + } + return value; +} + +function writeJson(file, value) { + fs.writeFileSync(file, `${JSON.stringify(value, null, 2)}\n`); +} + +// Newest line first, by numeric major then minor. +function byLineDescending(a, b) { + const [, aMajor, aMinor] = LINE.exec(a); + const [, bMajor, bMinor] = LINE.exec(b); + return Number(bMajor) - Number(aMajor) || Number(bMinor) - Number(aMinor); +} + +function cut({ root, tag }) { + // -C does not override GIT_DIR or GIT_INDEX_FILE. Run from a hook or a + // wrapper that exports them, git would read some other repository than root. + const env = Object.fromEntries( + Object.entries(process.env).filter(([name]) => !name.startsWith("GIT_")), + ); + const git = (...args) => + execFileSync("git", ["-C", root, ...args], { + encoding: "utf8", + env, + maxBuffer: 64 << 20, + }).trim(); + const gitBuffer = (...args) => + execFileSync("git", ["-C", root, ...args], { env, maxBuffer: 64 << 20 }); + + const match = TAG.exec(tag); + if (!match) fail(`${tag} is not a release tag; expected vX.Y.Z`); + const [, major, minor, patch] = match; + const line = `v${major}.${minor}`; + + let commit; + try { + commit = git("rev-parse", "--verify", "--quiet", `refs/tags/${tag}^{commit}`); + } catch { + fail(`tag ${tag} does not exist here; fetch tags first`); + } + + // The archive must be the end of its line, or the next patch's docs changes + // would be missing from "the docs for v0.9". + const laterPatches = git("tag", "-l", `v${major}.${minor}.*`) + .split("\n") + .filter((name) => TAG.test(name)) + .filter((name) => Number(TAG.exec(name)[3]) > Number(patch)); + if (laterPatches.length > 0) { + const newest = laterPatches + .sort((a, b) => Number(TAG.exec(a)[3]) - Number(TAG.exec(b)[3])) + .at(-1); + fail(`${tag} is not the last tag of ${line}; cut from ${newest} instead`); + } + + const docsRoot = path.join(root, DOCS_DIR); + const archiveDir = path.join(docsRoot, line); + if (fs.existsSync(archiveDir)) + fail(`${DOCS_DIR}/${line}/ already exists; archives are immutable`); + + const currentMeta = readJsonObject( + path.join(docsRoot, "current", "meta.json"), + "current/meta.json", + ); + if (!LINE.test(currentMeta.title ?? "")) { + fail( + `${DOCS_DIR}/current/meta.json title must be a line like v0.9, found ${currentMeta.title}`, + ); + } + if (currentMeta.title === line) { + fail(`${line} is the line current/ documents; archive it only after current/ moves on`); + } + + const provenancePath = path.join(root, PROVENANCE); + let provenanceBefore; + try { + provenanceBefore = fs.readFileSync(provenancePath, "utf8"); + } catch (error) { + if (error.code !== "ENOENT") throw error; + } + const provenance = + provenanceBefore === undefined ? {} : readJsonObject(provenancePath, PROVENANCE); + if (provenance[line]) fail(`${PROVENANCE} already has an entry for ${line}`); + + // Every input is read and validated before the first write, so a bad file + // cannot leave a half-done cut that the retry then refuses as immutable. + const rootMetaPath = path.join(docsRoot, "meta.json"); + const rootMetaBefore = fs.readFileSync(rootMetaPath, "utf8"); + const rootMeta = readJsonObject(rootMetaPath, `${DOCS_DIR}/meta.json`); + const archives = [ + ...fs + .readdirSync(docsRoot, { withFileTypes: true }) + .filter((entry) => entry.isDirectory() && LINE.test(entry.name)) + .map((entry) => entry.name), + line, + ].sort(byLineDescending); + + // `current/` only exists in tags cut after docs moved under it. Older tags + // have a flat docs/content/docs, which is not byte-identical to any archive + // layout, so they are refused rather than converted. + let sourceTree; + try { + sourceTree = git("rev-parse", "--verify", "--quiet", `${commit}:${SOURCE_PATH}`); + } catch { + fail(`${tag} has no ${SOURCE_PATH}; tags from before the current/ move cannot be archived`); + } + const entries = gitBuffer("ls-tree", "-r", "-z", commit, "--", SOURCE_PATH) + .toString("utf8") + .split("\0") + .filter(Boolean) + .map((row) => { + const tab = row.indexOf("\t"); + const [mode, type, sha] = row.slice(0, tab).split(" "); + return { mode, type, sha, rel: row.slice(tab + 1 + SOURCE_PATH.length + 1) }; + }); + const odd = entries.find((entry) => entry.type !== "blob" || entry.mode !== "100644"); + if (odd) fail(`${tag}:${SOURCE_PATH}/${odd.rel} is not a regular file (${odd.mode} ${odd.type})`); + if (entries.length === 0) fail(`${tag}:${SOURCE_PATH} is empty`); + + const staging = fs.mkdtempSync(path.join(os.tmpdir(), "docs-archive-")); + try { + for (const entry of entries) { + const target = path.join(staging, entry.rel); + fs.mkdirSync(path.dirname(target), { recursive: true }); + fs.writeFileSync(target, gitBuffer("cat-file", "blob", entry.sha)); + } + // Byte-identity check against the tree itself, without git's line-ending + // or attribute filters, so a checkout setting cannot alter the copy. + for (const entry of entries) { + const written = execFileSync( + "git", + ["hash-object", "--no-filters", path.join(staging, entry.rel)], + { encoding: "utf8", env }, + ).trim(); + if (written !== entry.sha) fail(`${entry.rel} did not copy byte-for-byte`); + } + // Writes start here. If any step fails, undo only what this run did. + try { + fs.cpSync(staging, archiveDir, { recursive: true, errorOnExist: true, force: false }); + + provenance[line] = { + sourceTag: tag, + sourceCommit: commit, + sourcePath: SOURCE_PATH, + sourceTree, + }; + const sortedProvenance = Object.fromEntries( + Object.keys(provenance) + .sort(byLineDescending) + .map((key) => [key, provenance[key]]), + ); + fs.mkdirSync(path.dirname(provenancePath), { recursive: true }); + writeJson(provenancePath, sortedProvenance); + + // current first, then archives newest first. Other keys in meta.json stay. + writeJson(rootMetaPath, { ...rootMeta, pages: ["current", ...archives] }); + } catch (error) { + // The cut refused up front if this directory existed, so whatever is + // there now came from this run. + fs.rmSync(archiveDir, { recursive: true, force: true }); + if (provenanceBefore === undefined) fs.rmSync(provenancePath, { force: true }); + else fs.writeFileSync(provenancePath, provenanceBefore); + fs.writeFileSync(rootMetaPath, rootMetaBefore); + throw error; + } + } finally { + fs.rmSync(staging, { recursive: true, force: true }); + } + + return { line, commit, sourceTree, files: entries.length }; +} + +try { + const result = cut(parseArgs(process.argv.slice(2))); + console.log( + `Archived ${result.files} files from ${result.commit.slice(0, 12)} as ${DOCS_DIR}/${result.line}/ (tree ${result.sourceTree}).`, + ); + console.log(`Next: review the diff and commit docs/content/docs and ${PROVENANCE} together.`); +} catch (error) { + if (!(error instanceof CutError)) throw error; + console.error(`FAIL: ${error.message}`); + process.exit(1); +} diff --git a/scripts/docs-archive-cut.test.mjs b/scripts/docs-archive-cut.test.mjs new file mode 100644 index 00000000..10d1d749 --- /dev/null +++ b/scripts/docs-archive-cut.test.mjs @@ -0,0 +1,232 @@ +import assert from "node:assert/strict"; +import { execFileSync, spawnSync } from "node:child_process"; +import fs from "node:fs"; +import os from "node:os"; +import path from "node:path"; +import test from "node:test"; + +const SCRIPT = path.resolve(import.meta.dirname, "docs-archive-cut.mjs"); +const DOCS = "docs/content/docs"; + +// A git hook exports GIT_DIR and GIT_INDEX_FILE, and lefthook runs this suite +// from pre-push. With those inherited, every git call below would land in the +// real repository instead of the fixture, so no GIT_* variable is passed on. +const ENV = Object.fromEntries( + Object.entries(process.env).filter(([name]) => !name.startsWith("GIT_")), +); + +function git(cwd, ...args) { + return execFileSync( + "git", + [ + "-c", + "user.email=test@example.com", + "-c", + "user.name=test", + "-c", + "commit.gpgsign=false", + "-c", + "tag.gpgsign=false", + ...args, + ], + { cwd, encoding: "utf8", env: ENV }, + ).trim(); +} + +function write(root, rel, content) { + const file = path.join(root, rel); + fs.mkdirSync(path.dirname(file), { recursive: true }); + fs.writeFileSync(file, content); +} + +function setCurrent(root, line, pages) { + fs.rmSync(path.join(root, DOCS, "current"), { recursive: true, force: true }); + write(root, `${DOCS}/current/meta.json`, JSON.stringify({ title: line, root: true })); + for (const [name, body] of Object.entries(pages)) { + write(root, `${DOCS}/current/${name}`, body); + } +} + +function commit(root, message, tag) { + git(root, "add", "-A"); + git(root, "commit", "-q", "-m", message); + if (tag) git(root, "tag", tag); +} + +// History: a flat-layout tag, v0.7.0, v0.8.0 and v0.8.1 (whose docs differ from +// the working tree), then HEAD documenting v0.9 with a v0.9.0 tag. +function fixture(t) { + const root = fs.mkdtempSync(path.join(os.tmpdir(), "docs-archive-cut-test-")); + t.after(() => fs.rmSync(root, { recursive: true, force: true })); + git(root, "init", "-q", "-b", "main"); + // Never commit or tag unless git is really working inside the fixture. + assert.equal( + fs.realpathSync(git(root, "rev-parse", "--absolute-git-dir")), + path.join(fs.realpathSync(root), ".git"), + ); + write(root, `${DOCS}/meta.json`, JSON.stringify({ pages: ["current"] })); + write(root, `${DOCS}/index.mdx`, "flat layout\n"); + commit(root, "flat docs", "v0.1.0"); + fs.rmSync(path.join(root, DOCS, "index.mdx")); + setCurrent(root, "v0.7", { "index.mdx": "seven index\n", "setup.mdx": "seven setup\n" }); + commit(root, "v0.7 docs", "v0.7.0"); + setCurrent(root, "v0.8", { + "index.mdx": "eight index\n", + "guide/deep.mdx": "eight deep\n", + "zero.mdx": "", + }); + commit(root, "v0.8 docs", "v0.8.0"); + write(root, `${DOCS}/current/index.mdx`, "eight index, revised\n"); + commit(root, "v0.8.1 docs", "v0.8.1"); + setCurrent(root, "v0.9", { "index.mdx": "nine index\n" }); + commit(root, "v0.9 docs", "v0.9.0"); + return root; +} + +function cut(root, ...args) { + const result = spawnSync("node", [SCRIPT, "--root", root, ...args], { + encoding: "utf8", + env: ENV, + }); + return { status: result.status, out: `${result.stdout}${result.stderr}` }; +} + +function snapshot(root) { + return git(root, "status", "--porcelain", "--untracked-files=all"); +} + +function readJson(root, rel) { + return JSON.parse(fs.readFileSync(path.join(root, rel), "utf8")); +} + +test("cuts the archive from the tag, not the working tree", (t) => { + const root = fixture(t); + const result = cut(root, "v0.8.1"); + assert.equal(result.status, 0, result.out); + + const archive = path.join(root, DOCS, "v0.8"); + assert.equal(fs.readFileSync(path.join(archive, "index.mdx"), "utf8"), "eight index, revised\n"); + assert.equal(fs.readFileSync(path.join(archive, "guide/deep.mdx"), "utf8"), "eight deep\n"); + assert.equal(fs.readFileSync(path.join(archive, "zero.mdx"), "utf8"), ""); + assert.equal(readJson(root, `${DOCS}/v0.8/meta.json`).title, "v0.8"); + // The working tree's current/ was not touched. + assert.equal( + fs.readFileSync(path.join(root, DOCS, "current", "index.mdx"), "utf8"), + "nine index\n", + ); + + const commitSha = git(root, "rev-list", "-n", "1", "v0.8.1"); + assert.deepEqual(readJson(root, "docs/content/archive-provenance.json"), { + "v0.8": { + sourceTag: "v0.8.1", + sourceCommit: commitSha, + sourcePath: `${DOCS}/current`, + sourceTree: git(root, "rev-parse", `v0.8.1:${DOCS}/current`), + }, + }); + assert.deepEqual(readJson(root, `${DOCS}/meta.json`), { pages: ["current", "v0.8"] }); + + // Provenance is a tree-hash equality: the committed archive is the tag's tree. + git(root, "add", "-A"); + const staged = git(root, "write-tree"); + assert.equal( + git(root, "rev-parse", `${staged}:${DOCS}/v0.8`), + git(root, "rev-parse", `v0.8.1:${DOCS}/current`), + ); +}); + +test("lists archives newest line first, current first of all", (t) => { + const root = fixture(t); + assert.equal(cut(root, "v0.7.0").status, 0); + assert.equal(cut(root, "v0.8.1").status, 0); + assert.deepEqual(readJson(root, `${DOCS}/meta.json`).pages, ["current", "v0.8", "v0.7"]); + assert.deepEqual(Object.keys(readJson(root, "docs/content/archive-provenance.json")), [ + "v0.8", + "v0.7", + ]); +}); + +test("refuses without changing anything", (t) => { + const root = fixture(t); + const before = snapshot(root); + const cases = [ + [["0.8.1"], "not a release tag"], + [["v0.8"], "not a release tag"], + [["v0.8.1-rc.1"], "not a release tag"], + [["v0.8.9"], "does not exist"], + [["v0.8.0"], "not the last tag of v0.8"], + [["v0.9.0"], "line current/ documents"], + [["v0.1.0"], "tags from before the current/ move"], + [[], "usage"], + [["v0.8.1", "extra"], "unexpected argument"], + ]; + for (const [args, message] of cases) { + const result = cut(root, ...args); + assert.notEqual(result.status, 0, `${args.join(" ")} should fail`); + assert.ok(result.out.includes(message), `${args.join(" ")}: ${result.out}`); + assert.equal(snapshot(root), before, `${args.join(" ")} left changes behind`); + } +}); + +test("refuses to overwrite an existing archive or provenance entry", (t) => { + const root = fixture(t); + assert.equal(cut(root, "v0.8.1").status, 0); + const before = snapshot(root); + const again = cut(root, "v0.8.1"); + assert.notEqual(again.status, 0); + assert.ok(again.out.includes("already exists"), again.out); + + fs.rmSync(path.join(root, DOCS, "v0.8"), { recursive: true }); + const orphan = cut(root, "v0.8.1"); + assert.notEqual(orphan.status, 0); + assert.ok(orphan.out.includes("already has an entry"), orphan.out); + assert.notEqual(before, ""); +}); + +test("an invalid root meta.json leaves no archive and no provenance change", (t) => { + const root = fixture(t); + write(root, "docs/content/archive-provenance.json", "{}\n"); + fs.writeFileSync(path.join(root, DOCS, "meta.json"), "{ not json"); + const before = snapshot(root); + const provenanceBefore = fs.readFileSync( + path.join(root, "docs/content/archive-provenance.json"), + "utf8", + ); + + const failed = cut(root, "v0.8.1"); + assert.notEqual(failed.status, 0); + assert.ok(failed.out.includes("meta.json"), failed.out); + assert.equal(fs.existsSync(path.join(root, DOCS, "v0.8")), false); + assert.equal( + fs.readFileSync(path.join(root, "docs/content/archive-provenance.json"), "utf8"), + provenanceBefore, + ); + assert.equal(snapshot(root), before); + + // Fixing the file makes the retry succeed instead of hitting "immutable". + write(root, `${DOCS}/meta.json`, JSON.stringify({ pages: ["current"] })); + const retry = cut(root, "v0.8.1"); + assert.equal(retry.status, 0, retry.out); +}); + +test("a write that fails midway undoes what the run created", (t) => { + const root = fixture(t); + // A directory where the provenance file must go makes the second write fail + // after the archive has already been copied. + fs.mkdirSync(path.join(root, "docs/content/archive-provenance.json"), { recursive: true }); + const failed = cut(root, "v0.8.1"); + assert.notEqual(failed.status, 0); + assert.equal(fs.existsSync(path.join(root, DOCS, "v0.8")), false); + assert.deepEqual(readJson(root, `${DOCS}/meta.json`), { pages: ["current"] }); +}); + +test("the replacement tag is the numerically newest patch", (t) => { + const root = fixture(t); + for (const name of ["v0.8.9", "v0.8.10"]) { + git(root, "commit", "-q", "--allow-empty", "-m", name); + git(root, "tag", name); + } + const result = cut(root, "v0.8.1"); + assert.notEqual(result.status, 0); + assert.ok(result.out.includes("cut from v0.8.10 instead"), result.out); +}); diff --git a/scripts/docs-export-contract.mjs b/scripts/docs-export-contract.mjs index 634876c4..cb4b53e6 100644 --- a/scripts/docs-export-contract.mjs +++ b/scripts/docs-export-contract.mjs @@ -1,14 +1,18 @@ // After `next build` in docs/, the static export is what ships at /docs. Moving // the content under content/docs/current/ must not leak that directory into -// the output, and every current page must still land at its unversioned URL. +// the output, every current page must still land at its unversioned URL, and +// each archived content/docs/vX.Y/ must export under its own segment: pages at +// vX.Y/.html and the version's index at vX.Y.html (the segment is the +// slug of its index page, so Next names it like a page, not a directory index). import fs from "node:fs"; import path from "node:path"; import process from "node:process"; import { fileURLToPath } from "node:url"; const CURRENT_DIR = "current"; +const VERSION_DIR = /^v\d+\.\d+$/u; const ROOT = path.resolve(path.dirname(fileURLToPath(import.meta.url)), ".."); -const CONTENT = path.join(ROOT, "docs", "content", "docs", CURRENT_DIR); +const DOCS_CONTENT = path.join(ROOT, "docs", "content", "docs"); const OUT = path.join(ROOT, "docs", "out"); // Mirrors slugsForPath in docs/src/lib/docs-versions.ts, which the loader uses. @@ -31,39 +35,53 @@ function listFiles(dir) { // Pages Next emits for itself, not from content. const FRAMEWORK_PAGES = new Set(["404.html", "_not-found.html"]); -// An archived release line exports under its own segment and is checked by its -// own contract; this one only owns the unversioned pages. -const ARCHIVE = /^v\d+\.\d+(?:\/|\.html$)/u; + +// Every page file of one version dir, keyed by the HTML file it must export as. +// `prefix` is the URL segment the version serves under: none for current/. +function expectedPages(dir, prefix) { + const expected = new Map(); + const pages = listFiles(path.join(DOCS_CONTENT, dir)).filter((file) => /\.mdx?$/u.test(file)); + for (const page of pages) { + const slugs = [...prefix, ...slugsFor(page)]; + expected.set(slugs.length === 0 ? "index.html" : `${slugs.join("/")}.html`, `${dir}/${page}`); + } + return expected; +} const problems = []; if (!fs.existsSync(OUT)) { problems.push(`${path.relative(ROOT, OUT)} is missing; run the docs build first`); } else { - const pages = listFiles(CONTENT).filter((file) => /\.mdx?$/u.test(file)); - const expected = new Map(); - for (const page of pages) { - const slugs = slugsFor(page); - expected.set(slugs.length === 0 ? "index.html" : `${slugs.join("/")}.html`, page); - } + const expected = expectedPages(CURRENT_DIR, []); if (!expected.has("index.html")) { problems.push(`${CURRENT_DIR}/index.mdx is missing; /docs would have no landing page`); } + const archives = fs + .readdirSync(DOCS_CONTENT, { withFileTypes: true }) + .filter((entry) => entry.isDirectory() && VERSION_DIR.test(entry.name)) + .map((entry) => entry.name); + for (const version of archives) { + for (const [html, page] of expectedPages(version, [version])) expected.set(html, page); + if (!fs.existsSync(path.join(DOCS_CONTENT, version, "index.mdx"))) { + problems.push(`${version}/index.mdx is missing; /docs/${version} would have no landing page`); + } + } const exported = new Set( listFiles(OUT) .map((file) => file.split(path.sep).join("/")) .filter((file) => file.endsWith(".html") && !file.startsWith("_next/")) - .filter((file) => !FRAMEWORK_PAGES.has(file) && !ARCHIVE.test(file)), + .filter((file) => !FRAMEWORK_PAGES.has(file)), ); for (const [html, page] of expected) { if (!exported.has(html)) { - problems.push(`${CURRENT_DIR}/${page} must export as docs/out/${html}`); + problems.push(`${page} must export as docs/out/${html}`); } } for (const html of exported) { if (!expected.has(html)) { problems.push( - `docs/out/${html} has no page in ${CURRENT_DIR}/; a directory leaked into the URL or a page went missing from the source`, + `docs/out/${html} has no page in ${CURRENT_DIR}/ or an archive; a directory leaked into the URL, a page went missing from the source, or an archive was removed without a clean build`, ); } } diff --git a/scripts/docs-export-contract.test.mjs b/scripts/docs-export-contract.test.mjs new file mode 100644 index 00000000..9e78b961 --- /dev/null +++ b/scripts/docs-export-contract.test.mjs @@ -0,0 +1,103 @@ +import assert from "node:assert/strict"; +import { spawnSync } from "node:child_process"; +import fs from "node:fs"; +import os from "node:os"; +import path from "node:path"; +import test, { after } from "node:test"; + +const SCRIPT = path.resolve(import.meta.dirname, "docs-export-contract.mjs"); + +// The contract finds the repo root from its own location, so a copy placed in a +// temp tree checks that tree instead of the real one. +const made = []; +after(() => { + for (const root of made) fs.rmSync(root, { recursive: true, force: true }); +}); + +function tree({ content, out }) { + const root = fs.mkdtempSync(path.join(os.tmpdir(), "docs-export-contract-test-")); + made.push(root); + fs.mkdirSync(path.join(root, "scripts")); + fs.copyFileSync(SCRIPT, path.join(root, "scripts", "docs-export-contract.mjs")); + for (const file of content) touch(root, `docs/content/docs/${file}`); + for (const file of out) touch(root, `docs/out/${file}`); + return root; +} + +function touch(root, rel) { + const file = path.join(root, rel); + fs.mkdirSync(path.dirname(file), { recursive: true }); + fs.writeFileSync(file, ""); +} + +function run(root) { + const result = spawnSync("node", [path.join(root, "scripts", "docs-export-contract.mjs")], { + encoding: "utf8", + }); + return { status: result.status, out: `${result.stdout}${result.stderr}` }; +} + +const CURRENT = ["current/index.mdx", "current/auth.mdx", "current/guide/deep.mdx"]; +const CURRENT_OUT = ["index.html", "auth.html", "guide/deep.html", "404.html", "_next/x.html"]; +const ARCHIVE = ["v0.8/index.mdx", "v0.8/auth.mdx", "v0.8/guide/deep.mdx"]; +const ARCHIVE_OUT = ["v0.8.html", "v0.8/auth.html", "v0.8/guide/deep.html"]; + +test("passes with zero archives and with a complete archive", () => { + assert.equal(run(tree({ content: CURRENT, out: CURRENT_OUT })).status, 0); + const withArchive = run( + tree({ content: [...CURRENT, ...ARCHIVE], out: [...CURRENT_OUT, ...ARCHIVE_OUT] }), + ); + assert.equal(withArchive.status, 0, withArchive.out); +}); + +test("an archive's index exports as vX.Y.html, not vX.Y/index.html", () => { + const result = run( + tree({ + content: [...CURRENT, ...ARCHIVE], + out: [...CURRENT_OUT, "v0.8/index.html", "v0.8/auth.html", "v0.8/guide/deep.html"], + }), + ); + assert.notEqual(result.status, 0); + assert.ok(result.out.includes("v0.8/index.mdx must export as docs/out/v0.8.html"), result.out); + assert.ok(result.out.includes("docs/out/v0.8/index.html has no page"), result.out); +}); + +test("fails when an archive page is missing from the export", () => { + const result = run( + tree({ + content: [...CURRENT, ...ARCHIVE], + out: [...CURRENT_OUT, "v0.8.html", "v0.8/guide/deep.html"], + }), + ); + assert.notEqual(result.status, 0); + assert.ok( + result.out.includes("v0.8/auth.mdx must export as docs/out/v0.8/auth.html"), + result.out, + ); +}); + +test("fails on exported versions that have no archive", () => { + const result = run( + tree({ content: CURRENT, out: [...CURRENT_OUT, "v0.8.html", "v0.8/auth.html"] }), + ); + assert.notEqual(result.status, 0); + assert.ok(result.out.includes("docs/out/v0.8.html has no page"), result.out); + assert.ok(result.out.includes("docs/out/v0.8/auth.html has no page"), result.out); +}); + +test("fails when an archive has no index page", () => { + const result = run( + tree({ + content: [...CURRENT, "v0.8/auth.mdx"], + out: [...CURRENT_OUT, "v0.8/auth.html"], + }), + ); + assert.notEqual(result.status, 0); + assert.ok(result.out.includes("v0.8/index.mdx is missing"), result.out); +}); + +test("still catches a leaked current/ directory", () => { + const result = run(tree({ content: CURRENT, out: [...CURRENT_OUT, "current/auth.html"] })); + assert.notEqual(result.status, 0); + assert.ok(result.out.includes("docs/out/current/auth.html has no page"), result.out); +}); diff --git a/scripts/docs-versions.test.mjs b/scripts/docs-versions.test.mjs index 22203730..cc4d0def 100644 --- a/scripts/docs-versions.test.mjs +++ b/scripts/docs-versions.test.mjs @@ -85,23 +85,65 @@ test("the analytics docs allowlist is exactly the current docs pages", () => { assert.deepEqual(listed, expected); }); -test("root-relative links resolve to a page in the same version dir", () => { - const dead = []; +// Source of one page with fenced and inline code removed, so examples that +// show a link do not count as one. +function proseOf(file) { + return fs + .readFileSync(file, "utf8") + .replace(/```[\s\S]*?```/gu, "") + .replace(/`[^`\n]*`/gu, ""); +} + +function allPages() { + const pages = []; for (const dir of versionDirs()) { - const slugs = new Set(pageSlugs(dir)); - // current serves at the docs root; an archived vX.Y serves under /vX.Y. - const prefix = dir === "current" ? "/" : `/${dir}/`; - for (const slug of slugs) { + for (const slug of pageSlugs(dir)) { + for (const extension of ["mdx", "md"]) { + const file = path.join(DOCS_ROOT, dir, `${slug}.${extension}`); + if (fs.existsSync(file)) pages.push({ label: `${dir}/${slug}`, file }); + } + } + } + return pages; +} + +test("root-relative links resolve to a page in the version they name or sit in", () => { + // Archived pages keep the unscoped links they had at their release tag; + // docs/src/lib/remark-versioned-links.ts rewrites them to /vX.Y/... at build. + // So an unversioned link is checked against its own dir's page set. A link + // that names another version on purpose (/v0.8/x inside v0.9/) is checked + // against that version's page set, and fails if that archive does not exist. + const dirs = new Set(versionDirs()); + const sets = new Map(); + const setFor = (dir) => { + if (!sets.has(dir)) sets.set(dir, new Set(pageSlugs(dir))); + return sets.get(dir); + }; + const dead = []; + for (const dir of dirs) { + for (const slug of setFor(dir)) { const file = path.join(DOCS_ROOT, dir, `${slug}.mdx`); const source = (fs.existsSync(file) ? fs.readFileSync(file, "utf8") : "").replace( /```[\s\S]*?```/gu, "", ); for (const match of source.matchAll(/\]\((\/[^)\s]*)\)/gu)) { - const [target] = match[1].split("#"); - const relative = target.startsWith(prefix) ? target.slice(prefix.length) : null; - const resolved = relative === null ? null : relative.replace(/\/$/u, "") || "index"; - if (resolved === null || !slugs.has(resolved)) { + const [target] = match[1].split(/[?#]/u); + let resolvedDir = dir; + let relative = target.slice(1); + const named = /^(v\d+\.\d+)(?:\/|$)/u.exec(relative); + if (named) { + resolvedDir = named[1]; + relative = relative.slice(named[0].length); + if (!dirs.has(resolvedDir)) { + dead.push( + `${dir}/${slug}: ${match[1]} (no ${resolvedDir}/ archive exists to link into)`, + ); + continue; + } + } + const resolved = relative.replace(/\/$/u, "") || "index"; + if (!setFor(resolvedDir).has(resolved)) { dead.push(`${dir}/${slug}: ${match[1]}`); } } @@ -109,3 +151,25 @@ test("root-relative links resolve to a page in the same version dir", () => { } assert.deepEqual(dead, []); }); + +test("no page uses a raw with a root-relative href", () => { + // A native gets no basePath from Next, so /authentication points at the + // site root, not /docs/authentication, and the archive link scoping skips + // native anchors. Use a markdown link: [text](/page). + const raw = /]*\bhref\s*=\s*(?:\{\s*)?["'`]\//u; + const found = allPages() + .filter(({ file }) => raw.test(proseOf(file))) + .map(({ label }) => label); + assert.deepEqual(found, [], "use a markdown link instead of a raw "); +}); + +test("no page links to /docs/... directly", () => { + // Next adds the /docs basePath to a root-relative link, so /docs/x becomes + // /docs/docs/x. Archive scoping also leaves /docs/... untouched. Write the + // target without the prefix: [text](/page). + const prefixed = /(?:\]\(|\bhref\s*=\s*(?:\{\s*)?["'`])\/docs(?=[/#?)"'`\s]|$)/u; + const found = allPages() + .filter(({ file }) => prefixed.test(proseOf(file))) + .map(({ label }) => label); + assert.deepEqual(found, [], "drop the /docs prefix from root-relative links"); +}); diff --git a/scripts/package-release-config-test-test.sh b/scripts/package-release-config-test-test.sh index 012a2ab3..5a8a6edd 100644 --- a/scripts/package-release-config-test-test.sh +++ b/scripts/package-release-config-test-test.sh @@ -424,6 +424,27 @@ expect_release_contract_failure \ "the package release contract must reject the 'since v' phrase outside README.md" mv "${releasing_since_backup}" "${fixture}/RELEASING.md" +# Docs archives (docs/content/docs/vX.Y/) are frozen copies of older releases +# and name old versions on purpose, so the stale-version grep skips them. The +# same literal in current/ is still a forgotten bump and must still fail. The +# files are staged because git grep only searches tracked paths. +archive_stale_dir="${fixture}/docs/content/docs/v0.0" +mkdir -p "${archive_stale_dir}" +printf 'Installed with VERSION=%s in this release.\n' "${previous_version}" >"${archive_stale_dir}/index.mdx" +git -C "${fixture}" add docs/content/docs/v0.0 +if ! (cd "${fixture}" && bash scripts/package-release-config-test.sh >/dev/null); then + echo "FAIL: a previous-version literal inside docs/content/docs/vX.Y/ must not fail the stale-version check" >&2 + exit 1 +fi +current_stale_file="${fixture}/docs/content/docs/current/stale-probe.mdx" +cp "${archive_stale_dir}/index.mdx" "${current_stale_file}" +git -C "${fixture}" add docs/content/docs/current/stale-probe.mdx +expect_release_contract_failure \ + "FAIL: stale references to v${previous_version} remain outside the changelog" \ + "the package release contract must still reject a previous-version literal in docs/content/docs/current/" +git -C "${fixture}" rm -rfq --cached docs/content/docs/v0.0 docs/content/docs/current/stale-probe.mdx +rm -rf "${archive_stale_dir}" "${current_stale_file}" + # The CHANGELOG-vs-tag guard only sees tags the checkout actually has, so # exercising it needs a real tag in the fixture repo. It guards against # portwing#283: a prep PR renamed the v0.9.12 heading to v0.9.13 instead of diff --git a/scripts/package-release-config-test.sh b/scripts/package-release-config-test.sh index 5509e40c..06b7800a 100755 --- a/scripts/package-release-config-test.sh +++ b/scripts/package-release-config-test.sh @@ -419,9 +419,16 @@ require_current_release_examples "docs/content/docs/current/security-model.mdx" # an rpm example, a doc's attestation command - still fails; a broader # file-wide exemption would let a stale version hide behind an unrelated # "since v" line anywhere in the repo. +# +# The other exemption is docs/content/docs/vX.Y/: an archive is a frozen copy of +# the docs at an older release, so it names that release's versions on purpose, +# and editing them would break its byte-for-byte tie to the tag. The archive +# contract (docs-archive-config-test.sh) polices those files instead. current/ +# gets no such pass. stale_previous_version_matches="$(git grep -n -F -- "$previous_version" -- \ '*.md' '*.mdx' '*.ts' '*.tsx' '*.yaml' '*.yml' '*.txt' \ ':(exclude)CHANGELOG.md' \ + ':(exclude)docs/content/docs/v[0-9]*' \ ':(exclude)scripts/package-release-config-test.sh' || true)" if [ -n "${stale_previous_version_matches}" ]; then stale_previous_version_matches="$(awk -v exempt="since v${previous_version}" ' diff --git a/scripts/remark-versioned-links.test.mjs b/scripts/remark-versioned-links.test.mjs new file mode 100644 index 00000000..9ab89112 --- /dev/null +++ b/scripts/remark-versioned-links.test.mjs @@ -0,0 +1,194 @@ +import assert from "node:assert/strict"; +import fs from "node:fs"; +import path from "node:path"; +import test from "node:test"; +import remarkMdx from "remark-mdx"; +import remarkParse from "remark-parse"; +import remarkStringify from "remark-stringify"; +import { unified } from "unified"; +import { archiveVersionOf } from "../docs/src/lib/docs-versions.ts"; +import { remarkVersionedLinks, scopeLink } from "../docs/src/lib/remark-versioned-links.ts"; + +const ROOT = path.resolve(import.meta.dirname, ".."); +const CURRENT = path.join(ROOT, "docs", "content", "docs", "current"); + +// Parse, run the plugin as Fumadocs would (with the source file's path on the +// vfile), and write the tree back to MDX so assertions read like the source. +function run(source, filePath) { + const processor = unified() + .use(remarkParse) + .use(remarkMdx) + .use(remarkVersionedLinks) + .use(remarkStringify); + return String(processor.processSync({ value: source, path: filePath })); +} + +test("archiveVersionOf recognises vX.Y directories and nothing else", () => { + assert.equal(archiveVersionOf("v0.9/authentication.mdx"), "v0.9"); + assert.equal(archiveVersionOf("v1.10/guides/setup.mdx"), "v1.10"); + assert.equal(archiveVersionOf("/repo/docs/content/docs/v0.9/index.mdx"), "v0.9"); + assert.equal( + archiveVersionOf("C:\\repo\\docs\\content\\docs\\v0.9\\index.mdx".replaceAll("\\", "/")), + "v0.9", + ); + assert.equal(archiveVersionOf("current/authentication.mdx"), undefined); + assert.equal(archiveVersionOf("/repo/docs/content/docs/current/v0.9/x.mdx"), undefined); + assert.equal(archiveVersionOf("v0.9.1/x.mdx"), undefined); + assert.equal(archiveVersionOf("v0.9"), undefined); + assert.equal(archiveVersionOf("index.mdx"), undefined); +}); + +test("scopeLink rewrites unversioned docs paths only", () => { + assert.equal(scopeLink("/authentication", "v0.9"), "/v0.9/authentication"); + assert.equal(scopeLink("/authentication#keys", "v0.9"), "/v0.9/authentication#keys"); + assert.equal(scopeLink("/authentication?x=1#keys", "v0.9"), "/v0.9/authentication?x=1#keys"); + assert.equal(scopeLink("/", "v0.9"), "/v0.9"); + assert.equal(scopeLink("/#top", "v0.9"), "/v0.9#top"); + assert.equal(scopeLink("/v0.9/authentication", "v0.9"), "/v0.9/authentication"); + assert.equal(scopeLink("/v0.8/authentication", "v0.9"), "/v0.8/authentication"); + assert.equal(scopeLink("/v0.9", "v0.9"), "/v0.9"); + assert.equal(scopeLink("/portwing.png", "v0.9"), "/portwing.png"); + assert.equal(scopeLink("/files/portwing.service", "v0.9"), "/files/portwing.service"); + assert.equal(scopeLink("https://example.com/x", "v0.9"), "https://example.com/x"); + assert.equal(scopeLink("//example.com/x", "v0.9"), "//example.com/x"); + assert.equal(scopeLink("mailto:a@b.c", "v0.9"), "mailto:a@b.c"); + assert.equal(scopeLink("#anchor", "v0.9"), "#anchor"); + assert.equal(scopeLink("./sibling", "v0.9"), "./sibling"); + assert.equal(scopeLink("sibling", "v0.9"), "sibling"); +}); + +const SAMPLE = [ + "[Auth](/authentication) and [anchor](/authentication#keys).", + "", + "[External](https://example.com/authentication) and [asset](/portwing.png).", + "", + "![logo](/portwing.png)", + "", + "[ref][r]", + "", + "[r]: /configuration", + "", + '', + "", + "```md", + "[not a link](/authentication)", + "```", + "", + "`[inline](/authentication)`", + "", +].join("\n"); + +test("links in an archived page are scoped to its version", () => { + const out = run(SAMPLE, "/repo/docs/content/docs/v0.9/authentication.mdx"); + assert.match(out, /\[Auth\]\(\/v0\.9\/authentication\)/u); + assert.match(out, /\[anchor\]\(\/v0\.9\/authentication#keys\)/u); + assert.match(out, /\[External\]\(https:\/\/example\.com\/authentication\)/u); + assert.match(out, /\[asset\]\(\/portwing\.png\)/u); + assert.match(out, /!\[logo\]\(\/portwing\.png\)/u); + assert.match(out, /\[r\]: \/v0\.9\/configuration/u); + assert.match(out, /href="\/v0\.9\/security-model"/u); + // Code is not a link: both forms are left exactly as written. + assert.match(out, /\[not a link\]\(\/authentication\)/u); + assert.match(out, /`\[inline\]\(\/authentication\)`/u); +}); + +test("native elements and basePath-prefixed targets are left alone", () => { + const file = "/repo/docs/content/docs/v0.9/x.mdx"; + const out = run( + ['raw', "", "[doc](/docs/authentication) [root](/docs)", ""].join( + "\n", + ), + file, + ); + assert.match(out, /raw<\/a>/u); + assert.match(out, /\]\(\/docs\/authentication\)/u); + assert.match(out, /\]\(\/docs\)/u); + assert.equal(scopeLink("/docs", "v0.9"), "/docs"); + assert.equal(scopeLink("/docs/authentication#x", "v0.9"), "/docs/authentication#x"); + assert.equal(scopeLink("/docsify", "v0.9"), "/v0.9/docsify"); +}); + +test("expression href attributes are scoped only when statically a string", () => { + const file = "/repo/docs/content/docs/v0.9/x.mdx"; + const out = run( + [ + '', + "", + "", + "", + "", + "", + "", + "", + // biome-ignore lint/suspicious/noTemplateCurlyInString: MDX source, not a JS template + "", + "", + "", + "", + '', + "", + ].join("\n"), + file, + ); + assert.match(out, /href=\{"\/v0\.9\/authentication"\}/u); + assert.match(out, /href=\{"\/v0\.9\/configuration"\}/u); + assert.match(out, /href=\{`\/v0\.9\/security-model#a`\}/u); + assert.match(out, /href=\{base \+ '\/authentication'\}/u); + assert.match(out, /href=\{`\/\$\{slug\}`\}/u); + assert.match(out, /href=\{link\}/u); + assert.match(out, /href=\{"https:\/\/example\.com\/x"\}/u); +}); + +test("the plugin is a no-op for current/ and for paths with no version", () => { + for (const file of [ + "/repo/docs/content/docs/current/authentication.mdx", + "current/authentication.mdx", + undefined, + ]) { + const scoped = run(SAMPLE, file); + const untouched = String( + unified().use(remarkParse).use(remarkMdx).use(remarkStringify).processSync(SAMPLE), + ); + assert.equal(scoped, untouched, `${file} must come through unchanged`); + } +}); + +test("every real current/ page comes through the plugin byte-for-byte", () => { + const pages = fs.readdirSync(CURRENT).filter((name) => name.endsWith(".mdx")); + assert.ok(pages.length > 0); + for (const name of pages) { + const source = fs.readFileSync(path.join(CURRENT, name), "utf8"); + const file = path.join(CURRENT, name); + const plain = String( + unified() + .use(remarkParse) + .use(remarkMdx) + .use(remarkStringify) + .processSync({ value: source, path: file }), + ); + assert.equal(run(source, file), plain, `${name} changed under the plugin`); + } +}); + +test("a real current page scoped as an archive keeps every link inside the archive", () => { + const source = fs.readFileSync(path.join(CURRENT, "authentication.mdx"), "utf8"); + const out = run(source, "/repo/docs/content/docs/v9.9/authentication.mdx"); + const internal = [...out.matchAll(/\]\((\/[^)\s]*)\)/gu)].map((match) => match[1]); + assert.ok(internal.length > 0, "authentication.mdx should link to other docs pages"); + for (const target of internal) { + assert.ok(target.startsWith("/v9.9/"), `${target} escaped the archive`); + } +}); + +test("the plugin is registered globally so the Fumadocs preset stays on", () => { + // A collection-level mdxOptions replaces the preset instead of extending it: + // the build stays green but code blocks lose highlighting and pages lose + // their table of contents. Only defineConfig({ mdxOptions }) layers on top. + const config = fs.readFileSync(path.join(ROOT, "docs", "source.config.ts"), "utf8"); + const docsBlock = config.slice( + config.indexOf("defineDocs("), + config.indexOf("});", config.indexOf("defineDocs(")), + ); + assert.doesNotMatch(docsBlock, /mdxOptions/u); + assert.match(config, /defineConfig\(\{\s*mdxOptions:\s*\{[^}]*remarkVersionedLinks/u); +}); diff --git a/website/scripts/security-headers.test.mjs b/website/scripts/security-headers.test.mjs index 41828959..0619b06f 100644 --- a/website/scripts/security-headers.test.mjs +++ b/website/scripts/security-headers.test.mjs @@ -53,6 +53,50 @@ test("static output paths map to clean public routes", () => { assert.equal(routeForOutputPath("out/docs/security-model.html"), "/docs/security-model"); }); +test("archived docs keep their dotted version segment in the public route", () => { + // An archive's index exports as docs/v1.0.html, a sibling of its docs/v1.0/ pages. + assert.equal(routeForOutputPath("out/docs/v1.0.html"), "/docs/v1.0"); + assert.equal( + routeForOutputPath("out/docs/v1.0/authentication.html"), + "/docs/v1.0/authentication", + ); +}); + +test("build output serves archived docs at their dotted routes with per-page CSP", () => { + const root = fs.mkdtempSync(path.join(os.tmpdir(), "portwing-bop-archive-")); + const source = path.join(root, "source"); + const target = path.join(root, ".vercel", "output"); + try { + fs.mkdirSync(path.join(source, "docs", "v1.0"), { recursive: true }); + fs.writeFileSync(path.join(source, "docs", "v1.0.html"), ""); + fs.writeFileSync( + path.join(source, "docs", "v1.0", "auth.html"), + "", + ); + + generateBuildOutput(source, target); + + const config = JSON.parse(fs.readFileSync(path.join(target, "config.json"), "utf8")); + assert.deepEqual(config.overrides["docs/v1.0.html"], { path: "docs/v1.0" }); + assert.deepEqual(config.overrides["docs/v1.0/auth.html"], { path: "docs/v1.0/auth" }); + + const indexRoute = config.routes.find((route) => route.src === "^/docs/v1\\.0/?$"); + const authRoute = config.routes.find((route) => route.src === "^/docs/v1\\.0/auth/?$"); + assert.ok(indexRoute, "the archive index needs its own route"); + assert.ok(authRoute, "an archive page needs its own route"); + assert.match(indexRoute.headers["Content-Security-Policy"], /script-src[^;]*'sha256-/); + assert.notEqual( + indexRoute.headers["Content-Security-Policy"], + authRoute.headers["Content-Security-Policy"], + ); + // The dot is escaped, so the route cannot match some other path with any character there. + assert.equal(new RegExp(indexRoute.src).test("/docs/v1x0"), false); + assert.equal(new RegExp(indexRoute.src).test("/docs/v1.0"), true); + } finally { + fs.rmSync(root, { recursive: true, force: true }); + } +}); + test("CSP permits only image origins present in the rendered page", () => { const headers = headersForHTML( 'link',