Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 7 additions & 7 deletions .github/workflows/post-deploy-smoke.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ on:

permissions:
contents: read
checks: read

jobs:
sentinel:
Expand Down Expand Up @@ -50,10 +51,9 @@ jobs:
cache: npm
- name: Install locked dependencies
run: npm ci
- name: Wait for deployed SHA and smoke production
run: >-
npm run smoke --
--environment production
--base-url https://ratemyplace.org
--expected-release ${{ github.event.workflow_run.head_sha }}
--wait-for-release-ms 600000
- name: Resolve trusted immutable Pages deployment and smoke it
env:
GITHUB_TOKEN: ${{ github.token }}
run: |
deployment_origin="$(npx tsx scripts/pages-deployment-url.ts --repository "${{ github.repository }}" --sha "${{ github.event.workflow_run.head_sha }}" --wait-ms 600000)"
npm run smoke -- --environment preview --base-url "$deployment_origin" --expected-release ${{ github.event.workflow_run.head_sha }} --wait-for-release-ms 600000
12 changes: 10 additions & 2 deletions .planning/codebase/INTEGRATIONS.md
Original file line number Diff line number Diff line change
Expand Up @@ -133,8 +133,16 @@
explicitly fails red when `quality` did not succeed and explicitly passes on success;
it never checks out, installs dependencies, or runs smoke. The separate, success-only
`smoke` job needs that sentinel and alone owns cancellable `production-smoke`
concurrency. It then waits for Cloudflare Pages to serve the exact commit SHA and runs
the read-only production smoke suite.
concurrency. With only `contents: read`, `checks: read`, and the built-in GitHub token,
it resolves the exact commit's single successful `Cloudflare Pages` check from
`cloudflare-workers-and-pages`, validates the advertised immutable eight-hex
`ratemyplace-64y.pages.dev` deployment origin, and runs the read-only preview smoke suite
against that exact release SHA. The canonical `ratemyplace.org` smoke remains a separate
manual check: Free-plan Bot Fight Mode can managed-challenge GitHub-hosted runners at its
health route, and this workflow neither changes Bot Fight Mode nor uses a Cloudflare API
credential. Future Cloudflare synthetic monitoring, if configured, is auxiliary
availability monitoring rather than an exact-release gate; a future machine-health
boundary must stay separately specified from exact-release verification.
- The repository workflows do not deploy or roll back Cloudflare Pages. A `main` branch
ruleset/required-check activation is not asserted here; Task 7 must verify that external
configuration separately.
Expand Down
31 changes: 26 additions & 5 deletions docs/runbooks/release-smoke.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,9 @@ $prHeadSha = (git rev-parse HEAD).Trim()
npm run smoke -- --environment preview --base-url $previewOrigin --expected-release $prHeadSha
```

For production, use the merged `main` SHA and the canonical production origin.
For an independent manual check of the canonical production domain, use the merged `main`
SHA and the canonical production origin. This is deliberately separate from the automated
exact-release gate.

```powershell
npm ci
Expand Down Expand Up @@ -76,10 +78,29 @@ red; a successful completion explicitly passes it. The sentinel has no checkout,
dependency installation, or smoke step, so failed CI never performs those actions.

Only the separate, success-gated `smoke` job needs the passing sentinel. It alone owns the
cancellable `production-smoke` concurrency group, then waits for Cloudflare Pages,
verifies that `/api/health` reports the triggering release SHA, and runs the full
read-only smoke suite. A failed post-deploy smoke means the release must not be called
healthy. It does not cause an automatic rollback.
cancellable `production-smoke` concurrency group. Using the built-in GitHub token with
read-only `checks: read`, it polls the triggering commit's GitHub check runs for exactly
one successful `Cloudflare Pages` check from `cloudflare-workers-and-pages`. The resolver
accepts only the immutable `https://<8-hex>.ratemyplace-64y.pages.dev` origin advertised by
that check, then verifies `/api/health` reports the triggering release SHA and runs the
full read-only smoke suite there. It fails closed for an untrusted, ambiguous, incomplete,
or malformed resolution and never deploys, rolls back, mutates Cloudflare configuration, or
uses a Cloudflare credential.

Cloudflare Free-plan Bot Fight Mode may managed-challenge GitHub-hosted runners at the
canonical `/api/health` route. It remains unchanged: the automated exact-release gate uses
the trusted immutable Pages deployment instead. After an automated pass, perform the
separate manual canonical-domain command above from an independent network when the
canonical routing check is needed. Cloudflare synthetic monitoring may be added later as
auxiliary availability monitoring, but it is not an exact-release gate.

Keep the machine-health boundary narrow: this release smoke proves public responses from a
specific immutable deployment. Any future machine-health endpoint or synthetic monitor must
be designed and documented as separate availability telemetry, not treated as evidence that
the canonical domain served a particular release.

A failed post-deploy smoke means the release must not be called healthy. It does not cause
an automatic rollback.

## Failure triage and approval boundary

Expand Down
33 changes: 33 additions & 0 deletions scripts/pages-deployment-url.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
import { resolvePagesDeploymentOrigin, type PagesDeploymentOptions } from '../src/lib/pagesDeployment';

const flags = new Set(['--repository', '--sha', '--wait-ms']);

function parseArgs(args: string[]): Omit<PagesDeploymentOptions, 'token'> {
const values = new Map<string, string>();
for (let index = 0; index < args.length; index += 2) {
const flag = args[index];
const value = args[index + 1];
if (!flags.has(flag)) throw new Error('Unknown argument');
if (!value || value.startsWith('--') || values.has(flag)) throw new Error('Invalid argument');
values.set(flag, value);
}
const repository = values.get('--repository');
const sha = values.get('--sha');
const waitMs = values.get('--wait-ms');
if (!repository || !sha || !waitMs || values.size !== flags.size) throw new Error('Missing required argument');
if (!/^\d+$/.test(waitMs)) throw new Error('Invalid wait');
return { repository, sha, waitMs: Number(waitMs) };
}

async function main(): Promise<void> {
try {
const origin = await resolvePagesDeploymentOrigin({ ...parseArgs(process.argv.slice(2)), token: process.env.GITHUB_TOKEN ?? '' });
console.log(origin);
} catch (error) {
const message = error instanceof Error ? error.message : 'Failed to resolve Pages deployment';
console.error(`Pages deployment resolver error: ${message}`);
process.exitCode = 1;
}
}

void main();
164 changes: 164 additions & 0 deletions src/lib/__tests__/pagesDeployment.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,164 @@
import { describe, expect, it, vi } from 'vitest';
import { resolvePagesDeploymentOrigin, type PagesDeploymentDependencies } from '../pagesDeployment';

const SHA = '3c0350327bef3ed8bf6afa34a3723294cf49b59d';
const ORIGIN = 'https://a1b2c3d4.ratemyplace-64y.pages.dev';

interface CheckRunFixture {
app?: { slug?: unknown };
name?: unknown;
head_sha?: unknown;
status?: unknown;
conclusion?: unknown;
output?: { summary?: unknown };
}

const apiResponse = (checkRuns: CheckRunFixture[], totalCount = checkRuns.length) => new Response(JSON.stringify({
total_count: totalCount,
check_runs: checkRuns,
}), { status: 200, headers: { 'content-type': 'application/json' } });

const trustedCheck = (overrides: CheckRunFixture = {}): CheckRunFixture => ({
app: { slug: 'cloudflare-workers-and-pages' },
name: 'Cloudflare Pages',
head_sha: SHA,
status: 'completed',
conclusion: 'success',
output: { summary: `Deployment URL: ${ORIGIN}` },
...overrides,
});

const dependencies = (responses: Array<Response | Error>, start = 0): PagesDeploymentDependencies => {
let now = start;
return {
fetch: vi.fn(async () => {
const next = responses.shift();
if (!next) throw new Error('unexpected request');
if (next instanceof Error) throw next;
return next;
}),
now: () => now,
sleep: vi.fn(async (milliseconds: number) => { now += milliseconds; }),
};
};

const resolve = (deps: PagesDeploymentDependencies, waitMs = 600_000) => resolvePagesDeploymentOrigin({
repository: 'example/ratemyplace',
sha: SHA,
token: 'test-token',
waitMs,
}, deps);

describe('resolvePagesDeploymentOrigin', () => {
it('returns the single trusted immutable deployment origin for the requested SHA', async () => {
await expect(resolve(dependencies([apiResponse([trustedCheck()])]))).resolves.toBe(ORIGIN);
});

it('selects the immutable URL from a trusted Pages summary that also advertises its branch alias', async () => {
const summary = [
'Preview URL: https://0d4541c6.ratemyplace-64y.pages.dev',
'Branch Preview URL: https://codex-phase-22a-smoke-delive.ratemyplace-64y.pages.dev',
].join('\n');

await expect(resolve(dependencies([apiResponse([trustedCheck({ output: { summary } })])]))).resolves.toBe(
'https://0d4541c6.ratemyplace-64y.pages.dev',
);
});

it('rejects a malformed branch alias even when the same summary has a valid immutable URL', async () => {
const summary = `${ORIGIN} https://branch-.ratemyplace-64y.pages.dev`;

await expect(resolve(dependencies([apiResponse([trustedCheck({ output: { summary } })])]))).rejects.toThrow(
/invalid Pages hostname/i,
);
});

it.each([
['missing trusted check', [apiResponse([]), apiResponse([trustedCheck()])]],
['queued trusted check', [apiResponse([trustedCheck({ status: 'queued', conclusion: null })]), apiResponse([trustedCheck()])]],
['in-progress trusted check', [apiResponse([trustedCheck({ status: 'in_progress', conclusion: null })]), apiResponse([trustedCheck()])]],
])('retries %s inside the wait budget', async (_name, responses) => {
await expect(resolve(dependencies(responses), 20_000)).resolves.toBe(ORIGIN);
});

it('ignores untrusted checks while waiting for the Cloudflare Pages check', async () => {
const untrusted = trustedCheck({ app: { slug: 'someone-else' } });
await expect(resolve(dependencies([apiResponse([untrusted]), apiResponse([trustedCheck()])]), 20_000)).resolves.toBe(ORIGIN);
});

it.each([
['incomplete pagination', apiResponse([trustedCheck()], 2), /incomplete check-run pagination/i],
['wrong trusted head SHA', apiResponse([trustedCheck({ head_sha: 'a'.repeat(40) })]), /head SHA/i],
['unsuccessful completion', apiResponse([trustedCheck({ conclusion: 'failure' })]), /did not succeed/i],
['ambiguous trusted matches', apiResponse([trustedCheck(), trustedCheck()]), /multiple trusted/i],
['missing summary', apiResponse([trustedCheck({ output: {} })]), /missing check summary/i],
['malformed summary', apiResponse([trustedCheck({ output: { summary: 7 } })]), /malformed check summary/i],
['invalid project hostname', apiResponse([trustedCheck({ output: { summary: 'https://a1b2c3d4.wrong-project.pages.dev' } })]), /invalid Pages hostname/i],
['explicit default Pages port', apiResponse([trustedCheck({ output: { summary: 'https://a1b2c3d4.ratemyplace-64y.pages.dev:443' } })]), /invalid Pages hostname/i],
['wrong-project branch alias', apiResponse([trustedCheck({ output: { summary: 'https://codex-phase-22a-smoke-delive.wrong-project.pages.dev' } })]), /invalid Pages hostname/i],
['branch alias with explicit default port', apiResponse([trustedCheck({ output: { summary: 'https://codex-phase-22a-smoke-delive.ratemyplace-64y.pages.dev:443' } })]), /invalid Pages hostname/i],
['multiple deployment origins', apiResponse([trustedCheck({ output: { summary: `${ORIGIN} https://b2c3d4e5.ratemyplace-64y.pages.dev` } })]), /multiple immutable deployment origins/i],
])('fails closed for %s', async (_name, response, expected) => {
await expect(resolve(dependencies([response]))).rejects.toThrow(expected);
});

it('fails safely when the GitHub API request fails', async () => {
await expect(resolve(dependencies([new Error('network unavailable')]))).rejects.toThrow(/GitHub check-run request failed/i);
});

it('rejects a trusted response delivered after the resolver deadline', async () => {
let now = 0;
const delayedResponse: PagesDeploymentDependencies = {
fetch: vi.fn(async () => {
now = 11;
return apiResponse([trustedCheck()]);
}),
now: () => now,
sleep: vi.fn(async () => undefined),
};

await expect(resolve(delayedResponse, 10)).rejects.toThrow(/deadline/i);
});

it('aborts an unresolved GitHub request when the resolver deadline expires', async () => {
vi.useFakeTimers();
try {
let now = 0;
let outcome: unknown;
const unresolvedRequest: PagesDeploymentDependencies = {
fetch: vi.fn((_input, init) => new Promise<Response>((_resolve, reject) => {
init?.signal?.addEventListener('abort', () => {
now = 10;
reject(new Error('request aborted'));
});
})),
now: () => now,
sleep: vi.fn(async () => undefined),
};
void resolve(unresolvedRequest, 10).then(
() => { outcome = 'resolved'; },
(error: unknown) => { outcome = error; },
);

await vi.advanceTimersByTimeAsync(10);

expect(outcome).toBeInstanceOf(Error);
expect((outcome as Error).message).toMatch(/deadline/i);
} finally {
vi.useRealTimers();
}
});

it.each([
['missing trusted head SHA', trustedCheck({ head_sha: undefined })],
['missing trusted status', trustedCheck({ status: undefined })],
])('fails immediately for %s', async (_name, malformedTrustedCheck) => {
await expect(resolve(dependencies([apiResponse([malformedTrustedCheck])]), 10_000))
.rejects.toThrow(/malformed trusted Cloudflare Pages check/i);
});

it('fails after the bounded deadline while a trusted check remains pending', async () => {
const pending = apiResponse([trustedCheck({ status: 'queued', conclusion: null })]);
await expect(resolve(dependencies([pending]), 10_000)).rejects.toThrow(/deadline/i);
});
});
40 changes: 30 additions & 10 deletions src/lib/__tests__/workflowContracts.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ import { resolve } from 'node:path';
import { describe, expect, it } from 'vitest';

const workflowPath = (...parts: string[]) => resolve(process.cwd(), '.github', 'workflows', ...parts);
const readWorkflow = (name: string) => readFileSync(workflowPath(name), 'utf8');
const readWorkflow = (name: string) => readFileSync(workflowPath(name), 'utf8').replace(/\r\n/g, '\n');

const getWorkflowJob = (workflow: string, name: string) => {
const jobsStart = workflow.indexOf('jobs:\n');
Expand Down Expand Up @@ -50,7 +50,7 @@ const findPermissionsDeclarations = (lines: string[]): PermissionsDeclaration[]
return declarations;
};

const assertReadOnlyPermissionsBlock = (workflow: string) => {
const assertReadOnlyPermissionsBlock = (workflow: string, allowedReadPermissions: string[] = ['contents']) => {
const lines = workflow.split(/\r?\n/);
const declarations = findPermissionsDeclarations(lines);

Expand All @@ -71,15 +71,19 @@ const assertReadOnlyPermissionsBlock = (workflow: string) => {
blockEntries.push(line);
}

expect(blockEntries).toHaveLength(1);
expect(blockEntries[0]).toMatch(/^[ \t]+contents[ \t]*:[ \t]*read[ \t]*(?:#.*)?$/);
expect(blockEntries).toHaveLength(allowedReadPermissions.length);
expect(blockEntries).toEqual(expect.arrayContaining(
allowedReadPermissions.map((permission) => expect.stringMatching(
new RegExp(`^[ \\t]+${permission}[ \\t]*:[ \\t]*read[ \\t]*(?:#.*)?$`),
)),
));
};

const assertLeastPrivilege = (workflow: string) => {
assertReadOnlyPermissionsBlock(workflow);
const assertLeastPrivilege = (workflow: string, allowedReadPermissions?: string[]) => {
assertReadOnlyPermissionsBlock(workflow, allowedReadPermissions);
expect(workflow).not.toMatch(/\bwrangler\b/i);
expect(workflow).not.toMatch(/\b(?:d1|r2)\b/i);
expect(workflow).not.toMatch(/(?:CLOUDFLARE|CF)_[A-Z_]*TOKEN/i);
expect(workflow).not.toMatch(/(?:CLOUDFLARE|CF)_[A-Z_]*(?:TOKEN|KEY)\b/i);
expect(workflow).not.toMatch(/secrets\./i);
expect(workflow).not.toMatch(/pull_request_target/i);
expect(workflow).not.toMatch(/permissions:\s*write-all/i);
Expand Down Expand Up @@ -113,7 +117,7 @@ describe('release workflow contracts', () => {

expect(workflow).toMatch(/^name: Post-deploy smoke$/m);
expect(workflow).toMatch(/^ workflow_run:\n workflows: \[CI\]\n types: \[completed\]$/m);
assertLeastPrivilege(workflow);
assertLeastPrivilege(workflow, ['contents', 'checks']);
expect(beforeJobs).not.toMatch(/^concurrency:/m);

expect(sentinel).toMatch(/github\.event\.workflow_run\.event == 'push'/);
Expand All @@ -139,14 +143,21 @@ describe('release workflow contracts', () => {
expect(smoke).toMatch(/uses: actions\/setup-node@v7\n with:\n node-version-file: \.node-version\n cache: npm/);
expect(smoke.indexOf('run: npm ci')).toBeGreaterThan(checkout);
expect(smoke.indexOf('run: npm ci')).toBeLessThan(smokeCommand);
expect(smoke).toMatch(/--base-url https:\/\/ratemyplace\.org/);
expect(smoke).toMatch(/GITHUB_TOKEN: \$\{\{ github\.token \}\}/);
expect(smoke).toMatch(/scripts\/pages-deployment-url\.ts/);
expect(smoke).toMatch(/--repository \"\$\{\{ github\.repository \}\}\"/);
expect(smoke).toMatch(/--sha \"\$\{\{ github\.event\.workflow_run\.head_sha \}\}\"/);
expect(smoke).toMatch(/--wait-ms 600000/);
expect(smoke).toMatch(/--environment preview/);
expect(smoke).toMatch(/--base-url "\$deployment_origin"/);
expect(smoke).not.toMatch(/--base-url https:\/\/ratemyplace\.org/);
expect(smoke).toMatch(/--expected-release \$\{\{ github\.event\.workflow_run\.head_sha \}\}/);
expect(smoke).toMatch(/--wait-for-release-ms 600000/);
});

it('keeps Cloudflare deployment and privileged operations out of repository workflows', () => {
for (const name of ['ci.yml', 'post-deploy-smoke.yml']) {
assertLeastPrivilege(readWorkflow(name));
assertLeastPrivilege(readWorkflow(name), name === 'post-deploy-smoke.yml' ? ['contents', 'checks'] : undefined);
}
});

Expand Down Expand Up @@ -200,6 +211,15 @@ describe('release workflow contracts', () => {
expect(() => assertLeastPrivilege(dangerousWorkflow)).toThrow();
});

it('rejects a Cloudflare API key credential', () => {
const dangerousWorkflow = readWorkflow('ci.yml').replace(
' quality:\n',
' quality:\n env:\n CLOUDFLARE_API_KEY: unsafe\n',
);

expect(() => assertLeastPrivilege(dangerousWorkflow)).toThrow();
});

it('allows harmless write-like shell text inside a run block', () => {
const harmlessWorkflow = readWorkflow('ci.yml').replace(
'run: npm ci',
Expand Down
Loading