Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions PLAN.md
Original file line number Diff line number Diff line change
Expand Up @@ -553,8 +553,8 @@ The roadmap in §4 is what we're shipping. This section is *where we are right n
| M5 | Privacy via Cloudflare Access | 🧪 implemented, not yet live-validated | Worker: `accessGuard` middleware verifies the `Cf-Access-Jwt-Assertion` JWT (RS256 via WebCrypto + JWKS cache, no `jose`) and gates `/` + `/r/*` + `/api/*`; enforces only when `ACCESS_AUD` var is set, so public mirrors stay open. `/webhooks/github` + `/health` left unauth. 8 unit tests. CLI: opt-in `gitflare access enable/disable` creates/deletes a self-hosted Access app + allow-list policy and redeploys with `ACCESS_AUD`/`ACCESS_TEAM_DOMAIN` vars. **Caveat:** this gates the dashboard/API only — `git clone` hits Artifacts directly (`*.artifacts.cloudflare.net`), so it is NOT Access-gated; private-clone is a later (v0.4+) item per §6/§11. **TODO before ✅:** verify the live Access apps/policies API shapes + `aud` field, and that `*.workers.dev` hosts are accepted, against a real account. |
| M5.5 | v0.1 polish | 🧪 implemented | Syntax highlighting in the blob viewer (highlight.js core + ~20 curated grammars, server-side; 512 KB cap with `<pre>` fallback; bundle 681→901 KB). README image proxy: new `GET /r/:name/raw/*` serves blob bytes from the Artifacts mirror so images render for private repos and survive GitHub outages — README rewriting now points there instead of `raw.githubusercontent.com`. Styled empty/error states (`ui/states.tsx`): home shows the `npx gitflare init` command, browse 404/500 render through the layout with a way back. |
| M6 | v0.2 CD (MVP slice) | 🧪 implemented | **Self-deploy model (user chose Worker-Secret path).** On push, after sync, the webhook fires `DeployDO` (`waitUntil`), which clones the repo, parses `.gitflare/deploy.yml`, and uploads the pre-built `cloudflare/deploy { kind: worker }` entry via the Workers Scripts multipart API. History in `DeployDO`; Deployments UI; CLI `gitflare deploy enable/disable` stores `CF_DEPLOY_TOKEN` + `CD_ENABLED`. Superseded by M7. |
| M7 | v0.2 CD — complete (per §4) | 🧪 implemented, not yet live-validated | Everything in §4 v0.2: **(1)** real YAML-subset parser (`deploy/yaml.ts`, no heavyweight dep) → **worker bindings** (vars/KV/R2/D1/DO/services) in the Scripts metadata; **(2)** **Pages** deploys via Direct Upload (upload-token → check-missing → upload → create-deployment), with per-branch **previews** (`production_branch`); **(3)** **D1 migrations** applied in order via the D1 query API, opt-in (`apply: true`), idempotent (applied set tracked per-DB in the DO); **(4)** **live deploy-log streaming** over a hibernatable WebSocket (`/r/:name/deployments/stream`) with the Deployments page subscribing; **(5)** **manual / GitHub-down trigger** `gitflare deploy run` + `/control/deploy/run` (auth'd by a `CONTROL_SECRET`, outside Access); **(6)** `gitflare deploy list` + `gitflare deploy rollback [--to <id>]` (rollback redeploys a previous successful commit via a full clone; migrations are forward-only and skipped). 56 unit tests across yaml/workflow/cf-deploy/highlight/access. **TODO before ✅:** one live run against a real Cloudflare account to confirm the Scripts multipart, Pages Direct Upload, and D1 query wire shapes. Known quirk (kept in M8): the v0.2 push path reads workflow/content at the clone's HEAD while recording the webhook sha — same-branch seconds-apart race only; the v0.3 CI path reads strictly at the pushed sha by construction. |
| M8 | v0.3 core CI — sandbox jobs, needs-gated deploys | ✅ core live-validated (2026-07-17, `sinameraji/kimiflare`) | **Format** (`.gitflare/ci.yml`, parsed by `src/ci/workflow.ts` reusing the yaml.ts subset + the shared `cloudflare/deploy` step validator): `on`/`branches`, `jobs:` with `needs:` (parse-time unknown/cycle validation, Kahn topo order), `run:` steps (multi-line via new `\|`/`\|-` block-scalar support in yaml.ts), per-job `env:` + `timeout_minutes` (default 15, max 60); a job is either a run job or a deploy job (mixing = parse error). **Execution**: new `CiDO` (binding `CI`, migration v3) — POST `/run` answers 202 and the pipeline runs as detached DO work; ONE sandbox per run (`@cloudflare/sandbox@0.12.3`, image pinned to match) cloned once from the Artifacts mirror via env-scoped `http.extraHeader` auth (token never in argv/.git/config/stderr; every log line scrubbed for the secret + its base64), then jobs execute in topo order with streamed line-buffered logs (hibernatable WS at `/r/:name/ci/stream`, cap 1000 lines/job). **Deploy jobs delegate to DeployDO** (`mode: "ci"`, strict at-sha ref-aware clone via new `cloneRepoAtRef`, feature branches included) so deploy history/rollback stay in one place; `DeployRecord` gains `workflow`/`job` for cross-format rollback. **Artifact handover**: worker entries built by run jobs are read out of the sandbox (5 MB cap) and shipped via `entryOverrides` — deploys ship what CI built, not the stale committed file. **Safety**: alarm watchdog fails interrupted/over-budget runs + destroys orphaned sandboxes; cancel (CLI `gitflare ci cancel`, `/control/ci/cancel`, runs-page button); push-storm supersede; fail-closed — committed ci.yml + CI disabled means deploy.yml does NOT run ungated; delegated deploys count as green ONLY on `status: "success"` (a "CD not enabled" skip fails the job loudly). **Status postback** via the legacy Commit Status API (context `gitflare/ci`; classic PAT can't use Checks), soft-fail. **CLI**: `gitflare ci enable/disable/run/list/cancel`; enable provisions `[[containers]]` (public `docker.io/cloudflare/sandbox` image — no local Docker; `--instance-type`, default `standard-1`); migrations v3+v4 are emitted UNconditionally so cross-machine redeploys never miss applied tags; CONTROL_SECRET reuse is symmetric with `deploy enable`; disable drops only `CI_ENABLED` (idle containers cost $0). 51 new unit tests (97 total worker-side). Worker bundle 901 KB → 1557 KB (sandbox SDK +433 KB, capnweb +104 KB, containers +75 KB); ~400 KB gzipped, fine for Workers limits. **Deliberate deviations from §4 v0.3**: `runtime: worker` rejected with a clear error (Dynamic Workers can't run npm — no Linux userland); per-job `image:` rejected (containers pin the image at Worker-deploy time); ArtifactFS mount not used (sandbox `git clone`; cold container start pulls the image from Docker Hub — minutes, not the §4 \<500ms); jobs in one run share a sandbox/workspace (GHA users expect isolation — M9); Pages artifact handover, R2 build cache, Browser Run, Actions importer, per-job status contexts, `gitflare sandbox` verb → M9. **Post-review hardening** (a 4-lens adversarial review of the diff, 18 findings, all verifier-confirmed): FIXED — watchdog/pipeline verdict races (a `finalizedRunIds` fence stops a lagging coroutine resurrecting a run the watchdog failed, and the watchdog no longer erases a concurrent cancel); a fallback alarm armed in `begin()` so eviction before the deadline alarm can't leave a zombie "running" record; `ARTIFACTS.get` moved inside the try (+ a last-resort commit status) so a push can't vanish traceless; tag/non-branch pushes skipped at the webhook (were minting failed runs + red statuses); artifact handover gated on the whole `needs` closure of run jobs succeeding (was shipping a failed build's output); the dead "fall back to deploy.yml" path on missing SANDBOX replaced with an honest fail-closed error; delegated-deploy response mapping extracted to a tested pure fn (`ci/delegation.ts`, all four shapes incl. skipped-is-not-green); the remote-URL allowlist tightened to reject shell metacharacters; duplicate webhook deliveries de-duped by branch+sha; dashboard cancel gated behind Access (was open on public mirrors — CLI `gitflare ci cancel` uses CONTROL_SECRET); CONTROL_SECRET now always (re)written on enable and config persisted the moment the Worker goes live (was strandable across machines / on a partial enable). RESIDUAL (documented, not yet fixed — acceptable for a not-live-validated milestone): a run still queued in-memory when the DO is evicted (e.g. by a redeploy) is dropped with no record; webhook delivery order isn't guaranteed by GitHub, so the supersede optimization is best-effort (out-of-order delivery can run the older sha last); cancel is a no-op during a run's brief pre-clone window and cannot un-ship a deploy job that already returned success (the run message says so). **Live-validated end-to-end** (2026-07-17, `sinameraji/kimiflare` on a real Workers Paid account): `gitflare ci enable` provisions the `[[containers]]` app from wrangler TOML; a push → webhook → sync → CI run #1 succeeded in 12s — the Sandbox booted (`node v22`), cloned the mirror at the pushed sha with a short-lived read token (URL logged WITHOUT credentials — env-scoped auth + scrubbing confirmed), and ran all 4 `run:` steps with streamed line-buffered logs, resolved env vars, and captured exit codes. Findings fixed along the way (all on this branch): (1) container deploy needs BOTH **Account → Cloudchamber:Edit** and **Account → Containers:Edit** token permissions (Containers alone → `Forbidden`); (2) the container app name can't contain consecutive dashes, so we emit an explicit sanitized `name` (the `owner--repo` worker name has `--`); (3) a latent M1 sync bug — `artifactsRepo.remote` on the current Artifacts beta is a lazy RPC proxy that stringifies to `[object JsRpcProperty]`; now threads the REPO_MAP remote string (would have broken any live v0.2 deploy too); (4) the push webhook awaited the full sync and blew past GitHub's 10s delivery timeout on a real repo (504 + redeliveries) — now responds 202 immediately, sync + dispatch in `waitUntil`; (5) commit-status postback soft-fails observably (the run logs *why* — here, an expired GITHUB_TOKEN). Config also cross-checked via `wrangler deploy --dry-run` (both CI-provisioned and default TOML resolve all four DO exports incl. unbound-Sandbox migration v4). **Not yet exercised live** (deferred soak): the deploy-job → DeployDO delegation, the alarm watchdog firing, cancel mid-run, hour-scale runs / `sleepAfter` container-activity behavior, and commit statuses actually rendering (needs a fresh GITHUB_TOKEN). |
| M7 | v0.2 CD — complete (per §4) | 🧪 worker-deploy path live-validated; Pages + D1 pending | Everything in §4 v0.2: **(1)** real YAML-subset parser (`deploy/yaml.ts`, no heavyweight dep) → **worker bindings** (vars/KV/R2/D1/DO/services) in the Scripts metadata; **(2)** **Pages** deploys via Direct Upload (upload-token → check-missing → upload → create-deployment), with per-branch **previews** (`production_branch`); **(3)** **D1 migrations** applied in order via the D1 query API, opt-in (`apply: true`), idempotent (applied set tracked per-DB in the DO); **(4)** **live deploy-log streaming** over a hibernatable WebSocket (`/r/:name/deployments/stream`) with the Deployments page subscribing; **(5)** **manual / GitHub-down trigger** `gitflare deploy run` + `/control/deploy/run` (auth'd by a `CONTROL_SECRET`, outside Access); **(6)** `gitflare deploy list` + `gitflare deploy rollback [--to <id>]` (rollback redeploys a previous successful commit via a full clone; migrations are forward-only and skipped). 56 unit tests across yaml/workflow/cf-deploy/highlight/access. **Live status (2026-07-17):** the **worker** deploy path (`uploadWorkerScript`: Scripts multipart upload + bindings + the workers.dev-subdomain enable fix) is validated — the v0.3 CI deploy job delegates through this exact code and shipped a reachable worker (see M8). The **sync** the deploy runs on top of is validated too (the `JsRpcProperty` remote fix). **Still pending a live run:** Pages Direct Upload and the D1 migration query path (both untouched by the CI validation). Known quirk (kept in M8): the v0.2 push path reads workflow/content at the clone's HEAD while recording the webhook sha — same-branch seconds-apart race only; the v0.3 CI path reads strictly at the pushed sha by construction. |
| M8 | v0.3 core CI — sandbox jobs, needs-gated deploys | ✅ core live-validated (2026-07-17, `sinameraji/kimiflare`) | **Format** (`.gitflare/ci.yml`, parsed by `src/ci/workflow.ts` reusing the yaml.ts subset + the shared `cloudflare/deploy` step validator): `on`/`branches`, `jobs:` with `needs:` (parse-time unknown/cycle validation, Kahn topo order), `run:` steps (multi-line via new `\|`/`\|-` block-scalar support in yaml.ts), per-job `env:` + `timeout_minutes` (default 15, max 60); a job is either a run job or a deploy job (mixing = parse error). **Execution**: new `CiDO` (binding `CI`, migration v3) — POST `/run` answers 202 and the pipeline runs as detached DO work; ONE sandbox per run (`@cloudflare/sandbox@0.12.3`, image pinned to match) cloned once from the Artifacts mirror via env-scoped `http.extraHeader` auth (token never in argv/.git/config/stderr; every log line scrubbed for the secret + its base64), then jobs execute in topo order with streamed line-buffered logs (hibernatable WS at `/r/:name/ci/stream`, cap 1000 lines/job). **Deploy jobs delegate to DeployDO** (`mode: "ci"`, strict at-sha ref-aware clone via new `cloneRepoAtRef`, feature branches included) so deploy history/rollback stay in one place; `DeployRecord` gains `workflow`/`job` for cross-format rollback. **Artifact handover**: worker entries built by run jobs are read out of the sandbox (5 MB cap) and shipped via `entryOverrides` — deploys ship what CI built, not the stale committed file. **Safety**: alarm watchdog fails interrupted/over-budget runs + destroys orphaned sandboxes; cancel (CLI `gitflare ci cancel`, `/control/ci/cancel`, runs-page button); push-storm supersede; fail-closed — committed ci.yml + CI disabled means deploy.yml does NOT run ungated; delegated deploys count as green ONLY on `status: "success"` (a "CD not enabled" skip fails the job loudly). **Status postback** via the legacy Commit Status API (context `gitflare/ci`; classic PAT can't use Checks), soft-fail. **CLI**: `gitflare ci enable/disable/run/list/cancel`; enable provisions `[[containers]]` (public `docker.io/cloudflare/sandbox` image — no local Docker; `--instance-type`, default `standard-1`); migrations v3+v4 are emitted UNconditionally so cross-machine redeploys never miss applied tags; CONTROL_SECRET reuse is symmetric with `deploy enable`; disable drops only `CI_ENABLED` (idle containers cost $0). 51 new unit tests (97 total worker-side). Worker bundle 901 KB → 1557 KB (sandbox SDK +433 KB, capnweb +104 KB, containers +75 KB); ~400 KB gzipped, fine for Workers limits. **Deliberate deviations from §4 v0.3**: `runtime: worker` rejected with a clear error (Dynamic Workers can't run npm — no Linux userland); per-job `image:` rejected (containers pin the image at Worker-deploy time); ArtifactFS mount not used (sandbox `git clone`; cold container start pulls the image from Docker Hub — minutes, not the §4 \<500ms); jobs in one run share a sandbox/workspace (GHA users expect isolation — M9); Pages artifact handover, R2 build cache, Browser Run, Actions importer, per-job status contexts, `gitflare sandbox` verb → M9. **Post-review hardening** (a 4-lens adversarial review of the diff, 18 findings, all verifier-confirmed): FIXED — watchdog/pipeline verdict races (a `finalizedRunIds` fence stops a lagging coroutine resurrecting a run the watchdog failed, and the watchdog no longer erases a concurrent cancel); a fallback alarm armed in `begin()` so eviction before the deadline alarm can't leave a zombie "running" record; `ARTIFACTS.get` moved inside the try (+ a last-resort commit status) so a push can't vanish traceless; tag/non-branch pushes skipped at the webhook (were minting failed runs + red statuses); artifact handover gated on the whole `needs` closure of run jobs succeeding (was shipping a failed build's output); the dead "fall back to deploy.yml" path on missing SANDBOX replaced with an honest fail-closed error; delegated-deploy response mapping extracted to a tested pure fn (`ci/delegation.ts`, all four shapes incl. skipped-is-not-green); the remote-URL allowlist tightened to reject shell metacharacters; duplicate webhook deliveries de-duped by branch+sha; dashboard cancel gated behind Access (was open on public mirrors — CLI `gitflare ci cancel` uses CONTROL_SECRET); CONTROL_SECRET now always (re)written on enable and config persisted the moment the Worker goes live (was strandable across machines / on a partial enable). RESIDUAL (documented, not yet fixed — acceptable for a not-live-validated milestone): a run still queued in-memory when the DO is evicted (e.g. by a redeploy) is dropped with no record; webhook delivery order isn't guaranteed by GitHub, so the supersede optimization is best-effort (out-of-order delivery can run the older sha last); cancel is a no-op during a run's brief pre-clone window and cannot un-ship a deploy job that already returned success (the run message says so). **Live-validated end-to-end** (2026-07-17, `sinameraji/kimiflare` on a real Workers Paid account): `gitflare ci enable` provisions the `[[containers]]` app from wrangler TOML; a push → webhook → sync → CI run #1 succeeded in 12s — the Sandbox booted (`node v22`), cloned the mirror at the pushed sha with a short-lived read token (URL logged WITHOUT credentials — env-scoped auth + scrubbing confirmed), and ran all 4 `run:` steps with streamed line-buffered logs, resolved env vars, and captured exit codes. Findings fixed along the way (all on this branch): (1) container deploy needs BOTH **Account → Cloudchamber:Edit** and **Account → Containers:Edit** token permissions (Containers alone → `Forbidden`); (2) the container app name can't contain consecutive dashes, so we emit an explicit sanitized `name` (the `owner--repo` worker name has `--`); (3) a latent M1 sync bug — `artifactsRepo.remote` on the current Artifacts beta is a lazy RPC proxy that stringifies to `[object JsRpcProperty]`; now threads the REPO_MAP remote string (would have broken any live v0.2 deploy too); (4) the push webhook awaited the full sync and blew past GitHub's 10s delivery timeout on a real repo (504 + redeliveries) — now responds 202 immediately, sync + dispatch in `waitUntil`; (5) commit-status postback soft-fails observably (the run logs *why* — here, an expired GITHUB_TOKEN). Config also cross-checked via `wrangler deploy --dry-run` (both CI-provisioned and default TOML resolve all four DO exports incl. unbound-Sandbox migration v4). **Deploy-job delegation also validated** (2026-07-17): a two-job `ci.yml` (a `run` job builds `dist/worker.js` in the sandbox — NOT committed — then a `needs`-gated deploy job) ran green end-to-end; the log shows `using CI-built dist/worker.js (103 bytes)` → the sandbox artifact flowed through `entryOverrides` to `DeployDO` and shipped, and the deployed worker served the CI-built body at **HTTP 200**. This surfaced + fixed a **latent v0.2 bug**: `DeployDO`'s raw Scripts-API upload never enabled the `workers.dev` subdomain (wrangler does; the API doesn't), so deployed workers 404'd — `uploadWorkerScript` now enables it and returns the reachable URL. Commit statuses also confirmed rendering on GitHub once a fresh `GITHUB_TOKEN` was set. **Not yet exercised live** (deferred soak): the alarm watchdog firing, cancel mid-run, and hour-scale runs / `sleepAfter` container-activity behavior. |

### What's in the repo right now (as of M0)

Expand Down
Loading
Loading