Skip to content

Launch, cancel, and OAuth consent from the web UI (Phase C)#31

Merged
zbuc merged 3 commits into
mainfrom
issue-30
Jul 22, 2026
Merged

Launch, cancel, and OAuth consent from the web UI (Phase C)#31
zbuc merged 3 commits into
mainfrom
issue-30

Conversation

@zbuc

@zbuc zbuc commented Jul 22, 2026

Copy link
Copy Markdown
Owner

Closes #30 — Phase C of the #26 design: the viewer gains its state-changing surface behind the full CSRF stack, onboarding buttons go live, and OAuth consent is drivable from the browser.

What's here

Viewer (now gmail_stats::webapp, with src/bin/web.rs a thin main so tests drive the exact served router):

  • POST /api/runs{"source":"gmail_api"}, {"source":"mbox","path":...}, or {"source":..., "resume_run_id":N}. Spawns the ingester as a supervised child: fixed argv (never a shell), stdin null, stderr into a 200-line in-memory ring buffer, kill_on_drop(false); binary located as a sibling of current_exe() with GMAIL_STATS_INGEST_BIN override; spawned runs get --quiet and the viewer's --db. 202 {run_id} (the child mints the id by inserting its run row; the viewer waits for it, matched by the child's pid), 409 run_active (friendly flock pre-check, plus mapping the child's own authoritative flock refusal after a lost TOCTOU race), 422 bad_path/bad_resume/bad_source (absolute + regular + readable + From magic sniff), 500 spawn_failed with a cargo build hint.
  • POST /api/runs/{id}/cancel — owned runs only; SIGTERM via the supervisor that owns the child handle (never a DB-supplied pid), SIGKILL after 10s, abandoned marking left to the existing janitor (the viewer still performs zero DB writes). 409 not_owned otherwise.
  • GET /api/runs/{id}/log — owned child's stderr ring buffer, memory-only, gone on viewer restart.
  • CSRF per Design proposal: drive scans and imports from the web UI #26's Security section, all three layers on every mutating route: strict Content-Type: application/json (415), X-Gmail-Stats-Csrf equal to the per-process token from /api/status (403, constant-time compare), Origin/Sec-Fetch-Site validated when present, absent passes. No CORS headers anywhere; Host guard and security headers unchanged. One deliberate reading: the Origin allow-list checks scheme http + host 127.0.0.1/localhost without pinning the port (mirroring the Host guard — any loopback origin is the same local user, and the token is the unforgeable layer).
  • /api/status now reports owns_active_run truthfully and exposes auth_url on run rows.

Scanner:

  • Custom InstalledFlowDelegate writes awaiting_auth + auth_url onto the run row (new AuthState writer-channel message, single-writer discipline intact) and returns it to running once tokens arrive; token acquisition is eager and bounded by GMAIL_STATS_AUTH_TIMEOUT (default 600s) → failed/auth_required (policy_enforced keeps its kind). No migration 2 needed — auth_url shipped with migration 1. Terminal UX unchanged (URL still printed unless --quiet). The delegate holds a weak channel sender: the authenticator outlives the scan loop, and a strong clone would have deadlocked the end-of-scan writer drain (caught by a test hang).
  • scan --resume <run_id> (Design proposal: drive scans and imports from the web UI #26 Q4, best-effort): starts listing from the persisted page token; a rejected/stale token falls back to page one silently (logged in verbose mode), correct either way via seen_mails dedupe.

Front-end: launch buttons on the onboarding cards (mbox absolute-path input, inline 422 errors), Cancel + Log only when owns_active_run, validated Authorize in Google link (https://accounts.google.com/ origin only, target=_blank rel="noopener noreferrer", inert text otherwise), Retry on failed runs, Resume import on interrupted imports, Resume — may restart from the beginning on failed scans, policy_enforced deep-link to the Takeout card. All POSTs carry the CSRF header from the latest status payload; all dynamic strings via textContent. Launch controls live in .live-only blocks that start hidden and are revealed only after /api/status responds, so the static demo never shows ingestion UI or fires a mutating request. demo/build_demo.py anchors intact.

Tests (all passing locally: 90 Rust tests + 6 harness scenarios + live E2E)

  • CSRF matrix (tests/web_mutating.rs, via tower::ServiceExt::oneshot on both mutating routes): form/multipart/text-plain and missing content types → 415; JSON without token → 403; wrong token → 403; hostile Origins (https://evil.example, http://evil.example, https://127.0.0.1, null) → 403; Sec-Fetch-Site: cross-site/same-site → 403; valid trio → 202; second start while active → 409 run_active; relative/missing/directory/non-mbox paths and bad resume ids → 422; not-owned cancel → 409 not_owned; malformed JSON → 400. Every response asserted to carry no Access-Control-* headers.
  • Supervision via a test-only fake_ingester binary (takes the real flock, writes real run rows; modes: complete / crash / hang+trap / hang+ignore / stderr / flockfail / norow): normal completion; crash mid-run → open row, freed flock, abandoned by the next real ingester's janitor; cancel SIGTERM honored → clean cancelled row; SIGTERM ignored → SIGKILL escalation (shortened grace), then janitor abandonment; viewer restart → owns_active_run:false, cancel/log refused, run continues and the original owner still cancels it; log ring capped at the last 200 lines; both flock-conflict shapes (pre-check 409, and the child losing the race post-pre-check still mapping to 409).
  • Auth flow unit tests: the delegate call writes awaiting_auth + URL to the row and the post-consent message returns it to running (no real OAuth — the delegate is invoked directly); consent timeout maps to auth_required, a policy_enforced message keeps its kind.
  • Front-end harness (python3 tests/frontend/harness.py, headless Chrome + a stub server that records every request): launch clicks POST with the CSRF token and JSON content type (asserted on the wire); an accounts.google.com auth_url renders a _blank/noopener noreferrer link while a hostile lookalike (accounts.google.com.evil.example) renders as inert text; Cancel/Log hidden when not owned (and shown + functional when owned, cancel POST captured with the token); the no-status-endpoint scenario shows zero ingestion UI and fires zero requests.
  • E2E against the real binaries in a scratch directory: started target/debug/web, fetched the real token from /api/status, POST /api/runs (mbox, 2000-message synthetic file) with curl → 202 {"run_id":1} → watched /api/status to done (2000 seen / 2000 new, bytes_done == bytes_total) → /api/summary reported 2000 messages across 17 senders. Forged cross-site-style requests against the same live server were all rejected (415/403 across six probe shapes). Scan-source E2E, honestly scoped since no real OAuth consent was run: with a fake client secret and GMAIL_STATS_AUTH_TIMEOUT=8, POST {"source":"gmail_api"} → 202, status showed awaiting_auth with an https://accounts.google.com/o/oauth2/auth?... auth_url and owns_active_run:true, then failed/auth_required after the timeout. The full consent happy path and the stale-resume-token fallback against the live Gmail API were not exercised.
  • Gates: cargo build, cargo test (90 tests), cargo clippy --all-targets -- -D warnings, cargo fmt --check, python3 demo/build_demo.py into a temp dir — all clean.

Notes

  • The repo-root stats.db/credentials.json/tokencache.json were never touched; all E2E ran against scratch paths.
  • GET /api/runs/{id}/log for a non-owned run returns 404 not_owned (409 felt wrong for a GET; the issue's 409 contract applies to cancel).
  • Import landed before the OAuth job within the phase, per the delivery plan in Design proposal: drive scans and imports from the web UI #26.

zbuc added 3 commits July 22, 2026 13:30
…esume token

The scanner now installs a custom InstalledFlow delegate that writes
state='awaiting_auth' plus the consent URL onto its ingest_runs row (via a
new AuthState writer-channel message, so single-writer discipline holds) and
returns the row to 'running' once tokens arrive. The delegate holds only a
weak sender: the authenticator outlives the scan loop, and a strong clone
would keep the writer channel open forever and deadlock the end-of-scan
drain. Token acquisition happens eagerly, bounded by GMAIL_STATS_AUTH_TIMEOUT
(default 600s); an unattended consent becomes failed/auth_required
(policy_enforced keeps its dedicated kind). Terminal UX is unchanged: the URL
is still printed unless --quiet, and no schema migration was needed (the
auth_url column shipped with migration 1).

scan --resume <run_id> (issue #26 Q4, best effort) starts listing from the
run's persisted page token. A token Gmail rejects falls back to page one
silently (logged in verbose mode); seen_mails dedupe keeps the counts correct
either way. --resume on a bare scan is therefore no longer an error.
The viewer application moves from src/bin/web.rs into a library module
(gmail_stats::webapp) so integration tests can drive the exact router the
binary serves; src/bin/web.rs is now a thin main. Phase B behavior is
unchanged.

New surface (issue #30, per the #26 design):

- POST /api/runs spawns the ingester as a supervised child: fixed argv (never
  a shell), stdin null, stdout null, stderr piped into a 200-line in-memory
  ring buffer, kill_on_drop(false) so runs survive viewer restarts. The
  binary is located as a sibling of current_exe() (GMAIL_STATS_INGEST_BIN
  override); spawned runs get --quiet and the viewer's --db path. Responses:
  202 {run_id} (the child mints the id by inserting its run row; the viewer
  waits for it, matched by child pid), 409 run_active (friendly flock
  pre-check, plus mapping the child's own authoritative flock refusal), 422
  bad_path/bad_resume/bad_source (absolute + regular + readable + mbox magic
  sniff), 500 spawn_failed with a cargo build hint.
- POST /api/runs/{id}/cancel: owned runs only. SIGTERM through the supervisor
  that owns the child handle (never a pid read from the database), SIGKILL
  after a 10s grace; a SIGKILLed child's open row is abandoned by the next
  ingester's janitor - the viewer still never writes. 409 not_owned
  otherwise.
- GET /api/runs/{id}/log: the owned child's stderr ring buffer, memory-only,
  gone on viewer restart.
- CSRF stack on every mutating route, all three layers required: strict
  Content-Type application/json (415), X-Gmail-Stats-Csrf equal to the
  per-process token from /api/status (403, constant-time compare), and
  Origin / Sec-Fetch-Site validation when present (absent passes - curl is
  the same local user). No CORS headers anywhere. Host guard and the existing
  security headers stay on every route.
- /api/status now reports owns_active_run truthfully and exposes auth_url on
  run rows (the client refuses to hyperlink anything that is not
  https://accounts.google.com/...).

src/bin/fake_ingester.rs is a test-only stand-in driven by tests: it takes
the real flock and writes real run rows, with modes for completing, crashing
mid-run, honoring SIGTERM, ignoring SIGTERM, emitting stderr, and simulating
a lost flock race. tests/web_mutating.rs covers the full 415/403/409/422/202
matrix via tower::ServiceExt::oneshot (asserting no Access-Control-* headers)
and the supervision paths: normal completion, crash then janitor abandonment,
cancel honored, SIGTERM-ignored SIGKILL escalation, viewer restart losing
ownership while the run continues, log ring capping, and both flock-conflict
shapes.
…rize link

Onboarding cards go live against the real viewer: Start scan, and an
absolute-path text input plus Import button (422 reasons render inline). The
copyable CLI commands remain, and every launch control sits in a .live-only
block that starts hidden and is only revealed once /api/status responds - the
static Pages demo (404 status) never shows ingestion UI or fires a mutating
request.

Every POST sends Content-Type: application/json and the X-Gmail-Stats-Csrf
token from the latest status payload. Cancel and Show log appear only when
owns_active_run is true; non-owned runs keep the cancel-from-terminal hint.
The log renders the stderr ring buffer via textContent into a pre block.

awaiting_auth runs show an Authorize in Google link only when auth_url parses
with origin https://accounts.google.com (target=_blank, rel=noopener
noreferrer); anything else renders as inert text. History entries gain Retry
(failed runs), Resume import (interrupted imports, fingerprint-validated by
the ingester), and Resume - may restart from the beginning (failed scans,
best-effort per #26 Q4); policy_enforced failures deep-link to the Takeout
card, which stays pinned open until a run starts. All dynamic strings are
rendered with textContent only.

tests/frontend/ is a headless-Chrome harness (python3
tests/frontend/harness.py): a stub server serves the real web/ assets and
records every request, so both sides are asserted - the DOM (launch controls
hidden without a status endpoint, Cancel/Log visibility by ownership, link vs
inert text for good vs hostile auth_url) and the wire (launch and cancel
POSTs carry the CSRF token and JSON content type; the demo scenario fires no
requests at all). demo/build_demo.py anchors are untouched and the build
still passes.
@zbuc
zbuc merged commit 145ae42 into main Jul 22, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Phase C: launch, cancel, and OAuth consent from the web UI

1 participant