Skip to content

Document the no-Docker on-ramp ladder (audit → URL-vs-URL compare → twin servers) #75

Description

@justin808

Problem

Every quickstart leads with the hardest path — twin Docker servers (production Dockerfile, embedded services, DB seeded at build time; honestly "~2 hours"). But the tool already supports much cheaper first wins, none of them documented as an adoption path:

  1. shaka-perf audit --url <u> — zero config, minutes to a full report (perf + a11y + agent-readiness).
  2. URL-vs-URL comparecompare --controlURL <a> --experimentURL <b> works against any two servers (packages/shaka-perf/src/compare/cli/program.ts): two preview deployments (Vercel/Netlify/Heroku review apps), staging vs production, or two local checkouts on two ports. Paired-simultaneous sampling still cancels shared client-side noise; the caveat (server-side isolation is now the user's job) belongs in the doc.
  3. Single-server validation — control URL == experiment URL to validate test structure (the discover-abtests skill already uses this mode).
  4. Twin Docker servers — full A/B rigor.

Teams on serverless/preview-deploy stacks (a huge share of frontend) may never need step 4, and today the docs give them nothing.

Direction

  • A "Choose your setup" page: the four rungs, time-to-value, what each can/can't conclude (statistical validity caveats per rung), and when to climb.
  • Lead the README/site quickstarts with rung 1–2; link rung 4 as "the full harness."
  • Add a preview-deploy example (e.g. GitHub Actions computing the two preview URLs and passing them via --controlURL/--experimentURL).

Refs

Complements #36 (adoption playbook = the guided version of this ladder).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions