test(smoke): UAT smoke suite for CI + TEST#426
Merged
Conversation
Black-box GraphQL smoke tests automating the API-observable rows of the integrated UAT matrix (24 of 54 fully, 7 partially — see test/flash/smoke/README.md for the row-by-row map). No app imports; the same suite targets the local quickstart stack, CI, or TEST after a deploy via SMOKE_ENDPOINT/SMOKE_PHONE_A/B env vars. - specs ordered by UAT phase: environment, auth/account, wallets/home, receive, external funding via lnd-outside (docker-gated), two-account intraledger payments, validation errors, feature-flag/bridge surface - flags-off assertions match the v0.6.0 launch baseline; set SMOKE_EXPECT_FLAGS_OFF=false for flags-on environments - yarn test:smoke, make smoke / smoke-env-up / smoke-all - .github/workflows/smoke-test.yml boots quickstart (local image build) and runs the suite on pushes/PRs to main Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Validated the suite against a live quickstart stack and split it into two tiers based on what the IBEX-mock environment can actually resolve: - CI tier (SMOKE_BACKEND_FULL=false, default): API health, login/session, account+wallet identity, tx-history shape, username-set, recipient non-resolution, payment-validation rejections, globals.topupEnabled, and bridge/cashout resolvers responding structurally (no 5xx). 14 tests, green against quickstart mock. - Full tier (SMOKE_BACKEND_FULL=true): balances, invoice creation, onchain address, default-wallet mutation, device-token, username resolution, and (docker + payments) funding + two-account sends. For TEST / real IBEX. Key adjustments from the live run: - getMe no longer selects balance (IBEX-mock has no balance path and it poisoned every identity assertion); getBalance is a separate best-effort fetch used only by the full tier. - bridge/cashout mutation specs assert a structured response (no resolver crash) rather than exact flag-off semantics — the reliable flag signal is globals.topupEnabled, which is asserted strictly. - walletId-scoped mutations moved to the full tier: the mock's synthetic wallet ids don't satisfy the WalletId scalar. CI workflow runs the CI tier only. README documents both tiers + the row-by-row UAT coverage map. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Running against a live backend surfaced that globals.topupEnabled (flash #421) and the Bridge mutations (flash #413) aren't in every deployed schema — an older backend fails the query at GraphQL validation, which hard-failed BRIDGE-01. A smoke suite pointed at "whatever's deployed" must tolerate schema-version drift. Add isUnknownFieldError(errors, field) — detects a GRAPHQL_VALIDATION_FAILED error naming a field — and route BRIDGE-01/03/05 through an `evaluate` helper that logs + skips (passes as not-applicable) when the field is absent, while still asserting the real check when present. Verified the detector against the exact error payload from the live run, and that a genuine runtime error is NOT misclassified as a missing field (so real resolver crashes still fail). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
bobodread876
approved these changes
Jul 7, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Automates the API-observable slice of the integrated UAT matrix (the spreadsheet) as a black-box GraphQL smoke suite in
test/flash/smoke. No app imports — everything runs over HTTP against a running endpoint, so the same suite targets the local quickstart stack, CI, or TEST after a deploy.Verified against a live quickstart stack, not just written — the tiering below reflects what actually resolves in that environment.
Two tiers
The quickstart stack mocks IBEX, so balances/invoices/device-tokens/payments don't resolve and its synthetic wallet ids don't satisfy the
WalletIdscalar. The suite is split accordingly:SMOKE_BACKEND_FULL=false, default) — API health, login/session, account+wallet identity, tx-history shape, username-set, recipient non-resolution, payment-validation rejections,globals.topupEnabled, and bridge/cashout resolvers responding structurally (no 5xx). 14 tests, green against quickstart mock. This is the crash / API-contract / auth / flag safety net.SMOKE_BACKEND_FULL=true) — balances, invoice creation, onchain address, default-wallet mutation, device-token, username resolution, and (with docker + payments) external funding + two-account sends. Point it at TEST or any real-IBEX environment.Coverage: ~14 rows in CI, ~10 more in the full tier, the rest device-bound UI (camera, push delivery, settings, widgets, watch) that stays manual. Full row-by-row map in
test/flash/smoke/README.md.Running
CI
.github/workflows/smoke-test.ymlboots the quickstart stack (local image build) and runs the CI tier on pushes/PRs tomain.Test Plan
yarn tsc-check, ✅eslint test/flash/smoke/**/*.ts🤖 Generated with Claude Code