fix(e2e): resolve the 29 'dead' lasuite selectors — 2 real defects, 27 a blind survey #553
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| name: Code Quality | |
| on: | |
| push: | |
| branches: [main, development, feature/**, bugfix/**, hotfix/**] | |
| pull_request: | |
| branches: [main, beta, development] | |
| workflow_dispatch: | |
| # Deduplicating a `push` run against the `pull_request` run for the SAME head | |
| # ref is the point of this block, and for a feature branch it is exactly right: | |
| # two runs of identical jobs, one of them wasted. | |
| # | |
| # It is wrong for `main` and `development`, because the push run there is NOT a | |
| # duplicate — it is the only carrier of the push-only jobs: "Coverage Baseline | |
| # Check" (`github.event_name == 'push'`), "SBOM" and "Features Extract". And | |
| # those two branches always have an open PR whose `head_ref` IS the branch | |
| # name: the standing "Release: merge development into beta" (#45 here). | |
| # `github.head_ref` on that PR run and `github.ref_name` on the push run both | |
| # render `development`, so both landed in the identical group | |
| # `quality-development`, and `cancel-in-progress` killed whichever started | |
| # first — always the push run, by a few seconds. | |
| # | |
| # Measured on this repo: 9 of the last 20 `development` push runs were | |
| # cancelled within ~96s of starting — e.g. 31045976452 (68s), 31037532276 | |
| # (6s), 31037517535 (51s), 31039915779 (55s). That duration is the | |
| # discriminator: the shared workflow's `timeout-minutes: 45` cancellation lands | |
| # at 45m16s–45m28s, so these are concurrency kills. | |
| # | |
| # On the surviving PR run "Coverage Baseline Check" reports `skipped`, which is | |
| # CORRECT for a pull_request event and renders exactly like a pass. So the gate | |
| # appears on both runs and executes on neither — a dead gate of the | |
| # permanently-pending shape. | |
| # | |
| # Suffixing only the default-branch push keeps feature-branch dedup untouched | |
| # (`quality-feature/x` for both events, exactly as before) and gives the two | |
| # default branches' push runs a lane of their own. | |
| # | |
| # Proven in openconnector#1158: its first-ever completed `development` push run | |
| # (31048998594) executed Coverage Baseline Check, SBOM and Features Extract. | |
| concurrency: | |
| group: quality-${{ github.head_ref || github.ref_name }}${{ (github.event_name == 'push' && (github.ref_name == 'main' || github.ref_name == 'development')) && '-push' || '' }} | |
| cancel-in-progress: true | |
| # Permission CEILING for the called quality pipeline. GitHub statically | |
| # validates the called workflow's declared job permissions against this | |
| # grant — even for jobs that are disabled — so it must cover the maximum | |
| # any nested job declares: journeydoc-capture (contents+actions write), | |
| # update-baseline / features-extract (contents write), and the Quality | |
| # Report PR comment (issues / pull-requests write). | |
| permissions: | |
| contents: write | |
| actions: write | |
| issues: write | |
| pull-requests: write | |
| jobs: | |
| quality: | |
| uses: ConductionNL/.github/.github/workflows/quality.yml@main | |
| with: | |
| app-name: nldesign | |
| php-version: "8.3" | |
| # Pinned, because the shared workflow's DEFAULT is '["stable31", "stable32"]' | |
| # and stable31 CANNOT WORK here. `additional-apps` below installs | |
| # openregister, which declares min-version="32" (its | |
| # lib/ContextChat/ContentProvider.php implements | |
| # OCP\ContextChat\IContentProvider, absent from core before NC32). On NC31 | |
| # `occ app:enable openregister` fails with | |
| # App "Open Register" cannot be installed because it is not compatible | |
| # with this version of the server. | |
| # and the run continues anyway, because that failure is only a ::warning::. | |
| # Every /apps/openregister/... call then returns Nextcloud's HTML 404 page. | |
| # | |
| # Order matters as much as membership: newman, playwright and | |
| # journeydoc-capture all check out `fromJSON(nextcloud-test-refs)[0]` as | |
| # their single server, so with the inherited default those three jobs ran | |
| # entirely on the version openregister cannot load. | |
| # | |
| # nldesign's own appinfo/info.xml floor stays at 28: this app declares no | |
| # <app> dependency and src/manifest.json declares none either — the NC32 | |
| # constraint is a property of the CI fixture, not of nldesign's code. | |
| nextcloud-test-refs: '["stable32"]' | |
| # These three were `false`, and that is worse than it sounds. The shared | |
| # workflow builds one matrix leg per tool unconditionally; a disabled leg | |
| # prints "<tool> is disabled — skipping." and `exit 0`. The job therefore | |
| # COMPLETES SUCCESSFULLY, and GitHub publishes a check context named | |
| # `quality / PHP Quality (psalm)` with a green tick — for a run in which | |
| # psalm was never invoked. On PR #206 that leg finished in ~13s with no | |
| # psalm output at all. Three checks named after three analysers were | |
| # reporting the analysers' ABSENCE as their SUCCESS. | |
| # | |
| # Turning them on rather than dressing up the skip, because all three | |
| # already pass on this repo today — the configs (phpcs.xml, phpstan.neon, | |
| # psalm.xml) are present, complete and were clearly once in use. Measured | |
| # on 9a0fbac under PHP 8.4: | |
| # | |
| # phpcs exit 0 0 errors, 5 warnings (warnings fixed in this PR) | |
| # phpstan exit 0 [OK] No errors (level 5) | |
| # psalm exit 0 No errors found | |
| # | |
| # Each was positive-controlled before being trusted, because a tool that | |
| # cannot START is indistinguishable from a tool that found nothing — the | |
| # first local attempt at phpcs exited 255 in vendor/composer/platform_check.php | |
| # (host PHP 8.2 vs the required 8.3) and printed no findings, which reads | |
| # exactly like a clean run. Injecting one deliberately broken class into | |
| # lib/Service/ turned all three red — phpcs 2, phpstan 1, psalm 2 — so | |
| # the green above is a verdict, not a silence. | |
| enable-psalm: true | |
| enable-phpstan: true | |
| enable-phpcs: true | |
| enable-phpmetrics: true | |
| enable-frontend: true | |
| enable-eslint: true | |
| enable-sbom: true | |
| enable-phpunit: true | |
| # nldesign ships ZERO .vue files — nc-vue is a build-only devDependency | |
| # used for icon generation. That is precisely why this app NEEDS e2e | |
| # rather than being exempt from it: nldesign's entire product surface is | |
| # runtime CSS — design tokens, token sets, dark mode, high-contrast, the | |
| # NL Design System variable mapping onto Nextcloud's own theming. None of | |
| # that is observable from a unit test, because there is no component to | |
| # mount; it only exists once a browser has resolved the cascade against a | |
| # running Nextcloud. tests/e2e/spec-coverage/ holds 31 specs that do | |
| # exactly that. | |
| # | |
| # `playwright-test-path` is left at its default (`tests/e2e`). The shared | |
| # workflow uses it twice: to assert tests exist, and to locate a config — | |
| # first `tests/e2e/playwright.config.ts`, then the repo root. nldesign has | |
| # no config under tests/e2e, so the root `playwright.config.ts` is used, | |
| # whose `chromium` project runs spec-coverage/ and testIgnores visual/ | |
| # (PNG baselines are host-font/GPU specific and cannot gate on a runner). | |
| # | |
| # workers stays at 1 deliberately. These specs mutate GLOBAL admin | |
| # theming state — active token set, dark mode, hidden slogan, custom CSS. | |
| # Two workers racing on one Nextcloud would cross-contaminate, so this | |
| # suite is not a candidate for the `fullyParallel` treatment used | |
| # elsewhere in the fleet. | |
| enable-playwright: true | |
| # A fresh Nextcloud has nldesign's `token_set` unset, which resolves to | |
| # the `nextcloud` set, whose design_system is `none` — and `none` means | |
| # CssInjectionService emits NO design-system layer and NO token layer at | |
| # all. That is a correct, deliberate stock-Nextcloud state, and it is also | |
| # not the state these specs describe: they assert the cascade | |
| # systems/… → tokens/<set>.css → custom-overrides.css, and a dark variant | |
| # next to the light one. On the first CI execution three of them failed | |
| # with a bare index of -1 for a stylesheet that was never supposed to be | |
| # there yet (run 30889958278). | |
| # | |
| # Seeding an actually-themed set is the precondition, not a workaround — | |
| # `rijkshuisstijl` has no explicit design_system, so it resolves to | |
| # `nldesign`, and it is one of the 41 sets that ships a generated dark | |
| # variant under css/tokens/dark/, which the dark-mode specs need. | |
| # dark_variants already defaults to '1'. | |
| playwright-seed-command: 'php occ config:app:set nldesign token_set --value rijkshuisstijl' | |
| # OpenRegister is required for nldesign to serve ANY route at all today. | |
| # | |
| # lib/Controller/HealthController.php declares | |
| # class HealthController extends | |
| # OCA\OpenRegister\AppHost\Controller\GenericHealthController | |
| # and its docblock asserts "OpenRegister is a soft dependency; the parent | |
| # class autoloads only on route dispatch, not at bootstrap." | |
| # | |
| # The first run of this job disproved that: with OpenRegister absent, | |
| # `/apps/nldesign/` returned **HTTP 500** — `Class | |
| # "OCA\OpenRegister\AppHost\Controller\GenericHealthController" not | |
| # found`. Nextcloud's router reflects over every registered controller | |
| # while MATCHING a route, so the missing parent takes down every nldesign | |
| # route, not just /api/health. The login page then never finished | |
| # rendering and global-setup timed out waiting for input[name="user"] — | |
| # a failure that names the login form and says nothing about the cause. | |
| # | |
| # Installing OpenRegister makes the suite able to run. It does NOT fix | |
| # the underlying coupling: nldesign is currently broken standalone, and | |
| # an `extends` cannot be made lazy by DI because it is resolved by the | |
| # autoloader, not the container. That is tracked separately. | |
| additional-apps: '[{"repo":"ConductionNL/openregister","app":"openregister","ref":"development"}]' | |
| # ── Integration tests ──────────────────────────────────────────────── | |
| # `enable-newman` defaults to FALSE and this caller never set it, so | |
| # tests/integration/nldesign.postman_collection.json had never executed in | |
| # CI. It sits at the default `newman-collection-path` (tests/integration), | |
| # which is both the directory the validator counts collections in and the | |
| # directory the run step globs `*.postman_collection.json` in — flat, not | |
| # recursively. | |
| enable-newman: true | |
| # ── Frontend Check legs ────────────────────────────────────────────── | |
| # `frontend-checks` defaults to `[]`, and an empty list means the shared | |
| # workflow emits NO "Frontend Check" job at all — so these two validators | |
| # ran nowhere while the run still looked complete. Both are self-contained | |
| # `node` scripts, which is what a leg has to be. | |
| # Measured on this tree before enabling: BOTH FAIL — `check:manifest` on | |
| # `top-level: menu (array) is required` + `pages (array) is required` | |
| # (nldesign's src/manifest.json does not carry the fleet manifest shape at | |
| # all), `test:l10n` on source strings missing from l10n/en.json. Real | |
| # pre-existing defects; the gate is what makes them visible. | |
| # `test` / `test:unit` are NOT listed: "Frontend Tests (unit)" runs them. | |
| frontend-checks: '["check:manifest", "test:l10n"]' | |
| # ── Coverage ratchet ───────────────────────────────────────────────── | |
| # `enable-coverage-guard` defaults to FALSE, which is why both | |
| # "Coverage Baseline Protection" and "Coverage Baseline Check" have only | |
| # ever reported `skipped`. It needs two inputs this repo did not have, | |
| # both added in this commit: `scripts/coverage-guard.php` (byte-identical | |
| # to openregister's) and `.coverage-baseline` = 77.48, this repo's own | |
| # measured coverage (3410 of 4401 statements) read from clover.xml in the | |
| # `coverage-report` artifact of run 30911331815. | |
| enable-coverage-guard: true | |
| # ── Hydra mechanical gates ─────────────────────────────────────────── | |
| # `enable-hydra-gates` defaults to FALSE, so this tier has never executed | |
| # here — the job reported `skipped`, which the Quality Report renders | |
| # identically to a pass. | |
| # `enable-axe` deliberately NOT set: a vanilla Nextcloud 34 already carries | |
| # serious/critical violations on core's OWN routes that DOM scoping does | |
| # not remove. Enabling axe is a separate decision. | |
| enable-hydra-gates: true | |
| # No `hydra-gates-ref` here on purpose. The shared workflow defaults it | |
| # to @main, and this workflow is itself consumed at @main, so the two | |
| # sides move together and a gate fix reaches this repo without a commit | |
| # in this repo. A pin is a silent expiry date: 22 repos sat on v1.0.1 and | |
| # 16 gates were dead fleet-wide while every one reported PASS (.github#159), | |
| # and a default flipped at @main later reached those old runners and made | |
| # them red on gates they had no subject matter for (.github#173) — that is | |
| # what blocked this repo's #220 and #217. | |
| # To hold this repo still for a specific reason, set the input explicitly | |
| # and say why — it is still honoured. To roll back for everyone, revert on | |
| # ConductionNL/.github main. | |
| # | |
| # THIRD CAUSE, and the one that is failing this repo RIGHT NOW | |
| # (.github#177): quality.yml@main began executing three gate helpers BY | |
| # NAME — check_spec_anchors.py, check_form_labels.py and | |
| # check_license_triangle.py — which exist in NO tag before v1.5.0. | |
| # Verified by DIRECTORY LISTING of each tag, not by per-file lookups: | |
| # those answered "present" uniformly across v1.0.0..v1.5.0, and the | |
| # uniformity across independent inputs was the tell that the instrument | |
| # was wrong. So the Hydra Gates job here fails at "Verify the pinned gates | |
| # package satisfies this workflow", before a single gate runs, with the | |
| # workflow's own words: "This is NOT a code-quality finding about your | |
| # repository." Removing the pin is the repair. | |
| # | |
| # Unpinning also picks up v1.5.1's push scoping (.github#179): on a push to | |
| # `development`, `origin/development` IS `HEAD`, so the diff was empty by | |
| # construction — <= v1.4.0 passed over it (permanently green) and v1.5.0 | |
| # refused with exit 99 (permanently red). The scope is now | |
| # `github.event.before...HEAD`, what the push actually changed. |