Debora — 3rd Task Checklist

[MuhammadKhalilzadeh]

Debora — 3rd Task Checklist

---

1. Fix Playwright project matching so ALL specs execute (top priority)

• Description: Clients/playwright.config.ts projects match only ~9 of 45 spec files (~344 tests). Roughly 36 spec files (~270 tests: assessments, model-inventory, file-manager, ai-gateway, policy-editor, public-intake-form, compliance-tracker, reporting, settings, incidents, datasets, shadow-ai, training, frameworks, intake-forms, onboarding, notifications…) are matched by no project and never run — not in CI, not locally.
• General guidance: Map every spec file to a project with the correct auth state (e2e/.auth/user.json super-admin vs admin.json org admin — see e2e/helpers/auth.helper.ts and e2e/factories/api.factory.ts). For specs that fail when activated: fix, quarantine with a linked issue, or propose deletion. Aryaman owns the config change — pair with him; you own the spec triage results.
• Where to start: List Clients/e2e/*.spec.ts, cross-reference against the projects in playwright.config.ts.
• What to expect: A triage spreadsheet/issue list + all surviving specs green under npx playwright test. This is the single highest-value QA fix in the repo.

2. Investigate and close the frontend coverage gap vs. its own gate

• Description: Committed frontend coverage is ~32% statements / 22.6% branches / 27% functions / 33% lines (Clients/coverage/coverage-summary.json), but vite.config.ts enforces thresholds of 50/40/45/50 in the test:ci job. Either the CI gate is silently red, the committed report is stale, or thresholds are bypassed — find out which.
• General guidance: Run npm run test:ci locally exactly as CI does; compare with the coverage CI job output. Document the true current numbers per area (coverage report HTML in Clients/coverage/). Produce a gap report naming the worst-covered directories (presentation/pages big components, domain/, application/services). Feed the worst offenders to Inna's task 6 and the team generally.
• Where to start: Clients/vite.config.ts (test.thresholds), Clients/coverage/coverage-summary.json, a manual test:ci run.
• What to expect: A written gap report (attach to the team wiki/docs) + an answer on why the gate doesn't fail. Do not lower thresholds without team sign-off.

3. Add ESLint to CI workflows

• Description: No CI workflow runs ESLint — only Prettier. Lint regressions in non-staged files slip through entirely (pre-commit lint-staged only covers staged files).
• General guidance: Add a lint job (npm run lint) to frontend-checks.yml and backend-checks.yml. First run locally and fix or file issues for existing violations — decide with the team whether to start with --max-warnings tolerance. Add npm caching while you're in backend-checks.yml (Aryaman's task 10 overlaps — split the work).
• Where to start: Clients/package.json and Servers/package.json lint scripts; both workflow files.
• What to expect: A workflow PR that fails on lint errors; existing violations triaged into fix-now vs. suppress-with-issue.

4. Expand AIGateway CI test coverage

• Description: AIGateway/tests/ has ~19 suites but CI runs only test_cache_service.py. Self-contained suites (test_encryption.py, role-enforcement, guardrails) never run in CI.
• General guidance: Aryaman owns the deep rework; you own verification: run each suite locally, classify (self-contained vs. needs live services), and add the self-contained ones to .github/workflows/aigateway-checks.yml's pytest step. Report failures with repro steps.
• Where to start: AIGateway/tests/ (run pytest per file), the workflow's pytest step.
• What to expect: CI runs N>1 suites; a classification list added to AIGateway/CLAUDE.md.

5. Add Helm/Kubernetes validation to IaC scanning

• Description: kubernetes/ has hand-written manifests and a Helm chart (kubernetes/helm/verifywise), but the only validation is Trivy config scan (iac-scan.yml). No helm lint, no template-render check, no kubeval/kube-linter. YAML errors only surface at release time.
• General guidance: Add a CI step: helm lint kubernetes/helm/verifywise + helm template render (fails on template errors). Optionally add kube-linter on the raw manifests. Keep the existing Trivy gate untouched.
• Where to start: .github/workflows/iac-scan.yml, kubernetes/helm/verifywise/ (verify helm is installed in CI runners or add setup-helm action).
• What to expect: A workflow PR; if lint surfaces existing chart issues, fix the trivial ones and file the rest.

6. Extend axe accessibility scans beyond 4 pages

• Description: A11y is only enforced on 4 pages (dashboard, model-inventory, vendors, policies) via the A11y_SCAN_SPECS Playwright project and e2e/helpers/axe.ts. DEFERRED_AXE_RULES documents currently-skipped rules.
• General guidance: As task 1 activates more specs, add their axe scans to the a11y project (the helper already exists). Then evaluate the DEFERRED_AXE_RULES list one rule at a time: fix real violations or keep the documented deferral.
• Where to start: Clients/e2e/helpers/axe.ts, playwright.config.ts (A11Y_SCAN_SPECS project).
• What to expect: More pages under a11y enforcement; issues filed for violations needing product/design decisions.

7. Author e2e tests for uncovered critical journeys

• Description: Even after task 1, key business journeys lack e2e depth: reporting generation → download, compliance-tracker state changes, approval-workflow end-to-end, file upload → evidence linkage.
• General guidance: Write 1–2 new Playwright specs following existing patterns (auth state, factories, test-ids.ts). Reuse e2e/helpers/; add axe scan to each new spec (per task 6). Backend knowledge needed: check Servers/swagger.yaml for the API calls the UI makes so you can seed/assert via API (e2e/factories/api.factory.ts).
• Where to start: An existing well-written spec like e2e/critical-journey.spec.ts as the template.
• What to expect: New spec files running green in CI; each covers a full user journey, not just page loads.

8. Write the missing backend/e2e test-plan documentation

• Description: Only the frontend has a test plan (Clients/PLAN_TESTS.md). There is no backend test plan (unit vs. integration vs. smoke boundaries, how to run the Postgres-backed suites locally) and no e2e strategy doc (spec organization, auth states, seeding approach, quarantine policy).
• General guidance: Write two docs: Servers/docs/TESTING.md (framework, test:unit vs test:integration vs test:smoke, DB setup for integration tests, factories in tests/factories/) and Clients/e2e/README.md (projects, auth setup flow, factories, how to add a spec, quarantine process). Verify every command you document actually works.
• Where to start: Servers/jest.config.js, Servers/tests/integration/, Clients/playwright.config.ts, EvalServer/tests/README.md (style reference).
• What to expect: Docs-only PRs — perfect for establishing the team's QA source of truth.

9. Build a flake-management process for e2e

• Description: CI e2e runs single-worker with 1 retry and a 60-minute timeout; there is no quarantine process for flaky specs, and the only_last_failed manual workflow + last-failed cache exist but their effectiveness is unverified.
• General guidance: After task 1 activates all specs, run the full suite repeatedly (3–5 runs) and record flaky tests. Propose and implement: a @flaky tag convention with a tracking issue requirement, and a weekly flake report script (can start as a simple CI artifact review checklist). Verify the last-failed cache actually skips greens.
• Where to start: Clients/playwright.config.ts (retries/trace config), .github/workflows/e2e-tests.yml.
• What to expect: A documented flake process + a list of stabilized-or-quarantined specs.

10. Verify backend integration suites run in CI (and fix the regex)

• Description: Overlap with Inna's task 4 — you verify: after her PR, confirm login, projects, tasks, vendors, approval-workflows, governance-os.cross-tenant suites actually execute in the backend-checks.yml tenant-isolation job (check run logs), and that auditTenantIsolationCoverage.ts still passes.
• General guidance: Read the CI run's step output to confirm each suite's test count appears. If any suite is red in CI, reproduce locally with the workflow's Postgres 16 service setup and file/fix.
• Where to start: A CI run log of the tenant-isolation job; Servers/scripts/auditTenantIsolationCoverage.ts.
• What to expect: A verification comment/note on Inna's PR + zero red suites.

11. Validate observability configs in CI

• Description: observability/ (Prometheus prometheus.yml, Loki loki-config.yaml, OTel otel-collector-config.yaml, Grafana dashboards, nginx) has no validation at all — not even in iac-scan paths. A broken YAML only fails after deployment.
• General guidance: Add CI validation: promtool check config + amtool check-config (if alertmanager config exists), loki config validation or at minimum YAML schema lint, and JSON validation for Grafana dashboard files. Add observability/** to the iac-scan paths so Trivy covers it too.
• Where to start: .github/workflows/iac-scan.yml (paths + new step), observability/ contents.
• What to expect: A workflow PR catching config errors before deploy; fix trivial existing issues, file the rest.

12. Performance smoke test for heavy backend endpoints

• Description: Endpoints known to be heavy (N+1 patterns, unbounded lists, 30MB memory-stored uploads) have no performance regression checks. Any PR can silently double their cost.
• General guidance: Build a small smoke script (k6, autocannon, or plain fetch + timing in a Jest test marked @perf) that hits 5–10 critical endpoints (login, project list, file upload, report generation, vendor list) against a seeded local stack and asserts response-time budgets. Run it nightly via scheduled workflow (non-blocking at first, report-only).
• Where to start: docker-compose.yml for the local stack, Servers/database/seeders/ for data, existing npm run test:smoke (deadline-summary) as the pattern.
• What to expect: A report-only nightly job + a doc of baseline timings; blocking budgets come later once baselines are trusted.

13. Expand test-data factories and seed hygiene

• Description: E2E relies on e2e/factories/api.factory.ts + a global-setup seed; backend integration tests use Servers/tests/factories. Coverage of entity types is thin (many entities lack factories), and old root-level files (Servers/seed-automation-history.js, seed-demo-logs.sql, SQL_Commands.sql) linger unowned.
• General guidance: Add factories for the entities used by your new e2e specs (task 7): risks, controls, policies, vendors with risks, intake-form submissions. Follow existing factory patterns. Assess the root-level seed files: move useful ones into database/seeders/ or delete — propose before deleting.
• Where to start: e2e/factories/, Servers/tests/factories/, the root-level seed files.
• What to expect: Factory PRs + a cleanup proposal; new specs (task 7) should need zero ad-hoc seeding.