Aryaman — 3rd Task Checklist

[MuhammadKhalilzadeh]

Aryaman — 3rd Task Checklist

---

1. Frontend bundle diet: kill the multi-MB chunks

• Description: Production build is 36.7 MB with several oversized JS chunks: index-*.js 2.6 MB, customAxios-*.js 1.8 MB (an axios wrapper should never be that large — heavy deps are being pulled into it), vendor-react 1.0 MB, RiskManagement 0.96 MB, EvalsDashboard 0.73 MB. chunkSizeWarningLimit: 600 is set just to silence warnings.
• General guidance: Investigate with npm run analyze (bundle visualizer). For customAxios: check what imports it drags in (xlsx/exceljs are common culprits). Split the biggest pages internally (lazy-load tabs/modals in EvalsDashboard/NewExperimentModal.tsx at 108 KB and EvalsDashboard.tsx at 102 KB, ModelInventory/index.tsx at 89 KB). Restore chunkSizeWarningLimit toward the default once chunks are under control.
• Where to start: Clients/vite.config.ts (build config), Clients/src/infrastructure/api/customAxios.ts, then src/presentation/pages/EvalsDashboard/.
• What to expect: Measurable KB reduction in dist/; lazy-loading must not break the routes (lazyRoute retry logic in src/application/utils/lazyRoute.tsx). Verify e2e admin project still passes after splitting. No backend changes expected, but verify no page silently depends on a side-effect import.

2. Wire the ~36 orphaned Playwright specs into execution

• Description: Clients/playwright.config.ts defines projects that only match ~9 of 45 spec files (~344 tests). Roughly 36 spec files (~270 tests: assessments, model-inventory feature tests, file-manager, ai-gateway, policy-editor, public-intake-form, compliance-tracker, reporting, settings, incidents, datasets, shadow-ai, training, frameworks, intake-forms, onboarding…) are matched by no project and never run — in CI or locally.
• General guidance: Add project entries mapping each spec to the correct auth state (e2e/.auth/user.json super-admin or admin.json org admin — see e2e/helpers/auth.helper.ts). Group by auth needs to keep runtime sane (CI already runs single-worker, 60-min cap). Some may need triage: fix, quarantine with an issue link, or delete if obsolete. Coordinate with Debora — she owns the ongoing spec maintenance.
• Where to start: Clients/playwright.config.ts projects section; enumerate specs via ls Clients/e2e and cross-reference.
• What to expect: All surviving specs run green in npx playwright test. CI runtime will grow — propose sharding in e2e-tests.yml as a follow-up if it exceeds the timeout.

3. Execute Security Remediation Plan — SAST waves (Wave 2+)

• Description: docs/plans/SECURITY_REMEDIATION_PLAN.md (2026-08-12) shows Wave 1 dependency fixes completed, but 293 code-scanning findings remain (Trivy 197, Semgrep 77, CodeQL 19; 90 at error severity), plus 2 low-severity @ai-sdk/provider-utils Dependabot alerts via @mastra/core.
• General guidance: Triage the backlog in severity order: fix true positives ( Semgrep/ CodeQL findings in Servers/ and Clients/), document accepted-risk suppressions with justifications (pattern already used in .trivyignore). For @ai-sdk/provider-utils: upgrade or patch via override in the relevant package.json. Every fix needs a regression test where feasible.
• Where to start: GitHub Security tab (Trivy/Semgrep/CodeQL) cross-referenced with docs/plans/SECURITY_REMEDIATION_PLAN.md and docs/security/code-scanning-remediation-2026-08-13.md.
• What to expect: PRs grouped by scanner/rule family. The Semgrep PR gate is baseline-diff (--baseline-commit), so only new findings block — confirm the count trends down. Backend + frontend findings both exist.

4. Backfill tests for the Advisor AI agent framework

• Description: Servers/advisor/ (~60 files: agents, orchestrator, planner, tool bridge, approval rule engine) has 0% test coverage despite being mounted at /api/advisor and powering the frontend Advisor chat (Clients uses @assistant-ui + @ai-sdk/react for it).
• General guidance: Start with pure units: planner/parsers (input → plan), approval-rule engine decisions, tool-bridge parameter validation. Mock LLM calls (the ai SDK is already in the jest ESM allowlist in Servers/jest.config.js). Then an orchestrator integration test with a stubbed model. Frontend: add tests for the Advisor chat hooks/components in Clients/src if missing.
• Where to start: Servers/advisor/ — identify pure functions first (parsers, rule engine); look at advisor/**/__tests__ referenced in the test agent's report (orchestrator/planner/roadmap tests were mentioned — verify what exists and fill the rest).
• What to expect: Meaningful branch/function coverage for advisor/; a mocked-LLM harness other devs can reuse. Expect this to move the backend function-coverage number noticeably (currently ~33%).

5. Expand AI Gateway CI beyond a single test file

• Description: AIGateway/tests/ has ~19 test suites (API keys, endpoints, virtual keys, budget, guardrails, spend, risk detection, prompts, cache, encryption, MCP hooks ×5, MCP runs) but CI (.github/workflows/aigateway-checks.yml) runs only test_cache_service.py. test_encryption.py and role-enforcement/guardrail suites are self-contained and could run without live services.
• General guidance: Audit each suite for external dependencies (live LiteLLM providers, Redis, Postgres). Make the self-contained ones run by stubbing/mocking network (existing tests/conftest.py patterns). Suites genuinely needing a live stack stay manual but should be wired to the ephemeral-stack pattern used by e2e-tests.yml if cheap.
• Where to start: AIGateway/tests/ (read conftest.py and test_encryption.py), .github/workflows/aigateway-checks.yml.
• What to expect: The pytest job grows from 1 to N suites; keep pip-audit and caching intact. Document (in AIGateway/CLAUDE.md) which suites remain e2e-only and why.

6. Route-level auth tests for the 6 unauthenticated routers

• Description: 103 of 109 route files use authenticateJWT. The 6 exceptions are intentional but un-audited at the route level: internal.route.ts, telemetry.route.ts, version.route.ts, shadowAiIngestion.route.ts (token middleware), virtualKeyProxy.route.ts (API key), webhook.route.ts (signature-verified).
• General guidance: For each, write route tests asserting: (a) unauthenticated requests are rejected, (b) the intended alternative auth works, (c) cross-tenant access is denied. For webhook.route.ts: assert signature validation fails on bad signatures. These complement the tenant-isolation suites — check Servers/tests/integration/ for existing patterns first.
• Where to start: Servers/routes/__tests__/ (only 4 route test files exist today) and Servers/tests/integration/ patterns like the workflow approval-gate suite.
• What to expect: 6 new test files; no production code changes unless a test reveals a real hole (fix + report).

7. Own the ZAP findings remediation loop

• Description: ZAP runs nightly + on PRs: passive baseline against the frontend (non-blocking) and backend /api (blocking via .zap/rules.tsv), plus an authenticated API scan against Servers/swagger.yaml that fails on High/Critical. When findings appear, someone must triage and fix.
• General guidance: Set up a weekly triage of ZAP reports (CI artifacts). Fix root causes in the Express app (headers via helmet config, cookie flags, input reflection) or tighten .zap/rules.tsv only with documented justification. Since backend ZAP is a blocking gate, this directly protects merge velocity.
• Where to start: .zap/rules.tsv (current rules), .github/workflows/zap-baseline.yml + zap-api-scan.yml.
• What to expect: Zero unexplained ZAP failures on develop; a short triage runbook appended to docs/security/.

8. Drive the i18n native-review pipeline to completion

• Description: Clients/src/i18n/translations.ts (~27,600 lines) has large blocks of English-passthrough entries marked "TODO native review" (de/fr/es) at lines ~8146 and ~17345. The custom DOM translator (domTranslator.ts) + i18n:audit tooling exist, but passthroughs mean non-English users see raw English strings.
• General guidance: Extract the passthrough list via npm run i18n:audit:strict, batch translations through the team's native reviewers (or a reviewed LLM pre-translate + human sign-off workflow), and land them in translations.ts. Keep the i18n:audit CI gate green. Do not switch i18n libraries — that decision is explicitly out of scope.
• Where to start: Clients/scripts/i18n-audit.mjs output; the two TODO markers in translations.ts.
• What to expect: Audited locale files with zero unreviewed passthroughs in touched batches; audit tooling untouched.

9. Harden the EvalServer ↔ backend contract

• Description: The backend talks to EvalServer (port 8000) via Servers/routes/deepEvalRoutes.route.ts (/api/deepeval) with contract types in Servers/types/evalServer.types.ts (EVAL_SERVER_REQUEST_SHAPES) and one contract test (routes/__tests__/evalServerContract.test.ts). EvalServer has a full openapi.json (110KB) — the contract surface is large relative to its test coverage.
• General guidance: Generate/verify request-shape coverage for every EvalServer endpoint the backend calls; extend the contract test to cover each shape (valid → 2xx mapping, invalid → 400). Add error-path tests for gateway failures (AIGateway down, timeout). Consider a CI check that fails when EvalServer/openapi.json and evalServer.types.ts drift.
• Where to start: Servers/types/evalServer.types.ts, Servers/routes/deepEvalRoutes.route.ts lines ~316–420 (internal chat completions), EvalServer/openapi.json.
• What to expect: Contract tests for all request shapes; a drift-check script wired into backend-checks.yml. Frontend impact: EvalsDashboard calls these routes — note any contract changes in the PR description for reviewers.

10. Optimize CI: npm caching, workflow dedup, build cache

• Description: Backend CI jobs don't use npm caching (unlike frontend jobs); the env block is copy-pasted 4× in backend-checks.yml; docker-image-test.yml builds the frontend with no-cache: true every hour.
• General guidance: Add cache: npm with per-workspace lockfile paths to backend setup-node steps; extract the repeated env block into a YAML anchor or reusable composite action; remove/conditionalize the frontend no-cache (cache by lockfile hash, invalidate on dependency changes). Keep the tenant-isolation job's Postgres service config untouched.
• Where to start: .github/workflows/backend-checks.yml, .github/workflows/docker-image-test.yml.
• What to expect: Shorter, cheaper CI runs — measure before/after on a few runs. No product code changes; this is pure developer-velocity work.

11. Unify the duplicated risk-form variants

• Description: The frontend has two parallel risk-form implementations: AddNewRiskForm and the AddNewRiskIBMForm/AddNewRiskMITForm variants (IBM vs MIT risk-model flavors), plus @deprecated styling on AddNewRiskForm/style.ts. Relatedly, AddNewVendorRiskForm/index.tsx:308 hardcodes risk_level: "High Risk" with a "TODO: Make this dynamic" marker.
• General guidance: Design a single form core with model-specific schema/config (the risk-model differences should be data-driven, not forked components), migrate all three call sites, then delete the forks. Fix the vendor-risk hardcoding by deriving the level from the computed score — check what Servers/routes/vendorRisk.route.ts + utils/vendorRisk.utils.ts accept/return so the frontend matches the backend's risk-level logic.
• Where to start: Clients/src/presentation/components/AddNewRiskForm/ vs AddNewRiskIBMForm/AddNewRiskMITForm/; AddNewVendorRiskForm/index.tsx:308; backend Servers/controllers/ vendor-risk handlers.
• What to expect: One form component + config objects; existing unit tests for these forms (in presentation/components/__tests__) updated; backend unchanged unless the level-derivation logic is missing (then add it server-side with tests).

12. (On approval) Feature: detach regulations from use-cases — frontend lead

• Description: docs/plans/PLAN-detach-regulations-from-use-cases.md is fully planned and awaiting approval. You own the frontend workstream in coordination with Harsh (backend) if both are approved, or take the whole MVP if assigned solo.
• General guidance: Per the plan: make regulation selection optional in use-case creation/editing (pages/ProjectView/), render EU-AI-Act-only sections conditionally, and add the regulation-independent classification UI. Follow react-hook-form patterns already in the project; keep the change behind clear commits so partial rollout is possible.
• Where to start: docs/plans/PLAN-detach-regulations-from-use-cases.md (task decomposition inside), then Clients/src/presentation/pages/ProjectView/V1.0ProjectView.tsx and src/application/config/routes.tsx project routes.
• What to expect: Frontend PRs per plan phase with unit tests; API contract changes consume Harsh's backend work (or define the contract first per the API Contract Protocol in agents/00-TEAM_WORKFLOW.md if you lead solo).