Skip to content

test(release): cover live onboarding, viewer recovery, and integration drift regressions #628

Description

@oaslananka

Summary

Add non-destructive release-confidence coverage for the user-facing failures and drift classes found during the 2026-08-30 live extension audit.

Context

The repository already has strong unit, security, accessibility, webview, visual, integration, real-pair, canary, mutation, and package checks. The live audit still found defects that are difficult to catch with isolated tests: installer/runtime mismatch, first-load viewer recovery, profile catalog drift, and multi-surface onboarding state.

This issue should extend the existing strategy rather than introduce a parallel test framework.

Scope

Add focused coverage for:

  • kicad-mcp-pro installer selection across Python/runtime combinations;
  • post-install re-detection and stale-failure recovery;
  • initial KiCanvas/WebGL failure with automatic CLI SVG fallback;
  • Task Hub canonical actions and duplicate setup-state prevention;
  • BoardReadyOps disabled/configured/incompatible contract states;
  • MCP profile discovery/migration against a published server artifact;
  • BOM/Netlist and integration empty-state behavior.

Acceptance criteria

  • Every live-audit regression above has a deterministic automated test at the lowest appropriate layer.
  • At least one extension-host or equivalent end-to-end lane proves the complete install/detect or simulated-install state transition without modifying production user data.
  • Viewer fallback is tested on Linux with WebGL unavailable or deterministically mocked unavailable.
  • Real-pair tests consume published/packaged server artifacts for compatibility claims.
  • Tests do not depend on external paid services, personal credentials, or mutable local state.
  • No existing test is skipped, weakened, or removed to make the new lane pass.
  • CI runtime and bundle/performance budgets remain within the repository's existing policy.

Validation

  • Existing check:kicad-studio, security, accessibility, webview/visual, integration, and real-pair suites.
  • New focused regression tests run in the appropriate required or canary lane.
  • Repeatable VSIX/package validation remains green.

Metadata

Metadata

Assignees

Labels

area:testingAutomated test coverage, fixtures, and regression testspriority:P1High-priority work after foundationproduct:vscode-extensionKiCad Studio VS Code extension workspaceregressionPrevents known bugs from returningrisk:mediumMinor, build-tool, action, or container dependency update requiring normal reviewtestingAutomated test coverage, fixtures, and regression teststype:taskOperational or maintenance task

Projects

No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions