feat: expansion scope — cross-framework heal, MCP server, watch mode, heal history (v0.2.0) - #8
Conversation
… heal history (v0.2.0) Four expansion features that widen 9lives beyond a Playwright-only CI tool: * Cross-framework heal (QUA-1272): framework adapter layer under src/ninelives/frameworks/. Cypress specs run via 'npx cypress run --reporter json --quiet' in the user's own project; Selenium (Python) specs run via the user's own pytest. Auto-detected per spec (.cy.* / .py / package.json deps), overridable with --framework. Cypress locator misses (wrapped in AssertionError) are classified as selector drift so the behavior-vs-drift guard doesn't eat them; Selenium's NoSuchElementException JSON payload feeds selector extraction. Tier 1 never injects Playwright waits into non-Playwright code; Tier 2 prompts are framework-labelled. * MCP server (QUA-1271): '9l mcp' — zero-dependency stdio JSON-RPC implementing initialize / tools/list / tools/call with heal_test and run_test, so Claude Code / Cursor / Codex heal red tests in-loop. Heal-loop output is rerouted to stderr to keep the protocol stream clean; non-interactive mode saves a .healed copy instead of prompting. * 9l watch + pre-commit hooks: poll-based heal-on-save watcher (no OS watcher deps) and .pre-commit-hooks.yaml with 9lives-heal / 9lives-run. * Heal history + 9l report: every heal attempt appends to .9lives/history.jsonl (local only); '9l report' aggregates it into a brittle-selector report with anchor stats and pin-a-testid recommendations, terminal or --md. Also: sync __version__ (was stuck at 0.1.0 since first release — '9l --version' lied) and bump to 0.2.0. 48 tests green (20 new), ruff clean.
Sigilix OverviewEffort: 4/5 (large) Quality gates
Summary — latest pushIntroduces cross-framework healing (Cypress, Selenium/pytest) via an adapter layer, an MCP server for agent-driven healing, poll-based watch mode, and heal-history tracking with a report command. The heal loop is now framework-agnostic, routing through adapters that handle preflight, execution, and artifact collection, while Tier 1 guards framework-specific injection (e.g., Playwright waits) and classification overrides ensure Cypress/Selenium locator misses are treated as selector drift rather than assertion failures. Important files
Sequence diagramsequenceDiagram
participant CLI
participant Adapter as Framework Adapter
participant Runner as Test Runner
participant Tier1 as Tier 1 Healer
participant History as .9lives/history.jsonl
CLI->>Adapter: get_adapter(spec, framework)
Adapter->>Adapter: preflight(spec)
CLI->>Adapter: run(spec)
Adapter->>Runner: execute spec (pytest/npx/cypress)
Runner-->>Adapter: RunResult
CLI->>Adapter: collect(result, dest_root)
CLI->>Tier1: heal(failure)
Tier1-->>CLI: HealingResult
CLI->>History: record_heal(spec, status, ...)
Confidence: 3/5The multi-framework adapter pattern and MCP server substantially expand the attack surface with subprocess invocation and agent-callable paths that rely on input validation and path sanitization rather than comprehensive hardening tests.
Suggested labels:
|
There was a problem hiding this comment.
QualityMax Security Scan
Found 2 security issue(s).
Scanned by LLM SAST (gemini-3.1-flash-lite).
Fix all findings with your LLM agent
<your-llm-agent> "Fix all QualityMax security findings in this PR:\n- injection in src/ninelives/frameworks/selenium.py:45: User-controlled spec path is passed to subprocess.run via shell-like command list\n- path-traversal in src/ninelives/mcp_server.py:165: User-provided 'spec' path is expanded and used without validation, allowing access to arbitrary files on the system."|
Warning Sigilix couldn't generate a review for this PR. Reason: Re-run |
|
/sigilix review |
|
Re-running ensemble review on the current head. Standard sticky comment will update when complete. |
- MCP heal_test/run_test: validate the tool-supplied spec path — must be an existing file with a recognized test suffix, contained in the server's working directory (NINELIVES_MCP_UNRESTRICTED=1 to lift). The real risk isn't reading files: healing a spec ultimately EXECUTES it (pytest import / test run), so cwd containment is the meaningful boundary for a prompt-injected agent. - Selenium adapter: reject non-.py specs in preflight. The subprocess call was already injection-safe (argv list, no shell, resolved absolute path can't parse as a pytest flag) — this is defense-in-depth for the MCP path. 3 new tests (51 total).
Response to QualityMax Security Scan — both findings addressed in 0e537ceFinding 1 — Finding 2 — Note for the SAST loop: the check-run summary said "3 security finding(s)" but this comment lists 2 — if there's a third below the posting threshold, surfacing it would help. 3 new regression tests; suite at 51 green. |
✅ QualityMax Pipeline
Powered by QualityMax — AI-Powered Test Automation |
Wires the four expansion-scope features (QUA-1271, QUA-1272 + watch/report):
1. Cross-framework heal — Cypress + Selenium (QUA-1272)
New
src/ninelives/frameworks/adapter layer; the run→classify→heal→verify loop is now framework-agnostic.npx cypress run --reporter json --quietin the user's own project (never scaffolds a Cypress install). Locator misses arrive wrapped inAssertionError— newLOCATOR_OVERRIDE_PATTERNSclassify them as selector drift so the behavior-vs-drift guard doesn't refuse to heal them.NoSuchElementException's{"method":…,"selector":…}payload andBy.*calls feed selector extraction..cy.*/.py/ package.json deps),--frameworkto force. Tier 1 never injects PlaywrightwaitForinto non-Playwright code; Tier 2 prompts carry the framework label + right code fence.2.
9l mcp— the agent-native wedge (QUA-1271)Zero-dependency stdio MCP server (newline-delimited JSON-RPC):
heal_test+run_test, so Claude Code / Cursor / Codex heal a red test mid-session.claude mcp add 9lives -- 9l mcp. Heal-loop output rerouted to stderr; non-interactive mode saves<spec>.healedinstead of prompting; assertion failures come backneeds-humanwith an explicit don't-mask-the-bug note.3.
9l watch+ pre-commit hooksPoll-based heal-on-save (no OS watcher deps, self-writes don't retrigger) and
.pre-commit-hooks.yamlwith9lives-heal/9lives-run.4. Heal history +
9l reportEvery heal attempt appends to
.9lives/history.jsonlnext to the spec (local only, best-effort — can never break healing).9l report [--md]aggregates into a brittle-selector report: heal counts, winning anchors, pin-a-testid recommendations.Housekeeping
__version__was stuck at0.1.0since the first release (9l --versionlied on 0.1.1–0.1.3); synced and bumped to 0.2.0 with pyproject.Tests: 48 green (20 new — detection, Cypress/Selenium parsing + classification, tier-1 framework guard, history/report, MCP protocol roundtrip, watch scanner). Ruff clean. Not tagged/released yet — v0.2.0 release is a separate step after merge.