|
| 1 | +# RFC 0002 — Post-Edit Verification MCP Tools |
| 2 | + |
| 3 | +- **Status:** Implemented (v1: delta-based verification) |
| 4 | +- **Packages:** `@domscribe/core`, `@domscribe/verify`, `@domscribe/relay`, `@domscribe/overlay` |
| 5 | +- **Depends on:** RFC 0001 (component-style capture) |
| 6 | + |
| 7 | +> This document reconstructs the RFC 0002 spec that earlier commits cite |
| 8 | +> (annotation schema v3, `verifyHistory`, `VerifyResult`) and records the |
| 9 | +> design as implemented. The original draft was removed in a repo cleanup; |
| 10 | +> the normative constraints below match the code. |
| 11 | +
|
| 12 | +## Problem |
| 13 | + |
| 14 | +Coding agents editing UI code cannot reliably tell whether their edit took |
| 15 | +effect. The universal failure modes: |
| 16 | + |
| 17 | +1. **Silent no-ops** — the agent edits the wrong file, a non-applying style |
| 18 | + path (specificity, conditional class), or HMR fails, and the agent |
| 19 | + declares success anyway. |
| 20 | +2. **Vision-blind regressions** — agents that verify by screenshot rely on |
| 21 | + a vision model to compare images, and vision models are demonstrably |
| 22 | + unreliable at exactly the deltas that matter for styling work (small |
| 23 | + offsets, near-identical shades, spacing changes). |
| 24 | + |
| 25 | +Every mainstream agent stack verifies by "screenshot + eyeball" as of |
| 26 | +mid-2026. None offers a deterministic, element-scoped comparison in the dev |
| 27 | +inner loop. |
| 28 | + |
| 29 | +## Design |
| 30 | + |
| 31 | +Two MCP tools on the relay close the loop deterministically: |
| 32 | + |
| 33 | +1. **`domscribe.verify.baseline`** — called *before* the edit. Captures a |
| 34 | + snapshot of the target element from the live page over the existing |
| 35 | + relay↔browser WS channel: the RFC 0001 computed-style allowlist |
| 36 | + (≤32 properties) plus the element's bounding rect. Returns an opaque |
| 37 | + `baselineId`. Baselines are in-memory and session-scoped (max 100, |
| 38 | + oldest evicted) — they describe the live page *now*, so persisting them |
| 39 | + would only produce stale comparisons. |
| 40 | + |
| 41 | +2. **`domscribe.verify.afterEdit`** — called *after* the edit and HMR. |
| 42 | + Re-captures the same element, computes deltas against the baseline, and |
| 43 | + returns a `VerifyResult` (core schema, annotation schema v3): |
| 44 | + - `verdict`: `match` | `partial` | `no_change` | `regression` |
| 45 | + - `componentStylesDelta`: per-property `{ property, before, after }` |
| 46 | + - `boundingRectDelta`: per-field `{ field, before, after }` (0.5 px |
| 47 | + epsilon for sub-pixel jitter) |
| 48 | + - `notes`: deterministic explanation |
| 49 | + When an `annotationId` is supplied, the result is appended to that |
| 50 | + annotation's `context.verifyHistory` (append-only). |
| 51 | + |
| 52 | +### Deltas first, pixels later |
| 53 | + |
| 54 | +The v1 verdict is computed from **style and geometry deltas only** — not |
| 55 | +pixel diffs. Grounds for this ordering: |
| 56 | + |
| 57 | +- Computed-style deltas are exact, cheap (no screenshot pipeline), immune |
| 58 | + to anti-aliasing noise, and directly actionable in code |
| 59 | + (`padding: 8px → 12px` tells the agent what to fix; a red pixel overlay |
| 60 | + does not). |
| 61 | +- The research consensus (Design2Code's low-level metrics, UI2Code^N's |
| 62 | + finding that CLIP-similarity rewards *degrade* refinement, the |
| 63 | + VLM-blindness benchmarks) is that element-level structured deltas are the |
| 64 | + reliable regression oracle, while holistic visual judgment belongs to |
| 65 | + the calling agent. |
| 66 | +- The pixel path stays open: `VerifyResultSchema.pixelDiffRatio` and |
| 67 | + `screenshotRef` are reserved, and `@domscribe/verify` already ships the |
| 68 | + pixelmatch comparator used by the falsifier harness. A future revision |
| 69 | + can add element-scoped screenshot capture without changing the contract. |
| 70 | + |
| 71 | +### Verdict semantics (deterministic, intent-agnostic) |
| 72 | + |
| 73 | +The tool measures; the agent judges intent. The caller may declare |
| 74 | +`expectedChanges: [{ property, value? }]` — the style changes the edit was |
| 75 | +meant to make (values compare as `getComputedStyle` strings). |
| 76 | + |
| 77 | +| Situation | Verdict | |
| 78 | +| --- | --- | |
| 79 | +| No style and no geometry delta | `no_change` | |
| 80 | +| No expectations declared, something changed | `match` + caveat note (change detection only) | |
| 81 | +| All expectations met, nothing unexpected | `match` | |
| 82 | +| All expectations met, extra properties changed | `partial` (unexpected properties listed) | |
| 83 | +| Some expectations met (incl. wrong-value changes) | `partial` | |
| 84 | +| No expectation met, other properties changed | `regression` | |
| 85 | + |
| 86 | +Geometry deltas never downgrade a verdict on their own — rect movement is |
| 87 | +usually a consequence of an intended style change (padding grows the box). |
| 88 | +They are always reported. |
| 89 | + |
| 90 | +### Degradation without `captureStyles` |
| 91 | + |
| 92 | +Style capture is gated on the runtime's `captureStyles` flag (default off |
| 93 | +in v0.x per RFC 0001). Without it, verification falls back to geometry-only |
| 94 | +change detection: `expectedChanges` cannot be evaluated and the result |
| 95 | +notes say so. The baseline response reports `hasComponentStyles` so agents |
| 96 | +can prompt the user to enable the flag when full deltas matter. |
| 97 | + |
| 98 | +## Wire surface |
| 99 | + |
| 100 | +- `POST /api/v1/verify/baseline` `{ entryId }` → |
| 101 | + `{ captured, baselineId?, browserConnected, hasComponentStyles?, hasBoundingRect?, error? }` |
| 102 | +- `POST /api/v1/verify/check` `{ baselineId, expectedChanges?, annotationId? }` → |
| 103 | + `{ verified, browserConnected, result?: VerifyResult, error? }` |
| 104 | + |
| 105 | +The WS `context:response` payload gains an optional |
| 106 | +`elementInfo.boundingRect` (serialized `DOMRect`), captured by the overlay's |
| 107 | +relay service. |
| 108 | + |
| 109 | +## Delta engine |
| 110 | + |
| 111 | +`@domscribe/verify` exports the pure functions the relay uses (no DOM, no |
| 112 | +I/O — unit-testable in isolation): |
| 113 | + |
| 114 | +- `diffStyleMaps(before, after): StylePropertyDelta[]` |
| 115 | +- `diffBoundingRects(before, after, epsilon?): BoundingRectDelta[]` |
| 116 | +- `resolveVerdict({ styleDeltas, rectDeltas, expectedChanges? })` |
| 117 | + |
| 118 | +## Falsifier gate |
| 119 | + |
| 120 | +RFC 0002's success criterion is a ≥60% retry-resolution rate on the styling |
| 121 | +falsifier corpus: after a failed first attempt, an agent given the |
| 122 | +`VerifyResult` deltas should resolve the task on retry at least 60% of the |
| 123 | +time. Measuring this requires the agent-driving falsifier mode |
| 124 | +(`--mode=agent`), which is tracked separately — see |
| 125 | +`docs/sprints/3071-rfc-0001-baseline.md` for the harness gap analysis. |
| 126 | + |
| 127 | +## Agent workflow |
| 128 | + |
| 129 | +``` |
| 130 | +1. domscribe.query.bySource / domscribe.resolve → entryId |
| 131 | +2. domscribe.verify.baseline { entryId } → baselineId |
| 132 | +3. edit source, wait for HMR |
| 133 | +4. domscribe.verify.afterEdit { baselineId, expectedChanges } |
| 134 | +5. verdict = no_change? → the edit did not land; fix and repeat 3–4 |
| 135 | + verdict = partial/regression? → consult deltas; fix and repeat 3–4 |
| 136 | + verdict = match? → done (agent confirms intent visually if it can) |
| 137 | +``` |
0 commit comments