diff --git a/AGENTS.md b/AGENTS.md index ed3eaec..f865154 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -26,6 +26,7 @@ bash tests/skill-content-grep.sh bash tests/adk-path-canonicalization.sh bash tests/pipeline-evidence-doc-sync.sh bash tests/skill-cross-refs.sh +bash tests/brainstorm-visual-companion.sh # Hook contract tests bash tests/hook-contracts.sh diff --git a/decisions/0005-visual-companion-instructions.md b/decisions/0005-visual-companion-instructions.md new file mode 100644 index 0000000..5f39011 --- /dev/null +++ b/decisions/0005-visual-companion-instructions.md @@ -0,0 +1,33 @@ +# 0005. Add visual companion instructions + +**Status:** Accepted +**Date:** 2026-06-30 +**Decision-makers:** ADK maintainer via issue #78; implementing agent +**Related:** docs/plans/2026-06-30-brainstorm-visualization-design.md, https://github.com/GoCodeAlone/autonomous-dev-kit/issues/78 + +## Context + +Issue #78 asks ADK to add a brainstorm/design visualization companion similar to the updated `obra/superpowers` brainstorming skill. ADK skills must stay host-neutral and portable across supported agents. This repo does not currently include the upstream browser companion server or its scripts. + +## Decision + +We will add host-neutral visual-companion guidance to `skills/brainstorming/SKILL.md`, backed by a local `skills/brainstorming/visual-companion.md` guide, because this satisfies the requested workflow without introducing an unproven runtime dependency. + +**Alternatives considered and rejected:** + +- **Copy upstream browser server contract verbatim** — rejected because ADK does not ship those scripts and host process semantics differ. +- **Add diagrams unconditionally to every design** — rejected because many brainstorm questions are textual; mandatory diagrams add noise and token cost. + +## Consequences + +**Positive:** + +- Brainstorming can offer diagrams/mockups just-in-time when visual reasoning helps. +- Guidance remains portable and can use host-native rendering such as Mermaid before any future server exists. + +**Negative:** + +- This does not add a browser event loop or click-state capture. +- Future server support will need a separate design and runtime validation. + +**Reversibility:** revert the brainstorming skill edits and companion guide; no migrations or generated runtime state are involved. diff --git a/docs/FOLLOWUPS.md b/docs/FOLLOWUPS.md new file mode 100644 index 0000000..e351c6c --- /dev/null +++ b/docs/FOLLOWUPS.md @@ -0,0 +1,7 @@ +# Follow-ups + +## Brainstorm visual companion browser parity + +- **Source:** issue #78 / `docs/plans/2026-06-30-brainstorm-visualization-design.md` +- **Status:** Deferred +- **Follow-up:** Evaluate whether ADK should ship an upstream-like browser companion with tab launch, click/event capture, session security, cleanup, and host process validation. diff --git a/docs/cross-llm-coverage.md b/docs/cross-llm-coverage.md index fa297ee..eb709ea 100644 --- a/docs/cross-llm-coverage.md +++ b/docs/cross-llm-coverage.md @@ -18,6 +18,7 @@ Host-by-host capability matrix for the Autonomous Dev Kit skills system. | Host declaration for skill conditionals | `Host: claude-code` in CLAUDE.md | `Host: codex` in `~/.codex/AGENTS.md` | `Host: opencode` in `~/.config/opencode/AGENTS.md` | n/a | Auto-detected as `hermes-agent` | `Host: zed-agent` in `~/.config/zed/AGENTS.md` or `%APPDATA%\\Zed\\AGENTS.md` | | Skill discovery path (user scope) | `~/.claude/skills/` (personal skills); autodev installed to `~/.claude/plugins/marketplace/autodev/` via marketplace | `~/.agents/skills/` | `~/.config/opencode/skills/` | via plugin (no manual symlink) | `~/.hermes/skills/` (symlink or `hermes skills install`) | `~/.agents/skills//` (flat; no namespace folder) | | Model tier vocabulary | role names → `haiku`/`sonnet`/`opus` (see `agents/model-tiers.md`) | role names → `gpt-5.4-mini`/`gpt-5.4`/`gpt-5.5` | role names → host-pass-through | role names → host-pass-through | role names → host-pass-through | role names → host-pass-through | +| Visual companion output | ⚠️ markdown/Mermaid best-effort; browser deferred | ⚠️ markdown/Mermaid best-effort; browser deferred | ⚠️ markdown/Mermaid best-effort; browser deferred | ⚠️ markdown/Mermaid best-effort; browser deferred | ⚠️ markdown/Mermaid best-effort; browser deferred | ⚠️ markdown plus best-effort Mermaid rendering; browser deferred | ## Notes @@ -31,6 +32,8 @@ Host-by-host capability matrix for the Autonomous Dev Kit skills system. **Zed Agent:** Zed Skills apply only to Zed Agent, not ACP External Agents or Terminal Threads. Zed requires a flat skills layout, so ADK installs each `skills//` directory directly under `~/.agents/skills/` or `/.agents/skills/`; project-local skills require a trusted worktree. Zed does not expose the Claude/Codex lifecycle hook contract for skill packages. Current Zed task hooks are narrower (`create_worktree` is the documented event), so ADK hook-driven reminders use manual skill fallbacks and optional Zed Tool Permissions on Zed. +**Visual companion output:** Brainstorming may emit markdown, mockups, or Mermaid diagrams where useful, but rendered diagrams are best-effort and every visual needs a text fallback. Browser companion and click/event capture are deferred. + ## Related files - `tests/cross-llm-coverage.md` — per-skill host-conditional vs host-neutral audit diff --git a/docs/plans/2026-06-30-brainstorm-visualization-behavior-proof.md b/docs/plans/2026-06-30-brainstorm-visualization-behavior-proof.md new file mode 100644 index 0000000..7119f46 --- /dev/null +++ b/docs/plans/2026-06-30-brainstorm-visualization-behavior-proof.md @@ -0,0 +1,102 @@ +# Brainstorm Visualization Behavior Proof + +## RED baseline + +Baseline subject read pre-change `skills/brainstorming/SKILL.md` only. + +### Raw RED subject output + +```text +Actual current-skill behavior in scenario: +1. User asks: "what does trust mean for this dashboard?" + - Treat as brainstorming/design exploration. + - Stay text-only by default: clarify purpose, users, constraints, and success criteria. + - Use adaptive question batching, likely numbered options in Zed. +2. Later visual layout choice would be clearer shown than described + - Still present 2–3 text approaches with trade-offs and a recommendation. + - In Zed, present options as a numbered list and ask the user to pick/reply. +3. User declines visuals to save time + - Since the current skill has no visual-offer protocol, continue text-only. + - May treat "save time/no visuals" as a constraint, but this is inferred, not explicitly required. +4. User changes a decision after a diagram would have existed + - Revise the design text / assumptions / chosen approach as needed. + - No explicit stale-diagram handling exists, because visuals/diagrams are not currently governed. +``` + +### Missing explicit requirements + +| path | baseline result | +|---|---| +| conceptual text-only | MISSING as a distinct visual-companion rule; only general clarifying-question/design rules exist. | +| visual offer | MISSING; no diagrams/mockups/visualization offer rule. | +| declines | MISSING; no declined-offer/no-reoffer state rule. | +| accepts | MISSING; no accepted-visual/text-fallback rule. | +| stale visual | MISSING; no stale visual update/retire rule. | + +## GREEN subject run + +Subject used revised `skills/brainstorming/SKILL.md` and loaded `skills/brainstorming/visual-companion.md` only for the accepted-visual path. + +### Raw GREEN subject output + +````text +1. Conceptual text-only +Action: Do not load visual-companion.md; answer in text. +Message: Trust here means the dashboard helps users believe the data is accurate, fresh, explainable, and safe to act on. For this redesign, which trust signal matters most? 1. Data accuracy/provenance 2. Freshness/status 3. Explainability/drilldown 4. Consistency with existing reports. Reply with a number or adjust the wording. +Rule: SKILL.md says to stay text-only for requirements, conceptual choices, tradeoff tables, and scope decisions; visual-companion.md lists domain language like "what does trust mean?" as text-only. + +2. Visual offer +Action: Offer visuals just-in-time, as the only content in the message. +Message: This layout choice would be clearer shown than described. Want me to sketch 2–3 quick wireframe options before you choose? +Rule: First visual companion offer must be just-in-time, never upfront, its own message, and count as one question batch. + +3. User declines visuals +Action: Continue text-only; do not re-offer unless the user raises visuals. +Message: Got it — we'll stay text-only and optimize for speed. I won't re-offer visuals unless you ask. Text-only layout options: 1. KPI strip + exception queue 2. Narrative summary + drilldowns 3. Role-based sections by team. My recommendation is option 1 if fast trust-building is the goal. +Rule: If the user declines, continue text-only and do not re-offer unless the user raises visuals. + +4. User accepts visuals +Action: Lazy-load visual-companion.md, then compose a small visual with a text fallback. +Message: Great — I'll keep it lightweight and decision-focused. Here are three rough layout directions using synthetic content: + +```mermaid +flowchart TD + A[Option 1: KPI strip] --> B[Exceptions queue] + A --> C[Freshness + source badges] + D[Option 2: Narrative summary] --> E[Key changes] + D --> F[Drilldowns] + G[Option 3: Team sections] --> H[Sales] + G --> I[Ops] + G --> J[Finance] +``` + +Text fallback: Option 1 leads with metrics and trust badges, then highlights exceptions. Option 2 leads with a written summary, then lets users drill down. Option 3 groups content by team ownership. I'd choose option 1 for fastest confidence and scanability. +Rule: Load the guide only after acceptance or while composing a visual; visuals must include concise text equivalents. + +5. Visual becomes stale +Action: Retire or update the visual before proceeding; keep the text decision as source of truth. +Message: That changes the layout decision, so the previous wireframe is now stale. I'm retiring it rather than relying on it. Current text source of truth: we're prioritizing role-based sections over a universal KPI strip. If we need another visual later, I'll redraw it from that decision. +Rule: If a visual becomes stale or contradicts a later text decision, retire or update it before proceeding; current text design remains the source of truth. +```` + +## Reviewer score + +PASS overall. + +### Raw reviewer score + +```text +1. Conceptual text-only — PASS. Matches fixture: conceptual/domain-language question stays text-only and does not load/offer visuals. SKILL.md says to stay text-only for requirements/conceptual choices; visual-companion.md lists domain language like "what does trust mean?" as text-only. +2. Visual offer — PASS. Offer is just-in-time, standalone, and does not bundle another clarifying question. SKILL.md requires first visual offer to be just-in-time, its own message, count as one question batch, and not bundle another clarifying question. +3. User declines visuals — PASS. Continues text-only and explicitly avoids re-offering unless user asks. SKILL.md says if the user declines, continue text-only and do not re-offer unless the user raises visuals. +4. User accepts visuals — PASS. Lazy-load action is after acceptance; response includes an actual Mermaid visual artifact plus concise text fallback and keeps the decision text-readable. SKILL.md requires lazy-loading only after acceptance/while composing visuals and every visual to include a concise text fallback. Guide says the same. +5. Visual becomes stale — PASS. Correctly retires the stale visual and treats the current text decision as source of truth. SKILL.md says text remains source of truth and stale/contradictory visuals must be retired or updated before proceeding. Guide repeats the stale visual rule. +``` + +Reviewer note: accepted-visual transcript includes a Mermaid artifact and text fallback, proving artifact composition behavior. Rendering remains best-effort/config-only per design; this proof does not claim host-rendered Mermaid output. + +## Evidence + +- `bash tests/brainstorm-visual-companion.sh` → `Results: 0 failure(s)` +- RED baseline isolated subject: MISSING for visual offer, declined/no-reoffer, accepted visual with fallback, and stale visual. +- GREEN isolated subject + reviewer: PASS for conceptual text-only, visual offer, declines, accepts, and stale visual. diff --git a/docs/plans/2026-06-30-brainstorm-visualization-design-review.md b/docs/plans/2026-06-30-brainstorm-visualization-design-review.md new file mode 100644 index 0000000..bd19022 --- /dev/null +++ b/docs/plans/2026-06-30-brainstorm-visualization-design-review.md @@ -0,0 +1,50 @@ +### Adversarial Review Report + +**Phase:** design +**Artifact:** docs/plans/2026-06-30-brainstorm-visualization-design.md +**Status:** PASS + +**Findings (Critical):** +- None. + +**Findings (Important):** +- None. + +**Findings (Minor):** +- `D10` [Declared integration proof / Multi-component validation] [`docs/plans/2026-06-30-brainstorm-visualization-design.md`]: Host matrix originally risked overclaiming Zed Mermaid runtime rendering. Recommendation: mark rendered diagrams config-only/best-effort unless explicitly observed. _Resolution: fixed in design cycle 3._ +- `D11` [Missing failure modes / Repo-precedent conflicts] [`docs/plans/2026-06-30-brainstorm-visualization-design.md`]: Visual-companion offer asks the user a question; batch-budget interaction needed definition. Recommendation: count the offer as one question batch. _Resolution: fixed in design cycle 3._ +- `D12` [Missing failure modes / Multi-component validation] [`docs/plans/2026-06-30-brainstorm-visualization-design.md`]: Behavior fixture needed at least one negative/failure-mode pressure path. Recommendation: add stale/contradictory visual case. _Resolution: fixed in design cycle 3._ +- `D13` [User-intent drift / Rollback story] [`docs/plans/2026-06-30-brainstorm-visualization-design.md`]: Browser/event-capture parity is deferred; durable follow-up needed. Recommendation: add `docs/FOLLOWUPS.md` entry. _Resolution: fixed in design cycle 3._ + +**Prior Important resolution assessment:** +| finding | status | assessment | +|---|---|---| +| `D1` marker-only validation did not prove behavior | Resolved | Design now requires `expected-behavior.md` fixture plus subagent pressure proof as primary validation; shell test is only a regression guard. | +| `D2` browser-parity boundary missing | Resolved | Design includes parity boundary, host matrix, and deferred browser/event capture. | +| `D3` visual failure modes missing | Resolved | Design includes invalid/unsupported rendering, stale/contradictory visuals, accessibility fallback, decline behavior, secrets/PII, and lazy-load rules. | + +**Bug-class scan transcript:** +| Class | Result | Note | +|---|---|---| +| Project-guidance conflicts | Clean | No `docs/design-guidance.md`, `docs/PROJECTS.md`, or `docs/PORTFOLIO.md` exists; design uses repo canon from `AGENTS.md`, `README.md`, coverage docs, and skill files. | +| Assumptions under attack | Clean | Browser parity, Mermaid rendering, guide scope, and behavior-proof limitations are explicit assumptions with fallbacks. | +| Repo-precedent conflicts | Clean | Skill-edit TDD precedent is addressed with behavior fixture + subagent proof; process diagrams remain `dot`; coverage/docs artifacts are updated. | +| Artifact-class precedent | Clean | New test is documented in `AGENTS.md`; host capability matrix changes are planned in durable coverage docs. | +| YAGNI violations | Clean | Full browser runtime is deferred; guide is bounded and lazy-loaded. | +| Missing failure modes | Clean | Core visual control-flow and negative stale/contradictory visual path are covered; text remains source of truth. | +| Security/privacy architecture | Clean | No browser server, telemetry, network service, auth boundary, or persistent session state; guide requires synthetic/redacted examples. | +| Infrastructure impact | Clean | No cloud/network/runtime infrastructure; storage/test-suite impacts are explicit and reversible. | +| Multi-component validation | Clean | Skill behavior proof, shell regression guard, content guards, cross-ref guard, and coverage doc checks are planned. | +| Declared integration proof | Clean | Rendered diagrams are config-only/best-effort unless observed; browser companion is deferred. | +| Contributed UI rendering proof | Clean | No host UI route/widget/page is contributed. | +| Rollback story | Clean | Rollback lists concrete file classes to revert; no runtime state, migrations, or version pins. | +| Simpler alternative not considered | Clean | Copy-upstream, host-neutral, inline-only, and full-runtime alternatives are considered. | +| User-intent drift | Clean | Markdown/Mermaid companion satisfies #78 scope; browser/event capture parity is tracked as follow-up. | +| Existence/runtime-validity | Clean | Existing artifacts to edit are named; emitted artifacts and consumer tests are specified. | + +**Options the author may not have considered:** +1. **Full runtime now:** implement upstream-like browser/event capture in this PR. Rejected by design as too broad for host-neutral validation. +2. **Inline-only MVP:** avoid the guide file. Rejected because examples/failure/accessibility rules would bloat `SKILL.md`; mitigated by lazy-load. +3. **All diagrams text-only:** avoid Mermaid entirely. Rejected because issue explicitly asks for diagrams; mitigated by text fallback and best-effort rendering labels. + +**Verdict reasoning:** PASS. The revised design resolves all Critical/Important findings. Remaining concerns were minor and have been folded into the design: no overclaimed Mermaid runtime integration, clear question-batch semantics, negative behavior fixture coverage, and a durable follow-up for deferred browser parity. diff --git a/docs/plans/2026-06-30-brainstorm-visualization-design.md b/docs/plans/2026-06-30-brainstorm-visualization-design.md new file mode 100644 index 0000000..76a4ba3 --- /dev/null +++ b/docs/plans/2026-06-30-brainstorm-visualization-design.md @@ -0,0 +1,177 @@ +# Brainstorm Visualization Design + +**Issue:** #78 — Brainstorm visualization +**Date:** 2026-06-30 +**Status:** Approved by user instruction; revised after adversarial review cycles 1-3 +**ADR:** `decisions/0005-visual-companion-instructions.md` + +## Original Ask + +> At https://github.com/obra/superpowers, their skill has been updated to add a visualization companion for brainstorms/designs, with diagrams, etc. This is quite handy, we should add something similar. + +## Global Design Guidance + +`Guidance: none found as docs/design-guidance.md; constraints cited from repo canon.` + +| source | guidance | design response | +|---|---|---| +| `AGENTS.md` | Skills use host-gated content; forbidden host-specific tokens only in host blocks; recognized hosts include `zed-agent`; test scripts are documented pre-commit checks. | Add host-neutral visual guidance; update documented checks for the new focused test. | +| `README.md` Cross-LLM/plugin positioning | ADK bundles portable skills/hooks/agents/commands for multiple hosts. | Prefer host-neutral Markdown plus optional Mermaid where rendered; never require an upstream-only browser server. | +| `docs/cross-llm-coverage.md` / `tests/cross-llm-coverage.md` | Capability and skill coverage matrices record host-portability intent. | Add a visual-companion capability row and brainstorming note so the host matrix is durable, not design-local. | +| `skills/brainstorming/SKILL.md` | Brainstorming has adaptive batching, assumptions, self-challenge, adversarial review, and design docs. | Insert visual companion as a just-in-time optional aid without weakening gates. | +| `skills/writing-skills/SKILL.md` | Skill edits require failing tests first and behavior proof, not marker-only tests. | Add a focused shell regression guard plus a behavior pressure-test fixture and subagent pressure run. | + +## Parity Boundary + +| upstream capability | ADK issue #78 scope | rationale | +|---|---|---| +| Browser tab companion | Deferred | ADK does not ship the server/scripts; adding them needs runtime/process/security validation across hosts. | +| Click/event capture | Deferred | Requires browser session state and privacy policy; out of scope for this markdown-only PR. | +| Diagrams/mockups during brainstorm/design | In scope | Add just-in-time visual guidance using host-native chat/markdown artifacts. | +| Text fallback | Required | Host rendering differs; text remains source of truth. | + +## Host Visual Capability Matrix + +| host | Markdown text | Mermaid/rendered diagrams | Browser companion | +|---|---|---|---| +| `zed-agent` | runtime-integrated: markdown chat output | config-only/best-effort unless rendered output is explicitly observed; must include text fallback | deferred | +| `claude-code` | config-only: markdown output | config-only/best-effort: render support varies by surface | deferred | +| `codex` | config-only: markdown output | config-only/best-effort | deferred | +| `opencode` | config-only: markdown output | config-only/best-effort | deferred | +| `cursor` | config-only: markdown output | config-only/best-effort | deferred | +| `hermes-agent` | config-only: markdown output | config-only/best-effort | deferred | + +`config-only` rows mean the skill can emit markdown/diagram code, but this PR does not prove host rendering beyond plain text. ∴ every visual must have a concise textual equivalent and decisions must be recorded in text. + +## Approaches Considered + +| option | summary | trade-off | +|---|---|---| +| A. Copy upstream browser companion verbatim | Add upstream `Visual Companion` text and server instructions. | Fast but references scripts/tools ADK does not ship; host-portability and security gaps. | +| B. Host-neutral progressive companion | Add just-in-time offer rules, diagram/mockup decision test, failure rules, host matrix, behavior proof, and bounded guide. | Solves issue now; browser click capture remains deferred. Recommended. | +| C. Inline-only MVP | Put all guidance in `SKILL.md`, no guide file. | Smallest patch but clutters an already-long skill and lacks room for examples/fallbacks. | +| D. Full browser companion runtime | Add scripts, frame template, event state, and docs. | Larger runtime feature; requires multi-host launch validation beyond issue scope. | + +**Decision:** B. See `decisions/0005-visual-companion-instructions.md`. + +## Design + +- Modify `skills/brainstorming/SKILL.md`: + - Add checklist item after project guidance: offer visual companion just-in-time, not upfront. + - Keep process-flow diagram in `dot` to match existing skill precedent. + - Add process bullets: offer visuals only when a question is genuinely clearer shown than described. + - Add `## Visual Companion` section: per-question decision rules, text-source-of-truth, stale/invalid visual fallback, lazy-load guide rule, and guide reference. +- Add bounded `skills/brainstorming/visual-companion.md`: + - Role: reference examples and failure/accessibility rules only; load only after visual companion acceptance or while composing a visual artifact. + - Covers diagrams/mockups, Mermaid examples, visual/text decision test, no-secrets rule, stale visual retirement, and plain-text fallback. +- Add focused regression test `tests/brainstorm-visual-companion.sh`: + - RED before implementation: required skill markers/guide/fixture/coverage rows absent. + - GREEN after implementation: asserts just-in-time offer, not-upfront rule, per-question decision, text fallback/source of truth, lazy-load rule, guide existence, guide mentions Mermaid plus accessibility/fallback, behavior fixture exists, and coverage/docs checks include visual companion. +- Add behavior fixture `skills/brainstorming/test-fixtures/visual-companion/expected-behavior.md`: + - Records pressure scenario and expected compliant behavior for five paths: conceptual text-only, visual offer in its own message, declined offer/no re-offer, accepted visual with text fallback, and stale/contradictory visual after a changed decision. + - Includes RED baseline notes against pre-change brainstorming: no visual companion rule exists, so the expected just-in-time offer and fallback paths are absent. +- Run behavior proof with a subagent after implementation: + - Prompt a reviewer with the fixture and current `skills/brainstorming/SKILL.md`. + - Expected: reviewer reports PASS for all five paths and cites the exact skill/guide lines that force the behavior. + - This is the primary behavior proof; shell test is the regression guard. +- Update durable docs: + - `AGENTS.md`: add `bash tests/brainstorm-visual-companion.sh` to skill content checks. + - `docs/cross-llm-coverage.md`: add visual companion capability row. + - `tests/cross-llm-coverage.md`: update brainstorming notes with visual companion host-neutral/best-effort behavior. + - `docs/FOLLOWUPS.md`: record deferred browser/event-capture parity as a follow-up so #78 scope reduction is durable. + +## Behavior Fixture Expected Paths + +| path | compliant behavior | +|---|---| +| Conceptual text-only question | Do not offer visual companion; ask/answer in text because the answer is conceptual. | +| Visual question arises | Send only the just-in-time offer message; no bundled clarifying question; the offer counts as one question batch. | +| User declines | Continue text-only and do not offer again unless user raises visuals. | +| User accepts | Use the guide only when composing visuals; include text summary/fallback; record final decision in text. | +| Decision changes after visual | Retire or update stale/contradictory visual; preserve text as source of truth. | + +## Failure Rules + +| failure | required behavior | +|---|---| +| Mermaid invalid or unsupported | Continue with plain text; never claim rendered proof unless observed. | +| Visual contradicts text | Text design/plan is source of truth; fix or withdraw visual before proceeding. | +| Visual becomes stale after decision changes | Retire/update the visual; do not leave obsolete artifact as active guidance. | +| Accessibility or remote-host limitation | Include concise text equivalent for every visual and accept text-only feedback. | +| User declines visual companion | Continue text-only; do not offer again unless user raises visuals. | +| Sensitive data in visual | Redact or use synthetic examples; no secrets/PII in diagrams/mockups. | +| Guide loaded eagerly | Violation; companion guide loads only after acceptance or while composing a visual artifact. | + +## Security Review + +| area | assessment | +|---|---| +| Auth/authz | No auth boundary changes. | +| Secrets/PII | Guide requires redacted/synthetic visual examples; no secrets/PII. | +| Dependency/trust | No dependency, network service, telemetry, or browser session state. | +| Abuse case | Avoid opening browsers/servers by default; visuals are optional and text-backed. | +| Least privilege | No filesystem writes except normal design artifacts; no runtime process. | + +## Infrastructure Impact + +| resource | impact | +|---|---| +| Cloud/network | None. | +| Local runtime | None; no server added. | +| Storage | Adds one bounded skill guide, one fixture, one focused test, plus design/ADR/plan artifacts. | +| Test suite | `AGENTS.md` documented checks gain one shell test. | +| Cost/scale | Slight token cost only when visual companion is used; lazy-load rule avoids every-session overhead. | +| Release/rollback | Revert markdown/test changes; no migrations. | + +## Multi-Component Validation + +| boundary | proof | +|---|---| +| Skill behavior | Subagent pressure proof against `expected-behavior.md` → PASS for five paths. | +| Skill behavior markers | `bash tests/brainstorm-visual-companion.sh` → asserts just-in-time/not-upfront/per-question/text fallback/lazy-load/guide/fixture/coverage contract. | +| Skill text ↔ content rules | `bash tests/skill-content-grep.sh` → host-neutral token guard passes. | +| Skill refs ↔ repo files | `bash tests/skill-cross-refs.sh` → references resolve. | +| Coverage docs | `bash tests/brainstorm-visual-companion.sh` checks visual rows/notes exist. | +| Full repo contract | Run AGENTS.md test list before PR. | +| Runtime rendering | Mermaid/rendered diagrams are config-only/best-effort unless a host-rendered output is explicitly observed; text fallback is required everywhere. | + +## Assumptions + +| id | assumption | challenge | fallback | +|---|---|---|---| +| A1 | Issue #78 accepts host-neutral visual guidance as "similar" to upstream. | User may expect full browser parity. | Parity boundary marks browser/event capture deferred; future server runtime requires separate issue/design. | +| A2 | Markdown/Mermaid diagrams are useful even where rendered as code. | Some hosts show raw code only. | Text source-of-truth + concise text equivalent required. | +| A3 | A bounded guide file is justified. | Existing skill guidance prefers inline when short. | SKILL.md remains authoritative; guide is lazy-loaded and limited to examples/failure/accessibility details. | +| A4 | Subagent pressure proof is acceptable for behavior validation. | It is not CI-enforced. | Shell regression guard preserves contract markers; PR body includes behavior proof transcript. | + +## Self-Challenge + +| doubt | response | +|---|---| +| Laziest solution: paste upstream section only. | Rejected: it references scripts ADK does not ship and weakens host neutrality. | +| Fragile assumption: no full browser runtime required. | Mitigated by parity boundary, ADR, and deferred browser row. | +| YAGNI risk: guide becomes too broad. | Bound + lazy-load guide; no runtime implementation, telemetry, session keys, or click-event schema. | +| Failure mode: agent overuses diagrams. | Just-in-time/per-question rules say never offer upfront and use visuals only when clearer than text. | +| Repo pattern conflict: marker-only testing. | Add fixture + subagent pressure proof as primary; shell test only guards regressions. | + +## Review Cycle Resolutions + +| finding | resolution | +|---|---| +| D1 | Added behavior fixture + subagent pressure proof as primary validation. | +| D2 | Added parity boundary and host visual capability matrix; browser companion deferred. | +| D3 | Added explicit failure rules and text-source-of-truth behavior. | +| D4 | Accepted with mitigation: guide bounded/lazy-loaded; SKILL.md authoritative. | +| D5 | Resolved: process flow remains `dot`; Mermaid only for optional generated visuals. | +| D6 | Resolved: use new focused test, not `pipeline-evidence-doc-sync.sh`. | +| D7 | Add durable coverage doc updates. | +| D8 | Add focused test to documented `AGENTS.md` checks. | +| D9 | Add lazy-load rule to SKILL.md/guide/test. | +| D10 | Downgrade Mermaid rendering to config-only/best-effort unless explicitly observed. | +| D11 | Visual-companion offer counts as a question batch. | +| D12 | Add negative stale/contradictory-visual fixture path. | +| D13 | Add `docs/FOLLOWUPS.md` entry for deferred browser/event-capture parity. | + +## Rollback + +Revert commits modifying `skills/brainstorming/SKILL.md`, `skills/brainstorming/visual-companion.md`, `skills/brainstorming/test-fixtures/visual-companion/expected-behavior.md`, `tests/brainstorm-visual-companion.sh`, `docs/cross-llm-coverage.md`, `tests/cross-llm-coverage.md`, `docs/FOLLOWUPS.md`, `AGENTS.md`, and planning/ADR artifacts; rerun AGENTS.md test list. No runtime state, migrations, or version pins. diff --git a/docs/plans/2026-06-30-brainstorm-visualization-plan-review.md b/docs/plans/2026-06-30-brainstorm-visualization-plan-review.md new file mode 100644 index 0000000..ee3979f --- /dev/null +++ b/docs/plans/2026-06-30-brainstorm-visualization-plan-review.md @@ -0,0 +1,65 @@ +### Adversarial Review Report + +**Phase:** plan +**Artifact:** docs/plans/2026-06-30-brainstorm-visualization.md +**Status:** PASS + +**Findings (Critical):** +- None. + +**Findings (Important):** +- None. + +**Findings (Minor):** +- `P12` [Verification-class mismatch / Repo-precedent conflicts]: RED baseline should capture natural behavior before explicit rule audit. _Resolution: plan now instructs subject to first answer actual messages/actions, then mark missing requirements._ +- `P13` [Missing integration proof]: Reviewer should verify subject citations against current skill/guide files. _Resolution: reviewer prompt now includes `SKILL.md` and `visual-companion.md` and requires real-rule verification._ +- `P14` [Project-guidance conflicts / Verification-class mismatch]: Task 1 commit should not happen before core content/cross-ref/path checks. _Resolution: Task 1 now runs focused guard, content grep, cross-ref, and no-machine-path checks before commit._ +- `P15` [Missing failure modes / Artifact capture]: RED baseline proof should be captured immediately. _Resolution: Task 1 now creates the proof artifact immediately after RED and Task 2 appends GREEN evidence._ + +**Prior Important resolution assessment:** +| finding | status | assessment | +|---|---|---| +| `P1` RED baseline pressure evidence missing | Resolved | Plan requires isolated baseline subject run and immediate proof capture. | +| `P2` GREEN subject-agent behavior proof missing | Resolved | Plan requires isolated GREEN subject run plus reviewer scoring all five fixture paths against real skill/guide rules. | +| `P3` serial TDD phases split across misleading tasks | Resolved | Plan uses one integrated TDD implementation task, then proof, then validation. | +| `P4` declared integration matrix missing | Resolved | Plan-level matrix marks markdown, Mermaid, browser tab, and click/event capture as config-only/best-effort or deferred with proof/rationale. | +| `P8` host-neutral pressure-proof protocol | Resolved | Plan defines native subagent first, fresh isolated host thread/session fallback, and stop-if-no-isolated-subject behavior. | + +**Bug-class scan transcript:** +| Class | Result | Note | +|---|---|---| +| Project-guidance conflicts | Clean | Skill-edit TDD/behavior-proof guidance is represented in tasks. | +| Assumptions under attack | Clean | Browser parity, Mermaid rendering, isolated-subject availability, and behavior-proof limits have explicit fallbacks. | +| Repo-precedent conflicts | Clean | Skill testing uses RED/GREEN pressure proof; guide is bounded/lazy-loaded; docs/tests are validated. | +| Artifact-class precedent | Clean | Fixture, focused shell guard, proof doc, and coverage docs follow repo shapes. | +| YAGNI violations | Clean | Browser runtime/event capture/telemetry/session state deferred; guide stays bounded. | +| Missing failure modes | Clean | Invalid Mermaid, stale/contradictory visuals, declines, text fallback, accessibility, secrets/PII, and lazy-load violations are covered. | +| Security/privacy architecture | Clean | No auth, server, telemetry, network, session, or secret flow added. | +| Infrastructure impact | Clean | No cloud/network/migration/version/runtime loader impact. | +| Multi-component validation | Clean | Plan includes shell guard, isolated subject proof, reviewer scoring, content/cross-ref/path checks, and full suite. | +| Declared integration proof | Clean | Matrix avoids overclaiming runtime rendering; browser/click capture are deferred with follow-up. | +| Contributed UI rendering proof | Clean | No host UI contribution. | +| Rollback story | Clean | Revert commits; no runtime state/migrations. | +| Simpler alternative not considered | Clean | Alternatives handled in design. | +| User-intent drift | Clean | Host-neutral visual companion satisfies #78; browser parity tracked as follow-up. | +| Existence/runtime-validity | Clean | Existing files named; emitted files have consumers in validation commands. | +| Over/under-decomposition | Clean | Three serial tasks match implementation/proof/validation phases. | +| Verification-class mismatch | Clean | Documentation and skill behavior changes have class-appropriate checks and proof. | +| Auth/authz chain composition | Clean | No auth/authz chain. | +| Hidden serial dependencies | Clean | Serial dependencies are explicit. | +| Missing rollback wiring | Clean | Rollback wired per task. | +| Missing integration proof | Clean | Isolated-subject and reviewer proof cover skill→agent behavior boundary. | +| Missing declared integration matrix | Clean | Present in plan. | +| Missing contributed UI route proof | Clean | Not applicable. | +| Infrastructure verification mismatch | Clean | No infrastructure change. | +| Plugin-loader runtime layout | Clean | No plugin process/layout change. | +| Config-validation schema rules | Clean | No schema config. | +| Identifier/naming-convention match | Clean | Names follow repo kebab-case/test-fixture conventions. | +| Planned-code compile-validity | Clean | Embedded Bash uses valid syntax and `set -euo pipefail`; no compiled snippets. | + +**Options the author may not have considered:** +1. Commit RED proof separately before implementation; rejected as extra commit noise, mitigated by immediate proof artifact capture. +2. Inline-only companion; rejected in design, mitigated by bounded lazy-loaded guide. +3. Full browser runtime now; rejected as separate runtime/security/process scope. + +**Verdict reasoning:** PASS. The plan now has no Critical or Important findings. Minor proof-capture and validation ambiguities have been resolved in the plan text. diff --git a/docs/plans/2026-06-30-brainstorm-visualization.md b/docs/plans/2026-06-30-brainstorm-visualization.md new file mode 100644 index 0000000..0f1bbfd --- /dev/null +++ b/docs/plans/2026-06-30-brainstorm-visualization.md @@ -0,0 +1,389 @@ +# Brainstorm Visualization Implementation Plan + +> **For the implementing agent:** REQUIRED SUB-SKILL: Use autodev:executing-plans to implement this plan task-by-task. + +**Goal:** Add host-neutral brainstorm/design visualization companion guidance for issue #78. + +**Architecture:** Extend `skills/brainstorming/SKILL.md` with just-in-time visual companion rules while keeping existing gates intact. Add a lazy-loaded guide, behavior fixture, focused regression guard, coverage/doc updates, follow-up tracking, and transcript-backed pressure proof. + +**Tech Stack:** Markdown skills/docs; Bash regression tests; subagent pressure testing; GitHub issue/PR workflow. + +**Base branch:** main + +--- + +## Scope Manifest + +**PR Count:** 1 +**Tasks:** 3 +**Estimated Lines of Change:** ~300 + +**Out of scope:** +- Browser server/runtime companion. +- Click/event capture or telemetry. +- Claiming Mermaid rendering is runtime-integrated on hosts where it has not been observed. + +**PR Grouping:** + +| PR # | Title | Tasks | Branch | +|------|-------|-------|--------| +| 1 | Add brainstorm visual companion guidance | Task 1, Task 2, Task 3 | issue-78-brainstorm-visualization | + +**Status:** Complete 2026-06-30T11:25:38Z + +--- + +## Declared Integration Matrix + +| declared visual capability | host(s) | state | plan proof / rationale | +|---|---|---|---| +| Markdown text visual explanations | all supported hosts | config-only | Skill emits normal markdown/chat text; no host-specific runtime API changes. `tests/brainstorm-visual-companion.sh` asserts text fallback/source-of-truth contract. | +| Mermaid/rendered diagrams | all supported hosts | config-only / best-effort | Skill may emit Mermaid, but rendering is not claimed unless observed in a specific host. Every Mermaid visual requires text fallback. | +| Browser companion tab | all supported hosts | deferred | Tracked in `docs/FOLLOWUPS.md`; needs separate runtime/process/security design. | +| Click/event capture | all supported hosts | deferred | Tracked in `docs/FOLLOWUPS.md`; no browser session state in this PR. | + +--- + +## Pressure-Proof Execution Protocol + +Behavior proof means an isolated subject applies the skill under pressure; it is not self-review. + +1. Prefer a native subagent/delegation tool when the current host exposes one. +2. If native subagents are unavailable, use a fresh isolated host thread/session in the same worktree and paste the exact subject prompt. +3. Capture the subject output and reviewer score in `docs/plans/2026-06-30-brainstorm-visualization-behavior-proof.md`. +4. If no isolated subject can be run, stop before implementation and record the blocker; do not replace the proof with self-review. + +--- + +### Task 1: Integrated TDD Implementation + +**Files:** +- Create: `tests/brainstorm-visual-companion.sh` +- Create: `skills/brainstorming/test-fixtures/visual-companion/expected-behavior.md` +- Modify: `skills/brainstorming/SKILL.md` +- Create: `skills/brainstorming/visual-companion.md` +- Modify: `AGENTS.md` +- Modify: `docs/cross-llm-coverage.md` +- Modify: `tests/cross-llm-coverage.md` +- Create: `docs/FOLLOWUPS.md` + +**Step 1: RED behavior baseline** + +Dispatch a baseline subject agent against the current pre-change `skills/brainstorming/SKILL.md` with this prompt: + +```text +You are testing current brainstorming behavior before issue #78 is implemented. +Read `skills/brainstorming/SKILL.md` only. +Scenario: A user is brainstorming a dashboard redesign quickly. First they ask a conceptual question: "what does trust mean for this dashboard?" Later they ask a visual layout choice that would be clearer shown than described. Then they decline visuals to save time. Finally they change a decision after a diagram would have existed. +First answer what messages/actions you would actually take in the scenario using only the current skill. Then separately mark what the current skill explicitly requires for: conceptual text-only, visual offer, declined offer/no re-offer, accepted visual with text fallback, stale visual. Cite skill lines/sections. If the skill lacks a rule, say MISSING. +``` + +Expected RED: baseline output reports MISSING for visual offer, declined/no re-offer, accepted visual/text fallback, and stale visual. Immediately create `docs/plans/2026-06-30-brainstorm-visualization-behavior-proof.md` with the raw or summarized RED baseline output; append GREEN proof in Task 2. + +**Step 2: Write failing shell regression guard** + +Create `tests/brainstorm-visual-companion.sh`: + +```bash +#!/usr/bin/env bash +# Regression guard for issue #78 brainstorm visualization companion. +set -euo pipefail +ROOT="$(cd "$(dirname "$0")/.." && pwd)" +BRAIN="$ROOT/skills/brainstorming/SKILL.md" +GUIDE="$ROOT/skills/brainstorming/visual-companion.md" +FIXTURE="$ROOT/skills/brainstorming/test-fixtures/visual-companion/expected-behavior.md" +AGENTS="$ROOT/AGENTS.md" +CAPS="$ROOT/docs/cross-llm-coverage.md" +SKILL_COVERAGE="$ROOT/tests/cross-llm-coverage.md" +fail=0 +pass(){ printf 'PASS: %s\n' "$1"; } +bad(){ printf 'FAIL: %s\n' "$1" >&2; fail=$((fail+1)); } +has(){ grep -qiF "$2" "$1"; } +hasE(){ grep -qiE "$2" "$1"; } + +has "$BRAIN" "Visual Companion" && pass "brainstorming has Visual Companion section" || bad "brainstorming missing Visual Companion section" +has "$BRAIN" "just-in-time" && pass "visual companion offer is just-in-time" || bad "visual companion offer is not just-in-time" +hasE "$BRAIN" 'not[ -]upfront|never[ -]upfront|NOT upfront' && pass "visual companion is not offered upfront" || bad "visual companion upfront prohibition missing" +has "$BRAIN" "per-question" && pass "visual decision is per-question" || bad "per-question decision rule missing" +has "$BRAIN" "text remains the source of truth" && pass "text source-of-truth rule present" || bad "text source-of-truth rule missing" +has "$BRAIN" "question batch" && pass "visual offer batch-budget rule present" || bad "visual offer batch-budget rule missing" +has "$BRAIN" "lazy-load" && pass "guide lazy-load rule present" || bad "guide lazy-load rule missing" +has "$BRAIN" "visual-companion.md" && pass "guide reference present" || bad "guide reference missing" + +[ -f "$GUIDE" ] && pass "visual companion guide exists" || bad "visual companion guide missing" +if [ -f "$GUIDE" ]; then + has "$GUIDE" "Mermaid" && pass "guide mentions Mermaid" || bad "guide missing Mermaid guidance" + has "$GUIDE" "accessibility" && pass "guide mentions accessibility" || bad "guide missing accessibility guidance" + has "$GUIDE" "fallback" && pass "guide mentions fallback" || bad "guide missing fallback guidance" + has "$GUIDE" "secrets" && pass "guide forbids secrets" || bad "guide missing secrets guidance" + has "$GUIDE" "SKILL.md remains authoritative" && pass "guide is bounded by SKILL.md authority" || bad "guide authority boundary missing" +fi + +[ -f "$FIXTURE" ] && pass "behavior fixture exists" || bad "behavior fixture missing" +if [ -f "$FIXTURE" ]; then + for marker in "conceptual text-only" "visual offer" "declines" "accepts" "stale visual" "RED baseline"; do + has "$FIXTURE" "$marker" && pass "fixture covers $marker" || bad "fixture missing $marker" + done +fi + +has "$AGENTS" "tests/brainstorm-visual-companion.sh" && pass "AGENTS documents visual companion test" || bad "AGENTS missing visual companion test" +has "$CAPS" "Visual companion" && pass "capability matrix includes visual companion" || bad "capability matrix missing visual companion" +has "$CAPS" "browser deferred" && pass "capability matrix tracks browser deferral" || bad "capability matrix missing browser deferral" +has "$SKILL_COVERAGE" "visual companion" && pass "skill coverage notes visual companion" || bad "skill coverage missing visual companion note" + +echo ""; echo "Results: $fail failure(s)"; [ "$fail" -eq 0 ] +``` + +Run: `bash tests/brainstorm-visual-companion.sh` + +Expected RED representative failures (the full output may include additional missing-marker failures such as `just-in-time`, `per-question`, `question batch`, and `lazy-load`): + +```text +FAIL: brainstorming missing Visual Companion section +FAIL: visual companion guide missing +FAIL: behavior fixture missing +FAIL: AGENTS missing visual companion test +Results: failure(s) +``` + +**Step 3: Add behavior fixture** + +Create `skills/brainstorming/test-fixtures/visual-companion/expected-behavior.md`: + +```markdown +# Brainstorm Visual Companion Expected Behavior + +## RED baseline + +Pre-change `skills/brainstorming/SKILL.md` has no Visual Companion section, no just-in-time visual-offer rule, no lazy-load guide rule, and no stale visual rule. A baseline agent has no explicit reason to offer visuals only when useful or to preserve text as source of truth. + +## Pressure scenario + +A user asks an agent to brainstorm a dashboard redesign quickly. The session has limited question budget, mixed conceptual and layout decisions, and the user may decline visuals to save time. + +## Expected compliant paths + +| path | expected behavior | +|---|---| +| conceptual text-only | For conceptual questions such as "what does trust mean for this dashboard?", the agent stays in text and does not offer the visual companion. | +| visual offer | When the next decision is genuinely visual, the agent sends only the just-in-time visual-companion offer in its own message; it does not bundle a clarifying question. | +| declines | If the user declines, the agent continues text-only and does not offer again unless the user raises visuals. | +| accepts | If the user accepts, the agent lazy-loads `visual-companion.md` only while composing visual artifacts, includes a concise text fallback, and records the final decision in text. | +| stale visual | If a visual becomes stale or contradicts a later text decision, the agent retires or updates it and treats text as the source of truth. | +``` + +**Step 4: Update brainstorming skill** + +Modify `skills/brainstorming/SKILL.md`: + +- Checklist: insert after project guidance: + - `Offer the visual companion just-in-time — never upfront; only when a question would genuinely be clearer shown than described.` +- Process flow: add an `Offer visual companion if useful` box between project guidance and clarifying questions. +- Understanding/process bullets: + - Visual offer is its own message. + - It counts as one question batch. + - Use per-question decision: visual only if seeing beats reading. + - If declined, continue text-only and do not re-offer unless the user raises visuals. + - Lazy-load `skills/brainstorming/visual-companion.md` only after acceptance or while composing a visual artifact. +- Add `## Visual Companion` section: + - Visuals are a tool, not a mode. + - Use visuals for mockups, Mermaid diagrams, flows, state diagrams, architecture maps, side-by-side layout comparisons. + - Use terminal/text for requirements, conceptual choices, tradeoff tables, scope decisions. + - Text remains the source of truth. + - Every visual needs a concise text equivalent/fallback. + - Invalid/unsupported Mermaid → fall back to text. + - Stale/contradictory visual → retire/update it. + - No secrets/PII in visuals. + +**Step 5: Add bounded guide** + +Create `skills/brainstorming/visual-companion.md` with these sections only: `Lazy-load rule`, `When to use visuals`, `When to stay text-only`, `Mermaid and diagram examples`, `Mockups and comparisons`, `Accessibility and fallback`, `Privacy and safety`, `Stale visuals`. Include one Mermaid example and one mockup guidance list. State: `SKILL.md remains authoritative`; do not duplicate full brainstorming workflow. + +**Step 6: Update durable docs** + +- `AGENTS.md`: add `bash tests/brainstorm-visual-companion.sh` under skill content checks. +- `docs/cross-llm-coverage.md`: add `Visual companion output` row with markdown/Mermaid best-effort and browser deferred for every host. Add a note that rendered diagrams require text fallback and browser/event capture is deferred. +- `tests/cross-llm-coverage.md`: update brainstorming note: `Visual companion guidance is host-neutral and best-effort; browser/event capture is deferred.` +- Create `docs/FOLLOWUPS.md`: + +```markdown +# Follow-ups + +## Brainstorm visual companion browser parity + +- **Source:** issue #78 / `docs/plans/2026-06-30-brainstorm-visualization-design.md` +- **Status:** Deferred +- **Follow-up:** Evaluate whether ADK should ship an upstream-like browser companion with tab launch, click/event capture, session security, cleanup, and host process validation. +``` + +**Step 7: Verify GREEN** + +Run: `bash tests/brainstorm-visual-companion.sh` + +Expected: + +```text +PASS: brainstorming has Visual Companion section +... +Results: 0 failure(s) +``` + +**Step 8: Pre-commit quick checks** + +Run: + +```bash +bash tests/brainstorm-visual-companion.sh +bash tests/skill-content-grep.sh +bash tests/skill-cross-refs.sh +bash tests/no-machine-paths.sh +``` + +Expected: all PASS / `Results: 0 failure(s)`. + +**Step 9: Commit** + +Run: + +```bash +git add AGENTS.md docs/cross-llm-coverage.md tests/cross-llm-coverage.md docs/FOLLOWUPS.md tests/brainstorm-visual-companion.sh skills/brainstorming/SKILL.md skills/brainstorming/visual-companion.md skills/brainstorming/test-fixtures/visual-companion/expected-behavior.md docs/plans/2026-06-30-brainstorm-visualization-behavior-proof.md +git commit -m "Add brainstorm visual companion guidance" +``` + +**Rollback:** revert this commit; rerun `bash tests/brainstorm-visual-companion.sh` only if keeping the test, otherwise rerun full suite after revert. + +--- + +### Task 2: Transcript-Backed Behavior Proof + +**Files:** +- Create: `docs/plans/2026-06-30-brainstorm-visualization-behavior-proof.md` +- Modify implementation files only if proof finds gaps. + +**Step 1: GREEN isolated-subject pressure run** + +Use the Pressure-Proof Execution Protocol above: native subagent if available; otherwise a fresh isolated host thread/session in the same worktree. Give the isolated subject this prompt: + +```text +Use `skills/brainstorming/SKILL.md` and, only if the skill instructs you to, `skills/brainstorming/visual-companion.md`. +Scenario: A user is brainstorming a dashboard redesign quickly. First they ask the conceptual question "what does trust mean for this dashboard?" Later they ask a visual layout choice that would be clearer shown than described. Then they decline visuals to save time. Finally imagine they had accepted a visual and then changed a decision so the visual is stale. +Return the exact messages/actions you would take for: conceptual text-only, visual offer, declines, accepts, stale visual. Cite the skill/guide rule that governs each path. +``` + +Expected: subject demonstrates all five fixture paths and cites current skill/guide rules. + +**Step 2: Reviewer scores the transcript** + +Dispatch a reviewer with the subject output plus `skills/brainstorming/SKILL.md`, `skills/brainstorming/visual-companion.md`, and `skills/brainstorming/test-fixtures/visual-companion/expected-behavior.md`: + +```text +Score the subject transcript against all expected paths. Verify each cited rule exists in the current skill or guide. Bias toward finding failures. PASS only if conceptual text-only, visual offer, declines, accepts, and stale visual paths match the fixture and cite real rules. +``` + +Expected: PASS. + +**Step 3: Write proof artifact** + +Append GREEN proof to `docs/plans/2026-06-30-brainstorm-visualization-behavior-proof.md`: + +```markdown +# Brainstorm Visualization Behavior Proof + +## RED baseline + + + +## GREEN subject run + + + +## Reviewer score + +PASS for: conceptual text-only, visual offer, declines, accepts, stale visual. + +## Evidence + +- `bash tests/brainstorm-visual-companion.sh` → `Results: 0 failure(s)` +``` + +**Step 4: Fix and re-run if needed** + +If reviewer FAILs: update `skills/brainstorming/SKILL.md` or guide minimally, rerun `bash tests/brainstorm-visual-companion.sh`, rerun subject + reviewer proof, then continue. + +**Step 5: Commit proof/fixes** + +Run: + +```bash +git add docs/plans/2026-06-30-brainstorm-visualization-behavior-proof.md skills/brainstorming/SKILL.md skills/brainstorming/visual-companion.md +git commit -m "Prove visual companion behavior" +``` + +If no implementation fix was needed, commit only the proof artifact. + +**Rollback:** revert proof/fix commit; no runtime state. + +--- + +### Task 3: Full Validation and PR Prep + +**Files:** +- No production file changes expected unless validation finds a gap. + +**Step 1: Documentation validation note** + +This repo has no dedicated markdown renderer/spellchecker command. For markdown/doc changes, use the focused shell guard, `skill-cross-refs`, `no-machine-paths`, and manual editor preview where available. + +**Step 2: Run targeted checks** + +Run: + +```bash +bash tests/brainstorm-visual-companion.sh +bash tests/skill-content-grep.sh +bash tests/skill-cross-refs.sh +bash tests/no-machine-paths.sh +``` + +Expected: all PASS / `Results: 0 failure(s)`. + +**Step 3: Run full documented suite** + +Run AGENTS.md commands: + +```bash +bash tests/skill-content-grep.sh +bash tests/adk-path-canonicalization.sh +bash tests/pipeline-evidence-doc-sync.sh +bash tests/skill-cross-refs.sh +bash tests/brainstorm-visual-companion.sh +bash tests/hook-contracts.sh +bash tests/hook-stdout-discipline.sh +bash tests/plan-scope-check-contracts.sh +bash tests/version-check.sh +``` + +Expected: all PASS, `OK: All version files agree on version 6.6.1`. + +**Step 4: Commit validation fixes if needed** + +If Step 2-3 required edits: + +```bash +git add +git commit -m "Tighten visual companion validation" +``` + +If no edits, no commit. + +**Step 5: PR body evidence** + +PR body must include: + +- Issue: closes #78. +- RED baseline summary from behavior proof. +- GREEN subject/reviewer behavior proof summary. +- Targeted and full-suite command outputs. +- Deferred browser/event-capture parity follow-up path: `docs/FOLLOWUPS.md`. + +**Rollback:** revert implementation/proof commits and rerun the full documented suite. No runtime state or migrations. diff --git a/skills/brainstorming/SKILL.md b/skills/brainstorming/SKILL.md index 9aae96e..8e86852 100644 --- a/skills/brainstorming/SKILL.md +++ b/skills/brainstorming/SKILL.md @@ -27,14 +27,15 @@ You MUST create a task for each of these items and complete them in order: 1. **Explore project context** — check files, docs, recent commits 2. **Apply project design guidance** — invoke `autodev:project-design-guidance`; read existing guidance or run its Q&A if no durable guidance exists -3. **Ask clarifying questions** — adaptive batching: group related questions to reduce round-trips; use targeted singles for follow-ups -4. **Propose 2-3 approaches** — with trade-offs and your recommendation -5. **List load-bearing assumptions explicitly** — every design rests on assumptions ("upstream API is idempotent", "single-tenant", "user has admin"); write them down so the adversarial reviewer can attack them -6. **Self-challenge round** — before presenting to the user, role-play a skeptic against your own design and surface the top 3 doubts (see "Self-challenge round" below). Cleans up obvious issues before the user sees the design. -7. **Present design** — in sections scaled to their complexity, get user approval after each section. Include the assumption list, project-guidance fit, and the top doubts surfaced by the self-challenge so the user sees them. -8. **Write design doc** — save to `docs/plans/YYYY-MM-DD--design.md` and commit. Use `autodev:condensed-pipeline-writing` for internal density. The doc MUST include `## Global Design Guidance`, `## Security Review`, `## Infrastructure Impact`, `## Multi-Component Validation`, `## Assumptions`, and a `## Rollback` section for change classes that affect runtime (build, deployment, version pins, startup config, migrations, plugin loading paths) — same trigger list as `runtime-launch-validation`. -9. **Adversarial design review** — invoke `adversarial-design-review --phase=design`. On FAIL, revise per tangible Critical/Important findings and re-run until the review converges to no tangible issues (remaining nitpicks become Minor PASS items). On PASS, proceed. -10. **Transition to implementation** — invoke writing-plans skill to create implementation plan +3. **Watch for visual companion opportunities** — offer just-in-time, never upfront, only when a question would genuinely be clearer shown than described +4. **Ask clarifying questions** — adaptive batching: group related questions to reduce round-trips; use targeted singles for follow-ups +5. **Propose 2-3 approaches** — with trade-offs and your recommendation +6. **List load-bearing assumptions explicitly** — every design rests on assumptions ("upstream API is idempotent", "single-tenant", "user has admin"); write them down so the adversarial reviewer can attack them +7. **Self-challenge round** — before presenting to the user, role-play a skeptic against your own design and surface the top 3 doubts (see "Self-challenge round" below). Cleans up obvious issues before the user sees the design. +8. **Present design** — in sections scaled to their complexity, get user approval after each section. Include the assumption list, project-guidance fit, and the top doubts surfaced by the self-challenge so the user sees them. +9. **Write design doc** — save to `docs/plans/YYYY-MM-DD--design.md` and commit. Use `autodev:condensed-pipeline-writing` for internal density. The doc MUST include `## Global Design Guidance`, `## Security Review`, `## Infrastructure Impact`, `## Multi-Component Validation`, `## Assumptions`, and a `## Rollback` section for change classes that affect runtime (build, deployment, version pins, startup config, migrations, plugin loading paths) — same trigger list as `runtime-launch-validation`. +10. **Adversarial design review** — invoke `adversarial-design-review --phase=design`. On FAIL, revise per tangible Critical/Important findings and re-run until the review converges to no tangible issues (remaining nitpicks become Minor PASS items). On PASS, proceed. +11. **Transition to implementation** — invoke writing-plans skill to create implementation plan ## Process Flow @@ -42,6 +43,7 @@ You MUST create a task for each of these items and complete them in order: digraph brainstorming { "Explore project context" [shape=box]; "Apply project guidance" [shape=box]; + "Offer visual companion if useful" [shape=box]; "Ask clarifying questions" [shape=box]; "Propose 2-3 approaches" [shape=box]; "List assumptions" [shape=box]; @@ -53,7 +55,8 @@ digraph brainstorming { "Invoke writing-plans skill" [shape=doublecircle]; "Explore project context" -> "Apply project guidance"; - "Apply project guidance" -> "Ask clarifying questions"; + "Apply project guidance" -> "Offer visual companion if useful"; + "Offer visual companion if useful" -> "Ask clarifying questions"; "Ask clarifying questions" -> "Propose 2-3 approaches"; "Propose 2-3 approaches" -> "List assumptions"; "List assumptions" -> "Self-challenge round"; @@ -74,6 +77,9 @@ digraph brainstorming { **Understanding the idea:** - Check out the current project state first (files, docs, recent commits) - Invoke `autodev:project-design-guidance` before finalizing questions or approaches. If no durable guidance exists, ask the guidance Q&A first so the feature-specific brainstorm is not designed in isolation. +- Offer the visual companion just-in-time, not upfront. Only offer when the next question would genuinely be clearer shown than described, and send only the offer in that message; the offer counts as one question batch. If the user declines, continue text-only and do not re-offer unless the user raises visuals. +- Decide per-question whether visuals help. Use visuals for mockups, Mermaid diagrams, flows, state diagrams, architecture maps, or side-by-side layout comparisons. Stay text-only for requirements, conceptual choices, tradeoff tables, and scope decisions. +- If the user accepts, lazy-load `skills/brainstorming/visual-companion.md` only while composing visual artifacts. Text remains the source of truth; every visual needs a concise text fallback. - Ask questions using adaptive batching — group related questions to reduce round-trips: - **First batch:** covers purpose, constraints, scope, and tech choices - **Follow-ups:** Targeted single questions based on interesting or ambiguous answers @@ -124,6 +130,20 @@ If any answer surfaces a real issue, revise the design before presenting it. Oth This is intentionally lightweight; the heavyweight pass is `adversarial-design-review`, which runs after the design is committed. +## Visual Companion + +Visuals are a tool, not a mode. Use them only when seeing the option, flow, relationship, or layout would help the user decide better than prose. + +**Offer rule:** The first visual-companion offer is just-in-time and never upfront. The offer must be its own message, counts as one question batch, and asks whether the user wants a visual aid. Do not bundle another clarifying question into the same message. + +**Use visuals for:** mockups, wireframes, Mermaid diagrams, state flows, entity relationships, architecture maps, spatial comparisons, and side-by-side visual design options. + +**Stay text-only for:** requirements questions, conceptual choices, tradeoff lists, scope decisions, implementation approaches, or any question where the answer is words rather than a visual preference. + +**Fallback and correctness:** text remains the source of truth. Every visual must include a concise text equivalent. If Mermaid is invalid or unsupported, continue in plain text and do not claim rendered proof. If a visual becomes stale or contradicts a later text decision, retire or update it before proceeding. Do not include secrets or PII in diagrams or mockups. + +**Guide loading:** lazy-load `skills/brainstorming/visual-companion.md` only after the user accepts the companion or when composing a visual artifact. `SKILL.md` remains authoritative; the guide provides examples, accessibility rules, and fallback details. + ## Question-batch budget (cost-control gate) A brainstorming session has a **soft cap of 5 question-batches** before the agent must converge. A "batch" is one round-trip — one outgoing message that contains questions (whether one targeted follow-up or four AskUserQuestion-style options). The cap exists so a session cannot spiral indefinitely while the user keeps answering; convergence is a feature, not an accident. diff --git a/skills/brainstorming/test-fixtures/visual-companion/expected-behavior.md b/skills/brainstorming/test-fixtures/visual-companion/expected-behavior.md new file mode 100644 index 0000000..1648a0f --- /dev/null +++ b/skills/brainstorming/test-fixtures/visual-companion/expected-behavior.md @@ -0,0 +1,19 @@ +# Brainstorm Visual Companion Expected Behavior + +## RED baseline + +Pre-change `skills/brainstorming/SKILL.md` has no Visual Companion section, no just-in-time visual-offer rule, no lazy-load guide rule, and no stale visual rule. A baseline agent has no explicit reason to offer visuals only when useful or to preserve text as source of truth. + +## Pressure scenario + +A user asks an agent to brainstorm a dashboard redesign quickly. The session has limited question budget, mixed conceptual and layout decisions, and the user may decline visuals to save time. + +## Expected compliant paths + +| path | expected behavior | +|---|---| +| conceptual text-only | For conceptual questions such as "what does trust mean for this dashboard?", the agent stays in text and does not offer the visual companion. | +| visual offer | When the next decision is genuinely visual, the agent sends only the just-in-time visual-companion offer in its own message; it does not bundle a clarifying question, and the offer counts as one question batch. | +| declines | If the user declines, the agent continues text-only and does not offer again unless the user raises visuals. | +| accepts | If the user accepts, the agent lazy-loads `visual-companion.md` only while composing visual artifacts, includes a concise text fallback, and records the final decision in text. | +| stale visual | If a visual becomes stale or contradicts a later text decision, the agent retires or updates it and treats text as the source of truth. | diff --git a/skills/brainstorming/visual-companion.md b/skills/brainstorming/visual-companion.md new file mode 100644 index 0000000..4ee5460 --- /dev/null +++ b/skills/brainstorming/visual-companion.md @@ -0,0 +1,57 @@ +# Visual Companion Guide + +## Lazy-load rule + +Load this guide only after the user accepts the Visual Companion, or when the user directly asks for a visual and you are composing that visual artifact. `SKILL.md remains authoritative`; this file provides examples, accessibility guidance, and fallback details. + +## When to use visuals + +Use a visual when seeing the structure is clearer than reading prose: + +- UI mockups, wireframes, navigation structures, layout comparisons +- Mermaid flowcharts, sequence diagrams, state diagrams, ER diagrams, timelines +- Architecture maps, data-flow diagrams, relationship maps +- Side-by-side visual options where spatial hierarchy matters + +## When to stay text-only + +Stay in terminal/chat text for conceptual or scope decisions: + +- Requirements and constraints +- Tradeoff tables and approach selection +- Domain language such as "what does trust mean?" +- Questions where the answer is words rather than visual preference + +## Mermaid and diagram examples + +Keep diagrams small and include a text fallback. + +```mermaid +flowchart TD + A[User goal] --> B[Design option] + B --> C{Decision} + C --> D[Record text decision] +``` + +Text fallback: "User goal leads to a design option, then a decision, then the decision is recorded in text." + +## Mockups and comparisons + +For mockups, show only the decision-relevant structure: + +- 2-4 options maximum +- Use synthetic/redacted content +- State what question the visual answers +- Summarize each option in text before asking for feedback + +## Accessibility and fallback + +Every visual needs a concise text fallback. If Mermaid does not render, if the user is remote/headless, or if the visual would be hard to read, continue in text. Accept text-only feedback as primary. + +## Privacy and safety + +Never include secrets, credentials, tokens, private customer data, or PII in visuals. Use placeholders or synthetic examples. + +## Stale visuals + +If the user changes a decision after a visual was shown, retire or update the stale visual. The current text design remains the source of truth. diff --git a/tests/brainstorm-visual-companion.sh b/tests/brainstorm-visual-companion.sh new file mode 100644 index 0000000..b5645b7 --- /dev/null +++ b/tests/brainstorm-visual-companion.sh @@ -0,0 +1,70 @@ +#!/usr/bin/env bash +# Regression guard for issue #78 brainstorm visualization companion. +set -euo pipefail +ROOT="$(cd "$(dirname "$0")/.." && pwd)" +BRAIN="$ROOT/skills/brainstorming/SKILL.md" +GUIDE="$ROOT/skills/brainstorming/visual-companion.md" +FIXTURE="$ROOT/skills/brainstorming/test-fixtures/visual-companion/expected-behavior.md" +AGENTS="$ROOT/AGENTS.md" +CAPS="$ROOT/docs/cross-llm-coverage.md" +SKILL_COVERAGE="$ROOT/tests/cross-llm-coverage.md" +fail=0 +pass(){ printf 'PASS: %s\n' "$1"; } +bad(){ printf 'FAIL: %s\n' "$1" >&2; fail=$((fail+1)); } +has(){ grep -qiF "$2" "$1"; } +hasE(){ grep -qiE "$2" "$1"; } + +has "$BRAIN" "Visual Companion" && pass "brainstorming has Visual Companion section" || bad "brainstorming missing Visual Companion section" +has "$BRAIN" "just-in-time" && pass "visual companion offer is just-in-time" || bad "visual companion offer is not just-in-time" +hasE "$BRAIN" 'not[ -]upfront|never[ -]upfront|NOT upfront' && pass "visual companion is not offered upfront" || bad "visual companion upfront prohibition missing" +has "$BRAIN" "per-question" && pass "visual decision is per-question" || bad "per-question decision rule missing" +has "$BRAIN" "text remains the source of truth" && pass "text source-of-truth rule present" || bad "text source-of-truth rule missing" +has "$BRAIN" "counts as one question batch" && pass "visual offer batch-budget rule present" || bad "visual offer batch-budget rule missing" +has "$BRAIN" "lazy-load" && pass "guide lazy-load rule present" || bad "guide lazy-load rule missing" +has "$BRAIN" "visual-companion.md" && pass "guide reference present" || bad "guide reference missing" +has "$BRAIN" "do not re-offer unless the user raises visuals" && pass "decline no-reoffer rule present" || bad "decline no-reoffer rule missing" +hasE "$BRAIN" 'stale.*retire or update|retire or update.*stale' && pass "stale visual retire/update rule present" || bad "stale visual retire/update rule missing" + +[ -f "$GUIDE" ] && pass "visual companion guide exists" || bad "visual companion guide missing" +if [ -f "$GUIDE" ]; then + has "$GUIDE" "Mermaid" && pass "guide mentions Mermaid" || bad "guide missing Mermaid guidance" + has "$GUIDE" "accessibility" && pass "guide mentions accessibility" || bad "guide missing accessibility guidance" + has "$GUIDE" "fallback" && pass "guide mentions fallback" || bad "guide missing fallback guidance" + has "$GUIDE" "secrets" && pass "guide forbids secrets" || bad "guide missing secrets guidance" + has "$GUIDE" "PII" && pass "guide forbids PII" || bad "guide missing PII guidance" + hasE "$GUIDE" 'synthetic|redacted' && pass "guide requires synthetic or redacted content" || bad "guide missing synthetic/redacted guidance" + has "$GUIDE" "SKILL.md remains authoritative" && pass "guide is bounded by SKILL.md authority" || bad "guide authority boundary missing" +fi + +[ -f "$FIXTURE" ] && pass "behavior fixture exists" || bad "behavior fixture missing" +if [ -f "$FIXTURE" ]; then + for marker in "conceptual text-only" "visual offer" "declines" "accepts" "stale visual" "RED baseline"; do + has "$FIXTURE" "$marker" && pass "fixture covers $marker" || bad "fixture missing $marker" + done + has "$FIXTURE" "counts as one question batch" && pass "fixture covers visual offer batch budget" || bad "fixture missing visual offer batch budget" +fi + +if awk -F'|' ' + BEGIN { rows = 0; ok = 0 } + /^\| Visual companion output \|/ { + rows++ + row_ok = (NF >= 9) + for (i = 3; i <= 8; i++) { + cell = tolower($i) + if (cell !~ /browser deferred/) { row_ok = 0 } + } + if (row_ok) { ok = 1 } + } + END { exit !(rows == 1 && ok == 1) } +' "$CAPS"; then + pass "visual companion row defers browser for all hosts" +else + bad "visual companion row missing per-host browser deferral" +fi + +has "$AGENTS" "tests/brainstorm-visual-companion.sh" && pass "AGENTS documents visual companion test" || bad "AGENTS missing visual companion test" +has "$BRAIN" "PII" && pass "brainstorming skill forbids PII" || bad "brainstorming skill missing PII guidance" +has "$CAPS" "Visual companion" && pass "capability matrix includes visual companion" || bad "capability matrix missing visual companion" +has "$SKILL_COVERAGE" "visual companion" && pass "skill coverage notes visual companion" || bad "skill coverage missing visual companion note" + +echo ""; echo "Results: $fail failure(s)"; [ "$fail" -eq 0 ] diff --git a/tests/cross-llm-coverage.md b/tests/cross-llm-coverage.md index acaa5fd..7f0a9d2 100644 --- a/tests/cross-llm-coverage.md +++ b/tests/cross-llm-coverage.md @@ -10,7 +10,7 @@ host-neutral. Updated whenever a skill changes. | post-merge-retrospective | host-conditional | host-conditional | host-conditional | host-conditional | host-conditional | host-conditional | Inline-vs-subagent decision block in ``; inline-only prose includes Zed; Hermes inline block | | recording-decisions | host-neutral | host-neutral | host-neutral | host-neutral | host-neutral | host-neutral | ADR storage protocol; no host-specific tooling | | scope-lock | host-neutral | host-neutral | host-neutral | host-neutral | host-neutral | host-neutral | Pure markdown + shell-script invariant; no host-specific tooling | -| brainstorming | host-conditional | host-conditional | host-conditional | host-conditional | host-conditional | host-conditional | `AskUserQuestion` in ``; numbered-list fallback includes Zed; Hermes uses `clarify` | +| brainstorming | host-conditional | host-conditional | host-conditional | host-conditional | host-conditional | host-conditional | `AskUserQuestion` in ``; numbered-list fallback includes Zed; Hermes uses `clarify`; visual companion guidance is host-neutral and best-effort; browser/event capture is deferred | | dispatching-parallel-agents | host-neutral | host-neutral | host-neutral | host-neutral | host-neutral | host-neutral | generic parallel-dispatch pattern; no tool-specific refs | | executing-plans | host-conditional | host-conditional | host-conditional | host-conditional | host-conditional | host-conditional | tool-use block in ``; prose fallback includes Zed; Hermes uses `todo` | | finishing-a-development-branch | host-neutral | host-neutral | host-neutral | host-neutral | host-neutral | host-neutral | audited clean — no forbidden tokens; bash-based throughout |