Skip to content

Commit ed96a18

Browse files
Bordaclaude
andcommitted
feat(foundry): add distill external mode
- Add `external` mode to `/distill` — analyses external plugin/skill/agent resource via URL or local path; reads structure (E1–E3), extracts mental model (E4–E6), compares against live local setup (E7–E10), scores candidates (E11), produces adoption brainstorm table with adopt-as-is/tweak/discuss/skip lanes (E12), and recommends install-as-is when Group A is thin or cherry-picking effort is large (E13–E15) - Bump foundry plugin version 0.3.0 → 0.4.0 - Add conflict pre-check and git diff safety gate to distill `lessons` mode — greps target file before writing, flags cross-proposal collisions with ⚠, shows diff for final review before commit - Auto-trigger fast benchmark in calibrate `apply` mode instead of dead-end error when no prior run exists for target - Add `/distill-external` calibration entry with two GT fixture cases (caveman plugin → install-as-is; Karpathy autoresearch → Group A maps to research plugin) - Clarify no-line-wrapping rule covers all message body content in git-commit.md --- Co-authored-by: Claude Code <noreply@anthropic.com>
1 parent 4b4d15f commit ed96a18

7 files changed

Lines changed: 167 additions & 37 deletions

File tree

README.md

Lines changed: 24 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -178,30 +178,30 @@ Skills are multi-agent workflows invoked via slash commands. Each skill composes
178178
179179
After running `/foundry:init link`, foundry skills are available without a prefix. OSS, develop, and research skills always use their plugin prefix.
180180
181-
| Skill | What It Does |
182-
| ---------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
183-
| 🟠 `/brainstorm` | `/brainstorm <idea>` — clarifying questions → approaches → spec → self-mentor review → approval gate; `breakdown <spec>` — ordered task table with per-task skill tags |
184-
| 🟠 `/manage` | Create, update, delete agents/skills/rules; manage `settings.json` permissions; auto type-detection and cross-ref propagation |
185-
| 🟠 `/investigate` | Systematic diagnosis for unknown failures — env, tools, hooks, CI divergence; ranks hypotheses and hands off to the right skill |
186-
| 🟠 `/session` | Parking lot for diverging ideas — auto-parks unanswered questions and deferred threads; `resume` shows pending, `archive` closes, `summary` digests the session |
187-
| 🟠 `/audit` | Config audit: broken refs, inventory drift, docs freshness; `fix [high\|medium\|all]` auto-fixes by severity; `upgrade` applies docs-sourced improvements |
188-
| 🟠 `/calibrate` | Synthetic benchmarks measuring recall vs confidence bias |
189-
| 🟠 `/distill` | Suggest new agents/skills, prune memory, consolidate lessons into rules |
190-
| 🔵 `/develop:plan` | Scope analysis and implementation planning without code changes |
191-
| 🔵 `/develop:feature` | TDD-first feature implementation: codebase analysis, demo test, TDD loop, docs, review |
192-
| 🔵 `/develop:fix` | Reproduce-first bug fixes: regression test, minimal fix, quality stack |
193-
| 🔵 `/develop:debug` | Systematic debugging for known test failures |
194-
| 🔵 `/develop:refactor` | Test-first refactors with scope analysis |
195-
| 🔵 `/develop:review` | Six-agent parallel review of local files or current git diff; no GitHub PR needed |
196-
| 🟢 `/oss:analyse` | GitHub thread analysis; `health` = repo overview + duplicate issue clustering |
197-
| 🟢 `/oss:review` | Tiered parallel review of GitHub PRs; `--reply` drafts welcoming contributor comments |
198-
| 🟢 `/oss:resolve` | OSS fast-close: resolving conflicts + applying review comments via codex-plugin-cc; three source modes: `pr`, `report`, `pr + report` |
199-
| 🟢 `/oss:release` | SemVer-disciplined release pipeline: notes, changelog with deprecation tracking, migration guides, full prepare pipeline |
200-
| 🟣 `/research:topic` | SOTA literature research with codebase-mapped implementation plan |
201-
| 🟣 `/research:plan` | Config wizard: profile-first bottleneck discovery → `program.md` |
202-
| 🟣 `/research:judge` | Research-supervisor review of experimental methodology (APPROVED/NEEDS-REVISION/BLOCKED) |
203-
| 🟣 `/research:run` | Metric-driven iteration loop; `--resume` continues after crash; `--team` for parallel exploration; `--colab` for GPU workloads |
204-
| 🟣 `/research:sweep` | Non-interactive pipeline: auto-plan → judge gate → run |
181+
| Skill | What It Does |
182+
| ---------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
183+
| 🟠 `/brainstorm` | `/brainstorm <idea>` — clarifying questions → approaches → spec → self-mentor review → approval gate; `breakdown <spec>` — ordered task table with per-task skill tags |
184+
| 🟠 `/manage` | Create, update, delete agents/skills/rules; manage `settings.json` permissions; auto type-detection and cross-ref propagation |
185+
| 🟠 `/investigate` | Systematic diagnosis for unknown failures — env, tools, hooks, CI divergence; ranks hypotheses and hands off to the right skill |
186+
| 🟠 `/session` | Parking lot for diverging ideas — auto-parks unanswered questions and deferred threads; `resume` shows pending, `archive` closes, `summary` digests the session |
187+
| 🟠 `/audit` | Config audit: broken refs, inventory drift, docs freshness; `fix [high\|medium\|all]` auto-fixes by severity; `upgrade` applies docs-sourced improvements |
188+
| 🟠 `/calibrate` | Synthetic benchmarks measuring recall vs confidence bias |
189+
| 🟠 `/distill` | Suggest new agents/skills, prune memory, consolidate lessons into rules; `external <source>` analyses an external plugin/skill/agent resource and produces a scored adoption proposal with install-as-is recommendation |
190+
| 🔵 `/develop:plan` | Scope analysis and implementation planning without code changes |
191+
| 🔵 `/develop:feature` | TDD-first feature implementation: codebase analysis, demo test, TDD loop, docs, review |
192+
| 🔵 `/develop:fix` | Reproduce-first bug fixes: regression test, minimal fix, quality stack |
193+
| 🔵 `/develop:debug` | Systematic debugging for known test failures |
194+
| 🔵 `/develop:refactor` | Test-first refactors with scope analysis |
195+
| 🔵 `/develop:review` | Six-agent parallel review of local files or current git diff; no GitHub PR needed |
196+
| 🟢 `/oss:analyse` | GitHub thread analysis; `health` = repo overview + duplicate issue clustering |
197+
| 🟢 `/oss:review` | Tiered parallel review of GitHub PRs; `--reply` drafts welcoming contributor comments |
198+
| 🟢 `/oss:resolve` | OSS fast-close: resolving conflicts + applying review comments via codex-plugin-cc; three source modes: `pr`, `report`, `pr + report` |
199+
| 🟢 `/oss:release` | SemVer-disciplined release pipeline: notes, changelog with deprecation tracking, migration guides, full prepare pipeline |
200+
| 🟣 `/research:topic` | SOTA literature research with codebase-mapped implementation plan |
201+
| 🟣 `/research:plan` | Config wizard: profile-first bottleneck discovery → `program.md` |
202+
| 🟣 `/research:judge` | Research-supervisor review of experimental methodology (APPROVED/NEEDS-REVISION/BLOCKED) |
203+
| 🟣 `/research:run` | Metric-driven iteration loop; `--resume` continues after crash; `--team` for parallel exploration; `--colab` for GPU workloads |
204+
| 🟣 `/research:sweep` | Non-interactive pipeline: auto-plan → judge gate → run |
205205
206206
→ Full command reference, orchestration flows, rules (13 auto-loaded rule files), architecture internals, status line — see [`.claude/README.md` → Skills](.claude/README.md#-skills)
207207

plugins/foundry/.claude-plugin/plugin.json

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -15,5 +15,5 @@
1515
"license": "MIT",
1616
"name": "foundry",
1717
"repository": "https://github.com/Borda/AI-Rig",
18-
"version": "0.3.0"
18+
"version": "0.4.0"
1919
}

plugins/foundry/README.md

Lines changed: 11 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -307,17 +307,21 @@ ______________________________________________________________________
307307

308308
### `/foundry:distill`
309309

310-
Extracts patterns from work history and corrections, then distills them into durable improvements — new agent or skill suggestions, roster quality review, memory pruning, or promoting lessons into rules and agent instruction updates.
310+
Extracts patterns from work history and corrections, then distills them into durable improvements — new agent or skill suggestions, roster quality review, memory pruning, promoting lessons into rules, or analysing external plugins and agentic resources for adoption.
311311

312312
```text
313-
/foundry:distill # analyze project patterns, suggest new agents/skills
314-
/foundry:distill review # review existing roster for quality and gaps (no new suggestions)
315-
/foundry:distill prune # trim stale/redundant entries from project MEMORY.md
316-
/foundry:distill lessons # promote patterns from .notes/lessons.md into rules/agents/skills
317-
/foundry:distill "I keep doing X manually" # use description as context for suggestions
313+
/foundry:distill # analyze project patterns, suggest new agents/skills
314+
/foundry:distill review # review existing roster for quality and gaps (no new suggestions)
315+
/foundry:distill prune # trim stale/redundant entries from project MEMORY.md
316+
/foundry:distill lessons # promote patterns from .notes/lessons.md into rules/agents/skills
317+
/foundry:distill "external https://..." # analyse external plugin/skill/agent resource, produce adoption proposal
318+
/foundry:distill "external ./path/to/plugin" # same — local path or directory
319+
/foundry:distill "I keep doing X manually" # use description as context for suggestions
318320
```
319321

320-
`lessons` mode is the primary post-correction consolidation path. It reads `.notes/lessons.md` and `feedback_*.md` memory files, clusters them by domain, classifies each entry as `-> rule`, `-> agent update`, `-> skill update`, `-> already covered`, or `-> too narrow`, generates proposals, and asks for confirmation before applying anything.
322+
**`lessons` mode** is the primary post-correction consolidation path. It reads `.notes/lessons.md` and `feedback_*.md` memory files, clusters them by domain, classifies each entry as `→ rule`, `→ agent update`, `→ skill update`, `→ already covered`, or `→ too narrow`, then generates proposals. Before applying, it runs a conflict pre-check — greps each target file for the section the delta would land in and flags cross-proposal collisions with ⚠. Confirmed changes are applied and followed by a `git diff` gate so you can inspect or revert before committing.
323+
324+
**`external` mode** does a fast + slow read of the source (URL, file, or directory), extracts the mental model and standout implementation details, compares against the live local setup, then splits candidates into two groups: *Align + improve* (maps cleanly onto existing agents/skills/rules) and *Differentiated highlights* (novel, structurally different — interesting but larger work). Each candidate is scored and assigned to an adoption lane: adopt-as-is / tweak / discuss / skip. When Group A is thin or cumulative edit effort is large, it recommends installing the source as a standalone plugin with justification, rather than cherry-picking. Nothing is written until you confirm.
321325

322326
After applying: run `/foundry:init` to propagate new rule files to `~/.claude/`.
323327

plugins/foundry/rules/git-commit.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,7 @@ paths:
1717
Co-authored-by: Claude Code <noreply@anthropic.com>
1818
```
1919

20-
- No line wrapping — each bullet single long line
20+
- No line wrapping anywhere in the message body — bullets, prose lines, all stay on single long lines; never break at any column width
2121

2222
## Gathering Diff Context
2323

plugins/foundry/skills/calibrate/SKILL.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -243,7 +243,7 @@ TIMESTAMP=$(basename "$LATEST")
243243

244244
For each target in target list, check whether `.reports/calibrate/<TIMESTAMP>/<target>/proposal.md` exists. Collect targets with proposal (`found`) and without (`missing`).
245245

246-
Print ` No proposal found for <target> — run /calibrate <target> [fast|full] first` for each missing target.
246+
For each **missing** target: do not stop — auto-trigger a `fast` benchmark inline. Print ` No prior run for <target> — running fast benchmark now…`, then execute Steps 2–5 for that target with `fast` mode using a fresh timestamp, and add to `found` list once proposals are written. This ensures `apply` alone always produces results rather than a dead-end error.
247247

248248
**Print run's report before applying**: for each found target, read and print `.reports/calibrate/<TIMESTAMP>/<target>/report.md` verbatim so user sees benchmark basis before any file changes.
249249

plugins/foundry/skills/calibrate/modes/skills.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -64,6 +64,7 @@ Modes evaluated for calibration but deferred — significant barriers. `/audit`
6464
| `/distill-review` | Reads real agent/skill files; synthetic roster possible but overlaps `/audit` calibration | Distinct synthetic scenarios identified |
6565
| `/distill-prune` | Likely calibratable — construct a synthetic memory corpus with known entries to drop (stale, redundant, duplicated-in-CLAUDE.md), then score recall of correct drop/trim/keep decisions; ground truth is constructable | Synthetic memory corpus fixtures built |
6666
| `/distill-lessons` | Reads real `.notes/lessons.md`; needs realistic synthetic lesson corpus | Lesson corpus fixtures exist |
67+
| `/distill-external` | Calibratable with two concrete GT fixture cases: **(1) caveman plugin** — narrow communication-mode tool, no local overlap → GT outcome: install-as-is recommendation; **(2) Karpathy autoresearch** — research automation with strong structural overlap to `research:` plugin → GT outcome: Group A candidates map to research plugin, digest recommended. Score whether adoption-table lane assignments (adopt-as-is/tweak/discuss/skip) and install-as-is flag match GT. Ground truth constructable without live external source — fixture = static snapshot of each tool's agent/skill/rule files. | GT fixture snapshots authored |
6768

6869
**Excluded** (inherently non-calibratable — documented to avoid recurring evaluation):
6970

0 commit comments

Comments
 (0)