From f2f47aecd593214877f27a146d17473d2d7dc057 Mon Sep 17 00:00:00 2001 From: Tester Date: Mon, 29 Jun 2026 09:02:49 +0200 Subject: [PATCH] feat(taxonomy): split capability into skill (capability:*) and tool (contract:*/substrate:*) axes MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Implements RFC-AI-0005. "Capability" was one vocabulary doing two jobs: a workflow-lifecycle phase (right for skills) stamped onto tools (wrong), which collapsed ~75% of tools into a meaningless capability:setup bucket and left four capabilities with no tool ever using them. Two orthogonal axes now: - Skill capability `capability:*` — the lifecycle phase (triage, review, fix, intake, reconciliation, resolve, reassess, stats) + setup split into `platform` (setup-* family) and `authoring` (write-skill, optimize-skill). - Tool capability — the interface a tool provides: `contract:` when it implements a capability contract under tools// (tracker, source-control, mail-archive, mail-source, mail-draft, cve-authority, report-relay, scan-format, project-metadata), or `substrate:` (analytics, sandbox, action-guard, privacy, framework-dev). Replaces the capability:setup grab-bag. Changes: - docs/rfcs/RFC-AI-0005.md — the taxonomy RFC (covers area / capability [two-axis] / kind / mode / organization / standalone labels + migration). - skill-and-tool-validator: ALLOWED_CAPABILITIES → SKILL_CAPABILITIES + TOOL_CAPABILITIES; per-axis validation; token regex widened for contract:/substrate: + hyphens; tests updated. - All 33 tools re-labelled to contract:/substrate: values. - The 10 capability:setup skills re-labelled platform/authoring. - docs/labels-and-capabilities.md — two capability tables + two maps; AGENTS.md + tools/AGENTS.md + docs/adapters/authoring.md + write-skill scaffold updated to the new vocab. Validator EXIT 0; validator pytest + mypy green. Generated-by: Claude Code (Opus 4.8) --- AGENTS.md | 33 +-- docs/adapters/authoring.md | 2 +- docs/labels-and-capabilities.md | 178 +++++++------ docs/rfcs/RFC-AI-0005.md | 237 ++++++++++++++++++ skills/optimize-skill/SKILL.md | 2 +- skills/setup-isolated-setup-doctor/SKILL.md | 2 +- skills/setup-isolated-setup-install/SKILL.md | 2 +- skills/setup-isolated-setup-update/SKILL.md | 2 +- skills/setup-isolated-setup-verify/SKILL.md | 2 +- skills/setup-override-upstream/SKILL.md | 2 +- skills/setup-shared-config-sync/SKILL.md | 2 +- skills/setup-status/SKILL.md | 2 +- skills/setup/SKILL.md | 2 +- skills/write-skill/SKILL.md | 7 +- tools/AGENTS.md | 11 +- tools/agent-guard/README.md | 2 +- tools/agent-isolation/README.md | 2 +- tools/apache-projects/README.md | 2 +- tools/cve-org/README.md | 2 +- tools/cve-tool-vulnogram/README.md | 2 +- tools/cve-tool/README.md | 2 +- tools/dashboard-generator/README.md | 2 +- tools/dev/README.md | 2 +- tools/egress-gateway/README.md | 2 +- tools/forwarder-relay/README.md | 2 +- tools/github-body-field/README.md | 2 +- tools/github-rollup/README.md | 2 +- tools/github/README.md | 2 +- tools/gmail/README.md | 2 +- tools/jira/README.md | 2 +- tools/mail-archive/README.md | 2 +- tools/mail-source/README.md | 2 +- tools/permission-audit/README.md | 2 +- tools/ponymail/README.md | 2 +- tools/pr-management-stats/README.md | 2 +- tools/preflight-audit/README.md | 2 +- tools/privacy-llm/README.md | 2 +- tools/probe-templates/README.md | 2 +- tools/sandbox-lint/README.md | 2 +- tools/scan-format/README.md | 2 +- .../README.md | 2 +- tools/skill-and-tool-validator/README.md | 2 +- .../src/skill_and_tool_validator/__init__.py | 67 +++-- .../tests/test_validator.py | 90 +++---- tools/skill-evals/README.md | 2 +- tools/spec-loop/README.md | 2 +- tools/spec-status-index/README.md | 2 +- tools/spec-validator/README.md | 2 +- tools/vcs/README.md | 2 +- 49 files changed, 497 insertions(+), 210 deletions(-) create mode 100644 docs/rfcs/RFC-AI-0005.md diff --git a/AGENTS.md b/AGENTS.md index cd2bc1d1..feabe9ed 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -494,27 +494,28 @@ dimensions on every issue and PR: - **`area:*`** — *what part of the framework does this touch?* (e.g. `area:pr-management`, `area:security`, `area:setup`, `area:issue`, `area:tools`, `area:ci`, `area:docs`). -- **`capability:*`** — *what does the tool / change actually do?* (e.g. - `capability:triage`, `capability:review`, `capability:fix`, - `capability:intake`, `capability:reconciliation`, - `capability:resolve`, `capability:reassess`, `capability:stats`, - `capability:setup`). - -The full taxonomy — every label dimension, every capability bucket, -the skill-to-capability and tool-to-capability maps — lives in -[`docs/labels-and-capabilities.md`](docs/labels-and-capabilities.md). -Read that page once; treat it as the source of truth. +- **capability** — *what does it do / provide?*, in two axes + (RFC-AI-0005): **skill capability** `capability:*` for skills + (`triage`, `review`, `fix`, `intake`, `reconciliation`, `resolve`, + `reassess`, `stats`, `platform`, `authoring`), and **tool capability** + `contract:*` / `substrate:*` for tools (the contract a tool implements, + e.g. `contract:tracker`, or a substrate kind, e.g. `substrate:privacy`). + +The full taxonomy — every label dimension, both capability axes, the +skill-capability and contract→adapter maps — lives in +[`docs/labels-and-capabilities.md`](docs/labels-and-capabilities.md) and +[RFC-AI-0005](docs/rfcs/RFC-AI-0005.md). Read those once; treat them as +the source of truth. **Rules** (full taxonomy and per-target details in [`docs/labels-and-capabilities.md`](docs/labels-and-capabilities.md)): - **Issues and PRs** get at least one `area:*` and every applicable - `capability:*` — match the capabilities the change *implements*, not - the file paths it touches; do not collapse multi-phase work to a single - "primary". -- **New tools** declare their capabilities in the first paragraph of the - tool README (`**Capability:** capability:NAME`); a tool is - `capability:setup` substrate by default. + capability — match what the change *implements*, not the file paths it + touches; do not collapse multi-phase work to a single "primary". +- **New tools** declare their capability in the first paragraph of the + tool README (`**Capability:** contract:NAME` or `substrate:NAME`) — the + contract the tool implements, or the substrate kind that fits. - **New skills** declare the capability in frontmatter (a string, or a YAML list for multi-capability skills); [`write-skill`](skills/write-skill/SKILL.md) prompts for it on every scaffold. diff --git a/docs/adapters/authoring.md b/docs/adapters/authoring.md index b9881ea9..0e148363 100644 --- a/docs/adapters/authoring.md +++ b/docs/adapters/authoring.md @@ -53,7 +53,7 @@ A tool adapter fulfils a capability *contract* for one backend. contract's operations. 3. **Declare the metadata** the validator requires (see [`tools/AGENTS.md`](../../tools/AGENTS.md)): - - a `**Capability:** capability:NAME` line in the README; + - a `**Capability:** contract:NAME` (or `substrate:NAME`) line in the README; - a `## Prerequisites` section (runtime, CLIs, credentials, network); - optionally an `**Organization:** ` line if the adapter belongs to a specific organization. diff --git a/docs/labels-and-capabilities.md b/docs/labels-and-capabilities.md index 4b81cd30..7d6d4a04 100644 --- a/docs/labels-and-capabilities.md +++ b/docs/labels-and-capabilities.md @@ -5,7 +5,7 @@ - [Labels and capabilities](#labels-and-capabilities) - [Label dimensions](#label-dimensions) - [1. `area:*` — subject](#1-area--subject) - - [2. `capability:*` — what the tool does](#2-capability--what-the-tool-does) + - [2. capability — two axes (skills vs tools)](#2-capability--two-axes-skills-vs-tools) - [3. `kind:*` — change type (pre-existing)](#3-kind--change-type-pre-existing) - [4. `mode:*` — handling mode (pre-existing)](#4-mode--handling-mode-pre-existing) - [Standalone labels](#standalone-labels) @@ -65,17 +65,16 @@ What part of the framework does this touch? | `area:ci` | `.github/` workflows, prek, validators | | `area:docs` | `docs/`, `MISSION.md`, READMEs | -### 2. `capability:*` — what the tool does +### 2. capability — two axes (skills vs tools) -Nine buckets. A tool or skill carries **one or more** `capability:*` -labels. Most map cleanly to a single bucket; dual-capability cases -are real and explicitly enumerated below. Issues and PRs follow the -same rule — apply every capability the change is implementing. +Per [RFC-AI-0005](rfcs/RFC-AI-0005.md), "capability" is **two orthogonal +vocabularies**, one per entity. A skill carries one or more **skill +capabilities** (`capability:*`); a tool carries one or more **tool +capabilities** (`contract:*` or `substrate:*`). List **all** that apply; +do not pick a single "primary". -When a skill or tool spans multiple capabilities, list **all** of -them in its frontmatter / README. Do not pick a single "primary" -to be neat; that loses information the label system exists to -surface. +**Axis 1 — skill capability** (`capability:*`) — the workflow-lifecycle +phase a skill performs: | Label | Definition | |---|---| @@ -87,9 +86,32 @@ surface. | `capability:resolve` | Close-out actions: invalidate, dedupe, CVE-allocate, post-announcement housekeeping. | | `capability:reassess` | Re-run resolved or end-of-life issues against current code to verify still-fixed / still-broken. | | `capability:stats` | Read-only dashboards, metrics, governance evidence, contributor nomination briefs. | -| `capability:setup` | Framework / agent / substrate infrastructure: install, verify, update, doctor, override-upstream, write-skill, optimize-skill, plus new tools under `tools/*`. | +| `capability:platform` | Framework / agent substrate skills: install, verify, update, doctor, override-upstream, status, shared-config-sync, the `setup` bootstrap. | +| `capability:authoring` | Skills that author or maintain other skills: `write-skill`, `optimize-skill`. | -The `capability:*` dimension is **orthogonal** to `area:*`. A single +**Axis 2 — tool capability** (`contract:*` / `substrate:*`) — the +interface a tool/adapter provides. `contract:` implements a +capability contract under `tools//`; `substrate:` is +framework substrate: + +| Label | Kind | Definition | +|---|---|---| +| `contract:tracker` | contract | Issue / PR / board / label backend. | +| `contract:source-control` | contract | Branch / commit / diff / push (VCS). | +| `contract:mail-archive` | contract | Public mailing-list / forum archive reads. | +| `contract:mail-source` | contract | Inbound-mail ingestion (mbox / IMAP / …). | +| `contract:mail-draft` | contract | Outbound mail composition (draft, never send). | +| `contract:cve-authority` | contract | CVE allocation / record management / publication. | +| `contract:report-relay` | contract | Inbound security-report relay detection. | +| `contract:scan-format` | contract | Security-scanner report parsing. | +| `contract:project-metadata` | contract | Governance rosters / people / releases. | +| `substrate:analytics` | substrate | Read-only metrics / dashboards / renderers. | +| `substrate:sandbox` | substrate | Agent isolation, egress control, settings audit. | +| `substrate:action-guard` | substrate | Deterministic pre-tool-use command guards. | +| `substrate:privacy` | substrate | PII redaction / approved-LLM gating. | +| `substrate:framework-dev` | substrate | Build / validate / eval the framework itself. | + +Both capability axes are **orthogonal** to `area:*`. A single query can answer "how is our triage stack doing across PR + issue + security?" by filtering on `capability:triage` alone, without enumerating per-area queries. @@ -158,7 +180,7 @@ Capabilities for every skill currently in | `security-issue-import-via-forwarder` | `capability:intake` | | `security-issue-import-from-scan` | `capability:intake` | | `security-issue-sync` | `capability:intake` *(+ `capability:reconciliation` once [#337](https://github.com/apache/magpie/issues/337) lands the ASF-dashboard step)* | -| `setup-shared-config-sync` | `capability:intake` + `capability:setup` *(reconciles user-scope config to a sync repo; the act is intake, the subject is setup)* | +| `setup-shared-config-sync` | `capability:intake` + `capability:platform` *(reconciles user-scope config to a sync repo; the act is intake, the subject is setup)* | | `release-vote-tally` | `capability:triage` *(reads the vote thread / approval signal, classifies each reply as binding or non-binding, tallies the result, and drafts the `[RESULT] [VOTE]` email for RM review — triage over the vote-thread queue)* | | `release-prepare` | `capability:resolve` *(drafts the planning issue, prep PR, and post-release bump PR that open the release lifecycle)* | | `release-announce-draft` | `capability:resolve` *(drafts the `[ANNOUNCE]` email and opens the site-bump PR that complete the release lifecycle)* | @@ -184,15 +206,15 @@ Capabilities for every skill currently in | `committer-onboarding` | `capability:stats` | | `list-skills` | `capability:stats` | | `release-audit-report` | `capability:stats` *(assembles the per-release audit record from the planning issue, vote thread, artefact list, and announce archive URL)* | -| `setup-status` | `capability:stats` + `capability:setup` *(reports the adoption configuration — stats — and delegates reconfiguration to the setup skill)* | -| `setup` | `capability:setup` | -| `setup-isolated-setup-install` | `capability:setup` | -| `setup-isolated-setup-verify` | `capability:setup` | -| `setup-isolated-setup-update` | `capability:setup` | -| `setup-isolated-setup-doctor` | `capability:setup` + `capability:reassess` *(re-checks an installed sandbox against current spec — the phase is reassess on subject setup)* | -| `setup-override-upstream` | `capability:setup` | -| `write-skill` | `capability:setup` | -| `optimize-skill` | `capability:setup` | +| `setup-status` | `capability:stats` + `capability:platform` *(reports the adoption configuration — stats — and delegates reconfiguration to the setup skill)* | +| `setup` | `capability:platform` | +| `setup-isolated-setup-install` | `capability:platform` | +| `setup-isolated-setup-verify` | `capability:platform` | +| `setup-isolated-setup-update` | `capability:platform` | +| `setup-isolated-setup-doctor` | `capability:platform` + `capability:reassess` *(re-checks an installed sandbox against current spec — the phase is reassess on subject setup)* | +| `setup-override-upstream` | `capability:platform` | +| `write-skill` | `capability:authoring` | +| `optimize-skill` | `capability:authoring` | | `skill-reconciler` | `capability:reconciliation` *(compares two near-duplicate skill copies and classifies every difference as ALLOWED, DRIFT, or SAFETY-BASELINE; proposes convergence; never writes either copy)* | ## Capability to tool map @@ -202,52 +224,46 @@ Tools under [`tools/`](../tools/). Tools with two values (separated by | Tool | Capability / capabilities | Role | |---|---|---| -| [`tools/agent-guard`](../tools/agent-guard/) | `capability:setup` | Deterministic `PreToolUse` guard dispatcher: blocks `gh`/`git` commands that would ping maintainers, carry a `Co-Authored-By` trailer, mark-ready prematurely, leak security language publicly, or empty a PR via force-push. Extensible — skills contribute guards via `guards.d` | -| [`tools/agent-isolation`](../tools/agent-isolation/) | `capability:setup` | Secure-agent sandbox helpers | -| [`tools/apache-projects`](../tools/apache-projects/) | `capability:stats` + `capability:intake` | ASF project-metadata substrate (`apache/comdev` `apache-projects-mcp`); read-only `projects.apache.org/json` rosters / people / releases. Backs `contributor-nomination` and the security roster-resolution paths; tracked at `main`, not pinned | -| [`tools/cve-org`](../tools/cve-org/) | `capability:resolve` + `capability:intake` | Publishes to CVE.org *(resolve)* and records the resulting CVE state back into the tracker *(intake)* | -| [`tools/cve-tool`](../tools/cve-tool/) | `capability:setup` | Adapter contract for CNA backends (Vulnogram, MITRE form, CVE.org direct, GHSA). Pure interface spec; no executable code — adapters under sibling `tools/cve-tool-*/` directories implement it. | -| [`tools/cve-tool-vulnogram`](../tools/cve-tool-vulnogram/) | `capability:resolve` | ASF Vulnogram CVE-allocation adapter. Implements the `tools/cve-tool/` contract. Previously named `tools/vulnogram/`. | -| [`tools/dashboard-generator`](../tools/dashboard-generator/) | `capability:stats` | Self-contained HTML dashboard generator | -| [`tools/dev`](../tools/dev/) | `capability:setup` | Framework dev-loop helpers | -| [`tools/egress-gateway`](../tools/egress-gateway/) | `capability:setup` | Egress-allowlist forward proxy (proxy.py plugin); host-level egress chokepoint — defence-in-depth for RFC-AI-0003 §4.4 | -| [`tools/forwarder-relay`](../tools/forwarder-relay/) | `capability:setup` | Adapter contract for inbound-relay backends (ASF Security relay, huntr.com, HackerOne triagers). Pure interface spec; adapters declare detection + credit-extraction + reporter-addressing rules. | -| [`tools/github`](../tools/github/) | `capability:setup` | GitHub REST / GraphQL substrate (called by every lifecycle phase — pure substrate, no single phase) | -| [`tools/github-body-field`](../tools/github-body-field/) | `capability:setup` | Read or rewrite one `### Field` section of a GitHub issue body without bringing the body into agent context — substrate helper for the security-sync skills | -| [`tools/github-rollup`](../tools/github-rollup/) | `capability:setup` | Append to (or create) the status-rollup comment on a GitHub issue without bringing the rollup body into agent context — substrate helper for every status-update-emitting skill | -| [`tools/gmail`](../tools/gmail/) | `capability:setup` | Gmail API substrate | -| [`tools/jira`](../tools/jira/) | `capability:setup` | JIRA REST substrate (read-only today; write subcommands tracked in [#301](https://github.com/apache/magpie/issues/301)) | -| [`tools/mail-archive`](../tools/mail-archive/) | `capability:setup` | Adapter contract for public mail-archive backends (PonyMail, Hyperkitty, Discourse, Google Groups, GitHub Discussions). Pure interface spec. | -| [`tools/mail-source`](../tools/mail-source/) | `capability:setup` + `capability:intake` | Mail-source backend abstraction (mbox / IMAP / Mailman 3); the abstraction is setup, every concrete read is part of the intake pipeline | -| [`tools/ponymail`](../tools/ponymail/) | `capability:setup` + `capability:intake` | PonyMail archive substrate; same dual role as `mail-source` — substrate plus an intake-pipeline component | -| [`tools/scan-format`](../tools/scan-format/) | `capability:intake` | Adapter contract for security-scanner report formats (ASVS reference); reads a scan's finding index + per-finding evidence for the `security-issue-import-from-scan` pipeline. | -| [`tools/permission-audit`](../tools/permission-audit/) | `capability:setup` | Audit + atomically edit Claude Code `permissions.allow[]` entries; backs `/magpie-setup verify --apply-permission-audit` (check 8d) | -| [`tools/pr-management-stats`](../tools/pr-management-stats/) | `capability:stats` | PR-backlog analytics engine | -| [`tools/preflight-audit`](../tools/preflight-audit/) | `capability:stats` | Dry-run the bulk-mode pre-flight classifier; measure skip-rate before / after any rule edit in the security-issue-sync skill | -| [`tools/privacy-llm`](../tools/privacy-llm/) | `capability:setup` | Privacy-LLM PII-scrubbing gate | -| [`tools/probe-templates`](../tools/probe-templates/) | `capability:setup` | Sandbox-doctor probe templates | -| [`tools/sandbox-lint`](../tools/sandbox-lint/) | `capability:setup` | Sandbox settings linter | -| [`tools/security-tracker-stats-dashboard`](../tools/security-tracker-stats-dashboard/) | `capability:stats` | Security-tracker analytics engine | -| [`tools/spec-loop`](../tools/spec-loop/) | `capability:setup` | Spec-driven build loop runner (Ralph-style) for framework development | -| [`tools/skill-evals`](../tools/skill-evals/) | `capability:setup` + `capability:stats` | Eval harness for skills; the harness is setup infrastructure, the run output is governance evidence | -| [`tools/skill-and-tool-validator`](../tools/skill-and-tool-validator/) | `capability:setup` | Skill-frontmatter and convention validator | -| [`tools/spec-status-index`](../tools/spec-status-index/) | `capability:setup` + `capability:stats` | Index of spec / RFC implementation status — substrate that also doubles as a governance/stats view | -| [`tools/spec-validator`](../tools/spec-validator/) | `capability:setup` | Spec-frontmatter and body-section validator — counterpart to `skill-and-tool-validator` for `tools/spec-loop/specs/` | -| [`tools/vcs`](../tools/vcs/) | `capability:setup` | Backend-dispatching implementation of the source-control (VCS) capability ([`tools/github/source-control.md`](../tools/github/source-control.md)); complete Git backend plus detected extension points for non-Git VCS bridges (#601 Hg, #602 SVN) | - -A tool's capabilities are determined by its **use-case lifecycle -phases**, not by which skills happen to consume it. `tools/github` is -called by every triage / intake / fix / resolve skill but is tagged -only `capability:setup` because it doesn't encode any one lifecycle -phase — it is pure substrate. `tools/cve-org`, by contrast, exists -specifically to *do* CVE publication and to record that result; both -the resolve action and the intake of state into the tracker are -first-class jobs of the tool, so it carries both labels. - -When a tool grows to serve a new lifecycle phase as a first-class -feature (rather than as generic substrate that other skills happen -to compose), add the new `capability:*` label to its README and to -the table above. +| [`tools/agent-guard`](../tools/agent-guard/) | `substrate:action-guard` | Deterministic `PreToolUse` guard dispatcher: blocks `gh`/`git` commands that would ping maintainers, carry a `Co-Authored-By` trailer, mark-ready prematurely, leak security language publicly, or empty a PR via force-push. Extensible — skills contribute guards via `guards.d` | +| [`tools/agent-isolation`](../tools/agent-isolation/) | `substrate:sandbox` | Secure-agent sandbox helpers | +| [`tools/apache-projects`](../tools/apache-projects/) | `contract:project-metadata` | ASF project-metadata substrate (`apache/comdev` `apache-projects-mcp`); read-only `projects.apache.org/json` rosters / people / releases. Backs `contributor-nomination` and the security roster-resolution paths; tracked at `main`, not pinned | +| [`tools/cve-org`](../tools/cve-org/) | `contract:cve-authority` | Publishes to CVE.org *(resolve)* and records the resulting CVE state back into the tracker *(intake)* | +| [`tools/cve-tool`](../tools/cve-tool/) | `contract:cve-authority` | Adapter contract for CNA backends (Vulnogram, MITRE form, CVE.org direct, GHSA). Pure interface spec; no executable code — adapters under sibling `tools/cve-tool-*/` directories implement it. | +| [`tools/cve-tool-vulnogram`](../tools/cve-tool-vulnogram/) | `contract:cve-authority` | ASF Vulnogram CVE-allocation adapter. Implements the `tools/cve-tool/` contract. Previously named `tools/vulnogram/`. | +| [`tools/dashboard-generator`](../tools/dashboard-generator/) | `substrate:analytics` | Self-contained HTML dashboard generator | +| [`tools/dev`](../tools/dev/) | `substrate:framework-dev` | Framework dev-loop helpers | +| [`tools/egress-gateway`](../tools/egress-gateway/) | `substrate:sandbox` | Egress-allowlist forward proxy (proxy.py plugin); host-level egress chokepoint — defence-in-depth for RFC-AI-0003 §4.4 | +| [`tools/forwarder-relay`](../tools/forwarder-relay/) | `contract:report-relay` | Adapter contract for inbound-relay backends (ASF Security relay, huntr.com, HackerOne triagers). Pure interface spec; adapters declare detection + credit-extraction + reporter-addressing rules. | +| [`tools/github`](../tools/github/) | `contract:tracker` | GitHub REST / GraphQL substrate (called by every lifecycle phase — pure substrate, no single phase) | +| [`tools/github-body-field`](../tools/github-body-field/) | `contract:tracker` | Read or rewrite one `### Field` section of a GitHub issue body without bringing the body into agent context — substrate helper for the security-sync skills | +| [`tools/github-rollup`](../tools/github-rollup/) | `contract:tracker` | Append to (or create) the status-rollup comment on a GitHub issue without bringing the rollup body into agent context — substrate helper for every status-update-emitting skill | +| [`tools/gmail`](../tools/gmail/) | `contract:mail-draft` | Gmail API substrate | +| [`tools/jira`](../tools/jira/) | `contract:tracker` | JIRA REST substrate (read-only today; write subcommands tracked in [#301](https://github.com/apache/magpie/issues/301)) | +| [`tools/mail-archive`](../tools/mail-archive/) | `contract:mail-archive` | Adapter contract for public mail-archive backends (PonyMail, Hyperkitty, Discourse, Google Groups, GitHub Discussions). Pure interface spec. | +| [`tools/mail-source`](../tools/mail-source/) | `contract:mail-source` | Mail-source backend abstraction (mbox / IMAP / Mailman 3); the abstraction is setup, every concrete read is part of the intake pipeline | +| [`tools/ponymail`](../tools/ponymail/) | `contract:mail-archive` | PonyMail archive substrate; same dual role as `mail-source` — substrate plus an intake-pipeline component | +| [`tools/scan-format`](../tools/scan-format/) | `contract:scan-format` | Adapter contract for security-scanner report formats (ASVS reference); reads a scan's finding index + per-finding evidence for the `security-issue-import-from-scan` pipeline. | +| [`tools/permission-audit`](../tools/permission-audit/) | `substrate:sandbox` | Audit + atomically edit Claude Code `permissions.allow[]` entries; backs `/magpie-setup verify --apply-permission-audit` (check 8d) | +| [`tools/pr-management-stats`](../tools/pr-management-stats/) | `substrate:analytics` | PR-backlog analytics engine | +| [`tools/preflight-audit`](../tools/preflight-audit/) | `substrate:analytics` | Dry-run the bulk-mode pre-flight classifier; measure skip-rate before / after any rule edit in the security-issue-sync skill | +| [`tools/privacy-llm`](../tools/privacy-llm/) | `substrate:privacy` | Privacy-LLM PII-scrubbing gate | +| [`tools/probe-templates`](../tools/probe-templates/) | `substrate:sandbox` | Sandbox-doctor probe templates | +| [`tools/sandbox-lint`](../tools/sandbox-lint/) | `substrate:sandbox` | Sandbox settings linter | +| [`tools/security-tracker-stats-dashboard`](../tools/security-tracker-stats-dashboard/) | `substrate:analytics` | Security-tracker analytics engine | +| [`tools/spec-loop`](../tools/spec-loop/) | `substrate:framework-dev` | Spec-driven build loop runner (Ralph-style) for framework development | +| [`tools/skill-evals`](../tools/skill-evals/) | `substrate:framework-dev` | Eval harness for skills; the harness is setup infrastructure, the run output is governance evidence | +| [`tools/skill-and-tool-validator`](../tools/skill-and-tool-validator/) | `substrate:framework-dev` | Skill-frontmatter and convention validator | +| [`tools/spec-status-index`](../tools/spec-status-index/) | `substrate:framework-dev` | Index of spec / RFC implementation status — substrate that also doubles as a governance/stats view | +| [`tools/spec-validator`](../tools/spec-validator/) | `substrate:framework-dev` | Spec-frontmatter and body-section validator — counterpart to `skill-and-tool-validator` for `tools/spec-loop/specs/` | +| [`tools/vcs`](../tools/vcs/) | `contract:source-control` | Backend-dispatching implementation of the source-control (VCS) capability ([`tools/github/source-control.md`](../tools/github/source-control.md)); complete Git backend plus detected extension points for non-Git VCS bridges (#601 Hg, #602 SVN) | + +A tool's capability is the **interface it provides**, not which skills +happen to consume it (RFC-AI-0005). `tools/github` provides the +`contract:tracker` interface; `tools/cve-tool-vulnogram` provides +`contract:cve-authority`; `tools/privacy-llm` is `substrate:privacy`. +Use a `contract:` value when the tool implements a capability +contract under `tools//`, and a `substrate:` value for +framework substrate. A tool may carry more than one (rare). --- @@ -258,18 +274,17 @@ capability: ### A GitHub issue -Apply at least one `area:*` AND one `capability:*` label. If the issue -genuinely spans capabilities, apply both — for example, -[#337](https://github.com/apache/magpie/issues/337) carries -both `capability:reconciliation` and `capability:setup` because it -covers a new substrate tool *and* a new sync-flow integration. +Apply at least one `area:*` AND one capability label — a skill +capability (`capability:*`) for skill work, a tool capability +(`contract:*` / `substrate:*`) for tool work. If the issue genuinely +spans capabilities, apply all that apply. ### A pull request -Same: `area:*` AND `capability:*`. Match the capability the change is -*implementing*, not the file paths it happens to touch. A PR that -adjusts the validator config to support a new triage rule is -`capability:triage` (the change's purpose), not `capability:setup` +Same: `area:*` AND the matching capability. Match the capability the +change is *implementing*, not the file paths it happens to touch. A PR +that adjusts the validator config to support a new triage rule is +`capability:triage` (the change's purpose), not `substrate:framework-dev` (the file it edited). ### A new tool under `tools/` @@ -278,12 +293,13 @@ Declare the tool's capability in the **first paragraph of its README** using the line: ```markdown -**Capability:** capability:NAME +**Capability:** contract:NAME ``` -If the tool serves more than one capability, list both. Substrate -bridges (`tools/github`, `tools/gmail`, …) default to -`capability:setup` unless they encode a specific lifecycle capability. +…or `substrate:NAME` for framework substrate. If the tool serves more +than one, list them (`contract:a + substrate:b`). Pick the +`contract:` that matches the capability contract the tool +implements, or the `substrate:` kind that fits. ### A new skill under `.claude/skills/` diff --git a/docs/rfcs/RFC-AI-0005.md b/docs/rfcs/RFC-AI-0005.md new file mode 100644 index 00000000..b5a4e8d9 --- /dev/null +++ b/docs/rfcs/RFC-AI-0005.md @@ -0,0 +1,237 @@ + + +**Table of Contents** *generated with [DocToc](https://github.com/thlorenz/doctoc)* + +- [RFC-AI-0005: Framework taxonomy](#rfc-ai-0005-framework-taxonomy) + - [Abstract](#abstract) + - [Status of this document](#status-of-this-document) + - [Motivation](#motivation) + - [The taxonomies at a glance](#the-taxonomies-at-a-glance) + - [Capability — the two-axis model](#capability--the-two-axis-model) + - [Axis 1 — Skill capability (what a workflow *does*)](#axis-1--skill-capability-what-a-workflow-does) + - [Axis 2 — Tool capability (what a backend *provides*)](#axis-2--tool-capability-what-a-backend-provides) + - [How the two axes link](#how-the-two-axes-link) + - [The other taxonomies (unchanged, documented here for completeness)](#the-other-taxonomies-unchanged-documented-here-for-completeness) + - [`area:*` — subject](#area--subject) + - [`kind:*` — change type](#kind--change-type) + - [`mode:*` — agentic mode](#mode--agentic-mode) + - [`organization:` — organization membership / inheritance](#organization--organization-membership--inheritance) + - [Standalone labels](#standalone-labels) + - [Migration](#migration) + - [Out of scope](#out-of-scope) + - [References](#references) + + + + + +# RFC-AI-0005: Framework taxonomy + +## Abstract + +Magpie classifies its issues, skills, and tools with several label +dimensions — `area:`, `capability:`, `kind:`, `mode:`, and the newer +`organization:` scope. This RFC documents the **whole** taxonomy in one +place and fixes the one dimension that has drifted: **`capability:`**. + +Today a single capability vocabulary is stamped on two very different +entities — *skills* (where it names a workflow-lifecycle phase) and +*tools* (where it should name a technical interface). The mismatch +collapses ~75% of tools into a meaningless `capability:setup` bucket and +leaves four capabilities with no tool ever using them. This RFC splits +`capability:` into two orthogonal axes — **skill capability** (what a +workflow does) and **tool capability** (what a backend provides, = the +contract it implements) — and specifies the migration. + +## Status of this document + +**Proposed.** Implemented in the same change set that lands this RFC +(`skill-and-tool-validator`, every `tools/*/README.md`, the +`capability:setup` skills, `docs/labels-and-capabilities.md`, and the +`AGENTS.md` / `tools/AGENTS.md` labeling sections). Supersedes the single +`capability:` vocabulary described in earlier revisions of +`docs/labels-and-capabilities.md`. + +## Motivation + +`docs/labels-and-capabilities.md` defines nine capabilities — `triage`, +`review`, `fix`, `intake`, `reconciliation`, `resolve`, `reassess`, +`stats`, `setup` — and applies them to both skills and tools. Measured +against the live tree: + +- **`capability:setup` is a grab-bag — 24 of 33 tools.** It is stamped + on entities with nothing in common: API substrate (`github`, `jira`, + `gmail`, `vcs`), adapter *contracts* (`cve-tool`, `mail-archive`, + `forwarder-relay`, `scan-format`, `mail-source`), a command guard + (`agent-guard`), sandbox/isolation (`agent-isolation`, + `egress-gateway`, `probe-templates`, `sandbox-lint`, + `permission-audit`), PII redaction (`privacy-llm`), the framework + dev-loop (`dev`, `spec-loop`, `spec-validator`, + `skill-and-tool-validator`, `spec-status-index`, `skill-evals`), and + context-safe helpers (`github-body-field`, `github-rollup`). +- **Four capabilities have zero tools** — `triage`, `review`, `fix`, + `reassess`. They only ever describe a skill. +- **The surviving tool buckets are too coarse** — `stats` lumps a data + *source* (`apache-projects`) with HTML *renderers* + (`dashboard-generator`); `resolve` on a tool means only "CVE + authority". + +**Root cause.** The nine capabilities are *workflow-lifecycle phases* — +the right model for **skills** (orthogonal to `area:`). A lifecycle phase +is the wrong model for a **tool**, which has a *technical interface*, not +a phase. The framework already has the right concept for a tool's +capability: the **capability contract** (`tools/cve-tool/`, +`mail-archive/`, `forwarder-relay/`, `scan-format/`, plus +`source-control` and the tracker interface). A tool's real capability is +*the contract it implements* — but that is not what `**Capability:**` +records. + +## The taxonomies at a glance + +| Dimension | Applies to | Answers | Source of truth | +|---|---|---|---| +| `area:*` | issues, PRs | *which part of the framework?* | this RFC + `docs/labels-and-capabilities.md` | +| **skill capability** | skills, issues, PRs | *what lifecycle phase does the workflow perform?* | this RFC + `docs/labels-and-capabilities.md` | +| **tool capability** | tools / adapters | *what interface/contract does the backend provide?* | this RFC + the adapter [registry](../adapters/registry.md) | +| `kind:*` | issues, PRs | *what type of change?* | `docs/labels-and-capabilities.md` | +| `mode:*` | skills, issues | *which agentic mode / risk tier?* | `docs/modes.md` | +| `organization:` | skills, families, tools, projects | *which organization does this belong to / inherit from?* | `organizations/README.md` | + +The change in this RFC is splitting the third row out of the second. + +## Capability — the two-axis model + +### Axis 1 — Skill capability (what a workflow *does*) + +The lifecycle phase a skill performs, orthogonal to `area:`: + +`triage · review · fix · intake · reconciliation · resolve · reassess · +stats · platform · authoring` + +This is the previous list with `setup` **split** into: + +- **`platform`** — framework/agent substrate skills: install, verify, + update, doctor, override-upstream, status, shared-config-sync, the + `setup` bootstrap. +- **`authoring`** — skills that author or maintain *other* skills: + `write-skill`, `optimize-skill`. + +Splitting `setup` removes the last overloaded skill bucket: "stand up the +agent" (`platform`) and "write a workflow" (`authoring`) are different +jobs and were both `setup`. + +### Axis 2 — Tool capability (what a backend *provides*) + +A tool's capability is **the contract / interface it implements**, drawn +from a controlled vocabulary that mirrors the capability contracts plus a +small set of substrate kinds: + +| Tool capability | Kind | What it provides | +|---|---|---| +| `tracker` | contract | issue / PR / board / label backend | +| `source-control` | contract | branch / commit / diff / push (VCS) | +| `mail-archive` | contract | public mailing-list / forum archive reads | +| `mail-source` | contract | inbound-mail ingestion (mbox / IMAP / …) | +| `mail-draft` | contract | outbound mail composition (draft, never send) | +| `cve-authority` | contract | CVE allocation / record management / publication | +| `report-relay` | contract | inbound security-report relay detection | +| `scan-format` | contract | security-scanner report parsing | +| `project-metadata` | contract | governance rosters / people / releases | +| `analytics` | substrate | read-only metrics / dashboards / renderers | +| `sandbox` | substrate | agent isolation, egress control, settings audit | +| `action-guard` | substrate | deterministic pre-tool-use command guards | +| `privacy` | substrate | PII redaction / approved-LLM gating | +| `framework-dev` | substrate | build / validate / eval the framework itself | + +The *contract* rows are exactly the seams an [adapter](../vendor-neutrality.md#tool-adapters) +plugs into; the tool capability of an adapter is the contract it +fulfils. The *substrate* rows replace the old `capability:setup` +catch-all with meaningful kinds. + +### How the two axes link + +A **skill consumes** tool capabilities (the contracts it needs); a +**tool provides** one. This is the edge the single vocabulary could not +express: + +> `security-issue-import` (skill capability `intake`) consumes the +> `mail-archive` + `mail-source` tool capabilities; `ponymail` provides +> `mail-archive`. + +`docs/labels-and-capabilities.md` therefore carries **two** maps: a +*skill → skill-capability* map, and a *contract → adapters* map (the same +table as the adapter [registry](../adapters/registry.md)). + +## The other taxonomies (unchanged, documented here for completeness) + +### `area:*` — subject + +`area:pr-management`, `area:security`, `area:setup`, `area:issue`, +`area:tools`, `area:ci`, `area:docs`. Orthogonal to capability: a +triage-rule change in PR management and one in security are both skill +capability `triage`, in different `area:`s. + +### `kind:*` — change type + +`kind:dx` (maintainer dev-loop / CLI UX), `kind:policy` (rule changes), +`kind:perf` (token / latency / API-call budget), `kind:adopter-config` +(per-adopter knob). + +### `mode:*` — agentic mode + +The five modes from `docs/modes.md`: **Agentic Triage**, **Agentic +Mentoring**, **Agentic Drafting**, **Agentic Pairing**, **Agentic +Autonomous** (off by default), plus `mode:cross-cutting` and +`mode:platform` for substrate that is not a mode. `mode:` is the *risk +tier* of an action; skill capability is the *phase*. A skill carries +both (e.g. `pr-management-code-review`: capability `review`, mode +Pairing/Drafting). + +### `organization:` — organization membership / inheritance + +Per RFC context in `organizations/README.md`: a skill, skill family, +tool, or project may declare the organization it belongs to / inherits +from. Absent = organization-agnostic. Distinct from the dimensions above +— it scopes *which governing body's defaults apply*, not what the entity +does. + +### Standalone labels + +`marketing`, `dependencies`, `python:uv`, and the default GitHub labels +(`bug`, `enhancement`, `documentation`, `good first issue`, …). + +## Migration + +1. **Validator** (`skill-and-tool-validator`): replace the single + `ALLOWED_CAPABILITIES` with `SKILL_CAPABILITIES` (Axis 1) and + `TOOL_CAPABILITIES` (Axis 2); skill-frontmatter validation checks + Axis 1, tool-README validation checks Axis 2; the capability-sync + check splits into the two maps. Tests updated. +2. **Tools**: rewrite every `tools/*/README.md` `**Capability:**` line to + its Axis-2 value (see the registry / `labels-and-capabilities.md` map). +3. **Skills**: re-label the `capability:setup` skills to `platform` or + `authoring`. +4. **Docs**: `docs/labels-and-capabilities.md` becomes two maps; + `AGENTS.md` labeling + `tools/AGENTS.md` updated. +5. **Evals**: any eval fixture asserting a capability value updated. +6. **GitHub labels** (follow-up, optional): the `capability:*` issue + labels are renamed/added to match; existing issues relabelled. + +The change is breaking for the `**Capability:**` declaration format but +mechanical once the vocabulary is fixed; the skill *lifecycle* phases are +unchanged except for the `setup` split. + +## Out of scope + +- Reworking `area:`, `kind:`, or `mode:` — documented here, unchanged. +- A per-capability *floor* spec for tools (which contract version an + adapter targets) — a possible future RFC. + +## References + +- `docs/labels-and-capabilities.md` — the implementing taxonomy doc. +- `docs/vendor-neutrality.md` — tool adapters + capability contracts. +- `docs/adapters/registry.md` — the contract → adapters map. +- `docs/modes.md` — the agentic modes. +- `organizations/README.md` — the organization dimension. diff --git a/skills/optimize-skill/SKILL.md b/skills/optimize-skill/SKILL.md index 16f9529f..f8bb2010 100644 --- a/skills/optimize-skill/SKILL.md +++ b/skills/optimize-skill/SKILL.md @@ -23,7 +23,7 @@ when_to_use: | that is `write-skill`. Skip when the request is a behavior change dressed up as an optimization; route those through normal skill editing + review. -capability: capability:setup +capability: capability:authoring license: Apache-2.0 --- diff --git a/skills/setup-isolated-setup-doctor/SKILL.md b/skills/setup-isolated-setup-doctor/SKILL.md index 830084c5..b55771c2 100644 --- a/skills/setup-isolated-setup-doctor/SKILL.md +++ b/skills/setup-isolated-setup-doctor/SKILL.md @@ -18,7 +18,7 @@ when_to_use: | Claude Code upgrade — the sandbox profile evolves and a previously-working call may have moved into deny. capability: - - capability:setup + - capability:platform - capability:reassess license: Apache-2.0 --- diff --git a/skills/setup-isolated-setup-install/SKILL.md b/skills/setup-isolated-setup-install/SKILL.md index 7a1b585c..bfb5236a 100644 --- a/skills/setup-isolated-setup-install/SKILL.md +++ b/skills/setup-isolated-setup-install/SKILL.md @@ -17,7 +17,7 @@ when_to_use: | already in place — use `setup-isolated-setup-verify` (to confirm completeness) or `setup-isolated-setup-update` (to refresh against the framework's latest) instead. -capability: capability:setup +capability: capability:platform license: Apache-2.0 --- diff --git a/skills/setup-isolated-setup-update/SKILL.md b/skills/setup-isolated-setup-update/SKILL.md index d64dc4f8..6041f66b 100644 --- a/skills/setup-isolated-setup-update/SKILL.md +++ b/skills/setup-isolated-setup-update/SKILL.md @@ -14,7 +14,7 @@ when_to_use: | blocked Bash call now appears to succeed. Recommended cadence per the doc: once per Claude Code upgrade or once a month, whichever comes first. Cheap to re-run; never destructive. -capability: capability:setup +capability: capability:platform license: Apache-2.0 --- diff --git a/skills/setup-isolated-setup-verify/SKILL.md b/skills/setup-isolated-setup-verify/SKILL.md index 125876ee..3f806088 100644 --- a/skills/setup-isolated-setup-verify/SKILL.md +++ b/skills/setup-isolated-setup-verify/SKILL.md @@ -17,7 +17,7 @@ when_to_use: | time a previously-blocked Bash call appears to have succeeded (the "did a denial silently turn into an allow?" canary). Cheap to re-run; never destructive. -capability: capability:setup +capability: capability:platform license: Apache-2.0 --- diff --git a/skills/setup-override-upstream/SKILL.md b/skills/setup-override-upstream/SKILL.md index f3964dad..0a2ee30d 100644 --- a/skills/setup-override-upstream/SKILL.md +++ b/skills/setup-override-upstream/SKILL.md @@ -15,7 +15,7 @@ when_to_use: | override locally for a while and deciding the change is worth contributing back. argument-hint: "[skill-name]" -capability: capability:setup +capability: capability:platform license: Apache-2.0 --- diff --git a/skills/setup-shared-config-sync/SKILL.md b/skills/setup-shared-config-sync/SKILL.md index 3d016ae1..821638d1 100644 --- a/skills/setup-shared-config-sync/SKILL.md +++ b/skills/setup-shared-config-sync/SKILL.md @@ -20,7 +20,7 @@ when_to_use: | other machines. capability: - capability:intake - - capability:setup + - capability:platform license: Apache-2.0 --- diff --git a/skills/setup-status/SKILL.md b/skills/setup-status/SKILL.md index aa773f9a..fb9d6f85 100644 --- a/skills/setup-status/SKILL.md +++ b/skills/setup-status/SKILL.md @@ -18,7 +18,7 @@ when_to_use: | state", "which families are installed", "add the github target". capability: - capability:stats - - capability:setup + - capability:platform license: Apache-2.0 --- diff --git a/skills/setup/SKILL.md b/skills/setup/SKILL.md index 7a58ed6f..4458c03a 100644 --- a/skills/setup/SKILL.md +++ b/skills/setup/SKILL.md @@ -20,7 +20,7 @@ when_to_use: | maintenance: "upgrade magpie", "verify magpie setup", "check magpie drift", "the snapshot is stale". argument-hint: "[adopt|upgrade|worktree-init|verify|override skill-name|unadopt]" -capability: capability:setup +capability: capability:platform license: Apache-2.0 --- diff --git a/skills/write-skill/SKILL.md b/skills/write-skill/SKILL.md index 7484b401..5263f8ad 100644 --- a/skills/write-skill/SKILL.md +++ b/skills/write-skill/SKILL.md @@ -14,7 +14,7 @@ when_to_use: | variations thereof. Also when refactoring or expanding an existing skill that should pick up the framework's current conventions (e.g. the prompt-injection-defence patterns). -capability: capability:setup +capability: capability:authoring license: Apache-2.0 --- @@ -113,7 +113,8 @@ skill bundles: │ │ │ `capability:triage`, `capability:review`, `capability:fix`, │ │ │ `capability:intake`, `capability:reconciliation`, │ │ │ `capability:resolve`, `capability:reassess`, -│ │ │ `capability:stats`, `capability:setup` — see +│ │ │ `capability:stats`, `capability:platform`, +│ │ │ `capability:authoring` — see │ │ │ [`docs/labels-and-capabilities.md`](../../docs/labels-and-capabilities.md)) │ │ └── license: Apache-2.0 (required, exact string) │ ├── SPDX header comment + placeholder-convention comment @@ -373,7 +374,7 @@ for the override → upstream loop. lifecycle phases (e.g. `security-issue-fix` does `capability:fix` + `capability:resolve`, `setup-isolated-setup-doctor` does - `capability:setup` + `capability:reassess`), use the YAML list + `capability:authoring` + `capability:reassess`), use the YAML list form and list **all** that apply — do not collapse to one to be neat. If the skill doesn't fit any of the nine buckets at all, treat that as a design signal worth pausing for — either the diff --git a/tools/AGENTS.md b/tools/AGENTS.md index 8455c01a..fd9b9c6d 100644 --- a/tools/AGENTS.md +++ b/tools/AGENTS.md @@ -35,12 +35,15 @@ declares, up front: 1. **Its capability** — a line of the exact form ```markdown - **Capability:** capability:NAME + **Capability:** contract:NAME ``` - (multi-value: `capability:NAME + capability:NAME`), with NAME drawn - from the taxonomy in `docs/labels-and-capabilities.md`. Substrate - bridges default to `capability:setup`. + …or `substrate:NAME` (multi-value: `contract:a + substrate:b`). A + **tool capability** is the interface the tool provides (RFC-AI-0005): + `contract:` when it implements a capability contract under + `tools//` (e.g. `contract:tracker`), or `substrate:` + for framework substrate (e.g. `substrate:sandbox`). Draw the value + from the taxonomy in `docs/labels-and-capabilities.md`. 2. **Its prerequisites** — a `## Prerequisites` section stating what the tool needs *before it can run*, so an adopter never discovers a diff --git a/tools/agent-guard/README.md b/tools/agent-guard/README.md index 5c94200a..0d5e1244 100644 --- a/tools/agent-guard/README.md +++ b/tools/agent-guard/README.md @@ -17,7 +17,7 @@ # agent-guard -**Capability:** capability:setup +**Capability:** substrate:action-guard A deterministic Claude Code [`PreToolUse`](https://code.claude.com/docs/en/hooks) guard dispatcher. It inspects every `Bash` command **before it runs** and diff --git a/tools/agent-isolation/README.md b/tools/agent-isolation/README.md index 2d32108d..2d4ee2d5 100644 --- a/tools/agent-isolation/README.md +++ b/tools/agent-isolation/README.md @@ -15,7 +15,7 @@ # `tools/agent-isolation/` — secure agent setup helpers -**Capability:** capability:setup +**Capability:** substrate:sandbox This directory ships the moving pieces the framework's [`docs/setup/secure-agent-setup.md`](../../docs/setup/secure-agent-setup.md) document diff --git a/tools/apache-projects/README.md b/tools/apache-projects/README.md index 7d1be226..4f70f2a8 100644 --- a/tools/apache-projects/README.md +++ b/tools/apache-projects/README.md @@ -12,7 +12,7 @@ # `tools/apache-projects/` -**Capability:** capability:stats + capability:intake +**Capability:** contract:project-metadata **Organization:** ASF diff --git a/tools/cve-org/README.md b/tools/cve-org/README.md index eaaefd0a..2306a768 100644 --- a/tools/cve-org/README.md +++ b/tools/cve-org/README.md @@ -12,7 +12,7 @@ # `tools/cve-org/` -**Capability:** capability:resolve + capability:intake +**Capability:** contract:cve-authority CVE.org publication client. Submits CVE records via the CVE.org REST API; consumed by `security-cve-allocate` once a CVE has been allocated via the ASF Vulnogram path. See [`tool.md`](tool.md) for the protocol detail and `cve.org` field mapping. diff --git a/tools/cve-tool-vulnogram/README.md b/tools/cve-tool-vulnogram/README.md index 2cafaed1..417e7a7a 100644 --- a/tools/cve-tool-vulnogram/README.md +++ b/tools/cve-tool-vulnogram/README.md @@ -12,7 +12,7 @@ # `tools/cve-tool-vulnogram/` -**Capability:** capability:resolve +**Capability:** contract:cve-authority **Organization:** ASF diff --git a/tools/cve-tool/README.md b/tools/cve-tool/README.md index be94687d..d72ae50f 100644 --- a/tools/cve-tool/README.md +++ b/tools/cve-tool/README.md @@ -24,7 +24,7 @@ # `tools/cve-tool/` -**Capability:** capability:setup +**Capability:** contract:cve-authority ## Prerequisites diff --git a/tools/dashboard-generator/README.md b/tools/dashboard-generator/README.md index e1c61de7..918abc27 100644 --- a/tools/dashboard-generator/README.md +++ b/tools/dashboard-generator/README.md @@ -18,7 +18,7 @@ # Dashboard generator -**Capability:** capability:stats +**Capability:** substrate:analytics Deterministic reference implementations of the dashboard that [`issue-reassess-stats`](../../skills/issue-reassess-stats/SKILL.md) diff --git a/tools/dev/README.md b/tools/dev/README.md index 90d0911f..74f081cf 100644 --- a/tools/dev/README.md +++ b/tools/dev/README.md @@ -12,7 +12,7 @@ # `tools/dev/` -**Capability:** capability:setup +**Capability:** substrate:framework-dev Framework dev-loop helpers (placeholder check, agent pre-commit hook). Invoked by prek and CI; not consumed by any skill directly. See the individual scripts in this directory for usage. diff --git a/tools/egress-gateway/README.md b/tools/egress-gateway/README.md index 7d360ead..114674b7 100644 --- a/tools/egress-gateway/README.md +++ b/tools/egress-gateway/README.md @@ -17,7 +17,7 @@ # egress-gateway -**Capability:** capability:setup +**Capability:** substrate:sandbox A local **host-allowlisting HTTP(S) forward proxy** for the Magpie framework. It is the egress-control chokepoint: framework diff --git a/tools/forwarder-relay/README.md b/tools/forwarder-relay/README.md index 638ba470..2c6e574c 100644 --- a/tools/forwarder-relay/README.md +++ b/tools/forwarder-relay/README.md @@ -28,7 +28,7 @@ # tools/forwarder-relay/ — adapter contract -**Capability:** capability:setup +**Capability:** contract:report-relay A forwarder-relay adapter is a pluggable seam that teaches the security skills how to recognise an inbound report that arrived diff --git a/tools/github-body-field/README.md b/tools/github-body-field/README.md index 84cff654..898f6f9a 100644 --- a/tools/github-body-field/README.md +++ b/tools/github-body-field/README.md @@ -19,7 +19,7 @@ # github-body-field -**Capability:** capability:setup +**Capability:** contract:tracker Read or rewrite a single `### Field` section of a GitHub issue body **without bringing the body into agent context**. diff --git a/tools/github-rollup/README.md b/tools/github-rollup/README.md index 61dd98c3..e417cf79 100644 --- a/tools/github-rollup/README.md +++ b/tools/github-rollup/README.md @@ -18,7 +18,7 @@ # github-rollup -**Capability:** capability:setup +**Capability:** contract:tracker Append to (or create) the status-rollup comment on a GitHub issue **without bringing the rollup body into agent context**. diff --git a/tools/github/README.md b/tools/github/README.md index c0c9fa3a..f60ef1ef 100644 --- a/tools/github/README.md +++ b/tools/github/README.md @@ -12,7 +12,7 @@ # `tools/github/` -**Capability:** capability:setup +**Capability:** contract:tracker GitHub REST + GraphQL substrate. Pure read/write wrapper used by every lifecycle phase (triage / intake / fix / resolve / stats). See [`tool.md`](tool.md) for the operation catalogue and the per-area files ([`issue-template.md`](issue-template.md), [`labels.md`](labels.md), [`operations.md`](operations.md), [`project-board.md`](project-board.md), [`status-rollup.md`](status-rollup.md)) for specifics. diff --git a/tools/gmail/README.md b/tools/gmail/README.md index 78df170b..d847ee65 100644 --- a/tools/gmail/README.md +++ b/tools/gmail/README.md @@ -12,7 +12,7 @@ # `tools/gmail/` -**Capability:** capability:setup +**Capability:** contract:mail-draft Gmail API substrate. Read + draft-only — never sends. Used by the security-issue-import / sync / invalidate flows for inbound report intake and outbound courtesy-reply drafting. See [`tool.md`](tool.md) for the operation catalogue and the per-area files for ASF relay routing, draft backends, threading, search queries. diff --git a/tools/jira/README.md b/tools/jira/README.md index 4f836204..1eebba88 100644 --- a/tools/jira/README.md +++ b/tools/jira/README.md @@ -30,7 +30,7 @@ # JIRA bridge -**Capability:** capability:setup +**Capability:** contract:tracker JIRA REST helpers for the `issue-*` skill family. Adopters with JIRA-based issue trackers wire this in as their diff --git a/tools/mail-archive/README.md b/tools/mail-archive/README.md index 65c878af..a1710ed5 100644 --- a/tools/mail-archive/README.md +++ b/tools/mail-archive/README.md @@ -22,7 +22,7 @@ # tools/mail-archive/ -**Capability:** capability:setup +**Capability:** contract:mail-archive This file defines the adapter contract for **public mail-archive backends** — the seam that lets adopting projects plug a non-ASF diff --git a/tools/mail-source/README.md b/tools/mail-source/README.md index 77acd3f7..3e77b89d 100644 --- a/tools/mail-source/README.md +++ b/tools/mail-source/README.md @@ -12,7 +12,7 @@ # `tools/mail-source/` -**Capability:** capability:setup + capability:intake +**Capability:** contract:mail-source Mail-source backend abstraction. Pluggable backends (mbox, IMAP, future Mailman 3 / Hyperkitty) that feed the security-issue-import intake pipeline a uniform thread/message view. See [`contract.md`](contract.md) for the backend interface. diff --git a/tools/permission-audit/README.md b/tools/permission-audit/README.md index 5ec3f3e8..ba9fce74 100644 --- a/tools/permission-audit/README.md +++ b/tools/permission-audit/README.md @@ -21,7 +21,7 @@ # permission-audit -**Capability:** capability:setup +**Capability:** substrate:sandbox Audit + atomically edit Claude Code's `permissions.allow[]` entries in `/.claude/settings.json` and `/.claude/settings.local.json`. diff --git a/tools/ponymail/README.md b/tools/ponymail/README.md index c3456a11..3a6e7541 100644 --- a/tools/ponymail/README.md +++ b/tools/ponymail/README.md @@ -12,7 +12,7 @@ # `tools/ponymail/` -**Capability:** capability:setup + capability:intake +**Capability:** contract:mail-archive **Organization:** ASF diff --git a/tools/pr-management-stats/README.md b/tools/pr-management-stats/README.md index c0ebe239..b77637ed 100644 --- a/tools/pr-management-stats/README.md +++ b/tools/pr-management-stats/README.md @@ -19,7 +19,7 @@ # pr-management-stats reference implementation -**Capability:** capability:stats +**Capability:** substrate:analytics Deterministic reference implementation of the data-fetch + classification contract that backs the diff --git a/tools/preflight-audit/README.md b/tools/preflight-audit/README.md index 385f2259..3f6ed5e4 100644 --- a/tools/preflight-audit/README.md +++ b/tools/preflight-audit/README.md @@ -19,7 +19,7 @@ # preflight-audit -**Capability:** capability:stats +**Capability:** substrate:analytics Dry-run the bulk-mode pre-flight classifier against a real or replayed tracker. Use to **measure skip-rate before / after any diff --git a/tools/privacy-llm/README.md b/tools/privacy-llm/README.md index dfbd072a..69eb7f2e 100644 --- a/tools/privacy-llm/README.md +++ b/tools/privacy-llm/README.md @@ -12,7 +12,7 @@ # `tools/privacy-llm/` -**Capability:** capability:setup +**Capability:** substrate:privacy Privacy-LLM PII-scrubbing gate. Standalone redactor / checker pair that screens content for PII before it reaches an external LLM. See [`tool.md`](tool.md) and [`wiring.md`](wiring.md) for integration details, [`models.md`](models.md) for the model catalogue, and [`pii.md`](pii.md) for the PII taxonomy. diff --git a/tools/probe-templates/README.md b/tools/probe-templates/README.md index 07df41c7..e760ba80 100644 --- a/tools/probe-templates/README.md +++ b/tools/probe-templates/README.md @@ -16,7 +16,7 @@ # Probe templates -**Capability:** capability:setup +**Capability:** substrate:sandbox Runnable cross-family probe scripts that the [`issue-reproducer`](../../skills/issue-reproducer/SKILL.md) diff --git a/tools/sandbox-lint/README.md b/tools/sandbox-lint/README.md index ef29ac2c..ed9e8fd6 100644 --- a/tools/sandbox-lint/README.md +++ b/tools/sandbox-lint/README.md @@ -17,7 +17,7 @@ # `sandbox-lint` -**Capability:** capability:setup +**Capability:** substrate:sandbox Lints `.claude/settings.json` against the shipped baseline at `tools/sandbox-lint/expected.json`, and against the security diff --git a/tools/scan-format/README.md b/tools/scan-format/README.md index 41ffd44d..632e0492 100644 --- a/tools/scan-format/README.md +++ b/tools/scan-format/README.md @@ -21,7 +21,7 @@ # tools/scan-format/ — adapter contract -**Capability:** capability:intake +**Capability:** contract:scan-format A **scan-format adapter** teaches [`security-issue-import-from-scan`](../../skills/security-issue-import-from-scan/SKILL.md) diff --git a/tools/security-tracker-stats-dashboard/README.md b/tools/security-tracker-stats-dashboard/README.md index 07d43cb9..6e68822b 100644 --- a/tools/security-tracker-stats-dashboard/README.md +++ b/tools/security-tracker-stats-dashboard/README.md @@ -22,7 +22,7 @@ # security-tracker-stats-dashboard -**Capability:** capability:stats +**Capability:** substrate:analytics Generate a self-contained HTML dashboard of `` repository statistics — issue-lifecycle bands (untriaged / triaged / PR-merged / diff --git a/tools/skill-and-tool-validator/README.md b/tools/skill-and-tool-validator/README.md index d0a989aa..71ba616f 100644 --- a/tools/skill-and-tool-validator/README.md +++ b/tools/skill-and-tool-validator/README.md @@ -17,7 +17,7 @@ # skill-and-tool-validator -**Capability:** capability:setup +**Capability:** substrate:framework-dev Validate framework skill definitions — YAML frontmatter, internal link integrity, and placeholder conventions. diff --git a/tools/skill-and-tool-validator/src/skill_and_tool_validator/__init__.py b/tools/skill-and-tool-validator/src/skill_and_tool_validator/__init__.py index c976d4d9..58233d46 100644 --- a/tools/skill-and-tool-validator/src/skill_and_tool_validator/__init__.py +++ b/tools/skill-and-tool-validator/src/skill_and_tool_validator/__init__.py @@ -94,8 +94,9 @@ TOOL_CAPABILITY_CATEGORY = "tool-capability" TOOL_PREREQUISITES_CATEGORY = "tool-prerequisites" -# Matches `**Capability:** capability:NAME` (and multi-value -# `capability:NAME + capability:NAME + …`) on a single line. +# Matches the `**Capability:** ` line (tool capability = +# `contract:NAME` / `substrate:NAME`, multi-value `a + b`) regardless of +# the token value; the value is validated against TOOL_CAPABILITIES. TOOL_CAPABILITY_RE = re.compile(r"^\*\*Capability:\*\*[ \t]+(.+)$", re.MULTILINE) # Matches a level-2 `## Prerequisites` heading. Every tool README must carry @@ -122,8 +123,9 @@ EVAL_COVERAGE_CATEGORY = "eval-coverage" _SKILL_TABLE_HEADER = "## Capability to skill map" _TOOL_TABLE_HEADER = "## Capability to tool map" -# Tokens like `capability:setup`. Optional backticks around the token. -_CAPABILITY_TOKEN_RE = re.compile(r"`?(capability:[a-z]+)`?") +# Tokens like `capability:triage`, `contract:source-control`, +# `substrate:sandbox`. Optional backticks. Hyphens allowed for multi-word names. +_CAPABILITY_TOKEN_RE = re.compile(r"`?((?:capability|contract|substrate):[a-z-]+)`?") # Italic-parenthetical annotation in the docs tables: `*( … )*` — used for # future-state notes (e.g. "*(+ capability:reconciliation once #337 lands)*"). # Stripped before extracting authoritative capability tokens. The terminator @@ -136,9 +138,12 @@ OPTIONAL_FRONTMATTER_KEYS = {"when_to_use", "mode", "organization"} ALLOWED_LICENSES = {"Apache-2.0"} -# Canonical capability taxonomy — docs/labels-and-capabilities.md is authoritative. -# Skills may declare a single capability (string form) or several (YAML list form). -ALLOWED_CAPABILITIES = { +# Canonical capability taxonomy — two orthogonal axes per RFC-AI-0005; +# docs/labels-and-capabilities.md is authoritative. +# +# Axis 1 — SKILL capability: the workflow-lifecycle phase a skill performs. +# Skills may declare a single capability (string form) or several (YAML list). +SKILL_CAPABILITIES = { "capability:triage", "capability:review", "capability:fix", @@ -147,7 +152,29 @@ "capability:resolve", "capability:reassess", "capability:stats", - "capability:setup", + "capability:platform", + "capability:authoring", +} + +# Axis 2 — TOOL capability: the interface a tool (adapter) provides, in two +# kinds distinguished by prefix (RFC-AI-0005): +# contract: — implements a capability contract under tools// +# substrate: — framework substrate (replaces the old capability:setup) +TOOL_CAPABILITIES = { + "contract:tracker", + "contract:source-control", + "contract:mail-archive", + "contract:mail-source", + "contract:mail-draft", + "contract:cve-authority", + "contract:report-relay", + "contract:scan-format", + "contract:project-metadata", + "substrate:analytics", + "substrate:sandbox", + "substrate:action-guard", + "substrate:privacy", + "substrate:framework-dev", } @@ -594,7 +621,7 @@ def validate_frontmatter(path: Path, text: str, root: Path | None = None) -> Ite if fm.get("capability"): # The frontmatter parser stores both forms as a single string: # single — `capability: capability:triage` → "capability:triage" - # list — `capability:\n - capability:intake\n …` → "- capability:intake\n- capability:setup" + # list — `capability:\n - capability:intake\n …` → "- capability:intake\n- capability:platform" # Split on lines, strip `- ` prefix when present. entries: list[str] = [] for raw_line in fm["capability"].splitlines(): @@ -608,12 +635,12 @@ def validate_frontmatter(path: Path, text: str, root: Path | None = None) -> Ite if not entries: yield Violation(path, 1, "frontmatter key 'capability' is empty") for entry in entries: - if entry not in ALLOWED_CAPABILITIES: + if entry not in SKILL_CAPABILITIES: yield Violation( path, 1, - f"frontmatter capability '{entry}' not in {sorted(ALLOWED_CAPABILITIES)} " - f"(see docs/labels-and-capabilities.md)", + f"frontmatter capability '{entry}' not in {sorted(SKILL_CAPABILITIES)} " + f"(skills use Axis-1 capability:* values; see docs/labels-and-capabilities.md)", ) desc_len = len(fm.get("description", "")) @@ -1349,9 +1376,10 @@ def validate_tools(root: Path | None = None) -> Iterable[Violation]: """For each ``tools//`` directory, require: 1. A ``README.md`` to exist at the tool root. - 2. The README to contain a ``**Capability:** capability:NAME`` line, - with NAME drawn from ``ALLOWED_CAPABILITIES``. Multi-value form is - ``**Capability:** capability:NAME + capability:NAME``. + 2. The README to contain a ``**Capability:** `` line, with the + token drawn from ``TOOL_CAPABILITIES`` (a ``contract:`` or + ``substrate:`` value per RFC-AI-0005). Multi-value form is + ``**Capability:** contract:a + substrate:b``. 3. The README to contain a ``## Prerequisites`` section so the tool's runtime / CLI / credential / network requirements are stated up front. @@ -1408,8 +1436,8 @@ def validate_tools(root: Path | None = None) -> Iterable[Violation]: yield Violation( readme, 1, - f"tool '{tool_dir.name}' README missing '**Capability:** capability:NAME' " - f"declaration (see docs/labels-and-capabilities.md)", + f"tool '{tool_dir.name}' README missing '**Capability:** contract:NAME' " + f"(or substrate:NAME) declaration (see docs/labels-and-capabilities.md)", category=TOOL_CAPABILITY_CATEGORY, ) continue @@ -1427,12 +1455,13 @@ def validate_tools(root: Path | None = None) -> Iterable[Violation]: ) continue for entry in entries: - if entry not in ALLOWED_CAPABILITIES: + if entry not in TOOL_CAPABILITIES: yield Violation( readme, line_no, f"tool '{tool_dir.name}' capability '{entry}' not in " - f"{sorted(ALLOWED_CAPABILITIES)} (see docs/labels-and-capabilities.md)", + f"{sorted(TOOL_CAPABILITIES)} (tools use contract:* / substrate:* " + f"values; see docs/labels-and-capabilities.md)", category=TOOL_CAPABILITY_CATEGORY, ) diff --git a/tools/skill-and-tool-validator/tests/test_validator.py b/tools/skill-and-tool-validator/tests/test_validator.py index 68b530bf..9dc53f26 100644 --- a/tools/skill-and-tool-validator/tests/test_validator.py +++ b/tools/skill-and-tool-validator/tests/test_validator.py @@ -88,7 +88,7 @@ class TestParseFrontmatter: def test_valid_frontmatter(self) -> None: - text = "---\nname: foo\ndescription: bar\ncapability: capability:setup\nlicense: Apache-2.0\n---\n# heading\n" + text = "---\nname: foo\ndescription: bar\ncapability: capability:platform\nlicense: Apache-2.0\n---\n# heading\n" fm = parse_frontmatter(text) assert fm is not None assert fm["name"] == "foo" @@ -102,7 +102,7 @@ def test_folded_scalar(self) -> None: "description: |\n" " First line of description.\n" " Second line.\n" - "capability: capability:setup\nlicense: Apache-2.0\n" + "capability: capability:platform\nlicense: Apache-2.0\n" "---\n" ) fm = parse_frontmatter(text) @@ -125,7 +125,7 @@ def test_block_scalar_preserves_internal_blank_line(self) -> None: " Paragraph one.\n" "\n" " Paragraph two, which used to be dropped.\n" - "capability: capability:setup\nlicense: Apache-2.0\n" + "capability: capability:platform\nlicense: Apache-2.0\n" "---\n" ) fm = parse_frontmatter(text) @@ -149,13 +149,13 @@ def test_no_closing_delimiter(self) -> None: class TestValidateFrontmatter: def test_valid(self, tmp_path: Path) -> None: path = tmp_path / "SKILL.md" - text = "---\nname: foo\ndescription: bar\ncapability: capability:setup\nlicense: Apache-2.0\n---\n" + text = "---\nname: foo\ndescription: bar\ncapability: capability:platform\nlicense: Apache-2.0\n---\n" violations = list(validate_frontmatter(path, text)) assert violations == [] def test_missing_name(self, tmp_path: Path) -> None: path = tmp_path / "SKILL.md" - text = "---\ndescription: bar\ncapability: capability:setup\nlicense: Apache-2.0\n---\n" + text = "---\ndescription: bar\ncapability: capability:platform\nlicense: Apache-2.0\n---\n" violations = list(validate_frontmatter(path, text)) assert len(violations) == 1 assert "name" in violations[0].message @@ -171,7 +171,7 @@ def test_missing_multiple_keys(self, tmp_path: Path) -> None: def test_empty_value(self, tmp_path: Path) -> None: path = tmp_path / "SKILL.md" - text = "---\nname: \ndescription: bar\ncapability: capability:setup\nlicense: Apache-2.0\n---\n" + text = "---\nname: \ndescription: bar\ncapability: capability:platform\nlicense: Apache-2.0\n---\n" violations = list(validate_frontmatter(path, text)) assert any("name' is empty" in v.message for v in violations) @@ -184,20 +184,20 @@ def test_invalid_license(self, tmp_path: Path) -> None: def test_valid_mode(self, tmp_path: Path) -> None: path = tmp_path / "SKILL.md" for mode in ("Triage", "Mentoring", "Drafting", "Pairing"): - text = f"---\nname: foo\ndescription: bar\ncapability: capability:setup\nlicense: Apache-2.0\nmode: {mode}\n---\n" + text = f"---\nname: foo\ndescription: bar\ncapability: capability:platform\nlicense: Apache-2.0\nmode: {mode}\n---\n" violations = list(validate_frontmatter(path, text)) assert violations == [], f"mode '{mode}' should be valid" def test_invalid_mode(self, tmp_path: Path) -> None: path = tmp_path / "SKILL.md" - text = "---\nname: foo\ndescription: bar\ncapability: capability:setup\nlicense: Apache-2.0\nmode: Auto-merge\n---\n" + text = "---\nname: foo\ndescription: bar\ncapability: capability:platform\nlicense: Apache-2.0\nmode: Auto-merge\n---\n" violations = list(validate_frontmatter(path, text)) assert any("mode" in v.message and "'Auto-merge'" in v.message for v in violations) def test_mode_optional(self, tmp_path: Path) -> None: # Skills without a mode (e.g. setup-* infrastructure) must not fail. path = tmp_path / "SKILL.md" - text = "---\nname: foo\ndescription: bar\ncapability: capability:setup\nlicense: Apache-2.0\n---\n" + text = "---\nname: foo\ndescription: bar\ncapability: capability:platform\nlicense: Apache-2.0\n---\n" violations = list(validate_frontmatter(path, text)) assert violations == [] @@ -225,7 +225,7 @@ def test_metadata_under_limit(self, tmp_path: Path) -> None: path = tmp_path / "SKILL.md" desc = "a" * 800 wtu = "b" * 700 - text = f"---\nname: foo\ndescription: {desc}\nwhen_to_use: {wtu}\ncapability: capability:setup\nlicense: Apache-2.0\n---\n" + text = f"---\nname: foo\ndescription: {desc}\nwhen_to_use: {wtu}\ncapability: capability:platform\nlicense: Apache-2.0\n---\n" violations = list(validate_frontmatter(path, text)) assert violations == [] @@ -233,7 +233,7 @@ def test_metadata_over_limit(self, tmp_path: Path) -> None: path = tmp_path / "SKILL.md" desc = "a" * 1000 wtu = "b" * (MAX_METADATA_CHARS - 1000 + 1) - text = f"---\nname: foo\ndescription: {desc}\nwhen_to_use: {wtu}\ncapability: capability:setup\nlicense: Apache-2.0\n---\n" + text = f"---\nname: foo\ndescription: {desc}\nwhen_to_use: {wtu}\ncapability: capability:platform\nlicense: Apache-2.0\n---\n" violations = list(validate_frontmatter(path, text)) assert any("truncates" in v.message and str(MAX_METADATA_CHARS) in v.message for v in violations) @@ -246,7 +246,7 @@ def test_argument_hint_accepted(self, tmp_path: Path) -> None: "---\n" "name: foo\n" "description: bar\n" - "capability: capability:setup\nlicense: Apache-2.0\n" + "capability: capability:platform\nlicense: Apache-2.0\n" "argument-hint: [--quick|--standard|--deep] \n" "---\n" ) @@ -262,7 +262,7 @@ def test_argument_hint_pipe_notation_with_spaces_in_option(self, tmp_path: Path) "---\n" "name: setup\n" "description: bar\n" - "capability: capability:setup\nlicense: Apache-2.0\n" + "capability: capability:platform\nlicense: Apache-2.0\n" "argument-hint: [adopt|upgrade|worktree-init|verify|override skill-name|unadopt]\n" "---\n" ) @@ -285,7 +285,7 @@ def test_argument_hint_does_not_inflate_metadata_budget(self, tmp_path: Path) -> f"name: foo\n" f"description: {desc}\n" f"when_to_use: {wtu}\n" - f"capability: capability:setup\nlicense: Apache-2.0\n" + f"capability: capability:platform\nlicense: Apache-2.0\n" f"argument-hint: {hint}\n" f"---\n" ) @@ -293,7 +293,7 @@ def test_argument_hint_does_not_inflate_metadata_budget(self, tmp_path: Path) -> assert violations == [], "argument-hint must not count toward description+when_to_use budget" def test_metadata_block_scalar_indicator_not_counted(self) -> None: - text = f"---\nname: foo\ndescription: |\n {'a' * 100}\ncapability: capability:setup\nlicense: Apache-2.0\n---\n" + text = f"---\nname: foo\ndescription: |\n {'a' * 100}\ncapability: capability:platform\nlicense: Apache-2.0\n---\n" fm = parse_frontmatter(text) assert fm is not None assert not fm["description"].startswith("|") @@ -309,7 +309,7 @@ def test_capability_yaml_list(self, tmp_path: Path) -> None: path = tmp_path / "SKILL.md" text = ( "---\nname: foo\ndescription: bar\n" - "capability:\n - capability:intake\n - capability:setup\n" + "capability:\n - capability:intake\n - capability:platform\n" "license: Apache-2.0\n---\n" ) violations = list(validate_frontmatter(path, text)) @@ -331,7 +331,7 @@ def test_capability_list_with_one_invalid_value(self, tmp_path: Path) -> None: path = tmp_path / "SKILL.md" text = ( "---\nname: foo\ndescription: bar\n" - "capability:\n - capability:setup\n - capability:invented\n" + "capability:\n - capability:platform\n - capability:invented\n" "license: Apache-2.0\n---\n" ) violations = list(validate_frontmatter(path, text)) @@ -357,7 +357,7 @@ def _skill(self, root: Path, dir_name: str, name: str) -> Path: skill_dir.mkdir(parents=True) path = skill_dir / "SKILL.md" path.write_text( - f"---\nname: {name}\ndescription: bar\ncapability: capability:setup\nlicense: Apache-2.0\n---\n# body\n", + f"---\nname: {name}\ndescription: bar\ncapability: capability:platform\nlicense: Apache-2.0\n---\n# body\n", encoding="utf-8", ) return path @@ -386,7 +386,7 @@ def test_missing_name_is_skipped(self, tmp_path: Path) -> None: skill_dir.mkdir(parents=True) path = skill_dir / "SKILL.md" path.write_text( - "---\ndescription: bar\ncapability: capability:setup\nlicense: Apache-2.0\n---\n# body\n", + "---\ndescription: bar\ncapability: capability:platform\nlicense: Apache-2.0\n---\n# body\n", encoding="utf-8", ) assert list(validate_name_convention(path, path.read_text())) == [] @@ -639,7 +639,7 @@ def _make_skill_dir(self, root: Path, skill_name: str = "setup-foo") -> Path: skill_dir = root / "skills" / skill_name skill_dir.mkdir(parents=True) (skill_dir / "SKILL.md").write_text( - f"---\nname: magpie-{skill_name}\ndescription: bar\ncapability: capability:setup\nlicense: Apache-2.0\n---\n" + f"---\nname: magpie-{skill_name}\ndescription: bar\ncapability: capability:platform\nlicense: Apache-2.0\n---\n" "\n" "# body\n", encoding="utf-8", @@ -651,7 +651,7 @@ def _make_skill_dir(self, root: Path, skill_name: str = "setup-foo") -> Path: "## Capability to skill map\n\n" "| Skill | Capability / capabilities |\n" "|---|---|\n" - f"| `{skill_name}` | `capability:setup` |\n\n" + f"| `{skill_name}` | `capability:platform` |\n\n" "## Capability to tool map\n\n" "| Tool | Capability / capabilities | Role |\n" "|---|---|---|\n", @@ -964,7 +964,7 @@ def test_quoted_phrase_diff_reports_missing(self, tmp_path: Path) -> None: # Minimal valid SKILL.md frontmatter used across injection-guard tests. _GUARD_FM = ( - "---\nname: test-skill\ndescription: bar\ncapability: capability:setup\nlicense: Apache-2.0\n---\n" + "---\nname: test-skill\ndescription: bar\ncapability: capability:platform\nlicense: Apache-2.0\n---\n" ) # A gh-pr-view signal that unambiguously looks like a workflow fetch step. @@ -1375,7 +1375,7 @@ def test_all_violations_are_soft_category(self, tmp_path: Path) -> None: def _fenced_skill_lf(cmd: str) -> str: """Wrap *cmd* in a minimal SKILL.md with a fenced bash block.""" - return f"---\nname: test\ndescription: test\ncapability: capability:setup\nlicense: Apache-2.0\n---\n\n```bash\n{cmd}\n```\n" + return f"---\nname: test\ndescription: test\ncapability: capability:platform\nlicense: Apache-2.0\n---\n\n```bash\n{cmd}\n```\n" class TestLowercaseFField: @@ -1445,7 +1445,7 @@ def test_prose_mention_not_flagged(self, tmp_path: Path) -> None: """Inline backtick prose like ``-f title='...'`` must not fire.""" path = tmp_path / "SKILL.md" text = ( - "---\nname: test\ndescription: test\ncapability: capability:setup\nlicense: Apache-2.0\n---\n\n" + "---\nname: test\ndescription: test\ncapability: capability:platform\nlicense: Apache-2.0\n---\n\n" "Avoid using `-f title='value'` — use `-F title=@file` instead.\n" ) violations = list(validate_lowercase_f_field(path, text)) @@ -1455,7 +1455,7 @@ def test_outside_fenced_block_not_flagged(self, tmp_path: Path) -> None: """Bare prose outside a fenced block must not fire.""" path = tmp_path / "SKILL.md" text = ( - "---\nname: test\ndescription: test\ncapability: capability:setup\nlicense: Apache-2.0\n---\n\n" + "---\nname: test\ndescription: test\ncapability: capability:platform\nlicense: Apache-2.0\n---\n\n" "Run: gh api milestones -f title='v1'\n" ) violations = list(validate_lowercase_f_field(path, text)) @@ -1518,7 +1518,7 @@ def test_license_header_violation_is_hard_category(self) -> None: def test_md_file_is_exempt(self, tmp_path: Path) -> None: """Skill .md files declare license via frontmatter, so they need no header.""" path = tmp_path / "SKILL.md" - text = "---\nname: foo\ndescription: bar\ncapability: capability:setup\nlicense: Apache-2.0\n---\n# Body\n" + text = "---\nname: foo\ndescription: bar\ncapability: capability:platform\nlicense: Apache-2.0\n---\n# Body\n" violations = list(validate_license_header(path, text)) assert violations == [] @@ -2089,7 +2089,7 @@ def _make_valid_skill(root: Path, name: str) -> Path: skill_dir = root / "skills" / name skill_dir.mkdir(parents=True, exist_ok=True) (skill_dir / "SKILL.md").write_text( - f"---\nname: magpie-{name}\ndescription: A test skill.\ncapability: capability:setup\nlicense: Apache-2.0\n---\n" + f"---\nname: magpie-{name}\ndescription: A test skill.\ncapability: capability:platform\nlicense: Apache-2.0\n---\n" "\n" "# Body\nSome content.\n" ) @@ -2097,7 +2097,7 @@ def _make_valid_skill(root: Path, name: str) -> Path: doc = root / "docs" / "labels-and-capabilities.md" if doc.exists(): text = doc.read_text() - row = f"| `{name}` | `capability:setup` |\n" + row = f"| `{name}` | `capability:platform` |\n" # Insert right after the skill table's separator row. marker = "## Capability to skill map\n\n| Skill | Capability / capabilities |\n|---|---|\n" if marker in text and row not in text: @@ -2162,7 +2162,7 @@ def test_strict_promotes_soft_violations_to_hard( "---\n" "name: magpie-soft-skill\n" "description: A test skill.\n" - "capability: capability:setup\nlicense: Apache-2.0\n" + "capability: capability:platform\nlicense: Apache-2.0\n" "---\n" "\n" "```bash\n" @@ -2173,7 +2173,7 @@ def test_strict_promotes_soft_violations_to_hard( doc = root / "docs" / "labels-and-capabilities.md" text = doc.read_text() marker = "## Capability to skill map\n\n| Skill | Capability / capabilities |\n|---|---|\n" - doc.write_text(text.replace(marker, marker + "| `soft-skill` | `capability:setup` |\n", 1)) + doc.write_text(text.replace(marker, marker + "| `soft-skill` | `capability:platform` |\n", 1)) monkeypatch.chdir(root) rc_normal = main([]) @@ -2349,7 +2349,7 @@ def test_tool_with_valid_readme(self, tmp_path: Path) -> None: tool = root / "tools" / "foo" tool.mkdir() (tool / "README.md").write_text( - "# tools/foo\n\n**Capability:** capability:setup\n\nFoo tool.\n\n" + "# tools/foo\n\n**Capability:** substrate:framework-dev\n\nFoo tool.\n\n" "## Prerequisites\n\n- Python 3.11+ via uv.\n" ) violations = list(validate_tools(root)) @@ -2388,7 +2388,7 @@ def test_tool_capability_multi_value(self, tmp_path: Path) -> None: tool = root / "tools" / "dual" tool.mkdir() (tool / "README.md").write_text( - "# dual\n\n**Capability:** capability:setup + capability:intake\n\n## Prerequisites\n\n- None.\n" + "# dual\n\n**Capability:** contract:tracker + substrate:analytics\n\n## Prerequisites\n\n- None.\n" ) violations = list(validate_tools(root)) assert violations == [] @@ -2402,7 +2402,7 @@ def test_tool_capability_regex_does_not_slurp_past_line(self, tmp_path: Path) -> tool.mkdir() (tool / "README.md").write_text( "# tools/with-prose\n\n" - "**Capability:** capability:setup\n\n" + "**Capability:** substrate:framework-dev\n\n" "Some prose that follows the capability line and should NOT be parsed as part of it.\n\n" "## Prerequisites\n\n- None.\n" ) @@ -2413,7 +2413,7 @@ def test_tool_missing_prerequisites(self, tmp_path: Path) -> None: root = _make_tools_root(tmp_path) tool = root / "tools" / "no-prereq" tool.mkdir() - (tool / "README.md").write_text("# no-prereq\n\n**Capability:** capability:setup\n\nA tool.\n") + (tool / "README.md").write_text("# no-prereq\n\n**Capability:** substrate:framework-dev\n\nA tool.\n") violations = list(validate_tools(root)) assert len(violations) == 1 assert "'## Prerequisites'" in violations[0].message @@ -2433,7 +2433,7 @@ def test_tool_organization_known(self, tmp_path: Path) -> None: tool = root / "tools" / "asf-backend" tool.mkdir() (tool / "README.md").write_text( - "# asf-backend\n\n**Capability:** capability:setup\n\n" + "# asf-backend\n\n**Capability:** substrate:framework-dev\n\n" "**Organization:** ASF\n\nAn ASF backend.\n\n## Prerequisites\n\n- None.\n" ) violations = list(validate_tools(root)) @@ -2445,7 +2445,7 @@ def test_tool_organization_unknown(self, tmp_path: Path) -> None: tool = root / "tools" / "bogus-org" tool.mkdir() (tool / "README.md").write_text( - "# bogus-org\n\n**Capability:** capability:setup\n\n" + "# bogus-org\n\n**Capability:** substrate:framework-dev\n\n" "**Organization:** Nope\n\nA tool.\n\n## Prerequisites\n\n- None.\n" ) violations = [v for v in validate_tools(root) if v.category == "organization"] @@ -2463,7 +2463,7 @@ def test_frontmatter_organization_known(self, tmp_path: Path) -> None: (tmp_path / "organizations" / "ASF").mkdir(parents=True) text = ( "---\nname: magpie-x\ndescription: d\nlicense: Apache-2.0\n" - "capability: capability:setup\norganization: ASF\n---\n\nBody.\n" + "capability: capability:platform\norganization: ASF\n---\n\nBody.\n" ) violations = [ v @@ -2476,7 +2476,7 @@ def test_frontmatter_organization_unknown(self, tmp_path: Path) -> None: (tmp_path / "organizations" / "ASF").mkdir(parents=True) text = ( "---\nname: magpie-x\ndescription: d\nlicense: Apache-2.0\n" - "capability: capability:setup\norganization: Nope\n---\n\nBody.\n" + "capability: capability:platform\norganization: Nope\n---\n\nBody.\n" ) violations = [ v @@ -2502,7 +2502,7 @@ def _seed_capability_repo( """Build a tiny repo with a labels-and-capabilities.md doc, skills, and tool READMEs. `*_skills` maps skill-name → capability cell text (e.g. ``capability:triage``). - `*_tools` maps tool-name → capability cell text (e.g. ``capability:setup + capability:intake``). + `*_tools` maps tool-name → capability cell text (e.g. ``capability:platform + capability:intake``). """ root = tmp_path / "repo" (root / "docs").mkdir(parents=True) @@ -2544,9 +2544,9 @@ def test_aligned_passes(self, tmp_path: Path) -> None: root = _seed_capability_repo( tmp_path, doc_skills={"alpha": "capability:triage"}, - doc_tools={"omega": "capability:setup"}, + doc_tools={"omega": "capability:platform"}, live_skills={"alpha": "capability:triage"}, - live_tools={"omega": "capability:setup"}, + live_tools={"omega": "capability:platform"}, ) violations = list(validate_capability_sync(root)) assert violations == [] @@ -2588,9 +2588,9 @@ def test_tool_in_doc_but_not_live(self, tmp_path: Path) -> None: root = _seed_capability_repo( tmp_path, doc_skills={}, - doc_tools={"omega": "capability:setup", "ghost-tool": "capability:setup"}, + doc_tools={"omega": "capability:platform", "ghost-tool": "capability:platform"}, live_skills={}, - live_tools={"omega": "capability:setup"}, + live_tools={"omega": "capability:platform"}, ) violations = list(validate_capability_sync(root)) assert any("'ghost-tool'" in v.message and "no live tools/" in v.message for v in violations) @@ -2599,9 +2599,9 @@ def test_live_tool_missing_from_doc(self, tmp_path: Path) -> None: root = _seed_capability_repo( tmp_path, doc_skills={}, - doc_tools={"omega": "capability:setup"}, + doc_tools={"omega": "capability:platform"}, live_skills={}, - live_tools={"omega": "capability:setup", "extra-tool": "capability:stats"}, + live_tools={"omega": "capability:platform", "extra-tool": "capability:stats"}, ) violations = list(validate_capability_sync(root)) assert any( diff --git a/tools/skill-evals/README.md b/tools/skill-evals/README.md index 0d74d7d9..e1a7090f 100644 --- a/tools/skill-evals/README.md +++ b/tools/skill-evals/README.md @@ -1,6 +1,6 @@ # skill-evals -**Capability:** capability:setup + capability:stats +**Capability:** substrate:framework-dev Behavioral eval harness for Apache Magpie skills. Each eval suite tests a skill pipeline step by step, verifying that the model produces the correct structured JSON output for a fixed set of fixture cases. diff --git a/tools/spec-loop/README.md b/tools/spec-loop/README.md index eadb230e..96dc8b48 100644 --- a/tools/spec-loop/README.md +++ b/tools/spec-loop/README.md @@ -3,7 +3,7 @@ # spec-loop -**Capability:** capability:setup +**Capability:** substrate:framework-dev A spec-driven build loop for this framework, in the general [Ralph](https://ghuntley.com/ralph/) style (run a fresh agent context diff --git a/tools/spec-status-index/README.md b/tools/spec-status-index/README.md index 5346a9a3..c3c59eda 100644 --- a/tools/spec-status-index/README.md +++ b/tools/spec-status-index/README.md @@ -15,7 +15,7 @@ # spec-status-index -**Capability:** capability:setup + capability:stats +**Capability:** substrate:framework-dev A deterministic `uv` tool that reads spec-loop specs from `tools/spec-loop/specs/` and prints them grouped by status, so build diff --git a/tools/spec-validator/README.md b/tools/spec-validator/README.md index accdaace..2cf5823d 100644 --- a/tools/spec-validator/README.md +++ b/tools/spec-validator/README.md @@ -14,7 +14,7 @@ # spec-validator -**Capability:** capability:setup +**Capability:** substrate:framework-dev Validates spec files in `tools/spec-loop/specs/` — the counterpart to `tools/skill-and-tool-validator/` for the spec side of the framework. diff --git a/tools/vcs/README.md b/tools/vcs/README.md index 0d55a05c..79bc5432 100644 --- a/tools/vcs/README.md +++ b/tools/vcs/README.md @@ -16,7 +16,7 @@ # `magpie-vcs` -**Capability:** capability:setup +**Capability:** contract:source-control Runnable implementation of the **source-control (VCS) capability** documented in