feat(meta): add adopter pilot-report template and validator#646
Open
justinmclean wants to merge 4 commits into
Open
feat(meta): add adopter pilot-report template and validator#646justinmclean wants to merge 4 commits into
justinmclean wants to merge 4 commits into
Conversation
Add tools/pilot-report-validator — a stdlib-only Python tool that validates adopter pilot-report Markdown files against a required schema (frontmatter keys skill/date/target_repo/profile, valid profile value asf|non-asf|custom, and seven required body sections). Add docs/pilot-report-template.md — the template adopters copy and fill in to record a skill-family pilot run (blocked preflights, false positives, confirmation points, privacy/adapter notes, proposed spec changes). Wire the template reference into the experimental family Status sections for mentoring, pairing, repo-health, and issue-management so pilot evidence is captured consistently. Add the tool to docs/labels-and-capabilities.md and pyproject.toml workspace members; add a .gitignore exception so the committed framework tool is not caught by the evidence-package pilot-*/ pattern. Generated-by: Claude (Opus 4.7)
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
justinmclean
force-pushed
the
adopter-pilot-feedback-harness
branch
from
be760b2 to
3892676
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds the adopter pilot-report artefact and a tool to validate it. A pilot
report documents an adopter's end-to-end run of an experimental skill family
and is the primary evidence source for advancing a skill from
experimentalto
stable.docs/pilot-report-template.md: the template adopters copy and fill in(frontmatter plus seven required sections: blocked preflights, false
positives, confirmation points, privacy/adapter notes, proposed spec
changes, and so on).
tools/pilot-report-validator/: a stdlib-only Python tool (no runtimedeps) that validates filled-in reports against the required schema.
mentoring, pairing, repo-health, and issue-management, and registers the
tool in
docs/labels-and-capabilities.mdand the workspacepyproject.toml.Type of change
.claude/skills/<name>/) — eval fixtures updated belowtools/<system>/*.md)tools/*/withpyproject.toml)docs/,README.md,CONTRIBUTING.md)projects/_template/)prek, workflows, validators)Test plan
prek run --all-filespassesuv run pytest/ruff check/mypypasses(
PYTHONPATH=tools/skill-evals/src python3 -m skill_evals.runner tools/skill-evals/evals/<skill>/)(a regression test for the bug fixed / the behaviour added — see CONTRIBUTING.md)