v0.6.0 — verbatim filter extraction + per-flag probe gate - #6
Merged
Conversation
Closes #5: generated CLIs were systematically blanket-marking list filters as broken from a single failed probe with the wrong name. The fix lands in three layers: instructions, substrate, and validator. Generator instructions (skills/clify/SKILL.md, references/scaffold-pipeline.md) - Promote list-filter handling to its own subsection of Phase 2 + 5. - Five rules: verbatim flag extraction (no collapsing *_startswith / *_contains into bare names), enum surfacing in flag specs, auto-emit sort_column/sort_order, per-flag probe protocol, probe log written to .clify.json.filterProbes. Substrate (examples/exemplar-cli/) - lib/quirks.mjs: pickBrokenFilters now returns { name: { value, match } } so the client-side fallback honors documented match modes (equals, startswith, contains). Backwards-compatible with the v0.5 string-list form (defaults to equals-or-contains). - lib/help.mjs: render flags.<name>.enum inline as --filter_by <All|NotShipped|Shipped>. - lib/args.mjs: new checkEnum helper; bin runtime rejects out-of-enum values with a validation_error. - knowledge/query-flags-and-broken-list-filters.md: full discovery checklist with the Zoho Inventory packages.list worked example. Validator gate (lib/validate.mjs) - New filter-coverage check (HARD-FAIL): * list action declares filter-shaped flags but .clify.json.filterProbes has zero entries for the resource → fail (Phase 5 skipped probe). * every filter on a list action is in brokenListFilters AND no probe verified ANY filter → fail (the blanket-mark anti-pattern). * a filter status:"untested" → warn, not fail. - Path params (e.g. itemId in /items/:itemId/variants) are excluded from the filter-shaped set. Tests - examples/exemplar-cli/test/integration.test.mjs: filter-probe pattern asserting the wire URL carries the filter and the row count differs. - examples/exemplar-cli/test/quirks.test.mjs: per-filter match-mode tests. - test/clify.test.mjs: four new validator tests covering pass, missing probes, blanket-mark, and untested-warn cases. Version bumped to 0.6.0 in package.json + .claude-plugin/plugin.json + .claude-plugin/marketplace.json (npm test enforces consistency). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #5.
Summary
Fixes the generator-side drift where generated CLIs blanket-mark list
filters as BROKEN after a single probe with the wrong name. The fix
lands in all three layers — instructions, runtime substrate, and the
validator gate — so guidance, behavior, and enforcement stay aligned.
Generator instructions
skills/clify/SKILL.mdandreferences/scaffold-pipeline.mdpromote list-filter handling from a one-liner to its own subsection of Phase 2 + Phase 5.*_startswith/*_containsinto bare names), enum surfacing, auto-emitsort_column/sort_order, per-flag probe protocol, probe log to.clify.json.filterProbes.Substrate (examples/exemplar-cli/)
lib/quirks.mjs—pickBrokenFiltersreturns{ name: { value, match } }so client-side fallback honors documented match modes (equals,startswith,contains). Backwards-compatible with the v0.5 string-list form.lib/help.mjs— rendersflags.<name>.enuminline:--filter_by <All|NotShipped|Shipped>.lib/args.mjs— newcheckEnumhelper; bin runtime rejects out-of-enum values with avalidation_error.knowledge/query-flags-and-broken-list-filters.md— full discovery checklist with the Zoho Inventorypackages listworked example.Validator gate (
lib/validate.mjs)New
filter-coveragecheck, in thecoveragecategory:.clify.json.filterProbeshas zero entries for the resource (Phase 5 skipped the probe step).brokenListFiltersAND no probe verified ANY filter (the blanket-mark anti-pattern).status: "untested"(probe was skipped at gen time).Path parameters (e.g.
itemIdin/items/:itemId/variants) are excluded from the filter-shaped set so sub-resource list actions don't false-positive.Tests
examples/exemplar-cli/test/integration.test.mjs—filter-probe patterntest asserting the wire URL carries the filter AND the row count differs from baseline. This is the test pattern any generated CLI inherits — would have caught the Zoho miss at gen time.examples/exemplar-cli/test/quirks.test.mjs— per-filter match-mode tests for the new substrate.test/clify.test.mjs— four new validator tests: pass, missing probes, blanket-mark, untested-warn.Version
Bumped to
0.6.0inpackage.json+.claude-plugin/plugin.json+.claude-plugin/marketplace.json(npm testenforces consistency viascripts/check-plugin-version.mjs).Test plan
cd examples/exemplar-cli && npm test— 56/56 passnpm test(clify root) — 19/19 pass, includes 4 new filter-coverage testsnpm run validate-exemplar— 27/27 checks pass, 0 warningsnpm test)🤖 Generated with Claude Code