Skip to content

CONTRIBUTING.md

Latest commit

 

History

History
37 lines (27 loc) · 2.33 KB

CONTRIBUTING.md

File metadata and controls

37 lines (27 loc) · 2.33 KB
Raw

Contributing to Caliber

Issues and pull requests are welcome. Caliber is small on purpose, so most contributions are a data edit, a wording fix, or a new golden task. Read this page before opening a pull request; the house rules below are checked on every change.

House rules

These apply to every output-facing file: the skill, the command files, the README, and the docs.

  • No em dashes and no standalone hyphens as punctuation. Hyphenated compound words are the only acceptable use. This keeps the tool's output consistent.
  • Keep the skill description at or under 200 characters. claude.ai rejects skill uploads whose frontmatter description exceeds 200 characters. Check the length after any edit to the frontmatter of skills/caliber/SKILL.md.
  • Keep the tier-ordering invariant. The three options are ordered on model tier always: Budget is at or below Optimal is at or below Splurge. Effort is ordered only when two options share a model, because a level name is not a fixed amount of thinking across models. n/a (the small tier) is the floor, max and ultracode are the ceiling, and a stronger model at a lower named effort still counts as the higher option.
  • Keep the engine model agnostic. A new model is a data edit in skills/caliber/models.md, not a logic edit in skills/caliber/SKILL.md. Concrete model names, prices, and effort support live only in the catalog.
  • Bump the "Last updated" date at the top of models.md on every change to that file.

Maintenance flow

When a model ships, an effort level changes, or a price moves, edit only skills/caliber/models.md. Verify every fact against the official doc links at the top of that file before committing; nothing is fetched at runtime, so the catalog is only as current as its last verified edit.

Validating changes

The output is a judgment, not a computed value, so validation is structural:

  1. Run the golden tasks in evals/golden-tasks.md against the edited skill.
  2. Check every output against the checklist in evals/invariants.md.

Outputs are nondeterministic, so check structure, not verbatim text: tier ordering holds, confidence is honest, model names and prices match models.md, and the format matches the templates in SKILL.md. If a recommendation feels off on a real task, that is a finding worth an issue even without a rule violation to point at.