Repository navigation
State where requirements on input materials live, and whether a violation is an error or a warning #46
Description
Reviewing waldronlab/agent-protocols#53 (Shannon diversity), the rule about rarefaction ended up in three places: a Materials bullet, a sentence in Step 2, and a Notes paragraph. None of them says what an agent should do when it gets a non-rarefied count table. That's not the author's fault; the standard doesn't say where input requirements belong or what happens when one fails.
What the standard currently says about Materials is that the heading must exist, plus a recommended skeleton with Software & Repositories and Databases & Reference Data. Steps may contain anything. The only error/warning vocabulary is in the protocol-runner skill and is about protocol status, not inputs. Diagnostic tests refer to "a requirement the protocol states" without saying where it's stated, and can only check refusal (expected_absent), so there's no way to express "proceed, but warn". Existing protocols have improvised: constraints in Materials prose, then a "Step 1: Validate" that restates them.
Proposal, three short additions:
-
Placement rule, under Materials in PROTOCOL_STANDARD.md:
Materials lists every input the protocol consumes (data, parameters, software, reference data) and every condition an input must or should satisfy. Steps consume materials; they don't introduce new conditions on inputs. Statements about what the protocol does or doesn't do belong in the overview or Notes, not in Materials.
-
Two severity tiers, using RFC 2119 words:
A condition written with must is a requirement: an executor that finds it violated stops before Step 1, reports which condition failed, and produces none of the protocol's outputs. A condition written with should is an advisory: the executor proceeds, warns the user naming the condition and its consequence, and records the warning in the execution metadata.
This applies to conditions written in a protocol's Materials section, not to the standard's own prose, which already uses these words in their ordinary sense. Because checking happens before Step 1, protocols stop needing a hand-written validation step for anything already in Materials. Checks that only arise during computation (a count sample whose total is zero) stay in the step where they arise.
-
Matching hooks in the two places that execute or test protocols:
skills/protocol-runner/SKILL.md: a step before Step 1 that checks every Materials condition; must stops, should warns and continues.test.yamldiagnostic mode: addexpected_warning(a substring or pattern) alongsideexpected_absent, so a test can assert both the results and the warning.
Under this rule the Shannon example becomes one bullet:
- **Input table:** CSV, one row per feature, one column per sample. The first column
must contain unique, non-missing feature identifiers. All other cells must be
non-negative numbers with no missing values. When `Input scale` is `counts`, samples
should be rarefied or otherwise normalized to a common sequencing depth; if they are
not, warn that Shannon values aren't comparable across samples and advise rarefaction.and the Step 2 sentence and Notes paragraph go away.
I'd leave the validator alone; it can't check prose semantics and this is enforceable in review. Related: #15, since the before-Step-1 check is part of the execution contract.
If this direction is right I'll write it up as an ADR.
Drafted with Claude Code; decisions and edits are mine.
Activity
Conditions on materials: placement rule, must/should tiers, expected_…