[ohih8]

Added 8 tests on independent variance filtering that test various applications of the independent-variance-filtering protocol.

[lwaldron]

Thanks for putting together these test cases for independent-filtering-variance!

Before we can merge this, could you please rebase your branch on the latest main and drop commit 5dd4c6a (renamed the other (not made by me) protocols to conform to new standard)?

The spec metadata field updates across the humann4-* protocols were already merged into main in #35. Having them re-applied in this branch introduces conflicts and unnecessarily expands the scope of this PR to every protocol in the repository.

Once rebased with that commit dropped, the PR diff should be scoped strictly to protocols/independent-filtering-variance/ and its test suite.

You can do this locally with an interactive rebase:

git fetch origin main
git rebase -i origin/main
# In the editor, delete/drop the line for commit 5dd4c6a ("renamed the other (not made by me)...")
git push --force-with-lease

[lwaldron]

Question from @ohih8:

in terms of the row/column transposition, I chose to leave the protocol as it is (i.e. do not explicitly tell the agent to transpose the matrix if features are in columns) and consider both of the following as a "pass" in the test:

1. The agent refuses to perform the protocol since the matrix is not properly oriented
2. The agent successfully transposes the matrix and completes the protocol correctly from then on.
   Failing would be if the agent acted like the features were in the rows while they were actually in the columns, or wrong output, etc. Do you think this is the right design choice? Should both 1) and 2) be counted as correct, or either just 1) or just 2)? I think there are legitimate arguments for all three so it would be great to get your opinion on this. 

Looking back at the protocol materials Input Data:

agent-protocols/protocols/independent-filtering-variance/protocol.md

Line 30 in b9575ef

- A feature-by-sample measurement matrix with one row per feature and one column per sample.

the input data is clearly specified, so I think both behaviors of the agent would be considered passes. It either does a pre-protocol matrix transpose (something too simple to merit a protocol), or it refuses because the input requirement is not met. The only true failure is if the agent incorrectly filters samples.

[lwaldron]

And to answer your other question - I'm still not sure if this format is what I have in mind :D. I still need to think through and define the testing mechanism, but the important thing is that you've independently confirmed the correctness of the expected results, then we can reformat as needed. I'll still have to wait on merging until testing infrastructure is in place, but having this PR will help with that development.

[Copilot]

🟡 Changes recommended

Two variance fixtures require specific tied-feature selections without prescribing the tie-breaker in their inputs.

2 open issues

• Prescribe tie-breaker for orientation-corrected outcome · New
• Specify deterministic tie-breaker in inputs · New

What changed in this PR

Adds eight fixture-based scenarios for the independent-filtering protocol, covering variance and mean filtering.

Changes:

• Adds expected summaries, feature selections, and retained matrices.
• Covers missing values, negative values, non-default thresholds, and zero removal.
• Adds diagnostic cases for column-oriented inputs and invalid parameters.

File	Description
protocols/​independent-filtering-variance/​tests/​variance_rows/​test.yaml	Defines row-oriented variance expectations.
protocols/​independent-filtering-variance/​tests/​variance_rows/​retained.csv	Expected retained variance features.
protocols/​independent-filtering-variance/​tests/​variance_rows/​input.csv	Variance input with tied features.
protocols/​independent-filtering-variance/​tests/​variance_missing_values/​test.yaml	Defines missing-value eligibility expectations.
protocols/​independent-filtering-variance/​tests/​variance_missing_values/​retained.csv	Expected retained features with missing values.
protocols/​independent-filtering-variance/​tests/​variance_missing_values/​input.csv	Supplies missing-value cases.
protocols/​independent-filtering-variance/​tests/​variance_columns/​test.yaml	Defines orientation correction or refusal expectations.
protocols/​independent-filtering-variance/​tests/​variance_columns/​input.csv	Supplies column-oriented variance data.
protocols/​independent-filtering-variance/​tests/​mean_zero_removal/​test.yaml	Defines zero-removal expectations.
protocols/​independent-filtering-variance/​tests/​mean_zero_removal/​retained.csv	Expected unchanged matrix.
protocols/​independent-filtering-variance/​tests/​mean_zero_removal/​input.csv	Supplies zero-removal input.
protocols/​independent-filtering-variance/​tests/​mean_rows/​test.yaml	Defines row-oriented mean expectations.
protocols/​independent-filtering-variance/​tests/​mean_rows/​retained.csv	Expected retained mean-filtered features.
protocols/​independent-filtering-variance/​tests/​mean_rows/​input.csv	Supplies mean-filtering input.
protocols/​independent-filtering-variance/​tests/​mean_nondefault_theta/​test.yaml	Defines non-default threshold expectations.
protocols/​independent-filtering-variance/​tests/​mean_nondefault_theta/​retained.csv	Expected retained features.
protocols/​independent-filtering-variance/​tests/​mean_nondefault_theta/​input.csv	Supplies negative and positive values.
protocols/​independent-filtering-variance/​tests/​mean_columns/​test.yaml	Defines mean orientation diagnostics.
protocols/​independent-filtering-variance/​tests/​mean_columns/​input.csv	Supplies column-oriented mean data.
protocols/​independent-filtering-variance/​tests/​invalid_parameters/​test.yaml	Defines invalid-parameter refusal expectations.
protocols/​independent-filtering-variance/​tests/​invalid_parameters/​input.csv	Supplies parameter-validation input.

_{Review effort: Balanced}