Protocol/CBEA by ohih8 · Pull Request #47 · waldronlab/agent-protocols

ohih8 · 2026-10-15T02:58:33Z

Closes #18

Added a protocol for CBEA. As usual with the protocols, I did some internal testing but more formal testing is needed to ensure the accuracy and efficacy of the protocol.

This reverts commit 14308ed.

Copilot

🟡 Changes recommended

Taxonomic aggregation and null-inference instructions leave choices that can change enrichment results.

2 open findings

What changed in this PR

Adds an atomic CBEA protocol for testing BugSigDB microbial signatures against compositional abundance data, addressing issue #18.

Changes:

Defines input requirements, taxonomy harmonization, background universe, and set-size bounds.
Describes balance scoring, significance testing, multiple-testing adjustment, and reporting.
Documents pooled versus per-dataset execution and interpretation limits.

File	Description
protocols/cbea-bugsigdb-enrichment/protocol.md	Adds the CBEA enrichment workflow, citations, parameters, and reporting requirements.

🧠 Review effort: Balanced

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

lwaldron · 2026-10-17T20:57:51Z

I know the issue said to do this for BugSigDB signatures, but I'm now thinking this should be an enrichment protocol for any microbial signatures, with BugSigDB as an example.

lwaldron

I know the issue said to do this for BugSigDB signatures, but I'm now thinking this should be an enrichment protocol for any microbial signatures, with BugSigDB as an example. But the protocol looks good!

Rename to cbea-signature-enrichment with BugSigDB as the worked example. Address Copilot review: define aggregation by summing relative abundances with warnings for excluded rows, and specify the pooled cross-sample null, correlation adjustment, and upper-tail p-values by naming the CBEA functions. Drafted with Claude Code.

Copilot

🟡 Changes recommended

The example does not reliably apply the recorded seed, and the stated null conflicts with the sampling procedure.

2 open issues

2 resolved since last review

_{Review effort: Balanced}

The exported cbea() methods do not forward parallel_backend, so seed with set.seed() instead. State the null as random subsets drawn from the whole universe, matching fit_scores(). Drafted with Claude Code.

Copilot

🔵 Needs a closer look

The statistical inference needs domain-expert validation, and the executable examples require corrections.

No open issues 🎉

2 previously missed issues

In code that hasn't changed since last review

Pass the imported BugSigDB result to getSignatures()

protocols/cbea-signature-enrichment/protocol.md:73

getSignatures() needs the imported BugSigDB data; it does not retrieve the preceding import result automatically. This example therefore fails with a missing-argument error. Bind the import result and pass it to getSignatures(), as the cited Figure3.Rmd does.

Use the selected distribution instead of hardcoded norm

protocols/cbea-signature-enrichment/protocol.md:244

The parameters permit selecting mnorm, but this call always uses norm. Following the snippet after selecting the mixture model produces normal-model p-values that disagree with the recorded distribution. Pass the selected distribution to keep the call consistent with Steps 4.3–4.5.

2 resolved since last review

_{Review effort: Balanced}

Pass the imported object to getSignatures() and use the selected distribution in the cbea() call. Drafted with Claude Code.

Copilot

🟡 Changes recommended

The mixture-model adjustment can miscalibrate significance calls, and pooled taxon alignment remains undefined.

1 open issue

mnorm correlation adjustment is incorrect in CBEA 0.99.4 · New

1 previously missed issue

In code that hasn't changed since last review

Pooled mode lacks a policy for aligning taxa across datasets

protocols/cbea-signature-enrichment/protocol.md:115

Pooled mode does not define how to align taxa before combining datasets. Positive tables with taxa {A, B, C} and {B, C, D} need a merge policy: a union introduces missing values or zeros forbidden by Step 1, while an intersection changes the universe and subsequent balances. Require identical, pre-aligned taxon identifiers, or explicitly define the merge and upstream preprocessing policy. Revalidate the combined table before proceeding so pooling cannot bypass the finite, strictly positive input requirement.

_{Review effort: Balanced}

+  - `distribution`: `norm`, a normal distribution fitted to the null scores.
+    `mnorm`, the two-component normal mixture from the same paper, is the only
+    permitted alternative and must be recorded if chosen.
+  - `correlation_adjustment`: enabled (`adj = TRUE` in `CBEA::cbea()`).


Drafted with Claude Code.

The mnorm correlation adjustment in CBEA 0.99.4 computes the mixture standard deviation without squaring component terms, so only norm is permitted. Pooled mode now merges datasets on the union of taxa with zero replacement applied to the combined table. Drafted with Claude Code.

Drafted with Claude Code.

lwaldron · 2026-10-17T21:27:26Z

OK looking good after some avoidance of upstream issues! Thanks @ohih8 , merging.

ohih8 added 3 commits October 14, 2026 22:14

Created CBEA Protocol

14308ed

Revert "Created CBEA Protocol"

a6986aa

This reverts commit 14308ed.

Add CBEA BugSigDB enrichment protocol

5304603

ohih8 requested a review from lwaldron as a code owner October 15, 2026 02:58

lwaldron requested a balanced review from Copilot October 16, 2026 17:17

Copilot started reviewing on behalf of lwaldron October 16, 2026 17:17

View session

Copilot AI reviewed Oct 16, 2026

View reviewed changes

Comment thread protocols/cbea-bugsigdb-enrichment/protocol.md Outdated

Comment thread protocols/cbea-bugsigdb-enrichment/protocol.md Outdated

lwaldron requested changes Oct 17, 2026

View reviewed changes

lwaldron requested a balanced review from Copilot October 17, 2026 21:04

Copilot started reviewing on behalf of lwaldron October 17, 2026 21:05

View session

Copilot AI reviewed Oct 17, 2026

View reviewed changes

Comment thread protocols/cbea-signature-enrichment/protocol.md Outdated

Comment thread protocols/cbea-signature-enrichment/protocol.md Outdated

Fix seed handling and null statement after Copilot review

5b45eb0

The exported cbea() methods do not forward parallel_backend, so seed with set.seed() instead. State the null as random subsets drawn from the whole universe, matching fit_scores(). Drafted with Claude Code.

lwaldron requested a balanced review from Copilot October 17, 2026 21:08

Copilot started reviewing on behalf of lwaldron October 17, 2026 21:09

View session

Copilot AI reviewed Oct 17, 2026

View reviewed changes

Fix BugSigDB and cbea() snippets after Copilot review

4e6c76a

Pass the imported object to getSignatures() and use the selected distribution in the cbea() call. Drafted with Claude Code.

lwaldron requested a balanced review from Copilot October 17, 2026 21:13

Copilot started reviewing on behalf of lwaldron October 17, 2026 21:14

View session

Copilot AI reviewed Oct 17, 2026

View reviewed changes

lwaldron added 2 commits October 17, 2026 17:18

Record review by Levi Waldron

b41683f

Drafted with Claude Code.

This was referenced Oct 17, 2026

get_sd() computes the mixture-normal standard deviation without squaring the component terms qpmnguyen/CBEA#12

Open

cbea() methods do not forward parallel_backend, so RNGseed on the backend has no effect qpmnguyen/CBEA#13

Open

Link upstream CBEA issues from the protocol

3011f74

Drafted with Claude Code.

lwaldron merged commit fc373d1 into waldronlab:main Oct 17, 2026
1 check passed

lwaldron mentioned this pull request Oct 18, 2026

Say when code belongs in a protocol #48

Merged

Navigation Menu

Repository navigation

Protocol/CBEA - #47

Protocol/CBEA #47
lwaldron merged 9 commits into
waldronlab:mainfrom
ohih8:protocol/cbea-bugsigdb-enrichment-pr

ohih8 commented Oct 15, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

lwaldron commented Oct 17, 2026

Uh oh!

lwaldron left a comment

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI left a comment

Uh oh!

lwaldron commented Oct 17, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

ohih8 commented Oct 15, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Uh oh!

Uh oh!

Uh oh!

lwaldron commented Oct 17, 2026

Uh oh!

lwaldron left a comment

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🔵 Needs a closer look

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Uh oh!

lwaldron commented Oct 17, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants