From 14308ed2210ba58b470c20b3668722de772a0e8d Mon Sep 17 00:00:00 2001 From: Otto Date: Wed, 14 Oct 2026 22:14:56 -0400 Subject: [PATCH 1/9] Created CBEA Protocol --- .../cbea-bugsigdb-enrichment/protocol.md | 222 ++++++++++++++++++ 1 file changed, 222 insertions(+) create mode 100644 protocols/cbea-bugsigdb-enrichment/protocol.md diff --git a/protocols/cbea-bugsigdb-enrichment/protocol.md b/protocols/cbea-bugsigdb-enrichment/protocol.md new file mode 100644 index 0000000..63f7ace --- /dev/null +++ b/protocols/cbea-bugsigdb-enrichment/protocol.md @@ -0,0 +1,222 @@ +--- +name: "cbea-bugsigdb-enrichment" +description: "Test harmonized BugSigDB microbial signatures for taxonomic enrichment with competitive balances of compositional abundances." +version: "1.0.0" +authors: + - name: "Otto Infield-Harm" + orcid: "0009-0009-8629-6124" + - name: "Levi Waldron" + orcid: "0000-0003-2725-0694" +date: "2026-10-14" +status: draft +type: "atomic" + +artifact_doi: ~ +collection_doi: "10.5281/zenodo.22731694" +protocol_citation: "10.1371/journal.pcbi.1010091" +method_origin_citation: "10.1371/journal.pcbi.1010091" +license: "CC-BY-4.0" +protocols_used: [] +upstream_repositories: + - "https://github.com/qpmnguyen/CBEA" + - "https://github.com/waldronlab/BugSigDBPaper" +database_urls: + - "https://bugsigdbr.bioconductor.org/" +key_packages: + - "CBEA" + - "bugsigdbr" + - "BiocSet" +category: "Statistical Analysis" +tags: [enrichment, microbe-sets, cbea, bugsigdb, compositional-data] +--- + +# CBEA Enrichment of BugSigDB Microbial Signatures + +This protocol calculates competitive balance enrichment scores and sample-level +significance for BugSigDB microbial signatures. For each signature and sample, +the score contrasts the geometric mean abundance of the signature taxa with the +geometric mean abundance of all other taxa in the declared background universe. + +## Materials + +- **Abundance data** + - A taxa-by-sample matrix of strictly positive relative abundances, or another + positive compositional representation whose scale and construction are + recorded. Counts must be converted to a positive compositional representation + before this protocol begins. + - The matrix must have unique taxon and sample identifiers. This protocol does + not define sequencing quality control, normalization, zero replacement, or + the choice of an upstream transformation. Record those decisions and the + resulting abundance scale. +- **BugSigDB signatures** + - A collection of signatures, each identified by a stable signature identifier + and containing taxa at a declared taxonomic rank. + - Obtain the signatures from BugSigDB or a versioned export and record the + release, retrieval date, source URL, and any filters used to select them. +- **Parameters** + - `rank`: exactly one taxonomic rank shared by the abundance table and all + signatures in an execution. + - `analysis_scope`: exactly one of `pooled` or `per_dataset`. + - `min_set_size`: 2 matched taxa. + - `max_set_size`: `|U| - 1`, where `U` is the execution's harmonized + background universe. + - `n_perm`: 100 permutations for the empirical null distribution. + - `distribution`: normal parametric fit to the permutation scores. + - `variance_inflation_adjustment`: enabled to account for inter-taxa + correlation. + - `multiple_testing_method`: Benjamini-Hochberg false-discovery-rate + adjustment across all retained signatures tested in the same execution. + - `significance_threshold`: adjusted p-value less than 0.05. + - `random_seed`: a recorded integer seed used for permutation reproducibility. + +## Steps + +### Step 1: Confirm the abundance matrix and analysis scope + +Verify that the abundance matrix is taxa by sample, that identifiers are unique, +and that every value supplied to CBEA is finite and strictly positive. Reject +the input and report the failed check if any value is zero, negative, missing, +or non-finite; do not silently add a pseudocount or replace a value. A +pseudocount or zero-replacement method must be selected and applied before this +protocol, with its value or algorithm recorded. + +Select `pooled` or `per_dataset` before examining enrichment results. Use +`pooled` only when datasets have compatible taxonomic processing, abundance +scales, and sample-level metadata. In pooled mode, combine the compatible +datasets and define one universe across the combined table. In per-dataset +mode, keep datasets separate and repeat Steps 2 through 5 independently for +each dataset, defining a separate universe and set of tested signatures for +each one. Do not treat a pooled analysis as an aggregation of independent +dataset results. + +Record the number of datasets, samples, taxa, the abundance representation, +the upstream preprocessing, the selected scope, and the random seed. + +### Step 2: Harmonize taxonomy and define the background universe + +Choose one `rank`, such as species or genus, before matching signatures. Convert +the taxon identifiers in the abundance matrix and the BugSigDB signatures to +that rank using the same naming convention and taxonomy mapping. Do not infer a +lower-rank identity from a higher-rank name, and do not combine identifiers +whose rank is ambiguous. + +For each signature, retain only members that match a taxon in the abundance +matrix after harmonization. Record every unmatched signature member and its +reason for exclusion. Define `U` as the set of all unique harmonized taxa +remaining in the abundance matrix for the selected analysis scope. The +complement for a signature `S` is `U \ S`; it is not the set of taxa appearing +in other BugSigDB signatures. + +Record the rank, taxonomy mapping or normalization rules, the size and +identifiers of `U`, and the matched and unmatched members of every signature. +Discard an empty signature before testing and record that reason. + +### Step 3: Apply the set-size bounds + +For each signature, let `S` be its unique matched taxa and let `k = |S|`. +Retain the signature only when: + +```text +2 <= k <= |U| - 1 +``` + +The lower bound excludes singleton signatures, for which the protocol does not +define a stable set-level balance. The upper bound guarantees a non-empty +complement. If `|U| < 3`, no signature can be tested. Remove duplicate +signature members before calculating `k`; do not count repeated BugSigDB rows as +distinct taxa. + +Record the original member count, matched member count, duplicate count, +unmatched members, `|U|`, the final set size, and the exclusion reason for every +discarded signature. Do not change the bounds after looking at the results. + +### Step 4: Calculate competitive balance scores and sample-level inference + +For every retained signature `S` and sample, calculate the CBEA competitive +balance using the positive abundances: + +```text +score(S, sample) = + sqrt(|S| * |U \ S| / |U|) * + (mean(log(abundance in S)) - mean(log(abundance in U \ S))) +``` + +This is the isometric log-ratio contrast between the signature subcomposition +and its complement. A positive score means that the signature taxa have a +higher geometric-mean abundance than the complement in that sample; a negative +score means the reverse. Do not replace this contrast with a sum or arithmetic +mean of signature abundances. + +Estimate the empirical null by permuting the taxon labels within each sample +`n_perm = 100` times while preserving the sample's abundance values and the +signature size. Use the recorded random seed. Fit the specified normal +distribution to the permutation scores and use the resulting sample-level +null inference. Apply the variance-inflation adjustment for inter-taxa +correlation. Return the CBEA score, the unadjusted p-value, and the adjusted +p-value for every retained signature and sample. + +The null hypothesis is competitive: within a sample, the taxa in the signature +have the same balance-level abundance as the measured taxa outside the +signature. It is not a null that every signature taxon has no association with +the sample condition, and it is not a test that every member of the signature +is individually differentially abundant. + +Record the CBEA implementation and version, all parameters, the random seed, +the number of successful permutations, and any signature-sample result that +could not be computed. + +### Step 5: Adjust across signatures and report the results + +Within each execution (the pooled table or one per-dataset table), apply the +Benjamini-Hochberg procedure to the complete family of unadjusted p-values for +the retained signatures. Use the same family definition for every sample in +that execution and do not adjust only the signatures that were significant +before correction. Mark a signature-sample result significant when its adjusted +p-value is less than 0.05. + +Produce a result table containing the dataset or pooled-scope identifier, +sample identifier, signature identifier, matched taxa, complement size, CBEA +score, unadjusted p-value, adjusted p-value, and significance decision. Also +produce an execution manifest containing the rank, universe, set-size bounds, +preprocessing description, inference parameters, exclusion counts, and +software versions. + +## Notes + +The CBEA paper describes this method as a single-sample taxon-set enrichment +method based on the competitive null hypothesis and isometric log-ratio +transformation. The BugSigDB paper's `Figure3.Rmd` supplies the application +context for running CBEA on pooled and individual microbiome datasets, but it +does not replace the method's primary citation. + +This protocol requires positive compositional abundances because the balance +uses logarithms. The choice of zero replacement, pseudocount, normalization, +taxonomic database, and differential-abundance input is upstream and must be +specified separately. The `0.5` addition shown in the application vignette is +not silently prescribed here; if it is used, record its scale and rationale +before running this protocol. + +A significant result licenses only the conclusion that the signature's +within-sample balance is unusual under the declared competitive null and +preprocessing. It does not establish causality, biological activity, disease +mechanism, differential abundance of every member, or generalization beyond +the analyzed samples. + +This protocol does not produce the abundance input, retrieve or curate +BugSigDB signatures, compare CBEA with ORA or PADOG, perform downstream +between-group differential analysis of the CBEA scores, or interpret a +signature's biological meaning. Those operations require separate protocols. + +## History & Reviews + + +### Version 1.0.0 (2026-10-14) + +#### Changes +- Initial protocol creation. +- Specified taxonomy harmonization, background universe, set-size bounds, + competitive balance calculation, permutation inference, multiple-testing + adjustment, and pooled versus per-dataset execution. + +#### Reviews +*No reviews yet.* From a6986aa099d2206278cc34650ab59dc1df184f27 Mon Sep 17 00:00:00 2001 From: Otto Date: Wed, 14 Oct 2026 22:45:26 -0400 Subject: [PATCH 2/9] Revert "Created CBEA Protocol" This reverts commit 14308ed2210ba58b470c20b3668722de772a0e8d. --- .../cbea-bugsigdb-enrichment/protocol.md | 222 ------------------ 1 file changed, 222 deletions(-) delete mode 100644 protocols/cbea-bugsigdb-enrichment/protocol.md diff --git a/protocols/cbea-bugsigdb-enrichment/protocol.md b/protocols/cbea-bugsigdb-enrichment/protocol.md deleted file mode 100644 index 63f7ace..0000000 --- a/protocols/cbea-bugsigdb-enrichment/protocol.md +++ /dev/null @@ -1,222 +0,0 @@ ---- -name: "cbea-bugsigdb-enrichment" -description: "Test harmonized BugSigDB microbial signatures for taxonomic enrichment with competitive balances of compositional abundances." -version: "1.0.0" -authors: - - name: "Otto Infield-Harm" - orcid: "0009-0009-8629-6124" - - name: "Levi Waldron" - orcid: "0000-0003-2725-0694" -date: "2026-10-14" -status: draft -type: "atomic" - -artifact_doi: ~ -collection_doi: "10.5281/zenodo.22731694" -protocol_citation: "10.1371/journal.pcbi.1010091" -method_origin_citation: "10.1371/journal.pcbi.1010091" -license: "CC-BY-4.0" -protocols_used: [] -upstream_repositories: - - "https://github.com/qpmnguyen/CBEA" - - "https://github.com/waldronlab/BugSigDBPaper" -database_urls: - - "https://bugsigdbr.bioconductor.org/" -key_packages: - - "CBEA" - - "bugsigdbr" - - "BiocSet" -category: "Statistical Analysis" -tags: [enrichment, microbe-sets, cbea, bugsigdb, compositional-data] ---- - -# CBEA Enrichment of BugSigDB Microbial Signatures - -This protocol calculates competitive balance enrichment scores and sample-level -significance for BugSigDB microbial signatures. For each signature and sample, -the score contrasts the geometric mean abundance of the signature taxa with the -geometric mean abundance of all other taxa in the declared background universe. - -## Materials - -- **Abundance data** - - A taxa-by-sample matrix of strictly positive relative abundances, or another - positive compositional representation whose scale and construction are - recorded. Counts must be converted to a positive compositional representation - before this protocol begins. - - The matrix must have unique taxon and sample identifiers. This protocol does - not define sequencing quality control, normalization, zero replacement, or - the choice of an upstream transformation. Record those decisions and the - resulting abundance scale. -- **BugSigDB signatures** - - A collection of signatures, each identified by a stable signature identifier - and containing taxa at a declared taxonomic rank. - - Obtain the signatures from BugSigDB or a versioned export and record the - release, retrieval date, source URL, and any filters used to select them. -- **Parameters** - - `rank`: exactly one taxonomic rank shared by the abundance table and all - signatures in an execution. - - `analysis_scope`: exactly one of `pooled` or `per_dataset`. - - `min_set_size`: 2 matched taxa. - - `max_set_size`: `|U| - 1`, where `U` is the execution's harmonized - background universe. - - `n_perm`: 100 permutations for the empirical null distribution. - - `distribution`: normal parametric fit to the permutation scores. - - `variance_inflation_adjustment`: enabled to account for inter-taxa - correlation. - - `multiple_testing_method`: Benjamini-Hochberg false-discovery-rate - adjustment across all retained signatures tested in the same execution. - - `significance_threshold`: adjusted p-value less than 0.05. - - `random_seed`: a recorded integer seed used for permutation reproducibility. - -## Steps - -### Step 1: Confirm the abundance matrix and analysis scope - -Verify that the abundance matrix is taxa by sample, that identifiers are unique, -and that every value supplied to CBEA is finite and strictly positive. Reject -the input and report the failed check if any value is zero, negative, missing, -or non-finite; do not silently add a pseudocount or replace a value. A -pseudocount or zero-replacement method must be selected and applied before this -protocol, with its value or algorithm recorded. - -Select `pooled` or `per_dataset` before examining enrichment results. Use -`pooled` only when datasets have compatible taxonomic processing, abundance -scales, and sample-level metadata. In pooled mode, combine the compatible -datasets and define one universe across the combined table. In per-dataset -mode, keep datasets separate and repeat Steps 2 through 5 independently for -each dataset, defining a separate universe and set of tested signatures for -each one. Do not treat a pooled analysis as an aggregation of independent -dataset results. - -Record the number of datasets, samples, taxa, the abundance representation, -the upstream preprocessing, the selected scope, and the random seed. - -### Step 2: Harmonize taxonomy and define the background universe - -Choose one `rank`, such as species or genus, before matching signatures. Convert -the taxon identifiers in the abundance matrix and the BugSigDB signatures to -that rank using the same naming convention and taxonomy mapping. Do not infer a -lower-rank identity from a higher-rank name, and do not combine identifiers -whose rank is ambiguous. - -For each signature, retain only members that match a taxon in the abundance -matrix after harmonization. Record every unmatched signature member and its -reason for exclusion. Define `U` as the set of all unique harmonized taxa -remaining in the abundance matrix for the selected analysis scope. The -complement for a signature `S` is `U \ S`; it is not the set of taxa appearing -in other BugSigDB signatures. - -Record the rank, taxonomy mapping or normalization rules, the size and -identifiers of `U`, and the matched and unmatched members of every signature. -Discard an empty signature before testing and record that reason. - -### Step 3: Apply the set-size bounds - -For each signature, let `S` be its unique matched taxa and let `k = |S|`. -Retain the signature only when: - -```text -2 <= k <= |U| - 1 -``` - -The lower bound excludes singleton signatures, for which the protocol does not -define a stable set-level balance. The upper bound guarantees a non-empty -complement. If `|U| < 3`, no signature can be tested. Remove duplicate -signature members before calculating `k`; do not count repeated BugSigDB rows as -distinct taxa. - -Record the original member count, matched member count, duplicate count, -unmatched members, `|U|`, the final set size, and the exclusion reason for every -discarded signature. Do not change the bounds after looking at the results. - -### Step 4: Calculate competitive balance scores and sample-level inference - -For every retained signature `S` and sample, calculate the CBEA competitive -balance using the positive abundances: - -```text -score(S, sample) = - sqrt(|S| * |U \ S| / |U|) * - (mean(log(abundance in S)) - mean(log(abundance in U \ S))) -``` - -This is the isometric log-ratio contrast between the signature subcomposition -and its complement. A positive score means that the signature taxa have a -higher geometric-mean abundance than the complement in that sample; a negative -score means the reverse. Do not replace this contrast with a sum or arithmetic -mean of signature abundances. - -Estimate the empirical null by permuting the taxon labels within each sample -`n_perm = 100` times while preserving the sample's abundance values and the -signature size. Use the recorded random seed. Fit the specified normal -distribution to the permutation scores and use the resulting sample-level -null inference. Apply the variance-inflation adjustment for inter-taxa -correlation. Return the CBEA score, the unadjusted p-value, and the adjusted -p-value for every retained signature and sample. - -The null hypothesis is competitive: within a sample, the taxa in the signature -have the same balance-level abundance as the measured taxa outside the -signature. It is not a null that every signature taxon has no association with -the sample condition, and it is not a test that every member of the signature -is individually differentially abundant. - -Record the CBEA implementation and version, all parameters, the random seed, -the number of successful permutations, and any signature-sample result that -could not be computed. - -### Step 5: Adjust across signatures and report the results - -Within each execution (the pooled table or one per-dataset table), apply the -Benjamini-Hochberg procedure to the complete family of unadjusted p-values for -the retained signatures. Use the same family definition for every sample in -that execution and do not adjust only the signatures that were significant -before correction. Mark a signature-sample result significant when its adjusted -p-value is less than 0.05. - -Produce a result table containing the dataset or pooled-scope identifier, -sample identifier, signature identifier, matched taxa, complement size, CBEA -score, unadjusted p-value, adjusted p-value, and significance decision. Also -produce an execution manifest containing the rank, universe, set-size bounds, -preprocessing description, inference parameters, exclusion counts, and -software versions. - -## Notes - -The CBEA paper describes this method as a single-sample taxon-set enrichment -method based on the competitive null hypothesis and isometric log-ratio -transformation. The BugSigDB paper's `Figure3.Rmd` supplies the application -context for running CBEA on pooled and individual microbiome datasets, but it -does not replace the method's primary citation. - -This protocol requires positive compositional abundances because the balance -uses logarithms. The choice of zero replacement, pseudocount, normalization, -taxonomic database, and differential-abundance input is upstream and must be -specified separately. The `0.5` addition shown in the application vignette is -not silently prescribed here; if it is used, record its scale and rationale -before running this protocol. - -A significant result licenses only the conclusion that the signature's -within-sample balance is unusual under the declared competitive null and -preprocessing. It does not establish causality, biological activity, disease -mechanism, differential abundance of every member, or generalization beyond -the analyzed samples. - -This protocol does not produce the abundance input, retrieve or curate -BugSigDB signatures, compare CBEA with ORA or PADOG, perform downstream -between-group differential analysis of the CBEA scores, or interpret a -signature's biological meaning. Those operations require separate protocols. - -## History & Reviews - - -### Version 1.0.0 (2026-10-14) - -#### Changes -- Initial protocol creation. -- Specified taxonomy harmonization, background universe, set-size bounds, - competitive balance calculation, permutation inference, multiple-testing - adjustment, and pooled versus per-dataset execution. - -#### Reviews -*No reviews yet.* From 530460324f54f66f7cd0de50bdf19f193a010d44 Mon Sep 17 00:00:00 2001 From: Otto Date: Wed, 14 Oct 2026 22:54:44 -0400 Subject: [PATCH 3/9] Add CBEA BugSigDB enrichment protocol --- .../cbea-bugsigdb-enrichment/protocol.md | 222 ++++++++++++++++++ 1 file changed, 222 insertions(+) create mode 100644 protocols/cbea-bugsigdb-enrichment/protocol.md diff --git a/protocols/cbea-bugsigdb-enrichment/protocol.md b/protocols/cbea-bugsigdb-enrichment/protocol.md new file mode 100644 index 0000000..63f7ace --- /dev/null +++ b/protocols/cbea-bugsigdb-enrichment/protocol.md @@ -0,0 +1,222 @@ +--- +name: "cbea-bugsigdb-enrichment" +description: "Test harmonized BugSigDB microbial signatures for taxonomic enrichment with competitive balances of compositional abundances." +version: "1.0.0" +authors: + - name: "Otto Infield-Harm" + orcid: "0009-0009-8629-6124" + - name: "Levi Waldron" + orcid: "0000-0003-2725-0694" +date: "2026-10-14" +status: draft +type: "atomic" + +artifact_doi: ~ +collection_doi: "10.5281/zenodo.22731694" +protocol_citation: "10.1371/journal.pcbi.1010091" +method_origin_citation: "10.1371/journal.pcbi.1010091" +license: "CC-BY-4.0" +protocols_used: [] +upstream_repositories: + - "https://github.com/qpmnguyen/CBEA" + - "https://github.com/waldronlab/BugSigDBPaper" +database_urls: + - "https://bugsigdbr.bioconductor.org/" +key_packages: + - "CBEA" + - "bugsigdbr" + - "BiocSet" +category: "Statistical Analysis" +tags: [enrichment, microbe-sets, cbea, bugsigdb, compositional-data] +--- + +# CBEA Enrichment of BugSigDB Microbial Signatures + +This protocol calculates competitive balance enrichment scores and sample-level +significance for BugSigDB microbial signatures. For each signature and sample, +the score contrasts the geometric mean abundance of the signature taxa with the +geometric mean abundance of all other taxa in the declared background universe. + +## Materials + +- **Abundance data** + - A taxa-by-sample matrix of strictly positive relative abundances, or another + positive compositional representation whose scale and construction are + recorded. Counts must be converted to a positive compositional representation + before this protocol begins. + - The matrix must have unique taxon and sample identifiers. This protocol does + not define sequencing quality control, normalization, zero replacement, or + the choice of an upstream transformation. Record those decisions and the + resulting abundance scale. +- **BugSigDB signatures** + - A collection of signatures, each identified by a stable signature identifier + and containing taxa at a declared taxonomic rank. + - Obtain the signatures from BugSigDB or a versioned export and record the + release, retrieval date, source URL, and any filters used to select them. +- **Parameters** + - `rank`: exactly one taxonomic rank shared by the abundance table and all + signatures in an execution. + - `analysis_scope`: exactly one of `pooled` or `per_dataset`. + - `min_set_size`: 2 matched taxa. + - `max_set_size`: `|U| - 1`, where `U` is the execution's harmonized + background universe. + - `n_perm`: 100 permutations for the empirical null distribution. + - `distribution`: normal parametric fit to the permutation scores. + - `variance_inflation_adjustment`: enabled to account for inter-taxa + correlation. + - `multiple_testing_method`: Benjamini-Hochberg false-discovery-rate + adjustment across all retained signatures tested in the same execution. + - `significance_threshold`: adjusted p-value less than 0.05. + - `random_seed`: a recorded integer seed used for permutation reproducibility. + +## Steps + +### Step 1: Confirm the abundance matrix and analysis scope + +Verify that the abundance matrix is taxa by sample, that identifiers are unique, +and that every value supplied to CBEA is finite and strictly positive. Reject +the input and report the failed check if any value is zero, negative, missing, +or non-finite; do not silently add a pseudocount or replace a value. A +pseudocount or zero-replacement method must be selected and applied before this +protocol, with its value or algorithm recorded. + +Select `pooled` or `per_dataset` before examining enrichment results. Use +`pooled` only when datasets have compatible taxonomic processing, abundance +scales, and sample-level metadata. In pooled mode, combine the compatible +datasets and define one universe across the combined table. In per-dataset +mode, keep datasets separate and repeat Steps 2 through 5 independently for +each dataset, defining a separate universe and set of tested signatures for +each one. Do not treat a pooled analysis as an aggregation of independent +dataset results. + +Record the number of datasets, samples, taxa, the abundance representation, +the upstream preprocessing, the selected scope, and the random seed. + +### Step 2: Harmonize taxonomy and define the background universe + +Choose one `rank`, such as species or genus, before matching signatures. Convert +the taxon identifiers in the abundance matrix and the BugSigDB signatures to +that rank using the same naming convention and taxonomy mapping. Do not infer a +lower-rank identity from a higher-rank name, and do not combine identifiers +whose rank is ambiguous. + +For each signature, retain only members that match a taxon in the abundance +matrix after harmonization. Record every unmatched signature member and its +reason for exclusion. Define `U` as the set of all unique harmonized taxa +remaining in the abundance matrix for the selected analysis scope. The +complement for a signature `S` is `U \ S`; it is not the set of taxa appearing +in other BugSigDB signatures. + +Record the rank, taxonomy mapping or normalization rules, the size and +identifiers of `U`, and the matched and unmatched members of every signature. +Discard an empty signature before testing and record that reason. + +### Step 3: Apply the set-size bounds + +For each signature, let `S` be its unique matched taxa and let `k = |S|`. +Retain the signature only when: + +```text +2 <= k <= |U| - 1 +``` + +The lower bound excludes singleton signatures, for which the protocol does not +define a stable set-level balance. The upper bound guarantees a non-empty +complement. If `|U| < 3`, no signature can be tested. Remove duplicate +signature members before calculating `k`; do not count repeated BugSigDB rows as +distinct taxa. + +Record the original member count, matched member count, duplicate count, +unmatched members, `|U|`, the final set size, and the exclusion reason for every +discarded signature. Do not change the bounds after looking at the results. + +### Step 4: Calculate competitive balance scores and sample-level inference + +For every retained signature `S` and sample, calculate the CBEA competitive +balance using the positive abundances: + +```text +score(S, sample) = + sqrt(|S| * |U \ S| / |U|) * + (mean(log(abundance in S)) - mean(log(abundance in U \ S))) +``` + +This is the isometric log-ratio contrast between the signature subcomposition +and its complement. A positive score means that the signature taxa have a +higher geometric-mean abundance than the complement in that sample; a negative +score means the reverse. Do not replace this contrast with a sum or arithmetic +mean of signature abundances. + +Estimate the empirical null by permuting the taxon labels within each sample +`n_perm = 100` times while preserving the sample's abundance values and the +signature size. Use the recorded random seed. Fit the specified normal +distribution to the permutation scores and use the resulting sample-level +null inference. Apply the variance-inflation adjustment for inter-taxa +correlation. Return the CBEA score, the unadjusted p-value, and the adjusted +p-value for every retained signature and sample. + +The null hypothesis is competitive: within a sample, the taxa in the signature +have the same balance-level abundance as the measured taxa outside the +signature. It is not a null that every signature taxon has no association with +the sample condition, and it is not a test that every member of the signature +is individually differentially abundant. + +Record the CBEA implementation and version, all parameters, the random seed, +the number of successful permutations, and any signature-sample result that +could not be computed. + +### Step 5: Adjust across signatures and report the results + +Within each execution (the pooled table or one per-dataset table), apply the +Benjamini-Hochberg procedure to the complete family of unadjusted p-values for +the retained signatures. Use the same family definition for every sample in +that execution and do not adjust only the signatures that were significant +before correction. Mark a signature-sample result significant when its adjusted +p-value is less than 0.05. + +Produce a result table containing the dataset or pooled-scope identifier, +sample identifier, signature identifier, matched taxa, complement size, CBEA +score, unadjusted p-value, adjusted p-value, and significance decision. Also +produce an execution manifest containing the rank, universe, set-size bounds, +preprocessing description, inference parameters, exclusion counts, and +software versions. + +## Notes + +The CBEA paper describes this method as a single-sample taxon-set enrichment +method based on the competitive null hypothesis and isometric log-ratio +transformation. The BugSigDB paper's `Figure3.Rmd` supplies the application +context for running CBEA on pooled and individual microbiome datasets, but it +does not replace the method's primary citation. + +This protocol requires positive compositional abundances because the balance +uses logarithms. The choice of zero replacement, pseudocount, normalization, +taxonomic database, and differential-abundance input is upstream and must be +specified separately. The `0.5` addition shown in the application vignette is +not silently prescribed here; if it is used, record its scale and rationale +before running this protocol. + +A significant result licenses only the conclusion that the signature's +within-sample balance is unusual under the declared competitive null and +preprocessing. It does not establish causality, biological activity, disease +mechanism, differential abundance of every member, or generalization beyond +the analyzed samples. + +This protocol does not produce the abundance input, retrieve or curate +BugSigDB signatures, compare CBEA with ORA or PADOG, perform downstream +between-group differential analysis of the CBEA scores, or interpret a +signature's biological meaning. Those operations require separate protocols. + +## History & Reviews + + +### Version 1.0.0 (2026-10-14) + +#### Changes +- Initial protocol creation. +- Specified taxonomy harmonization, background universe, set-size bounds, + competitive balance calculation, permutation inference, multiple-testing + adjustment, and pooled versus per-dataset execution. + +#### Reviews +*No reviews yet.* From e74be6f2cd0b690eced77f6c372c6206e4de8322 Mon Sep 17 00:00:00 2001 From: Levi Waldron Date: Sat, 17 Oct 2026 17:04:18 -0400 Subject: [PATCH 4/9] Generalize CBEA protocol to any microbial signatures Rename to cbea-signature-enrichment with BugSigDB as the worked example. Address Copilot review: define aggregation by summing relative abundances with warnings for excluded rows, and specify the pooled cross-sample null, correlation adjustment, and upper-tail p-values by naming the CBEA functions. Drafted with Claude Code. --- .../cbea-bugsigdb-enrichment/protocol.md | 222 ------------ .../cbea-signature-enrichment/protocol.md | 338 ++++++++++++++++++ 2 files changed, 338 insertions(+), 222 deletions(-) delete mode 100644 protocols/cbea-bugsigdb-enrichment/protocol.md create mode 100644 protocols/cbea-signature-enrichment/protocol.md diff --git a/protocols/cbea-bugsigdb-enrichment/protocol.md b/protocols/cbea-bugsigdb-enrichment/protocol.md deleted file mode 100644 index 63f7ace..0000000 --- a/protocols/cbea-bugsigdb-enrichment/protocol.md +++ /dev/null @@ -1,222 +0,0 @@ ---- -name: "cbea-bugsigdb-enrichment" -description: "Test harmonized BugSigDB microbial signatures for taxonomic enrichment with competitive balances of compositional abundances." -version: "1.0.0" -authors: - - name: "Otto Infield-Harm" - orcid: "0009-0009-8629-6124" - - name: "Levi Waldron" - orcid: "0000-0003-2725-0694" -date: "2026-10-14" -status: draft -type: "atomic" - -artifact_doi: ~ -collection_doi: "10.5281/zenodo.22731694" -protocol_citation: "10.1371/journal.pcbi.1010091" -method_origin_citation: "10.1371/journal.pcbi.1010091" -license: "CC-BY-4.0" -protocols_used: [] -upstream_repositories: - - "https://github.com/qpmnguyen/CBEA" - - "https://github.com/waldronlab/BugSigDBPaper" -database_urls: - - "https://bugsigdbr.bioconductor.org/" -key_packages: - - "CBEA" - - "bugsigdbr" - - "BiocSet" -category: "Statistical Analysis" -tags: [enrichment, microbe-sets, cbea, bugsigdb, compositional-data] ---- - -# CBEA Enrichment of BugSigDB Microbial Signatures - -This protocol calculates competitive balance enrichment scores and sample-level -significance for BugSigDB microbial signatures. For each signature and sample, -the score contrasts the geometric mean abundance of the signature taxa with the -geometric mean abundance of all other taxa in the declared background universe. - -## Materials - -- **Abundance data** - - A taxa-by-sample matrix of strictly positive relative abundances, or another - positive compositional representation whose scale and construction are - recorded. Counts must be converted to a positive compositional representation - before this protocol begins. - - The matrix must have unique taxon and sample identifiers. This protocol does - not define sequencing quality control, normalization, zero replacement, or - the choice of an upstream transformation. Record those decisions and the - resulting abundance scale. -- **BugSigDB signatures** - - A collection of signatures, each identified by a stable signature identifier - and containing taxa at a declared taxonomic rank. - - Obtain the signatures from BugSigDB or a versioned export and record the - release, retrieval date, source URL, and any filters used to select them. -- **Parameters** - - `rank`: exactly one taxonomic rank shared by the abundance table and all - signatures in an execution. - - `analysis_scope`: exactly one of `pooled` or `per_dataset`. - - `min_set_size`: 2 matched taxa. - - `max_set_size`: `|U| - 1`, where `U` is the execution's harmonized - background universe. - - `n_perm`: 100 permutations for the empirical null distribution. - - `distribution`: normal parametric fit to the permutation scores. - - `variance_inflation_adjustment`: enabled to account for inter-taxa - correlation. - - `multiple_testing_method`: Benjamini-Hochberg false-discovery-rate - adjustment across all retained signatures tested in the same execution. - - `significance_threshold`: adjusted p-value less than 0.05. - - `random_seed`: a recorded integer seed used for permutation reproducibility. - -## Steps - -### Step 1: Confirm the abundance matrix and analysis scope - -Verify that the abundance matrix is taxa by sample, that identifiers are unique, -and that every value supplied to CBEA is finite and strictly positive. Reject -the input and report the failed check if any value is zero, negative, missing, -or non-finite; do not silently add a pseudocount or replace a value. A -pseudocount or zero-replacement method must be selected and applied before this -protocol, with its value or algorithm recorded. - -Select `pooled` or `per_dataset` before examining enrichment results. Use -`pooled` only when datasets have compatible taxonomic processing, abundance -scales, and sample-level metadata. In pooled mode, combine the compatible -datasets and define one universe across the combined table. In per-dataset -mode, keep datasets separate and repeat Steps 2 through 5 independently for -each dataset, defining a separate universe and set of tested signatures for -each one. Do not treat a pooled analysis as an aggregation of independent -dataset results. - -Record the number of datasets, samples, taxa, the abundance representation, -the upstream preprocessing, the selected scope, and the random seed. - -### Step 2: Harmonize taxonomy and define the background universe - -Choose one `rank`, such as species or genus, before matching signatures. Convert -the taxon identifiers in the abundance matrix and the BugSigDB signatures to -that rank using the same naming convention and taxonomy mapping. Do not infer a -lower-rank identity from a higher-rank name, and do not combine identifiers -whose rank is ambiguous. - -For each signature, retain only members that match a taxon in the abundance -matrix after harmonization. Record every unmatched signature member and its -reason for exclusion. Define `U` as the set of all unique harmonized taxa -remaining in the abundance matrix for the selected analysis scope. The -complement for a signature `S` is `U \ S`; it is not the set of taxa appearing -in other BugSigDB signatures. - -Record the rank, taxonomy mapping or normalization rules, the size and -identifiers of `U`, and the matched and unmatched members of every signature. -Discard an empty signature before testing and record that reason. - -### Step 3: Apply the set-size bounds - -For each signature, let `S` be its unique matched taxa and let `k = |S|`. -Retain the signature only when: - -```text -2 <= k <= |U| - 1 -``` - -The lower bound excludes singleton signatures, for which the protocol does not -define a stable set-level balance. The upper bound guarantees a non-empty -complement. If `|U| < 3`, no signature can be tested. Remove duplicate -signature members before calculating `k`; do not count repeated BugSigDB rows as -distinct taxa. - -Record the original member count, matched member count, duplicate count, -unmatched members, `|U|`, the final set size, and the exclusion reason for every -discarded signature. Do not change the bounds after looking at the results. - -### Step 4: Calculate competitive balance scores and sample-level inference - -For every retained signature `S` and sample, calculate the CBEA competitive -balance using the positive abundances: - -```text -score(S, sample) = - sqrt(|S| * |U \ S| / |U|) * - (mean(log(abundance in S)) - mean(log(abundance in U \ S))) -``` - -This is the isometric log-ratio contrast between the signature subcomposition -and its complement. A positive score means that the signature taxa have a -higher geometric-mean abundance than the complement in that sample; a negative -score means the reverse. Do not replace this contrast with a sum or arithmetic -mean of signature abundances. - -Estimate the empirical null by permuting the taxon labels within each sample -`n_perm = 100` times while preserving the sample's abundance values and the -signature size. Use the recorded random seed. Fit the specified normal -distribution to the permutation scores and use the resulting sample-level -null inference. Apply the variance-inflation adjustment for inter-taxa -correlation. Return the CBEA score, the unadjusted p-value, and the adjusted -p-value for every retained signature and sample. - -The null hypothesis is competitive: within a sample, the taxa in the signature -have the same balance-level abundance as the measured taxa outside the -signature. It is not a null that every signature taxon has no association with -the sample condition, and it is not a test that every member of the signature -is individually differentially abundant. - -Record the CBEA implementation and version, all parameters, the random seed, -the number of successful permutations, and any signature-sample result that -could not be computed. - -### Step 5: Adjust across signatures and report the results - -Within each execution (the pooled table or one per-dataset table), apply the -Benjamini-Hochberg procedure to the complete family of unadjusted p-values for -the retained signatures. Use the same family definition for every sample in -that execution and do not adjust only the signatures that were significant -before correction. Mark a signature-sample result significant when its adjusted -p-value is less than 0.05. - -Produce a result table containing the dataset or pooled-scope identifier, -sample identifier, signature identifier, matched taxa, complement size, CBEA -score, unadjusted p-value, adjusted p-value, and significance decision. Also -produce an execution manifest containing the rank, universe, set-size bounds, -preprocessing description, inference parameters, exclusion counts, and -software versions. - -## Notes - -The CBEA paper describes this method as a single-sample taxon-set enrichment -method based on the competitive null hypothesis and isometric log-ratio -transformation. The BugSigDB paper's `Figure3.Rmd` supplies the application -context for running CBEA on pooled and individual microbiome datasets, but it -does not replace the method's primary citation. - -This protocol requires positive compositional abundances because the balance -uses logarithms. The choice of zero replacement, pseudocount, normalization, -taxonomic database, and differential-abundance input is upstream and must be -specified separately. The `0.5` addition shown in the application vignette is -not silently prescribed here; if it is used, record its scale and rationale -before running this protocol. - -A significant result licenses only the conclusion that the signature's -within-sample balance is unusual under the declared competitive null and -preprocessing. It does not establish causality, biological activity, disease -mechanism, differential abundance of every member, or generalization beyond -the analyzed samples. - -This protocol does not produce the abundance input, retrieve or curate -BugSigDB signatures, compare CBEA with ORA or PADOG, perform downstream -between-group differential analysis of the CBEA scores, or interpret a -signature's biological meaning. Those operations require separate protocols. - -## History & Reviews - - -### Version 1.0.0 (2026-10-14) - -#### Changes -- Initial protocol creation. -- Specified taxonomy harmonization, background universe, set-size bounds, - competitive balance calculation, permutation inference, multiple-testing - adjustment, and pooled versus per-dataset execution. - -#### Reviews -*No reviews yet.* diff --git a/protocols/cbea-signature-enrichment/protocol.md b/protocols/cbea-signature-enrichment/protocol.md new file mode 100644 index 0000000..48aefe2 --- /dev/null +++ b/protocols/cbea-signature-enrichment/protocol.md @@ -0,0 +1,338 @@ +--- +name: "cbea-signature-enrichment" +description: "Test microbial signatures, such as those curated in BugSigDB, for taxonomic enrichment in each sample using competitive balances of compositional abundances (CBEA)." +version: "1.0.0" +authors: + - name: "Otto Infield-Harm" + orcid: "0009-0009-8629-6124" + - name: "Levi Waldron" + orcid: "0000-0003-2725-0694" +date: "2026-10-17" +status: draft +type: "atomic" + +artifact_doi: ~ +collection_doi: "10.5281/zenodo.22731694" +protocol_citation: "10.1371/journal.pcbi.1010091" +method_origin_citation: "10.1371/journal.pcbi.1010091" +license: "CC-BY-4.0" +protocols_used: [] +upstream_repositories: + - "https://github.com/qpmnguyen/CBEA" + - "https://github.com/waldronlab/BugSigDBPaper" +database_urls: + - "https://bugsigdb.org/" +key_packages: + - "CBEA" + - "BiocSet" + - "bugsigdbr" +category: "Statistical Analysis" +tags: [enrichment, microbe-sets, cbea, bugsigdb, compositional-data] +--- + +# CBEA Enrichment of Microbial Signatures + +This protocol calculates a competitive balance enrichment score and a +sample-level p-value for each microbial signature in each sample. A signature +is any named set of taxa at one taxonomic rank. The signatures curated in +BugSigDB are the worked example throughout, but the protocol applies to any +collection of signatures: sets from another database, taxa reported by a +previous study, or sets defined by the analyst before seeing the data. For each +signature and sample, the score contrasts the geometric mean relative abundance +of the signature taxa with the geometric mean relative abundance of all other +taxa in the declared background universe. + +## Materials + +- **Abundance data** + - A taxa-by-sample matrix of strictly positive relative abundances: each + column sums to a constant such as 1 or 100 before zero replacement, and no + value is zero after it. Counts must be converted to relative abundances, + and zeros must be handled, before this protocol begins. + - The matrix must be on the linear proportion scale. Do not supply + log-transformed, centered log-ratio (CLR), or other transformed values; the + balance in Step 4 takes logarithms itself, and Step 2 may sum rows, which + is meaningful only on the linear scale. + - Taxon and sample identifiers must be unique. Taxon identifiers must carry + the rank of each row, either in the identifier itself (for example a + lineage string) or in an accompanying taxonomy table. + - This protocol does not define sequencing quality control, normalization, + or zero replacement. Record those decisions and the resulting abundance + scale. +- **Signatures** + - A collection of signatures, each identified by a stable signature + identifier and containing taxa with a declared taxonomic rank. Any format + that preserves the identifier and the member taxa is acceptable, such as a + named list, a GMT file, or a `BiocSet`. + - Record the source, version or release, retrieval date, and any filters used + to select signatures. + - **BugSigDB example.** Import a versioned BugSigDB release with + `bugsigdbr::importBugSigDB(version = )` and extract + signatures at the chosen rank with + `bugsigdbr::getSignatures(tax.id.type = "taxname", tax.level = rank, + exact.tax.level = FALSE)`. The last argument maps members curated below + `rank` up to `rank`, which is the mapping Step 2 permits. The release + identifier is the version to record. +- **Software** + - `CBEA` (https://github.com/qpmnguyen/CBEA), the Bioconductor implementation + of the method. Function names below refer to this package; an + implementation in another language must reproduce the operations named in + Step 4, not only the function's description. +- **Parameters** + - `rank`: exactly one taxonomic rank shared by the abundance table and all + signatures in an execution. + - `analysis_scope`: exactly one of `pooled` or `per_dataset`. + - `min_set_size`: 2 matched taxa. + - `max_set_size`: `|U| - 1`, where `U` is the execution's background + universe. + - `n_perm`: 100 random taxon subsets for the null distribution. + - `distribution`: `norm`, a normal distribution fitted to the null scores. + `mnorm`, the two-component normal mixture from the same paper, is the only + permitted alternative and must be recorded if chosen. + - `correlation_adjustment`: enabled (`adj = TRUE` in `CBEA::cbea()`). + - `multiple_testing_method`: Benjamini-Hochberg false-discovery-rate + adjustment, with the family defined in Step 5. + - `significance_threshold`: adjusted p-value less than 0.05. + - `random_seed`: a recorded integer seed used for the random taxon subsets. + +## Steps + +### Step 1: Confirm the abundance matrix and analysis scope + +Verify that the abundance matrix is taxa by sample, that identifiers are +unique, that the values are relative abundances on the linear scale, and that +every value is finite and strictly positive. Reject the input and report the +failed check if any value is zero, negative, missing, or non-finite; do not +silently add a pseudocount or replace a value. A pseudocount or +zero-replacement method must be selected and applied before this protocol, +with its value or algorithm recorded. Note that `CBEA::cbea()` adds a +pseudocount of `1e-5` with only a warning when it finds zeros; this protocol's +check must run before that function so the silent replacement never happens. + +Select `pooled` or `per_dataset` before examining enrichment results. Use +`pooled` only when datasets have compatible taxonomic processing, abundance +scales, and sample-level metadata. In pooled mode, combine the compatible +datasets and define one universe across the combined table. In per-dataset +mode, keep datasets separate and repeat Steps 2 through 5 independently for +each dataset, defining a separate universe and set of tested signatures for +each one. Do not treat a pooled analysis as an aggregation of independent +dataset results. + +Record the number of datasets, samples, taxa, the abundance representation, +the upstream preprocessing including the zero-replacement method, the selected +scope, and the random seed. + +### Step 2: Harmonize taxonomy and define the background universe + +Choose one `rank`, such as species or genus, before matching signatures. +Express the taxon identifiers in the abundance matrix and in the signatures at +that rank using the same naming convention and the same taxonomy. The vignette +this protocol was extracted from used NCBI taxonomy names at the genus and +species ranks on both sides. + +**Abundance rows.** Prefer an abundance table already at `rank`. When the +table is at a lower rank, aggregate it to `rank` by summing the relative +abundances of all rows that map to the same `rank` taxon, within each sample. +Summing is valid only because the input is on the linear proportion scale. Do +not relabel rows without summing: two species rows relabeled to one genus +would leave duplicate identifiers and would be scored as two members of the +complement. Exclude rows that cannot be mapped to a single taxon at `rank`, +including rows whose classification stops above `rank` and rows whose +identifier is ambiguous, and emit a warning naming each excluded row and the +abundance it carried, because such rows may indicate incompatible or +incorrect taxonomic labeling. After aggregation and exclusion, confirm again +that identifiers are unique and that every value is strictly positive. Zero +replacement should have been applied to the table at `rank`; if it was +applied before aggregation, record that, since the summed pseudocounts then +differ between taxa. + +**Signature members.** Map each member to `rank` using the same taxonomy. A +member curated below `rank` maps up to its ancestor at `rank` (a species to +its genus). A member curated above `rank`, or one with no identity at `rank`, +is unmappable: exclude it and record it. Do not infer a lower-rank identity +from a higher-rank name. Then retain only members that match a taxon in the +harmonized abundance matrix. Record every unmatched member and its reason for +exclusion. In the `CBEA` package, `BiocSet::filter_elementset()` performs the +matching step; `CBEA::cbea()` would also drop unmatched members silently, but +the protocol requires them to be recorded, so match explicitly. + +**Universe.** Define `U` as the set of all unique harmonized taxa remaining in +the abundance matrix for the selected analysis scope. This is what +`CBEA::get_raw_score()` uses: the complement of a signature `S` is every other +column of the matrix, `U \ S`. It is not the set of taxa appearing in other +signatures, and it is not restricted to taxa appearing in any signature. + +Record the rank, the taxonomy and mapping rules, the number of abundance rows +aggregated and excluded, the size and identifiers of `U`, and the matched and +unmatched members of every signature. Discard an empty signature before +testing and record that reason. + +### Step 3: Apply the set-size bounds + +For each signature, let `S` be its unique matched taxa and let `k = |S|`. +Retain the signature only when: + +```text +2 <= k <= |U| - 1 +``` + +The lower bound excludes singleton signatures, for which the protocol does +not define a stable set-level balance. The upper bound guarantees a non-empty +complement. If `|U| < 3`, no signature can be tested. Remove duplicate +signature members before calculating `k`; do not count repeated source rows +as distinct taxa. + +Record the original member count, matched member count, duplicate count, +unmatched members, `|U|`, the final set size, and the exclusion reason for +every discarded signature. Do not change the bounds after looking at the +results. + +### Step 4: Calculate competitive balance scores and sample-level inference + +For every retained signature `S` and sample, calculate the CBEA competitive +balance from the positive relative abundances: + +```text +score(S, sample) = + sqrt(|S| * |U \ S| / |U|) * + (mean(log(abundance in S)) - mean(log(abundance in U \ S))) +``` + +This is the isometric log-ratio contrast between the signature subcomposition +and its complement, computed by `CBEA::get_raw_score()`. A positive score +means that the signature taxa have a higher geometric-mean abundance than the +complement in that sample; a negative score means the reverse. Do not replace +this contrast with a sum or arithmetic mean of signature abundances. + +Construct the null distribution for each signature as `CBEA::fit_scores()` +does, pooling across samples rather than building a separate null per sample: + +1. Draw `n_perm = 100` random subsets of `U`, each of size `k`, without + replacement within a subset, using the recorded seed. In the `CBEA` + package, the subsets are drawn inside a `BiocParallel` backend. When no + backend is supplied, the package seeds a serial backend from the current + R random stream, so calling `set.seed(random_seed)` immediately before + `CBEA::cbea()` reproduces the subsets. When a backend is supplied, its + `RNGseed` must be set to `random_seed`, as in the call below; a backend + without `RNGseed` is not reproducible. +2. Apply every subset to every sample with the formula above. This yields + `n_perm` times the number of samples null scores for the signature, + pooled into one vector. +3. Fit the chosen `distribution` to the pooled null scores by maximum + likelihood (`CBEA::estimate_distr()`, which calls + `fitdistrplus::fitdist()` for `norm`). +4. With `correlation_adjustment` enabled, fit the same distribution to the + observed scores of the signature across all samples, then combine the two + fits (`CBEA::combine_distr()`): for `norm`, the final null keeps the mean + of the permutation fit and takes the standard deviation from the fit to + the observed scores. This widens the null to account for correlation among + taxa, which the random subsets do not preserve. Because the observed fit + needs several samples, do not run this protocol on a single sample or on a + dataset too small to fit the distribution; record the number of samples + per execution. +5. Convert each observed score to an upper-tail p-value under the final null, + `p = 1 - F(score)` where `F` is the fitted cumulative distribution + function (`CBEA::scale_scores()` with `method = "pval"`). A small p-value + therefore means the signature is enriched, not depleted. Depletion is not + tested by this protocol; if it is of interest, record that the lower tail + was used instead, for every result. + +In the `CBEA` package the call that performs Steps 4.1 through 4.5 is: + +```r +CBEA::cbea(obj = abundance, set = signatures, output = "pval", + abund_values = "relative_abundance", distr = "norm", + adj = TRUE, n_perm = 100, + parallel_backend = BiocParallel::SerialParam(RNGseed = random_seed)) +``` + +where `abundance` is a `TreeSummarizedExperiment` whose named assay holds the +harmonized, strictly positive relative abundances, and `signatures` is the +`BiocSet` of retained signatures. The package documentation describes +`n_perm` as bootstrap resamples; the implementation draws random taxon +subsets as described above, and the protocol follows the implementation. + +The null hypothesis is competitive: within a sample, the taxa in the signature +have the same balance-level abundance as a random set of the same size drawn +from the measured taxa outside the signature. It is not a null that every +signature taxon has no association with the sample condition, and it is not a +test that every member of the signature is individually differentially +abundant. + +Record the CBEA implementation and version, all parameters, the random seed, +the number of samples, the number of null scores per signature, the fitted +null parameters per signature, and any signature-sample result that could not +be computed. + +### Step 5: Adjust across signatures and report the results + +Within each execution (the pooled table or one per-dataset table), apply the +Benjamini-Hochberg procedure separately within each sample, taking the family +to be the unadjusted p-values of all retained signatures in that sample. Use +the same family definition for every sample and do not adjust only the +signatures that were significant before correction. Mark a signature-sample +result significant when its adjusted p-value is less than 0.05. Neither the +CBEA paper nor the BugSigDB application adjusts the sample-level p-values; +the adjustment is a requirement of this protocol so that a set of per-sample +calls has a declared false-discovery rate. + +Produce a result table containing the dataset or pooled-scope identifier, +sample identifier, signature identifier, matched taxa, complement size, CBEA +score, unadjusted p-value, adjusted p-value, and significance decision. Also +produce an execution manifest containing the rank, universe, set-size bounds, +preprocessing description, aggregation and exclusion counts from Step 2, +inference parameters, number of samples, and software versions. + +## Notes + +The CBEA paper describes this method as a single-sample taxon-set enrichment +method based on the competitive null hypothesis and the isometric log-ratio +transformation. The BugSigDB paper's `Figure3.Rmd` supplies the application +context for running CBEA on pooled and individual microbiome datasets with +BugSigDB signatures, but it does not replace the method's primary citation, +and nothing in this protocol depends on the signatures coming from BugSigDB. + +This protocol requires positive relative abundances because the balance uses +logarithms and because aggregation to `rank` sums rows. The choice of zero +replacement, pseudocount, normalization, and taxonomic database is upstream +and must be specified separately. The `0.5` added to percentage-scale +relative abundances in the BugSigDB vignette is one such choice, not a +prescription of this protocol; if it is used, record its scale and rationale +before running this protocol. + +The vignette requested `output = "zscore"` from `CBEA::cbea()` and passed the +per-sample z-scores to a between-group linear model. That downstream analysis +is out of scope here, and this protocol requests `output = "pval"` so that the +sample-level inference is the deliverable. The z-score is the observed score +standardized by the final null's mean and standard deviation, so it is +available from the same fit if a downstream protocol needs it. + +A significant result licenses only the conclusion that the signature's +within-sample balance is unusually high under the declared competitive null +and preprocessing. It does not establish causality, biological activity, +disease mechanism, differential abundance of every member, or generalization +beyond the analyzed samples. + +This protocol does not produce the abundance input, retrieve or curate +signatures, compare CBEA with over-representation analysis or PADOG, perform +downstream between-group differential analysis of the CBEA scores, or +interpret a signature's biological meaning. Those operations require separate +protocols. + +## History & Reviews + + +### Version 1.0.0 (2026-10-17) + +#### Changes +- Initial protocol creation. +- Specified taxonomy harmonization with aggregation by summing relative + abundances, the background universe, set-size bounds, the competitive + balance, the pooled cross-sample null and its correlation adjustment, + upper-tail p-values, per-sample Benjamini-Hochberg adjustment, and pooled + versus per-dataset execution. +- Generalized from BugSigDB signatures to any collection of microbial + signatures, with BugSigDB retained as the worked example. + +#### Reviews +*No reviews yet.* From 5b45eb026d885088ef670defdad627910bb98c1b Mon Sep 17 00:00:00 2001 From: Levi Waldron Date: Sat, 17 Oct 2026 17:08:45 -0400 Subject: [PATCH 5/9] Fix seed handling and null statement after Copilot review The exported cbea() methods do not forward parallel_backend, so seed with set.seed() instead. State the null as random subsets drawn from the whole universe, matching fit_scores(). Drafted with Claude Code. --- .../cbea-signature-enrichment/protocol.md | 20 +++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/protocols/cbea-signature-enrichment/protocol.md b/protocols/cbea-signature-enrichment/protocol.md index 48aefe2..1d52719 100644 --- a/protocols/cbea-signature-enrichment/protocol.md +++ b/protocols/cbea-signature-enrichment/protocol.md @@ -209,12 +209,11 @@ does, pooling across samples rather than building a separate null per sample: 1. Draw `n_perm = 100` random subsets of `U`, each of size `k`, without replacement within a subset, using the recorded seed. In the `CBEA` - package, the subsets are drawn inside a `BiocParallel` backend. When no - backend is supplied, the package seeds a serial backend from the current - R random stream, so calling `set.seed(random_seed)` immediately before - `CBEA::cbea()` reproduces the subsets. When a backend is supplied, its - `RNGseed` must be set to `random_seed`, as in the call below; a backend - without `RNGseed` is not reproducible. + package (version 0.99.4), the exported `cbea()` methods accept a + `parallel_backend` argument but do not forward it to the internal + fitter, which seeds a serial backend from the current R random stream. + Call `set.seed(random_seed)` immediately before `CBEA::cbea()` and do + not rely on the `parallel_backend` argument for reproducibility. 2. Apply every subset to every sample with the formula above. This yields `n_perm` times the number of samples null scores for the signature, pooled into one vector. @@ -240,10 +239,10 @@ does, pooling across samples rather than building a separate null per sample: In the `CBEA` package the call that performs Steps 4.1 through 4.5 is: ```r +set.seed(random_seed) CBEA::cbea(obj = abundance, set = signatures, output = "pval", abund_values = "relative_abundance", distr = "norm", - adj = TRUE, n_perm = 100, - parallel_backend = BiocParallel::SerialParam(RNGseed = random_seed)) + adj = TRUE, n_perm = 100) ``` where `abundance` is a `TreeSummarizedExperiment` whose named assay holds the @@ -253,8 +252,9 @@ harmonized, strictly positive relative abundances, and `signatures` is the subsets as described above, and the protocol follows the implementation. The null hypothesis is competitive: within a sample, the taxa in the signature -have the same balance-level abundance as a random set of the same size drawn -from the measured taxa outside the signature. It is not a null that every +have the same balance against their complement as a random set of the same +size drawn from the whole measured universe `U`, signature members included, +has against its own complement. It is not a null that every signature taxon has no association with the sample condition, and it is not a test that every member of the signature is individually differentially abundant. From 4e6c76acf4b9ff5d8d13ca07c8b9fb9014d21411 Mon Sep 17 00:00:00 2001 From: Levi Waldron Date: Sat, 17 Oct 2026 17:13:45 -0400 Subject: [PATCH 6/9] Fix BugSigDB and cbea() snippets after Copilot review Pass the imported object to getSignatures() and use the selected distribution in the cbea() call. Drafted with Claude Code. --- protocols/cbea-signature-enrichment/protocol.md | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/protocols/cbea-signature-enrichment/protocol.md b/protocols/cbea-signature-enrichment/protocol.md index 1d52719..e10d472 100644 --- a/protocols/cbea-signature-enrichment/protocol.md +++ b/protocols/cbea-signature-enrichment/protocol.md @@ -67,10 +67,10 @@ taxa in the declared background universe. - Record the source, version or release, retrieval date, and any filters used to select signatures. - **BugSigDB example.** Import a versioned BugSigDB release with - `bugsigdbr::importBugSigDB(version = )` and extract - signatures at the chosen rank with - `bugsigdbr::getSignatures(tax.id.type = "taxname", tax.level = rank, - exact.tax.level = FALSE)`. The last argument maps members curated below + `bsdb <- bugsigdbr::importBugSigDB(version = )` + and extract signatures at the chosen rank with + `bugsigdbr::getSignatures(bsdb, tax.id.type = "taxname", tax.level = + rank, exact.tax.level = FALSE)`. The last argument maps members curated below `rank` up to `rank`, which is the mapping Step 2 permits. The release identifier is the version to record. - **Software** @@ -241,13 +241,14 @@ In the `CBEA` package the call that performs Steps 4.1 through 4.5 is: ```r set.seed(random_seed) CBEA::cbea(obj = abundance, set = signatures, output = "pval", - abund_values = "relative_abundance", distr = "norm", + abund_values = "relative_abundance", distr = distribution, adj = TRUE, n_perm = 100) ``` where `abundance` is a `TreeSummarizedExperiment` whose named assay holds the harmonized, strictly positive relative abundances, and `signatures` is the -`BiocSet` of retained signatures. The package documentation describes +`BiocSet` of retained signatures, and `distribution` is `"norm"` unless +`"mnorm"` was selected and recorded. The package documentation describes `n_perm` as bootstrap resamples; the implementation draws random taxon subsets as described above, and the protocol follows the implementation. From b41683f41404010184e6f5af981a249994ff7b3e Mon Sep 17 00:00:00 2001 From: Levi Waldron Date: Sat, 17 Oct 2026 17:18:19 -0400 Subject: [PATCH 7/9] Record review by Levi Waldron Drafted with Claude Code. --- protocols/cbea-signature-enrichment/protocol.md | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/protocols/cbea-signature-enrichment/protocol.md b/protocols/cbea-signature-enrichment/protocol.md index e10d472..a104da7 100644 --- a/protocols/cbea-signature-enrichment/protocol.md +++ b/protocols/cbea-signature-enrichment/protocol.md @@ -7,6 +7,12 @@ authors: orcid: "0009-0009-8629-6124" - name: "Levi Waldron" orcid: "0000-0003-2725-0694" +reviews: + - name: "Levi Waldron" + orcid: "0000-0003-2725-0694" + date: "2026-10-17" + protocol_version: "1.0.0" + status: "approved" date: "2026-10-17" status: draft type: "atomic" @@ -336,4 +342,8 @@ protocols. signatures, with BugSigDB retained as the worked example. #### Reviews -*No reviews yet.* + +**Review by Levi Waldron ([0000-0003-2725-0694](https://orcid.org/0000-0003-2725-0694))** +- **Date:** 2026-10-17 +- **Status:** `approved` +- **Notes:** Converted the protocol from a BugSigDB-specific protocol to a general CBEA protocol for any collection of microbial signatures, keeping BugSigDB as the worked example. Added code snippets naming the `bugsigdbr` and `CBEA` functions that perform each step. Restricted input to linear-scale relative abundance so that aggregation to the target rank can sum rows, with excluded rows logged as warnings. Specified the pooled cross-sample null, the correlation adjustment, upper-tail p-values, and seeding by `set.seed()` because the package does not forward its `parallel_backend` argument. Defined the multiple-testing family as all retained signatures within a sample. From 26dd1ae4e252fb1279f4053bdc11c4b87d6c3b3e Mon Sep 17 00:00:00 2001 From: Levi Waldron Date: Sat, 17 Oct 2026 17:20:39 -0400 Subject: [PATCH 8/9] Restrict null to norm and define pooled-mode taxon merging The mnorm correlation adjustment in CBEA 0.99.4 computes the mixture standard deviation without squaring component terms, so only norm is permitted. Pooled mode now merges datasets on the union of taxa with zero replacement applied to the combined table. Drafted with Claude Code. --- .../cbea-signature-enrichment/protocol.md | 43 +++++++++++++------ 1 file changed, 30 insertions(+), 13 deletions(-) diff --git a/protocols/cbea-signature-enrichment/protocol.md b/protocols/cbea-signature-enrichment/protocol.md index a104da7..2af28d3 100644 --- a/protocols/cbea-signature-enrichment/protocol.md +++ b/protocols/cbea-signature-enrichment/protocol.md @@ -93,8 +93,8 @@ taxa in the declared background universe. universe. - `n_perm`: 100 random taxon subsets for the null distribution. - `distribution`: `norm`, a normal distribution fitted to the null scores. - `mnorm`, the two-component normal mixture from the same paper, is the only - permitted alternative and must be recorded if chosen. + The two-component mixture (`mnorm`) described in the same paper is not + permitted; see Notes. - `correlation_adjustment`: enabled (`adj = TRUE` in `CBEA::cbea()`). - `multiple_testing_method`: Benjamini-Hochberg false-discovery-rate adjustment, with the family defined in Step 5. @@ -118,7 +118,12 @@ check must run before that function so the silent replacement never happens. Select `pooled` or `per_dataset` before examining enrichment results. Use `pooled` only when datasets have compatible taxonomic processing, abundance scales, and sample-level metadata. In pooled mode, combine the compatible -datasets and define one universe across the combined table. In per-dataset +datasets on the union of their taxon identifiers, so that a taxon absent +from one dataset is zero in that dataset's samples, then apply the upstream +zero replacement to the combined table rather than to each dataset +separately, and repeat the checks above on the combined table. Define one +universe across the combined table. Record the number of taxa in each +dataset and in the union. In per-dataset mode, keep datasets separate and repeat Steps 2 through 5 independently for each dataset, defining a separate universe and set of tested signatures for each one. Do not treat a pooled analysis as an aggregation of independent @@ -223,12 +228,12 @@ does, pooling across samples rather than building a separate null per sample: 2. Apply every subset to every sample with the formula above. This yields `n_perm` times the number of samples null scores for the signature, pooled into one vector. -3. Fit the chosen `distribution` to the pooled null scores by maximum - likelihood (`CBEA::estimate_distr()`, which calls - `fitdistrplus::fitdist()` for `norm`). -4. With `correlation_adjustment` enabled, fit the same distribution to the +3. Fit a normal distribution to the pooled null scores by maximum + likelihood (`CBEA::estimate_distr()` with `distr = "norm"`, which calls + `fitdistrplus::fitdist()`). +4. With `correlation_adjustment` enabled, fit a normal distribution to the observed scores of the signature across all samples, then combine the two - fits (`CBEA::combine_distr()`): for `norm`, the final null keeps the mean + fits (`CBEA::combine_distr()`): the final null keeps the mean of the permutation fit and takes the standard deviation from the fit to the observed scores. This widens the null to account for correlation among taxa, which the random subsets do not preserve. Because the observed fit @@ -247,14 +252,13 @@ In the `CBEA` package the call that performs Steps 4.1 through 4.5 is: ```r set.seed(random_seed) CBEA::cbea(obj = abundance, set = signatures, output = "pval", - abund_values = "relative_abundance", distr = distribution, + abund_values = "relative_abundance", distr = "norm", adj = TRUE, n_perm = 100) ``` where `abundance` is a `TreeSummarizedExperiment` whose named assay holds the harmonized, strictly positive relative abundances, and `signatures` is the -`BiocSet` of retained signatures, and `distribution` is `"norm"` unless -`"mnorm"` was selected and recorded. The package documentation describes +`BiocSet` of retained signatures. The package documentation describes `n_perm` as bootstrap resamples; the implementation draws random taxon subsets as described above, and the protocol follows the implementation. @@ -307,6 +311,17 @@ relative abundances in the BugSigDB vignette is one such choice, not a prescription of this protocol; if it is used, record its scale and rationale before running this protocol. +The CBEA paper also fits a two-component normal mixture to the null scores, +selectable with `distr = "mnorm"`. This protocol does not permit it because +the correlation adjustment for the mixture in `CBEA` version 0.99.4 +(`get_sd()` in `R/utils.R`, and the objective functions in +`get_adj_mnorm()`) computes the mixture standard deviation without squaring +the component standard deviations and mean offsets, so the adjusted null is +miscalibrated. For equally weighted components with means -3 and 3 and unit +standard deviations, it returns 1 where the correct value is the square root +of 10. If a corrected implementation is validated, a later version of this +protocol can permit `mnorm` as an alternative from the same paper. + The vignette requested `output = "zscore"` from `CBEA::cbea()` and passed the per-sample z-scores to a between-group linear model. That downstream analysis is out of scope here, and this protocol requests `output = "pval"` so that the @@ -337,7 +352,9 @@ protocols. abundances, the background universe, set-size bounds, the competitive balance, the pooled cross-sample null and its correlation adjustment, upper-tail p-values, per-sample Benjamini-Hochberg adjustment, and pooled - versus per-dataset execution. + versus per-dataset execution with union-of-taxa merging. +- Restricted the null to the normal distribution because the mixture + adjustment in `CBEA` 0.99.4 is miscalibrated. - Generalized from BugSigDB signatures to any collection of microbial signatures, with BugSigDB retained as the worked example. @@ -346,4 +363,4 @@ protocols. **Review by Levi Waldron ([0000-0003-2725-0694](https://orcid.org/0000-0003-2725-0694))** - **Date:** 2026-10-17 - **Status:** `approved` -- **Notes:** Converted the protocol from a BugSigDB-specific protocol to a general CBEA protocol for any collection of microbial signatures, keeping BugSigDB as the worked example. Added code snippets naming the `bugsigdbr` and `CBEA` functions that perform each step. Restricted input to linear-scale relative abundance so that aggregation to the target rank can sum rows, with excluded rows logged as warnings. Specified the pooled cross-sample null, the correlation adjustment, upper-tail p-values, and seeding by `set.seed()` because the package does not forward its `parallel_backend` argument. Defined the multiple-testing family as all retained signatures within a sample. +- **Notes:** Converted the protocol from a BugSigDB-specific protocol to a general CBEA protocol for any collection of microbial signatures, keeping BugSigDB as the worked example. Added code snippets naming the `bugsigdbr` and `CBEA` functions that perform each step. Restricted input to linear-scale relative abundance so that aggregation to the target rank can sum rows, with excluded rows logged as warnings. Specified the pooled cross-sample null, the correlation adjustment, upper-tail p-values, and seeding by `set.seed()` because the package does not forward its `parallel_backend` argument. Defined the multiple-testing family as all retained signatures within a sample. Restricted the null distribution to `norm` after a review comment showed the `mnorm` correlation adjustment in `CBEA` 0.99.4 is miscalibrated, and defined pooled-mode merging on the union of taxa. From 3011f749ef9d814df394d1d374c9ae8c065b8e9a Mon Sep 17 00:00:00 2001 From: Levi Waldron Date: Sat, 17 Oct 2026 17:23:37 -0400 Subject: [PATCH 9/9] Link upstream CBEA issues from the protocol Drafted with Claude Code. --- protocols/cbea-signature-enrichment/protocol.md | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/protocols/cbea-signature-enrichment/protocol.md b/protocols/cbea-signature-enrichment/protocol.md index 2af28d3..220f913 100644 --- a/protocols/cbea-signature-enrichment/protocol.md +++ b/protocols/cbea-signature-enrichment/protocol.md @@ -224,7 +224,8 @@ does, pooling across samples rather than building a separate null per sample: `parallel_backend` argument but do not forward it to the internal fitter, which seeds a serial backend from the current R random stream. Call `set.seed(random_seed)` immediately before `CBEA::cbea()` and do - not rely on the `parallel_backend` argument for reproducibility. + not rely on the `parallel_backend` argument for reproducibility + (reported upstream as https://github.com/qpmnguyen/CBEA/issues/13). 2. Apply every subset to every sample with the formula above. This yields `n_perm` times the number of samples null scores for the signature, pooled into one vector. @@ -319,8 +320,10 @@ the correlation adjustment for the mixture in `CBEA` version 0.99.4 the component standard deviations and mean offsets, so the adjusted null is miscalibrated. For equally weighted components with means -3 and 3 and unit standard deviations, it returns 1 where the correct value is the square root -of 10. If a corrected implementation is validated, a later version of this -protocol can permit `mnorm` as an alternative from the same paper. +of 10. This is reported upstream as +https://github.com/qpmnguyen/CBEA/issues/12. If a corrected implementation +is validated, a later version of this protocol can permit `mnorm` as an +alternative from the same paper. The vignette requested `output = "zscore"` from `CBEA::cbea()` and passed the per-sample z-scores to a between-group linear model. That downstream analysis