Add MMIDAS pipelines by jessicaway · Pull Request #1874 · broadinstitute/warp

jessicaway · 2026-06-29T17:32:15Z

Description

Adds MMIDAS pipelines to Warp
Corresponding Warp-tools PR is here: Jw mmidas warp-tools#201

Checklist

If you can answer "yes" to the following items, please add a checkmark next to the appropriate checklist item(s) and notify our WARP team by tagging @broadinstitute/warp-admins in a comment on this PR.

Did you add inputs, outputs, or tasks to a workflow?
Did you modify, delete or move: file paths, file names, input names, output names, or task names?
If you made a changelog update, did you update the pipeline version number?

github-actions · 2026-06-29T17:32:24Z

Remember to squash merge!

github-actions · 2026-06-29T17:32:34Z

🔍Version Validation Results:

Comparing versions and changelogs for pipelines that differ from the versions on 'origin/staging':
MMIDAS_Analyze.wdl and MMIDAS_Analyze.changelog.md do not have matching versions
MMIDAS_Train.wdl and MMIDAS_Train.changelog.md do not have matching versions
MMIDAS_DataPrep.wdl and MMIDAS_DataPrep.changelog.md do not have matching versions
Some WDLs or changelog files need updating. See output for details.
validation_failed

github-actions · 2026-06-29T17:32:35Z

🔍Changelog Validation Results:

Comparing changelogs for pipelines that differ from the versions on 'origin/develop':
All changelog files are valid for this release.

Copilot

Pull request overview

Adds the MMIDAS WDL pipelines to WARP, covering data preparation, model training/evaluation, and downstream analysis/figure generation, plus associated changelogs, example inputs, and registry/version tracking updates.

Changes:

Introduce three new MMIDAS workflows: MMIDAS_DataPrep, MMIDAS_Train, and MMIDAS_Analyze.
Add initial changelogs and example input JSONs for each workflow.
Register the workflows in .dockstore.yml and add entries to pipeline_versions.txt (currently needs fixes).

Reviewed changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
pipelines/wdl/mmidas/MMIDAS_DataPrep.wdl	New Stage 1 workflow/task to build the preprocessed AnnData `.h5ad` input for training.
pipelines/wdl/mmidas/MMIDAS_DataPrep.changelog.md	Initial changelog entry for `MMIDAS_DataPrep` v1.0.0.
pipelines/wdl/mmidas/example_inputs/MMIDAS_DataPrep.json	Example inputs for `MMIDAS_DataPrep`.
pipelines/wdl/mmidas/MMIDAS_Train.wdl	New Stage 2+3a workflow/tasks for optional augmenter training, core model training/pruning, and checkpoint evaluation.
pipelines/wdl/mmidas/MMIDAS_Train.changelog.md	Initial changelog entry for `MMIDAS_Train` v1.0.0.
pipelines/wdl/mmidas/example_inputs/MMIDAS_Train.json	Example inputs for `MMIDAS_Train`.
pipelines/wdl/mmidas/MMIDAS_Analyze.wdl	New Stage 3b–5 workflow/tasks for classification, traversal prep, and figure generation.
pipelines/wdl/mmidas/MMIDAS_Analyze.changelog.md	Initial changelog entry for `MMIDAS_Analyze` v1.0.0.
pipelines/wdl/mmidas/example_inputs/MMIDAS_Analyze.json	Example inputs for `MMIDAS_Analyze`.
pipeline_versions.txt	Adds MMIDAS pipeline entries (currently incorrect format/values).
.dockstore.yml	Registers the new workflows (currently contains a typo in `MMIDAS_Train` name).

github-actions · 2026-06-29T17:49:27Z

Remember to squash merge!

github-actions · 2026-06-29T17:49:43Z

🔍Version Validation Results:

Comparing versions and changelogs for pipelines that differ from the versions on 'origin/staging':
MMIDAS_Analyze.wdl and MMIDAS_Analyze.changelog.md do not have matching versions
MMIDAS_Train.wdl and MMIDAS_Train.changelog.md do not have matching versions
MMIDAS_DataPrep.wdl and MMIDAS_DataPrep.changelog.md do not have matching versions
Some WDLs or changelog files need updating. See output for details.
validation_failed

github-actions · 2026-06-29T17:49:46Z

🔍Changelog Validation Results:

Comparing changelogs for pipelines that differ from the versions on 'origin/develop':
All changelog files are valid for this release.

github-actions · 2026-06-29T20:12:19Z

Remember to squash merge!

github-actions · 2026-06-29T20:12:29Z

🔍Version Validation Results:

Comparing versions and changelogs for pipelines that differ from the versions on 'origin/staging':
MMIDAS_Analyze.wdl and MMIDAS_Analyze.changelog.md do not have matching versions
MMIDAS_Train.wdl and MMIDAS_Train.changelog.md do not have matching versions
MMIDAS_DataPrep.wdl and MMIDAS_DataPrep.changelog.md do not have matching versions
Some WDLs or changelog files need updating. See output for details.
validation_failed

github-actions · 2026-06-29T20:12:30Z

🔍Changelog Validation Results:

Comparing changelogs for pipelines that differ from the versions on 'origin/develop':
All changelog files are valid for this release.

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

github-actions · 2026-06-30T18:33:58Z

Remember to squash merge!

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

github-actions · 2026-06-30T18:34:08Z

🔍Version Validation Results:

Comparing versions and changelogs for pipelines that differ from the versions on 'origin/staging':
MMIDAS_Analyze.wdl and MMIDAS_Analyze.changelog.md do not have matching versions
MMIDAS_Train.wdl and MMIDAS_Train.changelog.md do not have matching versions
MMIDAS_DataPrep.wdl and MMIDAS_DataPrep.changelog.md do not have matching versions
Some WDLs or changelog files need updating. See output for details.
validation_failed

github-actions · 2026-06-30T18:34:11Z

Remember to squash merge!

github-actions · 2026-06-30T18:34:11Z

🔍Changelog Validation Results:

Comparing changelogs for pipelines that differ from the versions on 'origin/develop':
All changelog files are valid for this release.

github-actions · 2026-06-30T18:34:23Z

🔍Changelog Validation Results:

Comparing changelogs for pipelines that differ from the versions on 'origin/develop':
All changelog files are valid for this release.

github-actions · 2026-06-30T18:34:25Z

🔍Version Validation Results:

Comparing versions and changelogs for pipelines that differ from the versions on 'origin/staging':
MMIDAS_Analyze.wdl and MMIDAS_Analyze.changelog.md do not have matching versions
MMIDAS_Train.wdl and MMIDAS_Train.changelog.md do not have matching versions
MMIDAS_DataPrep.wdl and MMIDAS_DataPrep.changelog.md do not have matching versions
Some WDLs or changelog files need updating. See output for details.
validation_failed

github-actions · 2026-06-30T18:37:42Z

Remember to squash merge!

github-actions · 2026-06-30T18:37:53Z

🔍Changelog Validation Results:

Comparing changelogs for pipelines that differ from the versions on 'origin/develop':
All changelog files are valid for this release.

github-actions · 2026-06-30T18:37:55Z

🔍Version Validation Results:

Comparing versions and changelogs for pipelines that differ from the versions on 'origin/staging':
MMIDAS_Analyze.wdl and MMIDAS_Analyze.changelog.md do not have matching versions
MMIDAS_Train.wdl and MMIDAS_Train.changelog.md do not have matching versions
MMIDAS_DataPrep.wdl and MMIDAS_DataPrep.changelog.md do not have matching versions
Some WDLs or changelog files need updating. See output for details.
validation_failed

github-actions · 2026-07-06T19:46:41Z

Remember to squash merge!

github-actions · 2026-07-06T19:46:53Z

🔍Changelog Validation Results:

Comparing changelogs for pipelines that differ from the versions on 'origin/develop':
All changelog files are valid for this release.

github-actions · 2026-07-06T19:46:56Z

🔍Version Validation Results:

Comparing versions and changelogs for pipelines that differ from the versions on 'origin/staging':
MMIDAS_Analyze.wdl and MMIDAS_Analyze.changelog.md do not have matching versions
MMIDAS_Train.wdl and MMIDAS_Train.changelog.md do not have matching versions
MMIDAS_DataPrep.wdl and MMIDAS_DataPrep.changelog.md do not have matching versions
Some WDLs or changelog files need updating. See output for details.
validation_failed

github-actions · 2026-07-10T15:43:43Z

Remember to squash merge!

github-actions · 2026-07-10T15:43:54Z

🔍Version Validation Results:

Comparing versions and changelogs for pipelines that differ from the versions on 'origin/staging':
MMIDAS_Analyze.wdl and MMIDAS_Analyze.changelog.md do not have matching versions
MMIDAS_Train.wdl and MMIDAS_Train.changelog.md do not have matching versions
MMIDAS_DataPrep.wdl and MMIDAS_DataPrep.changelog.md do not have matching versions
Some WDLs or changelog files need updating. See output for details.
validation_failed

github-actions · 2026-07-10T15:43:55Z

🔍Changelog Validation Results:

Comparing changelogs for pipelines that differ from the versions on 'origin/develop':
All changelog files are valid for this release.

# Conflicts: # pipeline_versions.txt

github-actions · 2026-07-22T15:38:32Z

Remember to squash merge!

github-actions · 2026-07-22T15:38:43Z

🔍Version Validation Results:

Comparing versions and changelogs for pipelines that differ from the versions on 'origin/staging':
MMIDAS_Train.wdl and MMIDAS_Train.changelog.md do not have matching versions
MMIDAS_Analyze.wdl and MMIDAS_Analyze.changelog.md do not have matching versions
MMIDAS_DataPrep.wdl and MMIDAS_DataPrep.changelog.md do not have matching versions
Some WDLs or changelog files need updating. See output for details.
validation_failed

github-actions · 2026-07-22T15:38:46Z

🔍Changelog Validation Results:

Comparing changelogs for pipelines that differ from the versions on 'origin/develop':
All changelog files are valid for this release.

github-actions · 2026-08-03T17:16:41Z

Remember to squash merge!

github-actions · 2026-08-03T17:16:51Z

🔍Version Validation Results:

Comparing versions and changelogs for pipelines that differ from the versions on 'origin/staging':
MMIDAS_Train.wdl and MMIDAS_Train.changelog.md do not have matching versions
MMIDAS_Analyze.wdl and MMIDAS_Analyze.changelog.md do not have matching versions
MMIDAS_DataPrep.wdl and MMIDAS_DataPrep.changelog.md do not have matching versions
Some WDLs or changelog files need updating. See output for details.
validation_failed

github-actions · 2026-08-03T17:16:52Z

🔍Changelog Validation Results:

Comparing changelogs for pipelines that differ from the versions on 'origin/develop':
All changelog files are valid for this release.

github-actions · 2026-08-07T18:56:48Z

Remember to squash merge!

github-actions · 2026-08-07T18:57:01Z

🔍Changelog Validation Results:

Comparing changelogs for pipelines that differ from the versions on 'origin/develop':
All changelog files are valid for this release.

github-actions · 2026-08-07T18:57:04Z

🔍Version Validation Results:

Comparing versions and changelogs for pipelines that differ from the versions on 'origin/staging':
All WDLs and changelog files appear to be valid for this release.

github-actions · 2026-08-12T17:50:22Z

Remember to squash merge!

github-actions · 2026-08-12T17:50:34Z

🔍Version Validation Results:

Comparing versions and changelogs for pipelines that differ from the versions on 'origin/staging':
All WDLs and changelog files appear to be valid for this release.

github-actions · 2026-08-12T17:50:42Z

🔍Changelog Validation Results:

Comparing changelogs for pipelines that differ from the versions on 'origin/develop':
All changelog files are valid for this release.

Add MMIDAS pipelines

92b8b65

jessicaway requested a review from Copilot June 29, 2026 17:32

Updated pipeline_versions.txt with all pipeline version information

617cabf

Copilot started reviewing on behalf of jessicaway June 29, 2026 17:32 View session

jessicaway mentioned this pull request Jun 29, 2026

Jw mmidas broadinstitute/warp-tools#201

Draft

Copilot AI reviewed Jun 29, 2026

View reviewed changes

Comment thread pipeline_versions.txt Outdated

Comment thread .dockstore.yml Outdated

jessicaway added 2 commits June 29, 2026 11:47

update docker version

67c4d1c

Merge remote-tracking branch 'origin/jw_MMIDAS' into jw_MMIDAS

d24656e

update docker again

6e53ca0

Potential fix for pull request finding

895608e

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Potential fix for pull request finding

a307a29

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Updated pipeline_versions.txt with all pipeline version information

2ee7cc0

jessicaway added 2 commits June 30, 2026 12:36

update docker again

b935843

Merge remote-tracking branch 'origin/jw_MMIDAS' into jw_MMIDAS

9c7e530

update dockers again

4224818

updates

7cc18fb

jessicaway added 2 commits July 16, 2026 09:22

update docker

fb306f2

Merge branch 'develop' into jw_MMIDAS

71cb59c

# Conflicts: # pipeline_versions.txt

Updated pipeline_versions.txt with all pipeline version information

d1f1ec0

Latest updates

e3ed74f

MMIDAS updates

fce549a

more updates

5cd03cc

Uh oh!

Conversation

jessicaway commented Jun 29, 2026

Description

Checklist

Uh oh!

github-actions Bot commented Jun 29, 2026

Uh oh!

github-actions Bot commented Jun 29, 2026

🔍Version Validation Results:

Uh oh!

github-actions Bot commented Jun 29, 2026

🔍Changelog Validation Results:

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

github-actions Bot commented Jun 29, 2026

Uh oh!

github-actions Bot commented Jun 29, 2026

🔍Version Validation Results:

Uh oh!

github-actions Bot commented Jun 29, 2026

🔍Changelog Validation Results:

Uh oh!

github-actions Bot commented Jun 29, 2026

Uh oh!

github-actions Bot commented Jun 29, 2026

🔍Version Validation Results:

Uh oh!

github-actions Bot commented Jun 29, 2026

🔍Changelog Validation Results:

Uh oh!

github-actions Bot commented Jun 30, 2026

Uh oh!

github-actions Bot commented Jun 30, 2026

🔍Version Validation Results:

Uh oh!

github-actions Bot commented Jun 30, 2026

Uh oh!

github-actions Bot commented Jun 30, 2026

🔍Changelog Validation Results:

Uh oh!

github-actions Bot commented Jun 30, 2026

🔍Changelog Validation Results:

Uh oh!

github-actions Bot commented Jun 30, 2026

🔍Version Validation Results:

Uh oh!

github-actions Bot commented Jun 30, 2026

Uh oh!

github-actions Bot commented Jun 30, 2026

🔍Changelog Validation Results:

Uh oh!

github-actions Bot commented Jun 30, 2026

🔍Version Validation Results:

Uh oh!

github-actions Bot commented Jul 6, 2026

Uh oh!

github-actions Bot commented Jul 6, 2026

🔍Changelog Validation Results:

Uh oh!

github-actions Bot commented Jul 6, 2026

🔍Version Validation Results:

Uh oh!

github-actions Bot commented Jul 10, 2026

Uh oh!

github-actions Bot commented Jul 10, 2026

🔍Version Validation Results:

Uh oh!

github-actions Bot commented Jul 10, 2026

🔍Changelog Validation Results:

Uh oh!

github-actions Bot commented Jul 22, 2026

Uh oh!

github-actions Bot commented Jul 22, 2026