Report MONAI Model Zoo bundles awaiting a triage decision - #508
Open
garybrowne wants to merge 2 commits into
Open
garybrowne wants to merge 2 commits into
garybrowne wants to merge 2 commits into
Conversation
`filter_whitelist` intersects the whitelist with the Zoo listing, so a newly published bundle was invisible -- nobody learned about it without going and looking. This adds the reporting the #477 reviewer asked for. Splits what was an implicit on/off list into three states, only two of which are written down: approved `models:` generate and publish declined `declined:` reviewed, deliberately not published candidate derived not yet triaged Candidates are computed as (available - models - declined), so the whitelist records human decisions only and needs no edit when the Zoo changes. Declines carry `at_version`, scoping the decision to what was actually reviewed: a model declined as immature resurfaces once the Zoo moves past it, where a plain blacklist would bury it permanently. Omitting `at_version` declines every version, for reasons that will never change. Also surfaces approved models that have disappeared from the Zoo, which `filter_whitelist` skips silently -- leaving a spec in the tree building against a model that is no longer published. Not auto-generating a PR per new bundle: the Zoo has 35 bundles against 1 approved, so that would open ~34 PRs for humans to reject, and it inverts the default to "publish unless stopped". The anatomy fields are also human classification absent from bundle metadata, so an auto-generated spec would have to guess where in the tree it belongs. The report emits them paste-ready with the fields blank instead. Issue per run for now, as agreed -- a single updated issue avoids weekly noise but needs find-or-create logic; worth adding only if the noise is real. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
16:00 UTC Sunday, which is 02:00 Monday in AEST. Drifts to 03:00 over AEDT since cron has no timezone -- immaterial for a weekly background job. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #485.
The gap
filter_whitelistintersects the whitelist with the Zoo listing, so a newly published bundle is invisible — nobody learns about it without going and looking. That was the reviewer's question on #477: "How will we know when potential models are added to MONAI without checking them ourselves?"Approach
Splits what was an implicit on/off list into three states, only two of which are written down:
models:declined:Candidates are computed as
available − models − declined, somonai_whitelist.yamlrecords human decisions only and needs no edit when the Zoo changes.at_versionscopes a decline to what was actually reviewed — a model declined as immature resurfaces once the Zoo moves past it, where a plain blacklist buries it permanently. Omit it to decline every version.What gets reported
Three things, as an
S4issue per sync run:modality/species/regionblank, so approving is copy, fill three fields, commitat_versionto re-declinefilter_whitelist, leaving a spec building against a model that is no longer publishedNothing to report means no issue is opened.
Why not auto-generate a PR per new model
The reviewer's original suggestion. Two reasons it was not taken, both discussed on #485:
modality/species/regionare human anatomical classification absent from bundle metadata, and they determine where the spec lands in the tree. An auto-generated PR would have to guess them.Verification
at_versioncorrectly never going staleAlso in here
permissions:block added tomonai-sync.yml— it had none, so it ran on default token permissions.issues: writeis needed for the new step;contents/pull-requestsmade explicit forcreate-pull-request. Slightly beyond the issue scope, so worth a look.0 16 * * 0— 02:00 Monday Sydney (AEST). Drifts to 03:00 over AEDT since cron has no timezone; not worth two schedules for a weekly background job.Not covered
gh issue createstep is untested in CI — everything else was run locally, but issue creation only exercises on a real dispatch.🤖 Generated with Claude Code