Skip to content

fix(runspec): reject unsupported executors in nemo-run pipeline launcher#300

Open
andrewwhitecdw wants to merge 1 commit into
NVIDIA-NeMo:mainfrom
andrewwhitecdw:fix-pipeline-executor-dispatch
Open

fix(runspec): reject unsupported executors in nemo-run pipeline launcher#300
andrewwhitecdw wants to merge 1 commit into
NVIDIA-NeMo:mainfrom
andrewwhitecdw:fix-pipeline-executor-dispatch

Conversation

@andrewwhitecdw

Copy link
Copy Markdown
Contributor

Problem

PipelineConfig.executor is declared as Literal["local", "slurm", "dgxcloud", "lepton"], but run_nemo_run only special-cases "local":

if config.executor == "local":
    executor = run.LocalExecutor(...)
else:  # slurm
    ...
    executor = run.SlurmExecutor(...)

Every other value — including "dgxcloud" and "lepton" — falls into the else branch, so a user passing executor="dgxcloud" gets either a misleading --account required for Slurm executor error or, worse, a Slurm job silently submitted to the wrong backend.

Fix

Make the Slurm branch explicit (elif config.executor == "slurm") and reject anything else with a clear error:

[pipeline] ERROR: Executor 'dgxcloud' is not supported by the nemo-run pipeline launcher (supported: 'local', 'slurm')

Testing

  • Verified dgxcloud/lepton now return rc=1 with the clear error; slurm without --account still hits its proper validation
  • pytest tests/nemo_runspec — 186 passed, 3 pre-existing failures on main (unrelated data_mover/lepton tests, fail identically without this change)

PipelineConfig.executor accepts 'local' | 'slurm' | 'dgxcloud' | 'lepton',
but run_nemo_run dispatched every non-'local' value into the Slurm branch,
silently building a SlurmExecutor for 'dgxcloud'/'lepton' (or failing with
a misleading '--account required for Slurm executor'). Make the slurm
branch explicit and error out clearly on unsupported executors.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant