Support multi-target conditional switch cases

[npow]

Summary

Allows a conditional self.next(..., condition=...) case to target either one step or a non-empty list or tuple of steps. A selected multi-target case fans out only after the condition is evaluated; existing single-target switch behavior is unchanged.

self.next(
    {
        "hit": self.finalize,
        "miss": [self.clip_extract_and_score, self.face_analysis],
    },
    condition="cache_status",
)

This removes boilerplate steps whose only purpose is to fan out after a switch case.

Behavior and constraints

• Multi-target cases behave like case-local static splits and must rejoin before end.
• Multi-target cases must have disjoint target sets. Lint rejects overlapping targets to keep selected-case join accounting unambiguous.
• Nested static splits and foreaches inside selected branches are supported by the native runtime.
• Single-target cases may continue to share a target.
• Argo Workflows rejects multi-target switch cases explicitly for now; support is tracked in argo: implement fanout switch case support #3331.
• AWS Step Functions continues to reject all conditional switches as unsupported.

Implementation

• Extends FlowSpec.next validation to accept non-empty list and tuple case values.
• Preserves per-case target lists in the graph while exposing flattened unique graph edges for compatibility.
• Tracks case-local split ancestry through graph traversal and lint validation.
• Resolves the selected case from branch roots for native-runtime join scheduling and task input validation.
• Adds deploy-time and runtime guards for unsupported Argo execution.
• Updates DAG card types to represent scalar or list-valued switch targets.

Validation

• python -m pytest -q test/unit/test_switch_fanout_cases.py test/unit/test_argo_conditional_input_paths.py test/unit/test_graph_structure.py — 75 passed.
• End-to-end local runs cover both routes, different case cardinalities, a shared join, a nested static split, and a nested foreach.
• black --check and git diff --check pass.

[greptile-apps]

Greptile Summary

This PR adds multi-target conditional switch cases to the flow graph and native runtime while explicitly preventing unsupported Argo execution.

• Accepts non-empty list or tuple targets for individual switch cases.
• Preserves case-local fanout ancestry for graph validation, join scheduling, and task input validation.
• Rejects multi-target switch cases before Argo workflow compilation, with a runtime fallback guard.
• Updates DAG card types and adds graph, lint, runtime, and Argo coverage.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains; the previously reported Argo target-dropping path is prevented before workflow compilation and backed by a runtime rejection guard.

Important Files Changed

Filename	Overview
metaflow/flowspec.py	Accepts and validates scalar or non-empty list/tuple targets, preserving the selected case as the runtime transition.
metaflow/graph.py	Parses list-valued switch cases, exposes flattened graph edges, and tracks case-local split ancestry.
metaflow/lint.py	Validates case-local split/join balance and rejects overlapping targets involving multi-target cases.
metaflow/runtime.py	Queues every selected case target and derives join cardinality from the selected switch branch roots.
metaflow/task.py	Resolves the selected switch case from actual join inputs and validates its expected input count.
metaflow/plugins/argo/argo_workflows.py	Rejects multi-target switch cases before workflow-template compilation, fixing the previously reported target-dropping path.
metaflow/plugins/argo/argo_workflows_decorator.py	Adds a runtime fallback that refuses to serialize a multi-target transition as Argo's scalar switch output.
metaflow/plugins/cards/ui/src/types.ts	Extends the DAG UI contract to represent scalar and list-valued switch targets.
test/unit/test_switch_fanout_cases.py	Covers graph parsing, validation constraints, selected-case resolution, and native runtime fanout behavior.
test/unit/test_argo_conditional_input_paths.py	Verifies both deploy-time and runtime rejection of unsupported Argo switch fanout.

_{Reviews (10): Last reviewed commit: "Clean up switch lint handling" | Re-trigger Greptile}

[saikonen]

Changes look good. The argo-workflows implementation still needs modifications to support this though, or a raise as unsupported if its going to be a follow-up PR.

[npow]

@saikonen done — added the raise in 12e43cb.

Two guards:

• Deploy-time (argo_workflows.py): _parse_conditional_branches() now iterates switch nodes and raises ArgoWorkflowsException before the workflow is even compiled if any case maps to a list of targets.
• Runtime safety net (argo_workflows_decorator.py): raises MetaflowException before writing /mnt/out/switch_step if _out_funcs has more than one entry, so a flow that somehow slips past deploy-time validation fails loudly rather than silently dropping targets.

Will open a follow-up to implement Argo support and move the tests to OSS.

[npow]

Follow-up PR for Argo support: #3331

[talsperre]

Requesting changes based on two reproduced correctness problems:

1. A fanout case containing a nested split or foreach passes validation but fails during local execution. The scheduler loses the outer switch-case identity, and the task-side join cardinality check has the same assumption.
2. Two fanout cases cannot share one join because lint compares each individual case against the union of predecessors from every case.

The current unit tests only call FlowSpec.next() directly, so they do not exercise either failing runtime path.

Full review, reproductions, locations, and suggested fix direction:
https://gist.github.com/talsperre/1a44dee761af17e7f9d47b454b6bf393

Reviewed at 12e43cb5b230395a2fa0c8265b6fa8d338a1958f.

[npow]

Addressed the review findings in 3207803e:

• resolve switch branch identity by its position in split_parents/split_branches, shared across runtime scheduling, task join validation, and lint
• allow different multi-target cases to converge on one shared join with case-specific cardinality
• add real local-execution regression coverage for both selected cases, tuple/list targets, nested static split, nested foreach, and 2-vs-3 input shared joins
• add focused Argo rejection coverage and use “multi-target switch case” wording
• update the card UI switch_cases TypeScript type to accept arrays

Validation:

• focused unit/runtime suite: 66 passed
• Black and git diff --check: clean
• broader test/unit: 518 passed, 20 skipped; 5 unrelated spin failures/errors occurred because spawned processes loaded the machine-installed Metaflow, with the run also interrupted during that section

[npow]

triggered internal tests

[talsperre]

Re-reviewed at 4a3fafa4. The fixes in 3207803e do address all findings from my original review, and the new focused suite passes locally (66 passed).

I found one remaining blocking correctness issue: if two fanout cases share a target but have different cardinalities, local join scheduling depends on task completion order. With a=[shared, a_only] and b=[shared, b_one, b_two], route b either:

• succeeds after joining only shared and b_one, silently dropping b_two, or
• never schedules the join and fails to reach end.

The runtime reconstructs the selected case from the finishing branch and chooses the first matching case; shared is therefore ambiguous. The selected case needs to be retained explicitly, or overlapping targets need to be rejected during linting.

Updated review with the complete runnable flow, exact commands, observed output, and code locations:
https://gist.github.com/talsperre/1a44dee761af17e7f9d47b454b6bf393

[npow]

Addressed the overlapping-target blocker in 1faf6967.

Multi-target switch cases are now required to have disjoint target sets. Lint rejects the reproduced a=[shared, a_only] / b=[shared, b_one, b_two] shape deterministically before execution, avoiding completion-order-dependent join scheduling while keeping this change narrowly scoped. The public FlowSpec.next() documentation now states the restriction.

Added the exact overlap shape as a regression test. Focused graph, fanout-runtime, and Argo suite: 67 passed; Black and git diff --check are clean.