Explore Metaflow CLI improvements

[saikonen]

As Metaflow supports building extensions for the client, some authors might want to extend parts of the CLI that are difficult or impossible to reliably get to. Identifying such areas in the code base and proposing ways to make them extensible would prove useful.

Goals

• Getting familiar with the Metaflow CLI internals and Metaflow extensions structure.
  • Identifying non-customisable sections of the CLI.
• Propose a design for improving the Metaflow CLI in any way.
  • Proposal should contain considerations regarding compatibility, and extension support.

---

Instructions

This issue is meant to be a venue for asking further questions on the topic. Implementation details and proposals are to be kept separate.

When you have met the goals of the issue, it is time to show and tell.
You can do this for example with your own fork with an issue or pull request and messaging a mentor on Slack with a link.

[rawadhossain]

Hi @saikonen, I spent some time exploring the Metaflow CLI internals to understand where extension points might be useful, and I experimented with a few PRs mentioned.

I wrote up a more detailed exploration and proposal here: Proposal Link

The proposal describes a few areas where extending the CLI can be difficult, and I implemented three improvements on my fork:

• CLI lifecycle hooks during per flow CLI startup
• Decorator defined run/resume CLI options
• A shared CLI argument serializer to reduce duplicated logic

I kept the proposal itself separate from the issue comment to avoid cluttering the discussion here.

A few questions:

• Do lifecycle checkpoints during CLI startup (e.g. post_datastore, post_metadata, etc.) seem like reasonable extension points for plugins?
• Does allowing FlowDecorator to define run / resume CLI options feel like the right abstraction, or would you prefer these extensions to live somewhere else in the CLI architecture?
• Would consolidating CLI argument serialization into a shared helper be acceptable, or is the current separation between cli_args.py and runtime.py intentional?
• Are there other parts of the CLI that you consider particularly important to make extensible?

Happy to adjust based on your feedback.

[ommgh]

Hi @saikonen I've been going through the CLI internals, especially how start() in cli.py sets up infrastructure before sub-commands run.

I noticed there are ~4 places in start() that do string matching against ctx.saved_args[0] to figures out what kind of initialization to do.
Like, it checks if the command is "spin" or "spin-step" to set up local metadata/datastore, checks if it's "run"/"resume"/"spin" to skip default decospec handling, etc.

This works fine for the built in commands but I was wondering, if someone writing an extension adds a new command that needs spin-like treatment (say something that replays a step locally like spin does, but with extra behavior), there isn’t a way for them to declare that . They’d have to modify the core start() logic.

Would it make sense to have some way for commands to declare what kind of initialization they need, so start() can check that instead of hardcoded names? Or is there a reason this is kept simple on purpose?

[ali-ezz]

Hi! I’m preparing a GSoC 2026 proposal for Metaflow and I’d like to work on this starter issue.

I’m planning to focus first on the CLI startup / extension points side:

• identifying places where CLI behavior is hard-coded
• noting which parts are difficult for extensions to customize
• proposing a minimal compatibility-safe design for improving extensibility

I’ll keep the implementation details separate and first share a short design note / findings from my fork. If there’s a specific area of the CLI you’d prefer I inspect first, I’d appreciate a pointer.