Split out optional-dependencies

[ntjohnson1]

In 0.24 we added a datafusion dependency but marked it as optional to reduce pulling along extra packages. Conda doesn't really do optional-dependencies AFAICT but instead people just add extra targets. Worth exploring in case people run into issues with our optional dependencies (and if we continue to design around this approach).

Worth looking at how pandas manages this for conda since we cross referenced their optional depencies approach.

[jeongseok-meta]

I revisited this against Rerun 0.36.0 and the current conda packaging work.

The upstream runtime extras are now:

• notebook
• catalog (datafusion and dataplatform remain as deprecated aliases)
• dataloader
• tracing
• all

The current conda package intentionally bundles both rerun-sdk and rerun-notebook, but it does not expose the other groups. Pandas-style run_constrained entries would only constrain packages that happen to be installed; they would not provide installable dependency groups, so I do not think that is the right model here.

There is now a direct long-term solution: CEP 44 has been accepted and adds native requirements.extras to recipe format v1. It requires repodata schema v3, though. conda-forge does not emit v3 yet; the rollout mechanism is still being decided in conda/ceps#146. conda-forge/conda-forge.github.io#2888 documents -with-<extra> outputs as today's fallback, while conda-forge/imagemagick-feedstock#385 demonstrates that native extras currently need a draft/blocked PR.

My recommended sequence is:

1. Migrate this feedstock to recipe format v1 in a mechanical, no-semantic-change PR.
2. Correct base metadata drift separately (python >=3.10, numpy >=2, and ipykernel !=7.0.* in 0.36).
3. Once conda-forge enables v3, add native catalog, dataloader, tracing, deprecated aliases, and all groups, mapping upstream torch to conda-forge's pytorch. Tests should verify that the base solve stays lean and that each selected group solves and imports.
4. Avoid temporary rerun-sdk-with-* outputs unless the v3 rollout stalls materially.

There is one separate package-boundary question. We can add the non-notebook extras without changing today's bundle. If we later want PyPI-like notebook opt-in behavior, I would preserve compatibility using an actual rerun-sdk-core package plus rerun-notebook, with the existing rerun-sdk name retained as a metapackage depending on both.

@ntjohnson1, @jleibs, @traversaro, could you weigh in on these choices?

1. Keep notebook support bundled under rerun-sdk for compatibility (my recommendation), or plan the -core/metapackage split?
2. Mirror the deprecated datafusion and dataplatform aliases until upstream removes them (my recommendation), or expose only current extras?
3. Wait for repodata v3 (my recommendation), or is there an immediate user need that justifies temporary -with-* outputs?