Skip to content

feat(repodata_gateway): read virtual package plugin registrations - #2630

Open
hunger wants to merge 1 commit into
conda:mainfrom
hunger:push-svuwkkkmqnsv
Open

feat(repodata_gateway): read virtual package plugin registrations#2630
hunger wants to merge 1 commit into
conda:mainfrom
hunger:push-svuwkkkmqnsv

Conversation

@hunger

@hunger hunger commented Jul 30, 2026

Copy link
Copy Markdown
Collaborator

Channels can register virtual package detection plugins in info.virtual_package_plugins, mapping the name of the package providing the plugin to the virtual packages it provides. Keying by plugin lets one cuda-detect cover both __cuda and __cuda_arch instead of registering the same detector twice. Parsed from both repodata.json and the sharded index; names are deserialized unchecked so one malformed entry cannot make a channel unusable.

The gateway exposes the registrations two ways:

  • Gateway::virtual_package_plugins(channel, platform) mirrors Gateway::channel_relations and needs no specs, which matters because the plugin package names only exist inside the metadata being fetched.
  • RepoDataQueryOutput::virtual_package_plugins lists them per subdir in resolved channel-priority order, so channels a solve discovers through CEP-42 are covered too.

Registrations are reported exactly as declared: duplicate claims on the same virtual package are left for the caller to resolve, and no plugin is fetched or executed. Registrations stay per subdir rather than collapsed per channel, matching how CEP-42 relations are handled.

rattler virtual-packages -c <channel> prints them for manual testing against the new test-data/channels/virtual-package-plugins fixture.

The design, including the parts that are not implemented yet -- the plugin protocol, execution, caching and the trust model -- is written up in crates/rattler_repodata_gateway/docs/virtual-package-plugins.md.

All of it sits behind the experimental-virtual-package-plugins feature; with the feature off the public API and the serialized output are unchanged. Cargo features are additive, so every crate constructing ChannelInfo or ShardedSubdirInfo must enable it in lockstep, hence the forwarding entries in rattler_index and rattler-bin.

Also switches the sharded-index test mock to rmp_serde::to_vec_named, matching what the indexer writes. Positional encoding misaligns as soon as a skipped field precedes a present one.

This is just an initial attempt at the data structure in repo-data only and just enough extra code to get something out of the rattler binary.

The preliminary spec is in crates/rattler_repodata_gateway/docs/virtual-package-plugins.md, based on @wolfv suggestion with changes from claude to update it with the decisions taken in this change.

We can merge this into main: Everything is hidden behind experimental-virtual-package-plugins features.

How Has This Been Tested?

By running cargo run --bin rattler --features experimental-virtual-package-plugins -- virtual-packages -c ./test-data/channels/virtual-package-plugins and with unit tests.

AI Disclosure

  • This PR contains AI-generated content.
    • I have tested any AI-generated content in my PR.
    • I take responsibility for any AI-generated content in my PR.

Tools: Claude

Checklist:

  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have added sufficient tests to cover my changes.

Channels can register virtual package detection plugins in
`info.virtual_package_plugins`, mapping the name of the package providing
the plugin to the virtual packages it provides. Keying by plugin lets one
`cuda-detect` cover both `__cuda` and `__cuda_arch` instead of registering
the same detector twice. Parsed from both `repodata.json` and the sharded
index; names are deserialized unchecked so one malformed entry cannot make
a channel unusable.

The gateway exposes the registrations two ways:

- `Gateway::virtual_package_plugins(channel, platform)` mirrors
  `Gateway::channel_relations` and needs no specs, which matters because
  the plugin package names only exist inside the metadata being fetched.
- `RepoDataQueryOutput::virtual_package_plugins` lists them per subdir in
  resolved channel-priority order, so channels a solve discovers through
  CEP-42 are covered too.

Registrations are reported exactly as declared: duplicate claims on the
same virtual package are left for the caller to resolve, and no plugin is
fetched or executed. Registrations stay per subdir rather than collapsed
per channel, matching how CEP-42 relations are handled.

`rattler virtual-packages -c <channel>` prints them for manual testing
against the new `test-data/channels/virtual-package-plugins` fixture.

The design, including the parts that are not implemented yet -- the plugin
protocol, execution, caching and the trust model -- is written up in
`crates/rattler_repodata_gateway/docs/virtual-package-plugins.md`.

All of it sits behind the `experimental-virtual-package-plugins` feature;
with the feature off the public API and the serialized output are
unchanged. Cargo features are additive, so every crate constructing
`ChannelInfo` or `ShardedSubdirInfo` must enable it in lockstep, hence the
forwarding entries in `rattler_index` and `rattler-bin`.

Also switches the sharded-index test mock to `rmp_serde::to_vec_named`,
matching what the indexer writes. Positional encoding misaligns as soon as
a skipped field precedes a present one.
@github-actions

Copy link
Copy Markdown
Contributor

cargo-semver-checks detected API breaking changes compared with the pull request's base revision.

Details
--- failure constructible_struct_adds_field: externally-constructible struct adds field ---

Description:
A pub struct constructible with a struct literal has a new pub field. Existing struct literals must be updated to include the new field.
        ref: https://doc.rust-lang.org/reference/expressions/struct-expr.html
       impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.48.0/src/lints/constructible_struct_adds_field.ron

Failed in:
  field ShardedSubdirInfo.virtual_package_plugins in /home/runner/work/rattler/rattler/crates/rattler_conda_types/src/repo_data/sharded.rs:74
  field ChannelInfo.virtual_package_plugins in /home/runner/work/rattler/rattler/crates/rattler_conda_types/src/repo_data/mod.rs:117
    Building rattler_conda_types v0.49.0 (current)
       Built [  21.712s] (current)
     Parsing rattler_conda_types v0.49.0 (current)
      Parsed [   0.096s] (current)
    Building rattler_conda_types v0.49.0 (baseline)
       Built [  21.465s] (baseline)
     Parsing rattler_conda_types v0.49.0 (baseline)
      Parsed [   0.089s] (baseline)
    Checking rattler_conda_types v0.49.0 -> v0.49.0 (assume minor change)
     Checked [   0.699s] 196 checks: 195 pass, 1 fail, 0 warn, 57 skip

     Summary semver requires new major version: 1 major and 0 minor checks failed
    Finished [  45.849s] rattler_conda_types

    Building rattler_index v0.30.10 (current)
       Built [  76.968s] (current)
     Parsing rattler_index v0.30.10 (current)
      Parsed [   0.027s] (current)
    Building rattler_index v0.30.10 (baseline)
       Built [  75.836s] (baseline)
     Parsing rattler_index v0.30.10 (baseline)
      Parsed [   0.026s] (baseline)
    Checking rattler_index v0.30.10 -> v0.30.10 (assume minor change)
     Checked [   0.106s] 196 checks: 196 pass, 57 skip
     Summary no semver update required
    Finished [ 157.207s] rattler_index


--- failure constructible_struct_adds_field: externally-constructible struct adds field ---

Description:
A pub struct constructible with a struct literal has a new pub field. Existing struct literals must be updated to include the new field.
        ref: https://doc.rust-lang.org/reference/expressions/struct-expr.html
       impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.48.0/src/lints/constructible_struct_adds_field.ron

Failed in:
  field RepoDataQueryOutput.virtual_package_plugins in /home/runner/work/rattler/rattler/crates/rattler_repodata_gateway/src/gateway/query.rs:45
    Building rattler_repodata_gateway v0.31.0 (current)
       Built [  58.355s] (current)
     Parsing rattler_repodata_gateway v0.31.0 (current)
      Parsed [   0.059s] (current)
    Building rattler_repodata_gateway v0.31.0 (baseline)
       Built [  57.764s] (baseline)
     Parsing rattler_repodata_gateway v0.31.0 (baseline)
      Parsed [   0.060s] (baseline)
    Checking rattler_repodata_gateway v0.31.0 -> v0.31.0 (assume minor change)
     Checked [   0.212s] 196 checks: 195 pass, 1 fail, 0 warn, 57 skip

     Summary semver requires new major version: 1 major and 0 minor checks failed
    Finished [ 119.418s] rattler_repodata_gateway

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant