diff --git a/README.md b/README.md index f17ec0a0..be5ce39b 100644 --- a/README.md +++ b/README.md @@ -55,6 +55,7 @@ for conda's implementation, all major changes should be submitted as | [0042](cep-0042.md) | Channel relations in repodata | | [0043](cep-0043.md) | Conditional dependencies | | [0044](cep-0044.md) | Optional dependency groups | +| [0045](cep-0045.md) | Simplified variant selection | ## References diff --git a/cep-0045.md b/cep-0045.md new file mode 100644 index 00000000..31939855 --- /dev/null +++ b/cep-0045.md @@ -0,0 +1,99 @@ +# CEP 45 - Simplified variant selection + + + + + + + + + +
Title Simplified variant selection
Status Accepted
Author(s) + Wolf Vollprecht <w.vollprecht@gmail.com>, + Bas Zalmstra <bas@prefix.dev>, + Jaime Rodríguez-Guerra <jaime.rogue@gmail.com> +
Created Feb 5, 2025
Updated May 14, 2026
Discussion https://github.com/conda/ceps/pull/111, https://github.com/conda/ceps/pull/166
Implementation NA
+ +## Abstract + +This CEP proposes a simplified mechanism to select package variants without relying on partial build string globbing. A package carries a list of plain string flags; a solver constraint that names a flag excludes any package that does not carry it. This mechanism involves adding a new `flags` repodata record field along with the corresponding `MatchSpec` extensions. + +## Motivation + +Within a channel and subdir, a conda package for a given project release may have different builds or artifacts. These are used to either correct deficiencies in the build process, or to provide different build alternatives (e.g. linking to diverse library backends). These artifacts can be distinguished by their build string, which can be matched with glob strings. + +In principle, the solver is responsible for choosing the right variant for a given package version. However, the user may also want to force a particular variant to satisfy their needs. The solution so far has been to plant a special substring in the build string so it can be selected by the corresponding glob string. For example, given a package with a CPU and a GPU variant (`package-1.0-h123abc_cpu_0.conda` and `package-1.0-h453cbd_gpu_0.conda`), the GPU variant can be chosen by asking for `package=*=*gpu*`. + +The problem with this approach is that it doesn't scale well for more than one "flag" per build string. What if a package needs to distinguish among more than one feature? That is, not just GPU vs CPU, but also BLAS backend, MPI or licensing? +The glob strings are not expressive enough for a single spec, so several ones need to be supplied (e.g. `package=*=*gpu*`, `package=*=*mkl*`, `package=*=*mpich*` and `package=*=*nogpl*`), resulting in complicated lookahead regexes that balloon in computational complexity. + +The answer to this problem is to provide a specific field that is designed to provide such selection capabilities without the expressivity and complexity problems observed above: `flags`. + +## Rationale + +The chosen keyword is `flags` which makes sense as a "compile time flag". This feature is mainly relevant for compiled packages with different compile time feature selection, making `flags` a matching name. + +## Specification + +### Repodata record syntax + +The `info/index.json` file of each conda artifact MUST support two new fields, `flags`. + +The value of the `flags` field MUST be a list of non-empty strings matching the regex `^[a-z0-9_]+(:[a-z0-9_]+)?$`. We allow a _single_ `:` for `key:value` semantics. + +Subsequently, the CEP 34 `info/index.json`'s `schema_version` value MUST be bumped to `3`. + +In recipes, these values MUST be supported by a `flags` key in the `build` section of each output (i.e. sibling to `number` and `track_features`) with a list of strings as the value; i.e. `build.flags = [str]`. + +### MatchSpec syntax changes + +Values in the `flags` field MUST be matchable by the corresponding keyword in `MatchSpec`, placed in the square brackets section. Its value MUST be a string or list of strings. Each entry MUST match the regex `^[a-z0-9_\*]+(:[a-z0-9_\*]+)?$`. The `*` character is a glob operator, with the same meaning as in CEP 29 "String Matching". + +Flag matching is intentionally simple: a package is excluded from consideration if it does not contain every flag listed in the `flags` constraint. A flag either matches the string (exactly or globbed) or the package is filtered out. + +### `index.json` and `repodata_record` changes + +The records gain a new `flags`: + +```json +{ + "name": "foobar", + "version": "1.2.3", + ..., + "flags": ["cuda", "release", "blas:mkl"], +} +``` + +### Changes to the `recipe.yaml` file + +```yaml +build: + string: ... + flags: ["cuda", "blas:mkl", "release"] +``` + +## Examples + +In practice, this would look like the following from a user perspective: + +```shell +conda install 'pytorch[version=">=3.1", flags=["cuda", "blas:*"]]' +``` + +Any package that does not carry both the `cuda` and a flag starting with `blas:` is excluded from the candidate set. Among the remaining candidates the solver applies its normal version and build-number and variant order preference sorting. + +## Rejected ideas + +This proposal may remind the readers of the old `features` properties in the first iterations of conda packaging. This is not a reimplementation. + +## Future plans + +Several extensions are deferred to future revisions: + +- Richer flag matching: numeric values, key-value items, and comparison operators (`>`, `<`) for numeric matches. +- Additional operators: `?` for "match if present" and `!` to exclude a flag. +- Variant prioritization: a `variant_priority` field to disambiguate among many matching variants. We could not agree on the sort order, so this is deferred for further discussion. + +## Copyright + +All CEPs are explicitly [CC0 1.0 Universal](https://creativecommons.org/publicdomain/zero/1.0/).