From 145692713ea1167e9cb15a3c445c951b06f8d890 Mon Sep 17 00:00:00 2001 From: jaimergp Date: Sat, 20 Dec 2025 19:04:16 +0100 Subject: [PATCH 01/20] Split in three CEPs --- cep-XXX3.md | 74 +++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 74 insertions(+) create mode 100644 cep-XXX3.md diff --git a/cep-XXX3.md b/cep-XXX3.md new file mode 100644 index 00000000..b7956758 --- /dev/null +++ b/cep-XXX3.md @@ -0,0 +1,74 @@ +# CEP XXXX - Simplified variant selection + + + + + + + + + + +
Title Simplified variant selection
Status Draft
Author(s) + Wolf Vollprecht <w.vollprecht@gmail.com>, + Bas Zalmstra <bas@prefix.dev>, + Jaime Rodríguez-Guerra <jaime.rogue@gmail.com> +
Created Feb 5, 2025
Updated Dec 20, 2025
Discussion https://github.com/conda/ceps/pull/111
Implementation TBD
Requires https://github.com/conda/ceps/pull/82, https://github.com/conda/ceps/pull/133, https://github.com/conda/ceps/pull/135
+ +## Abstract + +This CEP proposes a simplified mechanism to select package variants without relying on partial build string globbing. Instead, a concrete keyword-matching strategy is devised to allow for more targeted granularity. This mechanism involves adding a new `flags` repodata record field along with the corresponding `MatchSpec` extensions. + +## Motivation + +Within a channel and subdir, a conda package for a given project release may have different builds or artifacts. These are used to either correct deficiencies in the build process, or to provide different build alternatives (e.g. linking to diverse library backends). These artifacts can be distinguished by their build string, which can be matched with glob strings. + +In principle, the solver is responsible for choosing the right variant for a given package version. However, the user may also want to force a particular variant to satisfy their needs. The solution so far has been to plant a special substring in the build string so it can be selected by the corresponding glob string. For example, given a package with a CPU and a GPU variant (`package-1.0-h123abc_cpu_0.conda` and `package-1.0-h453cbd_gpu_0.conda`), the GPU variant can be choosen by asking for `package=*=*gpu*`. + +The problem with this approach is that it doesn't scale well for more than one "flag" per build string. What if a package needs to distinguish among more than one feature? That is, not just GPU vs CPU, but also BLAS backend, MPI or licensing? The glob strings are not expressive enough for a single spec, so several ones need to be supplied (e.g. `package=*=*gpu*`, `package=*=*mkl*`, `package=*=*mpich*` and `package=*=*nogpl*`), resulting in complicated lookahead regexes that balloon in computational complexity. + +The answer to this problem is to provide a specific field that is designed to provide such selection capabilities without the expressivity and complexity problems observed above: `flags`. + +## Rationale + +... + +## Specification + +### Repodata record syntax + +The `info/index.json` file of each conda artifact MUST support a new field, `flags`, the value of which MUST be a list of non-empty strings matching the regex `^[a-z0-9\-_:+\.]+$`. Subsequently, the `schema_version` value MUST be bumped to 4. + +In recipes, this value MUST be supported in the `build` section of each output (i.e. sibling to `number` and `track_features`). + +### MatchSpec syntax changes + +Values in the `flags` field MUST be matchable by the corresponding keyword in `MatchSpec`, placed in the square brackets section. Its value MUST be a string or list of strings supporting the following syntax: + +- A string matching the regex `^[a-z0-9\-_:+\.]+$` is considered a literal match and will evaluate to true if the input string matches the target string fully. +- The asterisk symbol `*` works as a glob operator, with the same rules used for build string matching. +- A string query MAY be followed by one of the operators `=`, `!=`, `>`, `>=`, `<` and `<=`, which implement equality, inequality, greater than, greater than or equals, less than, less than or equals, respectively. They MUST be followed by a numeric value. When used, anything right side in the target strings will be interpreted as numbers and compared accordingly. The left side follows the same string matching rules as above. +- Two special prefix characters MUST be recognized with the following meaning: + - `~`: negates the match + - `?`: makes the match optional + +## Examples + +In practice, this would look like the following from a user perspective: + +```shell +conda install 'pytorch[version=">=3.1", flags=["gpu:*", "?release"]]' +``` + +## Rejected ideas + +This proposal may remind the readers of the old `features` properties in the first iterations of conda packaging. + +## References + +... + + +## Copyright + +All CEPs are explicitly [CC0 1.0 Universal](https://creativecommons.org/publicdomain/zero/1.0/). From eebc79c838885cd1eee351b325a1ffa53f6c9bbc Mon Sep 17 00:00:00 2001 From: jaimergp Date: Sat, 20 Dec 2025 23:39:57 +0100 Subject: [PATCH 02/20] Adjust index.json schema_version numbers --- cep-XXX3.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/cep-XXX3.md b/cep-XXX3.md index b7956758..fd346b5b 100644 --- a/cep-XXX3.md +++ b/cep-XXX3.md @@ -37,7 +37,7 @@ The answer to this problem is to provide a specific field that is designed to pr ### Repodata record syntax -The `info/index.json` file of each conda artifact MUST support a new field, `flags`, the value of which MUST be a list of non-empty strings matching the regex `^[a-z0-9\-_:+\.]+$`. Subsequently, the `schema_version` value MUST be bumped to 4. +The `info/index.json` file of each conda artifact MUST support a new field, `flags`, the value of which MUST be a list of non-empty strings matching the regex `^[a-z0-9\-_:+\.]+$`. Subsequently, the `schema_version` value MUST be bumped to `5`. In recipes, this value MUST be supported in the `build` section of each output (i.e. sibling to `number` and `track_features`). From 249af6e8f814220283499b62077a8a484e256545 Mon Sep 17 00:00:00 2001 From: jaimergp Date: Mon, 9 Mar 2026 17:01:05 +0100 Subject: [PATCH 03/20] Update Requires in Simplified variant selection --- cep-XXX3.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/cep-XXX3.md b/cep-XXX3.md index fd346b5b..66737b8c 100644 --- a/cep-XXX3.md +++ b/cep-XXX3.md @@ -9,10 +9,10 @@ Jaime Rodríguez-Guerra <jaime.rogue@gmail.com> Created Feb 5, 2025 - Updated Dec 20, 2025 + Updated Mar 9, 2026 Discussion https://github.com/conda/ceps/pull/111 Implementation TBD - Requires https://github.com/conda/ceps/pull/82, https://github.com/conda/ceps/pull/133, https://github.com/conda/ceps/pull/135 + Requires https://github.com/conda/ceps/pull/146 ## Abstract From b7735c56adc99051c3048c5a8115fdb399f07bf5 Mon Sep 17 00:00:00 2001 From: Wolf Vollprecht Date: Fri, 30 Jan 2026 14:36:13 +0100 Subject: [PATCH 04/20] update cep --- cep-XXX3.md | 46 ++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 46 insertions(+) diff --git a/cep-XXX3.md b/cep-XXX3.md index 66737b8c..f0498d8b 100644 --- a/cep-XXX3.md +++ b/cep-XXX3.md @@ -41,6 +41,8 @@ The `info/index.json` file of each conda artifact MUST support a new field, `fla In recipes, this value MUST be supported in the `build` section of each output (i.e. sibling to `number` and `track_features`). +Additionally, another field has to be added to the `index.json` file and `repodata_record`: `variant_order`. This field lists flags in descending priority order. During the resolution, artifacts that match on flags are sorted by the order in `variant_order`. + ### MatchSpec syntax changes Values in the `flags` field MUST be matchable by the corresponding keyword in `MatchSpec`, placed in the square brackets section. Its value MUST be a string or list of strings supporting the following syntax: @@ -52,6 +54,50 @@ Values in the `flags` field MUST be matchable by the corresponding keyword in `M - `~`: negates the match - `?`: makes the match optional +### Variant order sorting + +The variants are sorted lexicographically by matching flags from the variant order. Flags not mentioned in the variant order will be sorted lower than any flags mentioned in the variant order, and in alphabetical order. + +Example: + +``` +variant order: [release, cuda, blas:mkl, blas:openblas] + +sorted variants: + +1. [release, cuda, blas:mkl] +2. [release, cuda, blas:openblas] +3. [release, blas:mkl] +4. [release, blas:openblas] +5. [release, blas:blis] +6. [debug] +``` + +### `index.json` and `repodata_record` changes + +The records gain two new fields, `flags` and `variant_order`: + +```json +... +{ + "name": "foobar", + "version": "1.2.3", + "flags": ["gpu:cuda", "build_type:release", "blas:mkl"], + "variant_order": ["build_type:release", "blas:mkl", "gpu:*"] # rest of the flags, +} +``` + +### Changes to the `recipe.yaml` file + +```yaml +build: + string: ... + flags: ["gpu:cuda${{ cuda_version }}", "blas:${{ blas}}", "release"] + variant: + order: ["release", "blas:mkl", "blas:blis", "blas:openblas", "gpu:*"], + +``` + ## Examples In practice, this would look like the following from a user perspective: From 040ce288c3b974850bf460a91d8f85deebc961b9 Mon Sep 17 00:00:00 2001 From: Wolf Vollprecht Date: Mon, 30 Mar 2026 18:40:48 +0200 Subject: [PATCH 05/20] simplify flags for now --- cep-XXX3.md | 49 ++++++++++--------------------------------------- 1 file changed, 10 insertions(+), 39 deletions(-) diff --git a/cep-XXX3.md b/cep-XXX3.md index f0498d8b..2b07e4a8 100644 --- a/cep-XXX3.md +++ b/cep-XXX3.md @@ -17,7 +17,7 @@ ## Abstract -This CEP proposes a simplified mechanism to select package variants without relying on partial build string globbing. Instead, a concrete keyword-matching strategy is devised to allow for more targeted granularity. This mechanism involves adding a new `flags` repodata record field along with the corresponding `MatchSpec` extensions. +This CEP proposes a simplified mechanism to select package variants without relying on partial build string globbing. A package carries a list of plain string flags; a solver constraint that names a flag excludes any package that does not carry it. This mechanism involves adding a new `flags` repodata record field along with the corresponding `MatchSpec` extensions. ## Motivation @@ -37,53 +37,25 @@ The answer to this problem is to provide a specific field that is designed to pr ### Repodata record syntax -The `info/index.json` file of each conda artifact MUST support a new field, `flags`, the value of which MUST be a list of non-empty strings matching the regex `^[a-z0-9\-_:+\.]+$`. Subsequently, the `schema_version` value MUST be bumped to `5`. +The `info/index.json` file of each conda artifact MUST support a new field, `flags`, the value of which MUST be a list of non-empty strings matching the regex `^[a-z0-9_]+$`. Note that `:` is explicitly reserved for future use (e.g. `key:value` semantics) and MUST NOT appear in flag strings at this time. Subsequently, the `schema_version` value MUST be bumped to `5`. In recipes, this value MUST be supported in the `build` section of each output (i.e. sibling to `number` and `track_features`). -Additionally, another field has to be added to the `index.json` file and `repodata_record`: `variant_order`. This field lists flags in descending priority order. During the resolution, artifacts that match on flags are sorted by the order in `variant_order`. - ### MatchSpec syntax changes -Values in the `flags` field MUST be matchable by the corresponding keyword in `MatchSpec`, placed in the square brackets section. Its value MUST be a string or list of strings supporting the following syntax: - -- A string matching the regex `^[a-z0-9\-_:+\.]+$` is considered a literal match and will evaluate to true if the input string matches the target string fully. -- The asterisk symbol `*` works as a glob operator, with the same rules used for build string matching. -- A string query MAY be followed by one of the operators `=`, `!=`, `>`, `>=`, `<` and `<=`, which implement equality, inequality, greater than, greater than or equals, less than, less than or equals, respectively. They MUST be followed by a numeric value. When used, anything right side in the target strings will be interpreted as numbers and compared accordingly. The left side follows the same string matching rules as above. -- Two special prefix characters MUST be recognized with the following meaning: - - `~`: negates the match - - `?`: makes the match optional - -### Variant order sorting - -The variants are sorted lexicographically by matching flags from the variant order. Flags not mentioned in the variant order will be sorted lower than any flags mentioned in the variant order, and in alphabetical order. - -Example: - -``` -variant order: [release, cuda, blas:mkl, blas:openblas] - -sorted variants: +Values in the `flags` field MUST be matchable by the corresponding keyword in `MatchSpec`, placed in the square brackets section. Its value MUST be a string or list of strings. Each entry MUST match the regex `^[a-z0-9_]+$`. -1. [release, cuda, blas:mkl] -2. [release, cuda, blas:openblas] -3. [release, blas:mkl] -4. [release, blas:openblas] -5. [release, blas:blis] -6. [debug] -``` +Flag matching is intentionally simple: a package is excluded from consideration if it does not contain every flag listed in the `flags` constraint. There is no glob, negation, optional, or comparison syntax — a flag either matches exactly or the package is filtered out. ### `index.json` and `repodata_record` changes -The records gain two new fields, `flags` and `variant_order`: +The records gain a new `flags` field: ```json -... { "name": "foobar", "version": "1.2.3", - "flags": ["gpu:cuda", "build_type:release", "blas:mkl"], - "variant_order": ["build_type:release", "blas:mkl", "gpu:*"] # rest of the flags, + "flags": ["cuda", "release", "mkl"] } ``` @@ -92,10 +64,7 @@ The records gain two new fields, `flags` and `variant_order`: ```yaml build: string: ... - flags: ["gpu:cuda${{ cuda_version }}", "blas:${{ blas}}", "release"] - variant: - order: ["release", "blas:mkl", "blas:blis", "blas:openblas", "gpu:*"], - + flags: ["cuda", "mkl", "release"] ``` ## Examples @@ -103,9 +72,11 @@ build: In practice, this would look like the following from a user perspective: ```shell -conda install 'pytorch[version=">=3.1", flags=["gpu:*", "?release"]]' +conda install 'pytorch[version=">=3.1", flags=["cuda", "mkl"]]' ``` +Any package that does not carry both the `cuda` and `mkl` flags is excluded from the candidate set. Among the remaining candidates the solver applies its normal version and build-number preference. + ## Rejected ideas This proposal may remind the readers of the old `features` properties in the first iterations of conda packaging. From 5e3e06a9ac227a75111702df648655529a690a3f Mon Sep 17 00:00:00 2001 From: Wolf Vollprecht Date: Tue, 31 Mar 2026 19:18:28 +0200 Subject: [PATCH 06/20] update flags cep --- cep-XXX3.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/cep-XXX3.md b/cep-XXX3.md index 2b07e4a8..7a602fc5 100644 --- a/cep-XXX3.md +++ b/cep-XXX3.md @@ -31,7 +31,7 @@ The answer to this problem is to provide a specific field that is designed to pr ## Rationale -... +The chosen keyword is `flags` which makes sense as a "compile time flag". This feature is mainly relevant for compiled packages with different compile time feature selection, making `flags` a matching name. ## Specification @@ -40,12 +40,13 @@ The answer to this problem is to provide a specific field that is designed to pr The `info/index.json` file of each conda artifact MUST support a new field, `flags`, the value of which MUST be a list of non-empty strings matching the regex `^[a-z0-9_]+$`. Note that `:` is explicitly reserved for future use (e.g. `key:value` semantics) and MUST NOT appear in flag strings at this time. Subsequently, the `schema_version` value MUST be bumped to `5`. In recipes, this value MUST be supported in the `build` section of each output (i.e. sibling to `number` and `track_features`). +In recipes, it MUST be represented as a list of strings under the `build.flags = [str]` key for each package output. ### MatchSpec syntax changes Values in the `flags` field MUST be matchable by the corresponding keyword in `MatchSpec`, placed in the square brackets section. Its value MUST be a string or list of strings. Each entry MUST match the regex `^[a-z0-9_]+$`. -Flag matching is intentionally simple: a package is excluded from consideration if it does not contain every flag listed in the `flags` constraint. There is no glob, negation, optional, or comparison syntax — a flag either matches exactly or the package is filtered out. +Flag matching is intentionally simple: a package is excluded from consideration if it does not contain every flag listed in the `flags` constraint. A flag either matches the string exactly or the package is filtered out. ### `index.json` and `repodata_record` changes @@ -81,10 +82,9 @@ Any package that does not carry both the `cuda` and `mkl` flags is excluded from This proposal may remind the readers of the old `features` properties in the first iterations of conda packaging. -## References - -... +## Future plans +In the future we might extend the matching ergonomics of flags to include numeric values and key-value items. ## Copyright From 1e74dd7ff55a20b5a7bf7858e208c81c2db88241 Mon Sep 17 00:00:00 2001 From: Wolf Vollprecht Date: Wed, 15 Apr 2026 16:50:55 +0200 Subject: [PATCH 07/20] add variant order, allow single : --- cep-XXX3.md | 44 +++++++++++++++++++++++++++++++++++--------- 1 file changed, 35 insertions(+), 9 deletions(-) diff --git a/cep-XXX3.md b/cep-XXX3.md index 7a602fc5..714e20c6 100644 --- a/cep-XXX3.md +++ b/cep-XXX3.md @@ -23,12 +23,14 @@ This CEP proposes a simplified mechanism to select package variants without rely Within a channel and subdir, a conda package for a given project release may have different builds or artifacts. These are used to either correct deficiencies in the build process, or to provide different build alternatives (e.g. linking to diverse library backends). These artifacts can be distinguished by their build string, which can be matched with glob strings. -In principle, the solver is responsible for choosing the right variant for a given package version. However, the user may also want to force a particular variant to satisfy their needs. The solution so far has been to plant a special substring in the build string so it can be selected by the corresponding glob string. For example, given a package with a CPU and a GPU variant (`package-1.0-h123abc_cpu_0.conda` and `package-1.0-h453cbd_gpu_0.conda`), the GPU variant can be choosen by asking for `package=*=*gpu*`. +In principle, the solver is responsible for choosing the right variant for a given package version. However, the user may also want to force a particular variant to satisfy their needs. The solution so far has been to plant a special substring in the build string so it can be selected by the corresponding glob string. For example, given a package with a CPU and a GPU variant (`package-1.0-h123abc_cpu_0.conda` and `package-1.0-h453cbd_gpu_0.conda`), the GPU variant can be chosen by asking for `package=*=*gpu*`. The problem with this approach is that it doesn't scale well for more than one "flag" per build string. What if a package needs to distinguish among more than one feature? That is, not just GPU vs CPU, but also BLAS backend, MPI or licensing? The glob strings are not expressive enough for a single spec, so several ones need to be supplied (e.g. `package=*=*gpu*`, `package=*=*mkl*`, `package=*=*mpich*` and `package=*=*nogpl*`), resulting in complicated lookahead regexes that balloon in computational complexity. The answer to this problem is to provide a specific field that is designed to provide such selection capabilities without the expressivity and complexity problems observed above: `flags`. +Similarly, the `build.number` field is currently overloaded with "variant order". To remediate this, we introduce a new integer field specifically to order variants of the same package version - the variant order field. Like the build number, it is a single integer. + ## Rationale The chosen keyword is `flags` which makes sense as a "compile time flag". This feature is mainly relevant for compiled packages with different compile time feature selection, making `flags` a matching name. @@ -37,26 +39,49 @@ The chosen keyword is `flags` which makes sense as a "compile time flag". This f ### Repodata record syntax -The `info/index.json` file of each conda artifact MUST support a new field, `flags`, the value of which MUST be a list of non-empty strings matching the regex `^[a-z0-9_]+$`. Note that `:` is explicitly reserved for future use (e.g. `key:value` semantics) and MUST NOT appear in flag strings at this time. Subsequently, the `schema_version` value MUST be bumped to `5`. +The `info/index.json` file of each conda artifact MUST support two new fields, `flags` and `variant_order`. + +The value of the `flags` field MUST be a list of non-empty strings matching the regex `^[a-z0-9_]+(:[a-z0-9_]+)?$`. We allow a _single_ `:` for `key:value` semantics. + +The value of the `variant_order` field must be a integer. + +Subsequently, the `schema_version` value MUST be bumped to `3`. + +In recipes, these values MUST be supported in the `build` section of each output (i.e. sibling to `number` and `track_features`). -In recipes, this value MUST be supported in the `build` section of each output (i.e. sibling to `number` and `track_features`). In recipes, it MUST be represented as a list of strings under the `build.flags = [str]` key for each package output. +For the variant order `build.variant_order: int`. ### MatchSpec syntax changes -Values in the `flags` field MUST be matchable by the corresponding keyword in `MatchSpec`, placed in the square brackets section. Its value MUST be a string or list of strings. Each entry MUST match the regex `^[a-z0-9_]+$`. +Values in the `flags` field MUST be matchable by the corresponding keyword in `MatchSpec`, placed in the square brackets section. Its value MUST be a string or list of strings. Each entry MUST match the regex `^[a-z0-9_]+(:[a-z0-9_]+)?$`. Flag matching is intentionally simple: a package is excluded from consideration if it does not contain every flag listed in the `flags` constraint. A flag either matches the string exactly or the package is filtered out. +### Package solver sort order changes + +The packages sorting MUST take into account the `variant_order` field. If not present, it is assumed `0`. + +If the `variant_order` field is present, sorting MUST respect (in the following order): + +- tracked features +- version +- build number +- variant order +- first order dependency versions +- timestamp + ### `index.json` and `repodata_record` changes -The records gain a new `flags` field: +The records gain a new `flags` and `variant_order` field: ```json { "name": "foobar", "version": "1.2.3", - "flags": ["cuda", "release", "mkl"] + ..., + "flags": ["cuda", "release", "blas:mkl"], + "variant_order": 123 } ``` @@ -65,7 +90,8 @@ The records gain a new `flags` field: ```yaml build: string: ... - flags: ["cuda", "mkl", "release"] + flags: ["cuda", "blas:mkl", "release"] + variant_order: 123 ``` ## Examples @@ -73,10 +99,10 @@ build: In practice, this would look like the following from a user perspective: ```shell -conda install 'pytorch[version=">=3.1", flags=["cuda", "mkl"]]' +conda install 'pytorch[version=">=3.1", flags=["cuda", "blas:*"]]' ``` -Any package that does not carry both the `cuda` and `mkl` flags is excluded from the candidate set. Among the remaining candidates the solver applies its normal version and build-number preference. +Any package that does not carry both the `cuda` and a flag starting with `blas:` is excluded from the candidate set. Among the remaining candidates the solver applies its normal version and build-number and variant order preference sorting. ## Rejected ideas From 572922dd585821df78e6a8c69fff1e898ed50d53 Mon Sep 17 00:00:00 2001 From: Wolf Vollprecht Date: Wed, 15 Apr 2026 16:57:09 +0200 Subject: [PATCH 08/20] extend sorting --- cep-XXX3.md | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/cep-XXX3.md b/cep-XXX3.md index 714e20c6..434ec951 100644 --- a/cep-XXX3.md +++ b/cep-XXX3.md @@ -43,7 +43,7 @@ The `info/index.json` file of each conda artifact MUST support two new fields, ` The value of the `flags` field MUST be a list of non-empty strings matching the regex `^[a-z0-9_]+(:[a-z0-9_]+)?$`. We allow a _single_ `:` for `key:value` semantics. -The value of the `variant_order` field must be a integer. +The value of the `variant_order` field must be a positive or negative integer. Subsequently, the `schema_version` value MUST be bumped to `3`. @@ -60,16 +60,16 @@ Flag matching is intentionally simple: a package is excluded from consideration ### Package solver sort order changes -The packages sorting MUST take into account the `variant_order` field. If not present, it is assumed `0`. +The packages sorting MUST take into account the `variant_order` field. If not present, it is assumed `0`. A package with a higher `variant_order` is preferred over those with lower values. If the `variant_order` field is present, sorting MUST respect (in the following order): -- tracked features -- version -- build number -- variant order -- first order dependency versions -- timestamp +- tracked features comparison (fewer track features preferred) +- version (higher version preferred) +- build number (higher preferred) +- variant order (higher preferred) +- first order dependency versions (higher dependency versions preferred) +- timestamp (newer package preferred) ### `index.json` and `repodata_record` changes From 540054fcc1df1fa9b600b0d6627c04ecad8ec681 Mon Sep 17 00:00:00 2001 From: Wolf Vollprecht Date: Wed, 15 Apr 2026 17:03:39 +0200 Subject: [PATCH 09/20] improve sorting --- cep-XXX3.md | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/cep-XXX3.md b/cep-XXX3.md index 434ec951..5c4fb2d2 100644 --- a/cep-XXX3.md +++ b/cep-XXX3.md @@ -60,16 +60,16 @@ Flag matching is intentionally simple: a package is excluded from consideration ### Package solver sort order changes -The packages sorting MUST take into account the `variant_order` field. If not present, it is assumed `0`. A package with a higher `variant_order` is preferred over those with lower values. +Package sorting MUST take into account the `variant_order` field. If absent, it defaults to `0`. Higher values are preferred over lower values. -If the `variant_order` field is present, sorting MUST respect (in the following order): +Sorting MUST use the following criteria, in order of decreasing precedence: -- tracked features comparison (fewer track features preferred) -- version (higher version preferred) -- build number (higher preferred) -- variant order (higher preferred) -- first order dependency versions (higher dependency versions preferred) -- timestamp (newer package preferred) +1. Tracked features (fewer is preferred) +2. Version (higher is preferred) +3. Build number (higher is preferred) +4. Variant order (higher is preferred) +5. First-order dependency versions (higher is preferred) +6. Timestamp (newer is preferred) ### `index.json` and `repodata_record` changes From 1df22903edbb7007dbc3a3851493f96eedd6a5f2 Mon Sep 17 00:00:00 2001 From: Wolf Vollprecht Date: Fri, 17 Apr 2026 15:54:31 +0200 Subject: [PATCH 10/20] rename to variant_priority, and add under `variant` field in recipe --- cep-XXX3.md | 21 +++++++++++++-------- 1 file changed, 13 insertions(+), 8 deletions(-) diff --git a/cep-XXX3.md b/cep-XXX3.md index 5c4fb2d2..573ef3b0 100644 --- a/cep-XXX3.md +++ b/cep-XXX3.md @@ -39,18 +39,18 @@ The chosen keyword is `flags` which makes sense as a "compile time flag". This f ### Repodata record syntax -The `info/index.json` file of each conda artifact MUST support two new fields, `flags` and `variant_order`. +The `info/index.json` file of each conda artifact MUST support two new fields, `flags` and `variant_priority`. The value of the `flags` field MUST be a list of non-empty strings matching the regex `^[a-z0-9_]+(:[a-z0-9_]+)?$`. We allow a _single_ `:` for `key:value` semantics. -The value of the `variant_order` field must be a positive or negative integer. +The value of the `variant_priority` field must be a positive or negative integer. Subsequently, the `schema_version` value MUST be bumped to `3`. In recipes, these values MUST be supported in the `build` section of each output (i.e. sibling to `number` and `track_features`). In recipes, it MUST be represented as a list of strings under the `build.flags = [str]` key for each package output. -For the variant order `build.variant_order: int`. +For the variant order `build.variant_priority: int`. ### MatchSpec syntax changes @@ -60,20 +60,20 @@ Flag matching is intentionally simple: a package is excluded from consideration ### Package solver sort order changes -Package sorting MUST take into account the `variant_order` field. If absent, it defaults to `0`. Higher values are preferred over lower values. +Package sorting MUST take into account the `variant_priority` field. If absent, it defaults to `0`. Higher values are preferred over lower values. Sorting MUST use the following criteria, in order of decreasing precedence: 1. Tracked features (fewer is preferred) 2. Version (higher is preferred) 3. Build number (higher is preferred) -4. Variant order (higher is preferred) +4. Variant priority (higher is preferred) 5. First-order dependency versions (higher is preferred) 6. Timestamp (newer is preferred) ### `index.json` and `repodata_record` changes -The records gain a new `flags` and `variant_order` field: +The records gain a new `flags` and `variant_priority` field: ```json { @@ -81,17 +81,20 @@ The records gain a new `flags` and `variant_order` field: "version": "1.2.3", ..., "flags": ["cuda", "release", "blas:mkl"], - "variant_order": 123 + "variant_priority": 123 } ``` ### Changes to the `recipe.yaml` file +Note: the `priority` field is used to set the `variant_priority` value in the `info/index.json`. + ```yaml build: string: ... flags: ["cuda", "blas:mkl", "release"] - variant_order: 123 + variant: + priority: 123 ``` ## Examples @@ -112,6 +115,8 @@ This proposal may remind the readers of the old `features` properties in the fir In the future we might extend the matching ergonomics of flags to include numeric values and key-value items. +For the `variant.priority` field in recipes, we plan to integrate a smart algorithm in the recipe executor itself, that determines a variant order value based on the flags / variants used as a follow up CEP. + ## Copyright All CEPs are explicitly [CC0 1.0 Universal](https://creativecommons.org/publicdomain/zero/1.0/). From c6fbfc204b473c7ab7b3932bcd27c4a76a5468f9 Mon Sep 17 00:00:00 2001 From: Wolf Vollprecht Date: Fri, 24 Apr 2026 16:45:54 +0200 Subject: [PATCH 11/20] remove variant_priority for now --- cep-XXX3.md | 31 +++---------------------------- 1 file changed, 3 insertions(+), 28 deletions(-) diff --git a/cep-XXX3.md b/cep-XXX3.md index 573ef3b0..9c81c7d2 100644 --- a/cep-XXX3.md +++ b/cep-XXX3.md @@ -29,8 +29,6 @@ The problem with this approach is that it doesn't scale well for more than one " The answer to this problem is to provide a specific field that is designed to provide such selection capabilities without the expressivity and complexity problems observed above: `flags`. -Similarly, the `build.number` field is currently overloaded with "variant order". To remediate this, we introduce a new integer field specifically to order variants of the same package version - the variant order field. Like the build number, it is a single integer. - ## Rationale The chosen keyword is `flags` which makes sense as a "compile time flag". This feature is mainly relevant for compiled packages with different compile time feature selection, making `flags` a matching name. @@ -39,41 +37,25 @@ The chosen keyword is `flags` which makes sense as a "compile time flag". This f ### Repodata record syntax -The `info/index.json` file of each conda artifact MUST support two new fields, `flags` and `variant_priority`. +The `info/index.json` file of each conda artifact MUST support two new fields, `flags`. The value of the `flags` field MUST be a list of non-empty strings matching the regex `^[a-z0-9_]+(:[a-z0-9_]+)?$`. We allow a _single_ `:` for `key:value` semantics. -The value of the `variant_priority` field must be a positive or negative integer. - Subsequently, the `schema_version` value MUST be bumped to `3`. In recipes, these values MUST be supported in the `build` section of each output (i.e. sibling to `number` and `track_features`). In recipes, it MUST be represented as a list of strings under the `build.flags = [str]` key for each package output. -For the variant order `build.variant_priority: int`. ### MatchSpec syntax changes -Values in the `flags` field MUST be matchable by the corresponding keyword in `MatchSpec`, placed in the square brackets section. Its value MUST be a string or list of strings. Each entry MUST match the regex `^[a-z0-9_]+(:[a-z0-9_]+)?$`. +Values in the `flags` field MUST be matchable by the corresponding keyword in ``, placed in the square brackets section. Its value MUST be a string or list of strings. Each entry MUST match the regex `^[a-z0-9_]+(:[a-z0-9_]+)?$`. Flag matching is intentionally simple: a package is excluded from consideration if it does not contain every flag listed in the `flags` constraint. A flag either matches the string exactly or the package is filtered out. -### Package solver sort order changes - -Package sorting MUST take into account the `variant_priority` field. If absent, it defaults to `0`. Higher values are preferred over lower values. - -Sorting MUST use the following criteria, in order of decreasing precedence: - -1. Tracked features (fewer is preferred) -2. Version (higher is preferred) -3. Build number (higher is preferred) -4. Variant priority (higher is preferred) -5. First-order dependency versions (higher is preferred) -6. Timestamp (newer is preferred) - ### `index.json` and `repodata_record` changes -The records gain a new `flags` and `variant_priority` field: +The records gain a new `flags`: ```json { @@ -81,20 +63,15 @@ The records gain a new `flags` and `variant_priority` field: "version": "1.2.3", ..., "flags": ["cuda", "release", "blas:mkl"], - "variant_priority": 123 } ``` ### Changes to the `recipe.yaml` file -Note: the `priority` field is used to set the `variant_priority` value in the `info/index.json`. - ```yaml build: string: ... flags: ["cuda", "blas:mkl", "release"] - variant: - priority: 123 ``` ## Examples @@ -115,8 +92,6 @@ This proposal may remind the readers of the old `features` properties in the fir In the future we might extend the matching ergonomics of flags to include numeric values and key-value items. -For the `variant.priority` field in recipes, we plan to integrate a smart algorithm in the recipe executor itself, that determines a variant order value based on the flags / variants used as a follow up CEP. - ## Copyright All CEPs are explicitly [CC0 1.0 Universal](https://creativecommons.org/publicdomain/zero/1.0/). From 37fb9a8e96cfd8436c557c198b324948012be557 Mon Sep 17 00:00:00 2001 From: jaimergp Date: Fri, 24 Apr 2026 20:35:51 +0200 Subject: [PATCH 12/20] Some pre-commit fixes --- cep-XXX3.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/cep-XXX3.md b/cep-XXX3.md index 9c81c7d2..ad72764e 100644 --- a/cep-XXX3.md +++ b/cep-XXX3.md @@ -25,7 +25,8 @@ Within a channel and subdir, a conda package for a given project release may hav In principle, the solver is responsible for choosing the right variant for a given package version. However, the user may also want to force a particular variant to satisfy their needs. The solution so far has been to plant a special substring in the build string so it can be selected by the corresponding glob string. For example, given a package with a CPU and a GPU variant (`package-1.0-h123abc_cpu_0.conda` and `package-1.0-h453cbd_gpu_0.conda`), the GPU variant can be chosen by asking for `package=*=*gpu*`. -The problem with this approach is that it doesn't scale well for more than one "flag" per build string. What if a package needs to distinguish among more than one feature? That is, not just GPU vs CPU, but also BLAS backend, MPI or licensing? The glob strings are not expressive enough for a single spec, so several ones need to be supplied (e.g. `package=*=*gpu*`, `package=*=*mkl*`, `package=*=*mpich*` and `package=*=*nogpl*`), resulting in complicated lookahead regexes that balloon in computational complexity. +The problem with this approach is that it doesn't scale well for more than one "flag" per build string. What if a package needs to distinguish among more than one feature? That is, not just GPU vs CPU, but also BLAS backend, MPI or licensing? +The glob strings are not expressive enough for a single spec, so several ones need to be supplied (e.g. `package=*=*gpu*`, `package=*=*mkl*`, `package=*=*mpich*` and `package=*=*nogpl*`), resulting in complicated lookahead regexes that balloon in computational complexity. The answer to this problem is to provide a specific field that is designed to provide such selection capabilities without the expressivity and complexity problems observed above: `flags`. From b9fc0ccfce7ec10d55b10bcd14c36ce92aca6841 Mon Sep 17 00:00:00 2001 From: jaimergp Date: Mon, 27 Apr 2026 10:13:58 +0200 Subject: [PATCH 13/20] Add `variant_priority` to Rejected Ideas --- cep-XXX3.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/cep-XXX3.md b/cep-XXX3.md index ad72764e..51b1a2ac 100644 --- a/cep-XXX3.md +++ b/cep-XXX3.md @@ -87,7 +87,9 @@ Any package that does not carry both the `cuda` and a flag starting with `blas:` ## Rejected ideas -This proposal may remind the readers of the old `features` properties in the first iterations of conda packaging. +This proposal may remind the readers of the old `features` properties in the first iterations of conda packaging. This is not a reimplementation. + +A previous iteration of this CEP included a `variant_priority` field to clearly identify which the priority of each variant build, instead of relying on `build_number` or `track_features` hackery. This was removed because its complexity deserves its own separate CEP, to be tackled in the future. ## Future plans From d7fc2051bddb7ff5e6bd529568837f664df7fadd Mon Sep 17 00:00:00 2001 From: jaimergp Date: Mon, 27 Apr 2026 10:24:36 +0200 Subject: [PATCH 14/20] Clarify record syntax --- cep-XXX3.md | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/cep-XXX3.md b/cep-XXX3.md index 51b1a2ac..a3809e4f 100644 --- a/cep-XXX3.md +++ b/cep-XXX3.md @@ -42,11 +42,9 @@ The `info/index.json` file of each conda artifact MUST support two new fields, ` The value of the `flags` field MUST be a list of non-empty strings matching the regex `^[a-z0-9_]+(:[a-z0-9_]+)?$`. We allow a _single_ `:` for `key:value` semantics. -Subsequently, the `schema_version` value MUST be bumped to `3`. +Subsequently, the CEP 34 `info/index.json`'s `schema_version` value MUST be bumped to `3`. -In recipes, these values MUST be supported in the `build` section of each output (i.e. sibling to `number` and `track_features`). - -In recipes, it MUST be represented as a list of strings under the `build.flags = [str]` key for each package output. +In recipes, these values MUST be supported by a `flags` key in the `build` section of each output (i.e. sibling to `number` and `track_features`) with a list of strings as the value; i.e. `build.flags = [str]`. ### MatchSpec syntax changes From 1836f46634a843fb8e2d698dd73bc0f2af736b80 Mon Sep 17 00:00:00 2001 From: jaimergp Date: Mon, 27 Apr 2026 10:24:42 +0200 Subject: [PATCH 15/20] Clarify globbing --- cep-XXX3.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/cep-XXX3.md b/cep-XXX3.md index a3809e4f..6d7ea8d2 100644 --- a/cep-XXX3.md +++ b/cep-XXX3.md @@ -48,9 +48,9 @@ In recipes, these values MUST be supported by a `flags` key in the `build` secti ### MatchSpec syntax changes -Values in the `flags` field MUST be matchable by the corresponding keyword in ``, placed in the square brackets section. Its value MUST be a string or list of strings. Each entry MUST match the regex `^[a-z0-9_]+(:[a-z0-9_]+)?$`. +Values in the `flags` field MUST be matchable by the corresponding keyword in `MatchSpec`, placed in the square brackets section. Its value MUST be a string or list of strings. Each entry MUST match the regex `^[a-z0-9_\*]+(:[a-z0-9_\*]+)?$`. The `*` character is a glob operator, with the same meaning as in CEP 29 "String Matching". -Flag matching is intentionally simple: a package is excluded from consideration if it does not contain every flag listed in the `flags` constraint. A flag either matches the string exactly or the package is filtered out. +Flag matching is intentionally simple: a package is excluded from consideration if it does not contain every flag listed in the `flags` constraint. A flag either matches the string (exactly or globbed) or the package is filtered out. ### `index.json` and `repodata_record` changes From be2b1a8268b3308a37a944e32e6cafa7d03ff577 Mon Sep 17 00:00:00 2001 From: jaimergp Date: Mon, 27 Apr 2026 10:25:44 +0200 Subject: [PATCH 16/20] Merge branch 'repodata-next' of github.com:wolfv/ceps into pr/wolfv/111 --- cep-XXX3.md | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/cep-XXX3.md b/cep-XXX3.md index 6d7ea8d2..5a106e54 100644 --- a/cep-XXX3.md +++ b/cep-XXX3.md @@ -91,7 +91,11 @@ A previous iteration of this CEP included a `variant_priority` field to clearly ## Future plans -In the future we might extend the matching ergonomics of flags to include numeric values and key-value items. +Several extensions are deferred to future revisions: + +- Richer flag matching: numeric values, key-value items, and comparison operators (>, <) for numeric matches. +- Additional operators: ? for "match if present" and ! to exclude a flag. +- Variant prioritization: a variant_priority field to disambiguate among many matching variants. We could not agree on the sort order, so this is deferred for further discussion. ## Copyright From ff054a05df06df7c1b6938cfec2dd7609f3078a9 Mon Sep 17 00:00:00 2001 From: jaimergp Date: Mon, 27 Apr 2026 10:26:59 +0200 Subject: [PATCH 17/20] Bump update dates --- cep-XXX3.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/cep-XXX3.md b/cep-XXX3.md index 5a106e54..289af2e6 100644 --- a/cep-XXX3.md +++ b/cep-XXX3.md @@ -9,7 +9,7 @@ Jaime Rodríguez-Guerra <jaime.rogue@gmail.com> Created Feb 5, 2025 - Updated Mar 9, 2026 + Updated Apr 27, 2026 Discussion https://github.com/conda/ceps/pull/111 Implementation TBD Requires https://github.com/conda/ceps/pull/146 From dbb433967b44741bab21b65236b8488c1abf6047 Mon Sep 17 00:00:00 2001 From: jaimergp Date: Mon, 27 Apr 2026 10:30:33 +0200 Subject: [PATCH 18/20] Reconcile rejected ideas and future plans --- cep-XXX3.md | 10 ++++------ 1 file changed, 4 insertions(+), 6 deletions(-) diff --git a/cep-XXX3.md b/cep-XXX3.md index 289af2e6..f965afed 100644 --- a/cep-XXX3.md +++ b/cep-XXX3.md @@ -11,7 +11,7 @@ Created Feb 5, 2025 Updated Apr 27, 2026 Discussion https://github.com/conda/ceps/pull/111 - Implementation TBD + Implementation NA Requires https://github.com/conda/ceps/pull/146 @@ -87,15 +87,13 @@ Any package that does not carry both the `cuda` and a flag starting with `blas:` This proposal may remind the readers of the old `features` properties in the first iterations of conda packaging. This is not a reimplementation. -A previous iteration of this CEP included a `variant_priority` field to clearly identify which the priority of each variant build, instead of relying on `build_number` or `track_features` hackery. This was removed because its complexity deserves its own separate CEP, to be tackled in the future. - ## Future plans Several extensions are deferred to future revisions: -- Richer flag matching: numeric values, key-value items, and comparison operators (>, <) for numeric matches. -- Additional operators: ? for "match if present" and ! to exclude a flag. -- Variant prioritization: a variant_priority field to disambiguate among many matching variants. We could not agree on the sort order, so this is deferred for further discussion. +- Richer flag matching: numeric values, key-value items, and comparison operators (`>`, `<`) for numeric matches. +- Additional operators: `?` for "match if present" and `!` to exclude a flag. +- Variant prioritization: a `variant_priority` field to disambiguate among many matching variants. We could not agree on the sort order, so this is deferred for further discussion. ## Copyright From 5108f1378f4b6795a918bf06d7b1508103e4b138 Mon Sep 17 00:00:00 2001 From: jaimergp Date: Mon, 27 Apr 2026 10:56:02 +0200 Subject: [PATCH 19/20] Rename --- cep-XXX3.md => cep-XXXX.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) rename cep-XXX3.md => cep-XXXX.md (98%) diff --git a/cep-XXX3.md b/cep-XXXX.md similarity index 98% rename from cep-XXX3.md rename to cep-XXXX.md index f965afed..71a67706 100644 --- a/cep-XXX3.md +++ b/cep-XXXX.md @@ -10,7 +10,7 @@ Created Feb 5, 2025 Updated Apr 27, 2026 - Discussion https://github.com/conda/ceps/pull/111 + Discussion https://github.com/conda/ceps/pull/111, https://github.com/conda/ceps/pull/166 Implementation NA Requires https://github.com/conda/ceps/pull/146 From cf33d26eeaafe3cb16ae9c97c29db93842a11a0a Mon Sep 17 00:00:00 2001 From: jaimergp Date: Thu, 14 May 2026 16:26:02 +0200 Subject: [PATCH 20/20] Mint as CEP 45 --- README.md | 1 + cep-XXXX.md => cep-0045.md | 7 +++---- 2 files changed, 4 insertions(+), 4 deletions(-) rename cep-XXXX.md => cep-0045.md (96%) diff --git a/README.md b/README.md index f17ec0a0..be5ce39b 100644 --- a/README.md +++ b/README.md @@ -55,6 +55,7 @@ for conda's implementation, all major changes should be submitted as | [0042](cep-0042.md) | Channel relations in repodata | | [0043](cep-0043.md) | Conditional dependencies | | [0044](cep-0044.md) | Optional dependency groups | +| [0045](cep-0045.md) | Simplified variant selection | ## References diff --git a/cep-XXXX.md b/cep-0045.md similarity index 96% rename from cep-XXXX.md rename to cep-0045.md index 71a67706..31939855 100644 --- a/cep-XXXX.md +++ b/cep-0045.md @@ -1,18 +1,17 @@ -# CEP XXXX - Simplified variant selection +# CEP 45 - Simplified variant selection - + - + -
Title Simplified variant selection
Status Draft
Status Accepted
Author(s) Wolf Vollprecht <w.vollprecht@gmail.com>, Bas Zalmstra <bas@prefix.dev>, Jaime Rodríguez-Guerra <jaime.rogue@gmail.com>
Created Feb 5, 2025
Updated Apr 27, 2026
Updated May 14, 2026
Discussion https://github.com/conda/ceps/pull/111, https://github.com/conda/ceps/pull/166
Implementation NA
Requires https://github.com/conda/ceps/pull/146
## Abstract