CEP 39 - A new recipe format (part 3): `jinja` functions in recipes by wolfv · Pull Request #71 · conda/ceps

wolfv · 2024-04-12T15:50:41Z

No description provided.

h-vetinari · 2024-04-12T20:43:19Z

I do think that for the future (ie. rattler-build) we'll have an upcoming CEP to define this feature better.

Is this that CEP? In any case, I don't see {{ stdlib(...) }} in the diff, and I think it needs to be covered in some way, given that we're now rolling this out at scale.

wolfv · 2024-04-13T08:43:04Z

Yep, this is the one. Do you want to have it as a separate function or rolled into compiler? I prefer to roll it into compiler...

h-vetinari · 2024-04-13T23:30:23Z

Yep, this is the one. Do you want to have it as a separate function or rolled into compiler? I prefer to roll it into compiler...

I'd prefer a separate function, because it's less magic to have to understand (i.e. "why do I need to specify c_stdlib_version and how does that affect my build environment?" is much easier to answer if there is a {{ stdlib("c") }} under build:); also, the coupling of compiler and stdlib versions is entirely incidental (at least for C), so I'd like to not conflate them conceptually.

wolfv · 2024-04-14T05:59:57Z

My points:

There are already two variables for compilers (c_compiler and c_compiler_version). Adding additional 2 variables (that are non-mandatory) is fine IMO, especially if it's well-documented.
It complicates things for beginners. I think recipes would continue to work without adding the stdlib (correct?). So it's something they should add but don't have to add. For compiler it's clear - the recipe will not work without a compiler.
In most cases, it will always be added when there is a compiler.

I can imagine two behaviors:

We control the stdlib behavior from the variant_config. This means, if the variant config contains entries for c_stdlib / c_stdlib_version we use them from the compiler function.
We add additional arguments to the jinja function, so that compiler('c', stdlib='glibc').

Note that while you are correctly pointing out a dislike about magical behavior, we are printing a lot more information, clearly displayed, with rattler-build. So we can make it obvious where the stdlib comes from (we already point out that the clangxx_osx-arm64 package comes from the compiler, for example).

And lastly, I just think it's a matter of writing really good documentation.

h-vetinari · 2024-04-14T08:29:36Z

I think recipes would continue to work without adding the stdlib (correct?).

Define "work". When we increase the c_stdlib_version on linux and macos later this year, the artefacts produced by a recipe without {{ stdlib("c") }} would continue to work on modern systems, but would be broken on systems with a too-old C standard library, because it's through the meta-packages pulled in by stdlib that we set the correct run-exports in the artefact metadata.

It would only affect very old systems, but I'd still consider that broken.

So it's something they should add but don't have to add. For compiler it's clear - the recipe will not work without a compiler.

For me it's the opposite here - a compiled recipe needs (in 99.9% of cases) a C standard library just as much as it needs a compiler. Up until now this was just an ambient assumption built-in to our infrastructure, but I'd prefer people to learn this at the same time as the necessity of {{ compiler(...) }} - the two go hand in hand (as implied by having the same "c" argument, and the same way the keys are constructed), even though the relevant versions are completely independent from each other.

I can imagine two behaviors:

We add additional arguments to the jinja function, so that compiler('c', stdlib='glibc').

I really dislike this, because it would replace the nicely universal

    - {{ compiler('c') }}
    - {{ stdlib('c') }}

with a horrible mess of platform-specific selectors (or however the syntax for that ends up looking like in the new recipe format):

    - {{ compiler('c', stdlib='glibc') }}                     # [linux]
    - {{ compiler('c', stdlib='macosx_deployment_target') }}  # [osx]
    - {{ compiler('c', stdlib='vs') }}                        # [win]

We control the stdlib behavior from the variant_config. This means, if the variant config contains entries for c_stdlib / c_stdlib_version we use them from the compiler function.

I agree that this is technically feasible, and with good output and documentation would be a workable solution. When I say I prefer to keep {{ stdlib("c") }} explicit, this is not an all-or-nothing statement, but more a 70:30 kind of preference over the implicit handling through the compiler.

One reason that pushes this from a 60:40 (based on "less magic") to a 70:30 preference is that the C stdlib has a special role, in the sense that it can become necessary even without an explicit C compiler. In other words, we'd have to cover cases where a recipe with only {{ compiler("cxx") }} or {{ compiler("rust") }} also includes c_stdlib{,_version}.

JeanChristopheMorinPerso · 2024-05-06T16:59:32Z

+
+## The `cdt` function
+
+cdt stands for "core dependency tree" packages. These are repackaged from Centos.


Does "centos" have to be mentioned here? CDT packages could technically contain binaries from any Linux distribution right? I'm also wondering if we should add a link to some documentation that describe CDT more in details?

+1 to the above.

JeanChristopheMorinPerso · 2024-05-06T17:03:00Z

+
+Where `cdt_name` and `cdt_arch` are loaded from the variant config. If they are undefined, they default to:
+
+- `cos6` for `cdt_name` on `x86_64` and `x86`, otherwise `cos7`


I feel a little bit uneasy with having default values hardcoded here. Do we really need to define default values in the spec? I feel like this kind of thing should be explicitly defined in an ecosystem (the central variant config in an ecosystem) instead of relying on some defaults.

I agree that we should NOT have default values for CDTs. The choice of CentOS 6 and 7 was made by Anaconda and conda-forge long ago, but I don't think we should necessarily impose such choices on the ecosystem as a whole.

Further, imposing these defaults means we either have to indefinitely support CDTs from EOL Linux distributions (CentOS 6 hit EOL in 2020-Nov, and CentOS 7 will hit EOL in 2024-Jun), or the evaluation of ${{ cdt("...") }} changes depending on the version of {conda,rattler)-build you use. Neither of those outcomes seems ideal.

Sure, we can get rid of this. I think we adopted this from conda-build where these values are also there by default.

FWIW, in the new CDTs for Alma 8, we're planning to get rid of cdt_name.

I'd also leave/move these out (see comment regarding defaults for compilers above).

@h-vetinari - curious what that means exactly? A change to conda-build?

I'll remove the defaults from the CEP!

JeanChristopheMorinPerso · 2024-05-06T17:10:27Z

+- `cos6` for `cdt_name` on `x86_64` and `x86`, otherwise `cos7`
+- To the `platform::arch` for `cdt_arch`, except for `x86` where it defaults to `i686`.
+
+## The `pin_subpackage` function


pin_subpackage is defined in another section. This section seems to only define the arguments of pin_subpackage and pin_compatible?

JeanChristopheMorinPerso · 2024-05-06T17:11:44Z

+  host:
+    - numpy
+  run:
+    - ${{ pin_compatible('numpy', exact=True) }}


Does it support min_pin and max_pin? It's unclear right now in this spec.

Yes, it does. Both use the same pin specification as mentioned earlier.

JeanChristopheMorinPerso · 2024-05-06T17:12:09Z

+      version: "1.2.3"
+    requirements:
+      run:
+        - ${{ pin_subpackage('libfoo', exact=True) }}


Does it support min_pin and max_pin? It's unclear right now in this spec.

They both are derived from the same "pin" function. So yeah, they both support the same inputs.

JeanChristopheMorinPerso · 2024-05-06T17:12:39Z

+
+## The `cmp` function
+
+The `cmp` function is used to compare two versions. It returns `True` if the comparison is true and `False` otherwise.


Should we add an example of how it can be used?

Yes, what operators are available and what is the syntax?

Added some examples!

Ah so it's more like a version matcher. When I read compare I thought we were comparing two packages together like "is numpy's version greater than python's", which didn't make much sense to me.

But this is more like asking "Does the version of Python match this version expression?". So in that sense the name could be better expressed as:

version(python, "<3.8")

version_match(python, "<3.8")

match_version(python, "<3.8")

satisfy(python, "<3.8")

Also this assumes that python will be represented by a concrete version x.y.z and not a spec like x.y.* right? IOW, it's already resolved to a specific package record.

If that's the case, should we specify that this function can only return after the solve has completed?

JeanChristopheMorinPerso · 2024-05-06T17:15:21Z

+
+You can also check for the existence of an environment variable:
+
+- `${{ env.exists("MY_ENV_VAR") }}` will return `true` if the environment variable `MY_ENV_VAR` is set and `false` otherwise.


Nitpick. false and False are both used in the spec. It's hard to know if it returns a string or a bool.

I added the word boolean true to make it clear that this is a bool value.

chenghlee · 2024-05-08T16:25:32Z

+The variant config could look something like:
+
+```
+c_compiler:
+  - gcc
+c_compiler_version:
+  - "8.9"
+cxx_compiler:
+  - clang
+cxx_compiler_version:
+  - "12"
+```


This seems a bit out of place; I think the example variant config should be provided before using its values to evaluate the function.

Additionally, we should provide an example variant config showing how to select different compilers for different platforms; e.g., are we keeping the conda_build_config.yaml mechanism?

c_compiler: - gcc # [linux] - clang # [osx] - vc # [win] c_compiler_version: - "14.1" # [linux] - "17.0" # [osx] - "2022" # [win]

yeah we need to write a spec for the variant config (or even more global "config"). But IMO that shouldn't be part of this CEP.

chenghlee · 2024-05-08T16:28:12Z

+
+## The compiler function
+
+The compiler function is used to stick together a compiler from {lang}_compiler and {lang}_compiler_version


Which, if any, of {lang}_compiler and {lang}_compiler_version must be specified in the variant configuration? If either or both are optional, the CEP should describe what happens in the case when the user/recipe does not provide a value (i.e., are there default values?).

chenghlee · 2024-05-08T16:28:55Z

+
+## The `cdt` function
+
+cdt stands for "core dependency tree" packages. These are repackaged from Centos.


+1 to the above.

chenghlee · 2024-05-08T16:45:52Z

+
+Where `cdt_name` and `cdt_arch` are loaded from the variant config. If they are undefined, they default to:
+
+- `cos6` for `cdt_name` on `x86_64` and `x86`, otherwise `cos7`


I agree that we should NOT have default values for CDTs. The choice of CentOS 6 and 7 was made by Anaconda and conda-forge long ago, but I don't think we should necessarily impose such choices on the ecosystem as a whole.

Further, imposing these defaults means we either have to indefinitely support CDTs from EOL Linux distributions (CentOS 6 hit EOL in 2020-Nov, and CentOS 7 will hit EOL in 2024-Jun), or the evaluation of ${{ cdt("...") }} changes depending on the version of {conda,rattler)-build you use. Neither of those outcomes seems ideal.

chenghlee · 2024-05-08T17:01:00Z

+- `min_pin`: The lower bound of the dependency spec. This is expressed as a `x.x....` version where the `x` are filled in from the corresponding version of the package. For example, `x.x` would be `1.2` for a package version `1.2.3`. The resulting pin spec would look like `>=1.2` for the `min_pin` argument of `x.x` and a version of `1.2.3` for the package.
+- `max_pin`: This defines the upper bound and follows the same `x.x` semantics but adds `+1` to the last segment. For example, `x.x` would be `1.(2 + 1)` for a package version `1.2.3`. The resulting pin spec would look like `<1.3` for the `max_pin` argument of `x.x` and a version of `1.2.3` for the package.


There seems a strong but unstated assumption here that version strings are integers separated by dots (.). If that is the case, that assumption should be explicitly stated and guidance provided as to what should happen if the assumption is violated. (Just sayin', because 1!24.4.0+91_gbfa8ccdce is a perfectly valid PEP-440/conda package version string.)

Also, as written, it seems like min_pin and max_pin do not support an explicit version; e.g., recipes cannot simply have max_pin="1.2.3".

Also, as written, it seems like min_pin and max_pin do not support an explicit version; e.g., recipes cannot simply have max_pin="1.2.3".

Conda's existing pin_compatible jinja has both {min,max}_pin as well as {lower,upper}_bound, in order to be able to distinguish 'x.x' from an explicit version limit.

Perhaps it's easier to understand to overload both styles into two kwargs (one for lower, one for upper), and allow either 'x(.x)*' or an explicit version number? It's more work to implement, but less confusing IMO (otherwise it's easy to specify conflicting things e.g. by setting both max_pin and upper_bound)

Wow, I wasn't aware! I found a few places where it's used but I don't know how it works. For example, in the onnx-feedstock it says:

- {{ pin_compatible('numpy', lower_bound='1.19', upper_bound='3.0') }} # [py>38]

Does this still take the lower bound from the build-time resolution? Or is this equivalent to just writing numpy >=1.19,<3.0?

(https://github.com/conda-forge/onnx-feedstock/blob/49b7f95ea64d2b0e174f23ac62fe0f59441491d7/recipe/meta.yaml#L49)

I also don't really understand why we would want this in the function. One can just as well write something like:

- numpy >=1.5 - ${{ pin_compatible("numpy", max_pin="x.x", min_pin="x.x.x") }}

If one needs additional (hard-coded) constraints. The solver doesn't really care about more constraints.

@h-vetinari I'm on board with your proposal.

I suppose that could in theory create a problem if some upstream project decided to the string literals x.x, x.y, and x.z as their versions, but if that ever happens, I'm happy to say "NOPE" to attempting to package it. 😆

@wolfv, given your changes in 0fda213, I presume you're not on board with this idea (or didn't see this thread)?

If you want to keep {min,max}_pin separate from {lower,upper}_bound, I think we need to enforce that at most one {min_pin,lower_bound} resp. {max_pin,upper_bound} each may be specified, everything else should raise an error.

I'd still prefer a unified interface, because I don't see the gain to keep these arguments separate (but rather requires a lot of complexity to ensure it is unambiguous/consistent)

I indeed didn't see it or didn't read carefully enough. I also think this is a good change. The logic is already in place, so I don't think it will be a big change on the rattler-build side.

OK, I just implemented that we only use lower_bound and upper_bound, but now I am wondering if we shouldn't have used min_pin and max_pin instead (as more people are using that these days). But this PR has the implementation: prefix-dev/rattler-build#918

I proposed the API that I think is more natural and IMO more desirable long-term.

Obviously the naming choice is just bike shedding and both would work, but max_pin=1.2.3 seems off to me, whereas upper_bound='x.x' is fine because it's a more generic term, where users don't have to guess what a pin is or isn't.

chenghlee · 2024-05-08T17:03:49Z

+
+## The `hash` variable
+
+`${{ hash }}` is the variant hash and is useful in the build string computation. This used to be `PKG_HASH` in the old recipe format. Since the `hash` variable depends on the variant computation, it is only available in the `build.string` field and is computed after the entire variant computation is finished.


Should we explicitly say that this is a read-only variable? E.g., is providing "hash: deadbeef123455" in the variant config file explicitly prohibited?

Shall we call it variant_hash?

it's not really prohibited in the implementation right now. But we can say that it should be...

I don't have strong feelings about the name. We can change it to variant_hash if that's preferred!

chenghlee · 2024-05-08T17:04:12Z

+
+## The `version_to_buildstring` function
+
+- `${{ python | version_to_buildstring }}` converts a version from the variant to a build string (it removes the `.` character and takes only the first two elements of the version).


Should we provide an example here?

This is the first time we introduce the | syntax as well. Can we do version_to_buildstring(python)?

Yeah using a function would also work. But we also expose the default minijinja filters (such as lower, replace, ...), that can be used like ${{ "mystring" | upper }}

or ${{ val | replace('foo', 'moo') }}

@jaimergp I added some more text regarding jinja filters to the bottom of the CEP.

jaimergp · 2024-05-09T11:08:09Z

@@ -0,0 +1,125 @@
+# Jinja functions in recipes
+
+The new recipe format has some Jinja functionalities. We want to specify what functions exist and their expected behavior.


I feel this is lacking a short intro. At least we should link into the previous CEPs.

Suggested change

The new recipe format has some Jinja functionalities. We want to specify what functions exist and their expected behavior.

The new recipe format (introduced by CEP-XX, CEP-XY) has some Jinja functionalities. We want to specify what functions exist and their expected behavior.

jaimergp · 2024-05-09T11:18:00Z

+- `${{ env.get("MY_ENV_VAR") }}` will return the value of the environment variable `MY_ENV_VAR` or throw an error if it is not set.
+- `${{ env.get_default("MY_ENV_VAR", "default_value") }}` will return the value of the environment variable `MY_ENV_VAR` or `"default_value"` if it is not set.


Could this be overloaded to a single function? With the 2nd param being optional.

yeah, that can work. We can also make it a named argument, e.g.

env.get("FOO", default="baz")

What do you think?

@jaimergp I implemented this change: prefix-dev/rattler-build#917

Let me know what you think.

Co-authored-by: jaimergp <jaimergp@users.noreply.github.com>

Co-authored-by: Jean-Christophe Morin <38703886+JeanChristopheMorinPerso@users.noreply.github.com>

Co-authored-by: jaimergp <jaimergp@users.noreply.github.com>

wolfv · 2024-05-20T17:08:04Z

@h-vetinari since we don't have a CEP for stdlib it might be good to define it here.

What are the default values for the different operating systems?

h-vetinari · 2024-05-20T22:15:55Z

since we don't have a CEP for stdlib it might be good to define it here.

Sounds good!

What are the default values for the different operating systems?

I was under the impression that defaults here aren't necessarily a good thing? Perhaps even more than the compiler packages, the stdlib-metapackages don't have a "canonical" naming. Are you planning to set defaults for {{ compiler(...) }} as well? Looking at the diff here it doesn't seem like there are default being proposed?

In any case, here are the specs that we're using in conda-forge. They won't change substantially anymore, barring unforeseen events.

c_stdlib:
  - sysroot                    # [linux]
  - macosx_deployment_target   # [osx]
  - vs                         # [win]
m2w64_c_stdlib:                # [win]
  - m2w64-toolchain            # [win]              <-- might change to m2w64-sysroot in the future
c_stdlib_version:              # [unix]
  - 2.17                       # [linux]            <-- per July 2024
  - 10.13                      # [osx and x86_64]
  - 11.0                       # [osx and arm64]

wolfv · 2024-05-21T16:17:36Z

I expanded the content a bit.

Some things that are left to decide:

Limit the filters from Jinja to a default subset?
Should we add a split(str) filter (useful for version numbers, so that one can do "${{ version | split(".") | slice(0, 2) | join(".") }} or something along those lines. We could also create a more specialized function/filter for version string manipulation of this sort.
Should env work more like Python env
- env["FOO"] instead of env.get("FOO")
- env.get("FOO", "default") instead of env.get_default("FOO", "default")
Or should it be more like Github Actions
- env.FOO instead of env.get("FOO")
- env.FOO or "default" for default values
Should ${{ compiler('c') }} evaluate directly to gcc_linux-64 or should there be an intermediate representation (as is now)
For pin_subpackage it will be impossible to evaluate it directly (due to self-referential nature of the function).
Should we add lower_bound and upper_bound to the pin functions? And how should they work (in conjunction with min_pin and max_pin)?
The final name for the cmp function.

mariusvniekerk · 2024-07-02T14:40:50Z

+```yaml
+requirements:
+  host:
+    - ${{ "cudatoolkit" if cuda == "have_cuda" }}


Should we add an explicit callout for the case like this where expression can evaluate into "" or i guess null and that means this is excluded.

We have something about that in line 500 - 510: https://github.com/conda/ceps/pull/71/files/c1028142a7f6e3e8a5d011667e0bec3b3f67ab3a#diff-0154c4169103373897853cfe5287bb7a1101289dd78c7fa30d2acba92b644938R500-R510

JeanChristopheMorinPerso · 2024-07-02T17:51:47Z

+
+## Error handling
+
+Build tools should be aggressive about Jinja errors:


Is "should" defined somewhere in the CEP repo? If not, an implementer could interpret it as optional compared to "must".

Ideally we determine these in a provisional CEP that others use

wolfv · 2024-07-16T10:24:27Z

@marcelotrevisani @jakirkham @mbargull @CJ-Wright last chance to vote - you can also choose abstain which would still help with the quorum.

schuylermartin45 · 2024-07-16T12:29:57Z

NP on the title of this PR: Since this is schema_version: 1 I've been referring to this as the V1 recipe format and the pre-CEP-13 format as V0. Do we want to keep consistent with that nomenclature?

I'm also not sure if it is worth codifying somewhere to prevent confusion. Maybe it is also worth it to add a section to the top of the CEP that specifies which schema_version a CEP belongs to, to prevent future confusion?

chenghlee · 2024-07-16T16:24:45Z

For the record: voted "no" because while I like the ideas presented in this CEP, I don't think the CEP as written is ready to be adopted as a specification. IMO, various unanswered questions/comments need to be addressed before adoption (e.g., top-down evaluation of the context object).

mbargull · 2024-07-17T04:53:22Z

+- `target_platform` is a string that represents the platform for which the package is built. It is a string of the form `os-arch` (e.g. `linux-64`, `osx-64`, `win-64`, `linux-aarch64`, ...).
+- `build_platform` is a string that represents the platform on which the package is built. Same format as `target_platform`.
+- `linux`, `osx`, `win`, `emscripten`: These are boolean variables that are `true` if the target platform is a Linux, macOS, Unix, or Windows platform, respectively. Note that this is the first part of the `target_platform` string.
+- `x86_64`, `aarch64`, `armv7l`, `ppc64le`, `s390x`, `sparc64`, `riscv64`: These are boolean variables that are `true` if the target platform is the respective architecture. Note that, except for `x86_64`, these are the second part of the `target_platform` string.


This does not capture the osx-arm64/win-arm64 cases.
Not a huge deal, but it would be good to mention here how those are handled.

added arm64.

mbargull · 2024-07-17T05:13:25Z

+
+Context evaluation must happen from top-to-bottom. That means a later value can
+reference an earlier one like so:
+
+```yaml
+context:
+  version: "1.0.5"
+  name_and_version: "pkg_${{ version | replace('.', '_') }}"  # evaluates to "pkg_1_0_5"
+```


Suggested change

Context evaluation must happen from top-to-bottom. That means a later value can

reference an earlier one like so:

```yaml

context:

version: "1.0.5"

name_and_version: "pkg_${{ version | replace('.', '_') }}" # evaluates to "pkg_1_0_5"

```

It would be better to leave this under-specified here instead of specifying it in conflict with the CEP-14.
This exact issue with the specification has been brought up by me in both previous CEPs for the new format and the wording here regresses on the issue instead of finally addressing it.
I do not understand why this simple issue is not being addressed, given that solutions have already been proposed.

jakirkham

Thanks Wolf! 🙏

Generally seems reasonable. Included some comments/suggestions. Otherwise would be good to add

jakirkham · 2024-07-17T10:21:46Z

+- `lower_bound`, defaults to `x.x.x.x.x.x`: the lower bound, either as a version
+  or as a "pin expression", or `None`
+- `upper_bound`, defaults to `x`: the upper bound, either as a version or as a
+  "pin expression" or `None`


This is specifying bounds...

jakirkham · 2024-07-17T10:22:53Z

+- The epoch is left untouched by the `max_pin` (or `min_pin`). If the epoch is
+  set, it will be included in the final version. E.g. `1!1.2.3` with a
+  `max_pin='x.x'` will result in `<1!1.3.0a0`.
+- When bumping the version with a `max_pin` the local version part is removed.
+  For example, `1.2.3+local` with a `max_pin='x.x'` will result in `<1.3.0a0`.


Whereas this is specifying pins. Should we consolidate to one terminology or should we define both? Are there cases we need both?

jakirkham · 2024-07-17T10:26:06Z

+> `conda-build` uses the `lower_bound` for the version that is used in
+> the `max_pin` pinning expression. `conda-build` also ignores the `min_pin`
+> expression when a `upper_bound` is used.


Am curious about the motivation for the differences here

FWIW do find users confuse or forget about the options (usually the bounds). So consolidating somehow would makes sense

jakirkham · 2024-07-17T10:39:01Z

+
+You can also check for the existence of an environment variable:
+
+- `${{ env.exists("MY_ENV_VAR") }}` will return a boolean `true` if the


get just returns None for undefined keys in Python and Jinja

https://docs.python.org/3/library/stdtypes.html#dict.get

jakirkham · 2024-07-17T10:41:25Z

+
+## Error handling
+
+Build tools should be aggressive about Jinja errors:


Ideally we determine these in a provisional CEP that others use

Co-authored-by: jakirkham <jakirkham@gmail.com>

wolfv · 2024-07-22T15:26:48Z

The vote has been closed with the following result:

Total voters: 15 (valid: 13 = 86.67%)

Yes votes (11 / 84.62%):

@xhochy (Uwe Korn)
@mariusvniekerk (Marius van Niekerk)
@goanpeca (Gonzalo Peña-Castellanos)
@ocefpaf (Filipe Fernandes)
@msarahan (Michael Sarahan)
@jakirkham (John Kirkham)
@jezdez (Jannis Leidel)
@wolfv (Wolf Vollprecht)
@jaimergp (Jaime Rodríguez-Guerra)
@kkraus14 (Keith Kraus)
@baszalmstra (Bas Zalmstra)

No votes (1 / 7.69%)):

@chenghlee (Cheng H. Lee)

Abstain votes (1 / 7.69%):

@mbargull (Marcel Bargull)

Not voted (2):

@CJ-Wright (Christopher J. 'CJ' Wright)
@marcelotrevisani (Marcelo Duarte Trevisani)

Invalid votes (0):

We reached quorum, and enough YES votes for this CEP to be accepted. 🎉

beeankha · 2024-10-03T21:11:24Z

+The "old" recipe format has allowed arbitrary Jinja syntax (including set,
+if/else or for loops). The new recipe format only allows a subset of Jinja with


Suggested change

The "old" recipe format has allowed arbitrary Jinja syntax (including set,

if/else or for loops). The new recipe format only allows a subset of Jinja with

The "old" (`V0`) recipe format has allowed arbitrary Jinja syntax (including `set`,

`if`/`else` or `for` loops). The "new" (`V1`) recipe format only allows a subset of Jinja with

We should probably also change old to V0 while we're at it.

The title of this PR should be changed to reference V1 recipe format vs V2

beeankha · 2024-10-03T21:44:13Z

+## The `hash` variable
+
+`${{ hash }}` is the variant hash and is useful in the build string computation.
+This used to be `PKG_HASH` in the old recipe format. Since the `hash` variable


Suggested change

This used to be `PKG_HASH` in the old recipe format. Since the `hash` variable

This used to be `PKG_HASH` in the `V0` recipe format. Since the `hash` variable

JeanChristopheMorinPerso · 2026-04-07T20:23:24Z

@wolfv @jaimergp I was searching for the CEP and couldn't find it. Should we merged this PR? It looks like the CEP was approved.

jaimergp · 2026-04-17T13:43:09Z

I don't know how we missed this but there we go, two years after approval, this CEP is hence minted as CEP 39. Merging!

JeanChristopheMorinPerso · 2026-04-17T13:54:47Z

Thank you!

add cep for all jinja functionality

f1ad3f2

JeanChristopheMorinPerso reviewed May 6, 2024

View reviewed changes

chenghlee requested changes May 8, 2024

View reviewed changes

jaimergp reviewed May 9, 2024

View reviewed changes

Comment thread cep-recipe-jinja.md Outdated

jaimergp reviewed May 9, 2024

View reviewed changes

Comment thread cep-recipe-jinja.md Outdated

jaimergp reviewed May 9, 2024

View reviewed changes

Comment thread cep-recipe-jinja.md Outdated

jaimergp reviewed May 9, 2024

View reviewed changes

Comment thread cep-recipe-jinja.md Outdated

jaimergp reviewed May 9, 2024

View reviewed changes

Comment thread cep-recipe-jinja.md Outdated

jaimergp reviewed May 9, 2024

View reviewed changes

Comment thread cep-recipe-jinja.md Outdated

jaimergp reviewed May 9, 2024

View reviewed changes

Comment thread cep-recipe-jinja.md Outdated

jaimergp reviewed May 9, 2024

View reviewed changes

Comment thread cep-recipe-jinja.md Outdated

jaimergp reviewed May 9, 2024

View reviewed changes

Comment thread cep-recipe-jinja.md Outdated

wolfv and others added 5 commits May 10, 2024 09:31

Update cep-recipe-jinja.md

07da5e7

Co-authored-by: jaimergp <jaimergp@users.noreply.github.com>

Update cep-recipe-jinja.md

e4e0a42

Co-authored-by: Jean-Christophe Morin <38703886+JeanChristopheMorinPerso@users.noreply.github.com>

Update cep-recipe-jinja.md

81bdf68

Co-authored-by: jaimergp <jaimergp@users.noreply.github.com>

Update cep-recipe-jinja.md

2694476

Co-authored-by: jaimergp <jaimergp@users.noreply.github.com>

Update cep-recipe-jinja.md

62ea3d4

Co-authored-by: jaimergp <jaimergp@users.noreply.github.com>

h-vetinari suggested changes May 10, 2024

View reviewed changes

Comment thread cep-recipe-jinja.md Outdated

add more text regarding jinja filters

0d3f301

add more content and context

dde40b1

mariusvniekerk reviewed Jul 2, 2024

View reviewed changes

JeanChristopheMorinPerso reviewed Jul 2, 2024

View reviewed changes

mariusvniekerk approved these changes Jul 3, 2024

View reviewed changes

jaimergp reviewed Jul 6, 2024

View reviewed changes

Comment thread cep-recipe-jinja.md

schuylermartin45 mentioned this pull request Jul 16, 2024

Upgrades commonly used selectors conda/conda-recipe-manager#80

Merged

mbargull reviewed Jul 17, 2024

View reviewed changes

h-vetinari mentioned this pull request Jul 17, 2024

CEP 40: Recipe serialization in packages #74

Merged

jakirkham approved these changes Jul 17, 2024

View reviewed changes

Update cep-recipe-jinja.md

81d9515

Co-authored-by: jakirkham <jakirkham@gmail.com>

baszalmstra mentioned this pull request Aug 1, 2024

fix: omit also jinja functions conda-forge/rattler-build-conda-compat#37

Merged

wolfv added 2 commits August 26, 2024 11:24

fix and finalize CEP

822558a

rename to cep-17

4141b17

beeankha reviewed Oct 3, 2024

View reviewed changes

Comment thread cep-17.md Outdated

beeankha reviewed Oct 3, 2024

View reviewed changes

Comment thread cep-17.md Outdated

jaimergp changed the title ~~CEP: Jinja functionality in v2 recipes~~ CEP: Jinja functionality in v1 recipes Oct 15, 2024

jaimergp added 3 commits April 17, 2026 15:21

Merge branch 'main' of github.com:conda/ceps into pr/wolfv/71

58341db

Mint as CEP 39 with some small editorial changes

39b7546

pre-commit

2deab64

jaimergp changed the title ~~CEP: Jinja functionality in v1 recipes~~ CEP 39 - A new recipe format (part 3): jinja functions in recipes Apr 17, 2026

jaimergp merged commit c12e64e into conda:main Apr 17, 2026
1 check passed


		## The `cdt` function

		cdt stands for "core dependency tree" packages. These are repackaged from Centos.


		Where `cdt_name` and `cdt_arch` are loaded from the variant config. If they are undefined, they default to:

		- `cos6` for `cdt_name` on `x86_64` and `x86`, otherwise `cos7`


		## The `cmp` function

		The `cmp` function is used to compare two versions. It returns `True` if the comparison is true and `False` otherwise.


		You can also check for the existence of an environment variable:

		- `${{ env.exists("MY_ENV_VAR") }}` will return `true` if the environment variable `MY_ENV_VAR` is set and `false` otherwise.


		## The compiler function

		The compiler function is used to stick together a compiler from {lang}_compiler and {lang}_compiler_version

		- `min_pin`: The lower bound of the dependency spec. This is expressed as a `x.x....` version where the `x` are filled in from the corresponding version of the package. For example, `x.x` would be `1.2` for a package version `1.2.3`. The resulting pin spec would look like `>=1.2` for the `min_pin` argument of `x.x` and a version of `1.2.3` for the package.
		- `max_pin`: This defines the upper bound and follows the same `x.x` semantics but adds `+1` to the last segment. For example, `x.x` would be `1.(2 + 1)` for a package version `1.2.3`. The resulting pin spec would look like `<1.3` for the `max_pin` argument of `x.x` and a version of `1.2.3` for the package.


		## The `hash` variable

		`${{ hash }}` is the variant hash and is useful in the build string computation. This used to be `PKG_HASH` in the old recipe format. Since the `hash` variable depends on the variant computation, it is only available in the `build.string` field and is computed after the entire variant computation is finished.


		## The `version_to_buildstring` function

		- `${{ python \| version_to_buildstring }}` converts a version from the variant to a build string (it removes the `.` character and takes only the first two elements of the version).

		@@ -0,0 +1,125 @@
		# Jinja functions in recipes

		The new recipe format has some Jinja functionalities. We want to specify what functions exist and their expected behavior.

	The new recipe format has some Jinja functionalities. We want to specify what functions exist and their expected behavior.
	The new recipe format (introduced by CEP-XX, CEP-XY) has some Jinja functionalities. We want to specify what functions exist and their expected behavior.

		- `${{ env.get("MY_ENV_VAR") }}` will return the value of the environment variable `MY_ENV_VAR` or throw an error if it is not set.
		- `${{ env.get_default("MY_ENV_VAR", "default_value") }}` will return the value of the environment variable `MY_ENV_VAR` or `"default_value"` if it is not set.


		## Error handling

		Build tools should be aggressive about Jinja errors:

		The "old" recipe format has allowed arbitrary Jinja syntax (including set,
		if/else or for loops). The new recipe format only allows a subset of Jinja with

	This used to be `PKG_HASH` in the old recipe format. Since the `hash` variable
	This used to be `PKG_HASH` in the `V0` recipe format. Since the `hash` variable

Uh oh!

Conversation

wolfv commented Apr 12, 2024

Uh oh!

h-vetinari commented Apr 12, 2024

Uh oh!

wolfv commented Apr 13, 2024

Uh oh!

h-vetinari commented Apr 13, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

wolfv commented Apr 14, 2024

Uh oh!

h-vetinari commented Apr 14, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

wolfv May 10, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

h-vetinari commented Apr 13, 2024 •

edited

Loading

h-vetinari commented Apr 14, 2024 •

edited

Loading

wolfv May 10, 2024 •

edited

Loading

h-vetinari Jun 10, 2024 •

edited

Loading