Skip to content

Draft another CEP for PURL annotation #114

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
jaimergp wants to merge 4 commits into
base: main
Choose a base branch
from
Open

Draft another CEP for PURL annotation #114

Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
36ce60f
Draft another CEP for PURL annotation
jaimergp Mar 10, 2025
5f14f5c
only lists
jaimergp Mar 10, 2025
22bf088
simpler example
jaimergp Mar 10, 2025
aea0f6c
Update cep-9999.md
jaimergp Mar 11, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
64 changes: 64 additions & 0 deletions cep-9999.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
# CEP ???? - Source provenance metadata

<table>
<tr><td> Title </td><td> CEP ???? - Source provenance metadata </td>
<tr><td> Status </td><td> Draft </td></tr>
<tr><td> Author(s) </td><td> Jaime Rodríguez-Guerra &lt;jaime.rogue@gmail.com&gt;</td></tr>
<tr><td> Created </td><td> Mar 10, 2025</td></tr>
<tr><td> Updated </td><td> Mar 10, 2025</td></tr>
<tr><td> Discussion </td><td> N/A </td></tr>
<tr><td> Implementation </td><td> N/A </td></tr>
</table>

## Abstract

This CEP aims to standardize how the conda ecosystem can leverage free-form
artifact metadata to annotate source and ecosystem provenance of the distributed
artifacts.

We propose using the `about` field in the recipe metadata to implement
a `purl` field where one or more [Package URLs](https://github.com/package-url/purl-spec)
(PURLs) can be specified.

## Motivation

Provenance metadata is useful to assess from which sources an artifact was built.
conda recipes already specify the necessary sources in their `source` section, but this field
is mostly concerned with source download and preparation for the build. In
most cases, it will contain one or more URLs or source-controlled repositories. These
URLs cannot be guaranteed to uniquely identify a project since they may change, and their
syntax might evolve across releases (e.g. different versioning schemes).

Instead we propose adding a new field that _uniquely_ identifies the project being built.
The Package URL project is chosen for this endeavour. This is speciallyuseful

@mgorny mgorny Apr 28, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
The Package URL project is chosen for this endeavour. This is speciallyuseful
The Package URL project is chosen for this endeavour. This is especially useful

for efforts like dependency mapping across ecosystems. See
[PEP 725](https://peps.python.org/pep-0725/) for practical applications in the
context of PyPI <> conda interoperability.

Example for `django`:

```yaml
# meta.yaml
{% set version = "1.10.1" %}
package:
name: django
version: {{ version }}
# ...
about:
# ...
purls:
- pkg:pypi/django@{{ version }}

@JeanChristopheMorinPerso JeanChristopheMorinPerso Mar 10, 2025

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should the CEP specification mandate that PURLs have a version or do we allow versionless PURLs? I'd also be curious to know what's your opinion on the cases where we'll need to use the generic type.

@jaimergp jaimergp Mar 11, 2025

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I initially thought of using versionless PURLs, but the examples in #63 (comment) made me consider adding the version because well, it has to be there anyway. That said, I don't think the version field should be required.

About generic, yes and no. Some packages won't have a canonical location or ecosystem (pretty common in C/C++), so the most convenient way is to annotate them as generic. PEP 725 (draft) goes into more detail about this. Check also https://github.com/jaimergp/external-metadata-mappings, where I am dabbing with the concept of cross-ecosystem mappings. There's a Streamlit app too. I do see the value in things like generic/libtiff, as well as PEP-725-proposed virtual:interface/* and virtual:compiler/*. Hopefully this can be integrated in PURL like pkg:abstract/* though.

@JeanChristopheMorinPerso JeanChristopheMorinPerso Mar 11, 2025

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I initially thought of using versionless PURLs, but the examples in #63 (comment) made me consider adding the version because well, it has to be there anyway.

I think it depends on how you want to use it. If you just want a standard way to map package names, then I guess versionless is fine. But if you want to map to a specific version, then you need the upstream version.

About generic, yes and no. Some packages won't have a canonical location or ecosystem (pretty common in C/C++), so the most convenient way is to annotate them as generic. PEP 725 (draft) goes into more detail about this. Check also https://github.com/jaimergp/external-metadata-mappings, where I am dabbing with the concept of cross-ecosystem mappings. There's a Streamlit app too. I do see the value in things like generic/libtiff, as well as PEP-725-proposed virtual:interface/* and virtual:compiler/*. Hopefully this can be package-url/purl-spec#222 though.

I have been following your work in https://github.com/jaimergp/external-metadata-mappings and the other repos, and that's great!

I'm asking about generics because we already started using PURLs to map our package names to upstream packages but decided to wait before adding any kind of support for generics. It gets messy pretty quickly. But more importantly, you need to know what you want to use the PURL for. Is it to map to an exact upstream location or is it to map something to something else and you don't really care about the exact source?

I don't know if we should define this in the CEP though.

@jaimergp jaimergp Mar 11, 2025

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm asking about generics because we already started using PURLs to map our package names to upstream packages but decided to wait before adding any kind of support for generics. It gets messy pretty quickly.

Do you have some examples of the messy parts? Really interested to see other use cases.

@JeanChristopheMorinPerso JeanChristopheMorinPerso Mar 11, 2025

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have to say that we didn't look very deeply into generics. But I feel like where it gets tricky is that the spec says that download_url and checksum can be provided. But the thing is that the download_url could differ based on the subdir or other things. There are also potential cases where do very different things could be named the same way. Since generic doesn't have a concept of namespace, I'm not too sure how we would handle these (though it's hypothetical mostly).

I might also overthinking it (which is very possible).

Comment on lines +49 to +50

@chenghlee chenghlee Mar 11, 2025

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we consider making the purls a mapping, rather than a simple list? This would make the field more easily interpretable and would give us more flexibility to further extend it.

E.g.,

purls:
  upstream:
    - pkg:pypi/foo
  vendors:
    - pkg:github/bar
    - pkg:github/baz

@jaimergp jaimergp Mar 11, 2025

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A free-form mapping or a mapping with well defined keys? If the latter, which ones? I don't want to end up in SBOM-like territory.

@bollwyvl bollwyvl Jun 5, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-phrasing here from #63: if not SBOM territory, then at least SBOM cartographic terms for the "functional equivalent" and "vendor/static link" cases, covered as part of closed namespaces in the SBOM specs:

  • CycloneDX
    • pedigree, one of: ancestors, descendents, variants, commits, patches, notes
  • SPDX
    • RelationshipType between artifacts... some of which are packages, but but may be licenses, vulnerability reports, patches, etc.

A strawman narrowing of those terms, maybe only the first needing to land initially:

what we're trying to say in CycloneDXese to SPDXish
this conda package fulfills the purpose of that pypi package ancestors descendantOf
this conda package statically contains, and therefore exposes you to the vulnerabilities of, that rust crate variants hasStaticLink

```

## Specification

The `about` section (top-level or output-specific) in the recipe file MAY
contain a `purls` field that takes a list of strings as specified
by the [PURL standard](https://github.com/package-url/purl-spec/blob/main/PURL-SPECIFICATION.rst).

If the build tool does not support this new field in `about`, the free-form `extra`

@JeanChristopheMorinPerso JeanChristopheMorinPerso Jun 30, 2025

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that extra can't be passed to an output. conda-build will simply ignore it and use the top-level extra section instead of the output-specific one.

@baszalmstra baszalmstra Jun 30, 2025

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could that be changed? Or is that not feasible?

@JeanChristopheMorinPerso JeanChristopheMorinPerso Jun 30, 2025

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we could change that. I think we would need to change https://github.com/conda/conda-build/blob/d27c97c11e2316905cefe24deebd3e445e853ece/conda_build/metadata.py#L2619-L2626, or something like that.

section can be used instead for backwards-compatibility.

## Copyright

All CEPs are explicitly [CC0 1.0 Universal](https://creativecommons.org/publicdomain/zero/1.0/).