Add CEP for a URL field for package records

cep-XXXX.md

@@ -0,0 +1,203 @@
+# CEP XXXX - URL field for package records
+
+<table>
+<tr><td> Title </td><td> URL field for package records</td></tr>
+<tr><td> Status </td><td> Draft </td></tr>
+<tr><td> Author(s) </td><td>
+  Dan Yeaw &lt;dyeaw@anaconda.com&gt;
+</td></tr>
+<tr><td> Created </td><td> Feb 2, 2026</td></tr>
+<tr><td> Updated </td><td> Feb 2, 2026</td></tr>
+<tr><td> Discussion </td><td> https://github.com/conda/ceps/pull/111 </td></tr>
+<tr><td> Implementation </td><td> TBD </td></tr>
+<tr><td> Requires </td><td> <a href="https://github.com/conda/ceps/pull/146">conda/ceps#146</a> (backwards-compatible repodata update strategy)</td></tr>
+</table>
+
+> The keywords "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT",
+"RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as
+described in [RFC2119][RFC2119] when, and only when, they appear in all capitals, as shown here.
+
+## Abstract
+
+This CEP introduces a new optional `url` field in package records to specify download locations for individual packages.
+
+## Motivation
+
+Currently, the download location for a package is constructed by combining the `base_url` field (defined in [CEP 16](https://conda.org/learn/ceps/cep-0016)) with the package filename (which serves as the dictionary key in repodata). This has a couple of limitations:
+
+1. Packages must be stored in the same directory as `repodata.json`
+2. It is not possible to serve the files from different servers or Content Delivery Networks (CDNs)
+
+## Rationale
+
+Adding a `url` field enables several use cases that are not possible with the current approach:
+
+- **Per-package directories**: Common in [PyPI simple repositories](https://packaging.python.org/en/latest/guides/hosting-your-own-index/#manual-repository) where each package has its own subdirectory
+- **CDN distribution**: Packages can be served from different CDNs with hash-based URLs
+- **Mixed sources**: Different packages in the same repodata can be hosted on different servers
+- **Backward compatibility**: Traditional flat directory structures continue to work by setting `url` to the filename
+
+Although wheels were the primary motivation, this change provides general flexibility for package hosting in the conda ecosystem.
+
+## Specification
+
+Package repodata records MAY contain a `url` field. The `url` field is NOT part of the package's `info/index.json`. It is added by the index generation tool when building repodata. When present, the value MUST be set to either a full URL or a relative POSIX path. Relative paths MUST NOT contain `..` path components.
+
+Conda clients SHALL construct the download URL as follows:
+
+1. Determine the base URL:
+   - If the repodata's `info` object contains a `base_url` field, use that value
+   - Otherwise, use an empty string (`""`)
+
+2. Determine the package path:
+   - If the package record contains a `url` field, use that value
+   - Otherwise, use the package filename (the dictionary key)
+
+3. Resolve the download URL by combining the base URL with the package path following [RFC 3986 URL resolution semantics](https://datatracker.ietf.org/doc/html/rfc3986#section-5) (equivalent to Python's `urllib.parse.urljoin(base_url, package_path)`).
+
+4. Determine the local cache filename:
+   - The basename of the resolved download URL SHALL be used as the local cache filename.
+
+This resolution means:
+
+- **Full URLs**: If the package path is a full URL (e.g., `https://cdn.example.com/package.conda`), it is used as-is, ignoring the base URL
+- **Relative path with `base_url`**: If the package path is relative (e.g., `subdir/package.conda`) and `base_url` is set (e.g., `https://repo.example.com/`), the result is `https://repo.example.com/subdir/package.conda`
+- **Relative path without `base_url`**: If the package path is relative and no `base_url` is specified, the package path remains relative and will be resolved by the HTTP client relative to the repodata's location
+
+## Examples
+
+### Absolute URL
+
+In this example, the package is served from a CDN with a full URL:
+
+```json
+{
+  "packages.conda": {
+    "numpy-2.4.2-py314hd4f4903_0.conda": {
+      "build": "py314hd4f4903_0",
+      "build_number": 0,
+      "constrains": [],
+      "depends": [
+        "libblas >=3.9.0,<4.0a0",
+        "libcblas >=3.9.0,<4.0a0",
+        "liblapack >=3.9.0,<4.0a0",
+        "python >=3.14,<3.15.0a0"
+      ],
+      "license": "BSD-3-Clause",
+      "md5": "c15ea513263c9a15d504ab6b087c6d81",
+      "name": "numpy",
+      "sha256": "1f39bde67c7d252f079260f871c1d6e67b9757e94953369081f5764423fb5e01",
+      "size": 8970626,
+      "subdir": "linux-64",
+      "timestamp": 1770020169383,
+      "url": "https://cdn.example.com/packages/numpy/numpy-2.4.2-py314hd4f4903_0.conda",
+      "version": "2.4.2"
+    }
+  }
+}
+```
+
+### Relative URL with base_url
+
+In this example, packages are organized in subdirectories by package name, using a relative URL combined with `base_url`. The repodata file is located at `https://repo.example.com/conda/linux-64/repodata.json`, where `linux-64` is the platform-specific subdirectory:
+
+```json
+{
+  "info": {
+    "base_url": "https://repo.example.com/conda/linux-64/"
+  },
+  "packages.conda": {
+    "packaging-25.0-pyh29332c3_1.conda": {
+      "build": "pyh29332c3_1",
+      "build_number": 1,
+      "depends": [
+        "python >=3.8"
+      ],
+      "license": "Apache-2.0 OR BSD-2-Clause",
+      "md5": "8da6e3f6a14a8f7b8e43f0e4b3e6b5c9",
+      "name": "packaging",
+      "noarch": "python",
+      "sha256": "a1b2c3d4e5f6a7b8c9d0e1f2a3b4c5d6e7f8a9b0c1d2e3f4a5b6c7d8e9f0a1b2",
+      "size": 51234,
+      "subdir": "noarch",
+      "timestamp": 1735689600000,
+      "url": "packaging/packaging-25.0-pyh29332c3_1.conda",
+      "version": "25.0"
+    }
+  }
+}
+```
+
+The client would resolve the relative `url` against the `base_url` to get: `https://repo.example.com/conda/linux-64/packaging/packaging-25.0-pyh29332c3_1.conda`
+
+## Backward Compatibility
+
+This CEP introduces a new optional `url` field to package records. Since the field is optional, this change is backwards-compatible:
+
+- Newer clients will use the `url` field when present, falling back to the traditional method when absent
+- Existing repodata without `url` fields will continue to work without modification
+
+However, older clients that don't recognize the `url` field will continue to construct download URLs using the existing method (combining `base_url` with the package filename). If a package is stored at a non-flat path (e.g., `packaging/packaging-25.0-pyh29332c3_1.conda`) and the file does not also exist at the traditional flat path, older clients will receive a 404 error.
+
+Therefore, channels MUST follow the backwards-compatible repodata update strategy specified in [conda/ceps#146](https://github.com/conda/ceps/pull/146) when introducing `url` fields for packages stored at non-flat paths. That strategy ensures older clients can still resolve and download artifacts correctly during the transition period.
+
+## Lockfile Considerations
+
+Previously, within a single channel's repodata, each package entry had a unique URL since it was derived directly from the unique dictionary key. This CEP decouples location from identity: a package record may now contain a `url` field pointing to an external location, meaning two entries in the same repodata can have different channel URLs and metadata while resolving to the same `url`.
+For example:
+
+```text
+https://conda.anaconda.org/my-channel/linux-64/numpy-2.4.2-py314hd4f4903_0.conda  ->  url: https://cdn.example.com/packages/numpy/numpy-2.4.2-py314hd4f4903_0.conda
+https://conda.anaconda.org/my-channel/linux-64/numpy-2.4.2-py313hd4f4903_0.conda  ->  url: https://cdn.example.com/packages/numpy/numpy-2.4.2-py314hd4f4903_0.conda
+```
+
+The resolved `url` is therefore no longer a guaranteed unique identifier for a package record.
+
+### Recommendation
+
+Lockfile implementations SHOULD adopt Package URLs (PURLs), as defined in [conda/ceps#159](https://github.com/conda/ceps/pull/159) (draft), as the canonical unique identifier for package records. A conda PURL encodes a package's identity (channel, name, version, build string, and subdir) in a standardized, location-independent form, for example:
+
+```text
+pkg:conda/conda-forge/numpy@2.4.2?build=py314hd4f4903_0&subdir=linux-64
+```
+
+PURLs separate identity from location: two channels that independently index the same external CDN artifact will produce distinct PURLs. The resolved `url` field SHOULD be stored separately in the lockfile for fetching purposes.
+
+## Security Considerations
+
+A package record's `url` field is authored by the channel. An attacker who can write to `repodata.json` can set an absolute `url` pointing to an attacker-controlled host and update the `md5`/`sha256` hashes to match. The client fetches the malicious artifact directly from that host, and could bypass any artifact proxy that the user may rely on for vulnerability scanning, egress control, and audit. Hash verification passes because the attacker controls both the artifact and the expected hashes.
+
+This attack requires only a write to `repodata.json`. In channel architectures where index generation and artifact storage use separate credentials or pipelines, such as a CI-driven indexing step backed by separate artifact storage, this represents a narrower attack surface than replacing an artifact directly.
+
+Client implementations MUST expose a `url_policy` configuration option with the following values:
+
+- `allow` (default): absolute and relative `url` values are resolved and fetched as specified.
+- `relative_only`: absolute URLs in `url` fields are ignored; the client falls back to constructing the download URL from `base_url` and the package filename. RECOMMENDED for environments using artifact proxies or with egress restrictions.
+- `disabled`: the `url` field is ignored entirely; behavior is equivalent to a client that does not implement this CEP.
+
+Channel operators using absolute URLs for CDN distribution SHOULD document the set of external hosts used so that consumers can permit them in egress allow-lists.
+
+## Rejected ideas
+
+### Using the dictionary key for paths
+
+The directory path could be embedded in the package's dictionary key (e.g., `packaging/packaging-25.0-pyh29332c3_1.conda`). While some clients may already support this, an explicit `url` field is clearer and maintains the convention that dictionary keys are filenames, not paths.
+
+### Adding an `fn` field alongside `url`
+
+A separate `fn` field could specify a different filename for saving locally. However, packages should not be renamed after downloading, and the filename can be obtained from either the `url` basename or the dictionary key. This would add no value while increasing repodata size.
+
+## References
+
+- [conda/ceps#146 - A backwards-compatible repodata update strategy](https://github.com/conda/ceps/pull/146): Repodata rollout pattern this CEP relies on when introducing optional package-record fields such as `url`
+- [conda/ceps#159 (draft) - PURLs for conda packages](https://github.com/conda/ceps/pull/159): Defines the standardized Package URL format for conda packages, recommended as the long-term unique identifier for lockfiles
+- [CEP 15 - Hosting repodata.json and packages separately by adding a `base_url` property](https://conda.org/learn/ceps/cep-0015): Introduced the `base_url` field for repodata
+- [PyPI Simple Repository API](https://peps.python.org/pep-0503/): Specification for simple package repositories
+- [Hosting your own simple repository](https://packaging.python.org/en/latest/guides/hosting-your-own-index/#manual-repository): Example of package repositories with per-package subdirectories
+
+## Copyright
+
+All CEPs are explicitly [CC0 1.0 Universal](https://creativecommons.org/publicdomain/zero/1.0/).
+
+<!-- links -->
+[RFC2119]: https://datatracker.ietf.org/doc/html/rfc2119