Skip to content

Latest commit

 

History

History
130 lines (99 loc) · 5.95 KB

File metadata and controls

130 lines (99 loc) · 5.95 KB

Taking on a repository that already exists

Most people arrive with packages already published somewhere. These commands record what is there, byte for byte, rather than asking anyone to start again.

Importing an existing repository

Most people arrive with a repository already published somewhere. import reads its index and records every artifact it names, rather than adopting each by hand:

snailmail import --project six --public-origin --dry-run python https://pypi.org/
snailmail import --project six --public-origin python https://pypi.org/

Each artifact goes through the same path as adopt: fetched, checked against the digest its index published, and recorded with its origin URL so it can be refetched later.

What a recorded digest is worth depends on where it came from — an index someone signed is not the same as one served over TLS alone — so the lock records that beside it, and status --json reports the counts:

  python -> {"index-stated": 2}

index-stated means the index published the digest and the fetched bytes matched, which is the strongest a simple index supports. Artifacts recorded by adopt read as operator, including in locks written before this field existed. PLAN.md §3.8 has the full set and what each level is worth. Anything the index names but does not publish a SHA-256 for is skipped and reported — a locked artifact is pinned to a digest someone stated in advance, and one computed from the bytes a download happened to return would prove only that the download was self-consistent.

One artifact failing does not abandon the rest, so a repository with a broken file imports the other 47 and names the one that failed.

PyPI and Helm today. A Helm repository has no per-project page, so importing one imports the repository:

snailmail import --public-origin --dry-run charts https://grafana.github.io/helm-charts

Where an index lists several mirrors of a chart, the origin recorded is the URL that actually served. Where it lists the same name and version twice, both entries are skipped and named — two entries claiming one identity cannot both be it, and picking one would record bytes nobody chose.

Debian too, which reads a suite rather than a project:

snailmail import --public-origin --dry-run --suite bookworm apt https://deb.debian.org/debian

A Debian import walks the chain rather than trusting the leaf: Release states the digest of Packages, and a Packages whose bytes disagree is refused outright — nothing in an index that failed its own root can be trusted. That is why Debian artifacts record index-chain where PyPI and Helm record index-stated. The Release signature itself is not verified yet, so the root of trust is still the transport, and the lock says exactly that.

And yum, which needs no extra flags because a repository root is one repository:

snailmail import --public-origin --dry-run rocky https://dl.rockylinux.org/pub/rocky/9/BaseOS/x86_64/os

yum walks the same chain as Debian: repomd.xml states the digest of primary.xml.gz, and a primary whose bytes disagree is refused. So rpm artifacts also record index-chain. If repomd.xml states only a sha1 or md5 for its primary, the import stops rather than quietly recording a weaker provenance than was asked for — signing repomd.xml is what would raise this to signed-index.

Multilib is preserved: i686 and x86_64 builds of one name-version are two artifacts, recorded as two blobs under the same package version rather than one overwriting the other.

And Alpine, which is the honest exception:

snailmail import --public-origin --dry-run alpine https://dl-cdn.alpinelinux.org/alpine/v3.19/main/x86_64

An APKINDEX entry carries C:Q1…, which decodes to a SHA-1 of the package's control section — not of the file. Checked against Alpine's own archive: the index states 6026787b… for 7zip-23.01-r0.apk, whose actual SHA-1 is 76a96042…. They differ because they are digests of different things. So there is nothing in an Alpine index to pin an artifact to, and an imported apk records computed: a digest of the bytes snailmail downloaded, and nothing more.

That is allowed but never hidden. If your workspace will not accept unauthenticated bytes, say so once:

snailmail import --public-origin --min-provenance index-stated alpine https://…

Every artifact then reports why it was refused, rather than being pinned to something weaker than you asked for. The floor works for any format — a Debian import establishes index-chain, so it passes an index-stated floor.

Adopting an artifact from a URL

snailmail adopt --sha256 HEX --public-origin REPOSITORY URL records one explicitly selected artifact in an existing owned repository. The lowercase SHA-256 pin is mandatory; --public-origin confirms that the complete requested URL is non-secret and may be committed and printed. Lock schema 2 retains that URL, plans display every visible adopted acquisition, and --dry-run validates without changing CAS or lock state. An adopted artifact is streamed to disk rather than held in memory, so its size costs disk and time rather than resident memory; SNAILMAIL_MAX_ARTIFACT_BYTES raises the 2 GiB ceiling that remains. Adoption requires the local blob store and does not claim authorship, build provenance, source signatures, or historical snailmail publication.

Inspecting someone else's repository

snailmail doctor URL needs no workspace and inspects a public HTTPS PyPI, Debian, or Helm repository. It parses bounded native indexes, follows at most the configured artifact limit, and checks referenced availability, size, SHA-256, archive validity, and package identity. Use --project for PyPI artifacts and --suite for a Debian base URL. Debian Release signatures and Helm provenance are explicitly reported as unverified in this initial slice. Runs inspect at most four artifacts, cap each expanded archive at 64 MiB, and stop after two minutes.


Back to snailmail.