Skip to content

hosts.md

Latest commit

 

History

History
174 lines (141 loc) · 8.19 KB

hosts.md

File metadata and controls

174 lines (141 loc) · 8.19 KB
Raw

Where repositories are published

Every host publishes the same built tree; what differs is how a revision becomes live, and that decides which formats each can serve. The table in the README is the summary; this is the detail.

Publishing over ssh

The rsync host publishes to a directory on a machine you reach over ssh — the shape most people already have, where a web server serves a filesystem path.

snailmail setup rpm --name yum --host rsync \
  --target deploy@packages.example \
  --output /srv/www/yum \
  --base-url https://packages.example/yum

--output is the absolute path on the far side, where for a local host it is workspace-relative. --target is the ssh destination; port, key and jump host belong in ssh_config.

This is the only host that serves every format, including a signed yum repository. A revision goes live by renaming one symlink, so the whole tree switches at once and the number of files that must change together does not matter. An object store commits one object and no more, which is why it cannot serve Debian, Alpine, or signed yum at all.

Two POSIX primitives carry the guarantees, neither of them from the transport:

  • rename(2) makes a publication atomic. The published path is a symlink into .snailmail/releases/<tree>/, and a revision goes live when a new symlink is renamed over the old one. A client follows one whole revision or the other, never a half-copied tree. The tree is copied to a staging path and renamed into its release first, so the symlink never points at a partial release.
  • mkdir(2) makes it conditional. There is no compare-and-swap on a symlink, so the expected revision is checked and the swap performed while holding a lock directory whose creation fails if it already exists. Two runners publishing at once means one fails, not one waits — the same answer every other host gives.

snailmail collect works here too. Every revision leaves a tree under .snailmail/releases/, and the swap that makes a new one live does not remove the one it replaced. Collection keeps the live revision — read from the far side, not trusted from the workspace — and whatever retention names. Unlike object storage there is no restore target held back, because rollback is not offered, so an older revision survives only if you ask for it.

Requirements and limits, stated because they are not checked from this side:

  • The published path and its release directory must be on one filesystem, since rename is not atomic across mount points.
  • ssh options — port, key, jump host — belong in ssh_config. A second place to configure ssh is a second place for it to be wrong.
  • A path that already exists and was not published by snailmail is refused rather than replaced, so a publication cannot unpublish somebody else's files.
  • Rollback is not offered. Pointing the symlink at an earlier release is easy; establishing that the release is still intact is not, because nothing on the far side verifies it. The adapter declines rather than claiming a rollback it cannot check.
  • No preview site, so the pr and approval gates are refused. Nothing is copied to the far side before the commit, so there is no URL a reviewer could install from. Under the auto gate a real client still installs from the exact staged bytes before they are published; what goes unchecked is that the far side serves them correctly, which is what a preview buys.

Publishing to object storage

Object storage serves PyPI, Helm and unsigned yum repositories. Setup uses the standard AWS credential chain; no credential values are written to the manifest or plan:

go run ./cmd/snailmail setup pypi \
  --name python \
  --host s3 \
  --bucket example-packages \
  --prefix python \
  --region us-east-1 \
  --base-url https://packages.example.com/python
git add snailmail.toml repos/python.lock.toml docs/install-python.md
git commit -m "configure hosted Python repository"
go run ./cmd/snailmail plan
go run ./cmd/snailmail apply

What the credential has to be allowed to do

Publishing needs, scoped to the configured prefix:

  • s3:GetObject — reading back what it wrote, to verify it
  • s3:PutObject — writing artifacts, indexes and the root object
  • s3:DeleteObject — removing an abandoned stage, and removing the root object when a restore has to leave a repository with none

It does not need s3:ListBucket. Every operation on the publishing path addresses an object by a key snailmail already knows, which is what lets a publication be verified without trusting a listing.

s3:ListBucket is needed only to discover state a publication has superseded — collecting old releases. That is a separate operation and may run under a separate credential, so a publishing role stays as narrow as the list above.

The bucket or gateway must serve the configured prefix, including .snailmail/stages/ during pre-publication verification. Use --endpoint and --use-path-style for compatible object stores. All non-loopback S3 API and package client endpoints must use HTTPS. Configure a bucket lifecycle rule to expire abandoned .snailmail/stages/ objects after a grace period; immutable .snailmail/releases/, .snailmail/manifests/, and .snailmail/restores/ objects must not use that short-lived rule.

A private object-storage repository uses a Basic-auth gateway and a short-lived credential broker. The broker is a compiled executable selected at runtime by SNAILMAIL_CREDENTIAL_BROKER; set --visibility private --read-auth basic --credential-broker default during setup. Snailmail sends the reviewed workspace, host, plan, change, tree, and object-prefix scope as JSON on stdin. The helper returns username, password, and RFC3339 expires_at JSON, with a maximum lifetime of 15 minutes. The helper and gateway are trusted to enforce the supplied scope. The helper receives only selected profile, web-identity, TLS, and SNAILMAIL_BROKER_* environment variables, not the complete snailmail environment. Credentials stay out of Git, plans, URLs, and argv; pip receives them through an isolated temporary netrc that is destroyed after verification.

Shared S3 blob storage keeps Git locks provider-neutral while treating the local CAS as a verified disposable cache:

go run ./cmd/snailmail blob-store s3 \
  --bucket example-artifacts \
  --prefix snailmail/cas \
  --region us-east-1

Publishing to GitHub Pages

Public GitHub Pages requires distinct pre-provisioned production and preview sites that deploy the root of the configured branch. Authenticate gh, then:

go run ./cmd/snailmail setup pypi \
  --name python \
  --host github-pages \
  --github-repo example/packages \
  --github-preview-repo example/packages-preview \
  --base-url https://example.github.io/packages \
  --preview-url https://example.github.io/packages-preview

Pages publication uses exact orphan commits, immutable stage and restore refs, force-with-lease compare-and-swap, .nojekyll, and bounded propagation polling. Private Pages repositories are rejected.

Browsing a bucket-hosted repository

A repository published to object storage serves index.html at its root, so https://packages.example/apt/ opens in a browser and shows what is published, how to install it, and each artifact's digest.

That page is a convenience copy and is not covered by the publication's guarantees. Nothing verifies it, observe does not read it, and a rollback does not restore it — it is refreshed by the next publication. The verified copy lives inside the release directory, because it is regenerated for every revision and writing it canonically would leave the previous revision unverifiable after a rollback. Clients are unaffected either way: they read simple/, index.yaml, repodata/ or SHA256SUMS, and those are the complete, verified answer.

The page shows the 500 most recently published artifacts and says so when there are more. A rendered row costs about 610 bytes, so a Debian suite of 63,440 artifacts would otherwise be a 38 MB page rebuilt and re-uploaded on every publication. Set it with Window if you want a different size; the footer always reports the repository's true total.

Back to snailmail.