Skip to content

feat(scripts): curl-installer harvester to close the Linux-install gap#66

Merged
noahhyden merged 1 commit into
mainfrom
feat/curl-installer-harvester
Jul 21, 2026
Merged

feat(scripts): curl-installer harvester to close the Linux-install gap#66
noahhyden merged 1 commit into
mainfrom
feat/curl-installer-harvester

Conversation

@noahhyden

Copy link
Copy Markdown
Owner

The gap

240 cli-tool recipes install nothing on a Homebrew-less Linux box — their only methods are brew/winget, so every guard fails and bootstrap dead-ends. That was the ollama bug (#65), registry-wide.

What this adds

scripts/curl_candidates.py — the maintainer helper that closes the mechanical subset, in the same propose → review → merge idiom as winget_candidates.py.

For each Linux-gapped, curl-less row whose id is in a vetted KNOWN_INSTALLERS allowlist, it proposes the official install script. Unlike winget there's no authoritative catalog, and a wrong installer runs the wrong software — so this is deliberately an allowlist, not a repo-probing guess. Every URL was confirmed to serve a real script (HTTP 200 + shebang). Each proposal is corroborated against the row's gh:

  • high — URL served from the tool's own repo (raw.githubusercontent.com/<owner>/<repo>/...), or a domain matching the gh owner/repo (e.g. deno.land ~ denoland/deno).
  • medium — a plausible official domain that doesn't corroborate; a human should eyeball it.

Only gapped rows lacking a curl key are proposed (idempotent against prior backfills); --apply stages keys into recipe_data.py for git review.

Generator

Adds a curl_shell field to gen_recipes.py so bash-only scripts render | bash instead of | sh (default stays sh) — needed because ~40% of official installers require bash.

Seeded allowlist (10, all HIGH, verified)

sh: deno, chezmoi, tailscale, pulumi, golangci-lint, trivy
bash: rclone, k3d, direnv, volta

$ python3 scripts/curl_candidates.py
resolved 10 curl installer(s) (high=10 medium=0) of 240 Linux-gapped row(s) -> curl_proposals.json

Not yet backfilled into recipe_data.py — left for your review (run --apply, then gen_recipes.py --rewrite). The remaining ~230 gapped tools need per-tool decisions (system package managers like apt/dnf, or GitHub-release binaries) that this curl harvester intentionally doesn't guess at.

Tests

tests/test_curl_candidates.py — 25 tests pinning the gap filter, the corroboration tiers (the security-relevant surface), idempotency, and apply_proposals staging. No network (allowlist is static). Full suite: 3645 passed, lint clean across 520 recipes.

🤖 Generated with Claude Code

240 cli-tool recipes install nothing on a Homebrew-less Linux box — their
only methods are brew/winget, so every guard fails and bootstrap dead-ends
(the ollama bug, registry-wide). This adds the maintainer helper that closes
the mechanical subset, in the same "propose, a human merges" idiom as the
winget harvester.

scripts/curl_candidates.py proposes an official `curl | sh` installer for
each Linux-gapped, curl-less row whose id is in a vetted KNOWN_INSTALLERS
allowlist. Unlike winget there's no authoritative catalog and a wrong
installer runs the WRONG software, so this is deliberately an allowlist (not
a repo-probing guess); each URL was confirmed to serve a real script. Every
proposal is corroborated against the row's `gh`:
  high    URL served from the tool's own repo (raw.githubusercontent.com/
          owner/repo/...) or a domain matching the gh owner/repo
  medium  a plausible official domain that doesn't corroborate — eyeball it
Only gapped rows lacking a curl key are proposed (idempotent); `--apply`
stages keys into recipe_data.py for git review. Stdlib only.

Also add a `curl_shell` field to the generator (gen_recipes.py) so bash-only
install scripts render `| bash` instead of `| sh` (default stays sh).

Seeded allowlist (10, all HIGH, verified 200 + shebang): deno, chezmoi,
tailscale, pulumi, golangci-lint, trivy (sh) and rclone, k3d, direnv, volta
(bash). Not yet backfilled into recipe_data.py — left for review. Lint green
(520), pytest 3645 passed (25 new).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@noahhyden
noahhyden merged commit 31a3834 into main Jul 21, 2026
5 checks passed
@noahhyden
noahhyden deleted the feat/curl-installer-harvester branch July 21, 2026 11:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant