Skip to content

feat(scripts): winget id harvester to close the Windows-install gap#61

Merged
noahhyden merged 3 commits into
mainfrom
feat/winget-harvester
Jul 20, 2026
Merged

feat(scripts): winget id harvester to close the Windows-install gap#61
noahhyden merged 3 commits into
mainfrom
feat/winget-harvester

Conversation

@noahhyden

Copy link
Copy Markdown
Owner

What & why

The generator already renders a winget install/update method whenever a
recipe_data.py row carries a winget key — but only 13 of 520 recipes do, so
the ~483 bulk-generated recipes install nothing on Windows. This adds a
maintainer harvester that resolves winget PackageIdentifiers for the rows that
lack one, closing that gap. (Addresses the "generated recipes are POSIX-first
(no winget methods yet)" note under 0.2.1.)

How it resolves ids (confidence tiers)

Queries the winget community-source index — the SQLite catalog (Public/index.db
inside source2.msix) that the winget CLI itself uses — not the msstore source.
Each row is tiered by trust:

tier rule
high id is the row's gh owner/repo exactly, or a command/moniker match whose publisher == the GitHub owner
medium a unique installed-command match, no gh corroboration
low moniker-only (weakest — monikers collide across unrelated apps)

Publisher corroboration is what makes high trustworthy: it rejects look-alike
false positives — e.g. the moniker pastel matches the Windows app
Japplis.Pastel, but the gh: sharkdp/pastel corroboration resolves the correct
sharkdp.pastel instead.

Usage

python3 scripts/winget_candidates.py                     # HIGH proposals -> winget_proposals.json
python3 scripts/winget_candidates.py --min-confidence medium
python3 scripts/winget_candidates.py --apply             # stage keys into recipe_data.py (review via git diff)

Stdlib only, installs nothing, runs anywhere. Default emits a JSON artifact for
review (same propose→human-merges idiom as the brew ingester); --apply stages
the keys as a reviewable diff. A new manual winget-ingest workflow uploads the
proposals artifact. Nothing about recipe_data.py/recipes/ changes in this PR
applying the keys is the follow-up you approve after reviewing the table below.

Verification (per the review discipline)

  • Tests-first, 31 new tests, 100% coverage of the module; full suite 3604 passed, 511 skipped.
  • Live: real CDN download and cached-index runs both resolve 145 HIGH ids; --apply on a copy of the real recipe_data.py produces valid Python (py_compile OK); a resolved row renders a lint-clean recipe with guarded winget install/upgrade methods through the real gen_recipes + lint_recipes.
  • Mutation red-teaming: 5 injected mutations (publisher substring-vs-equality, exact-id anchoring, truthy-vs-key-presence, gh-id precedence, threshold >=) each turn the suite red.
  • ruff 0.15.20 clean, py_compile OK, recipe linter clean.

Impact

A HIGH-confidence run resolves 145 ids → winget coverage 13 → ~158 recipes (~30%).
The 145 proposed keys for review:

145 HIGH-confidence proposals:

recipe winget id matched_by
act nektos.act gh-id-exact
age FiloSottile.age gh-id-exact
aichat sigoden.AIChat gh-id-exact
argocd argoproj.argocd cmd+publisher
aria2 aria2.aria2 gh-id-exact
ast-grep ast-grep.ast-grep gh-id-exact
atuin Atuinsh.Atuin gh-id-exact
aws-vault 99designs.aws-vault gh-id-exact
biome BiomeJS.Biome gh-id-exact
black psf.black gh-id-exact
brotli Google.Brotli gh-id-exact
btop aristocratos.btop4win moniker+publisher
buf bufbuild.buf gh-id-exact
bun Oven-sh.Bun gh-id-exact
caddy CaddyServer.Caddy gh-id-exact
ccache Ccache.Ccache gh-id-exact
chafa hpjansson.Chafa gh-id-exact
chezmoi twpayne.chezmoi gh-id-exact
cloudflared Cloudflare.cloudflared gh-id-exact
cmake Kitware.CMake gh-id-exact
consul Hashicorp.Consul gh-id-exact
cosign Sigstore.Cosign gh-id-exact
croc schollz.croc gh-id-exact
crystal CrystalLang.Crystal cmd+publisher
curl cURL.cURL gh-id-exact
dbmate amacneil.dbmate gh-id-exact
deno DenoLand.Deno gh-id-exact
direnv direnv.direnv gh-id-exact
dive wagoodman.dive gh-id-exact
doctl DigitalOcean.Doctl gh-id-exact
dog ogham.dog gh-id-exact
dprint dprint.dprint gh-id-exact
dua Byron.dua-cli gh-id-exact
duf muesli.duf gh-id-exact
fastfetch Fastfetch-cli.Fastfetch gh-id-exact
fend printfn.fend gh-id-exact
ffuf ffuf.ffuf gh-id-exact
flatbuffers Google.flatbuffers gh-id-exact
fnm Schniz.fnm gh-id-exact
freeze charmbracelet.freeze gh-id-exact
fswatch emcrisostomo.fswatch gh-id-exact
fzf junegunn.fzf gh-id-exact
gallery-dl mikf.gallery-dl gh-id-exact
gdu dundee.gdu gh-id-exact
genact svenstaro.genact gh-id-exact
ghq x-motemen.ghq gh-id-exact
gifski ImageOptim.gifski gh-id-exact
git-absorb tummychow.git-absorb gh-id-exact
git-cliff orhun.git-cliff gh-id-exact
gitleaks Gitleaks.Gitleaks gh-id-exact
glow charmbracelet.glow gh-id-exact
golangci-lint GolangCI.golangci-lint gh-id-exact
goreleaser goreleaser.goreleaser gh-id-exact
gping orf.gping gh-id-exact
grpcurl fullstorydev.grpcurl gh-id-exact
grype Anchore.Grype gh-id-exact
gum charmbracelet.gum gh-id-exact
hadolint hadolint.hadolint gh-id-exact
hatch PyPA.Hatch gh-id-exact
hcloud HetznerCloud.CLI gh-id-exact
helm Helm.Helm gh-id-exact
hexyl sharkdp.hexyl gh-id-exact
hurl Orange-OpenSource.Hurl gh-id-exact
imagemagick ImageMagick.ImageMagick gh-id-exact
infracost Infracost.Infracost gh-id-exact
julia Julialang.Julia gh-id-exact
just Casey.Just gh-id-exact
k9s Derailed.k9s gh-id-exact
kalker PaddiM8.kalker gh-id-exact
kompose Kubernetes.kompose gh-id-exact
kondo tbillington.kondo gh-id-exact
kopia Kopia.KopiaCLI cmd+publisher
kube-linter stackrox.kube-linter gh-id-exact
kubectl Kubernetes.kubectl cmd+publisher
kubectx ahmetb.kubectx gh-id-exact
kubescape kubescape.kubescape gh-id-exact
lazydocker JesseDuffield.Lazydocker gh-id-exact
lefthook evilmartians.lefthook gh-id-exact
lsd lsd-rs.lsd gh-id-exact
mc MinIO.Client cmd+publisher
meson mesonbuild.meson gh-id-exact
micro zyedidia.micro gh-id-exact
minikube Kubernetes.minikube gh-id-exact
miniserve svenstaro.miniserve gh-id-exact
mise jdx.mise gh-id-exact
mitmproxy mitmproxy.mitmproxy gh-id-exact
mkcert FiloSottile.mkcert gh-id-exact
mods charmbracelet.mods gh-id-exact
mprocs pvolok.mprocs gh-id-exact
neovim Neovim.Neovim gh-id-exact
ninja Ninja-build.Ninja gh-id-exact
nomad Hashicorp.Nomad gh-id-exact
nushell Nushell.Nushell gh-id-exact
oha hatoo.oha gh-id-exact
ollama Ollama.Ollama gh-id-exact
onefetch o2sh.onefetch gh-id-exact
opa open-policy-agent.opa gh-id-exact
opam OCaml.opam gh-id-exact
oras ORASProject.ORAS moniker+publisher
ov noborus.ov gh-id-exact
oxipng Shssoichiro.Oxipng gh-id-exact
packer Hashicorp.Packer gh-id-exact
pastel sharkdp.pastel gh-id-exact
pnpm pnpm.pnpm gh-id-exact
presenterm mfontanini.presenterm gh-id-exact
pulumi Pulumi.Pulumi gh-id-exact
qpdf QPDF.QPDF gh-id-exact
rclone Rclone.Rclone gh-id-exact
redis Redis.Redis gh-id-exact
restic restic.restic gh-id-exact
ruff astral-sh.ruff gh-id-exact
sccache Mozilla.sccache gh-id-exact
serverless Serverless.Serverless gh-id-exact
shellcheck koalaman.shellcheck gh-id-exact
shfmt mvdan.shfmt cmd+publisher
stack commercialhaskell.stack gh-id-exact
starship Starship.Starship gh-id-exact
step Smallstep.step moniker+publisher
stern stern.stern gh-id-exact
streamlink Streamlink.Streamlink gh-id-exact
stylua JohnnyMorganz.StyLua gh-id-exact
syft Anchore.Syft gh-id-exact
tailscale Tailscale.Tailscale gh-id-exact
tailwindcss TailwindLabs.TailwindCSS gh-id-exact
taplo tamasfe.taplo gh-id-exact
television alexpasmantier.television gh-id-exact
terraform-docs Terraform-docs.Terraform-docs gh-id-exact
tesseract tesseract-ocr.tesseract gh-id-exact
tflint TerraformLinters.tflint cmd+publisher
topgrade topgrade-rs.topgrade gh-id-exact
trivy AquaSecurity.Trivy gh-id-exact
typos Crate-CI.Typos gh-id-exact
typst Typst.Typst gh-id-exact
ugrep Genivia.ugrep gh-id-exact
vale errata-ai.Vale gh-id-exact
vault Hashicorp.Vault gh-id-exact
vhs charmbracelet.vhs gh-id-exact
vivid sharkdp.vivid gh-id-exact
wasmtime BytecodeAlliance.Wasmtime gh-id-exact
watchman facebook.watchman gh-id-exact
xh ducaale.xh gh-id-exact
xonsh xonsh.xonsh-winget moniker+publisher
yazi sxyazi.yazi gh-id-exact
yt-dlp yt-dlp.yt-dlp gh-id-exact
zola getzola.zola gh-id-exact

🤖 Generated with Claude Code

noahhyden and others added 3 commits July 20, 2026 09:01
The generator already renders a winget install/update method whenever a
recipe_data.py row carries a `winget` key, but almost no rows do — so the
bulk-generated recipes install nothing on Windows. scripts/winget_candidates.py
resolves winget PackageIdentifiers against the winget community-source index
(the SQLite catalog winget itself uses), tiered by confidence:

  high    gh owner/repo exact id, or command/moniker match whose publisher == gh owner
  medium  a unique installed-command match, no gh corroboration
  low     moniker-only (weakest — monikers collide across unrelated apps)

Publisher corroboration rejects look-alike false positives (e.g. the Windows
app Japplis.Pastel for the Rust sharkdp.pastel). Default emits
winget_proposals.json for review; --apply stages keys into recipe_data.py as a
reviewable git diff. Stdlib only, installs nothing, runs anywhere.

A HIGH run resolves ~145 ids (winget coverage 13 → ~158 recipes). Adds a manual
winget-ingest workflow (uploads proposals artifact) mirroring the brew ingester.

Tests: 31 new (100% coverage of the module), including mutation guards on every
confidence boundary; full suite 3604 passed. conftest puts scripts/ on sys.path.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…port

Applied all resolved winget PackageIdentifiers (145 high / 29 medium / 20 low)
into recipe_data.py and regenerated the recipes, taking winget coverage from 13
to 207 recipes. Windows is best-effort: bulk-resolved ids are only spot-checked,
so some may be wrong/stale/look-alikes (a winget install can fail or install the
wrong tool). Each method's `when:` guard makes a bad winget method fail cleanly
rather than break a POSIX bootstrap. Added a README warning + CHANGELOG note.

test_every_winget_recipe_is_covered now pins winget recipes to two sources of
truth — the hand-curated flagship map (proven by the windows real-bootstrap job)
and recipe_data.py (bulk, best-effort) — instead of the flagship-only lock.

Full suite: 3604 passed, 511 skipped. Recipes lint clean; ruff clean.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@noahhyden
noahhyden merged commit eaa149c into main Jul 20, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant