Summary
Replace the os/exec calls to a system git binary with the pure-Go go-git library, so GitGrab has no runtime dependency on an external git executable.
Today GitGrab is a Go program that shells out to git for every clone/pull/fetch. That means any environment running GitGrab must also ship a full git install (and its transitive shared libraries). For a CGO_ENABLED=0 static binary this is an unnecessary and painful dependency — most acutely when packaging GitGrab into a minimal container image.
Motivation
GitGrab is consumed by KOHO's Hopper repo-sync task, which wants to run it from a minimal (scratch / distroless) container. Because GitGrab requires a real git at runtime, that image can't be truly minimal:
scratch/distroless ship no git and no package manager, so git + git-core helpers + every transitive .so must be hand-copied via ldd scraping — fragile and high-churn (it broke repeatedly across several fixup commits).
- glibc's DNS resolution
dlopens libnss_dns.so at runtime, which ldd does not report, so HTTPS clones silently fail in a hand-assembled image.
- The workaround is to fall back to Alpine +
apk add git, which defeats the goal of a minimal, low-CVE-surface image.
A pure-Go implementation makes GitGrab a single self-contained static binary that drops cleanly into FROM scratch (plus CA certs), eliminating all of the above.
Current behavior
git is invoked via os/exec in two places:
cmd/gitgrab/main.go
exec.LookPath("git") — hard preflight check that aborts if git isn't on PATH.
gitgrab.go
getCurrentBranch() → git -C <path> branch --show-current
CloneRepo():
git clone <url> <path> for new repos
git -C <path> pull when on the default branch (main/master)
git -C <path> fetch when on a non-default branch, or as a fallback when branch detection fails / default branch is unknown
Auth is handled by URL construction:
- HTTP: private repos use
https://<token>@github.com/<org>/<repo>.git
- SSH:
git@github.com:<org>/<repo>.git (relies on the host's SSH agent/keys)
Proposed change
Swap the subprocess calls for go-git (github.com/go-git/go-git/v5), keeping the existing CLI, flags, and behavior identical.
Rough operation mapping:
| Current subprocess |
go-git equivalent |
git clone <url> <path> |
git.PlainClone(path, false, &git.CloneOptions{URL, Auth}) |
git branch --show-current |
repo.Head() → ref.Name().Short() |
git pull |
worktree.Pull(&git.PullOptions{Auth, ...}), treating git.NoErrAlreadyUpToDate as success |
git fetch |
repo.Fetch(&git.FetchOptions{Auth, ...}), treating git.NoErrAlreadyUpToDate as success |
| open existing repo |
git.PlainOpen(path) |
Auth:
- HTTP/token:
&githttp.BasicAuth{Username: "x-access-token", Password: <token>} instead of embedding the token in the URL (also keeps the token out of any URL that might get logged).
- SSH:
go-git's ssh transport (gitssh.NewSSHAgentAuth or NewPublicKeysFromFile) with an explicit known-hosts / host-key callback policy.
Also remove the exec.LookPath("git") preflight in main.go — it becomes meaningless once there's no external git.
Design considerations & edge cases
- SSH host-key verification: the shelled-out
git relied on the host's known_hosts. With go-git we must decide the HostKeyCallback policy explicitly. Container use will likely need known-hosts seeded or a documented policy.
pull semantics: go-git Pull fast-forwards the current branch; ensure divergent/non-fast-forward cases are handled the same way the CLI currently tolerates them (surface as a per-repo failure, keep going).
git.NoErrAlreadyUpToDate: both Pull and Fetch return this sentinel when there's nothing to do — must be treated as success, not an error, to preserve current output.
- Shallow clones: if we want to keep sync fast/cheap, evaluate
CloneOptions.Depth (note go-git's historical rough edges with shallow fetch).
- Git LFS: go-git does not support LFS. Confirm none of the target repos rely on LFS, or document the limitation.
- Submodules: default behavior differs; set
SubmoduleRescursivity/recurse options intentionally if submodules matter.
- Performance/memory: go-git can use more memory and be slower than C git on very large repos/monorepos — worth a sanity check against the largest KOHO repos.
- CGO: go-git is pure Go, so
CGO_ENABLED=0 static builds continue to work (and now actually deliver a dependency-free binary).
Acceptance criteria
Testing
- Unit tests around branch detection and the clone/pull/fetch decision logic (the existing
HTTPClient interface pattern makes the GitHub API side already testable; do the same for the git operations behind an interface so they can be faked).
- Integration test against a throwaway org (or a local bare repo fixture) exercising: fresh clone, no-op update, update-with-changes, non-default-branch fetch.
- Verify the resulting binary runs in
scratch.
Out of scope
- Changing the CLI surface, flags, or auth model.
- Adding LFS support.
Summary
Replace the
os/execcalls to a systemgitbinary with the pure-Gogo-gitlibrary, so GitGrab has no runtime dependency on an externalgitexecutable.Today GitGrab is a Go program that shells out to
gitfor every clone/pull/fetch. That means any environment running GitGrab must also ship a fullgitinstall (and its transitive shared libraries). For aCGO_ENABLED=0static binary this is an unnecessary and painful dependency — most acutely when packaging GitGrab into a minimal container image.Motivation
GitGrab is consumed by KOHO's Hopper repo-sync task, which wants to run it from a minimal (
scratch/ distroless) container. Because GitGrab requires a realgitat runtime, that image can't be truly minimal:scratch/distroless ship nogitand no package manager, sogit+git-corehelpers + every transitive.somust be hand-copied vialddscraping — fragile and high-churn (it broke repeatedly across several fixup commits).dlopenslibnss_dns.soat runtime, whichldddoes not report, so HTTPS clones silently fail in a hand-assembled image.apk add git, which defeats the goal of a minimal, low-CVE-surface image.A pure-Go implementation makes GitGrab a single self-contained static binary that drops cleanly into
FROM scratch(plus CA certs), eliminating all of the above.Current behavior
gitis invoked viaos/execin two places:cmd/gitgrab/main.goexec.LookPath("git")— hard preflight check that aborts ifgitisn't onPATH.gitgrab.gogetCurrentBranch()→git -C <path> branch --show-currentCloneRepo():git clone <url> <path>for new reposgit -C <path> pullwhen on the default branch (main/master)git -C <path> fetchwhen on a non-default branch, or as a fallback when branch detection fails / default branch is unknownAuth is handled by URL construction:
https://<token>@github.com/<org>/<repo>.gitgit@github.com:<org>/<repo>.git(relies on the host's SSH agent/keys)Proposed change
Swap the subprocess calls for
go-git(github.com/go-git/go-git/v5), keeping the existing CLI, flags, and behavior identical.Rough operation mapping:
git clone <url> <path>git.PlainClone(path, false, &git.CloneOptions{URL, Auth})git branch --show-currentrepo.Head()→ref.Name().Short()git pullworktree.Pull(&git.PullOptions{Auth, ...}), treatinggit.NoErrAlreadyUpToDateas successgit fetchrepo.Fetch(&git.FetchOptions{Auth, ...}), treatinggit.NoErrAlreadyUpToDateas successgit.PlainOpen(path)Auth:
&githttp.BasicAuth{Username: "x-access-token", Password: <token>}instead of embedding the token in the URL (also keeps the token out of any URL that might get logged).go-git'ssshtransport (gitssh.NewSSHAgentAuthorNewPublicKeysFromFile) with an explicit known-hosts / host-key callback policy.Also remove the
exec.LookPath("git")preflight inmain.go— it becomes meaningless once there's no externalgit.Design considerations & edge cases
gitrelied on the host'sknown_hosts. With go-git we must decide theHostKeyCallbackpolicy explicitly. Container use will likely need known-hosts seeded or a documented policy.pullsemantics: go-gitPullfast-forwards the current branch; ensure divergent/non-fast-forward cases are handled the same way the CLI currently tolerates them (surface as a per-repo failure, keep going).git.NoErrAlreadyUpToDate: both Pull and Fetch return this sentinel when there's nothing to do — must be treated as success, not an error, to preserve current output.CloneOptions.Depth(note go-git's historical rough edges with shallow fetch).SubmoduleRescursivity/recurse options intentionally if submodules matter.CGO_ENABLED=0static builds continue to work (and now actually deliver a dependency-free binary).Acceptance criteria
os/execusage remains; GitGrab runs with nogitbinary onPATH.gitgrab -o <org> -m http <dir>clones new repos and updates existing ones (pull on default branch, fetch otherwise), matching current behavior.gitgrab -o <org> -m ssh <dir>works with a documented host-key policy.CGO_ENABLED=0) and runs in aFROM scratchimage with only CA certs added.Testing
HTTPClientinterface pattern makes the GitHub API side already testable; do the same for the git operations behind an interface so they can be faked).scratch.Out of scope