Skip to content

Tighten README wording #137

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
0lut merged 1 commit into from
Jul 4, 2026
Merged

Tighten README wording #137

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
82 changes: 39 additions & 43 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,28 +3,26 @@
[![Latest release](https://img.shields.io/github/v/tag/0lut/gitmirrorcache?sort=semver&label=release)](https://github.com/0lut/gitmirrorcache/tags)
[![Container image](https://img.shields.io/badge/ghcr.io-0lut%2Fgitmirrorcache-blue?logo=docker)](https://github.com/0lut/gitmirrorcache/pkgs/container/gitmirrorcache)

A read-only Git fetch cache that sits between clone-heavy automation — CI
runners, coding agents, sandboxes, build farms — and allowlisted HTTPS
upstreams addressable as `host/owner/repo` (GitHub-style remotes, including
top-level GitLab and Bitbucket repos). Instead of hammering the upstream with
thousands of identical clones, clients fetch from the cache: an S3-compatible object store is the durable source of truth, and
local disk is just a disposable hot layer that can be rebuilt at any time. The
result is faster clones, fewer upstream rate-limit headaches, and a cache you
can throw away without losing anything.
A read-only Git fetch cache for clone-heavy automation (CI runners, coding
agents, build farms). It sits in front of allowlisted HTTPS upstreams
addressable as `host/owner/repo`, so clients fetch from the cache instead of
hitting the upstream with thousands of identical clones. An S3-compatible
object store is the durable source of truth; local disk is a disposable hot
layer that can be rebuilt at any time.

It exposes two interfaces:

- A **materialization API** (`/v1/materialize`, `/v1/resolve`) for warming and
- A materialization API (`/v1/materialize`, `/v1/resolve`) for warming and
inspecting cache state by commit, branch, or default branch.
- A **read-through Git remote** at `/git/{host}/{owner}/{repo}.git` that
standard Git clients can clone and fetch from over Smart HTTP, including
shallow and blobless fetches. Pushes (`git-receive-pack`) are rejected.
- A read-through Git remote at `/git/{host}/{owner}/{repo}.git` that standard
Git clients can clone and fetch from over Smart HTTP, including shallow and
blobless fetches. Pushes (`git-receive-pack`) are rejected.

On a cold miss, the server proxies upstream's response straight to the client
(so first-clone latency stays close to a direct clone) while warming the cache
in the background. Once the cache is warm, pack data is served from the local
bare repo; ref advertisements and branch/default-branch selectors still verify
refs against upstream so clients never see stale tips.
and warms the cache in the background, so first-clone latency stays close to a
direct clone. Once the cache is warm, pack data is served from the local bare
repo. Ref advertisements and branch selectors still verify refs against
upstream, so clients never see stale tips.

## Quick Start

Expand Down Expand Up @@ -86,7 +84,7 @@ Clients can opt out of cold-miss proxying per request with the
The server reads either a TOML config file (set `GIT_CACHE_CONFIG` to its path;
see `config/*.example.toml`) or individual environment variables. When
`GIT_CACHE_CONFIG` is set, the file wins and the other configuration variables
are ignored — except S3 credentials (`GIT_CACHE_S3_ACCESS_KEY`,
are ignored. The exception is S3 credentials (`GIT_CACHE_S3_ACCESS_KEY`,
`GIT_CACHE_S3_SECRET_KEY`, `GIT_CACHE_S3_SESSION_TOKEN`,
`GIT_CACHE_S3_REGION`), which are always read from the environment and never
from the file.
Expand All @@ -95,11 +93,11 @@ from the file.

| Variable | Default | What it does |
| --- | --- | --- |
| `GIT_CACHE_CONFIG` | – | Path to a TOML config file. If set, other `GIT_CACHE_*` variables are ignored, except the S3 credential variables noted above. |
| `GIT_CACHE_CONFIG` | unset | Path to a TOML config file. If set, other `GIT_CACHE_*` variables are ignored, except the S3 credential variables noted above. |
| `GIT_CACHE_BIND_ADDR` | `127.0.0.1:8080` | Address and port the HTTP server listens on. |
| `GIT_CACHE_ROOT` | `./cache` | Directory for the local hot cache (bare repos, temp files, repo index). |
| `GIT_CACHE_ALLOWED_UPSTREAM_HOSTS` | `github.com` | Comma-separated allowlist of upstream hosts the cache will talk to (e.g. `github.com,gitlab.com,git.internal.example`). |
| `GIT_CACHE_UPSTREAM_ROOT` | – | Optional local directory of bare upstream repos, used instead of real network upstreams (mainly for tests and local dev). |
| `GIT_CACHE_UPSTREAM_ROOT` | unset | Optional local directory of bare upstream repos, used instead of real network upstreams (mainly for tests and local dev). |
| `GIT_CACHE_RATE_LIMIT_PER_MINUTE` | `120` | Global rate limit for materialize requests. |

### Object store
Expand All @@ -108,11 +106,11 @@ from the file.
| --- | --- | --- |
| `GIT_CACHE_OBJECT_STORE_KIND` | `local` | `local` (filesystem) or `s3`. S3 requires building with the `s3` feature. |
| `GIT_CACHE_OBJECT_STORE_ROOT` | `./tmp/object-store` | Root directory for the `local` object store. |
| `GIT_CACHE_S3_BUCKET` | – | S3 bucket name. Required when kind is `s3`. |
| `GIT_CACHE_S3_BUCKET` | unset | S3 bucket name. Required when kind is `s3`. |
| `GIT_CACHE_S3_PREFIX` | `repos` | Key prefix inside the bucket. A schema-version suffix is appended automatically (e.g. `repos` stores under `repos-v3`). |
| `GIT_CACHE_S3_ENDPOINT` | – | Custom S3 endpoint for compatible stores (MinIO, Cloudflare R2, ...). Enables path-style addressing. |
| `GIT_CACHE_S3_ENDPOINT` | unset | Custom S3 endpoint for compatible stores (MinIO, Cloudflare R2, ...). Enables path-style addressing. |
| `GIT_CACHE_S3_REGION` | falls back to `AWS_REGION` / `AWS_DEFAULT_REGION` | AWS region for the bucket. |
| `GIT_CACHE_S3_ACCESS_KEY` / `GIT_CACHE_S3_SECRET_KEY` | – | Static S3 credentials. If unset, the standard AWS credential chain is used (env vars, profiles, IAM roles, workload identity). |
| `GIT_CACHE_S3_ACCESS_KEY` / `GIT_CACHE_S3_SECRET_KEY` | unset | Static S3 credentials. If unset, the standard AWS credential chain is used (env vars, profiles, IAM roles, workload identity). |
| `GIT_CACHE_S3_SESSION_TOKEN` | falls back to `AWS_SESSION_TOKEN` | Session token for temporary credentials. |

### Git remote
Expand All @@ -134,13 +132,13 @@ from the file.
| `GIT_CACHE_MAX_CONCURRENT_GIT_PROCESSES` | `64` | Semaphore limiting concurrent Git subprocesses (the main CPU/memory knob). |
| `GIT_CACHE_ASYNC_MATERIALIZE_CONCURRENCY` | `2` | Concurrency for background materialization work. |
| `GIT_CACHE_USE_GITOXIDE` | `true` | Use in-process gitoxide for local read-only Git operations; disable as a kill switch to fall back to the `git` binary. |
| `GIT_CACHE_UPSTREAM_AUTH_TOKEN_ENV` | – | Name of another env var holding a deployment-wide upstream token, injected via Git config env (never argv or manifests). |
| `GIT_CACHE_UPSTREAM_AUTH_TOKEN_ENV` | unset | Name of another env var holding a deployment-wide upstream token, injected via Git config env (never argv or manifests). |

### Disk

| Variable | Default | What it does |
| --- | --- | --- |
| `GIT_CACHE_DISK_QUOTA_BYTES` | 10 GiB | Hot-cache disk quota; LRU eviction keeps usage under this. The Helm chart sets this to 100 GiB to match its default 100Gi PVC — keep the quota at or below the volume size. |
| `GIT_CACHE_DISK_QUOTA_BYTES` | 10 GiB | Hot-cache disk quota; LRU eviction keeps usage under this. The Helm chart sets this to 100 GiB to match its default 100Gi PVC. Keep the quota at or below the volume size. |
| `GIT_CACHE_DISK_MIN_FREE_BYTES` | 1 GiB | Minimum free disk space to preserve on the cache volume. |
| `GIT_CACHE_DISK_ACCESS_FLUSH_SECS` | `60` | How often buffered repo-access timestamps are flushed to the on-disk index. |

Expand All @@ -158,14 +156,13 @@ from the file.

### Helm (Kubernetes)

The project ships as a Helm chart that lives in this repository at
The Helm chart lives at
[`deploy/helm/gitmirrorcache`](deploy/helm/gitmirrorcache). Release charts are
published to GHCR as OCI artifacts alongside the container image. The chart runs
the server as a StatefulSet with a persistent volume for the hot cache and an
hourly CronJob for compaction:
published to GHCR as OCI artifacts alongside the container image. The chart
runs the server as a StatefulSet with a persistent volume for the hot cache
and an hourly CronJob for compaction.

Helm install commands take both a release name and a chart reference. From the
repository root, use the chart directory as the chart reference:
From the repository root:

```sh
helm install git-cache deploy/helm/gitmirrorcache \
Expand All @@ -184,9 +181,9 @@ helm install git-cache "${CHART_REF}" \
--set config.objectStore.s3.bucket=my-git-cache-bucket
```

For release tag `vX.Y.Z`, use chart version `X.Y.Z`. See the
Release tag `vX.Y.Z` maps to chart version `X.Y.Z`. See the
[chart README](deploy/helm/gitmirrorcache/README.md) for credentials, local
checkout installs, sizing, and scaling guidance.
checkout installs, sizing, and scaling.

### AWS (ECS on EC2)

Expand All @@ -203,16 +200,16 @@ Preview stacks for any branch, tag, or commit can be deployed behind a shared
ALB with `scripts/aws/deploy-preview.sh`. See
[docs/deployment.md](docs/deployment.md).

The production target pulls the latest published GHCR image for this repository
and skips local Docker/ECR image builds:
The production target pulls the latest published GHCR image and skips local
Docker/ECR image builds:

```sh
AWS_REGION=us-west-2 scripts/aws/deploy-prod.sh
```

The Cloudflare static front door serves the landing page at `gitcache.sh` and
proxies `/git/*` plus API paths to the production ALB, so public clones use the
apex hostname:
The Cloudflare front door serves the landing page at `gitcache.sh` and proxies
`/git/*` plus API paths to the production ALB, so public clones use the apex
hostname:

```sh
AWS_REGION=us-west-2 scripts/cloudflare/deploy-static-site.sh
Expand All @@ -222,8 +219,7 @@ git clone https://gitcache.sh/git/github.com/org/repo.git
### Docker

Prebuilt multi-arch images are published to `ghcr.io/0lut/gitmirrorcache` on
`v*` release tags — the badge at the top of this README always shows the
newest release:
`v*` release tags:

```sh
docker pull ghcr.io/0lut/gitmirrorcache:latest # most recent release
Expand All @@ -238,11 +234,11 @@ testing there is a MinIO compose file:
docker compose -f docker-compose.minio.yml up -d
```

### Bare metal / anything else
### Bare metal

It's a single binary plus the `git` CLI. Build with
It is a single binary plus the `git` CLI. Build with
`cargo build --release -p git-cache-api --features s3`, point it at a config
file or env vars, and put it behind whatever process supervisor you like.
file or env vars, and run it under any process supervisor.

## CLI

Expand Down Expand Up @@ -270,7 +266,7 @@ RUN_GITHUB_INTEGRATION=1 python3 -m unittest -v integration_tests.test_git_remot
```

See [integration_tests/README.md](integration_tests/README.md) for MinIO/S3
variants. The S3 adapter itself is feature-gated:
variants. The S3 adapter is feature-gated:

```sh
cargo test -p git-cache-objectstore --features s3
Expand Down
Loading