Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
87 changes: 87 additions & 0 deletions .github/workflows/docker.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,87 @@
name: Docker

on:
push:
pull_request:
branches: ["main"]

permissions:
contents: read
packages: write

concurrency:
group: docker-${{ github.ref }}
cancel-in-progress: true

jobs:
build:
runs-on: ubuntu-24.04
timeout-minutes: 30
# A pull request from a branch of this repository skips this job. The
# push event for that branch already built and published the commit, so
# a second run doubles the cost and adds nothing. A pull request from a
# fork has no push event here, so it runs.
if: "${{ github.event_name != 'pull_request' || github.event.pull_request.head.repo.full_name != github.repository }}"
steps:
- uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
with:
persist-credentials: false

- uses: docker/setup-buildx-action@f87e5991a6d7451dcb8d9637bfbc97413f497069 # v4.4.1

- name: Log in to the GitHub Container Registry
if: github.event_name == 'push'
uses: docker/login-action@dbcb813823bdd20940b903addbd779551569679f # v4.6.0
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}

- name: Image tags
id: meta
uses: docker/metadata-action@dc802804100637a589fabce1cb79ff13a1411302 # v6.2.0
with:
images: ghcr.io/${{ github.repository }}
tags: |
type=sha
type=ref,event=branch
type=semver,pattern={{version}}
type=raw,value=latest,enable={{is_default_branch}}

# Build one platform and load it, so the smoke test runs before any
# push. A broken image then never reaches the registry.
- name: Build the amd64 image
uses: docker/build-push-action@c3c9e263c25d99ce0380d002d59b67737d91b0dc # v7.4.0
with:
context: .
platforms: linux/amd64
load: true
tags: objgitd:smoke
cache-from: type=gha
cache-to: type=gha,mode=max

- name: Smoke test the health endpoint
run: |
docker run -d --name objgitd -e BUCKET=ci-dummy -p 127.0.0.1:9090:9090 objgitd:smoke
for i in $(seq 1 30); do
if curl -fsS http://127.0.0.1:9090/healthz; then
exit 0
fi
sleep 1
done
docker logs objgitd
exit 1

# The build stage is pinned to the runner platform (see the Dockerfile),
# so both targets compile without emulation. The builder cache already
# holds the amd64 layers from the smoke build. Push events publish.
# Pull requests build both platforms and push nothing.
- name: Build every platform
uses: docker/build-push-action@c3c9e263c25d99ce0380d002d59b67737d91b0dc # v7.4.0
with:
context: .
platforms: linux/amd64,linux/arm64
push: ${{ github.event_name == 'push' }}
tags: ${{ steps.meta.outputs.tags }}
cache-from: type=gha
cache-to: type=gha,mode=max
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
.env
/objgitd
manifest/secrets/

node_modules
44 changes: 22 additions & 22 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,8 +18,8 @@ One backend answers three transports:
All three funnel authorization through one pluggable
`internal/auth.Authorizer`.

A fourth listener serves Prometheus metrics at `/metrics` (`-metrics-bind`,
default `:9090`). An empty value disables it.
A fourth listener serves Prometheus metrics at `/metrics` and a `/healthz`
probe endpoint (`-metrics-bind`, default `:9090`). An empty value disables it.

## Commands

Expand Down Expand Up @@ -50,36 +50,36 @@ Notes on the tests and on configuration:

## Where the code lives

| Path | Purpose |
| ----------------------------- | ---------------------------------------------------------------------- |
| `cmd/objgitd/main.go` | Builds the one `*daemon` and starts every listener. |
| `cmd/objgitd/git_protocol.go` | The git:// server. Also holds `operationFor` and `(*daemon).authorize`. |
| `cmd/objgitd/http.go` | Smart HTTP. `*daemon` is the `http.Handler` itself. |
| `cmd/objgitd/ssh.go` | The SSH server and its per-session dispatch. |
| `cmd/objgitd/receivepack.go` | The go-git fork that streams hook output, plus `writePack`. |
| `cmd/objgitd/hooks.go` | Ref diffing and the sandboxed hook run. |
| `internal/auth` | The one authorization interface. |
| `internal/repofs` | Maps a repository path to a `storage.Storer`. |
| `internal/storage/tigris` | Repository storage. A `storage.Storer` on the bucket. |
| `internal/bundler` | The async upload queue behind that storer. |
| `internal/s3fs` | Daemon-level state only, which is the SSH host key. |
| `internal/mountfs`, `internal/treefs`, `internal/kefkash` | The hook sandbox filesystem and shell wiring. |
| `internal/metrics` | Every Prometheus vector, plus thin helpers. |
| `internal/slog.go` | JSON handler init. |
| `cmd/membench/` | Push memory benchmark harness. Not shipped; see `docs/usage/memory-benchmark.md`. |
| Path | Purpose |
| --------------------------------------------------------- | --------------------------------------------------------------------------------- |
| `cmd/objgitd/main.go` | Builds the one `*daemon` and starts every listener. |
| `cmd/objgitd/git_protocol.go` | The git:// server. Also holds `operationFor` and `(*daemon).authorize`. |
| `cmd/objgitd/http.go` | Smart HTTP. `*daemon` is the `http.Handler` itself. |
| `cmd/objgitd/ssh.go` | The SSH server and its per-session dispatch. |
| `cmd/objgitd/receivepack.go` | The go-git fork that streams hook output, plus `writePack`. |
| `cmd/objgitd/hooks.go` | Ref diffing and the sandboxed hook run. |
| `internal/auth` | The one authorization interface. |
| `internal/repofs` | Maps a repository path to a `storage.Storer`. |
| `internal/storage/tigris` | Repository storage. A `storage.Storer` on the bucket. |
| `internal/bundler` | The async upload queue behind that storer. |
| `internal/s3fs` | Daemon-level state only, which is the SSH host key. |
| `internal/mountfs`, `internal/treefs`, `internal/kefkash` | The hook sandbox filesystem and shell wiring. |
| `internal/metrics` | Every Prometheus vector, plus thin helpers. |
| `internal/slog.go` | JSON handler init. |
| `cmd/membench/` | Push memory benchmark harness. Not shipped; see `docs/usage/memory-benchmark.md`. |

## Architecture

Read [docs/architecture/README.md](docs/architecture/README.md) first. It
describes the daemon and links to one page for each subsystem.

| Page | Read it before you change... |
| ------------------------------------------------------ | ---------------------------------------------------------------------- |
| Page | Read it before you change... |
| ------------------------------------------------------ | ----------------------------------------------------------------------- |
| [transports.md](docs/architecture/transports.md) | Any transport. It holds two protocol points that are easy to get wrong. |
| [auth.md](docs/architecture/auth.md) | Credentials, decisions, or a new `Authorizer`. |
| [hooks.md](docs/architecture/hooks.md) | Push hooks, output streaming, or the sandbox. |
| [metrics.md](docs/architecture/metrics.md) | Any metric or instrumentation seam. |
| [tigris-storer.md](docs/architecture/tigris-storer.md) | Object layout, refs, packs, the pack cache, or the upload path. |
| [tigris-storer.md](docs/architecture/tigris-storer.md) | Object layout, refs, packs, the pack cache, or the upload path. |
| [s3fs.md](docs/architecture/s3fs.md) | The `billy.Filesystem` over the bucket. |

Two more directories carry detail:
Expand Down
12 changes: 10 additions & 2 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,7 +1,14 @@
# syntax=docker/dockerfile:1

# --- build stage -------------------------------------------------------------
FROM golang:1.26 AS build
# Pin the build stage to the platform of the runner ($BUILDPLATFORM) and
# cross-compile through TARGETOS and TARGETARCH. One native runner then
# builds every target platform, and no emulation is necessary. A plain
# `docker build` sets these values to the host platform, which keeps the
# previous behavior.
FROM --platform=$BUILDPLATFORM golang:1.26 AS build
ARG TARGETOS
ARG TARGETARCH

WORKDIR /src

Expand All @@ -16,7 +23,8 @@ COPY . .
# protocol natively (no `git` binary at runtime).
RUN --mount=type=cache,target=/go/pkg/mod \
--mount=type=cache,target=/root/.cache/go-build \
CGO_ENABLED=0 go build -trimpath -ldflags="-s -w" \
CGO_ENABLED=0 GOOS=$TARGETOS GOARCH=$TARGETARCH \
go build -trimpath -ldflags="-s -w" \
-o /objgitd ./cmd/objgitd

# --- runtime stage -----------------------------------------------------------
Expand Down
4 changes: 3 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -123,4 +123,6 @@ To ssh://localhost:2222/xeiaso.net/objgit.git

### Deployment to production

TODO(Xe): re-evaluate performance-based life choices and write.
There is a kustomize deployment in [`manifest/`](./manifest/). See
[docs/usage/kubernetes.md](./docs/usage/kubernetes.md) for configuration and
caveats.
33 changes: 25 additions & 8 deletions cmd/objgitd/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -170,14 +170,7 @@ func main() {
os.Exit(1)
}
runtime.SetBlockProfileRate(100)
mux := http.NewServeMux()
mux.Handle("/metrics", promhttp.Handler())
mux.HandleFunc("GET /debug/pprof/", pprof.Index)
mux.HandleFunc("GET /debug/pprof/cmdline", pprof.Cmdline)
mux.HandleFunc("GET /debug/pprof/profile", pprof.Profile)
mux.HandleFunc("GET /debug/pprof/symbol", pprof.Symbol)
mux.HandleFunc("GET /debug/pprof/trace", pprof.Trace)
srv := &http.Server{Handler: mux}
srv := &http.Server{Handler: newMetricsMux()}
g.Go(func() error {
if err := srv.Serve(ln); err != nil && !errors.Is(err, http.ErrServerClosed) {
return err
Expand Down Expand Up @@ -246,3 +239,27 @@ func main() {
os.Exit(1)
}
}

// newMetricsMux builds the metrics listener's handler: the Prometheus scrape
// endpoint, pprof, and /healthz for probes and load balancers.
func newMetricsMux() *http.ServeMux {
mux := http.NewServeMux()
mux.Handle("/metrics", promhttp.Handler())
mux.HandleFunc("GET /healthz", handleHealthz)
mux.HandleFunc("GET /debug/pprof/", pprof.Index)
mux.HandleFunc("GET /debug/pprof/cmdline", pprof.Cmdline)
mux.HandleFunc("GET /debug/pprof/profile", pprof.Profile)
mux.HandleFunc("GET /debug/pprof/symbol", pprof.Symbol)
mux.HandleFunc("GET /debug/pprof/trace", pprof.Trace)
return mux
}

// handleHealthz answers 200 while the process is serving. The daemon has no
// deeper signal to report. A dead transport listener fails the errgroup and
// exits the process, and the probe turns that exit into a restart. The
// handler reads no bucket state, so a Tigris outage does not fail the probes
// of every pod at the same time.
func handleHealthz(w http.ResponseWriter, _ *http.Request) {
w.Header().Set("Content-Type", "text/plain; charset=utf-8")
fmt.Fprintln(w, "ok")
}
70 changes: 70 additions & 0 deletions cmd/objgitd/main_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
package main

import (
"io"
"net/http"
"net/http/httptest"
"testing"
)

// TestMetricsMuxHealthz covers the health endpoint of the metrics listener.
// GET /healthz answers 200 with an ok body for probes. Other methods are
// rejected. /metrics keeps serving the scrape endpoint beside it.
func TestMetricsMuxHealthz(t *testing.T) {
ts := httptest.NewServer(newMetricsMux())
t.Cleanup(ts.Close)

for _, tt := range []struct {
name string
method string
path string
wantStatus int
wantBody string
}{
{
name: "GET /healthz answers 200 ok",
method: http.MethodGet,
path: "/healthz",
wantStatus: http.StatusOK,
wantBody: "ok\n",
},
{
name: "POST /healthz is not allowed",
method: http.MethodPost,
path: "/healthz",
wantStatus: http.StatusMethodNotAllowed,
},
{
name: "GET /metrics still serves the scrape endpoint",
method: http.MethodGet,
path: "/metrics",
wantStatus: http.StatusOK,
},
} {
t.Run(tt.name, func(t *testing.T) {
req, err := http.NewRequest(tt.method, ts.URL+tt.path, nil)
if err != nil {
t.Fatalf("NewRequest: %v", err)
}
resp, err := http.DefaultClient.Do(req)
if err != nil {
t.Fatalf("Do: %v", err)
}
defer resp.Body.Close()

if resp.StatusCode != tt.wantStatus {
t.Errorf("status = %d, want %d", resp.StatusCode, tt.wantStatus)
}
if tt.wantBody == "" {
return
}
body, err := io.ReadAll(resp.Body)
if err != nil {
t.Fatalf("ReadAll: %v", err)
}
if string(body) != tt.wantBody {
t.Errorf("body = %q, want %q", body, tt.wantBody)
}
})
}
}
3 changes: 2 additions & 1 deletion docs/architecture/metrics.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,8 @@ the **default registry**. The Go-runtime and process collectors from
`main.go` serves `promhttp.Handler()` on its own listener. The flag is
`-metrics-bind`, the default is `:9090`, and an empty value disables the
listener. It uses the same errgroup Serve and Shutdown idiom as the HTTP
transport.
transport. The same listener also answers `GET /healthz` with `200 ok` for
probes and load balancers, and serves pprof under `/debug/pprof/`.

All series carry the prefix `objgit_`. There is no `repo` label, because
repository names have unbounded cardinality. Git operations are keyed by
Expand Down
Loading
Loading