Repository navigation
Conversation
Large blobs are the worst case for the git storer: EncodedObject buffers a whole body into memory, and an object over the 128 MiB pack cap gets a container to itself. LFS moves those bytes out of the repository entirely, and objgitd never carries them: the batch API answers with presigned URLs and the client talks to the bucket directly. Object bytes are shared bucket-wide at lfs/blobs/<oid>, so two repositories holding the same file store it once. Membership is separate, at <org>/<repo>/lfs/objects/<oid>, and every batch decision reads the marker rather than the blob. Without that split, knowing an oid would be enough to pull any repository's data through a repository you can reach. A client uploads into its own <org>/<repo>/lfs/staging/<oid>, and verify promotes it with Tigris' RenameObject. That indirection is what makes a marker mean anything: reading the shared blob would let anyone who knows an oid and a size -- both of which sit side by side in a pointer file -- claim an object they never had. Staging also keeps a rejected upload from touching the blob other repositories read. Upload integrity is enforced by the bucket. The presigned PUT carries ChecksumSHA256, and the checksum is signed rather than hoisted into the query string, so a client cannot drop the header to escape the check. Measured against a live bucket: wrong bytes 400, correct bytes 200, missing header 403. Downloads sign nothing but Host, so the URL stands alone. Locking, the verify endpoint, and git-lfs-authenticate over SSH are all here. The SSH command exits 127 when LFS is off, which is the only exit status git-lfs treats as "fall back" rather than as a hard failure. The feature is off by default; without -allow-lfs no route is registered and every LFS path is a 404. Assisted-by: Claude Opus 4.5 via Claude Code Signed-off-by: Xe Iaso <xe@tigrisdata.com>
.claude/ holds per-user settings and scratch git worktrees. The worktrees are themselves git repositories, so committing the directory would nest a repo inside this one and break clones. Assisted-by: Claude Opus 4.5 via Claude Code Signed-off-by: Xe Iaso <xe@tigrisdata.com>
Commits f4a1419 and ebfe4e3 moved objgitd off git's native bare-repository layout and onto packs/<id>.bin plus packs/<id>.cue containers. Nothing measured what that cost or bought. formatbench builds a daemon from the release before the change and one from the current checkout, pushes the same real repositories into both, and clones them back out. Both builds already report objgit_s3_requests_total with the same operation labels, so the request count each format costs compares directly and needs no proxy metric. Every cell gets a fresh repository prefix, pack cache, and daemon process, so nothing is measured warm by accident. Builds interleave rather than batch: network conditions drift over an afternoon, and running every "before" cell first would let that drift read as a result. The harness never deletes bucket data mid-run. It writes repos.txt and leaves cleanup to a later -cleanup invocation. Not shipped in the daemon; see docs/usage/format-benchmark.md. Assisted-by: Claude Opus 4.5 via Claude Code Signed-off-by: Xe Iaso <xe@tigrisdata.com>
Packs accumulate forever: there is no repacking and no garbage collection, so a repository's read cost grows with its push count rather than its size. pack-compaction.md is the design — how whole packs merge, how selection avoids a rewrite treadmill, the tombstone object, and how compaction behaves against a live push. pack-compaction-implementation.md is the task-by-task plan derived from it. Design only. No compaction code lands here. Assisted-by: Claude Opus 4.5 via Claude Code Signed-off-by: Xe Iaso <xe@tigrisdata.com>
efirs
approved these changes
Sep 22, 2026
A failed closing /metrics scrape set after = before and left Err and DNF empty, so the cell entered the report as a measurement of zero S3 requests. With the default two reps the median takes the lower index, so one timed-out scrape after a long push became the published figure. A partial bucket listing landed the same way, understating whichever format has the most keys to list. The before side also ran v1.0.2's listing-cache warmer, which lists on a wall-clock timer into the same counter the harness brackets. That inflation grows with elapsed time, on the axis the report ranks builds on. Pass -s3-cache-refresh=0 to a build whose own source defines it, detected by reading the checkout rather than by matching a ref name. Leave -s3-cache-ttl alone: zeroing it disables the cache outright and would inflate LIST counts with requests the release never made. Readiness now fails when the child dies, and a pre-flight scrape catches a leftover daemon holding the port, which otherwise answers the poll and serves every measurement from the wrong binary. Worktrees check out the requested ref before reuse, and the resolved commit reaches results.json and results.md so a published number can be checked afterwards. Assisted-by: Claude Opus 5 via Claude Code Signed-off-by: Xe Iaso <xe@tigrisdata.com>
Verify dropped the error from its marker read, so a transient HEAD failure answered 404 "object was not uploaded" and recorded the verify as missing, hiding the fault from the metric the docs say to alert on. A duplicate verify raced to a 500: the winner's rename consumed the staging key and the loser turned NoSuchKey into a server error. git-lfs reaches this by retrying a failed verify. Both entries now share one verifyPromoted helper, so a lost rename falls through to the marker check the staging-miss branch already used. An unparseable lfs-size read as size 0, which pinned the object behind a permanent 422 with no way for a client to correct it. An unreadable marker now reads as absent, so the next batch offers an upload action and the repository repairs itself on the next push. A warning names the raw value. A legitimate "0" still parses, so a zero-byte object is unaffected. Assisted-by: Claude Opus 5 via Claude Code Signed-off-by: Xe Iaso <xe@tigrisdata.com>
Only the batch handler capped its body. Verify, lock create, lock verify and unlock read theirs whole, and lock create stored the path verbatim into locks.json, which is re-read and re-encoded on every lock call and on every push. With -allow-push and anonymous access, one request could permanently degrade a repository's locking and push path. Every handler caps the body now, a lock path caps at 4096 bytes, and a repository holds at most 1000 locks. The LFS store ran on the unhardened client, so every marker, verify and lock call missed the retry and timeout wrapper the rest of the daemon uses, and a stale keep-alive connection could hang a request instead of failing fast. Only the presigner needs the concrete *s3.Client. The presigned-URL lifetime was clamped once at startup, which froze the one-minute floor for the life of the process when the daemon started against a nearly expired credential. It is evaluated per request now, and logs the clamp only on transition. Assisted-by: Claude Opus 5 via Claude Code Signed-off-by: Xe Iaso <xe@tigrisdata.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds Git LFS to
objgitd. Large files stay out of the git repository, and their bytes never pass through the daemon: the batch API answers with presigned Tigris URLs and the client transfers against the bucket directly.A 5 MB file pushed through this produced a 716-byte git pack.
Off by default. Without
-allow-lfsno LFS route is registered, so every LFS path is a 404 — which is what git-lfs and the locking spec expect from a server without LFS.Layout
Bytes are shared bucket-wide, so two repositories holding the same file store one copy. Membership is per repository, and every batch decision reads the marker rather than the blob.
A client uploads into its own staging key; verify promotes it with Tigris'
RenameObject. That indirection is what makes a marker mean anything — see below.Scope
Batch API,
verify, the file locking API, andgit-lfs-authenticateover SSH.-allow-lfsfalse-allow-lfs-lockstrue-external-url""(required for LFS overssh://)-lfs-url-ttl15m-lfs-max-size-lfs-max-batch500Three faults found during implementation
Each would have shipped as a silent failure. All three are pinned by tests.
Verify was a read oracle. As designed, verify granted membership when the shared blob existed at the declared size. But an oid and its size sit side by side in every pointer file, so anyone who could read a pointer could claim the object in their own repo and download it — exactly what the marker split was meant to prevent. The same path deleted the shared blob on a size mismatch, letting any writer destroy an object other repos depend on. Fixed with per-repo staging: membership now requires having actually written the bytes, and a rejected upload only touches the caller's own key.
The upload checksum was not signed. The AWS SDK hoists unrecognized
X-Amz-*headers into the query string, which would have leftx-amz-checksum-sha256unsigned and absent from the header map handed to the client — a checksum nobody sends and nothing enforces. Measured against a live bucket: wrong bytes400, correct bytes200, header omitted403. That last row is the point — the client cannot opt out.Fixing that broke downloads, and
git clonehid it. Disabling hoisting also signsx-amz-checksum-modeon GET, so the download URL only worked if the caller echoed that header back. git-lfs does echo it, so the clone passed; a plain fetch gotSignatureDoesNotMatch. Uploads and downloads now use separate signers.Verification
go build,go vet,staticcheck, andgo test ./...all clean, including-race.Against a real single-region Tigris bucket:
git lfs lock/locks/unlockworkinternal/lfs/livebucket_test.gocodifies the parts that need a bucket, behindOBJGIT_TIGRIS_LIVE_BUCKET. CI has no credentials, so everything else runs against a fake — except the presigner, which does no network I/O and is therefore tested for real.Known gaps
Documented in
docs/architecture/lfs.md:-lfs-max-sizecaps the declared size; an unverified upload lingers in staging (a bucket TTL rule on that prefix reaps them)Also on this branch
Two unrelated pieces of pre-existing working-tree work, committed separately so they can be split out if you'd rather:
feat(formatbench)(storage format comparison harness) anddocs(plans)(pack compaction design). Happy to move them to their own PR.Assisted-by: Claude Opus 4.5 via Claude Code