Repository navigation
feat(lfs): serve Git LFS and resolve EROFS pointers - #18
Merged
Merged
Conversation
Large blobs are the worst case for the git storer: EncodedObject buffers a whole body into memory, and an object over the 128 MiB pack cap gets a container to itself. LFS moves those bytes out of the repository entirely, and objgitd never carries them: the batch API answers with presigned URLs and the client talks to the bucket directly. Object bytes are shared bucket-wide at lfs/blobs/<oid>, so two repositories holding the same file store it once. Membership is separate, at <org>/<repo>/lfs/objects/<oid>, and every batch decision reads the marker rather than the blob. Without that split, knowing an oid would be enough to pull any repository's data through a repository you can reach. A client uploads into its own <org>/<repo>/lfs/staging/<oid>, and verify promotes it with Tigris' RenameObject. That indirection is what makes a marker mean anything: reading the shared blob would let anyone who knows an oid and a size -- both of which sit side by side in a pointer file -- claim an object they never had. Staging also keeps a rejected upload from touching the blob other repositories read. Upload integrity is enforced by the bucket. The presigned PUT carries ChecksumSHA256, and the checksum is signed rather than hoisted into the query string, so a client cannot drop the header to escape the check. Measured against a live bucket: wrong bytes 400, correct bytes 200, missing header 403. Downloads sign nothing but Host, so the URL stands alone. Locking, the verify endpoint, and git-lfs-authenticate over SSH are all here. The SSH command exits 127 when LFS is off, which is the only exit status git-lfs treats as "fall back" rather than as a hard failure. The feature is off by default; without -allow-lfs no route is registered and every LFS path is a 404. Assisted-by: Claude Opus 4.5 via Claude Code Signed-off-by: Xe Iaso <xe@tigrisdata.com>
.claude/ holds per-user settings and scratch git worktrees. The worktrees are themselves git repositories, so committing the directory would nest a repo inside this one and break clones. Assisted-by: Claude Opus 4.5 via Claude Code Signed-off-by: Xe Iaso <xe@tigrisdata.com>
Commits f4a1419 and ebfe4e3 moved objgitd off git's native bare-repository layout and onto packs/<id>.bin plus packs/<id>.cue containers. Nothing measured what that cost or bought. formatbench builds a daemon from the release before the change and one from the current checkout, pushes the same real repositories into both, and clones them back out. Both builds already report objgit_s3_requests_total with the same operation labels, so the request count each format costs compares directly and needs no proxy metric. Every cell gets a fresh repository prefix, pack cache, and daemon process, so nothing is measured warm by accident. Builds interleave rather than batch: network conditions drift over an afternoon, and running every "before" cell first would let that drift read as a result. The harness never deletes bucket data mid-run. It writes repos.txt and leaves cleanup to a later -cleanup invocation. Not shipped in the daemon; see docs/usage/format-benchmark.md. Assisted-by: Claude Opus 4.5 via Claude Code Signed-off-by: Xe Iaso <xe@tigrisdata.com>
Packs accumulate forever: there is no repacking and no garbage collection, so a repository's read cost grows with its push count rather than its size. pack-compaction.md is the design — how whole packs merge, how selection avoids a rewrite treadmill, the tombstone object, and how compaction behaves against a live push. pack-compaction-implementation.md is the task-by-task plan derived from it. Design only. No compaction code lands here. Assisted-by: Claude Opus 4.5 via Claude Code Signed-off-by: Xe Iaso <xe@tigrisdata.com>
A failed closing /metrics scrape set after = before and left Err and DNF empty, so the cell entered the report as a measurement of zero S3 requests. With the default two reps the median takes the lower index, so one timed-out scrape after a long push became the published figure. A partial bucket listing landed the same way, understating whichever format has the most keys to list. The before side also ran v1.0.2's listing-cache warmer, which lists on a wall-clock timer into the same counter the harness brackets. That inflation grows with elapsed time, on the axis the report ranks builds on. Pass -s3-cache-refresh=0 to a build whose own source defines it, detected by reading the checkout rather than by matching a ref name. Leave -s3-cache-ttl alone: zeroing it disables the cache outright and would inflate LIST counts with requests the release never made. Readiness now fails when the child dies, and a pre-flight scrape catches a leftover daemon holding the port, which otherwise answers the poll and serves every measurement from the wrong binary. Worktrees check out the requested ref before reuse, and the resolved commit reaches results.json and results.md so a published number can be checked afterwards. Assisted-by: Claude Opus 5 via Claude Code Signed-off-by: Xe Iaso <xe@tigrisdata.com>
Verify dropped the error from its marker read, so a transient HEAD failure answered 404 "object was not uploaded" and recorded the verify as missing, hiding the fault from the metric the docs say to alert on. A duplicate verify raced to a 500: the winner's rename consumed the staging key and the loser turned NoSuchKey into a server error. git-lfs reaches this by retrying a failed verify. Both entries now share one verifyPromoted helper, so a lost rename falls through to the marker check the staging-miss branch already used. An unparseable lfs-size read as size 0, which pinned the object behind a permanent 422 with no way for a client to correct it. An unreadable marker now reads as absent, so the next batch offers an upload action and the repository repairs itself on the next push. A warning names the raw value. A legitimate "0" still parses, so a zero-byte object is unaffected. Assisted-by: Claude Opus 5 via Claude Code Signed-off-by: Xe Iaso <xe@tigrisdata.com>
Only the batch handler capped its body. Verify, lock create, lock verify and unlock read theirs whole, and lock create stored the path verbatim into locks.json, which is re-read and re-encoded on every lock call and on every push. With -allow-push and anonymous access, one request could permanently degrade a repository's locking and push path. Every handler caps the body now, a lock path caps at 4096 bytes, and a repository holds at most 1000 locks. The LFS store ran on the unhardened client, so every marker, verify and lock call missed the retry and timeout wrapper the rest of the daemon uses, and a stale keep-alive connection could hang a request instead of failing fast. Only the presigner needs the concrete *s3.Client. The presigned-URL lifetime was clamped once at startup, which froze the one-minute floor for the life of the process when the daemon started against a nearly expired credential. It is evaluated per request now, and logs the clamp only on transition. Assisted-by: Claude Opus 5 via Claude Code Signed-off-by: Xe Iaso <xe@tigrisdata.com>
Read canonical pointers during the tree walk and stream verified LFS objects into the image builder. Use a new image version so existing snapshots that contain pointer text are rebuilt. Assisted-by: GPT-6 via Codex Signed-off-by: Xe Iaso <xe@tigrisdata.com>
efirs
approved these changes
Sep 23, 2026
Xe
changed the title
Xe
changed the title
Xe
pushed a commit
that referenced
this pull request
Sep 23, 2026
# [1.10.0](v1.9.0...v1.10.0) (2026-09-23) ### Features * **lfs:** serve Git LFS and resolve EROFS pointers ([#18](#18)) ([6fde39b](6fde39b)) Signed-off-by: Tigris Data <social@tigrisdata.com>
|
🎉 This PR is included in version 1.10.0 🎉 The release is available on GitHub release Your semantic-release bot 📦🚀 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Details
Resolve canonical LFS pointers while building EROFS images, check repository membership before reading shared blobs, and write version 2 images so prior snapshots are rebuilt.
The LFS handlers also bound request bodies and distinguish missing objects from bucket faults.
Test plan
go build ./...npm run formatgo test ./...