Sync → Prevent concurrent syncs from silently deleting files#106
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Resolves #83
Problem
PUTat the end of every sync, so when two devices synced at overlapping times the last writer wonChanges
If-Matchon its ETag, orIf-None-Match: *on a first syncstate.jsondoes not advance, and the next sync re-reads the fresh manifest and reconciles both devices' work with nothing lostWhy
Greptile Summary
This PR fixes the silent-deletion race (#83) by making the manifest upload conditional on the ETag read at the start of each sync pass —
If-Matchfor an existing manifest,If-None-Match: *for a first sync. Losing the race now fails the pass loudly without advancingstate.json, so the next sync re-reads the winner's manifest and reconciles both devices' work with nothing lost.sync.ts:readRemoteManifestnow returns the manifest's ETag and refuses to proceed if none is present;syncOncethreads the ETag through to a conditionalputObjectcall and surfaces a 412 as an explicit "sync again" error.storage.ts/errors.ts:GetResultgains anetagfield,PutConditionandconditionHeadersare added, and 412 is mapped to the new"conflict"status.fake.ts: Fake storage grows a revision-based ETag map that enforcesifAbsent/ifMatchwith real S3 semantics, including cleaning up etags on delete.Confidence Score: 5/5
Safe to merge — the change is a well-scoped optimistic concurrency fix with no regressions to the happy path and comprehensive test coverage at both the unit and integration level.
The conditional-write logic in
syncOnceis correct: it reads the ETag once, threads it through asIf-Match(orIf-None-Match: *on a first sync), and treats 412 as a clean retry signal without advancing state. The fake storage faithfully models real S3 semantics, the no-etag guard closes the unsafe-fallback path, and the two new race-condition unit tests plus the live-bucket interleaved test leave no obvious untested paths in the changed code.No files require special attention.
Important Files Changed
etagtoGetResult,PutConditiontype, andconditionHeadershelper; passes headers tos3PutObjectand surfaces the ETag from GET responses. Implementation is correct.ifAbsent/ifMatchconditions correctly, and cleans up the etag map ondeleteObject— matching real S3 semantics in all tested paths.racingStoragewrapper correctly triggers B's full pass during A's manifest write, and the test verifies that both devices converge with no files lost after A retries.ifAbsentreject-on-existing, andifMatchstale-reject; the fixed-keyifMatchtest is now wrapped intry/finallyfor hermeticity.Comments Outside Diff (1)
src/sync/fake.ts, line 111-113 (link)deleteObjectleaves stale etag in the mapstore.delete(key)removes the content, butetagsstill holds the old revision for that key. If a future test doesdeleteObjectthen a conditionalputObjectwith{ kind: "ifMatch", etag: oldEtag }, the fake will succeed (old etag still matches), whereas real S3/R2 returns 412 because the object no longer exists. This diverges from the real behaviour the fake is meant to model, and would silently mask any bug in that path. Addingetags.delete(key)alongsidestore.delete(key)would close the gap.Reviews (2): Last reviewed commit: "Address comments" | Re-trigger Greptile