Skip to content

Make first remote uploads exclusive and recover safely - #35

Merged
mikecerisano merged 1 commit into
mainfrom
fix/sftp-first-upload-20261017
Oct 17, 2026
Merged

mikecerisano merged 1 commit into
mainfrom
fix/sftp-first-upload-20261017

Conversation

@mikecerisano

@mikecerisano mikecerisano commented Oct 17, 2026 •

Copy link
Copy Markdown
Collaborator

A first remote backup could fail before sending any file because put -a tried to resume a new UUID temporary path that did not yet exist. The first upload now claims a fresh remote file exclusively, streams the verified local backup into it, and publishes the final name without replacing an existing file. An interrupted upload is never presented as complete.

The Mac provider sends a bundled Python 3 helper over SSH exec. The helper pins parent directory descriptors, refuses symlink leaves, creates the temporary file with exclusive/no-follow flags, reads it back when the chosen verification mode requires that, and publishes with a no-replace hard link. The app rehashes the exact opened local file descriptor against the manifest before sending bytes, bounds streamed input/output and command diagnostics, and keeps cancellation tied to the process-group runner merged in #36. Final upload-only status still says uploaded, not downloaded and checked; the SHA-256 mode downloads and checks remote bytes.

Recovery is deliberately conservative. A saved promotion intent only accepts an existing final when the retained temporary path and final are the same remote inode. A missing, swapped, or same-size unrelated object becomes a conflict or needs review. Interrupted uploads restart under a new UUID on explicit retry; uncertain old temporary paths are left untouched and identified privately to the operator. After a successful promotion, the hidden .bitmatch-upload-UUID name also remains as a hard link to the final file (the same inode, not a second copy of the bytes) for ownership proof and optional manual cleanup. BitMatch does not unlink an object that another writer may have replaced; queue and provider tests assert that pathname-only cleanup is not called.

Supported-server requirements and limits:

  • First uploads require SSH exec, Python 3 with POSIX openat/linkat support, and a remote destination filesystem that supports hard links for exclusive no-replace publication. Shell-less/SFTP-only accounts are unsupported by this path. Preflight checks the helper, but cannot prove a particular destination filesystem supports hard links; unsupported promotion fails closed when attempted.
  • This does not defend against another process with the same account privileges (or root) changing an already-open inode or moving a pinned directory. Upload-only remains unverified remote bytes, and interrupted uploads no longer resume from a partial temporary file.

Local validation of the change:

  • Focused Mac queue/provider/process/view-model suites: 145 passed, 0 failed.
  • Full serialized Mac app suite: TEST SUCCEEDED; xcresult overall 1,191 passed, 14 skipped, 0 failed (its device/configuration view counts 1,238 passed because Xcode counts parameterized cases differently).
  • Disposable localhost sshd with temporary keys, agent, pinned known-hosts, and remote root: helper adversarial probes passed; actual provider upload/readback 1/1 passed. This tests SSH exec on one local POSIX filesystem, not an external NAS or every server type.
  • iPad simulator Debug build: BUILD SUCCEEDED. git diff --check and helper syntax checks passed.

The tested commit was rebased onto newer independent main changes with an identical stable patch ID and an equal git range-diff; the full gates above were not rerun after that patch-identical base refresh. No release or external-server compatibility is claimed here.

@mikecerisano
mikecerisano force-pushed the fix/sftp-first-upload-20261017 branch from 9740848 to 07608f3 Compare October 17, 2026 15:19
@mikecerisano
mikecerisano force-pushed the fix/sftp-first-upload-20261017 branch from 07608f3 to d8ece04 Compare October 17, 2026 15:19
@mikecerisano mikecerisano changed the title Fix first SFTP upload without replacing remote files Make first remote uploads exclusive and recover safely Oct 17, 2026
@mikecerisano
mikecerisano marked this pull request as ready for review October 17, 2026 15:20
@mikecerisano
mikecerisano merged commit 00660ec into main Oct 17, 2026
4 of 8 checks passed
@mikecerisano
mikecerisano deleted the fix/sftp-first-upload-20261017 branch October 17, 2026 15:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant