Skip to content

spec.md

Latest commit

 

History

History
962 lines (792 loc) · 42.9 KB

spec.md

File metadata and controls

962 lines (792 loc) · 42.9 KB
Raw

DeltaGit product specification

Hard prerequisite: no DeltaGit product implementation, benchmark campaign, storage schema, runtime, adapter, or Git exporter may begin until gate DG-L0 establishes a terminal, reusable LayerFS maturity baseline. DeltaGit is currently a requirements consumer for LayerFS, not an active development lane.

Status: PLANNED — IMPLEMENTATION BLOCKED BY LAYERFS MATURITY

Version: DeltaGitSpecV1-draft

Date: 2026-08-23

Normative terms MUST, MUST NOT, SHOULD, and MAY have their usual requirements meanings. A requirement marked Planned has no implementation or acceptance authority yet.

This specification and architecture.md are the current DeltaGit planning authorities. Earlier files under docs/ are exploratory history and are superseded wherever they conflict with these documents.

1. Product contract

1.1 Purpose

one logical tool operation
        |
        v
one isolated mutable view
        |
        v
one sealed ChangedSet
        |
        v
one immutable LayerFS checkpoint
        |
        +--> rollback / fork / inspect locally
        |
        `--> promote selected state to ordinary Git

DeltaGit records work in motion between Git commits. It adds operation-level checkpoints, cheap local forks, logical rollback, and provenance around mature LayerFS roots while retaining Git as the compatibility and collaboration plane.

1.2 Product positioning

Plane Authoritative for Not authoritative for
LayerFS Canonical file state, immutable roots, CAS/CDC/COW identity, range resolution, durable root publication Tool, actor, session, Git branch, or product policy
DeltaGit Operations, sessions, mutation journals, checkpoints, local refs, rollback/fork, provenance, promotion policy Canonical object bytes, chunk identity, Git merge semantics
Git Blobs, trees, commits, refs, tags, merge/rebase/cherry-pick, fetch/push, remotes Per-tool pre-commit workspace history
Workspace backend Process-visible projection and mutation observation Canonical identity or operation history
Agent/tool adapter Provider input translation and correlation metadata Operation completion, filesystem truth, checkpoint durability 
flowchart TD
    T[Developer or tool] --> R[DeltaGit operation runtime]
    R --> J[Mutation journal and ChangedSet]
    J --> C[DeltaGit checkpoint coordinator]
    C --> L[Mature LayerFS]
    L --> V[Immutable RootId]
    V --> P[SDK / virtual / native projection]
    V --> E[Git exporter]
    E --> G[Ordinary Git commit and ref]
Loading

1.3 Non-goals

DeltaGit V1 MUST NOT:

Non-goal Delegate or later owner
Implement LayerFS CAS, CDC, COW, canonical extents, or root identity LayerFS
Put actor, agent, command, prompt, session, or Git data into LayerFS identity Never
Replace Git merge, rebase, cherry-pick, tags, remotes, packs, or hosting protocols Git
Guarantee process/container isolation Sandbox/runtime product
Parse or rewrite arbitrary shell command strings to remap paths Virtual workspace backend
Treat OS watcher events as complete truth without a completeness proof Mutation-aware backend or scan fallback
Make physical native rollback or cold export constant-time Projection-specific operation
Provide automatic semantic conflict resolution Later collaboration layer
Start as an agent-specific product Adapters remain optional and thin
Add remote synchronization of operation history Planned after local terminal acceptance

2. LayerFS maturity prerequisite

2.1 DG-L0 admission gate

DeltaGit implementation MAY start only after every row is independently evidenced against the same reusable LayerFS product bytes.

ID Required LayerFS capability Admission evidence Current status
L0.1 Terminal Phase-4/G5/G6 correctness and performance disposition Sealed manifests, raw evidence, independent audit Blocked / evolving
L0.2 Arbitrary insert/delete/replace and bounded multi-splice G6 expected-local ordinary path plus honest suffix fallback Planned
L0.3 Detached child-root commit, direct multi-root read, and root pinning Two sessions commit children of one parent and read both without moving a singleton head Planned
L0.4 Range resolver 1/10/100-MiB arbitrary-range correctness and bounded work Planned
L0.5 Deterministic detached-child prepare/commit and reconciliation Requested root/delta known before dispatch; one root-durability COMMIT; exact read-back Durability machinery partial; detached semantics planned
L0.6 Verified default and explicit trusted-local policy Threat boundary, no identity-check removal, verified-after-trusted scrub G5 evidence pending terminal closure
L0.7 Product SDK/VFS boundary Benchmark and consumer call the same core/engine implementation Planned
L0.8 Long-history stability 1/10/100/1,000 revisions; direct root lookup; bounded Q/RSS/storage Planned
L0.9 Reachability and retention Root pins, in-flight roots, retained roots, safe reclamation contract Planned
L0.10 Projection separation Canonical durable, virtual visible, native durable, native complete, cold export endpoints Planned
L0.11 Portable logical format No SQLite/APFS/FUSE/SDK/product metadata in canonical identity Architecturally required; terminal audit pending
L0.12 Exact resource and error contracts Checked caps, exact errors, cleanup, terminal ownership zeroes Evidence incomplete

Failure of any L0.* item keeps DeltaGit at design-only status.

2.2 Required LayerFS semantic surface

Exact names are not frozen. DeltaGit requires semantic equivalents of:

open_store(policy)
load_root(root_id)
pin_root(root_id) -> RootPin
read_path(root_id, path)
read_range(root_id, path, range)
prepare_detached_child(parent_root, changes) -> PreparedChild
commit_prepared_child(prepared_child) -> RootDurabilityOutcome
diff_roots(parent_root, child_root)
verify_root(root_id)
materialize_exact(root_id, destination)
observe_storage_and_resources()
release_pin(root_id)

Preparation deterministically returns the requested root, LayerDeltaId, and a bounded/restartable prepared handle without making the root durable. Commit makes immutable objects, transition, and child root durable but moves no singleton product-visible head. DeltaGit's SessionHead is the sole product-visible selection pointer. A future admitted joint transaction MAY combine root durability and head publication, but the V1 contract assumes and measures one LayerFS durability transaction followed by one recoverable DeltaGit metadata/head transaction.

The surface MUST NOT expose SQLite rows, canonical byte construction, APFS extent details, benchmark fixture IDs, or G5/G6 experiment types to DeltaGit.

3. Terminology and identities

Term Meaning Authority
RootId Content-derived identity of one immutable filesystem state LayerFS
LayerDeltaId Authenticated parent-to-child filesystem transition LayerFS
SessionId DeltaGit-local mutable line of work DeltaGit
SessionHead Durable DeltaGit ref to a checkpoint/root DeltaGit
OperationId Stable identity of one logical tool execution DeltaGit
OperationRecord Lifecycle, execution result, provenance reference, and checkpoint binding DeltaGit
MutationJournal Ordered observation log produced while one operation runs DeltaGit/workspace backend
ChangedSet Canonicalized final net filesystem intent for one operation DeltaGit
CheckpointId DeltaGit identity binding an operation and immutable resulting root DeltaGit
Projection Process-visible view of a selected immutable root Workspace backend
Logical rollback Durable SessionHead switch to an existing checkpoint/root DeltaGit
Physical reconciliation Make a virtual/native workspace represent the selected root Projection backend
Promotion Export a selected LayerFS root as ordinary Git objects and update a Git ref DeltaGit + Git

3.1 Identity separation

LayerFS RootId preimage
    = canonical filesystem state only

DeltaGit CheckpointId preimage
    = versioned DeltaGit record binding OperationId + parent checkpoint
      + resulting RootId + normalized ChangedSet digest

Git commit ID preimage
    = Git tree + parents + Git commit metadata

Two histories producing identical files MUST share the same RootId and MAY have different CheckpointId and Git commit IDs.

The following MUST remain outside every LayerFS canonical identity:

OperationId  SessionId  actor/tool/provider IDs  command/environment
timestamps   messages   Git refs/commits          sandbox/task IDs

4. Normative records

These are semantic records, not a frozen database or wire schema.

4.1 Operation record

Field Required Rule
operation_id Yes Stable before child process starts
session_id Yes Names the owning DeltaGit session
expected_checkpoint Yes CAS guard against lost update
base_root Yes Immutable root used to create the view
state Yes Closed vocabulary in section 6
execution_outcome Yes after drain Exit/signal/timeout/cancel, separate from capture outcome
checkpoint_id Conditional Present after a successful seal, including no-change operation record
result_root Conditional Equal to base root for no-change; new root for state change
journal_receipt Conditional Completeness/custody result, not filesystem identity
checkpoint_intent_id Conditional Present from pre-root-commit intent through final reconciliation
provenance_ref Optional Points to opaque product metadata
created_at / sealed_at Yes Observability only; not LayerFS identity

Provider-specific tool-call, task, actor, or session fields MUST live in an optional versioned provenance envelope. Adapters MAY populate it; DeltaGit correctness MUST NOT require it.

The exact command is retained only in a prospectively size-capped private operation record with restrictive local permissions and default report redaction. DeltaGit MUST NOT retain full environment values by default; it stores only admitted non-secret metadata and a digest unless a separately qualified private/encrypted policy says otherwise. Oversize command, environment, or provenance input fails before child launch. Runtime-generated stdout/stderr is streamed through prospectively bounded buffers with explicit truncation/backpressure/cancellation or failure status; it cannot be preflighted and MUST NOT grow unbounded. Recovery MUST NOT rerun the command.

4.2 Checkpoint intent

After deterministic construction and before LayerFS COMMIT dispatch, DeltaGit durably records:

CheckpointIntent {
    operation_id
    session_id
    expected_checkpoint
    expected_session_head
    parent_root
    requested_root
    requested_layer_delta
    changed_set_digest
    prepared_child_receipt
    state: PreparedForRootCommit | RootCommitDispatched
}

This write-ahead intent supplies the exact requested identity for lost-ACK and crash reconciliation. A durable intent does not make the root or checkpoint visible. No-change operations need no LayerFS intent because preparation proves the existing root; their final metadata transaction creates the checkpoint and moves SessionHead directly.

4.3 Checkpoint record

CheckpointRecord {
    checkpoint_id
    session_id
    operation_id
    parent_checkpoint
    parent_root
    result_root
    layer_delta_id | NoChange
    changed_set_digest
    publication_outcome
}

Required relations:

parent_checkpoint.root == parent_root
state-changing: parent_root != result_root
no-change:       parent_root == result_root and layer_delta_id == NoChange
session-head CAS expected == parent_checkpoint

4.4 Session record

SessionRecord {
    session_id
    base_checkpoint
    head_checkpoint
    state
    projection_target
    projection_observation
}

A session ref names a checkpoint; it is not a workspace copy.

5. Functional requirements

5.1 Sessions and views

ID Requirement Status
DG-S01 Creating a session MUST durably point at an admitted checkpoint/root without copying unchanged content. Planned
DG-S02 A session MUST pin its base/head roots while operations or projections can read them. Planned
DG-S03 V1 MUST permit at most one state-changing operation per session at a time. Planned
DG-S04 Independent sessions MAY construct changes concurrently; publication follows LayerFS writer rules. Planned
DG-S05 Every process-visible view MUST report the exact root it represents. Planned
DG-S06 A view MUST be classified virtual, native_exact, native_latest, or copy_fallback. Planned
DG-S07 Relative-path isolation by changed working directory MUST NOT be claimed as absolute-path isolation. Planned

5.2 Operation lifecycle

ID Requirement Status
DG-O01 The operation record MUST be durable before the child starts. Planned
DG-O02 The original command MUST be stored as data and launched without reconstructed shell quoting. Planned
DG-O03 Tool-return, polling timeout, and operation completion MUST remain distinct. Planned
DG-O04 Completion authority is drained process tree plus workspace quiescence, not a provider post-tool callback. Planned
DG-O05 Background descendants MUST keep the operation open or produce Escaped; they MUST NOT silently mutate after seal. Planned
DG-O06 A non-zero command exit MUST NOT imply no changes; capture policy and execution result are independent. Planned
DG-O07 Read-only/no-change operations MAY create operation history but MUST NOT create a new LayerFS root. Planned
DG-O08 One logical tool operation produces at most one state-changing detached LayerFS root commit. Planned
DG-O09 write_stdin/polling continuation MUST address the existing operation, never allocate a second one. Planned
DG-O10 Cancellation MUST terminate/drain the owned process tree before capture or discard policy executes. Planned
DG-O11 Execution outcome and filesystem disposition MUST be separate fields; cancellation/timeout MUST NOT imply discard. Planned

5.3 Mutation observation

ID Requirement Status
DG-J01 Journal events MUST be ordered, operation-scoped, path-contained, and sequence-checked. Planned
DG-J02 Dirty byte ranges are optimization hints; LayerFS canonical identities remain authoritative. Planned
DG-J03 A mutation-aware backend MUST issue a completeness receipt or fail closed. Planned
DG-J04 Watcher-only/native mode MUST use an immutable-snapshot/proven-writer-exclusion exact scan or label the operation incomplete. Planned
DG-J05 Bypassed or escaped writes MUST never be omitted from a claimed complete checkpoint. Planned
DG-J06 Journal parsing and normalization MUST be streaming/bounded. Planned
DG-J07 Unknown events, malformed paths, gaps, or overflow revoke journal completeness; sealing then requires independent frozen exact reconciliation or fails closed. Planned

5.4 Checkpoint publication

ID Requirement Status
DG-C01 Checkpoint input is the pinned base_root plus one normalized final ChangedSet; deterministic preparation returns requested root/delta before dispatch. Planned
DG-C02 Before a state-changing LayerFS COMMIT, DeltaGit MUST durably persist a CheckpointIntent binding the exact requested root/delta and expected head. Planned
DG-C03 Projection MUST issue zero LayerFS writer transactions and zero LayerFS root-durability COMMITs. Planned
DG-C04 One DeltaGit metadata transaction MUST atomically bind the checkpoint and compare-and-swap SessionHead; this is the sole product-visible selection transition. Planned
DG-C05 Repeating an already sealed OperationId MUST return its exact prior disposition, not create another checkpoint. Planned
DG-C06 A stale session head MUST fail exactly; implicit rebase/retry is forbidden. Planned
DG-C07 Checkpoint acknowledgement MUST distinguish canonical durability from projection visibility. Planned
DG-C08 Root durability and SessionHead publication MUST reconcile independently before another decision in their authority domain. Planned
DG-C09 A durable child root whose SessionHead CAS fails remains unreachable/recoverable and MUST NOT become visible to that session. Planned
DG-C10 State-changing checkpoint durability uses one intent transaction, one LayerFS root transaction, and one DeltaGit checkpoint/SessionHead transaction unless a later admitted joint transaction proves equivalence. Planned
DG-C11 No-change uses zero LayerFS/intent transactions and one DeltaGit transaction advancing SessionHead to a new checkpoint bound to the same root. Planned

5.5 Rollback and fork

ID Requirement Status
DG-R01 Logical rollback MUST be a durable CAS update of SessionHead to an existing checkpoint. Planned
DG-R02 Logical rollback MUST write zero payload objects and perform zero native projection bytes in its timer. Planned
DG-R03 Rollback MUST NOT delete the abandoned future; retention/GC decides later reclamation. Planned
DG-R04 Fork MUST create a new SessionHead at an existing checkpoint/root without copying unchanged content. Planned
DG-R05 Advancing one fork MUST not change the root observed by another fork. Planned
DG-R06 Logical acknowledgement, virtual visibility, native durability, and full verification MUST be separate observations. Planned
DG-R07 Rollback freshness MUST be reported honestly; no external monotonic authority means rollback protection is unavailable. Planned

5.6 Git promotion

ID Requirement Status
DG-G01 Promotion MUST export one explicitly selected immutable LayerFS root. Planned
DG-G02 File content MUST stream to Git blobs; a temporary full directory MUST NOT be required. Planned
DG-G03 Unchanged LayerFS file/directory identities SHOULD reuse verified LayerFS-to-Git object mappings. Planned
DG-G04 Promotion MUST preserve every property in the admitted Git-export subset: path, regular-file content, executable bit, and tree ordering. Planned
DG-G05 The Git commit and branch update MUST use ordinary Git objects and compare-and-swap ref update. Planned
DG-G06 Conversion occurs at explicit commit/publish, before git push; push remains ordinary Git. Planned
DG-G07 Default promotion MAY squash many operations into one Git commit. Planned
DG-G08 Operation-preserving Git commits/notes/custom refs are optional and MUST NOT block compatibility. Deferred
DG-G09 Merge, rebase, cherry-pick, diff, fetch, push, tags, and remotes MUST delegate to Git. Planned boundary
DG-G10 A Git export failure MUST leave the selected DeltaGit checkpoint intact and must not advance the Git ref. Planned
DG-G11 V1 lossless export admits regular files, directories with representable descendants, frozen executable-bit mapping, and the admitted Git path set only. Planned
DG-G12 Empty directories, symlinks until LayerFS admits them, and other unrepresentable state MUST return typed UnsupportedGitExport; silent omission and Git-ref update are forbidden. Planned

6. State machines

6.1 Operation state

stateDiagram-v2
    [*] --> Prepared: durable operation intent
    Prepared --> Running: child started
    Running --> Draining: primary child exited / cancel requested
    Draining --> Quiescing: owned descendants drained
    Quiescing --> Sealing: journal complete
    Sealing --> Sealed: operation + checkpoint bound
    Sealing --> NoChange: same root; operation retained
    Sealing --> Discarded: private view discarded by policy
    Running --> Escaped: unowned writer may remain
    Escaped --> RecoveryRequired: ownership/completeness unresolved
    Prepared --> Failed: launch failed
    Quiescing --> Failed: journal/capture failed
    Sealing --> Reconciling: publication/session outcome uncertain
    Reconciling --> Sealed: requested checkpoint visible
    Reconciling --> RecoveryRequired: prior/different/ambiguous preserved
    Failed --> RecoveryRequired: retained evidence is recoverable
    RecoveryRequired --> Sealing: ownership and frozen-view authority re-established
    RecoveryRequired --> Abandoned: cannot prove safe completion
Loading

Execution outcome and filesystem disposition are orthogonal:

ExecutionOutcome Meaning
Success / NonZero Primary process exited normally
Signaled / TimedOut / Cancelled Process tree was terminated and drained
LaunchFailed No child was admitted
FilesystemDisposition Meaning
Sealed Exact checkpoint bound and head published
NoChange Operation retained; RootId unchanged; SessionHead advances to a new checkpoint bound to that root
Discarded Private final view intentionally not checkpointed
Failed No claimed checkpoint; evidence retained
IncompleteEscaped Writer lifetime or view completeness unproved; sealing prohibited
RecoveryRequired Nonterminal evidence requires exact reconciliation
Abandoned Terminal manual disposition; no checkpoint claim

Any execution outcome may pair with a valid filesystem disposition. In particular, a cancelled or non-zero operation may still be Sealed after the normal journal, root-durability, and SessionHead reconciliation path.

6.2 Session state

             +---------------- rollback/fork ----------------+
             |                                                |
             v                                                |
Clean(head=H0) -> Operation(H0) -> Checkpointing -> Clean(head=H1)
      |                  |               |
      |                  |               +--> Reconciling
      |                  +--> Escaped/RecoveryRequired
      +--> ProjectionPending ----------------> ProjectionCurrent

Session state invariants:

at most one state-changing operation per session
head_checkpoint always resolves to a durable LayerFS root
projection_target may lead or equal projected_root
projected_root never implies session head unless exact IDs agree

6.3 Projection state

stateDiagram-v2
    [*] --> Idle
    Idle --> Building: target root requested
    Building --> Current: exact root atomically published
    Building --> Building: latest pending target replaces older pending
    Building --> Failed: private output discarded
    Current --> Building: newer target requested
    Failed --> Building: explicit retry/new target
Loading

The first admissible scheduler is one in-flight projection plus one replaceable latest pending root. Every started, cancelled, superseded, failed, and published build MUST be counted.

7. Mutation journal and ChangedSet

7.1 Event vocabulary

Event Required data Net intent
Create canonical path, kind Path did not exist at base
WriteRange path, old/new range hint Content may differ
Truncate path, new length Content length differs
Delete path, observed kind Tombstone relative to base
Rename old path, new path Atomic namespace intent
Metadata path, changed fields Metadata-only candidate
ReplaceKind path, old kind, new kind File/directory/symlink type change

Events describe observations. The sealed ChangedSet describes final net state.

7.2 Normalization

ordered journal
    |
    +--> validate sequence, operation, containment, overflow
    +--> resolve rename chains
    +--> coalesce overlapping/adjacent dirty ranges
    +--> collapse repeated writes and metadata updates
    +--> compare final path state with pinned base root
    +--> remove zero-effect paths
    `--> stable canonical path order
            |
            v
        ChangedSet

Required normalizations:

Event sequence Final net effect
create → writes → delete No effect if path absent at base
modify → modify One final replace with coalesced hints
delete → create Replace or type change against base
rename A→B → rename B→C One A→C intent after collision checks
rename → modify destination Rename plus final destination content
metadata → restore original metadata No effect
path changed but final identity equals base No effect

Normalization MUST NOT infer correctness from event hints alone. The final captured state and LayerFS identity checks decide the root.

Journal gaps, unknown records, and overflow invalidate journal completeness. They do not become complete through a quiet interval. A seal after such a failure is permitted only when an independently frozen reconciliation proves the exact final view; otherwise the operation remains incomplete.

7.3 Completeness classes

Class Meaning May seal?
MutationAwareComplete All mutations traversed an authoritative workspace boundary Yes
StructuredToolComplete Operation used only proven structured file APIs Yes, for admitted scope
NativeScanReconciled Events were hints and an immutable snapshot or proven writer exclusion was scanned exactly Yes
IncompleteEscapedWriter Writer may outlive operation or bypass view No
IncompleteJournalGap Sequence/custody gap No
UnsupportedObject Unrepresentable path/type/metadata No

NativeScanReconciled retains a receipt binding:

operation_id
view_or_snapshot_identity
base_root
scan boundary and start/end
writer-exclusion or immutable-snapshot proof
journal sequence/custody status
final root/digest

Quiescence timing alone is never writer-exclusion authority.

8. Checkpoint, rollback, and fork semantics

8.1 Per-tool checkpoint

tool call
  +-- many writes
  +-- many paths
  +-- child processes
  `-- temporary create/rename/delete
        |
        v
one final ChangedSet
        |
        v
prepare requested RootId / LayerDeltaId
        |
        v
durable CheckpointIntent
        |
        v
one detached LayerFS root-durability COMMIT
        |
        v
one DeltaGit metadata transaction: checkpoint binding + SessionHead CAS

The checkpoint boundary is the logical operation, not each syscall. Read-only and net-zero operations retain operation history without manufacturing roots.

Expected ordinary cost model after LayerFS G6 maturity:

checkpoint work =
    O(journal bytes + changed paths)
  + O(changed content + local CDC resynchronization)
  + O(changed path/file tree paths)
  + one DeltaGit CheckpointIntent durability transaction
  + one LayerFS detached-root durability transaction
  + one DeltaGit checkpoint/SessionHead visibility transaction

These are three honest durability boundaries, not three canonical publications. SessionHead is the only DeltaGit-visible selection authority. If its CAS fails, the child root remains durable but unreachable until recovery or reclamation. The write-ahead CheckpointIntent closes crashes and lost acknowledgements between the transactions. A future admitted joint transaction may combine them, but V1 does not assume it.

For a no-change operation, preparation returns the base root: skip intent and LayerFS COMMIT, create a new CheckpointId, and atomically move SessionHead C0/R0 -> C1/R0 in the single DeltaGit metadata transaction.

Honest exceptions:

missing mutation authority -> frozen exact reconciliation or reject
adversarial CDC non-rejoin  -> LayerFS suffix fallback
cold native projection      -> output-relative work
full verification           -> reachable-state work

8.2 Logical rollback

flowchart LR
    H3[Head C3 / Root R3] -->|CAS rollback| H1[Head C1 / Root R1]
    H3 -. retained .-> F[Future C2/C3]
    H1 --> V[Virtual view selects R1]
    H1 --> N[Optional native reconciliation]
Loading

Timing boundaries MUST be separate:

Boundary Includes Excludes
logical_ack Validate target, durable SessionHead CAS Projection, full verification
virtual_visible Root installed in resolver/view Native file publication
native_durable Native file/dir sync and atomic publish Full independent scan
verification_complete Selected verification policy Earlier ACK timers

8.3 Fork

checkpoint C10 / root R10
       |
       +--> session A head C10 --op--> C11 / R11
       |
       `--> session B head C10 --op--> C12 / R12

unchanged objects/subtrees: shared
session refs and changed objects: distinct

Fork storage MUST initially consist only of DeltaGit metadata and root pins.

9. Git compatibility contract

9.1 Promotion pipeline

flowchart LR
    O[DeltaGit operations] --> S[Selected checkpoint]
    S --> R[LayerFS root]
    R --> B[Git blob stream]
    B --> T[Git trees]
    T --> C[git commit-tree]
    C --> U[git update-ref CAS]
    U --> P[ordinary git push]
Loading

Initial implementation SHOULD use ordinary Git plumbing before adding any custom Git object writer:

git hash-object --stdin -w
git mktree
git commit-tree
git update-ref <ref> <new> <expected-old>

git fast-import MAY replace repeated process startup only after profiling.

9.2 Lossless V1 export subset

LayerFS state V1 Git export
Regular file content Lossless Git blob
Executable bit under frozen mapping Lossless Git mode
Directory with representable descendants Lossless Git tree
Empty directory UnsupportedGitExport
Symlink before LayerFS admits/qualifies it UnsupportedGitExport
Hard link, device, socket, FIFO, xattr, ACL, unsupported mode/path UnsupportedGitExport

Failure is typed, the selected checkpoint remains intact, and the Git ref does not move. A later explicitly lossy policy may choose markers or omission, but it is outside V1 and may never be mislabeled lossless.

9.3 Delegation matrix

Operation Owner
Per-tool checkpoint, local rollback/fork DeltaGit
Root/file/range content LayerFS
Export selected state to blobs/trees/commit DeltaGit using Git plumbing
status, diff, show, log, grep Git against a compatible view
merge, rebase, cherry-pick, apply Git, executed inside a DeltaGit operation when worktree-mutating
fetch, push, remote negotiation, tags Git
Optional operation provenance in notes/trailers Later DeltaGit feature

10. Failure, crash, and reconciliation

10.1 Failure boundaries

Failure point Required result
Before child starts Durable Failed; no workspace claim
Child exits non-zero Record exit; still seal exact final changes unless policy discards
Child/descendant escapes Escaped; no complete checkpoint claim
Journal gap/malformed event Preserve evidence; reject seal or independently frozen exact reconciliation
Prepared child but no durable intent Root COMMIT MUST NOT dispatch; operation remains recoverable/incomplete
Durable intent before root dispatch Reconcile exact requested root as absent, then dispatch once or abandon
LayerFS fails before detached-root durability Prior root/session head remains visible
LayerFS root COMMIT acknowledgement lost Read back the intent-bound requested root/delta exactly
Root durable, DeltaGit head not updated Prepared intent enables recovery; child root remains unreachable/pinned
DeltaGit head acknowledgement lost Read back exact SessionHead and operation binding
Projection fails Canonical checkpoint remains valid; private output discarded
Git object export fails DeltaGit checkpoint intact; Git ref unchanged
Git ref CAS fails Exported objects may remain unreachable; no silent overwrite

10.2 Reconciliation outcome vocabulary

LayerFS detached-root durability:

RequestedRootDurable
RequestedRootAbsent
ConflictingRootRecord
AmbiguousRootDurability

DeltaGit SessionHead, native projection, and Git-ref publication use their own pointer authority:

RequestedVisible
PriorVisible
DifferentVisible
Ambiguous

Each uncertain LayerFS root-durability COMMIT, DeltaGit SessionHead publication, native publication, and Git ref update MUST reconcile in its own authority domain. Wall time, filesystem residue, or a repeated write MUST NOT substitute for read-back authority.

10.3 Recovery

startup recovery
    |
    +--> scan nonterminal OperationRecords
    +--> load CheckpointIntent and exact requested root/delta when present
    +--> prove process-tree state or mark unknown
    +--> verify journal custody/completeness
    +--> reconcile LayerFS detached-root durability
    +--> reconcile SessionHead CAS
    +--> retain/discard private projections by exact root
    `--> emit terminal disposition or RecoveryRequired

Recovery MUST be idempotent and MUST NOT rerun the original command. After root COMMIT dispatch, recovery MUST NOT infer the requested root from the parent alone or choose among sibling roots. It reconciles only the exact requested_root bound by the durable intent. If no such intent exists, no state-changing SessionHead publication is allowed.

11. Resource, storage, and performance contracts

11.1 Bounded resources

Resource Normative bound
Journal writer memory Bounded append buffer; never all events
Journal normalization memory O(changed paths + bounded range batches)
Content capture memory LayerFS bounded streaming buffer; never whole large file by default
Operation concurrency One state-changing operation per session in V1
Projection queue One in-flight + one replaceable latest pending
Processes One owned process group/Job Object per operation
Descriptors/temps Checked caps; exact terminal cleanup
Storage New unique content + changed canonical nodes + DeltaGit metadata
History traversal Direct checkpoint/root lookup; never replay all earlier operations to read current state

Exact numeric caps MUST be prospectively frozen before implementation. No unbounded queue, map, journal, dirty-range set, output capture, or provenance blob is admissible.

11.2 Storage equation

For one checkpoint:

DeltaStorage =
    new_unique_LayerFS_payload_bytes
  + new_changed_canonical_tree_bytes
  + LayerFS_transition/root_bytes
  + DeltaGit_intent/operation/checkpoint/ref_bytes
  + optional_projection_delta
  - reclaimed_unreachable_bytes   # only by separately safe GC

Prohibited hidden terms:

complete workspace copy per checkpoint
complete flat manifest rewrite per checkpoint
duplicate unchanged payload
unbounded permanent operation overlay chain
native projection charged as canonical checkpoint storage

11.3 Performance claim boundaries

Operation Target asymptotic behavior Qualification
Journal discovery Changed events/paths Mutation recording itself still costs event bytes
Ordinary state-changing checkpoint Changed paths/content + local resync + changed tree paths Intent + root durability + head transactions timed separately; fallback reported
No-change checkpoint One DeltaGit checkpoint/SessionHead transaction Zero intent and LayerFS root transactions
Logical rollback/fork Durable metadata/ref update Projection excluded
Virtual range read Tree height + intersecting extents + returned bytes Cache class reported
Native incremental projection Route-specific delta/suffix work Platform capability dependent
Cold native export Full output bytes Never called delta-only
Git promotion Selected tree walk + new Git objects Cache reuse must be authenticated

No latency number is a gate until prospectively frozen after DG-L0. Existing experimental numbers are diagnostic only and MUST NOT be presented as product performance.

12. Security, trust, and portability

12.1 Security boundary

Requirement Rule
Path containment Reject traversal, root escape, malformed canonical path, and unsafe symlink following
Process authority DeltaGit runtime is not a container/security sandbox
Command handling Preserve exact command as data; do not construct a new shell string from parsed pieces
Command custody Private size-capped record, restrictive permissions, default redaction; oversize fails before launch
Environment/provenance Do not retain full values by default; cap admitted metadata, store a digest, and exclude secrets from reports
Journal custody Private permissions, operation binding, sequence/length checks, durable seal
Absolute paths Claim transparency only when a qualified virtual/namespace backend enforces it
LayerFS identity Always validate fetched/new/incumbent canonical objects under admitted policy
Trusted local mode Explicit opt-in; Verified remains default; does not protect against rollback/offline replacement/malicious same-UID mutation
Git export Never execute repository content merely to export objects; exact ref CAS required

12.2 Portability tiers

Tier Required behavior Example backends
Core Identical DeltaGit/LayerFS semantics, checkpoint records, rollback/fork Any supported OS
Virtual Root/range view without complete native reconstruction SDK, FUSE, FSKit, ProjFS-like adapter
Native accelerated Best safe projection route with capability receipt APFS clone/patch, Linux reflink
Compatibility Exact ordinary-copy/full-export fallback Any filesystem with basic file APIs

Platform accelerators MUST NOT change LayerFS roots, DeltaGit checkpoints, session semantics, durability, or exact errors.

13. Acceptance program

13.1 Ordered gates

DG-L0  LayerFS maturity
   |
   v
DG-1   Operation runtime + custody
   |
   v
DG-2   Journal + ChangedSet
   |
   v
DG-3   Checkpoint + rollback + fork
   |
   v
DG-4   Git promotion compatibility
   |
   v
DG-5   Recovery + security + portability
   |
   v
DG-6   Performance + storage + history
   |
   v
DeltaGit local MVP terminal audit

Each gate starts with a prospective contract, focused correctness tests, a short mechanism screen, and one bounded measured campaign only after signal. Failed attempts remain evidence; thresholds and populations are not changed after observation.

13.2 Functional test matrix

Case 1 MiB 10 MiB 100 MiB Required proof
No-change/read-only operation P S S Operation retained; no new root
Create/modify/delete/rename P P S Exact final root and ChangedSet
Random-position size-preserving edits P P P Expected-local ordinary work
Random insert/delete/shorter/longer P P P G6 route/fallback honest
Multi-file/multi-splice tool operation P P S One checkpoint/publication
Non-zero exit with mutations P S — Exit and checkpoint independent
Background child P S — No premature seal
Cancellation/timeout P S — Process tree drained; exact disposition
Absolute-path bypass P — — Reject/incomplete unless virtual boundary
Frozen native reconciliation P S — Immutable snapshot/writer exclusion receipt; gaps/overflow recovered exactly
Logical rollback/fork P P P Zero content/native work in logical timer
Projection exact/latest P P P Root-correct coalescing
Git promotion P P S Git fsck/tree/content/ref agreement
Unsupported Git state P S — Typed UnsupportedGitExport; checkpoint intact; Git ref unchanged

Legend: P primary, S semantic smoke, — not scheduled.

13.3 History and concurrency matrix

Workload Population Required observations
Serial checkpoints 1/10/100/1,000 Latency drift, direct lookup, storage slope, Q/RSS
Random edit trace Deterministic frozen trace Position/magnitude/shape, oracle root, fallback rate
Independent sessions 1/2/4/8 Detached children from one parent, independent heads, writer queue, aggregate resources
Same-session stale writer 2 contenders One exact winner, one typed stale-head failure
Pinned readers + writer 3 readers + 1 writer Old/new root isolation
Rollback/fork tree depth and breadth frozen No content copy; roots independently readable
Projection storm bounded root sequence One in-flight/one pending conservation
Reopen/recovery every nonterminal boundary Idempotent terminal reconciliation

13.4 Storage acceptance

Gate PASS condition
No-change Zero new LayerFS payload/tree/root bytes
Fork Zero copied payload and canonical tree bytes
Logical rollback Zero new payload/canonical content and zero native bytes in timer
Small checkpoint Growth explained exactly by storage equation
Return to prior content Existing canonical identities reused
1,000 revisions Measured bounded slope; no full-manifest rewrite/history replay
Retention/GC Pinned/retained roots survive; only proven unreachable objects reclaimed

13.5 Terminal audit

DeltaGit local MVP can be called PASS only if an independent audit verifies:

same reusable LayerFS product bytes as DG-L0
no benchmark-only semantic path
detached root durability and SessionHead publication are distinct and reconciled
SessionHead is the sole product-visible selection authority
all operation/session state transitions closed
all ambiguous outcomes reconciled
every dispatched root COMMIT has a prior durable requested-root intent
all old roots and forks exact
Git fsck and exported tree agreement
bounded Q/RSS/journal/descriptors/storage
no hidden full workspace copy in checkpoint/rollback/fork
no agent/provider metadata in LayerFS identity
no DeltaGit implementation of delegated Git algorithms
failed evidence preserved
terminal resources and process ownership zero

14. Current disposition and deferred work

Item Disposition
LayerFS fitness for DeltaGit Strong architectural fit; not yet proven mature
DeltaGit implementation WAIT for DG-L0
Per-tool checkpoint Planned: one operation → one checkpoint
Logical rollback/fork Planned: durable ref operations, projection separate
Storage efficiency Planned: CAS/COW/G6 sharing plus explicit retention/GC
Git compatibility Planned: export at explicit promotion, delegate Git algorithms
Agent adapters Deferred until provider-neutral runtime is accepted
Automatic merge/rebase of operation histories Deferred
Portable operation-history synchronization Deferred
Multi-agent sandbox product Separate future product above DeltaGit/LayerFS

The next authorized action is to finish and audit LayerFS against DG-L0. This specification authorizes no DeltaGit source implementation.