commit-window-refactor.md

Commit Window And Commit Planning

spec — current behaviour. The code depends on this document; change one, change the other. Index: ../INDEX.md

Status: implemented Date: 2026-05-01

This document is the source of truth for the current commit-window and commit planning implementation. It replaces the earlier batching design, planning analysis, and follow-up cleanup notes. Those documents were useful while the refactor was in flight; the implementation has now landed.

Goals

The branch worker turns live watch events and snapshot resyncs into Git commits. The design has three main goals:

• Keep Git history readable during bursty operations such as kubectl apply -k, helm upgrade, ArgoCD sync waves, or GUI-driven edits.
• Treat a write as complete only after the push to the remote succeeds.
• Keep replay stable even if a GitTarget or its encryption Secret changes while work is locally committed but not yet pushed.

The user-facing commit-shaping control is GitTarget.spec.commit.window: an idleTimeout (default 5s) of silence, or a maxDuration (default 1m) after the window opened. An idleTimeout of 0s gives per-event local commits in the normal no-conflict path. Push cadence is intentionally separate and uses a fixed 5 second cooldown in the branch worker.

Use Cases

Burst collapse

A short burst from one logical operation should usually become one commit per author and target, not one commit per Kubernetes object. For example, one kubectl apply -k that creates four ConfigMaps for the same GitTarget and same user becomes one grouped commit once the window's idleTimeout passes with no write.

Honest authorship

A window's commit is authored by its events' author, from author attribution. The operator remains the committer because it physically writes the Git commit. Resync writes are authored by the configured committer because they represent a snapshot of the cluster, not a single human action.

Target isolation

A grouped commit covers exactly one GitTarget. That keeps path selection, bootstrap files, and SOPS encryption recipients unambiguous. Events for different targets split into different commit units even when they arrive in the same burst.

Safe replay

If the remote branch moves before our push lands, unpublished local commits are discarded and rebuilt on top of the new remote tip. Replay uses the metadata resolved when the work entered the pending lifecycle, so a deleted or changed GitTarget does not change how already-pending work is written. The one exception is the prune mode, which is re-read before a replay and only ever tightened. Replay also applies the actions each write decided when it was first committed, rather than planning the cluster's picture again onto the new tip.

Current Flow

flowchart TD
    A[Live watch event or resync] --> B{Kind}
    B -->|live event| C[Open window: same author and target]
    C --> E{Finalize trigger}
    E -->|author or target change| F[PendingWriteCommit]
    E -->|idleTimeout silence| F
    E -->|maxDuration| F
    E -->|byte cap| F
    E -->|attach: Next| F
    E -->|apply due: flush| F
    E -->|Git host push notice| F
    E -->|shutdown| F
    B -->|resync| Q[Finalize open window if present]
    Q -.-> F
    Q -->|then| D[PendingWriteResync]
    F --> G[pendingWrites]
    D --> G
    G --> I[First commit decides each write's actions on its tree]
    I -->|no action left| S[settled: not retained, not pushed]
    I --> J{Push}
    J -->|success| K[clear pendingWrites and advance lastPushAt]
    J -->|remote moved| L[reset to remote tip]
    L --> M[apply the decided actions onto the new tip]
    M --> J
    J -->|transient failure| N[keep local commits and pendingWrites for retry]

Loading

BranchWorker owns the live open window, timers, repository preparation, push cooldowns, and retry scheduling. It retains []PendingWrite until a push succeeds. Local commit creation does not clear pending work. A write whose first commit changes nothing in Git is the exception: it settles at once and is neither retained nor pushed. A save (a write carrying a CommitRequest) is always retained, and the push decides its outcome.

Per-event writes are processed as a stream. The branch worker keeps one open window at a time, and that window contains only one author and one target. The window finalizes on its idle timeout or its maximum duration, and immediately on author change, target change, the byte cap, an attach: Next request, a resync, shutdown, an apply coming due, or a Git host's push notice. An apply coming due closes the window so its edits are published before the apply request is sent (the flush, close_reason="apply_due"). A push notice closes it so its writes are published, and replayed onto the move, at once. Neither applies while a CommitRequest holds the window: no flush is made while a request is registered, and a push notice leaves an attached window to its save. A timer of 0s closes the window in the same step that collected the write. Repeated writes to the same Git path inside the open window are last-write-wins while preserving first-seen path order.

Resyncs and atomic writes are different: they bypass the live-event window, but they do not jump ahead of already-windowed live events. If one arrives while the branch worker has an open event window, the worker finalizes that window into a commit pending write first, then appends its own pending write. This preserves arrival order while keeping a resync or an atomic write as one batch.

Core Types

WriteRequest is the external input contract for Git writes. Callers choose between:

• CommitModePerEvent, used for live watch events that may be windowed. With idleTimeout: 0s, each event finalizes immediately.
• CommitModeAtomic, one commit for all events in the request. No production caller sends one today: a snapshot arrives as a resync (ResyncRequest).

PendingWrite is the durability unit retained until push succeeds:

• PendingWriteCommit represents one finalized commit-shaped event window.
• PendingWriteAtomic represents one caller-defined atomic write.
• PendingWriteResync represents one GitTarget's snapshot, applied by mark-and-sweep.
• PendingWriteRequestRecord records a CommitRequest's message in an empty commit when no window reached the request and it asked for whenNothingToCommit: CommitEmpty.

Pending writes carry resolved target metadata (ResolvedTargetMetadata): target identity, destination path, bootstrap options, encryption configuration, and prune mode. That metadata is the replay contract.

commit_executor.go commits the pending writes in order, at most one local Git commit per write. A write's first commit decides its actions on that tree: a window or atomic write keeps only the events whose plan changed the tree, a resync keeps the upserts that changed it and the documents its sweep removed, and a save keeps every object it captured. A replay (replayPendingWrites) applies those actions onto the newer tip. The executor stages bootstrap templates, configures encryption from the write's metadata, skips a write that changes nothing, renders the message, chooses commit metadata, and creates the local commit.

Window Rules

The branch worker is the only owner of commit grouping for live events. It appends to or finalizes the open window on event arrival.

Rules:

• A new window starts when there is no open window.
• The current window finalizes when the author changes.
• The current window finalizes when the GitTarget changes.
• Same-author, same-target edits of the same path stay in the same window.
• Within a window, the latest event for a path wins.
• First-seen path order is preserved for commit message resource ordering.

This keeps GUI-style repeated edits compact while making commit boundaries visible at the same point where events enter the pending lifecycle. There is no planner-time regrouping pass.

Messages And Authorship

Message precedence is a non-empty literal override, then commit.message.liveTemplate for live windows or commit.message.reconcileTemplate for atomic snapshots and resyncs. Every live window uses the same context, including singleton and 0s windows. A CommitRequest's message is the literal override, and commit.message.requestTemplate can frame it (see commitrequest-design.md). Literal text never changes authorship. Resolved templates and literal overrides travel with pending writes through retries and conflict replay, so editing a target does not change an already finalized write's message configuration.

The live template receives LiveCommitMessageData:

• Author
• GitTarget
• Count
• Operations
• Resources
• RequestMessage

Resources is deduplicated by Git path and rendered in first-seen order.

Push And Replay

The branch worker records the remote root hash for the current push cycle before creating local commits. PushAtomic uses that root hash to detect whether the remote moved.

Push outcomes:

• Success: clear pendingWrites, clear the push-cycle root hash, and advance lastPushAt.
• Transient failure: if the remote tip still matches the recorded root hash, leave local commits and pending writes untouched. The next cooldown-driven push retries the same commits.
• Push error plus failed remote-state fetch: treat it like transient. Preserve state and surface the original push error.
• Conflict: if the remote tip differs, reset onto the latest remote tip, apply the retained writes' decided actions onto it, and retry in the same call.

Replay correctness is defined by final tree correctness and a successful push, not by preserving unpublished local commit objects. If replay finds that the new remote tree already contains the desired content, the write is skipped and no replacement commit is created. A write the new tree refuses is skipped and settled as a refusal, and the writes after it are still replayed.

Operational Controls

commit.window lives on GitTarget.spec.commit, and it used to live on GitProvider.spec.push. The old placement said commit shaping belonged to the branch writer for a provider/branch; that was wrong in a way the object model made visible, because a branch worker serves every GitTarget sharing a (provider, branch) pair and those targets had no way to disagree. Commit shaping describes the FOLDER being written, so it is a GitTarget field.

A branch worker is still per (provider, branch), so the timers are resolved per OPEN WINDOW rather than once per worker, and snapshotted when the window opens: editing a target never moves a deadline already running. That is affordable because a window is already bound to exactly one GitTarget by construction: it finalizes the moment the target changes. An unreadable target, or one declaring no window, takes the 5s and 1m defaults. A CommitRequest that attaches replaces both timers for that window.

The byte cap is an operator startup setting, --branch-buffer-max-size (default 8Mi), because it protects pod memory rather than describing user-facing Git history.

The push cooldown is fixed at 5 seconds. It keeps fast local commits from spamming the remote while still keeping ordinary single-change latency close to idleTimeout + push RTT. A flush skips it: when an apply comes due, the writes up to the flush mark are published at once.

Tests

The implementation is covered at three useful levels:

• Pending-write tests assert metadata derivation, arrival order, one-resolution-per-target encryption lookup, prune-mode tightening, and that a decided write replays as decided.
• Executor tests assert message-kind routing, no-op skipping, and encryption coming from the pending write's metadata.
• Branch-worker tests assert open-window finalize triggers, retained pending writes, transient vs conflict push handling, replay with deleted target metadata, atomic/commit interleave order, and no-op replay behavior.

The e2e smoke test for commit-window batching covers the full pipeline from Kubernetes event to Git commit. The deeper replay and failure-classification cases stay in unit/split-worker tests where they can be made deterministic.

Short History

The original implementation had separate grouped and atomic write paths, plus test-facing compatibility helpers in git.go. That made push failure handling, replay, and encryption resolution harder to reason about. The refactor moved both grouped and atomic writes through retained PendingWrites, a shared planner, and a shared executor. It also removed the old request-writing compatibility path and the transitional test helpers.

No CRD shape changed as part of the refactor beyond the already-landed commit-window and grouped-message fields.