diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index 9d29ae8f9..a1157836a 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -1,494 +1,404 @@ -# git-stage-batch Architecture - -This document gives a project-level view of how `git-stage-batch` is built. -It is intended for contributors who want to understand where behavior lives, -how data moves through the system, and which modules own which parts of the -workflow. - -For the specialized architecture of named batch storage and batch operations, -see [BATCHES.md](BATCHES.md). - -## What the Program Is - -`git-stage-batch` is a command-line tool for constructing clean Git history -from messy working-tree changes. At its core, it does three things: - -- discovers changes in the working tree and index -- presents those changes as navigable hunks or files -- applies user decisions back to Git state and session state - -Most commands are not long-running daemons. The program persists enough state -on disk to make a multi-step interactive workflow feel continuous across -separate invocations. - -The project is pure Python and leans on Git itself for repository truth. -Instead of reimplementing repository storage, it shells out to `git`, parses -its output, and stores tool-specific metadata alongside that workflow. - -## High-Level Design - -The codebase is organized as a set of layers: - -- `cli/` - Argument parsing, help, completion, and command-specific dispatch helpers. -- `runtime.py` - Top-level mode dispatch between parsed CLI arguments and the interactive TUI. -- `commands/` - User-facing command implementations. -- `data/` - Session-persistent state, selected-change caches, progress tracking, - undo/redo, page-aware file review safety state, and other workflow - bookkeeping. -- `core/` - Neutral models and parsing logic for diffs, hunks, hashes, and line - selections. -- `staging/` - Helpers that build exact target file content for selective index or working - tree updates. -- `batch/` - The advanced named-batch subsystem: ownership, storage, merge, attribution, - source refresh, and batch-specific selection logic. -- `output/` - Rendering of hunks, page-aware file reviews, multi-file review lists, - patches, and colors for terminal display. -- `tui/` - The interactive single-key interface built on top of the command layer. -- `utils/` - Git subprocess wrappers, file I/O helpers, path layout, journaling, and - text/process utilities. - -That split is intentional: - -- `core/` tries to describe changes -- `commands/` decide what to do about them -- `data/` remembers where the workflow is -- `batch/` adds a second persistence model for deferred changes -- `utils/` hides subprocess and filesystem details from the rest of the code - -## The Main Execution Flow - -The usual entry path is: - -1. `cli.argument_parser` builds the command-line interface. -2. `runtime.dispatch_cli_mode` chooses interactive mode or noninteractive - execution. -3. `cli.execution` invokes the parsed command function. -4. A function in `commands/` performs the operation. -5. That command uses `data/`, `core/`, `staging/`, `batch/`, and `utils/` - modules as needed. - -For a typical non-batch staging workflow: - -1. `start` initializes session state and discovers the first pending change. -2. `show` or the cached hunk state renders the current change. -3. `include`, `skip`, or `discard` records a decision. -4. Navigation advances to the next pending change. -5. `again` clears per-pass progress while preserving broader session context. -6. `stop` or `abort` ends the session. - -The program therefore behaves like a state machine spread across CLI -invocations. The on-disk state in `data/` is what keeps the workflow coherent. - -## Command Layer Structure - -Command modules at `commands/.py` are the entry points for public -operations such as `include`, `skip`, and `discard`. Those files should keep -argument-level guards, command-level scope resolution, and delegation to the -operation owner. - -Support code lives in command subpackages named for the kind of workflow they -own: - -- `commands.selection` - Selected-hunk, selected-line, replacement, and selected-change helpers. -- `commands.file_scope` - Explicit file-scoped include, skip, discard, and batch-transfer helpers. -- `commands.batch_source` - Batch-source action planning, candidate selection, previews, and completion. -- `commands.batch_transform` - Batch-to-batch transform persistence and result reporting. -- `commands.session` - Session iteration helpers shared by command entry points. -- `commands.fixup` - Suggest-fixup history helpers. - -File-scoped `show --file` and single-file `show --from --file` add a -page-aware review layer on top of the selected-change cache. Large file reviews -can show only one page or a page range. When not all pages are shown, pathless -or omitted-path actions such as `include`, `skip`, `discard`, or `include ---file` without a path refuse unless the user names a file explicitly or selects -complete actionable changes that were actually shown. Multi-file `show --files` -and multi-file `show --from ` are navigational lists; they intentionally -clear selected-change state instead of leaving a hidden selected file behind, -and later pathless or omitted-path actions refuse until the user shows a hunk or -opens one listed file. - -## Core Representation of Changes - -The `core/` package defines the models and parsing logic used throughout the -project. - -Important pieces include: - -- `core.models` - Shared dataclasses such as `LineLevelChange`, `LineEntry`, - `BinaryFileChange`, and hunk/header representations. -- `core.diff_parser` - Streaming unified-diff parsing from Git output into structured models. -- `core.line_selection` - Parsing and formatting of line ID selections like `1,3,5-7`. -- `core.hashing` - Stable hunk hashing for tracking progress across commands. - -The diff parser is especially central. Many workflows begin with raw `git diff` -output and convert it into structured hunk objects that later modules can -filter, render, and apply. - -The key design choice here is that parsing and structural modeling are kept -separate from command policy. `core/` should describe "what the diff says," -not "what the command should do." - -## Session and Workflow State - -The `data/` package is the backbone of the interactive workflow. - -It stores information such as: - -- whether a session is active -- the starting `HEAD` -- stash-like abort restoration data -- the currently selected hunk or file -- the last page-aware file review, including which pages and complete - actionable selections were shown -- included, skipped, or discarded progress -- iteration state for `again` -- undo/redo checkpoints -- hidden consumed selections -- cached batch source mappings for the current session - -Important modules include: - -- `data.session` - Session lifecycle, abort initialization, snapshotting, and cleanup. -- `data.hunk_tracking` - Discovery, navigation, selected-change orchestration, and live filtering. -- `data.selected_change` - Persistence, loading, snapshots, stale-cache validation, and file-scoped views - for the current selected change. -- `data.undo_checkpoints` - Undo/redo checkpoints for session operations. -- `data.line_state` - Serialization of the currently selected line-level view. -- `data.line_id_files` - Persistence for processed include, skip, and batch line ID files. -- `data.file_review.state` - Persistence for page-aware file review safety state. -- `data.file_review.records` - Dataclasses and enums that describe persisted file review state. -- `data.file_review.pages` - Parsing and normalization for persisted review page selections. -- `data.file_review.fingerprints` - Fingerprints of selected file views so later pathless actions can refuse - stale or ambiguous operations. -- `data.progress`, `data.file_tracking` - Progress bookkeeping across files and hunks. - -This layer is what makes the tool feel interactive even though many commands -are separate processes. A later invocation does not recompute everything from -scratch; it resumes from persisted session state. - -## How Normal Staging Works - -For ordinary include/skip/discard workflows, the architecture is: - -1. Discover a diff hunk with `git diff`. -2. Parse it into `LineLevelChange` or `BinaryFileChange`. -3. Cache the selected change in session state. -4. Render it for the user. -5. Apply the user's decision back to Git state and progress state. - -There are two broad paths: - -- text hunks -- binary file changes - -### Text hunks - -Text hunks are handled either as whole hunks or line-scoped selections. - -- Whole-hunk include/discard often delegates directly to Git patch application. -- Line-level include/discard uses `staging.content_buffers` to build exact - target content from the parsed hunk plus selected line IDs, then - `staging.index_update` writes index targets through Git objects. - -That split is important. The program does not always ask Git to apply a smaller -patch. For fine-grained line operations, it often computes the intended file -content itself and then writes that result back through Git-aware helpers. - -### Binary changes - -Binary file changes are treated as file-level units. They are detected in the -diff parser and routed through separate command handling because line-level -operations do not make sense there. - -## The Batch Subsystem - -The `batch/` package is the largest specialized subsystem in the project. -It exists to let users defer, recall, split, reapply, and reconcile saved -changes independently of the immediate staging pass. - -At a high level, the batch subsystem is responsible for: - -- storing named batch state -- representing batch ownership over file content -- reconstructing batch views for display -- merging batch-owned changes into the current working tree or index -- reversing batch-owned changes out of the working tree -- filtering already-batched content out of live diffs -- handling stale batch sources as files evolve -- reconciling batches against newer tip state with `sift` - -The most important conceptual rule is that batches are not just stored diffs. -They are a constraint-based model over per-file source snapshots. - -The deeper model, data layout, and merge behavior are documented in -[BATCHES.md](BATCHES.md). This file only describes where that logic sits in the -project. - -Key modules: - -- `batch.text_file_storage` - Persisting text file ownership and realized text content. -- `batch.binary_file_storage` - Persisting binary file entries. -- `batch.gitlink_storage` - Persisting submodule pointer entries. -- `batch.file_entry_storage` - Copying and removing generic batch file entries. -- `batch.content_commits` - Publishing realized batch content trees. -- `batch.realized_file_content` - Streaming realized text content from baseline, source, and ownership. -- `batch.state_refs` - Publishing authoritative batch state into Git refs. -- `batch.query` - Reading batch metadata and refs. -- `batch.ownership` - Ownership data structures and ownership transformations. -- `batch.ownership_units` - Ownership-unit grouping, filtering, validation, and rebuild logic. -- `batch.ownership_remapping` - Ownership remapping across matched and lineage-tracked source spaces. -- `batch.ownership_translation` - Selected-line translation into batch ownership claims. -- `batch.absence_content` - Streaming builders for absence-claim content buffers. -- `batch.replacement_line_runs` - Replacement line-run derivation from old and new file content. -- `batch.merge` - Structural merge and reverse-merge logic. -- `batch.merge_validation` - Structural safety checks for claimed-line placement during merge. -- `batch.absence_constraints` - Absence-claim suppression and deletion-choice resolution. -- `batch.baseline_edits` - Exact baseline-coordinate edit fallback for replacement merge round trips. -- `batch.baseline_correspondence` - Baseline restoration mapping for discarding batch-owned changes. -- `batch.discard_reversal` - Presence-constraint reversal for batch discard operations. -- `batch.realized_boundaries` - Boundary lookup and sequence checks over realized entries. -- `batch.attribution` - Ownership attribution for filtering live diffs. -- `batch.source_refresh` - Stale-source detection and repair orchestration. -- `batch.source_cache`, `batch.source_buffers`, `batch.source_snapshots` - Session source-commit caching, session-start buffer loading, and batch source - commit construction. -- `batch.selection` - File scope and line-level batch selection rules. -- `batch.display` - Reconstructing batch views for `show --from` and related workflows. -- `batch.comparison` - Shared semantic-run comparison logic used by attribution and sift. - -## Display, Output, and TUI - -The program separates data modeling from terminal rendering. - -- `output/` - Knows how to print line-level changes, page-aware file reviews, multi-file - review lists, binary changes, patches, and colors. Page-selection state - belongs to `data.file_review`, while `output/` owns terminal models and - rendering. -- `tui/` - Adds the interactive menu-driven front end. - -The interactive UI is not a separate architecture stack. It mostly sits on top -of the same command and state machinery used by the plain CLI. That keeps the -behavior consistent: the TUI is another way to drive the same underlying -workflow rather than a parallel implementation. - -## Git Integration Philosophy - -The `utils.git_command`, `utils.git_refs`, and related `utils.git_*` modules -wrap subprocess calls to Git and provide streaming and transactional helpers -such as: - -- `run_git_command()` -- `stream_git_command()` -- `update_git_refs()` -- blob/object readers and writers - -The project relies on Git as the source of truth for: - -- the working tree -- the index -- committed history -- object storage -- ref updates - -This is a deliberate architectural choice. The tool owns workflow state and -batch-specific metadata, but it delegates repository semantics to Git whenever -possible. - -## State Layout and Persistence Strategy - -There are two broad kinds of state in the project: - -- session/workflow state -- Git-backed batch state - -Session state lives under the tool's state directory and supports: - -- resuming selected hunks -- `again` -- `undo` / `redo` -- `abort` -- hidden masking of already-consumed selections - -Batch state is more durable and increasingly Git-native: - -- content refs under `refs/git-stage-batch/batches/*` -- state refs under `refs/git-stage-batch/state/*` - -That split lets the tool provide both: - -- ephemeral workflow control for the current pass -- durable saved changes that outlive one pass - -## Error Handling and Safety - -A recurring theme in the codebase is conservative refusal. - -This shows up in several places: - -- stale selected hunks are rejected and recalculated -- stale or mismatched file-review safety state is rejected before pathless or - omitted-path line actions are accepted -- partial file reviews block ambiguous pathless or omitted-path actions until - the user shows all pages, names the file explicitly, or selects complete shown - changes -- batch merge paths fail when structure is ambiguous -- atomic ownership units cannot be partially selected -- abort snapshots are captured up front so destructive operations can be rolled - back to session start - -The general rule is that the project prefers refusing an operation over -guessing when the guess could silently damage the working tree, the index, or -saved batch state. - -## Build, Packaging, and Documentation - -Top-level project files provide the surrounding system: - -- `pyproject.toml` - Python project metadata and development tooling config. -- `meson.build` - Build and packaging integration. -- `docs/` - User-facing website documentation. -- `completions/` - Shell completion support. -- `assets/` - Bundled assistant assets and related material. - -The architecture documents at the repository root serve a different purpose from -the website docs: - -- website docs explain how to use the tool -- root-level architecture docs explain how the implementation is organized - -## Testing Strategy - -The test suite mirrors the code structure closely: - -- `tests/cli/` - CLI parsing and dispatch behavior. -- `tests/commands/` - User-facing command behavior. -- `tests/core/` - Diff parsing, models, line selection, encoding, and hashing. -- `tests/data/` - Session and state-management logic. -- `tests/staging/` - Selective content construction and application helpers. -- `tests/batch/` - Batch storage, merge, ownership, sift, validation, and attribution behavior. -- `tests/functional/` - Multi-step end-to-end workflows. -- `tests/tui/` - Interactive UI behavior. - -That layout is useful when making changes: - -- if you are touching diff structure, start in `tests/core/` -- if you are touching session flow, look in `tests/data/` and - `tests/functional/` -- if you are touching named batches, expect most coverage in `tests/batch/` - plus command-level tests - -## Where to Start Reading - -For a contributor new to the codebase, a good reading order is: - -1. `README.md` - Understand the product-level workflow. -2. `src/git_stage_batch/cli/argument_parser.py` - See the public command surface. -3. `src/git_stage_batch/runtime.py` and - `src/git_stage_batch/cli/execution.py` - See how parsed arguments enter interactive mode or command execution. -4. `src/git_stage_batch/commands/start.py`, - `src/git_stage_batch/commands/show.py`, - `src/git_stage_batch/commands/include.py`, - `src/git_stage_batch/commands/skip.py`, - `src/git_stage_batch/commands/discard.py` - Understand the core non-batch workflow. -5. `src/git_stage_batch/data/session.py` and - `src/git_stage_batch/data/hunk_tracking.py`, then - `src/git_stage_batch/data/selected_change/` - Understand state, navigation, and selected-change persistence. -6. `src/git_stage_batch/core/diff_parser.py` and - `src/git_stage_batch/core/models.py` - Understand the shared representation of change. -7. `BATCHES.md` and then `src/git_stage_batch/batch/*` - Understand the advanced deferred-change architecture. - -## Summary - -At the project level, `git-stage-batch` is an interactive workflow engine built -on top of Git: - -- `core/` models change -- `commands/` define behavior -- `data/` preserves workflow state -- `staging/` computes selective file results -- `batch/` adds durable deferred-change semantics -- `output/` and `tui/` present the workflow to the user -- `utils/` keeps Git and filesystem integration manageable - -If you keep that separation in mind, most of the codebase becomes easier to -navigate. The batch subsystem is the deepest specialized part, but it still sits -inside that larger pattern rather than replacing it. +# Codebase guide for contributors + +This guide explains where the program starts, how one user action moves through +the source tree, and which files normally change when behavior is added or +removed. It describes the current source. The architecture tests remain the +authority when a statement here and the code disagree. + +Paths in the command walkthroughs are places to start reading the current +implementation. They are not a required module inventory. Adding a small helper +or combining helpers does not change the organization described here when the +same directory still owns the same work. + +Start with this guide for ordinary command, session, display, and interactive +work. Do not start in `src/git_stage_batch/batch/`. That directory implements +named batches and the merge rules used by a small number of line operations. +Read [Batch internals](BATCHES.md) only when a change reaches one of those +behaviors. + +## Terms used in this guide + +The following terms have their Git meanings: + +- **Working tree**: the files checked out on disk. +- **Index**: the Git staging area that will supply the next commit. +- **Current commit**: the commit named `HEAD` by Git. +- **Hunk**: one contiguous section of a file diff. + +The following terms name state maintained by this program: + +- **Session**: the persisted work started by `git-stage-batch start` and ended + by `stop` or `abort`. +- **Selected change**: the hunk, file, rename, deletion, binary file, submodule + pointer, or file mode change saved for the next action. +- **File review**: a saved view of one file, including which pages and line + identifiers were shown. Later actions use that record to reject stale or + unseen selections. + +## Read these files first + +The shortest reading path for an ordinary command is: + +1. [`src/git_stage_batch/cli/main.py`](src/git_stage_batch/cli/main.py) parses + arguments, changes directory when `-C` is present, acquires the session lock, + and handles errors. +2. [`src/git_stage_batch/cli/subcommand_registry.py`](src/git_stage_batch/cli/subcommand_registry.py) + lists every command that the parser registers. +3. [`src/git_stage_batch/runtime.py`](src/git_stage_batch/runtime.py) chooses + interactive or non-interactive execution. +4. [`src/git_stage_batch/cli/execution.py`](src/git_stage_batch/cli/execution.py) + calls the function stored on the parsed arguments. +5. [`src/git_stage_batch/commands/start.py`](src/git_stage_batch/commands/start.py), + [`show.py`](src/git_stage_batch/commands/show.py), and + [`include.py`](src/git_stage_batch/commands/include.py) show the main session + workflow. +6. [`src/git_stage_batch/data/hunk_tracking.py`](src/git_stage_batch/data/hunk_tracking.py) + finds and saves the next change. +7. [`src/git_stage_batch/core/diff_parser.py`](src/git_stage_batch/core/diff_parser.py) + converts Git diff output into Python values. +8. [`src/git_stage_batch/commands/selection/selected_change_staging.py`](src/git_stage_batch/commands/selection/selected_change_staging.py) + shows how a selected change reaches the index and how the next change is + selected. + +That path is enough to understand whole-hunk `start`, `show`, `include`, `skip`, +and `discard`. It does not require reading the named-batch implementation. + +## What each source directory contains + +| Path | What belongs there | Change it when | +| --- | --- | --- | +| `cli/` | Parser construction, argument normalization, help routing, completion, and choosing which command function matches the supplied options | A command, option, alias, or argument combination changes | +| `commands/` | Preconditions and the complete sequence for one user-requested operation | The effect of a command changes | +| `core/` | Diff values, diff parsing, line-selection parsing, hashing, replacement values, and byte-preserving buffers | The representation of a change changes without reading or writing session state | +| `data/` | Reading and writing session state, finding the next change, checking freshness, progress, undo and redo records, and file-review records | Behavior must survive another program invocation or must be derived from persisted session state | +| `staging/` | Building exact target file content and writing that content to the index | A selective text operation changes the content staged for one file | +| `output/` | Converting prepared values into terminal text | Wording, colors, rows, summaries, or machine-readable output changes | +| `tui/` | Prompts, key handling, menus, and calls into `commands/` for interactive mode | A behavior must be reachable or presented differently in interactive mode | +| `utils/` | Git command execution, Git object access, Git references, paths, file access, process streaming, and journals | Several callers need the same low-level Git, process, path, or file operation | +| `editor/` | The in-memory line editor, piece table, line endings, and line export | Byte-preserving line editing changes | +| `batch/` | Named-batch storage, ownership records, display reconstruction, matching, merge, discard reversal, and source refresh | A named-batch behavior changes, already-batched lines are filtered incorrectly, or live line inclusion reaches the temporary batch merge described below | + +The package root also contains modules with narrow jobs: + +- [`runtime.py`](src/git_stage_batch/runtime.py) chooses interactive or + non-interactive execution. +- [`exceptions.py`](src/git_stage_batch/exceptions.py) defines errors and + top-level command exit behavior. +- [`git_paths.py`](src/git_stage_batch/git_paths.py) encodes, decodes, and + parses Git pathnames without losing raw bytes. +- [`i18n.py`](src/git_stage_batch/i18n.py) loads translated messages. + +## How a command reaches its implementation + +Every explicit non-interactive command follows this sequence: + +1. The installed `git-stage-batch` program calls `main()` in + [`cli/main.py`](src/git_stage_batch/cli/main.py). +2. `parse_command_line()` in + [`cli/argument_parser.py`](src/git_stage_batch/cli/argument_parser.py) + expands short actions, builds the parser, parses the arguments, and + normalizes file arguments. +3. [`cli/root_parser.py`](src/git_stage_batch/cli/root_parser.py) creates the + root parser and asks `add_cli_subcommands()` to register commands. +4. One parser registration function stores a callable in `args.func` with + `set_defaults(func=...)`. +5. `dispatch_cli_mode()` in [`runtime.py`](src/git_stage_batch/runtime.py) + sends non-interactive work to `execute_non_interactive_args()`. +6. [`cli/execution.py`](src/git_stage_batch/cli/execution.py) calls + `args.func(args)`. +7. That callable invokes a function under `commands/`, either directly or + through a dispatch module under `cli/` when several option combinations + share one command name. + +The parser chooses *which* operation the user requested. The function under +`commands/` owns *how* that operation behaves. + +### Example: `status` + +`git-stage-batch status` has a short path that shows the responsibility of each +directory: + +1. `add_status_subcommand()` in + [`cli/session_subcommands.py`](src/git_stage_batch/cli/session_subcommands.py) + declares `status`, its options, and the call to `command_status()`. +2. `command_status()` in + [`commands/status.py`](src/git_stage_batch/commands/status.py) checks whether + a session is active and chooses human-readable, machine-readable, or shell + prompt output. +3. `read_status_summary()` in + [`data/status_summary.py`](src/git_stage_batch/data/status_summary.py) reads + persisted state and returns one dictionary. +4. `print_status_summary()` in + [`output/status.py`](src/git_stage_batch/output/status.py) prints the + human-readable form. [`output/status_prompt.py`](src/git_stage_batch/output/status_prompt.py) + renders the shell prompt form. + +The parser does not read session files. The data module does not print. The +output module does not decide whether the command is allowed. + +### Example: `include` for one selected hunk + +`git-stage-batch include` shows a mutating command: + +1. `add_include_subcommand()` in + [`cli/selection_subcommands.py`](src/git_stage_batch/cli/selection_subcommands.py) + declares the arguments and stores `dispatch_include_command()` as the + callable. +2. [`cli/include_dispatch.py`](src/git_stage_batch/cli/include_dispatch.py) + distinguishes whole-hunk, whole-file, selected-line, replacement, named-batch + source, and named-batch destination forms. +3. With no scope option, it calls `command_include()` in + [`commands/include.py`](src/git_stage_batch/commands/include.py). +4. `command_include()` checks repository and session requirements and rejects + a stale or ambiguous selection. +5. `include_selected_change()` in + [`commands/selection/selected_change_staging.py`](src/git_stage_batch/commands/selection/selected_change_staging.py) + loads the selected change or asks `fetch_next_change()` to find one. It also + opens an undo checkpoint. +6. The same module handles each concrete change type. A text hunk is applied to + the index with `git_apply_to_index()`. Binary files, renames, file modes, + deletions, and submodule pointers use their corresponding Git helpers. +7. [`data/progress.py`](src/git_stage_batch/data/progress.py) records the + completed hunk. +8. `finish_selected_change_action()` in + [`commands/selection/action_completion.py`](src/git_stage_batch/commands/selection/action_completion.py) + selects and displays the next change when automatic advancement is enabled. + +Whole-hunk inclusion does not use the named-batch merge implementation. + +### Exception: `include --line` + +Line inclusion has an additional safety path: + +1. `command_include_line()` resolves the selected file and line identifiers. +2. `include_live_line_selection()` in + [`commands/selection/include_line_action.py`](src/git_stage_batch/commands/selection/include_line_action.py) + loads the current index, the session snapshot, and the working-tree + snapshot. +3. [`commands/selection/include_line_selection.py`](src/git_stage_batch/commands/selection/include_line_selection.py) + creates a temporary named batch, translates the selected lines into batch + ownership, and asks the batch merge code to build the new index content. +4. The function verifies that applying the same temporary ownership to the + working tree would leave the working tree unchanged. +5. [`staging/index_update.py`](src/git_stage_batch/staging/index_update.py) + writes the accepted content to the index. The temporary batch is removed + before the command returns. + +Read [Batch internals](BATCHES.md) when changing steps 3 or 4. A change to +argument parsing, session checks, selected-line loading, or the final index +write can usually be understood without reading the rest of `batch/`. + +## How interactive mode reaches the same commands + +Interactive mode changes how the user chooses an action; it does not provide a +second implementation of that action. + +1. `dispatch_cli_mode()` imports and calls `start_interactive_mode()` from + [`tui/interactive.py`](src/git_stage_batch/tui/interactive.py). +2. [`tui/action_dispatch.py`](src/git_stage_batch/tui/action_dispatch.py) maps + a key to one handler. +3. A handler for the selected action calls a function under `commands/`. For example, + [`tui/hunk_actions.py`](src/git_stage_batch/tui/hunk_actions.py) calls + `command_include()`, `command_skip()`, or `command_discard()`. + +Add shared behavior under `commands/`, then call it from interactive mode. Do +not duplicate the operation in `tui/`. + +## Where a change goes + +Use the narrowest row that describes the change: + +| Requested change | Start here | Other places that commonly change | +| --- | --- | --- | +| Add an option to an existing command | Its registration function under `cli/` | Its dispatch module, command function, command tests, manual page, website command reference, shell completion | +| Change what a command does | Its module under `commands/` | The command subpackage that owns the affected sequence, data reads or writes, output renderer, command and functional tests | +| Change how Git diff text is understood | `core/diff_parser.py` and the relevant value in `core/models.py` | Core tests, data discovery code, output code for a new value type | +| Add persisted session information | The module under `data/` that reads and writes that record, plus a path helper under `utils/paths.py` | Session cleanup, abort or undo handling when applicable, data tests, functional tests | +| Change human-readable text layout | The module under `output/` that prints that form | The command that prepares its input, output tests, translated source string | +| Add an interactive key or menu item | A handler under `tui/` that calls the command | `tui/action_dispatch.py`, prompt or help modules, a command function, interactive tests | +| Add a reusable Git operation | A specific `utils/git_*` module | Utility tests and the command or data caller | +| Change line content written to the index | `staging/content_buffers.py` or `staging/index_update.py` | The command that chooses the content, staging tests, command tests | +| Change named-batch persistence or merge behavior | The owning module under `batch/` | A batch-facing command, batch tests, command tests, [Batch internals](BATCHES.md) | + +## Add a command users can run + +A command is complete when every public way to discover and run it agrees. +Follow these steps: + +1. Add the operation under `src/git_stage_batch/commands/`. Put supporting + functions in an existing command subpackage when that subpackage already + owns the same behavior. +2. Register the command in the matching file under `src/git_stage_batch/cli/`. + Current groups include session commands, selected-change commands, + file-blocking commands, fixup commands, asset commands, and named-batch + commands. +3. Add that registration function to + `cli/subcommand_registry.py`. A new option on an existing command does not + need another registry entry. +4. Call the command function directly from the registration function when the + argument mapping is simple. Add or extend a dispatch module under `cli/` + when mutually exclusive forms such as `--file`, `--line`, `--from`, and + `--to` choose different command functions. +5. Put state reads and writes in `data/`, and terminal rendering in `output/`, + when the command needs them. +6. If interactive mode exposes the command, add a handler under `tui/` that + calls the same command function. Update the visible prompt and help text. +7. Add or update `man/git-stage-batch-.1.in`. For a new page, add it + to `manpage_specs` in `meson.build`. Update `man/git-stage-batch.1.in` when + the root manual page lists the command. +8. Update `completions/git-stage-batch`, `docs/commands.md`, and any workflow + guide that teaches the affected behavior. +9. Wrap user-visible Python strings with the translation function imported + from `i18n.py`. The build runs `scripts/generate_potfiles.py` to find source + files containing translated strings. +10. Add focused tests in the directory matching the changed source, then add a + functional test when correctness depends on several invocations or on the + combined working tree, index, and session state. + +## Add an option to an existing command + +An option normally touches fewer places: + +1. Declare it in the command's registration function under `cli/`. +2. Pass the parsed value through the existing callable or dispatch module. +3. Add the behavior to the command function or to the module that already owns + that part of the command. +4. Update shell completion, the command's manual page, and `docs/commands.md`. +5. Test parser acceptance and rejection in `tests/cli/`. +6. Test the resulting behavior in `tests/commands/` or `tests/functional/`. +7. If interactive mode offers the same choice, update its prompt, handler, and + tests explicitly. Do not assume a command-line option becomes interactive + automatically. + +## Remove a command or option + +Remove the behavior from the outside inward so each remaining call has an +implementation until its caller is removed: + +1. Remove interactive prompts, menu choices, key handlers, and interactive + tests that call the behavior. +2. Remove the parser argument or command registration and its parser tests. +3. Remove the call from `cli/subcommand_registry.py` when deleting a command. +4. Remove the command function and then any support functions with no callers. +5. Remove persisted state only after accounting for repositories that may still + contain it. If old state must remain readable, keep the reader or add an + explicit migration. +6. Remove the command or option from shell completion, manual pages, website + documentation, examples, and assistant assets that name it. +7. Remove a deleted manual page from `manpage_specs` in `meson.build`. +8. Search for the command spelling, aliases, option spelling, command function, + state filenames, and translated messages before considering the removal + complete. +9. Run the architecture tests after deleting support modules. They catch stale + imports and accidental movement of behavior into the wrong package. + +## Change persisted session state + +Each kind of persisted session record has one data module that reads and writes +it. Use this sequence when adding or changing a record: + +1. Add the path in [`utils/paths.py`](src/git_stage_batch/utils/paths.py). +2. Add serialization and validation in the module under `data/` that owns the + record. +3. Decide when the record is created, refreshed, and cleared. Selected-change + files are enumerated by + [`data/selected_change/store.py`](src/git_stage_batch/data/selected_change/store.py) + and cleared through + [`data/selected_change/lifecycle.py`](src/git_stage_batch/data/selected_change/lifecycle.py). +4. Decide whether undo, redo, abort, `again`, or `stop` must restore or delete + the record. These operations do not cover new state automatically. +5. Reject malformed or stale records at the read boundary. Do not let callers + guess whether a partially valid record is safe. +6. Test creation, loading, stale input, cleanup, and the relevant recovery + operation under `tests/data/` and `tests/functional/`. + +## Keep prepared data separate from rendering + +The `status` path demonstrates the expected division: + +- `data/status_summary.py` prepares values. +- `output/status.py` prints those values. +- `commands/status.py` chooses the form and controls errors. + +Use the same division for new output. A data module may return strings that are +stored data, but it should not print. An output module may format and print, but +it should not mutate the index, working tree, or session. + +## Tests that match each source area + +| Source changed | First test directory | Add a functional test when | +| --- | --- | --- | +| `cli/` | `tests/cli/` | Argument routing must be proven with repository state | +| `commands/` | `tests/commands/` | The operation spans several commands or recovery steps | +| `core/` | `tests/core/` | The parsed value must also be applied to a real repository | +| `data/` | `tests/data/` | State must survive several program invocations | +| `staging/` | `tests/staging/` | The index and working tree must be compared after a full command | +| `output/` | `tests/output/` | Usually not needed unless command context changes the rendering | +| `tui/` | `tests/tui/` | The same behavior must be proven across interactive and non-interactive use | +| `utils/` | `tests/utils/` | A real repository is required to prove the helper contract | +| `editor/` | `tests/editor/` | Edited content must pass through a complete command | +| `batch/` | `tests/batch/` | A named batch must survive or interact with a session workflow | + +Use `tests/functional/` for multi-step repository behavior, not as a substitute +for a focused test of the owning module. + +## Import rules checked by the test suite + +[`tests/architecture/test_import_boundaries.py`](tests/architecture/test_import_boundaries.py) +checks the current package boundaries. The broad rules are: + +- top-level packages must not form an import cycle +- modules inside `batch/` must not form an import cycle +- `batch/` must not import workflow storage from `data/` +- `commands/` must not import `tui/` +- `batch/`, `core/`, `data/`, and `utils/` must raise errors without importing + the command exit helper +- modules named by an architecture test must be imported directly rather than + hidden behind package re-exports + +Many tests in that file protect a specific ownership decision. Read the failing +test before moving a function between modules. Its name and assertions state +the required location and allowed import direction. + +Run the architecture checks with: + +```console +uv run pytest -n auto tests/architecture/test_import_boundaries.py +``` + +## When `batch/` is relevant + +Read [Batch internals](BATCHES.md) before changing any of the following: + +- commands that use `--to` or `--from` +- `new`, `list`, `drop`, `annotate`, `validate`, `reset`, or `sift` +- storage below `refs/git-stage-batch/` +- translation between displayed lines and saved batch ownership +- hiding already-batched changes from the live diff +- merge or discard behavior for a saved batch +- source refresh after a file changes +- the temporary batch merge used by `include --line` + +For ordinary parser work, whole-hunk actions, session lifecycle, progress, +output, and most interactive menus, stay in the directories described in this +guide and treat `batch/` as an implementation dependency reached through an +existing function. + +## Validation before submitting a change + +Run focused tests while editing. Before submitting a broad source change, run: + +```console +uv run pytest -n auto +uv build +``` + +For documentation changes, build the website with strict link and navigation +checking: + +```console +uv run mkdocs build --strict +``` + +The complete test suite is intentionally run in parallel. A serial run is most +useful for focused debugging. diff --git a/BATCHES.md b/BATCHES.md index c83c6ade7..63f79a93f 100644 --- a/BATCHES.md +++ b/BATCHES.md @@ -1,456 +1,67 @@ -# Git Stage Batch: Current Batch Architecture - -This document explains how the current batch system works internally. -It is meant to be usable both by contributors and by readers who have not used -the tool before. - -It reflects the implementation under `src/git_stage_batch/batch/`, -`src/git_stage_batch/data/`, and the batch-facing commands in -`src/git_stage_batch/commands/`. - -## Table of Contents - -1. Introduction -2. Typical User Workflow -3. Mental Model -4. Storage Model -5. Session Model and Abort Semantics -6. Ownership Model -7. Materialization and Realized Batch Content -8. Applying and Discarding a Batch -9. Stale Batch Sources and Source Advancement -10. Display, Attribution, and Hidden Changes -11. Semantic Units and Line-Level Batch Selection -12. Reset and Moving Claims Between Batches -13. Sift -14. Binary Files -15. Important Invariants -16. Key Code Paths - ---- - -## Introduction - -Git itself gives you one mutable staging area: the index. That is enough when -you want to stage a change now or leave it unstaged for later. It is less -helpful when one working tree contains several unrelated changes and you want to -separate them over time. - -`git-stage-batch` adds named, persistent buckets called batches. A batch lets -you set aside part of the current change set without committing it yet. Later, -you can bring that saved change back into the index or the working tree, remove -it from the working tree, move it to another batch, or reconcile it against -newer code. - -Concretely, a batch can later: - -- stage its changes into the index and working tree -- apply its changes back to the working tree -- discard its changes from the working tree -- move or reset its claims -- reconcile itself against the current tip with `sift` - -Internally, the implementation is not patch-replay based. It stores ownership as -constraints: - -- presence constraints: source lines a batch claims -- absence constraints: baseline sequences a batch says must not appear - -Those constraints are stored in batch metadata and interpreted relative to a -stable per-file snapshot. The rest of this document explains that model from -the outside in. - ---- - -## Typical User Workflow - -Before the internal details, it helps to see the feature from the user's point -of view. - -A common workflow looks like this: - -1. Run `git-stage-batch start` to begin a staging session. -2. Review the current unstaged changes hunk by hunk. -3. Save some changes into a named batch with `include --to ` or - `discard --to ` instead of staging or discarding them permanently. -4. Keep editing the working tree, possibly across multiple `again` passes. -5. Later, operate on the saved batch with commands such as - `show --from `, `include --from `, `apply --from `, - `discard --from `, `reset --from `, or `sift`. - -Two facts drive the architecture: - -- a batch is persistent state, not just a temporary UI view -- the working tree can keep changing after content has been saved into a batch - -Most of the complexity in the implementation comes from preserving that saved -content accurately while the file around it keeps evolving. - ---- - -## Mental Model - -For one file in one batch, there are four states worth keeping in mind: - -1. Baseline - The committed starting point, taken from `HEAD` when the batch is created. -2. Batch source - A stable per-file snapshot used as the coordinate space for the batch's saved - claims. -3. Realized batch content - The stored file view produced by applying only this batch's ownership to the - baseline. -4. Current working tree - The file as it exists now, which may have drifted from the source snapshot. - -If you are new to the tool, the shortest useful summary is: - -- the baseline is where the batch started from -- the batch source is the snapshot the batch uses to remember "which lines" -- the realized batch content is what the batch itself stores as its file view -- the working tree is whatever the user has on disk right now - -The batch does not store "apply this diff later". It stores: - -- which source lines must be present -- which deleted baseline sequences must stay absent - -Later operations use those four states to decide how to reapply the saved change -to today's working tree, or how to remove that saved change from today's working -tree. - ---- - -## Storage Model - -### Ref layout - -Current authoritative refs live under: - -- `refs/git-stage-batch/batches/`: realized batch content commit -- `refs/git-stage-batch/state/`: normalized batch metadata plus embedded - source snapshots - -See `src/git_stage_batch/batch/ref_names.py` and -`src/git_stage_batch/batch/state_refs.py`. - -Legacy `refs/batches/` refs are still read as migration inputs, but once a -batch is rewritten by current code the authoritative refs above replace them. - -### What the content ref stores - -The content ref points at a normal Git commit whose tree contains the realized -batch files described in the previous section. For text files, realized content -is built from: - -- batch baseline -- batch source file content -- batch ownership - -Text materialization lives in `src/git_stage_batch/batch/text_file_storage.py` -and `src/git_stage_batch/batch/realized_file_content.py`. Batch content trees -are published through `src/git_stage_batch/batch/content_commits.py`. - -### What the state ref stores - -The state ref stores a tree containing: - -- `batch.json`: normalized metadata -- `sources/` entries: embedded source snapshots for files in the batch - -`sync_batch_state_refs()` publishes that state from file-backed metadata into -Git, then removes the old file-backed metadata directory. - -This means the authoritative batch state is now Git-native, not just files under -the local state directory. - -### Metadata shape - -Per batch: - -- `note` -- `created_at` -- `baseline` / `baseline_commit` -- `files` - -Per text file: - -- `batch_source_commit` -- `presence_claims` -- `deletions` -- `replacement_units` (optional; omitted when empty) -- `mode` -- `change_type` (optional; `added` or `deleted` for whole-path lifecycle changes) - -Per binary file: - -- `file_type = "binary"` -- `change_type` -- `batch_source_commit` -- `mode` - -`deletions` are serialized as anchored blobs, not inline text. -`presence_claims` is the first-class representation for source content that must -exist after the batch is applied. Each claim stores `source_lines`; when needed, -it can also store `baseline_references` keyed by source line with blob-backed -before/after boundary bytes. -`replacement_units` records explicit coupling between presence source ranges and -deletion indexes so replacement atomicity does not need to be rediscovered from -display adjacency. The field is omitted when no explicit replacement units are -stored. -Text `change_type` is omitted for ordinary modified-file batches. It is only -stored when a text batch owns the whole added path or the whole deleted path; -deleted text paths are absent from the batch content tree just like deleted -binary paths. - ---- - -## Session Model and Abort Semantics - -Batch behavior is coupled to the interactive session model started by -`git-stage-batch start`. - -When a session starts, the tool snapshots: - -- the starting `HEAD` -- a stash-like snapshot of tracked changes -- snapshots for untracked and intent-to-add files as needed -- the current batch refs, so batch mutations can be rolled back on `abort` - -See `src/git_stage_batch/data/session.py` and -`src/git_stage_batch/data/batch_refs.py`. - -Within that session model, batch-affecting commands also participate in the -tool's `undo` / `redo` checkpoint flow. That is a narrower mechanism than -`abort`: - -- `undo` and `redo` step backward or forward through recent session operations -- `abort` restores the repository and batch refs to the state captured at - session start - -For someone learning the tool, the practical point is that batch operations are -not isolated from the rest of the session machinery. They are part of the same -reversible interaction model. - -This matters because the initial batch source for a file is not an arbitrary -later working-tree snapshot. It is derived from session-start buffers loaded by -`src/git_stage_batch/batch/source_buffers.py`, then persisted as per-file source -commits by `src/git_stage_batch/batch/source_snapshots.py`. The active session's -per-file source commit cache lives in `src/git_stage_batch/batch/source_cache.py`. -For files that did not exist at session start, initial source creation falls -back to the current working-tree file content so the new file's claimed lines -actually exist in source space. - -That design keeps later discard/abort behavior anchored to the same session -snapshot even if the user keeps editing. - ---- - -## Ownership Model - -With the session model in place, ownership can be stated precisely. -Ownership is defined in `src/git_stage_batch/batch/ownership.py`. - -`BatchOwnership` has three persistent fields: - -- `presence_claims` - `PresenceClaim(source_lines, baseline_references)` records source-space line - ranges like `["3-7", "10"]` that must exist after application. Optional - `baseline_references` record the old-file boundary around individual presence - lines when that extra proof is available. -- `deletions` - `DeletionClaim(anchor_line, content_lines)` records that a specific baseline - sequence must be absent after a given source line, or at start-of-file if the - anchor is `None`. Optional `baseline_reference` metadata records the same - old-file coordinate shape used by presence claims. -- `replacement_units` - Optional metadata linking presence source ranges to entries in `deletions` when - the capture path knows they form one replacement. These persisted units are - selected atomically as a whole; display-adjacency grouping is only the fallback - for ownership without explicit replacement metadata. - -This distinction is central: - -- presence claims say "this source content belongs to the batch" -- absence claims say "this baseline content was removed by the batch" - -The metadata key is still `deletions` for historical reasons, but the system -treats those entries as absence constraints, not as negative patches to replay. - -### Ownership versus attribution - -This document uses two similar words for two different layers: - -- ownership - The persistent claims stored in batch metadata. -- attribution - The derived answer to "which visible parts of the current working tree appear - to belong to which batches right now?" - -Ownership is stored and declarative. -Attribution is derived and ephemeral. - -That distinction matters because the tool can hide already-batched content from -the live diff even though the batch metadata itself has not changed. Attribution -is one view over the current file state. Ownership is the underlying saved -state. - -### Coordinate space - -Ownership is always expressed in batch-source coordinates, not working-tree -coordinates. - -That means: - -- line numbers remain stable while the working tree evolves -- line-level batch operations must translate between display IDs, - working-tree lines, and source-space lines - ---- - -## Materialization and Realized Batch Content - -Once baseline, source, and ownership exist, the realized text content for a -batch is built by `build_realized_buffer_from_lines()` in -`src/git_stage_batch/batch/realized_file_content.py` and persisted by -`src/git_stage_batch/batch/text_file_storage.py`. - -That function: - -1. reads baseline bytes -2. reads batch source bytes -3. resolves ownership -4. calls `satisfy_constraints()` from - `src/git_stage_batch/batch/presence_constraints.py` -5. emits full file bytes while preserving the source line-ending style - -Important detail: realization uses lenient absence enforcement. If a deletion's -baseline sequence is not found exactly at the expected boundary while building -the stored batch view, realization does not fail. It simply avoids suppressing a -non-matching sequence. This is deliberate because the stored view answers -"what does this batch claim?" rather than "can it be merged into today's -working tree right now?" - ---- - -## Applying and Discarding a Batch - -The user-facing commands that consume this model fall into three groups: - -- commands that stage batch content into the index and working tree -- commands that apply batch content to the working tree -- commands that remove batch content from the working tree - -### `include --from` - -`include --from ` stages batch content into the index and writes it to -the working tree. - -Implementation: - -- `src/git_stage_batch/commands/include_from.py` -- `src/git_stage_batch/batch/merge.py` - -For each file, the command: - -1. reads batch metadata -2. reads the file's batch source content -3. optionally narrows ownership by display line IDs or file scope -4. merges source constraints into current index bytes with `merge_batch()` -5. merges source constraints into current working-tree bytes with `merge_batch()` -6. writes the merged targets to the index and working tree - -### `apply --from` - -`apply --from ` uses the same merge model, but writes the merged result -only to the working tree, leaving the index untouched. - -Implementation: - -- `src/git_stage_batch/commands/apply_from.py` - -### `discard --from` - -`discard --from ` is the structural inverse. It removes the batch's -effects from the working tree by calling `discard_batch()`. - -Implementation: - -- `src/git_stage_batch/commands/discard_from.py` -- `src/git_stage_batch/batch/merge.py` - -The discard path uses source-to-baseline correspondence rather than raw patch -reversal. It classifies baseline/source regions as: - -- `EQUAL` -- `INSERT` -- `REPLACE_LINE_BY_LINE` -- `REPLACE_BY_HUNK` - -That classification determines whether content can be restored line-by-line or -must be restored atomically as a hunk. - -### Merge-time safety checks - -`merge_batch()` does not blindly insert missing lines. It first validates that -the batch can still be applied safely: - -- claimed source lines must either still be present or have enough surrounding - mapped context -- deletion anchors must still be structurally meaningful -- missing claimed runs must still fit coherently into the target structure - -Those checks are implemented in `_check_structural_validity()` and -`_check_claimed_region_compatibility()` in `src/git_stage_batch/batch/merge.py`. - -The design is intentionally conservative. When context is lost or ambiguous, the -tool fails instead of guessing. - -### Baseline-coordinate round trips - -If the target tree has not changed between capturing a selection into a batch and -applying that batch, the operation should round-trip through batch metadata. The -merge path supports this without making general fuzzy guesses: - -- replacement units can use `baseline_reference.after_line` plus the exact - absence bytes to replace a baseline run only when those bytes still exist at - that coordinate -- absence-only claims can use the same baseline coordinate and exact absence - bytes when their source anchor is no longer present in the target -- presence-only additions can use per-claim `baseline_references` only when the - recorded before/after boundary bytes still match the target -- presence-only baseline-coordinate application is still constraint - satisfaction, not blind insertion; if the target already equals the batch - source, or if the exact claimed insertion is already present at the recorded - boundary, the merge is a no-op for that claim -- if the target exactly equals the batch source and every claimed presence or - absence constraint carries baseline-coordinate metadata, the merge is a no-op - before structural absence suppression runs. This keeps a freshly captured - batch idempotent when round-tripped back onto the same tree, even when old - absence bytes also appear as unrelated or newly claimed source bytes. -- if those exact baseline-coordinate checks fail, the fallback declines and the - normal structural merge path decides whether the batch is still safely - mergeable - -This is deliberately narrower than raw patch replay. It handles the no-change -round trip while preserving conservative behavior when the target has drifted. - -This is worth emphasizing. A wrong guess here would not just produce a merge -conflict. It could silently place saved lines into the wrong structural context -or remove content the batch never actually owned. The implementation therefore -prefers false negatives over silent corruption: - -- if claimed lines no longer have trustworthy surrounding context, fail -- if a deletion anchor can no longer be placed reliably, fail -- if a missing claimed run seems to come from a structurally incompatible region, - fail - -From a user perspective, that can feel strict. From an architecture perspective, -it is one of the core safety properties of the system. - -### One concrete example - -Suppose `HEAD:file.txt` contains: +# Batch internals + +This document explains the code that saves changes in named batches and later +applies, stages, displays, moves, or removes those changes. + +Read the [codebase guide](ARCHITECTURE.md) first. Return to that guide if the +change concerns only whole-hunk staging, ordinary session progress, argument +parsing, terminal output, or an interactive menu. Those subjects do not require +a tour of the sizeable `src/git_stage_batch/batch/` package. + +Read this document when changing: + +- commands that use `--to` or `--from` +- `new`, `list`, `drop`, `annotate`, `validate`, `reset`, or `sift` +- storage below `refs/git-stage-batch/` +- saved-line ownership, batch display, merge, discard reversal, or source + refresh +- filtering that hides already-saved changes from the live diff +- the temporary batch merge used by `include --line` + +The implementation described here is spread across: + +- `src/git_stage_batch/batch/` for saved-batch data and operations +- `src/git_stage_batch/commands/batch_source/` for applying actions to a saved + batch +- `src/git_stage_batch/commands/selection/` and + `src/git_stage_batch/commands/file_scope/` for saving live changes into a + batch +- selected modules under `src/git_stage_batch/data/` for session recovery, + file-review safety, and hiding changes already handled during a session + +Paths named below are current places to start following a behavior. They do not +prescribe how many helper modules the package must contain. Adding or combining +a helper does not change the described design when storage, ownership, command, +and session responsibilities remain in the same packages. + +## Terms used by the code + +These names refer to different file contents. They are not interchangeable. + +| Term | Exact meaning | +| --- | --- | +| **Named batch** | A saved set of changes identified by a user-supplied name | +| **Baseline** | The current commit, named `HEAD` by Git, when the batch is created. A repository without a commit uses Git's empty tree. | +| **Batch source** | A complete, stable snapshot of one file. Saved line numbers refer to this snapshot. The initial source normally comes from the session-start file. | +| **Current working tree** | The file on disk now. It may differ from both the baseline and the batch source. | +| **Batch ownership** | A `BatchOwnership` value containing the saved requirements for one text file | +| **Presence claim** | Batch-source lines that must be present after the saved change is applied | +| **Absence claim** | Baseline bytes that must be absent after the saved change is applied. The stored metadata key is `deletions`. | +| **Replacement unit** | Stored metadata that ties presence claims and absence claims together because they are the new and old sides of one replacement | +| **Ownership unit** | An `OwnershipUnit` value derived for display and selection. A replacement or deletion-only value must be selected in full. | +| **Realized batch content** | The complete file content produced from the baseline, batch source, and ownership. The source uses the word “realized” in names such as `realized_file_content.py`. | +| **Baseline reference** | Exact before-and-after baseline line positions and bytes recorded for a claim. Merge code uses them only when those positions and bytes still match. | +| **Attribution** | A calculated answer to which visible working-tree changes are already owned by which batches. It is recalculated; it is not stored as ownership. | +| **Display line identifier** | A number printed beside a changed line for a later user selection. It is not a batch-source line number. | +| **Merge candidate** | One numbered result calculated after ordinary merge cannot choose a single structural placement. A later command can name the reviewed result. | + +The phrase **batch-source line number** below always means a one-based line +number in the stable batch-source file. It never means a displayed line +identifier or a current working-tree line number. + +## One saved text change + +Suppose the current commit contains: ```text line1 @@ -458,7 +69,7 @@ line2 line3 ``` -During a session, the working tree becomes: +The working tree then becomes: ```text line1 @@ -467,393 +78,544 @@ line3 line4-new ``` -The user saves the modification and the new line into a batch. - -At that point: - -- baseline: the `HEAD` version with `line1 / line2 / line3` -- batch source: a stable snapshot that contains - `line1 / line2-modified / line3 / line4-new` -- ownership: claims the source lines for `line2-modified` and `line4-new`, plus - a deletion claim that says the original baseline `line2` should be absent -- realized batch content: the file view the batch itself stores, which is - effectively "baseline plus this batch's claimed changes" - -Later, if the user edits the file again, the batch does not forget what it saved. -Instead, `include --from`, `apply --from`, and `discard --from` interpret those -saved claims against the new working-tree state. That is why the source snapshot -and the ownership model exist in the first place. - ---- - -## Stale Batch Sources and Source Advancement - -The initial batch source comes from the session-start snapshot, but later -selections may reference working-tree lines that no longer exist in the current -source coordinate space. That is the stale-source problem. - -The authoritative repair path lives in -`src/git_stage_batch/batch/source_refresh.py`. - -### When a source is considered stale - -`detect_stale_batch_source_for_selection()` reports staleness when a selected -context or addition line cannot be expressed in source space, meaning its -`source_line` is `None`. - -### How source advancement works - -The stale-source repair path coordinates -`advance_batch_source_for_file_with_provenance()` in -`src/git_stage_batch/batch/ownership.py` with selection refresh in -`src/git_stage_batch/batch/source_refresh.py`: - -1. reads the old source content -2. reads the current working-tree content -3. builds a new synthetic source that preserves already-owned presence lines, - even if they have been removed from the working tree by earlier discard-to- - batch operations -4. records provenance maps while building that source: - - old source line -> advanced source line - - working-tree line -> advanced source line -5. creates a new batch source commit from that advanced content -6. remaps existing ownership into the new source coordinate space -7. returns the working-tree provenance map so the selected lines can be - re-annotated without matching synthesized source text again - -This is a major detail the old document missed: advanced sources are not -necessarily equal to the live working tree. They can intentionally carry forward -already-owned lines that are absent from the current file. - -The provenance maps are part of the safety model. Existing ownership is remapped -through the old-source map, while newly selected lines are re-annotated through -the working-tree map when that map is available. That avoids rediscovering line -identity from text in synthesized sources, especially when repeated lines would -make structural matching ambiguous. - -### Session cache - -Per-file source commits are cached for the active session in -`session-batch-sources.json`. `add_file_to_batch()` uses that cache so repeated -operations on the same file reuse the current source commit unless stale-source -repair advances it. - ---- - -## Display, Attribution, and Hidden Changes - -Two different mechanisms drive what the user sees: - -1. batch display reconstruction -2. attribution-based filtering of the live diff - -### Batch display reconstruction - -`show --from ` reconstructs a file's batch view from source content plus -ownership using `build_display_lines_from_batch_source_lines()` in -`src/git_stage_batch/batch/display.py`. - -For a single file, the command caches that reconstructed view as a selected -batch-file view and renders it through the page-aware file review output. For -multiple files, `show --from ` prints a navigational file list instead -of caching one hidden selected file. Pathless or omitted-path actions after -that list refuse until the user opens a specific file with -`show --from --file `. - -The reconstructed single-file view contains: - -- claimed lines -- deleted lines represented from deletion claims -- optional context -- ephemeral display IDs - -Those display IDs are not source line numbers. They are UI identifiers used for -line-level batch operations. Batch display has two related line-ID spaces: - -- mergeable gutter IDs, used by historical line actions when there is no fresh - matching file review -- review gutter IDs, used by page-aware file reviews - -The review gutter ID space can include resettable lines that are not currently -mergeable into the working tree. Review selections therefore persist -action-specific groups: `include --from`, `apply --from`, and `discard --from` -only accept groups that are mergeable for that action, while `reset --from` can -also accept reset-only groups. - -Page-aware batch reviews also persist short-lived safety state in -`data.file_review.state`, with record types in `data.file_review.records` and -batch review selection translation in `data.file_review.batch_selection`. That -state records the batch name, file path, shown pages, complete action-specific -review selections, and fingerprints of the selected batch view. Pathless line -actions from batch commands, the -corresponding omitted-path `--file` forms, and explicit `--file --line` -forms with a fresh matching review validate against this state so users cannot -accidentally act on unshown pages, stale display IDs, or a selection that is not -supported by the requested action. Whole-file batch actions may use the reviewed -file only after a fresh full-file review; partial reviews refuse until all pages -are shown or the file path is named explicitly. - -This is an important separation of concerns: - -- the display model is for presenting batch-owned content to the user -- the merge model is for satisfying ownership constraints against a target file - -The display model creates ephemeral IDs and groups adjacent visible lines in a -way that is useful for selection. The merge model works in source-space -coordinates and structural alignment. They are related, but they are not the -same layer and should not be treated as interchangeable. - -### Live `include --line` - -Live line inclusion uses the batch constraint model directly. The command creates -transient batch ownership for the selected live hunk lines, merges that ownership -into the index target with `merge_batch()`, and independently verifies that -merging the same ownership into the current working tree would leave the working -tree unchanged. The transient batch state is deleted before the command returns. +The user saves the modification and new line into a batch. The program records: -The transient ownership builder scans the full live hunk, not just the selected -rows, so unselected context can still provide source and baseline boundaries for -selected absence claims. Replacement coupling is supplied from before/after file -comparison as semantic replacement runs. It is not inferred from the displayed -diff layout: same-sized runs may expose positional row units, while 1-to-N and -N-to-1 replacements are represented as one run and only become a replacement -unit when the whole run is selected. +- the current commit as the baseline +- a batch source containing `line1`, `line2-modified`, `line3`, and + `line4-new` +- presence claims for the batch-source lines containing `line2-modified` and + `line4-new` +- an absence claim for the baseline bytes containing `line2` +- a replacement unit tying `line2-modified` to the absence of `line2`, when + the capture path can prove that relationship -If the transient batch path cannot prove that working-tree unchanged property, -the command refuses the line selection. It does not fall back to the older -semantic matcher or raw line staging path, because those paths can reinterpret -stale or ambiguous display IDs through a different model. +The realized batch content is the baseline with those saved requirements +satisfied. If the working tree changes again, the ownership does not change to +use new working-tree line numbers. Later commands translate the stored +batch-source line numbers into the current file only when the surrounding +content provides a safe placement. -### Attribution for the live working tree +## Where a batch is stored -Live `show` filtering uses file-centric attribution built in -`src/git_stage_batch/batch/attribution.py`. +A Git reference is a named pointer to a Git object. Current batches use two +references: -The attribution layer: +- `refs/git-stage-batch/batches/` points to a commit containing realized + batch files. +- `refs/git-stage-batch/state/` points to a commit containing validated + metadata and embedded batch-source files. -1. compares `HEAD:file` to the working tree -2. derives semantic change units -3. supplements that with stored batch-owned units that may no longer be visible - in the working tree -4. determines which batches currently own which working-tree fragments -5. projects that ownership back onto rendered diff lines +[`batch/ref_names.py`](src/git_stage_batch/batch/ref_names.py) defines those +names. [`batch/state_refs.py`](src/git_stage_batch/batch/state_refs.py) reads, +writes, and deletes them. -This is how already-batched content can be hidden from the interactive staging -pass while still allowing the user to keep editing nearby code. +The state commit contains: -### Consumed selections +- `batch.json`, which contains the batch name, schema version, revision, note, + creation time, baseline, content reference, content commit, and per-file + metadata +- `sources/`, which contains the embedded batch source for each file that + has one -The same attribution machinery also incorporates hidden consumed selections -recorded during the current session in -`src/git_stage_batch/data/consumed_selections.py`. +The revision changes each time `sync_batch_state_refs()` publishes state. The +function compares the revision it read with the current state reference before +updating both references. If another writer changed the batch first, publication +fails instead of overwriting that newer state. -Those hidden claims persist across `again` and participate in masking so the UI -does not keep surfacing changes the user already consumed. +Historical `refs/batches/` references and file-backed metadata are read as +migration input. A successful write through current code publishes both current +references and removes the historical storage for that batch. ---- +### Per-file metadata -## Semantic Units and Line-Level Batch Selection +[`batch/metadata_schema.py`](src/git_stage_batch/batch/metadata_schema.py) +validates stored fields before the rest of the program uses them. -Line-level batch operations are not raw line slicing. They are semantic-unit -selection derived from the reconstructed batch display described above. +A text-file entry normally contains: -### Ownership units - -For text batches, `select_batch_ownership_for_display_ids_from_lines()` uses -`build_ownership_units_from_batch_source_lines()` to group reconstructed display -lines into ownership units: - -- `PRESENCE_ONLY` -- `REPLACEMENT` -- `DELETION_ONLY` - -Grouping is based on adjacency in the reconstructed batch display, not simple -source-line proximity. - -### Atomicity rules - -`REPLACEMENT` and `DELETION_ONLY` units are atomic. If the user selects only part -of one, the command raises an `AtomicUnitError` and reports the required gutter -ID range. - -This prevents: - -- dropping the addition side of a replacement while leaving its deletion claim -- removing only part of a deletion block - -That same semantic-unit model is reused by `reset --from` so line-level resets -cannot create orphaned deletion claims. - ---- - -## Reset and Moving Claims Between Batches - -`reset --from ` removes claims from a batch. With `--to`, it moves those -claims into another batch instead of dropping them. - -Implementation: - -- `src/git_stage_batch/commands/reset.py` - -Key behaviors: - -- full-file resets remove that file from the batch -- line-level resets operate through ownership units, not raw line ranges -- `reset --from A --to B` requires compatible baselines -- moving claims between batches reuses the same batch source commit when - possible, and refuses incompatible source mismatches - -This keeps ownership moves lossless and prevents two batches from silently -interpreting the same line ranges in different source spaces. - ---- - -## Sift - -`git-stage-batch sift --from --to ` is not just "drop lines -already present at tip". It creates a new batch representation for the remaining -delta between the batch's target content and the current working tree. - -Implementation: - -- `src/git_stage_batch/commands/sift.py` -- `src/git_stage_batch/batch/comparison.py` - -For text files, the sifted batch intentionally uses different persistence -semantics than ordinary batches: - -- the synthetic batch source commit stores the target content directly -- the realized batch file also stores that target content directly -- ownership is then expressed in that target-content coordinate space so that - merging the sifted batch with the current working tree reproduces the intended - target - -This is deliberate and validated. A sifted batch is still a valid batch, but -its source snapshot is synthetic rather than derived from the session-start file -snapshot. - -The reason this matters is that sift is answering a different question from -ordinary batch creation. - -Ordinary batch persistence starts from "what did the user save out of the -session's file snapshot?" Sift starts from "what part of this batch's intended -result is still missing from the current tree?" Once that becomes the problem, -persisting the target content directly is the simplest faithful representation. - -So the sifted representation is different on purpose: - -- it keeps only the still-needed portion of the batch -- it re-expresses ownership relative to that reduced target -- it preserves the guarantee that merging the sifted batch should reconstruct - the intended remaining change - ---- - -## Binary Files - -Binary files are stored as atomic file changes. - -Implementation: - -- `add_binary_file_to_batch()` in - `src/git_stage_batch/batch/binary_file_storage.py` - -There is no line-level ownership for binary files. A binary batch entry records: - -- added / modified / deleted state -- source commit -- file mode - -Commands can move or apply those files as whole units only. - ---- - -## Important Invariants - -The current implementation relies on these invariants: - -1. Ownership is always interpreted in batch-source coordinate space. -2. The initial batch source is derived from the session-start snapshot, not from - an arbitrary later working-tree state, except that files absent at session - start use their current working-tree content. -3. Advanced batch sources may preserve already-owned lines that are absent from - the live working tree. -4. When source content is synthesized with known provenance, remapping and - selected-line re-annotation should treat those provenance maps as - authoritative. Text matching is a fallback only when no provenance map is - available. -5. The content ref and state ref must stay in sync. -6. Deletion claims are anchored structural constraints, not generic search and - delete instructions. -7. Line-level batch operations must preserve semantic atomicity. -8. Merge and discard favor refusal over ambiguous structural guesses. - -If any of these change, the command behavior and the safety model change with -them. - ---- - -## Key Code Paths - -Core batch state: - -- `src/git_stage_batch/batch/state_refs.py` -- `src/git_stage_batch/batch/query.py` -- `src/git_stage_batch/batch/content_commits.py` -- `src/git_stage_batch/batch/text_file_storage.py` -- `src/git_stage_batch/batch/binary_file_storage.py` -- `src/git_stage_batch/batch/gitlink_storage.py` -- `src/git_stage_batch/batch/file_entry_storage.py` -- `src/git_stage_batch/batch/realized_file_content.py` -- `src/git_stage_batch/batch/lifecycle.py` - -Ownership and repair: - -- `src/git_stage_batch/batch/ownership.py` -- `src/git_stage_batch/batch/source_refresh.py` -- `src/git_stage_batch/batch/source_cache.py` -- `src/git_stage_batch/batch/source_buffers.py` -- `src/git_stage_batch/batch/source_snapshots.py` -- `src/git_stage_batch/batch/selection.py` - -Merge / discard engine: +- `batch_source_commit` +- `source_path` in stored state +- `presence_claims` +- `deletions` +- `replacement_units` when a replacement relationship was recorded +- `mode` +- `change_type` only for a complete added or deleted path + +An absence claim stores the removed bytes in a Git blob and records that blob's +object identifier. It does not repeat those bytes inline in `batch.json`. + +Binary files, submodule pointers, and file mode changes use separate file types +and store only fields applicable to the complete file change. They do not use +text line ownership. + +## How a new batch is created + +`git-stage-batch new ` reaches +[`batch/lifecycle.py`](src/git_stage_batch/batch/lifecycle.py): + +1. `create_batch()` validates the name and refuses an existing batch. +2. It resolves the current commit. A repository without a commit uses Git's + empty tree. +3. It writes initial metadata with an empty `files` mapping. +4. It creates a content commit from the baseline tree. +5. `sync_batch_state_refs()` publishes the content and state references. + +The same module owns deletion and note updates through `delete_batch()` and +`update_batch_note()`. Read-only listing and metadata lookup live in +[`batch/query.py`](src/git_stage_batch/batch/query.py). + +## How session recovery includes batch changes + +A staging session includes batch references in its recovery state. + +- [`data/session.py`](src/git_stage_batch/data/session.py) calls + `snapshot_batch_refs()` when a session starts. +- [`data/batch_refs.py`](src/git_stage_batch/data/batch_refs.py) records every + current batch content and state reference. `abort` restores those references, + restores a dropped batch, and removes a batch created after session start. +- [`data/undo_checkpoints.py`](src/git_stage_batch/data/undo_checkpoints.py) + records references changed by one command. `undo` and `redo` restore only the + command checkpoints they traverse. + +An action that changes a batch must run inside the existing command checkpoint +flow. `abort` and `undo` serve different scopes: `abort` returns all batches to +session-start state, while `undo` reverses one recorded operation. + +## How `include --to` saves a live change + +For `git-stage-batch include --to `, execution crosses the following +modules: + +1. [`cli/selection_subcommands.py`](src/git_stage_batch/cli/selection_subcommands.py) + declares `--to`. +2. [`cli/include_dispatch.py`](src/git_stage_batch/cli/include_dispatch.py) + resolves file scope and calls `command_include_to_batch()`. +3. [`commands/include.py`](src/git_stage_batch/commands/include.py) validates + the repository, batch name, and file-review scope. +4. [`commands/selection/include_to_batch_action.py`](src/git_stage_batch/commands/selection/include_to_batch_action.py) + opens an undo checkpoint and routes text, binary, deletion, submodule + pointer, file mode, file-scoped, and selected-line forms to their owners. +5. A text selection reaches + `acquire_batch_ownership_update_for_selection()` in + [`batch/ownership_update.py`](src/git_stage_batch/batch/ownership_update.py). + That function refreshes a stale source when necessary, translates selected + lines into ownership, and combines new ownership with existing ownership for + the file. +6. `add_file_to_batch()` in + [`batch/text_file_storage.py`](src/git_stage_batch/batch/text_file_storage.py) + obtains the baseline and source files, builds realized content, updates the + content tree, writes metadata, and publishes both current references. +7. The command records the live hunk as handled and selects the next change + when automatic advancement is enabled. + +Whole-file binary, submodule pointer, and file mode changes use their specific +storage modules instead of `text_file_storage.py`. + +`discard --to ` records the same saved ownership but also removes the +selected content from the working tree. Its command path lives in the matching +discard modules under `commands/selection/` and `commands/file_scope/`. + +## How text ownership is stored + +[`batch/ownership.py`](src/git_stage_batch/batch/ownership.py) defines +`BatchOwnership` with three fields: + +- `presence_claims`: source line ranges and optional baseline references +- `deletions`: separate `AbsenceClaim` values containing an anchor and removed + baseline bytes +- `replacement_units`: optional links between presence ranges and entries in + `deletions` + +An absence anchor is either a batch-source line after which the baseline bytes +were removed or `None` for the beginning of the file. The anchor is a placement +boundary, not an instruction to search the whole file and delete the first +matching text. + +The metadata key remains `deletions` for compatibility. Code that loads it +constructs `AbsenceClaim` values because the stored requirement is “these exact +baseline bytes must be absent at this boundary.” + +[`batch/hunk_ownership_translation.py`](src/git_stage_batch/batch/hunk_ownership_translation.py) +and [`batch/ownership_translation.py`](src/git_stage_batch/batch/ownership_translation.py) +translate selected displayed lines into this stored form. The hunk translator +can use the complete old and new replacement run so it does not decide +replacement membership from display adjacency alone. + +## How the realized batch file is built + +`build_realized_buffer_from_lines()` in +[`batch/realized_file_content.py`](src/git_stage_batch/batch/realized_file_content.py) +receives: + +1. the baseline bytes +2. the batch-source bytes +3. `BatchOwnership` + +It selects every claimed source line, removes baseline sequences described by +applicable absence claims, and preserves the source line-ending style. The +result becomes the file stored in the batch content commit. + +Building stored content is less strict than merging into a current working +file. If an absence claim does not match its expected boundary while the stored +view is being built, the builder leaves the non-matching bytes alone. The +stored view describes what the batch owns. A later merge separately decides +whether that ownership can be placed safely in a current file. + +## How a saved batch is displayed + +`show --from ` reconstructs visible changes from the batch source and +ownership. It does not use the realized batch file as a ready-made patch. + +The main path is: + +1. [`commands/show_from.py`](src/git_stage_batch/commands/show_from.py) resolves + batch and file scope. +2. [`batch/display.py`](src/git_stage_batch/batch/display.py) builds changed + lines from source content and ownership. +3. [`batch/file_display_model.py`](src/git_stage_batch/batch/file_display_model.py) + prepares the complete review model for one file. +4. Modules under `data/file_review/` save the batch name, file path, shown + pages, permitted complete selections, and comparison values used to detect + whether the view later changed. +5. Modules under `output/` render that prepared review. + +Displayed line identifiers exist only to refer back to this saved view. Batch +operations translate them to complete ownership units before changing stored +ownership or applying a batch. + +A multi-file `show --from` prints a file list and clears the selected file. The +user must open a specific file before a later pathless action. A partial +single-file review permits only action groups completely shown on the selected +pages. These checks prevent an identifier from acting on an unseen or stale +line. + +## How `include --from` and `apply --from` add saved changes + +`include --from ` changes both the index and working tree. +`apply --from ` changes only the working tree. + +Their command modules are: + +- [`commands/include_from.py`](src/git_stage_batch/commands/include_from.py) +- [`commands/apply_from.py`](src/git_stage_batch/commands/apply_from.py) + +Both commands use modules under `commands/batch_source/` to: + +1. resolve the batch name or reviewed candidate +2. resolve file and displayed-line scope +3. reject a stale review or an incomplete ownership unit +4. build one action plan for each selected file +5. ask the batch merge code for target file content +6. write accepted targets +7. refresh review and selected-change state + +For text files, [`batch/merge.py`](src/git_stage_batch/batch/merge.py) receives +the batch source, ownership, and current target bytes. It tries to satisfy the +presence and absence requirements against that target. + +### Checks before merge writes content + +The merge code requires enough exact structure to place every missing claimed +line and every absence claim: + +- an already-present claimed line may remain where it is +- a missing claimed run needs surrounding mapped content or an exact recorded + baseline boundary +- an absence anchor must still identify the intended structural boundary +- a replacement fallback requires the recorded baseline bytes at the recorded + baseline position + +[`batch/merge_validation.py`](src/git_stage_batch/batch/merge_validation.py) +owns structural validation. Helpers called by `merge.py` separate three kinds +of exact fallback work: + +- [`batch/baseline_edits.py`](src/git_stage_batch/batch/baseline_edits.py) + applies replacement edits only at matching baseline positions. +- [`batch/absence_constraints.py`](src/git_stage_batch/batch/absence_constraints.py) + resolves which matching baseline bytes an absence claim may suppress. +- [`batch/presence_constraints.py`](src/git_stage_batch/batch/presence_constraints.py) + places missing claimed content. + +If the required boundary is missing or several placements are possible, the +operation refuses. It does not choose the first similar text. That refusal +prevents saved content from being inserted into or removed from the wrong part +of a file. + +Applying a batch to an unchanged copy of its batch source also does not add the +same content twice. The merge checks whether claimed content and required +absences are already satisfied before writing. + +## How `discard --from` removes saved changes + +`discard --from ` removes the selected batch's effect from the working +tree and does not change the index. + +[`commands/discard_from.py`](src/git_stage_batch/commands/discard_from.py) uses +the same action-context, selection, planning, refusal, and completion modules as +the other batch-source actions. Text reversal reaches the discard functions +under `batch/`. + +The reversal code compares the batch source with the baseline. It determines +whether each region is unchanged, inserted, a line-for-line replacement, or a +replacement that must be handled as one complete hunk. The implementation uses +the enumeration values `EQUAL`, `INSERT`, `REPLACE_LINE_BY_LINE`, and +`REPLACE_BY_HUNK` for those cases. -- `src/git_stage_batch/batch/merge.py` -- `src/git_stage_batch/batch/match.py` +Start with these files when following that reversal: -Display and attribution: +- [`batch/baseline_correspondence.py`](src/git_stage_batch/batch/baseline_correspondence.py) + maps batch-source regions back to baseline regions. +- [`batch/discard_reversal.py`](src/git_stage_batch/batch/discard_reversal.py) + reverses presence requirements. +- [`batch/realized_boundaries.py`](src/git_stage_batch/batch/realized_boundaries.py) + checks exact boundaries in the current realized sequence. + +The same refusal rule applies: when the current file no longer provides one +safe reversal, the command stops without guessing. + +## What happens when the working file changes + +Saved ownership uses batch-source line numbers. A later selection may contain a +working-tree line that has no corresponding line in that older source. In the +selected `LineEntry`, that condition appears as `source_line is None`. -- `src/git_stage_batch/batch/display.py` -- `src/git_stage_batch/batch/attribution.py` -- `src/git_stage_batch/batch/comparison.py` +`ensure_batch_source_current_for_selection()` in +[`batch/source_refresh.py`](src/git_stage_batch/batch/source_refresh.py) handles +that condition: -Session coupling: +1. It reads the old batch source and current working file. +2. [`batch/source_advancement.py`](src/git_stage_batch/batch/source_advancement.py) + constructs a new source. It preserves previously claimed lines even when an + earlier `discard --to` removed them from the working file. +3. While constructing that source, it records two exact line maps: old source + line to new source line, and working-tree line to new source line. +4. It creates a new source commit. +5. It remaps existing ownership with the old-source map. +6. It annotates the new selection with the working-tree map. +7. It updates the active session's per-file source cache. -- `src/git_stage_batch/data/session.py` -- `src/git_stage_batch/data/batch_refs.py` -- `src/git_stage_batch/data/consumed_selections.py` +The constructed source is therefore not always identical to the working file. +The two maps record where every carried or current line came from. Text matching +is used only when the construction path did not provide one of those maps. + +Initial source loading, storage, and caching are split across: + +- [`batch/source_buffers.py`](src/git_stage_batch/batch/source_buffers.py) for + session-start file buffers +- [`batch/source_snapshots.py`](src/git_stage_batch/batch/source_snapshots.py) + for source commits +- [`batch/source_cache.py`](src/git_stage_batch/batch/source_cache.py) for the + active session's per-file source commit mapping + +A file absent at session start uses its current working-tree content for the +initial source so every newly claimed line exists in that source. + +## Why some displayed lines must be selected together + +Line-level batch actions do not remove arbitrary individual metadata rows. +[`batch/ownership_units.py`](src/git_stage_batch/batch/ownership_units.py) +builds `OwnershipUnit` values from the reconstructed display. The code records three +kinds in [`batch/ownership_unit_types.py`](src/git_stage_batch/batch/ownership_unit_types.py): + +- `PRESENCE_ONLY`: claimed source lines with no coupled absence claim +- `REPLACEMENT`: claimed source lines coupled to one or more absence claims +- `DELETION_ONLY`: one or more absence claims with no claimed source line -User-facing commands: +A replacement or deletion-only ownership unit must be selected in full. Partial +selection would leave only one side of a replacement or only part of one +removed baseline sequence. The command reports the full displayed identifier +range that must be selected. -- `src/git_stage_batch/commands/include_from.py` -- `src/git_stage_batch/commands/apply_from.py` -- `src/git_stage_batch/commands/discard_from.py` -- `src/git_stage_batch/commands/reset.py` -- `src/git_stage_batch/commands/sift.py` +`reset --from` uses the same ownership units, so removing ownership cannot leave a saved +absence without its coupled presence or leave a partial deletion claim. ---- +## How live changes are hidden after they are saved + +The live `show` command must avoid presenting changes already owned by a batch. +[`batch/attribution.py`](src/git_stage_batch/batch/attribution.py) calculates +that answer for one file: -## Summary +1. compare the baseline file with the current working file +2. divide that comparison into changed regions +3. load saved ownership that can affect the file +4. determine which batch owns each visible changed region +5. project the answer onto displayed diff lines -The current batch system is a Git-backed, constraint-based persistence layer on -top of the interactive staging workflow. Its key design choices are: +[`batch/attribution_projection.py`](src/git_stage_batch/batch/attribution_projection.py) +owns the final projection onto line entries. Selected-change filtering under +`data/selected_change/` removes fully owned lines or hunks before display. -- authoritative content and state refs under `refs/git-stage-batch/*` -- ownership stored in source-space, not as replayable patches -- session-start source snapshots with explicit stale-source advancement -- structural merge/discard with conservative failure modes -- attribution and semantic-unit filtering to keep the UI coherent as the working - tree evolves +The same calculation includes consumed selections from +[`data/consumed_selections.py`](src/git_stage_batch/data/consumed_selections.py). +Those records hide changes already handled during the current session even when +they are not persistent named-batch ownership. + +Ownership and attribution answer different questions: + +- ownership records what a batch saved in batch-source line numbers +- attribution calculates which current visible changes satisfy that saved + ownership now + +## Why `include --line` creates a temporary batch + +Live selected-line inclusion uses the same ownership and merge checks without +leaving a named batch behind. + +[`commands/selection/include_line_selection.py`](src/git_stage_batch/commands/selection/include_line_selection.py): + +1. creates a uniquely named temporary batch +2. translates the complete live hunk and selected displayed line identifiers + into ownership +3. asks `batch/merge.py` to apply that ownership to the current index content +4. asks the same merge code to apply it to the current working-tree content +5. accepts the index result only when the working-tree result is byte-for-byte + unchanged +6. deletes the temporary batch and restores the session source cache before + returning + +This path ensures that selected-line staging and named-batch merge agree about +replacements and structural placement. Whole-hunk inclusion does not use this +path. + +## How ownership is removed or moved + +`reset --from ` removes selected ownership from a batch. +`reset --from --to ` moves it to another batch. + +[`commands/reset.py`](src/git_stage_batch/commands/reset.py) owns the command +sequence. Selection uses the complete ownership units described above. + +- A complete-file reset removes that file from the source batch. +- A selected-line reset removes complete ownership units. +- A move requires compatible baselines. +- A move reuses the same batch source when possible and refuses incompatible + source files. + +These checks prevent one saved line number from being interpreted against two +different source files. + +## How `sift` rewrites a batch for the current working tree + +`sift --from --to ` keeps only the portion of a saved +batch still missing from the current working tree. + +The command starts in +[`commands/sift.py`](src/git_stage_batch/commands/sift.py). Comparison lives in +[`batch/comparison.py`](src/git_stage_batch/batch/comparison.py). Persistence of +the result is split between +[`commands/batch_transform/sift_results.py`](src/git_stage_batch/commands/batch_transform/sift_results.py) +and [`commands/batch_transform/sift_persistence.py`](src/git_stage_batch/commands/batch_transform/sift_persistence.py). + +For a text file, the destination batch uses constructed source content equal to +the saved target content. Ownership is expressed in that constructed source's +line numbers. This differs from ordinary capture, whose initial source normally +comes from the session-start file. + +The difference is required by the command's input: + +- ordinary capture asks which live change the user selected +- `sift` asks which part of an existing saved result is not present now + +Merging the destination batch with the current working file must produce the +still-missing target content. + +## Complete-file changes + +Text line ownership does not represent every Git change. + +- [`batch/binary_file_storage.py`](src/git_stage_batch/batch/binary_file_storage.py) + stores binary additions, modifications, and deletions as complete-file + changes. +- [`batch/gitlink_storage.py`](src/git_stage_batch/batch/gitlink_storage.py) + stores submodule pointer changes. +- [`batch/file_mode_storage.py`](src/git_stage_batch/batch/file_mode_storage.py) + stores executable-mode changes. +- [`batch/file_entry_storage.py`](src/git_stage_batch/batch/file_entry_storage.py) + copies and removes generic stored entries. + +Line options cannot select part of these changes. + +## Where to make a batch change + +| Change | Owning code | +| --- | --- | +| Create or delete a batch, or update its note | `batch/lifecycle.py` | +| Validate a batch name | `batch/validation.py` | +| List batches or read metadata | `batch/query.py` | +| Validate or encode stored metadata | `batch/metadata_schema.py` | +| Publish or delete Git references | `batch/state_refs.py` | +| Persist text content and ownership | `batch/text_file_storage.py` | +| Persist binary, submodule pointer, or mode content | The matching `*_storage.py` module | +| Build the stored text file | `batch/realized_file_content.py` | +| Translate a live selection into ownership | `batch/hunk_ownership_translation.py` or `batch/ownership_translation.py` | +| Combine a new selection with stored ownership | `batch/ownership_update.py` and `batch/ownership_merging.py` | +| Refresh an old source | `batch/source_refresh.py` and `batch/source_advancement.py` | +| Display a saved file | `batch/display.py` and `batch/file_display_model.py` | +| Apply saved text to a current target | `batch/merge.py` and the validation and constraint helpers it calls | +| Remove saved text from a current working file | The correspondence, reversal, and boundary modules named in the discard section | +| Decide which lines must be selected together | `batch/ownership_units.py` and the ownership-unit support modules | +| Hide already-saved live changes | `batch/attribution.py` and `batch/attribution_projection.py` | +| Coordinate an action from a saved batch | `commands/batch_source/` | +| Move or remove ownership | `commands/reset.py` plus ownership modules | +| Rewrite only the still-missing saved result | `commands/sift.py`, `commands/batch_transform/`, and `batch/comparison.py` | + +Do not put session storage in `batch/`. The architecture test +`test_batch_package_stays_below_workflow_data` requires `batch/` not to import +`data/`. Command modules may coordinate both packages. + +## Add or remove a batch feature + +For a new user-visible batch behavior: + +1. Add parser options under `cli/batch_subcommands.py` for a batch command, or + under `cli/selection_subcommands.py` for a new `--from` or `--to` form. +2. Add or update the command entry under `commands/`. Use + `commands/batch_source/` when the operation consumes a saved batch and needs + shared context, selection, planning, refusal, or completion behavior. +3. Change `batch/` only when the stored representation, source translation, + ownership, display, comparison, merge, or reversal rule changes. +4. If stored metadata changes, update schema validation and migration before + writing the new field. Keep older stored batches readable or reject them + with an explicit version error. +5. Update the manual page, `docs/batches.md`, `docs/commands.md`, shell + completion, and interactive batch menus when they expose the behavior. +6. Add focused tests under `tests/batch/`, command tests under + `tests/commands/`, and a functional test when the behavior spans several + invocations or recovery operations. + +When removing a batch behavior, remove interactive and parser callers first, +then the command sequence, then unreferenced batch functions. Do not remove a +metadata reader merely because current code no longer writes that field; stored +batches may still contain it. Search manual pages, website documentation, +completion, assistant assets, state migration, and tests for both the command +spelling and the stored field name. + +## Rules protected by tests + +The implementation and architecture tests rely on these rules: + +1. Every text ownership line number refers to the file's batch source. +2. The initial batch source comes from the session-start file, except for a file + that did not exist then. +3. A refreshed source may retain previously claimed lines that are no longer in + the working tree. +4. When source construction returns exact origin maps, remapping uses those + maps. Text matching is only a fallback when a map is unavailable. +5. The content reference and state reference are published as one checked + update. +6. An absence claim applies at its stored structural boundary, not at the first + equal text anywhere in the file. +7. A replacement or deletion-only ownership unit cannot be partly selected. +8. Merge and discard stop when one safe placement or reversal cannot be + established. +9. Modules under `batch/` do not import workflow storage from `data/`. +10. Imports among modules under `batch/` do not form a cycle. + +Run the focused architecture and batch tests with: + +```console +uv run pytest -n auto tests/architecture/test_import_boundaries.py tests/batch +``` -That is the model contributors should use when reasoning about bugs or adding -new batch features. +Run affected command and functional tests whenever a change crosses the +`batch/` boundary into user-visible behavior. diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 1ed98fe67..7c005d5d0 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -15,7 +15,7 @@ This project uses [uv](https://docs.astral.sh/uv/) for development workflow and # Install uv if you haven't already curl -LsSf https://astral.sh/uv/install.sh | sh -# Install meson and ninja (example for Fedora/RHEL) +# Install meson and ninja (example for Fedora or Red Hat Enterprise Linux) sudo dnf install meson ninja-build # Clone the repository @@ -33,6 +33,20 @@ Use the xdist form (`-n auto`) for full-suite runs. The suite is large enough that serial `uv run pytest` is mainly useful for focused debugging or when reproducing ordering-sensitive failures. +## Find the Code for a Change + +Read [the codebase guide](ARCHITECTURE.md) before changing source. It follows +real commands from argument parsing through command behavior, persisted state, +Git updates, and output. It also lists the files that normally change when a +command or option is added or removed. + +Do not begin with `src/git_stage_batch/batch/` for ordinary command or session +work. That package is documented separately because its saved-change storage +and merge rules are much larger than the introductory path. Read +[the batch internals guide](BATCHES.md) when changing a command that uses +`--to` or `--from`, saved ownership, batch display, batch merge, source refresh, +or the temporary batch merge used by `include --line`. + ## Commit Message Guidelines We follow strict commit message conventions to maintain a clear and understandable project history. @@ -53,7 +67,7 @@ We follow strict commit message conventions to maintain a clear and understandab - **If the commit is a step toward a larger feature, say so explicitly.** Describe the end goal briefly, then explain how this commit moves toward it. - **Name the feature goal in early groundwork commits.** If a commit mainly exists to enable a later user-facing feature, say what that feature is and why it matters instead of presenting the commit as isolated infrastructure work. - **Prefer concrete limitations over vague judgments.** Avoid words like "cumbersome", "better", or "improved" without explaining why. -- **Do not use `Co-Authored-By` for contributions produced from AI.** Only use it for human co-authors. +- **Do not use `Co-Authored-By` for contributions produced by artificial intelligence.** Only use it for human co-authors. - **Only use the word `this` when referring to the commit itself.** Use `that` or similar for other contexts. - **Wrap body paragraphs at 75 characters.** - **Be humble and forward thinking.** Avoid words like "comprehensive" or "crucial", and avoid a tone that could sound like bragging or seem short-sighted. @@ -125,7 +139,7 @@ project does not provide generic helpers for transformed selections." Describe how the commit addresses one part of that problem. Be precise about scope. If the commit only addresses one path (such as the man -page, CLI help, or internal structure), say so clearly rather than implying +page, command help, or internal structure), say so clearly rather than implying the entire problem is solved. If the commit introduces infrastructure or an early step toward a larger @@ -186,8 +200,9 @@ Before finalizing a commit message, check: ``` cli: Add --verbose flag for detailed output -The CLI currently provides minimal feedback during operation, only showing -the selected hunk without any indication of progress or internal state. +The command-line interface currently provides minimal feedback during +operation, only showing the selected hunk without any indication of progress +or internal state. Users working with large changesets cannot easily determine how much work remains or what has already been processed, making it difficult to gauge @@ -208,7 +223,7 @@ Notice how the first paragraph evolves to reflect the cumulative state, and how i18n: Add Spanish translation (es) The program has gettext infrastructure in place but only contains -English messages in the POT template. +English messages in the translation template. Without translations, the program cannot serve non-English speaking users. Spanish is one of the most widely spoken languages globally. @@ -280,12 +295,13 @@ The code currently provides minimal output... ❌ **Don't describe the change in the first paragraph:** ``` -This commit adds verbose output to the CLI... +This commit adds verbose output to the command-line interface... ``` ✅ **Do describe what exists today:** ``` -The CLI currently provides minimal feedback during operation... +The command-line interface currently provides minimal feedback during +operation... ``` ❌ **Don't confuse a symptom with the real problem:** @@ -311,18 +327,18 @@ Without an organized directory, the code may become harder to maintain. ✅ **Do describe the missing capability:** ``` -The project does not yet provide a TUI for interactive use. +The project does not yet provide an interactive terminal interface. ``` ❌ **Don't use vague value judgments:** ``` -The CLI is cumbersome to use. +The command-line interface is cumbersome to use. ``` ✅ **Do describe concrete limitations:** ``` -The CLI requires repeated command invocation and does not provide a -continuous hunk-by-hunk workflow. +The command-line interface requires repeated command invocation and does not +provide a continuous hunk-by-hunk workflow. ``` ❌ **Don't overstate the impact of the commit:** diff --git a/README.md b/README.md index 721a216c3..99d487a3d 100644 --- a/README.md +++ b/README.md @@ -187,6 +187,8 @@ pip install git-stage-batch Contributions are welcome! Please see [CONTRIBUTING.md](CONTRIBUTING.md) for: - Development setup instructions +- A [codebase guide](ARCHITECTURE.md) with command paths and change checklists +- A separate [batch internals guide](BATCHES.md) for saved-change storage and merge behavior - Commit message guidelines - Code style conventions