Add git_numstat backend (RepoState): session-retrospective change digest - #18
Merged
Conversation
New adapters/git_numstat/ (reader + mapping.py + backend.py), registered via
one import line in server/app.py, plus DOMAIN_REPO_CHANGE in core/metrics.py
and a PROFILER_TOOLS entry ("git_numstat": "git") in platform/detect.py.
Owner semantics kept strict: RepoState is a token-efficient BACKWARD look at
what a session did to a repo (reorientation without re-reading raw diffs)
and the provenance axis pinning perf/build/CI reports to code states — NOT
GitHub/API monitoring, and facts only, never verdicts (no readiness
judgments baked in; ranking by lines_added is the model reading a fact).
Artifact-first: binds to a SAVED `git diff --numstat -M <base>.. >
session.numstat` file. Line grammar verified against REAL git 2.53 output
before writing the parser (the real-report lesson): renames arrive as
whole-path `old => new` OR braced `pre{old => new}post` with either brace
side possibly empty ({ => api}); empty-side rebuilds collapse the resulting
'//' exactly as git's own display does; a renamed unit is NAMED BY ITS NEW
PATH (the reorientation answer) with old_path/renamed facts in expand.
Non-ASCII paths arrive C-quoted by git and are kept verbatim (honest to the
artifact). duration_us is None for every unit — a change digest has no time
dimension — so summarize_report falls back to file order, asserted.
Headline honesty case, end-to-end through get_metrics: a BINARY file prints
'-<TAB>-' — no line counts exist, so lines_added/deleted surface as
'not_available_in_this_export', never 0.0 — while a pure no-edit rename is
a genuine measured 0.0. Also observed and documented: git splits a rename
into add+delete below its similarity threshold; that split is git's real
answer, not a parse bug (probe note + test docstring).
Fixtures, both genuinely produced: (1) repo_session_early/…_sample.numstat —
real captures on THIS repo of v1.1.1..499fb2c and v1.1.1..a82e7fd (the v1.2
lane's own footprint; paths+counts only, scanned); the compare test shows
gha_log/backend.py growing 96->110 added lines between the two merges — the
session's own history read back through the tool. (2)
repo_lab_renames_sample.numstat — real capture from a throwaway lab repo
with real commits engineered to exhibit every numstat shape at once
(binaries, all rename forms, space path, C-quoted Turkish filename).
Loud ValueError on non-numstat input including the empty-diff-empty-file
case, and on MIXED content (numstat pasted into a bigger log) — partial
digests never happen silently. 14 new tests; full suite 182 passed + 16
skipped (baseline 168 + 16).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
TASK 3, closing the schematic: the
git_numstatbackend (RepoState family). Newsrc/perfdigest/adapters/git_numstat/(reader +mapping.py+backend.py),DOMAIN_REPO_CHANGEincore/metrics.py, one import line inserver/app.py, onePROFILER_TOOLSentry ("git_numstat": "git").Owner semantics, kept strict
git diff --numstat -M <base>.. > session.numstatfile — the raw diff is the token sink, numstat is its digest. An agent reorienting after compaction reads the footprint in onesummarize_reportcall instead of re-reading diffs.duration_usisNonefor every unit (a change digest has no time dimension), sosummarize_reportfalls back tofile_order— asserted — and size ranking happens by the model readinglines_added, not by a hotness heuristic we bake in.Real-format verification (the real-report lesson, again load-bearing)
The parser grammar was written FROM real
git diff --numstat -Moutput (git 2.53, throwaway lab repo with real commits), not from documentation. Real findings baked into reader + tests:NOTES => TASKS.md, bracedsrc/{parser.py => parser_v2.py}/{src => docs}/util.py, and empty-side bracesdocs/{ => api}/config.py(rebuilding a path over the empty side can produce//, which git's own display collapses — so we do too). Renamed units are named by their NEW path (the reorientation answer);old_path/renamedfacts are inexpand."docs/\303\266l\303\247\303\274m.txt", core.quotePath default — a real case on any Turkish-named file) and are kept verbatim as printed: the name is honest to the artifact, we don't unquote.-Mon modern git (diff.renamesdefault), but the capture command still advises-Mexplicitly for portability.Headline honesty case
A binary file prints
-<TAB>-: no line counts exist.lines_added/lines_deletedsurface as'not_available_in_this_export'end-to-end throughget_metrics— never 0.0 — while a pure no-edit rename is a genuine measured0.0. Both directions tested side by side (the dash and the zero must never blur).Fixtures (all genuinely produced, none hand-written)
repo_session_early_sample.numstatv1.1.1..499fb2c(27 files — the lane after the gha_log merge)repo_session_sample.numstatv1.1.1..a82e7fd(46 files — after the previous-green merge)repo_lab_renames_sample.numstatBoth session fixtures are paths+counts only, scanned before committing. The meta touch: the compare test reads the lane's own history through its own tool —
adapters/gha_log/backend.pygrew 96 → 110 added lines between the two snapshots (task 2's usage-prompt extension), delta+14.0withdelta = b - a.Error behavior
Loud, named
ValueErroron: prose, JSON, the empty file (an empty diff writes an empty file — "if the session made no changes there is nothing to digest" is said out loud, not returned as a silent[]), and mixed content (a numstat pasted into a bigger log would digest partially — refused loudly instead of silently dropping lines).Test evidence
14 new tests in
tests/test_git_numstat.py: registry dispatch (both formats, domain, platforms, suffixes); real session fixture parse (46 units, file order, all durations None); per-file metrics vs merged reality; summarize file-order fallback with explicit metrics arg; two-snapshot footprint evolution viacompare_metrics; full lab listing (all 11 shapes); binary None honesty end-to-end; genuine-zero rename contrast; all four rename resolutions with old_path in expand; binary/printed-path facts in expand; mixed add+delete counts; three non-numstat error cases; mixed-content loudness; capture-command shape (range-based, file-bound, uncommitted variant advertised).Full suite after rebasing onto main (which had merged cargo_diag #16 meanwhile; conflicts in metrics.py/detect.py resolved keeping both backends): 196 passed, 16 skipped (baseline 168 + 14 cargo_diag + 14 here). Ruff clean. No edits to
pyproject.tomlversion,README.md,CLAUDE.md.Test plan
uv run --extra dev pytest -q— 196 passed, 16 skippedruff checkon new files — cleansuggest_profile_command('git_numstat', 'v1.1.1')returns the range capture command with the uncommitted-only variant;platform_capabilitieslists git_numstat as universally digestiblegit remote -vverified before push (github.com/onlyxItachi/PerfDigest-MCP)🤖 Generated with Claude Code