fix(core): prune stale changed_files rows on refresh (#100)

[josephschmitt]

Bug

In repos with many untracked files (e.g. a .venv not yet in .gitignore, thousands of files), Monocle freezes/slows and the changed_files table fills with thousands of rows. After adding the directory to .gitignore the TUI recovers, but stale rows remain in changed_files. Likewise, deleting untracked files leaves their rows behind.

Root cause

SessionManager.RefreshChangedFiles (internal/core/session.go) runs every 2s. It computed the current changed set via git.Diff() and UPSERTed every file (UpsertChangedFile — an additive INSERT ... ON CONFLICT DO UPDATE), but never deleted rows whose paths were no longer in the current set. DeleteChangedFiles existed but was only called from tests, so stale rows persisted forever.

Secondary perf issue: each file was a separate auto-committed write every 2s (WAL, no batching) — a write storm under thousands of files.

Fix (sync + batched transaction)

Added DB.ReplaceChangedFiles(sessionID, files) (internal/db/queries.go): within a single *sql.Tx it DELETEs the session's changed_files rows then batch-INSERTs the current set (carrying each file's already-merged Reviewed state), committing once and rolling back on error.

RefreshChangedFiles now reads existing reviewed state from the DB (as before), merges it onto the current set, and calls ReplaceChangedFiles once instead of looping per-file upserts. This:

• removes rows for files no longer in the changed set (deleted / gitignored / reverted untracked files), and
• collapses the per-cycle writes into one transaction.

The engine's post-refresh snapshot path (Engine.filesRelativeToSnapshot in engine_impl.go) runs after RefreshChangedFiles returns and re-adds reverted-but-in-snapshot files via its own upsert, so that behavior is preserved — the prune happens before, not inside, that path.

Tests

• internal/db/db_test.go — TestReplaceChangedFiles: seed 3 files (one reviewed), replace with a 2-file set → third row pruned, surviving reviewed flag preserved.
• internal/core/session_test.go:
  • TestRefreshChangedFiles_PrunesStaleRows (gitStub): A,B,C changed → 3 rows; C leaves the set → C's row gone, A,B retain reviewed state.
  • TestRefreshChangedFiles_GitignorePrune (setupTestRepo + real git client): untracked file row appears, add to .gitignore, refresh → row removed.

All of go build ./..., go test ./internal/..., and go vet ./... pass.

Closes #100

🤖 Generated with Claude Code

[greptile-apps]

Greptile Summary

Fixes stale changed_files rows that accumulated when untracked files were deleted or gitignored, and collapses the per-file write storm into a single transactional replace. Also resolves the previously-flagged reviewed-state regression by threading the pre-replace reviewed map (priorReviewed) through to filesRelativeToSnapshot, so reverted files re-added after the delete retain the user's reviewed flag.

• ReplaceChangedFiles (internal/db/queries.go): new method that DELETE-then-batch-INSERTs the session's changed_files set in one *sql.Tx, rolling back on any error.
• RefreshChangedFiles (internal/core/session.go): captures the pre-replace reviewed state, calls ReplaceChangedFiles once instead of looping UpsertChangedFile, and returns the captured map as a second return value.
• filesRelativeToSnapshot (internal/core/engine_impl.go): accepts priorReviewed map[string]bool and uses it when re-inserting reverted files, preserving reviewed state that the replace deleted.

Confidence Score: 5/5

Safe to merge. The transactional replace is correctly implemented, the previously-flagged reviewed-state loss for reverted files is resolved, and the new test suite covers the core scenarios end-to-end.

The change is narrowly scoped: one new DB method with a straightforward DELETE+INSERT transaction, a signature change on RefreshChangedFiles to return the pre-replace reviewed map, and a one-parameter addition to filesRelativeToSnapshot that uses it. Reading from a nil map in Go is safe, so existing test call-sites that pass nil are correct. No new data races or loss scenarios were found.

No files require special attention.

Important Files Changed

Filename	Overview
internal/db/queries.go	Adds ReplaceChangedFiles: atomically DELETE + batch-INSERT within a single transaction; deferred Rollback + Commit pattern is correct.
internal/core/session.go	RefreshChangedFiles now returns the pre-replace reviewed map alongside the file list; switches from per-file UpsertChangedFile to a single ReplaceChangedFiles call.
internal/core/engine_impl.go	Threads priorReviewed through filesRelativeToSnapshot so reverted-file rows re-inserted after the transactional replace preserve the user's reviewed flag; nil-map read in Go is safe for callers that pass nil.
internal/core/engine_test.go	Adds regression test TestFilesRelativeToSnapshot_RevertedFilePreservesReviewedState; updates three existing call-sites to pass nil for priorReviewed.
internal/core/session_test.go	Adds two new tests covering stale-row pruning and .gitignore-triggered prune; updates existing call-sites for the new return signature.
internal/db/db_test.go	Adds TestReplaceChangedFiles verifying the delete+insert behaviour, stale-row pruning, and reviewed-flag preservation.

Sequence Diagram

sequenceDiagram
    participant BG as Background ticker (2s)
    participant E as Engine.RefreshChangedFiles
    participant SM as SessionManager.RefreshChangedFiles
    participant DB as DB
    participant Git as git.Diff

    BG->>E: tick
    E->>SM: RefreshChangedFiles(session)
    SM->>Git: Diff(baseRef)
    Git-->>SM: currentFiles[]
    SM->>DB: GetChangedFiles(sessionID)
    DB-->>SM: existingRows (priorReviewed map)
    note over SM: merge reviewed flags onto currentFiles
    SM->>DB: ReplaceChangedFiles(sessionID, ptrs) BEGIN TX / DELETE all rows / INSERT currentFiles / COMMIT
    DB-->>SM: ok
    SM-->>E: files, priorReviewed, nil
    E->>E: autoUnmarkChangedFiles(files, latestSnap)
    alt "reviewBase != nil"
        E->>E: filesRelativeToSnapshot(session, files, reviewBase, priorReviewed)
        note over E: for each reverted file: Reviewed = priorReviewed[path]
        E->>DB: UpsertChangedFile(revertedFile)
    end
    E-->>BG: updatedFiles

Loading

_{Reviews (2): Last reviewed commit: "fix(core): preserve reviewed state for r..." | Re-trigger Greptile}