fix: review the change that was made, not the one that was proposed - #134
Merged
Conversation
`Executor` showed the reviewer the diff text the model produced. The tolerance ladder exists to rescue malformed patches, so that text and the change it produces are routinely different: a `@@ -0,0` hunk git apply would refuse becomes, under --unidiff-zero, a real insertion with real context. Reviewing the text had both failure directions. Good work was rejected for an artefact of the plumbing -- the reviewer said, correctly about what it was shown and falsely about what happened, that it "could not verify that existing functions were kept untouched, because both files are shown as being created from empty". And a diff that claimed more than it did would have been reviewed as truth, which is the one thing the review prompt exists to catch. `SessionExecutor` already reviewed `git diff HEAD`. This makes the two agree. Live effect on the same three-item backlog: an item completed end to end for the first time, and the two that still fail now fail on their merits -- the reviewer can see that a rescued hunk landed above the file's existing imports, which is a real defect (#133) that the proposed diff could not express.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #132.
Executorpassed the model's own diff text to the reviewer.SessionExecutoralready passed
git diff HEAD. The plain one was the wrong way round.This matters because the tolerance ladder's whole purpose is to rescue patches
whose text is malformed — so the text and the resulting change are routinely
different.
Evidence, from a live run
Model's text (what the reviewer used to see):
What actually landed:
The reviewer rejected the item because it "could not verify that existing
functions and tests were kept untouched, because both files are shown as being
created from empty". Correct about the text; false about the change. The
checks had already passed against the applied tree, including the pre-existing
tests — the harness had proof the file was intact and showed the reviewer
something else.
Live effect
Same three-item backlog, before and after:
then failed review on the artefact.
review → commit — and the remaining two now fail on their merits: the
reviewer can see that a rescued hunk landed above the file's existing
imports. That is Bug: a zero-context hunk is applied at line 1, putting code above module docstrings while checks stay green #133, a real defect the proposed diff could not express.
Rejecting good work was the visible half. The invisible half is worse: a gate
told to catch a change that "claims more than it did" could not, because it
was reading the claim.
Tests
Two, both failing without the change: the reviewer's prompt must contain the
surviving context and must not contain the proposed
@@ -0,0header; and theprompt must show where a rescued hunk landed relative to existing content.
548+ tests, ruff,
mypy .green.