Recover the removal a compound take-off loses - #2
Open
GetBeholder wants to merge 1 commit into
Open
Conversation
Same fix as GetBeholder/Beholder-ME#2; this extractor carries the identical five-pass path. When one sentence both takes a garment off and puts another on, the trained extractor emits one half and drops the other: 2/8 on compound prose against 3/3 when the take-off stands alone. The lost garment stays in state and is fed back into the next prompt, so one miss compounds for the rest of the scene. Re-asking the SAME compound prose with removal-only framing recovers nothing (0 of 6) — the compound sentence itself is what blinds the model. Re-asking with just the take-off clause recovers all of them. So when the five-pass reply carries no worn_remove, the prose shows something coming off, and the sentence is compound, the worn lane runs once more on the take-off clause alone and only its worn_remove entries are merged in. Verified through the real extract() against Beholder-Q8_0: compound removal 2/8 -> 8/8, controls unmoved, extra call only on the six sentences that needed it. Interim; the real fix is a training round. COMPOUND_ADD_REMOVE.md records the evidence, including the five prompt variants that did not move the number. Co-Authored-By: Claude <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Port of GetBeholder/Beholder-ME#2 — this extractor carries the identical five-pass path.
The bug
One sentence that both takes a garment off and puts another on loses one half. Against the trained extractor (Beholder-Q8_0):
The lost garment stays in state and is fed back into the next prompt, so one miss compounds for the rest of the scene.
What does not fix it
Five prompt variants, all measured, all left it at 2/8 — an explicit rule, a worked example with expected JSON, "read each clause separately", a terse
worn_remove MUST appear, and reordering the existing bullet. Controls never moved.Re-asking the same compound prose with removal-only framing recovers 0 of 6: the compound sentence itself blinds the model, not the instruction.
The repair
takeoffClause()splits the sentence and asks the worn lane again about the take-off half alone — prose of the shape the model handles 3/3. Onlyworn_removeis merged from that reply; it never adds or replaces aworn, and it fires only when the reply carried no removal, the prose shows something coming off, and the sentence is compound.Verified through the real
extract()against the live model:Ordinary turns cost nothing.
Scope
Interim — the real fix is a training round.
COMPOUND_ADD_REMOVE.mdcarries the evidence so the prompt experiment is not repeated.Still unfixed: the addition can land on the slots being removed from (
bootonchest); that belongs wherever the merge runs.Tests: 224 pass, 0 fail (4 new for
takeoffClause).