Skip to content

Fix robust evidence matching and extraction recovery - #4

Merged
FFFFFFpy merged 6 commits into
mainfrom
fix/robust-evidence-schema
Jul 10, 2026
Merged

FFFFFFpy merged 6 commits into
mainfrom
fix/robust-evidence-schema

Conversation

@FFFFFFpy

Copy link
Copy Markdown
Owner

Summary

This is the second-pass fix driven by the supplied debug bundle. The previous quote-based evidence design exposed three new failure modes: typography-only quote differences were rejected, YAML/H1 title duplication was treated as ambiguous, and malformed stage responses such as {} were counted as successful empty output.

Changes

Evidence resolution

  • normalize Unicode/full-width/curly quotation marks and compatibility characters while preserving a mapping to original source positions
  • keep the original Markdown slice in resolved evidence rather than the model-normalized quote
  • when a quote appears in both YAML frontmatter and visible Markdown, prefer the unique visible-body occurrence
  • retain strict ambiguity errors when multiple visible-body matches remain

Stage response contract and retry

  • require summary, concepts, entities, or relations at the top level with the expected value type
  • treat {}, misspelled keys, null values, and wrong container types as malformed responses
  • retry a malformed stage once with an explicit corrective schema
  • mark the stage failed after a second malformed response instead of reporting 0/0 ok
  • record attempt/retry counts and preserve second-attempt debug payloads under attempt-2/

Concept/entity separation

  • strengthen prompts so concepts exclude named entities and entities use a canonical type set
  • reject entity outputs whose type describes a concept, mechanism, strategy, gameplay pattern, hook, or business model
  • reclassify name collisions in favor of valid concrete entities

Relations

  • provide unique typed node IDs to the relation stage
  • require subject_id / object_id while retaining legacy name-based relation output compatibility
  • deduplicate relation candidates before prompting

Validation

  • targeted local tests: 6 passed
  • legacy name-based relation compatibility: passed
  • supplied debug payload replay:
    • all 12 concept quotes resolve
    • title entity resolves to visible H1 at source line 9
    • final classification: 11 concepts, 6 concrete entities
    • malformed {} relation response triggers exactly one corrective retry

Version bumped to 0.2.1.

@FFFFFFpy
FFFFFFpy marked this pull request as ready for review July 10, 2026 19:15
@FFFFFFpy
FFFFFFpy merged commit 126e36d into main Jul 10, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant