fix: tag extraction candidates with conversational grounding and drop constructed content

[gorkem2020]

Problem

Extraction has no notion of conversational register. An in-game rule, a persona's claim, or drafted fiction is textually indistinguishable from a durable user fact once it reaches the extraction LLM, so a well-formed invented rule gets extracted with exactly the same shape as a genuine preference. Downstream, anything that scores candidates by shape rather than reality (for example a utility/confidence-style admission heuristic) will tend to prefer the invented content, because in-fiction text is maximally grounded in the transcript and reads as durably useful.

The fix belongs at the source: the extractor should know, per candidate, whether the underlying assertion is actually about the real user/world or only true inside an in-conversation constructed context (a game, roleplay, drafted fiction, a hypothetical, or sample/test data being manipulated).

The grounding mechanism

buildExtractionPrompt (src/extraction-prompts.ts) now asks the model to tag every extracted memory with a grounding field:

• "real": an assertion about the actual user, the real world, or this real working session, including a genuine first-person aside stated in passing during a game (for example, mentioning a real appointment while playing).
• "constructed": an assertion whose truth exists only inside an in-conversation constructed context: a game's rules/scores/bets, a persona's claims, drafted fiction, a hypothetical, or sample data.

A deterministic post-filter in SmartExtractor.extractCandidates's validation loop (src/smart-extractor.ts) then enforces the contract:

• A candidate tagged "constructed" is dropped unless its category is events.
• At most one constructed events note survives per extraction (an episodic record that the real participants did the activity, e.g. "ran a puzzle exercise", not the invented rules/scores themselves).
• A candidate tagged "real" is never affected by this filter, regardless of category, including a real aside stated during play.
• A missing, non-string, or unrecognized grounding value fails open to "real", so a model that ignores the new field cannot regress extraction behavior. This keeps the change fully backward compatible.

The scope-glob extraction-policy knob

As a cheap, deterministic companion, SmartExtractorConfig.extractionPolicy accepts a map from a scope string (or glob, using *) to one of:

• "full" (default): today's behavior, unchanged.
• "episodic-only": only events-class candidates are kept, independent of grounding. Useful as a blunt backstop for scopes known to be pure play/roleplay.
• "none": extraction is skipped entirely for that scope, with zero LLM calls.

Resolution (resolveExtractionPolicy, exported from src/smart-extractor.ts) prefers an exact scope match over a glob match, and defaults unmatched scopes to "full". This knob needs no host-side plumbing: it is read entirely from SmartExtractorConfig, keyed on scope strings the extractor already has in hand.

Test coverage

New suite: test/extraction-grounding-register.test.mjs (wired into package.json's test script and into scripts/ci-test-manifest.mjs under core-regression).

• Constructed game content (an invented house rule, a bet outcome) is dropped from preferences/cases, keeping only the single constructed events note.
• Two constructed events candidates in one extraction: only the first survives (cap enforcement).
• A genuine real first-person aside stated during play survives under its natural category, alongside the constructed episodic note (false-positive guard).
• A purely factual transcript is extracted unchanged (regression guard against the default/no-grounding-field path).
• A malformed or missing grounding value (non-string, unrecognized string) fails open to "real" and is kept.
• buildExtractionPrompt documents the grounding contract (structural check against prompt regressions).
• resolveExtractionPolicy: exact match wins over glob match; unmatched scope defaults to "full".
• Scope mapped "none": extraction is skipped with zero LLM calls.
• Scope mapped "episodic-only": only events-class candidates survive, independent of grounding.
• Unmatched scope: behavior is unchanged from "full".

All fixtures are synthetic two-agent transcripts (agent-one/agent-two) with invented game content.

Non-goals

• The admission-control gate (src/admission-control.ts) is untouched. A reality-style admission feature is a reasonable independent defense-in-depth follow-up, but it is downstream of this fix, costs a second per-candidate signal, and only helps deployments that enable admission control (off by default). Better evaluated after this change has had time to prove out.
• Speaker attribution (distinguishing the real user from an agent labeled "User" in-transcript) is a separate, independently tracked fix. grounding and speaker attribution are orthogonal per-candidate filters in the same validation loop and are designed to stack without either blocking the other.
• Host-side config wiring to expose extractionPolicy through the plugin's own configuration file is left as a fast-follow; the knob is fully functional today via SmartExtractorConfig for any caller that constructs SmartExtractor directly.

Live fleet verification (2026-07-12)

Deployed this branch to a live gateway and probed it with a fresh two-turn webchat session against a fleet test agent:

1. Turn 1: a one-round roleplay message carrying five extractable-looking in-game "facts" (a favorite drink, a residence, a pet, a persona identity, and a house rule).
2. Turn 2: game over, plus a genuine first-person aside ("doing a routine memory-system check this Sunday morning").

Observed (extraction model: openai/gpt-oss-120b):

• Turn 1 raw extractor output contained exactly one candidate, the allowed episodic events note about the roleplay activity itself; the model's reasoning applied the new grounding rules ("...seems like part of the roleplay persona. So it's constructed."). None of the five in-game propositions surfaced as profile/preferences/entities/cases/patterns candidates.
• Turn 2 extracted the real aside as a real events candidate, showing the rule does not over-suppress genuine asides spoken around play.
• Downstream admission scoring then rejected both episodic notes on utility grounds, so the net store delta was zero rows.
• The deterministic post-filter was not exercised in this run (the prompt level already suppressed everything durable) and remains in place as the backstop for weaker extraction models.

For contrast: before this change, an equivalent game session on the same fleet persisted in-game rules, scores, and persona claims as durable preference and fact rows.

v2 hardening (added after extended live testing)

Repeated runs of the same roleplay fixture surfaced a failure mode in v1: grounding self-tagging is not deterministic. On one of three identical-fixture runs, the model tagged three in-fiction canon items real (matching the shape of the few-shot examples, which all showed the same three-real-plus-one-constructed pattern) while correctly tagging the session note constructed, a batch self-contradiction the deterministic filter could not catch because it trusts the per-item tags and fails open to real.

v2 therefore stops relying on per-item self-tags alone:

1. Batch-level register: the extraction schema gains a top-level conversation_register: "real" | "mixed" | "fiction", judged once per batch (whole-session classification proved stable across runs where per-item tags wobbled). The filter enforces it deterministically: fiction drops all durable candidates regardless of per-item tags and keeps at most the single constructed events note; a missing register is treated as mixed (toward scrutiny, not open).
2. Batch contradiction check: in a non-real register, an explicit constructed tag in the batch demotes surviving real-tagged durables (deterministic, no extra LLM call). Tradeoff, stated plainly: a genuine durable aside made mid-fiction without an exit cue is sacrificed; such asides normally classify as events, and the demotion is audit-logged.
3. Grounding reaches admission and storage: candidates now carry grounding (and the batch register) past the filter into the admission evaluation and into persisted row metadata, so there is an audit trail of which stored events used the constructed allowance. Constructed candidates targeting durable registers are auto-rejected before any admission LLM call.
4. Fiction-aware admission: the utility prompt receives the grounding and register lines and scores in-fiction claims near zero for durable categories; the durable-category type prior is capped to the events level under a fiction register so priors cannot outvote reality.
5. Prompt rework: the grounding rule is now explicitly per item with no expected tag counts ("the storage cap is a storage rule applied after tagging, it is NOT a tagging quota"), with a batch self-consistency instruction, and the few-shot examples were reshaped into varied batch shapes including an all-constructed mid-game example.
6. Vocabulary alignment: the admission guidance names all six registers correctly, and the extraction criteria bullets use only the output enum.

v2 live verification

Deployed on a live gateway and re-probed with the original fixture: extraction produced exactly one real aside plus one constructed session note (both with persisted grounding and register metadata), zero fiction durables, an honest "I don't have that information" on the recall probe that previously returned the fictional answer, and an admission rejection whose logged reasoning explicitly cited the fictional frame. The deterministic layers are covered by red-proven unit tests reproducing the exact observed failure shape.

[app3apps]

This PR currently conflicts with the latest master (mergeable=CONFLICTING, merge_state_status=DIRTY). R0 verification and R1 value assessment completed, but the conflict gate deferred deep review because the branch cannot be merged as-is.

Please rebase onto or merge the latest master, resolve the conflicts, and push an updated head. I will run the full review after the branch is clean.

One item to keep in scope during the rebase: R1 found that the new extractionPolicy scope-glob feature is broader than the grounding bug fix and is exposed without normal plugin-configuration wiring. The full review will evaluate whether it should be split or fully integrated.

[app3apps]

Re-reviewed the rebased head e993727.

There is one functional regression to address before merge:

• In src/smart-extractor.ts:971-980, once a non-real batch contains any constructed candidate, the contradiction guard removes every surviving durable candidate without checking that candidate's own grounding. A mixed conversation containing a real-grounded preference plus a constructed event therefore loses the genuine preference. That conflicts with the prompt contract that real asides in mixed conversations should survive under their natural durable category.

Please preserve real-grounded durable candidates and add a regression covering a mixed batch with both real and constructed candidates.

Verification note: the new 21-test grounding suite passes in isolation. The serial full suite hit the orchestrator's 180-second timeout before reaching that file; no assertion failure was observed. As a separate follow-up, extractionPolicy values should be validated because an unknown value currently fails open to full extraction.