feat(trajectory): measure observable compaction content-byte retention

[anthony-chaudhary]

For: operators comparing context retention on GPT Astra FAK sessions (priority 3 harness, Enabling).
Problem: canonical trajectory audit accounts tokens but does not expose Codex compacted replacement-history byte retention. The primary pinned 10 top-level Astra sessions contain 15 compaction markers (8 sessions); a secondary all-source cohort contains one viable marker, so a 3x retained-context objective currently lacks a measured baseline.
Today: manual transcript inspection can identify compacted markers but cannot produce a canonical content-free replayable retention receipt.
Better because: the existing Go audit will report exact decoded UTF8 content bytes reconstructed from explicit response items and replacement histories, with unsupported/missing-history coverage. This is not provider prompt bytes, tokens, or FAK-origin compaction attribution.

Scope: internal/trajectory audit row, parser, new bounded helper and one regression test. Preserve token accounting and privacy. Opaque items, unsupported content and missing replacement histories make comparability unavailable. A valid replacement resets history and restores supported coverage.
Witness: one deterministic Go regression spanning multibyte text/tool content, two replacements, missing-history recovery, plus real CLI replay of the private 10-session pinned corpus with no transcript contents in output.
Done: canonical JSONL emits per-compaction before/after observable content-byte measurements and explicit comparability; original usage totals remain unchanged. No 3x performance or quality claim from instrumentation alone.
Related: #8584 net-true context burden; #3027 fidelity probes; #8509 typed usage resets. Open/closed duplicate searches found no matching per-compaction audit receipt unit.

Gate remediation: canonical CompactionEvents positioning adds its narrow concept row, reciprocal sibling distinction, glossary anchor, and generated INDEX/README through fak concept position. The new field originally increased context-ctx coverage debt by one; required positioning restores the prior debt without weakening the gate.

Verified completion

Shipped internal/trajectory@r79+g7cbd0560a in commit 7cbd0560a997bfb8e6ab598b93eed5da7c8fac32, verified on remote main. One targeted regression, full WSL trajectory tests and vet, independent changed-CLI replay, prospective validation, and commit audit passed. The same immutable ten-session corpus now emits 15 complete per-file compaction receipts; per-session token totals are unchanged. These are reconstructed transcript-content UTF8 bytes, not provider request tokens or compaction-engine attribution. The operator's broader 3x optimization goal remains unmet.

Core through-line: decode recorded compaction replacements -> emit typed per-file retention receipts -> replay the frozen corpus through the real CLI.
Gold-plating boundary: live gateway compaction, provider-cost attribution, broader quality optimization, and presentation changes are follow-ons, not completion claims for this issue.

Follow-on reconciliation

Existing work remains in #11431 (Responses compaction), #3027 (fidelity), and #8584 (net-true context burden). Execution findings were reconciled in #9798 (restore boundary), #11813 (missing worker metadata), and #11815 (dirty-worker post-merge validation). Do not duplicate those issues.

The canonical fan-out generated the following three QA candidates. They remain a reviewable plan, not new confirmed defects or dispatched workers; dedupe against existing witnesses and tickets before filing. Further optimization depends on resolving the retained-content target and its standing-instruction floor.

Canonical follow-on plan

{
  "schema": "fak.issue-fanout-plan.v1",
  "input": {
    "title": "Observable compaction retention receipts",
    "leaf": "trajectory",
    "spine_ref": "7cbd0560a997bfb8e6ab598b93eed5da7c8fac32",
    "parent_ref": "#11809",
    "areas": [
      "qa",
      "observability",
      "integration"
    ],
    "max": 3,
    "completion_standard": "production"
  },
  "area_counts": {
    "qa": 3
  },
  "candidates": [
    {
      "key": "fanout-trajectory-spine-93eb7fa5b8856143-qa-edge-sweep",
      "title": "qa: adversarial + edge-case sweep for Observable compaction retention receipts",
      "generation": "gen/now",
      "parent_ref": "#11809",
      "current_state": "The working spine of Observable compaction retention receipts exists (7cbd0560a997bfb8e6ab598b93eed5da7c8fac32); this follow-on has not started.",
      "why_now": "Spine-first default: the spine just shipped, so its follow-on backlog is filed at creation time while context is hot.",
      "working_spine": "Every documented input class of Observable compaction retention receipts keeps working under hostile and edge input; the test table is the map of what is proven.",
      "work_unit": "leaf",
      "expected_steps": 3,
      "assumptions": [
        "The spine at 7cbd0560a997bfb8e6ab598b93eed5da7c8fac32 still builds/runs on trunk."
      ],
      "confusion_risks": [
        "A green test that never drives the real seam is not a witness.",
        "Harden what exists; do not grow the spine's scope inside this follow-on."
      ],
      "coordination": [
        "Stay inside internal/trajectory/; check the lane lease before dispatch."
      ],
      "trigger": "one-shot: filed at spine-ship time by fak issue fanout",
      "batch_policy": "one batch per spine, capped at --max candidates; marker-key (fanout-trajectory-*) dedupe against existing issues before filing",
      "in_scope": "Table-driven tests over internal/trajectory/ covering empty, oversized, malformed, and hostile inputs and every error path.",
      "out_of_scope": "New features; refactors beyond what a failing test forces.",
      "done_condition": "Edge/adversarial table tests land in internal/trajectory/ and fail before / pass after any fix they force.",
      "witness": "go test ./internal/trajectory -run 'Edge|Adversarial' -v output captured in the commit or issue.",
      "acceptance_gate": "go test ./internal/trajectory -run 'Edge|Adversarial' -v and make test-fast green.",
      "lane": "trajectory",
      "paths": [
        "internal/trajectory/"
      ],
      "labels": [
        "fanout",
        "qa",
        "class:infra",
        "priority/P1"
      ],
      "priority": "priority/P1",
      "closure_binding": "Close via a commit whose body carries `Closes #\u003cn\u003e` and whose diff lands the witness above.",
      "problem_frame": {
        "schema": "fak-problem-frame/1",
        "ready": true,
        "enforced": true,
        "centrality": "enabling",
        "centrality_target": "the shipped spine's observable Core outcome",
        "checks": {
          "p1": {
            "id": "p1",
            "status": "advanced",
            "evidence": "the child captures its own bounded follow-on context while reusing the shipped spine",
            "valid": true
          },
          "p2": {
            "id": "p2",
            "status": "advanced",
            "evidence": "the child closes a named operating or evidence gap instead of expanding the spine",
            "valid": true
          },
          "p3": {
            "id": "p3",
            "status": "preserved",
            "evidence": "the child stays independently dispatchable and can be reprioritized without changing sibling scope",
            "valid": true
          },
          "p4": {
            "id": "p4",
            "status": "advanced",
            "evidence": "the child carries its witness through the real follow-on planning and dispatch path",
            "valid": true
          }
        }
      },
      "required_model_tier": "T1",
      "optimal_model_tier": "T0"
    },
    {
      "key": "fanout-trajectory-spine-93eb7fa5b8856143-qa-failure-paths",
      "title": "qa: failure-path + refusal coverage for Observable compaction retention receipts",
      "generation": "gen/now",
      "parent_ref": "#11809",
      "current_state": "The working spine of Observable compaction retention receipts exists (7cbd0560a997bfb8e6ab598b93eed5da7c8fac32); this follow-on has not started.",
      "why_now": "Spine-first default: the spine just shipped, so its follow-on backlog is filed at creation time while context is hot.",
      "working_spine": "Every refusal or error Observable compaction retention receipts can return names the fix in its message, and a test asserts each one.",
      "work_unit": "leaf",
      "expected_steps": 3,
      "assumptions": [
        "The spine at 7cbd0560a997bfb8e6ab598b93eed5da7c8fac32 still builds/runs on trunk."
      ],
      "confusion_risks": [
        "Do not weaken a refusal to make a test pass; the refusal is the contract.",
        "Harden what exists; do not grow the spine's scope inside this follow-on."
      ],
      "coordination": [
        "Stay inside internal/trajectory/; check the lane lease before dispatch."
      ],
      "trigger": "one-shot: filed at spine-ship time by fak issue fanout",
      "batch_policy": "one batch per spine, capped at --max candidates; marker-key (fanout-trajectory-*) dedupe against existing issues before filing",
      "in_scope": "Tests asserting each error return of internal/trajectory/; message-quality fixes those tests force.",
      "out_of_scope": "Happy-path behavior changes.",
      "done_condition": "Each error return has a test asserting both the refusal and that its message names the recovery.",
      "witness": "go test ./internal/trajectory -run 'Refus|Error|Requir' -v output captured in the commit or issue.",
      "acceptance_gate": "go test ./internal/trajectory -run 'Refus|Error|Requir' -v and make test-fast green.",
      "lane": "trajectory",
      "paths": [
        "internal/trajectory/"
      ],
      "labels": [
        "fanout",
        "qa",
        "class:infra",
        "priority/P2"
      ],
      "priority": "priority/P2",
      "closure_binding": "Close via a commit whose body carries `Closes #\u003cn\u003e` and whose diff lands the witness above.",
      "problem_frame": {
        "schema": "fak-problem-frame/1",
        "ready": true,
        "enforced": true,
        "centrality": "enabling",
        "centrality_target": "the shipped spine's observable Core outcome",
        "checks": {
          "p1": {
            "id": "p1",
            "status": "advanced",
            "evidence": "the child captures its own bounded follow-on context while reusing the shipped spine",
            "valid": true
          },
          "p2": {
            "id": "p2",
            "status": "advanced",
            "evidence": "the child closes a named operating or evidence gap instead of expanding the spine",
            "valid": true
          },
          "p3": {
            "id": "p3",
            "status": "preserved",
            "evidence": "the child stays independently dispatchable and can be reprioritized without changing sibling scope",
            "valid": true
          },
          "p4": {
            "id": "p4",
            "status": "advanced",
            "evidence": "the child carries its witness through the real follow-on planning and dispatch path",
            "valid": true
          }
        }
      },
      "required_model_tier": "T2",
      "optimal_model_tier": "T1"
    },
    {
      "key": "fanout-trajectory-spine-93eb7fa5b8856143-qa-determinism",
      "title": "qa: determinism + race witness for Observable compaction retention receipts",
      "generation": "gen/now",
      "parent_ref": "#11809",
      "current_state": "The working spine of Observable compaction retention receipts exists (7cbd0560a997bfb8e6ab598b93eed5da7c8fac32); this follow-on has not started.",
      "why_now": "Spine-first default: the spine just shipped, so its follow-on backlog is filed at creation time while context is hot.",
      "working_spine": "The same input yields byte-identical output every run, and the package is race-clean, so cohort/replay layers can trust it.",
      "work_unit": "leaf",
      "expected_steps": 2,
      "assumptions": [
        "The spine at 7cbd0560a997bfb8e6ab598b93eed5da7c8fac32 still builds/runs on trunk."
      ],
      "confusion_risks": [
        "Map iteration order and clock/rand reads are the usual determinism leaks.",
        "Harden what exists; do not grow the spine's scope inside this follow-on."
      ],
      "coordination": [
        "Stay inside internal/trajectory/; check the lane lease before dispatch."
      ],
      "trigger": "one-shot: filed at spine-ship time by fak issue fanout",
      "batch_policy": "one batch per spine, capped at --max candidates; marker-key (fanout-trajectory-*) dedupe against existing issues before filing",
      "in_scope": "A determinism test (two runs, deep-equal) and a -race run over internal/trajectory/.",
      "out_of_scope": "Performance tuning.",
      "done_condition": "Determinism test lands; -race run over the package is clean.",
      "witness": "go test -race ./internal/trajectory -run Determinism -v output captured in the commit or issue.",
      "acceptance_gate": "go test -race ./internal/trajectory -run Determinism -v and make test-race green.",
      "lane": "trajectory",
      "paths": [
        "internal/trajectory/"
      ],
      "labels": [
        "fanout",
        "qa",
        "class:infra",
        "priority/P2"
      ],
      "priority": "priority/P2",
      "closure_binding": "Close via a commit whose body carries `Closes #\u003cn\u003e` and whose diff lands the witness above.",
      "problem_frame": {
        "schema": "fak-problem-frame/1",
        "ready": true,
        "enforced": true,
        "centrality": "enabling",
        "centrality_target": "the shipped spine's observable Core outcome",
        "checks": {
          "p1": {
            "id": "p1",
            "status": "advanced",
            "evidence": "the child captures its own bounded follow-on context while reusing the shipped spine",
            "valid": true
          },
          "p2": {
            "id": "p2",
            "status": "advanced",
            "evidence": "the child closes a named operating or evidence gap instead of expanding the spine",
            "valid": true
          },
          "p3": {
            "id": "p3",
            "status": "preserved",
            "evidence": "the child stays independently dispatchable and can be reprioritized without changing sibling scope",
            "valid": true
          },
          "p4": {
            "id": "p4",
            "status": "advanced",
            "evidence": "the child carries its witness through the real follow-on planning and dispatch path",
            "valid": true
          }
        }
      },
      "required_model_tier": "T2",
      "optimal_model_tier": "T1"
    }
  ]
}

[anthony-chaudhary]

Implementation landed locally as internal/trajectory@r79+g7cbd0560a (commit 7cbd056), exact nine paths including required canonical concept positioning. It is awaiting publication: safe sync refuses the shared checkout on pre-existing PUBLIC_LEAK findings outside this issue, so no push was attempted and this issue remains open.

Witnesses passed: complete trajectory package tests and vet under WSL; independent regression execution; real canonical CLI on the same verified private 10-session top-level Astra corpus. Original audit emitted no compaction receipts; new audit emits 15 complete reconstructed-content events. Before content: 4,148,616 UTF8 bytes; retained: 1,024,200; separately reported summaries: 159,232. Every session's token totals remain unchanged. Original and new-output snapshots have identical corpus digest f1d37aa9b291d60726f5d870939641c24872c3361a8b2a65618daf67babec32e. Raw inputs remain private.

These are observable transcript-content bytes, not provider prompt tokens, an attributed FAK compaction result, or a 3x gain. Originating-task fidelity remains unproven. Managed landing passed all admission, disambiguation, compilation, CAS and synchronization gates. Recovery follow-ons are tracked separately in #11813 and #11815.