Skip to content

Deferred research: audit and version the 13 frozen-harness findings from PR #84 #85

Description

@Taz33m

Agreed disposition

The maintainer explicitly authorized moving these 13 findings from PR #84 into a deferred follow-up on 2026-09-05, preserving all frozen code and evidence, so the release-code PR can merge after its remaining fix and checks.

These findings are deferred, not fixed or dismissed as false positives. Resolving their original PR conversations records this agreed transfer only. This issue remains open until the work below has an evidence-backed disposition.

Findings (original review links)

  • 3939088982 — V2 Claude auth-route probe inherits ambient credentials/routing. Audit existing route bindings locally; sanitize and bind the environment in a newly versioned runner.
  • 3939088984 — V2 finalization can quarantine a completed provider turn as restartable. Audit completion receipts without replaying providers; add a durable completion barrier in a future runner.
  • 3939088987 — V2 snapshot freezing permits symlinks. Audit existing inventories/read boundaries locally; preregister an explicit link policy for future runs.
  • 3939088994 — Baseline live workspace copy is not bound to the frozen snapshot before launch. Audit existing bindings; hash and verify initialized copies in a future runner.
  • 3939089000 — Baseline run-command success can outlive a failed/timed-out provider. Audit transport lifecycle receipts separately from semantic outcomes.
  • 3939089002 — Treatment technical verdicts can exempt unsuccessful exits/timeouts. Audit treatment receipts separately from V2; define explicit success predicates in a new validator.
  • 3939089010 — Missing/unreadable V2 bound files can escape as OSError. Add controlled validation errors in a future version without rewriting frozen hash bindings.
  • 3939089043 — Baseline missing model metadata falls back to opus. Treat absent model identity as missing evidence, not observed identity, in a separately versioned audit/fix.
  • 3939089045 — Baseline allows execution-time timeout overrides. Compare existing receipts with preregistered budgets before judging whether runs are affected.
  • 3939089048 — Baseline candidate/gold paths lack private-root confinement. Audit declared paths locally; enforce resolved boundaries in a new runner.
  • 3939089050 — Baseline case IDs admit path components. Audit existing manifest IDs; enforce a safe identifier grammar in a future runner.
  • 3939089056 — Treatment validation omits some workspace evidence artifacts. Audit existing completeness locally; specify all required artifacts in a new validator.
  • 3939089060 — Baseline quarantine checks a completion sentinel that baseline does not write. Audit lifecycle receipts rather than treating a missing sentinel as permission to retry.

Scope and safety boundaries

  • Preserve historical/hash-bound runners, frozen snapshots/prompts/skills, private gold, evaluator artifacts, interruptions, and retained results byte-for-byte.
  • No completed provider turn may be retried. No new provider run or external evaluation is authorized by this issue or the PR merge.
  • Any approved evidence audit is local. Never send private gold or evaluator artifacts to external providers or attach private material to this public issue.
  • Separate baseline, treatment, and V2 harness generations; do not pool them or invent missing metadata.
  • These are source-level risks, not proof that any particular completed cohort was contaminated.
  • Any future runner/validator changes require a new versioned protocol and tests, not an in-place rewrite of a frozen runner. Any necessary future collection requires explicit authorization.

Completion criteria

  • Record a per-finding audit/disposition, separating confirmed impact, no observed impact, and missing evidence.
  • Define and test required fixes in a separately versioned harness where applicable.
  • Preserve the audit trail and explain any impact on published conclusions without retroactively modifying source evidence.
  • Review the dispositions before closing this issue.

The existing public review disposition records the original scope and risks. This issue supersedes its instruction to keep these specific PR conversations open pending an agreed disposition; it does not grant release/tag/publication authority.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions