Skip to content

Freeze calibrated verifier and prepare Unseen-v1 evaluation - #30

Merged
samarth1412 merged 7 commits into
mainfrom
agent/unseen-v1-foundation
Jul 22, 2026
Merged

Freeze calibrated verifier and prepare Unseen-v1 evaluation#30
samarth1412 merged 7 commits into
mainfrom
agent/unseen-v1-foundation

Conversation

@samarth1412

Copy link
Copy Markdown
Owner

What changed

  • freezes the existing semantic verifier as semantic_v1_calibrated and adds versioned public artifact schemas and rule-pack boundaries;
  • completes strict privacy controls, streaming-safe bounded FastAPI capture, and deterministic resource cleanup;
  • adds bounded/batch verification and concurrent run isolation for LangChain, LangGraph, and LlamaIndex;
  • prepares 1.1.0rc1 with Python 3.10--3.13, optional-integration, 80% coverage, dependency-audit, OIDC, attestation, and cross-platform wheel gates;
  • defines the ContextTrace-Unseen-v1 acquisition, annotation, sealing, and preregistration protocol and updates overlapping related work.

Why

Existing RAGTruth, Diag-150, and Naturalistic Eval v2 results were repeatedly inspected during verifier development and must be treated as calibration evidence. This change freezes that implementation before collecting a source-family/domain/publication-window-disjoint test corpus.

Validation

  • clean committed worktree: 483 tests passed;
  • 85.90% coverage with an 80% CI floor;
  • zero ResourceWarnings under warning-strict execution;
  • Ruff correctness checks and scoped public-contract type checks passed;
  • pip-audit reported no known vulnerabilities;
  • 1.1.0rc1 wheel and sdist built and installed successfully in a clean environment;
  • paper compiled successfully with the new bibliography.

Follow-up

Publish 1.1.0rc1 to TestPyPI, then acquire and freeze ContextTrace-Unseen-v1 before implementing or scoring semantic_core_v2.

@samarth1412
samarth1412 marked this pull request as ready for review July 22, 2026 17:49
@samarth1412
samarth1412 merged commit 274bf67 into main Jul 22, 2026
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant