Skip to content

feat(eval): consume sharded corpus-v2 exams - #16

Merged
vex0209-bt merged 1 commit into
mainfrom
feat/corpus-v2-evaluator
Jul 24, 2026
Merged

feat(eval): consume sharded corpus-v2 exams#16
vex0209-bt merged 1 commit into
mainfrom
feat/corpus-v2-evaluator

Conversation

@vex0209-bt

Copy link
Copy Markdown
Contributor

Summary

  • add version-gated loading for corpus manifest v1 and sharded v2 manifests
  • fetch only shards required by the deterministic exam sample selection
  • verify root, shard, MP4, PNG, decoded-video, and first-frame integrity before evaluation
  • use the corpus caption and verified PNG first frame as TI2V conditioning inputs
  • extend corpus verification, audit records, calibration support, tests, and the production runbook

Safety

  • hash root manifests and shards before parsing
  • fail closed on schema, content-address, decoding, or conditioning-frame mismatches
  • preserve v1 compatibility and leave the active chain.toml manifest pin unchanged
  • require no runtime captioning model, leaving all eight H100 GPUs available for four evaluation pairs

Verification

  • 672 passed in the full test suite
  • adversarial corpus-v2 parser, shard-selection, and artifact-integrity coverage
  • wheel build and isolated wheel import passed
  • git diff --check and changed-file static checks passed

Load digest-pinned corpus-v2 roots and only the shards selected by deterministic global indices. Validate the strict builder schema and fail closed on root, shard, caption, decode, or object-key drift.

Verify content-addressed MP4 and PNG bytes, decoded truth and RGB digests, and the conditioning PNGs exact relationship to truth frame zero. Extend audit and operator verification paths while preserving corpus-v1 behavior and leaving chain.toml unchanged.
@vex0209-bt
vex0209-bt merged commit b927ec8 into main Jul 24, 2026
3 checks passed
@vex0209-bt
vex0209-bt deleted the feat/corpus-v2-evaluator branch July 24, 2026 16:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant