Physics-semantics evaluation layer and the 6-scorer architecture - #5

Merged

sherryzyh merged 28 commits into

mainfrom

Jun 22, 2026

Commits on Jun 18, 2026

Add prkit.verify facade, enrich Verdict, make AccuracyEvaluator scorer-backed

sherryzyh
and
claude
committed
8f32eaf
Make pre-commit formatting convergent with the venv/CI toolchain

sherryzyh
and
claude
committed
8ab26e3
Add EED/SEED partial-credit scorer filling Verdict.partial_credit

sherryzyh
and
claude
committed
b533ffd

Commits on Jun 19, 2026

Improve relation equivalence in the comparison engine

sherryzyh
and
claude
committed
8965814
Document the physics-semantics equivalence judgement

sherryzyh
and
claude
committed
7dcee05
Judge symbolic equivalence over declared symbol domains

sherryzyh
and
claude
committed
6fdef84
Add the answer-structure decision layer (canonicalize, then gate to atomic)

sherryzyh
and
claude
committed
337799e
Explain the per-structure signature columns in STRUCTURE.md

sherryzyh
and
claude
committed
f006e85
Add the objective semantics-build ecosystem: q_ref/q_prob, a_ref, a_pred, reference-free judgement

sherryzyh
and
claude
committed
8a3937f

Commits on Jun 20, 2026

Reconcile sign-convention flips between directional answers

sherryzyh
and
claude
committed
8c47f93
Document the sign-convention equivalence lane

sherryzyh
and
claude
committed
de33a01
Capture sign conventions at build time so the equivalence lane fires on built data

sherryzyh
and
claude
committed
a3158ad
Add a single-source dataset license registry and gate auto-download

sherryzyh
and
claude
committed
2092877
Ignore the incidental uv.lock (project standardizes on pip/.venv)

sherryzyh
and
claude
committed
84b7601
Restructure the semantics build/extraction public API

sherryzyh
and
claude
committed
90d3776
Refresh the docs for the physics-semantics layer

sherryzyh
and
claude
committed
ce26e20
Unify the answer taxonomy on AnswerObjectKind and retire AnswerCategory

sherryzyh
and
claude
committed
0c6b7d0

Commits on Jun 21, 2026

Streamline the Answer record, contract version, and edit-distance core

sherryzyh
and
claude
committed
f45b506
Add the model-graded LLMJudgeScorer wrapping the LLM-judge engine

sherryzyh
and
claude
committed
2f366a5
Drop the dead "legacy" entry from the ruff extend-exclude

sherryzyh
and
claude
committed
3789281
Vendor the PHYBench EED and CMPhysBench SEED edit-distance baselines

sherryzyh
and
claude
committed
a87a0ce
Forbid pint on the verify/scoring import path

sherryzyh
and
claude
committed
0d5a5a1
Add the our-semantics EED/SEED scorers and the N/A score sentinel

sherryzyh
and
claude
committed
ce1eb2c
Load only the toolkit's own .env via the nearest pyproject ancestor

sherryzyh
and
claude
committed
775b279
Collapse the prkit.verify single-file package into a module

sherryzyh
and
claude
committed
5800424
Rename the answer ontology module to answer_taxonomy

sherryzyh
and
claude
committed
4708190
Rename the Answer and PhysicalDataset domain classes to Physics*

sherryzyh
and
claude
committed
6a3a443

Commits on Jun 22, 2026

Fix CI gate failures and make CI reproducible against local

sherryzyh
and
claude
committed
5199005