User problem
Qarinah publishes reproducible local benchmark fixtures, but external developers need a standard way to test their own public or synthetic repositories without contributing private project history or accepting a marketing-only aggregate.
Proposed contract
Create a privacy-preserving reproduction kit that:
- imports only user-selected public or synthetic evidence;
- records fixture identity, command, package version, Node version, platform, estimator, and exact result hashes;
- compares equivalent required current sources on both sides;
- reports retrieval coverage, rank, conflicts, supersession, pack size, and estimated context reduction separately;
- emits a signed or digest-addressed result bundle suitable for an issue attachment or independent archive;
- refuses provider-cost, latency, and answer-quality conclusions unless those measurements are actually supplied.
Acceptance criteria
- At least three redistributable software projects with distinct task shapes.
- One command to reproduce each result from a clean install.
- Machine-readable schema and human-readable report.
- Negative controls for missing evidence, misleading similarity, private paths, and changed fixtures.
- CI verifies arithmetic, source hashes, and result-to-version identity.
- Contribution guide for independent results and responsible claim wording.
User problem
Qarinah publishes reproducible local benchmark fixtures, but external developers need a standard way to test their own public or synthetic repositories without contributing private project history or accepting a marketing-only aggregate.
Proposed contract
Create a privacy-preserving reproduction kit that:
Acceptance criteria