Skip to content

CHANGELOG.md

Latest commit

 

History

History
94 lines (73 loc) · 4.09 KB

CHANGELOG.md

File metadata and controls

94 lines (73 loc) · 4.09 KB
Raw

Changelog

All notable changes to JudgeCalibrationKit will be documented here.

0.3.0

Added

  • Typed imports for xceval 0.4 report documents, including their complete normalized sample rows and mapping diagnostics.
  • Canonical artifactID and exact-source byteDigest provenance on inspect and report imports when xceval supplies them.

Changed

  • Updated the canonical XCEvalFormat dependency to xceval 0.4.x.
  • Expanded the typed document boundary so every xceval 0.4 command is handled explicitly. Non-sample command documents return a deterministic unsupportedCommand import error.

Compatibility

  • Existing samples JSON, samples JSON Lines, and sample-bearing inspect JSON remain supported, including the checked-in canonical xceval 0.3.0 fixture.
  • Metrics, evidence, selections, comparisons, gates, and dataset-lifecycle documents are not treated as calibration populations. They are summary-only or may represent filtered subsets, so importing them would bias agreement statistics.

0.2.0

Added

  • Typed normalized xceval imports through the upstream XCEvalFormat product, including source provenance and deterministic mapping diagnostics.
  • Explicit single-key metadata slice analysis with distinct missing-value slices, bounded cardinality, and deterministic reports.
  • Worst-slice judge-human CI gates with selection-aware clustered-bootstrap intervals, eligible-slice counts, and discarded-replicate attribution.
  • A deterministic iOS 27 simulator harness for runtime verification of the typed Apple Evaluations adapter. The harness cross-target typechecks, but its local run found no installed iOS 27 simulator runtime; a live runtime PASS remains pending.

Changed

  • Raised the package baseline to Swift 6.2 and macOS 14 while retaining the existing iOS 16, tvOS 16, watchOS 9, and visionOS 1 declarations.
  • JudgeCalibrationXCEval now delegates ownership of the normalized xceval/v1 envelope to XCEvalFormat 0.3.x. Calibration mappings, stable sample identity, scale validation, and policy remain project-owned.
  • The package-level dependency statement is now scoped precisely: the JudgeCalibration core target links no third-party dependencies, while the optional xceval bridge uses the canonical upstream format product.

Compatibility

  • Existing judge-calibration/v1 datasets and data-import convenience APIs remain supported.
  • Existing analysis and gate configurations retain overall-analysis behavior when slice options are absent.
  • Worst-slice humanPanel gates are explicitly unsupported in this release; slice gates support judgeHuman and everyJudgeHuman comparisons.
  • Worst-slice mean absolute and signed error results are explicitly unavailable because a minimum-value gate cannot represent their metric direction safely.

Migrating from 0.1.x

  • Swift 6.2 and macOS 14 are now the minimum toolchain and macOS deployment baselines; the other declared Apple platform floors are unchanged.
  • SwiftPM now resolves Evaluations-Framework-CLI 0.3.x across the package graph. The JudgeCalibration target itself still links no third-party module.
  • importData, importJSON, and importJSONLines remain available. Typed XCEvalDocument and [XCEvalSampleLine] import-result APIs add provenance and diagnostics without changing judge-calibration/v1.
  • Configurations without slices retain 0.1.x overall-analysis behavior.
  • Envelope/schema/command/sample-count and JSON Lines decode failures now use XCEvalFormat's error surface; JCK continues to own mapping and calibration errors.

Thanks to Rudrank Riyam for publishing XCEvalFormat in xceval 0.3.0 and for making the normalized interoperability boundary a shared public product.

0.1.0

Added

  • Dependency-free calibration schema, validation, dataset merging, metrics, reports, clustered-bootstrap intervals, and gates.
  • Swift Testing failure helper.
  • Typed Apple Evaluations adapter with dynamic Xcode discovery scripts.
  • Optional importer for normalized xceval/v1 output, pinned to xceval 0.2.1.
  • Hand-authored examples, design documentation, and DocC guides.