Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/ISSUE_TEMPLATE/bug.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ body:
attributes:
label: Version
description: Package version or exact commit.
placeholder: 0.5.0
placeholder: 0.6.0
validations:
required: true
- type: dropdown
Expand Down
5 changes: 4 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@ All notable changes to Skivolve are documented in this file. The format follows

## [Unreleased]

## [0.6.0] - 2026-08-07

### Added

- Added four calibrated final-output cases for compatibility decisions, read-only diagnoses, surgical plans, and evidence-gap research, with transient workspace-write evidence and unseen positive paraphrases guarding against oracle overfitting.
Expand Down Expand Up @@ -129,7 +131,8 @@ All notable changes to Skivolve are documented in this file. The format follows
- Repository-local `engineering` and `testing` reference bundles for exercising Git-bound comparison variants.
- CI, CodeQL, OpenSSF Scorecard, Dependabot, issue forms, contribution policy, security reporting, governance, support, and release documentation.

[Unreleased]: https://github.com/Dhi13man/skivolve/compare/v0.5.0...HEAD
[Unreleased]: https://github.com/Dhi13man/skivolve/compare/v0.6.0...HEAD
[0.6.0]: https://github.com/Dhi13man/skivolve/compare/v0.5.0...v0.6.0
[0.5.0]: https://github.com/Dhi13man/skivolve/compare/v0.4.0...v0.5.0
[0.4.0]: https://github.com/Dhi13man/skivolve/compare/v0.3.0...v0.4.0
[0.3.0]: https://github.com/Dhi13man/skivolve/compare/v0.2.0...v0.3.0
Expand Down
4 changes: 2 additions & 2 deletions CITATION.cff
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,8 @@ cff-version: 1.2.0
message: If you use Skivolve in published work, cite this software release.
title: Skivolve
type: software
version: 0.5.0
date-released: 2026-07-29
version: 0.6.0
date-released: 2026-08-07
authors:
- family-names: Seal
given-names: Dhiman
Expand Down
18 changes: 9 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@

Skivolve runs reproducible A/B evaluations of agent skills and instruction bundles through isolated harnesses, objective case verifiers, and calibrated blinded comparison.

Version `0.5.0` is an alpha release for expert evaluation work on Linux. The public repository contains train and validation cases, not a private holdout, and ships no live comparator certification. It does not claim that one harness or bundle is superior. The production `software-engineering-v1` comparator profile is calibrated for software changes; the bundled plain-language profile has test authority and author-authored labels, not independent production calibration.
Version `0.6.0` is an alpha release for expert evaluation work on Linux. The public repository contains train and validation cases, not a private holdout, and ships no live comparator certification. It does not claim that one harness or bundle is superior. The production `software-engineering-v1` comparator profile is calibrated for software changes; the bundled plain-language profile has test authority and author-authored labels, not independent production calibration.

## What Skivolve Provides

Expand All @@ -32,7 +32,7 @@ The runtime package has one exact third-party dependency, `rfc8785==0.1.4`, for
Clone the tagged source so the reference suite, cases, and pinned Git baseline are available:

```bash
git clone --branch v0.5.0 https://github.com/Dhi13man/skivolve.git
git clone --branch v0.6.0 https://github.com/Dhi13man/skivolve.git
cd skivolve
python3 -m venv .venv
. .venv/bin/activate
Expand All @@ -41,16 +41,16 @@ python3 -m venv .venv
Download, verify, and install the release wheel:

```bash
mkdir -p /tmp/skivolve-0.5.0
gh release download v0.5.0 --repo Dhi13man/skivolve \
--pattern "skivolve-0.5.0*" --pattern SHA256SUMS \
--dir /tmp/skivolve-0.5.0
(cd /tmp/skivolve-0.5.0 && sha256sum --check SHA256SUMS)
mkdir -p /tmp/skivolve-0.6.0
gh release download v0.6.0 --repo Dhi13man/skivolve \
--pattern "skivolve-0.6.0*" --pattern SHA256SUMS \
--dir /tmp/skivolve-0.6.0
(cd /tmp/skivolve-0.6.0 && sha256sum --check SHA256SUMS)
gh attestation verify \
/tmp/skivolve-0.5.0/skivolve-0.5.0-py3-none-any.whl \
/tmp/skivolve-0.6.0/skivolve-0.6.0-py3-none-any.whl \
--repo Dhi13man/skivolve
python -m pip install \
/tmp/skivolve-0.5.0/skivolve-0.5.0-py3-none-any.whl
/tmp/skivolve-0.6.0/skivolve-0.6.0-py3-none-any.whl
```

## Run A Comparison
Expand Down
4 changes: 2 additions & 2 deletions SECURITY.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,8 @@ Skivolve executes untrusted generated code and integrates with authenticated cod

| Version | Supported |
| --------- | --------- |
| `0.4.x` | Yes |
| `<=0.3.x` | No |
| `0.6.x` | Yes |
| `<=0.5.x` | No |

Only the latest patch release receives security fixes. A security fix may change a provider protocol lock, release authority, schema, corpus, or result contract when preserving the old behavior would remain unsafe.

Expand Down
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"

[project]
name = "skivolve"
version = "0.5.0"
version = "0.6.0"
description = "Open-source framework for reproducible A/B evaluation of agent skills and instructions"
readme = "README.md"
requires-python = ">=3.11"
Expand Down
18 changes: 9 additions & 9 deletions site/docs/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@

<main id="main-content" tabindex="-1">
<header class="page-hero shell">
<p class="eyebrow">Documentation · Version 0.5.0</p>
<p class="eyebrow">Documentation · Version 0.6.0</p>
<h1>Run your first evidence-bound evaluation.</h1>
<p class="lede">Start with an offline dry run. It validates the suite, tools, exact sources, cases, provider locks, and selection before a model is invoked or result artifact is written.</p>
</header>
Expand All @@ -77,20 +77,20 @@ <h2 id="requirements">Requirements</h2>
<p>Skivolve runs generated code. Read the <a href="../security/">security model</a> before using proprietary fixtures, private holdouts, or valuable provider credentials.</p>

<h2 id="install">Install the verified release</h2>
<pre><code>git clone --branch v0.5.0 https://github.com/Dhi13man/skivolve.git
<pre><code>git clone --branch v0.6.0 https://github.com/Dhi13man/skivolve.git
cd skivolve
python3 -m venv .venv
. .venv/bin/activate
mkdir -p /tmp/skivolve-0.5.0
gh release download v0.5.0 --repo Dhi13man/skivolve \
--pattern "skivolve-0.5.0*" --pattern SHA256SUMS \
--dir /tmp/skivolve-0.5.0
(cd /tmp/skivolve-0.5.0 &amp;&amp; sha256sum --check SHA256SUMS)
mkdir -p /tmp/skivolve-0.6.0
gh release download v0.6.0 --repo Dhi13man/skivolve \
--pattern "skivolve-0.6.0*" --pattern SHA256SUMS \
--dir /tmp/skivolve-0.6.0
(cd /tmp/skivolve-0.6.0 &amp;&amp; sha256sum --check SHA256SUMS)
gh attestation verify \
/tmp/skivolve-0.5.0/skivolve-0.5.0-py3-none-any.whl \
/tmp/skivolve-0.6.0/skivolve-0.6.0-py3-none-any.whl \
--repo Dhi13man/skivolve
python -m pip install \
/tmp/skivolve-0.5.0/skivolve-0.5.0-py3-none-any.whl</code></pre>
/tmp/skivolve-0.6.0/skivolve-0.6.0-py3-none-any.whl</code></pre>
<p>This leaves the tagged reference suite beside a provenance-verified installation. The runtime has one exact third-party dependency for RFC 8785 JSON canonicalization.</p>

<h2 id="dry-run">Validate without dispatch</h2>
Expand Down
2 changes: 1 addition & 1 deletion site/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@
"license": "https://opensource.org/license/mit",
"programmingLanguage": "Python",
"runtimePlatform": "Linux and Python 3.11 or newer",
"version": "0.5.0",
"version": "0.6.0",
"isAccessibleForFree": true,
"keywords": "agent skill evaluation, instruction evaluation, agent harness evaluation, reproducible A/B evaluation, blinded pairwise comparison"
}
Expand Down
4 changes: 2 additions & 2 deletions site/sitemap.xml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
<url><loc>https://dhi13man.github.io/skivolve/</loc><lastmod>2026-07-17</lastmod></url>
<url><loc>https://dhi13man.github.io/skivolve/docs/</loc><lastmod>2026-07-17</lastmod></url>
<url><loc>https://dhi13man.github.io/skivolve/</loc><lastmod>2026-08-07</lastmod></url>
<url><loc>https://dhi13man.github.io/skivolve/docs/</loc><lastmod>2026-08-07</lastmod></url>
<url><loc>https://dhi13man.github.io/skivolve/corpus/</loc><lastmod>2026-07-17</lastmod></url>
<url><loc>https://dhi13man.github.io/skivolve/architecture/</loc><lastmod>2026-07-17</lastmod></url>
<url><loc>https://dhi13man.github.io/skivolve/security/</loc><lastmod>2026-07-17</lastmod></url>
Expand Down
2 changes: 1 addition & 1 deletion skivolve/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@
)
from .runner import EvalRunner, RunSelection, RunnerError

__version__ = "0.5.0"
__version__ = "0.6.0"

__all__ = [
"ClaudeCliProvider",
Expand Down
8 changes: 4 additions & 4 deletions skivolve/comparator-profile-authority.json
Original file line number Diff line number Diff line change
Expand Up @@ -4,17 +4,17 @@
{
"id": "plain-language-revision-v1",
"descriptor_sha256": "60806e964d6c415d1ebb007976acedd1286a2ec6dd788442e58f345cbbd26b30",
"production_release_sha256": "da012be22caeb1c76b37b181537a8f4e061419b35c520bde6d4dce484a8c9c40",
"test_release_sha256": "fdf61ed4ff33650a28c4431b98e8553205fb0c3aafc457844efcaad6f4c0958b",
"production_release_sha256": "aece988498ecc3427d1ca22073d7e6507306ffd79229350730055b8b649b6376",
"test_release_sha256": "6323bd82b5be4c1a232fbd892dbe8a6adbe1d5e92287443e313dac35533f3c8f",
"certification_contract_sha256": "48acdc90136e871ab9cea000c76f2ec94c517ed4466d6cba5beb57f5b4a9c265",
"requires_live_certification": true,
"authority_scope": "test"
},
{
"id": "software-engineering-v1",
"descriptor_sha256": "6a166ed3a313ee145d00519382c93c9fb56223aaf9b80f29806428f53f505e35",
"production_release_sha256": "177f65fe98ea11ca11e9f5998070ff4c5d9a5233ba362d97e43356422c6f01f0",
"test_release_sha256": "ba789cdad72490699914d2046565e8a2fc4132252e31b2d8c5e4e279ffd2554e",
"production_release_sha256": "42696ed92dcbc7e896a41458e6e229b29bf240de56da66f0fbc6730db24a709f",
"test_release_sha256": "a8809f54dea239c209d51b36dc7a4427e7fe37fbf9470489f0a8f5af72d4bad7",
"certification_contract_sha256": "48acdc90136e871ab9cea000c76f2ec94c517ed4466d6cba5beb57f5b4a9c265",
"requires_live_certification": true,
"authority_scope": "production"
Expand Down
2 changes: 1 addition & 1 deletion skivolve/comparator_calibration/release.json
Original file line number Diff line number Diff line change
Expand Up @@ -184,7 +184,7 @@
"profile_registry_source_sha256": "1582fa2eaebdf3c645f16918e2d5a62a279c176f55c1451d81aa48072b27648a",
"provider_capability_registry_source_sha256": "90871a063abcd1061c4d336589667837dd2d07ecfabcb69aa642ed5db283c07e",
"harness_manifest_source_sha256": "2d2924e7168a789cfa37ba513a30b4ceef28135d830127e833c7d3e1dc5c0569",
"harness_package_source_sha256": "eb4d6316c4ed472038178dc19ef6f10dc76916ff7d2b3075b3a88100074bbba2",
"harness_package_source_sha256": "df29ab44d3cdd23f3b668a1c3605d7d7ffed8dadd750e41d5062e0f430f3f367",
"run_evals_source_sha256": "7f1c9665fa0b5715b0ff82bb7a1449545422533fee8927ae693970bed20802d5",
"holdout_plan_source_sha256": "38981b6a9f272d02418ca99e2f2a31666a22988e4ca67606f53e50fae6332b2b",
"prepare_holdout_plan_source_sha256": "b26f8592d541a32f250699f6be77a9229b349c7f6bf893e76a1533610d1661a2",
Expand Down
2 changes: 1 addition & 1 deletion skivolve/comparator_calibration/tests/test-release.json
Original file line number Diff line number Diff line change
Expand Up @@ -166,7 +166,7 @@
"profile_registry_source_sha256": "1582fa2eaebdf3c645f16918e2d5a62a279c176f55c1451d81aa48072b27648a",
"provider_capability_registry_source_sha256": "90871a063abcd1061c4d336589667837dd2d07ecfabcb69aa642ed5db283c07e",
"harness_manifest_source_sha256": "2d2924e7168a789cfa37ba513a30b4ceef28135d830127e833c7d3e1dc5c0569",
"harness_package_source_sha256": "eb4d6316c4ed472038178dc19ef6f10dc76916ff7d2b3075b3a88100074bbba2",
"harness_package_source_sha256": "df29ab44d3cdd23f3b668a1c3605d7d7ffed8dadd750e41d5062e0f430f3f367",
"run_evals_source_sha256": "7f1c9665fa0b5715b0ff82bb7a1449545422533fee8927ae693970bed20802d5",
"holdout_plan_source_sha256": "38981b6a9f272d02418ca99e2f2a31666a22988e4ca67606f53e50fae6332b2b",
"prepare_holdout_plan_source_sha256": "b26f8592d541a32f250699f6be77a9229b349c7f6bf893e76a1533610d1661a2",
Expand Down
2 changes: 1 addition & 1 deletion skivolve/plain_language_calibration/release.json
Original file line number Diff line number Diff line change
Expand Up @@ -166,7 +166,7 @@
"profile_registry_source_sha256": "1582fa2eaebdf3c645f16918e2d5a62a279c176f55c1451d81aa48072b27648a",
"provider_capability_registry_source_sha256": "90871a063abcd1061c4d336589667837dd2d07ecfabcb69aa642ed5db283c07e",
"harness_manifest_source_sha256": "2d2924e7168a789cfa37ba513a30b4ceef28135d830127e833c7d3e1dc5c0569",
"harness_package_source_sha256": "eb4d6316c4ed472038178dc19ef6f10dc76916ff7d2b3075b3a88100074bbba2",
"harness_package_source_sha256": "df29ab44d3cdd23f3b668a1c3605d7d7ffed8dadd750e41d5062e0f430f3f367",
"run_evals_source_sha256": "7f1c9665fa0b5715b0ff82bb7a1449545422533fee8927ae693970bed20802d5",
"holdout_plan_source_sha256": "38981b6a9f272d02418ca99e2f2a31666a22988e4ca67606f53e50fae6332b2b",
"prepare_holdout_plan_source_sha256": "b26f8592d541a32f250699f6be77a9229b349c7f6bf893e76a1533610d1661a2",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -148,7 +148,7 @@
"profile_registry_source_sha256": "1582fa2eaebdf3c645f16918e2d5a62a279c176f55c1451d81aa48072b27648a",
"provider_capability_registry_source_sha256": "90871a063abcd1061c4d336589667837dd2d07ecfabcb69aa642ed5db283c07e",
"harness_manifest_source_sha256": "2d2924e7168a789cfa37ba513a30b4ceef28135d830127e833c7d3e1dc5c0569",
"harness_package_source_sha256": "eb4d6316c4ed472038178dc19ef6f10dc76916ff7d2b3075b3a88100074bbba2",
"harness_package_source_sha256": "df29ab44d3cdd23f3b668a1c3605d7d7ffed8dadd750e41d5062e0f430f3f367",
"run_evals_source_sha256": "7f1c9665fa0b5715b0ff82bb7a1449545422533fee8927ae693970bed20802d5",
"holdout_plan_source_sha256": "38981b6a9f272d02418ca99e2f2a31666a22988e4ca67606f53e50fae6332b2b",
"prepare_holdout_plan_source_sha256": "b26f8592d541a32f250699f6be77a9229b349c7f6bf893e76a1533610d1661a2",
Expand Down
Loading