verifiablelabs · stelioszach03 · Jul 10, 2026 · Jul 10, 2026
diff --git a/.github/dependabot.yml b/.github/dependabot.yml
@@ -0,0 +1,9 @@
+version: 2
+updates:
+  - package-ecosystem: "github-actions"
+    directory: "/"
+    schedule:
+      interval: "weekly"
+    open-pull-requests-limit: 5
+    commit-message:
+      prefix: "ci"
diff --git a/.github/scripts/run-gitleaks.sh b/.github/scripts/run-gitleaks.sh
@@ -0,0 +1,15 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+version="8.30.1"
+archive="gitleaks_${version}_linux_x64.tar.gz"
+sha256="551f6fc83ea457d62a0d98237cbad105af8d557003051f41f3e7ca7b3f2470eb"
+tool_dir="${RUNNER_TEMP:-${TMPDIR:-/tmp}}/vlabs-gitleaks-${version}"
+
+mkdir -p "$tool_dir"
+curl --fail --silent --show-error --location --retry 3 \
+  --output "$tool_dir/$archive" \
+  "https://github.com/gitleaks/gitleaks/releases/download/v${version}/${archive}"
+printf '%s  %s\n' "$sha256" "$tool_dir/$archive" | sha256sum --check --status
+tar -xzf "$tool_dir/$archive" -C "$tool_dir" gitleaks
+"$tool_dir/gitleaks" git . --redact=100 --no-banner --no-color --timeout 300
diff --git a/.github/workflows/validate.yml b/.github/workflows/validate.yml
@@ -2,11 +2,17 @@ name: validate-evidence
 on:
   push: { branches: [main] }
   pull_request: { branches: [main] }
+permissions:
+  contents: read
 jobs:
   validate:
     runs-on: ubuntu-latest
     steps:
-      - uses: actions/checkout@v4
-      - uses: actions/setup-python@v5
+      - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4.3.1
+        with: { fetch-depth: 0, persist-credentials: false }
+      - uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
         with: { python-version: "3.12" }
+      - name: Gitleaks (blocking, full history, redacted)
+        run: bash .github/scripts/run-gitleaks.sh
+      - run: python -m unittest discover -s tests -v
       - run: python scripts/validate_evidence.py
diff --git a/PROVENANCE.md b/PROVENANCE.md
@@ -1,8 +1,10 @@
 # Provenance
 
-Clean import (no history rewrite) from `verifiablelabs/verifiable-labs-envs`
-at commit `a0f30dc547a73aaae8608d193f94035192404627` (main). Validation tooling authored fresh; evidence artifacts are synthetic/illustrative until first consented evidence pack.
+Clean import (no history rewrite) from the legacy source repository at commit
+`a0f30dc547a73aaae8608d193f94035192404627`. Validation tooling was authored
+fresh. `evidence/` and export manifests are synthetic/illustrative; `results/`
+contains explicitly labelled real measurements on public benchmark data, not
+customer evidence.
 
-The source monorepo remains canonical until the split flips; this mirror is
-refreshed by the migration tooling documented in
-`verifiable-labs-private/docs/ops/github-repo-split-migration.md`.
+This split repository is canonical for public evidence artifacts. Historical
+source references are retained only for provenance.
diff --git a/README.md b/README.md
@@ -5,10 +5,10 @@ sample assurance cards, aggregate metrics, and sanitized export manifests.
 
 > Verifiable Labs builds clean feedback and promotion gates for increasingly general AI agents.
 
-**Everything currently in this repository is illustrative** — synthetic
-numbers published to show the *shape* of the evidence, not real customer
-results. Real (consented, redacted) evidence packs will be added here as
-they are produced, and will be clearly labeled.
+Artifacts under `evidence/` and the HF/W&B manifests are **illustrative** —
+synthetic numbers published to show the evidence format, not customer results.
+`results/` separately contains clearly labelled measurements reproduced on
+public benchmark datasets. There are no customer results in this repository.
 
 What is never published, here or anywhere: hidden evaluation content, gold
 answers, anti-hack detection details, raw traces, customer data, secrets,
@@ -34,9 +34,9 @@ evidence to Hugging Face and Weights & Biases:
 
 The artifacts they reference are **synthetic / redacted**. They contain
 **no** hidden evals, gold answers, raw traces, customer data, secrets,
-private anti-hack traps, or private engine internals — every record is
-routed through the platform's `assert_public_export_safe` policy gate, which
-refuses any such content and any `restricted`/`secret` classification.
+private anti-hack traps, or private engine internals. Export tooling routes
+candidates through the platform's `assert_public_export_safe` policy gate,
+which refuses such content and any `restricted`/`secret` classification.
 
 HF and W&B uploads are **gated by explicit approval flags**
 (`VLABS_APPROVE_HF_UPLOAD=1`, `VLABS_APPROVE_WANDB_UPLOAD=1`); the exporters
@@ -49,9 +49,13 @@ Public, **synthetic / redacted** demo evidence:
 - **Hugging Face dataset** — <https://huggingface.co/datasets/verifiablelabs/vlabs-clean-gate-evidence>
 - **Weights & Biases** (entity `verifiable-labs`): [clean-generalization-gate](https://wandb.ai/verifiable-labs/clean-generalization-gate) · [contamination-firewall](https://wandb.ai/verifiable-labs/contamination-firewall) · [anti-hack-engine](https://wandb.ai/verifiable-labs/anti-hack-engine) · [scenario-compiler](https://wandb.ai/verifiable-labs/scenario-compiler) · [runpod-costs](https://wandb.ai/verifiable-labs/runpod-costs)
 
-All published evidence is **synthetic / redacted** and is **not a training
-dataset**. It contains **no** customer data, hidden evaluations, gold
-answers, raw traces, private anti-hack traps, or private engine internals.
+The HF/W&B demo evidence is **synthetic / redacted** and is **not a training
+dataset**. The separate JSON files under `results/` are real public-benchmark
+measurements and contain no customer data, hidden evaluations, gold answers,
+raw traces, private anti-hack traps, or private engine internals.
+
+`SHA256SUMS` commits the exact bytes of every JSON/Markdown artifact under
+`evidence/`, `results/`, and `manifests/`; CI rejects missing or stale hashes.
 
 Install the SDK: `pip install "vlabs-sdk==0.0.2"`
 
@@ -63,4 +67,4 @@ pip install "vlabs-sdk==0.0.2"
 
 ## Formal scope
 
-Selected mathematical properties behind the contamination-resistant promotion gate are machine-verified in Lean 4. The implementation is property-tested against the formal specification.
+Selected mathematical properties behind the contamination-resistant promotion gate are machine-verified in Lean 4. A hand-maintained Python mirror has property tests derived from selected definitions; no mechanized code-to-proof parity is claimed.
diff --git a/SHA256SUMS b/SHA256SUMS
@@ -0,0 +1,7 @@
+c40311c3b39c1c65ae37773b5ec17822201eda773eb23aacc0f65a2f405a8edc  evidence/aggregate_metrics_sample.md
+18aa69a44def4c0d52c5a31917200c61aacd35f54b3bd24ccf7ad39e7a96063c  evidence/sample_assurance_card_redacted.json
+cce9d30b37e38c3124297e76cc63fe5aece4a193b2f97130ccf987e053f2214e  manifests/evidence_safety_report.md
+e6b8d44851ed167ef96a466aa4fd96223a620157a143010d293050a708ee2217  manifests/hf_upload_manifest.json
+9282a42690917c9328bc8f2a327c47e284a5f383dc9a0acd1cd62322877610d3  manifests/wandb_upload_manifest.json
+9a097306fc1d4b0106f16f76b9ad219b211ef1b36ded26cff81fd49cfac618ff  results/atlas_honeypot_ipt.json
+e3333f51e21e3390c347faf304ce54d5c93a5ec5059fce654e676375154b5c59  results/conformal_fp_control_mbpp.json
diff --git a/evidence/aggregate_metrics_sample.md b/evidence/aggregate_metrics_sample.md
@@ -10,7 +10,7 @@
 | OOD score | 0.69 | 0.71 | +0.02 |
 | Adversarial score | 0.62 | 0.63 | +0.01 |
 | Data-contamination risk (DCR) | 0.03 | 0.03 | 0.00 |
-| Clean VGS | 0.61 | 0.68 | +0.07 |
+| Clean VGS | 0.61 | 0.679 | +0.069 |
 | Generalization gap (public − hidden) | 0.07 | 0.06 | −0.01 |
 
 Gate decision for this (synthetic) pair: **ACCEPT** — clean VGS improved

diff --git a/evidence/sample_assurance_card_redacted.json b/evidence/sample_assurance_card_redacted.json
@@ -1,14 +1,28 @@
 {
-  "_comment": "ILLUSTRATIVE EXAMPLE — synthetic numbers, fake IDs, fields redacted as they would be for a real customer. Not a real evaluation.",
   "card_version": "v2",
   "run_id": "run_redacted_xxxx",
-  "org": "REDACTED",
-  "agent": "REDACTED",
-  "scores": { "public": 0.77, "hidden": 0.70, "ood": 0.66, "adversarial": 0.58 },
-  "contamination": { "dcr": 0.05 },
-  "clean_vgs": { "baseline": 0.55, "candidate": 0.59 },
+  "agent_id": "REDACTED",
+  "baseline_id": "REDACTED",
+  "candidate_id": "REDACTED",
+  "decision": "LIMITED_ROLLOUT",
+  "raw_vgs": 0.65,
+  "dcr": 0.05,
+  "clean_vgs": 0.5925,
+  "public_score": 0.77,
+  "hidden_score": 0.70,
+  "ood_score": 0.66,
   "generalization_gap": 0.07,
-  "gate": { "outcome": "LIMITED_ROLLOUT", "reasons": ["ood_regressed"] },
+  "reject_reasons": ["ood_regressed"],
   "redaction_status": "redacted_public_safe",
-  "formal_claim": "Selected mathematical properties behind the contamination-resistant promotion gate are machine-verified in Lean 4. The implementation is property-tested against the formal specification."
+  "hf_public_safe": true,
+  "formal_claim": "Selected mathematical properties behind the contamination-resistant promotion gate are machine-verified in Lean 4. A hand-maintained Python mirror has property tests derived from selected definitions; no mechanized code-to-proof parity is claimed.",
+  "formal_scope": "Selected mathematical properties behind the contamination-resistant promotion gate are machine-verified in Lean 4. A hand-maintained Python mirror has property tests derived from selected definitions; no mechanized code-to-proof parity is claimed.",
+  "metadata": {
+    "illustrative": true,
+    "comment": "Synthetic numbers, fake IDs, and redacted fields; not a real evaluation.",
+    "org": "REDACTED",
+    "baseline_clean_vgs": 0.55,
+    "adversarial_score": 0.58,
+    "clean_vgs_beta": 0.5
+  }
 }
diff --git a/manifests/evidence_safety_report.md b/manifests/evidence_safety_report.md
@@ -14,7 +14,10 @@ Generated locally. **No upload performed.** Approval flags VLABS_APPROVE_HF_UPLO
 | docs_claims | PASS |
 | synthetic_markers | PASS |
 
-¹ The only matches were the manifest safety-flags  and  — redaction metadata asserting *absence*, not content. No hidden-eval/gold-answer/raw-trace content is present.
+¹ The only matches were the manifest safety flags
+`hidden_evals_included=false` and `raw_traces_included=false` — metadata
+asserting absence, not content. No hidden-eval, gold-answer, or raw-trace
+content is present.
 
 ## Export-guard negative tests (assert_public_export_safe)
 - hidden_eval → REFUSED · gold_answer → REFUSED · raw_trace → REFUSED · secret → REFUSED

diff --git a/manifests/hf_upload_manifest.json b/manifests/hf_upload_manifest.json
@@ -5,39 +5,39 @@
     "classification": "public_demo",
     "records": [
       {
-        "_comment": "ILLUSTRATIVE synthetic example \u2014 fake IDs, redacted fields.",
         "card_version": "v2",
         "run_id": "run_redacted_xxxx",
-        "org": "REDACTED",
-        "agent": "REDACTED",
-        "scores": {
-          "public": 0.77,
-          "hidden": 0.7,
-          "ood": 0.66,
-          "adversarial": 0.58
-        },
-        "contamination": {
-          "dcr": 0.05
-        },
-        "clean_vgs": {
-          "baseline": 0.55,
-          "candidate": 0.59
-        },
+        "agent_id": "REDACTED",
+        "baseline_id": "REDACTED",
+        "candidate_id": "REDACTED",
+        "decision": "LIMITED_ROLLOUT",
+        "raw_vgs": 0.65,
+        "dcr": 0.05,
+        "clean_vgs": 0.5925,
+        "public_score": 0.77,
+        "hidden_score": 0.7,
+        "ood_score": 0.66,
         "generalization_gap": 0.07,
-        "gate": {
-          "outcome": "LIMITED_ROLLOUT",
-          "reasons": [
-            "ood_regressed"
-          ]
-        },
+        "reject_reasons": ["ood_regressed"],
         "redaction_status": "redacted_public_safe",
-        "formal_claim": "Selected mathematical properties behind the contamination-resistant promotion gate are machine-verified in Lean 4. The implementation is property-tested against the formal specification."
+        "hf_public_safe": true,
+        "formal_claim": "Selected mathematical properties behind the contamination-resistant promotion gate are machine-verified in Lean 4. A hand-maintained Python mirror has property tests derived from selected definitions; no mechanized code-to-proof parity is claimed.",
+        "formal_scope": "Selected mathematical properties behind the contamination-resistant promotion gate are machine-verified in Lean 4. A hand-maintained Python mirror has property tests derived from selected definitions; no mechanized code-to-proof parity is claimed.",
+        "metadata": {
+          "illustrative": true,
+          "comment": "Synthetic numbers, fake IDs, and redacted fields; not a real evaluation.",
+          "org": "REDACTED",
+          "baseline_clean_vgs": 0.55,
+          "adversarial_score": 0.58,
+          "clean_vgs_beta": 0.5
+        }
       },
       {
         "kind": "aggregate_metrics",
-        "raw_vgs": 0.61,
+        "raw_vgs": 0.7,
         "dcr": 0.03,
-        "clean_vgs": 0.68,
+        "clean_vgs": 0.679,
+        "clean_vgs_beta": 0.0,
         "public": 0.84,
         "hidden": 0.78,
         "ood": 0.71,
@@ -54,23 +54,18 @@
     "card": {
       "pretty_name": "vlabs-clean-gate-evidence",
       "license": "apache-2.0",
-      "task_categories": [
-        "text-generation",
-        "feature-extraction"
-      ],
-      "intended_use": "Public evaluation evidence for the contamination-resistant promotion gate: aggregate scores, generalization-gap deltas, and redacted assurance cards for reproducing the published comparisons. Not a training set.",
-      "limitations": "Contains only public-demo records. Hidden evaluations, gold answers, raw agent traces, anti-hack traps, and customer data are excluded by policy. Scores are aggregate; per-instance private content is never published.",
-      "contamination_policy": "Public instances are generated after the evaluation freeze; hidden splits are never released. See the contamination firewall documentation.",
-      "formal_scope": "Selected mathematical properties behind the contamination-resistant promotion gate are machine-verified in Lean 4. The implementation is property-tested against the formal specification.",
+      "task_categories": ["text-generation", "feature-extraction"],
+      "intended_use": "Public synthetic evaluation evidence for the contamination-resistant promotion gate. Not a training set.",
+      "limitations": "Contains only public-demo records. Hidden evaluations, gold answers, raw agent traces, anti-hack traps, and customer data are excluded by policy.",
+      "contamination_policy": "Public instances are generated after the evaluation freeze; hidden splits are never released.",
+      "formal_scope": "Selected mathematical properties behind the contamination-resistant promotion gate are machine-verified in Lean 4. A hand-maintained Python mirror has property tests derived from selected definitions; no mechanized code-to-proof parity is claimed.",
       "classification": "public_demo",
       "redaction_status": "redacted"
     },
     "export_decision": {
       "allowed": false,
       "dry_run": true,
-      "reasons": [
-        "approval flag VLABS_APPROVE_HF_UPLOAD not set \u2014 dry-run only"
-      ]
+      "reasons": ["approval flag VLABS_APPROVE_HF_UPLOAD not set — dry-run only"]
     },
     "redaction": {
       "status": "redacted",
@@ -86,4 +81,4 @@
     "dataset": "vlabs-clean-gate-evidence",
     "reason": "dry-run (default)"
   }
-}
+}
diff --git a/manifests/wandb_upload_manifest.json b/manifests/wandb_upload_manifest.json
@@ -3,9 +3,10 @@
     "target": "wandb",
     "project": "clean-generalization-gate",
     "metrics": {
-      "raw_vgs": 0.61,
+      "raw_vgs": 0.7,
       "dcr": 0.03,
-      "clean_vgs": 0.68,
+      "clean_vgs": 0.679,
+      "clean_vgs_beta": 0.0,
       "public": 0.84,
       "hidden": 0.78,
       "ood": 0.71,
@@ -19,39 +20,38 @@
     },
     "artifacts": {
       "assurance_card": {
-        "_comment": "ILLUSTRATIVE synthetic example \u2014 fake IDs, redacted fields.",
         "card_version": "v2",
         "run_id": "run_redacted_xxxx",
-        "org": "REDACTED",
-        "agent": "REDACTED",
-        "scores": {
-          "public": 0.77,
-          "hidden": 0.7,
-          "ood": 0.66,
-          "adversarial": 0.58
-        },
-        "contamination": {
-          "dcr": 0.05
-        },
-        "clean_vgs": {
-          "baseline": 0.55,
-          "candidate": 0.59
-        },
+        "agent_id": "REDACTED",
+        "baseline_id": "REDACTED",
+        "candidate_id": "REDACTED",
+        "decision": "LIMITED_ROLLOUT",
+        "raw_vgs": 0.65,
+        "dcr": 0.05,
+        "clean_vgs": 0.5925,
+        "public_score": 0.77,
+        "hidden_score": 0.7,
+        "ood_score": 0.66,
         "generalization_gap": 0.07,
-        "gate": {
-          "outcome": "LIMITED_ROLLOUT",
-          "reasons": [
-            "ood_regressed"
-          ]
-        },
+        "reject_reasons": ["ood_regressed"],
         "redaction_status": "redacted",
-        "formal_claim": "Selected mathematical properties behind the contamination-resistant promotion gate are machine-verified in Lean 4. The implementation is property-tested against the formal specification.",
-        "formal_scope": "Selected mathematical properties behind the contamination-resistant promotion gate are machine-verified in Lean 4. The implementation is property-tested against the formal specification."
+        "hf_public_safe": true,
+        "formal_claim": "Selected mathematical properties behind the contamination-resistant promotion gate are machine-verified in Lean 4. A hand-maintained Python mirror has property tests derived from selected definitions; no mechanized code-to-proof parity is claimed.",
+        "formal_scope": "Selected mathematical properties behind the contamination-resistant promotion gate are machine-verified in Lean 4. A hand-maintained Python mirror has property tests derived from selected definitions; no mechanized code-to-proof parity is claimed.",
+        "metadata": {
+          "illustrative": true,
+          "comment": "Synthetic numbers, fake IDs, and redacted fields; not a real evaluation.",
+          "org": "REDACTED",
+          "baseline_clean_vgs": 0.55,
+          "adversarial_score": 0.58,
+          "clean_vgs_beta": 0.5
+        }
       },
       "aggregate_metrics": {
-        "raw_vgs": 0.61,
+        "raw_vgs": 0.7,
         "dcr": 0.03,
-        "clean_vgs": 0.68,
+        "clean_vgs": 0.679,
+        "clean_vgs_beta": 0.0,
         "public": 0.84,
         "hidden": 0.78,
         "ood": 0.71,
@@ -78,4 +78,4 @@
     "project": "clean-generalization-gate",
     "reason": "dry-run (default)"
   }
-}
+}
diff --git a/reproducibility-notes.md b/reproducibility-notes.md
@@ -1,17 +1,26 @@
 # Reproducibility notes
 
-Every future evidence pack in this repository will state, per artifact:
+Every new evidence pack in this repository must state, per artifact:
 
 - the evaluation-contract hash and run id it came from,
 - the seed policy (train / val / test split seeds never overlap),
-- which numbers are measured vs derived (clean score = raw x (1 - DCR)),
+- which numbers are measured vs derived
+  (`clean_vgs = raw_vgs × (1 − DCR) − beta × DCR`),
 - the redaction profile applied before publication.
 
 Current artifacts under [`evidence/`](evidence/) are **synthetic/illustrative**
 and exist to fix the format. Links to W&B dashboards / HF datasets will be added
 only after the corresponding export approvals
 (`VLABS_APPROVE_{WANDB_UPLOAD,HF_UPLOAD}`).
 
+[`SHA256SUMS`](SHA256SUMS) covers every current JSON/Markdown artifact under
+`evidence/`, `results/`, and `manifests/`. It provides repository artifact
+integrity, not independent attestation of the historical benchmark execution.
+The two legacy public-benchmark results below predate the hash policy and did
+not record an immutable source commit or dataset revision at run time; their
+reproduction commands are retained, and a future rerun should add those exact
+revisions rather than inventing retrospective provenance.
+
 ---
 
 ## Real, reproducible result — IPT on the Obfuscation Atlas MBPP-Honeypot (public data)