feat: complete the verifiable research core - #1
Open
SHi-ON wants to merge 58 commits into
Open
Conversation
The book manifest recorded the SHA-256 of RESEARCH.md as read from a Windows checkout with CRLF line endings, while git stores LF. The Research book integrity workflow therefore failed on every Linux CI run. Normalize line endings before hashing in both the build script and the test, and re-stamp the manifest with the LF hash. Page images and PDF are unchanged because the rendered content is identical. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Define the service-boundary contracts every runtime package implements against (signers, evidence writer/reader, checkpoint service, anchor publisher, run state machine, scenario engine, learner adapter, symbol gateway, nursery runtime), the hash domain separators and stream/tree naming shared by the evidence chain, zod schemas for checkpoint manifests, run manifests, anchor receipts, Merkle proofs, turn records, intervention events, and generated-analysis entries, and the evidence bundle byte-level format. Add @ald/hashing with domain-separated SHA-256, RFC 8785 canonical JSON, Ed25519 sign/verify with raw-key encodings, per-run in-memory signer registry, and a deterministic seeded PRNG. Scaffold the remaining packages (merkle, lifecycle, scenario, learners, gateway, checkpoint, anchor, verifier, orchestrator) with build wiring; vitest now aliases every packages/* directory automatically. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…erkle, lifecycle, scenario, learners, gateway Seven packages implemented in parallel against the shared contracts: - @ald/hashing: chain-walk validator with the full LEDGER §17 single-stream mutation matrix, signed-event builder bound to the stream's signer domain, canonical JSONL parser, and a file-backed per-run Ed25519 key store (ALD-008, ALD-009). - @ald/evidence: SqliteEvidenceWriter/Reader with serialized writes, atomic sender-ledger + channel-event turn commits, rejection and oracle control events, interpretation binding to the recorded delivery, turn records, intervention and generated-analysis streams, write-time fork detection with preserved conflicting artifacts, recovery, and deterministic evidence bundle export; migration v2 adds turn_records, run_signers, fork_artifacts; child-process crash-safety tests (ALD-010, ALD-011, ALD-016). - @ald/merkle: RFC 6962 ordered trees with the LEDGER §7 leaf/node domains, inclusion and consistency proofs and verifiers (ALD-012). - @ald/lifecycle: SPEC §7.2 run state machine with exhaustive transition tests, run configuration validation and defaults builder, derived-run configuration lineage (ALD-023, ALD-024, ALD-026, ALD-028). - @ald/scenario: deterministic referential scenario engine with held-out splits, opaque references, independent receiver ordering, oracle encoding, all five §9.5 interaction profiles, and the observation hygiene filter (ALD-037, ALD-038, ALD-041). - @ald/learners: learner adapter conformance harness, no-learning reference adapter, tabular REINFORCE scratch-RL adapter with agent-native ledger drafts and ledger-to-prediction function, versioned learner contracts with a banned-pattern lint (ALD-042, ALD-043, ALD-045). - @ald/gateway: Symbol Gateway with the fixed-token protocol, all six §9.6 communication-control conditions, protocol-independent rejection framework, interpretation binding, and a conformance vector suite (ALD-029, ALD-030, ALD-034, ALD-035, ALD-036). Whole-repo build, lint, and 561 tests pass. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
- AgentActionProposalSchema is now an explicit discriminated union so proposal.kind narrows publicArtifact for every consumer. - ledgerStreamForRole returns the narrowed LedgerStream type. - EvidenceWriter gains appendAffectEvent; EvidenceReader gains readRunSigners, matching the SQLite writer's existing surface; the gateway in-memory test double implements both. - RunConfig gains an optional evaluationTurns stage budget (SPEC §8.3). - LearnerContract carries an optional track. - viem added to @ald/anchor; contract lint wired into npm run check. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…dent verifier, Nursery orchestrator - @ald/checkpoint: EvidenceCheckpointService builds LEDGER §8 manifests from the Evidence Store with independently recomputed ordered Merkle roots, witness signatures, chained previous-checkpoint hashes, interval idempotency, inclusion/consistency proof records, bundle proof-file writer, and a background CheckpointScheduler (ALD-013, ALD-014). - @ald/anchor: ChainTransport abstraction with an in-memory fake chain and a viem-backed Base transport, BaseAnchorPublisher with idempotent submission, bounded retry, confirmation polling, single terminal receipt insert, pending-file crash safety, double-opt-in mainnet gate, anchor key file handling, and the verifier-side anchor checks (ALD-018..ALD-022). - @ald/verifier: standalone verifyBundle library and ald-verify CLI that re-derives every hash, chain link, signature, Merkle root, proof, checkpoint chain, cross-binding, configuration hash, claim statement, and anchor binding from an exported bundle alone, with the full LEDGER §17 mutation matrix as tests (ALD-015, ALD-017). - @ald/orchestrator: NurseryRuntime implementing the SPEC §8.1 turn cycle across scenario engine, learner adapters, Symbol Gateway, Evidence Writer, checkpoints, experiment records, pause/resume/abort, crash recovery, fork handling, replay digest, and sealing along the spec's unanchored path (ALD-025, ALD-026, ALD-027, partial ALD-059/ALD-071). Whole-repo build, lint, and 737 tests pass. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ugh, orchestrator wiring - validateChain now honours each stream's link field (previousChannelHash for the channel transcript), removing the verifier and orchestrator workarounds and letting the pre-resume integrity check cover the channel stream. - buildRunConfig passes evaluationTurns through. - @ald/orchestrator depends on @ald/checkpoint, @ald/anchor, @ald/verifier, and the new @ald/analysis skeleton; experiment and production modules are stubbed for Phase D. - CONFIGURATION.md documents the mainnet double opt-in and key-store layout. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…or hardening - @ald/analysis: special functions (log-gamma, regularized incomplete beta, Student t, normal), descriptive statistics, one-sample t-test, TOST equivalence, Holm-Bonferroni, Wilson intervals, seeded bootstrap intervals, Cohen's h and rank-biserial effect sizes, the E03 Appendix D §D.6-§D.10 analysis, and E11 training/evaluation summaries (ALD-072 statistics scaffold). - Twin packs: baby-a/baby-b learner routes (§12.4), nursery controller, evidence, and verification routes (§12.5, §12.6), role guards and the §12.3 response envelope, a prototype-mode runtime registry, and route/role matrix and isolation tests (ALD-048..ALD-052). - Orchestrator hardening: SPEC §14.5 adapter-failure retry-then-pause, idempotent seal with retrySeal/abandonSeal, empty-message receiver path in both learner adapters so the disabled condition works with scratch-rl. Whole-repo build, lint, and 866 tests pass. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ency proofs Checkpoint 0 is created before any turn, so the auxiliary turns tree is omitted from its manifest (LEDGER §8). The checkpoint service correctly emits a consistency proof from that checkpoint with fromSize 0, but the verifier rejected it as a tree not committed by both checkpoints, failing every run with two or more checkpoints. The verifier now resolves the omission to the documented empty tree; the bundle format states the rule; the fixture ships the fromSize-0 proof and a regression test pins it. Also export the orchestrator's adapter-failure and seal-recovery symbols. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…t writer, and run script createProductionRuntime wires the Nursery runtime to the real checkpoint service, proof writer, and verifier (anchoring skipped in this environment). runE03Controls runs the six §9.6 communication conditions per Appendix D seed slot with no-learning learners and feeds seed-level success into the E03 analysis; runE11NamingGame runs the tabular scratch-RL naming game with a frozen evaluation phase and reports windowed learning curves, held-out success, vocabulary utilization, symbol entropy, policy-hash constancy, and verifier results. writeQualificationReport emits summaries and a REPORT.md that carries the Prototype Mode claim boundary and the non-confirmatory qualification label verbatim. scripts/run-qualification.mjs drives both. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Anchor: no receipt row on the confirmation give-up path (pending sidecar kept so a later poll records the confirmation), unconditional per-checkpoint idempotency with in-flight coalescing, nonce-pinned viem submissions that re-broadcast identical bytes on retry, not-found classification by viem error class only, redacted transport errors that never carry the RPC URL, a 192-block safe-tag proxy with a separate confirmation-poll budget, key-file errors that never echo the key, and an exclusive fsynced 0600 sidecar write. Checkpoint: cross-package regression test verifying an exported bundle with proof files through the independent verifier, tick interval defaulting to a quarter of the time interval, and scheduler observers that cannot poison the promise chain or re-add checkpointed events. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signature checks, hash domains, tree names, and file names are taken from the normative constants rather than trusted from run-manifest.json; every hash is rebuilt from the raw parsed file value and unknown keys are reported for manifests, configuration, checkpoints, proofs, receipts, and experiment records; bundle-relative names are contained to single segments inside the bundle; prompts/ is rebuilt into the prompt bundle hash and bound to the configuration, manifest, and every checkpoint; receipts are bound to the configured network and finality policy; experiment-record references are bound to bundle checkpoints and receipts; cross-bindings check turn agreement; the report records whether the on-chain half ran; RPC URLs are redacted; report persistence and CLI failures degrade to exit code 1 instead of losing the result. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…rom learners - hashing: canonicalJson rejects every non-JSON-native value instead of dropping data; signer objects no longer carry the raw seed. - evidence: recursive triggers so REPLACE cannot bypass append-only guards; recover() is checkpoint-aware and blocks a run whose committed prefix is shorter than, or differs from, its last checkpoint; fork-artifact stream guard fixed and constrained by migration v3. - merkle: proof verification bounds sizes and path lengths; range-root cache keyed by leaf-array identity. - lifecycle: finality policy derived from the anchor network; fixed-glyph accepts inventory and message bounds. - gateway: complexity budget on proposal inspection with a payload-too-complex rejection instead of a stack overflow. - scenario: whitespace-free prose caught by the hygiene filter. - learners: intrinsic/extrinsic configuration mismatch rejected at init; tool-only assertion checks exact artifact keys and non-enumerable fields. - types: LearnerInitContext.config is LearnerVisibleRunConfig (no randomSeed), so an adapter cannot regenerate researcher-only ground truth. Whole-repo build, lint, and 1014 tests pass. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Recovery reconstructs terminal states from evidence and is read-only for sealed, aborted, or forked runs; it refuses a signer registry that does not match the run's registered public keys. Seal idempotence rests on facts later checkpoints cannot erase, a retried abort seal keeps its abort disposition, and stage transitions are evaluated independently of pauses so a safety trigger during evaluation escalates through the §7.2 abort row instead of looping or buying an extra turn. Annotations cannot chain a checkpoint after the final one. The shuffled pre-pass stamps ledger events with their slot turn and turns a malformed envelope into a committed rejection. A nonzero verifier exit is audited with a safety trigger and deviation. The runtime no longer inserts anchor receipts itself, deferring to the publisher's single terminal row. Learners receive the configuration without randomSeed. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The E03/E11 harness and scratch-RL runtime tests drive thousands of signed turns through SQLite and exceed 60 seconds on a loaded machine. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Hypothesis and first-use records are committed to the ledger before the adapter marks them, so a retried call re-emits the same event instead of losing it. Blinding nonces are position-independent hashes of the event's own identity, so a restart cannot reuse a nonce. The exported tabular policy (version 2) carries the first-use and hypothesis registries and the intrinsic predictor, restored on same-run recovery and reset for derived runs, while the checkpoint hash still stays constant through evaluation. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Viewer-only ledger filtering on the nursery ledgers route, malformed path encoding mapped into the §12.3 envelope, inbound observations passed through the hygiene filter and bound to the pack's own role with an audited rejection, Baby tool routes gated on a live run, snapshot ids validated and contained under the session directory with fixed error messages, path-safe runId validation on run creation, the 501 code removed in favour of the §12.3 set, human-view audit events on list and delta reads, and the §14.2 annotate route. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…s, stricter conformance, shape-checked policy loading The scratch-RL receiver scores the delivered prefix and records the message verbatim, so control conditions that deliver a different length no longer abort the run. act() and onOutcome() replay the same draw and reward on a retried turn so seeded replay stays exact. The conformance harness rejects keys the schema would strip and scans agent-native ledger content for human language. Policy checkpoints from a different attribute space are refused. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ion state reports/qualification/20260907-2228-e9a8abe: non-confirmatory Prototype Mode run on commit e9a8abe — 30 E03 control runs (5 seeds x 6 conditions x 200 episodes) and 3 E11 naming-game runs (3000 training + 200 evaluation turns), all 33 evidence bundles independently verified. E03 non-oracle conditions sit at chance (0.238) with oracle adequacy 1.0 and separation lower bound 0.75; E11 evaluation success 0.93/0.975/0.98 against chance 0.25 with policy hashes constant through evaluation. Every run is disposition invalid by construction (unanchored) and nothing here is a research finding. README status describes the verifiable-core phase; BACKLOG §1/§2/§15 record the verified state and implementation decisions, with acceptance checkboxes set from agent-verified evidence; reports/README notes that E03 controls are trajectory-identical with a no-learning receiver under the Appendix D design. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…tion binding, bundle attachments, isolation and provenance seams, affect windows, causal probes; add learner track stubs and package skeletons Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Complete and stabilize the interrupted Phase E wave across alternate carriers, affect routing, learner adapters, isolation, operations, interventions, and observation red-teaming. Add evidence-backed acceptance coverage, centralize new hash domains, and bump the working patch version to 0.1.1.
Register built-in synthetic scenarios through the quarantine pipeline and require every engine bundle to be approved before run state is created. Persist the registry in production and Nursery twin wiring, add fail-closed integration tests, and bump the working patch version to 0.1.2.
Bind held-out scenario types and fixed curriculum transitions into Nursery runs, restore active stages after restart, and ship contracts for every implemented learner track. Preserve intervention-related RunConfig overrides and record transition policy hashes in the audit stream.
Freeze ledger-derived probe schedules before evaluation, apply symbolic ablations and substitutions through the Gateway, and bind each applied probe into signed turn evidence. Rebuild schedules on recovery and independently verify descriptors, artifacts, transformations, and turn links.
Schedule one extra attempt after a failed primary episode, preserve episode budgets and carrier limits, and restore pending attempts after restart. Bind repairs to signed turn records and make the standalone verifier reject missing, duplicated, or scenario-changing repair links.
Exercise the retention job against real exported bundles and SQLite evidence. Cover eligibility, disabled and active windows, public-release and Base-mainnet guards, preserved evidence rows, and the chained retention audit log.
Require Research-Grade runs to use normalized adapters that self-report distinct container IDs. Add the locked-down two-network Docker topology and an automated smoke gate that verifies capability denial, route denial, and learner failure isolation.
Exercise all enumerated side-channel categories through real Gateway submissions, transport measurements, host capability probes, and a deliberate hidden-state correlation. Reuse the ALD-040 measurement primitives and execute the joined suite in the Mode R container gate while preserving the Mode P claim boundary.
Commit the unsigned intervention chain as an auxiliary Merkle tree so attachment bindings and governance events can be proven through witness-signed checkpoints and anchors.
Store canonical analysis artifacts atomically with intervention events, witness their prefixes, export indexed hashes, link them from Experiment Records, and verify every file, binding, anchor boundary, and record reference independently.
Use an IPC readiness handshake before sending SIGTERM so the integration test cannot race the child process signal-handler installation.
Add the E40 three-role synthetic-message harness, hash-linked scheme-change and Eve-attempt records, explicit non-security reporting, and an enforced import boundary protecting production hashing and anchoring.
Validate complete interpretation batches against delayed agent-native source events, append them only to the separately signed audit stream, and checkpoint the resulting auxiliary-tree prefix. Expose the generated audit layer only through researcher-authorized reads and independently verify every exported source binding.
Add hash-bound pre-registration for recognizable-glyph and unintended-feature probes, immutable mark-level metrics across all five carrier conditions, and an evidence-bound claim gate. Preserve failed analyses as independently verifiable negative evidence and replace a timestamp-sensitive affect assertion with a structural check.
Initialize each scratch-RL Baby from an independent deterministic random stream, export the exact pre-turn policies, and witness their hashes in checkpoint zero evidence. Add deterministic timing injection to stabilize the active side-channel conformance test.
Bind child initialization evidence and run manifests to immutable parent checkpoint and policy artifacts. Export policy files and require the independent verifier to validate the parent bundle supplied for a derived run.
Record machine-readable hygiene-block intervention evidence before rejecting an observation that violates the learner-visible schema, without persisting the prohibited text.
Map every enumerated side-channel category to its active mitigation, reject any route result outside the standard response envelope, and keep the exhaustive route matrix synchronized with the exported surface.
Commit exact frozen-model provenance in checkpoint-zero initialization evidence and enforce every Research-Grade-only claim label against the recorded deployment mode before export.
Record the predictive loss and outcome-label exclusion in checkpoint-zero evidence, and independently verify both claims against the exported initial policy.
Run the full tokenizer, label-shuffled linear-probe, and text-aligned encoder battery against hash-bound learner provenance, then persist per-Baby results as witnessed evidence.
Exercise scratch RL, self-supervised, and hybrid updates in distinct containers and prove injected counterpart state is never read while local buffered updates still change policy.
Expose authoritative researcher-only status, telemetry, anchor, observation, and replay routes; add the vanilla console and a hash-bound prohibited-UX release check.
Protect checkpoint scheduling with process failure handlers, bind verifier reports to the exact manifest, persist intervention deviations, and enforce anchored confirmatory pre-registration before run start.
Close the software-readiness checks for Gates G2-G5 with executable evidence mappings, cross-track scenario equivalence, derived learner substitution, multi-seed aggregation, and the exact Mode P/Mode R comparison contract. Keep publication research judgments explicitly outside the software claim boundary.
Run the full integrity and Mode R suites for every proposed change, retain timing and JUnit evidence, enforce acceptance and API-documentation coverage, and document architecture, operations, reliability, and release controls.
Bind the first green hosted consolidated and Mode R workflow run to its exact commit and close the 20-trial CI crash-safety criterion without overstating upstream branch protection.
Inspect plaintext-bearing rejection fields while excluding opaque hashes and signatures, where short fixed-token strings can occur by chance. This preserves the raw-artifact leakage assertion without nondeterministic false positives.
Define a single evidence-driven path from the verified core through dependency integrity, reproducible E03 design, preregistration artifacts, research preflight, real open-weight qualification, and synchronized public status. Keep funded-chain, upstream-policy, and independent-operator requirements explicit as external gates.
Update the vulnerable js-yaml lock entry, explicitly approve the two reviewed native install scripts, block high and critical npm advisories, and retain the audit JSON with hosted CI evidence.
Check in the deterministic 30,000-replicate power simulation and primary/reserve seed manifest from Appendix D. Require the lower 95% Monte Carlo bound to clear 90% power and correct the SD 0.15 design row from 150 to 155 seeds.
Compile the full E03 condition matrix, primary and reserve seeds, canonical pre-registration JSON, and domain-separated hash before collection. Bind all 498 default run configurations to that one hash while retaining an explicit draft-only claim boundary.
Add a 17-check research preflight over immutable commits, Mode R, isolation, input hashes, seed coverage, canonical registration, external binding, and the matching confirmed pre-run anchor. Emit complete human and JSON blocker reports without asserting chain or scientific validity.
Expose the loopback-only local-model client and add a privacy-minimized operator command that hashes local weights, exercises both Baby roles through the shared learner contract, and writes a non-confirmatory provenance report without prompts or private outputs.
Allow the shared conformance harness to carry an exact model reference and make frozen-model qualification derive it from the hashed local weights. Exercise the path with a weights-file provenance double rather than relying on the scripted exemption.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Validation
Exact commit:
d1b4eabcde991de06eb94e6f260a62c7d4b00f8cnpm run check— 134 test files, 1,735 tests passed; secret scan passed for 578 files; zero high-severity dependency advisoriesnpm run test:mode-r— passed Docker network/process isolation, survivor behavior, twelve side-channel categories, and local-only policy updates for scratch-RL, self-supervised, and hybrid tracks9bf36f3f2e4e324c46274a0873f23e63692eb4f2fa4fc8f57ff4109a96164c997485fe6f11af29433bc51cab58009521f205840f5b4ae3a32fa7f92e8534fdf5, llama.cpp archive SHA-256819c30050d77e4305a978cacccddac8b44f21b365fe7ae67a20213f4ae4e7e92consolidated-suiteandmode-r— both passed on the exact commit; audit, runtime, JUnit, and Mode R evidence uploadedRemaining external evidence gates
The exact-head upstream workflow is awaiting maintainer approval: run 34414725835.
External registration and governance approval remain prerequisites to confirmatory collection. No experiment result is claimed.