Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions experimental/authority_resolution/CONTRIBUTION_NOTES.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# Contribution review notes

Candidate only; not submitted. GovKM is fixture author. Intended upstream licensing is ADEBench repository MIT, subject to separate approval. This archive contains public fictional fixtures only, zero holdout seed/material, no scoring or model-adapter integration. Avoid publishing until maintainer agrees exact directory/schema and oracle visibility rules.
28 changes: 28 additions & 0 deletions experimental/authority_resolution/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
# GovKM / ADEBench authority-resolution fixture generator v0.2.0

**Scope:** Public, purely synthetic fixture-generation candidate for ADEBench's proposed experimental non-Core authority-resolution track. **Not** an ADEBench scorer, model-memory adapter, or completed benchmark. No scored tests, upstream submission, private holdout, or external communication.

## Quick start (non-scored fixture self-validation)

```sh
python -m unittest discover -s tests -v
python generate_authority_cases.py --seed 20261022 --count 2 --set public --output-dir examples/public_seed_20261022
```

`--count N` generates N variants **per** six-pattern family (6N total). The seed is a nonnegative integer <2^63. The output directory must not exist; no overwrite. Holdout export is explicitly rejected. Python's PRNG stream should be replayed using the Python runtime version recorded in metadata.

## v0.2 protocol corrections

- `cases.json` includes `source_inventory_text`, which is **only the input record collection** offered for indexing. It must never be counted as delivered-to-model evidence. The only scoring boundary accepted by Simone is ADEBench's independently captured `delivered_text_after_cut`. This implementation does not create, simulate or evaluate that field. `expected_outcomes.json` is a separated private-to-evaluator oracle; never provide it to a model or retrieval system.
- Authorized supersession AR-01 requires source E1, superseding E2, and source grant G1 as **required** evidence rather than assuming supersession alone suffices. Revocation AR-04 likewise requires E1, E2 and G1.
- AR-06 recognizes two different outcomes: `NO_AUTHORITY_IF_COVERAGE_PROVEN` only where grant-register coverage (SOURCE-REGISTRY) is independently supported in model-delivered evidence; otherwise `AUTHORITY_UNRESOLVED`. `source_coverage` is a fictional source assertion, **not proof that the memory delivered it**. External score thresholds and semantic paraphrase matching remain ADEBench decisions.
- All scenarios preserve distinct `valid_at` (effective authority) versus `recorded_at` (ingestion/source chronology). AR-05 historical as-of t2 must not be superseded retroactively by t3. Exact boundary conditions remain subject to ADEBench's scoring contract.
- Three separate evaluator dimensions are declared: operative value, authority outcome and source event delivery. This generator DOES NOT implement any evaluator, cutoff algorithm, or score.

## Files and provenance

`cases.json` public fictional events and coverage assumptions; `expected_outcomes.json` candidate oracle (never feed to memory); `metadata.json` seed/version/runtime; `sha256_manifest.json` SHA-256 for output files and generator/test source; `tests/test_generator.py` local non-scored consistency checks. Source and tests are included in manifest rather than embedding self-referential hashes into source.

**Packaging boundary:** No partner records, GovKM doctrine, internal scoring methods, or proprietary AIL logic may be contributed. ADEBench controls its own scoring and experimental leaderboard; attribution of GovKM as fixture author and licensing under actual repo license must be verified at the separate upstream approval gate. ADEBench, not GovKM, retains any private holdout seeds.

**Protocol note:** `source_coverage` currently declares a fictional complete registry as a source assertion. It does not prove actual retrieved or delivered coverage. This deliberately prevents false-positive claims that "no grant found" means "no authority." An independent evaluator must define a falsifiable model-visible coverage proof before scoring NO_AUTHORITY. Do not infer a passing result from this metadata.
9 changes: 9 additions & 0 deletions experimental/authority_resolution/SOURCE_SHA256.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
{
"README.md": "9db79f9da2668a144982acb21e1c9f1097c44c05517e2de2ba6f14f45188f4db",
"examples/public_seed_20261022/cases.json": "f6c2515f64ed6ddc2226cfef19c0e9db0426092893ee8d211195cbd119f1e0a5",
"examples/public_seed_20261022/expected_outcomes.json": "b8f64ef4f018958db56a4ed0a828dab006a56cfc09c26093e91cb4410232c192",
"examples/public_seed_20261022/metadata.json": "e169842f562b59cd3bad28e0ac8c710338f29df2712ee797cd761c9a02895908",
"examples/public_seed_20261022/sha256_manifest.json": "361ac71f6d0ccfb8cb5431b3290d0860a77dbbd5deab236874b5b8921c411a5b",
"generate_authority_cases.py": "a2321ffed53aaa7a9c42fb47293c8f4875581effe76bf9760b83e676fbfe4d58",
"tests/test_generator.py": "f9d4570b94002eac7ee51ccf4cf25994554b35ae9d207a73727a2d2e79e77dbc"
}
11 changes: 11 additions & 0 deletions experimental/authority_resolution/SUBMISSION.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
# GovKM experimental authority-resolution contribution

GovKM is fixture author. The synthetic contribution is offered under this repository's MIT license; see ../../LICENSE. The original v0.2 source, example files and manifests are preserved byte for byte. README.md and CONTRIBUTION_NOTES.md describe the original candidate preparation status; this submission note records the subsequent authorized proposal.

This directory is a proposed experimental location outside Core, subject to maintainer review under issue #2. It adds no scorer, adapter, package dependency, CI configuration or leaderboard change. ADEBench retains scoring, metrics, experimental leaderboard and private holdout custody.

Only source_inventory_text is offered as unscored source input. expected_outcomes.json is a public-example evaluation oracle and must never enter memory/model input. Only independently captured delivered_text_after_cut can support future ADEBench scores. No private holdout was generated or included.

Validation: Python 3.12.14, 17/17 offline generator tests passed. Public seed 20261022, count 2 per family gives 12 cases. Cases and expected outcomes reproduce approved sample bytes. Original metadata records Python 3.13.5; current-runtime replay changes python_version and its manifest hash only. No scored benchmark or native AI-memory/model experiment was run.

Maintainer review remains needed for directory/schema, public-oracle placement, delivered coverage proof for no-authority cases, temporal boundaries and matching/cut rules. No claim of adoption before merge.

Large diffs are not rendered by default.

Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
[{"case_id":"AR-01-S20261022-V0001","conflicting_event_ids":[],"delivery_boundary":"ADEBench measured delivered_text_after_cut only; source_inventory_text never scores","expected_resolution":"RESOLVED","historical_event_ids":["E1"],"independent_dimensions":["operative_value","authority_outcome","source_event_delivery"],"interpretation_note":"Expected event and authority evidence pertains to delivered text after cut, not retrieval pool or final model answer.","operative_values":["Fern"],"required_event_ids":["E1","E2"],"required_grant_ids":["G1"],"supersession_event_ids":["E2"],"temporal_basis":"valid_at; recorded_at provenance only"},{"case_id":"AR-01-S20261022-V0002","conflicting_event_ids":[],"delivery_boundary":"ADEBench measured delivered_text_after_cut only; source_inventory_text never scores","expected_resolution":"RESOLVED","historical_event_ids":["E1"],"independent_dimensions":["operative_value","authority_outcome","source_event_delivery"],"interpretation_note":"Expected event and authority evidence pertains to delivered text after cut, not retrieval pool or final model answer.","operative_values":["Amber"],"required_event_ids":["E1","E2"],"required_grant_ids":["G1"],"supersession_event_ids":["E2"],"temporal_basis":"valid_at; recorded_at provenance only"},{"case_id":"AR-02-S20261022-V0001","conflicting_event_ids":[],"delivery_boundary":"ADEBench measured delivered_text_after_cut only; source_inventory_text never scores","disallowed_event_ids":["E2"],"expected_resolution":"RESOLVED","historical_event_ids":[],"independent_dimensions":["operative_value","authority_outcome","source_event_delivery"],"interpretation_note":"Expected event and authority evidence pertains to delivered text after cut, not retrieval pool or final model answer.","operative_values":["Ember"],"required_event_ids":["E1"],"required_grant_ids":["G1"],"supersession_event_ids":[],"temporal_basis":"valid_at; recorded_at provenance only"},{"case_id":"AR-02-S20261022-V0002","conflicting_event_ids":[],"delivery_boundary":"ADEBench measured delivered_text_after_cut only; source_inventory_text never scores","disallowed_event_ids":["E2"],"expected_resolution":"RESOLVED","historical_event_ids":[],"independent_dimensions":["operative_value","authority_outcome","source_event_delivery"],"interpretation_note":"Expected event and authority evidence pertains to delivered text after cut, not retrieval pool or final model answer.","operative_values":["Jade"],"required_event_ids":["E1"],"required_grant_ids":["G1"],"supersession_event_ids":[],"temporal_basis":"valid_at; recorded_at provenance only"},{"case_id":"AR-03-S20261022-V0001","conflicting_event_ids":["E1","E2"],"delivery_boundary":"ADEBench measured delivered_text_after_cut only; source_inventory_text never scores","expected_resolution":"CONFLICT_UNRESOLVED","historical_event_ids":[],"independent_dimensions":["operative_value","authority_outcome","source_event_delivery"],"interpretation_note":"Expected event and authority evidence pertains to delivered text after cut, not retrieval pool or final model answer.","operative_values":[],"required_event_ids":["E1","E2"],"required_grant_ids":["G1","G2"],"supersession_event_ids":[],"temporal_basis":"valid_at; recorded_at provenance only"},{"case_id":"AR-03-S20261022-V0002","conflicting_event_ids":["E1","E2"],"delivery_boundary":"ADEBench measured delivered_text_after_cut only; source_inventory_text never scores","expected_resolution":"CONFLICT_UNRESOLVED","historical_event_ids":[],"independent_dimensions":["operative_value","authority_outcome","source_event_delivery"],"interpretation_note":"Expected event and authority evidence pertains to delivered text after cut, not retrieval pool or final model answer.","operative_values":[],"required_event_ids":["E1","E2"],"required_grant_ids":["G1","G2"],"supersession_event_ids":[],"temporal_basis":"valid_at; recorded_at provenance only"},{"case_id":"AR-04-S20261022-V0001","conflicting_event_ids":[],"delivery_boundary":"ADEBench measured delivered_text_after_cut only; source_inventory_text never scores","expected_resolution":"NO_AUTHORITY","historical_event_ids":["E1"],"independent_dimensions":["operative_value","authority_outcome","source_event_delivery"],"interpretation_note":"Expected event and authority evidence pertains to delivered text after cut, not retrieval pool or final model answer.","operative_values":[],"required_event_ids":["E1","E2"],"required_grant_ids":["G1"],"supersession_event_ids":["E2"],"temporal_basis":"valid_at; recorded_at provenance only"},{"case_id":"AR-04-S20261022-V0002","conflicting_event_ids":[],"delivery_boundary":"ADEBench measured delivered_text_after_cut only; source_inventory_text never scores","expected_resolution":"NO_AUTHORITY","historical_event_ids":["E1"],"independent_dimensions":["operative_value","authority_outcome","source_event_delivery"],"interpretation_note":"Expected event and authority evidence pertains to delivered text after cut, not retrieval pool or final model answer.","operative_values":[],"required_event_ids":["E1","E2"],"required_grant_ids":["G1"],"supersession_event_ids":["E2"],"temporal_basis":"valid_at; recorded_at provenance only"},{"case_id":"AR-05-S20261022-V0001","conflicting_event_ids":[],"delivery_boundary":"ADEBench measured delivered_text_after_cut only; source_inventory_text never scores","expected_resolution":"RESOLVED","future_event_ids":["E2"],"historical_event_ids":[],"independent_dimensions":["operative_value","authority_outcome","source_event_delivery"],"interpretation_note":"Expected event and authority evidence pertains to delivered text after cut, not retrieval pool or final model answer.","operative_values":["Fern"],"required_event_ids":["E1"],"required_grant_ids":["G1"],"supersession_event_ids":[],"temporal_basis":"valid_at; recorded_at provenance only"},{"case_id":"AR-05-S20261022-V0002","conflicting_event_ids":[],"delivery_boundary":"ADEBench measured delivered_text_after_cut only; source_inventory_text never scores","expected_resolution":"RESOLVED","future_event_ids":["E2"],"historical_event_ids":[],"independent_dimensions":["operative_value","authority_outcome","source_event_delivery"],"interpretation_note":"Expected event and authority evidence pertains to delivered text after cut, not retrieval pool or final model answer.","operative_values":["Granite"],"required_event_ids":["E1"],"required_grant_ids":["G1"],"supersession_event_ids":[],"temporal_basis":"valid_at; recorded_at provenance only"},{"case_id":"AR-06-S20261022-V0001","conflicting_event_ids":[],"delivery_boundary":"ADEBench measured delivered_text_after_cut only; source_inventory_text never scores","expected_resolution":"NO_AUTHORITY_IF_COVERAGE_PROVEN","historical_event_ids":[],"incomplete_coverage_resolution":"AUTHORITY_UNRESOLVED","independent_dimensions":["operative_value","authority_outcome","source_event_delivery"],"interpretation_note":"Expected event and authority evidence pertains to delivered text after cut, not retrieval pool or final model answer.","operative_values":[],"required_coverage_refs":["SOURCE-REGISTRY"],"required_event_ids":["E1"],"required_grant_ids":[],"supersession_event_ids":[],"temporal_basis":"valid_at; recorded_at provenance only"},{"case_id":"AR-06-S20261022-V0002","conflicting_event_ids":[],"delivery_boundary":"ADEBench measured delivered_text_after_cut only; source_inventory_text never scores","expected_resolution":"NO_AUTHORITY_IF_COVERAGE_PROVEN","historical_event_ids":[],"incomplete_coverage_resolution":"AUTHORITY_UNRESOLVED","independent_dimensions":["operative_value","authority_outcome","source_event_delivery"],"interpretation_note":"Expected event and authority evidence pertains to delivered text after cut, not retrieval pool or final model answer.","operative_values":[],"required_coverage_refs":["SOURCE-REGISTRY"],"required_event_ids":["E1"],"required_grant_ids":[],"supersession_event_ids":[],"temporal_basis":"valid_at; recorded_at provenance only"}]
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
{"case_count":12,"classification":"PUBLIC_SYNTHETIC","count_per_pattern":2,"generator_version":"0.2.0","patterns":["AR-01","AR-02","AR-03","AR-04","AR-05","AR-06"],"prng":"python.random.Random","python_version":"3.13.5","schema_version":"2.0","seed":20261022,"source_inventory_is_not_measured_delivery":true}
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
{"algorithm":"SHA-256","files":{"cases.json":"f6c2515f64ed6ddc2226cfef19c0e9db0426092893ee8d211195cbd119f1e0a5","expected_outcomes.json":"b8f64ef4f018958db56a4ed0a828dab006a56cfc09c26093e91cb4410232c192","generate_authority_cases.py":"a2321ffed53aaa7a9c42fb47293c8f4875581effe76bf9760b83e676fbfe4d58","metadata.json":"e169842f562b59cd3bad28e0ac8c710338f29df2712ee797cd761c9a02895908","tests/test_generator.py":"f9d4570b94002eac7ee51ccf4cf25994554b35ae9d207a73727a2d2e79e77dbc"},"generator_version":"0.2.0","scope":"public fixture files and local generator/test sources only"}
Loading
Loading