Evidence before adjectives. A vendor-neutral vocabulary for evaluating authority and execution controls in agentic systems.
Status: Draft v0.2 (opt-in; v0.1 preserved) · No aggregate score · Not a certification · Not a product ranking
When an agent runtime says “the tool call was authorized”, that statement hides several independent security and governance questions. Agent Authority Conformance separates those questions so implementations can state precisely what they have demonstrated, what remains untested, and what they deliberately do not claim.
The project evaluates evidence, not marketing language.
| ID | Property | Core question |
|---|---|---|
| A | Receipt Integrity | Is the authorization artifact authentic, unmodified, valid and correctly replay-bounded? |
| B | Authority Provenance | Can the system establish who or what had authority to approve the action? |
| C | Exact-Call Integrity | Is authorization bound to the exact tool call that is executed? |
| D | Semantic Authority | Was the action permitted in its real operational meaning? |
| E | Execution-Boundary Integrity | Can the protected effect be reached only through the governed path? |
| F | TOCTOU Resistance | Can approved conditions change between authorization and execution? |
| G | Effect Verification | Does the system verify the actual external effect after execution? |
These properties are deliberately separable. Evidence for one property grants no credit in another.
v0.2 records verification execution separately from the property result:
| Verification | Property status | Meaning |
|---|---|---|
NOT_RUN |
UNTESTED |
No applicable verification completed. |
NOT_RUN |
OUT_OF_SCOPE |
Explicit non-claim within the assessment scope. |
COMPLETED |
PASS / FAIL |
Resolved evidence establishes the bounded property or its violation. |
COMPLETED |
NOT_ESTABLISHED |
The procedure ran, but evidence supports neither conclusion. |
UNSUPPORTED / INVALID_INPUT / ERROR |
null |
A non-verdict with a structured verifier error. |
A runtime success, valid receipt or passing test expectation is not a property verdict. Missing evidence alone is never a demonstrated violation.
- RESOLVED — the assessor inspected the named test, fixture, artifact or code at a stated revision.
- REPORTED — the evidence was described but not independently resolved to an artifact.
- NONE — no evidence was offered.
In v0.2, PASS and FAIL require nonempty, pinned RESOLVED evidence.
Resolving an artifact does not alone prove that it is sufficient.
Do not aggregate A–G into a score.
No percentage, star rating, “5/7 conformant”, security grade or ranking is valid under this model. The properties are not commensurable and deployment priorities differ.
| Path | Purpose |
|---|---|
SPECIFICATION-v0.2.md |
Draft v0.2 semantics and bounded E evidence rules |
schema/assessment-v0.2.schema.json |
Opt-in v0.2 assessment schema |
MIGRATION-v0.2.md |
Per-record migration and compatibility rules |
SPECIFICATION.md |
Preserved normative v0.1 specification |
schema/assessment.schema.json |
Preserved v0.1 schema |
examples/ |
Versioned examples and historical assessments |
conformance/ |
Validation and bounded E reference inference |
tests/fixtures/ |
Synthetic, committed evidence-sufficiency cases |
CONTRIBUTING.md |
Evidence and contribution discipline |
Python 3.12 or newer:
python -m venv .venv
# Linux/macOS: . .venv/bin/activate
# Windows PowerShell: .venv/Scripts/Activate.ps1
python -m pip install -r requirements-dev.txt
python -m conformance.check
python -m unittest discover -s tests -vActivate the environment with the command for your platform before installing. The checks run offline after dependency installation and do not invoke external tools or regenerate committed expectations.
The E reference rule consumes already accepted evidence. It does not verify artifacts, inspect credential custody or establish coverage itself. Its synthetic fixtures test inference semantics, not the security of a deployed product. See the input trust boundary.
This complete, schema-valid v0.2 skeleton intentionally grants no conformance
credit. It is also committed as examples/v0.2/minimal.json.
{
"assessed_at": "2026-09-15",
"properties": [
{
"caveat": "This is a valid assessment skeleton and grants no conformance credit.",
"evidence": [],
"evidence_tier": "NONE",
"id": "C",
"property": "Exact-Call Integrity",
"reasoning": "No verification procedure has been run for this example.",
"status": "UNTESTED",
"verification_status": "NOT_RUN"
}
],
"revision": "not-assessed",
"scope": {
"explicit_non_claims": [],
"external_effects": "none",
"process_model": "not-assessed",
"trust_assumptions": []
},
"spec_version": "0.2",
"system": "Example Runtime (unassessed)",
"system_version": "not-assessed"
}The schema intentionally contains no aggregate score field. Schema validation checks report structure; the assessor remains responsible for the evidence.
- Separate properties. Receipt correctness is not authority provenance; exact binding is not semantic correctness; dispatch success is not effect verification.
- Resolve claims to immutable evidence. Prefer test path + revision + command over README prose.
- Declare scope. Process model, deployment assumptions, simulated vs real effects and explicit non-claims belong in the assessment.
- Treat missing evidence conservatively. Distinguish
UNTESTEDfrom completed-but-inconclusiveNOT_ESTABLISHED; neither is a demonstrated violation. - Record disagreement. An implementer should be able to see exactly where an assessor's interpretation differs from their own claim.
The vocabulary emerged from execution-assurance work in REMORA-research and a cross-system dialogue around AEGIS Core. This repository deliberately separates the conformance model from any one implementation so the methodology can be challenged, reproduced and applied independently.
The success criterion is simple: an implementer should be able to read an assessment and say, “yes, this describes exactly what we proved, what we did not prove, and what we are not trying to solve.”
The v0.2 evidence-sufficiency distinction is informed by
CoSAI/OASIS WS4 RFC #189,
where contributions by @darklordVirtual address verifier semantics and the
separation of evidence inputs from test expectations. This repository is an
independent draft; the link does not imply CoSAI adoption or endorsement.
The most valuable contributions are not new adjectives or broader claims. They are:
- adversarial examples that show two properties are not actually separable,
- reproducible fixtures that sharpen a property boundary,
- assessments of unrelated runtimes,
- evidence that a definition is biased toward one architecture,
- proposals for missing dimensions that cannot be represented by A–G.
See CONTRIBUTING.md.
Apache-2.0. A vocabulary nobody may lawfully reuse is not vendor-neutral, and the repository shipped without a licence file until 2026-08-29. Contributions are accepted under the same terms.