Skip to content

Repository files navigation

Agent Authority Conformance

Evidence before adjectives. A vendor-neutral vocabulary for evaluating authority and execution controls in agentic systems.

Status: Draft v0.2 (opt-in; v0.1 preserved) · No aggregate score · Not a certification · Not a product ranking

When an agent runtime says “the tool call was authorized”, that statement hides several independent security and governance questions. Agent Authority Conformance separates those questions so implementations can state precisely what they have demonstrated, what remains untested, and what they deliberately do not claim.

The project evaluates evidence, not marketing language.

The seven properties

ID Property Core question
A Receipt Integrity Is the authorization artifact authentic, unmodified, valid and correctly replay-bounded?
B Authority Provenance Can the system establish who or what had authority to approve the action?
C Exact-Call Integrity Is authorization bound to the exact tool call that is executed?
D Semantic Authority Was the action permitted in its real operational meaning?
E Execution-Boundary Integrity Can the protected effect be reached only through the governed path?
F TOCTOU Resistance Can approved conditions change between authorization and execution?
G Effect Verification Does the system verify the actual external effect after execution?

These properties are deliberately separable. Evidence for one property grants no credit in another.

Assessment statuses

v0.2 records verification execution separately from the property result:

Verification Property status Meaning
NOT_RUN UNTESTED No applicable verification completed.
NOT_RUN OUT_OF_SCOPE Explicit non-claim within the assessment scope.
COMPLETED PASS / FAIL Resolved evidence establishes the bounded property or its violation.
COMPLETED NOT_ESTABLISHED The procedure ran, but evidence supports neither conclusion.
UNSUPPORTED / INVALID_INPUT / ERROR null A non-verdict with a structured verifier error.

A runtime success, valid receipt or passing test expectation is not a property verdict. Missing evidence alone is never a demonstrated violation.

Evidence tiers

  • RESOLVED — the assessor inspected the named test, fixture, artifact or code at a stated revision.
  • REPORTED — the evidence was described but not independently resolved to an artifact.
  • NONE — no evidence was offered.

In v0.2, PASS and FAIL require nonempty, pinned RESOLVED evidence. Resolving an artifact does not alone prove that it is sufficient.

One rule matters most

Do not aggregate A–G into a score.

No percentage, star rating, “5/7 conformant”, security grade or ranking is valid under this model. The properties are not commensurable and deployment priorities differ.

Repository map

Path Purpose
SPECIFICATION-v0.2.md Draft v0.2 semantics and bounded E evidence rules
schema/assessment-v0.2.schema.json Opt-in v0.2 assessment schema
MIGRATION-v0.2.md Per-record migration and compatibility rules
SPECIFICATION.md Preserved normative v0.1 specification
schema/assessment.schema.json Preserved v0.1 schema
examples/ Versioned examples and historical assessments
conformance/ Validation and bounded E reference inference
tests/fixtures/ Synthetic, committed evidence-sufficiency cases
CONTRIBUTING.md Evidence and contribution discipline

Quick start

Python 3.12 or newer:

python -m venv .venv
# Linux/macOS: . .venv/bin/activate
# Windows PowerShell: .venv/Scripts/Activate.ps1
python -m pip install -r requirements-dev.txt
python -m conformance.check
python -m unittest discover -s tests -v

Activate the environment with the command for your platform before installing. The checks run offline after dependency installation and do not invoke external tools or regenerate committed expectations.

The E reference rule consumes already accepted evidence. It does not verify artifacts, inspect credential custody or establish coverage itself. Its synthetic fixtures test inference semantics, not the security of a deployed product. See the input trust boundary.

Minimal assessment shape

This complete, schema-valid v0.2 skeleton intentionally grants no conformance credit. It is also committed as examples/v0.2/minimal.json.

{
  "assessed_at": "2026-09-15",
  "properties": [
    {
      "caveat": "This is a valid assessment skeleton and grants no conformance credit.",
      "evidence": [],
      "evidence_tier": "NONE",
      "id": "C",
      "property": "Exact-Call Integrity",
      "reasoning": "No verification procedure has been run for this example.",
      "status": "UNTESTED",
      "verification_status": "NOT_RUN"
    }
  ],
  "revision": "not-assessed",
  "scope": {
    "explicit_non_claims": [],
    "external_effects": "none",
    "process_model": "not-assessed",
    "trust_assumptions": []
  },
  "spec_version": "0.2",
  "system": "Example Runtime (unassessed)",
  "system_version": "not-assessed"
}

The schema intentionally contains no aggregate score field. Schema validation checks report structure; the assessor remains responsible for the evidence.

Design principles

  1. Separate properties. Receipt correctness is not authority provenance; exact binding is not semantic correctness; dispatch success is not effect verification.
  2. Resolve claims to immutable evidence. Prefer test path + revision + command over README prose.
  3. Declare scope. Process model, deployment assumptions, simulated vs real effects and explicit non-claims belong in the assessment.
  4. Treat missing evidence conservatively. Distinguish UNTESTED from completed-but-inconclusive NOT_ESTABLISHED; neither is a demonstrated violation.
  5. Record disagreement. An implementer should be able to see exactly where an assessor's interpretation differs from their own claim.

Origin

The vocabulary emerged from execution-assurance work in REMORA-research and a cross-system dialogue around AEGIS Core. This repository deliberately separates the conformance model from any one implementation so the methodology can be challenged, reproduced and applied independently.

The success criterion is simple: an implementer should be able to read an assessment and say, “yes, this describes exactly what we proved, what we did not prove, and what we are not trying to solve.”

Standards discussion

The v0.2 evidence-sufficiency distinction is informed by CoSAI/OASIS WS4 RFC #189, where contributions by @darklordVirtual address verifier semantics and the separation of evidence inputs from test expectations. This repository is an independent draft; the link does not imply CoSAI adoption or endorsement.

Contributing

The most valuable contributions are not new adjectives or broader claims. They are:

  • adversarial examples that show two properties are not actually separable,
  • reproducible fixtures that sharpen a property boundary,
  • assessments of unrelated runtimes,
  • evidence that a definition is biased toward one architecture,
  • proposals for missing dimensions that cannot be represented by A–G.

See CONTRIBUTING.md.

Licence

Apache-2.0. A vocabulary nobody may lawfully reuse is not vendor-neutral, and the repository shipped without a licence file until 2026-08-29. Contributions are accepted under the same terms.

About

A vendor-neutral vocabulary for evaluating authority and execution controls in agentic systems. Seven properties, never aggregated into a score.

Topics

Resources

Contributing

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages