Repository navigation
test: add cross-framework governed failure conformance - #189
Merged
brunovicco merged 4 commits intoSep 6, 2026
Merged
Conversation
brunovicco
marked this pull request as ready for review
September 6, 2026 04:56
brunovicco
deleted the
test/phase-54-cross-framework-executor-failure-conformance
branch
September 6, 2026 13:52
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Phase 54 — cross-framework governed executor failure conformance
This phase extends the existing framework conformance contract with one post-executor failure scenario and required no production-code change.
Concrete gap
The previous conformance suite covered successful execution and pre-executor security states, but not the Phase 49
GovernedActionExecutionError/ActionExecutionFailureEvidencepath. Phase 53 demonstrated that a framework could satisfy the existing conformance suite while still replacing application-owned failure provenance.Design
GovernedActionRuntimeand through LangGraph, CrewAI, LlamaIndex and Agno;GovernedActionExecutionErrorfrom every framework;ActionExecutionFailureEvidenceto equal the application baseline exactly;execution_attempted=true,failure_reason=executor_error, andexternal_side_effect_state=unknown;Result
All four framework adapters already conform after the Phase 53 Agno fix. No LangGraph, CrewAI, LlamaIndex, Agno, application-runtime, authorization, approval, or policy code had to change.
The resulting contract is:
authorized executor failure -> exactly one attempt + baseline-equivalent application-owned failure evidence across frameworksand:
framework conformance includes failure provenance, not only successful/pre-executor statesCI evidence
Functional CI #486 passed on
b18528ea6dbf98f813cdb583134afbc33002ce6a, proving the new failure scenario across the direct application baseline and all four frameworks before documentation.Final clean CI #489 passed on
d6a22ba4dd6a425a7014e3e370470b93db6d5524after documentation and removal of the temporary docs workflow.Final quality evidence:
Documentation
CHANGELOG.mdrecords the cross-framework governed-failure conformance evidence;docs/security/GOVERNED_AGENT_ACTIONS.mdadds authorized executor failure to the cross-framework matrix and documents baseline-equivalentActionExecutionFailureEvidence, exactly one attempt, raw-error exclusion, andexternal_side_effect_state=unknown.Final scope
The final diff contains exactly three files:
tests/integration/test_governed_action_framework_conformance.py;CHANGELOG.md;docs/security/GOVERNED_AGENT_ACTIONS.md.No production code, framework adapter, application runtime, dependency, workflow helper, historical artifact, or published release remains changed. This phase adds no retry, idempotency, rollback, compensation, two-phase commit, or distributed transaction semantics.
Closes #188