Skip to content

test: validate CrewAI native task retry and adversarial recovery - #317

Draft
stringsofthemind-oss wants to merge 1 commit into
mainfrom
test/crewai-5802-native-retry
Draft

stringsofthemind-oss wants to merge 1 commit into
mainfrom
test/crewai-5802-native-retry

Conversation

@stringsofthemind-oss

@stringsofthemind-oss stringsofthemind-oss commented Nov 5, 2026 •

Copy link
Copy Markdown
Owner

The existing CrewAI lab exercises native structured tools but owns redispatch itself. This adds an actual Agent.execute_task retry: the simulated provider commits a payment and loses its acknowledgement, a subsequent scripted executor failure reaches CrewAI's task retry handler, and the same admitted intent is invoked again. The independent provider ledger records 2 effects without protection versus 1 with Once.

The 10-case suite also covers eight competing processes, twelve changed-field conflicts across CLAIMED/UNKNOWN/CONFIRMED, distinct intents with identical fields, unavailable provider truth and later reconciliation, confirmed-record recovery-transition refusals, non-idempotent ABSENT refusal, hard process exit before confirmation, and loss of a confirmed result before framework delivery. Process-death redispatch is explicitly harness-owned. Python Once has no release/force API; the regression tests its actual recovery transitions, not a fictional release contract.

Changes are confined to the existing lab and its dedicated Windows/Linux CI matrix. No SDK production code, unrelated examples, or FIRST 10 adoption records change.

Validation

  • Windows / Python 3.12.10 / CrewAI 1.15.22: 10/10 passed with published once-agent-sdk 0.1.1, and 10/10 passed in a separate clean environment with a built checkout wheel.
  • Loaded core SHA-256 matches in both: aa37acded51c75a08edfd9d81350f725b1cfe292336d833bec1506cdc8bb65d6.
  • Existing CrewAI hostile-retry and identity labs passed; Python checkout SDK suite: 44 passed.
  • FIRST 10 2 passed, packed FIRST 10 passed, wrappers 17 passed, packed wrappers 14 checks plus type contracts passed, local SQLite 23 passed, site funnel 4 passed, proof regeneration and source/site guards passed.
  • Initial restricted runs could not use local event-loop sockets or fetch the existing public proof; normal local runs passed.
  • CrewAI CI: all four Windows/Linux × published/checkout-wheel jobs passed, each running all 10 cases; evidence JSON and resolved dependency versions are retained as artifacts. Full Regression CI also passed, including the TypeScript release/safety suite, Python SDK, MCP package, and plugin checks. All five triggered workflows are green.

Scope

Synthetic local payment authority, scripted LLM, one trusted host, persistent shared SQLite, correct logical IDs and complete payment-field binding, authoritative fixture lookup. No live payment/model test, universal exactly-once, multi-host, state-loss, independent-adoption, or retention claim. Neither CrewAI PR head #5822/#6048 was tested; their open/unmerged status and exact heads are documented as of 2026-11-05.

Reproduce: lab instructions. Related upstream discussion: crewAIInc/crewAI#5802. This does not claim to fix that issue.

AI assistance was used to implement and validate this contribution.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant