Skip to content

feat(core): add configurable retry policy - #22

Draft
pnlabs-dev wants to merge 2 commits into
zaber-dev:mainfrom
pnlabs-dev:feat/resilience-retry-policy
Draft

pnlabs-dev wants to merge 2 commits into
zaber-dev:mainfrom
pnlabs-dev:feat/resilience-retry-policy

Conversation

@pnlabs-dev

@pnlabs-dev pnlabs-dev commented Aug 21, 2026 •

Copy link
Copy Markdown

Description

Adds a configurable retry policy for @free-ai-gateway/core while keeping transport retries and provider failover as separate, bounded concerns.

This PR:

  • introduces a typed RetryPolicy with configurable backoff and jitter;
  • supports full and decorrelated jitter with injectable RNG/sleep for deterministic tests;
  • adds per-capability provider-attempt limits in CapabilityRouter;
  • makes retryable: false stop provider failover instead of continuing to the next candidate;
  • preserves existing behavior when no retry policy is supplied;
  • keeps HTTP 429 handling at the router/quota boundary rather than silently retrying it at the transport layer;
  • adds focused regression coverage for jitter bounds, retry exhaustion, failover limits, non-retryable failures, and backward compatibility.

Closes #18

Type of Change

  • 🐛 Bug fix (non-breaking change fixing an issue)
  • ✨ New provider adapter
  • ⚡ Performance / Resilience improvement
  • 📝 Documentation update
  • 🔧 Refactoring / Code hygiene

Validation

Focused local validation completed:

  • strict TypeScript compilation of the changed surface: PASS
  • retry-policy behavioral regression suite: 7/7 PASS

Full repository npm run typecheck and npm test are intentionally left to the pull-request CI matrix before this draft is marked ready for review.

Checklist

  • My code follows the project's code style and standards.
  • I have run npm run typecheck and npm test and all checks passed.
  • I have updated documentation (README.md, .env.example, CONTRIBUTING.md) if applicable.
  • If adding a new provider adapter, I verified it with providers.schema.json and registered it in src/core/registry.ts.

@zaber-dev zaber-dev left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for this excellent, well-architected PR! The separation of concerns between transport retries and router failover strictly aligns with our @free-ai-gateway/core architectural principles (AGENTS.md).

However, we detected a failing test during local validation:

❌ Test Failure in packages/core/tests/retry-policy.test.ts:199

ext ✖ bounds failover by capability and applies router jitter AssertionError [ERR_ASSERTION]: Expected values to be strictly deep-equal: actual: [ 1, 0, 1 ] expected: [ 1, 1, 0 ]

🔍 Root Cause Analysis

In CapabilityRouter, candidates are sorted using AdaptiveHealthStrategy.rank(). When multiple candidate adapters have identical confidence tiers (\official) and no metrics history, their ranking scores are identical. In that case, AdaptiveHealthStrategy applies tie-breaker jitter:
\\ s
// packages/core/src/router/adaptive-health-strategy.ts:63
return Math.random() - 0.5;
\
Because the candidate ranking is randomized on equal rank, second and hird are nondeterministically swapped. When candidate order becomes [first, third, second], capping attempts at 2 executes irst (fail) then hird (fail), resulting in call counts [1, 0, 1] instead of [1, 1, 0].

💡 Suggested Fix

In packages/core/tests/retry-policy.test.ts:

  1. Assign distinct confidence tiers in providerConfig for the test adapters (e.g., irst -> live_console, second -> official, hird -> unverified) so AdaptiveHealthStrategy ranks them deterministically without triggering tie-breaker jitter, OR
  2. Pass a deterministic mock IRoutingStrategy (returning candidates in fixed order) to CapabilityRouter inside makeRouter().
  3. Please re-run
    pm test to verify the entire monorepo test suite passes.

Once this test is updated, we will be happy to approve and merge!

@pnlabs-dev

Copy link
Copy Markdown
Author

Thank you very much for the detailed review and for reproducing the failure.

You're right — the test was incorrectly assuming a deterministic candidate order while the default "AdaptiveHealthStrategy" intentionally randomizes equal-ranked candidates. I missed that interaction in the focused local run.

I'll fix the test by injecting a deterministic routing strategy rather than changing production ranking behavior just to make the fixture stable.

While reviewing this, I also noticed a couple of adjacent retry/failover edge cases that are worth pinning down with regression coverage, particularly around non-retryable provider failures and keeping transport-retry policy separate from provider-failover behavior. I’ll keep those changes bounded to the retry-policy scope rather than broadening the PR unnecessarily.

Before marking the PR ready for review, I'll re-run the full repository "npm run typecheck" and "npm test" suite and make sure the focused retry-policy tests are deterministic as well.

Thanks again for the clear root-cause analysis and suggested direction — it was very helpful.

@pnlabs-dev

Copy link
Copy Markdown
Author

I’ve pushed the deterministic test fix in "29c1555637cb538bfda15e1f70905ea829719888".

The retry-policy test helper now injects a fixed-order "IRoutingStrategy", so the fixture no longer depends on the production "AdaptiveHealthStrategy" tie-breaker jitter. No production routing behavior was changed by this follow-up.

The upstream Actions run for this exact head is currently "action_required" with no jobs started, so I don’t want to claim the full matrix as passing yet. Once the workflow is approved and runs, I’ll treat the repository build/typecheck/test matrix as the final gate before marking the PR ready.

Thank you again for pointing out the nondeterminism.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(resilience): Add configurable sliding-window jitter and per-capability retry multipliers

2 participants