Skip to content

feat: resilient retry sandbox for profile aggregation and write-back jobs #30

Description

@ibloud

Summary

Add a retry sandbox for asynchronous profile pipeline steps (aggregation and external write-back) to isolate transient failures and guarantee eventual consistency.

Why

Cloud/network/API failures should not drop profile updates or corrupt state.

Scope

  • Introduce retry policy with bounded exponential backoff + jitter.
  • Classify retryable vs terminal errors.
  • Persist retry metadata (attempt count, last error, next retry time).
  • Add dead-letter path after max attempts.
  • Ensure idempotent reprocessing of the same match/profile event.
  • Add observability counters:
    • success, retry, terminal-fail, dead-letter, duplicate-suppressed

Data/Schema

  • Define retry sandbox collection/document shape.
  • Record correlation IDs to trace event lineage end-to-end.

Acceptance Criteria

  • Transient failures recover automatically within policy bounds.
  • Terminal failures are quarantined with actionable diagnostics.
  • Replays do not double-apply trait/profile mutations.
  • Metrics/logs allow operators to inspect retry health.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    player-profileplayerProfiles schema/persistence

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions