Skip to content

Saga pattern support #248

Description

@Krever

Context

Workflow4s has always assumed that runIO steps are idempotent. Idempotency is usually a desirable property regardless of the orchestrator and can typically be achieved on the client side (e.g. with an idempotency key), so the library has deliberately not offered compensation primitives. This issue is a placeholder to gather interest and design input.

For effects that truly cannot be made idempotent, the saga pattern is the standard answer: declare non-idempotent work together with its compensation, so that failure triggers a reverse sequence of compensating actions. HandleError / HandleErrorWith today are forward-recovery only — they continue the workflow, they don't undo steps that already ran.

Sketch of the idea

Introduce a parallel type — tentatively SWIO (saga WIO) — that represents a workflow fragment whose forward steps may be non-idempotent, and which carries enough information to compensate them.

  • SWIO cannot be used directly as a WIO. To convert it back, the user must supply a compensation handler (analogous to how HandleError converts an erroring WIO into a total one).
  • A non-idempotent runIO would:
    1. Persist an "intent" event before executing the IO.
    2. Execute the IO.
    3. Persist a "completed" event after success.
    4. On replay: if "intent" is present but "completed" is not, treat the step as failed and trigger compensation of everything that completed earlier in the saga.
  • Compensation actions must themselves be idempotent (standard saga requirement), so they are standard WIO.
// rough shape — naming and API entirely open
val charge: SWIO[Input, ChargeFailed, Charged] = SWIO.runIO(...)
val ship:   SWIO[Charged, ShipFailed, Shipped]  = SWIO.runIO(...)

val saga: WIO[Input, OrderFailed, Shipped] =
  (charge >>> ship).compensateWith {
    case Charged(...) => SWIO.runIO(refund)
    case Shipped(...) => SWIO.runIO(recall)
  }

Open questions (no decisions yet)

  • Naming. SWIO works as a placeholder. Better options? (SagaWIO, TxWIO, keep it in WIO with a capability marker?)
  • Granularity. Does the saga boundary wrap a single non-idempotent step, or an arbitrary composition? Can sagas nest?
  • Type-level enforcement. Should SWIO be a distinct sealed hierarchy, a phantom-typed WIO, or a capability in the context? Trade-off between API separation and implementation duplication.
  • Intent/completed events. Should this be a new pair of event types in the journal, or reuse Checkpoint? How does it interact with existing event handlers?
  • Partial failure in compensation. What happens if a compensation action itself fails? Retry forever? Dead-letter? User-configurable?
  • Interaction with HandleError. If a saga is inside a handleErrorWith, does error recovery run before or after compensation? Does compensation produce a recoverable error?
  • Interaction with signals / interrupts. If a workflow is interrupted mid-saga (e.g. by interruptWith), does compensation fire?
  • Retry vs compensate. Who decides? Per-step policy, or saga-level?
  • Prior art. Worth surveying: Temporal's saga (explicit Saga object with addCompensation), Akka Projections / Cats Saga, zio-saga, Eventuate Tram. None map 1:1 onto an event-sourced, replay-based model like workflow4s, but the API shapes are informative.

Encoding idea: opaque alias over WIO

Instead of a new hierarchy, SWIO could be an opaque alias:

opaque type SWIO[I, E, O] = WIO[I, CompensationErr | E, O]

The "must provide compensation before use" property then falls out of the error channel: converting back to WIO is just a handleErrorWith on CompensationErr. Loose ends: where the stack of prior completion records lives (doesn't fit in a single error value), and how the saga scope is delimited (nesting, interaction with outer handleErrorWith).

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions