Context
Workflow4s has always assumed that runIO steps are idempotent. Idempotency is usually a desirable property regardless of the orchestrator and can typically be achieved on the client side (e.g. with an idempotency key), so the library has deliberately not offered compensation primitives. This issue is a placeholder to gather interest and design input.
For effects that truly cannot be made idempotent, the saga pattern is the standard answer: declare non-idempotent work together with its compensation, so that failure triggers a reverse sequence of compensating actions. HandleError / HandleErrorWith today are forward-recovery only — they continue the workflow, they don't undo steps that already ran.
Sketch of the idea
Introduce a parallel type — tentatively SWIO (saga WIO) — that represents a workflow fragment whose forward steps may be non-idempotent, and which carries enough information to compensate them.
SWIO cannot be used directly as a WIO. To convert it back, the user must supply a compensation handler (analogous to how HandleError converts an erroring WIO into a total one).
- A non-idempotent
runIO would:
- Persist an "intent" event before executing the IO.
- Execute the IO.
- Persist a "completed" event after success.
- On replay: if "intent" is present but "completed" is not, treat the step as failed and trigger compensation of everything that completed earlier in the saga.
- Compensation actions must themselves be idempotent (standard saga requirement), so they are standard WIO.
// rough shape — naming and API entirely open
val charge: SWIO[Input, ChargeFailed, Charged] = SWIO.runIO(...)
val ship: SWIO[Charged, ShipFailed, Shipped] = SWIO.runIO(...)
val saga: WIO[Input, OrderFailed, Shipped] =
(charge >>> ship).compensateWith {
case Charged(...) => SWIO.runIO(refund)
case Shipped(...) => SWIO.runIO(recall)
}
Open questions (no decisions yet)
- Naming.
SWIO works as a placeholder. Better options? (SagaWIO, TxWIO, keep it in WIO with a capability marker?)
- Granularity. Does the saga boundary wrap a single non-idempotent step, or an arbitrary composition? Can sagas nest?
- Type-level enforcement. Should
SWIO be a distinct sealed hierarchy, a phantom-typed WIO, or a capability in the context? Trade-off between API separation and implementation duplication.
- Intent/completed events. Should this be a new pair of event types in the journal, or reuse
Checkpoint? How does it interact with existing event handlers?
- Partial failure in compensation. What happens if a compensation action itself fails? Retry forever? Dead-letter? User-configurable?
- Interaction with
HandleError. If a saga is inside a handleErrorWith, does error recovery run before or after compensation? Does compensation produce a recoverable error?
- Interaction with signals / interrupts. If a workflow is interrupted mid-saga (e.g. by
interruptWith), does compensation fire?
- Retry vs compensate. Who decides? Per-step policy, or saga-level?
- Prior art. Worth surveying: Temporal's saga (explicit
Saga object with addCompensation), Akka Projections / Cats Saga, zio-saga, Eventuate Tram. None map 1:1 onto an event-sourced, replay-based model like workflow4s, but the API shapes are informative.
Encoding idea: opaque alias over WIO
Instead of a new hierarchy, SWIO could be an opaque alias:
opaque type SWIO[I, E, O] = WIO[I, CompensationErr | E, O]
The "must provide compensation before use" property then falls out of the error channel: converting back to WIO is just a handleErrorWith on CompensationErr. Loose ends: where the stack of prior completion records lives (doesn't fit in a single error value), and how the saga scope is delimited (nesting, interaction with outer handleErrorWith).
Context
Workflow4s has always assumed that
runIOsteps are idempotent. Idempotency is usually a desirable property regardless of the orchestrator and can typically be achieved on the client side (e.g. with an idempotency key), so the library has deliberately not offered compensation primitives. This issue is a placeholder to gather interest and design input.For effects that truly cannot be made idempotent, the saga pattern is the standard answer: declare non-idempotent work together with its compensation, so that failure triggers a reverse sequence of compensating actions.
HandleError/HandleErrorWithtoday are forward-recovery only — they continue the workflow, they don't undo steps that already ran.Sketch of the idea
Introduce a parallel type — tentatively
SWIO(saga WIO) — that represents a workflow fragment whose forward steps may be non-idempotent, and which carries enough information to compensate them.SWIOcannot be used directly as aWIO. To convert it back, the user must supply a compensation handler (analogous to howHandleErrorconverts an erroringWIOinto a total one).runIOwould:Open questions (no decisions yet)
SWIOworks as a placeholder. Better options? (SagaWIO,TxWIO, keep it inWIOwith a capability marker?)SWIObe a distinct sealed hierarchy, a phantom-typedWIO, or a capability in the context? Trade-off between API separation and implementation duplication.Checkpoint? How does it interact with existing event handlers?HandleError. If a saga is inside ahandleErrorWith, does error recovery run before or after compensation? Does compensation produce a recoverable error?interruptWith), does compensation fire?Sagaobject withaddCompensation), Akka Projections / Cats Saga, zio-saga, Eventuate Tram. None map 1:1 onto an event-sourced, replay-based model like workflow4s, but the API shapes are informative.Encoding idea: opaque alias over
WIOInstead of a new hierarchy,
SWIOcould be an opaque alias:The "must provide compensation before use" property then falls out of the error channel: converting back to
WIOis just ahandleErrorWithonCompensationErr. Loose ends: where the stack of prior completion records lives (doesn't fit in a single error value), and how the saga scope is delimited (nesting, interaction with outerhandleErrorWith).