Skip to content

Commit 0dbb538

Browse files
Add semantic machine testing and causal runtime probes (#85)
* feat: add semantic machine invariants * feat: add bounded machine exploration * feat: add causal runtime probes
1 parent 031fa9d commit 0dbb538

20 files changed

Lines changed: 3192 additions & 156 deletions

File tree

.changeset/fair-laws-check.md

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
---
2+
"@typeonce/effect-machine": minor
3+
---
4+
5+
Add state, step, and trace invariants for checking application semantics over planner traces, including machine-inferred builders, conditional observation requirements, structured reports, and property-test assertions. Add bounded breadth-first exploration with state-dependent event representatives, shortest witnesses, explicit truncation frontiers, and fail-closed reachability assertions. Add testing-only runtime probes with acknowledged event delivery so tests can causally inspect ignored, targetless, changing, and failed live macrosteps without adding a production `sendAndAwait` API.

README.md

Lines changed: 120 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -21,10 +21,11 @@ require upgrading Effect in lockstep; do not override the peer to another beta.
2121
import { Machine } from "@typeonce/effect-machine"
2222
import { ClusterMachine } from "@typeonce/effect-machine/cluster"
2323
import { AtomMachine } from "@typeonce/effect-machine/reactivity"
24+
import { MachineTest } from "@typeonce/effect-machine/testing"
2425
```
2526

2627
Each ESM entrypoint is independent and tree-shakeable. Importing the root does
27-
not load the reactivity or cluster adapters.
28+
not load the reactivity, cluster, or testing modules.
2829

2930
## First machine
3031

@@ -655,6 +656,124 @@ restrictions, checkpoint planning, and delivery guarantees are documented on
655656
that API. `Machine.resume` is logical resumption, not durable process or cluster
656657
restoration.
657658

659+
## Property-based semantic invariants
660+
661+
`MachineTest.verify` checks statechart structure and planner lifecycle laws.
662+
Application semantics belong in invariants that can be reused across generated
663+
scenarios and, in future, bounded exploration:
664+
665+
```ts
666+
import { MachineTest } from "@typeonce/effect-machine/testing"
667+
import { Effect } from "effect"
668+
669+
const invariant = MachineTest.invariants(accountMachine)
670+
const laws = [
671+
invariant.state(
672+
"balance is never negative",
673+
({ snapshot }) =>
674+
snapshot.value.balance >= 0 ||
675+
`negative balance: ${snapshot.value.balance}`
676+
),
677+
invariant.step(
678+
"withdrawal removes exactly its amount",
679+
({ before, event, after }) =>
680+
event._tag !== "Withdraw" ||
681+
after.value.balance === before.value.balance - event.amount
682+
)
683+
]
684+
685+
const generated = MachineTest.scenarios(accountMachine, {
686+
minEvents: 0,
687+
maxEvents: 30
688+
})
689+
690+
it.effect.prop(
691+
"preserves account laws",
692+
{ scenario: generated.arbitrary },
693+
({ scenario }) =>
694+
MachineTest.run(accountMachine, scenario).pipe(
695+
Effect.tap((trace) => MachineTest.verify(accountMachine, trace)),
696+
Effect.flatMap((trace) => MachineTest.assertInvariants(accountMachine, trace, laws))
697+
)
698+
)
699+
```
700+
701+
State invariants observe settled startup and public-event states by default.
702+
Set `observe` to `"microsteps"`, `"all"`, or `"final"` for a different scope.
703+
Use `when` for conditional laws. A condition with no matches is reported as
704+
`untested`; add `require: { minObservations: 1 }` when a particular trace must
705+
exercise it. `checkInvariants` returns this report, while `assertInvariants`
706+
returns `void` for direct use in property tests. Failures retain the complete
707+
shrunk trace and precise event, microstep, configuration, and observation
708+
location.
709+
710+
These APIs inspect planner evidence. Staged action effects, invokes, timing,
711+
and process scheduling require the runtime command-model APIs instead.
712+
713+
Use bounded exploration when random scenarios should be complemented by a
714+
systematic search over concrete event representatives:
715+
716+
```ts
717+
const explored = yield * MachineTest.explore(accountMachine, {
718+
events: ({ snapshot }) => [
719+
new Deposit({ amount: 1 }),
720+
new Withdraw({ amount: snapshot.value.balance }),
721+
new Withdraw({ amount: snapshot.value.balance + 1 })
722+
],
723+
stateKey: ({ snapshot }) => `${snapshot.value._tag}:${snapshot.value.balance}`,
724+
limits: {
725+
maxDepth: 20,
726+
maxStates: 1_000,
727+
maxTransitions: 10_000
728+
},
729+
invariants: laws
730+
})
731+
732+
const rejected = yield * MachineTest.assertReachable(
733+
explored,
734+
"insufficient funds rejection",
735+
({ configuration }) => configuration.includes("Rejected")
736+
)
737+
738+
console.log(rejected.trace.scenario.events) // shortest witness
739+
```
740+
741+
Exploration is breadth-first, so each retained node owns its shortest trace.
742+
It is exhaustive only for the concrete events returned by `events` and the
743+
equivalence relation defined by `stateKey`. Equal keys intentionally collapse
744+
snapshots and only the first representative is expanded. Results distinguish
745+
`Complete` from `Truncated` and retain the depth, state, or transition frontier
746+
that hit a limit. An unreachability assertion succeeds only for a complete
747+
result; otherwise it fails as inconclusive. Cycles are retained as graph edges,
748+
but exploration does not enumerate every cyclic path. Invariants are checked
749+
on startup and on each planned edge extending a node's shortest trace.
750+
751+
## Causal runtime probes
752+
753+
Pure traces do not execute invokes or the managed runtime. When a test needs to
754+
prove that one live event has actually left the mailbox, attach a testing-only
755+
probe to a statechart reference:
756+
757+
```ts
758+
const ref = yield * Machine.start(machine)
759+
const probe = yield * MachineTest.probe(machine, ref)
760+
761+
const step = yield * probe.sendAndAwait(new CancelRequested({}))
762+
763+
assert.strictEqual(step.handled, false)
764+
assert.deepStrictEqual(step.before, step.after)
765+
```
766+
767+
`sendAndAwait` completes after that event's synchronous macrostep and managed
768+
commit work. It also completes for ignored events, which publish no snapshot
769+
and therefore cannot be synchronized by waiting for `ref.changes`.
770+
771+
The step retains the exact runtime plan, before/after logical snapshots, and
772+
whether the event was handled or changed/reentered the active configuration.
773+
It does not wait for timers or invoked processes to finish. Production code
774+
continues to use enqueue-only `ref.send`; probes are exported only from the
775+
separate testing entry point.
776+
658777
## Current limits
659778

660779
Declarative first-class guards are not part of the current API. Ordinary

docs/agent-guide.md

Lines changed: 91 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -724,6 +724,97 @@ This is not durable runtime restoration. `ClusterMachine` has a separate
724724
checkpoint/planning contract and process-local restrictions; do not substitute
725725
`Machine.resume` for cluster recovery.
726726

727+
## Testing machine semantics
728+
729+
Import planner testing tools from the dedicated entrypoint:
730+
731+
```ts
732+
import { MachineTest } from "@typeonce/effect-machine/testing"
733+
```
734+
735+
Use three distinct layers:
736+
737+
1. `MachineTest.verify(machine, trace)` checks structural statechart and
738+
planner lifecycle laws.
739+
2. `MachineTest.assertInvariants(machine, trace, laws)` checks application
740+
semantics such as conservation, authorization, and exact state updates.
741+
3. Runtime command models check executed actions, invokes, timing, process
742+
publication, and cancellation. Planner traces do not execute this work.
743+
744+
Define semantic laws with a machine-bound builder so the callback receives the
745+
exact state and event types:
746+
747+
```ts
748+
const invariant = MachineTest.invariants(machine)
749+
750+
const laws = [
751+
invariant.state("balance is never negative", ({ snapshot }) =>
752+
snapshot.value.balance >= 0 || "negative balance"),
753+
invariant.step("withdrawal is exact", ({ before, event, after }) =>
754+
event._tag !== "Withdraw" ||
755+
after.value.balance === before.value.balance - event.amount),
756+
invariant.trace("all inputs were planned", ({ trace }) =>
757+
trace.steps.length === trace.scenario.events.length)
758+
]
759+
```
760+
761+
State laws observe settled states by default. Select `"microsteps"`, `"all"`,
762+
or `"final"` only when the law requires that evidence. A `when` condition with
763+
no matches is explicitly `untested`; use
764+
`require: { minObservations: 1 }` when the current trace must exercise it.
765+
Prefer `assertInvariants` inside FastCheck properties because it succeeds with
766+
`void`. Use `checkInvariants` when the test needs the per-law report.
767+
768+
For systematic planner exploration, provide a finite abstraction explicitly:
769+
770+
```ts
771+
const explored = yield * MachineTest.explore(machine, {
772+
events: ({ snapshot }) => eventRepresentatives(snapshot),
773+
stateKey: ({ snapshot }) => logicalStateKey(snapshot),
774+
limits: { maxDepth: 20, maxStates: 1_000 },
775+
invariants: laws
776+
})
777+
```
778+
779+
The event callback returns concrete representatives, not schemas or
780+
arbitraries. Include meaningful boundary values based on the current snapshot.
781+
The key defines which snapshots are treated as equivalent; it must retain every
782+
piece of data that can change the future behavior being tested. A coarse key
783+
can make exploration finite but under-approximate behavior.
784+
785+
`assertReachable` returns the shortest witness. `assertUnreachable` succeeds
786+
only when `explored.completeness` is `Complete`. Never interpret a truncated
787+
depth, state, or transition frontier as an unreachability proof. The explorer
788+
retains cycles as graph edges but does not enumerate every path around them;
789+
use a separate temporal/path model when a law depends on repeated traversal
790+
rather than logical-state reachability.
791+
792+
Do not encode application invariants as guards merely to make them testable.
793+
Keep ordinary TypeScript branching in transition handlers unless a choice is
794+
part of the statechart topology. Invariants independently verify the resulting
795+
trace without changing production transition selection.
796+
797+
### Live event causality
798+
799+
Use a probe when a test must establish that one event was processed by a
800+
running statechart rather than merely accepted by its mailbox:
801+
802+
```ts
803+
const ref = yield * Machine.start(machine)
804+
const probe = yield * MachineTest.probe(machine, ref)
805+
const step = yield * probe.sendAndAwait(event)
806+
```
807+
808+
Inspect `step.before`, `step.after`, `step.plan`, `step.handled`, and
809+
`step.configurationChanged`. An ignored event has `handled: false` and an
810+
empty microstep list, but still completes its acknowledgement. A targetless
811+
handler has `handled: true` even if its before and after snapshots are equal.
812+
813+
Do not use a probe as a substitute for a domain completion event. The
814+
acknowledgement covers the submitted event's synchronous macrostep, state
815+
commit, emissions, and invoke startup; it does not wait for an invoke or timer
816+
to complete. Application code should continue to use `MachineRef.send`.
817+
727818
## Common compiler errors
728819

729820
### `initial` is not callable

0 commit comments

Comments
 (0)