Skip to content

Update Effect and XState benchmark runtimes - #75

Merged
SandroMaglione merged 1 commit into
mainfrom
codex/update-runtime-dependencies
Aug 10, 2026
Merged

Update Effect and XState benchmark runtimes#75
SandroMaglione merged 1 commit into
mainfrom
codex/update-runtime-dependencies

Conversation

@SandroMaglione

Copy link
Copy Markdown
Member

Summary

  • update Effect, @effect/vitest, and example Effect packages from 4.0.0-beta.105 to 4.0.0-beta.107
  • update the XState v6 benchmark alias from 6.0.0-alpha.27 to 6.0.0-alpha.31; XState v5 remains current at 5.32.5
  • adapt schema-derived test arbitraries to Effect's new FastCheck factory API while preserving opaque-filter diagnostics
  • refresh all workspace manifests, lockfiles, installation docs, benchmark docs, and the consumer compatibility fixture

Changeset

  • Added or updated for a library or package-metadata change
  • Not required because this PR does not change src/ or package.json

Validation

  • pnpm check
  • Relevant example checks, when examples changed
  • Reviewed the automated type-performance report, when the public TypeScript API or inference changed
  • Reviewed the automated runtime-performance report, when runtime behavior changed

Local validation also included pnpm perf:types, frozen-lockfile installation, and a full pnpm perf:runtime run. The automated reports remain the merge gate for the repeatable base-versus-PR comparison.

@github-actions

Copy link
Copy Markdown
Contributor

Type performance

Measured with TypeScript 6.0.3 and skipLibCheck=true.

Scenario Base PR Difference
Effect only 55 55 0 (0.0%)
Import effect-machine 55 55 0 (0.0%)
Machine.defineStates (3 states) 2,965 2,965 0 (0.0%)
Machine.make (3 states, 2 events) 8,715 8,715 0 (0.0%)
machine.handle (3 states, 2 transitions) 23,492 23,492 0 (0.0%)
machine.handle (depth 8) 117,075 117,075 0 (0.0%)
machine.handle (depth 12) 129,713 129,713 0 (0.0%)
machine.handle (depth 16) 145,055 145,055 0 (0.0%)
machine.handle (depth 24) 183,851 183,851 0 (0.0%)
machine.handle (wide depth 16) 214,189 214,189 0 (0.0%)
machine.handle (parallel/history/choice) 132,667 132,667 0 (0.0%)
machine.handle (4 successive calls) 128,674 128,674 0 (0.0%)
machine exact input/output/error/services 110,584 110,584 0 (0.0%)
execution adapter readiness 127,854 127,854 0 (0.0%)

Marginal instantiations are measured against the matching setup without that API call:

Scenario Base PR Difference
Import effect-machine 0 0 0
Machine.defineStates (3 states) 2,910 2,910 0 (0.0%)
Machine.make (3 states, 2 events) 5,742 5,742 0 (0.0%)
machine.handle (3 states, 2 transitions) 14,777 14,777 0 (0.0%)
machine.handle (depth 8) 109,601 109,601 0 (0.0%)
machine.handle (depth 12) 121,039 121,039 0 (0.0%)
machine.handle (depth 16) 135,181 135,181 0 (0.0%)
machine.handle (depth 24) 171,577 171,577 0 (0.0%)
machine.handle (wide depth 16) 202,224 202,224 0 (0.0%)
machine.handle (parallel/history/choice) 114,331 114,331 0 (0.0%)
machine.handle (4 successive calls) 113,703 113,703 0 (0.0%)
machine exact input/output/error/services 100,670 100,670 0 (0.0%)
execution adapter readiness 102,092 102,092 0 (0.0%)
Check times (informational)
Scenario Base PR
Effect only 0.03s 0.04s
Import effect-machine 0.03s 0.03s
Machine.defineStates (3 states) 0.10s 0.11s
Machine.make (3 states, 2 events) 0.17s 0.17s
machine.handle (3 states, 2 transitions) 0.25s 0.26s
machine.handle (depth 8) 0.52s 0.56s
machine.handle (depth 12) 0.58s 0.59s
machine.handle (depth 16) 0.64s 0.65s
machine.handle (depth 24) 0.71s 0.73s
machine.handle (wide depth 16) 0.80s 0.77s
machine.handle (parallel/history/choice) 0.65s 0.65s
machine.handle (4 successive calls) 0.64s 0.59s
machine exact input/output/error/services 0.55s 0.54s
execution adapter readiness 0.61s 0.61s

Type instantiations are the comparison metric. Check time varies with runner load and is informational only.

@github-actions

Copy link
Copy Markdown
Contributor

Runtime performance

Median of 5 independent benchmark processes on AMD EPYC 7763 64-Core Processor with Node v24.18.0.

Pull request baseline

Scenario Effect Machine XState 5 XState 6 alpha
Plan counter transitions 125,288 transitions/s 131,528 transitions/s 13,387 transitions/s
Drain burst with terminal fence 505,054 increments/s 389,376 increments/s 191,883 increments/s
Drain burst with a change observer 472,849 increments/s 387,274 increments/s 191,152 increments/s
Lookup and send to one child 408,851 increments/s 385,214 increments/s 184,446 increments/s
Start and stop a machine 173,581 machines/s 185,185 machines/s 141,183 machines/s
Start and stop a parent with one child 39,638 families/s 68,460 families/s 40,394 families/s
Drain burst through a compound state 530,174 events/s 343,547 events/s 189,417 events/s
Drain burst through two parallel regions 478,433 events/s 259,006 events/s 144,027 events/s

Effect runtime reference points

Scenario Effect Machine Effect runtime primitives
Start and stop a raw compiled process 69,750 processes/s
Start and interrupt a suspended fiber 166,058 fibers/s
Start and stop a queue worker 116,877 workers/s
Start and stop an actor shell 107,910 actors/s
Start and stop two actor shells 64,645 families/s
Update an owner-only mutable snapshot 316,856,781 updates/s
Update a synchronized snapshot 1,584,809 updates/s
Create, resolve, and await a terminal latch 1,451,739 latches/s
Memory profile Effect Machine XState 5 XState 6 alpha Effect runtime primitives
Idle machine 1.5 KiB 3.8 KiB 2.2 KiB
Raw generic managed process 12.1 KiB
Raw compiled process 2.6 KiB
Two independent idle machines 3.0 KiB 7.3 KiB 3.9 KiB
Idle parent with one child 4.8 KiB 5.5 KiB 4.0 KiB
Parent with observed child registry 9.0 KiB
Parent with observed invoked child snapshots 5.0 KiB
Suspended Effect fiber 0.7 KiB
Effect queue with waiting fiber 2.8 KiB
Effect mailbox actor shell 3.2 KiB
Two Effect actor shells 6.9 KiB

Effect Machine change from base

Metric Base Base variability PR PR variability Difference
Plan counter transitions 134,227 transitions/s 0.7% MAD 125,288 transitions/s 0.8% MAD -6.7%
Drain burst with terminal fence 532,812 increments/s 0.7% MAD 505,054 increments/s 1.4% MAD -5.2%
Drain burst with a change observer 481,527 increments/s 1.1% MAD 472,849 increments/s 2.3% MAD -1.8%
Lookup and send to one child 406,769 increments/s 1.0% MAD 408,851 increments/s 2.6% MAD +0.5%
Start and stop a machine 164,177 machines/s 0.2% MAD 173,581 machines/s 2.5% MAD +5.7%
Start and stop a parent with one child 32,898 families/s 0.8% MAD 39,638 families/s 2.0% MAD +20.5%
Drain burst through a compound state 507,323 events/s 0.7% MAD 530,174 events/s 0.9% MAD +4.5%
Drain burst through two parallel regions 465,414 events/s 0.6% MAD 478,433 events/s 1.4% MAD +2.8%
Idle machine heap per unit 1.5 KiB 0.0% MAD 1.5 KiB 0.0% MAD +0.0%
Raw generic managed process heap per unit 12.1 KiB 0.1% MAD 12.1 KiB 0.0% MAD -0.0%
Raw compiled process heap per unit 2.6 KiB 0.1% MAD 2.6 KiB 0.1% MAD +0.0%
Two independent idle machines heap per unit 3.0 KiB 0.0% MAD 3.0 KiB 0.0% MAD -0.0%
Idle parent with one child heap per unit 4.8 KiB 0.0% MAD 4.8 KiB 0.0% MAD -0.0%
Parent with observed child registry heap per unit 9.0 KiB 0.0% MAD 9.0 KiB 0.0% MAD -0.0%
Parent with observed invoked child snapshots heap per unit 5.1 KiB 0.0% MAD 5.0 KiB 0.1% MAD -0.1%

Effect runtime reference change from base

Metric Base Base variability PR PR variability Difference
Start and stop a raw compiled process 69,075 processes/s 0.1% MAD 69,750 processes/s 1.2% MAD +1.0%
Start and interrupt a suspended fiber 164,989 fibers/s 1.3% MAD 166,058 fibers/s 0.1% MAD +0.6%
Start and stop a queue worker 114,325 workers/s 1.4% MAD 116,877 workers/s 0.6% MAD +2.2%
Start and stop an actor shell 104,954 actors/s 0.3% MAD 107,910 actors/s 1.8% MAD +2.8%
Start and stop two actor shells 65,885 families/s 0.8% MAD 64,645 families/s 4.3% MAD -1.9%
Update an owner-only mutable snapshot 316,856,781 updates/s 0.0% MAD 316,856,781 updates/s 0.0% MAD 0.0%
Update a synchronized snapshot 1,523,654 updates/s 2.3% MAD 1,584,809 updates/s 1.1% MAD +4.0%
Create, resolve, and await a terminal latch 1,459,914 latches/s 2.1% MAD 1,451,739 latches/s 1.0% MAD -0.6%
Versions and interpretation
  • Effect Machine: 0.3.0
  • XState 5: 5.32.5
  • XState 6 alpha: 6.0.0-alpha.31
  • Effect runtime primitives: 4.0.0-beta.107

Higher throughput is better; lower heap is better. Variability is the median absolute deviation across independent processes, relative to their median. Runtime measurements on shared GitHub-hosted hardware remain informational, so small differences should be confirmed across multiple workflow runs.

@SandroMaglione
SandroMaglione merged commit fcddd68 into main Aug 10, 2026
9 checks passed
@SandroMaglione
SandroMaglione deleted the codex/update-runtime-dependencies branch August 10, 2026 08:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant