Skip to content

Add Effect runtime benchmark ceilings - #50

Merged
SandroMaglione merged 1 commit into
mainfrom
codex/effect-runtime-ceilings
Aug 8, 2026
Merged

Add Effect runtime benchmark ceilings#50
SandroMaglione merged 1 commit into
mainfrom
codex/effect-runtime-ceilings

Conversation

@SandroMaglione

Copy link
Copy Markdown
Member

Summary

  • add Effect-only lifecycle reference points for fibers, queue workers, actor shells, and two-shell families
  • add coordination reference points for mutable snapshots, synchronized snapshots, and terminal latches
  • report raw compiled-process lifecycle and memory alongside the existing generic-process and statechart profiles
  • render runtime reference points separately from cross-library scenarios and document their interpretation

Changeset

  • Added or updated for a library or package-metadata change
  • Not required because this PR does not change src/ or package.json

Validation

  • pnpm check
  • Relevant example checks, when examples changed (not applicable; no examples changed)
  • Reviewed the automated type-performance report, when the public TypeScript API or inference changed (not applicable; no public API change)
  • Reviewed the automated runtime-performance report, when runtime behavior changed (three independent local full processes plus the quick report renderer)

@github-actions

github-actions Bot commented Aug 8, 2026

Copy link
Copy Markdown
Contributor

Type performance

Measured with TypeScript 6.0.3 and skipLibCheck=true.

Scenario Base PR Difference
Effect only 55 55 0 (0.0%)
Import effect-machine 55 55 0 (0.0%)
Machine.defineStates (3 states) 2,965 2,965 0 (0.0%)
Machine.make (3 states, 2 events) 8,715 8,715 0 (0.0%)
machine.handle (3 states, 2 transitions) 23,492 23,492 0 (0.0%)
machine.handle (depth 8) 117,075 117,075 0 (0.0%)
machine.handle (depth 12) 129,713 129,713 0 (0.0%)
machine.handle (depth 16) 145,055 145,055 0 (0.0%)
machine.handle (depth 24) 183,851 183,851 0 (0.0%)
machine.handle (wide depth 16) 214,189 214,189 0 (0.0%)
machine.handle (parallel/history/choice) 132,667 132,667 0 (0.0%)
machine.handle (4 successive calls) 128,674 128,674 0 (0.0%)
machine exact input/output/error/services 110,584 110,584 0 (0.0%)
execution adapter readiness 127,854 127,854 0 (0.0%)

Marginal instantiations are measured against the matching setup without that API call:

Scenario Base PR Difference
Import effect-machine 0 0 0
Machine.defineStates (3 states) 2,910 2,910 0 (0.0%)
Machine.make (3 states, 2 events) 5,742 5,742 0 (0.0%)
machine.handle (3 states, 2 transitions) 14,777 14,777 0 (0.0%)
machine.handle (depth 8) 109,601 109,601 0 (0.0%)
machine.handle (depth 12) 121,039 121,039 0 (0.0%)
machine.handle (depth 16) 135,181 135,181 0 (0.0%)
machine.handle (depth 24) 171,577 171,577 0 (0.0%)
machine.handle (wide depth 16) 202,224 202,224 0 (0.0%)
machine.handle (parallel/history/choice) 114,331 114,331 0 (0.0%)
machine.handle (4 successive calls) 113,703 113,703 0 (0.0%)
machine exact input/output/error/services 100,670 100,670 0 (0.0%)
execution adapter readiness 102,092 102,092 0 (0.0%)
Check times (informational)
Scenario Base PR
Effect only 0.03s 0.04s
Import effect-machine 0.03s 0.03s
Machine.defineStates (3 states) 0.11s 0.10s
Machine.make (3 states, 2 events) 0.16s 0.16s
machine.handle (3 states, 2 transitions) 0.22s 0.23s
machine.handle (depth 8) 0.52s 0.52s
machine.handle (depth 12) 0.55s 0.57s
machine.handle (depth 16) 0.61s 0.61s
machine.handle (depth 24) 0.70s 0.71s
machine.handle (wide depth 16) 0.76s 0.77s
machine.handle (parallel/history/choice) 0.60s 0.62s
machine.handle (4 successive calls) 0.58s 0.58s
machine exact input/output/error/services 0.51s 0.50s
execution adapter readiness 0.58s 0.58s

Type instantiations are the comparison metric. Check time varies with runner load and is informational only.

@github-actions

github-actions Bot commented Aug 8, 2026

Copy link
Copy Markdown
Contributor

Runtime performance

Median of 5 independent benchmark processes on AMD EPYC 9V74 80-Core Processor with Node v24.18.0.

Pull request baseline

Scenario Effect Machine XState 5 XState 6 alpha Effect runtime primitives
Plan counter transitions 132,775 transitions/s 178,864 transitions/s 147,715 transitions/s
Drain burst with terminal fence 105,954 increments/s 507,168 increments/s 137,371 increments/s
Drain burst with a change observer 86,300 increments/s 503,029 increments/s 136,991 increments/s
Lookup and send to one child 97,609 increments/s 500,999 increments/s 133,227 increments/s
Start and stop a machine 29,394 machines/s 233,263 machines/s 93,484 machines/s
Start and stop a parent with one child 8,655 families/s 93,145 families/s 38,867 families/s
Start and stop a raw compiled process 38,023 processes/s
Start and interrupt a suspended fiber 202,102 fibers/s
Start and stop a queue worker 146,199 workers/s
Start and stop an actor shell 136,780 actors/s
Start and stop two actor shells 85,346 families/s
Update an owner-only mutable snapshot 401,123,145 updates/s
Update a synchronized snapshot 2,039,613 updates/s
Create, resolve, and await a terminal latch 2,165,219 latches/s
Memory profile Effect Machine XState 5 XState 6 alpha Effect runtime primitives
Idle machine 14.0 KiB 3.8 KiB 6.1 KiB
Raw generic managed process 13.2 KiB
Raw compiled process 12.4 KiB
Two independent idle machines 27.4 KiB 7.3 KiB 11.8 KiB
Idle parent with one child 35.2 KiB 5.5 KiB 8.8 KiB
Parent with observed child registry 46.7 KiB
Parent with observed invoked child snapshots 35.4 KiB
Suspended Effect fiber 0.6 KiB
Effect queue with waiting fiber 2.8 KiB
Effect mailbox actor shell 3.2 KiB
Two Effect actor shells 6.9 KiB

Effect Machine change from base

Metric Base Base variability PR PR variability Difference
Plan counter transitions 140,394 transitions/s 0.8% MAD 132,775 transitions/s 1.9% MAD -5.4%
Drain burst with terminal fence 105,538 increments/s 0.8% MAD 105,954 increments/s 2.0% MAD +0.4%
Drain burst with a change observer 85,242 increments/s 1.2% MAD 86,300 increments/s 0.5% MAD +1.2%
Lookup and send to one child 97,093 increments/s 0.7% MAD 97,609 increments/s 0.9% MAD +0.5%
Start and stop a machine 29,273 machines/s 3.7% MAD 29,394 machines/s 2.7% MAD +0.4%
Start and stop a parent with one child 8,630 families/s 1.9% MAD 8,655 families/s 0.8% MAD +0.3%
Start and stop a raw compiled process 38,671 processes/s 0.7% MAD 38,023 processes/s 0.6% MAD -1.7%
Idle machine heap per unit 14.0 KiB 0.0% MAD 14.0 KiB 0.0% MAD +0.0%
Raw generic managed process heap per unit 13.2 KiB 0.0% MAD 13.2 KiB 0.0% MAD -0.0%
Raw compiled process heap per unit 12.4 KiB 0.0% MAD 12.4 KiB 0.0% MAD -0.1%
Two independent idle machines heap per unit 27.4 KiB 0.0% MAD 27.4 KiB 0.0% MAD -0.0%
Idle parent with one child heap per unit 35.2 KiB 0.0% MAD 35.2 KiB 0.0% MAD 0.0%
Parent with observed child registry heap per unit 46.7 KiB 0.0% MAD 46.7 KiB 0.0% MAD 0.0%
Parent with observed invoked child snapshots heap per unit 35.4 KiB 0.0% MAD 35.4 KiB 0.0% MAD 0.0%
Versions and interpretation
  • Effect Machine: 0.3.0
  • XState 5: 5.32.5
  • XState 6 alpha: 6.0.0-alpha.27
  • Effect runtime primitives: 4.0.0-beta.105

Higher throughput is better; lower heap is better. Variability is the median absolute deviation across independent processes, relative to their median. Runtime measurements on shared GitHub-hosted hardware remain informational, so small differences should be confirmed across multiple workflow runs.

@SandroMaglione
SandroMaglione merged commit ced53ca into main Aug 8, 2026
9 checks passed
@SandroMaglione
SandroMaglione deleted the codex/effect-runtime-ceilings branch August 8, 2026 17:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant