Skip to content

Add runtime performance benchmarks - #32

Merged
SandroMaglione merged 2 commits into
mainfrom
codex/runtime-benchmarks
Aug 7, 2026
Merged

Add runtime performance benchmarks#32
SandroMaglione merged 2 commits into
mainfrom
codex/runtime-benchmarks

Conversation

@SandroMaglione

@SandroMaglione SandroMaglione commented Aug 7, 2026

Copy link
Copy Markdown
Member

Summary

  • add repeatable local runtime benchmarks for planning, event drainage, lifecycle, and idle memory
  • compare the compiled library with pinned XState 5 and XState 6 alpha baselines
  • produce human-readable output and versioned JSON reports for future automation
  • run base and pull request benchmarks three times on the same GitHub-hosted runner
  • publish the median results through a read-only benchmark workflow and a separate trusted sticky-comment workflow

Changeset

  • Added or updated for a library or package-metadata change
  • Not required because this PR does not change src/ or package.json

Validation

  • pnpm check
  • pnpm perf:types
  • Relevant example checks, when examples changed — not applicable
  • Reviewed the automated type-performance report, when the public TypeScript API or inference changed — no public API change
  • Reviewed the automated runtime-performance report, when runtime behavior changed — bootstrap job and artifact passed; sticky comments activate after the trusted workflow lands on main

@github-actions

github-actions Bot commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

Type performance

Measured with TypeScript 6.0.3 and skipLibCheck=true.

Scenario Base PR Difference
Effect only 55 55 0 (0.0%)
Import effect-machine 55 55 0 (0.0%)
Machine.defineStates (3 states) 2,965 2,965 0 (0.0%)
Machine.make (3 states, 2 events) 8,731 8,731 0 (0.0%)
machine.handle (3 states, 2 transitions) 23,634 23,634 0 (0.0%)
machine.handle (depth 8) 113,023 113,023 0 (0.0%)
machine.handle (depth 12) 125,661 125,661 0 (0.0%)
machine.handle (depth 16) 141,003 141,003 0 (0.0%)
machine.handle (depth 24) 179,799 179,799 0 (0.0%)
machine.handle (wide depth 16) 210,137 210,137 0 (0.0%)
machine.handle (parallel/history/choice) 128,876 128,876 0 (0.0%)
machine.handle (4 successive calls) 129,498 129,498 0 (0.0%)
machine exact input/output/error/services 107,371 107,371 0 (0.0%)
execution adapter readiness 122,890 122,890 0 (0.0%)

Marginal instantiations are measured against the matching setup without that API call:

Scenario Base PR Difference
Import effect-machine 0 0 0
Machine.defineStates (3 states) 2,910 2,910 0 (0.0%)
Machine.make (3 states, 2 events) 5,758 5,758 0 (0.0%)
machine.handle (3 states, 2 transitions) 14,903 14,903 0 (0.0%)
machine.handle (depth 8) 105,546 105,546 0 (0.0%)
machine.handle (depth 12) 116,984 116,984 0 (0.0%)
machine.handle (depth 16) 131,126 131,126 0 (0.0%)
machine.handle (depth 24) 167,522 167,522 0 (0.0%)
machine.handle (wide depth 16) 198,169 198,169 0 (0.0%)
machine.handle (parallel/history/choice) 110,514 110,514 0 (0.0%)
machine.handle (4 successive calls) 114,497 114,497 0 (0.0%)
machine exact input/output/error/services 95,819 95,819 0 (0.0%)
execution adapter readiness 96,968 96,968 0 (0.0%)
Check times (informational)
Scenario Base PR
Effect only 0.02s 0.02s
Import effect-machine 0.02s 0.02s
Machine.defineStates (3 states) 0.08s 0.07s
Machine.make (3 states, 2 events) 0.12s 0.11s
machine.handle (3 states, 2 transitions) 0.18s 0.16s
machine.handle (depth 8) 0.38s 0.38s
machine.handle (depth 12) 0.42s 0.41s
machine.handle (depth 16) 0.45s 0.45s
machine.handle (depth 24) 0.51s 0.53s
machine.handle (wide depth 16) 0.56s 0.57s
machine.handle (parallel/history/choice) 0.46s 0.47s
machine.handle (4 successive calls) 0.44s 0.45s
machine exact input/output/error/services 0.39s 0.40s
execution adapter readiness 0.42s 0.42s

Type instantiations are the comparison metric. Check time varies with runner load and is informational only.

@SandroMaglione
SandroMaglione marked this pull request as ready for review August 7, 2026 16:29
@SandroMaglione
SandroMaglione merged commit 757c10c into main Aug 7, 2026
9 checks passed
@SandroMaglione
SandroMaglione deleted the codex/runtime-benchmarks branch August 7, 2026 16:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant