Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/lazy-children-rest.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"@typeonce/effect-machine": patch
---

Allocate child process scopes and observable child registries only when a machine uses child-management capabilities.
30 changes: 14 additions & 16 deletions .github/workflows/runtime-performance.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ jobs:
runtime-performance:
name: runtime-performance
runs-on: ubuntu-latest
timeout-minutes: 15
timeout-minutes: 25
steps:
- name: Check out base
uses: actions/checkout@v7
Expand Down Expand Up @@ -46,8 +46,7 @@ jobs:
pnpm --dir head install --frozen-lockfile
pnpm --dir head build

- name: Install and build base when benchmarkable
if: ${{ hashFiles('base/scripts/runtime-performance.mjs') != '' }}
- name: Install and build base
run: |
pnpm --dir base install --frozen-lockfile
pnpm --dir base build
Expand All @@ -57,29 +56,28 @@ jobs:
run: |
set -euo pipefail
reports="$RUNNER_TEMP/runtime-performance"
harness="$GITHUB_WORKSPACE/head/scripts/runtime-performance.mjs"
mkdir -p "$reports/base" "$reports/head"

measure() {
local checkout="$1"
local output="$2"
(
cd "$checkout"
node --expose-gc scripts/runtime-performance.mjs --json
EFFECT_MACHINE_BENCHMARK_ROOT="$PWD" node --expose-gc "$harness" --json
) > "$output"
}

if [[ -f base/scripts/runtime-performance.mjs ]]; then
measure base "$reports/base/1.json"
measure head "$reports/head/1.json"
measure head "$reports/head/2.json"
measure base "$reports/base/2.json"
measure base "$reports/base/3.json"
measure head "$reports/head/3.json"
else
measure head "$reports/head/1.json"
measure head "$reports/head/2.json"
measure head "$reports/head/3.json"
fi
measure base "$reports/base/1.json"
measure head "$reports/head/1.json"
measure head "$reports/head/2.json"
measure base "$reports/base/2.json"
measure base "$reports/base/3.json"
measure head "$reports/head/3.json"
measure head "$reports/head/4.json"
measure base "$reports/base/4.json"
measure base "$reports/base/5.json"
measure head "$reports/head/5.json"

node head/scripts/compare-runtime-performance.mjs \
"$reports/base" \
Expand Down
21 changes: 14 additions & 7 deletions perf/runtime/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,12 @@ The command reports:

- pure `Machine.plan` counter-transition throughput;
- end-to-end useful-increment throughput for a burst sent to one running machine;
- the same running burst while a consumer drains every published snapshot;
- repeated child lookup and delivery to one running child;
- machine start-and-stop throughput;
- idle heap and resident-memory growth at 100, 500, and 1,000 live machines.
- parent-with-child start-and-stop throughput;
- heap and resident-memory growth for both idle machines and idle parents with
one child, at 100, 500, and 1,000 live units.

The comparison dependencies use package aliases, so XState 5 and 6 can be
loaded by the same process:
Expand Down Expand Up @@ -68,12 +72,15 @@ pnpm perf:runtime -- --output runtime-performance.json
Prefer `--output` for scripts: pnpm and the preceding build may add their own
lines to standard output before `--json` is printed.

Pull requests run the suite three times for both the base and pull request
revisions on the same GitHub-hosted runner. The workflow publishes the median
of those process-level results to the job summary and a sticky pull request
comment. The benchmark workflow has read-only repository access; a separate
trusted `workflow_run` workflow validates the uploaded JSON before receiving
permission to update the comment.
Pull requests run the pull request's benchmark harness five times against both
the base and pull request library revisions on the same GitHub-hosted runner.
Using one harness revision means a newly added scenario can compare both
implementations immediately. The runs are interleaved to reduce time-dependent
machine drift. The workflow publishes the process-level median and its median
absolute deviation to the job summary and a sticky pull request comment. The
benchmark workflow has read-only repository access; a separate trusted
`workflow_run` workflow validates the uploaded JSON before receiving permission
to update the comment.

The implementation lives in `scripts/runtime-performance.mjs`; the Effect
Machine fixture is in `perf/runtime/counter.mjs`, and the comparison adapter is
Expand Down
134 changes: 132 additions & 2 deletions perf/runtime/counter.mjs
Original file line number Diff line number Diff line change
@@ -1,5 +1,17 @@
import { Effect, Schema } from "effect"
import { Machine } from "../../dist/index.js"
import { readFileSync } from "node:fs"
import { createRequire } from "node:module"
import { dirname, join, resolve } from "node:path"
import { fileURLToPath, pathToFileURL } from "node:url"

const implementationRoot = resolve(
process.env.EFFECT_MACHINE_BENCHMARK_ROOT ?? fileURLToPath(new URL("../..", import.meta.url))
)
const implementationRequire = createRequire(pathToFileURL(join(implementationRoot, "package.json")))
const effectPackagePath = implementationRequire.resolve("effect/package.json")
const effectPackage = JSON.parse(readFileSync(effectPackagePath, "utf8"))
const effect = await import(pathToFileURL(resolve(dirname(effectPackagePath), effectPackage.exports["."])).href)
const { Machine } = await import(pathToFileURL(join(implementationRoot, "dist/index.js")).href)
const { Effect, Fiber, Option, Schema, Stream } = effect

const CounterState = Schema.TaggedUnion({
Count: {
Expand Down Expand Up @@ -42,6 +54,20 @@ export const counterMachine = Machine.make({
}
})

const ParentState = Schema.TaggedUnion({ Active: {} })
const ParentStates = Machine.defineStates({ Active: ParentState.cases.Active })
const CounterChild = Machine.child("counter", counterMachine)
const counterParentMachine = Machine.make({
id: "RuntimeBenchmarkCounterParent",
states: ParentStates.states,
events: [],
initial: () => ParentStates.initial.Active(ParentState.cases.Active.make({}))
}).handle({
Active: {
invoke: Machine.invokeMachine({ child: CounterChild })
}
})

export const incrementEvent = CounterEvent.cases.Increment.make({})
const finishEvent = CounterEvent.cases.Finish.make({})

Expand All @@ -67,6 +93,76 @@ export const startCounter = () => Effect.runPromise(Machine.start(counterMachine

export const stopCounter = (ref) => Effect.runPromise(ref.stop)

export const startObservedCounter = () =>
Effect.runPromise(
Effect.gen(function*() {
const ref = yield* Machine.start(counterMachine)
const observer = yield* ref.changes.pipe(Stream.runDrain, Effect.forkDetach)
yield* Effect.yieldNow
return { ref, observer }
})
)

export const stopObservedCounter = ({ ref, observer }) =>
Effect.runPromise(
ref.stop.pipe(Effect.ensuring(Fiber.interrupt(observer)))
)

export const runObservedCounterBurst = ({ ref, observer }, size) =>
Effect.runPromise(
Effect.gen(function*() {
for (let index = 0; index < size; index += 1) {
yield* ref.send(incrementEvent)
}
yield* ref.send(finishEvent)
const value = yield* ref.join
yield* Fiber.join(observer)
return value
})
)

const waitForCounterChild = (parent) =>
Effect.gen(function*() {
for (let attempt = 0; attempt < 1_000; attempt += 1) {
const child = yield* parent.child(CounterChild)
if (Option.isSome(child)) {
return child.value
}
yield* Effect.yieldNow
}
return yield* Effect.dieMessage("Effect Machine child did not become ready")
})

export const startChildCounter = () =>
Effect.runPromise(
Effect.gen(function*() {
const parent = yield* Machine.start(counterParentMachine)
yield* waitForCounterChild(parent)
return parent
})
)

export const stopChildCounter = (parent) => Effect.runPromise(parent.stop)

export const runChildCounterBurst = (parent, size) =>
Effect.runPromise(
Effect.gen(function*() {
for (let index = 0; index < size; index += 1) {
const child = yield* parent.child(CounterChild)
if (Option.isNone(child)) {
return yield* Effect.dieMessage("Effect Machine child disappeared during the benchmark")
}
yield* child.value.send(incrementEvent)
}
const child = yield* parent.child(CounterChild)
if (Option.isNone(child)) {
return yield* Effect.dieMessage("Effect Machine child disappeared before the terminal fence")
}
yield* child.value.send(finishEvent)
return yield* child.value.join
})
)

export const runCounterBurst = (ref, size) =>
Effect.runPromise(
Effect.gen(function*() {
Expand Down Expand Up @@ -95,13 +191,31 @@ export const stopCounters = (refs) =>
})
)

export const startChildCounters = (count) =>
Effect.runPromise(
Effect.forEach(
Array.from({ length: count }),
() =>
Effect.gen(function*() {
const parent = yield* Machine.start(counterParentMachine)
yield* waitForCounterChild(parent)
return parent
}),
{ concurrency: 1 }
)
)

export const stopChildCounters = stopCounters

export const effectMachineAdapter = {
implementation: "effect-machine",
label: "Effect Machine",
version: undefined,
async: true,
planCounterBatch,
runCounterBurst,
runObservedCounterBurst,
runChildCounterBurst,
runLifecycle: async () => {
const ref = await startCounter()
try {
Expand All @@ -112,8 +226,24 @@ export const effectMachineAdapter = {
await stopCounter(ref)
}
},
runChildLifecycle: async () => {
const parent = await startChildCounter()
try {
if (!parent.sessionId.startsWith("machine:")) {
throw new Error(`Child lifecycle benchmark produced invalid session id ${parent.sessionId}`)
}
} finally {
await stopChildCounter(parent)
}
},
startCounter,
startObservedCounter,
startChildCounter,
startCounters,
startChildCounters,
stopCounter,
stopChildCounter,
stopChildCounters,
stopObservedCounter,
stopCounters
}
9 changes: 7 additions & 2 deletions perf/runtime/implementations.mjs
Original file line number Diff line number Diff line change
@@ -1,14 +1,19 @@
import { readFileSync } from "node:fs"
import { resolve } from "node:path"
import { fileURLToPath } from "node:url"
import * as XStateV5 from "xstate-v5"
import * as XStateV6 from "xstate-v6"
import { effectMachineAdapter } from "./counter.mjs"
import { makeXStateAdapter } from "./xstate.mjs"

const readPackageVersion = (path) => JSON.parse(readFileSync(path, "utf8")).version
const implementationRoot = resolve(
process.env.EFFECT_MACHINE_BENCHMARK_ROOT ?? fileURLToPath(new URL("../..", import.meta.url))
)

export const packageVersions = {
effectMachine: readPackageVersion(new URL("../../package.json", import.meta.url)),
effect: readPackageVersion(new URL("../../node_modules/effect/package.json", import.meta.url)),
effectMachine: readPackageVersion(resolve(implementationRoot, "package.json")),
effect: readPackageVersion(resolve(implementationRoot, "node_modules/effect/package.json")),
tinybench: readPackageVersion(new URL("../../node_modules/tinybench/package.json", import.meta.url)),
xstateV5: readPackageVersion(new URL("../../node_modules/xstate-v5/package.json", import.meta.url)),
xstateV6: readPackageVersion(new URL("../../node_modules/xstate-v6/package.json", import.meta.url))
Expand Down
31 changes: 27 additions & 4 deletions perf/runtime/memory-worker.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -8,12 +8,33 @@ if (typeof globalThis.gc !== "function") {

const implementationId = process.argv[2]
const counts = JSON.parse(process.argv[3] ?? "[]")
const profileId = process.argv[4] ?? "idle"
const implementation = implementations.find((candidate) => candidate.implementation === implementationId)

if (implementation === undefined) {
throw new Error(`Unknown runtime benchmark implementation: ${implementationId}`)
}

const profile = profileId === "idle"
? {
id: "idle",
label: "Idle machine",
start: implementation.startCounters,
stop: implementation.stopCounters
}
: profileId === "parent-with-child"
? {
id: "parent-with-child",
label: "Idle parent with one child",
start: implementation.startChildCounters,
stop: implementation.stopChildCounters
}
: undefined

if (profile === undefined) {
throw new Error(`Unknown runtime memory profile: ${profileId}`)
}

const collectGarbage = async () => {
for (let index = 0; index < 3; index += 1) {
globalThis.gc()
Expand All @@ -35,8 +56,8 @@ const linearSlope = (points, value) => {
}

// Trigger implementation-specific lazy initialization before taking the baseline.
const warmup = await implementation.startCounter()
await implementation.stopCounter(warmup)
const warmup = await profile.start(1)
await profile.stop(warmup)
await collectGarbage()

const baseline = process.memoryUsage()
Expand All @@ -45,7 +66,7 @@ const refs = []

try {
for (const count of counts) {
refs.push(...await implementation.startCounters(count - refs.length))
refs.push(...await profile.start(count - refs.length))
await collectGarbage()
const usage = process.memoryUsage()
points.push({
Expand All @@ -57,14 +78,16 @@ try {
})
}
} finally {
await implementation.stopCounters(refs)
await profile.stop(refs)
await collectGarbage()
}

process.stdout.write(JSON.stringify({
implementation: implementation.implementation,
implementationLabel: implementation.label,
implementationVersion: implementation.version,
id: profile.id,
label: profile.label,
baseline: {
heapUsedBytes: baseline.heapUsed,
rssBytes: baseline.rss
Expand Down
Loading