Skip to content

Log cold start time#890

Draft
KVSRoyal wants to merge 1 commit into
mainfrom
cold-start-metrics
Draft

Log cold start time#890
KVSRoyal wants to merge 1 commit into
mainfrom
cold-start-metrics

Conversation

@KVSRoyal

@KVSRoyal KVSRoyal commented Jun 26, 2026

Copy link
Copy Markdown
Member

WIP. Do not review / merge.

Change Type (required)

Indicate the type of change your pull request is:

  • patch
  • minor
  • major
  • N/A

Does your PR have any documentation updates?

  • Updated docs
  • No Update needed
  • Unable to update docs
📦 Published PR as canary version: 1.0.2--canary.890.38467

Try this version out locally by upgrading relevant packages to 1.0.2--canary.890.38467

Comment thread .husky/pre-commit
. "$(dirname "$0")/_/husky.sh"

yarn lint-staged
pnpm lint-staged

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This hasn't been working because the repo uses pnpm, not yarn.

@intuit-svc

intuit-svc commented Jun 26, 2026

Copy link
Copy Markdown
Contributor

Benchmark Results

Comparison against baseline from main. ⚠️ = regression (>10% slower), ✅ = improvement (>5% faster)

core/player ⚠️

Benchmark Current Baseline Change
core/player/src/binding/__tests__/parser.bench.ts > parser benchmarks > Resolving binding: foo.bar 848.61K ops/s 876.50K ops/s -3.2%
core/player/src/binding/__tests__/parser.bench.ts > parser benchmarks > Resolving binding: foo.pets.1.name 441.09K ops/s 449.71K ops/s -1.9%
core/player/src/binding/__tests__/parser.bench.ts > parser benchmarks > Resolving binding: foo.pets.01.name 474.18K ops/s 476.07K ops/s -0.4%
core/player/src/binding/__tests__/parser.bench.ts > parser benchmarks > Resolving binding: foo.pets['01'].name 394.72K ops/s 448.21K ops/s -11.9% ⚠️
core/player/src/binding/__tests__/parser.bench.ts > parser benchmarks > Resolving binding: foo.pets[01].name 467.74K ops/s 514.64K ops/s -9.1%
core/player/src/binding/__tests__/parser.bench.ts > parser benchmarks > Resolving binding: foo.pets[name = "frodo"].type 275.16K ops/s 300.59K ops/s -8.5%
core/player/src/binding/__tests__/parser.bench.ts > parser benchmarks > Resolving binding: foo.pets["name" = "sprinkles"].type 205.24K ops/s 227.64K ops/s -9.8%
core/player/src/binding/__tests__/parser.bench.ts > parser benchmarks > Resolving binding: foo.pets["isDog" = false].type 255.43K ops/s 297.85K ops/s -14.2% ⚠️
core/player/src/binding/__tests__/parser.bench.ts > parser benchmarks > Resolving binding: foo.pets["isDog" = true].type 259.40K ops/s 288.53K ops/s -10.1% ⚠️
core/player/src/binding/__tests__/parser.bench.ts > binding creation benchmarks > Resolving binding: foo.bar 548.12K ops/s 553.90K ops/s -1.0%
core/player/src/binding/__tests__/parser.bench.ts > binding creation benchmarks > Resolving binding: foo.pets.1.name 340.81K ops/s 377.98K ops/s -9.8%
core/player/src/binding/__tests__/parser.bench.ts > binding creation benchmarks > Resolving binding: foo.pets.01.name 314.07K ops/s 369.83K ops/s -15.1% ⚠️
core/player/src/binding/__tests__/parser.bench.ts > binding creation benchmarks > Resolving binding: foo.pets['01'].name 282.38K ops/s 316.97K ops/s -10.9% ⚠️
core/player/src/binding/__tests__/parser.bench.ts > binding creation benchmarks > Resolving binding: foo.pets[01].name 311.74K ops/s 365.07K ops/s -14.6% ⚠️
core/player/src/binding/__tests__/parser.bench.ts > binding creation benchmarks > Resolving binding: foo.pets[name = "frodo"].type 222.46K ops/s 239.33K ops/s -7.1%
core/player/src/binding/__tests__/parser.bench.ts > binding creation benchmarks > Resolving binding: foo.pets["name" = "sprinkles"].type 148.90K ops/s 192.12K ops/s -22.5% ⚠️
core/player/src/binding/__tests__/parser.bench.ts > binding creation benchmarks > Resolving binding: foo.pets["isDog" = false].type 161.95K ops/s 232.73K ops/s -30.4% ⚠️
core/player/src/binding/__tests__/parser.bench.ts > binding creation benchmarks > Resolving binding: foo.pets["isDog" = true].type 247.62K ops/s 245.16K ops/s +1.0%
core/player/src/expressions/__tests__/performance.bench.ts > Expression Parsing/Execution Benchmark > Parsing: {{foo}} = 1 + 3 (sync) 426.07K ops/s 452.51K ops/s -5.8%
core/player/src/expressions/__tests__/performance.bench.ts > Expression Parsing/Execution Benchmark > Parsing: {{foo}} = 1 + 3 (async) 309.11K ops/s 372.16K ops/s -16.9% ⚠️
core/player/src/expressions/__tests__/performance.bench.ts > Expression Parsing/Execution Benchmark > Parsing: conditional(true, true, false) (sync) 419.36K ops/s 472.88K ops/s -11.3% ⚠️
core/player/src/expressions/__tests__/performance.bench.ts > Expression Parsing/Execution Benchmark > Parsing: conditional(true, true, false) (async) 366.44K ops/s 419.72K ops/s -12.7% ⚠️
core/player/src/expressions/__tests__/performance.bench.ts > Expression Parsing/Execution Benchmark > Parsing: {{foo}} = conditional({{bar}} > 0, true, false) (sync) 183.37K ops/s 175.26K ops/s +4.6%
core/player/src/expressions/__tests__/performance.bench.ts > Expression Parsing/Execution Benchmark > Parsing: {{foo}} = conditional({{bar}} > 0, true, false) (async) 172.76K ops/s 151.14K ops/s +14.3% ✅
core/player/src/expressions/__tests__/performance.bench.ts > Expression Parsing/Execution Benchmark > Parsing: {{foo}} = conditional(conditional(true = false, false, true), conditional(false = false, true, false), conditional(true = true, false, true)) (sync) 142.76K ops/s 160.57K ops/s -11.1% ⚠️
core/player/src/expressions/__tests__/performance.bench.ts > Expression Parsing/Execution Benchmark > Parsing: {{foo}} = conditional(conditional(true = false, false, true), conditional(false = false, true, false), conditional(true = true, false, true)) (async) 137.21K ops/s 142.90K ops/s -4.0%
core/player/src/expressions/__tests__/performance.bench.ts > Expression Parsing/Execution Benchmark > Parsing: {{foo}} = await(asyncTestFunction(1)) (sync) N/A N/A N/A
core/player/src/expressions/__tests__/performance.bench.ts > Expression Parsing/Execution Benchmark > Parsing: {{foo}} = await(asyncTestFunction(1)) (async) 227.27K ops/s 279.31K ops/s -18.6% ⚠️
core/player/src/expressions/__tests__/performance.bench.ts > Expression Parsing/Execution Benchmark > Parsing: {{foo}} = asyncTestFunction(1) (sync) 311.32K ops/s 292.82K ops/s +6.3% ✅
core/player/src/expressions/__tests__/performance.bench.ts > Expression Parsing/Execution Benchmark > Parsing: {{foo}} = asyncTestFunction(1) (async) 287.78K ops/s 282.35K ops/s +1.9%
core/player/src/expressions/__tests__/performance.bench.ts > Expression Parsing/Execution Benchmark > Parsing: asyncTestFunction(1) (sync) 760.50K ops/s 868.99K ops/s -12.5% ⚠️
core/player/src/expressions/__tests__/performance.bench.ts > Expression Parsing/Execution Benchmark > Parsing: asyncTestFunction(1) (async) 552.13K ops/s 691.51K ops/s -20.2% ⚠️
core/player/src/expressions/__tests__/performance.bench.ts > Expression Parsing/Execution Benchmark > Parsing: {{foo}} = conditional(!{{bar}} == false, await(asyncTestFunction(1)), false) (sync) 163.03K ops/s 174.27K ops/s -6.5%
core/player/src/expressions/__tests__/performance.bench.ts > Expression Parsing/Execution Benchmark > Parsing: {{foo}} = conditional(!{{bar}} == false, await(asyncTestFunction(1)), false) (async) 164.37K ops/s 147.52K ops/s +11.4% ✅
core/player/src/view/resolver/__tests__/index.bench.ts > resolver benchmarks > initial resolve 609.92 ops/s 660.87 ops/s -7.7%
core/player/src/view/resolver/__tests__/index.bench.ts > resolver benchmarks > Resolving from cache 15.02K ops/s 18.64K ops/s -19.4% ⚠️
core/player/src/view/resolver/__tests__/index.bench.ts > resolver benchmarks > data changes 2.72K ops/s 2.85K ops/s -4.7%
core/player/src/view/resolver/__tests__/index.bench.ts > resolver benchmarks > data changes slow 535.96 ops/s 595.54 ops/s -10.0% ⚠️

plugins/async-node/core ⚠️

Benchmark Current Baseline Change
plugins/async-node/core/src/__tests__/index.bench.ts > async node benchmarks > Resolve Async Node 1 times 12.31K ops/s 14.87K ops/s -17.2% ⚠️
plugins/async-node/core/src/__tests__/index.bench.ts > async node benchmarks > Resolve Async Node 5 times 12.34K ops/s 14.30K ops/s -13.7% ⚠️
plugins/async-node/core/src/__tests__/index.bench.ts > async node benchmarks > Resolve Async Node 10 times 9.70K ops/s 10.31K ops/s -5.9%
plugins/async-node/core/src/__tests__/index.bench.ts > async node benchmarks > Resolve Async Node 50 times 3.02K ops/s 3.54K ops/s -14.6% ⚠️
plugins/async-node/core/src/__tests__/index.bench.ts > async node benchmarks > Resolve Async Node 100 times 1.48K ops/s 1.86K ops/s -20.4% ⚠️
plugins/async-node/core/src/__tests__/transform.bench.ts > async transform benchmarks > Resolve Async Node 1 times 6.73K ops/s 7.48K ops/s -10.0%
plugins/async-node/core/src/__tests__/transform.bench.ts > async transform benchmarks > Resolve Async Node 5 times 7.69K ops/s 8.19K ops/s -6.1%
plugins/async-node/core/src/__tests__/transform.bench.ts > async transform benchmarks > Resolve Async Node 10 times 6.79K ops/s 7.06K ops/s -3.8%
plugins/async-node/core/src/__tests__/transform.bench.ts > async transform benchmarks > Resolve Async Node 50 times 2.52K ops/s 2.60K ops/s -3.0%
plugins/async-node/core/src/__tests__/transform.bench.ts > async transform benchmarks > Resolve Async Node 100 times 1.56K ops/s 1.65K ops/s -5.6%

react/player ⚠️

Benchmark Current Baseline Change
react/player/src/asset/__tests__/index.bench.tsx > ReactAsset benchmarks > Render asset nested in 1 ReactAssets 529.98 ops/s 634.79 ops/s -16.5% ⚠️
react/player/src/asset/__tests__/index.bench.tsx > ReactAsset benchmarks > Bubble errors nested in 1 ReactAssets 673.25 ops/s 1.17K ops/s -42.3% ⚠️
react/player/src/asset/__tests__/index.bench.tsx > ReactAsset benchmarks > Render asset nested in 5 ReactAssets 511.73 ops/s 645.24 ops/s -20.7% ⚠️
react/player/src/asset/__tests__/index.bench.tsx > ReactAsset benchmarks > Bubble errors nested in 5 ReactAssets 670.95 ops/s 1.16K ops/s -42.0% ⚠️
react/player/src/asset/__tests__/index.bench.tsx > ReactAsset benchmarks > Render asset nested in 10 ReactAssets 473.40 ops/s 648.21 ops/s -27.0% ⚠️
react/player/src/asset/__tests__/index.bench.tsx > ReactAsset benchmarks > Bubble errors nested in 10 ReactAssets 809.97 ops/s 942.18 ops/s -14.0% ⚠️
react/player/src/asset/__tests__/index.bench.tsx > ReactAsset benchmarks > Render asset nested in 50 ReactAssets 465.99 ops/s 531.30 ops/s -12.3% ⚠️
react/player/src/asset/__tests__/index.bench.tsx > ReactAsset benchmarks > Bubble errors nested in 50 ReactAssets 219.65 ops/s 270.47 ops/s -18.8% ⚠️
react/player/src/asset/__tests__/index.bench.tsx > ReactAsset benchmarks > Render asset nested in 100 ReactAssets 345.81 ops/s 429.00 ops/s -19.4% ⚠️
react/player/src/asset/__tests__/index.bench.tsx > ReactAsset benchmarks > Bubble errors nested in 100 ReactAssets 86.23 ops/s 126.43 ops/s -31.8% ⚠️

@KVSRoyal KVSRoyal force-pushed the cold-start-metrics branch from 15b3c52 to 9c98289 Compare June 26, 2026 22:18
@codecov

codecov Bot commented Jun 26, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 0.00%. Comparing base (6fe28cc) to head (4f330c5).
⚠️ Report is 3 commits behind head on main.

Additional details and impacted files
@@     Coverage Diff     @@
##   main   #890   +/-   ##
===========================
===========================

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@KVSRoyal KVSRoyal added the patch Increment the patch version when merged label Jun 26, 2026
@KVSRoyal KVSRoyal marked this pull request as ready for review June 29, 2026 17:19
@KVSRoyal KVSRoyal requested review from a team as code owners June 29, 2026 17:19
@KVSRoyal

Copy link
Copy Markdown
Member Author

/canary

private reactPlayerInfo: ReactPlayerInfo;

constructor(options?: ReactPlayerOptions) {
const startTime = new Date();

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For react, we may want to use the performance API which should give is a higher fidelity value

intuit-svc added a commit to player-ui/player-ui.github.io that referenced this pull request Jun 29, 2026
@KVSRoyal KVSRoyal force-pushed the cold-start-metrics branch from 9c98289 to 4f330c5 Compare July 1, 2026 16:19
@KVSRoyal KVSRoyal marked this pull request as draft July 1, 2026 16:37
@KVSRoyal

KVSRoyal commented Jul 1, 2026

Copy link
Copy Markdown
Member Author

/canary

intuit-svc added a commit to player-ui/player-ui.github.io that referenced this pull request Jul 1, 2026
@intuit-svc

Copy link
Copy Markdown
Contributor

Build Preview

Your PR was deployed by CircleCI #38467 on Wed, 01 Jul 2026 17:27:25 GMT with this version:

1.0.2--canary.890.38467

📖 Docs (View site)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

patch Increment the patch version when merged

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants