ci: add dedicated Release benchmark app and performance reporting

.github/PERFORMANCE.md

@@ -0,0 +1,95 @@
+# Release performance CI
+
+`Nitro Performance` builds `apps/benchmark` in its ordinary Release/Hermes configuration,
+then runs base → head → head → base on one booted target. A noisy comparison can
+run one additional head/base pair. Each suite measures 40 cases with five warmup
+batches and twenty samples, calibrated toward 150 ms (roughly 100–200 ms) of
+timed work per sample. Iteration counts are rounded to two significant digits
+and frozen after warmup; measured outliers are retained. Allocation-heavy samples
+sum bounded timed chunks with explicit GC and native-cleanup yields between chunks excluded from timing,
+so memory-safety limits do not shorten the sample. See `iterations` and
+`chunkIterations` in raw results; sample milliseconds are `ns/op * iterations / 1e6`.
+Calibration changes and version-2 benchmark definitions require a new baseline.
+Each binary is installed once per suite run. Each case uses a fresh app process
+so the runtime-scoped JSI reference cache does not accumulate millions of weak
+reference records across unrelated cases. Startup is outside measured work;
+the host validates and combines the per-case results, preserving their raw samples.
+The host computes confidence intervals by resampling whole matched base/head
+process runs, then batches within each selected run. Bencher median bounds use
+the same run-aware approach. Treating all batches from multiple processes as
+independent gave a false-positive control result in the initial A/A validation;
+process-level variability must be preserved in the uncertainty estimate.
+Robust CV remains visible and causes a neutral result when uncertainty overlaps
+zero. It is not an unconditional veto: an effect beyond the budget whose full
+run-aware confidence interval excludes zero is still reported in its direction.
+
+The benchmark app is separate from `apps/example`, which keeps the demos and
+Harness correctness tests. It shares the real Nitro test packages, but has no
+Harness, navigation, screens, or safe-area dependencies. The TurboModule control
+and all benchmark cases belong to the benchmark app. Android permits
+cleartext only to loopback; the example app does not contain the CI entrypoint.
+See [`apps/benchmark/README.md`](../apps/benchmark/README.md) for local commands.
+
+## Android host requirements
+
+The API 36 x86_64 emulator **must use KVM CPU acceleration**. The workflow grants
+the ephemeral runner access to `/dev/kvm`, verifies acceleration before boot, and
+launches with `-accel on`. Missing acceleration fails the job rather than silently
+falling back to software CPU emulation. Software GPU rendering is separate and is
+still used on the headless runner.
+
+The initial bootstrap run accidentally used `-accel off`: boot alone took 8½
+minutes and six suites took another 32½ minutes. Those Android timings are not a
+performance baseline. Accelerated results use a distinct `...-kvm` Bencher testbed.
+
+Host-side log lines show each suite's start, completion, measurement duration,
+total wall time (including app installation/launch), and why a repeat pair ran.
+These logs are outside the app's timed regions. Raw JSON, BMF, and comparison
+Markdown are retained in the workflow artifacts for 30 days.
+Both targets have a five-minute boot limit; individual device installation and
+launch commands have a two-minute limit. Android app exits also retain logcat and
+process-exit diagnostics, so native crashes do not disappear at emulator teardown.
+
+Same-repository PRs publish from a separate clean job after both device jobs
+finish. This works before merge: it checks out an immutable, reviewed reporting
+commit, downloads only result data, validates it against GitHub API metadata,
+and rebuilds the table/BMF without installing or executing either app checkout.
+Fork PRs never enter this privileged job; their `workflow_run` reporter becomes
+active once its definition reaches the default branch. That reporter skips
+same-repository PRs to avoid duplicate uploads. Both paths post the rebuilt
+paired-comparison table as one updatable PR comment, independently of Bencher's
+historical comparison. Stale results do
+not overwrite a newer PR revision, and user-authored comments are never edited.
+Revoke the previously exposed credential and replace the
+repository's `BENCHER_KEY` secret before enabling publishing. PR jobs never receive
+that secret in device/build jobs or any fork job. After rotation, set the repository variable `NITRO_BENCHER_ENABLED`
+to `true` to enable Bencher uploads. The paired PR comment does not need that key.
+Verdicts remain advisory during noise calibration.
+
+The pre-merge publisher requires a real base benchmark app. The infrastructure
+PR's bootstrap A/A runs remain diagnostic artifacts, not Bencher baselines.
+For paired PR reports, first upload the measured base to `baseline-<full-base-sha>`
+on each testbed, then upload `pr-<number>` with that exact baseline as its start
+point. This works with an empty Bencher project and with stacked PRs without
+pretending their base is `main`. Pushes/scheduled main runs record main history.
+Both platform baselines are uploaded before either head. The publisher does not
+reset the PR branch for each platform, preserving the other testbed's reports;
+a changed base SHA naturally selects a different start point.
+The Bencher action and downloaded CLI version are both pinned. Both publishers
+verify the reviewed Linux CLI SHA-256 before the step receiving the API key.
+
+## Promoting performance verdicts to a gate
+
+This initial workflow always passes `--mode advisory`; merging it does **not**
+enable performance enforcement. Collect at least 30 successful main/no-change
+runs on each unchanged suite and testbed before a separate reviewed promotion.
+Use only same-commit comparisons to estimate noise; ordinary base/head deltas
+may include real code changes and must not inflate the noise allowance.
+
+For each synchronous case, the promotion must set a per-case budget of
+`max(5%, 1.5 × p95(abs(no-change delta)))`. Cases requiring over 10%, Promise
+metrics, and inconclusive comparisons remain advisory. The current 5% table
+threshold is provisional, not a calibrated gate. Do not simply change `--mode`
+to `enforce`: the initial comparator uses that single provisional threshold, and
+the trusted reporter deliberately rejects PR attempts to enable enforcement.
+Promotion needs the reviewed per-case policy and matching reporter support.

.github/workflows/performance-report.yml

@@ -0,0 +1,85 @@
+name: Publish Nitro Performance
+
+on:
+  workflow_run:
+    workflows: [Nitro Performance]
+    types: [completed]
+
+permissions:
+  actions: read
+  checks: write
+  contents: read
+  pull-requests: write
+
+jobs:
+  publish:
+    # Same-repository PRs publish in a separate job of Nitro Performance, which
+    # also works before this workflow reaches the default branch.
+    if: >-
+      github.event.workflow_run.event != 'pull_request' ||
+      github.event.workflow_run.head_repository.full_name != github.repository
+    runs-on: ubuntu-24.04
+    steps:
+      - name: Checkout trusted reporting code
+        uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
+      - uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 # v2
+        with:
+          bun-version: 1.3.14
+
+      - name: Download performance report
+        id: download
+        continue-on-error: true
+        uses: actions/download-artifact@37930b1c2abaa49bbe596cd826c3c89aef350131 # v7
+        with:
+          name: performance-report
+          path: untrusted-artifact
+          github-token: ${{ secrets.GITHUB_TOKEN }}
+          run-id: ${{ github.event.workflow_run.id }}
+
+      - name: Validate untrusted report
+        if: steps.download.outcome == 'success'
+        env:
+          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+        run: |
+          EVENT_NAME=$(jq -r '.workflow_run.event' "$GITHUB_EVENT_PATH")
+          TRUSTED_PR_ARGUMENTS=()
+          if [[ "$EVENT_NAME" == 'pull_request' ]]; then
+            PR_NUMBER=$(jq -r '.pullRequestNumber' untrusted-artifact/performance-report.json)
+            if [[ ! "$PR_NUMBER" =~ ^[1-9][0-9]*$ ]]; then
+              echo 'Invalid pull request number in performance artifact.' >&2
+              exit 1
+            fi
+            gh api "repos/$GITHUB_REPOSITORY/pulls/$PR_NUMBER" > trusted-pull-request.json
+            TRUSTED_PR_ARGUMENTS=(--trusted-pull-request trusted-pull-request.json)
+          fi
+          bun scripts/performance/validate-report.ts \
+            --artifact-directory untrusted-artifact \
+            --output-directory validated-report \
+            --expected-repository "$GITHUB_REPOSITORY" \
+            --trusted-workflow-event "$GITHUB_EVENT_PATH" \
+            "${TRUSTED_PR_ARGUMENTS[@]}"
+
+      - name: Post paired comparison to the PR
+        if: steps.download.outcome == 'success'
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+        run: bun scripts/performance/github-report.ts --directory validated-report
+
+      - name: Install Bencher CLI
+        if: steps.download.outcome == 'success' && vars.NITRO_BENCHER_ENABLED == 'true'
+        uses: bencherdev/bencher@8d75325c3bc59403a2186a056b472c4f49d42838 # v0.6.12
+        with:
+          version: '0.6.12'
+
+      - name: Verify pinned Bencher binary
+        if: steps.download.outcome == 'success' && vars.NITRO_BENCHER_ENABLED == 'true'
+        run: |
+          printf '%s  %s\n' 'c2d3a6a7fae654246134e5ced1408bdb9ba4e198b0ac3b903af17a06574a7e08' "$(command -v bencher)" | sha256sum --check -
+
+      - name: Publish to Bencher and GitHub
+        if: steps.download.outcome == 'success' && vars.NITRO_BENCHER_ENABLED == 'true'
+        env:
+          BENCHER_API_KEY: ${{ secrets.BENCHER_KEY }}
+          BENCHER_PROJECT: nitro
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+        run: bun scripts/performance/publish.ts --directory validated-report