feat(observability-map): static observability scorer for webapp route entry points (#4455)

A static observability scorer for the webapp's route entry points,
Lighthouse-style. The idea comes from evlog's `map` command, but that
tool has no Remix adapter and checks for its own logging API, so the
idea is ported rather than the tool.

It scans all 427 loader/action entry points in `apps/webapp/app/routes`
with the TypeScript compiler API and scores each against five checks:
error-classification, auth-boundary, auth-scope, request-context and
audit-trail. Current output on the real tree is **19/100** over 412
measured entry points.

```
cd internal-packages/observability-map
pnpm exec tsx src/cli.ts               # terminal report
pnpm exec tsx src/cli.ts --json        # machine output
pnpm exec tsx src/cli.ts api/v1/token  # one entry, per-check detail
```

The two findings at the top of the fix list are real: `/auth/sso` and
`/api/v1/authorization-code` mint or exchange credentials
unauthenticated, and `/_app/orgs/:organizationSlug/settings/team`
resolves its org from a URL slug and gates each mutating branch on an
RBAC check alone, which per `apps/webapp/CLAUDE.md` is not the tenant
floor on self-hosted.

Decisions worth knowing, all with the reasoning in the README:

- The score started at 83 during development and fell to 19. Every drop
was a perverse incentive being removed, not a regression: routes were
being paid for having no error handling, two checks were reading the
same fact, suppressing a failure raised the score, and a no-op `catch
(e) { throw e }` was worth 50 points a route.
- **A mutation corpus is the tool's main defence.** 44 entries apply
semantics-preserving edits to a copy of the real route tree and assert
the score cannot rise, per route as well as globally, because a mean can
hide one route going up by taking another down. One entry runs as a live
expected failure: `try { String(0); }` with a deciding catch is a known
open hole worth 19 to 44, and it is disclosed rather than quietly
excluded.
- `audit-trail` and `request-context` are reported as headline figures
rather than one finding repeated hundreds of times. Both still count in
full where they should.
- A cohort change moves the number without anything in the codebase
getting better. Widening the sensitive cohort from 26 to 67 took the
global from 15 to 19 with no webapp change at all, so the report prints
per-check applicability and what the global would be without each one.

CI: a report-only job posts a sticky comment when a PR moves the report,
and says nothing when it does not. The package's own tests gate through
`pr_checks.yml`. The diff-scoped merge gate is still deferred until the
report has been used in anger.

524 tests plus the corpus. No runtime or dependency changes to anything
that ships.

<!-- GitButler Footer Boundary Top -->
---
This is **part 1 of 4 in a stack** made with GitButler:
- <kbd>&nbsp;4&nbsp;</kbd> #4485
- <kbd>&nbsp;3&nbsp;</kbd> #4484
- <kbd>&nbsp;2&nbsp;</kbd> #4483
- <kbd>&nbsp;1&nbsp;</kbd> #4455 👈 
<!-- GitButler Footer Boundary Bottom -->

Author: 1stvamp

.github/workflows/observability-map.yml

@@ -22,6 +22,7 @@ jobs:
       webapp: ${{ steps.filter.outputs.webapp }}
       packages: ${{ steps.filter.outputs.packages }}
       internal: ${{ steps.filter.outputs.internal }}
+      obsmap: ${{ steps.filter.outputs.obsmap }}
       cli: ${{ steps.filter.outputs.cli }}
       sdk: ${{ steps.filter.outputs.sdk }}
     steps:
@@ -81,6 +82,36 @@ jobs:
               - 'pnpm-lock.yaml'
               - 'pnpm-workspace.yaml'
               - 'turbo.json'
+            # The whole webapp app tree, not just its routes, and that is the whole reason this
+            # filter exists. Two tests in @internal/observability-map read it: integration.test.ts
+            # scans the live route tree, and webappSymbols.test.ts walks all of apps/webapp/app and
+            # fails when a guard, sensitive or audit symbol stops resolving. Routes-only was this
+            # filter's own bug: renaming e.g. requireUserId in app/services/session.server.ts
+            # matched `webapp` and nothing else, so no job ran the suite and the break landed on
+            # main, or on the next unrelated internal-packages PR.
+            #
+            # The cost of the wider set, measured over the last 400 commits on main: 31% touch
+            # routes, 52% touch apps/webapp/app, so the job goes from firing on roughly a third of
+            # PRs to roughly a half. It is the cheap one -- a single 4x runner, no containers, no
+            # database, no prisma generate -- which is what makes that affordable.
+            #
+            # observability-map.yml is here because integration.test.ts asserts on its text and no
+            # other filter watches it, so editing the report workflow alone ran nothing at all.
+            #
+            # Deliberately NOT here: this package's own paths, and packages/plugins/src and
+            # internal-packages/rbac/src, the other two trees webappSymbols.test.ts reads.
+            # `internal` above already matches `internal-packages/**` and `packages/**`, and
+            # `unit-tests-internal.yml` runs `turbo run test --filter "@internal/*"`, which picks up
+            # @internal/observability-map and runs the same vitest suite. Listing them here as well
+            # ran the suite twice on every PR touching them, which was this filter's own doing.
+            obsmap:
+              - 'apps/webapp/app/**'
+              - '.github/workflows/pr_checks.yml'
+              - '.github/workflows/unit-tests-observability-map.yml'
+              - '.github/workflows/observability-map.yml'
+              - 'package.json'
+              - 'pnpm-lock.yaml'
+              - 'pnpm-workspace.yaml'
             cli:
               - 'packages/cli-v3/**'
               - 'packages/build/**'
@@ -149,6 +180,11 @@ jobs:
       DOCKERHUB_USERNAME: ${{ secrets.DOCKERHUB_USERNAME }}
       DOCKERHUB_TOKEN: ${{ secrets.DOCKERHUB_TOKEN }}
 
+  obsmap:
+    needs: changes
+    if: needs.changes.outputs.obsmap == 'true'
+    uses: ./.github/workflows/unit-tests-observability-map.yml
+
   e2e:
     needs: changes
     if: needs.changes.outputs.cli == 'true'
@@ -172,6 +208,7 @@ jobs:
       - e2e-webapp
       - packages
       - internal
+      - obsmap
       - e2e
       - sdk-compat
     if: always()

.github/workflows/pr_checks.yml

@@ -0,0 +1,43 @@
+name: "🧪 Unit Tests: Observability Map"
+
+permissions:
+  contents: read
+
+# Its own workflow rather than a job inside observability-map.yml, because that workflow is not
+# reachable from pr_checks.yml's all-checks aggregate and so gates nothing. Called from there
+# instead, behind a paths filter, which is how every other test suite in this repo is gated.
+on:
+  workflow_call:
+
+jobs:
+  unitTests:
+    name: "🧪 Unit Tests: Observability Map"
+    # No containers and no database: the package is a static analyser over source text, so the
+    # suite is CPU bound on parsing the route tree and needs nothing the runner does not have.
+    runs-on: warp-ubuntu-latest-x64-4x
+    steps:
+      - name: ⬇️ Checkout repo
+        uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
+        with:
+          fetch-depth: 1
+          persist-credentials: false
+
+      - name: ⎔ Setup pnpm
+        uses: pnpm/action-setup@fc06bc1257f339d1d5d8b3a19a8cae5388b55320 # v5.0.0
+        with:
+          version: 10.33.2
+
+      - name: ⎔ Setup node
+        uses: WarpBuilds/setup-node@bc639b444d583175926b588962199c247d23e8d3 # v6
+        with:
+          node-version: 24.18.0
+          cache: "pnpm"
+
+      - name: 📥 Download deps
+        run: pnpm install --frozen-lockfile
+
+      # This suite reads apps/webapp/app (the route tree for the scan, the whole app tree for the
+      # symbol check) and the report workflow's text, which is why the filter that gates this
+      # workflow watches all of those and not only the routes folder.
+      - name: 🧪 Run tests
+        run: pnpm --filter @internal/observability-map run test

.github/workflows/unit-tests-observability-map.yml

@@ -82,3 +82,6 @@ ailogger-output.log
 
 # local planning/design docs, not committed
 **/docs/superpowers/
+
+# observability-map CLI output artifact, not committed
+observability-map.json