Skip to content

Reporter overhaul (3/6): shadow structured emission - #5867

Merged
Sean Larkin (TheLarkInn) merged 11 commits into
mainfrom
reporter/3-shadow-emission
Aug 27, 2026
Merged

Reporter overhaul (3/6): shadow structured emission#5867
Sean Larkin (TheLarkInn) merged 11 commits into
mainfrom
reporter/3-shadow-emission

Conversation

@TheLarkInn

@TheLarkInn Sean Larkin (TheLarkInn) commented Jul 15, 2026

Copy link
Copy Markdown
Member

📚 Reporter Overhaul PR stack (merge bottom-up)

# PR Phase Base
1 #5865 Contracts & baselines (package) main
2 #5866 Bootstrap & compatibility #5865
3 #5867 Shadow structured emission #5866
4 #5868 Opt-in reporters #5867
5 #5869 Heft protocol track #5868
6 #5870 Perf budgets & default flip #5869

Each PR's diff is scoped to its phase; review independently, merge from #5865 upward.

👉 This is PR 3 of 6.

Phase 3 — Shadow structured emission (RFC §8.1)

First-party lifecycle and diagnostic events are emitted without changing visible output, and telemetry/exit-code parity is validated.

What's included

  • RushSession event sink + scoped producers exposed to actions and plugins (plugins cannot inspect modes/destinations/thresholds).
  • Shadow lifecycle emission — structured lifecycle + diagnostic events run alongside the legacy renderer with no output change.
  • Telemetry projection subscriber — allowlist + beforeLog adapter.
  • Reporter-independent exit codes — command success/exit semantics preserved regardless of the active reporter.

Validation

  • rush build --to @rushstack/reporterrush test --only @rushstack/reporter ✅ (adds Session, Lifecycle, Telemetry, ExitStatus suites)
  • All exports @beta.

Scope note

Standalone package; not yet wired into the live CLI. Independently releasable/revertible per RFC §8.1.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds phase-3 shadow structured reporting to the standalone Rush reporter package without changing visible CLI output.

Changes:

  • Adds scoped session, plugin, and lifecycle reporting APIs.
  • Adds telemetry projection and legacy hook adaptation.
  • Adds reporter-independent exit-status helpers and validation suites.

Reviewed changes

Copilot reviewed 26 out of 26 changed files in this pull request and generated 7 comments.

Show a summary per file
File Description
research/progress.txt Records phase-3 implementation progress.
research/feature-list.json Marks features 11–14 complete.
libraries/reporter/src/test/Telemetry.test.ts Tests telemetry projection and privacy.
libraries/reporter/src/test/Session.test.ts Tests scoped session and plugin APIs.
libraries/reporter/src/test/Lifecycle.test.ts Tests shadow lifecycle parity.
libraries/reporter/src/test/ExitStatus.test.ts Tests exit-status and JSON controls.
libraries/reporter/src/telemetry/TelemetrySubscriber.ts Aggregates telemetry from events.
libraries/reporter/src/telemetry/TelemetryAggregate.ts Defines the telemetry allowlist.
libraries/reporter/src/telemetry/BeforeLogAdapter.ts Adapts legacy telemetry hooks.
libraries/reporter/src/session/ScopedReporterFactory.ts Creates scoped event producers.
libraries/reporter/src/session/ScopedLogger.ts Adds a presentation-free logger.
libraries/reporter/src/session/RushSessionReporting.ts Exposes session reporting contexts.
libraries/reporter/src/session/PluginApi.ts Adds plugin compatibility contracts.
libraries/reporter/src/lifecycle/ShadowParity.ts Derives shadow result parity.
libraries/reporter/src/lifecycle/LifecycleEvents.ts Defines lifecycle payloads and statuses.
libraries/reporter/src/lifecycle/LifecycleEmitter.ts Emits structured lifecycle events.
libraries/reporter/src/index.ts Exports the new APIs.
libraries/reporter/src/exit/ExitStatus.ts Resolves reporter-independent exit status.
libraries/reporter/src/exit/CommandJson.ts Separates command and reporter JSON flags.
libraries/reporter/src/diagnostics/templates/configuration.ts Adds the plugin incompatibility message.
libraries/reporter/src/diagnostics/RushDiagnosticCodeRegistry.ts Registers the plugin diagnostic.
common/reviews/api/rush-reporter.api.md Updates the public API report.
common/changes/@rushstack/rush-reporter/docs-rush-reporter-overhaul-spec_2026-07-15-00-35-03.json Records exit-status APIs.
common/changes/@rushstack/rush-reporter/docs-rush-reporter-overhaul-spec_2026-07-15-00-30-21.json Records telemetry APIs.
common/changes/@rushstack/rush-reporter/docs-rush-reporter-overhaul-spec_2026-07-15-00-24-15.json Records lifecycle APIs.
common/changes/@rushstack/rush-reporter/docs-rush-reporter-overhaul-spec_2026-07-15-00-18-26.json Records session and plugin APIs.
Suppressed comments (2)

libraries/reporter/src/telemetry/TelemetrySubscriber.ts:104

  • This counts status-transition events rather than operations by final status. A normal operation that transitions readyexecutingsuccess contributes to three buckets, whereas the frozen legacy telemetry counts each operation once from its final operationResult.status (PhasedScriptAction.ts:1100-1145). Track the latest status by operationId and derive the counts from that map when building the aggregate.
      case 'operationStatusChanged': {
        const status: string = (event.payload as { status: string }).status;
        this._operationStatusCounts[status] = (this._operationStatusCounts[status] ?? 0) + 1;
        break;

libraries/reporter/src/lifecycle/ShadowParity.ts:93

  • Like the telemetry subscriber, this summary counts every status transition rather than each operation's final status. That inflates parity counts for any operation that emits normal intermediate states and cannot match the legacy final-result summary. Retain the latest status per operationId, then count the final map values.
  for (const event of events) {
    if (event.type === 'operationStatusChanged') {
      const payload: IOperationStatusChangedPayload = event.payload as IOperationStatusChangedPayload;
      operationCounts[payload.status] = (operationCounts[payload.status] ?? 0) + 1;

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread libraries/reporter/src/lifecycle/LifecycleEvents.ts
Comment thread libraries/reporter/src/session/PluginApi.ts
Comment thread libraries/reporter/src/session/ScopedReporterFactory.ts
Comment thread libraries/reporter/src/telemetry/TelemetrySubscriber.ts
Comment thread libraries/reporter/src/telemetry/BeforeLogAdapter.ts Outdated
Comment thread libraries/reporter/src/exit/ExitStatus.ts
Comment thread libraries/reporter/src/lifecycle/ShadowParity.ts Outdated
Base automatically changed from reporter/2-bootstrap-and-compat to main August 26, 2026 15:37
Comment thread research/progress.txt
Comment thread research/feature-list.json
Comment thread libraries/reporter/src/telemetry/TelemetrySubscriber.ts

@mojaza Mo Jazayeri (mojaza) left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall, LGTM with a minor comment.

Expose the producer-facing reporting surface for @rushstack/reporter (#5858).

- Add createScopedReporter and RushSessionReporting so RushSession can create
  scoped reporters and loggers bound to a command, operation, project, and phase
- Add IScopedLogger, a presentation-free logger with no terminal handle
- Pass the sink to actions through IReporterExecutionContext while exposing only
  emit methods, so plugins cannot inspect modes, destinations, or thresholds
- Add the Rush plugin API version, manifest field, a support check, and a
  structured migration diagnostic for incompatible plugins
- Cover scoped emission, the privacy floor, the emit-only surface, logging, the
  session facade, and plugin compatibility with tests

Assistant-model: GitHub Copilot
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 897dcf7e-e6e8-4a84-85ca-34b93fa29be3
Assistant-model: GitHub Copilot
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 897dcf7e-e6e8-4a84-85ca-34b93fa29be3
Add first-party lifecycle emission for @rushstack/reporter (#5858).

- Add typed lifecycle payloads and a LifecycleEmitter that actions, the
  scheduler, and plugins use to publish session, command, and operation events
  plus structured diagnostics
- Mark every lifecycle, result, and diagnostic event required, and keep the
  emitter output-neutral so legacy rendering stays the sole visible output
- Add deriveExitCodeFromEvents and summarizeShadowResult to validate exit-code
  and result parity against the frozen baselines
- Cover emission, scope merging, output neutrality, and parity with tests

Assistant-model: GitHub Copilot
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 897dcf7e-e6e8-4a84-85ca-34b93fa29be3
Assistant-model: GitHub Copilot
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 897dcf7e-e6e8-4a84-85ca-34b93fa29be3
Add the telemetry subscriber for @rushstack/reporter (#5858).

- Add TelemetrySubscriber, which consumes canonical events before reporter
  filtering and produces an allowlisted aggregate at command completion
- Keep only allowlisted values: command and result, timing, operation status
  counts, diagnostic codes and categories, reporter mode, and protocol and
  producer versions
- Exclude messages, paths, raw output, command arguments, remediation
  parameters, stacks, and local-sensitive and secret values by construction
- Add createTelemetryReporter to wire the subscriber into the manager
- Preserve the legacy beforeLog hook through createBeforeLogAdapter
- Add allowlist schema and leakage tests

Assistant-model: GitHub Copilot
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 897dcf7e-e6e8-4a84-85ca-34b93fa29be3
Assistant-model: GitHub Copilot
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 897dcf7e-e6e8-4a84-85ca-34b93fa29be3
Preserve command success and exit-code semantics for @rushstack/reporter (#5858).

- Add resolveExitStatus and resolveExitStatusFromEvents, which return 0 for
  success including warning-only success, 1 for failure and logical
  cancellation, and the conventional signal-derived status for OS signals
- Take only failure, cancellation, and signal state as inputs, so the reporter
  mode and diagnostic categories can never select the exit code
- Add separateJsonControls so the command-specific --json flag stays distinct
  from the json reporter and keeps its own schema
- Cover success, failure, cancellation, signal precedence, category
  independence, and JSON-control separation with tests

Assistant-model: GitHub Copilot
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 897dcf7e-e6e8-4a84-85ca-34b93fa29be3
Assistant-model: GitHub Copilot
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 897dcf7e-e6e8-4a84-85ca-34b93fa29be3
- Scoped reporters emit human messages on the messageEmitted channel with a
  fail-safe local-sensitive privacy default (was: activityChanged with a
  producer-set required flag, which was coalescible under pressure)
- Producers no longer set 'required'; the manager derives it from event type
- Register RUSH_PLUGIN_API_INCOMPATIBLE in the diagnostic registry with its
  template

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@TheLarkInn
Sean Larkin (TheLarkInn) merged commit 410fb62 into main Aug 27, 2026
10 checks passed
@github-project-automation github-project-automation Bot moved this from Needs triage to Closed in Bug Triage Aug 27, 2026
@TheLarkInn
Sean Larkin (TheLarkInn) deleted the reporter/3-shadow-emission branch August 27, 2026 02:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Closed

Development

Successfully merging this pull request may close these issues.

4 participants