Reporter overhaul (3/6): shadow structured emission - #5867
Merged
Conversation
This was referenced Jul 15, 2026
Sean Larkin (TheLarkInn)
force-pushed
the
reporter/3-shadow-emission
branch
from
July 15, 2026 04:53
73b545b to
9a7365d
Compare
Sean Larkin (TheLarkInn)
force-pushed
the
reporter/3-shadow-emission
branch
from
August 19, 2026 13:56
9a7365d to
044bcfe
Compare
Contributor
There was a problem hiding this comment.
Pull request overview
Adds phase-3 shadow structured reporting to the standalone Rush reporter package without changing visible CLI output.
Changes:
- Adds scoped session, plugin, and lifecycle reporting APIs.
- Adds telemetry projection and legacy hook adaptation.
- Adds reporter-independent exit-status helpers and validation suites.
Reviewed changes
Copilot reviewed 26 out of 26 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
research/progress.txt |
Records phase-3 implementation progress. |
research/feature-list.json |
Marks features 11–14 complete. |
libraries/reporter/src/test/Telemetry.test.ts |
Tests telemetry projection and privacy. |
libraries/reporter/src/test/Session.test.ts |
Tests scoped session and plugin APIs. |
libraries/reporter/src/test/Lifecycle.test.ts |
Tests shadow lifecycle parity. |
libraries/reporter/src/test/ExitStatus.test.ts |
Tests exit-status and JSON controls. |
libraries/reporter/src/telemetry/TelemetrySubscriber.ts |
Aggregates telemetry from events. |
libraries/reporter/src/telemetry/TelemetryAggregate.ts |
Defines the telemetry allowlist. |
libraries/reporter/src/telemetry/BeforeLogAdapter.ts |
Adapts legacy telemetry hooks. |
libraries/reporter/src/session/ScopedReporterFactory.ts |
Creates scoped event producers. |
libraries/reporter/src/session/ScopedLogger.ts |
Adds a presentation-free logger. |
libraries/reporter/src/session/RushSessionReporting.ts |
Exposes session reporting contexts. |
libraries/reporter/src/session/PluginApi.ts |
Adds plugin compatibility contracts. |
libraries/reporter/src/lifecycle/ShadowParity.ts |
Derives shadow result parity. |
libraries/reporter/src/lifecycle/LifecycleEvents.ts |
Defines lifecycle payloads and statuses. |
libraries/reporter/src/lifecycle/LifecycleEmitter.ts |
Emits structured lifecycle events. |
libraries/reporter/src/index.ts |
Exports the new APIs. |
libraries/reporter/src/exit/ExitStatus.ts |
Resolves reporter-independent exit status. |
libraries/reporter/src/exit/CommandJson.ts |
Separates command and reporter JSON flags. |
libraries/reporter/src/diagnostics/templates/configuration.ts |
Adds the plugin incompatibility message. |
libraries/reporter/src/diagnostics/RushDiagnosticCodeRegistry.ts |
Registers the plugin diagnostic. |
common/reviews/api/rush-reporter.api.md |
Updates the public API report. |
common/changes/@rushstack/rush-reporter/docs-rush-reporter-overhaul-spec_2026-07-15-00-35-03.json |
Records exit-status APIs. |
common/changes/@rushstack/rush-reporter/docs-rush-reporter-overhaul-spec_2026-07-15-00-30-21.json |
Records telemetry APIs. |
common/changes/@rushstack/rush-reporter/docs-rush-reporter-overhaul-spec_2026-07-15-00-24-15.json |
Records lifecycle APIs. |
common/changes/@rushstack/rush-reporter/docs-rush-reporter-overhaul-spec_2026-07-15-00-18-26.json |
Records session and plugin APIs. |
Suppressed comments (2)
libraries/reporter/src/telemetry/TelemetrySubscriber.ts:104
- This counts status-transition events rather than operations by final status. A normal operation that transitions
ready→executing→successcontributes to three buckets, whereas the frozen legacy telemetry counts each operation once from its finaloperationResult.status(PhasedScriptAction.ts:1100-1145). Track the latest status byoperationIdand derive the counts from that map when building the aggregate.
case 'operationStatusChanged': {
const status: string = (event.payload as { status: string }).status;
this._operationStatusCounts[status] = (this._operationStatusCounts[status] ?? 0) + 1;
break;
libraries/reporter/src/lifecycle/ShadowParity.ts:93
- Like the telemetry subscriber, this summary counts every status transition rather than each operation's final status. That inflates parity counts for any operation that emits normal intermediate states and cannot match the legacy final-result summary. Retain the latest status per
operationId, then count the final map values.
for (const event of events) {
if (event.type === 'operationStatusChanged') {
const payload: IOperationStatusChangedPayload = event.payload as IOperationStatusChangedPayload;
operationCounts[payload.status] = (operationCounts[payload.status] ?? 0) + 1;
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Sean Larkin (TheLarkInn)
force-pushed
the
reporter/3-shadow-emission
branch
from
August 26, 2026 15:37
4c1b56d to
5d903ee
Compare
Mo Jazayeri (mojaza)
approved these changes
Aug 26, 2026
Mo Jazayeri (mojaza)
left a comment
Contributor
There was a problem hiding this comment.
Overall, LGTM with a minor comment.
Expose the producer-facing reporting surface for @rushstack/reporter (#5858). - Add createScopedReporter and RushSessionReporting so RushSession can create scoped reporters and loggers bound to a command, operation, project, and phase - Add IScopedLogger, a presentation-free logger with no terminal handle - Pass the sink to actions through IReporterExecutionContext while exposing only emit methods, so plugins cannot inspect modes, destinations, or thresholds - Add the Rush plugin API version, manifest field, a support check, and a structured migration diagnostic for incompatible plugins - Cover scoped emission, the privacy floor, the emit-only surface, logging, the session facade, and plugin compatibility with tests Assistant-model: GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 897dcf7e-e6e8-4a84-85ca-34b93fa29be3
Assistant-model: GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 897dcf7e-e6e8-4a84-85ca-34b93fa29be3
Add first-party lifecycle emission for @rushstack/reporter (#5858). - Add typed lifecycle payloads and a LifecycleEmitter that actions, the scheduler, and plugins use to publish session, command, and operation events plus structured diagnostics - Mark every lifecycle, result, and diagnostic event required, and keep the emitter output-neutral so legacy rendering stays the sole visible output - Add deriveExitCodeFromEvents and summarizeShadowResult to validate exit-code and result parity against the frozen baselines - Cover emission, scope merging, output neutrality, and parity with tests Assistant-model: GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 897dcf7e-e6e8-4a84-85ca-34b93fa29be3
Assistant-model: GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 897dcf7e-e6e8-4a84-85ca-34b93fa29be3
Add the telemetry subscriber for @rushstack/reporter (#5858). - Add TelemetrySubscriber, which consumes canonical events before reporter filtering and produces an allowlisted aggregate at command completion - Keep only allowlisted values: command and result, timing, operation status counts, diagnostic codes and categories, reporter mode, and protocol and producer versions - Exclude messages, paths, raw output, command arguments, remediation parameters, stacks, and local-sensitive and secret values by construction - Add createTelemetryReporter to wire the subscriber into the manager - Preserve the legacy beforeLog hook through createBeforeLogAdapter - Add allowlist schema and leakage tests Assistant-model: GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 897dcf7e-e6e8-4a84-85ca-34b93fa29be3
Assistant-model: GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 897dcf7e-e6e8-4a84-85ca-34b93fa29be3
Preserve command success and exit-code semantics for @rushstack/reporter (#5858). - Add resolveExitStatus and resolveExitStatusFromEvents, which return 0 for success including warning-only success, 1 for failure and logical cancellation, and the conventional signal-derived status for OS signals - Take only failure, cancellation, and signal state as inputs, so the reporter mode and diagnostic categories can never select the exit code - Add separateJsonControls so the command-specific --json flag stays distinct from the json reporter and keeps its own schema - Cover success, failure, cancellation, signal precedence, category independence, and JSON-control separation with tests Assistant-model: GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 897dcf7e-e6e8-4a84-85ca-34b93fa29be3
Assistant-model: GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 897dcf7e-e6e8-4a84-85ca-34b93fa29be3
- Scoped reporters emit human messages on the messageEmitted channel with a fail-safe local-sensitive privacy default (was: activityChanged with a producer-set required flag, which was coalescible under pressure) - Producers no longer set 'required'; the manager derives it from event type - Register RUSH_PLUGIN_API_INCOMPATIBLE in the diagnostic registry with its template Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Sean Larkin (TheLarkInn)
force-pushed
the
reporter/3-shadow-emission
branch
from
August 27, 2026 00:26
4b791cb to
48391eb
Compare
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
📚 Reporter Overhaul PR stack (merge bottom-up)
mainEach PR's diff is scoped to its phase; review independently, merge from #5865 upward.
👉 This is PR 3 of 6.
Phase 3 — Shadow structured emission (RFC §8.1)
First-party lifecycle and diagnostic events are emitted without changing visible output, and telemetry/exit-code parity is validated.
What's included
RushSessionevent sink + scoped producers exposed to actions and plugins (plugins cannot inspect modes/destinations/thresholds).beforeLogadapter.Validation
rush build --to @rushstack/reporter✅rush test --only @rushstack/reporter✅ (adds Session, Lifecycle, Telemetry, ExitStatus suites)@beta.Scope note
Standalone package; not yet wired into the live CLI. Independently releasable/revertible per RFC §8.1.