Reporter overhaul (4/6): opt-in reporters, selection, StreamCollator replacement - #5868
Open
Sean Larkin (TheLarkInn) wants to merge 68 commits into
Open
Reporter overhaul (4/6): opt-in reporters, selection, StreamCollator replacement#5868Sean Larkin (TheLarkInn) wants to merge 68 commits into
Sean Larkin (TheLarkInn) wants to merge 68 commits into
Conversation
Document the reviewed reporter architecture, structured event protocol, compatibility model, rollout plan, and supporting codebase research for issue #5858. Assistant-model: GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: ba539363-ff7b-44c9-b4f6-e49ef7fc6c92
Introduce the @rushstack/reporter package skeleton that will hold the canonical event protocol, reporter manager, and built-in reporters described in the Rush reporter overhaul RFC (#5858). - Scaffold libraries/reporter with rig-based Heft build config - Mark exported contracts as public beta via API Extractor (@beta) - Declare no dependency on rush-lib - Register the package in rush.json and update the pnpm lockfile - Track reporter-overhaul feature-list progress Assistant-model: GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 897dcf7e-e6e8-4a84-85ca-34b93fa29be3
Assistant-model: GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 897dcf7e-e6e8-4a84-85ca-34b93fa29be3
Define the presentation-free event contract for @rushstack/reporter as described in the Rush reporter overhaul RFC (#5858). - Add IReporterProtocolVersion, ReporterPrivacyClassification, and the immutable IReporterEventEnvelope with source and scope - Add the closed ReporterEventType union and REPORTER_EVENT_TYPES list - Add ReporterJsonValue to type JSON-serializable payloads - Document sequence as authoritative ordering and timestamp as informational, and preserve child producer order via sourceSequence - Add a golden envelope schema test covering snapshot, JSON round-trip, the closed union, and Error payload handling Assistant-model: GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 897dcf7e-e6e8-4a84-85ca-34b93fa29be3
Assistant-model: GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 897dcf7e-e6e8-4a84-85ca-34b93fa29be3
Introduce the presentation-free producer contract for @rushstack/reporter (#5858). - Add IReporterEventSink.emit, which accepts an envelope input that omits the sink-assigned eventId, sequence, and timestamp - Add IScopedReporter with emitMessage, emitDiagnostic, and emitExtension, none of which expose reporter instances, destinations, or thresholds - Add namespaced extension event names with an isReporterExtensionEventName validator so producer events cannot collide with the closed core set - Add a minimal IRushDiagnostic contract, expanded by the structured diagnostics feature - Cover the sink omit contract, scoped emit ids, emit-only surface, and extension name validation with tests Assistant-model: GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 897dcf7e-e6e8-4a84-85ca-34b93fa29be3
Assistant-model: GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 897dcf7e-e6e8-4a84-85ca-34b93fa29be3
Complete the structured diagnostics contract for @rushstack/reporter (#5858). - Expand IRushDiagnostic with category, remediation, source, classified parameters, cause chaining, and related artifacts - Add RushDiagnosticCategory, IRushRemediationAction, IRushDiagnosticSource, and IClassifiedDiagnosticValue - Add a central, never-reused RUSH_<DOMAIN>_<NAME> code registry with English templates keyed by resource key and a stable internal-error code - Add computeEnvelopePrivacyFloor and getPrivacyClassificationRank so the envelope classification is the least-sensitive floor while field-level classification stays authoritative - Add createRushDiagnostic and RushError so emission returns a diagnostic id that propagated failures reference - Add registry, privacy, factory, and RushError tests Assistant-model: GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 897dcf7e-e6e8-4a84-85ca-34b93fa29be3
Assistant-model: GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 897dcf7e-e6e8-4a84-85ca-34b93fa29be3
Add the cross-process wire protocol for @rushstack/reporter (#5858). - Add REPORTER_PROTOCOL_VERSION, REPORTER_PROTOCOL_LIMITS (1 MiB bootstrap and record, 64 KiB external chunk), and isReporterProtocolCompatible - Add an NDJSON encoder and a streaming decoder that enforce the record limit - Add hello/helloAck negotiation that accepts the capability intersection, tolerates additive minor versions, and rejects only unknown required features or unsupported majors - Emit an update-global-Rush diagnostic (RUSH_PROTOCOL_UPDATE_REQUIRED) on rejection - Cover encode/decode framing, size limits, and negotiation with tests Assistant-model: GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 897dcf7e-e6e8-4a84-85ca-34b93fa29be3
Assistant-model: GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 897dcf7e-e6e8-4a84-85ca-34b93fa29be3
Add the reporter fan-out engine for @rushstack/reporter (#5858). - Add the IReporter interface and IReporterContext - Add ReporterManager, the authoritative in-process sink: it assigns one monotonic session sequence, eventId, and timestamp, rehomes foreign child envelopes while preserving sourceSequence, and delivers events to each reporter through an independent ordered queue - Enforce exclusive destination ownership and add ReporterMultiplexer for explicit sharing - Flush on normal, error, and signal completion with 10s and 2s timeouts - Coalesce replaceable status events under pressure while never dropping lifecycle, diagnostic, result, artifact, or external-output events - Disable optional reporters on runtime failure with an emergency diagnostic and make required-reporter and initialization failures fatal - Cover ordering, destinations, failure handling, coalescing, and flush/close Assistant-model: GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 897dcf7e-e6e8-4a84-85ca-34b93fa29be3
Assistant-model: GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 897dcf7e-e6e8-4a84-85ca-34b93fa29be3
Freeze baselines for @rushstack/reporter (#5858). - Add golden schema tests that freeze the serialized event envelope, diagnostic, and hello/helloAck wire forms and assert NDJSON round-trips - Add compatibility goldens for negotiation across supported majors and minors, including additive-minor tolerance and forward-compatible field retention - Freeze representative legacy success and failure output as the parity baseline for the future legacy reporter - Wire all goldens into the existing jest test runner Assistant-model: GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 897dcf7e-e6e8-4a84-85ca-34b93fa29be3
Assistant-model: GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 897dcf7e-e6e8-4a84-85ca-34b93fa29be3
Sean Larkin (TheLarkInn)
requested review from
Ian Clanton-Thuon (iclanton) and
Pete Gonzalez (octogonz)
as code owners
July 15, 2026 03:13
This was referenced Jul 15, 2026
Assistant-model: GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 897dcf7e-e6e8-4a84-85ca-34b93fa29be3
Assistant-model: GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 897dcf7e-e6e8-4a84-85ca-34b93fa29be3
Add the machine reporters for @rushstack/reporter (#5858). - Add JsonReporter, which emits the complete versioned NDJSON event stream on exclusive stdout and replaces an oversized record with a valid marker - Add AiReporter, a bounded projection that emits a status record and a final record with the result, scope, error codes and categories, structured remediation, aggregate counts, log reference, and artifact completeness - Cap the AI record at 64 KiB and 20 detailed diagnostics, represent warnings by count when failures exist, and exclude raw output and stacks - Keep the absolute log path in AI output while telemetry continues to exclude it - Add stdout-purity tests for both reporters and a telemetry path cross-check Assistant-model: GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 897dcf7e-e6e8-4a84-85ca-34b93fa29be3
Assistant-model: GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 897dcf7e-e6e8-4a84-85ca-34b93fa29be3
Add the full-detail file reporter for @rushstack/reporter (#5858). - Add FileReporter, which writes a debug NDJSON invocation log to <commonTempFolder>/rush-logs/<UTC timestamp>-<pid>-<action>.log with owner-only permissions and redacts fields classified as secret - Maintain a latest.log pointer for both successful and failed commands - Buffer events until the first flush and fall back to the OS temp folder - Delete logs older than 14 days and cap retention at 20 sessions - Treat failure at both paths as nonfatal, emitting an emergency warning and marking the artifact unavailable - Cover writing, permissions, redaction, retention, fallback, and failure with tests against real temp directories Assistant-model: GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 897dcf7e-e6e8-4a84-85ca-34b93fa29be3
Assistant-model: GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 897dcf7e-e6e8-4a84-85ca-34b93fa29be3
Add the legacy reporter for @rushstack/reporter (#5858). - Add LegacyReporter, which reproduces the current Rush output: the start line, the parallelism line, StreamCollator-style operation headers with grouped output, and a success or failure summary with durations - Add isLegacyEmergencyFallbackRequested and confirm RUSH_REPORTER=legacy selects the legacy reporter as an emergency fallback - Validate the reporter against the frozen legacy output snapshots Assistant-model: GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 897dcf7e-e6e8-4a84-85ca-34b93fa29be3
Assistant-model: GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 897dcf7e-e6e8-4a84-85ca-34b93fa29be3
Add the uncollated operation event stream for @rushstack/reporter (#5858). - Add OperationStreamEmitter so the scheduler emits operation registration, status transitions, raw output chunks, and the aggregate command result - Emit output chunks immediately in call order and never collate them, so the concise reporter derives activity without buffering while the detailed and file reporters own grouping - Add iterateExternalOutput and regroupOperationOutput so problem matchers consume the uncollated stream and reporters reconstruct StreamCollator-parity grouping - Cover emission, chunking, uncollated ordering, and reporter parity with tests Assistant-model: GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 897dcf7e-e6e8-4a84-85ca-34b93fa29be3
Assistant-model: GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 897dcf7e-e6e8-4a84-85ca-34b93fa29be3
Recover diagnostics from raw external output for @rushstack/reporter (#5858). - Preserve raw stdout and stderr as ordered chunks and run matchers over an ANSI-normalized copy so the raw evidence and process status are never modified - Add a tool- and version-scoped problem matcher registry gated on default enablement, routing older Heft versions through the version predicate - Add runProblemMatchers, which reassembles lines split across chunks, links recovered diagnostics to the operation and source location, preserves unmatched text, and caps duplicate diagnostics - Cover ANSI stripping, scoping, recovery, split chunks, the duplicate cap, and a representative corpus with tests Assistant-model: GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 897dcf7e-e6e8-4a84-85ca-34b93fa29be3
Assistant-model: GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 897dcf7e-e6e8-4a84-85ca-34b93fa29be3
Bridge the legacy sentinel error pattern for @rushstack/reporter (#5858). - Deprecate AlreadyReportedError so new usage is prohibited now that structured diagnostics and RushError are available - Add LegacyErrorBridge, which observes emitted diagnostics, correlates legacy sentinels with them, and suppresses duplicate rendering of failures that are already represented - Document the bridge removal criteria for a later major - Cover sentinel detection, suppression, correlation, and the removal criteria with tests Assistant-model: GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 897dcf7e-e6e8-4a84-85ca-34b93fa29be3
Assistant-model: GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 897dcf7e-e6e8-4a84-85ca-34b93fa29be3
- OperationStreamEmitter no longer sets 'required' (manager derives it from the event type) - Problem matcher diagnostics use the kind-tagged source union and the registered RUSH_EXTERNAL_TOOL_PROBLEM code with its template Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Sean Larkin (TheLarkInn)
force-pushed
the
reporter/4-opt-in-reporters
branch
from
August 19, 2026 14:08
291cbc2 to
58a8fd2
Compare
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
… copilot/pr/5868/8a1d8d95-fb24-44d8-8700-9fdb3dbdaef3
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
… copilot/pr/5868/8a1d8d95-fb24-44d8-8700-9fdb3dbdaef3
Preserve legacy telemetry hook output without mutating the allowlisted aggregate, keep root session telemetry authoritative, retain inherited lifecycle scope, and make shadow result summaries internally consistent. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Preserve streamed output, enforce reporter privacy and lifecycle contracts, restore legacy compatibility, and correct reporter change metadata. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Resolve the generated reporter API report with the updated telemetry adapter contract. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
Pull request overview
Adds standalone opt-in reporter selection, rendering, logging, semantic operation events, problem matching, and legacy compatibility.
Changes:
- Adds environment-based reporter planning and log-level controls.
- Implements interactive, plaintext, JSON, AI, file, and legacy reporters.
- Adds uncollated operation events, diagnostic recovery, and legacy error bridging.
Reviewed changes
Copilot reviewed 51 out of 51 changed files in this pull request and generated 17 comments.
Show a summary per file
| File | Description |
|---|---|
research/progress.txt |
Records phase progress. |
research/feature-list.json |
Marks phase features complete. |
libraries/reporter/src/test/ReporterSelection.test.ts |
Tests reporter selection. |
libraries/reporter/src/test/ProblemMatchers.test.ts |
Tests problem matching. |
libraries/reporter/src/test/PlaintextReporter.test.ts |
Tests plaintext output. |
libraries/reporter/src/test/OperationStreamEmitter.test.ts |
Tests semantic event emission. |
libraries/reporter/src/test/LogLevelFilter.test.ts |
Tests log filtering. |
libraries/reporter/src/test/LegacyReporter.test.ts |
Tests legacy rendering. |
libraries/reporter/src/test/LegacyErrorBridge.test.ts |
Tests error bridging. |
libraries/reporter/src/test/JsonAiReporter.test.ts |
Tests machine reporters. |
libraries/reporter/src/test/FileReporter.test.ts |
Tests file logging. |
libraries/reporter/src/test/DefaultInteractiveReporter.test.ts |
Tests interactive rendering. |
libraries/reporter/src/test/AutomaticReporterMatrix.test.ts |
Tests automatic planning. |
libraries/reporter/src/test/__snapshots__/PlaintextReporter.test.ts.snap |
Captures plaintext output. |
libraries/reporter/src/test/__snapshots__/LegacyReporter.test.ts.snap |
Captures legacy output. |
libraries/reporter/src/scheduler/OperationStreamEmitter.ts |
Emits uncollated operation events. |
libraries/reporter/src/scheduler/OperationOutputGrouping.ts |
Regroups operation output. |
libraries/reporter/src/reporters/ReporterRedaction.ts |
Redacts secret fields. |
libraries/reporter/src/reporters/PlaintextReporter.ts |
Implements plaintext reporting. |
libraries/reporter/src/reporters/LegacyReporter.ts |
Implements legacy reporting. |
libraries/reporter/src/reporters/JsonReporter.ts |
Implements NDJSON reporting. |
libraries/reporter/src/reporters/InteractiveRendering.ts |
Provides live-region helpers. |
libraries/reporter/src/reporters/FileReporter.ts |
Implements retained file logs. |
libraries/reporter/src/reporters/DefaultInteractiveReporter.ts |
Implements interactive output. |
libraries/reporter/src/reporters/AiReporter.ts |
Implements bounded AI output. |
libraries/reporter/src/matchers/ProblemMatcherRunner.ts |
Recovers diagnostics from output. |
libraries/reporter/src/matchers/ProblemMatcherRegistry.ts |
Selects applicable matchers. |
libraries/reporter/src/matchers/ProblemMatcher.ts |
Defines matcher contracts. |
libraries/reporter/src/matchers/AnsiNormalization.ts |
Normalizes ANSI output. |
libraries/reporter/src/index.ts |
Exports new APIs. |
libraries/reporter/src/diagnostics/templates/operation.ts |
Adds external-tool text. |
libraries/reporter/src/diagnostics/RushDiagnosticCodeRegistry.ts |
Registers external-tool diagnostics. |
libraries/reporter/src/config/ReporterSelection.ts |
Resolves reporter controls. |
libraries/reporter/src/config/ReporterNames.ts |
Defines reporter names and levels. |
libraries/reporter/src/config/OutputControl.ts |
Parses output destinations. |
libraries/reporter/src/config/LogLevelFilter.ts |
Classifies event visibility. |
libraries/reporter/src/config/AutomaticReporterMatrix.ts |
Builds environment-specific plans. |
libraries/reporter/src/config/AgentDetection.ts |
Detects agents and CI. |
libraries/reporter/src/compat/LegacyErrorBridge.ts |
Correlates legacy sentinels. |
common/reviews/api/rush-reporter.api.md |
Updates the API report. |
common/changes/@rushstack/rush-reporter/docs-rush-reporter-overhaul-spec_2026-07-15-01-40-55.json |
Documents error bridging. |
common/changes/@rushstack/rush-reporter/docs-rush-reporter-overhaul-spec_2026-07-15-01-36-33.json |
Documents problem matchers. |
common/changes/@rushstack/rush-reporter/docs-rush-reporter-overhaul-spec_2026-07-15-01-30-02.json |
Documents operation events. |
common/changes/@rushstack/rush-reporter/docs-rush-reporter-overhaul-spec_2026-07-15-01-24-04.json |
Documents legacy reporting. |
common/changes/@rushstack/rush-reporter/docs-rush-reporter-overhaul-spec_2026-07-15-01-18-53.json |
Documents file reporting. |
common/changes/@rushstack/rush-reporter/docs-rush-reporter-overhaul-spec_2026-07-15-01-12-29.json |
Documents machine reporters. |
common/changes/@rushstack/rush-reporter/docs-rush-reporter-overhaul-spec_2026-07-15-01-05-03.json |
Documents plaintext reporting. |
common/changes/@rushstack/rush-reporter/docs-rush-reporter-overhaul-spec_2026-07-15-01-00-00.json |
Documents interactive reporting. |
common/changes/@rushstack/rush-reporter/docs-rush-reporter-overhaul-spec_2026-07-15-00-50-08.json |
Documents reporter planning. |
common/changes/@rushstack/rush-reporter/docs-rush-reporter-overhaul-spec_2026-07-15-00-45-15.json |
Documents log filtering. |
common/changes/@rushstack/rush-reporter/docs-rush-reporter-overhaul-spec_2026-07-15-00-41-00.json |
Documents reporter selection. |
Suppressed comments (1)
libraries/reporter/src/reporters/AiReporter.ts:312
- After all three arrays are emptied, the record can still exceed
maxBytesbecausecommandName, the log path, category/count maps, and other fixed fields remain unbounded. The method then writes the oversized record anyway, violating the documented maximum. Add a final size check and a bounded fallback record or bound the remaining fields.
for (const target of trimTargets) {
while (Buffer.byteLength(JSON.stringify(record), 'utf8') > this._maxBytes && target.get().length > 0) {
target.set(target.get().slice(0, target.get().length - 1));
record.truncated = true;
}
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Complete lifecycle status coverage, root-session parity, telemetry compatibility, extension privacy, and Rush semver-range plugin gating. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Harden reporter filtering, protocol bounds, terminal rendering, file streaming, matcher recovery, and legacy summaries. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Combine plugin version support with the reviewed reporter contracts and expanded operation statuses. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
GitHub continued to report conflicts even though the current base is the direct second parent of the PR head. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
📚 Reporter Overhaul PR stack (merge bottom-up)
mainEach PR's diff is scoped to its phase; review independently, merge from #5865 upward.
👉 This is PR 4 of 6.
Phase 4 — Opt-in reporters (RFC §8.1)
Adds the actual reporters plus selection/config, and replaces StreamCollator only on the new reporter path.
What's included
--reporter/RUSH_REPORTER/agent/CI/TTY), independent per-reporter log levels, and the automatic environment matrix.defaultinteractive (3-row live region),plaintext/non-TTY,json, boundedai, full-detailfile(owner-only perms, retention, OS-temp fallback), andlegacy(selectable + emergency fallback).AlreadyReportedErrorusage; correlate legacy sentinels with emitted diagnostics.Validation
rush build --to @rushstack/reporter✅rush test --only @rushstack/reporter✅ (adds ReporterSelection, LogLevelFilter, AutomaticReporterMatrix, the reporter suites, OperationStreamEmitter, ProblemMatchers, LegacyErrorBridge)@beta.Scope note
Standalone package; not yet wired into the live CLI. Independently releasable/revertible per RFC §8.1.