[relloyd]

Summary

Adds OpenTelemetry metrics and traces for GitHub workflow attempts. Telemetry is scoped to work performed in each attempt: partial reruns omit jobs that succeeded earlier, while jobs actually rerun are counted again. github.workflow.runs counts every collected attempt in workflow_run mode, including partial reruns; current-workflow mode counts only attempts where the telemetry job runs.

Metric points include workflow.run_id and workflow.run_attempt, so independent runs and attempts use separate metric series. The default OTLP exporter uses cumulative temporality. To calculate totals by workflow, sum per-attempt points over the desired time range by workflow.name, dropping run, attempt, outcome, and event dimensions. Keep repository when totals should remain separate across repositories.

Key Changes

1. Workflow and job metrics

• Adds workflow and job counters for runs, execution and queue duration, billable time, raw billable time, and rounding waste.
• Applies GitHub-hosted runner multipliers (Linux 1x, Windows 2x, macOS 10x). Self-hosted jobs report zero GitHub-hosted billing and rounding waste while retaining execution duration and runner metadata.
• Infers macOS larger-runner architecture correctly: -large is amd64 and -xlarge is arm64.

2. Attempt-scoped collection

• Filters job results to the requested run_attempt, excluding jobs retained from earlier attempts while preserving all jobs in the selected attempt, including matrix jobs with duplicate names.
• Uses the attempt's run_started_at for workflow duration, falling back to the earliest selected job creation time.
• Includes workflow.run_id and workflow.run_attempt on metric points so independent cumulative streams do not collide; aggregate by workflow name for totals.

3. Attempt-aware traces

• Each root workflow span represents one attempt and includes only jobs executed in that attempt.
• Job and step spans include attempt context; step spans also include job.id.
• Queue spans remain queryable as job.queue, with wrapper spans retained for a collapsible waterfall without duplicating job cost attributes.

Test Plan

• npm run all — formatting, ESLint, strict TypeScript, 12 Vitest suites / 135 tests, 97.88% statement coverage, and dist/index.js packaging.
• Regression test: two independent cumulative providers with the same workflow name export distinct run-ID series whose github.workflow.runs values sum to 2.
• Regression tests: self-hosted jobs retain duration/runner metadata but report zero hosted billing; macOS -large/-xlarge architecture cases are covered.
• PR CI on the updated head (runs after the follow-up commit is pushed).
• Workflow Run Tests are triggered by a target workflow on main; that integration run remains unverified.

[gemini-code-assist]

Code Review

This pull request migrates OpenTelemetry metrics from Gauges to Counters to support cleaner aggregation, introduces several new workflow and job metrics (such as billable time and rounding waste), and adds structured trace attributes with a span.type discriminator. Feedback on these changes highlights a critical issue with the re-run filtering logic, where filtering single jobs in toWorkflowJob incorrectly excludes successful jobs from previous attempts; deduplicating jobs at the collection level is recommended instead. Additionally, since OpenTelemetry Counters are monotonic, job and workflow durations must be guarded with Math.max(0, ...) to prevent negative values caused by GitHub API timing anomalies.

[Copilot]

🟡 Changes recommended

Cross-invocation aggregation and runner billing and architecture calculations still produce inaccurate telemetry.

3 open issues

• Default OTLP configuration fails to aggregate counter metrics · New
• Classify macOS large runners as x64, not ARM64 · New
• Exclude self-hosted jobs from hosted billing minutes · New

What changed in this PR

This PR expands the action’s OpenTelemetry output with attempt-scoped CI metrics, billable-time estimates, and queryable queue spans.

Changes:

• Replaces gauges with counters for workflow/job counts, durations, and billing totals.
• Adds paginated job collection and attempt filtering to avoid repeating earlier work on partial reruns.
• Enriches trace attributes and updates tests, documentation, output validation, and the packaged action.

File	Description
src/​utils/​calc-billable.ts	Adds runner detection and billable-time calculations.
src/​utils/​calc-billable.test.ts	Tests runner inference, rounding, and multipliers.
src/​traces/​create-trace.test.ts	Verifies attempt timing and layered span attributes.
src/​traces/​create-spans.ts	Adds queue spans, attempt attributes, and workflow rollups.
src/​traces/​create-spans.test.ts	Covers negative queue-duration handling.
src/​metrics/​create-metrics.ts	Switches metric creation to counters.
src/​metrics/​create-metrics.test.ts	Verifies counter values and telemetry attributes.
src/​metrics/​create-gauges.ts	Removes the gauge implementation.
src/​metrics/​create-counters.ts	Implements workflow and job counters.
src/​metrics/​create-counters.test.ts	Tests counter export and duration safeguards.
src/​metrics/​constants.ts	Defines new metrics and attribute keys.
src/​main.test.ts	Updates fixtures with attempt metadata.
src/​github/​types.ts	Preserves runner labels, attempts, events, and step numbers.
src/​github/​github.ts	Adds pagination, attempt filtering, and start-time fallback.
src/​github/​github.test.ts	Tests attempt selection and metadata conversion.
README.md	Documents counters, trace attributes, and rerun behavior.
docs/​specification.md	Updates metric and trace specifications.
dist/​index.js	Packages the updated action implementation.
badges/​coverage.svg	Updates reported coverage to 97.84%.
.github/​workflows/​validate-action-output.yml	Normalizes counter output and new span attributes.

_{Review effort: Balanced}

[relloyd]

/gemini review

[gemini-code-assist]

Code Review

This pull request transitions the telemetry metrics from Gauges to Counters and introduces structured attributes with a span.type discriminator to prevent double-counting in traces. It also implements logic to filter out reused jobs from previous attempts during workflow reruns and adds utilities to calculate billable minutes and rounding waste. Feedback on the changes highlights a logical bug where falling back to the earliest job creation time when run_started_at is missing fails to filter out reused jobs, as well as a potential TypeScript type mismatch if job.labels is null.

[Copilot]

🟡 Changes recommended

The first-attempt fallback cutoff can discard valid completed jobs with anomalous timestamps, losing execution and billing telemetry and potentially causing collection to fail.

1 open issue

• Avoid filtering anomalous first-attempt jobs by created_at · New

3 resolved since last review

• Exclude self-hosted jobs from hosted billing minutes
• Classify macOS large runners as x64, not ARM64
• Default OTLP configuration fails to aggregate counter metrics

_{Review effort: Balanced}