Skip to content

Aggregate gen-sdk parity across providers from per-repo compare results #2287

Description

@pose

Context

Part of the gen-sdk migration epic, pulumi/home#4787. The per-repo comparison tooling (#2282) compares a single provider+language, both locally (make compare_sdk_<lang>) and in that provider's CI, emitting a structured report plus an exit code. It deliberately does not try to compare every provider in one place.

Scope

A separate aggregation step that, once the per-repo comparison job is rolled out across provider repos, collects each repo's compare_sdk_<lang> results (workflow run conclusions and/or the uploaded report artifacts) and produces an aggregate parity summary: percentage clean per provider and per language, suitable for tracking rollout progress.

Why this is separate from #2282

Making the per-repo tool itself produce a cross-provider summary would require it to obtain every provider's schema.json and legacy codegen binary in one place, i.e. clone and build every provider. That is expensive and just duplicates work each provider's CI already does. Collecting the already-computed per-repo results from GitHub Actions keeps the per-repo tool simple and uses each repo as the source of truth for its own parity.

Acceptance criteria

  • A script (in provider-ci) that enumerates provider repos from providers.json and gathers their latest compare_sdk_<lang> results across languages.
  • Produces an aggregate parity summary (% clean per provider/language) to stdout and a shareable format (e.g. markdown and/or CSV).
  • Depends on the per-repo comparison job from Add gen-sdk vs legacy SDK comparison tooling (shadow-gen diff) #2282 being present in the target repos.

Part of pulumi/home#4787.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions