Skip to content

Add gen-sdk vs legacy SDK comparison tooling (shadow-gen diff) #2282

Description

@pose

Context

Part of the gen-sdk migration epic, pulumi/home#4787. Before flipping a provider+language to gen-sdk we must prove its output matches the legacy codegen output.

Scope

A reusable comparison ("shadow-gen diff") script in provider-ci, a make compare_sdk_<lang> target, and a CI job.

Acceptance criteria

Out of scope: cross-provider aggregate parity (% per provider/language). The tool stays per-provider, per-language. Once this job is rolled out everywhere, aggregate parity is collected from the per-repo workflow results by a separate script, tracked in #2287.

Self-contained: independent of the codegen fix (it only reports the diff).

Part of pulumi/home#4787.

Interim path: compare against tfgen's per-language schema

The shipped tool feeds gen-sdk the canonical schema.json, which assumes the end-state fix where gen-sdk resolves the chooser markup itself. That fix is further out (#2240), so the parity here sits at 7-35%, dominated by the leak. The shorter interim path is to have tfgen emit a per-language schema (tfgen <lang>-schema, pulumi/pulumi-terraform-bridge#3487) with the markup already resolved, and render from that. The existing --schema-cmd hook is the seam.

  • compare-sdk runs the per-language schema emitter and feeds it to gen-sdk, reporting parity per provider/language.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions