Skip to content

add SHA-based artifact-freshness check (#527)#532

Draft
PolyphonyRequiem wants to merge 1 commit into
mainfrom
squad/527-schema-freshness-test
Draft

add SHA-based artifact-freshness check (#527)#532
PolyphonyRequiem wants to merge 1 commit into
mainfrom
squad/527-schema-freshness-test

Conversation

@PolyphonyRequiem

Copy link
Copy Markdown
Owner

Summary

Adds VerbOutputSchemas_ArtifactIsFresh to VerbCatalogSanityTests — a SHA-based CI gate that fails when artifacts/verb-output-schemas.json drifts from what the current verb annotations would generate.

⚠️ RED UNTIL #527 Mozart-half merges

This test will fail until Mozart's [VerbResult] backfill PR (squad/527-verbresult-attrs) lands and regenerates artifacts/verb-output-schemas.json. That is expected and documented in the test's XML doc comment. Do not merge this PR to main until Mozart's half is also merged.

What this does

  • In-process comparison: reads VerbOutputSchemaCatalog.Json (the compile-time constant produced by the VerbSchemaGenerator Roslyn source generator) and compares it against the on-disk artifacts/verb-output-schemas.json.
  • Canonicalization before hashing: both sides are parsed and re-serialized with sorted object keys and consistent indentation before SHA-256 hashing. This prevents whitespace-only reformats from triggering false positives.
  • Actionable failure message: reports both SHA values and tells the developer exactly what to run:
    Run: dotnet build src/Polyphony.SchemaExporter -c Release
    

Why SHA-based and not byte-for-byte?

The existing Catalog_ArtifactFile_ExistsAtRepoRoot_AndMatchesEmbeddedJson test already does a byte-for-byte comparison. This new test adds a normalized (canonical JSON) comparison layer on top, so that:

  1. Formatting-only diffs don't create confusing "stale artifact" failures.
  2. The failure message is unambiguous — SHA mismatch means semantic drift, not a trailing-newline difference.

Files changed

  • tests/Polyphony.Tests/Annotations/VerbCatalogSanityTests.cs — new test + CanonicalizeJson/SortNode/Sha256Hex helpers

Part of #527 (Brahms half)

Mozart's half: squad/527-verbresult-attrs (backfill [VerbResult] on ~50 command methods and regenerate the artifact).

Fails CI when artifacts/verb-output-schemas.json drifts from
what the current verb annotations would generate. Closes a
silent-staleness gap flagged in the 2026-05-28 squad fan-out.

Note: this test will be RED until Mozart's [VerbResult] backfill
PR (squad/527-verbresult-attrs) lands and regenerates the artifact.
That is expected.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@PolyphonyRequiem PolyphonyRequiem added the squad:brahms Owner: Brahms (Testability) label May 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

squad:brahms Owner: Brahms (Testability)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant