Skip to content

⚙️ [Maintenance]: PowerShell changes gain production-like validation - #569

Draft
MariusStorhaug wants to merge 2 commits into
dependabot:mainfrom
MariusStorhaug:powershell-smoke-tests
Draft

⚙️ [Maintenance]: PowerShell changes gain production-like validation#569
MariusStorhaug wants to merge 2 commits into
dependabot:mainfrom
MariusStorhaug:powershell-smoke-tests

Conversation

@MariusStorhaug

@MariusStorhaug MariusStorhaug commented Aug 30, 2026

Copy link
Copy Markdown

PowerShell ecosystem changes can now be validated through Dependabot's official smoke-test architecture before they ship. This contribution establishes a canonical, production-like test target for ordinary module updates, grouped update rules, and repositories that configure more than one directory.

New: Official PowerShell update validation

Maintainers can exercise real PowerShell updater jobs through the same Dependabot CLI recording and cache workflow used by established ecosystems. Once the companion CLI release and Core smoke wiring land, PowerShell changes can run against this coverage without a custom harness.

The contribution makes it possible to detect regressions across the updater behaviors that matter most:

  • Native PowerShell dependency declarations in module manifests, using module statements, and #Requires -Modules directives.
  • Version discovery through Microsoft Artifact Registry, including PowerShell Gallery fallback when a module is not available from MAR.
  • Exact generated files, dependency payloads, pull-request titles, descriptions, commit messages, and group metadata.
  • Minor and patch updates combined into one dependency-group pull request while an eligible major update remains independent.
  • Explicit multi-directory processing that includes every configured directory without treating nested, unconfigured directories as implicitly in scope.

Scope: Updater behavior rather than hosted-service orchestration

These suites validate normalized updater jobs and their recorded API output. Raw dependabot.yml translation, schedules, open-pull-request limits, and hosted-service label application remain outside predeployment smoke-test scope.


Technical details
  • tests/smoke-powershell.yaml covers exact RequiredModules, using module, and #Requires -Modules updates for Az, GitHub, and Pester.
  • tests/smoke-powershell-group-rules.yaml records grouped GitHub/Pester patch and minor updates plus an independent major Az update.
  • tests/smoke-powershell-version-multidir.yaml records one grouped update across the configured root and nested directories while the unconfigured Az fixture remains absent.
  • All jobs use package-manager: powershell, beta-ecosystem enablement, deterministic version caps, and cooldown exclusion for MAR packages without publication timestamps.
  • Fixture commit cc0cbe27e4ee3360d553578a56b1bcda4fba9b79 is pinned through dependabot/smoke-tests. Merge with Create a merge commit because squash or rebase would make that fixture SHA unreachable.
  • Recordings were generated with Dependabot CLI v1.92.0 and the PowerShell updater built from exact Core SHA 1a60ac7cb2223b66224907594b638ab323886be0 (local image manifest sha256:4a9c785c6de9a264015003ae6902d34e88cfbf6794cb83ca37b526ff955c5cdb). Regeneration against that corrected image was byte-identical to commit daa716bb68173b914b18a9568bba0aa3bc2a3301.
  • Cached validation passed with 76/76 calls for smoke-powershell, 76/76 for smoke-powershell-group-rules, and 65/65 for smoke-powershell-version-multidir; yamllint also passes.
  • The updater emits record_ecosystem_meta after each pull-request call. Dependabot CLI intentionally excludes those informational calls from smoke recordings under Ignore record_ecosystem_meta calls in smoke tests cli#587, so the committed expectations match stock CLI output and current smoke-test convention.
  • The three smoke jobs currently stop before updater execution with unknown package manager: powershell. Add PowerShell package manager mapping cli#652 adds the missing mapping, but a CLI release newer than v1.92.0 is required before this repository's @latest workflow can pass. Rerunning before that release would reproduce the same expected failure.
  • Core follow-up wiring uses matrix entry { "core": "powershell", "test": "powershell", "ecosystem": "powershell" }; prefix discovery selects all three smoke-powershell*.yaml suites, and the powershell filter includes the common paths plus powershell/**.
  • Implementation plan progress: fixtures, recordings, exact-image regeneration, cached validation, and Core matrix/filter coordination are complete. Publishing official cache artifacts remains blocked on the CLI release.
  • Issue convergence sweep: the finished diff was checked against the PowerShell Core and CLI bootstrap work; it does not independently close either pull request.
Changed surface Standards checked Framework docs checked Result
powershell/** PowerShell fixture syntax and declaration forms Dependabot smoke fixture conventions Aligned
tests/smoke-powershell*.yaml YAML formatting and generated payload fidelity Dependabot CLI recording and cache workflow Aligned
Relevant issues (or links)

Related work

@MariusStorhaug

Copy link
Copy Markdown
Author

@robaiken, could you provide a focused review of the immutable fixture pin/merge-commit requirement and the three generated PowerShell expectations? The corrected Core image produced byte-identical recordings with full cache coverage; the remaining CI failure is the separately tracked CLI registration bootstrap.

@MariusStorhaug MariusStorhaug changed the title Add PowerShell smoke tests ⚙️ [Maintenance]: PowerShell updates gain official smoke coverage Aug 30, 2026
@MariusStorhaug MariusStorhaug changed the title ⚙️ [Maintenance]: PowerShell updates gain official smoke coverage ⚙️ [Maintenance]: PowerShell changes gain production-like validation Aug 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant