Add the integration test suite - #13
Closed
BiosSystem wants to merge 55 commits into
Closed
Conversation
…ge and ad suppression
…able and auto-update check
Bumps the all group with 2 updates: [actions/checkout](https://github.com/actions/checkout) and [softprops/action-gh-release](https://github.com/softprops/action-gh-release). Updates `actions/checkout` from 4 to 7 - [Release notes](https://github.com/actions/checkout/releases) - [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md) - [Commits](actions/checkout@v4...v7) Updates `softprops/action-gh-release` from 2 to 3 - [Release notes](https://github.com/softprops/action-gh-release/releases) - [Changelog](https://github.com/softprops/action-gh-release/blob/master/CHANGELOG.md) - [Commits](softprops/action-gh-release@v2...v3) --- updated-dependencies: - dependency-name: actions/checkout dependency-version: '7' dependency-type: direct:production update-type: version-update:semver-major dependency-group: all - dependency-name: softprops/action-gh-release dependency-version: '3' dependency-type: direct:production update-type: version-update:semver-major dependency-group: all ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps the all group with 1 update: [actions/checkout](https://github.com/actions/checkout). Updates `actions/checkout` from 4 to 7 - [Release notes](https://github.com/actions/checkout/releases) - [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md) - [Commits](actions/checkout@v4...v7) --- updated-dependencies: - dependency-name: actions/checkout dependency-version: '7' dependency-type: direct:production update-type: version-update:semver-major dependency-group: all ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* Update WinSwift logic and docs for Windows 11 24H2 telemetry and Copilot purge * Update standalone executable for 3.2.0
…elease documentation
…ecture specifications
…t architectural state
Release/ holds generated packaging artifacts that are recreated by build.ps1 and published to GitHub Releases, so it does not belong in version control. The existing *.zip rule caught only the archive and left the expanded payload and checksum file showing as untracked noise.
Ports the ForceRemoveEdge portion of upstream Raphire/Win11Debloat
c8a4069 ("Improve error reporting & handling", #741). The commit cannot
be cherry-picked because WinSwift renamed every file it touches, so this
was applied by hand to the one file that had not also diverged in
content.
- Guard the routine behind WhatIf. ForceRemoveEdge is exposed as a
FeatureId and could previously run destructively under dry-run when
invoked directly rather than through the app removal path.
- Wrap the routine in try/catch/finally and dispose the three registry
keys it opens.
- Create the Edge stub with -Force -ErrorAction Stop so a partial stub
is not silently skipped.
- Capture the uninstaller exit code and report a nonzero result instead
of continuing into cleanup as though removal succeeded.
- Report per-path failures when removing leftover shortcuts rather than
discarding them with -ErrorAction SilentlyContinue.
- Replace four blind "reg delete ... *>$null" calls with
Remove-EdgeAutostartValue, which distinguishes an already-absent value
from a failure to inspect or remove an existing one.
- Return a boolean result. Both Request-EdgeForceRemove call sites
discard it with "$null =" so the caller's pipeline output is unchanged.
Rebuilds the standalone bundle, which inlines Scripts/.
Records the 2026-09-05 review of the five upstream commits landed since the 383f4f3 baseline, with a disposition for each. Documents why c8a4069 cannot be cherry-picked, including the full upstream-to-WinSwift path mapping, so the finding is not rediscovered on the next reconciliation pass. Lists the files whose ports remain deferred because WinSwift content has diverged from upstream.
The AllUsers branch of Remove-AppxApp ran the 24H2 Copilot/DevHome/MSTeams DISM fallback twice. The first copy checks the exit code and catches errors; the second was a bare repeat of the same call with its output discarded, so a package that resisted the first attempt was simply retried with no reporting, and every affected removal paid for a second DISM invocation. Keeps the guarded copy and drops the bare one. Also realigns the Get-AppxProvisionedPackage line, which sat 24 columns too deep, and normalizes this block to CRLF so it matches the rest of the file. Rebuilds the standalone bundle, which inlines Scripts/.
The DISM fallback passed the caller's wildcard glob straight through:
DISM /Online /Remove-ProvisionedAppxPackage /PackageName:*Copilot*
/PackageName requires an exact provisioned package name and rejects
wildcards, so every invocation failed with exit code 87
(ERROR_INVALID_PARAMETER). The check tolerated 87 explicitly, which
masked the failure: the fallback reported success while removing
nothing. Copilot, Dev Home, and the new Teams were left provisioned and
free to return on the next servicing pass, which is exactly what this
path exists to prevent.
Resolve the real package names with the same Get-AppxProvisionedPackage
query already used a line above, and run DISM once per match.
The query runs after the Remove-ProvisionedAppxPackage attempt, so it
returns only packages that survived it. When the cmdlet succeeds, which
is the common case, DISM is not invoked at all.
Exit code handling:
- Drop the 87 tolerance. With a real package name, 87 is a genuine
failure and should be reported, not swallowed.
- Accept 3010 alongside 0. DISM returns it for success pending reboot.
- Reset $global:LASTEXITCODE before each call, matching the convention
in Import-RegistryFile and Set-StoreSearchSuggestions, so a stale
value from an earlier native command cannot be misread as this one's
result.
Rebuilds the standalone bundle, which inlines Scripts/.
The v3.1.0/v3.2.0 modernization plan was the last roadmap on file and v3.3.0 shipped without a successor. This adds one and points the old plan at it. v3.4.0 is scoped as recoverability and proof. The last three releases each added half a safety story: v3.2.0 added registry backup, v3.3.0 added desired-state verification, but nothing consumes the backup automatically and ten features cannot be verified at all. Every track in this plan extends infrastructure that already ships rather than adding a new subsystem. Tracks: - Automatic rollback on failed apply. The backup is already taken in Phase 1 before the apply phase, and Restore-RegistryBackupState is a working entry point, but InvokeChanges.ps1 has no reference to any restore function. The plan covers the run-scoped backup handle, the failure policy, and where the call belongs. - Verification coverage from 102/112 to 112/112, following the Appx-absence pattern DisableWidgets already uses in Test-FeatureApplied. - An integration suite for Tests/Integration, which has been empty since July, targeting apply, undo, and both rollback branches. - Field validation of the 24H2 DISM fallback, which was diagnosed from DISM's documented contract rather than observed failing. Records the sequencing constraint that tracks 2 and 3 land before track 1, the InvokeChanges.ps1 collision with the deferred upstream port backlog, and four open decisions that need answers before implementation.
Closes the Track 2 gap in the v3.4.0 plan. All 112 features in
Features.json now declare how they are verified, and the routing is
driven by that metadata rather than by a hardcoded list.
The gap was smaller than the plan claimed. Three of the ten features
counted as unverifiable were already verified at runtime by a hardcoded
FeatureId list inside Test-WinSwiftFeature:
if ($FeatureId -in @('RemoveApps','RemoveGamingApps','RemoveHPApps'))
The metadata was incomplete, not the coverage. That is its own problem:
routing by hardcoded list means Features.json does not describe how a
feature is verified, and the unit suite cannot tell a declared gap from
an undeclared one. Real new verification work was five features.
Changes:
- Replace the hardcoded FeatureId branch with dispatch on the declared
VerificationAdapter, making Features.json the single source of truth.
- Add the AppxAbsence adapter for the three app-removal features,
checking installed and provisioned state. Provisioned state matters
because a package can be uninstalled per-user while still provisioned
and due to return on the next servicing pass.
- Add the EdgeRemoved adapter for ForceRemoveEdge, checking the Edge
uninstall key in the 32-bit registry view plus the four autostart
values Remove-EdgeAutostartValue clears. Those are exactly the
artifacts ForceRemoveEdge manipulates.
- Add the StartLayout adapter for the four start menu features,
SHA-256 comparing the on-disk start2.bin against the expected
template. The all-users variants check every user profile plus the
default profile, since new users inherit from it.
- Add the NotApplicable status for entries with no persistent desired
state, reported distinctly and feeding neither the failure nor the
error count so it cannot affect exit code 2. Apps is a value-carrying
CLI parameter and CreateRestorePoint is a one-shot action; neither has
a state to read back.
Tests:
- Assert every feature declares a RegistryKey or a VerificationAdapter,
so a new feature cannot be added without a verification story.
- Pin the exemption list to exactly Apps and CreateRestorePoint, so
NotApplicable cannot become a dumping ground.
- Cover the new adapters, the NotApplicable counting rule, the
no-adapter Unsupported path, and unknown-adapter error handling.
Corrects section 4 of the v3.4.0 plan, which listed the three
already-covered features as gaps, and rebuilds the standalone bundle.
Ensure-Admin.ps1 is dot-sourced, and `exit` inside a dot-sourced script
does not terminate the caller. Every exit in that guard was therefore
inert: WinSwift printed "must be run as Administrator" and then carried
on into the apply pipeline.
Reproduced non-elevated with stdin redirected, which is what any
scripted or CI invocation looks like. Read-Host returns an empty string
immediately, the prompt reads as declined, `exit 1` does nothing, and
the run proceeds:
WinSwift must be run as Administrator.
...
[WhatIf] Create registry backup
[WhatIf] Apply 15 registry changes from 'Disable_Telemetry.reg'
[WhatIf] Disable Scheduled Task: ...Microsoft Compatibility Appraiser
exit code 0
Under -DryRun nothing is written. Without it, an unprivileged process
would attempt real registry imports and scheduled task changes, fail
partway through on access denied, and leave a half-applied system.
The same defect breaks the relaunch path: after Start-Process -Verb
RunAs succeeds, `exit 0` does not stop the parent, so the elevated child
and the original non-elevated process run WinSwift concurrently.
Confirmed the mechanism with a minimal repro. A dot-sourced script
invoked with arguments cannot terminate its caller through `exit`, with
or without [CmdletBinding()]; `throw` and caller-side handling both do.
Fix: the guard reports its outcome through $script:ElevationOutcome,
which propagates to the caller because dot-sourcing shares scope, and
WinSwift.ps1 exits on anything other than 'Elevated' before any runtime
module loads. 'Relaunched' exits 0 because the elevated child owns the
run; 'Denied' and 'Failed' exit 1.
Also stop prompting when no console can answer. A redirected read
returned instantly and was indistinguishable from a declined prompt, so
that case now reports why it cannot continue.
Verified after the fix: both the read-only -Verify path and the -DryRun
apply path exit 1 without reaching the pipeline.
Adds source assertions for the outcome contract and its position ahead
of module loading, plus a behavioral test that runs the entry script
unelevated and asserts the pipeline is never reached. That test skips
when already elevated, which is the case on CI runners.
Track 3 of the v3.4.0 plan. Unit tests exercise functions in isolation; these run WinSwift.ps1 as a real process, which is the only way to cover startup guards, parameter binding, config loading, and exit codes together. Tests are gated by what they can do to the host: - ReadOnly runs the -Verify path, which reads state and exits before applying anything, so it cannot write even if the engine regresses. Safe anywhere, and the default. - DryRun asserts that -DryRun writes nothing. A regression in the dry-run guard would write to the host, so it needs an ephemeral machine. - Mutating deliberately applies changes and is Windows Sandbox only. Nothing beyond ReadOnly runs unless asked for. Coverage: - Test-VerifyContract.ps1 pins the -Verify exit codes and profile parsing. Every case is deterministic on any machine, since whether a real tweak is applied depends on host configuration: an exempt-only profile, an unknown feature, an empty profile, a missing path, and an app id that cannot exist. - Test-DryRunSafety.ps1 proves -DryRun reaches the apply pipeline and still changes nothing, asserting against the exact values the feature would write rather than sweeping the registry. - Test-ApplyRoundTrip.ps1 applies a registry-backed feature and checks both the verification verdict and the individual values underneath it, so a partially applied .reg file cannot pass as compliant. - Test-RollbackContract.ps1 is the executable specification for Track 1. Rollback does not exist yet, so the block skips itself until InvokeChanges.ps1 references Restore-RegistryBackupState, and activates on its own when Track 1 lands. It pins the failure policy, including that an app removal failure must not trigger rollback. Two findings recorded in the README rather than papered over: - Undo cannot be tested. WinSwift.ps1 initialises $script:UndoParams empty and only Show-MainWindow.ps1 ever populates it, so no command line can select a feature for undo. Invoke-UndoFeatures and the RegistryUndoKey metadata all exist and all 112 features declare undo text, but the CLI cannot reach any of it. The test is present and skipped with the reason inline. The larger problem it stands in for is that unattended deployments cannot revert either. - CI runs more than section 5.2 of the plan expected. The windows-2025 runner is build 26100 and elevated, so it clears both the version gate and the administrator guard, and both ReadOnly and DryRun run there. Sandbox is still required for Mutating. Adds a Sandbox configuration and bootstrap for the mutating tests, which maps the repository read-only so a test cannot modify the working tree it was launched from, and wires the two CI-runnable tags into the workflow.
The first CI run of the suite reported success while running nothing:
Pester v5.7.1
##[error]No test files were found and no scriptblocks were provided.
Integration: passed, failed, skipped
Two defects, both in this suite.
Pester only discovers files matching *.Tests.ps1 when handed a
directory. The files followed the Test-*.ps1 convention used in
Tests/Unit, which works there only because CI invokes each of those by
explicit path. Renamed to *.Tests.ps1.
Worse, the failure passed. Invoke-Pester threw, the empty result gave
$result.FailedCount the value $null, and `$null -gt 0` is false, so the
step went green having executed no tests. A suite that discovers nothing
has to fail rather than report success.
The runner now catches discovery errors, treats a null or zero-total
result as a failure, and prints the total alongside the pass and fail
counts so an empty run is visible rather than inferred. The CI step
checks the same condition instead of trusting FailedCount alone.
BiosSystem
force-pushed
the
feat/integration-suite
branch
from
September 5, 2026 18:50
9911978 to
75055be
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Track 3 of the v3.4.0 plan. Unit tests exercise functions in isolation; these run WinSwift.ps1 as a real process, which is the only way to cover startup guards, parameter binding, config loading and exit codes together.
That difference already paid for itself: this work found the admin guard bypass fixed in #12. Three releases of unit tests and static validation never caught it, because none of them ran the script.
Risk tags
Every Describe carries one tag, chosen by what it can do to the machine:
ReadOnly-Verify, which reads state and exits before applying anythingDryRun-DryRunwrites nothing. A regression would write hereMutatingNothing beyond
ReadOnlyruns unless asked for.Coverage
VerifyContract.Tests.ps1pins the-Verifyexit codes and profile parsing. Every case is deterministic on any machine, which took some care: whether a real tweak is applied depends on host configuration, so none of these assert on one. The machine-independent cases are an exempt-only profile (provingNotApplicablenever fails a run, end to end), an unknown feature, an empty profile, a missing path, and an app id that cannot exist.DryRunSafety.Tests.ps1proves-DryRunreaches the apply pipeline and still changes nothing, asserting against the exact values the feature would write, read from its .reg file.ApplyRoundTrip.Tests.ps1applies a registry-backed feature and checks both the verification verdict and the individual values underneath it, so a partially applied .reg cannot pass as compliant.RollbackContract.Tests.ps1is the executable spec for Track 1. Rollback does not exist yet, so the block skips itself until InvokeChanges.ps1 references Restore-RegistryBackupState, then activates on its own. It pins the failure policy from section 3.2, including the load-bearing case that an app removal failure must not trigger rollback.Writing that contract first is the point: Track 1's open decisions are now assertions rather than prose.
Two findings, recorded rather than papered over
Undo cannot be tested, so plan scenario 2 is not writable. WinSwift.ps1 initialises
$script:UndoParamsto an empty hashtable, and the only code that ever populates it is Scripts/GUI/Show-MainWindow.ps1. Invoke-UndoFeatures exists, the per-feature RegistryUndoKey metadata exists, and all 112 features declare undo text, but nothing on the command line can select a feature for undo. The test is present and skipped with the reason inline. The larger problem it stands in for is that unattended deployments cannot revert either.CI runs more than section 5.2 expected. The plan assumed CI would be limited to non-mutating cases. In practice the windows-2025 runner is build 26100 and runs elevated, so it clears both the version gate and the admin guard, and
ReadOnlyandDryRunboth run there. Sandbox is still needed forMutating, since Server 2025 has no Start menu start2.bin and a limited Appx stack.Sandbox harness
Sandbox/WinSwift-Tests.wsb maps the repository read-only so a mutating test cannot modify the working tree it was launched from. The bootstrap copies it somewhere writable, installs Pester, and runs the full suite. Everything is discarded when the window closes.
Three defects CI found in the suite itself
*.Tests.ps1when handed a directory. The files used theTest-*.ps1convention from Tests/Unit, which works there only because CI invokes each by explicit path.$result.FailedCountnull, and$null -gt 0is false. A suite that discovers nothing now fails.-Skip:during discovery, but Test-IsElevated was only dot-sourced in BeforeAll. Resolved in BeforeDiscovery instead.Static validation covers 110 files, up from 103.