From 906b342d407db1a90d42746cc0ecb8b7d16c324e Mon Sep 17 00:00:00 2001 From: LuigiLink Date: Thu, 11 Jun 2026 19:21:21 +0200 Subject: [PATCH] Enhance SPSUpdate for version 3.2.1: implement LPT algorithm for content database distribution, add distribution report and timestamped snapshots, update patch installation method, and fix edge case in database assignment; bump version to 3.2.1 and add corresponding tests. --- CHANGELOG.md | 27 +++++ RELEASE-NOTES.md | 36 +++---- scripts/Modules/sps.util.psm1 | 105 ++++++++++++------- scripts/SPSUpdate.ps1 | 6 +- tests/Modules/sps.util.Tests.ps1 | 166 +++++++++++++++++++++++++++++++ tests/SPSUpdate.Tests.ps1 | 2 +- 6 files changed, 281 insertions(+), 61 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 2214cd2..ba35623 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -3,6 +3,33 @@ The format is based on and uses the types of changes according to [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). +## [3.2.1] - 2026-06-11 + +### Changed + +scripts/Modules/sps.util.psm1: + +- `Initialize-SPSContentDbJsonFile` now distributes content databases across the four sequences using a **Longest Processing Time First (LPT)** algorithm based on `DiskSizeRequired`, instead of splitting them evenly by count (`floor(count / 4)`). Sequences are now balanced by total database size so parallel upgrade workloads finish closer to the same time. +- `Initialize-SPSContentDbJsonFile` now prints a distribution report to the transcript showing the count, total size (MB) and percentage for each of the four sequences. +- `Initialize-SPSContentDbJsonFile` now also writes a timestamped snapshot of the inventory (`_yyyy-MM-dd_HH-mm-ss.json`) next to the canonical file each time it runs. The canonical file (consumed by `SPSUpdate.ps1`) is still overwritten in place, while the dated snapshots accumulate in `scripts/Config/` so previous inventories can be reviewed or restored. Snapshot failures are logged via `Write-Verbose` and never block the canonical write. +- `Start-SPSProductUpdate` now invokes the SharePoint patch setup with `/passive` instead of `/quiet`. `/passive` still runs the patch without user interaction but displays a progress UI, which gives administrators visibility on the installation progress when the script is run interactively. Behavior of the returned exit code and the post-install service-restoration logic is unchanged. + +scripts/SPSUpdate.ps1: + +- Bumped `$SPSUpdateVersion` to `3.2.1`. + +### Fixed + +scripts/Modules/sps.util.psm1: + +- Fixed `Initialize-SPSContentDbJsonFile` edge case where fewer than 4 content databases caused `groupSize` to evaluate to `0`, dumping every database into `SPContentDatabase4` and leaving `SPContentDatabase1..3` empty. + +### Tests + +tests/Modules/sps.util.Tests.ps1: + +- Added Pester tests for `Initialize-SPSContentDbJsonFile` covering: no file written when `Get-SPContentDatabase` returns `$null`, fair distribution of fewer than 4 databases (regression for the `floor(count / 4)` bug), LPT balancing by size (the largest database lands alone in its sequence), and the `SPContentDatabase1..4` JSON property contract consumed by `SPSUpdate.ps1`. + ## [3.2.0] - 2026-05-26 ### Added diff --git a/RELEASE-NOTES.md b/RELEASE-NOTES.md index 20f6c7d..40ef187 100644 --- a/RELEASE-NOTES.md +++ b/RELEASE-NOTES.md @@ -1,38 +1,30 @@ # SPSUpdate - Release Notes -## [3.2.0] - 2026-05-26 +## [3.2.1] - 2026-06-11 -### Added - -scripts/SPSUpdate.ps1: - -- Added new `InitContentDB` value for the `Action` parameter that (re)generates the ContentDatabase inventory JSON file for the local farm. Intended to be run on the source farm before a farm upgrade (for example SharePoint Server 2019 → Subscription Edition). -- Added new `MountContentDatabase` configuration property in the JSON config file. When `true`, the master server iterates through the ContentDatabase inventory JSON file and mounts every database that is not already attached to the farm. Mounts are performed sequentially to avoid concurrent writes to the configuration database. -- Loader of the ContentDatabase inventory JSON file now also runs when `MountContentDatabase` is `true` (previously only when `UpgradeContentDatabase` was `true`). -- Added dedicated transcript log file naming for the `InitContentDB` action. +### Changed scripts/Modules/sps.util.psm1: -- Added `Mount-SPSContentDatabase` wrapper around `Mount-SPContentDatabase`. The wrapper validates the target web application, skips databases that are already attached, and accepts an optional `DatabaseServer` parameter. +- `Initialize-SPSContentDbJsonFile` now distributes content databases across the four sequences using a **Longest Processing Time First (LPT)** algorithm based on `DiskSizeRequired`, instead of splitting them evenly by count (`floor(count / 4)`). Sequences are now balanced by total database size so parallel upgrade workloads finish closer to the same time. +- `Initialize-SPSContentDbJsonFile` now prints a distribution report to the transcript showing the count, total size (MB) and percentage for each of the four sequences. +- `Initialize-SPSContentDbJsonFile` now also writes a timestamped snapshot of the inventory (`_yyyy-MM-dd_HH-mm-ss.json`) next to the canonical file each time it runs. The canonical file (consumed by `SPSUpdate.ps1`) is still overwritten in place, while the dated snapshots accumulate in `scripts/Config/` so previous inventories can be reviewed or restored. Snapshot failures are logged via `Write-Verbose` and never block the canonical write. +- `Start-SPSProductUpdate` now invokes the SharePoint patch setup with `/passive` instead of `/quiet`. `/passive` still runs the patch without user interaction but displays a progress UI, which gives administrators visibility on the installation progress when the script is run interactively. Behavior of the returned exit code and the post-install service-restoration logic is unchanged. -tests/SPSUpdate.Tests.ps1: +scripts/SPSUpdate.ps1: -- Added assertions for the new `InitContentDB` action and the `MountContentDatabase` flow. -- Added regression test that the ProductUpdate flow no longer calls `Test-SPSPendingReboot`. +- Bumped `$SPSUpdateVersion` to `3.2.1`. -tests/Modules/sps.util.Tests.ps1: +### Fixed -- Added tests for `Mount-SPSContentDatabase` (export, parameters, idempotency when the DB is already attached, mounting when missing, error when the web application is not found). +scripts/Modules/sps.util.psm1: -### Changed +- Fixed `Initialize-SPSContentDbJsonFile` edge case where fewer than 4 content databases caused `groupSize` to evaluate to `0`, dumping every database into `SPContentDatabase4` and leaving `SPContentDatabase1..3` empty. -scripts/SPSUpdate.ps1: +### Tests -- Bumped `$SPSUpdateVersion` to `3.2.0`. -- Removed the blocking pending-reboot check from the `ProductUpdate` action. On production farms the Windows reboot markers (Component Based Servicing, `PendingFileRenameOperations`, etc.) commonly remain set after several reboots, which was causing legitimate updates to be aborted. The `Test-SPSPendingReboot` helper is kept available in `util.psm1` for ad-hoc usage. - -### Documentation +tests/Modules/sps.util.Tests.ps1: -- Updated `scripts/SPSUpdate_README.md`, `wiki/Configuration.md` and `wiki/Usage.md` to document the new `InitContentDB` action, the new `MountContentDatabase` JSON property and the removal of the blocking pending-reboot check from `ProductUpdate`. +- Added Pester tests for `Initialize-SPSContentDbJsonFile` covering: no file written when `Get-SPContentDatabase` returns `$null`, fair distribution of fewer than 4 databases (regression for the `floor(count / 4)` bug), LPT balancing by size (the largest database lands alone in its sequence), and the `SPContentDatabase1..4` JSON property contract consumed by `SPSUpdate.ps1`. A full list of changes in each version can be found in the [change log](CHANGELOG.md) diff --git a/scripts/Modules/sps.util.psm1 b/scripts/Modules/sps.util.psm1 index 80ea5d3..f6e763c 100644 --- a/scripts/Modules/sps.util.psm1 +++ b/scripts/Modules/sps.util.psm1 @@ -418,41 +418,54 @@ function Initialize-SPSContentDbJsonFile { $spAllDatabases = Get-SPContentDatabase -ErrorAction SilentlyContinue if ($null -ne $spAllDatabases) { - #Calculate the number of databases in each group - $groupSize = [math]::Floor($spAllDatabases.Count / 4) - #Loop through each content database and assign to groups - for ($i = 0; $i -lt $spAllDatabases.Count; $i++) { - $spDatabase = $spAllDatabases[$i] - #Determine which group to add the database to - if ($i -lt $groupSize) { - [void]$tbSPContentDb1.Add([SPDbContent]@{ - Name = $spDatabase.Name; - Server = $spDatabase.Server; - WebAppUrl = $spDatabase.WebApplication.Url; - }) - } - elseif ($i -lt ($groupSize * 2)) { - [void]$tbSPContentDb2.Add([SPDbContent]@{ - Name = $spDatabase.Name; - Server = $spDatabase.Server; - WebAppUrl = $spDatabase.WebApplication.Url; - }) - } - elseif ($i -lt ($groupSize * 3)) { - [void]$tbSPContentDb3.Add([SPDbContent]@{ - Name = $spDatabase.Name; - Server = $spDatabase.Server; - WebAppUrl = $spDatabase.WebApplication.Url; - }) + # --- LPT (Longest Processing Time First) scheduling --- + # Balance databases across 4 sequences by total DiskSizeRequired + # rather than by count, so parallel upgrade workloads finish closer + # to the same time. + + # 1. Sort databases by size descending + $spSortedDatabases = $spAllDatabases | + Sort-Object -Property DiskSizeRequired -Descending + + # 2. Track cumulative load (bytes) per sequence (index 0 = Seq1 .. 3 = Seq4) + $sequenceLoad = @(0.0, 0.0, 0.0, 0.0) + $sequenceLists = @($tbSPContentDb1, $tbSPContentDb2, $tbSPContentDb3, $tbSPContentDb4) + + # 3. Assign each database to the sequence with the lowest current load + foreach ($spDatabase in $spSortedDatabases) { + $minLoad = $sequenceLoad[0] + $minIndex = 0 + for ($s = 1; $s -lt 4; $s++) { + if ($sequenceLoad[$s] -lt $minLoad) { + $minLoad = $sequenceLoad[$s] + $minIndex = $s + } } - else { - [void]$tbSPContentDb4.Add([SPDbContent]@{ - Name = $spDatabase.Name; - Server = $spDatabase.Server; - WebAppUrl = $spDatabase.WebApplication.Url; - }) + [void]$sequenceLists[$minIndex].Add([SPDbContent]@{ + Name = $spDatabase.Name; + Server = $spDatabase.Server; + WebAppUrl = $spDatabase.WebApplication.Url; + }) + $sequenceLoad[$minIndex] += $spDatabase.DiskSizeRequired + } + + # --- Distribution report (visible in transcript) --- + $totalBytes = ($sequenceLoad | Measure-Object -Sum).Sum + $totalMB = [math]::Round($totalBytes / 1MB, 0) + $dbCount = @($spSortedDatabases).Count + Write-Output '--- ContentDatabase Distribution Report ---' + Write-Output ("Total : {0} database(s) | {1:N0} MB" -f $dbCount, $totalMB) + for ($s = 0; $s -lt 4; $s++) { + $loadMB = [math]::Round($sequenceLoad[$s] / 1MB, 0) + $pct = if ($totalBytes -gt 0) { + [math]::Round($sequenceLoad[$s] / $totalBytes * 100, 1) } + else { 0 } + Write-Output (" Sequence {0} : {1,3} database(s) | {2,7:N0} MB | {3,5:N1}%" ` + -f ($s + 1), $sequenceLists[$s].Count, $loadMB, $pct) } + Write-Output '-------------------------------------------' + #Add each array to jsonObject $jsonObject | Add-Member -MemberType NoteProperty ` -Name 'SPContentDatabase1' ` @@ -470,8 +483,30 @@ function Initialize-SPSContentDbJsonFile { -Name 'SPContentDatabase4' ` -Value $tbSPContentDb4 - #Convert jsonObject to JSON and save to a file - $jsonObject | ConvertTo-Json | Set-Content -Path $Path -Force + # Serialize once and write both the canonical file (consumed by SPSUpdate.ps1) + # and a timestamped snapshot in the same folder so previous inventories are + # retained for troubleshooting and rollback. + $jsonPayload = $jsonObject | ConvertTo-Json + $jsonPayload | Set-Content -Path $Path -Force + + try { + $snapshotDir = [System.IO.Path]::GetDirectoryName($Path) + $snapshotBaseName = [System.IO.Path]::GetFileNameWithoutExtension($Path) + $snapshotExtension = [System.IO.Path]::GetExtension($Path) + $snapshotTimestamp = Get-Date -Format 'yyyy-MM-dd_HH-mm-ss' + $snapshotFileName = '{0}_{1}{2}' -f $snapshotBaseName, $snapshotTimestamp, $snapshotExtension + $snapshotPath = if ([string]::IsNullOrEmpty($snapshotDir)) { + $snapshotFileName + } + else { + Join-Path -Path $snapshotDir -ChildPath $snapshotFileName + } + $jsonPayload | Set-Content -Path $snapshotPath -Force + Write-Output "ContentDatabase inventory snapshot saved to: $snapshotPath" + } + catch { + Write-Verbose -Message "Failed to write ContentDatabase inventory snapshot: $($_.Exception.Message)" + } } } @@ -682,7 +717,7 @@ Setup file is blocked! Please use 'Unblock-File -Path $SetupFile' to unblock the } - $setupInstall = Start-Process -FilePath $SetupFile -ArgumentList '/quiet /passive' -Wait -PassThru + $setupInstall = Start-Process -FilePath $SetupFile -ArgumentList '/passive' -Wait -PassThru # Error codes: https://aka.ms/installerrorcodes switch ($setupInstall.ExitCode) { 0 { diff --git a/scripts/SPSUpdate.ps1 b/scripts/SPSUpdate.ps1 index 076d0cb..72d4555 100644 --- a/scripts/SPSUpdate.ps1 +++ b/scripts/SPSUpdate.ps1 @@ -44,8 +44,8 @@ .NOTES FileName: SPSUpdate.ps1 Author: Jean-Cyril DROUHIN - Date: May 26, 2026 - Version: 3.2.0 + Date: June 11, 2026 + Version: 3.2.1 .LINK https://spjc.fr/ @@ -183,7 +183,7 @@ catch { } # Define variables -$SPSUpdateVersion = '3.2.0' +$SPSUpdateVersion = '3.2.1' $getDateFormatted = Get-Date -Format yyyy-MM-dd_H-mm $spsUpdateFileName = "$($Application)-$($Environment)_$($getDateFormatted)" $spsUpdateDBsFile = "$($Application)-$($Environment)-$($spFarmName)-ContentDBs.json" diff --git a/tests/Modules/sps.util.Tests.ps1 b/tests/Modules/sps.util.Tests.ps1 index 19a3768..09382f0 100644 --- a/tests/Modules/sps.util.Tests.ps1 +++ b/tests/Modules/sps.util.Tests.ps1 @@ -184,3 +184,169 @@ Describe 'Mount-SPSContentDatabase' { Assert-MockCalled -ModuleName sps.util -CommandName Add-SPSUpdateEvent -Times 1 -Exactly } } + +Describe 'Initialize-SPSContentDbJsonFile' { + BeforeEach { + $script:tempJsonPath = Join-Path -Path ([System.IO.Path]::GetTempPath()) ` + -ChildPath ("sps-contentdb-{0}.json" -f [guid]::NewGuid()) + } + + AfterEach { + if ($script:tempJsonPath) { + # Remove the canonical file AND any timestamped snapshots produced alongside it. + $dir = Split-Path -Path $script:tempJsonPath -Parent + $base = [System.IO.Path]::GetFileNameWithoutExtension($script:tempJsonPath) + Get-ChildItem -Path $dir -Filter "$base*.json" -ErrorAction SilentlyContinue | + Remove-Item -Force -ErrorAction SilentlyContinue + } + } + + function script:New-FakeSPDb { + param( + [string]$Name, + [long]$Size, + [string]$Server = 'SQL01', + [string]$Url = 'https://intranet.contoso.com' + ) + [pscustomobject]@{ + Name = $Name + Server = $Server + DiskSizeRequired = $Size + WebApplication = [pscustomobject]@{ Url = $Url } + } + } + + It 'writes no file when Get-SPContentDatabase returns $null' { + Mock -ModuleName sps.util -CommandName Get-SPContentDatabase -MockWith { $null } + + Initialize-SPSContentDbJsonFile -Path $script:tempJsonPath + + Test-Path -Path $script:tempJsonPath | Should -Be $false + + # And no timestamped snapshot either + $dir = Split-Path -Path $script:tempJsonPath -Parent + $base = [System.IO.Path]::GetFileNameWithoutExtension($script:tempJsonPath) + @(Get-ChildItem -Path $dir -Filter "$base*.json" -ErrorAction SilentlyContinue).Count | Should -Be 0 + } + + It 'distributes 3 databases across 3 distinct sequences (regression: old code put them all in sequence 4)' { + Mock -ModuleName sps.util -CommandName Get-SPContentDatabase -MockWith { + @( + (New-FakeSPDb -Name 'WSS_Content_A' -Size 30MB) + (New-FakeSPDb -Name 'WSS_Content_B' -Size 20MB) + (New-FakeSPDb -Name 'WSS_Content_C' -Size 10MB) + ) + } + + Initialize-SPSContentDbJsonFile -Path $script:tempJsonPath + + $json = Get-Content -Path $script:tempJsonPath -Raw | ConvertFrom-Json + + # Use .Where({$_}) to safely count: ConvertFrom-Json yields $null for "[]", + # and @($null).Count is 1 (not 0) so a naive @(...).Count would mislead. + @($json.SPContentDatabase1).Where({ $_ }).Count | Should -Be 1 + @($json.SPContentDatabase2).Where({ $_ }).Count | Should -Be 1 + @($json.SPContentDatabase3).Where({ $_ }).Count | Should -Be 1 + @($json.SPContentDatabase4).Where({ $_ }).Count | Should -Be 0 + + # LPT: largest first → Seq1 gets the 30MB DB + $json.SPContentDatabase1.Name | Should -Be 'WSS_Content_A' + $json.SPContentDatabase2.Name | Should -Be 'WSS_Content_B' + $json.SPContentDatabase3.Name | Should -Be 'WSS_Content_C' + } + + It 'balances databases by total DiskSizeRequired rather than by count' { + # Worst case for the old "floor(count/4)" splitter: one large DB plus many small ones. + # Old code would put DBs 1..(N/4) into Seq1 (incl. the giant one) and dump the rest into Seq4. + # LPT should land the giant DB alone in one sequence and spread the small ones across the others. + Mock -ModuleName sps.util -CommandName Get-SPContentDatabase -MockWith { + @( + (New-FakeSPDb -Name 'WSS_Content_BIG' -Size 100MB) + (New-FakeSPDb -Name 'WSS_Content_S1' -Size 10MB) + (New-FakeSPDb -Name 'WSS_Content_S2' -Size 10MB) + (New-FakeSPDb -Name 'WSS_Content_S3' -Size 10MB) + (New-FakeSPDb -Name 'WSS_Content_S4' -Size 10MB) + (New-FakeSPDb -Name 'WSS_Content_S5' -Size 10MB) + (New-FakeSPDb -Name 'WSS_Content_S6' -Size 10MB) + ) + } + + Initialize-SPSContentDbJsonFile -Path $script:tempJsonPath + + $json = Get-Content -Path $script:tempJsonPath -Raw | ConvertFrom-Json + + $sequences = @( + $json.SPContentDatabase1 + $json.SPContentDatabase2 + $json.SPContentDatabase3 + $json.SPContentDatabase4 + ) + + # All 7 databases must be present exactly once across the 4 sequences. + $allNames = $sequences | ForEach-Object { @($_).Where({ $_ }).Name } + @($allNames).Count | Should -Be 7 + @($allNames | Sort-Object -Unique).Count | Should -Be 7 + + # Sequence containing the big DB must hold ONLY the big DB + # (otherwise the splitter never beat the makespan of the largest single job). + $bigSequences = @( + foreach ($seq in $sequences) { + $names = @($seq).Where({ $_ }).Name + if ($names -contains 'WSS_Content_BIG') { , $names } + } + ) + $bigSequences.Count | Should -Be 1 + $bigSequences[0].Count | Should -Be 1 + } + + It 'produces the SPContentDatabase1..4 JSON contract consumed by SPSUpdate.ps1' { + Mock -ModuleName sps.util -CommandName Get-SPContentDatabase -MockWith { + @( + (New-FakeSPDb -Name 'WSS_Content_X' -Size 5MB) + (New-FakeSPDb -Name 'WSS_Content_Y' -Size 5MB) + (New-FakeSPDb -Name 'WSS_Content_Z' -Size 5MB) + (New-FakeSPDb -Name 'WSS_Content_W' -Size 5MB) + ) + } + + Initialize-SPSContentDbJsonFile -Path $script:tempJsonPath + + $json = Get-Content -Path $script:tempJsonPath -Raw | ConvertFrom-Json + $json.PSObject.Properties.Name | Should -Contain 'SPContentDatabase1' + $json.PSObject.Properties.Name | Should -Contain 'SPContentDatabase2' + $json.PSObject.Properties.Name | Should -Contain 'SPContentDatabase3' + $json.PSObject.Properties.Name | Should -Contain 'SPContentDatabase4' + + # Each entry preserves Name / Server / WebAppUrl + $first = $json.SPContentDatabase1 + $first.Name | Should -Not -BeNullOrEmpty + $first.Server | Should -Be 'SQL01' + $first.WebAppUrl | Should -Be 'https://intranet.contoso.com' + } + + It 'writes a timestamped snapshot alongside the canonical file with matching content' { + Mock -ModuleName sps.util -CommandName Get-SPContentDatabase -MockWith { + @( (New-FakeSPDb -Name 'WSS_Content_A' -Size 10MB) ) + } + + Initialize-SPSContentDbJsonFile -Path $script:tempJsonPath + + # Canonical file is written (existing contract) + Test-Path -Path $script:tempJsonPath | Should -Be $true + + # A snapshot named _.json lives next to it + $dir = Split-Path -Path $script:tempJsonPath -Parent + $base = [System.IO.Path]::GetFileNameWithoutExtension($script:tempJsonPath) + $ext = [System.IO.Path]::GetExtension($script:tempJsonPath) + $snapshotPattern = "^$([regex]::Escape($base))_\d{4}-\d{2}-\d{2}_\d{2}-\d{2}-\d{2}$([regex]::Escape($ext))$" + $snapshots = @( + Get-ChildItem -Path $dir -Filter "$base*$ext" -ErrorAction SilentlyContinue | + Where-Object { $_.Name -match $snapshotPattern } + ) + $snapshots.Count | Should -Be 1 + + # Snapshot content is identical to the canonical file + (Get-Content -Path $script:tempJsonPath -Raw) | + Should -Be (Get-Content -Path $snapshots[0].FullName -Raw) + } +} diff --git a/tests/SPSUpdate.Tests.ps1 b/tests/SPSUpdate.Tests.ps1 index bfc47fb..d855cf5 100644 --- a/tests/SPSUpdate.Tests.ps1 +++ b/tests/SPSUpdate.Tests.ps1 @@ -185,7 +185,7 @@ Describe 'SPSUpdate.ps1 Content Validation' { } It 'defines the current script version' { - $scriptContent | Should -Match '\$SPSUpdateVersion\s*=\s*''3\.2\.0''' + $scriptContent | Should -Match '\$SPSUpdateVersion\s*=\s*''3\.2\.1''' } It 'imports util module' {