diff --git a/CHANGELOG.md b/CHANGELOG.md index 66a62cf..2c1e6eb 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,17 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] +## [1.3.4] - 2026-07-10 + +### Changed + +- **`SPSyncUserProfile.ps1` performance** (#24). The User Profile service context and `UserProfileManager` are now built **once** and reused for every user, instead of being reconstructed on every iteration inside `Add-SPSUserProfile`. On a large (~100k-user) farm the per-user reconstruction was the dominant cost of the run and drove a steady throughput decay through object churn/GC pressure. No behaviour change: the same profiles are created/updated and the same JSON/report outputs are produced. +- The per-user transcript output is condensed from a ~14-line before/after dump to a **single status line** per user (`[UPDATE] DOMAIN\user (changed: …)`, `[CREATE]`, `[INFO]`, `[UNKNOWN_USER]`), cutting the synchronous I/O and the transcript size dramatically on large runs. (#24) + +### Added + +- `SPSyncUserProfile.ps1` prints an **end-of-run timing summary** — users processed, wall time, average ms/user, and the CREATE/UPDATE/INFO/UNKNOWN_USER breakdown — so run performance is measurable from the log. (#24) + ## [1.3.3] - 2026-07-09 ### Added diff --git a/RELEASE-NOTES.md b/RELEASE-NOTES.md index 8e02a19..a461c76 100644 --- a/RELEASE-NOTES.md +++ b/RELEASE-NOTES.md @@ -1,48 +1,37 @@ # SPSUserSync - Release Notes -## [1.3.3] - 2026-07-09 +## [1.3.4] - 2026-07-10 -This release adds **AD account status detection** so departed employees are handled -correctly, and it works for **every customer** because it relies only on universal -Active Directory signals — never on a customer-specific leaver process. +This is a **performance** release for `SPSyncUserProfile.ps1`, with no change in +behaviour: the same profiles are created and updated, and the same JSON files and +HTML report are produced. -Two kinds of departed accounts are now distinguished and handled: +On a large (~100k-user) farm, a full profile reconciliation took over an hour. A +transcript analysis showed the time was spent almost entirely in the per-user loop, +and that the loop rebuilt the User Profile service context and `UserProfileManager` +**on every single user** — the dominant cost, and the cause of a steady throughput +decay over the run (object churn / GC pressure). -- **Disabled** accounts (kept in AD, `userAccountControl` bit `0x2`): detected and, - with the new opt-in `SkipDisabledUsers`, reported instead of being given a profile. -- **Deleted** accounts (no longer in AD): already not provisioned, now explicitly - tagged so they can be told apart from an actionable configuration gap. - -### Added +### Changed -- **Account status on every record** — `ConvertTo-SPSUserRecord` reads - `userAccountControl` and exposes `AccountStatus` (`Active` / `Disabled` / - `NotFound`) and `Enabled`; `SPSyncUserInfoList.ps1` writes `AccountStatus` into the - JSON snapshot. Universal by design: no dedicated OU, naming convention, HR feed or - retention assumption. A resolved account with no `userAccountControl` (some non-AD - LDAP directories) stays `Active`, exactly as before. (#21) -- **`SkipDisabledUsers`** (`sync-settings.psd1`, default `$false`) — when `$true`, - `SPSyncUserProfile.ps1` reports `Disabled` accounts as Not Added instead of creating - or updating their profile. Default preserves the previous behaviour. (#21) -- **`NotAddedReason`** on each Not-Added entry — `AD_NOT_FOUND`, `MISSING_ATTRIBUTES` - or `DISABLED` — with a per-reason breakdown in the run summary, so an expected miss - (a departed account) is easy to tell apart from an actionable one (e.g. a forest - missing from `ad-domains.psd1`). (#21) -- The **`SPSyncUserInfoList` HTML report** gains an *AD Status* column and a - *Disabled in AD* card (with a note pointing at `SkipDisabledUsers`), so disabled — - departed-but-retained — accounts are visible before the profile sync runs. (#21) +- **The `UserProfileManager` is now built once and reused for the whole run** + (#24), instead of being reconstructed for every user inside `Add-SPSUserProfile`. + This removes the dominant per-user cost and the associated memory churn. +- **The per-user transcript output is condensed to a single status line** — + `[UPDATE] DOMAIN\user (changed: WorkEmail, Country)`, `[CREATE]`, `[INFO]` or + `[UNKNOWN_USER]` — replacing the previous ~14-line before/after dump. On a + ~100k-user run this cuts a multi-million-line, tens-of-MB transcript down to one + line per user and removes the corresponding synchronous I/O. (#24) -### Changed +### Added -- CI: bump the GitHub Actions that ran on the deprecated Node.js 20 runtime — - `actions/checkout` `v4`→`v7`, `actions/upload-artifact` `v4`→`v7`, - `softprops/action-gh-release` `v2`→`v3`. CI-only; the packaged module and scripts - are unchanged. (#22) +- **End-of-run timing summary** in `SPSyncUserProfile.ps1`: users processed, wall + time, average milliseconds per user, and the CREATE / UPDATE / INFO / + UNKNOWN_USER breakdown — so each run's performance is visible directly in the + log. (#24) ### Upgrade note -`SkipDisabledUsers` acts on the `AccountStatus` written by `SPSyncUserInfoList.ps1` -1.3.3+. **Regenerate the JSON snapshot after upgrading** for the flag to take effect; -a pre-1.3.3 snapshot carries no status, so no user is skipped as disabled until then. -There is no behaviour change for a correctly-deployed farm when `SkipDisabledUsers` -is left at its default `$false`. +No configuration change is required and the output files are unchanged. Simply +deploy the new version; the next `SPSyncUserProfile.ps1` run is faster and its log +is far smaller, ending with the new timing summary. diff --git a/src/Modules/SPSUserSync.Common/SPSUserSync.Common.psd1 b/src/Modules/SPSUserSync.Common/SPSUserSync.Common.psd1 index 6858e9a..d2bb9d8 100644 --- a/src/Modules/SPSUserSync.Common/SPSUserSync.Common.psd1 +++ b/src/Modules/SPSUserSync.Common/SPSUserSync.Common.psd1 @@ -1,6 +1,6 @@ @{ RootModule = 'SPSUserSync.Common.psm1' - ModuleVersion = '1.3.3' + ModuleVersion = '1.3.4' GUID = '97e2ce6d-509f-4916-846e-da2d5780765e' Author = 'Jean-Cyril DROUHIN' CompanyName = 'luigilink' diff --git a/src/SPSyncUserProfile.ps1 b/src/SPSyncUserProfile.ps1 index 076c116..cbb253b 100644 --- a/src/SPSyncUserProfile.ps1 +++ b/src/SPSyncUserProfile.ps1 @@ -31,6 +31,12 @@ every forest because the AD lookup has already been performed on the application farms. + For performance, the User Profile service context and UserProfileManager are + created once and reused for every user (rather than rebuilt per user), and the + per-user transcript output is condensed to a single status line; the run ends + with a timing summary (users, wall time, average ms/user, and the + CREATE/UPDATE/INFO/UNKNOWN_USER breakdown). + .PARAMETER InputFile Absolute path of the JSON file produced by SPSyncUserInfoList.ps1. @@ -110,8 +116,7 @@ function Add-SPSUserProfile { $Country, [Parameter(Mandatory = $true)] - [System.String] - $MySiteUrl, + $UserProfileManager, [Parameter(Mandatory = $true)] [System.String] @@ -126,51 +131,40 @@ function Add-SPSUserProfile { $UserLogin = $UserLogin.Replace($ClaimPrefix, '') $userProfileUpdated = $false $spUserProfileStatus = 'INFO' - $funcStarted = Get-Date - - Write-Output '--------------------------------------------------------------' - Write-Output "Manage $UserLogin in User Profile Service App - Started at $funcStarted" + $changedFields = New-Object System.Collections.Generic.List[System.String] try { - $spServiceContext = Get-SPServiceContext -Site $MySiteUrl - $spUserProfileManager = New-Object -TypeName Microsoft.Office.Server.UserProfiles.UserProfileManager($spServiceContext) - - if ($spUserProfileManager.UserExists($UserLogin)) { - $getUserProfile = $spUserProfileManager.GetUserProfile($UserLogin) - Write-Output '[SPUserProfiles.UserProfileManager.GetUserProfile] Current Values:' - Write-Output "AccountName: $UserLogin" - Write-Output "PreferredName: $($getUserProfile['PreferredName'].Value)" - Write-Output "FirstName: $($getUserProfile['FirstName'].Value)" - Write-Output "LastName: $($getUserProfile['LastName'].Value)" - Write-Output "WorkEmail: $($getUserProfile['WorkEmail'].Value)" - Write-Output "Country: $($getUserProfile['Country'].Value)" - Write-Output '[SPUserProfiles.UserProfileManager.GetUserProfile] Target Values:' - Write-Output "AccountName: $UserLogin" - Write-Output "PreferredName: $PreferredName" - Write-Output "FirstName: $FirstName" - Write-Output "LastName: $LastName" - Write-Output "WorkEmail: $WorkEmail" - Write-Output "Country: $Country" + # The UserProfileManager is built once by the caller and passed in, rather + # than rebuilt per user. Rebuilding Get-SPServiceContext + UserProfileManager + # on every user was the dominant cost of the run on a large farm and drove a + # steady throughput decay (object churn / GC pressure). (#24) + if ($UserProfileManager.UserExists($UserLogin)) { + $getUserProfile = $UserProfileManager.GetUserProfile($UserLogin) if ($getUserProfile['WorkEmail'].Value -ne $WorkEmail) { $getUserProfile['WorkEmail'].Value = $WorkEmail $userProfileUpdated = $true + [void]$changedFields.Add('WorkEmail') } if ($getUserProfile['PreferredName'].Value -ne $PreferredName) { $getUserProfile['PreferredName'].Value = $PreferredName $userProfileUpdated = $true + [void]$changedFields.Add('PreferredName') } if ($getUserProfile['FirstName'].Value -ne $FirstName) { $getUserProfile['FirstName'].Value = $FirstName $userProfileUpdated = $true + [void]$changedFields.Add('FirstName') } if ($getUserProfile['LastName'].Value -ne $LastName) { $getUserProfile['LastName'].Value = $LastName $userProfileUpdated = $true + [void]$changedFields.Add('LastName') } if ($getUserProfile['Country'].Value -ne $Country -and -not [string]::IsNullOrEmpty($Country)) { $getUserProfile['Country'].Value = $Country $userProfileUpdated = $true + [void]$changedFields.Add('Country') } if ($userProfileUpdated) { $getUserProfile.Commit() @@ -178,25 +172,11 @@ function Add-SPSUserProfile { } } else { - Write-Output '[SPUserProfiles.UserProfileManager.GetUserProfile] Current Values:' - Write-Output "AccountName: $UserLogin" - Write-Output 'PreferredName: NULL' - Write-Output 'FirstName: NULL' - Write-Output 'LastName: NULL' - Write-Output 'WorkEmail: NULL' - Write-Output 'Country: NULL' - Write-Output '[SPUserProfiles.UserProfileManager.GetUserProfile] Target Values:' - Write-Output "AccountName: $UserLogin" - Write-Output "PreferredName: $PreferredName" - Write-Output "FirstName: $FirstName" - Write-Output "LastName: $LastName" - Write-Output "WorkEmail: $WorkEmail" - Write-Output "Country: $Country" $spUserProfileStatus = 'CREATE' if (Test-SPSADUser -UserLogin $UserLogin) { - $spUserProfileManager.CreateUserProfile($UserLogin, $PreferredName) + $UserProfileManager.CreateUserProfile($UserLogin, $PreferredName) if (-not [string]::IsNullOrEmpty($WorkEmail)) { - $getUserProfile = $spUserProfileManager.GetUserProfile($UserLogin) + $getUserProfile = $UserProfileManager.GetUserProfile($UserLogin) $getUserProfile['WorkEmail'].Value = $WorkEmail if (-not [string]::IsNullOrEmpty($Country)) { $getUserProfile['Country'].Value = $Country } $getUserProfile['FirstName'].Value = $FirstName @@ -209,6 +189,17 @@ function Add-SPSUserProfile { } } + # One concise line per user instead of the previous ~14-line before/after + # dump (~14 lines x ~100k users produced a multi-million-line, tens-of-MB + # transcript). On an UPDATE the changed fields are listed - the only + # actionable detail; CREATE / UNKNOWN_USER are self-explanatory. (#24) + if ($spUserProfileStatus -eq 'UPDATE') { + Write-Output "[$spUserProfileStatus] $UserLogin (changed: $($changedFields -join ', '))" + } + else { + Write-Output "[$spUserProfileStatus] $UserLogin" + } + [void]$ResultCollection.Add([SPSUserProfileMgmt]@{ AccountName = $UserLogin PreferredName = $PreferredName @@ -224,7 +215,6 @@ function Add-SPSUserProfile { catch { $catchMessage = @" An error occurred while managing user '$UserLogin' in User Profile Service App -MySiteUrl: $MySiteUrl Exception: $_ "@ Add-SPSUserSyncEvent -Message $catchMessage -Source 'Add-SPSUserProfile' -EntryType 'Error' @@ -432,8 +422,26 @@ secrets.psd1 must be regenerated on THIS server, as the service account that run Exit 1 } +# Build the User Profile service context and manager ONCE, before the loop. Rebuilding +# them inside Add-SPSUserProfile for every user was the dominant cost of the run on a +# large farm (and the source of a steady throughput decay through object churn). (#24) +try { + $spServiceContext = Get-SPServiceContext -Site $spMySiteHostUrl + $spUserProfileManager = New-Object -TypeName Microsoft.Office.Server.UserProfiles.UserProfileManager($spServiceContext) +} +catch { + $catchMessage = @" +An error occurred while creating the UserProfileManager for MySite: $spMySiteHostUrl +Exception: $_ +"@ + Add-SPSUserSyncEvent -Message $catchMessage -Source 'UserProfileManager' -EntryType 'Error' + Stop-Transcript | Out-Null + Exit 1 +} + # Process eligible users $tbSPSUserProfileMgmt = New-Object -TypeName System.Collections.ArrayList +$profileLoopStarted = Get-Date foreach ($psoSPSiteUser in $usersWithPrerequisites) { try { Add-SPSUserProfile ` @@ -443,7 +451,7 @@ foreach ($psoSPSiteUser in $usersWithPrerequisites) { -LastName $psoSPSiteUser.LastName ` -WorkEmail $psoSPSiteUser.Email ` -Country $psoSPSiteUser.Country ` - -MySiteUrl $spMySiteHostUrl ` + -UserProfileManager $spUserProfileManager ` -ClaimPrefix $settings.ClaimPrefix ` -ResultCollection $tbSPSUserProfileMgmt } @@ -457,6 +465,23 @@ Exception: $_ Add-SPSUserSyncEvent -Message $catchMessage -Source 'Add-SPSUserProfile' -EntryType 'Error' } } +$profileLoopEnded = Get-Date + +# Timing summary (#24): each run reports its own throughput and the +# CREATE / UPDATE / INFO / UNKNOWN_USER split, so future performance work is +# driven by fresh numbers rather than guesswork. +$profileElapsed = $profileLoopEnded - $profileLoopStarted +Write-Output '-----------------------------------------------' +Write-Output "Profile loop: $($tbSPSUserProfileMgmt.Count) user(s) in $([System.Math]::Round($profileElapsed.TotalMinutes, 1)) min ($([System.Math]::Round($profileElapsed.TotalSeconds, 1)) s)" +if ($tbSPSUserProfileMgmt.Count -gt 0) { + $msPerUser = [System.Math]::Round($profileElapsed.TotalMilliseconds / $tbSPSUserProfileMgmt.Count, 1) + Write-Output "Average: $msPerUser ms/user" + foreach ($statusGroup in ($tbSPSUserProfileMgmt | Group-Object -Property Status | Sort-Object Count -Descending)) { + $statusLabel = if ([string]::IsNullOrEmpty($statusGroup.Name)) { '(none)' } else { $statusGroup.Name } + Write-Output " $statusLabel : $($statusGroup.Count)" + } +} +Write-Output '-----------------------------------------------' if ($usersWithoutPrerequisites.Count -ne 0) { $missingCount = @($usersWithoutPrerequisites | Where-Object { $_.NotAddedReason -eq 'MISSING_ATTRIBUTES' }).Count diff --git a/tests/SPSyncUserProfile.Tests.ps1 b/tests/SPSyncUserProfile.Tests.ps1 index 698abf4..823c5a7 100644 --- a/tests/SPSyncUserProfile.Tests.ps1 +++ b/tests/SPSyncUserProfile.Tests.ps1 @@ -48,6 +48,16 @@ Describe 'Add-SPSUserProfile parameter contract' { $hasAllowEmpty = [bool]($p.Attributes | Where-Object { $_ -is [System.Management.Automation.AllowEmptyCollectionAttribute] }) $hasAllowEmpty | Should -BeTrue } + + It 'takes a pre-built UserProfileManager instead of rebuilding it per user (#24)' { + # The manager is now built once in Main and passed in; the old per-user + # MySiteUrl parameter (which drove the per-user Get-SPServiceContext + + # UserProfileManager construction) is gone. + $cmd = Get-Command Add-SPSUserProfile + $cmd.Parameters.Keys | Should -Contain 'UserProfileManager' + $cmd.Parameters['UserProfileManager'].Attributes.Where{ $_ -is [System.Management.Automation.ParameterAttribute] }[0].Mandatory | Should -BeTrue + $cmd.Parameters.Keys | Should -Not -Contain 'MySiteUrl' + } } Describe 'Split-SPSProfileUser' {