release: v1.3.4 — SPSyncUserProfile performance: build UserProfileManager once (#24)#25
Merged
Conversation
…ng, add timing summary Field run on a large (~100k-user) farm took ~1h11. Transcript analysis showed the time was spent almost entirely in the per-user loop, which rebuilt the User Profile service context and UserProfileManager on EVERY user (~76k times) — the dominant cost and the cause of a ~30% throughput decay over the run (object churn / GC). - Build Get-SPServiceContext + UserProfileManager ONCE in Main and pass the manager into Add-SPSUserProfile (new -UserProfileManager parameter, replacing -MySiteUrl), instead of reconstructing them per user. Fail fast (Exit 1) if the manager cannot be created. - Condense the per-user transcript from a ~14-line before/after dump to a single status line ([UPDATE] login (changed: ...), [CREATE], [INFO], [UNKNOWN_USER]), slashing synchronous I/O and transcript size on large runs. - Add an end-of-run timing summary (users, wall time, avg ms/user, and the CREATE/UPDATE/INFO/UNKNOWN_USER breakdown) so runs are measurable from the log. No behaviour change: same profiles created/updated, same JSON and HTML outputs. Tests lock the new parameter contract (UserProfileManager mandatory, MySiteUrl gone). Fixes #24
Bump module to 1.3.4; changelog and release notes for the SPSyncUserProfile performance work (#24). RELEASE-NOTES.md holds the current version only.
Pester Test Results101 tests 101 ✅ 3s ⏱️ Results for commit 1be20b8. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #24. Performance release for
SPSyncUserProfile.ps1. No behaviour change — same profiles created/updated, same JSON and HTML outputs.Why
A field run on a large (~100k-user) farm took ~1h11. Transcript analysis of that run:
Test-SPSADUser(LDAP)Root cause:
Add-SPSUserProfilerebuiltGet-SPServiceContext+UserProfileManageron every user (~76k constructions, each loading UPA config) — the dominant per-user cost and the driver of the throughput decay (object churn / GC pressure).What
UserProfileManageronce in Main and pass the manager intoAdd-SPSUserProfile(new-UserProfileManagerparameter, replacing-MySiteUrl). Fail fast (Exit 1) if it can't be created.[UPDATE] DOMAIN\user (changed: WorkEmail, Country),[CREATE],[INFO],[UNKNOWN_USER]. On a ~100k-user run this turns a multi-million-line, tens-of-MB transcript into one line per user and removes the matching synchronous I/O.Not in this PR (deferred)
Parallelising the profile write with a RunspacePool. It is not a copy-paste of the InfoList AD path:
UserProfileManageris not thread-safe, each runspace needs its own context + SharePoint command surface, and the mutating calls (CreateUserProfile/Commit) need isolation and on-farm SQL-contention testing. To be decided with fresh numbers from the new timing summary, only if the single-threaded win here is not enough.Tests
Add-SPSUserProfilenow has a mandatoryUserProfileManagerparameter and no longer exposesMySiteUrl.Split-SPSProfileUser, account-status, report) unchanged and still covered.Upgrade note
No configuration change; output files unchanged. Deploy and run — the next
SPSyncUserProfile.ps1run is faster, its log far smaller, and ends with the timing summary.