Send task-host build process environment as delta

[OvesN]

Fixes #14097

Context

Out-of-proc task hosts re-send the same build-invariant data in every TaskHostConfiguration (and echo the environment back in every TaskHostTaskComplete). Two payloads dominate this redundancy on large -mt builds:

1. The build process environment (~6 KB), shipped forward in every config and back in every result.
2. On solution builds, the CurrentSolutionConfigurationContents global property — an XML blob describing every project's configuration (~57 KB on OrchardCore) — shipped forward in every config. It is ~99% of the global-properties payload and ~48% of all task-host config bytes.

Both are invariant for the vast majority of tasks, yet re-transmitted in full to connections that already received them.

Changes Made

• Send the environment as a delta. Under a negotiated wire format (PacketVersion 5) the environment is sent in full only once per task-host connection; subsequent packets whose environment is unchanged carry a 1-byte "identical" marker instead of the full dictionary, on both the forward (config) and return (result) paths. The receiver reconstructs it from the last full environment seen on that connection.

• Deduplicate the solution configuration blob. Solution (.sln/.slnx) builds inject CurrentSolutionConfigurationContents as a global property on every project. It is build-invariant (it describes the solution, not the project), but it is re-serialized into every TaskHostConfiguration. It cannot simply be dropped — a task may read it at runtime via IBuildEngine6.GetGlobalProperties() — so the same per-connection delta is applied (PacketVersion 65: the value is carried in its own field, excluded from the global-properties dictionary on the wire (so the large blob is never serialized twice), preceded by a 1-byte Full/Identical marker. It is sent once per connection, then only a marker; the task host reconstructs it from the connection baseline.

• Skip redundant per-task environment apply/restore in the task host when the next task's environment is unchanged from the previous one.

• Bug fixes:

  • Self-clear: feeding the reused (unchanged) environment back into SetEnvironment cleared it, because in -mt it was the driver's own backing dictionary. Fixed with a reference-equality guard.

Testing

• Round-trip serialization tests for the new Full/Identical wire forms: environment and CurrentSolutionConfigurationContents at PacketVersion 5
• -mt end-to-end regression tests: environment observed across consecutive task-host tasks, and an environment change made by one task-host task is observed by the next (verified failing before the aliasing fix, passing after).
• Validated end-to-end: full build incl. net35, and OrchardCore.slnx -mt Rebuild succeeds.
• Existing task-host test suites pass.

Measurements

Workload: OrchardCore (OrchardCore.slnx), -t:Rebuild -mt — 17,975 task-host task invocations. Serialized payload sizes measured with ThProfile (write-side stream-position deltas; deterministic, single run). Baseline = upstream main; patched = this PR.

Task-host IPC payload	Baseline	This PR	Saved
Build process environment (forward + return)	122.0 MB	0.1 MB	≈ 122 MB
CurrentSolutionConfigurationContents (forward)	647.4 MB	1.8 MB	≈ 646 MB
Total task-host IPC (config + result)	≈ 1,548 MB	≈ 764 MB	≈ 785 MB (−51%)

Per direction:

Field	Baseline	This PR	Reduction
TaskHostConfiguration total (forward)	1,342 MB	620 MB	−54%
↳ environment	61.0 MB	0.1 MB	−99.8%
↳ CurrentSolutionConfigurationContents	647.4 MB	1.8 MB	−99.7%
↳ global properties (incl. the blob)	653.6 MB	7.9 MB	−98.8%
TaskHostTaskComplete total (return)	206.4 MB	143.3 MB	−31% (all from the environment)

The environment was ~4.6% of every config packet and ~29.7% of every result packet; CurrentSolutionConfigurationContents was ~48% of every config packet. After the change each carries only a 1-byte "unchanged" marker once the connection baseline is established. The remaining config payload is dominated by per-task task parameters, which are not invariant and so are not deduplicated.

Wall-clock / overhead

Sending the environment as a delta, deduplicating the solution configuration, and skipping the redundant per-task environment apply/restore did not measurably change total wall-clock time or task-host overhead in my measurements — the IPC savings are below the run-to-run noise floor on my machine. This is an IPC-volume optimization (fewer bytes serialized/transmitted/deserialized), not a critical-path one.

I measured TaskHost overhead on OrchardCore after restore on my machine three times and took the average, using this guide to compute task host overhead. (Note: in this guide all MSBuildMultiThreadableTask parameters were deliberately deleted; I kept them, of course.)

The measurement showed no significant improvement on total time for task and overhead:
Baseline:

Patched version:

Metric	Before → After	Delta	Surface reading
Total task-host time	871,916 → 1,008,858 ms	+136,942 ms (+15.7%)	looks like regression
Overhead share	58.6% → 54.4%	−4.2 pp (−7.2% rel.)	looks like improvement

Both deltas are likely noise (the two metrics point in opposite directions, and execute_ms — task work this change cannot affect — moved by a similar magnitude, indicating machine-load drift rather than a real effect).

[JanProvaznik]

does this also avoid the setting of environment in cases when it's not needed? not only the transmission?

[Copilot]

Pull request overview

Reduces out-of-proc task host IPC volume in -mt builds by introducing per-connection “full vs identical” transfer modes for build-invariant payloads (build process environment + CurrentSolutionConfigurationContents), and by avoiding redundant environment apply/restore work inside the task host.

Changes:

• Introduces packet-version-gated delta/identical wire forms for the task-host build process environment (forward + return paths) and for solution configuration contents (forward path).
• Adds TranslateDictionaryExcludingKeys support to keep large invariant global properties out of the per-task global-properties dictionary payload.
• Skips redundant per-task environment apply/restore in the task host when the next task’s environment is unchanged, and adds regression/unit tests covering the new wire format and aliasing edge cases.

Reviewed changes

Copilot reviewed 19 out of 19 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
src/Shared/TaskHostTaskComplete.cs	Adds environment “full vs identical” wire mode for the task completion packet.
src/Shared/TaskHostConfiguration.cs	Adds environment + solution-config delta fields/modes and excludes CurrentSolutionConfigurationContents from global-properties dictionary on the wire.
src/Shared/NodeEndpointOutOfProcBase.cs	Ensures outgoing packets use the negotiated (min) packet version when serializing.
src/MSBuild/OutOfProcTaskHostNode.cs	Reconstructs omitted invariants from per-connection baselines; skips redundant environment apply/restore between tasks.
src/Framework/MultiThreadedTaskEnvironmentDriver.cs	Guards against self-clearing when SetEnvironment is called with the driver’s own backing dictionary.
src/Framework/ITranslator.cs	Adds TranslateDictionaryExcludingKeys to the translator interface.
src/Framework/BinaryTranslator.cs	Implements TranslateDictionaryExcludingKeys in binary translators (write-side omits excluded keys).
src/Framework/BackEnd/NodePacketTypeExtensions.cs	Bumps packet version to 5 and documents the new delta capability.
src/Framework/BackEnd/CommunicationsUtilities.cs	Adds AreEnvironmentsEquivalent helper used to decide whether to send/apply/restore environment.
src/Build/Instance/TaskFactories/TaskHostTask.cs	Reconstructs “identical” returned environment from the sent config instead of re-receiving it.
src/Build/BackEnd/Components/Communications/NodeProviderOutOfProcBase.cs	Tracks per-connection baselines and marks outgoing configs as “identical” when invariants match.
src/Build.UnitTests/BackEnd/TranslationHelpers.cs	Adds helper to measure last serialization size for wire-size assertions.
src/Build.UnitTests/BackEnd/TaskHostTaskComplete_Tests.cs	Adds tests for v5 environment full/identical and legacy fallback behavior.
src/Build.UnitTests/BackEnd/TaskHostFactory_Tests.cs	Adds end-to-end regression tests for env reuse and env delta correctness in -mt.
src/Build.UnitTests/BackEnd/TaskHostConfiguration_Tests.cs	Adds tests for v5 environment delta and solution-config dedupe + legacy fallback.
src/Build.UnitTests/BackEnd/TaskEnvironment_Tests.cs	Adds regression test for the aliased-dictionary self-clear scenario.
src/Build.UnitTests/BackEnd/SetEnvironmentVariableTask.cs	Adds a test task that mutates environment inside the (possibly OOP) task process.
src/Build.UnitTests/BackEnd/ReadEnvironmentVariableTask.cs	Adds a test task that reads environment + reports PID for OOP validation.
src/Build.UnitTests/BackEnd/BinaryTranslator_Tests.cs	Adds unit tests proving excluded keys are omitted from the wire and round-trip behavior is correct.

[JanProvaznik]

I think the Global properties should be deduplicated for the general case rather than special casing the one found in the solution build

[OvesN]

Manual end-to-end tests passed:

Scenario: a .NET Framework MSBuild.exe (as used by Visual Studio) drives a Runtime="NET"
task, and that task runs in a .NET task host launched from the released .NET SDK 10.0.300 (not the
bootstrap).

• Sample project: src/Build.UnitTests/TestAssets/ExampleNetTask/TestNetTask/TestNetTask.csproj
  (declares TaskFactory="TaskHostFactory" Runtime="NET").
• The Runtime="NET" task it runs: ExampleTask (src/Build.UnitTests/TestAssets/ExampleNetTask/ExampleTask),
  which logs its process name, PID and full path so you can prove where it ran.

1. Pin the sample to released 10.0.300

TestNetTask ships a global.json that rolls forward to the newest SDK on the machine, so pin it to
10.0.300. in this global.json file:
.\artifacts\bin\Microsoft.Build.Engine.UnitTests\Debug\net10.0\TestAssets\ExampleNetTask\TestNetTask\global.json

2. Set the host environment

$env:DOTNET_HOST_PATH = "C:\Program Files\dotnet\dotnet.exe"
$env:DOTNET_ROOT      = "C:\Program Files\dotnet"

---

3. Run the sample with the Framework MSBuild.exe

-restore and the two -p: values are required by the sample (they let it locate ExampleTask.dll).

$proj   = ".\artifacts\bin\Microsoft.Build.Engine.UnitTests\Debug\net10.0\TestAssets\ExampleNetTask\TestNetTask\TestNetTask.csproj"
$asmLoc = ".\artifacts\bin\Microsoft.Build.Engine.UnitTests\Debug\net10.0"

& ".\artifacts\bin\bootstrap\net472\MSBuild\Current\Bin\MSBuild.exe" $proj `
    -restore -v:n -nologo `
    "-p:LatestDotNetCoreForMSBuild=net10.0" "-p:AssemblyLocation=$asmLoc"

Result:

1. Process path is under dotnet\sdk\10.0.300 → the task ran in the released 10.0.300 NET task
   host (not the bootstrap, not the parent process).
2. The build succeeded

[JanProvaznik]

since this is a node communication change with nontrivial blast radius, I'd prefer a second positive review from @AR-May or @rainersigwald

[AR-May]

Overall, I like the changes, this is a great PR. I have a couple of minor comments and one discussion point.

Since this change is risky, I'd also like to do an experimental insertion for it. As part of that insertion, I'd like to force the tasks to run in the task host to get additional test coverage. Assuming the tests pass, I'm approving this PR.

[AR-May]

Placing the Full/Identical decision here in DrainPacketQueue  has a bit of a layering smell for me. This loop is generic transport "write any packet's bytes to the pipe" but now it ends up knowing that a specific packet's payload can be delta-encoded.

I don't think it has to live here. The baseline is really just per-connection state, and the decision only needs to (a) use a per-connection baseline - so we do not mix different types of the task host. We need to be extra careful with that. And (b) happen before the config is serialized.

From another point of view, with this code here, we are extra sure we did not mix up baselines for the connection.

Option I'd consider: Compute the mode where the config is built/enqueued (e.g. in  TaskHostTask ), backed by a per-connection baseline held in the provider keyed by TaskHostNodeKey . This puts the payload knowledge in the layer that owns the config, and keeps the drain loop pure transport.

@JanProvaznik what do you think?

[OvesN]

I agree on smell, but honestly I'm a little afraid to move it outside  DrainPacketQueue. As you mentioned placing it here guarantees that we will not mix up baselines and it we can have it here without any locking