Remove legacy object-only messaging overloads in NServiceBus 11

[danielmarbach]

Goal

Remove the legacy object-only messaging overloads in NServiceBus 11 without allowing an existing call to silently change its routing behavior.

This issue now includes the removal previously tracked by #7892. A separate warning-only major release is not needed if NServiceBus 11 fails the build for calls where generic type inference could change the logical message type.

Why remove the overloads in NServiceBus 11

The object-only overloads determine the logical message type from message.GetType(). The generic overloads use typeof(T). Removing the object overloads changes which method an ordinary call selects, and these two types are not always the same.

For example:

ICommand message = new PlaceOrder();
await session.Send(message);

The object overload routes this call as PlaceOrder. After removal, the generic overload would infer ICommand and route it as ICommand.

An upgrade guide is not enough protection for this behavior change. NServiceBus 11 must report an error when the analyzer cannot prove that generic inference preserves the previous runtime-type routing. The application must then choose the intended logical message type explicitly.

This lets us remove the legacy overloads in one major release. Calls that are safe compile without changes, while calls that may behave differently cannot pass unnoticed.

NServiceBus 10.x

The work in #7889 remains unchanged:

• Object-only overloads remain available and continue to win overload resolution.
• NSB0039 identifies routing-equivalent migrations and offers a code fix such as Send<MyMessage>(new MyMessage()).
• NSB0040 identifies calls where moving to a typed overload may change routing or logical message identity. It has no code fix because the caller must make that decision.
• NSB0039 and NSB0040 run when the project enables trimming or NativeAOT, or when the project explicitly enables the migration audit.
• NSB0041 continues to report explicit Send<object>(...)-style calls for every project.

Since 10.3, the same diagnostics also cover the incoming logical-message and message-mutator paths, whose trimming-safe replacements landed with the strict message metadata work in #7918:

• The object-only IIncomingLogicalMessageContext.UpdateMessageInstance(object) invocation is analyzed like UpdateMessage(object), including the stricter same-instance and logical-message-identity cases.
• Property assignment to the logical mutator contexts (mutatorContext.Message = ... on MutateIncomingMessageContext, mutatorContext.OutgoingMessage = ... on MutateOutgoingMessageContext) is analyzed through simple-assignment analysis. NSB0039 offers a fix that rewrites safe assignments to the typed UpdateMessage<T>(...) / UpdateMessageInstance<T>(...) calls; unprovable assignments report NSB0040.
• NSB0041 does not apply to assignments. An object-typed assignment never receives a fixable NSB0039.

Users who do not enable trimming, NativeAOT, or the migration audit do not need to change their code in 10.x.

Users who migrate early may have explicit generic arguments that become unnecessary after upgrading to NServiceBus 11. Those calls remain valid and preserve the chosen logical message type. Removing a redundant type argument is optional.

NServiceBus 11 API changes

Remove the object-only overloads and the corresponding [OverloadResolutionPriority(-1)] attributes from:

• IMessageSession and MessageSession
• IPipelineContext
• IMessageProcessingContext
• Message-session and pipeline convenience extensions
• RunningEndpointInstance
• IOutgoingLogicalMessageContext and OutgoingLogicalMessageContext
• IIncomingLogicalMessageContext
• Related testing fakes, including TestableIncomingLogicalMessageContext

Also remove the instance-replacement setters from the logical mutator context classes:

• MutateIncomingMessageContext.Message (the setter)
• MutateOutgoingMessageContext.OutgoingMessage (the setter)

The getters remain for reading and in-place mutation of the current message. Instance replacement requires the typed UpdateMessage<T> / UpdateMessageInstance<T> methods or their object plus explicit Type overloads. Mutators that only mutate properties of the existing instance do not need the setter.

(The mutator contexts are concrete classes in the public IMutateIncomingMessages / IMutateOutgoingMessages signatures, so customers assign these setters directly. The setters were deprecated with warning in 10.3 using accessor-level obsoletion, which warns only on assignments and leaves the getters untouched. The 10.x analyzer diagnostics additionally risk-grade the remaining usages and offer code fixes until the setter path is removed with the object-overload removal batch.)

The generic overloads continue to route using typeof(T). Keep the overloads whose signatures accept both an object instance and an explicit, annotated Type. They are not legacy object-only overloads. Middleware and other dynamic code need them to state the logical message type after the compile-time generic type has been lost.

The default interface implementations added for 10.x binary compatibility currently delegate to the object-only overloads. NServiceBus 11 must replace that compatibility shape with the final typed contract. Use the object plus explicit Type overload as the canonical abstract interface operation, and keep the generic overload as a default interface method that forwards message and typeof(T) to it. This gives third-party implementations one operation to implement while supporting both generic and dynamically typed callers. Third-party implementations of the affected interfaces will need to implement that contract as part of the major-version upgrade.

NServiceBus 11 analyzer behavior

Calls that preserve behavior

When the analyzer can prove that the inferred generic type is the same as the runtime type used by the old overload, the call compiles normally:

await session.Send(new PlaceOrder());

No explicit generic argument is required.

Calls that may change behavior

When the runtime type can differ from the inferred generic type, report a new NServiceBus 11 diagnostic as an error by default. This includes interface, base-class, unsealed, nullable, proxy, and generic-forwarding scenarios, plus the stricter identity cases for UpdateMessage and UpdateMessageInstance.

Do not reuse NSB0040 for this error. A project may have suppressed NSB0040 while running the optional 10.x migration audit. Reusing the ID would carry that suppression into NServiceBus 11 and could disable the guard against a silent routing change. Retire NSB0040 with the object-only overloads and allocate a fresh diagnostic ID for ambiguous inferred generic calls.

The error must require an explicit decision:

• Use an explicit generic argument when the static contract is intended, for example Send<ICommand>(message).
• Use the overload that accepts an explicit Type when the logical message type is selected dynamically.

Do not offer an automatic code fix for the new diagnostic.

Explicit generic calls are deliberate and should not receive the new diagnostic. NSB0041 remains for explicit Send<object>(...)-style mistakes.

Analyzer coverage

The NServiceBus 11 analyzer must cover both ordinary invocations and method-group conversions. For example:

Func<object, SendOptions, CancellationToken, Task> send = session.Send;

In 10.x, this method group can bind to the object-only overload. After removal, it can bind to the generic overload with inferred T equal to object. The later invocation targets Func.Invoke, so invocation-only analysis cannot recover the original NServiceBus method selection.

The implementation and tests must cover:

• Method references, delegate creation, and reduced extension-method groups.
• Calls through concrete NServiceBus implementations, testing fakes, and interface implementations, rather than matching only methods declared directly on the known interfaces.
• Generic forwarding methods and other type-parameter scenarios.
• Inferred T equal to object versus an explicit Send<object>(...). The first requires the new ambiguity error; the second remains NSB0041. Roslyn exposes T as object in both cases, so the analyzer must also inspect invocation syntax.

Property assignment to the mutator context setters is covered in 10.x by NSB0039/NSB0040 through simple-assignment analysis. It needs no NServiceBus 11 diagnostic because the setters themselves are removed; un-upgraded code fails to compile against the removed setter.

Expression trees written as lambdas contain an invocation and remain covered by invocation analysis. C# does not allow direct conversion of a method group to Expression<TDelegate>. Reflection-built expression trees and analyzer-disabled builds are outside this compile-time guard.

Diagnostic lifecycle

• Remove NSB0039 with the object-only overloads because safe calls bind naturally to the generic API.
• Retire NSB0040 with the 10.x object-overload migration diagnostic, including its setter-assignment coverage.
• Add a fresh diagnostic ID, enabled as an error by default, for ambiguous inferred generic calls in NServiceBus 11.
• Retain NSB0041.

Projects can still suppress or reconfigure analyzer diagnostics, but the default upgrade path must not allow a routing change to remain hidden.

Acceptance criteria

• All listed object-only overloads are removed in NServiceBus 11.
• The instance-replacement setters of MutateIncomingMessageContext and MutateOutgoingMessageContext are removed; the typed UpdateMessage<T> / UpdateMessageInstance<T> methods and their object plus explicit Type overloads remain, and the getters stay readable.
• Matching generic overloads no longer carry [OverloadResolutionPriority(-1)].
• Generic overloads consistently use typeof(T) as the logical message type.
• The object plus explicit Type overload is the canonical abstract interface operation, and the generic default interface method forwards to it using typeof(T).
• Compatibility default interface bodies no longer delegate to removed object-only overloads.
• Provably safe inferred generic calls compile without requiring an explicit type argument.
• A fresh diagnostic ID is an error by default for inferred calls where runtime-type routing and static-type routing may differ.
• Method-group and delegate conversions cannot bypass the ambiguity diagnostic.
• Inferred T equal to object receives the new ambiguity diagnostic, while explicit Send<object>(...) remains NSB0041.
• Explicit generic and explicit Type calls are recognized as deliberate routing choices.
• UpdateMessage and UpdateMessageInstance retain their more conservative analysis for same-instance and logical-message-identity cases.
• NSB0039 and NSB0040 are removed, and NSB0041 remains.
• Public API approvals, analyzer tests, acceptance tests, testing fakes, and upgrade documentation are updated.
• Third-party interface implementation scenarios are covered by compile-time and binary-compatibility tests appropriate for the major-version change.

Out of scope

• Changing the NServiceBus 10.x rollout in Add trimming-safe messaging overloads and migration analyzer #7889 or the mutator setter analyzer additions in 10.3.
• Transport body mutators, which operate on ReadOnlyMemory<byte> and do not lose CLR type information.