diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index fdf7ee4..ff39803 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -42,11 +42,11 @@ jobs: uses: actions/cache@v6 with: path: ~/.nuget/packages - key: nuget-${{ runner.os }}-${{ hashFiles('**/*.csproj', '**/Directory.*.props', 'global.json') }} + key: nuget-${{ runner.os }}-${{ hashFiles('**/*.csproj', '**/Directory.*.props', '**/packages.lock.json', 'global.json') }} restore-keys: nuget-${{ runner.os }}- - name: Restore - run: dotnet restore + run: dotnet restore -p:RestoreLockedMode=true - name: Build run: dotnet build --no-restore -c Release diff --git a/.gitignore b/.gitignore index 51b4c07..06e30f3 100644 --- a/.gitignore +++ b/.gitignore @@ -1,7 +1,5 @@ -# .gitignore — RequestFlow -# Modern SDK-style .NET library solution (.slnx), multi-target -# netstandard2.0;net8.0;net10.0. NuGet-published. No project.json / -# packages.config legacy cruft on purpose. +# RequestFlow: SDK-style .NET library solution (.slnx), NuGet-published. +# No project.json / packages.config legacy cruft on purpose. # ── Build output ───────────────────────────────────────────── [Bb]in/ @@ -24,17 +22,13 @@ artifacts/ !.vscode/launch.json !.vscode/extensions.json -# ── Local build settings (per-machine) ─────────────────────── -# Optional override file. Directory.Build.props imports it when it exists. -Directory.Local.props - # ── NuGet package output ───────────────────────────────────── # Packing RequestFlow / RequestFlow.Cqrs. Symbols too (.snupkg). *.nupkg *.snupkg # ── Test results & coverage ────────────────────────────────── -# Unit test project + coverage (v1 checklist #7). +# Unit test project + coverage. [Tt]est[Rr]esults/ *.trx *.coverage @@ -44,7 +38,7 @@ coverage*.xml coverage*.info # ── BenchmarkDotNet ────────────────────────────────────────── -# Dispatch-cost benchmarks vs MediatR (plan §"Targets and quality bar"). +# Dispatch-cost benchmarks vs MediatR. BenchmarkDotNet.Artifacts/ # ── MSBuild / binary logs ──────────────────────────────────── diff --git a/CHANGELOG.md b/CHANGELOG.md index 0493ef3..0282210 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,6 +6,20 @@ Pre-1.0: the public API can still change between previews. The `release` workflow uses the section matching the pushed tag as the GitHub Release body. A missing section fails the build before publishing. Before tagging, rename `[Unreleased]` to the release version and add its date. +## [Unreleased] + +### Added + +- [Benchmark suites](benchmarks/README.md) on .NET 10. [`RequestFlow.Benchmarks.MediatR`](benchmarks/RequestFlow.Benchmarks.MediatR/README.md) compares requests, void requests, streams, events, catalog size, mixed traffic, and short-circuit pipelines against MediatR 12.5.0, the last Apache-2.0 release. `RequestFlow.Benchmarks` compares RequestFlow's Task and ValueTask paths and stage chains. + +### Changed + +- The MediatR notes in [events.md](docs/events.md#notes-for-mediatr-migrations) describe MediatR 12.5.0 probes only. + +### Performance + +- Dispatch allocates less for a stream sent without a cancelable token and for a `ParallelPublishStrategy` event with one applicable handler. Exception types, parameter names, and synchronous throws are unchanged. + ## [1.0.0-preview.9] - 2026-09-08 ### Added diff --git a/Directory.Build.props b/Directory.Build.props index 10a6d7f..637e008 100644 --- a/Directory.Build.props +++ b/Directory.Build.props @@ -23,7 +23,4 @@ - - - diff --git a/README.md b/README.md index 551c3af..98cca68 100644 --- a/README.md +++ b/README.md @@ -88,6 +88,10 @@ Void handlers return plain `Task`; `Unit` does not appear in user code. Existing The [MediatR migration guide](https://github.com/illia1f/RequestFlow/blob/main/docs/migrating-from-mediatr.md) covers the file-by-file sequence, event differences, conditional registration, and unsupported extension points. +## Benchmarks + +The [MediatR comparison](https://github.com/illia1f/RequestFlow/blob/main/benchmarks/RequestFlow.Benchmarks.MediatR/README.md) measures requests, stages, streams, events, and short-circuit paths against MediatR 12.5.0 on .NET 10, with correctness probes and package lock files. Its results describe MediatR 12.5.0 only, not later releases. The [benchmark guide](https://github.com/illia1f/RequestFlow/blob/main/benchmarks/README.md) explains how to run the suites. + ## Packages - **[`RequestFlow.Abstractions`](https://www.nuget.org/packages/RequestFlow.Abstractions)** holds Task and ValueTask requests, handlers, dispatchers, stages, streaming, events, publish strategies, validation models, and exceptions. It has no package dependency on `net8.0` or `net10.0`. diff --git a/ROADMAP.md b/ROADMAP.md index 36ca6e9..21562d7 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -22,7 +22,7 @@ Release: [1.0.0-preview.9](CHANGELOG.md#100-preview9---2026-09-08). - [x] Streaming requests via `IAsyncEnumerable` - [x] Opt-in ValueTask request family: `IValueRequest`, ValueTask stages, and CQRS command/query twins - [x] MediatR migration and target compatibility guides -- [ ] BenchmarkDotNet suite against pinned MediatR, martinothamar/Mediator, LiteBus, and DispatchR packages, with raw results +- [ ] BenchmarkDotNet suite against pinned MediatR, martinothamar/Mediator, and DispatchR packages, with raw results Targets: `netstandard2.0;net462;net8.0;net10.0`. The `net462` target has shipped since `1.0.0-preview.2`. diff --git a/RequestFlow.slnx b/RequestFlow.slnx index 56ff1cb..e8fb28d 100644 --- a/RequestFlow.slnx +++ b/RequestFlow.slnx @@ -1,4 +1,12 @@ + + + + + + + + diff --git a/benchmarks/Directory.Build.props b/benchmarks/Directory.Build.props new file mode 100644 index 0000000..0089dcf --- /dev/null +++ b/benchmarks/Directory.Build.props @@ -0,0 +1,11 @@ + + + + false + false + true + + + + + diff --git a/benchmarks/Directory.Build.targets b/benchmarks/Directory.Build.targets new file mode 100644 index 0000000..39767e8 --- /dev/null +++ b/benchmarks/Directory.Build.targets @@ -0,0 +1,8 @@ + + + + + + + diff --git a/benchmarks/README.md b/benchmarks/README.md new file mode 100644 index 0000000..f68971e --- /dev/null +++ b/benchmarks/README.md @@ -0,0 +1,139 @@ +# Benchmarks + +Two BenchmarkDotNet suites, both on .NET 10: + +| Suite | Cases | Measures | +| --- | --- | --- | +| `RequestFlow.Benchmarks.MediatR` | 224 | RequestFlow against MediatR 12.5.0 in one executable, with matching workloads and service lifetimes. See the [MediatR comparison](RequestFlow.Benchmarks.MediatR/README.md). | +| `RequestFlow.Benchmarks` | 34 | RequestFlow alone: Task vs ValueTask paths and stage chains. | + +- A case is one benchmark method with one parameter set. +- No timing results are published yet. +- RequestFlow is built from the checked-out source, so record the commit when sharing results. +- BenchmarkDotNet 0.15.8 and Microsoft DI 10.0.9. Package lock files pin the resolved dependencies. +- CI builds both suites but does not run them. + +## Quick start + +Requires the .NET 10 SDK. Run from the repository root: + +```powershell +dotnet restore benchmarks/RequestFlow.Benchmarks.MediatR -p:RestoreLockedMode=true +dotnet build benchmarks/RequestFlow.Benchmarks.MediatR -c Release --no-restore + +# Check results, execution order, lifetimes, tokens, and disposal without timing. +dotnet run --project benchmarks/RequestFlow.Benchmarks.MediatR -c Release -f net10.0 --no-build --no-restore -- probe + +# Rough timings for one group. Not publishable. +dotnet run --project benchmarks/RequestFlow.Benchmarks.MediatR -c Release -f net10.0 --no-build --no-restore -- --quick --filter '*TaskRetainedScopeBenchmarks*' + +# Full run of every case into an unused directory. +dotnet run --project benchmarks/RequestFlow.Benchmarks.MediatR -c Release -f net10.0 --no-build --no-restore -- --filter '*' --artifacts benchmarks/artifacts/YYYY-MM-DD-mediatr-full +``` + +- These commands build only the suite and the `net10.0` build of RequestFlow. To build the whole solution, run `dotnet restore -p:RestoreLockedMode=true` and `dotnet build -c Release --no-restore`. +- Single-quoted filters work in PowerShell and bash. In cmd.exe, use double quotes. + +## RequestFlow suite + +```powershell +dotnet restore benchmarks/RequestFlow.Benchmarks -p:RestoreLockedMode=true +dotnet build benchmarks/RequestFlow.Benchmarks -c Release --no-restore + +# Verify the workloads without timing. +dotnet run --project benchmarks/RequestFlow.Benchmarks -c Release -f net10.0 --no-build --no-restore -- probe + +# Rough timings for Task and ValueTask void dispatch. +dotnet run --project benchmarks/RequestFlow.Benchmarks -c Release -f net10.0 --no-build --no-restore -- --quick --filter '*VoidCompletionBenchmarks*' +``` + +- For a full run, use `--filter '*'` without `--quick`. +- Each class uses the Task path as the baseline for the ValueTask path. +- `ChainLevelBenchmarks` bypasses the dispatcher. It enters the Task, ValueTask, and void plans that `AddRequestFlow` froze with 0, 1, or 5 stages. +- Void completion and short-circuit benchmarks dispatch through RequestFlow. +- Rows use the shared fixture handlers and stages, so the handler work matches the comparison suite. + +## Run modes + +| Argument | What it does | +| --- | --- | +| `probe` | Checks every workload without timing. A pass ends with `Probe passed:` and the number of classes, parameter sets, and cases checked. | +| `--smoke` | Runs each case once to check it executes. | +| `--quick` | Rough timings. Not publishable. | +| `--filter` without `--smoke` or `--quick` | Runs `probe`, then the full profile: two launches, 15 to 50 warm-up and 60 to 100 measured iterations, 5% relative error target. | + +- Without arguments, `dotnet run` applies the MediatR suite's first launch profile, **Verify comparison**, so it only runs `probe`. +- The RequestFlow suite has no launch profile. Without arguments it runs `probe`, then shows BenchmarkDotNet's selection prompt. + +## Filters + +- `--filter '*'` selects every case. +- When narrowing, keep both libraries selected within each scenario. +- Pass several patterns to one `--filter` to combine scenarios: `--filter '*MediatRStream*' '*MediatRVoidRequest*'`. + +| MediatR scenario | Filter | Cases | +| --- | --- | --- | +| Task requests | `*MediatRTask*Scope*` | 36 | +| Void requests | `*MediatRVoidRequest*` | 24 | +| Streams | `*MediatRStream*` | 72 | +| Sequential events | `*MediatREventSequential*` | 16 | +| Parallel events | `*MediatREventParallel*` | 16 | +| Catalog size | `*MediatRCatalogSize*` | 18 | +| Mixed traffic | `*MediatRMixedTraffic*` | 18 | +| Short circuit | `*MediatRShortCircuit*` | 18 | +| Direct-handler reference | `*MediatRTaskDirectHandler*` | 6 | + +## Reports + +- `--artifacts ` sets the output directory. Without it, reports go to `BenchmarkDotNet.Artifacts////` under the current directory, where `` is `mediatr` or `requestflow`. +- Readable tables are the `results/*-report-github.md` files. `run-manifest.json` records the run status and host details. +- Git ignores `BenchmarkDotNet.Artifacts/` and `benchmarks/artifacts/`. + +## Publishing a run + +Publish only a run whose `run-manifest.json` shows `status: completed` and `publicationCandidate: true`. That requires: + +- the full profile, with no measurement overrides +- a passing probe +- Git can read HEAD and the status of every build input +- a clean Git status for every build input, with HEAD and that status unchanged from start to end of the run +- no `DOTNET_*`, `COMPlus_*`, or `CORECLR_*` environment variables, except the SDK and host settings the harness ignores + +Before the run: + +- Build the suite after your last commit. A full run fails before timing anything when the build's commit differs from HEAD. +- Delete stray files under `src/`, the suite folder, and each benchmark project the suite references, such as old reports and `.csproj.user` files. The source check includes ignored files and skips only `bin` and `obj`. +- Start without a processor affinity mask. The jobs set none, but child processes inherit any mask set with `start /affinity`, `taskset`, or a similar tool. +- Leave the machine otherwise idle. + +After the run: + +- Check that `hostEnvironment.processorCount` in the manifest equals the logical core count in the report header. `publicationCandidate` does not check affinity, and parallel event and suspending rows depend on how many cores the process can use. +- A run stopped with Ctrl+C stays at `status: running` and is not publishable. + +## Visual Studio + +Use Visual Studio for exploration only. Publish from the command line. + +1. Open `RequestFlow.slnx` in a Visual Studio version that supports the .NET 10 SDK and `.slnx` solutions. +2. In the `benchmarks` solution folder, set `RequestFlow.Benchmarks.MediatR` as the startup project. +3. Select the **Release** configuration. +4. Pick a launch profile: **Verify comparison** (`probe`), **Smoke test** (`--smoke` on the Task retained-scope class only), or **Full comparison** (`--filter *`). +5. Start without debugging (**Ctrl+F5**). + +- Reports go under `BenchmarkDotNet.Artifacts/mediatr/` at the repository root, because the profiles set `workingDirectory` to `../..`. `dotnet run` ignores that setting. +- A Visual Studio run is never a publication candidate. Visual Studio saves the selected profile in `RequestFlow.Benchmarks.MediatR.csproj.user`, which the source check flags. + +## Projects + +All five are in the root solution. + +| Project | Contents | +| --- | --- | +| `RequestFlow.Benchmarks` | Stage-chain measurements, Task vs ValueTask dispatch, and probes | +| `RequestFlow.Benchmarks.MediatR` | MediatR fixtures, paired benchmarks, and probes. Does not depend on `RequestFlow.Benchmarks`. | +| `RequestFlow.Benchmarks.Harness` | BenchmarkDotNet configuration, runner, run manifest, and provenance records. Knows nothing about workloads. | +| `RequestFlow.Benchmarks.Core` | Probe engine, trace names, provider hosts, shared workloads, and the short-circuit dependency graph | +| `RequestFlow.Benchmarks.RequestFlowFixtures` | RequestFlow handlers, stages, registration, and message catalogs | + +- Benchmark classes are public and inheritable so BenchmarkDotNet can generate its harness. diff --git a/benchmarks/RequestFlow.Benchmarks.Core/Catalog/RuntimeCatalog.cs b/benchmarks/RequestFlow.Benchmarks.Core/Catalog/RuntimeCatalog.cs new file mode 100644 index 0000000..0dc5632 --- /dev/null +++ b/benchmarks/RequestFlow.Benchmarks.Core/Catalog/RuntimeCatalog.cs @@ -0,0 +1,172 @@ +namespace RequestFlow.Benchmarks.Core; + +public static class RuntimeCatalog +{ + /// + /// The number of sends in one mixed-traffic invocation. + /// + public const int MixedTrafficBatch = 64; + + private const int MixedTrafficSeed = 20260926; + + private static readonly int[] Sizes = [4, 16, 64]; + + public static Type[] Tags(int count) + { + if (!Sizes.Contains(count)) + throw new ArgumentOutOfRangeException(nameof(count)); + + return AllTags.Take(count).ToArray(); + } + + /// + /// Returns the tag of every send in one mixed-traffic invocation over a catalog of types. + /// Each tag appears equally often, in an order shuffled once with a fixed seed. + /// + public static IReadOnlyList MixedTrafficOrder(int count) + => MixedTrafficOrders.TryGetValue(count, out Type[]? order) ? order : throw new ArgumentOutOfRangeException(nameof(count)); + + private static readonly Type[] AllTags = + [ + typeof(RuntimeTag1), + typeof(RuntimeTag2), + typeof(RuntimeTag3), + typeof(RuntimeTag4), + typeof(RuntimeTag5), + typeof(RuntimeTag6), + typeof(RuntimeTag7), + typeof(RuntimeTag8), + typeof(RuntimeTag9), + typeof(RuntimeTag10), + typeof(RuntimeTag11), + typeof(RuntimeTag12), + typeof(RuntimeTag13), + typeof(RuntimeTag14), + typeof(RuntimeTag15), + typeof(RuntimeTag16), + typeof(RuntimeTag17), + typeof(RuntimeTag18), + typeof(RuntimeTag19), + typeof(RuntimeTag20), + typeof(RuntimeTag21), + typeof(RuntimeTag22), + typeof(RuntimeTag23), + typeof(RuntimeTag24), + typeof(RuntimeTag25), + typeof(RuntimeTag26), + typeof(RuntimeTag27), + typeof(RuntimeTag28), + typeof(RuntimeTag29), + typeof(RuntimeTag30), + typeof(RuntimeTag31), + typeof(RuntimeTag32), + typeof(RuntimeTag33), + typeof(RuntimeTag34), + typeof(RuntimeTag35), + typeof(RuntimeTag36), + typeof(RuntimeTag37), + typeof(RuntimeTag38), + typeof(RuntimeTag39), + typeof(RuntimeTag40), + typeof(RuntimeTag41), + typeof(RuntimeTag42), + typeof(RuntimeTag43), + typeof(RuntimeTag44), + typeof(RuntimeTag45), + typeof(RuntimeTag46), + typeof(RuntimeTag47), + typeof(RuntimeTag48), + typeof(RuntimeTag49), + typeof(RuntimeTag50), + typeof(RuntimeTag51), + typeof(RuntimeTag52), + typeof(RuntimeTag53), + typeof(RuntimeTag54), + typeof(RuntimeTag55), + typeof(RuntimeTag56), + typeof(RuntimeTag57), + typeof(RuntimeTag58), + typeof(RuntimeTag59), + typeof(RuntimeTag60), + typeof(RuntimeTag61), + typeof(RuntimeTag62), + typeof(RuntimeTag63), + typeof(RuntimeTag64), + ]; + + private static readonly Dictionary MixedTrafficOrders = Sizes.ToDictionary(size => size, CreateMixedTrafficOrder); + + private static Type[] CreateMixedTrafficOrder(int count) + { + Type[] order = [.. Enumerable.Repeat(Tags(count), MixedTrafficBatch / count).SelectMany(tags => tags)]; + + new Random(MixedTrafficSeed).Shuffle(order); + + return order; + } +} + +public sealed class RuntimeTag1; +public sealed class RuntimeTag2; +public sealed class RuntimeTag3; +public sealed class RuntimeTag4; +public sealed class RuntimeTag5; +public sealed class RuntimeTag6; +public sealed class RuntimeTag7; +public sealed class RuntimeTag8; +public sealed class RuntimeTag9; +public sealed class RuntimeTag10; +public sealed class RuntimeTag11; +public sealed class RuntimeTag12; +public sealed class RuntimeTag13; +public sealed class RuntimeTag14; +public sealed class RuntimeTag15; +public sealed class RuntimeTag16; +public sealed class RuntimeTag17; +public sealed class RuntimeTag18; +public sealed class RuntimeTag19; +public sealed class RuntimeTag20; +public sealed class RuntimeTag21; +public sealed class RuntimeTag22; +public sealed class RuntimeTag23; +public sealed class RuntimeTag24; +public sealed class RuntimeTag25; +public sealed class RuntimeTag26; +public sealed class RuntimeTag27; +public sealed class RuntimeTag28; +public sealed class RuntimeTag29; +public sealed class RuntimeTag30; +public sealed class RuntimeTag31; +public sealed class RuntimeTag32; +public sealed class RuntimeTag33; +public sealed class RuntimeTag34; +public sealed class RuntimeTag35; +public sealed class RuntimeTag36; +public sealed class RuntimeTag37; +public sealed class RuntimeTag38; +public sealed class RuntimeTag39; +public sealed class RuntimeTag40; +public sealed class RuntimeTag41; +public sealed class RuntimeTag42; +public sealed class RuntimeTag43; +public sealed class RuntimeTag44; +public sealed class RuntimeTag45; +public sealed class RuntimeTag46; +public sealed class RuntimeTag47; +public sealed class RuntimeTag48; +public sealed class RuntimeTag49; +public sealed class RuntimeTag50; +public sealed class RuntimeTag51; +public sealed class RuntimeTag52; +public sealed class RuntimeTag53; +public sealed class RuntimeTag54; +public sealed class RuntimeTag55; +public sealed class RuntimeTag56; +public sealed class RuntimeTag57; +public sealed class RuntimeTag58; +public sealed class RuntimeTag59; +public sealed class RuntimeTag60; +public sealed class RuntimeTag61; +public sealed class RuntimeTag62; +public sealed class RuntimeTag63; +public sealed class RuntimeTag64; diff --git a/benchmarks/RequestFlow.Benchmarks.Core/Hosting/RetainedScopeHost.cs b/benchmarks/RequestFlow.Benchmarks.Core/Hosting/RetainedScopeHost.cs new file mode 100644 index 0000000..a34a624 --- /dev/null +++ b/benchmarks/RequestFlow.Benchmarks.Core/Hosting/RetainedScopeHost.cs @@ -0,0 +1,31 @@ +using Microsoft.Extensions.DependencyInjection; + +namespace RequestFlow.Benchmarks.Core; + +/// +/// A provider with one retained scope and the service resolved from it. +/// +public sealed class RetainedScopeHost : IDisposable + where TService : notnull +{ + private readonly ServiceProvider _provider; + private readonly IServiceScope _scope; + + /// + /// The service resolved from the retained scope. + /// + public readonly TService Service; + + public RetainedScopeHost(ServiceProvider provider) + { + _provider = provider; + _scope = provider.CreateScope(); + Service = _scope.ServiceProvider.GetRequiredService(); + } + + public void Dispose() + { + _scope.Dispose(); + _provider.Dispose(); + } +} diff --git a/benchmarks/RequestFlow.Benchmarks.Core/Hosting/ScopePerOperationHost.cs b/benchmarks/RequestFlow.Benchmarks.Core/Hosting/ScopePerOperationHost.cs new file mode 100644 index 0000000..b00aa21 --- /dev/null +++ b/benchmarks/RequestFlow.Benchmarks.Core/Hosting/ScopePerOperationHost.cs @@ -0,0 +1,13 @@ +using Microsoft.Extensions.DependencyInjection; + +namespace RequestFlow.Benchmarks.Core; + +/// +/// A provider whose benchmark methods create and dispose a scope per operation. +/// +public sealed class ScopePerOperationHost(ServiceProvider provider) : IDisposable +{ + public readonly ServiceProvider Provider = provider; + + public void Dispose() => Provider.Dispose(); +} diff --git a/benchmarks/RequestFlow.Benchmarks.Core/RequestFlow.Benchmarks.Core.csproj b/benchmarks/RequestFlow.Benchmarks.Core/RequestFlow.Benchmarks.Core.csproj new file mode 100644 index 0000000..175552f --- /dev/null +++ b/benchmarks/RequestFlow.Benchmarks.Core/RequestFlow.Benchmarks.Core.csproj @@ -0,0 +1,18 @@ + + + + net10.0 + enable + enable + + + + + + + + + + + + diff --git a/benchmarks/RequestFlow.Benchmarks.Core/ShortCircuit/DependencyComponent.cs b/benchmarks/RequestFlow.Benchmarks.Core/ShortCircuit/DependencyComponent.cs new file mode 100644 index 0000000..1820e9c --- /dev/null +++ b/benchmarks/RequestFlow.Benchmarks.Core/ShortCircuit/DependencyComponent.cs @@ -0,0 +1,28 @@ +namespace RequestFlow.Benchmarks.Core; + +public sealed class DependencyState(ScopeWitness scope, ShortCircuitSettings settings) +{ + public ScopeWitness Scope { get; } = scope; + public ShortCircuitSettings Settings { get; } = settings; +} + +public abstract class DependencyComponent : WorkloadComponent +{ + public DependencyState State { get; } + public string Role { get; } + + protected DependencyComponent(DependencyState state, string role) : base(state.Scope) + { + State = state; + Role = role; + state.Settings.Trace?.Constructed.Add(this); + } + + protected void Enter(ShortCircuitMessage message, CancellationToken token) => Observe(message, Role, token); + + public override void Dispose() + { + State.Settings.Trace?.Disposed.Add(this); + base.Dispose(); + } +} diff --git a/benchmarks/RequestFlow.Benchmarks.Core/ShortCircuit/HandlerDependencies.cs b/benchmarks/RequestFlow.Benchmarks.Core/ShortCircuit/HandlerDependencies.cs new file mode 100644 index 0000000..a7a02c2 --- /dev/null +++ b/benchmarks/RequestFlow.Benchmarks.Core/ShortCircuit/HandlerDependencies.cs @@ -0,0 +1,27 @@ +namespace RequestFlow.Benchmarks.Core; + +public sealed class CommandBuffer(DependencyState state) : DependencyComponent(state, "command-buffer") +{ + public byte[] Bytes { get; } = state.Settings.BufferBytes == 0 + ? Array.Empty() + : new byte[state.Settings.BufferBytes]; +} + +public sealed class DataSession(DependencyState state, CommandBuffer buffer) + : DependencyComponent(state, "data-session") +{ + public int Capacity => buffer.Bytes.Length; +} + +public sealed class Repository(DependencyState state, DataSession session) : DependencyComponent(state, "repository") +{ + public Task Read() + => session.Capacity >= 0 + ? ShortCircuitWorkload.Handled + : throw new InvalidOperationException(); + + public ValueTask ReadValue() + => session.Capacity >= 0 + ? new(ShortCircuitWorkload.HandledResult) + : throw new InvalidOperationException(); +} diff --git a/benchmarks/RequestFlow.Benchmarks.Core/ShortCircuit/ShortCircuitMessage.cs b/benchmarks/RequestFlow.Benchmarks.Core/ShortCircuit/ShortCircuitMessage.cs new file mode 100644 index 0000000..0c6e0f2 --- /dev/null +++ b/benchmarks/RequestFlow.Benchmarks.Core/ShortCircuit/ShortCircuitMessage.cs @@ -0,0 +1,13 @@ +namespace RequestFlow.Benchmarks.Core; + +public enum ShortCircuitOutcome +{ + AuthorizationReject, + CacheHit, + FullExecution +} + +public class ShortCircuitMessage(ShortCircuitOutcome outcome) : WorkloadMessage +{ + public ShortCircuitOutcome Outcome { get; } = outcome; +} diff --git a/benchmarks/RequestFlow.Benchmarks.Core/ShortCircuit/ShortCircuitSettings.cs b/benchmarks/RequestFlow.Benchmarks.Core/ShortCircuit/ShortCircuitSettings.cs new file mode 100644 index 0000000..5d83fbf --- /dev/null +++ b/benchmarks/RequestFlow.Benchmarks.Core/ShortCircuit/ShortCircuitSettings.cs @@ -0,0 +1,9 @@ +namespace RequestFlow.Benchmarks.Core; + +public sealed record ShortCircuitSettings(int BufferBytes, ConstructionTrace? Trace = null); + +public sealed class ConstructionTrace +{ + public List Constructed { get; } = []; + public List Disposed { get; } = []; +} diff --git a/benchmarks/RequestFlow.Benchmarks.Core/ShortCircuit/ShortCircuitVerification.cs b/benchmarks/RequestFlow.Benchmarks.Core/ShortCircuit/ShortCircuitVerification.cs new file mode 100644 index 0000000..bd48d71 --- /dev/null +++ b/benchmarks/RequestFlow.Benchmarks.Core/ShortCircuit/ShortCircuitVerification.cs @@ -0,0 +1,151 @@ +using Microsoft.Extensions.DependencyInjection; + +namespace RequestFlow.Benchmarks.Core; + +public enum DependencyResolution +{ + EnteredOnly, + AllStages, + EntireChain +} + +public static class ShortCircuitVerification +{ + public static void Verify( + string library, + Func build, + Func createMessage, + Func> send, + DependencyResolution resolution, + bool childScopePerSend = false, + int authorizationDisposals = 1, + ServiceLifetime dispatcherLifetime = ServiceLifetime.Scoped) + where TDispatcher : notnull + where TMessage : ShortCircuitMessage + { + foreach (int bufferBytes in new[] { 0, 4096 }) + foreach (ShortCircuitOutcome outcome in Enum.GetValues()) + { + var construction = new ConstructionTrace(); + using ServiceProvider provider = build(bufferBytes, construction); + + var prior = new List(); + using var cancellation = new CancellationTokenSource(); + + for (int scopeIndex = 0; scopeIndex < 2; scopeIndex++) + { + int scopeStart = construction.Constructed.Count; + + using (IServiceScope scope = provider.CreateScope()) + { + TDispatcher dispatcher = scope.ServiceProvider.GetRequiredService(); + + Require( + ReferenceEquals(dispatcher, scope.ServiceProvider.GetRequiredService()) + == ScopeLifetimeVerification.SharesWithinScope(dispatcherLifetime), + "dispatcher lifetime"); + + for (int sendIndex = 0; sendIndex < 3; sendIndex++) + { + ShortCircuitOutcome selected = sendIndex == 2 ? ShortCircuitOutcome.FullExecution : outcome; + var trace = new ExecutionTrace(); + TMessage message = createMessage(selected); + message.Trace = trace; + + int start = construction.Constructed.Count; + ValueTask pending = send(dispatcher, message, cancellation.Token); + + Require(pending.IsCompletedSuccessfully, "workload did not complete synchronously"); + + int result = pending.GetAwaiter().GetResult(); + BenchmarkExpectation expectation = ShortCircuitWorkload.Expectation(selected); + + Require(result == expectation.Result, "response"); + + Require(trace.Entries.Select(entry => entry.Name).SequenceEqual(expectation.Names), "execution order"); + Require(trace.Entries.All(entry => entry.Token == cancellation.Token), "cancellation token"); + + var activated = construction.Constructed.Skip(childScopePerSend ? start : scopeStart).ToArray(); + string[] expected = ExpectedRoles(selected, resolution); + + Require( + activated.Select(component => component.Role).Order().SequenceEqual(expected.Order()), + "activation: " + string.Join(", ", activated.Select(component => component.Role))); + + Require( + activated.Select(component => component.State).Distinct().Count() == 1, + "dependency scope identity"); + + Require( + trace.Entries.All(entry => activated.Contains(entry.Component)), + "executed untracked component"); + + Require(!activated.Any(prior.Contains), "component crossed a scope boundary"); + + Require( + !activated.Any(component => + prior.Any(previous => ReferenceEquals(previous.State, component.State))), + "dependency state crossed a scope boundary"); + + if (!childScopePerSend && sendIndex == 1) + Require(construction.Constructed.Count == start, "scoped component was reconstructed"); + + Require( + activated.All(component => component.IsDisposed == childScopePerSend), + "premature or missing disposal"); + + if (childScopePerSend) + prior.AddRange(activated); + + foreach (DependencyComponent component in activated) + { + if (component is ChangeBuffer change) + Require(change.Bytes.Length == bufferBytes, "transaction buffer size"); + + if (component is CommandBuffer command) + Require(command.Bytes.Length == bufferBytes, "handler buffer size"); + } + } + } + + DependencyComponent[] completed = construction.Constructed.Skip(scopeStart).ToArray(); + + foreach (DependencyComponent component in completed) + { + Require(component.IsDisposed && component.Scope.IsDisposed, "scope disposal"); + + int expectedCount = component.Role == "authorization" ? authorizationDisposals : 1; + + Require( + construction.Disposed.Count(item => ReferenceEquals(item, component)) == expectedCount, + "disposal count: " + component.Role); + } + + if (!childScopePerSend) + prior.AddRange(completed); + } + + void Require(bool condition, string detail) + { + if (!condition) + throw new InvalidOperationException($"{library} short circuit ({outcome}, buffer={bufferBytes}): {detail}."); + } + } + } + + private static string[] ExpectedRoles(ShortCircuitOutcome outcome, DependencyResolution resolution) + { + string[] authorization = ["access-policy", "authorization"]; + string[] cache = [.. authorization, "cache-policy", "cache"]; + string[] stages = [.. cache, "change-buffer", "unit-of-work", "transaction"]; + string[] full = [.. stages, "command-buffer", "data-session", "repository", TraceNames.Handler]; + + if (outcome == ShortCircuitOutcome.FullExecution || resolution == DependencyResolution.EntireChain) + return full; + + if (resolution == DependencyResolution.AllStages) + return stages; + + return outcome == ShortCircuitOutcome.AuthorizationReject ? authorization : cache; + } +} diff --git a/benchmarks/RequestFlow.Benchmarks.Core/ShortCircuit/ShortCircuitWorkload.cs b/benchmarks/RequestFlow.Benchmarks.Core/ShortCircuit/ShortCircuitWorkload.cs new file mode 100644 index 0000000..9976508 --- /dev/null +++ b/benchmarks/RequestFlow.Benchmarks.Core/ShortCircuit/ShortCircuitWorkload.cs @@ -0,0 +1,45 @@ +using Microsoft.Extensions.DependencyInjection; + +namespace RequestFlow.Benchmarks.Core; + +public static class ShortCircuitWorkload +{ + public const int RejectedResult = 71; + public const int CachedResult = 73; + public const int HandledResult = 79; + + public static readonly Task Rejected = Task.FromResult(RejectedResult); + public static readonly Task Cached = Task.FromResult(CachedResult); + public static readonly Task Handled = Task.FromResult(HandledResult); + + public static BenchmarkExpectation Expectation(IReadOnlyDictionary parameters) + => Expectation((ShortCircuitOutcome)parameters["Outcome"]!); + + public static BenchmarkExpectation Expectation(ShortCircuitOutcome outcome) => outcome switch + { + ShortCircuitOutcome.AuthorizationReject => new(["authorization"], RejectedResult), + ShortCircuitOutcome.CacheHit => new(["authorization", "cache"], CachedResult), + ShortCircuitOutcome.FullExecution => new(["authorization", "cache", "transaction", TraceNames.Handler], HandledResult), + _ => throw new ArgumentOutOfRangeException(nameof(outcome)), + }; + + public static void AddDependencies( + IServiceCollection services, + int bufferBytes = 0, + ConstructionTrace? trace = null) + { + services.AddScoped(); + services.AddSingleton(new ShortCircuitSettings(bufferBytes, trace)); + services.AddScoped(); + + services.AddScoped(); + services.AddScoped(); + + services.AddScoped(); + services.AddScoped(); + + services.AddScoped(); + services.AddScoped(); + services.AddScoped(); + } +} diff --git a/benchmarks/RequestFlow.Benchmarks.Core/ShortCircuit/StageDependencies.cs b/benchmarks/RequestFlow.Benchmarks.Core/ShortCircuit/StageDependencies.cs new file mode 100644 index 0000000..82d42c7 --- /dev/null +++ b/benchmarks/RequestFlow.Benchmarks.Core/ShortCircuit/StageDependencies.cs @@ -0,0 +1,23 @@ +namespace RequestFlow.Benchmarks.Core; + +public sealed class AccessPolicy(DependencyState state) : DependencyComponent(state, "access-policy") +{ + public bool Allows(ShortCircuitMessage message) => message.Outcome != ShortCircuitOutcome.AuthorizationReject; +} + +public sealed class CachePolicy(DependencyState state) : DependencyComponent(state, "cache-policy") +{ + public bool Contains(ShortCircuitMessage message) => message.Outcome == ShortCircuitOutcome.CacheHit; +} + +public sealed class ChangeBuffer(DependencyState state) : DependencyComponent(state, "change-buffer") +{ + public byte[] Bytes { get; } = state.Settings.BufferBytes == 0 + ? Array.Empty() + : new byte[state.Settings.BufferBytes]; +} + +public sealed class UnitOfWork(DependencyState state, ChangeBuffer buffer) : DependencyComponent(state, "unit-of-work") +{ + public int Capacity => buffer.Bytes.Length; +} diff --git a/benchmarks/RequestFlow.Benchmarks.Core/Verification/BenchmarkExpectation.cs b/benchmarks/RequestFlow.Benchmarks.Core/Verification/BenchmarkExpectation.cs new file mode 100644 index 0000000..ff226f1 --- /dev/null +++ b/benchmarks/RequestFlow.Benchmarks.Core/Verification/BenchmarkExpectation.cs @@ -0,0 +1,60 @@ +namespace RequestFlow.Benchmarks.Core; + +/// +/// The expected workload for one benchmark parameter set. +/// +/// The benchmark method whose library resolves handlers in a child scope per send. +/// How many child scopes one call of opens. +/// Benchmark methods that return no response, so does not apply to them. +public sealed record BenchmarkExpectation( + string[] Names, + int? Result = null, + bool Ordered = true, + int ParallelHandlers = 0, + IReadOnlyDictionary? HandlerTypes = null, + string? ChildScopeMethod = null, + int ChildScopes = 1, + IReadOnlyCollection? VoidMethods = null) +{ + /// + /// A request that returns the shared workload result. + /// + public static BenchmarkExpectation Request(int stages = 0, string prefix = TraceNames.StagePrefix) + => new(BenchmarkVerification.RequestNames(stages, prefix), BenchmarkWorkload.Expected); + + /// + /// A request without a response value. + /// + public static BenchmarkExpectation VoidRequest(int stages = 0, string prefix = TraceNames.StagePrefix) + => new(BenchmarkVerification.RequestNames(stages, prefix)); + + /// + /// A fully consumed stream with its checksum. + /// + public static BenchmarkExpectation Stream(int stages, int count) + => new(BenchmarkVerification.StreamNames(stages, count), count * (count - 1) / 2); + + /// + /// An event with the declared handler ordering and fan-out gate. + /// + public static BenchmarkExpectation Event(int handlers, bool ordered = true, int parallelHandlers = 0) + => new(BenchmarkVerification.EventNames(handlers), Ordered: ordered, ParallelHandlers: parallelHandlers); + + /// + /// Checks the trace, result and declared handler types. + /// + public void Verify(BenchmarkObservation observation) + { + int? result = VoidMethods?.Contains(observation.Method) == true ? null : Result; + + BenchmarkVerification.VerifyWorkload(observation, Names, result, ordered: Ordered); + + if (HandlerTypes is null) + return; + + if (!HandlerTypes.TryGetValue(observation.Method, out Type[]? expectedTypes)) + throw new InvalidOperationException($"Benchmark '{observation.BenchmarkType.Name}.{observation.Method}' has no handler type expectations."); + + BenchmarkVerification.VerifyHandlerTypes(observation, expectedTypes); + } +} diff --git a/benchmarks/RequestFlow.Benchmarks.Core/Verification/BenchmarkObservation.cs b/benchmarks/RequestFlow.Benchmarks.Core/Verification/BenchmarkObservation.cs new file mode 100644 index 0000000..836a7e5 --- /dev/null +++ b/benchmarks/RequestFlow.Benchmarks.Core/Verification/BenchmarkObservation.cs @@ -0,0 +1,8 @@ +namespace RequestFlow.Benchmarks.Core; + +public sealed record BenchmarkObservation( + Type BenchmarkType, + string Method, + IReadOnlyDictionary Parameters, + object? Result, + IReadOnlyList? Trace = null); diff --git a/benchmarks/RequestFlow.Benchmarks.Core/Verification/BenchmarkScopeAttribute.cs b/benchmarks/RequestFlow.Benchmarks.Core/Verification/BenchmarkScopeAttribute.cs new file mode 100644 index 0000000..662a0d1 --- /dev/null +++ b/benchmarks/RequestFlow.Benchmarks.Core/Verification/BenchmarkScopeAttribute.cs @@ -0,0 +1,10 @@ +namespace RequestFlow.Benchmarks.Core; + +/// +/// The scope lifetime that the probe checks for every method of a benchmark class. +/// +[AttributeUsage(AttributeTargets.Class)] +public sealed class BenchmarkScopeAttribute(ScopeLifetime lifetime) : Attribute +{ + public ScopeLifetime Lifetime { get; } = lifetime; +} diff --git a/benchmarks/RequestFlow.Benchmarks.Core/Verification/BenchmarkVerification.cs b/benchmarks/RequestFlow.Benchmarks.Core/Verification/BenchmarkVerification.cs new file mode 100644 index 0000000..bd7ba07 --- /dev/null +++ b/benchmarks/RequestFlow.Benchmarks.Core/Verification/BenchmarkVerification.cs @@ -0,0 +1,368 @@ +using System.Reflection; +using BenchmarkDotNet.Attributes; +using RequestFlow.Benchmarks.Harness; + +namespace RequestFlow.Benchmarks.Core; + +public static class BenchmarkVerification +{ + /// + /// Checks the result and trace of a benchmark call, which passes . + /// + public static void VerifyWorkload( + BenchmarkObservation observation, string[] expectedNames, int? expectedResult = null, bool ordered = true) + { + string label = Label(observation); + + if (expectedResult is not null && !Equals(observation.Result, expectedResult.Value)) + throw new InvalidOperationException($"{label} returned {observation.Result ?? "no result"}, expected {expectedResult}."); + + TraceAssertions.VerifyRecords(label, observation.Trace ?? [], expectedNames, CancellationToken.None, ordered); + } + + public static string[] RequestNames(int stages = 0, string prefix = TraceNames.StagePrefix) + => [.. Enumerable.Range(1, stages).Select(index => prefix + index), TraceNames.Handler]; + + /// + /// Returns the stream trace names. An early stop reads one item and records no completion. + /// + public static string[] StreamNames(int stages, int count, bool early = false) + => [.. RequestNames(stages), .. Enumerable.Range(0, early ? 1 : count).Select(index => TraceNames.ItemPrefix + index), + .. early ? Array.Empty() : new[] { TraceNames.StreamComplete }, TraceNames.StreamDispose]; + + public static string[] EventNames(int handlers) + => Enumerable.Range(1, handlers) + .SelectMany(index => new[] + { + TraceNames.EventHandlerPrefix + index, + TraceNames.EventHandlerPrefix + index + TraceNames.CompletionSuffix, + }) + .ToArray(); + + public static void VerifyHandlerTypes(BenchmarkObservation observation, Type[] expectedTypes) + { + Type[] actual = (observation.Trace ?? []) + .Where(entry => entry.Name == TraceNames.Handler || TraceNames.IsEventHandlerStart(entry.Name)) + .Select(entry => entry.Component.GetType()).ToArray(); + + if (!actual.SequenceEqual(expectedTypes)) + { + string expectedTypeNames = string.Join(", ", expectedTypes.Select(type => type.ToString())); + string actualTypeNames = string.Join(", ", actual.Select(type => type.ToString())); + + throw new InvalidOperationException( + $"Benchmark '{observation.BenchmarkType.Name}.{observation.Method}' executed the wrong handler types. " + + $"Expected: {expectedTypeNames}. Actual: {actualTypeNames}."); + } + } + + /// + /// Returns the values of a on a benchmark property or field. + /// + public static T[] Params(Type benchmarkType, string member) + { + MemberInfo declaration = benchmarkType.GetMember(member, BindingFlags.Public | BindingFlags.Instance).SingleOrDefault() + ?? throw new InvalidOperationException($"'{benchmarkType.Name}' has no member '{member}'."); + ParamsAttribute attribute = declaration.GetCustomAttribute() + ?? throw new InvalidOperationException($"'{benchmarkType.Name}.{member}' has no [Params] attribute."); + + return attribute.Values.Cast().ToArray(); + } + + /// + /// Runs every benchmark method of every parameter set twice and checks it against its class expectation. + /// + /// + /// Every class declares its scope lifetime with . + /// A class must use a new scope per call and dispose it before the call completes. + /// A class must retain one scope across calls. + /// A class is exempt from scope checks. + /// A Completion parameter fixes whether the returned awaitable is complete when the call returns. + /// + public static BenchmarkProbeResult Run( + Assembly assembly, + IReadOnlyDictionary, BenchmarkExpectation>> expectations) + => Run( + assembly.GetTypes().Where(type => type.IsPublic && !type.IsAbstract && + FindMethodsWithAttribute(type, typeof(BenchmarkAttribute)).Length > 0).ToArray(), + expectations); + + internal static BenchmarkProbeResult Run( + Type[] benchmarkTypes, + IReadOnlyDictionary, BenchmarkExpectation>> expectations) + { + foreach (Type type in benchmarkTypes) + { + if (!expectations.ContainsKey(type)) + throw new InvalidOperationException($"Benchmark '{type.Name}' has no workload expectations."); + + // Throws when the class declares no scope lifetime. + ScopeVerification.For(type); + } + + int parameterSets = 0; + int cases = 0; + + foreach (Type type in benchmarkTypes) + { + MethodInfo[] methods = FindMethodsWithAttribute(type, typeof(BenchmarkAttribute)); + + VerifyBaselines(type, methods); + + if (methods.Select(method => method.GetCustomAttribute()!.OperationsPerInvoke).Distinct().Count() != 1) + throw new InvalidOperationException($"'{type.Name}' must use one OperationsPerInvoke for every method."); + + foreach (Dictionary parameters in EnumerateParameterSets(type)) + { + VerifyParameterSet(type, methods, parameters, expectations[type](parameters)); + + parameterSets++; + cases += methods.Length; + } + } + + if (cases == 0) + throw new InvalidOperationException("No benchmark methods were verified."); + + return new BenchmarkProbeResult(benchmarkTypes.Length, parameterSets, cases); + } + + /// + /// Requires one baseline per group. Methods without a category form one group. + /// + internal static void VerifyBaselines(Type type, MethodInfo[] methods) + { + foreach (IGrouping group in methods.GroupBy(CategoryKey, StringComparer.Ordinal)) + { + if (group.Count(method => method.GetCustomAttribute()!.Baseline) != 1) + { + string category = group.Key.Length == 0 ? "its uncategorized methods" : $"category '{group.Key}'"; + + throw new InvalidOperationException($"'{type.Name}' must declare one baseline for {category}."); + } + } + } + + private static string CategoryKey(MethodInfo method) + => string.Join(",", method.GetCustomAttributes(true) + .SelectMany(attribute => attribute.Categories).Distinct().Order(StringComparer.Ordinal)); + + // BenchmarkDotNet gives every benchmark method its own instance and setup, so the probe does too. + private static void VerifyParameterSet( + Type type, + MethodInfo[] methods, + Dictionary parameters, + BenchmarkExpectation expectation) + { + foreach (MethodInfo method in methods) + { + object instance = CreateInstance(type, parameters); + + try + { + InvokeLifecycleMethod(instance, method, typeof(GlobalSetupAttribute)); + + BenchmarkObservation first = ObserveMethod(instance, method, parameters, expectation); + BenchmarkObservation second = ObserveMethod(instance, method, parameters, expectation); + + ScopeVerification.VerifyAcrossCalls(first, second, expectation); + + expectation.Verify(first); + expectation.Verify(second); + } + catch (Exception exception) + { + string parameterDescription = string.Join(", ", parameters.Select(pair => pair.Key + "=" + pair.Value)); + Exception cause = exception is TargetInvocationException invocation + ? invocation.InnerException ?? exception + : exception; + + throw new InvalidOperationException( + $"Benchmark correctness verification failed for '{type.Name}.{method.Name}' ({parameterDescription}).", cause); + } + finally + { + InvokeLifecycleMethod(instance, method, typeof(GlobalCleanupAttribute)); + } + } + } + + private static object CreateInstance(Type type, IReadOnlyDictionary parameters) + { + object instance = Activator.CreateInstance(type) + ?? throw new InvalidOperationException($"Cannot construct '{type.FullName}'."); + + foreach ((string name, object? value) in parameters) + { + if (type.GetProperty(name) is PropertyInfo property) + property.SetValue(instance, value); + else + type.GetField(name)!.SetValue(instance, value); + } + + return instance; + } + + // Matches BenchmarkDotNet: a lifecycle method that targets the benchmark wins over an untargeted one. + private static void InvokeLifecycleMethod(object instance, MethodInfo benchmark, Type attribute) + { + MethodInfo? lifecycle = FindMethodsWithAttribute(instance.GetType(), attribute) + .Select(method => (Method: method, Attribute: (TargetedAttribute)method.GetCustomAttribute(attribute, true)!)) + .OrderByDescending(candidate => candidate.Attribute.Targets.Length) + .FirstOrDefault(candidate => candidate.Attribute.Match(benchmark)) + .Method; + + if (lifecycle is not null) + WaitForResult(lifecycle.Invoke(instance, null)); + } + + private static BenchmarkObservation ObserveMethod( + object instance, MethodInfo method, IReadOnlyDictionary parameters, BenchmarkExpectation expectation) + { + Type type = instance.GetType(); + var trace = new ExecutionTrace(expectation.ParallelHandlers); + // A field holds one message or an array of them. + var messages = InstanceFields(type) + .Select(field => field.GetValue(instance)) + .SelectMany(value => value as object?[] ?? [value]) + .OfType() + .Distinct() + .Select(message => (Message: message, OriginalTrace: message.Trace)) + .ToArray(); + + try + { + foreach (var message in messages) + message.Message.Trace = trace; + + object? operation = null; + object? result; + bool? completedOnReturn = null; + + try + { + using (var continuations = new DeferredContinuations()) + { + operation = method.Invoke(instance, null); + completedOnReturn = IsCompleted(operation); + + if (continuations.DeadlineExpired) + throw new InvalidOperationException($"'{type.Name}.{method.Name}' blocked on its own continuation."); + } + + trace.VerifyFanOut(); + } + finally + { + trace.ReleaseHandlers(); + result = WaitForResult(operation); + } + + trace.VerifyCompletion(); + + var observation = new BenchmarkObservation(type, method.Name, parameters, result, trace.Entries); + + VerifyCompletionShape(observation, completedOnReturn); + ScopeVerification.VerifyAfterCall(observation, expectation); + + return observation; + } + finally + { + foreach (var message in messages) + message.Message.Trace = message.OriginalTrace; + } + } + + private static void VerifyCompletionShape(BenchmarkObservation observation, bool? completedOnReturn) + { + bool? synchronous = observation.Parameters.TryGetValue("Completion", out object? completion) && completion is CompletionKind kind + ? kind != CompletionKind.Suspending + : null; + + if (synchronous is null) + return; + + if (completedOnReturn is null) + throw new InvalidOperationException($"{Label(observation)} returns no awaitable, so its completion shape cannot be checked."); + + if (completedOnReturn != synchronous) + { + string expected = synchronous.Value ? "complete synchronously" : "suspend"; + + throw new InvalidOperationException($"{Label(observation)} did not {expected}."); + } + } + + private static bool? IsCompleted(object? operation) + => operation?.GetType().GetProperty(nameof(Task.IsCompleted))?.GetValue(operation) as bool?; + + internal static string Label(BenchmarkObservation observation) + { + string parameterDescription = string.Join( + ", ", + observation.Parameters.Select(pair => pair.Key + "=" + pair.Value)); + + return $"Benchmark '{observation.BenchmarkType.Name}.{observation.Method}' ({parameterDescription})"; + } + + // Abstract benchmark bases declare private fields that GetFields does not return for the derived type. + private static IEnumerable InstanceFields(Type type) + { + for (Type? current = type; current is not null; current = current.BaseType) + { + foreach (FieldInfo field in current.GetFields( + BindingFlags.Public | BindingFlags.NonPublic | BindingFlags.Instance | BindingFlags.DeclaredOnly)) + yield return field; + } + } + + private static MethodInfo[] FindMethodsWithAttribute(Type type, Type attribute) + => type.GetMethods(BindingFlags.Public | BindingFlags.Instance) + .Where(method => method.IsDefined(attribute, true)).ToArray(); + + private static IEnumerable> EnumerateParameterSets(Type type) + { + MemberInfo[] members = type.GetMembers(BindingFlags.Public | BindingFlags.Instance) + .Where(member => member.GetCustomAttribute() is not null).ToArray(); + IEnumerable> sets = [new Dictionary()]; + + foreach (MemberInfo member in members) + { + object?[] values = member.GetCustomAttribute()!.Values; + sets = sets.SelectMany(previous => values.Select(value => + { + var next = new Dictionary(previous) { [member.Name] = value }; + + return next; + })).ToArray(); + } + + return sets; + } + + private static object? WaitForResult(object? result) + { + if (result is ValueTask valueTask) + { + valueTask.AsTask().GetAwaiter().GetResult(); + + return null; + } + + if (result is not null && result.GetType().IsGenericType && + result.GetType().GetGenericTypeDefinition() == typeof(ValueTask<>)) + { + result = result.GetType().GetMethod("AsTask")!.Invoke(result, null); + } + + if (result is Task task) + { + task.GetAwaiter().GetResult(); + + return task.GetType().GetProperty("Result")?.GetValue(task); + } + + return result; + } +} diff --git a/benchmarks/RequestFlow.Benchmarks.Core/Verification/ComparedLibrary.cs b/benchmarks/RequestFlow.Benchmarks.Core/Verification/ComparedLibrary.cs new file mode 100644 index 0000000..7ef7d9e --- /dev/null +++ b/benchmarks/RequestFlow.Benchmarks.Core/Verification/ComparedLibrary.cs @@ -0,0 +1,14 @@ +using Microsoft.Extensions.DependencyInjection; + +namespace RequestFlow.Benchmarks.Core; + +/// +/// One library in a paired scope check: its label, provider build, and the registrations its report prints. +/// +/// Service types the report prints besides the dispatcher. +/// Open generic interfaces whose implementations the report prints. +public sealed record ComparedLibrary( + string Name, + Func Build, + IReadOnlyCollection? ReportedServices = null, + IReadOnlyCollection? ReportedContracts = null); diff --git a/benchmarks/RequestFlow.Benchmarks.Core/Verification/DeferredContinuations.cs b/benchmarks/RequestFlow.Benchmarks.Core/Verification/DeferredContinuations.cs new file mode 100644 index 0000000..9ce287e --- /dev/null +++ b/benchmarks/RequestFlow.Benchmarks.Core/Verification/DeferredContinuations.cs @@ -0,0 +1,70 @@ +namespace RequestFlow.Benchmarks.Core; + +// Holds continuations posted during a probe call, so a suspending operation cannot finish before the probe reads its state. +// A call that blocks on its own deferred continuation is released after the deadline and fails the probe. +internal sealed class DeferredContinuations : SynchronizationContext, IDisposable +{ + private static readonly TimeSpan Deadline = TimeSpan.FromSeconds(1); + + private readonly SynchronizationContext? _previous = Current; + private readonly Queue<(SendOrPostCallback Callback, object? State)> _pending = new(); + private readonly Timer _deadline; + private bool _released; + + public bool DeadlineExpired { get; private set; } + + public DeferredContinuations() + { + _deadline = new Timer(_ => Expire(), null, Timeout.Infinite, Timeout.Infinite); + SetSynchronizationContext(this); + } + + public override void Post(SendOrPostCallback d, object? state) + { + lock (_pending) + { + if (!_released) + { + _pending.Enqueue((d, state)); + _deadline.Change(Deadline, Timeout.InfiniteTimeSpan); + + return; + } + } + + Forward((d, state)); + } + + public override SynchronizationContext CreateCopy() => this; + + public void Dispose() + { + SetSynchronizationContext(_previous); + Release(); + _deadline.Dispose(); + } + + private void Expire() + { + DeadlineExpired = true; + Release(); + } + + private void Release() + { + (SendOrPostCallback Callback, object? State)[] pending; + + lock (_pending) + { + _released = true; + pending = _pending.ToArray(); + _pending.Clear(); + } + + foreach ((SendOrPostCallback Callback, object? State) item in pending) + Forward(item); + } + + private static void Forward((SendOrPostCallback Callback, object? State) item) + => ThreadPool.QueueUserWorkItem(static item => item.Callback(item.State), item, preferLocal: false); +} diff --git a/benchmarks/RequestFlow.Benchmarks.Core/Verification/ExecutionRecord.cs b/benchmarks/RequestFlow.Benchmarks.Core/Verification/ExecutionRecord.cs new file mode 100644 index 0000000..2e01a98 --- /dev/null +++ b/benchmarks/RequestFlow.Benchmarks.Core/Verification/ExecutionRecord.cs @@ -0,0 +1,4 @@ +namespace RequestFlow.Benchmarks.Core; + +public sealed record ExecutionRecord( + string Name, WorkloadComponent Component, ScopeWitness Scope, CancellationToken Token); diff --git a/benchmarks/RequestFlow.Benchmarks.Core/Verification/ExecutionTrace.cs b/benchmarks/RequestFlow.Benchmarks.Core/Verification/ExecutionTrace.cs new file mode 100644 index 0000000..c0042d1 --- /dev/null +++ b/benchmarks/RequestFlow.Benchmarks.Core/Verification/ExecutionTrace.cs @@ -0,0 +1,96 @@ +using System.Collections.Concurrent; + +namespace RequestFlow.Benchmarks.Core; + +public sealed class ExecutionTrace(int parallelHandlers = 0) +{ + private readonly ConcurrentQueue _entries = new(); + private readonly ConcurrentQueue _operations = new(); + private int _completedHolds; + private readonly TaskCompletionSource? _handlerRelease = parallelHandlers > 0 + ? new(TaskCreationOptions.RunContinuationsAsynchronously) : null; + + public IReadOnlyList Entries => _entries.ToArray(); + + public void Record(ExecutionRecord entry) => _entries.Enqueue(entry); + + public TTask Track(TTask operation) where TTask : Task + { + _operations.Enqueue(operation); + + return operation; + } + + public ValueTask Track(ValueTask operation) => new(Track(operation.AsTask())); + + public ValueTask Track(ValueTask operation) => new(Track(operation.AsTask())); + + // Handlers are held only on suspending rows, so a held handler that already completed ignored its completion shape. + internal Task HoldHandlers(Task operation) + => _handlerRelease is null ? operation : WaitForRelease(Held(operation)); + + internal ValueTask HoldHandlers(ValueTask operation) + => _handlerRelease is null ? operation : new ValueTask(WaitForRelease(Held(operation.AsTask()))); + + internal Task HoldHandlers(Task operation) + => _handlerRelease is null ? operation : WaitForRelease(Held(operation)); + + internal ValueTask HoldHandlers(ValueTask operation) + => _handlerRelease is null ? operation : new ValueTask(WaitForRelease(Held(operation.AsTask()))); + + private TTask Held(TTask operation) where TTask : Task + { + if (operation.IsCompleted) + Interlocked.Increment(ref _completedHolds); + + return operation; + } + + internal void VerifyFanOut() + { + if (_handlerRelease is null) + return; + + ExecutionRecord[] entries = _entries.ToArray(); + int started = entries.Count(entry => TraceNames.IsEventHandlerStart(entry.Name)); + bool hasCompletedOperations = entries.Any( + entry => entry.Name.EndsWith(TraceNames.CompletionSuffix, StringComparison.Ordinal)); + + if (started != parallelHandlers || hasCompletedOperations) + throw new InvalidOperationException($"Expected all {parallelHandlers} event handlers to start before completion was released; got {started} starts."); + + if (_completedHolds != 0) + throw new InvalidOperationException($"{_completedHolds} held event handlers completed synchronously on a suspending row."); + } + + internal void ReleaseHandlers() => _handlerRelease?.TrySetResult(); + + private async Task WaitForRelease(Task operation) + { + await _handlerRelease!.Task.ConfigureAwait(false); + await operation.ConfigureAwait(false); + } + + private async Task WaitForRelease(Task operation) + { + await _handlerRelease!.Task.ConfigureAwait(false); + + return await operation.ConfigureAwait(false); + } + + internal void VerifyCompletion() + { + if (_operations.Any(operation => !operation.IsCompletedSuccessfully)) + throw new InvalidOperationException("The benchmark returned before its workload completed successfully."); + + int handlerStarts = _entries.Count(IsHandlerStart); + int completedStreams = _entries.Count(entry => entry.Name == TraceNames.StreamComplete); + int expectedOperations = handlerStarts - completedStreams; + + if (_operations.Count != expectedOperations) + throw new InvalidOperationException("The benchmark did not complete one workload per handler."); + } + + private static bool IsHandlerStart(ExecutionRecord entry) + => entry.Name == TraceNames.Handler || TraceNames.IsEventHandlerStart(entry.Name); +} diff --git a/benchmarks/RequestFlow.Benchmarks.Core/Verification/RegistrationReport.cs b/benchmarks/RequestFlow.Benchmarks.Core/Verification/RegistrationReport.cs new file mode 100644 index 0000000..c1e07ca --- /dev/null +++ b/benchmarks/RequestFlow.Benchmarks.Core/Verification/RegistrationReport.cs @@ -0,0 +1,65 @@ +using Microsoft.Extensions.DependencyInjection; + +namespace RequestFlow.Benchmarks.Core; + +/// +/// Lists and filters the DI descriptors the benchmark fixtures register. +/// +public static class RegistrationReport +{ + /// + /// Prints workload components, the dispatcher, , and implementations of . + /// + /// Service types to print; an open generic type matches every closed registration of it. + /// Open generic interfaces whose implementations to print, whatever service type registers them. + public static void ReportRegistrations( + string label, + IServiceCollection registrations, + Type dispatcherType, + IReadOnlyCollection? reportedServices = null, + IReadOnlyCollection? reportedContracts = null) + { + Console.WriteLine(label); + + Type[] services = [typeof(ScopeWitness), dispatcherType, .. reportedServices ?? []]; + IReadOnlyCollection contracts = reportedContracts ?? []; + + foreach (ServiceDescriptor descriptor in registrations.Where(descriptor => + IsComponent(descriptor) || Implements(descriptor, contracts) || + services.Any(service => Matches(descriptor.ServiceType, service)))) + { + string implementation = GetImplementationType(descriptor)?.ToString() ?? + (descriptor.IsKeyedService ? descriptor.KeyedImplementationInstance : descriptor.ImplementationInstance)?.GetType().ToString() ?? + "factory"; + object? serviceKey = descriptor.IsKeyedService ? descriptor.ServiceKey : "none"; + + Console.WriteLine($" {descriptor.ServiceType}: {descriptor.Lifetime}; implementation={implementation}; key={serviceKey}"); + } + } + + /// + /// Returns registrations for workload components. + /// + public static IEnumerable GetComponentDescriptors(IServiceCollection services) + => services.Where(IsComponent); + + internal static Type? GetImplementationType(ServiceDescriptor descriptor) + => descriptor.IsKeyedService ? descriptor.KeyedImplementationType : descriptor.ImplementationType; + + private static bool IsComponent(ServiceDescriptor descriptor) + => GetImplementationType(descriptor) is Type implementation && + typeof(WorkloadComponent).IsAssignableFrom(implementation); + + private static bool Implements(ServiceDescriptor descriptor, IReadOnlyCollection contracts) + { + Type type = GetImplementationType(descriptor) ?? descriptor.ServiceType; + + return type.GetInterfaces().Any(contract => + contract.IsGenericType && contracts.Contains(contract.GetGenericTypeDefinition())); + } + + private static bool Matches(Type serviceType, Type reported) + => serviceType == reported || + (reported.IsGenericTypeDefinition && serviceType.IsGenericType && + serviceType.GetGenericTypeDefinition() == reported); +} diff --git a/benchmarks/RequestFlow.Benchmarks.Core/Verification/ScopeLifetime.cs b/benchmarks/RequestFlow.Benchmarks.Core/Verification/ScopeLifetime.cs new file mode 100644 index 0000000..ea366ca --- /dev/null +++ b/benchmarks/RequestFlow.Benchmarks.Core/Verification/ScopeLifetime.cs @@ -0,0 +1,19 @@ +namespace RequestFlow.Benchmarks.Core; + +public enum ScopeLifetime +{ + /// + /// Every call runs in one scope created before timing, which stays undisposed. + /// + RetainedScope, + + /// + /// Every call creates its own scope and disposes it before completing. + /// + ScopePerOperation, + + /// + /// Calls invoke a handler instance directly, so no scope is checked. + /// + DirectHandler, +} diff --git a/benchmarks/RequestFlow.Benchmarks.Core/Verification/ScopeLifetimeVerification.cs b/benchmarks/RequestFlow.Benchmarks.Core/Verification/ScopeLifetimeVerification.cs new file mode 100644 index 0000000..8f2d0ad --- /dev/null +++ b/benchmarks/RequestFlow.Benchmarks.Core/Verification/ScopeLifetimeVerification.cs @@ -0,0 +1,231 @@ +using Microsoft.Extensions.DependencyInjection; +using static RequestFlow.Benchmarks.Core.RegistrationReport; +using static RequestFlow.Benchmarks.Core.TraceAssertions; + +namespace RequestFlow.Benchmarks.Core; + +/// +/// Checks scoped component identities, execution traces and disposal. +/// +public static class ScopeLifetimeVerification +{ + /// + /// Checks registrations and execution across two scopes. + /// + public static void VerifyScopes( + string label, + ServiceCollection registrations, + Func build, + Type dispatcherType, + Func execute, + string[] expectedNames, + bool report, + bool ordered = true, + IReadOnlyCollection? reportedServices = null, + IReadOnlyCollection? reportedContracts = null, + ServiceLifetime dispatcherLifetime = ServiceLifetime.Scoped) + { + ServiceDescriptor[] components = GetComponentDescriptors(registrations).ToArray(); + + VerifyComponentRegistrations(label, components); + + Require( + registrations.LastOrDefault(descriptor => descriptor.ServiceType == dispatcherType)?.Lifetime == dispatcherLifetime, + label + " does not register its dispatcher as " + dispatcherLifetime + "."); + + if (report) + ReportRegistrations(label, registrations, dispatcherType, reportedServices, reportedContracts); + + using var cancellation = new CancellationTokenSource(); + using ServiceProvider provider = build(registrations); + using IServiceScope first = provider.CreateScope(); + using IServiceScope next = provider.CreateScope(); + object dispatcher = first.ServiceProvider.GetRequiredService(dispatcherType); + + Require( + ReferenceEquals(dispatcher, first.ServiceProvider.GetRequiredService(dispatcherType)) == SharesWithinScope(dispatcherLifetime), + label + " does not resolve a " + dispatcherLifetime + " dispatcher within one scope."); + + Require( + !ReferenceEquals(dispatcher, next.ServiceProvider.GetRequiredService(dispatcherType)), + label + " shares dispatcher across scopes."); + + var firstTrace = new ExecutionTrace(); + var repeatedTrace = new ExecutionTrace(); + var nextTrace = new ExecutionTrace(); + + Execute(first.ServiceProvider, firstTrace); + Execute(first.ServiceProvider, repeatedTrace); + Execute(next.ServiceProvider, nextTrace); + + VerifyRecords(label, firstTrace.Entries, expectedNames, cancellation.Token, ordered); + VerifyRecords(label + " (repeated call)", repeatedTrace.Entries, expectedNames, cancellation.Token, ordered); + VerifyRecords(label + " (next scope)", nextTrace.Entries, expectedNames, cancellation.Token, ordered); + + ScopeWitness firstWitness = first.ServiceProvider.GetRequiredService(); + ScopeWitness nextWitness = next.ServiceProvider.GetRequiredService(); + + VerifyComponentIdentities(label, firstTrace, repeatedTrace, nextTrace, firstWitness, nextWitness); + + first.Dispose(); + + Require( + firstTrace.Entries.All(entry => entry.Component.IsDisposed && entry.Scope.IsDisposed), + label + " did not dispose the first scope."); + + Require( + nextTrace.Entries.All(entry => !entry.Component.IsDisposed && !entry.Scope.IsDisposed), + label + " disposed another scope's component."); + + next.Dispose(); + + Require( + nextTrace.Entries.All(entry => entry.Component.IsDisposed && entry.Scope.IsDisposed), + label + " did not dispose the next scope."); + + if (report) + Console.WriteLine(" Verified dispatcher, component and dependency identities, tokens, traversal and disposal."); + + void Execute(IServiceProvider services, ExecutionTrace trace) + { + try + { + execute(services, trace, cancellation.Token).GetAwaiter().GetResult(); + } + catch (Exception exception) + { + throw new InvalidOperationException(label + " failed.", exception); + } + } + } + + /// + /// Runs on two libraries' registrations of the same workload. + /// + /// The workload description that follows each library name. + /// Required of both libraries, so a lifetime mismatch fails the pair. + public static void VerifyPair( + string label, + string[] expectedNames, + bool report, + ComparedLibrary first, + ServiceCollection firstServices, + Func firstExecute, + ComparedLibrary second, + ServiceCollection secondServices, + Func secondExecute, + bool ordered = true, + ServiceLifetime dispatcherLifetime = ServiceLifetime.Scoped) + where TFirst : notnull + where TSecond : notnull + { + Verify(first, firstServices, firstExecute); + Verify(second, secondServices, secondExecute); + + void Verify( + ComparedLibrary library, + ServiceCollection services, + Func execute) + where TDispatcher : notnull + => VerifyScopes( + label: library.Name + " " + label, + registrations: services, + build: library.Build, + dispatcherType: typeof(TDispatcher), + execute: (provider, trace, token) => execute(provider.GetRequiredService(), trace, token), + expectedNames: expectedNames, + report: report, + ordered: ordered, + reportedServices: library.ReportedServices, + reportedContracts: library.ReportedContracts, + dispatcherLifetime: dispatcherLifetime); + } + + internal static bool SharesWithinScope(ServiceLifetime dispatcherLifetime) => dispatcherLifetime switch + { + ServiceLifetime.Scoped => true, + ServiceLifetime.Transient => false, + _ => throw new ArgumentOutOfRangeException(nameof(dispatcherLifetime), dispatcherLifetime, "Dispatchers are scoped or transient."), + }; + + /// + /// Requires a request to return the shared workload result. + /// + public static async Task ExpectResult(Task operation) + { + int result = await operation.ConfigureAwait(false); + + Require(result == BenchmarkWorkload.Expected, $"Request returned {result}, expected {BenchmarkWorkload.Expected}."); + } + + /// + public static Task ExpectResult(ValueTask operation) => ExpectResult(operation.AsTask()); + + /// + /// Consumes a stream and checks its checksum, or reads only its first item when is set. + /// + public static async Task ReadStream(IAsyncEnumerable sequence, int count, bool early, CancellationToken token) + { + if (early) + { + await using IAsyncEnumerator enumerator = sequence.GetAsyncEnumerator(token); + + Require(await enumerator.MoveNextAsync().ConfigureAwait(false), "Stream ended before its first item."); + Require(enumerator.Current == 0, "Stream returned the wrong first item."); + + return; + } + + int sum = await BenchmarkWorkload.Consume(sequence).ConfigureAwait(false); + + Require(sum == count * (count - 1) / 2, "Stream returned the wrong checksum."); + } + + private static void VerifyComponentRegistrations(string label, ServiceDescriptor[] components) + { + Require( + components.All(descriptor => descriptor.Lifetime == ServiceLifetime.Scoped), + label + " has a non-scoped component."); + + bool hasUniqueRegistrations = components.GroupBy(descriptor => (descriptor.ServiceType, + descriptor.IsKeyedService ? descriptor.ServiceKey : null, + GetImplementationType(descriptor))).All(group => group.Count() == 1); + + Require(hasUniqueRegistrations, label + " has duplicate component registrations."); + } + + private static void VerifyComponentIdentities( + string label, + ExecutionTrace firstTrace, + ExecutionTrace repeatedTrace, + ExecutionTrace nextTrace, + ScopeWitness firstWitness, + ScopeWitness nextWitness) + { + foreach (ExecutionRecord entry in firstTrace.Entries) + { + ExecutionRecord repeated = repeatedTrace.Entries.Single(candidate => candidate.Name == entry.Name); + ExecutionRecord fresh = nextTrace.Entries.Single(candidate => candidate.Name == entry.Name); + + Require( + ReferenceEquals(entry.Component, repeated.Component), + label + " changes " + entry.Name + " within a scope."); + + Require( + !ReferenceEquals(entry.Component, fresh.Component), + label + " shares " + entry.Name + " across scopes."); + + Require( + ReferenceEquals(entry.Scope, firstWitness) && ReferenceEquals(repeated.Scope, firstWitness), + label + " changes its scoped dependency."); + + Require( + ReferenceEquals(fresh.Scope, nextWitness) && !ReferenceEquals(firstWitness, nextWitness), + label + " shares its scoped dependency across scopes."); + + Require(!entry.Component.IsDisposed && !fresh.Component.IsDisposed, label + " disposed a live component."); + } + + Require(!firstWitness.IsDisposed && !nextWitness.IsDisposed, label + " disposed a live dependency."); + } +} diff --git a/benchmarks/RequestFlow.Benchmarks.Core/Verification/ScopeVerification.cs b/benchmarks/RequestFlow.Benchmarks.Core/Verification/ScopeVerification.cs new file mode 100644 index 0000000..3717cbe --- /dev/null +++ b/benchmarks/RequestFlow.Benchmarks.Core/Verification/ScopeVerification.cs @@ -0,0 +1,70 @@ +using System.Reflection; +using static RequestFlow.Benchmarks.Core.TraceAssertions; + +namespace RequestFlow.Benchmarks.Core; + +internal static class ScopeVerification +{ + public static ScopeLifetime For(Type benchmarkType) + => benchmarkType.GetCustomAttribute()?.Lifetime + ?? throw new InvalidOperationException($"Benchmark '{benchmarkType.Name}' has no [BenchmarkScope] attribute."); + + public static void VerifyAfterCall(BenchmarkObservation observation, BenchmarkExpectation expectation) + { + ScopeLifetime lifetime = For(observation.BenchmarkType); + + if (lifetime == ScopeLifetime.DirectHandler) + return; + + IReadOnlyList records = observation.Trace ?? []; + string label = BenchmarkVerification.Label(observation); + bool childScopes = expectation.ChildScopeMethod == observation.Method; + + Require(records.Count > 0, label + " recorded no scoped components."); + + if (lifetime == ScopeLifetime.RetainedScope) + { + Require(!childScopes, label + " runs handlers in a child scope per send, so a retained scope cannot retain them."); + Require(Witnesses(records).Length == 1, label + " did not run every component in its retained scope."); + Require(records.All(record => !record.Scope.IsDisposed && !record.Component.IsDisposed), + label + " disposed a component of its retained scope."); + + return; + } + + int scopes = childScopes ? expectation.ChildScopes : 1; + + Require(Witnesses(records).Length == scopes, $"{label} ran its components in {Witnesses(records).Length} scopes, expected {scopes}."); + Require(records.All(record => record.Scope.IsDisposed && record.Component.IsDisposed), + label + " completed before disposing its operation scope."); + } + + public static void VerifyAcrossCalls(BenchmarkObservation first, BenchmarkObservation second, BenchmarkExpectation expectation) + { + ScopeLifetime lifetime = For(first.BenchmarkType); + + if (lifetime == ScopeLifetime.DirectHandler) + return; + + IReadOnlyList firstRecords = first.Trace ?? []; + IReadOnlyList secondRecords = second.Trace ?? []; + string label = BenchmarkVerification.Label(first); + + if (lifetime == ScopeLifetime.RetainedScope) + { + Require(Witnesses(firstRecords).SequenceEqual(Witnesses(secondRecords)), label + " changed its retained scope between calls."); + Require(Components(firstRecords).SetEquals(Components(secondRecords)), label + " changed a scoped component between calls."); + + return; + } + + Require(!Witnesses(firstRecords).Intersect(Witnesses(secondRecords)).Any(), label + " reused an operation scope."); + Require(!Components(firstRecords).Overlaps(Components(secondRecords)), label + " reused a scoped component across operations."); + } + + private static ScopeWitness[] Witnesses(IReadOnlyList records) + => records.Select(record => record.Scope).Distinct(ReferenceEqualityComparer.Instance).Cast().ToArray(); + + private static HashSet Components(IReadOnlyList records) + => new(records.Select(record => (object)record.Component), ReferenceEqualityComparer.Instance); +} diff --git a/benchmarks/RequestFlow.Benchmarks.Core/Verification/TraceAssertions.cs b/benchmarks/RequestFlow.Benchmarks.Core/Verification/TraceAssertions.cs new file mode 100644 index 0000000..534897c --- /dev/null +++ b/benchmarks/RequestFlow.Benchmarks.Core/Verification/TraceAssertions.cs @@ -0,0 +1,58 @@ +namespace RequestFlow.Benchmarks.Core; + +/// +/// Assertions over execution traces shared by the suite probes. +/// +public static class TraceAssertions +{ + /// + /// Checks trace names, cancellation tokens and event completion order. + /// + /// Whether the names must follow in order, not just match them. + public static void VerifyRecords( + string label, + IReadOnlyList entries, + string[] expectedNames, + CancellationToken token, + bool ordered = true) + { + string[] actual = entries.Select(entry => entry.Name).ToArray(); + bool namesMatch = ordered + ? actual.SequenceEqual(expectedNames) + : actual.Order(StringComparer.Ordinal).SequenceEqual(expectedNames.Order(StringComparer.Ordinal)); + + Require( + namesMatch, + $"{label} did not execute the expected workload. Expected: {string.Join(", ", expectedNames)}. Actual: {string.Join(", ", actual)}."); + Require(entries.All(entry => entry.Token == token), $"{label} passed the wrong cancellation token."); + Require(HandlersCompleteAfterStart(actual), $"{label} recorded an event handler completion before its start."); + } + + /// + /// Throws if a verification condition fails. + /// + public static void Require(bool condition, string message) + { + if (!condition) + throw new InvalidOperationException(message); + } + + private static bool HandlersCompleteAfterStart(string[] names) + { + for (int index = 0; index < names.Length; index++) + { + string name = names[index]; + + if (!name.StartsWith(TraceNames.EventHandlerPrefix, StringComparison.Ordinal) || + !name.EndsWith(TraceNames.CompletionSuffix, StringComparison.Ordinal)) + continue; + + int startIndex = Array.IndexOf(names, name[..^TraceNames.CompletionSuffix.Length]); + + if (startIndex < 0 || startIndex >= index) + return false; + } + + return true; + } +} diff --git a/benchmarks/RequestFlow.Benchmarks.Core/Verification/TraceNames.cs b/benchmarks/RequestFlow.Benchmarks.Core/Verification/TraceNames.cs new file mode 100644 index 0000000..8390203 --- /dev/null +++ b/benchmarks/RequestFlow.Benchmarks.Core/Verification/TraceNames.cs @@ -0,0 +1,46 @@ +namespace RequestFlow.Benchmarks.Core; + +/// +/// The trace record names that workload components write and the probe checks. +/// +public static class TraceNames +{ + /// + /// A request or stream handler. + /// + public const string Handler = "handler"; + + /// + /// Prefixes the one-based number of a stage. + /// + public const string StagePrefix = "stage:"; + + /// + /// Prefixes the one-based number of an event handler. + /// + public const string EventHandlerPrefix = "handler:"; + + /// + /// Follows an event handler name once its awaitable completes. + /// + public const string CompletionSuffix = ":complete"; + + /// + /// Prefixes the zero-based index of a stream item. + /// + public const string ItemPrefix = "item:"; + + /// + /// A stream that yielded every item. + /// + public const string StreamComplete = "stream:complete"; + + /// + /// A stream whose enumerator was disposed. + /// + public const string StreamDispose = "stream:dispose"; + + public static bool IsEventHandlerStart(string name) + => name.StartsWith(EventHandlerPrefix, StringComparison.Ordinal) && + !name.EndsWith(CompletionSuffix, StringComparison.Ordinal); +} diff --git a/benchmarks/RequestFlow.Benchmarks.Core/Workloads/BenchmarkWorkload.cs b/benchmarks/RequestFlow.Benchmarks.Core/Workloads/BenchmarkWorkload.cs new file mode 100644 index 0000000..e992112 --- /dev/null +++ b/benchmarks/RequestFlow.Benchmarks.Core/Workloads/BenchmarkWorkload.cs @@ -0,0 +1,155 @@ +namespace RequestFlow.Benchmarks.Core; + +public static class BenchmarkWorkload +{ + public const int Input = 42; + public const int Expected = Input + Increment; + + // What every response handler adds to its request value. + private const int Increment = 37; + + private static readonly Task CachedResponse = Task.FromResult(Expected); + + public static Task TaskResponse(WorkloadMessage request, CancellationToken token) + { + token.ThrowIfCancellationRequested(); + + Task operation = request.Completion switch + { + CompletionKind.Cached when request.Value == Input => CachedResponse, + CompletionKind.Suspending => SuspendTask(request.Value, token), + _ => Task.FromResult(request.Value + Increment), + }; + + return request.Trace is { } trace ? trace.Track(operation) : operation; + } + + public static ValueTask ValueResponse(WorkloadMessage request, CancellationToken token) + { + token.ThrowIfCancellationRequested(); + + ValueTask operation = request.Completion == CompletionKind.Suspending + ? SuspendValue(request.Value, token) + : new ValueTask(request.Value + Increment); + + return request.Trace is { } trace ? trace.Track(operation) : operation; + } + + public static Task TaskVoidRequest(WorkloadMessage request, CancellationToken token) + { + token.ThrowIfCancellationRequested(); + + Task operation = request.Completion == CompletionKind.Suspending ? SuspendVoidRequest(token) : Task.CompletedTask; + + return request.Trace is { } trace ? trace.Track(trace.HoldHandlers(operation)) : operation; + } + + public static ValueTask ValueVoidRequest(WorkloadMessage request, CancellationToken token) + { + token.ThrowIfCancellationRequested(); + + ValueTask operation = request.Completion == CompletionKind.Suspending ? SuspendValueVoidRequest(token) : default; + + return request.Trace is { } trace ? trace.Track(trace.HoldHandlers(operation)) : operation; + } + + /// + /// The work for a library whose void handler returns a unit value. + /// + public static Task TaskUnitVoidRequest(WorkloadMessage request, CancellationToken token) + where TUnit : struct + { + token.ThrowIfCancellationRequested(); + + Task operation = request.Completion == CompletionKind.Suspending + ? SuspendUnit(token) + : CompletedUnit.Task; + + return request.Trace is { } trace ? trace.Track(trace.HoldHandlers(operation)) : operation; + } + + /// + /// The work for a library whose void handler returns a unit value. + /// + public static ValueTask ValueUnitVoidRequest(WorkloadMessage request, CancellationToken token) + where TUnit : struct + { + token.ThrowIfCancellationRequested(); + + ValueTask operation = request.Completion == CompletionKind.Suspending + ? SuspendValueUnit(token) + : default; + + return request.Trace is { } trace ? trace.Track(trace.HoldHandlers(operation)) : operation; + } + + public static async Task Consume(IAsyncEnumerable sequence) + { + int sum = 0; + + await foreach (int item in sequence.ConfigureAwait(false)) + sum += item; + + return sum; + } + + public static async Task AwaitTask(Task task) => await task.ConfigureAwait(false); + + public static async ValueTask AwaitValue(ValueTask task) => await task.ConfigureAwait(false); + + public static async Task AwaitTask(Task task) => await task.ConfigureAwait(false); + + public static async ValueTask AwaitValue(ValueTask task) => await task.ConfigureAwait(false); + + private static async Task SuspendTask(int value, CancellationToken token) + { + await Task.Yield(); + token.ThrowIfCancellationRequested(); + + return value + Increment; + } + + private static async ValueTask SuspendValue(int value, CancellationToken token) + { + await Task.Yield(); + token.ThrowIfCancellationRequested(); + + return value + Increment; + } + + private static async Task SuspendVoidRequest(CancellationToken token) + { + await Task.Yield(); + token.ThrowIfCancellationRequested(); + } + + private static async ValueTask SuspendValueVoidRequest(CancellationToken token) + { + await Task.Yield(); + token.ThrowIfCancellationRequested(); + } + + private static async Task SuspendUnit(CancellationToken token) + where TUnit : struct + { + await Task.Yield(); + token.ThrowIfCancellationRequested(); + + return default; + } + + private static async ValueTask SuspendValueUnit(CancellationToken token) + where TUnit : struct + { + await Task.Yield(); + token.ThrowIfCancellationRequested(); + + return default; + } + + private static class CompletedUnit + where TUnit : struct + { + public static readonly Task Task = System.Threading.Tasks.Task.FromResult(default(TUnit)); + } +} diff --git a/benchmarks/RequestFlow.Benchmarks.Core/Workloads/CompletionKind.cs b/benchmarks/RequestFlow.Benchmarks.Core/Workloads/CompletionKind.cs new file mode 100644 index 0000000..20b2b06 --- /dev/null +++ b/benchmarks/RequestFlow.Benchmarks.Core/Workloads/CompletionKind.cs @@ -0,0 +1,8 @@ +namespace RequestFlow.Benchmarks.Core; + +public enum CompletionKind +{ + Cached, + Synchronous, + Suspending, +} diff --git a/benchmarks/RequestFlow.Benchmarks.Core/Workloads/RuntimeTracing.cs b/benchmarks/RequestFlow.Benchmarks.Core/Workloads/RuntimeTracing.cs new file mode 100644 index 0000000..8a8af8e --- /dev/null +++ b/benchmarks/RequestFlow.Benchmarks.Core/Workloads/RuntimeTracing.cs @@ -0,0 +1,63 @@ +using System.Runtime.CompilerServices; + +namespace RequestFlow.Benchmarks.Core; + +public static class RuntimeTracing +{ + public static Task EventTask( + WorkloadMessage message, WorkloadComponent component, string name, CancellationToken token) + { + Task operation = BenchmarkWorkload.TaskVoidRequest(message, token); + + return message.Trace is null ? operation : CompleteTask(operation, message.Trace, component, name, token); + } + + public static ValueTask EventValue( + WorkloadMessage message, WorkloadComponent component, string name, CancellationToken token) + { + ValueTask operation = BenchmarkWorkload.ValueVoidRequest(message, token); + + return message.Trace is null ? operation : CompleteValue(operation, message.Trace, component, name, token); + } + + public static async IAsyncEnumerable Stream( + WorkloadMessage message, int count, WorkloadComponent component, + [EnumeratorCancellation] CancellationToken token) + { + bool suspending = message.Completion == CompletionKind.Suspending; + + try + { + for (int i = 0; i < count; i++) + { + token.ThrowIfCancellationRequested(); + + if (suspending) + await Task.Yield(); + + message.Trace?.Record(new ExecutionRecord(TraceNames.ItemPrefix + i, component, component.Scope, token)); + yield return i; + } + + message.Trace?.Record(new ExecutionRecord(TraceNames.StreamComplete, component, component.Scope, token)); + } + finally + { + message.Trace?.Record(new ExecutionRecord(TraceNames.StreamDispose, component, component.Scope, token)); + } + } + + private static async Task CompleteTask( + Task operation, ExecutionTrace trace, WorkloadComponent component, string name, CancellationToken token) + { + await operation.ConfigureAwait(false); + trace.Record(new ExecutionRecord(name + TraceNames.CompletionSuffix, component, component.Scope, token)); + } + + private static async ValueTask CompleteValue( + ValueTask operation, ExecutionTrace trace, WorkloadComponent component, string name, CancellationToken token) + { + await operation.ConfigureAwait(false); + trace.Record(new ExecutionRecord(name + TraceNames.CompletionSuffix, component, component.Scope, token)); + } +} diff --git a/benchmarks/RequestFlow.Benchmarks.Core/Workloads/ScopeWitness.cs b/benchmarks/RequestFlow.Benchmarks.Core/Workloads/ScopeWitness.cs new file mode 100644 index 0000000..53a0736 --- /dev/null +++ b/benchmarks/RequestFlow.Benchmarks.Core/Workloads/ScopeWitness.cs @@ -0,0 +1,8 @@ +namespace RequestFlow.Benchmarks.Core; + +public sealed class ScopeWitness : IDisposable +{ + public bool IsDisposed { get; private set; } + + public void Dispose() => IsDisposed = true; +} diff --git a/benchmarks/RequestFlow.Benchmarks.Core/Workloads/StageSlots.cs b/benchmarks/RequestFlow.Benchmarks.Core/Workloads/StageSlots.cs new file mode 100644 index 0000000..f56e9ba --- /dev/null +++ b/benchmarks/RequestFlow.Benchmarks.Core/Workloads/StageSlots.cs @@ -0,0 +1,38 @@ +namespace RequestFlow.Benchmarks.Core; + +public interface IStageSlot +{ + static abstract int Number { get; } +} + +public sealed class Slot1 : IStageSlot +{ + public static int Number => 1; +} + +public sealed class Slot2 : IStageSlot +{ + public static int Number => 2; +} + +public sealed class Slot3 : IStageSlot +{ + public static int Number => 3; +} + +public sealed class Slot4 : IStageSlot +{ + public static int Number => 4; +} + +public sealed class Slot5 : IStageSlot +{ + public static int Number => 5; +} + +public static class StageSlot where TSlot : IStageSlot +{ + public static readonly string StageName = TraceNames.StagePrefix + TSlot.Number; + public static readonly string HandlerName = TraceNames.EventHandlerPrefix + TSlot.Number; + public static readonly string PreHandlerName = "prehandler:" + TSlot.Number; +} diff --git a/benchmarks/RequestFlow.Benchmarks.Core/Workloads/WorkloadComponent.cs b/benchmarks/RequestFlow.Benchmarks.Core/Workloads/WorkloadComponent.cs new file mode 100644 index 0000000..d40a762 --- /dev/null +++ b/benchmarks/RequestFlow.Benchmarks.Core/Workloads/WorkloadComponent.cs @@ -0,0 +1,13 @@ +namespace RequestFlow.Benchmarks.Core; + +public abstract class WorkloadComponent(ScopeWitness scope) : IDisposable +{ + public ScopeWitness Scope { get; } = scope; + + public bool IsDisposed { get; private set; } + + protected void Observe(WorkloadMessage message, string name, CancellationToken token) + => message.Trace?.Record(new ExecutionRecord(name, this, Scope, token)); + + public virtual void Dispose() => IsDisposed = true; +} diff --git a/benchmarks/RequestFlow.Benchmarks.Core/Workloads/WorkloadMessage.cs b/benchmarks/RequestFlow.Benchmarks.Core/Workloads/WorkloadMessage.cs new file mode 100644 index 0000000..5a2206e --- /dev/null +++ b/benchmarks/RequestFlow.Benchmarks.Core/Workloads/WorkloadMessage.cs @@ -0,0 +1,10 @@ +namespace RequestFlow.Benchmarks.Core; + +public class WorkloadMessage(CompletionKind completion = CompletionKind.Cached, int value = 42) +{ + public CompletionKind Completion { get; } = completion; + + public int Value { get; } = value; + + public ExecutionTrace? Trace { get; set; } +} diff --git a/benchmarks/RequestFlow.Benchmarks.Core/packages.lock.json b/benchmarks/RequestFlow.Benchmarks.Core/packages.lock.json new file mode 100644 index 0000000..a376bd0 --- /dev/null +++ b/benchmarks/RequestFlow.Benchmarks.Core/packages.lock.json @@ -0,0 +1,178 @@ +{ + "version": 1, + "dependencies": { + "net10.0": { + "BenchmarkDotNet": { + "type": "Direct", + "requested": "[0.15.8, )", + "resolved": "0.15.8", + "contentHash": "paCfrWxSeHqn3rUZc0spYXVFnHCF0nzRhG0nOLnyTjZYs8spsimBaaNmb3vwqvALKIplbYq/TF393vYiYSnh/Q==", + "dependencies": { + "BenchmarkDotNet.Annotations": "0.15.8", + "CommandLineParser": "2.9.1", + "Gee.External.Capstone": "2.3.0", + "Iced": "1.21.0", + "Microsoft.CodeAnalysis.CSharp": "4.14.0", + "Microsoft.Diagnostics.Runtime": "3.1.512801", + "Microsoft.Diagnostics.Tracing.TraceEvent": "3.1.21", + "Microsoft.DotNet.PlatformAbstractions": "3.1.6", + "Perfolizer": "[0.6.1]", + "System.Management": "9.0.5" + } + }, + "Microsoft.Extensions.DependencyInjection": { + "type": "Direct", + "requested": "[10.0.9, )", + "resolved": "10.0.9", + "contentHash": "NijozhERJDIaJ4k5TSMy1jOi0cSC2HfkvRD/Sl+kGSSKgVbFnF4GxgtMN/MrzHB8D1JxIrD4xSer9Blh9v3axQ==", + "dependencies": { + "Microsoft.Extensions.DependencyInjection.Abstractions": "10.0.9" + } + }, + "BenchmarkDotNet.Annotations": { + "type": "Transitive", + "resolved": "0.15.8", + "contentHash": "hfucY0ycAsB0SsoaZcaAp9oq5wlWBJcylvEJb9pmvdYUx6PD6S4mDiYnZWjdjAlLhIpe/xtGCwzORfzAzPqvzA==" + }, + "CommandLineParser": { + "type": "Transitive", + "resolved": "2.9.1", + "contentHash": "OE0sl1/sQ37bjVsPKKtwQlWDgqaxWgtme3xZz7JssWUzg5JpMIyHgCTY9MVMxOg48fJ1AgGT3tgdH5m/kQ5xhA==" + }, + "Gee.External.Capstone": { + "type": "Transitive", + "resolved": "2.3.0", + "contentHash": "2ap/rYmjtzCOT8hxrnEW/QeiOt+paD8iRrIcdKX0cxVwWLFa1e+JDBNeECakmccXrSFeBQuu5AV8SNkipFMMMw==" + }, + "Iced": { + "type": "Transitive", + "resolved": "1.21.0", + "contentHash": "dv5+81Q1TBQvVMSOOOmRcjJmvWcX3BZPZsIq31+RLc5cNft0IHAyNlkdb7ZarOWG913PyBoFDsDXoCIlKmLclg==" + }, + "Microsoft.CodeAnalysis.Analyzers": { + "type": "Transitive", + "resolved": "3.11.0", + "contentHash": "v/EW3UE8/lbEYHoC2Qq7AR/DnmvpgdtAMndfQNmpuIMx/Mto8L5JnuCfdBYtgvalQOtfNCnxFejxuRrryvUTsg==" + }, + "Microsoft.CodeAnalysis.Common": { + "type": "Transitive", + "resolved": "4.14.0", + "contentHash": "PC3tuwZYnC+idaPuoC/AZpEdwrtX7qFpmnrfQkgobGIWiYmGi5MCRtl5mx6QrfMGQpK78X2lfIEoZDLg/qnuHg==", + "dependencies": { + "Microsoft.CodeAnalysis.Analyzers": "3.11.0" + } + }, + "Microsoft.CodeAnalysis.CSharp": { + "type": "Transitive", + "resolved": "4.14.0", + "contentHash": "568a6wcTivauIhbeWcCwfWwIn7UV7MeHEBvFB2uzGIpM2OhJ4eM/FZ8KS0yhPoNxnSpjGzz7x7CIjTxhslojQA==", + "dependencies": { + "Microsoft.CodeAnalysis.Analyzers": "3.11.0", + "Microsoft.CodeAnalysis.Common": "[4.14.0]" + } + }, + "Microsoft.Diagnostics.NETCore.Client": { + "type": "Transitive", + "resolved": "0.2.510501", + "contentHash": "juoqJYMDs+lRrrZyOkXXMImJHneCF23cuvO4waFRd2Ds7j+ZuGIPbJm0Y/zz34BdeaGiiwGWraMUlln05W1PCQ==", + "dependencies": { + "Microsoft.Extensions.Logging": "6.0.0" + } + }, + "Microsoft.Diagnostics.Runtime": { + "type": "Transitive", + "resolved": "3.1.512801", + "contentHash": "0lMUDr2oxNZa28D6NH5BuSQEe5T9tZziIkvkD44YkkCGQXPJqvFjLq5ZQq1hYLl3RjQJrY+hR0jFgap+EWPDTw==", + "dependencies": { + "Microsoft.Diagnostics.NETCore.Client": "0.2.410101" + } + }, + "Microsoft.Diagnostics.Tracing.TraceEvent": { + "type": "Transitive", + "resolved": "3.1.21", + "contentHash": "/OrJFKaojSR6TkUKtwh8/qA9XWNtxLrXMqvEb89dBSKCWjaGVTbKMYodIUgF5deCEtmd6GXuRerciXGl5bhZ7Q==", + "dependencies": { + "Microsoft.Diagnostics.NETCore.Client": "0.2.510501", + "System.Reflection.TypeExtensions": "4.7.0" + } + }, + "Microsoft.DotNet.PlatformAbstractions": { + "type": "Transitive", + "resolved": "3.1.6", + "contentHash": "jek4XYaQ/PGUwDKKhwR8K47Uh1189PFzMeLqO83mXrXQVIpARZCcfuDedH50YDTepBkfijCZN5U/vZi++erxtg==" + }, + "Microsoft.Extensions.DependencyInjection.Abstractions": { + "type": "Transitive", + "resolved": "10.0.9", + "contentHash": "g41l/30G3K4B/d/L8kjux0+30e27c8D0FVQ/PFCpbekgfDpj9mnDhieP67EqXWvl1EWNeZh2rpR4F5B/jcDOHA==" + }, + "Microsoft.Extensions.Logging": { + "type": "Transitive", + "resolved": "6.0.0", + "contentHash": "eIbyj40QDg1NDz0HBW0S5f3wrLVnKWnDJ/JtZ+yJDFnDj90VoPuoPmFkeaXrtu+0cKm5GRAwoDf+dBWXK0TUdg==", + "dependencies": { + "Microsoft.Extensions.DependencyInjection": "6.0.0", + "Microsoft.Extensions.DependencyInjection.Abstractions": "6.0.0", + "Microsoft.Extensions.Logging.Abstractions": "6.0.0", + "Microsoft.Extensions.Options": "6.0.0" + } + }, + "Microsoft.Extensions.Logging.Abstractions": { + "type": "Transitive", + "resolved": "6.0.0", + "contentHash": "/HggWBbTwy8TgebGSX5DBZ24ndhzi93sHUBDvP1IxbZD7FDokYzdAr6+vbWGjw2XAfR2EJ1sfKUotpjHnFWPxA==" + }, + "Microsoft.Extensions.Options": { + "type": "Transitive", + "resolved": "6.0.0", + "contentHash": "dzXN0+V1AyjOe2xcJ86Qbo233KHuLEY0njf/P2Kw8SfJU+d45HNS2ctJdnEnrWbM9Ye2eFgaC5Mj9otRMU6IsQ==", + "dependencies": { + "Microsoft.Extensions.DependencyInjection.Abstractions": "6.0.0", + "Microsoft.Extensions.Primitives": "6.0.0" + } + }, + "Microsoft.Extensions.Primitives": { + "type": "Transitive", + "resolved": "6.0.0", + "contentHash": "9+PnzmQFfEFNR9J2aDTfJGGupShHjOuGw4VUv+JB044biSHrnmCIMD+mJHmb2H7YryrfBEXDurxQ47gJZdCKNQ==" + }, + "Perfolizer": { + "type": "Transitive", + "resolved": "0.6.1", + "contentHash": "CR1QmWg4XYBd1Pb7WseP+sDmV8nGPwvmowKynExTqr3OuckIGVMhvmN4LC5PGzfXqDlR295+hz/T7syA1CxEqA==", + "dependencies": { + "Pragmastat": "3.2.4" + } + }, + "Pragmastat": { + "type": "Transitive", + "resolved": "3.2.4", + "contentHash": "I5qFifWw/gaTQT52MhzjZpkm/JPlfjSeO/DTZJjO7+hTKI+0aGRgOgZ3NN6D96dDuuqbIAZSeA5RimtHjqrA2A==" + }, + "System.CodeDom": { + "type": "Transitive", + "resolved": "9.0.5", + "contentHash": "cuzLM2MWutf9ZBEMPYYfd0DXwYdvntp7VCT6a/wvbKCa2ZuvGmW74xi+YBa2mrfEieAXqM4TNKlMmSnfAfpUoQ==" + }, + "System.Management": { + "type": "Transitive", + "resolved": "9.0.5", + "contentHash": "n6o9PZm9p25+zAzC3/48K0oHnaPKTInRrxqFq1fi/5TPbMLjuoCm/h//mS3cUmSy+9AO1Z+qsC/Ilt/ZFatv5Q==", + "dependencies": { + "System.CodeDom": "9.0.5" + } + }, + "System.Reflection.TypeExtensions": { + "type": "Transitive", + "resolved": "4.7.0", + "contentHash": "VybpaOQQhqE6siHppMktjfGBw1GCwvCqiufqmP8F1nj7fTUNtW35LOEt3UZTEsECfo+ELAl/9o9nJx3U91i7vA==" + }, + "requestflow.benchmarks.harness": { + "type": "Project", + "dependencies": { + "BenchmarkDotNet": "[0.15.8, )" + } + } + } + } +} \ No newline at end of file diff --git a/benchmarks/RequestFlow.Benchmarks.Harness/Configuration/BenchmarkArgumentPolicy.cs b/benchmarks/RequestFlow.Benchmarks.Harness/Configuration/BenchmarkArgumentPolicy.cs new file mode 100644 index 0000000..4416490 --- /dev/null +++ b/benchmarks/RequestFlow.Benchmarks.Harness/Configuration/BenchmarkArgumentPolicy.cs @@ -0,0 +1,48 @@ +namespace RequestFlow.Benchmarks.Harness; + +internal static class BenchmarkArgumentPolicy +{ + private const string PublishedRuntime = "net10.0"; + + // Options that select benchmarks, place artifacts or pick the published runtime without changing measurement. + private static readonly string[] ValueOptions = + ["--filter", "-f", "--artifacts", "-a", "--runtimes", "-r", "--anyCategories", "--allCategories"]; + + private static readonly string[] FlagOptions = ["--join"]; + + /// + /// Returns whether any argument falls outside the options that leave measurement unchanged. + /// + internal static bool HasMeasurementOverrides(string[] arguments) + { + string? option = null; + + foreach (string argument in arguments) + { + string? value = argument; + + if (argument.StartsWith('-')) + { + int equals = argument.IndexOf('='); + + option = equals < 0 ? argument : argument[..equals]; + value = equals < 0 ? null : argument[(equals + 1)..]; + + if (!Contains(ValueOptions, option) && !Contains(FlagOptions, option)) + return true; + } + + // A value belongs to the last option and continues until the next token that starts with '-'. + if (value is not null && !IsAllowedValue(option, value)) + return true; + } + + return false; + } + + private static bool IsAllowedValue(string? option, string value) + => option is not null && Contains(ValueOptions, option) && + (!Contains(["--runtimes", "-r"], option) || value.Equals(PublishedRuntime, StringComparison.OrdinalIgnoreCase)); + + private static bool Contains(string[] options, string option) => options.Contains(option, StringComparer.OrdinalIgnoreCase); +} diff --git a/benchmarks/RequestFlow.Benchmarks.Harness/Configuration/BenchmarkConfig.cs b/benchmarks/RequestFlow.Benchmarks.Harness/Configuration/BenchmarkConfig.cs new file mode 100644 index 0000000..eb8a2ce --- /dev/null +++ b/benchmarks/RequestFlow.Benchmarks.Harness/Configuration/BenchmarkConfig.cs @@ -0,0 +1,52 @@ +using BenchmarkDotNet.Configs; +using BenchmarkDotNet.Diagnosers; +using BenchmarkDotNet.Exporters; +using BenchmarkDotNet.Exporters.Json; +using BenchmarkDotNet.Jobs; +using BenchmarkDotNet.Order; +using BenchmarkDotNet.Validators; +using Perfolizer.Horology; + +namespace RequestFlow.Benchmarks.Harness; + +public static class BenchmarkConfig +{ + internal const double MaximumRelativeError = 0.05; + + public static IConfig Create(BenchmarkTier tier, string artifactsPath) + { + Job job = tier switch + { + BenchmarkTier.Full => Job.Default + .WithMinIterationTime(TimeInterval.FromMilliseconds(500)) + .WithLaunchCount(2) + .WithMinWarmupCount(15) + .WithMaxWarmupCount(50) + .WithMinIterationCount(60) + .WithMaxIterationCount(100) + .WithMaxRelativeError(MaximumRelativeError), + BenchmarkTier.Quick => Job.Default + .WithIterationTime(TimeInterval.FromMilliseconds(250)) + .WithLaunchCount(1) + .WithWarmupCount(3) + .WithIterationCount(8), + BenchmarkTier.Smoke => Job.Dry, + _ => throw new ArgumentOutOfRangeException(nameof(tier), tier, "Unknown benchmark tier."), + }; + + ManualConfig config = ManualConfig.Create(DefaultConfig.Instance) + .AddJob(job.AsDefault()) + // With several --runtimes, BenchmarkDotNet makes the first runtime a baseline job; this keeps ratios within each runtime. + .AddJob(Job.Default.WithBaseline(false).AsMutator()) + .AddValidator(JitOptimizationsValidator.FailOnError) + .AddDiagnoser(MemoryDiagnoser.Default) + .AddDiagnoser(ThreadingDiagnoser.Default) + .AddExporter(MarkdownExporter.Default) + .AddExporter(JsonExporter.Full) + .WithOrderer(new DefaultOrderer(SummaryOrderPolicy.Declared)); + + config.ArtifactsPath = artifactsPath; + + return config; + } +} diff --git a/benchmarks/RequestFlow.Benchmarks.Harness/Configuration/BenchmarkTier.cs b/benchmarks/RequestFlow.Benchmarks.Harness/Configuration/BenchmarkTier.cs new file mode 100644 index 0000000..ef1fb0b --- /dev/null +++ b/benchmarks/RequestFlow.Benchmarks.Harness/Configuration/BenchmarkTier.cs @@ -0,0 +1,8 @@ +namespace RequestFlow.Benchmarks.Harness; + +public enum BenchmarkTier +{ + Full, + Quick, + Smoke, +} diff --git a/benchmarks/RequestFlow.Benchmarks.Harness/Execution/BenchmarkRun.cs b/benchmarks/RequestFlow.Benchmarks.Harness/Execution/BenchmarkRun.cs new file mode 100644 index 0000000..a54d1c6 --- /dev/null +++ b/benchmarks/RequestFlow.Benchmarks.Harness/Execution/BenchmarkRun.cs @@ -0,0 +1,109 @@ +using BenchmarkDotNet.Configs; +using BenchmarkDotNet.ConsoleArguments; +using BenchmarkDotNet.ConsoleArguments.ListBenchmarks; +using BenchmarkDotNet.Jobs; +using BenchmarkDotNet.Loggers; + +namespace RequestFlow.Benchmarks.Harness; + +public sealed record BenchmarkRun( + string Suite, + BenchmarkTier Tier, + IConfig Config, + Job[] Jobs, + string[] Arguments, + string[] OriginalArguments, + bool IsInformational) +{ + private static readonly string[] TierSwitches = ["--smoke", "--quick"]; + private static readonly string[] InformationalSwitches = ["--help", "-h", "--info", "--version"]; + private static readonly string[] PathOptions = + [ + "--artifacts", "-a", "--cli", "--packages", "--coreRun", "--monoPath", "--ilcPackages", + "--wasmEngine", "--customRuntimePack", "--AOTCompilerPath", "--wasmDataDir", + ]; + + public static BenchmarkRun Create(string suiteName, string[] args) + { + ArgumentException.ThrowIfNullOrWhiteSpace(suiteName); + ArgumentNullException.ThrowIfNull(args); + + if (args.Any(argument => argument.StartsWith('@'))) + throw new ArgumentException("Response files (@file) are not supported. Pass the options directly.", nameof(args)); + + BenchmarkTier tier = SelectTier(args); + string[] arguments = args.Where(argument => !TierSwitches.Contains(argument, StringComparer.OrdinalIgnoreCase)).ToArray(); + + MakePathsAbsolute(arguments); + + string artifactsPath = Path.GetFullPath(Path.Combine( + "BenchmarkDotNet.Artifacts", + suiteName, + tier.ToString().ToLowerInvariant(), + $"{DateTimeOffset.UtcNow:yyyyMMdd-HHmmss}-{Guid.NewGuid():N}")); + + IConfig config = BenchmarkConfig.Create(tier, artifactsPath); + var parserLog = new AccumulationLogger(); + var parsed = ConfigParser.Parse(arguments, parserLog, config); + bool help = arguments.Length == 1 && InformationalSwitches.Contains(arguments[0], StringComparer.OrdinalIgnoreCase); + + if (!parsed.isSuccess && !help) + throw new ArgumentException( + $"BenchmarkDotNet rejected the command arguments.{Environment.NewLine}{ParserErrors(parserLog.GetLog())}", nameof(args)); + + if (parsed.isSuccess && parsed.options.ArtifactsDirectory is { } artifacts) + config = BenchmarkConfig.Create(tier, artifacts.FullName); + + bool listing = parsed.isSuccess && parsed.options.ListBenchmarkCaseMode != ListBenchmarkCaseMode.Disabled; + Job[] jobs = parsed.isSuccess ? parsed.config.GetJobs().ToArray() : []; + + return new BenchmarkRun(suiteName, tier, config, jobs, arguments, args.ToArray(), help || listing); + } + + // Keeps the parser errors and drops the generic usage examples that follow them. + private static string ParserErrors(string log) + { + int usage = log.IndexOf("USAGE:", StringComparison.Ordinal); + + return (usage < 0 ? log : log[..usage]).Trim(); + } + + // The runner changes to the suite folder before BenchmarkDotNet parses the arguments again. + private static void MakePathsAbsolute(string[] arguments) + { + for (int index = 0; index < arguments.Length; index++) + { + string argument = arguments[index]; + int equals = argument.IndexOf('='); + string option = equals < 0 ? argument : argument[..equals]; + + if (!PathOptions.Contains(option, StringComparer.OrdinalIgnoreCase)) + { + if (argument.StartsWith("-a", StringComparison.Ordinal) && argument.Length > 2) + arguments[index] = "-a" + Path.GetFullPath(argument[2..]); + + continue; + } + + if (equals >= 0) + arguments[index] = argument[..(equals + 1)] + Path.GetFullPath(argument[(equals + 1)..]); + else if (index + 1 < arguments.Length && !arguments[index + 1].StartsWith('-')) + arguments[++index] = Path.GetFullPath(arguments[index]); + + if (option.Equals("--coreRun", StringComparison.OrdinalIgnoreCase)) + while (index + 1 < arguments.Length && !arguments[index + 1].StartsWith('-')) + arguments[++index] = Path.GetFullPath(arguments[index]); + } + } + + private static BenchmarkTier SelectTier(string[] args) + { + bool smoke = args.Contains("--smoke", StringComparer.OrdinalIgnoreCase); + bool quick = args.Contains("--quick", StringComparer.OrdinalIgnoreCase); + + if (smoke && quick) + throw new ArgumentException("Use either --smoke or --quick, not both.", nameof(args)); + + return smoke ? BenchmarkTier.Smoke : quick ? BenchmarkTier.Quick : BenchmarkTier.Full; + } +} diff --git a/benchmarks/RequestFlow.Benchmarks.Harness/Execution/BenchmarkRunStatus.cs b/benchmarks/RequestFlow.Benchmarks.Harness/Execution/BenchmarkRunStatus.cs new file mode 100644 index 0000000..19958f2 --- /dev/null +++ b/benchmarks/RequestFlow.Benchmarks.Harness/Execution/BenchmarkRunStatus.cs @@ -0,0 +1,23 @@ +using BenchmarkDotNet.Reports; + +namespace RequestFlow.Benchmarks.Harness; + +internal sealed record BenchmarkRunStatus(string Name, string? Error) +{ + internal static readonly BenchmarkRunStatus Completed = new("completed", null); + internal static readonly BenchmarkRunStatus Running = new("running", null); + + internal static BenchmarkRunStatus Failed(string error) => new("failed", error); + + internal static BenchmarkRunStatus Determine(Summary[] summaries) + { + if (summaries.Length == 0) + return Failed("No benchmark matched the filter. Check the pattern with --list flat."); + + bool hasFailures = summaries.Any(summary => summary.HasCriticalValidationErrors || summary.Reports.Length == 0) || + summaries.SelectMany(summary => summary.Reports).Any(report => + !report.Success || report.ExecuteResults.Any(result => result.ExitCode != 0)); + + return hasFailures ? Failed("BenchmarkDotNet did not complete every selected benchmark.") : Completed; + } +} diff --git a/benchmarks/RequestFlow.Benchmarks.Harness/Execution/SuiteRunner.cs b/benchmarks/RequestFlow.Benchmarks.Harness/Execution/SuiteRunner.cs new file mode 100644 index 0000000..13aac89 --- /dev/null +++ b/benchmarks/RequestFlow.Benchmarks.Harness/Execution/SuiteRunner.cs @@ -0,0 +1,234 @@ +using System.Reflection; +using BenchmarkDotNet.Configs; +using BenchmarkDotNet.Reports; +using BenchmarkDotNet.Running; + +namespace RequestFlow.Benchmarks.Harness; + +public static class SuiteRunner +{ + public static int Run( + Assembly assembly, + string suiteName, + string[] args, + Func probe, + Action? reportLifetimes = null) + { + ArgumentNullException.ThrowIfNull(assembly); + ArgumentException.ThrowIfNullOrWhiteSpace(suiteName); + ArgumentNullException.ThrowIfNull(args); + ArgumentNullException.ThrowIfNull(probe); + + if (IsCommand(args, "probe")) + return RunCommand("Probe", () => RunProbe(probe)); + + if (IsCommand(args, "lifetimes")) + { + if (reportLifetimes is null) + return Fail("This suite does not report lifetimes."); + + return RunCommand("Lifetime report", reportLifetimes); + } + + if (args.Any(argument => argument.Equals("probe", StringComparison.OrdinalIgnoreCase) || + argument.Equals("lifetimes", StringComparison.OrdinalIgnoreCase))) + return Fail("Use probe or lifetimes as the only command argument."); + + try + { + BenchmarkRun run = BenchmarkRun.Create(suiteName, args); + + if (run.IsInformational) + { + BenchmarkSwitcher.FromAssembly(assembly).Run(run.Arguments, run.Config); + + return 0; + } + + return RunBenchmarks(assembly, run, probe); + } + catch (Exception exception) + { + return Fail($"Benchmark run failed: {Describe(exception)}"); + } + } + + private static int RunCommand(string name, Action command) + { + try + { + command(); + + return 0; + } + catch (Exception exception) + { + return Fail($"{name} failed: {Describe(exception)}"); + } + } + + private static string Describe(Exception exception) + => exception.InnerException is { } inner ? exception.Message + " " + Describe(inner) : exception.Message; + + private static BenchmarkProbeResult RunProbe(Func probe) + { + BenchmarkProbeResult result = probe(); + + Console.WriteLine( + $"Probe passed: checked {result.Classes} benchmark classes, {result.ParameterSets} parameter sets and {result.Cases} cases."); + + return result; + } + + // Writes run-manifest.json before and after BenchmarkDotNet, so an interrupted run stays at status running. + // A full run probes first and fails without timing anything when the suite build is not from HEAD or the probe throws. + private static int RunBenchmarks(Assembly assembly, BenchmarkRun run, Func probe) + { + string artifactsPath = Path.GetFullPath(run.Config.ArtifactsPath); + string manifestPath = Path.Combine(artifactsPath, "run-manifest.json"); + + if (Directory.Exists(artifactsPath) && Directory.EnumerateFileSystemEntries(artifactsPath).Any()) + return Fail($"Artifacts path '{artifactsPath}' is not empty. Choose an unused directory."); + + Directory.CreateDirectory(artifactsPath); + + BenchmarkRunManifest manifest = BenchmarkRunManifest.Start( + run, + BenchmarkSource.Read(assembly, artifactsPath), + BenchmarkHostEnvironment.Read(), + BenchmarkPackageVersions.Read(assembly), + run.Jobs.Select(BenchmarkJobDescription.Create).ToArray()); + + manifest.Write(manifestPath); + + if (run.Tier == BenchmarkTier.Full) + { + // The probe runs the loaded build, but BenchmarkDotNet rebuilds the suite from the working tree. + string? buildCommit = BenchmarkSource.ReadBuildCommit(assembly); + + if (manifest.Source.IsIdentified && + !string.Equals(buildCommit, manifest.Source.Commit, StringComparison.OrdinalIgnoreCase)) + { + return Finish( + BenchmarkRunStatus.Failed( + $"The suite was built from commit {buildCommit ?? "unknown"}, but HEAD is {manifest.Source.Commit}. Rebuild the suite before a full run."), + manifest.EffectiveJobs, + []); + } + + BenchmarkProbeResult probeResult; + + try + { + probeResult = RunProbe(probe); + } + catch (Exception exception) + { + return Finish(BenchmarkRunStatus.Failed($"Probe failed: {Describe(exception)}"), manifest.EffectiveJobs, []); + } + + manifest = manifest.WithProbe(probeResult); + manifest.Write(manifestPath); + } + + // BenchmarkDotNet searches for the suite's project below the nearest solution above the working directory. + // Starting in the suite folder makes that the solution nearest the suite, whatever directory the run started in. + if (BenchmarkProjectDirectory.Read(assembly) is { } projectDirectory) + Directory.SetCurrentDirectory(projectDirectory); + + BenchmarkRunStatus status; + BenchmarkJobDescription[] jobs = manifest.EffectiveJobs; + string[] warnings = []; + + try + { + Summary[] summaries = BenchmarkSwitcher.FromAssembly(assembly).Run(run.Arguments, run.Config).ToArray(); + BenchmarkJobDescription[] executedJobs = DescribeJobs( + summaries.SelectMany(summary => summary.Reports).Select(report => report.BenchmarkCase)); + + if (executedJobs.Length != 0) + jobs = executedJobs; + + status = BenchmarkRunStatus.Determine(summaries); + warnings = CollectWarnings(summaries, run.Tier); + } + catch (Exception exception) + { + status = BenchmarkRunStatus.Failed(Describe(exception)); + } + + return Finish(status, jobs, warnings); + + int Finish(BenchmarkRunStatus status, BenchmarkJobDescription[] jobs, string[] warnings) + { + manifest + .Finish( + status, + jobs, + warnings, + BenchmarkSource.Read(assembly, artifactsPath), + BenchmarkArgumentPolicy.HasMeasurementOverrides(run.Arguments)) + .Write(manifestPath); + + return status == BenchmarkRunStatus.Completed ? 0 : Fail(status.Error!); + } + } + + private static string[] CollectWarnings(Summary[] summaries, BenchmarkTier tier) + { + IEnumerable validationWarnings = summaries.SelectMany(summary => summary.ValidationErrors) + .Where(error => !error.IsCritical) + .Select(error => error.Message); + IEnumerable accuracyWarnings = tier == BenchmarkTier.Full + ? summaries.SelectMany(AccuracyWarnings) + : []; + + return validationWarnings + .Concat(summaries.SelectMany(AnalysisWarnings)) + .Concat(accuracyWarnings) + .Distinct() + .ToArray(); + } + + private static IEnumerable AccuracyWarnings(Summary summary) + => summary.Reports + .Where(report => report.ResultStatistics is not null) + .Select(report => BenchmarkAccuracyWarnings.Describe( + report.BenchmarkCase.DisplayInfo, + report.ResultStatistics!.Mean, + report.ResultStatistics.ConfidenceInterval.Margin)) + .OfType(); + + private static IEnumerable AnalysisWarnings(Summary summary) + { + ImmutableConfig[] configs = summary.BenchmarksCases.Select(benchmark => benchmark.Config).Distinct().ToArray(); + var analyserConclusions = configs.SelectMany(config => config.GetAnalysers()).Distinct() + .SelectMany(analyser => analyser.Analyse(summary)); + + var configurationConclusions = configs.SelectMany(config => config.ConfigAnalysisConclusion); + + return analyserConclusions.Concat(configurationConclusions) + .Distinct() + .Select(conclusion => $"{conclusion.AnalyserId} ({conclusion.Kind}), " + + $"{conclusion.Report?.BenchmarkCase.DisplayInfo ?? summary.Title}: {conclusion.Message}"); + } + + private static BenchmarkJobDescription[] DescribeJobs(IEnumerable cases) + => cases.Select(benchmark => (Job: BenchmarkJobDescription.Create(benchmark.Job), Benchmark: benchmark.DisplayInfo)) + .GroupBy(item => (item.Job.Id, item.Job.Description)) + .Select(group => group.First().Job with + { + Benchmarks = group.Select(item => item.Benchmark).Distinct().ToArray(), + }) + .ToArray(); + + private static bool IsCommand(string[] args, string command) + => args.Length == 1 && args[0].Equals(command, StringComparison.OrdinalIgnoreCase); + + private static int Fail(string message) + { + Console.Error.WriteLine(message); + + return 1; + } +} diff --git a/benchmarks/RequestFlow.Benchmarks.Harness/Reporting/BenchmarkAccuracyWarnings.cs b/benchmarks/RequestFlow.Benchmarks.Harness/Reporting/BenchmarkAccuracyWarnings.cs new file mode 100644 index 0000000..9bd7ec7 --- /dev/null +++ b/benchmarks/RequestFlow.Benchmarks.Harness/Reporting/BenchmarkAccuracyWarnings.cs @@ -0,0 +1,29 @@ +namespace RequestFlow.Benchmarks.Harness; + +internal static class BenchmarkAccuracyWarnings +{ + internal static string? Describe(string benchmark, double mean, double confidenceIntervalHalfWidth) + { + if (!double.IsFinite(mean) || mean <= 0 || !double.IsFinite(confidenceIntervalHalfWidth)) + { + return FormattableString.Invariant( + $"{benchmark}: cannot calculate relative timing error from mean = {mean:G6} ns and 99.9% confidence interval half-width = {confidenceIntervalHalfWidth:G6} ns. ") + + "The mean must be finite and greater than zero, and the half-width must be finite. " + + "Inspect BenchmarkDotNet's validation messages and raw measurements to determine why these statistics are invalid. " + + "This row and ratios involving it cannot support timing claims."; + } + + double relativeError = confidenceIntervalHalfWidth / mean; + + if (relativeError <= BenchmarkConfig.MaximumRelativeError) + return null; + + return FormattableString.Invariant( + $"{benchmark}: mean = {mean:G6} ns; 99.9% confidence interval half-width = {confidenceIntervalHalfWidth:G6} ns. ") + + FormattableString.Invariant( + $"Relative error (half-width / mean) is {relativeError * 100:G6}%, above the {BenchmarkConfig.MaximumRelativeError * 100:G6}% target. ") + + "This measures uncertainty in the estimated mean, not the range of individual operation times. " + + "Inspect raw measurements for timing drift within each launch and disagreement between launches; these summary statistics do not identify the cause. " + + "Keep this row and ratios involving it out of timing claims until measurement precision is established."; + } +} diff --git a/benchmarks/RequestFlow.Benchmarks.Harness/Reporting/BenchmarkHostEnvironment.cs b/benchmarks/RequestFlow.Benchmarks.Harness/Reporting/BenchmarkHostEnvironment.cs new file mode 100644 index 0000000..f36d017 --- /dev/null +++ b/benchmarks/RequestFlow.Benchmarks.Harness/Reporting/BenchmarkHostEnvironment.cs @@ -0,0 +1,73 @@ +using System.Collections; +using System.Diagnostics; +using System.Runtime; +using System.Runtime.InteropServices; + +namespace RequestFlow.Benchmarks.Harness; + +internal sealed record BenchmarkHostEnvironment( + string OperatingSystem, + string Runtime, + string Architecture, + int ProcessorCount, + string? ProcessorAffinity, + bool IsServerGc, + IReadOnlyDictionary RuntimeVariables) +{ + private static readonly string[] RuntimePrefixes = ["DOTNET_", "COMPlus_", "CORECLR_"]; + + // SDK and host settings that leave code execution unchanged, as set by dotnet run, the CI workflow, + // setup-dotnet and the GitHub runner images. A trailing * matches a prefix. + private static readonly string[] IgnoredVariables = + [ + "DOTNET_CLI_*", + "DOTNET_LAUNCH_PROFILE", + "DOTNET_NOLOGO", + "DOTNET_ROOT*", + "DOTNET_SKIP_FIRST_TIME_EXPERIENCE", + "DOTNET_MULTILEVEL_LOOKUP", + ]; + + /// + /// The captured runtime variables outside the ignore list, which disqualify a run from publication. + /// + internal string[] ExecutionVariables => RuntimeVariables.Keys.Where(name => !IsIgnored(name)).ToArray(); + + internal static BenchmarkHostEnvironment Read() => Read(Environment.GetEnvironmentVariables()); + + internal static BenchmarkHostEnvironment Read(IDictionary environment) + { + var variables = new SortedDictionary(StringComparer.Ordinal); + + foreach (DictionaryEntry entry in environment) + { + if (entry.Key is string name && RuntimePrefixes.Any(prefix => name.StartsWith(prefix, StringComparison.OrdinalIgnoreCase))) + variables[name] = entry.Value as string ?? ""; + } + + return new( + RuntimeInformation.OSDescription, + RuntimeInformation.FrameworkDescription, + RuntimeInformation.ProcessArchitecture.ToString(), + Environment.ProcessorCount, + ReadProcessorAffinity(), + GCSettings.IsServerGC, + variables); + } + + // BenchmarkDotNet child processes inherit this mask when no job sets an affinity. + private static string? ReadProcessorAffinity() + { + if (!System.OperatingSystem.IsWindows() && !System.OperatingSystem.IsLinux()) + return null; + + using Process process = Process.GetCurrentProcess(); + + return $"0x{process.ProcessorAffinity:X}"; + } + + private static bool IsIgnored(string name) + => IgnoredVariables.Any(ignored => ignored.EndsWith('*') + ? name.StartsWith(ignored[..^1], StringComparison.OrdinalIgnoreCase) + : name.Equals(ignored, StringComparison.OrdinalIgnoreCase)); +} diff --git a/benchmarks/RequestFlow.Benchmarks.Harness/Reporting/BenchmarkJobDescription.cs b/benchmarks/RequestFlow.Benchmarks.Harness/Reporting/BenchmarkJobDescription.cs new file mode 100644 index 0000000..5a0a288 --- /dev/null +++ b/benchmarks/RequestFlow.Benchmarks.Harness/Reporting/BenchmarkJobDescription.cs @@ -0,0 +1,26 @@ +using BenchmarkDotNet.Characteristics; +using BenchmarkDotNet.Jobs; + +namespace RequestFlow.Benchmarks.Harness; + +internal sealed record BenchmarkJobDescription( + string Id, + string Description, + IReadOnlyDictionary Characteristics, + string[] Benchmarks) +{ + internal static BenchmarkJobDescription Create(Job job) + { + var characteristics = job.GetCharacteristicsWithValues() + .Where(characteristic => !characteristic.HasChildCharacteristics && + characteristic != CharacteristicObject.IdCharacteristic) + .OrderBy(characteristic => characteristic.DeclaringType.Name, StringComparer.Ordinal) + .ThenBy(characteristic => characteristic.Id, StringComparer.Ordinal) + .ToDictionary(characteristic => $"{characteristic.DeclaringType.Name}.{characteristic.Id}", + characteristic => CharacteristicPresenter.DefaultPresenter.ToPresentation(job, characteristic)); + + string description = string.Join(", ", characteristics.Select(pair => $"{pair.Key}={pair.Value}")); + + return new BenchmarkJobDescription(job.ResolvedId, description, characteristics, []); + } +} diff --git a/benchmarks/RequestFlow.Benchmarks.Harness/Reporting/BenchmarkPackageVersions.cs b/benchmarks/RequestFlow.Benchmarks.Harness/Reporting/BenchmarkPackageVersions.cs new file mode 100644 index 0000000..104dea5 --- /dev/null +++ b/benchmarks/RequestFlow.Benchmarks.Harness/Reporting/BenchmarkPackageVersions.cs @@ -0,0 +1,48 @@ +using System.Text.Json; + +namespace RequestFlow.Benchmarks.Harness; + +internal static class BenchmarkPackageVersions +{ + internal static IReadOnlyDictionary Read(System.Reflection.Assembly assembly) + { + var packages = new SortedDictionary(StringComparer.OrdinalIgnoreCase); + string depsPath = Path.ChangeExtension(assembly.Location, ".deps.json"); + + if (File.Exists(depsPath)) + { + using JsonDocument document = JsonDocument.Parse(File.ReadAllText(depsPath)); + + AddPackages(document, packages); + } + + using Stream? assets = assembly.GetManifestResourceStream("RequestFlow.Benchmarks.BuildDependencies.json"); + + if (assets is not null) + { + using JsonDocument document = JsonDocument.Parse(assets); + + AddPackages(document, packages); + } + + return packages; + } + + private static void AddPackages(JsonDocument document, IDictionary packages) + { + if (!document.RootElement.TryGetProperty("libraries", out JsonElement libraries)) + return; + + foreach (JsonProperty library in libraries.EnumerateObject()) + { + if (!library.Value.TryGetProperty("type", out JsonElement type) || + !type.GetString()!.Equals("package", StringComparison.OrdinalIgnoreCase)) + continue; + + int separator = library.Name.LastIndexOf('/'); + + if (separator > 0) + packages[library.Name[..separator]] = library.Name[(separator + 1)..]; + } + } +} diff --git a/benchmarks/RequestFlow.Benchmarks.Harness/Reporting/BenchmarkProbeResult.cs b/benchmarks/RequestFlow.Benchmarks.Harness/Reporting/BenchmarkProbeResult.cs new file mode 100644 index 0000000..44f9010 --- /dev/null +++ b/benchmarks/RequestFlow.Benchmarks.Harness/Reporting/BenchmarkProbeResult.cs @@ -0,0 +1,6 @@ +namespace RequestFlow.Benchmarks.Harness; + +/// +/// The counts a passed probe checked before timing. +/// +public sealed record BenchmarkProbeResult(int Classes, int ParameterSets, int Cases); diff --git a/benchmarks/RequestFlow.Benchmarks.Harness/Reporting/BenchmarkProjectDirectory.cs b/benchmarks/RequestFlow.Benchmarks.Harness/Reporting/BenchmarkProjectDirectory.cs new file mode 100644 index 0000000..bca9690 --- /dev/null +++ b/benchmarks/RequestFlow.Benchmarks.Harness/Reporting/BenchmarkProjectDirectory.cs @@ -0,0 +1,14 @@ +using System.Reflection; + +namespace RequestFlow.Benchmarks.Harness; + +internal static class BenchmarkProjectDirectory +{ + internal static string? Read(Assembly assembly) + { + string? directory = assembly.GetCustomAttributes() + .SingleOrDefault(attribute => attribute.Key == "RequestFlow.Benchmarks.ProjectDirectory")?.Value; + + return directory is not null && Path.IsPathFullyQualified(directory) ? directory : null; + } +} diff --git a/benchmarks/RequestFlow.Benchmarks.Harness/Reporting/BenchmarkRunManifest.cs b/benchmarks/RequestFlow.Benchmarks.Harness/Reporting/BenchmarkRunManifest.cs new file mode 100644 index 0000000..60ae181 --- /dev/null +++ b/benchmarks/RequestFlow.Benchmarks.Harness/Reporting/BenchmarkRunManifest.cs @@ -0,0 +1,92 @@ +using System.Text.Json; + +namespace RequestFlow.Benchmarks.Harness; + +internal sealed record BenchmarkRunManifest( + string Suite, + string Tier, + DateTimeOffset StartedUtc, + DateTimeOffset? EndedUtc, + string[] Arguments, + BenchmarkSource Source, + BenchmarkSource? FinalSource, + bool SourceUnchanged, + BenchmarkHostEnvironment HostEnvironment, + IReadOnlyDictionary Packages, + BenchmarkJobDescription[] EffectiveJobs, + BenchmarkProbeResult? Probe, + string[] Warnings, + string Status, + string? Error, + bool PublicationCandidate) +{ + private static readonly JsonSerializerOptions SerializerOptions = new() + { + PropertyNamingPolicy = JsonNamingPolicy.CamelCase, + WriteIndented = true, + }; + + internal static BenchmarkRunManifest Start( + BenchmarkRun run, + BenchmarkSource source, + BenchmarkHostEnvironment environment, + IReadOnlyDictionary packages, + BenchmarkJobDescription[] effectiveJobs) + => new( + run.Suite, + run.Tier.ToString().ToLowerInvariant(), + DateTimeOffset.UtcNow, + null, + run.OriginalArguments, + source, + null, + false, + environment, + packages, + effectiveJobs, + null, + [], + BenchmarkRunStatus.Running.Name, + null, + false); + + internal BenchmarkRunManifest WithProbe(BenchmarkProbeResult probe) => this with { Probe = probe }; + + /// + /// Records the outcome. + /// Only a completed full run with a passed probe, from clean, unchanged source without measurement overrides or runtime variables, is a publication candidate. + /// + internal BenchmarkRunManifest Finish( + BenchmarkRunStatus status, + BenchmarkJobDescription[] effectiveJobs, + string[] warnings, + BenchmarkSource finalSource, + bool hasMeasurementOverrides) + { + bool unchanged = Source.IsSameAs(finalSource); + string[] executionVariables = HostEnvironment.ExecutionVariables; + List allWarnings = [.. warnings]; + + if (!unchanged) + allWarnings.Add("Source revalidation failed: HEAD or the Git status of a build input changed during the run, or Git could not identify the source. This run is not a publication candidate."); + + if (executionVariables.Length > 0) + allWarnings.Add($"Runtime environment variables are set: {string.Join(", ", executionVariables)}. This run is not a publication candidate."); + + return this with + { + EndedUtc = DateTimeOffset.UtcNow, + FinalSource = finalSource, + SourceUnchanged = unchanged, + EffectiveJobs = effectiveJobs, + Warnings = [.. allWarnings], + Status = status.Name, + Error = status.Error, + PublicationCandidate = status == BenchmarkRunStatus.Completed && Tier == "full" && + Source.IsIdentified && !Source.IsDirty && unchanged && !hasMeasurementOverrides && + executionVariables.Length == 0 && Probe is not null, + }; + } + + internal void Write(string path) => File.WriteAllText(path, JsonSerializer.Serialize(this, SerializerOptions)); +} diff --git a/benchmarks/RequestFlow.Benchmarks.Harness/Reporting/BenchmarkSource.cs b/benchmarks/RequestFlow.Benchmarks.Harness/Reporting/BenchmarkSource.cs new file mode 100644 index 0000000..23974ad --- /dev/null +++ b/benchmarks/RequestFlow.Benchmarks.Harness/Reporting/BenchmarkSource.cs @@ -0,0 +1,172 @@ +using System.Diagnostics; +using System.Reflection; +using System.Text.Json.Serialization; + +namespace RequestFlow.Benchmarks.Harness; + +internal sealed record BenchmarkSource(string? Commit, bool IsDirty, string[]? Changes) +{ + private static readonly string[] BuildFiles = + [ + "Directory.Build.props", "Directory.Build.targets", "Directory.Packages.props", "global.json", "nuget.config", + "RequestFlow.snk", + ]; + + internal static readonly BenchmarkSource Unidentified = new(null, false, null); + + [JsonIgnore] + public bool IsIdentified => Commit is not null && Changes is not null; + + internal bool IsSameAs(BenchmarkSource other) + => IsIdentified && other.IsIdentified && Commit == other.Commit && Changes!.SequenceEqual(other.Changes!); + + /// + /// Reads HEAD and the Git status of every path that feeds the suite build, ignored files included. + /// + internal static BenchmarkSource Read(Assembly assembly, string artifactsPath) + { + string? projectDirectory = BenchmarkProjectDirectory.Read(assembly); + string? root = projectDirectory is null ? null : RunGit(projectDirectory, "rev-parse", "--show-toplevel"); + + if (root is null) + return Unidentified; + + root = Path.GetFullPath(root); + + List arguments = + [ + "status", "--porcelain", "--untracked-files=all", "--ignored", "--", + .. InputPaths(assembly, root), + ]; + + if (Relative(root, artifactsPath) is var artifacts && IsInside(artifacts)) + arguments.Add(":(exclude)" + artifacts); + + string? commit = RunGit(root, "rev-parse", "HEAD"); + string? status = RunGit(root, [.. arguments]); + + if (commit is null || status is null) + return Unidentified; + + string[] changes = ReadChanges(status); + + return new BenchmarkSource(commit.Trim(), changes.Length > 0, changes); + } + + /// + /// Reads the commit that the SDK appends after + in the informational version at build time. + /// + internal static string? ReadBuildCommit(Assembly assembly) + { + string? version = assembly.GetCustomAttribute()?.InformationalVersion; + int separator = version?.LastIndexOf('+') ?? -1; + + return separator < 0 ? null : version![(separator + 1)..]; + } + + /// + /// Splits porcelain status output into lines and drops build output below bin or obj folders. + /// + /// + /// A :(exclude,glob) pathspec would make Git also report nested repositories outside the input paths. + /// + internal static string[] ReadChanges(string status) + => status.Split('\n', StringSplitOptions.RemoveEmptyEntries) + .Select(line => line.TrimEnd('\r')) + .Where(line => !IsBuildOutput(line)) + .ToArray(); + + // A line is two status letters, a space and a path, or for a rename the old and new paths joined by " -> ". + private static bool IsBuildOutput(string line) + => line.Length > 3 && line[3..].Split(" -> ").All(path => path.Trim('"').Split('/').Any(segment => segment is "bin" or "obj")); + + private static SortedSet InputPaths(Assembly assembly, string root) + { + var paths = new SortedSet(StringComparer.Ordinal) { "src" }; + + foreach (string project in ProjectDirectories(assembly)) + { + paths.Add(Relative(root, project)); + + for (string? directory = Path.GetDirectoryName(project); + directory is not null && IsInside(Relative(root, directory)); + directory = Path.GetDirectoryName(directory)) + { + foreach (string file in BuildFiles.Select(name => Path.Combine(directory, name)).Where(File.Exists)) + paths.Add(Relative(root, file)); + } + } + + return paths; + } + + private static IEnumerable ProjectDirectories(Assembly assembly) + { + var visited = new HashSet(StringComparer.Ordinal); + var pending = new Queue([assembly]); + + while (pending.TryDequeue(out Assembly? current)) + { + if (!visited.Add(current.FullName!) || BenchmarkProjectDirectory.Read(current) is not { } directory) + continue; + + yield return directory; + + foreach (AssemblyName reference in current.GetReferencedAssemblies()) + { + if (TryLoad(reference) is { } referenced) + pending.Enqueue(referenced); + } + } + } + + private static Assembly? TryLoad(AssemblyName name) + { + try + { + return Assembly.Load(name); + } + catch + { + return null; + } + } + + private static string Relative(string root, string path) + => Path.GetRelativePath(root, Path.GetFullPath(path)).Replace('\\', '/'); + + private static bool IsInside(string relative) + => relative != ".." && !relative.StartsWith("../", StringComparison.Ordinal) && !Path.IsPathRooted(relative); + + private static string? RunGit(string workingDirectory, params string[] arguments) + { + try + { + var start = new ProcessStartInfo("git") + { + WorkingDirectory = workingDirectory, + RedirectStandardOutput = true, + UseShellExecute = false, + CreateNoWindow = true, + }; + + foreach (string argument in arguments) + start.ArgumentList.Add(argument); + + using Process? process = Process.Start(start); + + if (process is null) + return null; + + string output = process.StandardOutput.ReadToEnd(); + + process.WaitForExit(); + + return process.ExitCode == 0 ? output.TrimEnd() : null; + } + catch + { + return null; + } + } +} diff --git a/benchmarks/RequestFlow.Benchmarks.Harness/RequestFlow.Benchmarks.Harness.csproj b/benchmarks/RequestFlow.Benchmarks.Harness/RequestFlow.Benchmarks.Harness.csproj new file mode 100644 index 0000000..9e92273 --- /dev/null +++ b/benchmarks/RequestFlow.Benchmarks.Harness/RequestFlow.Benchmarks.Harness.csproj @@ -0,0 +1,13 @@ + + + + net10.0 + enable + enable + + + + + + + diff --git a/benchmarks/RequestFlow.Benchmarks.Harness/packages.lock.json b/benchmarks/RequestFlow.Benchmarks.Harness/packages.lock.json new file mode 100644 index 0000000..08415b1 --- /dev/null +++ b/benchmarks/RequestFlow.Benchmarks.Harness/packages.lock.json @@ -0,0 +1,171 @@ +{ + "version": 1, + "dependencies": { + "net10.0": { + "BenchmarkDotNet": { + "type": "Direct", + "requested": "[0.15.8, )", + "resolved": "0.15.8", + "contentHash": "paCfrWxSeHqn3rUZc0spYXVFnHCF0nzRhG0nOLnyTjZYs8spsimBaaNmb3vwqvALKIplbYq/TF393vYiYSnh/Q==", + "dependencies": { + "BenchmarkDotNet.Annotations": "0.15.8", + "CommandLineParser": "2.9.1", + "Gee.External.Capstone": "2.3.0", + "Iced": "1.21.0", + "Microsoft.CodeAnalysis.CSharp": "4.14.0", + "Microsoft.Diagnostics.Runtime": "3.1.512801", + "Microsoft.Diagnostics.Tracing.TraceEvent": "3.1.21", + "Microsoft.DotNet.PlatformAbstractions": "3.1.6", + "Perfolizer": "[0.6.1]", + "System.Management": "9.0.5" + } + }, + "BenchmarkDotNet.Annotations": { + "type": "Transitive", + "resolved": "0.15.8", + "contentHash": "hfucY0ycAsB0SsoaZcaAp9oq5wlWBJcylvEJb9pmvdYUx6PD6S4mDiYnZWjdjAlLhIpe/xtGCwzORfzAzPqvzA==" + }, + "CommandLineParser": { + "type": "Transitive", + "resolved": "2.9.1", + "contentHash": "OE0sl1/sQ37bjVsPKKtwQlWDgqaxWgtme3xZz7JssWUzg5JpMIyHgCTY9MVMxOg48fJ1AgGT3tgdH5m/kQ5xhA==" + }, + "Gee.External.Capstone": { + "type": "Transitive", + "resolved": "2.3.0", + "contentHash": "2ap/rYmjtzCOT8hxrnEW/QeiOt+paD8iRrIcdKX0cxVwWLFa1e+JDBNeECakmccXrSFeBQuu5AV8SNkipFMMMw==" + }, + "Iced": { + "type": "Transitive", + "resolved": "1.21.0", + "contentHash": "dv5+81Q1TBQvVMSOOOmRcjJmvWcX3BZPZsIq31+RLc5cNft0IHAyNlkdb7ZarOWG913PyBoFDsDXoCIlKmLclg==" + }, + "Microsoft.CodeAnalysis.Analyzers": { + "type": "Transitive", + "resolved": "3.11.0", + "contentHash": "v/EW3UE8/lbEYHoC2Qq7AR/DnmvpgdtAMndfQNmpuIMx/Mto8L5JnuCfdBYtgvalQOtfNCnxFejxuRrryvUTsg==" + }, + "Microsoft.CodeAnalysis.Common": { + "type": "Transitive", + "resolved": "4.14.0", + "contentHash": "PC3tuwZYnC+idaPuoC/AZpEdwrtX7qFpmnrfQkgobGIWiYmGi5MCRtl5mx6QrfMGQpK78X2lfIEoZDLg/qnuHg==", + "dependencies": { + "Microsoft.CodeAnalysis.Analyzers": "3.11.0" + } + }, + "Microsoft.CodeAnalysis.CSharp": { + "type": "Transitive", + "resolved": "4.14.0", + "contentHash": "568a6wcTivauIhbeWcCwfWwIn7UV7MeHEBvFB2uzGIpM2OhJ4eM/FZ8KS0yhPoNxnSpjGzz7x7CIjTxhslojQA==", + "dependencies": { + "Microsoft.CodeAnalysis.Analyzers": "3.11.0", + "Microsoft.CodeAnalysis.Common": "[4.14.0]" + } + }, + "Microsoft.Diagnostics.NETCore.Client": { + "type": "Transitive", + "resolved": "0.2.510501", + "contentHash": "juoqJYMDs+lRrrZyOkXXMImJHneCF23cuvO4waFRd2Ds7j+ZuGIPbJm0Y/zz34BdeaGiiwGWraMUlln05W1PCQ==", + "dependencies": { + "Microsoft.Extensions.Logging": "6.0.0" + } + }, + "Microsoft.Diagnostics.Runtime": { + "type": "Transitive", + "resolved": "3.1.512801", + "contentHash": "0lMUDr2oxNZa28D6NH5BuSQEe5T9tZziIkvkD44YkkCGQXPJqvFjLq5ZQq1hYLl3RjQJrY+hR0jFgap+EWPDTw==", + "dependencies": { + "Microsoft.Diagnostics.NETCore.Client": "0.2.410101" + } + }, + "Microsoft.Diagnostics.Tracing.TraceEvent": { + "type": "Transitive", + "resolved": "3.1.21", + "contentHash": "/OrJFKaojSR6TkUKtwh8/qA9XWNtxLrXMqvEb89dBSKCWjaGVTbKMYodIUgF5deCEtmd6GXuRerciXGl5bhZ7Q==", + "dependencies": { + "Microsoft.Diagnostics.NETCore.Client": "0.2.510501", + "System.Reflection.TypeExtensions": "4.7.0" + } + }, + "Microsoft.DotNet.PlatformAbstractions": { + "type": "Transitive", + "resolved": "3.1.6", + "contentHash": "jek4XYaQ/PGUwDKKhwR8K47Uh1189PFzMeLqO83mXrXQVIpARZCcfuDedH50YDTepBkfijCZN5U/vZi++erxtg==" + }, + "Microsoft.Extensions.DependencyInjection": { + "type": "Transitive", + "resolved": "6.0.0", + "contentHash": "k6PWQMuoBDGGHOQTtyois2u4AwyVcIwL2LaSLlTZQm2CYcJ1pxbt6jfAnpWmzENA/wfrYRI/X9DTLoUkE4AsLw==", + "dependencies": { + "Microsoft.Extensions.DependencyInjection.Abstractions": "6.0.0" + } + }, + "Microsoft.Extensions.DependencyInjection.Abstractions": { + "type": "Transitive", + "resolved": "6.0.0", + "contentHash": "xlzi2IYREJH3/m6+lUrQlujzX8wDitm4QGnUu6kUXTQAWPuZY8i+ticFJbzfqaetLA6KR/rO6Ew/HuYD+bxifg==" + }, + "Microsoft.Extensions.Logging": { + "type": "Transitive", + "resolved": "6.0.0", + "contentHash": "eIbyj40QDg1NDz0HBW0S5f3wrLVnKWnDJ/JtZ+yJDFnDj90VoPuoPmFkeaXrtu+0cKm5GRAwoDf+dBWXK0TUdg==", + "dependencies": { + "Microsoft.Extensions.DependencyInjection": "6.0.0", + "Microsoft.Extensions.DependencyInjection.Abstractions": "6.0.0", + "Microsoft.Extensions.Logging.Abstractions": "6.0.0", + "Microsoft.Extensions.Options": "6.0.0" + } + }, + "Microsoft.Extensions.Logging.Abstractions": { + "type": "Transitive", + "resolved": "6.0.0", + "contentHash": "/HggWBbTwy8TgebGSX5DBZ24ndhzi93sHUBDvP1IxbZD7FDokYzdAr6+vbWGjw2XAfR2EJ1sfKUotpjHnFWPxA==" + }, + "Microsoft.Extensions.Options": { + "type": "Transitive", + "resolved": "6.0.0", + "contentHash": "dzXN0+V1AyjOe2xcJ86Qbo233KHuLEY0njf/P2Kw8SfJU+d45HNS2ctJdnEnrWbM9Ye2eFgaC5Mj9otRMU6IsQ==", + "dependencies": { + "Microsoft.Extensions.DependencyInjection.Abstractions": "6.0.0", + "Microsoft.Extensions.Primitives": "6.0.0" + } + }, + "Microsoft.Extensions.Primitives": { + "type": "Transitive", + "resolved": "6.0.0", + "contentHash": "9+PnzmQFfEFNR9J2aDTfJGGupShHjOuGw4VUv+JB044biSHrnmCIMD+mJHmb2H7YryrfBEXDurxQ47gJZdCKNQ==" + }, + "Perfolizer": { + "type": "Transitive", + "resolved": "0.6.1", + "contentHash": "CR1QmWg4XYBd1Pb7WseP+sDmV8nGPwvmowKynExTqr3OuckIGVMhvmN4LC5PGzfXqDlR295+hz/T7syA1CxEqA==", + "dependencies": { + "Pragmastat": "3.2.4" + } + }, + "Pragmastat": { + "type": "Transitive", + "resolved": "3.2.4", + "contentHash": "I5qFifWw/gaTQT52MhzjZpkm/JPlfjSeO/DTZJjO7+hTKI+0aGRgOgZ3NN6D96dDuuqbIAZSeA5RimtHjqrA2A==" + }, + "System.CodeDom": { + "type": "Transitive", + "resolved": "9.0.5", + "contentHash": "cuzLM2MWutf9ZBEMPYYfd0DXwYdvntp7VCT6a/wvbKCa2ZuvGmW74xi+YBa2mrfEieAXqM4TNKlMmSnfAfpUoQ==" + }, + "System.Management": { + "type": "Transitive", + "resolved": "9.0.5", + "contentHash": "n6o9PZm9p25+zAzC3/48K0oHnaPKTInRrxqFq1fi/5TPbMLjuoCm/h//mS3cUmSy+9AO1Z+qsC/Ilt/ZFatv5Q==", + "dependencies": { + "System.CodeDom": "9.0.5" + } + }, + "System.Reflection.TypeExtensions": { + "type": "Transitive", + "resolved": "4.7.0", + "contentHash": "VybpaOQQhqE6siHppMktjfGBw1GCwvCqiufqmP8F1nj7fTUNtW35LOEt3UZTEsECfo+ELAl/9o9nJx3U91i7vA==" + } + } + } +} \ No newline at end of file diff --git a/benchmarks/RequestFlow.Benchmarks.MediatR/Benchmarks/MediatRCatalogSizeBenchmarks.cs b/benchmarks/RequestFlow.Benchmarks.MediatR/Benchmarks/MediatRCatalogSizeBenchmarks.cs new file mode 100644 index 0000000..758f2cb --- /dev/null +++ b/benchmarks/RequestFlow.Benchmarks.MediatR/Benchmarks/MediatRCatalogSizeBenchmarks.cs @@ -0,0 +1,51 @@ +using BenchmarkDotNet.Attributes; +using MediatR; +using RequestFlow.Benchmarks.Core; +using RequestFlow.Benchmarks.RequestFlowFixtures; +using RequestFlow.Benchmarks.RequestFlowFixtures.Catalog; + +namespace RequestFlow.Benchmarks.MediatR; + +[BenchmarkScope(ScopeLifetime.RetainedScope)] +public class MediatRCatalogSizeBenchmarks +{ + [Params(4, 16, 64)] + public int CatalogSize { get; set; } + + [Params(CompletionKind.Cached, CompletionKind.Synchronous, CompletionKind.Suspending)] + public CompletionKind Completion { get; set; } + + private RetainedScopeHost _requestFlow = null!; + private RetainedScopeHost _mediatR = null!; + + private RuntimeTaskRequest _requestFlow1 = null!; + private MediatRCatalogRequest _mediatR1 = null!; + + [GlobalSetup] + public void Setup() + { + _requestFlow = new(RequestFlowSetup.Build(RequestFlowSetup.CatalogServices(CatalogSize, MediatRSetup.DispatcherLifetime))); + _mediatR = new(MediatRSetup.Build(MediatRSetup.CatalogServices(CatalogSize))); + + _requestFlow1 = new RuntimeTaskRequest(Completion); + _mediatR1 = new MediatRCatalogRequest(Completion); + + RequestFlow().GetAwaiter().GetResult(); + MediatR().GetAwaiter().GetResult(); + } + + [Benchmark(Baseline = true)] + public Task RequestFlow() + => _requestFlow.Service.SendAsync(_requestFlow1, CancellationToken.None); + + [Benchmark] + public Task MediatR() + => _mediatR.Service.Send(_mediatR1, CancellationToken.None); + + [GlobalCleanup] + public void Cleanup() + { + _requestFlow.Dispose(); + _mediatR.Dispose(); + } +} diff --git a/benchmarks/RequestFlow.Benchmarks.MediatR/Benchmarks/MediatREventParallelRetainedScopeBenchmarks.cs b/benchmarks/RequestFlow.Benchmarks.MediatR/Benchmarks/MediatREventParallelRetainedScopeBenchmarks.cs new file mode 100644 index 0000000..e639ab1 --- /dev/null +++ b/benchmarks/RequestFlow.Benchmarks.MediatR/Benchmarks/MediatREventParallelRetainedScopeBenchmarks.cs @@ -0,0 +1,51 @@ +using BenchmarkDotNet.Attributes; +using MediatR; +using RequestFlow.Benchmarks.Core; +using RequestFlow.Benchmarks.RequestFlowFixtures; + +namespace RequestFlow.Benchmarks.MediatR; + +[BenchmarkScope(ScopeLifetime.RetainedScope)] +public class MediatREventParallelRetainedScopeBenchmarks +{ + [Params(CompletionKind.Synchronous, CompletionKind.Suspending)] + public CompletionKind Completion { get; set; } + + [Params(1, 5)] + public int HandlerCount { get; set; } + + private RetainedScopeHost _requestFlow = null!; + private RetainedScopeHost _mediatR = null!; + + private EventMessage _requestFlowRequest = null!; + private MediatREvent _mediatRRequest = null!; + + [GlobalSetup] + public void Setup() + { + _requestFlow = new(RequestFlowSetup.Build( + RequestFlowSetup.EventServices(HandlerCount, parallel: true, dispatcherLifetime: MediatRSetup.DispatcherLifetime))); + _mediatR = new(MediatRSetup.Build(MediatRSetup.EventServices(HandlerCount, parallel: true))); + + _requestFlowRequest = new EventMessage(Completion); + _mediatRRequest = new MediatREvent(Completion); + + RequestFlow().GetAwaiter().GetResult(); + MediatR().GetAwaiter().GetResult(); + } + + [Benchmark(Baseline = true)] + public Task RequestFlow() + => _requestFlow.Service.PublishAsync(_requestFlowRequest, CancellationToken.None); + + [Benchmark] + public Task MediatR() + => _mediatR.Service.Publish(_mediatRRequest, CancellationToken.None); + + [GlobalCleanup] + public void Cleanup() + { + _requestFlow.Dispose(); + _mediatR.Dispose(); + } +} diff --git a/benchmarks/RequestFlow.Benchmarks.MediatR/Benchmarks/MediatREventParallelScopePerOperationBenchmarks.cs b/benchmarks/RequestFlow.Benchmarks.MediatR/Benchmarks/MediatREventParallelScopePerOperationBenchmarks.cs new file mode 100644 index 0000000..16f335d --- /dev/null +++ b/benchmarks/RequestFlow.Benchmarks.MediatR/Benchmarks/MediatREventParallelScopePerOperationBenchmarks.cs @@ -0,0 +1,68 @@ +using BenchmarkDotNet.Attributes; +using MediatR; +using Microsoft.Extensions.DependencyInjection; +using RequestFlow.Benchmarks.Core; +using RequestFlow.Benchmarks.RequestFlowFixtures; + +namespace RequestFlow.Benchmarks.MediatR; + +[BenchmarkScope(ScopeLifetime.ScopePerOperation)] +public class MediatREventParallelScopePerOperationBenchmarks +{ + [Params(CompletionKind.Synchronous, CompletionKind.Suspending)] + public CompletionKind Completion { get; set; } + + [Params(1, 5)] + public int HandlerCount { get; set; } + + private ScopePerOperationHost _requestFlow = null!; + private ScopePerOperationHost _mediatR = null!; + + private EventMessage _requestFlowRequest = null!; + private MediatREvent _mediatRRequest = null!; + + [GlobalSetup] + public void Setup() + { + _requestFlow = new(RequestFlowSetup.Build( + RequestFlowSetup.EventServices(HandlerCount, parallel: true, dispatcherLifetime: MediatRSetup.DispatcherLifetime))); + _mediatR = new(MediatRSetup.Build(MediatRSetup.EventServices(HandlerCount, parallel: true))); + + _requestFlowRequest = new EventMessage(Completion); + _mediatRRequest = new MediatREvent(Completion); + + RequestFlow().GetAwaiter().GetResult(); + MediatR().GetAwaiter().GetResult(); + } + + [Benchmark(Baseline = true)] + public async Task RequestFlow() + { + using IServiceScope scope = _requestFlow.Provider.CreateScope(); + + var publisher = scope.ServiceProvider + .GetRequiredService(); + + await publisher + .PublishAsync(_requestFlowRequest, CancellationToken.None).ConfigureAwait(false); + } + + [Benchmark] + public async Task MediatR() + { + using IServiceScope scope = _mediatR.Provider.CreateScope(); + + var publisher = scope.ServiceProvider + .GetRequiredService(); + + await publisher + .Publish(_mediatRRequest, CancellationToken.None).ConfigureAwait(false); + } + + [GlobalCleanup] + public void Cleanup() + { + _requestFlow.Dispose(); + _mediatR.Dispose(); + } +} diff --git a/benchmarks/RequestFlow.Benchmarks.MediatR/Benchmarks/MediatREventSequentialRetainedScopeBenchmarks.cs b/benchmarks/RequestFlow.Benchmarks.MediatR/Benchmarks/MediatREventSequentialRetainedScopeBenchmarks.cs new file mode 100644 index 0000000..a7a801a --- /dev/null +++ b/benchmarks/RequestFlow.Benchmarks.MediatR/Benchmarks/MediatREventSequentialRetainedScopeBenchmarks.cs @@ -0,0 +1,51 @@ +using BenchmarkDotNet.Attributes; +using MediatR; +using RequestFlow.Benchmarks.Core; +using RequestFlow.Benchmarks.RequestFlowFixtures; + +namespace RequestFlow.Benchmarks.MediatR; + +[BenchmarkScope(ScopeLifetime.RetainedScope)] +public class MediatREventSequentialRetainedScopeBenchmarks +{ + [Params(CompletionKind.Synchronous, CompletionKind.Suspending)] + public CompletionKind Completion { get; set; } + + [Params(1, 5)] + public int HandlerCount { get; set; } + + private RetainedScopeHost _requestFlow = null!; + private RetainedScopeHost _mediatR = null!; + + private EventMessage _requestFlowRequest = null!; + private MediatREvent _mediatRRequest = null!; + + [GlobalSetup] + public void Setup() + { + _requestFlow = new(RequestFlowSetup.Build( + RequestFlowSetup.EventServices(HandlerCount, parallel: false, dispatcherLifetime: MediatRSetup.DispatcherLifetime))); + _mediatR = new(MediatRSetup.Build(MediatRSetup.EventServices(HandlerCount, parallel: false))); + + _requestFlowRequest = new EventMessage(Completion); + _mediatRRequest = new MediatREvent(Completion); + + RequestFlow().GetAwaiter().GetResult(); + MediatR().GetAwaiter().GetResult(); + } + + [Benchmark(Baseline = true)] + public Task RequestFlow() + => _requestFlow.Service.PublishAsync(_requestFlowRequest, CancellationToken.None); + + [Benchmark] + public Task MediatR() + => _mediatR.Service.Publish(_mediatRRequest, CancellationToken.None); + + [GlobalCleanup] + public void Cleanup() + { + _requestFlow.Dispose(); + _mediatR.Dispose(); + } +} diff --git a/benchmarks/RequestFlow.Benchmarks.MediatR/Benchmarks/MediatREventSequentialScopePerOperationBenchmarks.cs b/benchmarks/RequestFlow.Benchmarks.MediatR/Benchmarks/MediatREventSequentialScopePerOperationBenchmarks.cs new file mode 100644 index 0000000..615e21c --- /dev/null +++ b/benchmarks/RequestFlow.Benchmarks.MediatR/Benchmarks/MediatREventSequentialScopePerOperationBenchmarks.cs @@ -0,0 +1,68 @@ +using BenchmarkDotNet.Attributes; +using MediatR; +using Microsoft.Extensions.DependencyInjection; +using RequestFlow.Benchmarks.Core; +using RequestFlow.Benchmarks.RequestFlowFixtures; + +namespace RequestFlow.Benchmarks.MediatR; + +[BenchmarkScope(ScopeLifetime.ScopePerOperation)] +public class MediatREventSequentialScopePerOperationBenchmarks +{ + [Params(CompletionKind.Synchronous, CompletionKind.Suspending)] + public CompletionKind Completion { get; set; } + + [Params(1, 5)] + public int HandlerCount { get; set; } + + private ScopePerOperationHost _requestFlow = null!; + private ScopePerOperationHost _mediatR = null!; + + private EventMessage _requestFlowRequest = null!; + private MediatREvent _mediatRRequest = null!; + + [GlobalSetup] + public void Setup() + { + _requestFlow = new(RequestFlowSetup.Build( + RequestFlowSetup.EventServices(HandlerCount, parallel: false, dispatcherLifetime: MediatRSetup.DispatcherLifetime))); + _mediatR = new(MediatRSetup.Build(MediatRSetup.EventServices(HandlerCount, parallel: false))); + + _requestFlowRequest = new EventMessage(Completion); + _mediatRRequest = new MediatREvent(Completion); + + RequestFlow().GetAwaiter().GetResult(); + MediatR().GetAwaiter().GetResult(); + } + + [Benchmark(Baseline = true)] + public async Task RequestFlow() + { + using IServiceScope scope = _requestFlow.Provider.CreateScope(); + + var publisher = scope.ServiceProvider + .GetRequiredService(); + + await publisher + .PublishAsync(_requestFlowRequest, CancellationToken.None).ConfigureAwait(false); + } + + [Benchmark] + public async Task MediatR() + { + using IServiceScope scope = _mediatR.Provider.CreateScope(); + + var publisher = scope.ServiceProvider + .GetRequiredService(); + + await publisher + .Publish(_mediatRRequest, CancellationToken.None).ConfigureAwait(false); + } + + [GlobalCleanup] + public void Cleanup() + { + _requestFlow.Dispose(); + _mediatR.Dispose(); + } +} diff --git a/benchmarks/RequestFlow.Benchmarks.MediatR/Benchmarks/MediatRMixedTrafficBenchmarks.cs b/benchmarks/RequestFlow.Benchmarks.MediatR/Benchmarks/MediatRMixedTrafficBenchmarks.cs new file mode 100644 index 0000000..4e8c70c --- /dev/null +++ b/benchmarks/RequestFlow.Benchmarks.MediatR/Benchmarks/MediatRMixedTrafficBenchmarks.cs @@ -0,0 +1,70 @@ +using BenchmarkDotNet.Attributes; +using MediatR; +using RequestFlow.Benchmarks.Core; +using RequestFlow.Benchmarks.RequestFlowFixtures; +using RequestFlow.Benchmarks.RequestFlowFixtures.Catalog; + +namespace RequestFlow.Benchmarks.MediatR; + +[BenchmarkScope(ScopeLifetime.RetainedScope)] +public class MediatRMixedTrafficBenchmarks +{ + [Params(4, 16, 64)] + public int CatalogSize { get; set; } + + [Params(CompletionKind.Cached, CompletionKind.Synchronous, CompletionKind.Suspending)] + public CompletionKind Completion { get; set; } + + private RetainedScopeHost _requestFlow = null!; + private RetainedScopeHost _mediatR = null!; + + private global::RequestFlow.IRequest[] _requestFlowRequests = null!; + private global::MediatR.IRequest[] _mediatRRequests = null!; + + [GlobalSetup] + public void Setup() + { + _requestFlow = new(RequestFlowSetup.Build(RequestFlowSetup.CatalogServices(CatalogSize, MediatRSetup.DispatcherLifetime))); + _mediatR = new(MediatRSetup.Build(MediatRSetup.CatalogServices(CatalogSize))); + + IReadOnlyList order = RuntimeCatalog.MixedTrafficOrder(CatalogSize); + + _requestFlowRequests = CreateRequests>(typeof(RuntimeTaskRequest<>), order); + _mediatRRequests = CreateRequests>(typeof(MediatRCatalogRequest<>), order); + + RequestFlow().GetAwaiter().GetResult(); + MediatR().GetAwaiter().GetResult(); + } + + [Benchmark(Baseline = true, OperationsPerInvoke = RuntimeCatalog.MixedTrafficBatch)] + public async Task RequestFlow() + { + int sum = 0; + + foreach (global::RequestFlow.IRequest request in _requestFlowRequests) + sum += await _requestFlow.Service.SendAsync(request, CancellationToken.None).ConfigureAwait(false); + + return sum; + } + + [Benchmark(OperationsPerInvoke = RuntimeCatalog.MixedTrafficBatch)] + public async Task MediatR() + { + int sum = 0; + + foreach (global::MediatR.IRequest request in _mediatRRequests) + sum += await _mediatR.Service.Send(request, CancellationToken.None).ConfigureAwait(false); + + return sum; + } + + [GlobalCleanup] + public void Cleanup() + { + _requestFlow.Dispose(); + _mediatR.Dispose(); + } + + private TRequest[] CreateRequests(Type requestDefinition, IReadOnlyList order) + => [.. order.Select(tag => (TRequest)Activator.CreateInstance(requestDefinition.MakeGenericType(tag), Completion)!)]; +} diff --git a/benchmarks/RequestFlow.Benchmarks.MediatR/Benchmarks/MediatRStreamRetainedScopeBenchmarks.cs b/benchmarks/RequestFlow.Benchmarks.MediatR/Benchmarks/MediatRStreamRetainedScopeBenchmarks.cs new file mode 100644 index 0000000..43449c2 --- /dev/null +++ b/benchmarks/RequestFlow.Benchmarks.MediatR/Benchmarks/MediatRStreamRetainedScopeBenchmarks.cs @@ -0,0 +1,53 @@ +using BenchmarkDotNet.Attributes; +using MediatR; +using RequestFlow.Benchmarks.Core; +using RequestFlow.Benchmarks.RequestFlowFixtures; + +namespace RequestFlow.Benchmarks.MediatR; + +[BenchmarkScope(ScopeLifetime.RetainedScope)] +public class MediatRStreamRetainedScopeBenchmarks +{ + [Params(1, 10, 100)] + public int ItemCount { get; set; } + + [Params(CompletionKind.Synchronous, CompletionKind.Suspending)] + public CompletionKind Completion { get; set; } + + [Params(0, 1, 5)] + public int StageCount { get; set; } + + private RetainedScopeHost _requestFlow = null!; + private RetainedScopeHost _mediatR = null!; + + private StreamRequest _requestFlowRequest = null!; + private MediatRStreamRequest _mediatRRequest = null!; + + [GlobalSetup] + public void Setup() + { + _requestFlow = new(RequestFlowSetup.Build(RequestFlowSetup.StreamServices(StageCount, MediatRSetup.DispatcherLifetime))); + _mediatR = new(MediatRSetup.Build(MediatRSetup.StreamServices(StageCount))); + + _requestFlowRequest = new StreamRequest(ItemCount, Completion); + _mediatRRequest = new MediatRStreamRequest(ItemCount, Completion); + + RequestFlow().GetAwaiter().GetResult(); + MediatR().GetAwaiter().GetResult(); + } + + [Benchmark(Baseline = true)] + public Task RequestFlow() + => BenchmarkWorkload.Consume(_requestFlow.Service.Stream(_requestFlowRequest, CancellationToken.None)); + + [Benchmark] + public Task MediatR() + => BenchmarkWorkload.Consume(_mediatR.Service.CreateStream(_mediatRRequest, CancellationToken.None)); + + [GlobalCleanup] + public void Cleanup() + { + _requestFlow.Dispose(); + _mediatR.Dispose(); + } +} diff --git a/benchmarks/RequestFlow.Benchmarks.MediatR/Benchmarks/MediatRStreamScopePerOperationBenchmarks.cs b/benchmarks/RequestFlow.Benchmarks.MediatR/Benchmarks/MediatRStreamScopePerOperationBenchmarks.cs new file mode 100644 index 0000000..4c39e61 --- /dev/null +++ b/benchmarks/RequestFlow.Benchmarks.MediatR/Benchmarks/MediatRStreamScopePerOperationBenchmarks.cs @@ -0,0 +1,72 @@ +using BenchmarkDotNet.Attributes; +using MediatR; +using Microsoft.Extensions.DependencyInjection; +using RequestFlow.Benchmarks.Core; +using RequestFlow.Benchmarks.RequestFlowFixtures; + +namespace RequestFlow.Benchmarks.MediatR; + +[BenchmarkScope(ScopeLifetime.ScopePerOperation)] +public class MediatRStreamScopePerOperationBenchmarks +{ + [Params(1, 10, 100)] + public int ItemCount { get; set; } + + [Params(CompletionKind.Synchronous, CompletionKind.Suspending)] + public CompletionKind Completion { get; set; } + + [Params(0, 1, 5)] + public int StageCount { get; set; } + + private ScopePerOperationHost _requestFlow = null!; + private ScopePerOperationHost _mediatR = null!; + + private StreamRequest _requestFlowRequest = null!; + private MediatRStreamRequest _mediatRRequest = null!; + + [GlobalSetup] + public void Setup() + { + _requestFlow = new(RequestFlowSetup.Build(RequestFlowSetup.StreamServices(StageCount, MediatRSetup.DispatcherLifetime))); + _mediatR = new(MediatRSetup.Build(MediatRSetup.StreamServices(StageCount))); + + _requestFlowRequest = new StreamRequest(ItemCount, Completion); + _mediatRRequest = new MediatRStreamRequest(ItemCount, Completion); + + RequestFlow().GetAwaiter().GetResult(); + MediatR().GetAwaiter().GetResult(); + } + + [Benchmark(Baseline = true)] + public async Task RequestFlow() + { + using IServiceScope scope = _requestFlow.Provider.CreateScope(); + + var streamDispatcher = scope.ServiceProvider + .GetRequiredService(); + + var stream = streamDispatcher.Stream(_requestFlowRequest, CancellationToken.None); + + return await BenchmarkWorkload.Consume(stream).ConfigureAwait(false); + } + + [Benchmark] + public async Task MediatR() + { + using IServiceScope scope = _mediatR.Provider.CreateScope(); + + var dispatcher = scope.ServiceProvider + .GetRequiredService(); + + var stream = dispatcher.CreateStream(_mediatRRequest, CancellationToken.None); + + return await BenchmarkWorkload.Consume(stream).ConfigureAwait(false); + } + + [GlobalCleanup] + public void Cleanup() + { + _requestFlow.Dispose(); + _mediatR.Dispose(); + } +} diff --git a/benchmarks/RequestFlow.Benchmarks.MediatR/Benchmarks/MediatRTaskDirectHandlerBenchmarks.cs b/benchmarks/RequestFlow.Benchmarks.MediatR/Benchmarks/MediatRTaskDirectHandlerBenchmarks.cs new file mode 100644 index 0000000..2807246 --- /dev/null +++ b/benchmarks/RequestFlow.Benchmarks.MediatR/Benchmarks/MediatRTaskDirectHandlerBenchmarks.cs @@ -0,0 +1,52 @@ +using BenchmarkDotNet.Attributes; +using RequestFlow.Benchmarks.Core; +using RequestFlow.Benchmarks.RequestFlowFixtures; + +namespace RequestFlow.Benchmarks.MediatR; + +[BenchmarkScope(ScopeLifetime.DirectHandler)] +public class MediatRTaskDirectHandlerBenchmarks +{ + [Params(CompletionKind.Cached, CompletionKind.Synchronous, CompletionKind.Suspending)] + public CompletionKind Completion { get; set; } + + private readonly ScopeWitness _requestFlowWitness = new(); + private readonly ScopeWitness _mediatRWitness = new(); + + private TaskRequestHandler _requestFlowHandler = null!; + private MediatRTaskHandler _mediatRHandler = null!; + + private TaskRequest _requestFlowRequest = null!; + private MediatRTaskRequest _mediatRRequest = null!; + + [GlobalSetup] + public void Setup() + { + _requestFlowHandler = new TaskRequestHandler(_requestFlowWitness); + _mediatRHandler = new MediatRTaskHandler(_mediatRWitness); + + _requestFlowRequest = new TaskRequest(Completion); + _mediatRRequest = new MediatRTaskRequest(Completion); + + RequestFlowDirectHandler().GetAwaiter().GetResult(); + MediatRDirectHandler().GetAwaiter().GetResult(); + } + + [Benchmark(Baseline = true)] + public Task RequestFlowDirectHandler() + => _requestFlowHandler.HandleAsync(_requestFlowRequest, CancellationToken.None); + + [Benchmark] + public Task MediatRDirectHandler() + => _mediatRHandler.Handle(_mediatRRequest, CancellationToken.None); + + [GlobalCleanup] + public void Cleanup() + { + _requestFlowHandler.Dispose(); + _mediatRHandler.Dispose(); + + _requestFlowWitness.Dispose(); + _mediatRWitness.Dispose(); + } +} diff --git a/benchmarks/RequestFlow.Benchmarks.MediatR/Benchmarks/MediatRTaskRetainedScopeBenchmarks.cs b/benchmarks/RequestFlow.Benchmarks.MediatR/Benchmarks/MediatRTaskRetainedScopeBenchmarks.cs new file mode 100644 index 0000000..3f4b812 --- /dev/null +++ b/benchmarks/RequestFlow.Benchmarks.MediatR/Benchmarks/MediatRTaskRetainedScopeBenchmarks.cs @@ -0,0 +1,50 @@ +using BenchmarkDotNet.Attributes; +using MediatR; +using RequestFlow.Benchmarks.Core; +using RequestFlow.Benchmarks.RequestFlowFixtures; + +namespace RequestFlow.Benchmarks.MediatR; + +[BenchmarkScope(ScopeLifetime.RetainedScope)] +public class MediatRTaskRetainedScopeBenchmarks +{ + [Params(CompletionKind.Cached, CompletionKind.Synchronous, CompletionKind.Suspending)] + public CompletionKind Completion { get; set; } + + [Params(0, 1, 5)] + public int StageCount { get; set; } + + private RetainedScopeHost _requestFlow = null!; + private RetainedScopeHost _mediatR = null!; + + private TaskRequest _requestFlowRequest = null!; + private MediatRTaskRequest _mediatRRequest = null!; + + [GlobalSetup] + public void Setup() + { + _requestFlow = new(RequestFlowSetup.Build(RequestFlowSetup.TaskServices(StageCount, MediatRSetup.DispatcherLifetime))); + _mediatR = new(MediatRSetup.Build(MediatRSetup.TaskServices(StageCount))); + + _requestFlowRequest = new TaskRequest(Completion); + _mediatRRequest = new MediatRTaskRequest(Completion); + + RequestFlow().GetAwaiter().GetResult(); + MediatR().GetAwaiter().GetResult(); + } + + [Benchmark(Baseline = true)] + public Task RequestFlow() + => _requestFlow.Service.SendAsync(_requestFlowRequest, CancellationToken.None); + + [Benchmark] + public Task MediatR() + => _mediatR.Service.Send(_mediatRRequest, CancellationToken.None); + + [GlobalCleanup] + public void Cleanup() + { + _requestFlow.Dispose(); + _mediatR.Dispose(); + } +} diff --git a/benchmarks/RequestFlow.Benchmarks.MediatR/Benchmarks/MediatRTaskScopePerOperationBenchmarks.cs b/benchmarks/RequestFlow.Benchmarks.MediatR/Benchmarks/MediatRTaskScopePerOperationBenchmarks.cs new file mode 100644 index 0000000..3881586 --- /dev/null +++ b/benchmarks/RequestFlow.Benchmarks.MediatR/Benchmarks/MediatRTaskScopePerOperationBenchmarks.cs @@ -0,0 +1,67 @@ +using BenchmarkDotNet.Attributes; +using MediatR; +using Microsoft.Extensions.DependencyInjection; +using RequestFlow.Benchmarks.Core; +using RequestFlow.Benchmarks.RequestFlowFixtures; + +namespace RequestFlow.Benchmarks.MediatR; + +[BenchmarkScope(ScopeLifetime.ScopePerOperation)] +public class MediatRTaskScopePerOperationBenchmarks +{ + [Params(CompletionKind.Cached, CompletionKind.Synchronous, CompletionKind.Suspending)] + public CompletionKind Completion { get; set; } + + [Params(0, 1, 5)] + public int StageCount { get; set; } + + private ScopePerOperationHost _requestFlow = null!; + private ScopePerOperationHost _mediatR = null!; + + private TaskRequest _requestFlowRequest = null!; + private MediatRTaskRequest _mediatRRequest = null!; + + [GlobalSetup] + public void Setup() + { + _requestFlow = new(RequestFlowSetup.Build(RequestFlowSetup.TaskServices(StageCount, MediatRSetup.DispatcherLifetime))); + _mediatR = new(MediatRSetup.Build(MediatRSetup.TaskServices(StageCount))); + + _requestFlowRequest = new TaskRequest(Completion); + _mediatRRequest = new MediatRTaskRequest(Completion); + + RequestFlow().GetAwaiter().GetResult(); + MediatR().GetAwaiter().GetResult(); + } + + [Benchmark(Baseline = true)] + public async Task RequestFlow() + { + using IServiceScope scope = _requestFlow.Provider.CreateScope(); + + var dispatcher = scope.ServiceProvider + .GetRequiredService(); + + return await dispatcher + .SendAsync(_requestFlowRequest, CancellationToken.None).ConfigureAwait(false); + } + + [Benchmark] + public async Task MediatR() + { + using IServiceScope scope = _mediatR.Provider.CreateScope(); + + var dispatcher = scope.ServiceProvider + .GetRequiredService(); + + return await dispatcher + .Send(_mediatRRequest, CancellationToken.None).ConfigureAwait(false); + } + + [GlobalCleanup] + public void Cleanup() + { + _requestFlow.Dispose(); + _mediatR.Dispose(); + } +} diff --git a/benchmarks/RequestFlow.Benchmarks.MediatR/Benchmarks/MediatRVoidRequestRetainedScopeBenchmarks.cs b/benchmarks/RequestFlow.Benchmarks.MediatR/Benchmarks/MediatRVoidRequestRetainedScopeBenchmarks.cs new file mode 100644 index 0000000..58f0cef --- /dev/null +++ b/benchmarks/RequestFlow.Benchmarks.MediatR/Benchmarks/MediatRVoidRequestRetainedScopeBenchmarks.cs @@ -0,0 +1,50 @@ +using BenchmarkDotNet.Attributes; +using MediatR; +using RequestFlow.Benchmarks.Core; +using RequestFlow.Benchmarks.RequestFlowFixtures; + +namespace RequestFlow.Benchmarks.MediatR; + +[BenchmarkScope(ScopeLifetime.RetainedScope)] +public class MediatRVoidRequestRetainedScopeBenchmarks +{ + [Params(CompletionKind.Synchronous, CompletionKind.Suspending)] + public CompletionKind Completion { get; set; } + + [Params(0, 1, 5)] + public int StageCount { get; set; } + + private RetainedScopeHost _requestFlow = null!; + private RetainedScopeHost _mediatR = null!; + + private TaskVoidRequest _requestFlowRequest = null!; + private MediatRVoidRequest _mediatRRequest = null!; + + [GlobalSetup] + public void Setup() + { + _requestFlow = new(RequestFlowSetup.Build(RequestFlowSetup.TaskVoidRequestServices(StageCount, MediatRSetup.DispatcherLifetime))); + _mediatR = new(MediatRSetup.Build(MediatRSetup.VoidRequestServices(StageCount))); + + _requestFlowRequest = new TaskVoidRequest(Completion); + _mediatRRequest = new MediatRVoidRequest(Completion); + + RequestFlow().GetAwaiter().GetResult(); + MediatR().GetAwaiter().GetResult(); + } + + [Benchmark(Baseline = true)] + public Task RequestFlow() + => _requestFlow.Service.SendAsync(_requestFlowRequest, CancellationToken.None); + + [Benchmark] + public Task MediatR() + => _mediatR.Service.Send(_mediatRRequest, CancellationToken.None); + + [GlobalCleanup] + public void Cleanup() + { + _requestFlow.Dispose(); + _mediatR.Dispose(); + } +} diff --git a/benchmarks/RequestFlow.Benchmarks.MediatR/Benchmarks/MediatRVoidRequestScopePerOperationBenchmarks.cs b/benchmarks/RequestFlow.Benchmarks.MediatR/Benchmarks/MediatRVoidRequestScopePerOperationBenchmarks.cs new file mode 100644 index 0000000..c16f201 --- /dev/null +++ b/benchmarks/RequestFlow.Benchmarks.MediatR/Benchmarks/MediatRVoidRequestScopePerOperationBenchmarks.cs @@ -0,0 +1,67 @@ +using BenchmarkDotNet.Attributes; +using MediatR; +using Microsoft.Extensions.DependencyInjection; +using RequestFlow.Benchmarks.Core; +using RequestFlow.Benchmarks.RequestFlowFixtures; + +namespace RequestFlow.Benchmarks.MediatR; + +[BenchmarkScope(ScopeLifetime.ScopePerOperation)] +public class MediatRVoidRequestScopePerOperationBenchmarks +{ + [Params(CompletionKind.Synchronous, CompletionKind.Suspending)] + public CompletionKind Completion { get; set; } + + [Params(0, 1, 5)] + public int StageCount { get; set; } + + private ScopePerOperationHost _requestFlow = null!; + private ScopePerOperationHost _mediatR = null!; + + private TaskVoidRequest _requestFlowRequest = null!; + private MediatRVoidRequest _mediatRRequest = null!; + + [GlobalSetup] + public void Setup() + { + _requestFlow = new(RequestFlowSetup.Build(RequestFlowSetup.TaskVoidRequestServices(StageCount, MediatRSetup.DispatcherLifetime))); + _mediatR = new(MediatRSetup.Build(MediatRSetup.VoidRequestServices(StageCount))); + + _requestFlowRequest = new TaskVoidRequest(Completion); + _mediatRRequest = new MediatRVoidRequest(Completion); + + RequestFlow().GetAwaiter().GetResult(); + MediatR().GetAwaiter().GetResult(); + } + + [Benchmark(Baseline = true)] + public async Task RequestFlow() + { + using IServiceScope scope = _requestFlow.Provider.CreateScope(); + + var dispatcher = scope.ServiceProvider + .GetRequiredService(); + + await dispatcher + .SendAsync(_requestFlowRequest, CancellationToken.None).ConfigureAwait(false); + } + + [Benchmark] + public async Task MediatR() + { + using IServiceScope scope = _mediatR.Provider.CreateScope(); + + var dispatcher = scope.ServiceProvider + .GetRequiredService(); + + await dispatcher + .Send(_mediatRRequest, CancellationToken.None).ConfigureAwait(false); + } + + [GlobalCleanup] + public void Cleanup() + { + _requestFlow.Dispose(); + _mediatR.Dispose(); + } +} diff --git a/benchmarks/RequestFlow.Benchmarks.MediatR/Program.cs b/benchmarks/RequestFlow.Benchmarks.MediatR/Program.cs new file mode 100644 index 0000000..e9ca9a0 --- /dev/null +++ b/benchmarks/RequestFlow.Benchmarks.MediatR/Program.cs @@ -0,0 +1,10 @@ +using RequestFlow.Benchmarks.Harness; + +namespace RequestFlow.Benchmarks.MediatR; + +public static class Program +{ + public static int Main(string[] args) + => SuiteRunner.Run(typeof(Program).Assembly, "mediatr", args, + SuiteProbe.Run, SuiteProbe.ReportLifetimes); +} diff --git a/benchmarks/RequestFlow.Benchmarks.MediatR/Properties/launchSettings.json b/benchmarks/RequestFlow.Benchmarks.MediatR/Properties/launchSettings.json new file mode 100644 index 0000000..7a4e553 --- /dev/null +++ b/benchmarks/RequestFlow.Benchmarks.MediatR/Properties/launchSettings.json @@ -0,0 +1,19 @@ +{ + "profiles": { + "Verify comparison": { + "commandName": "Project", + "workingDirectory": "../..", + "commandLineArgs": "probe" + }, + "Smoke test": { + "commandName": "Project", + "workingDirectory": "../..", + "commandLineArgs": "--smoke --filter *TaskRetainedScopeBenchmarks*" + }, + "Full comparison": { + "commandName": "Project", + "workingDirectory": "../..", + "commandLineArgs": "--filter *" + } + } +} diff --git a/benchmarks/RequestFlow.Benchmarks.MediatR/README.md b/benchmarks/RequestFlow.Benchmarks.MediatR/README.md new file mode 100644 index 0000000..0d87749 --- /dev/null +++ b/benchmarks/RequestFlow.Benchmarks.MediatR/README.md @@ -0,0 +1,96 @@ +# RequestFlow vs MediatR 12.5.0 + +- Compares RequestFlow with `MediatR 12.5.0`, the last Apache-2.0 release, on .NET 10. +- Results and behavior notes on this page describe MediatR 12.5.0 only, not later releases. +- RequestFlow is not affiliated with or endorsed by MediatR or its maintainers. +- Each class pairs a RequestFlow baseline with the matching native MediatR operation. +- No timing results are published yet. Probes check behavior, not performance. +- Build and run with the [quick start](../README.md#quick-start). This page covers what is specific to MediatR. + +## Matrix + +| Group | Completion | Work | Lifetime | +| --- | --- | --- | --- | +| Task response | Cached, newly completed, suspending | Returns 79; 0/1/5 stages | Retained scope and new scope per send | +| Task void request | Synchronous, suspending | 0/1/5 stages | Retained scope and new scope per send | +| Stream | Synchronous, suspending | Fully consumes 1/10/100 items; 0/1/5 forwarding stages | Retained scope and new scope per enumeration | +| Sequential events | Synchronous, suspending | Awaits 1/5 handlers | Retained scope and new scope per publication | +| Parallel events | Synchronous, suspending | Awaits 1/5 handlers | Retained scope and new scope per publication | +| Direct Task handler reference | Cached, newly completed, suspending | One concrete handler call per row with a reused request | No dispatch or DI resolution in the method | +| Catalog size | Cached, newly completed, suspending | One typed send within equal 4/16/64 handler catalogs | Retained scope | +| Mixed traffic | Cached, newly completed, suspending | 64 sends covering every request type of equal 4/16/64 handler catalogs; checksum 5,056 | Retained scope | + +- Synchronous stages forward their native awaitable. Suspending request stages await it. Stream stages forward the sequence without adding their own iterator. +- Catalog-size rows always send `RuntimeTag1`, so the lookup key and call target stay fixed while the catalog grows. +- Mixed traffic varies both. It sends every request type equally often, in one fixed shuffled order from `RuntimeCatalog.MixedTrafficOrder` that both libraries share. Each send goes through the library's `IRequest` overload, which dispatches on the runtime type. +- Direct-handler rows compare matching concrete handler calls. Their ratios describe workload cost and do not rank dispatchers. + +## Reading a result + +- `Mean` is the time of the whole operation the row describes. `Allocated` is all managed memory that operation allocates, including handler allocations such as a new Task or a stream iterator. +- `RetainedScope` setup builds providers, creates scopes, resolves dispatchers, constructs requests, and runs both methods once. Timed calls reuse those dispatchers, requests, and activated handlers. +- `ScopePerOperation` setup constructs reusable requests and warms both methods. Each timed call creates a scope, resolves its dispatcher, awaits completion, and disposes the scope. `Allocated` includes the activated handlers, stages, shared disposable test dependency, and DI bookkeeping. +- New-scope Task response and short-circuit methods are `async Task` wrappers. Their results fall outside the runtime's cached tasks, so each operation allocates one extra `Task` on both sides. +- Mixed traffic sets `OperationsPerInvoke = 64`, so rows report cost per send. Each invocation also allocates one `Task` for the uncached checksum, so a cached mixed row never shows 0 B. +- Timed code keeps the null checks for optional tracing. Only probes create trace records. + +## Native API differences + +- MediatR 12.5.0 registers `IMediator`, `ISender` and `IPublisher` as transient by default, and the suite keeps that default. RequestFlow's dispatchers and event publisher are scoped by default, so the suite calls `WithTransientDispatcher()` to match. +- The suite registers the selected handlers and pipeline behaviors as scoped, like RequestFlow's handlers and stages, and leaves other MediatR services at their native lifetimes. +- MediatR 12.5.0 has Task request and notification handlers but no ValueTask request family, so ValueTask cells are excluded. +- Void handlers return plain Task. MediatR 12.5.0's void pipeline contract uses `Unit`; RequestFlow's void stage contract uses plain Task. +- Sequential events use MediatR 12.5.0's default publisher. Parallel events use its native `TaskWhenAllPublisher`, in separate classes. +- The catalog fixture registers public closed generic request-handler contracts through normal DI. No MediatR implementation source is used. +- Event rows cover successful publication of an exact event type. Failure, cancellation, and inheritance semantics differ between the libraries and are not compared. +- Timed stream rows always consume the full sequence. Probes also check early disposal. + +## Short-circuit requests + +The pipeline is authorization, cache, transaction, then handler. Authorization and cache can return without calling the next component. The transaction has a unit of work and change buffer; the handler has a repository, data session, and separate command buffer. + +The constructor probe checks this on the first send in a fresh scope: + +| Outcome | Components executed by both libraries | RequestFlow constructs | MediatR 12.5.0 constructs | +| --- | --- | --- | --- | +| Authorization rejects | Authorization | Authorization and its dependencies | All pipeline behaviors and their dependencies; no handler | +| Cache returns a result | Authorization and cache | Authorization, cache and their dependencies | All pipeline behaviors and their dependencies; no handler | +| Request reaches the handler | All three stages and handler | All stages, handler and their dependencies | All pipeline behaviors, handler and their dependencies | + +- `MediatRShortCircuitRetainedScopeBenchmarks` has 6 cases: three outcomes and two libraries, with 0-byte buffers. Setup sends the selected outcome once, and timed rows reuse the instances it created. +- `MediatRShortCircuitScopePerOperationBenchmarks` has 12 cases: three outcomes, 0 or 4,096 bytes per dependency buffer, and two libraries. Rows include scope creation, dispatcher resolution, activation, send, and disposal. Provider construction stays outside timing. +- Read the two classes as separate scenarios. +- All components and dependencies are scoped. The 0-byte case keeps the dependency graph without allocating the buffers. +- With 4,096-byte buffers, RequestFlow skips constructing the transaction buffer on either early return. MediatR 12.5.0 constructs it even though the transaction never runs. Neither library constructs the handler's buffer on those paths. +- `probe` checks every case for constructor reachability, result, execution order, token propagation, scoped reuse, dependency sizes, and disposal. + +## Commands + +Run from the repository root after the [quick start](../README.md#quick-start) build. + +```powershell +dotnet run --project benchmarks/RequestFlow.Benchmarks.MediatR -c Release -f net10.0 --no-build --no-restore -- lifetimes +dotnet run --project benchmarks/RequestFlow.Benchmarks.MediatR -c Release -f net10.0 --no-build --no-restore -- --list flat +dotnet run --project benchmarks/RequestFlow.Benchmarks.MediatR -c Release -f net10.0 --no-build --no-restore -- --smoke --filter '*ShortCircuit*' +``` + +- `probe` runs both methods of every parameter set twice. It validates results and checks stage order, tokens, completion shape, component and dispatcher identities within and across scopes, scope disposal, stream checksums and early disposal, and awaited event completion. +- `lifetimes` prints the DI descriptors for workload components, dispatchers, `IMediator`, `INotificationPublisher`, `IEventPublisher`, every `IPipelineBehavior<,>` and `IStreamPipelineBehavior<,>`, and RequestFlow stages, short-circuit graphs included. It then runs the identity and disposal checks, because descriptors alone do not prove lifetime parity. +- `--list flat` lists all 30 benchmark methods without timing. +- To review one group on its own, select it with `--filter`, for example `'*Event*'` or `'*ShortCircuit*'`. + +## Where each row lives + +Benchmark classes are in `Benchmarks/` and `ShortCircuit/`, one file per class. Class names below omit the `Benchmarks` suffix. RequestFlow fixtures live in `RequestFlow.Benchmarks.RequestFlowFixtures`. + +| Benchmark classes | MediatR setup | MediatR workload | RequestFlow setup and workload | +| --- | --- | --- | --- | +| `MediatRTaskRetainedScope`, `MediatRTaskScopePerOperation` | `MediatRSetup.TaskServices` | `Workloads/TaskWorkloads.cs` | `RequestFlowSetup.TaskServices`, `Requests/TaskRequests.cs` | +| `MediatRTaskDirectHandler` | None. Constructs `MediatRTaskHandler` with `new`. | `Workloads/TaskWorkloads.cs` | No setup. Constructs `TaskRequestHandler` from `Requests/TaskRequests.cs` with `new`. | +| `MediatRVoidRequestRetainedScope`, `MediatRVoidRequestScopePerOperation` | `MediatRSetup.VoidRequestServices` | `Workloads/VoidRequestWorkloads.cs` | `RequestFlowSetup.TaskVoidRequestServices`, `Requests/TaskVoidRequests.cs` | +| `MediatRStreamRetainedScope`, `MediatRStreamScopePerOperation` | `MediatRSetup.StreamServices` | `Workloads/StreamWorkloads.cs` | `RequestFlowSetup.StreamServices`, `Streams/Streams.cs` | +| `MediatREventSequential*`, `MediatREventParallel*` | `MediatRSetup.EventServices` | `Workloads/EventWorkloads.cs` | `RequestFlowSetup.EventServices`, `Events/Events.cs` | +| `MediatRCatalogSize`, `MediatRMixedTraffic` | `MediatRSetup.CatalogServices` | `Workloads/CatalogWorkloads.cs` | `RequestFlowSetup.CatalogServices`, `Catalog/RuntimeTaskRequests.cs` | +| `MediatRShortCircuitRetainedScope`, `MediatRShortCircuitScopePerOperation` | `MediatRShortCircuitSetup.Services` | `ShortCircuit/ShortCircuitWorkloads.cs` | `ShortCircuitServices.Services`, `ShortCircuit/ShortCircuitTaskWorkloads.cs` | + +- Request types are in `Workloads/Messages.cs`, except the catalog request in `Workloads/CatalogWorkloads.cs` and the short-circuit request in `ShortCircuit/ShortCircuitWorkloads.cs`. diff --git a/benchmarks/RequestFlow.Benchmarks.MediatR/RequestFlow.Benchmarks.MediatR.csproj b/benchmarks/RequestFlow.Benchmarks.MediatR/RequestFlow.Benchmarks.MediatR.csproj new file mode 100644 index 0000000..07e0eac --- /dev/null +++ b/benchmarks/RequestFlow.Benchmarks.MediatR/RequestFlow.Benchmarks.MediatR.csproj @@ -0,0 +1,16 @@ + + + Exe + net10.0 + enable + + + + + + + + + + + diff --git a/benchmarks/RequestFlow.Benchmarks.MediatR/Setup/MediatRSetup.cs b/benchmarks/RequestFlow.Benchmarks.MediatR/Setup/MediatRSetup.cs new file mode 100644 index 0000000..44a937c --- /dev/null +++ b/benchmarks/RequestFlow.Benchmarks.MediatR/Setup/MediatRSetup.cs @@ -0,0 +1,158 @@ +using MediatR; +using MediatR.NotificationPublishers; +using Microsoft.Extensions.DependencyInjection; +using RequestFlow.Benchmarks.Core; + +namespace RequestFlow.Benchmarks.MediatR; + +internal static class MediatRSetup +{ + /// + /// MediatR 12.5.0's default lifetime for , and . + /// RequestFlow's dispatchers in this suite use it too. + /// + public const ServiceLifetime DispatcherLifetime = ServiceLifetime.Transient; + + private static readonly Type[] TaskStages = + [ + typeof(MediatRTaskStage1), + typeof(MediatRTaskStage2), + typeof(MediatRTaskStage3), + typeof(MediatRTaskStage4), + typeof(MediatRTaskStage5), + ]; + + private static readonly Type[] VoidRequestStages = + [ + typeof(MediatRVoidRequestStage1), + typeof(MediatRVoidRequestStage2), + typeof(MediatRVoidRequestStage3), + typeof(MediatRVoidRequestStage4), + typeof(MediatRVoidRequestStage5), + ]; + + private static readonly Type[] StreamStages = + [ + typeof(MediatRStreamStage1), + typeof(MediatRStreamStage2), + typeof(MediatRStreamStage3), + typeof(MediatRStreamStage4), + typeof(MediatRStreamStage5), + ]; + + private static readonly Type[] EventHandlers = + [ + typeof(MediatREventHandler1), + typeof(MediatREventHandler2), + typeof(MediatREventHandler3), + typeof(MediatREventHandler4), + typeof(MediatREventHandler5), + ]; + + public static ServiceCollection TaskServices(int stageCount) + => RequestServices(stageCount, TaskStages, typeof(MediatRTaskHandler), + typeof(IPipelineBehavior)); + + public static ServiceCollection VoidRequestServices(int stageCount) + => RequestServices(stageCount, VoidRequestStages, typeof(MediatRVoidRequestHandler), + typeof(IPipelineBehavior)); + + public static ServiceCollection StreamServices(int stageCount) + { + CheckStageCount(stageCount); + + var services = BaseServices(parallel: false, typeof(MediatRStreamHandler)); + + for (int i = 0; i < stageCount; i++) + services.AddScoped( + typeof(IStreamPipelineBehavior), + StreamStages[i]); + + return services; + } + + public static ServiceCollection EventServices(int handlerCount, bool parallel) + { + if (handlerCount is not (1 or 5)) + throw new ArgumentOutOfRangeException(nameof(handlerCount)); + + return BaseServices(parallel, EventHandlers.Take(handlerCount).ToArray()); + } + + public static ServiceCollection CatalogServices(int catalogSize) + { + Type[] tags = RuntimeCatalog.Tags(catalogSize); + ServiceCollection services = BaseServices(parallel: false); + + foreach (Type tag in tags) + { + Type request = typeof(MediatRCatalogRequest<>).MakeGenericType(tag); + Type handler = typeof(MediatRCatalogHandler<>).MakeGenericType(tag); + + services.AddScoped( + typeof(global::MediatR.IRequestHandler<,>).MakeGenericType(request, typeof(int)), + handler); + } + + return services; + } + + public static ServiceProvider Build(IServiceCollection services) + => services.BuildServiceProvider(new ServiceProviderOptions + { + ValidateOnBuild = true, + ValidateScopes = true, + }); + + private static ServiceCollection RequestServices(int stageCount, Type[] stages, Type handler, Type stageContract) + { + CheckStageCount(stageCount); + + var services = BaseServices(parallel: false, handler); + + for (int i = 0; i < stageCount; i++) + services.AddScoped(stageContract, stages[i]); + + return services; + } + + private static ServiceCollection BaseServices(bool parallel, params Type[] includedTypes) + { + var services = new ServiceCollection(); + + services.AddScoped(); + AddMediatRServices(services, parallel, includedTypes); + + return services; + } + + /// + /// Adds MediatR with its default mediator lifetime and re-registers the included handlers as scoped. + /// + public static void AddMediatRServices(IServiceCollection services, bool parallel, params Type[] includedTypes) + { + services.AddMediatR(configuration => + { + configuration.RegisterServicesFromAssemblyContaining(); + configuration.TypeEvaluator = type => includedTypes.Contains(type); + + if (parallel) + configuration.NotificationPublisher = new TaskWhenAllPublisher(); + }); + + ServiceDescriptor[] handlers = services.Where(descriptor => + descriptor.ImplementationType is Type implementation && includedTypes.Contains(implementation)).ToArray(); + + foreach (ServiceDescriptor descriptor in handlers) + { + services.Remove(descriptor); + services.AddScoped(descriptor.ServiceType, descriptor.ImplementationType!); + } + } + + private static void CheckStageCount(int count) + { + if (count is not (0 or 1 or 5)) + throw new ArgumentOutOfRangeException(nameof(count)); + } +} diff --git a/benchmarks/RequestFlow.Benchmarks.MediatR/ShortCircuit/MediatRShortCircuitRetainedScopeBenchmarks.cs b/benchmarks/RequestFlow.Benchmarks.MediatR/ShortCircuit/MediatRShortCircuitRetainedScopeBenchmarks.cs new file mode 100644 index 0000000..27952c7 --- /dev/null +++ b/benchmarks/RequestFlow.Benchmarks.MediatR/ShortCircuit/MediatRShortCircuitRetainedScopeBenchmarks.cs @@ -0,0 +1,47 @@ +using BenchmarkDotNet.Attributes; +using MediatR; +using RequestFlow.Benchmarks.Core; +using RequestFlow.Benchmarks.RequestFlowFixtures.ShortCircuit; + +namespace RequestFlow.Benchmarks.MediatR; + +[BenchmarkScope(ScopeLifetime.RetainedScope)] +public class MediatRShortCircuitRetainedScopeBenchmarks +{ + [Params(ShortCircuitOutcome.AuthorizationReject, ShortCircuitOutcome.CacheHit, ShortCircuitOutcome.FullExecution)] + public ShortCircuitOutcome Outcome { get; set; } + + private RetainedScopeHost _requestFlow = null!; + private RetainedScopeHost _mediatR = null!; + + private ShortCircuitTaskRequest _requestFlowRequest = null!; + private MediatRShortCircuitRequest _mediatRRequest = null!; + + [GlobalSetup] + public void Setup() + { + _requestFlow = new(ShortCircuitServices.Build(bufferBytes: 0, valueTask: false, dispatcherLifetime: MediatRSetup.DispatcherLifetime)); + _mediatR = new(MediatRShortCircuitSetup.Build(bufferBytes: 0)); + + _requestFlowRequest = new(Outcome); + _mediatRRequest = new(Outcome); + + RequestFlow().GetAwaiter().GetResult(); + MediatR().GetAwaiter().GetResult(); + } + + [Benchmark(Baseline = true)] + public Task RequestFlow() + => _requestFlow.Service.SendAsync(_requestFlowRequest, CancellationToken.None); + + [Benchmark] + public Task MediatR() + => _mediatR.Service.Send(_mediatRRequest, CancellationToken.None); + + [GlobalCleanup] + public void Cleanup() + { + _requestFlow.Dispose(); + _mediatR.Dispose(); + } +} diff --git a/benchmarks/RequestFlow.Benchmarks.MediatR/ShortCircuit/MediatRShortCircuitScopePerOperationBenchmarks.cs b/benchmarks/RequestFlow.Benchmarks.MediatR/ShortCircuit/MediatRShortCircuitScopePerOperationBenchmarks.cs new file mode 100644 index 0000000..9d7f05c --- /dev/null +++ b/benchmarks/RequestFlow.Benchmarks.MediatR/ShortCircuit/MediatRShortCircuitScopePerOperationBenchmarks.cs @@ -0,0 +1,60 @@ +using BenchmarkDotNet.Attributes; +using MediatR; +using Microsoft.Extensions.DependencyInjection; +using RequestFlow.Benchmarks.Core; +using RequestFlow.Benchmarks.RequestFlowFixtures.ShortCircuit; + +namespace RequestFlow.Benchmarks.MediatR; + +[BenchmarkScope(ScopeLifetime.ScopePerOperation)] +public class MediatRShortCircuitScopePerOperationBenchmarks +{ + [Params(ShortCircuitOutcome.AuthorizationReject, ShortCircuitOutcome.CacheHit, ShortCircuitOutcome.FullExecution)] + public ShortCircuitOutcome Outcome { get; set; } + + [Params(0, 4096)] + public int BufferBytes { get; set; } + + private ScopePerOperationHost _requestFlow = null!; + private ScopePerOperationHost _mediatR = null!; + private ShortCircuitTaskRequest _requestFlowRequest = null!; + private MediatRShortCircuitRequest _mediatRRequest = null!; + + [GlobalSetup] + public void Setup() + { + _requestFlow = new(ShortCircuitServices.Build(BufferBytes, valueTask: false, dispatcherLifetime: MediatRSetup.DispatcherLifetime)); + _mediatR = new(MediatRShortCircuitSetup.Build(BufferBytes)); + + _requestFlowRequest = new(Outcome); + _mediatRRequest = new(Outcome); + + RequestFlow().GetAwaiter().GetResult(); + MediatR().GetAwaiter().GetResult(); + } + + [Benchmark(Baseline = true)] + public async Task RequestFlow() + { + using IServiceScope scope = _requestFlow.Provider.CreateScope(); + var dispatcher = scope.ServiceProvider.GetRequiredService(); + + return await dispatcher.SendAsync(_requestFlowRequest, CancellationToken.None).ConfigureAwait(false); + } + + [Benchmark] + public async Task MediatR() + { + using IServiceScope scope = _mediatR.Provider.CreateScope(); + var dispatcher = scope.ServiceProvider.GetRequiredService(); + + return await dispatcher.Send(_mediatRRequest, CancellationToken.None).ConfigureAwait(false); + } + + [GlobalCleanup] + public void Cleanup() + { + _requestFlow.Dispose(); + _mediatR.Dispose(); + } +} diff --git a/benchmarks/RequestFlow.Benchmarks.MediatR/ShortCircuit/MediatRShortCircuitSetup.cs b/benchmarks/RequestFlow.Benchmarks.MediatR/ShortCircuit/MediatRShortCircuitSetup.cs new file mode 100644 index 0000000..f73c109 --- /dev/null +++ b/benchmarks/RequestFlow.Benchmarks.MediatR/ShortCircuit/MediatRShortCircuitSetup.cs @@ -0,0 +1,39 @@ +using MediatR; +using Microsoft.Extensions.DependencyInjection; +using RequestFlow.Benchmarks.Core; +using RequestFlow.Benchmarks.RequestFlowFixtures.ShortCircuit; + +namespace RequestFlow.Benchmarks.MediatR; + +public static class MediatRShortCircuitSetup +{ + public static ServiceProvider Build(int bufferBytes, ConstructionTrace? trace = null) + => MediatRSetup.Build(Services(bufferBytes, trace)); + + public static ServiceCollection Services(int bufferBytes, ConstructionTrace? trace = null) + { + var services = new ServiceCollection(); + + ShortCircuitWorkload.AddDependencies(services, bufferBytes, trace); + MediatRSetup.AddMediatRServices(services, parallel: false, typeof(MediatRShortCircuitHandler)); + + services.AddScoped, MediatRShortCircuitAuthorization>(); + services.AddScoped, MediatRShortCircuitCache>(); + services.AddScoped, MediatRShortCircuitTransaction>(); + + return services; + } + + public static void Verify() + { + ShortCircuitServices.VerifyRequestFlow(valueTask: false, dispatcherLifetime: MediatRSetup.DispatcherLifetime); + + ShortCircuitVerification.Verify( + "MediatR", + (bytes, trace) => Build(bytes, trace), + outcome => new(outcome), + (dispatcher, message, token) => new(dispatcher.Send(message, token)), + DependencyResolution.AllStages, + dispatcherLifetime: MediatRSetup.DispatcherLifetime); + } +} diff --git a/benchmarks/RequestFlow.Benchmarks.MediatR/ShortCircuit/ShortCircuitWorkloads.cs b/benchmarks/RequestFlow.Benchmarks.MediatR/ShortCircuit/ShortCircuitWorkloads.cs new file mode 100644 index 0000000..d29c1e9 --- /dev/null +++ b/benchmarks/RequestFlow.Benchmarks.MediatR/ShortCircuit/ShortCircuitWorkloads.cs @@ -0,0 +1,62 @@ +using MediatR; +using RequestFlow.Benchmarks.Core; + +namespace RequestFlow.Benchmarks.MediatR; + +public sealed class MediatRShortCircuitRequest(ShortCircuitOutcome outcome) + : ShortCircuitMessage(outcome), global::MediatR.IRequest; + +public sealed class MediatRShortCircuitAuthorization(DependencyState state, AccessPolicy dependency) + : DependencyComponent(state, "authorization"), IPipelineBehavior +{ + public Task Handle( + MediatRShortCircuitRequest request, + RequestHandlerDelegate next, + CancellationToken token) + { + Enter(request, token); + + return dependency.Allows(request) ? next(token) : ShortCircuitWorkload.Rejected; + } +} + +public sealed class MediatRShortCircuitCache(DependencyState state, CachePolicy dependency) + : DependencyComponent(state, "cache"), IPipelineBehavior +{ + public Task Handle( + MediatRShortCircuitRequest request, + RequestHandlerDelegate next, + CancellationToken token) + { + Enter(request, token); + + return dependency.Contains(request) ? ShortCircuitWorkload.Cached : next(token); + } +} + +public sealed class MediatRShortCircuitTransaction(DependencyState state, UnitOfWork dependency) + : DependencyComponent(state, "transaction"), IPipelineBehavior +{ + public Task Handle( + MediatRShortCircuitRequest request, + RequestHandlerDelegate next, + CancellationToken token) + { + Enter(request, token); + + return dependency.Capacity >= 0 ? next(token) : throw new InvalidOperationException(); + } +} + +public sealed class MediatRShortCircuitHandler(DependencyState state, Repository dependency) + : DependencyComponent(state, TraceNames.Handler), global::MediatR.IRequestHandler +{ + public Task Handle(MediatRShortCircuitRequest request, CancellationToken token) + { + Enter(request, token); + + Task result = dependency.Read(); + + return request.Trace is { } trace ? trace.Track(result) : result; + } +} diff --git a/benchmarks/RequestFlow.Benchmarks.MediatR/Verification/SuiteProbe.cs b/benchmarks/RequestFlow.Benchmarks.MediatR/Verification/SuiteProbe.cs new file mode 100644 index 0000000..ca751ee --- /dev/null +++ b/benchmarks/RequestFlow.Benchmarks.MediatR/Verification/SuiteProbe.cs @@ -0,0 +1,226 @@ +using MediatR; +using Microsoft.Extensions.DependencyInjection; +using RequestFlow.Benchmarks.Core; +using RequestFlow.Benchmarks.Harness; +using RequestFlow.Benchmarks.RequestFlowFixtures; +using RequestFlow.Benchmarks.RequestFlowFixtures.Catalog; +using RequestFlow.Benchmarks.RequestFlowFixtures.ShortCircuit; +using static RequestFlow.Benchmarks.Core.BenchmarkVerification; +using static RequestFlow.Benchmarks.Core.RegistrationReport; +using static RequestFlow.Benchmarks.Core.ScopeLifetimeVerification; +using static RequestFlow.Benchmarks.Core.TraceAssertions; + +namespace RequestFlow.Benchmarks.MediatR; + +public static class SuiteProbe +{ + private static readonly Type[] MediatRServices = + [ + typeof(IMediator), + typeof(INotificationPublisher), + typeof(IPipelineBehavior<,>), + typeof(IStreamPipelineBehavior<,>), + ]; + + private static readonly ComparedLibrary MediatRLibrary = new("MediatR", MediatRSetup.Build, MediatRServices); + + public static BenchmarkProbeResult Run() + { + VerifyLifetimes(report: false); + MediatRShortCircuitSetup.Verify(); + + return VerifyBenchmarkMethods(); + } + + public static void ReportLifetimes() + { + VerifyLifetimes(report: true); + + ReportRegistrations( + "RequestFlow short circuit", + ShortCircuitServices.Services(bufferBytes: 0, dispatcherLifetime: MediatRSetup.DispatcherLifetime), + typeof(IRequestDispatcher), RequestFlowSetup.ReportedServices, RequestFlowSetup.StageContracts); + + ReportRegistrations( + "MediatR short circuit", MediatRShortCircuitSetup.Services(bufferBytes: 0), + typeof(ISender), MediatRServices); + } + + private static void VerifyLifetimes(bool report) + { + VerifyRequestLifetimes(report); + VerifyStreamLifetimes(report); + VerifyEventLifetimes(report); + VerifyCatalogs(report); + } + + private static void VerifyRequestLifetimes(bool report) + { + foreach (CompletionKind completion in Params(typeof(MediatRTaskRetainedScopeBenchmarks), nameof(MediatRTaskRetainedScopeBenchmarks.Completion))) + foreach (int stages in Params(typeof(MediatRTaskRetainedScopeBenchmarks), nameof(MediatRTaskRetainedScopeBenchmarks.StageCount))) + { + VerifyPair( + $"Task, completion={completion}, stages={stages}", RequestNames(stages), report, + RequestFlowSetup.Library, RequestFlowSetup.TaskServices(stages, MediatRSetup.DispatcherLifetime), + (dispatcher, trace, token) => ExpectResult(dispatcher.SendAsync(new TaskRequest(completion) { Trace = trace }, token)), + MediatRLibrary, MediatRSetup.TaskServices(stages), + (sender, trace, token) => ExpectResult(sender.Send(new MediatRTaskRequest(completion) { Trace = trace }, token)), + dispatcherLifetime: MediatRSetup.DispatcherLifetime); + } + + foreach (CompletionKind completion in Params(typeof(MediatRVoidRequestRetainedScopeBenchmarks), nameof(MediatRVoidRequestRetainedScopeBenchmarks.Completion))) + foreach (int stages in Params(typeof(MediatRVoidRequestRetainedScopeBenchmarks), nameof(MediatRVoidRequestRetainedScopeBenchmarks.StageCount))) + { + VerifyPair( + $"void request, completion={completion}, stages={stages}", RequestNames(stages), report, + RequestFlowSetup.Library, RequestFlowSetup.TaskVoidRequestServices(stages, MediatRSetup.DispatcherLifetime), + (dispatcher, trace, token) => dispatcher.SendAsync(new TaskVoidRequest(completion) { Trace = trace }, token), + MediatRLibrary, MediatRSetup.VoidRequestServices(stages), + (sender, trace, token) => sender.Send(new MediatRVoidRequest(completion) { Trace = trace }, token), + dispatcherLifetime: MediatRSetup.DispatcherLifetime); + } + } + + private static void VerifyStreamLifetimes(bool report) + { + foreach (int stages in Params(typeof(MediatRStreamRetainedScopeBenchmarks), nameof(MediatRStreamRetainedScopeBenchmarks.StageCount))) + foreach (int count in Params(typeof(MediatRStreamRetainedScopeBenchmarks), nameof(MediatRStreamRetainedScopeBenchmarks.ItemCount))) + foreach (CompletionKind completion in Params(typeof(MediatRStreamRetainedScopeBenchmarks), nameof(MediatRStreamRetainedScopeBenchmarks.Completion))) + foreach (bool early in new[] { false, true }) + { + VerifyPair( + $"Stream, items={count}, completion={completion}, stages={stages}, early={early}", + StreamNames(stages, count, early: early), report, + RequestFlowSetup.Library, RequestFlowSetup.StreamServices(stages, MediatRSetup.DispatcherLifetime), + (dispatcher, trace, token) => ReadStream( + dispatcher.Stream(new StreamRequest(count, completion) { Trace = trace }, token), count, early, token), + MediatRLibrary, MediatRSetup.StreamServices(stages), + (sender, trace, token) => ReadStream( + sender.CreateStream(new MediatRStreamRequest(count, completion) { Trace = trace }, token), count, early, token), + dispatcherLifetime: MediatRSetup.DispatcherLifetime); + } + } + + private static void VerifyEventLifetimes(bool report) + { + foreach ((Type benchmark, bool parallel) in new[] + { (typeof(MediatREventSequentialRetainedScopeBenchmarks), false), (typeof(MediatREventParallelRetainedScopeBenchmarks), true) }) + foreach (CompletionKind completion in Params(benchmark, nameof(MediatREventSequentialRetainedScopeBenchmarks.Completion))) + foreach (int handlers in Params(benchmark, nameof(MediatREventSequentialRetainedScopeBenchmarks.HandlerCount))) + { + VerifyPair( + $"Event, completion={completion}, handlers={handlers}, parallel={parallel}", EventNames(handlers), report, + RequestFlowSetup.Library, RequestFlowSetup.EventServices(handlers, parallel, MediatRSetup.DispatcherLifetime), + (publisher, trace, token) => publisher.PublishAsync(new EventMessage(completion) { Trace = trace }, token), + MediatRLibrary, MediatRSetup.EventServices(handlers, parallel: parallel), + (publisher, trace, token) => publisher.Publish(new MediatREvent(completion) { Trace = trace }, token), + ordered: !parallel, + dispatcherLifetime: MediatRSetup.DispatcherLifetime); + } + } + + private static void VerifyCatalogs(bool report) + { + foreach (int count in Params(typeof(MediatRCatalogSizeBenchmarks), nameof(MediatRCatalogSizeBenchmarks.CatalogSize))) + { + ServiceCollection requestFlowServices = RequestFlowSetup.CatalogServices(count, MediatRSetup.DispatcherLifetime); + ServiceCollection mediatRServices = MediatRSetup.CatalogServices(count); + + Require( + GetComponentDescriptors(requestFlowServices).Count() == count, + $"RequestFlow catalog={count} registered the wrong number of handlers."); + + Require( + GetComponentDescriptors(mediatRServices).Count() == count, + $"MediatR catalog={count} registered the wrong number of handlers."); + + foreach (Type tag in RuntimeCatalog.Tags(count)) + { + Type requestFlowRequest = typeof(RuntimeTaskRequest<>).MakeGenericType(tag); + Type mediatRRequest = typeof(MediatRCatalogRequest<>).MakeGenericType(tag); + + VerifyPair( + $"catalog request {tag.Name}, catalog={count}", [TraceNames.Handler], report, + RequestFlowSetup.Library, requestFlowServices, + (dispatcher, trace, token) => ExpectResult( + dispatcher.SendAsync(CatalogRequest>(requestFlowRequest, trace), token)), + MediatRLibrary, mediatRServices, + (sender, trace, token) => ExpectResult( + sender.Send(CatalogRequest>(mediatRRequest, trace), token)), + dispatcherLifetime: MediatRSetup.DispatcherLifetime); + } + } + } + + private static BenchmarkProbeResult VerifyBenchmarkMethods() + { + var expectations = new Dictionary, BenchmarkExpectation>> + { + [typeof(MediatRShortCircuitRetainedScopeBenchmarks)] = ShortCircuitWorkload.Expectation, + [typeof(MediatRShortCircuitScopePerOperationBenchmarks)] = ShortCircuitWorkload.Expectation, + [typeof(MediatRTaskRetainedScopeBenchmarks)] = RequestWithStages, + [typeof(MediatRTaskScopePerOperationBenchmarks)] = RequestWithStages, + [typeof(MediatRVoidRequestRetainedScopeBenchmarks)] = VoidRequestWithStages, + [typeof(MediatRVoidRequestScopePerOperationBenchmarks)] = VoidRequestWithStages, + [typeof(MediatRStreamRetainedScopeBenchmarks)] = StreamWithStages, + [typeof(MediatRStreamScopePerOperationBenchmarks)] = StreamWithStages, + [typeof(MediatREventSequentialRetainedScopeBenchmarks)] = SequentialEvent, + [typeof(MediatREventSequentialScopePerOperationBenchmarks)] = SequentialEvent, + [typeof(MediatREventParallelRetainedScopeBenchmarks)] = ParallelEvent, + [typeof(MediatREventParallelScopePerOperationBenchmarks)] = ParallelEvent, + [typeof(MediatRTaskDirectHandlerBenchmarks)] = _ => BenchmarkExpectation.Request(), + [typeof(MediatRCatalogSizeBenchmarks)] = _ => BenchmarkExpectation.Request(), + [typeof(MediatRMixedTrafficBenchmarks)] = MixedTrafficExpectation, + }; + + return BenchmarkVerification.Run(typeof(Program).Assembly, expectations); + + static BenchmarkExpectation RequestWithStages(IReadOnlyDictionary parameters) + => BenchmarkExpectation.Request(stages: (int)parameters[nameof(MediatRTaskRetainedScopeBenchmarks.StageCount)]!); + + static BenchmarkExpectation VoidRequestWithStages(IReadOnlyDictionary parameters) + => BenchmarkExpectation.VoidRequest(stages: (int)parameters[nameof(MediatRTaskRetainedScopeBenchmarks.StageCount)]!); + + static BenchmarkExpectation StreamWithStages(IReadOnlyDictionary parameters) + => BenchmarkExpectation.Stream( + stages: (int)parameters[nameof(MediatRTaskRetainedScopeBenchmarks.StageCount)]!, + count: (int)parameters[nameof(MediatRStreamRetainedScopeBenchmarks.ItemCount)]!); + + static BenchmarkExpectation SequentialEvent(IReadOnlyDictionary parameters) + => BenchmarkExpectation.Event(handlers: (int)parameters[nameof(MediatREventSequentialRetainedScopeBenchmarks.HandlerCount)]!); + + static BenchmarkExpectation ParallelEvent(IReadOnlyDictionary parameters) + { + int handlers = (int)parameters[nameof(MediatREventSequentialRetainedScopeBenchmarks.HandlerCount)]!; + bool verifyFanOut = (CompletionKind)parameters[nameof(MediatREventParallelRetainedScopeBenchmarks.Completion)]! == CompletionKind.Suspending; + + return BenchmarkExpectation.Event(handlers, ordered: false, + parallelHandlers: verifyFanOut ? handlers : 0); + } + } + + private static BenchmarkExpectation MixedTrafficExpectation(IReadOnlyDictionary parameters) + { + IReadOnlyList order = RuntimeCatalog.MixedTrafficOrder((int)parameters[nameof(MediatRMixedTrafficBenchmarks.CatalogSize)]!); + + return new( + Names: [.. Enumerable.Repeat(TraceNames.Handler, RuntimeCatalog.MixedTrafficBatch)], + Result: RuntimeCatalog.MixedTrafficBatch * BenchmarkWorkload.Expected, + HandlerTypes: new Dictionary + { + [nameof(MediatRMixedTrafficBenchmarks.RequestFlow)] = + [.. order.Select(tag => typeof(RuntimeTaskHandler<>).MakeGenericType(tag))], + [nameof(MediatRMixedTrafficBenchmarks.MediatR)] = + [.. order.Select(tag => typeof(MediatRCatalogHandler<>).MakeGenericType(tag))], + }); + } + + private static TRequest CatalogRequest(Type requestType, ExecutionTrace trace) + { + var request = (WorkloadMessage)Activator.CreateInstance(requestType, CompletionKind.Cached)!; + + request.Trace = trace; + + return (TRequest)(object)request; + } +} diff --git a/benchmarks/RequestFlow.Benchmarks.MediatR/Workloads/CatalogWorkloads.cs b/benchmarks/RequestFlow.Benchmarks.MediatR/Workloads/CatalogWorkloads.cs new file mode 100644 index 0000000..5dec642 --- /dev/null +++ b/benchmarks/RequestFlow.Benchmarks.MediatR/Workloads/CatalogWorkloads.cs @@ -0,0 +1,17 @@ +using RequestFlow.Benchmarks.Core; + +namespace RequestFlow.Benchmarks.MediatR; + +public sealed class MediatRCatalogRequest(CompletionKind completion = CompletionKind.Cached) + : WorkloadMessage(completion), global::MediatR.IRequest; + +public sealed class MediatRCatalogHandler(ScopeWitness scope) + : WorkloadComponent(scope), global::MediatR.IRequestHandler, int> +{ + public Task Handle(MediatRCatalogRequest request, CancellationToken cancellationToken) + { + Observe(request, TraceNames.Handler, cancellationToken); + + return BenchmarkWorkload.TaskResponse(request, cancellationToken); + } +} diff --git a/benchmarks/RequestFlow.Benchmarks.MediatR/Workloads/EventWorkloads.cs b/benchmarks/RequestFlow.Benchmarks.MediatR/Workloads/EventWorkloads.cs new file mode 100644 index 0000000..31af5d3 --- /dev/null +++ b/benchmarks/RequestFlow.Benchmarks.MediatR/Workloads/EventWorkloads.cs @@ -0,0 +1,22 @@ +using MediatR; +using RequestFlow.Benchmarks.Core; + +namespace RequestFlow.Benchmarks.MediatR; + +public sealed class MediatREventHandler1(ScopeWitness scope) : MediatREventHandler(scope); +public sealed class MediatREventHandler2(ScopeWitness scope) : MediatREventHandler(scope); +public sealed class MediatREventHandler3(ScopeWitness scope) : MediatREventHandler(scope); +public sealed class MediatREventHandler4(ScopeWitness scope) : MediatREventHandler(scope); +public sealed class MediatREventHandler5(ScopeWitness scope) : MediatREventHandler(scope); + +public abstract class MediatREventHandler(ScopeWitness scope) + : WorkloadComponent(scope), INotificationHandler + where TSlot : IStageSlot +{ + public Task Handle(MediatREvent notification, CancellationToken cancellationToken) + { + Observe(notification, StageSlot.HandlerName, cancellationToken); + + return RuntimeTracing.EventTask(notification, this, StageSlot.HandlerName, cancellationToken); + } +} diff --git a/benchmarks/RequestFlow.Benchmarks.MediatR/Workloads/Messages.cs b/benchmarks/RequestFlow.Benchmarks.MediatR/Workloads/Messages.cs new file mode 100644 index 0000000..c568d87 --- /dev/null +++ b/benchmarks/RequestFlow.Benchmarks.MediatR/Workloads/Messages.cs @@ -0,0 +1,18 @@ +using RequestFlow.Benchmarks.Core; + +namespace RequestFlow.Benchmarks.MediatR; + +public sealed class MediatRTaskRequest(CompletionKind completion = CompletionKind.Cached, int value = 42) + : WorkloadMessage(completion, value), global::MediatR.IRequest; + +public sealed class MediatRVoidRequest(CompletionKind completion = CompletionKind.Cached) + : WorkloadMessage(completion), global::MediatR.IRequest; + +public sealed class MediatRStreamRequest(int count = 10, CompletionKind completion = CompletionKind.Synchronous) + : WorkloadMessage(completion), global::MediatR.IStreamRequest +{ + public int Count { get; } = count; +} + +public sealed class MediatREvent(CompletionKind completion = CompletionKind.Cached) + : WorkloadMessage(completion), global::MediatR.INotification; diff --git a/benchmarks/RequestFlow.Benchmarks.MediatR/Workloads/StreamWorkloads.cs b/benchmarks/RequestFlow.Benchmarks.MediatR/Workloads/StreamWorkloads.cs new file mode 100644 index 0000000..ff6fb13 --- /dev/null +++ b/benchmarks/RequestFlow.Benchmarks.MediatR/Workloads/StreamWorkloads.cs @@ -0,0 +1,36 @@ +using MediatR; +using RequestFlow.Benchmarks.Core; + +namespace RequestFlow.Benchmarks.MediatR; + +public sealed class MediatRStreamHandler(ScopeWitness scope) + : WorkloadComponent(scope), global::MediatR.IStreamRequestHandler +{ + public IAsyncEnumerable Handle(MediatRStreamRequest request, CancellationToken cancellationToken) + { + Observe(request, TraceNames.Handler, cancellationToken); + + return RuntimeTracing.Stream(request, request.Count, this, cancellationToken); + } +} + +public sealed class MediatRStreamStage1(ScopeWitness scope) : MediatRStreamStage(scope); +public sealed class MediatRStreamStage2(ScopeWitness scope) : MediatRStreamStage(scope); +public sealed class MediatRStreamStage3(ScopeWitness scope) : MediatRStreamStage(scope); +public sealed class MediatRStreamStage4(ScopeWitness scope) : MediatRStreamStage(scope); +public sealed class MediatRStreamStage5(ScopeWitness scope) : MediatRStreamStage(scope); + +public abstract class MediatRStreamStage(ScopeWitness scope) + : WorkloadComponent(scope), IStreamPipelineBehavior + where TSlot : IStageSlot +{ + public IAsyncEnumerable Handle( + MediatRStreamRequest request, + StreamHandlerDelegate next, + CancellationToken cancellationToken) + { + Observe(request, StageSlot.StageName, cancellationToken); + + return next(); + } +} diff --git a/benchmarks/RequestFlow.Benchmarks.MediatR/Workloads/TaskWorkloads.cs b/benchmarks/RequestFlow.Benchmarks.MediatR/Workloads/TaskWorkloads.cs new file mode 100644 index 0000000..c7d29ec --- /dev/null +++ b/benchmarks/RequestFlow.Benchmarks.MediatR/Workloads/TaskWorkloads.cs @@ -0,0 +1,38 @@ +using MediatR; +using RequestFlow.Benchmarks.Core; + +namespace RequestFlow.Benchmarks.MediatR; + +public sealed class MediatRTaskHandler(ScopeWitness scope) + : WorkloadComponent(scope), global::MediatR.IRequestHandler +{ + public Task Handle(MediatRTaskRequest request, CancellationToken cancellationToken) + { + Observe(request, TraceNames.Handler, cancellationToken); + + return BenchmarkWorkload.TaskResponse(request, cancellationToken); + } +} + +public sealed class MediatRTaskStage1(ScopeWitness scope) : MediatRTaskStage(scope); +public sealed class MediatRTaskStage2(ScopeWitness scope) : MediatRTaskStage(scope); +public sealed class MediatRTaskStage3(ScopeWitness scope) : MediatRTaskStage(scope); +public sealed class MediatRTaskStage4(ScopeWitness scope) : MediatRTaskStage(scope); +public sealed class MediatRTaskStage5(ScopeWitness scope) : MediatRTaskStage(scope); + +public abstract class MediatRTaskStage(ScopeWitness scope) + : WorkloadComponent(scope), IPipelineBehavior + where TSlot : IStageSlot +{ + public Task Handle( + MediatRTaskRequest request, + RequestHandlerDelegate next, + CancellationToken cancellationToken) + { + Observe(request, StageSlot.StageName, cancellationToken); + + Task result = next(cancellationToken); + + return request.Completion == CompletionKind.Suspending ? BenchmarkWorkload.AwaitTask(result) : result; + } +} diff --git a/benchmarks/RequestFlow.Benchmarks.MediatR/Workloads/VoidRequestWorkloads.cs b/benchmarks/RequestFlow.Benchmarks.MediatR/Workloads/VoidRequestWorkloads.cs new file mode 100644 index 0000000..67028f4 --- /dev/null +++ b/benchmarks/RequestFlow.Benchmarks.MediatR/Workloads/VoidRequestWorkloads.cs @@ -0,0 +1,38 @@ +using MediatR; +using RequestFlow.Benchmarks.Core; + +namespace RequestFlow.Benchmarks.MediatR; + +public sealed class MediatRVoidRequestHandler(ScopeWitness scope) + : WorkloadComponent(scope), global::MediatR.IRequestHandler +{ + public Task Handle(MediatRVoidRequest request, CancellationToken cancellationToken) + { + Observe(request, TraceNames.Handler, cancellationToken); + + return BenchmarkWorkload.TaskVoidRequest(request, cancellationToken); + } +} + +public sealed class MediatRVoidRequestStage1(ScopeWitness scope) : MediatRVoidRequestStage(scope); +public sealed class MediatRVoidRequestStage2(ScopeWitness scope) : MediatRVoidRequestStage(scope); +public sealed class MediatRVoidRequestStage3(ScopeWitness scope) : MediatRVoidRequestStage(scope); +public sealed class MediatRVoidRequestStage4(ScopeWitness scope) : MediatRVoidRequestStage(scope); +public sealed class MediatRVoidRequestStage5(ScopeWitness scope) : MediatRVoidRequestStage(scope); + +public abstract class MediatRVoidRequestStage(ScopeWitness scope) + : WorkloadComponent(scope), IPipelineBehavior + where TSlot : IStageSlot +{ + public Task Handle( + MediatRVoidRequest request, + RequestHandlerDelegate next, + CancellationToken cancellationToken) + { + Observe(request, StageSlot.StageName, cancellationToken); + + Task result = next(cancellationToken); + + return request.Completion == CompletionKind.Suspending ? BenchmarkWorkload.AwaitTask(result) : result; + } +} diff --git a/benchmarks/RequestFlow.Benchmarks.MediatR/packages.lock.json b/benchmarks/RequestFlow.Benchmarks.MediatR/packages.lock.json new file mode 100644 index 0000000..98b9981 --- /dev/null +++ b/benchmarks/RequestFlow.Benchmarks.MediatR/packages.lock.json @@ -0,0 +1,218 @@ +{ + "version": 1, + "dependencies": { + "net10.0": { + "MediatR": { + "type": "Direct", + "requested": "[12.5.0, )", + "resolved": "12.5.0", + "contentHash": "vqm2H8/nqL5NAJHPhsG1JOPwfkmbVrPyh4svdoRzu+uZh6Ex7PRoHBGsLYC0/RWCEJFqD1ohHNpteQvql9OktA==", + "dependencies": { + "MediatR.Contracts": "[2.0.1, 3.0.0)", + "Microsoft.Extensions.DependencyInjection.Abstractions": "8.0.0" + } + }, + "Microsoft.Extensions.DependencyInjection": { + "type": "Direct", + "requested": "[10.0.9, )", + "resolved": "10.0.9", + "contentHash": "NijozhERJDIaJ4k5TSMy1jOi0cSC2HfkvRD/Sl+kGSSKgVbFnF4GxgtMN/MrzHB8D1JxIrD4xSer9Blh9v3axQ==", + "dependencies": { + "Microsoft.Extensions.DependencyInjection.Abstractions": "10.0.9" + } + }, + "BenchmarkDotNet": { + "type": "Transitive", + "resolved": "0.15.8", + "contentHash": "paCfrWxSeHqn3rUZc0spYXVFnHCF0nzRhG0nOLnyTjZYs8spsimBaaNmb3vwqvALKIplbYq/TF393vYiYSnh/Q==", + "dependencies": { + "BenchmarkDotNet.Annotations": "0.15.8", + "CommandLineParser": "2.9.1", + "Gee.External.Capstone": "2.3.0", + "Iced": "1.21.0", + "Microsoft.CodeAnalysis.CSharp": "4.14.0", + "Microsoft.Diagnostics.Runtime": "3.1.512801", + "Microsoft.Diagnostics.Tracing.TraceEvent": "3.1.21", + "Microsoft.DotNet.PlatformAbstractions": "3.1.6", + "Perfolizer": "[0.6.1]", + "System.Management": "9.0.5" + } + }, + "BenchmarkDotNet.Annotations": { + "type": "Transitive", + "resolved": "0.15.8", + "contentHash": "hfucY0ycAsB0SsoaZcaAp9oq5wlWBJcylvEJb9pmvdYUx6PD6S4mDiYnZWjdjAlLhIpe/xtGCwzORfzAzPqvzA==" + }, + "CommandLineParser": { + "type": "Transitive", + "resolved": "2.9.1", + "contentHash": "OE0sl1/sQ37bjVsPKKtwQlWDgqaxWgtme3xZz7JssWUzg5JpMIyHgCTY9MVMxOg48fJ1AgGT3tgdH5m/kQ5xhA==" + }, + "Gee.External.Capstone": { + "type": "Transitive", + "resolved": "2.3.0", + "contentHash": "2ap/rYmjtzCOT8hxrnEW/QeiOt+paD8iRrIcdKX0cxVwWLFa1e+JDBNeECakmccXrSFeBQuu5AV8SNkipFMMMw==" + }, + "Iced": { + "type": "Transitive", + "resolved": "1.21.0", + "contentHash": "dv5+81Q1TBQvVMSOOOmRcjJmvWcX3BZPZsIq31+RLc5cNft0IHAyNlkdb7ZarOWG913PyBoFDsDXoCIlKmLclg==" + }, + "MediatR.Contracts": { + "type": "Transitive", + "resolved": "2.0.1", + "contentHash": "FYv95bNT4UwcNA+G/J1oX5OpRiSUxteXaUt2BJbRSdRNiIUNbggJF69wy6mnk2wYToaanpdXZdCwVylt96MpwQ==" + }, + "Microsoft.CodeAnalysis.Analyzers": { + "type": "Transitive", + "resolved": "3.11.0", + "contentHash": "v/EW3UE8/lbEYHoC2Qq7AR/DnmvpgdtAMndfQNmpuIMx/Mto8L5JnuCfdBYtgvalQOtfNCnxFejxuRrryvUTsg==" + }, + "Microsoft.CodeAnalysis.Common": { + "type": "Transitive", + "resolved": "4.14.0", + "contentHash": "PC3tuwZYnC+idaPuoC/AZpEdwrtX7qFpmnrfQkgobGIWiYmGi5MCRtl5mx6QrfMGQpK78X2lfIEoZDLg/qnuHg==", + "dependencies": { + "Microsoft.CodeAnalysis.Analyzers": "3.11.0" + } + }, + "Microsoft.CodeAnalysis.CSharp": { + "type": "Transitive", + "resolved": "4.14.0", + "contentHash": "568a6wcTivauIhbeWcCwfWwIn7UV7MeHEBvFB2uzGIpM2OhJ4eM/FZ8KS0yhPoNxnSpjGzz7x7CIjTxhslojQA==", + "dependencies": { + "Microsoft.CodeAnalysis.Analyzers": "3.11.0", + "Microsoft.CodeAnalysis.Common": "[4.14.0]" + } + }, + "Microsoft.Diagnostics.NETCore.Client": { + "type": "Transitive", + "resolved": "0.2.510501", + "contentHash": "juoqJYMDs+lRrrZyOkXXMImJHneCF23cuvO4waFRd2Ds7j+ZuGIPbJm0Y/zz34BdeaGiiwGWraMUlln05W1PCQ==", + "dependencies": { + "Microsoft.Extensions.Logging": "6.0.0" + } + }, + "Microsoft.Diagnostics.Runtime": { + "type": "Transitive", + "resolved": "3.1.512801", + "contentHash": "0lMUDr2oxNZa28D6NH5BuSQEe5T9tZziIkvkD44YkkCGQXPJqvFjLq5ZQq1hYLl3RjQJrY+hR0jFgap+EWPDTw==", + "dependencies": { + "Microsoft.Diagnostics.NETCore.Client": "0.2.410101" + } + }, + "Microsoft.Diagnostics.Tracing.TraceEvent": { + "type": "Transitive", + "resolved": "3.1.21", + "contentHash": "/OrJFKaojSR6TkUKtwh8/qA9XWNtxLrXMqvEb89dBSKCWjaGVTbKMYodIUgF5deCEtmd6GXuRerciXGl5bhZ7Q==", + "dependencies": { + "Microsoft.Diagnostics.NETCore.Client": "0.2.510501", + "System.Reflection.TypeExtensions": "4.7.0" + } + }, + "Microsoft.DotNet.PlatformAbstractions": { + "type": "Transitive", + "resolved": "3.1.6", + "contentHash": "jek4XYaQ/PGUwDKKhwR8K47Uh1189PFzMeLqO83mXrXQVIpARZCcfuDedH50YDTepBkfijCZN5U/vZi++erxtg==" + }, + "Microsoft.Extensions.DependencyInjection.Abstractions": { + "type": "Transitive", + "resolved": "10.0.9", + "contentHash": "g41l/30G3K4B/d/L8kjux0+30e27c8D0FVQ/PFCpbekgfDpj9mnDhieP67EqXWvl1EWNeZh2rpR4F5B/jcDOHA==" + }, + "Microsoft.Extensions.Logging": { + "type": "Transitive", + "resolved": "6.0.0", + "contentHash": "eIbyj40QDg1NDz0HBW0S5f3wrLVnKWnDJ/JtZ+yJDFnDj90VoPuoPmFkeaXrtu+0cKm5GRAwoDf+dBWXK0TUdg==", + "dependencies": { + "Microsoft.Extensions.DependencyInjection": "6.0.0", + "Microsoft.Extensions.DependencyInjection.Abstractions": "6.0.0", + "Microsoft.Extensions.Logging.Abstractions": "6.0.0", + "Microsoft.Extensions.Options": "6.0.0" + } + }, + "Microsoft.Extensions.Logging.Abstractions": { + "type": "Transitive", + "resolved": "6.0.0", + "contentHash": "/HggWBbTwy8TgebGSX5DBZ24ndhzi93sHUBDvP1IxbZD7FDokYzdAr6+vbWGjw2XAfR2EJ1sfKUotpjHnFWPxA==" + }, + "Microsoft.Extensions.Options": { + "type": "Transitive", + "resolved": "6.0.0", + "contentHash": "dzXN0+V1AyjOe2xcJ86Qbo233KHuLEY0njf/P2Kw8SfJU+d45HNS2ctJdnEnrWbM9Ye2eFgaC5Mj9otRMU6IsQ==", + "dependencies": { + "Microsoft.Extensions.DependencyInjection.Abstractions": "6.0.0", + "Microsoft.Extensions.Primitives": "6.0.0" + } + }, + "Microsoft.Extensions.Primitives": { + "type": "Transitive", + "resolved": "6.0.0", + "contentHash": "9+PnzmQFfEFNR9J2aDTfJGGupShHjOuGw4VUv+JB044biSHrnmCIMD+mJHmb2H7YryrfBEXDurxQ47gJZdCKNQ==" + }, + "Perfolizer": { + "type": "Transitive", + "resolved": "0.6.1", + "contentHash": "CR1QmWg4XYBd1Pb7WseP+sDmV8nGPwvmowKynExTqr3OuckIGVMhvmN4LC5PGzfXqDlR295+hz/T7syA1CxEqA==", + "dependencies": { + "Pragmastat": "3.2.4" + } + }, + "Pragmastat": { + "type": "Transitive", + "resolved": "3.2.4", + "contentHash": "I5qFifWw/gaTQT52MhzjZpkm/JPlfjSeO/DTZJjO7+hTKI+0aGRgOgZ3NN6D96dDuuqbIAZSeA5RimtHjqrA2A==" + }, + "System.CodeDom": { + "type": "Transitive", + "resolved": "9.0.5", + "contentHash": "cuzLM2MWutf9ZBEMPYYfd0DXwYdvntp7VCT6a/wvbKCa2ZuvGmW74xi+YBa2mrfEieAXqM4TNKlMmSnfAfpUoQ==" + }, + "System.Management": { + "type": "Transitive", + "resolved": "9.0.5", + "contentHash": "n6o9PZm9p25+zAzC3/48K0oHnaPKTInRrxqFq1fi/5TPbMLjuoCm/h//mS3cUmSy+9AO1Z+qsC/Ilt/ZFatv5Q==", + "dependencies": { + "System.CodeDom": "9.0.5" + } + }, + "System.Reflection.TypeExtensions": { + "type": "Transitive", + "resolved": "4.7.0", + "contentHash": "VybpaOQQhqE6siHppMktjfGBw1GCwvCqiufqmP8F1nj7fTUNtW35LOEt3UZTEsECfo+ELAl/9o9nJx3U91i7vA==" + }, + "requestflow": { + "type": "Project", + "dependencies": { + "Microsoft.Extensions.DependencyInjection.Abstractions": "[8.0.1, )", + "RequestFlow.Abstractions": "[1.0.0-preview.9, )" + } + }, + "requestflow.abstractions": { + "type": "Project" + }, + "requestflow.benchmarks.core": { + "type": "Project", + "dependencies": { + "BenchmarkDotNet": "[0.15.8, )", + "Microsoft.Extensions.DependencyInjection": "[10.0.9, )", + "RequestFlow.Benchmarks.Harness": "[1.0.0, )" + } + }, + "requestflow.benchmarks.harness": { + "type": "Project", + "dependencies": { + "BenchmarkDotNet": "[0.15.8, )" + } + }, + "requestflow.benchmarks.requestflowfixtures": { + "type": "Project", + "dependencies": { + "Microsoft.Extensions.DependencyInjection": "[10.0.9, )", + "RequestFlow": "[1.0.0-preview.9, )", + "RequestFlow.Benchmarks.Core": "[1.0.0, )" + } + } + } + } +} \ No newline at end of file diff --git a/benchmarks/RequestFlow.Benchmarks.RequestFlowFixtures/Catalog/RuntimeTaskRequests.cs b/benchmarks/RequestFlow.Benchmarks.RequestFlowFixtures/Catalog/RuntimeTaskRequests.cs new file mode 100644 index 0000000..a981755 --- /dev/null +++ b/benchmarks/RequestFlow.Benchmarks.RequestFlowFixtures/Catalog/RuntimeTaskRequests.cs @@ -0,0 +1,17 @@ +using RequestFlow.Benchmarks.Core; + +namespace RequestFlow.Benchmarks.RequestFlowFixtures.Catalog; + +public sealed class RuntimeTaskRequest(CompletionKind completion = CompletionKind.Cached) + : WorkloadMessage(completion), IRequest; + +public sealed class RuntimeTaskHandler(ScopeWitness scope) + : WorkloadComponent(scope), IRequestHandler, int> +{ + public Task HandleAsync(RuntimeTaskRequest request, CancellationToken cancellationToken) + { + Observe(request, TraceNames.Handler, cancellationToken); + + return BenchmarkWorkload.TaskResponse(request, cancellationToken); + } +} diff --git a/benchmarks/RequestFlow.Benchmarks.RequestFlowFixtures/Events/Events.cs b/benchmarks/RequestFlow.Benchmarks.RequestFlowFixtures/Events/Events.cs new file mode 100644 index 0000000..e365a50 --- /dev/null +++ b/benchmarks/RequestFlow.Benchmarks.RequestFlowFixtures/Events/Events.cs @@ -0,0 +1,23 @@ +using RequestFlow.Benchmarks.Core; + +namespace RequestFlow.Benchmarks.RequestFlowFixtures; + +public sealed class EventMessage(CompletionKind completion = CompletionKind.Cached) + : WorkloadMessage(completion), IEvent; + +public abstract class EventMessageHandler(ScopeWitness scope) : WorkloadComponent(scope), IEventHandler + where TSlot : IStageSlot +{ + public Task HandleAsync(EventMessage @event, CancellationToken cancellationToken) + { + Observe(@event, StageSlot.HandlerName, cancellationToken); + + return RuntimeTracing.EventTask(@event, this, StageSlot.HandlerName, cancellationToken); + } +} + +public sealed class EventMessageHandler1(ScopeWitness scope) : EventMessageHandler(scope); +public sealed class EventMessageHandler2(ScopeWitness scope) : EventMessageHandler(scope); +public sealed class EventMessageHandler3(ScopeWitness scope) : EventMessageHandler(scope); +public sealed class EventMessageHandler4(ScopeWitness scope) : EventMessageHandler(scope); +public sealed class EventMessageHandler5(ScopeWitness scope) : EventMessageHandler(scope); diff --git a/benchmarks/RequestFlow.Benchmarks.RequestFlowFixtures/RequestFlow.Benchmarks.RequestFlowFixtures.csproj b/benchmarks/RequestFlow.Benchmarks.RequestFlowFixtures/RequestFlow.Benchmarks.RequestFlowFixtures.csproj new file mode 100644 index 0000000..e889aa3 --- /dev/null +++ b/benchmarks/RequestFlow.Benchmarks.RequestFlowFixtures/RequestFlow.Benchmarks.RequestFlowFixtures.csproj @@ -0,0 +1,14 @@ + + + + net10.0 + enable + + + + + + + + + diff --git a/benchmarks/RequestFlow.Benchmarks.RequestFlowFixtures/Requests/TaskRequests.cs b/benchmarks/RequestFlow.Benchmarks.RequestFlowFixtures/Requests/TaskRequests.cs new file mode 100644 index 0000000..7ae9157 --- /dev/null +++ b/benchmarks/RequestFlow.Benchmarks.RequestFlowFixtures/Requests/TaskRequests.cs @@ -0,0 +1,35 @@ +using RequestFlow.Benchmarks.Core; + +namespace RequestFlow.Benchmarks.RequestFlowFixtures; + +public sealed class TaskRequest(CompletionKind completion = CompletionKind.Cached, int value = 42) + : WorkloadMessage(completion, value), IRequest; + +public sealed class TaskRequestHandler(ScopeWitness scope) : WorkloadComponent(scope), IRequestHandler +{ + public Task HandleAsync(TaskRequest request, CancellationToken cancellationToken) + { + Observe(request, TraceNames.Handler, cancellationToken); + + return BenchmarkWorkload.TaskResponse(request, cancellationToken); + } +} + +public abstract class TaskStage(ScopeWitness scope) : WorkloadComponent(scope), IRequestStage + where TSlot : IStageSlot +{ + public Task HandleAsync(TaskRequest request, Continuation next, CancellationToken cancellationToken) + { + Observe(request, StageSlot.StageName, cancellationToken); + + Task result = next.InvokeAsync(cancellationToken); + + return request.Completion == CompletionKind.Suspending ? BenchmarkWorkload.AwaitTask(result) : result; + } +} + +public sealed class TaskStage1(ScopeWitness scope) : TaskStage(scope); +public sealed class TaskStage2(ScopeWitness scope) : TaskStage(scope); +public sealed class TaskStage3(ScopeWitness scope) : TaskStage(scope); +public sealed class TaskStage4(ScopeWitness scope) : TaskStage(scope); +public sealed class TaskStage5(ScopeWitness scope) : TaskStage(scope); diff --git a/benchmarks/RequestFlow.Benchmarks.RequestFlowFixtures/Requests/TaskVoidRequests.cs b/benchmarks/RequestFlow.Benchmarks.RequestFlowFixtures/Requests/TaskVoidRequests.cs new file mode 100644 index 0000000..ded01d8 --- /dev/null +++ b/benchmarks/RequestFlow.Benchmarks.RequestFlowFixtures/Requests/TaskVoidRequests.cs @@ -0,0 +1,35 @@ +using RequestFlow.Benchmarks.Core; + +namespace RequestFlow.Benchmarks.RequestFlowFixtures; + +public sealed class TaskVoidRequest(CompletionKind completion = CompletionKind.Cached) + : WorkloadMessage(completion), IRequest; + +public sealed class TaskVoidRequestHandler(ScopeWitness scope) : WorkloadComponent(scope), IRequestHandler +{ + public Task HandleAsync(TaskVoidRequest request, CancellationToken cancellationToken) + { + Observe(request, TraceNames.Handler, cancellationToken); + + return BenchmarkWorkload.TaskVoidRequest(request, cancellationToken); + } +} + +public abstract class TaskVoidRequestStage(ScopeWitness scope) : WorkloadComponent(scope), IRequestStage + where TSlot : IStageSlot +{ + public Task HandleAsync(TaskVoidRequest request, Continuation next, CancellationToken cancellationToken) + { + Observe(request, StageSlot.StageName, cancellationToken); + + Task result = next.InvokeAsync(cancellationToken); + + return request.Completion == CompletionKind.Suspending ? BenchmarkWorkload.AwaitTask(result) : result; + } +} + +public sealed class TaskVoidRequestStage1(ScopeWitness scope) : TaskVoidRequestStage(scope); +public sealed class TaskVoidRequestStage2(ScopeWitness scope) : TaskVoidRequestStage(scope); +public sealed class TaskVoidRequestStage3(ScopeWitness scope) : TaskVoidRequestStage(scope); +public sealed class TaskVoidRequestStage4(ScopeWitness scope) : TaskVoidRequestStage(scope); +public sealed class TaskVoidRequestStage5(ScopeWitness scope) : TaskVoidRequestStage(scope); diff --git a/benchmarks/RequestFlow.Benchmarks.RequestFlowFixtures/Requests/ValueRequests.cs b/benchmarks/RequestFlow.Benchmarks.RequestFlowFixtures/Requests/ValueRequests.cs new file mode 100644 index 0000000..032deb2 --- /dev/null +++ b/benchmarks/RequestFlow.Benchmarks.RequestFlowFixtures/Requests/ValueRequests.cs @@ -0,0 +1,40 @@ +using RequestFlow.Benchmarks.Core; + +namespace RequestFlow.Benchmarks.RequestFlowFixtures; + +public sealed class ValueRequest(CompletionKind completion = CompletionKind.Cached, int value = 42) + : WorkloadMessage(completion, value), IValueRequest; + +public sealed class ValueRequestHandler(ScopeWitness scope) + : WorkloadComponent(scope), IValueRequestHandler +{ + public ValueTask HandleAsync(ValueRequest request, CancellationToken cancellationToken) + { + Observe(request, TraceNames.Handler, cancellationToken); + + return BenchmarkWorkload.ValueResponse(request, cancellationToken); + } +} + +public abstract class ValueStage(ScopeWitness scope) + : WorkloadComponent(scope), IValueRequestStage + where TSlot : IStageSlot +{ + public ValueTask HandleAsync( + ValueRequest request, + ValueContinuation next, + CancellationToken cancellationToken) + { + Observe(request, StageSlot.StageName, cancellationToken); + + ValueTask result = next.InvokeAsync(cancellationToken); + + return request.Completion == CompletionKind.Suspending ? BenchmarkWorkload.AwaitValue(result) : result; + } +} + +public sealed class ValueStage1(ScopeWitness scope) : ValueStage(scope); +public sealed class ValueStage2(ScopeWitness scope) : ValueStage(scope); +public sealed class ValueStage3(ScopeWitness scope) : ValueStage(scope); +public sealed class ValueStage4(ScopeWitness scope) : ValueStage(scope); +public sealed class ValueStage5(ScopeWitness scope) : ValueStage(scope); diff --git a/benchmarks/RequestFlow.Benchmarks.RequestFlowFixtures/Requests/ValueVoidRequests.cs b/benchmarks/RequestFlow.Benchmarks.RequestFlowFixtures/Requests/ValueVoidRequests.cs new file mode 100644 index 0000000..83faecf --- /dev/null +++ b/benchmarks/RequestFlow.Benchmarks.RequestFlowFixtures/Requests/ValueVoidRequests.cs @@ -0,0 +1,36 @@ +using RequestFlow.Benchmarks.Core; + +namespace RequestFlow.Benchmarks.RequestFlowFixtures; + +public sealed class ValueVoidRequest(CompletionKind completion = CompletionKind.Cached) + : WorkloadMessage(completion), IValueRequest; + +public sealed class ValueVoidRequestHandler(ScopeWitness scope) : WorkloadComponent(scope), IValueRequestHandler +{ + public ValueTask HandleAsync(ValueVoidRequest request, CancellationToken cancellationToken) + { + Observe(request, TraceNames.Handler, cancellationToken); + + return BenchmarkWorkload.ValueVoidRequest(request, cancellationToken); + } +} + +public abstract class ValueVoidRequestStage(ScopeWitness scope) + : WorkloadComponent(scope), IValueRequestStage + where TSlot : IStageSlot +{ + public ValueTask HandleAsync(ValueVoidRequest request, ValueContinuation next, CancellationToken cancellationToken) + { + Observe(request, StageSlot.StageName, cancellationToken); + + ValueTask result = next.InvokeAsync(cancellationToken); + + return request.Completion == CompletionKind.Suspending ? BenchmarkWorkload.AwaitValue(result) : result; + } +} + +public sealed class ValueVoidRequestStage1(ScopeWitness scope) : ValueVoidRequestStage(scope); +public sealed class ValueVoidRequestStage2(ScopeWitness scope) : ValueVoidRequestStage(scope); +public sealed class ValueVoidRequestStage3(ScopeWitness scope) : ValueVoidRequestStage(scope); +public sealed class ValueVoidRequestStage4(ScopeWitness scope) : ValueVoidRequestStage(scope); +public sealed class ValueVoidRequestStage5(ScopeWitness scope) : ValueVoidRequestStage(scope); diff --git a/benchmarks/RequestFlow.Benchmarks.RequestFlowFixtures/Setup/RequestFlowSetup.cs b/benchmarks/RequestFlow.Benchmarks.RequestFlowFixtures/Setup/RequestFlowSetup.cs new file mode 100644 index 0000000..a6c35e1 --- /dev/null +++ b/benchmarks/RequestFlow.Benchmarks.RequestFlowFixtures/Setup/RequestFlowSetup.cs @@ -0,0 +1,246 @@ +using System.Reflection; +using Microsoft.Extensions.DependencyInjection; +using RequestFlow.Benchmarks.Core; +using RequestFlow.Benchmarks.RequestFlowFixtures.Catalog; + +namespace RequestFlow.Benchmarks.RequestFlowFixtures; + +public static class RequestFlowSetup +{ + /// + /// RequestFlow services that registration reports print besides the dispatcher. + /// + public static readonly Type[] ReportedServices = [typeof(IEventPublisher)]; + + /// + /// RequestFlow stage contracts whose implementations registration reports print. + /// + public static readonly Type[] StageContracts = + [ + typeof(IRequestStage<,>), + typeof(IRequestStage<>), + typeof(IValueRequestStage<,>), + typeof(IValueRequestStage<>), + typeof(IStreamRequestStage<,>), + ]; + + /// + /// RequestFlow as one side of a paired scope check. + /// + public static readonly ComparedLibrary Library = new("RequestFlow", Build, ReportedServices, StageContracts); + + private static readonly Type[] TaskStages = + [ + typeof(TaskStage1), + typeof(TaskStage2), + typeof(TaskStage3), + typeof(TaskStage4), + typeof(TaskStage5), + ]; + + private static readonly Type[] ValueStages = + [ + typeof(ValueStage1), + typeof(ValueStage2), + typeof(ValueStage3), + typeof(ValueStage4), + typeof(ValueStage5), + ]; + + private static readonly Type[] TaskVoidRequestStages = + [ + typeof(TaskVoidRequestStage1), + typeof(TaskVoidRequestStage2), + typeof(TaskVoidRequestStage3), + typeof(TaskVoidRequestStage4), + typeof(TaskVoidRequestStage5), + ]; + + private static readonly Type[] ValueVoidRequestStages = + [ + typeof(ValueVoidRequestStage1), + typeof(ValueVoidRequestStage2), + typeof(ValueVoidRequestStage3), + typeof(ValueVoidRequestStage4), + typeof(ValueVoidRequestStage5), + ]; + + private static readonly Type[] StreamStages = + [ + typeof(StreamStage1), + typeof(StreamStage2), + typeof(StreamStage3), + typeof(StreamStage4), + typeof(StreamStage5), + ]; + + private static readonly MethodInfo AddHandler = typeof(RequestFlowOptions) + .GetMethod(nameof(RequestFlowOptions.AddHandler), Type.EmptyTypes)!; + + public static ServiceCollection TaskServices(int stageCount, ServiceLifetime dispatcherLifetime = ServiceLifetime.Scoped) + => RequestServices( + stageCount, + options => options.AddHandler(), + TaskStages, + valueTask: false, + dispatcherLifetime); + + public static ServiceCollection ValueServices(int stageCount) + => RequestServices( + stageCount, + options => options.AddHandler(), + ValueStages, + valueTask: true, + ServiceLifetime.Scoped); + + public static ServiceCollection TaskVoidRequestServices(int stageCount, ServiceLifetime dispatcherLifetime = ServiceLifetime.Scoped) + => RequestServices( + stageCount, + options => options.AddHandler(), + TaskVoidRequestStages, + valueTask: false, + dispatcherLifetime); + + public static ServiceCollection ValueVoidRequestServices(int stageCount) + => RequestServices( + stageCount, + options => options.AddHandler(), + ValueVoidRequestStages, + valueTask: true, + ServiceLifetime.Scoped); + + public static ServiceCollection StreamServices(int stageCount, ServiceLifetime dispatcherLifetime = ServiceLifetime.Scoped) + { + CheckStageCount(stageCount); + + var services = CreateServices(); + + services.AddRequestFlow(options => + { + SetDispatcherLifetime(options, dispatcherLifetime); + options.WithScopedHandlers().AddHandler(); + + for (int i = 0; i < stageCount; i++) + options.AddStreamStage(StreamStages[i], stage => stage.AsScoped()); + }); + + return services; + } + + public static ServiceCollection EventServices( + int handlerCount, bool parallel = false, ServiceLifetime dispatcherLifetime = ServiceLifetime.Scoped) + { + if (handlerCount is not (1 or 5)) + throw new ArgumentOutOfRangeException(nameof(handlerCount)); + + var services = CreateServices(); + + services.AddRequestFlow(options => + { + SetDispatcherLifetime(options, dispatcherLifetime); + options.WithScopedHandlers().AddEventHandler(); + + if (handlerCount == 5) + { + options.AddEventHandler(); + options.AddEventHandler(); + options.AddEventHandler(); + options.AddEventHandler(); + } + + if (parallel) + options.PublishEventsInParallel(); + else + options.PublishEventsSequentially(); + }); + + return services; + } + + /// + /// Registers one scoped per tag in . + /// + public static ServiceCollection CatalogServices(int count, ServiceLifetime dispatcherLifetime = ServiceLifetime.Scoped) + { + Type[] tags = RuntimeCatalog.Tags(count); + var services = CreateServices(); + + services.AddRequestFlow(options => + { + SetDispatcherLifetime(options, dispatcherLifetime); + options.WithScopedHandlers(); + + foreach (Type tag in tags) + AddHandler.MakeGenericMethod(typeof(RuntimeTaskHandler<>).MakeGenericType(tag)).Invoke(options, null); + }); + + return services; + } + + public static ServiceProvider Build(IServiceCollection services) + { + var provider = services.BuildServiceProvider(new ServiceProviderOptions + { + ValidateOnBuild = true, + ValidateScopes = true, + }); + + provider.ValidateRequestFlow(); + + return provider; + } + + private static ServiceCollection RequestServices( + int stageCount, + Action registerHandler, + Type[] stageTypes, + bool valueTask, + ServiceLifetime dispatcherLifetime) + { + CheckStageCount(stageCount); + + var services = CreateServices(); + + services.AddRequestFlow(options => + { + SetDispatcherLifetime(options, dispatcherLifetime); + options.WithScopedHandlers(); + registerHandler(options); + + for (int i = 0; i < stageCount; i++) + { + Type stageType = stageTypes[i]; + + if (valueTask) + options.AddValueStage(stageType, stage => stage.AsScoped()); + else + options.AddStage(stageType, stage => stage.AsScoped()); + } + }); + + return services; + } + + internal static void SetDispatcherLifetime(RequestFlowOptions options, ServiceLifetime dispatcherLifetime) + { + if (dispatcherLifetime == ServiceLifetime.Transient) + options.WithTransientDispatcher(); + else if (dispatcherLifetime != ServiceLifetime.Scoped) + throw new ArgumentOutOfRangeException(nameof(dispatcherLifetime), dispatcherLifetime, "RequestFlow dispatchers are scoped or transient."); + } + + private static ServiceCollection CreateServices() + { + var services = new ServiceCollection(); + + services.AddScoped(); + + return services; + } + + private static void CheckStageCount(int stageCount) + { + if (stageCount is not (0 or 1 or 5)) + throw new ArgumentOutOfRangeException(nameof(stageCount)); + } +} diff --git a/benchmarks/RequestFlow.Benchmarks.RequestFlowFixtures/ShortCircuit/ShortCircuitServices.cs b/benchmarks/RequestFlow.Benchmarks.RequestFlowFixtures/ShortCircuit/ShortCircuitServices.cs new file mode 100644 index 0000000..dad57b6 --- /dev/null +++ b/benchmarks/RequestFlow.Benchmarks.RequestFlowFixtures/ShortCircuit/ShortCircuitServices.cs @@ -0,0 +1,71 @@ +using Microsoft.Extensions.DependencyInjection; +using RequestFlow.Benchmarks.Core; + +namespace RequestFlow.Benchmarks.RequestFlowFixtures.ShortCircuit; + +public static class ShortCircuitServices +{ + public static ServiceProvider Build( + int bufferBytes, + ConstructionTrace? trace = null, + bool valueTask = false, + ServiceLifetime dispatcherLifetime = ServiceLifetime.Scoped) + => RequestFlowSetup.Build(Services(bufferBytes, trace, valueTask, dispatcherLifetime)); + + public static ServiceCollection Services( + int bufferBytes, + ConstructionTrace? trace = null, + bool valueTask = false, + ServiceLifetime dispatcherLifetime = ServiceLifetime.Scoped) + { + var services = new ServiceCollection(); + + ShortCircuitWorkload.AddDependencies(services, bufferBytes, trace); + + services.AddRequestFlow(options => + { + RequestFlowSetup.SetDispatcherLifetime(options, dispatcherLifetime); + options.WithScopedHandlers(); + + if (valueTask) + { + options.AddHandler(); + options.AddValueStage(typeof(ShortCircuitValueAuthorization), stage => stage.AsScoped()); + options.AddValueStage(typeof(ShortCircuitValueCache), stage => stage.AsScoped()); + options.AddValueStage(typeof(ShortCircuitValueTransaction), stage => stage.AsScoped()); + } + else + { + options.AddHandler(); + options.AddStage(typeof(ShortCircuitTaskAuthorization), stage => stage.AsScoped()); + options.AddStage(typeof(ShortCircuitTaskCache), stage => stage.AsScoped()); + options.AddStage(typeof(ShortCircuitTaskTransaction), stage => stage.AsScoped()); + } + }); + + return services; + } + + public static void VerifyRequestFlow( + bool valueTask, + Func? build = null, + ServiceLifetime dispatcherLifetime = ServiceLifetime.Scoped) + { + if (valueTask) + ShortCircuitVerification.Verify( + "RequestFlow ValueTask", + build ?? ((bytes, trace) => Build(bytes, trace, valueTask: true, dispatcherLifetime)), + outcome => new(outcome), + (dispatcher, message, token) => dispatcher.SendAsync(message, token), + DependencyResolution.EnteredOnly, + dispatcherLifetime: dispatcherLifetime); + else + ShortCircuitVerification.Verify( + "RequestFlow Task", + build ?? ((bytes, trace) => Build(bytes, trace, dispatcherLifetime: dispatcherLifetime)), + outcome => new(outcome), + (dispatcher, message, token) => new(dispatcher.SendAsync(message, token)), + DependencyResolution.EnteredOnly, + dispatcherLifetime: dispatcherLifetime); + } +} diff --git a/benchmarks/RequestFlow.Benchmarks.RequestFlowFixtures/ShortCircuit/ShortCircuitTaskWorkloads.cs b/benchmarks/RequestFlow.Benchmarks.RequestFlowFixtures/ShortCircuit/ShortCircuitTaskWorkloads.cs new file mode 100644 index 0000000..130b944 --- /dev/null +++ b/benchmarks/RequestFlow.Benchmarks.RequestFlowFixtures/ShortCircuit/ShortCircuitTaskWorkloads.cs @@ -0,0 +1,61 @@ +using RequestFlow.Benchmarks.Core; + +namespace RequestFlow.Benchmarks.RequestFlowFixtures.ShortCircuit; + +public sealed class ShortCircuitTaskRequest(ShortCircuitOutcome outcome) + : ShortCircuitMessage(outcome), IRequest; + +public sealed class ShortCircuitTaskAuthorization(DependencyState state, AccessPolicy dependency) + : DependencyComponent(state, "authorization"), IRequestStage +{ + public Task HandleAsync( + ShortCircuitTaskRequest request, + Continuation next, + CancellationToken token) + { + Enter(request, token); + + return dependency.Allows(request) ? next.InvokeAsync(token) : ShortCircuitWorkload.Rejected; + } +} + +public sealed class ShortCircuitTaskCache(DependencyState state, CachePolicy dependency) + : DependencyComponent(state, "cache"), IRequestStage +{ + public Task HandleAsync( + ShortCircuitTaskRequest request, + Continuation next, + CancellationToken token) + { + Enter(request, token); + + return dependency.Contains(request) ? ShortCircuitWorkload.Cached : next.InvokeAsync(token); + } +} + +public sealed class ShortCircuitTaskTransaction(DependencyState state, UnitOfWork dependency) + : DependencyComponent(state, "transaction"), IRequestStage +{ + public Task HandleAsync( + ShortCircuitTaskRequest request, + Continuation next, + CancellationToken token) + { + Enter(request, token); + + return dependency.Capacity >= 0 ? next.InvokeAsync(token) : throw new InvalidOperationException(); + } +} + +public sealed class ShortCircuitTaskHandler(DependencyState state, Repository dependency) + : DependencyComponent(state, TraceNames.Handler), IRequestHandler +{ + public Task HandleAsync(ShortCircuitTaskRequest request, CancellationToken token) + { + Enter(request, token); + + Task result = dependency.Read(); + + return request.Trace is { } trace ? trace.Track(result) : result; + } +} diff --git a/benchmarks/RequestFlow.Benchmarks.RequestFlowFixtures/ShortCircuit/ShortCircuitValueWorkloads.cs b/benchmarks/RequestFlow.Benchmarks.RequestFlowFixtures/ShortCircuit/ShortCircuitValueWorkloads.cs new file mode 100644 index 0000000..57f3177 --- /dev/null +++ b/benchmarks/RequestFlow.Benchmarks.RequestFlowFixtures/ShortCircuit/ShortCircuitValueWorkloads.cs @@ -0,0 +1,61 @@ +using RequestFlow.Benchmarks.Core; + +namespace RequestFlow.Benchmarks.RequestFlowFixtures.ShortCircuit; + +public sealed class ShortCircuitValueRequest(ShortCircuitOutcome outcome) + : ShortCircuitMessage(outcome), IValueRequest; + +public sealed class ShortCircuitValueAuthorization(DependencyState state, AccessPolicy dependency) + : DependencyComponent(state, "authorization"), IValueRequestStage +{ + public ValueTask HandleAsync( + ShortCircuitValueRequest request, + ValueContinuation next, + CancellationToken token) + { + Enter(request, token); + + return dependency.Allows(request) ? next.InvokeAsync(token) : new ValueTask(ShortCircuitWorkload.RejectedResult); + } +} + +public sealed class ShortCircuitValueCache(DependencyState state, CachePolicy dependency) + : DependencyComponent(state, "cache"), IValueRequestStage +{ + public ValueTask HandleAsync( + ShortCircuitValueRequest request, + ValueContinuation next, + CancellationToken token) + { + Enter(request, token); + + return dependency.Contains(request) ? new ValueTask(ShortCircuitWorkload.CachedResult) : next.InvokeAsync(token); + } +} + +public sealed class ShortCircuitValueTransaction(DependencyState state, UnitOfWork dependency) + : DependencyComponent(state, "transaction"), IValueRequestStage +{ + public ValueTask HandleAsync( + ShortCircuitValueRequest request, + ValueContinuation next, + CancellationToken token) + { + Enter(request, token); + + return dependency.Capacity >= 0 ? next.InvokeAsync(token) : throw new InvalidOperationException(); + } +} + +public sealed class ShortCircuitValueHandler(DependencyState state, Repository dependency) + : DependencyComponent(state, TraceNames.Handler), IValueRequestHandler +{ + public ValueTask HandleAsync(ShortCircuitValueRequest request, CancellationToken token) + { + Enter(request, token); + + ValueTask result = dependency.ReadValue(); + + return request.Trace is { } trace ? trace.Track(result) : result; + } +} diff --git a/benchmarks/RequestFlow.Benchmarks.RequestFlowFixtures/Streams/Streams.cs b/benchmarks/RequestFlow.Benchmarks.RequestFlowFixtures/Streams/Streams.cs new file mode 100644 index 0000000..58a2497 --- /dev/null +++ b/benchmarks/RequestFlow.Benchmarks.RequestFlowFixtures/Streams/Streams.cs @@ -0,0 +1,41 @@ +using RequestFlow.Benchmarks.Core; + +namespace RequestFlow.Benchmarks.RequestFlowFixtures; + +public sealed class StreamRequest(int count = 10, CompletionKind completion = CompletionKind.Synchronous) + : WorkloadMessage(completion), IStreamRequest +{ + public int Count { get; } = count; +} + +public sealed class StreamHandler(ScopeWitness scope) + : WorkloadComponent(scope), IStreamRequestHandler +{ + public IAsyncEnumerable Handle(StreamRequest request, CancellationToken cancellationToken) + { + Observe(request, TraceNames.Handler, cancellationToken); + + return RuntimeTracing.Stream(request, request.Count, this, cancellationToken); + } +} + +public abstract class StreamStage(ScopeWitness scope) + : WorkloadComponent(scope), IStreamRequestStage + where TSlot : IStageSlot +{ + public IAsyncEnumerable Handle( + StreamRequest request, + StreamContinuation next, + CancellationToken cancellationToken) + { + Observe(request, StageSlot.StageName, cancellationToken); + + return next.Invoke(cancellationToken); + } +} + +public sealed class StreamStage1(ScopeWitness scope) : StreamStage(scope); +public sealed class StreamStage2(ScopeWitness scope) : StreamStage(scope); +public sealed class StreamStage3(ScopeWitness scope) : StreamStage(scope); +public sealed class StreamStage4(ScopeWitness scope) : StreamStage(scope); +public sealed class StreamStage5(ScopeWitness scope) : StreamStage(scope); diff --git a/benchmarks/RequestFlow.Benchmarks.RequestFlowFixtures/packages.lock.json b/benchmarks/RequestFlow.Benchmarks.RequestFlowFixtures/packages.lock.json new file mode 100644 index 0000000..0e10696 --- /dev/null +++ b/benchmarks/RequestFlow.Benchmarks.RequestFlowFixtures/packages.lock.json @@ -0,0 +1,195 @@ +{ + "version": 1, + "dependencies": { + "net10.0": { + "Microsoft.Extensions.DependencyInjection": { + "type": "Direct", + "requested": "[10.0.9, )", + "resolved": "10.0.9", + "contentHash": "NijozhERJDIaJ4k5TSMy1jOi0cSC2HfkvRD/Sl+kGSSKgVbFnF4GxgtMN/MrzHB8D1JxIrD4xSer9Blh9v3axQ==", + "dependencies": { + "Microsoft.Extensions.DependencyInjection.Abstractions": "10.0.9" + } + }, + "BenchmarkDotNet": { + "type": "Transitive", + "resolved": "0.15.8", + "contentHash": "paCfrWxSeHqn3rUZc0spYXVFnHCF0nzRhG0nOLnyTjZYs8spsimBaaNmb3vwqvALKIplbYq/TF393vYiYSnh/Q==", + "dependencies": { + "BenchmarkDotNet.Annotations": "0.15.8", + "CommandLineParser": "2.9.1", + "Gee.External.Capstone": "2.3.0", + "Iced": "1.21.0", + "Microsoft.CodeAnalysis.CSharp": "4.14.0", + "Microsoft.Diagnostics.Runtime": "3.1.512801", + "Microsoft.Diagnostics.Tracing.TraceEvent": "3.1.21", + "Microsoft.DotNet.PlatformAbstractions": "3.1.6", + "Perfolizer": "[0.6.1]", + "System.Management": "9.0.5" + } + }, + "BenchmarkDotNet.Annotations": { + "type": "Transitive", + "resolved": "0.15.8", + "contentHash": "hfucY0ycAsB0SsoaZcaAp9oq5wlWBJcylvEJb9pmvdYUx6PD6S4mDiYnZWjdjAlLhIpe/xtGCwzORfzAzPqvzA==" + }, + "CommandLineParser": { + "type": "Transitive", + "resolved": "2.9.1", + "contentHash": "OE0sl1/sQ37bjVsPKKtwQlWDgqaxWgtme3xZz7JssWUzg5JpMIyHgCTY9MVMxOg48fJ1AgGT3tgdH5m/kQ5xhA==" + }, + "Gee.External.Capstone": { + "type": "Transitive", + "resolved": "2.3.0", + "contentHash": "2ap/rYmjtzCOT8hxrnEW/QeiOt+paD8iRrIcdKX0cxVwWLFa1e+JDBNeECakmccXrSFeBQuu5AV8SNkipFMMMw==" + }, + "Iced": { + "type": "Transitive", + "resolved": "1.21.0", + "contentHash": "dv5+81Q1TBQvVMSOOOmRcjJmvWcX3BZPZsIq31+RLc5cNft0IHAyNlkdb7ZarOWG913PyBoFDsDXoCIlKmLclg==" + }, + "Microsoft.CodeAnalysis.Analyzers": { + "type": "Transitive", + "resolved": "3.11.0", + "contentHash": "v/EW3UE8/lbEYHoC2Qq7AR/DnmvpgdtAMndfQNmpuIMx/Mto8L5JnuCfdBYtgvalQOtfNCnxFejxuRrryvUTsg==" + }, + "Microsoft.CodeAnalysis.Common": { + "type": "Transitive", + "resolved": "4.14.0", + "contentHash": "PC3tuwZYnC+idaPuoC/AZpEdwrtX7qFpmnrfQkgobGIWiYmGi5MCRtl5mx6QrfMGQpK78X2lfIEoZDLg/qnuHg==", + "dependencies": { + "Microsoft.CodeAnalysis.Analyzers": "3.11.0" + } + }, + "Microsoft.CodeAnalysis.CSharp": { + "type": "Transitive", + "resolved": "4.14.0", + "contentHash": "568a6wcTivauIhbeWcCwfWwIn7UV7MeHEBvFB2uzGIpM2OhJ4eM/FZ8KS0yhPoNxnSpjGzz7x7CIjTxhslojQA==", + "dependencies": { + "Microsoft.CodeAnalysis.Analyzers": "3.11.0", + "Microsoft.CodeAnalysis.Common": "[4.14.0]" + } + }, + "Microsoft.Diagnostics.NETCore.Client": { + "type": "Transitive", + "resolved": "0.2.510501", + "contentHash": "juoqJYMDs+lRrrZyOkXXMImJHneCF23cuvO4waFRd2Ds7j+ZuGIPbJm0Y/zz34BdeaGiiwGWraMUlln05W1PCQ==", + "dependencies": { + "Microsoft.Extensions.Logging": "6.0.0" + } + }, + "Microsoft.Diagnostics.Runtime": { + "type": "Transitive", + "resolved": "3.1.512801", + "contentHash": "0lMUDr2oxNZa28D6NH5BuSQEe5T9tZziIkvkD44YkkCGQXPJqvFjLq5ZQq1hYLl3RjQJrY+hR0jFgap+EWPDTw==", + "dependencies": { + "Microsoft.Diagnostics.NETCore.Client": "0.2.410101" + } + }, + "Microsoft.Diagnostics.Tracing.TraceEvent": { + "type": "Transitive", + "resolved": "3.1.21", + "contentHash": "/OrJFKaojSR6TkUKtwh8/qA9XWNtxLrXMqvEb89dBSKCWjaGVTbKMYodIUgF5deCEtmd6GXuRerciXGl5bhZ7Q==", + "dependencies": { + "Microsoft.Diagnostics.NETCore.Client": "0.2.510501", + "System.Reflection.TypeExtensions": "4.7.0" + } + }, + "Microsoft.DotNet.PlatformAbstractions": { + "type": "Transitive", + "resolved": "3.1.6", + "contentHash": "jek4XYaQ/PGUwDKKhwR8K47Uh1189PFzMeLqO83mXrXQVIpARZCcfuDedH50YDTepBkfijCZN5U/vZi++erxtg==" + }, + "Microsoft.Extensions.DependencyInjection.Abstractions": { + "type": "Transitive", + "resolved": "10.0.9", + "contentHash": "g41l/30G3K4B/d/L8kjux0+30e27c8D0FVQ/PFCpbekgfDpj9mnDhieP67EqXWvl1EWNeZh2rpR4F5B/jcDOHA==" + }, + "Microsoft.Extensions.Logging": { + "type": "Transitive", + "resolved": "6.0.0", + "contentHash": "eIbyj40QDg1NDz0HBW0S5f3wrLVnKWnDJ/JtZ+yJDFnDj90VoPuoPmFkeaXrtu+0cKm5GRAwoDf+dBWXK0TUdg==", + "dependencies": { + "Microsoft.Extensions.DependencyInjection": "6.0.0", + "Microsoft.Extensions.DependencyInjection.Abstractions": "6.0.0", + "Microsoft.Extensions.Logging.Abstractions": "6.0.0", + "Microsoft.Extensions.Options": "6.0.0" + } + }, + "Microsoft.Extensions.Logging.Abstractions": { + "type": "Transitive", + "resolved": "6.0.0", + "contentHash": "/HggWBbTwy8TgebGSX5DBZ24ndhzi93sHUBDvP1IxbZD7FDokYzdAr6+vbWGjw2XAfR2EJ1sfKUotpjHnFWPxA==" + }, + "Microsoft.Extensions.Options": { + "type": "Transitive", + "resolved": "6.0.0", + "contentHash": "dzXN0+V1AyjOe2xcJ86Qbo233KHuLEY0njf/P2Kw8SfJU+d45HNS2ctJdnEnrWbM9Ye2eFgaC5Mj9otRMU6IsQ==", + "dependencies": { + "Microsoft.Extensions.DependencyInjection.Abstractions": "6.0.0", + "Microsoft.Extensions.Primitives": "6.0.0" + } + }, + "Microsoft.Extensions.Primitives": { + "type": "Transitive", + "resolved": "6.0.0", + "contentHash": "9+PnzmQFfEFNR9J2aDTfJGGupShHjOuGw4VUv+JB044biSHrnmCIMD+mJHmb2H7YryrfBEXDurxQ47gJZdCKNQ==" + }, + "Perfolizer": { + "type": "Transitive", + "resolved": "0.6.1", + "contentHash": "CR1QmWg4XYBd1Pb7WseP+sDmV8nGPwvmowKynExTqr3OuckIGVMhvmN4LC5PGzfXqDlR295+hz/T7syA1CxEqA==", + "dependencies": { + "Pragmastat": "3.2.4" + } + }, + "Pragmastat": { + "type": "Transitive", + "resolved": "3.2.4", + "contentHash": "I5qFifWw/gaTQT52MhzjZpkm/JPlfjSeO/DTZJjO7+hTKI+0aGRgOgZ3NN6D96dDuuqbIAZSeA5RimtHjqrA2A==" + }, + "System.CodeDom": { + "type": "Transitive", + "resolved": "9.0.5", + "contentHash": "cuzLM2MWutf9ZBEMPYYfd0DXwYdvntp7VCT6a/wvbKCa2ZuvGmW74xi+YBa2mrfEieAXqM4TNKlMmSnfAfpUoQ==" + }, + "System.Management": { + "type": "Transitive", + "resolved": "9.0.5", + "contentHash": "n6o9PZm9p25+zAzC3/48K0oHnaPKTInRrxqFq1fi/5TPbMLjuoCm/h//mS3cUmSy+9AO1Z+qsC/Ilt/ZFatv5Q==", + "dependencies": { + "System.CodeDom": "9.0.5" + } + }, + "System.Reflection.TypeExtensions": { + "type": "Transitive", + "resolved": "4.7.0", + "contentHash": "VybpaOQQhqE6siHppMktjfGBw1GCwvCqiufqmP8F1nj7fTUNtW35LOEt3UZTEsECfo+ELAl/9o9nJx3U91i7vA==" + }, + "requestflow": { + "type": "Project", + "dependencies": { + "Microsoft.Extensions.DependencyInjection.Abstractions": "[8.0.1, )", + "RequestFlow.Abstractions": "[1.0.0-preview.9, )" + } + }, + "requestflow.abstractions": { + "type": "Project" + }, + "requestflow.benchmarks.core": { + "type": "Project", + "dependencies": { + "BenchmarkDotNet": "[0.15.8, )", + "Microsoft.Extensions.DependencyInjection": "[10.0.9, )", + "RequestFlow.Benchmarks.Harness": "[1.0.0, )" + } + }, + "requestflow.benchmarks.harness": { + "type": "Project", + "dependencies": { + "BenchmarkDotNet": "[0.15.8, )" + } + } + } + } +} \ No newline at end of file diff --git a/benchmarks/RequestFlow.Benchmarks/Chains/ChainLevelBenchmarks.cs b/benchmarks/RequestFlow.Benchmarks/Chains/ChainLevelBenchmarks.cs new file mode 100644 index 0000000..801177f --- /dev/null +++ b/benchmarks/RequestFlow.Benchmarks/Chains/ChainLevelBenchmarks.cs @@ -0,0 +1,114 @@ +using BenchmarkDotNet.Attributes; +using BenchmarkDotNet.Configs; +using Microsoft.Extensions.DependencyInjection; +using RequestFlow.Benchmarks.Core; +using RequestFlow.Benchmarks.RequestFlowFixtures; + +namespace RequestFlow.Benchmarks; + +/// +/// The Task and ValueTask plans that AddRequestFlow froze, entered directly, so each row holds the stage chain and not the dispatcher around it. +/// Each stage adds costs a DI lookup and the fixture stage body as well as the chain step. +/// +[BenchmarkScope(ScopeLifetime.RetainedScope)] +[GroupBenchmarksBy(BenchmarkLogicalGroupRule.ByCategory)] +public class ChainLevelBenchmarks +{ + private const string ResponseCategory = "Response"; + private const string VoidCategory = "Void"; + + [Params(0, 1, 5)] + public int StageCount { get; set; } + + private PlanHost> _task = null!; + private PlanHost> _value = null!; + private PlanHost> _taskVoid = null!; + private PlanHost _valueVoid = null!; + + private RequestPlan _taskPlan = null!; + private ValueRequestPlan _valuePlan = null!; + private RequestPlan _taskVoidPlan = null!; + private ValueVoidRequestPlan _valueVoidPlan = null!; + + private IServiceProvider _taskServices = null!; + private IServiceProvider _valueServices = null!; + private IServiceProvider _taskVoidServices = null!; + private IServiceProvider _valueVoidServices = null!; + + private readonly TaskRequest _taskRequest = new(); + private readonly ValueRequest _valueRequest = new(); + private readonly TaskVoidRequest _taskVoidRequest = new(); + private readonly ValueVoidRequest _valueVoidRequest = new(); + + [GlobalSetup] + public void Setup() + { + _task = new(RequestFlowSetup.TaskServices(StageCount), typeof(TaskRequest)); + _value = new(RequestFlowSetup.ValueServices(StageCount), typeof(ValueRequest)); + _taskVoid = new(RequestFlowSetup.TaskVoidRequestServices(StageCount), typeof(TaskVoidRequest)); + _valueVoid = new(RequestFlowSetup.ValueVoidRequestServices(StageCount), typeof(ValueVoidRequest)); + + (_taskPlan, _taskServices) = (_task.Plan, _task.Services); + (_valuePlan, _valueServices) = (_value.Plan, _value.Services); + (_taskVoidPlan, _taskVoidServices) = (_taskVoid.Plan, _taskVoid.Services); + (_valueVoidPlan, _valueVoidServices) = (_valueVoid.Plan, _valueVoid.Services); + + TaskChain().GetAwaiter().GetResult(); + ValueChain().GetAwaiter().GetResult(); + TaskVoidChain().GetAwaiter().GetResult(); + ValueVoidChain().GetAwaiter().GetResult(); + } + + [Benchmark(Baseline = true), BenchmarkCategory(ResponseCategory)] + public Task TaskChain() + => _taskPlan.ExecuteAsync(_taskRequest, _taskServices, CancellationToken.None); + + [Benchmark, BenchmarkCategory(ResponseCategory)] + public ValueTask ValueChain() + => _valuePlan.ExecuteAsync(_valueRequest, _valueServices, CancellationToken.None); + + [Benchmark(Baseline = true), BenchmarkCategory(VoidCategory)] + public Task TaskVoidChain() + => _taskVoidPlan.ExecuteAsync(_taskVoidRequest, _taskVoidServices, CancellationToken.None); + + [Benchmark, BenchmarkCategory(VoidCategory)] + public ValueTask ValueVoidChain() + => _valueVoidPlan.ExecuteVoidAsync(_valueVoidRequest, _valueVoidServices, CancellationToken.None); + + [GlobalCleanup] + public void Cleanup() + { + _task.Dispose(); + _value.Dispose(); + _taskVoid.Dispose(); + _valueVoid.Dispose(); + } + + // A provider built from the fixture registrations, one retained scope, and the plan the provider froze for one request type. + private sealed class PlanHost : IDisposable + where TPlan : RequestPlanBase + { + private readonly ServiceProvider _provider; + private readonly IServiceScope _scope; + + public PlanHost(ServiceCollection services, Type requestType) + { + _provider = RequestFlowSetup.Build(services); + _scope = _provider.CreateScope(); + + Plan = _provider.GetRequiredService().TryGetPlanFor(requestType, out RequestPlanBase? plan) + ? (TPlan)plan + : throw new InvalidOperationException($"RequestFlow froze no plan for '{requestType.Name}'."); + } + + public TPlan Plan { get; } + + public IServiceProvider Services => _scope.ServiceProvider; + + public void Dispose() + { + _scope.Dispose(); + _provider.Dispose(); + } + } +} diff --git a/benchmarks/RequestFlow.Benchmarks/Program.cs b/benchmarks/RequestFlow.Benchmarks/Program.cs new file mode 100644 index 0000000..4a29ea8 --- /dev/null +++ b/benchmarks/RequestFlow.Benchmarks/Program.cs @@ -0,0 +1,9 @@ +using RequestFlow.Benchmarks.Harness; + +namespace RequestFlow.Benchmarks; + +public static class Program +{ + public static int Main(string[] args) + => SuiteRunner.Run(typeof(Program).Assembly, "requestflow", args, SuiteProbe.Run); +} diff --git a/benchmarks/RequestFlow.Benchmarks/RequestFlow.Benchmarks.csproj b/benchmarks/RequestFlow.Benchmarks/RequestFlow.Benchmarks.csproj new file mode 100644 index 0000000..0325b4b --- /dev/null +++ b/benchmarks/RequestFlow.Benchmarks/RequestFlow.Benchmarks.csproj @@ -0,0 +1,20 @@ + + + + Exe + net10.0 + enable + + + + + + + + + + + + + + diff --git a/benchmarks/RequestFlow.Benchmarks/ShortCircuit/ShortCircuitRetainedScopeBenchmarks.cs b/benchmarks/RequestFlow.Benchmarks/ShortCircuit/ShortCircuitRetainedScopeBenchmarks.cs new file mode 100644 index 0000000..e2be206 --- /dev/null +++ b/benchmarks/RequestFlow.Benchmarks/ShortCircuit/ShortCircuitRetainedScopeBenchmarks.cs @@ -0,0 +1,46 @@ +using BenchmarkDotNet.Attributes; +using RequestFlow.Benchmarks.Core; +using RequestFlow.Benchmarks.RequestFlowFixtures.ShortCircuit; + +namespace RequestFlow.Benchmarks; + +[BenchmarkScope(ScopeLifetime.RetainedScope)] +public class ShortCircuitRetainedScopeBenchmarks +{ + [Params(ShortCircuitOutcome.AuthorizationReject, ShortCircuitOutcome.CacheHit, ShortCircuitOutcome.FullExecution)] + public ShortCircuitOutcome Outcome { get; set; } + + private RetainedScopeHost _task = null!; + private RetainedScopeHost _value = null!; + + private ShortCircuitTaskRequest _taskRequest = null!; + private ShortCircuitValueRequest _valueRequest = null!; + + [GlobalSetup] + public void Setup() + { + _task = new(ShortCircuitServices.Build(bufferBytes: 0, valueTask: false)); + _value = new(ShortCircuitServices.Build(bufferBytes: 0, valueTask: true)); + + _taskRequest = new(Outcome); + _valueRequest = new(Outcome); + + TaskPath().GetAwaiter().GetResult(); + ValueTaskPath().GetAwaiter().GetResult(); + } + + [Benchmark(Baseline = true)] + public Task TaskPath() + => _task.Service.SendAsync(_taskRequest, CancellationToken.None); + + [Benchmark] + public ValueTask ValueTaskPath() + => _value.Service.SendAsync(_valueRequest, CancellationToken.None); + + [GlobalCleanup] + public void Cleanup() + { + _task.Dispose(); + _value.Dispose(); + } +} diff --git a/benchmarks/RequestFlow.Benchmarks/ShortCircuit/ShortCircuitScopePerOperationBenchmarks.cs b/benchmarks/RequestFlow.Benchmarks/ShortCircuit/ShortCircuitScopePerOperationBenchmarks.cs new file mode 100644 index 0000000..d92569c --- /dev/null +++ b/benchmarks/RequestFlow.Benchmarks/ShortCircuit/ShortCircuitScopePerOperationBenchmarks.cs @@ -0,0 +1,60 @@ +using BenchmarkDotNet.Attributes; +using Microsoft.Extensions.DependencyInjection; +using RequestFlow.Benchmarks.Core; +using RequestFlow.Benchmarks.RequestFlowFixtures.ShortCircuit; + +namespace RequestFlow.Benchmarks; + +[BenchmarkScope(ScopeLifetime.ScopePerOperation)] +public class ShortCircuitScopePerOperationBenchmarks +{ + [Params(ShortCircuitOutcome.AuthorizationReject, ShortCircuitOutcome.CacheHit, ShortCircuitOutcome.FullExecution)] + public ShortCircuitOutcome Outcome { get; set; } + + [Params(0, 4096)] + public int BufferBytes { get; set; } + + private ScopePerOperationHost _task = null!; + private ScopePerOperationHost _value = null!; + + private ShortCircuitTaskRequest _taskRequest = null!; + private ShortCircuitValueRequest _valueRequest = null!; + + [GlobalSetup] + public void Setup() + { + _task = new(ShortCircuitServices.Build(BufferBytes, valueTask: false)); + _value = new(ShortCircuitServices.Build(BufferBytes, valueTask: true)); + + _taskRequest = new(Outcome); + _valueRequest = new(Outcome); + + TaskPath().GetAwaiter().GetResult(); + ValueTaskPath().GetAwaiter().GetResult(); + } + + [Benchmark(Baseline = true)] + public async Task TaskPath() + { + using IServiceScope scope = _task.Provider.CreateScope(); + var dispatcher = scope.ServiceProvider.GetRequiredService(); + + return await dispatcher.SendAsync(_taskRequest, CancellationToken.None).ConfigureAwait(false); + } + + [Benchmark] + public async Task ValueTaskPath() + { + using IServiceScope scope = _value.Provider.CreateScope(); + var dispatcher = scope.ServiceProvider.GetRequiredService(); + + return await dispatcher.SendAsync(_valueRequest, CancellationToken.None).ConfigureAwait(false); + } + + [GlobalCleanup] + public void Cleanup() + { + _task.Dispose(); + _value.Dispose(); + } +} diff --git a/benchmarks/RequestFlow.Benchmarks/Verification/SuiteProbe.cs b/benchmarks/RequestFlow.Benchmarks/Verification/SuiteProbe.cs new file mode 100644 index 0000000..b49dab1 --- /dev/null +++ b/benchmarks/RequestFlow.Benchmarks/Verification/SuiteProbe.cs @@ -0,0 +1,29 @@ +using RequestFlow.Benchmarks.Core; +using RequestFlow.Benchmarks.Harness; +using RequestFlow.Benchmarks.RequestFlowFixtures.ShortCircuit; + +namespace RequestFlow.Benchmarks; + +public static class SuiteProbe +{ + public static BenchmarkProbeResult Run() + { + ShortCircuitServices.VerifyRequestFlow(valueTask: false); + ShortCircuitServices.VerifyRequestFlow(valueTask: true); + + var expectations = new Dictionary, BenchmarkExpectation>> + { + [typeof(ChainLevelBenchmarks)] = parameters => BenchmarkExpectation.Request( + stages: (int)parameters[nameof(ChainLevelBenchmarks.StageCount)]!) with + { + VoidMethods = [nameof(ChainLevelBenchmarks.TaskVoidChain), nameof(ChainLevelBenchmarks.ValueVoidChain)], + }, + [typeof(VoidCompletionBenchmarks)] = parameters => BenchmarkExpectation.VoidRequest( + stages: (bool)parameters[nameof(VoidCompletionBenchmarks.WithPipeline)]! ? 1 : 0), + [typeof(ShortCircuitRetainedScopeBenchmarks)] = ShortCircuitWorkload.Expectation, + [typeof(ShortCircuitScopePerOperationBenchmarks)] = ShortCircuitWorkload.Expectation, + }; + + return BenchmarkVerification.Run(typeof(Program).Assembly, expectations); + } +} diff --git a/benchmarks/RequestFlow.Benchmarks/VoidCompletion/VoidCompletionBenchmarks.cs b/benchmarks/RequestFlow.Benchmarks/VoidCompletion/VoidCompletionBenchmarks.cs new file mode 100644 index 0000000..a277b40 --- /dev/null +++ b/benchmarks/RequestFlow.Benchmarks/VoidCompletion/VoidCompletionBenchmarks.cs @@ -0,0 +1,51 @@ +using BenchmarkDotNet.Attributes; +using RequestFlow.Benchmarks.Core; +using RequestFlow.Benchmarks.RequestFlowFixtures; + +namespace RequestFlow.Benchmarks; + +/// +/// Compares synchronous Task and ValueTask void dispatch with matched scoped lifetimes. +/// +[BenchmarkScope(ScopeLifetime.RetainedScope)] +public class VoidCompletionBenchmarks +{ + /// + /// Whether one forwarding stage wraps each handler. + /// + [Params(false, true)] + public bool WithPipeline { get; set; } + + private RetainedScopeHost _task = null!; + private RetainedScopeHost _value = null!; + + private readonly TaskVoidRequest _taskVoidRequest = new(); + private readonly ValueVoidRequest _valueVoidRequest = new(); + + [GlobalSetup] + public void Setup() + { + int stages = WithPipeline ? 1 : 0; + + _task = new(RequestFlowSetup.Build(RequestFlowSetup.TaskVoidRequestServices(stages))); + _value = new(RequestFlowSetup.Build(RequestFlowSetup.ValueVoidRequestServices(stages))); + + TaskPath().GetAwaiter().GetResult(); + ValueTaskPath().GetAwaiter().GetResult(); + } + + [Benchmark(Baseline = true)] + public Task TaskPath() + => _task.Service.SendAsync(_taskVoidRequest, CancellationToken.None); + + [Benchmark] + public ValueTask ValueTaskPath() + => _value.Service.SendAsync(_valueVoidRequest, CancellationToken.None); + + [GlobalCleanup] + public void Cleanup() + { + _task.Dispose(); + _value.Dispose(); + } +} diff --git a/benchmarks/RequestFlow.Benchmarks/packages.lock.json b/benchmarks/RequestFlow.Benchmarks/packages.lock.json new file mode 100644 index 0000000..45ad6ae --- /dev/null +++ b/benchmarks/RequestFlow.Benchmarks/packages.lock.json @@ -0,0 +1,203 @@ +{ + "version": 1, + "dependencies": { + "net10.0": { + "Microsoft.Extensions.DependencyInjection": { + "type": "Direct", + "requested": "[10.0.9, )", + "resolved": "10.0.9", + "contentHash": "NijozhERJDIaJ4k5TSMy1jOi0cSC2HfkvRD/Sl+kGSSKgVbFnF4GxgtMN/MrzHB8D1JxIrD4xSer9Blh9v3axQ==", + "dependencies": { + "Microsoft.Extensions.DependencyInjection.Abstractions": "10.0.9" + } + }, + "BenchmarkDotNet": { + "type": "Transitive", + "resolved": "0.15.8", + "contentHash": "paCfrWxSeHqn3rUZc0spYXVFnHCF0nzRhG0nOLnyTjZYs8spsimBaaNmb3vwqvALKIplbYq/TF393vYiYSnh/Q==", + "dependencies": { + "BenchmarkDotNet.Annotations": "0.15.8", + "CommandLineParser": "2.9.1", + "Gee.External.Capstone": "2.3.0", + "Iced": "1.21.0", + "Microsoft.CodeAnalysis.CSharp": "4.14.0", + "Microsoft.Diagnostics.Runtime": "3.1.512801", + "Microsoft.Diagnostics.Tracing.TraceEvent": "3.1.21", + "Microsoft.DotNet.PlatformAbstractions": "3.1.6", + "Perfolizer": "[0.6.1]", + "System.Management": "9.0.5" + } + }, + "BenchmarkDotNet.Annotations": { + "type": "Transitive", + "resolved": "0.15.8", + "contentHash": "hfucY0ycAsB0SsoaZcaAp9oq5wlWBJcylvEJb9pmvdYUx6PD6S4mDiYnZWjdjAlLhIpe/xtGCwzORfzAzPqvzA==" + }, + "CommandLineParser": { + "type": "Transitive", + "resolved": "2.9.1", + "contentHash": "OE0sl1/sQ37bjVsPKKtwQlWDgqaxWgtme3xZz7JssWUzg5JpMIyHgCTY9MVMxOg48fJ1AgGT3tgdH5m/kQ5xhA==" + }, + "Gee.External.Capstone": { + "type": "Transitive", + "resolved": "2.3.0", + "contentHash": "2ap/rYmjtzCOT8hxrnEW/QeiOt+paD8iRrIcdKX0cxVwWLFa1e+JDBNeECakmccXrSFeBQuu5AV8SNkipFMMMw==" + }, + "Iced": { + "type": "Transitive", + "resolved": "1.21.0", + "contentHash": "dv5+81Q1TBQvVMSOOOmRcjJmvWcX3BZPZsIq31+RLc5cNft0IHAyNlkdb7ZarOWG913PyBoFDsDXoCIlKmLclg==" + }, + "Microsoft.CodeAnalysis.Analyzers": { + "type": "Transitive", + "resolved": "3.11.0", + "contentHash": "v/EW3UE8/lbEYHoC2Qq7AR/DnmvpgdtAMndfQNmpuIMx/Mto8L5JnuCfdBYtgvalQOtfNCnxFejxuRrryvUTsg==" + }, + "Microsoft.CodeAnalysis.Common": { + "type": "Transitive", + "resolved": "4.14.0", + "contentHash": "PC3tuwZYnC+idaPuoC/AZpEdwrtX7qFpmnrfQkgobGIWiYmGi5MCRtl5mx6QrfMGQpK78X2lfIEoZDLg/qnuHg==", + "dependencies": { + "Microsoft.CodeAnalysis.Analyzers": "3.11.0" + } + }, + "Microsoft.CodeAnalysis.CSharp": { + "type": "Transitive", + "resolved": "4.14.0", + "contentHash": "568a6wcTivauIhbeWcCwfWwIn7UV7MeHEBvFB2uzGIpM2OhJ4eM/FZ8KS0yhPoNxnSpjGzz7x7CIjTxhslojQA==", + "dependencies": { + "Microsoft.CodeAnalysis.Analyzers": "3.11.0", + "Microsoft.CodeAnalysis.Common": "[4.14.0]" + } + }, + "Microsoft.Diagnostics.NETCore.Client": { + "type": "Transitive", + "resolved": "0.2.510501", + "contentHash": "juoqJYMDs+lRrrZyOkXXMImJHneCF23cuvO4waFRd2Ds7j+ZuGIPbJm0Y/zz34BdeaGiiwGWraMUlln05W1PCQ==", + "dependencies": { + "Microsoft.Extensions.Logging": "6.0.0" + } + }, + "Microsoft.Diagnostics.Runtime": { + "type": "Transitive", + "resolved": "3.1.512801", + "contentHash": "0lMUDr2oxNZa28D6NH5BuSQEe5T9tZziIkvkD44YkkCGQXPJqvFjLq5ZQq1hYLl3RjQJrY+hR0jFgap+EWPDTw==", + "dependencies": { + "Microsoft.Diagnostics.NETCore.Client": "0.2.410101" + } + }, + "Microsoft.Diagnostics.Tracing.TraceEvent": { + "type": "Transitive", + "resolved": "3.1.21", + "contentHash": "/OrJFKaojSR6TkUKtwh8/qA9XWNtxLrXMqvEb89dBSKCWjaGVTbKMYodIUgF5deCEtmd6GXuRerciXGl5bhZ7Q==", + "dependencies": { + "Microsoft.Diagnostics.NETCore.Client": "0.2.510501", + "System.Reflection.TypeExtensions": "4.7.0" + } + }, + "Microsoft.DotNet.PlatformAbstractions": { + "type": "Transitive", + "resolved": "3.1.6", + "contentHash": "jek4XYaQ/PGUwDKKhwR8K47Uh1189PFzMeLqO83mXrXQVIpARZCcfuDedH50YDTepBkfijCZN5U/vZi++erxtg==" + }, + "Microsoft.Extensions.DependencyInjection.Abstractions": { + "type": "Transitive", + "resolved": "10.0.9", + "contentHash": "g41l/30G3K4B/d/L8kjux0+30e27c8D0FVQ/PFCpbekgfDpj9mnDhieP67EqXWvl1EWNeZh2rpR4F5B/jcDOHA==" + }, + "Microsoft.Extensions.Logging": { + "type": "Transitive", + "resolved": "6.0.0", + "contentHash": "eIbyj40QDg1NDz0HBW0S5f3wrLVnKWnDJ/JtZ+yJDFnDj90VoPuoPmFkeaXrtu+0cKm5GRAwoDf+dBWXK0TUdg==", + "dependencies": { + "Microsoft.Extensions.DependencyInjection": "6.0.0", + "Microsoft.Extensions.DependencyInjection.Abstractions": "6.0.0", + "Microsoft.Extensions.Logging.Abstractions": "6.0.0", + "Microsoft.Extensions.Options": "6.0.0" + } + }, + "Microsoft.Extensions.Logging.Abstractions": { + "type": "Transitive", + "resolved": "6.0.0", + "contentHash": "/HggWBbTwy8TgebGSX5DBZ24ndhzi93sHUBDvP1IxbZD7FDokYzdAr6+vbWGjw2XAfR2EJ1sfKUotpjHnFWPxA==" + }, + "Microsoft.Extensions.Options": { + "type": "Transitive", + "resolved": "6.0.0", + "contentHash": "dzXN0+V1AyjOe2xcJ86Qbo233KHuLEY0njf/P2Kw8SfJU+d45HNS2ctJdnEnrWbM9Ye2eFgaC5Mj9otRMU6IsQ==", + "dependencies": { + "Microsoft.Extensions.DependencyInjection.Abstractions": "6.0.0", + "Microsoft.Extensions.Primitives": "6.0.0" + } + }, + "Microsoft.Extensions.Primitives": { + "type": "Transitive", + "resolved": "6.0.0", + "contentHash": "9+PnzmQFfEFNR9J2aDTfJGGupShHjOuGw4VUv+JB044biSHrnmCIMD+mJHmb2H7YryrfBEXDurxQ47gJZdCKNQ==" + }, + "Perfolizer": { + "type": "Transitive", + "resolved": "0.6.1", + "contentHash": "CR1QmWg4XYBd1Pb7WseP+sDmV8nGPwvmowKynExTqr3OuckIGVMhvmN4LC5PGzfXqDlR295+hz/T7syA1CxEqA==", + "dependencies": { + "Pragmastat": "3.2.4" + } + }, + "Pragmastat": { + "type": "Transitive", + "resolved": "3.2.4", + "contentHash": "I5qFifWw/gaTQT52MhzjZpkm/JPlfjSeO/DTZJjO7+hTKI+0aGRgOgZ3NN6D96dDuuqbIAZSeA5RimtHjqrA2A==" + }, + "System.CodeDom": { + "type": "Transitive", + "resolved": "9.0.5", + "contentHash": "cuzLM2MWutf9ZBEMPYYfd0DXwYdvntp7VCT6a/wvbKCa2ZuvGmW74xi+YBa2mrfEieAXqM4TNKlMmSnfAfpUoQ==" + }, + "System.Management": { + "type": "Transitive", + "resolved": "9.0.5", + "contentHash": "n6o9PZm9p25+zAzC3/48K0oHnaPKTInRrxqFq1fi/5TPbMLjuoCm/h//mS3cUmSy+9AO1Z+qsC/Ilt/ZFatv5Q==", + "dependencies": { + "System.CodeDom": "9.0.5" + } + }, + "System.Reflection.TypeExtensions": { + "type": "Transitive", + "resolved": "4.7.0", + "contentHash": "VybpaOQQhqE6siHppMktjfGBw1GCwvCqiufqmP8F1nj7fTUNtW35LOEt3UZTEsECfo+ELAl/9o9nJx3U91i7vA==" + }, + "requestflow": { + "type": "Project", + "dependencies": { + "Microsoft.Extensions.DependencyInjection.Abstractions": "[8.0.1, )", + "RequestFlow.Abstractions": "[1.0.0-preview.9, )" + } + }, + "requestflow.abstractions": { + "type": "Project" + }, + "requestflow.benchmarks.core": { + "type": "Project", + "dependencies": { + "BenchmarkDotNet": "[0.15.8, )", + "Microsoft.Extensions.DependencyInjection": "[10.0.9, )", + "RequestFlow.Benchmarks.Harness": "[1.0.0, )" + } + }, + "requestflow.benchmarks.harness": { + "type": "Project", + "dependencies": { + "BenchmarkDotNet": "[0.15.8, )" + } + }, + "requestflow.benchmarks.requestflowfixtures": { + "type": "Project", + "dependencies": { + "Microsoft.Extensions.DependencyInjection": "[10.0.9, )", + "RequestFlow": "[1.0.0-preview.9, )", + "RequestFlow.Benchmarks.Core": "[1.0.0, )" + } + } + } + } +} \ No newline at end of file diff --git a/docs/events.md b/docs/events.md index 5a3c486..430c769 100644 --- a/docs/events.md +++ b/docs/events.md @@ -328,14 +328,14 @@ Events bypass request and stream stages. A `PublishAsync` call goes straight thr ## Notes for MediatR migrations -The behavior warnings below come from black-box probes against MediatR 12.5.0 and 14.2.0 on Microsoft.Extensions.DependencyInjection. They describe those versions and that container, not every MediatR release. +The behavior warnings below come from black-box probes against MediatR 12.5.0 on Microsoft.Extensions.DependencyInjection. They describe that version and that container, not every MediatR release. - Delivery to base-class, interface, and catch-all notification handlers was gated on the published type also having an exact-type handler. With one, all of them ran; with none, the publish reached no handler at all. RequestFlow's closure is unconditional: every applicable handler runs whether or not an exact-type handler exists. - A notification that reached nobody in the probes is delivered under RequestFlow when a base, interface, or `IEvent` handler applies, and becomes an `RF0114` startup problem when none does. - Do not mistake RequestFlow's same-tier order for registration order or a compatibility guarantee. Review handlers whose side effects depend on the order of exact-type registrations. - RequestFlow runs every applicable handler unless a publisher cancellation check stops the walk, and aggregates handler failures. Code that catches a handler exception directly around `Publish` needs to inspect `EventPublishException.Failures` instead. - Where MediatR 12.5.0 permitted a notification without handlers, RequestFlow needs a missing-handler exemption for that event. -- Repeated `AddMediatR` registration delivered notification handlers more than once in the 12.5.0 probes; MediatR 14.1.0 stopped that duplicate delivery. RequestFlow deduplicates scanned assemblies, and manual `AddEventHandler` entries, across additive registration calls. +- Repeated `AddMediatR` registration delivered notification handlers more than once in the 12.5.0 probes. RequestFlow deduplicates scanned assemblies, and manual `AddEventHandler` entries, across additive registration calls. - Conditional registration ports directly: `if (flag) services.AddTransient, H>()` becomes `if (flag) options.AddEventHandler()` inside the configure delegate. Leave the bare container registration behind, because under RequestFlow it does nothing. - A `TypeEvaluator` scan filter maps to `ExcludeEventHandler()` per handler kept out, and an open generic notification handler ports as manually added closings, `options.AddEventHandler>()` per event. - Map a MediatR notification publisher to `IEventPublishStrategy` ([the port recipe](#port-a-mediatr-notification-publisher)). Its callback throws on failure, while `EventDelivery.StartAsync` returns a failure. Use `RunAsync` or collect and call `ThrowIfAny`; discarding `StartAsync` results drops failures. diff --git a/src/RequestFlow/RequestFlow.csproj b/src/RequestFlow/RequestFlow.csproj index f96e77c..4259b2b 100644 --- a/src/RequestFlow/RequestFlow.csproj +++ b/src/RequestFlow/RequestFlow.csproj @@ -13,6 +13,7 @@ +