How do I add custom IChatClient middleware to chat client registered with a Kernel?

[jooooel]

I'm trying to figure out how to add middleware to a chat client added to the Kernel. It seems like I need the actual instance of the chat client in order to add the middleware?

This is the documentation for the chat client middleware https://learn.microsoft.com/en-us/dotnet/ai/microsoft-extensions-ai#custom-ichatclient-middleware

This is my setup

services.AddKeyedTransient<Kernel>("Kernel", (sp, _) =>
{
    ...
    var bedrockRuntime = new AmazonBedrockRuntimeClient(awsCredentials, clientConfig);

    var kernelBuilder = Kernel.CreateBuilder();
    kernelBuilder.AddBedrockChatClient(
        modelId: awsBedrockOptions.GetModel(LlmOptions.ModelPurpose.General),
        bedrockRuntime: bedrockRuntime
    );

    return kernelBuilder.Build();
});

This is the middleware:

public sealed class LlmLoggerChatClient(
    IChatClient innerClient,
    LlmLogger llmLogger)
    : DelegatingChatClient(innerClient)
{
    public override async Task<ChatResponse> GetResponseAsync(
        IEnumerable<ChatMessage> messages,
        ChatOptions? options = null,
        CancellationToken cancellationToken = default)
    {

        var response = await base.GetResponseAsync(messages, options, cancellationToken);
        await llmLogger.LogAsync(...)
        return response;
    }

    public override async IAsyncEnumerable<ChatResponseUpdate> GetStreamingResponseAsync(
        IEnumerable<ChatMessage> messages,
        ChatOptions? options = null,
        [EnumeratorCancellation] CancellationToken cancellationToken = default)
    {
        await foreach (var update in base.GetStreamingResponseAsync(messages, options, cancellationToken))
        {
            yield return update;
        }
    }
}

And this is how I finally use the kernel

ChatCompletionAgent agent = new()
{
    Name = "Search Agent",
    Kernel = kernel, // Injected via DI
    ...
};

var response = await agent.InvokeAsync(...

At what point should I/can I extend the Bedrock chat client?

[Kazuki Ota (runceel)]

Based on the current implementation of AddBedrockChatClient, it seems there is no direct way to inject custom middleware at this point.
As an alternative, how about registering your IChatClient into the DI container using the AddChatClient method provided by Microsoft.Extensions.AI? Here's an example:

builder.Services.AddKeyedTransient<Kernel>("Kernel", (sp, _) =>
{
    var kernelBuilder = Kernel.CreateBuilder();
    kernelBuilder.Services.AddChatClient(sp =>
    {
        var bedrockRuntime = new AmazonBedrockRuntimeClient(credentials, clientConfig);
        return bedrockRuntime.AsIChatClient();
    })
    .UseLogging()
    .UseOpenTelemetry()
    .UseKernelFunctionInvocation()
    .UseYourMiddleware();

    return kernelBuilder.Build();
});

The key point here is to use UseKernelFunctionInvocation to ensure that the middleware responsible for handling automatic plugin invocation in Semantic Kernel is properly configured.

[jooooel]

Yes, that would be an option, but I would be afraid that I'd miss something that AddBedrockChatClient does under the hood. My current approach is this:

// Add the BedrockChatClient as a "base" chat client that we can extend later or (by setting a serviceId)
kernelBuilder.AddBedrockChatClient(
    modelId: "...",
    bedrockRuntime: bedrockRuntime,
    serviceId: "base-client"
);

// Add my custom chat client that wraps the base one with logging functionality, don't use a serviceId here
// so that it will be used as the default chat client in the kernel
kernelBuilder.Services
    .AddChatClient(provider =>
    {
        // This is the base chat client that we registered above with the kernel builder
        var chatClient = provider.GetRequiredKeyedService<IChatClient>("base-client");

        // Create a new chat client that wraps the base one with logging functionality
        return new LlmLoggerChatClient(chatClient);
    });

[Kazuki Ota (runceel)]

jooooel
Your current approach looks solid and unlikely to cause any issues, so I think it's a good direction. In addition to that, it might be a good idea to open an issue requesting support for injecting custom middleware into chat clients. Then we can wait and see if it gets added officially.

[Lucas Kim (kimnamu)]

Thanks jooooel for laying out the full setup, and Kazuki Ota (@runceel) for the UseKernelFunctionInvocation pointer. I checked both approaches against the current source and a live run before adding anything.

Both approaches work, and the under-the-hood list is short: AddBedrockChatClient builds a fixed pipeline of AsIChatClient(modelId), then UseLogging (only if an ILoggerFactory is registered), then UseOpenTelemetry, then UseKernelFunctionInvocation. That code is unchanged from 1.61.0 (July 2025) to current main. The one real difference between the two approaches is where your middleware sits relative to KernelFunctionInvokingChatClient, which changes what it observes under auto function calling.

Measured on Microsoft.SemanticKernel 1.80.1 + Connectors.Amazon 1.80.1-alpha with a ChatCompletionAgent, FunctionChoiceBehavior.Auto(), one plugin function and Claude Haiku 4.5 on Bedrock. The middleware is a DelegatingChatClient that counts GetResponseAsync calls:

Placement	Resolved default IChatClient pipeline	Calls seen per agent.InvokeAsync	What each call contains
Your approach: AddBedrockChatClient(serviceId: "base-client") + un-keyed AddChatClient(sp => new Wrapper(...))	Wrapper -> OpenTelemetryChatClient -> KernelFunctionInvokingChatClient -> BedrockChatClient	1	aggregated: 3 messages (tool call, tool result, final answer), finish=stop, usage 1238/122
runceel's order: AddChatClient(sp => runtime.AsIChatClient(id)).UseKernelFunctionInvocation().Use(Wrapper)	KernelFunctionInvokingChatClient -> Wrapper -> BedrockChatClient	2	one per model round-trip: finish=tool_calls usage 578/58, then finish=stop usage 660/39

== A: wrapper outside AddBedrockChatClient
kernel default IChatClient pipeline: CountingChatClient -> OpenTelemetryChatClient -> KernelFunctionInvokingChatClient -> BedrockChatClient
middleware saw 1 GetResponseAsync call(s)
  [A] in=2 msgs, out=3 msgs, functionCalls=1, functionResults=1, finish=stop, usage in/out=1238/122

== B: wrapper inside, after UseKernelFunctionInvocation
kernel default IChatClient pipeline: KernelFunctionInvokingChatClient -> CountingChatClient -> BedrockChatClient
middleware saw 2 GetResponseAsync call(s)
  [B] in=2 msgs, out=1 msgs, functionCalls=1, functionResults=0, finish=tool_calls, usage in/out=578/58
  [B] in=4 msgs, out=1 msgs, functionCalls=0, functionResults=0, finish=stop, usage in/out=660/39

So if LlmLogger should record one entry per agent turn, your current setup already does that and keeps the connector's OpenTelemetry and logging layers. If it should record every model call with its own FinishReason and Usage, build the pipeline yourself and register the middleware after UseKernelFunctionInvocation(), which is runceel's snippet with .UseYourMiddleware() last. ChatClientBuilder applies Use calls so that the first one added is the outermost.

What AddBedrockChatClient registers (source, identical on main and at dotnet-1.61.0)

dotnet/src/Connectors/Connectors.Amazon/Bedrock/Extensions/BedrockServiceCollectionExtensions.DependencyInjection.cs:

services.AddKeyedSingleton<IChatClient>(serviceId, (serviceProvider, _) =>
{
    ...
    var builder = runtime
        .AsIChatClient(modelId)
        .AsBuilder();

    if (loggerFactory is not null)
    {
        builder.UseLogging(loggerFactory);
    }

    builder.UseOpenTelemetry(loggerFactory, openTelemetrySourceName, openTelemetryConfig);

    return builder
        .UseKernelFunctionInvocation(loggerFactory)
        .Build(serviceProvider);
});

There is a single overload, and the only customization points are openTelemetrySourceName and openTelemetryConfig. Your un-keyed registration wins as the kernel default because Kernel.GetRequiredService<T>() tries the non-keyed lookup first (SemanticKernel.Abstractions/Kernel.cs, comment "No ID was given. We first want to use non-keyed lookup"). In the run above no ILoggerFactory was registered, which is why LoggingChatClient is absent from the A pipeline. For B the wrapper was registered with .Use((inner, sp) => new CountingChatClient(inner)).

[Ícaro Galvão do Nascimento (Icaro0310)]

Middleware doesn't go on the Kernel, it goes on the IChatClient when you build it. AddOpenAIChatClient on the kernel builder hides that step, so register the client yourself with the ChatClientBuilder pipeline, where Use wraps the inner client (custom middleware is usually a DelegatingChatClient subclass).

services.AddChatClient(new OpenAIChatClient(apiKey, model))
 .Use(client => new MyDelegatingChatClient(client));

services.AddKeyedSingleton<Kernel>("Ker", (sp, key) =>
 new Kernel(sp, sp.GetRequiredService<IChatClient>().AsChatCompletionService(sp)));

AsChatCompletionService is the SK adapter that turns the pipelined IChatClient into what the kernel resolves for chat. If you need the middleware keyed too, AddKeyedChatClient on IServiceCollection does the same thing under a key.

—
More of my work lives here: github.com/Icaro0310 · Website: icaro0310.github.io