Convert testing suite to TestContainers.

[j7nw4r]

Summary

The current test suite fails due to VMware/Broadcom's deprecation of Bitnami. A Kafka source that is not affected by the Bitnami deprecation is required to restore a working test setup.

Motivation

Testing is currently blocked. Restoring a working test setup requires a Kafka source that is not affected by the Bitnami deprecation.

Proposal

Migrate testing to Kafka Testcontainers.

Each test compilation unit (each test file) would run its own Docker Kafka container. Multiple versions are available (Apache vs Confluent, Kraft vs Zookeeper).

The first pass would be an implementation using Apache Kraft Kafka. A later step could add a test macro that enumerates the tests over all of the necessary Kafka distributions.

This change brings in a dependency on local Docker/Podman being available for testing. The current test suite already depends on external Kafka.

https://docs.rs/testcontainers/latest/testcontainers/
https://docs.rs/testcontainers-modules/latest/testcontainers_modules/

@fede1024

[fede1024]

Yes, this would be great! The test suite for this library is overdue for a revamp. In hindsight moving to the bitnami images was not a great idea. No objections on my side, thank you for working on this!

…

On Thu, 2 Oct 2025, 19:42 Johnathan W, ***@***.***> wrote: *j7nw4r* created an issue (fede1024/rust-rdkafka#797) <#797> The current test suite seems to fail due to VMWare/Broadcom's deprecation of Binami (*shakes fist*). To get testing back on track, I suggest that we migrate the testing to use Kafka Testcontainers <https://testcontainers.com/modules/kafka/?language=rust>. Essentially, each test compilation unit (each test file) would run it's own docker Kafka container. There are even multiple versions available (Apache vs Confluent, Kraft vs Zookeeper). The first pass would would be implementation using Apache Kraft Kafka. Eventually I think we could create a test macro would would enumerate the tests over all of the necessary Kafka distributions. This change would bring in a dependency on local Docker/Podman being available for testing, but I don't think that is a big deal given that the current test suite depends on external Kafka. Thoughts? If we're in agreement, I can take this one. https://docs.rs/testcontainers/latest/testcontainers/ https://docs.rs/testcontainers-modules/0.13.0/testcontainers_modules/ @fede1024 <https://github.com/fede1024> — Reply to this email directly, view it on GitHub <#797>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AALY3T3Z3GNGLTYDKC3MZXD3VVWZ3AVCNFSM6AAAAACIEKE7NOVHI2DSMVQWIX3LMV43ASLTON2WKOZTGQ3TQNJTGY4DOMI> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[j7nw4r]

Work can be tracked here: #798

[j7nw4r]

@fede1024 Just for clarity before I continue, this is a what a basic produce test looks like: https://github.com/fede1024/rust-rdkafka/pull/798/files#diff-7f604488cb54e565cbbffe2475cac35b642abcd7bc9b2a2bcd26130c3219dcd5.

This should give an idea of how test containers are used.

[fede1024]

That looks great! Thank you for taking the time to implement tests properly instead of just updating the docker image. Hopefully once the initial setup code is implemented (and it looks like it is), porting more tests won't require too much work.