Skip to content

test(pulsar): speed up integration tests and reduce image footprint - #112

Merged
matthyx merged 1 commit into
kubescape:mainfrom
ANAMASGARD:perf/111-pulsar-integration-tests
Sep 9, 2026
Merged

matthyx merged 1 commit into
kubescape:mainfrom
ANAMASGARD:perf/111-pulsar-integration-tests

Conversation

@ANAMASGARD

Copy link
Copy Markdown
Member

Overview

Closes #111.

This PR separates broker-independent unit tests from tests that require a real Pulsar broker. Unit tests now use focused mocks and run
without Docker, while real broker behavior is tested through Testcontainers under the integration build tag.

Changes

  • Add a shared Testcontainers-based Pulsar test broker.
  • Upgrade the test image from apachepulsar/pulsar:2.11.0 to apachepulsar/pulsar:4.0.13.
  • Disable the Pulsar functions worker and limit JVM memory for test containers.
  • Use dynamically mapped broker and admin ports.
  • Add explicit admin API and broker-port readiness checks.
  • Add bounded container logs on startup failure and idempotent cleanup.
  • Move broker-independent producer and consumer coverage to focused mock unit tests.
  • Keep publish/subscribe, redelivery, DLQ, retry, and admin API behavior as integration tests.
  • Migrate the exported PulsarTestSuite to the shared Testcontainers implementation.
  • Preserve AppPortStart and AdminPortStart, updating them with the resolved host ports during setup.
  • Remove duplicated Docker shell scripts and obsolete port-process cleanup code.
  • Update CI to Go 1.24.1, matching go.mod.
  • Run Docker-free unit/race/coverage checks separately from a single integration-test job.
  • Document the unit, integration, Docker, and rootless Podman workflows.

Result

  • go test ./... no longer requires a Docker daemon.
  • PR CI needs to start and pull Pulsar only for the integration-test job.
  • The replacement Pulsar image is approximately 464 MB unpacked locally, compared with approximately 1.5 GB for the previous image.
  • Dynamic port mapping avoids collisions between concurrent test environments.

Compatibility note

PulsarTestSuite.AppPortStart and AdminPortStart remain publicly available for source compatibility. Testcontainers now selects
collision-free ports and replaces these fields with the resolved host ports during SetupSuite; values assigned before setup no longer
control port selection.

Verification

DOCKER_HOST=unix:///definitely-does-not-exist \
  go test ./... -count=1

go test -race ./... -count=1
go build ./...
go vet ./...

go test -tags=integration -p=1 -timeout=5m \
  ./pulsar/connector ./pulsar/test -count=1

go test -tags=integration -p=1 -timeout=15m \
  ./pulsar/connector ./pulsar/test -count=3

go vet -tags=integration ./pulsar/connector ./pulsar/test

Signed-off-by: Gaurav Chaudhary <chaudharygaurav2004@gmail.com>
@coderabbitai

coderabbitai Bot commented Sep 9, 2026

Copy link
Copy Markdown

Important

  • 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Advanced

Run ID: 7bb52369-db72-4400-b2bd-055aa6e7578d


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@matthyx matthyx left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the Testcontainers migration, unit/integration split, cleanup paths, exported suite compatibility, and CI wiring. No blocking findings. Verified locally at 0b92789 with go test ./..., go test -race ./..., go vet ./..., go mod tidy -diff, and the tagged Pulsar integration suites.

@matthyx
matthyx merged commit 5f9ba14 into kubescape:main Sep 9, 2026
8 checks passed
@matthyx matthyx moved this to To Archive in KS PRs tracking Sep 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Archived in project

Development

Successfully merging this pull request may close these issues.

Improve integration test speed and reduce Docker image footprint

2 participants