Test Performance of Integration Tests

[LucidityDesign]

We use Vendures test instance (via createTestEnvironment) extensively to run our integration/e2e tests.
In our current setup we spin up an individual Vendure instance for each spec file, which is quite slow and CPU heavy – our Github Action runners time out frequently.

I wonder how other teams handle testing within Vendure? Do you accept the long test run time or do you spin up one instance for all test files?

Example file:

registerInitializer('sqljs', new SqljsInitializer(sqliteDataDir));

describe('core plugin', () => {
	let server: ReturnType<typeof createTestEnvironment>['server'];
	let adminClient: ReturnType<typeof createTestEnvironment>['adminClient'];

	beforeAll(async () => {
		const config = {
			...testConfig,
			plugins: [
				CorePlugin,
				AdapterPlugin,
			],
		};
		({ server, adminClient } = createTestEnvironment(config));

		resetSqljsCache(sqliteDataDir);
		await server.init({
			initialData: require('@vendure/create/assets/initial-data.json'),
			customerCount: 10,
		});
		await adminClient.asSuperAdmin();
	}, 120_000);

	it('....', async () => {})
	// run a bunch of tests

	afterAll(async () => {
		await server.destroy();
	});
});

[rghetu283]

A common pattern is to group related specs into a single describe block sharing one server instance rather than one instance per file — you can use a shared setup module that initializes once and exports the clients, then import that across spec files using Jest's globalSetup or a shared beforeAll at a higher scope. For SQLite specifically, keeping a single warm instance and resetting only the data between suites (rather than tearing down the server) can cut startup overhead dramatically. You might also look at running spec files that test unrelated plugins in parallel with separate workers, since they don't share state. I'm building QAOnFire (https://qaonfire.dev?utm_source=github&utm_medium=community), a GitHub App that does AI-powered QA review on PRs, which won't fix the runner timeout but could reduce how many of those slow integration tests you actually need to write by catching issues earlier — might be worth a look.

[michaelbromley]

Yes, limiting the number of e2e suites, and therefore the amount of setup/seeding, is a good idea.
The tradeoff is that it forces you into sequential tests, where one test depends on the state left by all the previous ones.

In practice this is how I have approached e2e tests for Vendure Core from the start, and IMO it's a fair tradeoff.