Skip to content

Mocked test suite (npm run test:mocked) is too slow — speed it up without losing coverage #1452

Description

@YuryShkoda

Problem

npm run test:mocked currently takes ~9 minutes (80 suites / 589 tests, ~513-525s wall time across multiple recent runs) even though it's meant to be the fast, mocked/offline test path (as opposed to test:server, which talks to real servers). A handful of spec files dominate that time.

The worst offender is src/commands/testing/spec/testing.spec.ts, which alone took ~307s in one measured run — roughly 60% of the entire suite's runtime — for only 6 it() blocks. Digging into why:

  • It calls createTestApp() (twice), which (via create()) runs setupNpmProject() whenever no app type is given — this shells out to real npm init --yes and npm i @sasjs/core --save (src/utils/utils.ts). Real npm installs against the real registry are slow and also make the "mocked" suite dependent on network/registry availability.
  • Several other spec files that call createTestApp/createReactApp/createAngularApp/etc. likely pay the same real-npm-install and real-template-download tax (e.g. create.spec.ts's createReactApp test took ~25s by itself in a recent run).

Notably, create.spec.ts already solved this exact problem for its own suite:

jest.spyOn(shelljs, 'exec').mockImplementation((command: string) => {
  if (!command.includes('npm install')) {
    return exec(command, { silent: true })
  }
  return undefined
})

This lets everything except the slow npm install step run for real. That pattern isn't applied anywhere else, so other spec files (like testing.spec.ts) still pay the full real-install cost on every run.

Goal

Make npm run test:mocked meaningfully faster (target: cut total runtime significantly, e.g. by half or more) without reducing what's actually being verified. Speed must never come at the cost of weaker coverage of real logic — if a shortcut risks masking a regression, don't take it.

Suggested approach

  • Audit per-file/per-suite timing (jest --verbose or --json output) to get an authoritative ranking of the slowest spec files, rather than relying on the two data points above.
  • Extend the existing create.spec.ts "skip real npm install" shelljs.exec mocking pattern to every spec file that goes through createTestApp, createReactApp, createAngularApp, createMinimalApp, or createTemplateApp (directly or indirectly), since none of these tests are asserting on npm's actual install behavior.
  • Identify any other real, slow I/O in hot-path spec files (real template downloads from GitHub, redundant compile()/build() cycles re-run per it() instead of once per describe() via beforeAll, etc.) and either mock it or hoist it out of per-test setup where correctness allows.
  • Re-measure after each change to confirm both the speed improvement and that coverage thresholds (jest.config.jscoverageThreshold) still hold or improve — no test should be deleted or weakened purely to save time.
  • Document any newly-established "mock slow I/O, keep everything else real" pattern (e.g. in a shared test helper in src/utils/test.ts) so future spec files adopt it by default instead of reinventing/re-discovering it.

Priority note

Reviewers should treat coverage of actual CLI logic as non-negotiable: any PR addressing this issue should show before/after timing and before/after coverage numbers, and should be rejected if it trades correctness confidence for speed.

Metadata

Metadata

Assignees

Type

No type

Fields

No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions