How to configure NODE_PATH for the Electron instance?

[trusktr]

When karma-electron is symlinked into a project, the upward search path for node_modules folders may not include the application where we are running tests, therefore the tests will not be able to require dependencies because they won't be found.

I tried running karma with NODE_PATH=pwd/node_modules:$NODE_PATH karma ... in case this scenario happens (it isn't needed if karma-electron is under the project node_modules).

But this NODE_PATH does not propagate into the Electron instance.

Do you know how to set NODE_PATH (or something similar) for the Electron instance?

[trusktr]

Seems like this is the same issue (but not solution): electron/electron#11 EDIT: well even if that require('module').globalPaths trick works, I don't want to have to write that in every test file.

[twolfson]

To paraphrase your issue: You'd like to load a file for each test file execution, is that right? (e.g. auto-insert require/similar at the start of each test file)

If that's the case, then we need to reframe your understanding a bit =/

Karma is a browser test runner, not a Node.js one. We need to look at everything through the lens of "what if this was executing in Chrome or Firefox"? (where we don't have sweet features like NODE_PATH)

Once we reframe it, then we realize we need to find support for this either in karma itself or your chosen test runner (e.g. mocha, jasmine)

karma easily supports this feature (I use it all the time for loading CSS files for visual testing/debugging)

Documentation:

• included boolean in files config, https://karma-runner.github.io/4.0/config/files.html

Example:

• Including a CSS file in all test cases https://github.com/twolfson/multi-image-mergetool/blob/1.32.1/karma.conf.js#L17
• Note: JS will be loaded via <script but will behave identical to test files (i.e. share same global context)

If you'd like something even more custom, karma also supports custom HTML pages where scripts can be loaded as you wish:

Documentation:

• customContextFile and customDebugFile in https://karma-runner.github.io/4.0/config/configuration-file.html

Example:

• https://github.com/twolfson/karma-electron/blob/6.3.0/test/integration-test/karma.conf.js#L83-L86
• https://github.com/twolfson/karma-electron/blob/6.3.0/test/integration-test/test-files/custom-context.html

[trusktr]

To paraphrase your issue: You'd like to load a file for each test file execution, is that right? (e.g. auto-insert require/similar at the start of each test file)

That's not exactly my issue (but that is one way I thought I could possibly solve my issue).

The issue is that when a meta-package of mine (which contains karma and karma-electron, and this meta package has other things like TypeScript build steps, Webpack bundling steps, etc) is not installed normally in my project, but symlinked, then Node which is running in karma-electron's Electron instance will no longer find my project's node_modules.

To make it more clear, if I run console.log(module) in a test file, I see this:

Module {
...
  paths: [
    '/home/trusktr/src/lume+umbrella/packages/builder-js-package/node_modules/electron/dist/resources/default_app.asar/node_modules', 
    '/home/trusktr/src/lume+umbrella/packages/builder-js-package/node_modules/electron/dist/resources/node_modules'
  ],
...
}

But the thing is, my project is actually located at

/home/trusktr/src/lume+umbrella/packages/my-project

So Node will not find /home/trusktr/src/lume+umbrella/packages/my-project/node_modules where all my project dependencies are.

This is happening, because I tried to switch to Lerna for multi-project management, which symlinks projects together. Now karma and karme-electron are not directly inside of /home/trusktr/src/lume+umbrella/packages/my-project/node_modules like they are when nothing is symlinked.

/home/trusktr/src/lume+umbrella/packages/my-project/node_modules/builder-js-package is symlinked to /home/trusktr/src/lume+umbrella/packages/builder-js-package, where builder-js-package is my "meta package" that contains all my build scripts, test tools, etc.

Karma is a browser test runner, not a Node.js one

This I know, which is exactly why I'm using karma+karma-electron (because with Jest I can only get fake browser APIs which have bugs or missing features).

As for the rest of your previous comment, I think we can ignore it because it was misunderstanding.

I just want require calls in karma-electron's Electron instance to find my modules. (And this problem only happens when karma-electron is not located under my project node_modules like normal because it in another symlinked location)

EDIT: Or, wait, maybe I misunderstood: Are you saying, that if I add the files to karma includes that then this allows them to be required in Electron (effectively adding them to module.paths)?

[trusktr]

Alrighty! I think there might be an easy fix for this in karma-electron. I found this line:

karma-electron/lib/node-integration-iframe.mustache.js

Line 36 in 406248d

module.paths = module.paths.concat(__require('module')._nodeModulePaths('{{!karmaBasePath}}'));

Maybe, we can add an option for karma-electron called path, which allows us to specify additional paths that can be pushed onto that array.

Let me see about a pull request.

[trusktr]

In case I can make it more clear, in my project, I install some NPM packages:

npm install mkdirp

Now in my test code, this normally works fine:

const mkdirp = require('mkdirp')

describe('foo', () => {
  it('bar', () => { /* ... use mkdirp ...*/ })
})

But the problem happens only when karma-electron is no longer in my project node_modules (because it is in a symlinked location), and in this case I get an error

Electron 2.0.18 (Node 8.9.3) ERROR
  Uncaught Error: Cannot find module 'mkdirp'
  at module.js:545:5

[trusktr]

Thank you for your responsiveness on this by the way! You are more responsive than most any other project maintainer.

[twolfson]

Ah, I see. Thanks for explaining the issue in more detail =)

A PR won't be necessary, more detailed options coming up in next comment

[twolfson]

Great find! The mustache template you linked to is indeed how we correct electron dropping node_modules paths due to a similar problem

This template is loaded via a karma preprocessor to wrap our files listed in karma. Thus, it has the same browser context as any other JS in our tests =) This means any tricks used there, can be used more broadly

https://github.com/twolfson/karma-electron/blob/6.3.0/lib/karma-electron-preprocessor.js#L23-L91

To reiterate, module.paths was the solution you were looking for, not NODE_PATH, for adjusting node_modules path resolution inside of the JS runtime (e.g. Electron main, Electron renderer)

First off, I recommend giving it a try with a specific test file. To reuse your mkdirp example:

// I prefer to use `concat` to avoid contaminating the original array
module.paths = module.paths.concat(['path/to/custom/node_modules']);

// Normal `mkdirp` execution
const mkdirp = require('mkdirp')

describe('foo', () => {
  it('bar', () => { /* ... use mkdirp ...*/ })
})

If that works fine, then we've got 2 options for you:

• Use Electron's preload feature, https://github.com/twolfson/karma-electron/tree/6.3.0#using-preload-with-browserwindow
  • preload file should be as simple as contents module.paths = module.paths.concat(['path/to/custom/node_modules']);
  • I'm iffy if this will propagate properly to child windows/iframes (how karma launches fresh contexts for test runs)
    • In fact, it might not since I'd prob have avoided the mustache template in that case
• Use previously described options in How to configure NODE_PATH for the Electron instance? #44 (comment)
  • File should be loaded at start of files and use included: true flag
  • Same contents: module.paths = module.paths.concat(['path/to/custom/node_modules']);
  • If this doesn't work (pretty sure if should, otherwise CSS wouldn't be in same context as other test files), then there's probably a karma plugin which allows for appending content to each test file

[trusktr]

Aha!! So, I couldn't get the above module.paths trick to work, but I got the test to pass by doing the following

const path = require('path')
process.env.NODE_PATH = path.resolve(process.cwd(), 'node_modules') + ':' + process.env.NODE_PATH
require('module').Module._initPaths()

const mkdirp = require('mkdirp')

describe('TODO, tests', () => {
	it('needs to be done', () => {
		expect(mkdirp).toBeInstanceOf(Function)
	})
})

thanks to this hint on StackOverflow: https://stackoverflow.com/a/33976627/454780.

[twolfson]

Great! Glad to hear it all worked out =)

[trusktr]

Well so far that sample works, but I still want to avoid writing that inside every test file. Let me try your files/included suggestion.

Would that trick with NODE_PATH be worth putting in the mustache template to ensure that path.resolve(process.cwd(), 'node_modules') is always available, even when symlinked?

[twolfson]

The current code is meant to match normal Node.js resolution which traverses upwards based off the current filepath https://nodejs.org/api/modules.html I don't believe using cwd in that resolution is a common setup so I'd prefer to leave it as a one off setup If I'm mistaken in how your symlink is setup and it should match normal Node.js behavior, please correct me

…

On Sat, Mar 7, 2020, 8:08 PM Joe Pea ***@***.***> wrote: Well so far that sample works, but I still want to avoid writing that inside every test file. Let me try your files/included suggestion. Would that trick with NODE_PATH be worth putting in the mustache template to ensure that path.resolve(process.cwd(), 'node_modules') is always available, even when symlinked? — You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub <#44?email_source=notifications&email_token=AAG4KWAVY5VTUBRZP5K36XDRGMK3DA5CNFSM4LDR7SFKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEOEL6UI#issuecomment-596164433>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAG4KWAFTUIOOP4HAKNV6RTRGMK3DANCNFSM4LDR7SFA> .