How to use Meson-installed repository with Jupyter notebook/lab?

[adam2392]

Hi,

I'm building a project locally that consists of Python and Cython. Everything works fine when running things like:

./spin build
./spin test -- ./project/test_some_file.py

These are all installed in my <sktree> conda environment. I then open up Jupyter notebook and I have a kernel installed wrt <sktree> conda environment, but this does not work.

[rgommers]

Hi @adam2392, spin does a local install for developing the project itself within the source tree of the repo (defaulting to build-install). That's not site-packages, so it only works when you run things via the spin interface (including spin ipython, but there's no spin jupyter).

You may be better off using pip install -e . --no-build-isolation for everything if you want to use Jupyter. And instead of spin test, use pytest directly.

[adam2392]

Hmmm so that's what I did, but I get some obscure error:

pip install -e ./

# which seems to work...
Obtaining file:///Users/adam2392/Documents/scikit-tree
  Installing build dependencies ... done
  Checking if build backend supports build_editable ... done
  Getting requirements to build editable ... done
  Preparing editable metadata (pyproject.toml) ... done
Building wheels for collected packages: scikit-tree
  Building editable for scikit-tree (pyproject.toml) ... done
  Created wheel for scikit-tree: filename=scikit_tree-0.0.0.dev0-cp39-cp39-macosx_13_0_arm64.whl size=12245 sha256=e84dee8ad53b72c5970e91e7a02b8b022ac53f5f7f39f6d647f9a55647c22451
  Stored in directory: /private/var/folders/6_/sl83qtkd68x3_mvfys07_6qm0000gn/T/pip-ephem-wheel-cache-r4coca6w/wheels/04/0b/eb/5bf45d99f396bab0ff88a00a64fb349013242494ef712c3ddd
Successfully built scikit-tree
Installing collected packages: scikit-tree
Successfully installed scikit-tree-0.0.0.dev0

# then I run Jupyter lab but now get a super obscure error:

---------------------------------------------------------------------------
FileNotFoundError                         Traceback (most recent call last)
/var/folders/6_/sl83qtkd68x3_mvfys07_6qm0000gn/T/ipykernel_11187/3634873820.py in <cell line: 6>()
      4 
      5 from sklearn.neighbors import NearestNeighbors
----> 6 import sktree
      7 # from sktree.ensemble._unsupervised_forest import pairwise_forest_distance
      8 

~/miniforge3/envs/sktree/lib/python3.9/importlib/_bootstrap.py in _find_and_load(name, import_)

~/miniforge3/envs/sktree/lib/python3.9/importlib/_bootstrap.py in _find_and_load_unlocked(name, import_)

~/miniforge3/envs/sktree/lib/python3.9/importlib/_bootstrap.py in _find_spec(name, path, target)

~/miniforge3/envs/sktree/lib/python3.9/site-packages/_scikit_tree_editable_loader.py in find_spec(self, fullname, path, target)
    266                 return None
    267             namespace = False
--> 268             tree = self.rebuild()
    269             parts = fullname.split('.')
    270 

~/miniforge3/envs/sktree/lib/python3.9/site-packages/_scikit_tree_editable_loader.py in rebuild(self)
    307             stdout = subprocess.DEVNULL
    308 
--> 309         subprocess.run(self._build_cmd, cwd=self._build_path, env=env, stdout=stdout, check=True)
    310 
    311         install_plan_path = os.path.join(self._build_path, 'meson-info', 'intro-install_plan.json')

~/miniforge3/envs/sktree/lib/python3.9/subprocess.py in run(input, capture_output, timeout, check, *popenargs, **kwargs)
    503         kwargs['stderr'] = PIPE
    504 
--> 505     with Popen(*popenargs, **kwargs) as process:
    506         try:
    507             stdout, stderr = process.communicate(input, timeout=timeout)

~/miniforge3/envs/sktree/lib/python3.9/subprocess.py in __init__(self, args, bufsize, executable, stdin, stdout, stderr, preexec_fn, close_fds, shell, cwd, env, universal_newlines, startupinfo, creationflags, restore_signals, start_new_session, pass_fds, user, group, extra_groups, encoding, errors, text, umask)
    949                             encoding=encoding, errors=errors)
    950 
--> 951             self._execute_child(args, executable, preexec_fn, close_fds,
    952                                 pass_fds, cwd, env,
    953                                 startupinfo, creationflags, shell,

~/miniforge3/envs/sktree/lib/python3.9/subprocess.py in _execute_child(self, args, executable, preexec_fn, close_fds, pass_fds, cwd, env, startupinfo, creationflags, shell, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, restore_signals, gid, gids, uid, umask, start_new_session)
   1819                     if errno_num != 0:
   1820                         err_msg = os.strerror(errno_num)
-> 1821                     raise child_exception_type(errno_num, err_msg, err_filename)
   1822                 raise child_exception_type(err_msg)
   1823 

FileNotFoundError: [Errno 2] No such file or directory: '/private/var/folders/6_/sl83qtkd68x3_mvfys07_6qm0000gn/T/pip-build-env-zmsxguxu/overlay/bin/ninja'

If I do this would it work?

./spin shell

>>> (in new shell): jupyter lab

[rgommers]

Using the --no-build-isolation flag is essential - see https://meson-python.readthedocs.io/en/latest/how-to-guides/editable-installs.html for why. That pip doesn't do that by default is an unfortunate UX issue I'm afraid.