Pip install Error - Python 3.12

[awilliamson]

When attempting to run with Python 3.12, there is an error during pip install -e . step. This was within a virtualenv set up using python3.12 -m virtualenv .venv and then activated.

Full Error:

Collecting pufferlib@ git+https://github.com/thatguy11325/PufferLib.git@1.0 (from pokemonred-puffer==0.0.1)
  Cloning https://github.com/thatguy11325/PufferLib.git (to revision 1.0) to /tmp/pip-install-mb5akcfi/pufferlib_a4d4c203ed2f4a8e85370bb4f5b742c2
  Running command git clone --filter=blob:none --quiet https://github.com/thatguy11325/PufferLib.git /tmp/pip-install-mb5akcfi/pufferlib_a4d4c203ed2f4a8e85370bb4f5b742c2
  Resolved https://github.com/thatguy11325/PufferLib.git to commit 47ebd5e4ed49c7d735e3db56a7710802e67cf961
  Installing build dependencies ... done
  Getting requirements to build wheel ... error
  error: subprocess-exited-with-error

  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> [33 lines of output]
      Traceback (most recent call last):
        File "/home/ashley/GitHubProjects/pokemonred_puffer/.venv2/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 389, in <module>
          main()
        File "/home/ashley/GitHubProjects/pokemonred_puffer/.venv2/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 373, in main
          json_out["return_val"] = hook(**hook_input["kwargs"])
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/home/ashley/GitHubProjects/pokemonred_puffer/.venv2/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 137, in get_requires_for_build_wheel
          backend = _build_backend()
                    ^^^^^^^^^^^^^^^^
        File "/home/ashley/GitHubProjects/pokemonred_puffer/.venv2/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 70, in _build_backend
          obj = import_module(mod_path)
                ^^^^^^^^^^^^^^^^^^^^^^^
        File "/usr/lib/python3.12/importlib/__init__.py", line 90, in import_module
          return _bootstrap._gcd_import(name[level:], package, level)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
        File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
        File "<frozen importlib._bootstrap>", line 1310, in _find_and_load_unlocked
        File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
        File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
        File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
        File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked
        File "<frozen importlib._bootstrap>", line 935, in _load_unlocked
        File "<frozen importlib._bootstrap_external>", line 999, in exec_module
        File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
        File "/tmp/pip-build-env-i9achue3/overlay/lib/python3.12/site-packages/setuptools/__init__.py", line 16, in <module>
          import setuptools.version
        File "/tmp/pip-build-env-i9achue3/overlay/lib/python3.12/site-packages/setuptools/version.py", line 1, in <module>
          import pkg_resources
        File "/tmp/pip-build-env-i9achue3/overlay/lib/python3.12/site-packages/pkg_resources/__init__.py", line 2191, in <module>
          register_finder(pkgutil.ImpImporter, find_on_path)
                          ^^^^^^^^^^^^^^^^^^^
      AttributeError: module 'pkgutil' has no attribute 'ImpImporter'. Did you mean: 'zipimporter'?
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.

Partial Error:

file "/tmp/pip-build-env-_o6jctxg/overlay/lib/python3.12/site-packages/pkg_resources/__init__.py", line 2191, in <module>
          register_finder(pkgutil.ImpImporter, find_on_path)
                          ^^^^^^^^^^^^^^^^^^^
      AttributeError: module 'pkgutil' has no attribute 'ImpImporter'. Did you mean: 'zipimporter'?

This will cause the project build to fail.

Resolution I found, was to go to Python 3.11. In Python 3.12, the long-standing deprecated pkgutil.ImpImporter was finally removed. In Python 3.11, it still exists, and the project builds successfully.

Attempted the following, but the pip and setuptools don't update, and still dno't understand pkgutil.ImpImporter.

python3.12 -m ensurepip --upgrade
python3.12 -m pip install --upgrade setuptools
python3.12 -m pip install -e .

Recommendation would be to add Python running versions to the README for that people may know. Also logging this error here as there is no standing GitHub issue which users can search.

[thatguy11325]

Looked into this a but further. The error you saw is due to this repo's dependency on my PufferLib 1.0 fork. I can talk to Joseph (the PufferLib maintainer) to integrate some fixes from my fork into his 2.0 branch which does support Python 3.12. He merged some of the fixes to 1.0 but they never made it to 2.0 for some reason. Once that's done, I can upgrade puffer here and 3.12 should be supported.