Python version mismatch with Google Colab notebook

[RomainAbbelight]

@LREIN663 Google Colab now ships by default with Python 3.12+ which is incompatible with the requirements of the package currently.
So the installation fails:
ERROR: Package 'py-comet' requires a different Python: 3.12.12 not in '<3.12,>=3.8'

Have you ever tried to run Comet on Python 3.12+?

[LREIN663]

Yes indeed and I had problems. The main reason why COMET was pinned to py<3.12 were it's dependencies, most importantly numpy. Numpy had a big update with V2.0 and since most other packages depend on numpy this meant changing most of them.
Solving this reliably was a little bit more tricky, however I believe a found a reasonable solution.
Concerning the main repo:
Based upon my findings on the other packages (scipy, numba, ...) I set up a second set of dependencies to support newer python version (up to the lastest 3.14), but kept well tested old dependencies for older python versions in a first set. Pip will check the python version installed in the current environment and choose the corresponding set.
I tested this setup successfully in virtual conda environments using python 3.9, 3.10, 3.12, 3.13 and 3.14.
With this I pinned COMET to maximally python 3.14 ( newest version currently ) since a newer python version will likely break this again.
e9a9e3a
Concerning the google colab:
Since a full installation is for the COLAB notebook hosted on google is overkill I propose to for now stick to the old approach of pinning specifically the numba version (0.4 -> 0.61) that is compatible with the current Google colab setup and only install that (using the --no-deps flag). This however requires me to keep an eye out for major google colab updates. cd3d213

Please let me know if that solves the issue for you as well or if you see potential for further improvements.
Best,
Lenny

[RomainAbbelight]

I have just tested the Google Colab version (currently the most portable version for my cvolleagues) and it all seems to be running fine. I will let you know if I have further issues when I run it on various samples. Thanks a lot!

At some point, you might have to take the leap and stick to newer versions only of the packages, rather than supporting older versions. Are you mostly waiting to see if it passes the test of time?
Because you have a set of test, if you're happy with your tests and they pass with the new package+Python versions, you might as well only support that from now on. But it's obviously your call!

[superchromix]

I have just tested the Google Colab version (currently the most portable version for my cvolleagues) and it all seems to be running fine. I will let you know if I have further issues when I run it on various samples. Thanks a lot!

At some point, you might have to take the leap and stick to newer versions only of the packages, rather than supporting older versions. Are you mostly waiting to see if it passes the test of time? Because you have a set of test, if you're happy with your tests and they pass with the new package+Python versions, you might as well only support that from now on. But it's obviously your call!

We like to support old versions in our code ;) -- as far as I'm concerned Comet should support older versions than it currently does. Python's way of "flying forward while breaking everything" is not everyone's cup of tea. It makes it impossible to have stable code.

[superchromix]

Yes indeed and I had problems. The main reason why COMET was pinned to py<3.12 were it's dependencies, most importantly numpy. Numpy had a big update with V2.0 and since most other packages depend on numpy this meant changing most of them. Solving this reliably was a little bit more tricky, however I believe a found a reasonable solution. Concerning the main repo: Based upon my findings on the other packages (scipy, numba, ...) I set up a second set of dependencies to support newer python version (up to the lastest 3.14), but kept well tested old dependencies for older python versions in a first set. Pip will check the python version installed in the current environment and choose the corresponding set. I tested this setup successfully in virtual conda environments using python 3.9, 3.10, 3.12, 3.13 and 3.14. With this I pinned COMET to maximally python 3.14 ( newest version currently ) since a newer python version will likely break this again. e9a9e3a Concerning the google colab: Since a full installation is for the COLAB notebook hosted on google is overkill I propose to for now stick to the old approach of pinning specifically the numba version (0.4 -> 0.61) that is compatible with the current Google colab setup and only install that (using the --no-deps flag). This however requires me to keep an eye out for major google colab updates. cd3d213

Please let me know if that solves the issue for you as well or if you see potential for further improvements. Best, Lenny

Lenny, bear in mind that we need to maintain compatibility with early Numba versions. So, please don't make any changes which would result in raising the minimum Python version needed to run Comet...

[superchromix]

Guys, I would suggest that the best solution to this issue is described here:

https://developers.googleblog.com/en/google-colab-adds-more-back-to-school-improvements/

and here:

https://github.com/googlecolab/backend-info?tab=readme-ov-file#colabs-fallback-runtime-version

Look at the section on Colab runtime versions. A notebook can be pinned to a specific version, eg:

2025.07

Ubuntu 22.04.04 LTS
Python 3.11.13
numpy 2.0.2
PyTorch 2.6.0
Jax 0.5.2
Tensorflow 2.18.0 (not included in TPU runtimes)
R version 4.5.1 (2025-06-13) -- "Great Square Root"
julia version 1.10.9

..... This might make our lives simpler going forward?

[RomainAbbelight]

I have just tested the Google Colab version (currently the most portable version for my cvolleagues) and it all seems to be running fine. I will let you know if I have further issues when I run it on various samples. Thanks a lot!
At some point, you might have to take the leap and stick to newer versions only of the packages, rather than supporting older versions. Are you mostly waiting to see if it passes the test of time? Because you have a set of test, if you're happy with your tests and they pass with the new package+Python versions, you might as well only support that from now on. But it's obviously your call!

We like to support old versions in our code ;) -- as far as I'm concerned Comet should support older versions than it currently does. Python's way of "flying forward while breaking everything" is not everyone's cup of tea. It makes it impossible to have stable code.

Oh yes, totally @superchromix , COMET does not have to follow package updates. It's only if you want to provide the tool as a Google Colab solution, you probably need a solution: I really like the links you posted. Developping, testing and pinning to a specific runtime seems like an excellent idea!

[LREIN663]

Guys, I would suggest that the best solution to this issue is described here:

https://developers.googleblog.com/en/google-colab-adds-more-back-to-school-improvements/

and here:

https://github.com/googlecolab/backend-info?tab=readme-ov-file#colabs-fallback-runtime-version

Look at the section on Colab runtime versions. A notebook can be pinned to a specific version, eg:

2025.07

Ubuntu 22.04.04 LTS
Python 3.11.13
numpy 2.0.2
PyTorch 2.6.0
Jax 0.5.2
Tensorflow 2.18.0 (not included in TPU runtimes)
R version 4.5.1 (2025-06-13) -- "Great Square Root"
julia version 1.10.9

..... This might make our lives simpler going forward?

Yes that indeed makes live much simpler. I will then proceed and pin the colab notebook version - if someone ever wants to modify the notebook on their side and use functions from a newer version I'm sure they'd know how to 'unpin' the notebook and update the packages. I could also add a small paragraph in the docs concerning this.

[RomainAbbelight]

That's a great plan. I am happy to beta-test for you there if you need. Let me know.