Upgrade to JupyterHub 5

[tsnow03]

jupyterhub single user is 4.0 but needs to be 5.0 to maintain stability for the 2i2c deployment.

[github-actions]

👈 Test this PR on Binder

[tsnow03]

/condalock

[weiji14]

/condalock

https://github.com/CryoInTheCloud/hub-image/actions/runs/25188625598/job/73852512592

[weiji14]

@tsnow03, might need to revert f1d6150 and 83cfd30, don't think the default GITHUB_TOKEN is working either.

##[group]Run if [[ $(git ls-files --modified) ]]; then
�[36;1mif [[ $(git ls-files --modified) ]]; then�[0m
�[36;1m  git config --global user.name 'actions-bot'�[0m
�[36;1m  git config --global user.email '58130806+actions-bot@users.noreply.github.com'�[0m
�[36;1m  git remote set-url origin https://x-access-token:${GITHUB_TOKEN}@github.com/CryoInTheCloud/hub-image.git�[0m
�[36;1m  git commit --all --message "[condalock-command] autogenerated conda-lock files"�[0m
�[36;1m  git push�[0m
�[36;1mfi�[0m
shell: /usr/bin/bash -e {0}
env:
  MAMBA_ROOT_PREFIX: /home/runner/micromamba
  MAMBA_EXE: /home/runner/micromamba-bin/micromamba
  CONDARC: /home/runner/work/_temp/setup-micromamba/.condarc
  GITHUB_TOKEN: ***
##[endgroup]
[jh5 e777206] [condalock-command] autogenerated conda-lock files
 2 files changed, 711 insertions(+), 550 deletions(-)
remote: Invalid username or token. Password authentication is not supported for Git operations.
fatal: Authentication failed for 'https://github.com/CryoInTheCloud/hub-image.git/'
##[error]Process completed with exit code 128.

We should use the cryointhecloud-bot GitHub App token auth, but problem is that the token expires after 1 hour as mentioned in #157 (comment).

Edit: done in #185

[itcarroll]

Do you have to involve the github-app-token at all in the condalock-command workflow? The create-or-update-comment action should work with built-in GITHUB_TOKEN, as does git push.

[weiji14]

Do you have to involve the github-app-token at all in the condalock-command workflow? The create-or-update-comment action should work with built-in GITHUB_TOKEN, as does git push.

Probably not in this case since the PR is made on a branch under CryoInTheCloud/hub-image rather than a fork. The token should only be needed if we want things to work from forks if I'm not mistaken.

For context, I added the token stuff back in 2022 (#5), when GITHUB_TOKEN didn't quite work with create-or-update-comment I think, even from non-forks. But situation might be a little different now. If we don't mind not being able to run the workflow on forks (e.g. casual user not in the organisation wanting to update some dependency), then probably ok to just use GITHUB_TOKEN.

[tsnow03]

Thanks @weiji14. This is super weird. I think we occasionally have people from outside the org so it would be good to keep it. It doesn't seem like that is the issue we are having though.

[weiji14]

I'm seeing this error in the logs on another run in #189:

Error: [@octokit/auth-app] privateKey option is required
    at createAppAuth (/home/runner/work/_actions/actions/create-github-app-token/1b10c78c7865c340bc4f6099eb2f838309f1e8c3/dist/main.cjs:22902:11)
    at main (/home/runner/work/_actions/actions/create-github-app-token/1b10c78c7865c340bc4f6099eb2f838309f1e8c3/dist/main.cjs:23233:17)
    at run (/home/runner/work/_actions/actions/create-github-app-token/1b10c78c7865c340bc4f6099eb2f838309f1e8c3/dist/main.cjs:23362:10)
    at Object.<anonymous> (/home/runner/work/_actions/actions/create-github-app-token/1b10c78c7865c340bc4f6099eb2f838309f1e8c3/dist/main.cjs:23374:20)
    at Module._compile (node:internal/modules/cjs/loader:1830:14)
    at Object..js (node:internal/modules/cjs/loader:1961:10)
    at Module.load (node:internal/modules/cjs/loader:1553:32)
    at Module._load (node:internal/modules/cjs/loader:1355:12)
    at wrapModuleLoad (node:internal/modules/cjs/loader:255:19)
    at Module.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:154:5)
##[error][@octokit/auth-app] privateKey option is required

Thinking that there might be an issue with the bot's private key? I'm gonna regenerate it and refresh the APP_PRIVATE_KEY secret then try again.

[itcarroll]

The Error: [@octokit/auth-app] privateKey option is required happening in pr_comment.yml workflow may be happening because the contents: read permission is missing but is required to see the secrets.

[itcarroll]

I think we occasionally have people from outside the org so it would be good to keep it.

We could keep it for the create-or-update-comment action, while removing it from actions/checkout (which would also make the git push step use the GITHUB_TOKEN).

It doesn't seem like that is the issue we are having though.

No this doesn't fix the slow solve issue, but it might prevent the workflow from failing because of the slow solve. The create-github-app-token step could be after the locking step, so timeout wouldn't be an issue.

[itcarroll]

Draft with suggested permissions and reordering is in #191.

[itcarroll]

/condalock

[itcarroll]

Here are the packages in environment.yml that will have major or minor changes compared to quay.io/cryointhecloud/cryo-hub-image:b39d8d901d7b. Err.. ignore some nan's that I'm not going to debug right now.

	current	incoming
jupyterhub-singleuser	4.1.6	5.4.5
nbgitpuller	1.2.2	1.3.0
jupyter-resource-usage	nan	1.2.1
gh	2.87.0	2.92.0
ipython	8.17.2	8.37.0
ipywidgets	nan	8.1.8
jupyter-offlinenotebook	nan	0.3.2
jupyterlab-git	0.51.4	0.52.0
jupyterlab_pygments	nan	0.3.0
jupytergis	0.13.2	0.15.0
hypothesis	6.151.9	6.152.4
numba	0.63.1	0.65.1
numpy	1.26.4	2.4.3
xarray	2026.2.0	2026.4.0
bokeh	3.8.2	3.9.0
leafmap	0.60.1	0.61.1
lonboard	0.14.0	0.16.0
pygmt	0.17.0	0.18.0
rasterio	1.4.3	1.5.0
rioxarray	0.20.0	0.22.0
h5py	3.13.0	3.16.0
nco	5.1.9	5.3.6
awscli	2.25.8	2.33.8
boto3	1.42.49	1.43.0
s3fs	2026.2.0	2026.4.0
earthaccess	0.16.0	0.17.0
gcsfs	2026.2.0	2026.4.0
dask	2024.12.1	2026.3.0
pytest-cov	7.0.0	7.1.0
esmpy	8.8.1	8.9.1
mystmd	1.8.1	1.9.0
qgis	3.40.3	3.44.7

[weiji14]

Cool, thanks @itcarroll for figuring out those token permissions! I just have one note below, otherwise should be ok to merge today.

[itcarroll]

@tsnow03 The expected check can be ignored; haven't figured out how to fine tune that rule since we removed spurious triggers to re-build the image.

[weiji14]

The expected check can be ignored; haven't figured out how to fine tune that rule since we removed spurious triggers to re-build the image.

That call/build job should have been triggered no? It should have been triggered with a change to the conda-linux-64.lock file

hub-image/.github/workflows/test.yaml

Lines 3 to 7 in f449351

	on:
	pull_request:
	paths:
	- "appendix"
	- "conda-linux-64.lock"

but it doesn't look like the job is running somehow? This reminds me of icesat2py/icepyx#756 (comment) - wherein the commit needs to be made with a PAT instead of the default GITHUB_TOKEN to trigger a job, maybe #191 wasn't a good idea...

[itcarroll]

😢 Maybe the only good idea was requesting the token after running conda lock. (And fixing the "hooray"!)

So we change the step order again and put the git remote set-url back in? Like this:

https://github.com/CryoInTheCloud/hub-image/blob/itcarroll-patch-2/.github/workflows/conda-lock-command.yml

[weiji14]

I think we want to revert most of #191 (generate the token and use it for 'actions/checkout', so no need to use git remote set-url), but you can keep the pillow and reactions: hooray changes 🙂 Sorry for all the back and forth, these GH Actions are very trial and error...

[weiji14]

While you do that, I'll just push a tiny commit here, so the checks will run and we can get this PR merged.

[itcarroll]

generate the token and use it for 'actions/checkout'

I don't see how we do this, necessarily before the conda solve, without going back to having a push error if the token expires. The token has a max duration of one hour, right?

[weiji14]

generate the token and use it for 'actions/checkout'

I don't see how we do this, necessarily before the conda solve, without going back to having a push error if the token expires. The token has a max duration of one hour, right?

Yes, token expires in 1 hour (there's a way to extend it following actions/create-github-app-token#121 (comment) but too complicated).

The main issue is still the slow solver (it honestly shouldn't take more than an hour). I'm holding my breath for conda-lock to support mamba 2.6.0's repodata shards (#157 (comment)), hopefully by end of this week - conda/conda-lock#907 🤞 which should improve the situation.