Skip to content

Commit c0a72e3

Browse files
committed
fix: address review feedback on nebi version floor, image-ref sync, CI smoke test
- Document the nebi >= v0.15 floor where deployers look: comment at values.yaml nebi.image.tag, override prose in nebi-integration.md, and a troubleshooting row for the silent ~61s fallback (including the slow-start mode fallback race). - Fix the profiles NOTE comment to name all three refs a hand-bump must move (outer kubespawner_override.image, choices.default image, and its display_name). - Add tests/unit/test_image_ref_sync.py: profile images and choice display_names must match singleuser.image, and hub.image.tag must match singleuser.image.tag, so a half-bump fails the unit job instead of e2e passing against a stale image. - Add a post-build smoke step to build-image.yaml (opt-in via smoke_test_labextensions, enabled for the jupyterlab image): run jupyter labextension list in the pushed digest and assert the jupyterlab-launchpad version pinned in pixi.toml is enabled.
1 parent 26b59e9 commit c0a72e3

5 files changed

Lines changed: 125 additions & 4 deletions

File tree

.github/workflows/build-image.yaml

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -37,6 +37,11 @@ on:
3737
required: false
3838
default: ""
3939
type: string
40+
smoke_test_labextensions:
41+
description: "After each arch build, run `jupyter labextension list` in the pushed image and assert the jupyterlab-launchpad version pinned in images/jupyterlab/pixi.toml is enabled. Only meaningful for jupyterlab-target images. Skipped on fork PRs (no pushed image to run)."
42+
required: false
43+
default: false
44+
type: boolean
4045
secrets:
4146
QUAY_TOKEN:
4247
required: false
@@ -96,6 +101,15 @@ jobs:
96101
GPU=${{ inputs.gpu }}
97102
NEBI_IMAGE=${{ inputs.nebi_image }}
98103
104+
- name: "Smoke test: labextensions enabled at pinned versions"
105+
if: env.IS_FORK_PR != 'true' && inputs.smoke_test_labextensions
106+
run: |
107+
expected=$(sed -nE 's/^jupyterlab-launchpad = "==([^"]+)"$/\1/p' images/jupyterlab/pixi.toml)
108+
test -n "$expected" || { echo "could not read jupyterlab-launchpad pin from pixi.toml"; exit 1; }
109+
out=$(docker run --rm "ghcr.io/${{ github.repository_owner }}/${{ inputs.image }}@${{ steps.build.outputs.digest }}" jupyter labextension list 2>&1)
110+
printf '%s\n' "$out"
111+
grep -F "jupyterlab-launchpad v${expected}" <<< "$out"
112+
99113
- name: "Export digest"
100114
if: env.IS_FORK_PR != 'true'
101115
run: |
@@ -150,6 +164,15 @@ jobs:
150164
GPU=${{ inputs.gpu }}
151165
NEBI_IMAGE=${{ inputs.nebi_image }}
152166
167+
- name: "Smoke test: labextensions enabled at pinned versions"
168+
if: env.IS_FORK_PR != 'true' && inputs.smoke_test_labextensions
169+
run: |
170+
expected=$(sed -nE 's/^jupyterlab-launchpad = "==([^"]+)"$/\1/p' images/jupyterlab/pixi.toml)
171+
test -n "$expected" || { echo "could not read jupyterlab-launchpad pin from pixi.toml"; exit 1; }
172+
out=$(docker run --rm "ghcr.io/${{ github.repository_owner }}/${{ inputs.image }}@${{ steps.build.outputs.digest }}" jupyter labextension list 2>&1)
173+
printf '%s\n' "$out"
174+
grep -F "jupyterlab-launchpad v${expected}" <<< "$out"
175+
153176
- name: "Export digest"
154177
if: env.IS_FORK_PR != 'true'
155178
run: |

.github/workflows/build-images.yaml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -43,6 +43,7 @@ jobs:
4343
with:
4444
image: nebari-data-science-pack-jupyterlab
4545
target: jupyterlab
46+
smoke_test_labextensions: true
4647
secrets:
4748
QUAY_TOKEN: ${{ secrets.QUAY_TOKEN }}
4849

docs/src/content/docs/nebi-integration.md

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -78,6 +78,11 @@ Leaving `tag` empty disables the init container entirely — no Nebi in user pod
7878
pinned per chart release (`scripts/bump_image_tags.py` only handles the JupyterLab images),
7979
so override it to test a PR build or to roll forward between chart releases.
8080

81+
When overriding, keep the binary **≥ v0.15**: `jupyterlab-launchpad` (≥ 1.1.1, pinned in
82+
the JupyterLab image) listens for `nebi:job-completed` from this binary to refresh the
83+
kernel list immediately after an environment build. Older nebi never sends the message,
84+
so kernels silently fall back to the ~61s `KernelSpecManager` poll with no error logged.
85+
8186
## The token exchange
8287

8388
Nebi needs a per-user credential, and the hub is the only component holding the user's
@@ -212,6 +217,7 @@ CA merge step is ordered before it, so the bundle is ready. See
212217
| Empty environment dropdown in jhub-apps | Exchange failure, or `auth_state` missing — `kubectl logs deploy/hub \| grep nebi-envs`. |
213218
| Registry changes not visible | Hub not restarted since the change. |
214219
| Apps missing packages in a Nebi env | `jhub-app-proxy-version` below v0.2.3. |
220+
| New kernels take ~61s to appear after an env build | `nebi.image.tag` below v0.15 (no `nebi:job-completed` sender), launchpad below 1.1.1, the Nebi tab not open when the build finished, or nebi fell back to team mode after a slow start (`/version` unreachable for ~900ms at handshake time) — all fail silently to the poll. |
215221

216222
```bash
217223
# Is the binary in the pod?

tests/unit/test_image_ref_sync.py

Lines changed: 84 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,84 @@
1+
"""Structural tests that every hand-editable image reference in values.yaml
2+
agrees with ``jupyterhub.singleuser.image``.
3+
4+
e2e derives its cache key and kind side-load from ``singleuser.image``, but
5+
the pod that actually spawns comes from the *default profile*: the spawn
6+
POST has no body, so kubespawner falls through to the ``default: true``
7+
profile, whose ``profile_options`` default choice overwrites the image.
8+
A bump that moves ``singleuser.image.tag`` but misses a profile ref would
9+
therefore have e2e report the new tag while the pod pulls the old image —
10+
green CI on stale code. ``scripts/bump_image_tags.py`` keeps these in sync
11+
on the automated path; these asserts catch the hand-edit path.
12+
"""
13+
14+
from __future__ import annotations
15+
16+
from pathlib import Path
17+
18+
import yaml
19+
20+
REPO_ROOT = Path(__file__).resolve().parents[2]
21+
VALUES_YAML = REPO_ROOT / "values.yaml"
22+
23+
24+
def _jupyterhub_values():
25+
with VALUES_YAML.open() as f:
26+
return yaml.safe_load(f)["jupyterhub"]
27+
28+
29+
def _singleuser_ref(jh):
30+
image = jh["singleuser"]["image"]
31+
return f'{image["name"]}:{image["tag"]}'
32+
33+
34+
def test_profile_images_match_singleuser():
35+
"""Every profile's outer kubespawner_override.image AND its
36+
profile_options default-choice image must equal singleuser.image —
37+
the default choice is what the spawned pod actually runs."""
38+
jh = _jupyterhub_values()
39+
ref = _singleuser_ref(jh)
40+
profiles = jh["custom"]["profiles"]
41+
assert profiles, "no profiles found under jupyterhub.custom.profiles"
42+
for profile in profiles:
43+
slug = profile["slug"]
44+
assert profile["kubespawner_override"]["image"] == ref, (
45+
f"profile {slug!r}: kubespawner_override.image does not match "
46+
f"singleuser.image ({ref}) — jhub-apps' Create App shows this "
47+
"value; a half-bump here spawns a stale image"
48+
)
49+
choices = profile["profile_options"]["image"]["choices"]
50+
for name, choice in choices.items():
51+
assert choice["kubespawner_override"]["image"] == ref, (
52+
f"profile {slug!r} choice {name!r}: image does not match "
53+
f"singleuser.image ({ref}) — this choice overwrites the pod "
54+
"image at spawn, so e2e would report the new tag while the "
55+
"pod pulls the old one"
56+
)
57+
58+
59+
def test_profile_choice_display_names_match_image():
60+
"""The default choice's display_name embeds the tag the profile selector
61+
shows; it must name the image the choice actually spawns."""
62+
jh = _jupyterhub_values()
63+
image = jh["singleuser"]["image"]
64+
expected = f'{image["name"].rsplit("/", 1)[-1]}:{image["tag"]}'
65+
for profile in jh["custom"]["profiles"]:
66+
choices = profile["profile_options"]["image"]["choices"]
67+
for name, choice in choices.items():
68+
assert choice["display_name"] == expected, (
69+
f'profile {profile["slug"]!r} choice {name!r}: display_name '
70+
f'{choice["display_name"]!r} does not match the image it '
71+
f"spawns ({expected}) — the selector would show one tag and "
72+
"run another"
73+
)
74+
75+
76+
def test_hub_tag_matches_singleuser_tag():
77+
"""hub and jupyterlab images are built from the same commit and tagged
78+
with the same sha; a half-bump that moves the hub pair but not the
79+
jupyterlab refs (or vice versa) must not pass unnoticed."""
80+
jh = _jupyterhub_values()
81+
assert jh["hub"]["image"]["tag"] == jh["singleuser"]["image"]["tag"], (
82+
"hub.image.tag and singleuser.image.tag are bumped together from the "
83+
"same commit's build; a mismatch means a partial hand-bump"
84+
)

values.yaml

Lines changed: 11 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -253,6 +253,10 @@ nebi:
253253
# handles the jupyterlab images); deployers can override to test a PR
254254
# build or roll forward. Leave non-empty so the init container is wired
255255
# by default. sha-5ca877a == the nebi v0.15 tag commit.
256+
# Must be >= v0.15: jupyterlab-launchpad >= 1.1.1 (pinned in
257+
# images/jupyterlab/pixi.toml) listens for `nebi:job-completed` from this
258+
# binary. Older nebi sends nothing, so kernel refresh silently falls back
259+
# to the ~61s KernelSpecManager poll — no error is logged anywhere.
256260
tag: "sha-5ca877a"
257261
pullPolicy: IfNotPresent
258262
# External Nebi FQDN (browser-side, used for OIDC redirect).
@@ -489,10 +493,13 @@ jupyterhub:
489493
# kubespawner_override:
490494
# extra_resource_limits:
491495
# nvidia.com/gpu: 1
492-
# NOTE: when bumping singleuser.image.tag below, also bump the
493-
# ``image: ...`` lines inside each profile_options.image.choices.default
494-
# entry so the profile selector shows the right tag. (z2jh values.yaml
495-
# cannot reference other values, so the duplication is unavoidable.)
496+
# NOTE: when bumping singleuser.image.tag below, also bump — in EACH
497+
# profile — the outer ``kubespawner_override.image``, the ``image: ...``
498+
# line inside profile_options.image.choices.default, and that choice's
499+
# ``display_name`` (it embeds the tag the selector shows). (z2jh
500+
# values.yaml cannot reference other values, so the duplication is
501+
# unavoidable.) tests/unit/test_image_ref_sync.py fails if any of them
502+
# drift from singleuser.image.
496503
profiles:
497504
- slug: small-instance
498505
display_name: "Small Instance"

0 commit comments

Comments
 (0)