ci: smoke-test the shipped artifact on every platform - #3
Merged
Merged
Conversation
Every existing job exercises build/release/duckdb -- a shell with the extension statically linked in. That never loads the .duckdb_extension file users install, so nothing in this pipeline could catch an artifact that builds green and then fails to instantiate. quack-oauth's wasm side-module is the standing example of that gap biting. scripts/smoke_test.py downloads the OFFICIAL DuckDB CLI for the target version, installs the artifact into it, loads it, and calls tunnels(). The third step matters: loading only proves the shared object resolves, and symbols can resolve lazily and fail on first call. Wired as needs: [build] with deploy taking needs: [build, smoke-test], so a broken artifact on any platform blocks distribution. Verified locally against the v1.5.5 artifact -- passes on a good build, and exits non-zero for a truncated artifact, a missing file, an unknown arch, and a loadable extension whose function does not exist.
A job-level 'if:' is evaluated before the matrix expands, so matrix.arch is not in scope there. GitHub rejected the whole workflow at validation time -- the run reported failure with zero jobs and no logs, which reads like nothing happened rather than like a syntax error. Filter in a first step instead, where the matrix context is available.
This was referenced Aug 8, 2026
jrosskopf
added a commit
to DataZooDE/duckdb-gdrive
that referenced
this pull request
Aug 8, 2026
Every existing job exercises build/release/duckdb -- a shell with the extension statically linked in. That never loads the .duckdb_extension file users install, so nothing in this pipeline could catch an artifact that builds green and then fails to instantiate. quack-oauth's CLAUDE.md already records that gap biting: its wasm side-module "builds green but won't instantiate in the browser". scripts/smoke_test.py downloads the OFFICIAL DuckDB CLI for the target version, installs the artifact into it, loads it, and calls a real function. The third step matters on its own -- loading proves the shared object resolves, and symbols can resolve lazily and only fail on call. Wired as needs: [build], with each deploy job taking the smoke job in its needs, so a broken artifact on any platform blocks distribution. The smoke query was verified against a real build of this extension rather than read off the source. Three of the fleet-wide candidates were wrong when checked that way. Piloted in DataZooDE/erpl-tunnel#3: six smoke jobs green across linux_amd64, osx_arm64 and windows_amd64 on two DuckDB lines.
jrosskopf
added a commit
to DataZooDE/erpl-idoc
that referenced
this pull request
Aug 8, 2026
Every existing job exercises build/release/duckdb -- a shell with the extension statically linked in. That never loads the .duckdb_extension file users install, so nothing in this pipeline could catch an artifact that builds green and then fails to instantiate. quack-oauth's CLAUDE.md already records that gap biting: its wasm side-module "builds green but won't instantiate in the browser". scripts/smoke_test.py downloads the OFFICIAL DuckDB CLI for the target version, installs the artifact into it, loads it, and calls a real function. The third step matters on its own -- loading proves the shared object resolves, and symbols can resolve lazily and only fail on call. Wired as needs: [build], with each deploy job taking the smoke job in its needs, so a broken artifact on any platform blocks distribution. The smoke query was verified against a real build of this extension rather than read off the source. Three of the fleet-wide candidates were wrong when checked that way. Piloted in DataZooDE/erpl-tunnel#3: six smoke jobs green across linux_amd64, osx_arm64 and windows_amd64 on two DuckDB lines.
jrosskopf
added a commit
to DataZooDE/anofox-similarity
that referenced
this pull request
Aug 8, 2026
Every existing job exercises build/release/duckdb -- a shell with the extension statically linked in. That never loads the .duckdb_extension file users install, so nothing in this pipeline could catch an artifact that builds green and then fails to instantiate. quack-oauth's CLAUDE.md already records that gap biting: its wasm side-module "builds green but won't instantiate in the browser". scripts/smoke_test.py downloads the OFFICIAL DuckDB CLI for the target version, installs the artifact into it, loads it, and calls a real function. The third step matters on its own -- loading proves the shared object resolves, and symbols can resolve lazily and only fail on call. Wired as needs: [build], with each deploy job taking the smoke job in its needs, so a broken artifact on any platform blocks distribution. The smoke query was verified against a real build of this extension rather than read off the source. Three of the fleet-wide candidates were wrong when checked that way. Piloted in DataZooDE/erpl-tunnel#3: six smoke jobs green across linux_amd64, osx_arm64 and windows_amd64 on two DuckDB lines.
jrosskopf
added a commit
to DataZooDE/quack-oauth
that referenced
this pull request
Aug 9, 2026
* ci: smoke-test the shipped artifact on every platform Every existing job exercises build/release/duckdb -- a shell with the extension statically linked in. That never loads the .duckdb_extension file users install, so nothing in this pipeline could catch an artifact that builds green and then fails to instantiate. quack-oauth's CLAUDE.md already records that gap biting: its wasm side-module "builds green but won't instantiate in the browser". scripts/smoke_test.py downloads the OFFICIAL DuckDB CLI for the target version, installs the artifact into it, loads it, and calls a real function. The third step matters on its own -- loading proves the shared object resolves, and symbols can resolve lazily and only fail on call. Wired as needs: [build], with each deploy job taking the smoke job in its needs, so a broken artifact on any platform blocks distribution. The smoke query was verified against a real build of this extension rather than read off the source. Three of the fleet-wide candidates were wrong when checked that way. Piloted in DataZooDE/erpl-tunnel#3: six smoke jobs green across linux_amd64, osx_arm64 and windows_amd64 on two DuckDB lines. * fix(ci): match excluded archs exactly so Windows is really smoke tested The gate used a substring test against exclude_archs. Nearly every caller excludes `windows_amd64_mingw`, which also matches `windows_amd64` -- so the Windows smoke job printed "excluded by the build job" and reported success without ever downloading, loading or exercising the artifact. Green meant nothing ran, which is precisely what this workflow exists to prevent. Split the list on `;`/`,`/whitespace and compare whole tokens instead. The list is passed via env because some callers set exclude_archs from a GitHub expression whose value contains quotes.
jrosskopf
added a commit
to DataZooDE/anofox-bayes
that referenced
this pull request
Aug 9, 2026
* ci: smoke-test the shipped artifact on every platform Every existing job exercises build/release/duckdb -- a shell with the extension statically linked in. That never loads the .duckdb_extension file users install, so nothing in this pipeline could catch an artifact that builds green and then fails to instantiate. quack-oauth's CLAUDE.md already records that gap biting: its wasm side-module "builds green but won't instantiate in the browser". scripts/smoke_test.py downloads the OFFICIAL DuckDB CLI for the target version, installs the artifact into it, loads it, and calls a real function. The third step matters on its own -- loading proves the shared object resolves, and symbols can resolve lazily and only fail on call. Wired as needs: [build], with each deploy job taking the smoke job in its needs, so a broken artifact on any platform blocks distribution. The smoke query was verified against a real build of this extension rather than read off the source. Three of the fleet-wide candidates were wrong when checked that way. Piloted in DataZooDE/erpl-tunnel#3: six smoke jobs green across linux_amd64, osx_arm64 and windows_amd64 on two DuckDB lines. * fix(ci): match excluded archs exactly so Windows is really smoke tested The gate used a substring test against exclude_archs. Nearly every caller excludes `windows_amd64_mingw`, which also matches `windows_amd64` -- so the Windows smoke job printed "excluded by the build job" and reported success without ever downloading, loading or exercising the artifact. Green meant nothing ran, which is precisely what this workflow exists to prevent. Split the list on `;`/`,`/whitespace and compare whole tokens instead. The list is passed via env because some callers set exclude_archs from a GitHub expression whose value contains quotes. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
jrosskopf
added a commit
to DataZooDE/anofox-statistics
that referenced
this pull request
Aug 9, 2026
* ci: smoke-test the shipped artifact on every platform Every existing job exercises build/release/duckdb -- a shell with the extension statically linked in. That never loads the .duckdb_extension file users install, so nothing in this pipeline could catch an artifact that builds green and then fails to instantiate. quack-oauth's CLAUDE.md already records that gap biting: its wasm side-module "builds green but won't instantiate in the browser". scripts/smoke_test.py downloads the OFFICIAL DuckDB CLI for the target version, installs the artifact into it, loads it, and calls a real function. The third step matters on its own -- loading proves the shared object resolves, and symbols can resolve lazily and only fail on call. Wired as needs: [build], with each deploy job taking the smoke job in its needs, so a broken artifact on any platform blocks distribution. The smoke query was verified against a real build of this extension rather than read off the source. Three of the fleet-wide candidates were wrong when checked that way. Piloted in DataZooDE/erpl-tunnel#3: six smoke jobs green across linux_amd64, osx_arm64 and windows_amd64 on two DuckDB lines. * fix(ci): match excluded archs exactly so Windows is really smoke tested The gate used a substring test against exclude_archs. Nearly every caller excludes `windows_amd64_mingw`, which also matches `windows_amd64` -- so the Windows smoke job printed "excluded by the build job" and reported success without ever downloading, loading or exercising the artifact. Green meant nothing ran, which is precisely what this workflow exists to prevent. Split the list on `;`/`,`/whitespace and compare whole tokens instead. The list is passed via env because some callers set exclude_archs from a GitHub expression whose value contains quotes. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
jrosskopf
added a commit
to DataZooDE/anofox-scenario
that referenced
this pull request
Aug 9, 2026
* ci: smoke-test the shipped artifact on every platform Every existing job exercises build/release/duckdb -- a shell with the extension statically linked in. That never loads the .duckdb_extension file users install, so nothing in this pipeline could catch an artifact that builds green and then fails to instantiate. quack-oauth's CLAUDE.md already records that gap biting: its wasm side-module "builds green but won't instantiate in the browser". scripts/smoke_test.py downloads the OFFICIAL DuckDB CLI for the target version, installs the artifact into it, loads it, and calls a real function. The third step matters on its own -- loading proves the shared object resolves, and symbols can resolve lazily and only fail on call. Wired as needs: [build], with each deploy job taking the smoke job in its needs, so a broken artifact on any platform blocks distribution. The smoke query was verified against a real build of this extension rather than read off the source. Three of the fleet-wide candidates were wrong when checked that way. Piloted in DataZooDE/erpl-tunnel#3: six smoke jobs green across linux_amd64, osx_arm64 and windows_amd64 on two DuckDB lines. * fix(ci): match excluded archs exactly so Windows is really smoke tested The gate used a substring test against exclude_archs. Nearly every caller excludes `windows_amd64_mingw`, which also matches `windows_amd64` -- so the Windows smoke job printed "excluded by the build job" and reported success without ever downloading, loading or exercising the artifact. Green meant nothing ran, which is precisely what this workflow exists to prevent. Split the list on `;`/`,`/whitespace and compare whole tokens instead. The list is passed via env because some callers set exclude_archs from a GitHub expression whose value contains quotes. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
jrosskopf
added a commit
to DataZooDE/anofox-forecast
that referenced
this pull request
Aug 9, 2026
* ci: smoke-test the shipped artifact on every platform Every existing job exercises build/release/duckdb -- a shell with the extension statically linked in. That never loads the .duckdb_extension file users install, so nothing in this pipeline could catch an artifact that builds green and then fails to instantiate. quack-oauth's CLAUDE.md already records that gap biting: its wasm side-module "builds green but won't instantiate in the browser". scripts/smoke_test.py downloads the OFFICIAL DuckDB CLI for the target version, installs the artifact into it, loads it, and calls a real function. The third step matters on its own -- loading proves the shared object resolves, and symbols can resolve lazily and only fail on call. Wired as needs: [build], with each deploy job taking the smoke job in its needs, so a broken artifact on any platform blocks distribution. The smoke query was verified against a real build of this extension rather than read off the source. Three of the fleet-wide candidates were wrong when checked that way. Piloted in DataZooDE/erpl-tunnel#3: six smoke jobs green across linux_amd64, osx_arm64 and windows_amd64 on two DuckDB lines. * fix(ci): match excluded archs exactly so Windows is really smoke tested The gate used a substring test against exclude_archs. Nearly every caller excludes `windows_amd64_mingw`, which also matches `windows_amd64` -- so the Windows smoke job printed "excluded by the build job" and reported success without ever downloading, loading or exercising the artifact. Green meant nothing ran, which is precisely what this workflow exists to prevent. Split the list on `;`/`,`/whitespace and compare whole tokens instead. The list is passed via env because some callers set exclude_archs from a GitHub expression whose value contains quotes. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
jrosskopf
added a commit
to DataZooDE/anofox-tabular
that referenced
this pull request
Aug 9, 2026
* ci: smoke-test the shipped artifact on every platform Every existing job exercises build/release/duckdb -- a shell with the extension statically linked in. That never loads the .duckdb_extension file users install, so nothing in this pipeline could catch an artifact that builds green and then fails to instantiate. quack-oauth's CLAUDE.md already records that gap biting: its wasm side-module "builds green but won't instantiate in the browser". scripts/smoke_test.py downloads the OFFICIAL DuckDB CLI for the target version, installs the artifact into it, loads it, and calls a real function. The third step matters on its own -- loading proves the shared object resolves, and symbols can resolve lazily and only fail on call. Wired as needs: [build], with each deploy job taking the smoke job in its needs, so a broken artifact on any platform blocks distribution. The smoke query was verified against a real build of this extension rather than read off the source. Three of the fleet-wide candidates were wrong when checked that way. Piloted in DataZooDE/erpl-tunnel#3: six smoke jobs green across linux_amd64, osx_arm64 and windows_amd64 on two DuckDB lines. * fix(ci): match excluded archs exactly so Windows is really smoke tested The gate used a substring test against exclude_archs. Nearly every caller excludes `windows_amd64_mingw`, which also matches `windows_amd64` -- so the Windows smoke job printed "excluded by the build job" and reported success without ever downloading, loading or exercising the artifact. Green meant nothing ran, which is precisely what this workflow exists to prevent. Split the list on `;`/`,`/whitespace and compare whole tokens instead. The list is passed via env because some callers set exclude_archs from a GitHub expression whose value contains quotes.
jrosskopf
added a commit
to DataZooDE/anofox-tabfm
that referenced
this pull request
Aug 9, 2026
* ci: smoke-test the shipped artifact on every platform Every existing job exercises build/release/duckdb -- a shell with the extension statically linked in. That never loads the .duckdb_extension file users install, so nothing in this pipeline could catch an artifact that builds green and then fails to instantiate. quack-oauth's CLAUDE.md already records that gap biting: its wasm side-module "builds green but won't instantiate in the browser". scripts/smoke_test.py downloads the OFFICIAL DuckDB CLI for the target version, installs the artifact into it, loads it, and calls a real function. The third step matters on its own -- loading proves the shared object resolves, and symbols can resolve lazily and only fail on call. Wired as needs: [build], with each deploy job taking the smoke job in its needs, so a broken artifact on any platform blocks distribution. The smoke query was verified against a real build of this extension rather than read off the source. Three of the fleet-wide candidates were wrong when checked that way. Piloted in DataZooDE/erpl-tunnel#3: six smoke jobs green across linux_amd64, osx_arm64 and windows_amd64 on two DuckDB lines. * fix(ci): match excluded archs exactly so Windows is really smoke tested The gate used a substring test against exclude_archs. Nearly every caller excludes `windows_amd64_mingw`, which also matches `windows_amd64` -- so the Windows smoke job printed "excluded by the build job" and reported success without ever downloading, loading or exercising the artifact. Green meant nothing ran, which is precisely what this workflow exists to prevent. Split the list on `;`/`,`/whitespace and compare whole tokens instead. The list is passed via env because some callers set exclude_archs from a GitHub expression whose value contains quotes. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
jrosskopf
added a commit
that referenced
this pull request
Aug 22, 2026
The constant is stamped on telemetry and the feedback banner, so it has to match the tag being cut. Changelog covers everything unreleased since v2026.08.07: the replace-on-conflict registration fix (which erpl's tunnel removal depends on), the v1.4.5 Windows/arm64 vcpkg repair, and the per-platform smoke tests from #3 and #4.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The gap
Every job in this pipeline exercises
build/release/duckdb— a shell with the extension statically linked in. That never loads the.duckdb_extensionfile users actually install, so nothing here can catch an artifact that builds green and then fails to instantiate.That is not hypothetical. quack-oauth's
CLAUDE.mddocuments its wasm side-module as "builds green but won't instantiate in the browser — a load-time failure CI can't see." Same gap, already known.What this adds
scripts/smoke_test.pydownloads the official DuckDB CLI for the target version, installs the built artifact into it, loads it, and calls a real function:INSTALLLOAD+duckdb_extensions()SELECT count(*) FROM tunnels()The third step matters on its own: loading proves the shared object resolves, not that its registered functions run — symbols can resolve lazily and only fail when called.
tunnels()was chosen because it needs no network, no SSH, no credentials: with no tunnels open it returns zero rows, which is a successful call.Wiring
needs: [build], with deploy moved toneeds: [build, smoke-test]on both the v1.5.5 and v1.4.5 lines — a broken artifact on any platform now blocks distribution.Matrix:
linux_amd64,osx_arm64,windows_amd64, honouring the build job'sexclude_archs.Verified locally
Against the real v1.5.5 artifact:
Note on approach
This is the pilot for a fleet-wide rollout — 11 of 15 extensions currently have no load verification at all. It follows erpl-web's single cross-platform Python script rather than erpl's sh/ps1/musl trio, so the other repos need one file each rather than three.
Need help on this PR? Tag
@codesmith-botwith what you need. Autofix is disabled.