build: compile the duckvfs backend into the image - #434
Merged
Conversation
The published image was built `--features gemini,s3,gcs`, so the duckvfs
LaneStore was not in the binary at all. Setting
ESCUREL_STORAGE_BACKEND=duckvfs against it fails at boot with "requires the
'duckvfs' cargo feature; this binary was built without it" — the image simply
could not serve a Google Drive lane store.
It costs nothing to carry. The feature is `dep:duckdb` and libduckdb is
already linked for escurel-index, so this adds no native dependency and no
new system package; the backend was absent for no benefit.
Verified by running the image, not by reading the Dockerfile:
* `cargo check -p escurel-server --features gemini,s3,gcs,duckvfs` passes,
so the feature combination has no interaction problem.
* The container boots with ESCUREL_STORAGE_BACKEND=duckvfs and reports
healthy, where the old image would have exited on the feature error.
* The duckdb-gdrive extension is actually FETCHED inside the container:
/tmp/e/.duckdb/extensions/v1.5.5/linux_amd64/gdrive.duckdb_extension,
20MB, pulled from the community repository at first use.
That last check is the one worth keeping. The feature being compiled in only
means the backend exists; it still needs egress and a writable $HOME for
~/.duckdb at RUNTIME, and neither is visible from the Dockerfile. A pod whose
$HOME is not writable would boot, report healthy, and fail on first use — so
the Dockerfile now records both requirements next to the feature list.
Needs ESCUREL_STORAGE_DUCKVFS_EXTENSION_REPO=community in the deployment
(escurel#433).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The published image was built
--features gemini,s3,gcs, so theduckvfsLaneStore was not in the binary at all. SettingESCUREL_STORAGE_BACKEND=duckvfsagainst it fails at boot:So the image could not serve a Google Drive lane store, which is the backend's whole reason for existing.
Cost: none
The feature is
dep:duckdb, and libduckdb is already linked forescurel-index. No new native dependency, no new system package. The backend was absent for no benefit.Verified by running the image, not by reading the Dockerfile
cargo check -p escurel-server --features gemini,s3,gcs,duckvfspasses — no feature-interaction problemESCUREL_STORAGE_BACKEND=duckvfsand reports healthy, where the old image exits on the feature errorduckdb-gdriveextension is actually fetched inside the container:/tmp/e/.duckdb/extensions/v1.5.5/linux_amd64/gdrive.duckdb_extension, 20 MB, pulled from the community repository on first useThat last check is the one worth keeping. The feature being compiled in only means the backend exists; it still needs egress and a writable
$HOMEfor~/.duckdbat runtime, and neither is visible from the Dockerfile. A pod whose$HOMEis not writable would boot, report healthy, and fail on first use — so the Dockerfile now records both requirements next to the feature list.Deployment note
Needs
ESCUREL_STORAGE_DUCKVFS_EXTENSION_REPO=community(#433). Without it — and without an explicit..._EXTENSIONpath — the load is skipped rather than failing, so the store opens cleanly and the first write fails on a missingwrite_blob.Dockerfileis inpublish-image.yml'spaths:filter, so merging republishes:mainand a new:<sha>.Need help on this PR? Tag
@codesmith-botwith what you need. Autofix is disabled.