use slimmer image for init container

[sklarsa]

This PR swaps the data migration init container to a much smaller alpine distro to reduce the potential attack surface.

Since bash is not installed, we needed to update the migration script to make it POSIX-compliant.

[sklarsa]

I had claude run 3 tests locally using kind, and here are the results:

  Test 1: Fresh Install (test-fresh)                                                                               
                                                                                                                   
  Init container logs:                                                                                             
  File 'tables.d.*' not found. Nothing to move.                                                                    

  Pod status:
  NAME        READY   STATUS    RESTARTS   AGE   IP           NODE                       NOMINATED NODE   READINESS
   GATES
  questdb-0   1/1     Running   0          44s   10.244.0.6   test-fresh-control-plane   <none>           <none>

  ---
  Test 2: Old-to-New Migration (test-migrate)

  Seeded old data layout (PV root before migration):
  total 4
  drwxrwxrwx    1 root     root           142 Feb 18 16:20 .
  drwxr-xr-x    1 root     root            68 Feb 18 16:20 ..
  -rw-r--r--    1 root     root             0 Feb 18 16:20 .hidden_file
  -rw-r--r--    1 root     root             0 Feb 18 16:20 _tab_index.d
  -rw-r--r--    1 root     root             0 Feb 18 16:20 _txn
  drwxr-xr-x    1 root     root             0 Feb 18 16:20 conf
  -rw-r--r--    1 root     root             0 Feb 18 16:20 db.lock
  -rw-r--r--    1 root     root             9 Feb 18 16:20 somefile.txt
  -rw-r--r--    1 root     root             0 Feb 18 16:20 tables.d.0
  -rw-r--r--    1 root     root             0 Feb 18 16:20 tables.d.1
  SETUP-DONE

  Init container logs:
  Migration complete!

  /var/lib/questdb/ after migration (PV root — clean, only QuestDB-created dirs):
  drwxrwxrwx 1 questdb questdb  54 Feb 18 16:20 .
  drwxr-xr-x 1 root    root     48 Jan 28 15:24 ..
  drwxr-xr-x 1 questdb questdb  58 Feb 18 16:20 conf
  drwxr-xr-x 1 questdb questdb 770 Feb 18 16:20 db
  -rw-rw-r-- 1 questdb questdb 503 Feb 18 16:20 hello.txt
  drwxr-xr-x 1 questdb questdb  48 Feb 18 16:20 import
  drwxr-xr-x 1 questdb questdb  64 Feb 18 16:20 public

  /var/lib/questdb/db/ contents (all old files successfully moved in):
  -rw-r--r-- 1 questdb questdb    0 Feb 18 16:20 .hidden_file
  -rw-r--r-- 1 questdb questdb   16 Feb 18 16:20 .data_id
  drwxrwxr-x 1 questdb questdb   46 Feb 18 16:20 _query_trace
  -rw-r--r-- 1 questdb questdb 4096 Feb 18 16:20 _tab_index.d
  -rw-r--r-- 1 questdb questdb    0 Feb 18 16:20 _txn
  drwxr-xr-x 1 questdb questdb    0 Feb 18 16:20 conf
  -rw-r--r-- 1 questdb questdb    0 Feb 18 16:20 db.lock
  -rw-r--r-- 1 questdb questdb    9 Feb 18 16:20 somefile.txt
  -rw-r--r-- 1 questdb questdb    0 Feb 18 16:20 tables.d.0
  -rw-r--r-- 1 questdb questdb 4096 Feb 18 16:20 tables.d.1
  -rw-r--r-- 1 questdb questdb    0 Feb 18 16:20 tables.d.lock
  drwxrwxr-x 1 questdb questdb   66 Feb 18 16:20 telemetry
  ...

  ---
  Test 3: New-to-New Upgrade (test-upgrade)

  Old chart (debian+bash) installed, table created:
  {"ddl":"OK"}

  Upgraded to new chart (alpine+sh), init container logs:
  File 'tables.d.*' not found. Nothing to move.

  Table survived the upgrade:
  {"query":"SELECT * FROM test_table","columns":[{"name":"ts","type":"TIMESTAMP"},{"name":"val","type":"INT"}],"tim
  estamp":0,"dataset":[],"count":0}

[sklarsa]

Ran a few more tests...

  Test 4: Existing db/ directory — PASS                                                                            
                                                                                                                   
  - Seeded PV with tables.d.0, tables.d.1, AND an existing db/ dir containing existing_file.txt and                
  some_table/col.d                                                                                                 
  - Init log: "Migration complete!"                                                                                
  - Old db/ was moved to db/db/ (preserving existing_file.txt and some_table/ inside it)                           
  - All root files (rootfile.txt, _txn, .hidden, etc.) moved into db/                                              
  - QuestDB Running

  Test 5: Completely empty volume — PASS

  - PVC with zero files
  - Init log: "Nothing to move."
  - QuestDB started fine, created its own db/ directory
  - Pod 1/1 Running

  Test 6: Files with spaces and special characters — PASS

  - Seeded: file with spaces.txt, dir with spaces/inner.txt, quotefile.txt, .hidden_special
  - Init log: "Migration complete!"
  - All files moved into db/ with names intact: dir with spaces, file with spaces.txt, .hidden_special all present
  - QuestDB Running

[sklarsa]

More test results after code changes

  #: 1
  Scenario: Fresh install — empty PVC, no prior data
  Result: PASS
  Init Container Output: "Nothing to move."
  ────────────────────────────────────────
  #: 2
  Scenario: Old-to-new migration — seeded old layout with tables.d.*, hidden files
  Result: PASS
  Init Container Output: "Migration complete!" — all files including .hidden_file moved to db/
  ────────────────────────────────────────
  #: 3
  Scenario: New-to-new upgrade — install old chart (debian+bash), create table, upgrade to alpine+sh
  Result: PASS
  Init Container Output: "Nothing to move." — table survived upgrade
  ────────────────────────────────────────
  #: 4
  Scenario: Existing db/ directory — old layout + pre-existing db/ with files
  Result: PASS
  Init Container Output: "Migration complete!" — old db/ preserved at db/db/ with contents intact
  ────────────────────────────────────────
  #: 5
  Scenario: Empty volume — PVC with zero files
  Result: PASS
  Init Container Output: "Nothing to move." — QuestDB created its own db/
  ────────────────────────────────────────
  #: 6
  Scenario: Special characters — files with spaces, quotes, hidden dotfiles
  Result: PASS
  Init Container Output: "Migration complete!" — file with spaces.txt, dir with spaces, quote'file.txt, .hidden_special

[freddyrios]

is this also aimed for the next chart release?

[sklarsa]

yes, just needs review by @bluestreak01 when he gets a moment

[bluestreak01]

Hi @sklarsa 👋 — here's a critical review of this PR. I validated the rewritten script end-to-end on a real alpine:3.23 busybox shell (not just by reading it), ran helm lint, and ran shellcheck -s sh.

Verdict: solid, low-risk change with a sound goal. Approve after considering one minor behavioral regression.

Strengths

• Real, justified benefit — alpine:3.23 (~8 MB) vs debian:12.10-slim (~75 MB): less attack surface, faster pulls.
• Correct POSIX conversion — [[ ]]→[ ], dropped shopt -s extglob, replaced bash-only mv !($DEST_DIR) with a portable for loop, uses POSIX [!.] instead of [^.].
• Actually more robust than the original in two ways:
  • Proper quoting (cd "$SOURCE_DIR", mv "$item") now handles table dirs with spaces (tested; the old unquoted mv would break).
  • Better hidden-file coverage: the old .[^.]* silently missed ..foo-style names; the new ./..?* branch catches them.
• Cleanup is complete — no lingering bash/debian references anywhere; not bumping Chart.yaml matches the repo convention (bumps land in separate "chart update" PRs).

Issues (by severity)

1. Low — broken hidden symlinks are silently left behind (behavioral regression).
The guard [ -e "$item" ] || continue treats a broken symlink as non-existent (-e follows the link), so it's skipped. It's also asymmetric: the main loop has no -e guard, so a visible broken symlink is moved, but a hidden one is not. The old mv .[^.]* … || true moved it. Proof from an Alpine run:

--- source leftovers (should be only db) ---
.brokenhidden        <-- left behind, not migrated
db

Very unlikely inside a QuestDB data dir, but easy to fix:

for item in ./.[!.]* ./..?* ; do
  [ -e "$item" ] || [ -L "$item" ] || continue
  mv "$item" "$DEST_DIR/"
done

2. Low/style — [ … ] && continue under set -e. Fragile idiom in general, but I confirmed busybox ash does not prematurely exit here (loop completes, all files moved). Safe as written; a case/if would be less surprising to future readers.

3. Style — the main for item in ./* loop lacks the null-glob guard the hidden loop has. Safe today only because db (just created) and the marker always match ./*. A defensive [ -e "$item" ] || continue would make the two loops consistent and future-proof.

4. Nit — no automated test. This is a data-relocating script with no CI coverage; the change rests on manual verification. A tiny shell test (or a shellcheck -s sh gate) would be cheap insurance. shellcheck is otherwise clean — the only hit (SC2086 on ls $MARKER) is an intentional, required glob.

5. Nit — tag: "3.23" is a floating minor tag (still floats patch releases). Same practice as the old 12.10-slim, so not a regression, but a digest pin would be stronger for reproducibility/supply-chain.

Bottom line

Functionally correct and a net improvement in robustness and image size. None of the issues are blockers — I'd recommend just the one-line || [ -L "$item" ] fix (#1) to preserve exact parity with the old script's symlink handling; everything else is optional polish.

[sklarsa]

Addressed review feedback + re-validated on kind

Pushed 02a7e1d to address the review, then ran another automated review and re-ran the migration scenarios end-to-end on a local kind cluster (real chart install, alpine:3.23 init container, questdb/questdb:9.3.2).

Changes in this push

• Broken hidden symlink parity (the one recommended fix). [ -e "$item" ] follows the link, so a broken symlink was skipped. Added || [ -L "$item" ] to both loops so broken symlinks — hidden and visible — are migrated, matching the old bash mv .[^.]* … behavior.
• Dropped the fragile [ … ] && continue in the visible-files loop in favor of a plain if (no longer leaning on &&-under-set -e).
• Null-glob guard on the visible-files loop too, so both loops behave consistently.
• Documented the intentional unquoted $MARKER glob with # shellcheck disable=SC2086. shellcheck -s sh is now completely clean; helm lint passes.

Test matrix — all pass

#	Scenario	Init container log	Pod	Outcome
1	Fresh install (empty PVC)	Nothing to move.	1/1 Running	QuestDB creates its own db/; no nested dir
2	Old→new migration (markers, hidden file, conf/, db.lock, somefile.txt)	Migration complete!	1/1 Running	All files (incl. .hidden_file) moved into db/, contents intact
3	New→new upgrade (create table → helm upgrade → recreate pod)	Nothing to move.	1/1 Running	test_table + all 3 rows survived the upgrade
4	Pre-existing db/ directory	Migration complete!	1/1 Running	Root files → db/; old db/ relocated to db/db/ with existing_file.txt + some_table/ intact
6	Spaces, special chars, broken hidden + visible symlinks	Migration complete!	1/1 Running	Everything moved into db/, incl. both broken symlinks

(Test 5 "empty volume" is identical to Test 1.)

Evidence for the symlink fix (Test 6)

Seeded volume root:

lrwxrwxrwx  .brokenhidden -> /nonexistent/target      <- broken HIDDEN symlink
lrwxrwxrwx  brokenvisible -> /nonexistent/target      <- broken VISIBLE symlink
drwxr-xr-x  dir with spaces
-rw-r--r--  file with spaces.txt
-rw-r--r--  .hidden_special
-rw-r--r--  tables.d.0

After migration, db/ contains (volume root left clean):

lrwxrwxrwx  .brokenhidden -> /nonexistent/target      <- now migrated (was left behind before the fix)
lrwxrwxrwx  brokenvisible -> /nonexistent/target
drwxr-xr-x  dir with spaces
-rw-r--r--  file with spaces.txt
-rw-r--r--  .hidden_special
-rw-r--r--  tables.d.0

Test 3 data survival (host curl via port-forward)

before upgrade: SELECT count() FROM test_table -> [[3]]
after  upgrade: SHOW TABLES -> ["test_table"],  count -> [[3]],  rows -> [1],[2],[3]

Notes (non-blocking)

• The branch is behind master; its real diff is only the 3 init-container files, so a merge/squash keeps master's Chart.yaml/workflows (verified — no revert). Rebasing would just let CI run against current master.
• alpine:3.23 remains a floating minor tag (same practice as the old 12.10-slim); a digest pin would be stronger but isn't a regression.

[sklarsa]

Ready for review — summary of changes since the first review

Three commits address the review feedback, each verified end-to-end on a local kind cluster (alpine:3.23 init container) and under dash/busybox + shellcheck -s sh + helm lint:

Commit	Change
02a7e1d	Restore broken-hidden-symlink parity (|| [ -L "$item" ]), replace the fragile [ … ] && continue with a plain if, add the null-glob guard to the visible-files loop, and document the intentional $MARKER glob with a # shellcheck disable=SC2086.
807b9b1	Harden the marker check: ls -d $MARKER so an (empty) directory marker still registers as found instead of silently skipping migration.

Verification (all green)

• kind, real init container: fresh install → Nothing to move.; old→new migration (incl. hidden files, conf/, files with spaces) → all under db/; pre-existing db/ → relocated to db/db/ with contents intact; broken hidden + visible symlinks → migrated; empty-marker-dir → migrated; new→new upgrade → data survived. Every pod 1/1 Running.
• shellcheck -s sh clean; helm lint passes; three independent review passes found no correctness regressions.

Known follow-up (not in scope here)

The migration is not crash-resumable — if the init container is killed mid-run it can strand data on re-run. This is pre-existing (the old bash script had the identical flaw) and low-probability, so it's tracked and being fixed separately in #260 (stacked PR incoming) rather than bundled into this image-swap change.

This PR is a faithful, behavior-preserving POSIX/alpine port and is ready for review.