Skip to content

Migrate self-hosted Supabase and fix Auth TLS trust - #1508

Merged
JackWilb merged 15 commits into
devfrom
codex/supabase-upstream
Oct 22, 2026
Merged

JackWilb merged 15 commits into
devfrom
codex/supabase-upstream

Conversation

@JackWilb

@JackWilb JackWilb commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

Why

The bundled self-hosted stack uses unavailable MinIO images (or main's pgsty replacement), and its Auth image accepts certificate chains affected by the eMudhra compromise. Move deployments to a pinned upstream bundle with an upstream Auth TLS fix and a tested migration procedure.

Architecture

Deployment files and operating scripts come from Supabase self-hosted/v0.8.3, verified against commit bc27d2661edd09a5d0b8d76ce33da6d4071a756e, outside the application checkout. It supplies Postgres 17, Envoy and Storage 1.81 with persistent local files. reVISit owns setup SQL, GitHub configuration, migration helpers and instructions.

A required Compose override pins supabase/gotrue:v2.198.0-rc.25 by digest, including Supabase's Go 1.27.3 fix. The bundle's stable Auth 2.197 predates the fix. GitHub remains optional; the fixed Auth override applies to every deployment and persists through recreation. No custom Auth build is required.

Migration preserves existing signing/encryption secrets, sessions, application records, bucket settings and access policies. Original object bytes are exported through the old Storage API and installed with their versioned paths and HTTP metadata; database object records are not rewritten. Checksums, truncated responses, concurrent metadata changes and S3 file/directory conflicts are checked before upgrading the database. Atomic installs support retries after interruption, disk exhaustion or failed metadata writes.

A small guarded helper prepares a copy of the pinned upstream database-upgrade script. It selects this database's encryption volume, respects the saved Auth/GitHub overrides on first startup, and treats required migration failures as fatal. The original upstream script remains unchanged. Unexpected upstream source is rejected before producing a prepared copy.

User-facing impact

Existing self-hosted operators must follow the migration guide during a maintenance window. Updating the application checkout alone does not update running services. New deployments follow the setup guide; hosted Supabase does not require this migration.

Validation

At 5dc9fc5b53e9716efb8735ba5744e707bf5b9d81:

  • Python migration/helper suites: 18 passed. Focused Supabase unit suites: 105 passed. Lint, typecheck and production build passed. Documented shell blocks and prepared upgrade script parse successfully. New concurrency/path-conflict regressions fail against the previous helper and pass with the fix.
  • Real Linux/ARM64 Docker rehearsal from main's PG15+pgsty stack: 1,219 objects across multiple buckets, including 1,205 small objects, a 32 MiB recording, zero bytes, overwrites and unusual supported names. Every original byte/hash, HTTP metadata, application row, bucket setting, policy and synthetic Vault plaintext matched after migration. All 11 upgraded services healthy; exact pinned Auth verified.
  • Existing Designer/anonymous access and refresh tokens, password login, row/object access restrictions and empty/private buckets passed. Five Chromium checks passed after migration: new submissions/fresh-context persistence, retained data, unfinished participant continuation, retained Designer authorization/JSON+CSV exports and binary API operations. Four applicable checks passed after full container recreation and again after coordinated rollback.
  • Original MinIO empty-deployment migration and rollback passed, preserving the empty bucket and encrypted test value. Two Chromium checks passed on that upgraded deployment. A populated original-MinIO deployment also migrated all 20 source objects, including a legacy object with no version ID; an actual S3 file/directory conflict was rejected before the upgrade. The original MinIO server and a checksum-verified official released client binary were used; the client binary was packaged locally because its registry image was unavailable.
  • Real failure checks: interrupted export, partial copy, insufficient disk, unwritable destination, missing rollback-image archive and failed required database migration all stopped as intended. Retrying interrupted/failed installs preserved the expected object. An unrelated encryption-volume sentinel retained its bytes, ownership and permissions.
  • Mocked GitHub authorize/callback/code exchange/session issuance passed through Envoy. Fresh installation with generated publishable keys and ES256 anonymous sessions, plus recreation, passed in the earlier rehearsal with the same pinned images. The final pinned Auth image's native trust store rejected all 12 compromised-chain fixtures and accepted the valid GitHub control.
  • Main app compatibility: its own locked dependencies passed 66 focused Supabase unit tests, lint/typecheck/build, all five browser acceptance checks before and after migration, and the four applicable browser checks after recreation and again after the VM reboot. No application source or dependencies changed.
  • Independent adversarial review of the final head found no remaining findings; all 12 historical Codex findings were verified fixed or stale. GitHub checks are tracked separately on this head.

Rehearsals used synthetic data on Docker Desktop's Linux VM, Linux-volume paths and native Auth/Postgres images. Ports, checkout paths and noninteractive/root invocation were adapted for the fixture; the actual guide commands and prepared upgrade script were exercised without a volume-discovery shim. Full container removal/recreation and an actual Docker VM reboot passed. The kernel boot ID changed; all 11 services recovered healthy, all 29 current objects and database metadata matched the post-browser export, application/policy/Vault snapshots matched, and existing access/refresh tokens and password login passed. Unrelated containers retained their original stopped states and restart policies. Live external GitHub credentials/callback, SMTP delivery and hardware capture were not tested. The user confirmed live OAuth/SMTP are unavailable locally.

Risk and rollout

Pause all writes; make coordinated external database, object-store, encryption and configuration backups; save the original object-store images; rehearse restoration before changing a deployment. The guide checks Linux extended attributes and export compatibility before the database upgrade, checks required database migrations/grants and reindexes, compares application/bucket/policy snapshots and verifies every exported object before acceptance. Never regenerate existing keys or run fresh setup SQL during migration.

Storage's schema migration is one-way: rollback requires coordinated backups and loses subsequent writes. Rollback restores vulnerable Auth, so the documented certificate mitigation must succeed before normal operation and must be reapplied after recreation. File storage requires Linux extended attributes and does not supply multi-host failover. Custom databases/extensions/S3 layouts need separate rehearsal. Move retained copies and the upgrade binary cache outside the deployment before upstream updates.

Auth remains a release candidate; review a patched stable replacement before removing the override. Verify the deployed GitHub callback and SMTP settings during the deployment smoke test before reopening traffic.

@github-actions

github-actions Bot commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

🪓 PR closed, deleted preview.

@JackWilb
JackWilb marked this pull request as ready for review October 7, 2026 03:55
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 7, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-07T05:01:36.250891Z 504f7c0 Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 39abb1d481

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread supabase/utils/upgrade-pg17.sh Outdated
Comment thread supabase/MIGRATION.md Outdated
Comment thread supabase/migrate-storage.py Outdated
Comment thread supabase/update.sh Outdated
Comment thread supabase/utils/upgrade-pg17.sh Outdated
Comment thread supabase/utils/upgrade-pg17.sh Outdated
@JackWilb

JackWilb commented Oct 7, 2026

Copy link
Copy Markdown
Contributor Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 2584871f9c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread supabase/MIGRATION.md Outdated
Comment thread supabase/MIGRATION.md Outdated
Comment thread supabase/MIGRATION.md
Comment thread supabase/README.md Outdated
@JackWilb

JackWilb commented Oct 7, 2026

Copy link
Copy Markdown
Contributor Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 504f7c0651

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread supabase/tests/migrate-storage.spec.py Outdated
Comment thread supabase/README.md Outdated
@JackWilb JackWilb changed the title Migrate self-hosted Supabase off MinIO Migrate self-hosted Supabase and fix Auth TLS trust Oct 22, 2026
@JackWilb
JackWilb merged commit 09b4294 into dev Oct 22, 2026
11 checks passed
@JackWilb
JackWilb deleted the codex/supabase-upstream branch October 22, 2026 17:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant