Repository navigation
deps: upgrade to pgx v5 and tern v2 (MARSOHS-1643) - #15
Merged
Merged
Conversation
Semgrep flags any new file importing github.com/jackc/pgx/v4 ("Use
github.com/jackc/pgx/v5 or newer instead of other postgres driver
libraries"), which blocks #13 (cmd/tester/migrate.go). Move the whole repo
to pgx v5 and tern v2 rather than suppressing the finding.
API changes:
- github.com/jackc/pgx/v4 -> /v5, pgxpool and pgconn likewise
(github.com/jackc/pgconn is now github.com/jackc/pgx/v5/pgconn).
- github.com/jackc/tern/migrate -> github.com/jackc/tern/v2/migrate.
NewMigrator, AppendMigration, Migrate, MigrateTo, GetCurrentVersion and
OnStart are unchanged in shape; v2 just takes a v5 *pgx.Conn.
- pgxpool.Connect is gone; pgxpool.New connects lazily, so serve (and the
test harness) now Ping after constructing the pool to keep a bad DSN
failing at startup.
- github.com/lib/pq dropped: pq.Array was only used for runs.args
(varchar[]), which v5 encodes and scans as []string natively.
- pgx v5 encodes a nil slice/pointer for a json/jsonb parameter as SQL NULL
where v4 sent JSON null. tests.result, tests.logs and runs.meta are jsonb
NOT NULL and the runner produces nil Logs for a test with no output, so
those columns are now pre-marshalled with encoding/json (db.jsonb) to
keep the v4 wire behaviour. TestPG_AddTest_NoLogs pins this; without the
helper TestPG_ListRunSummariesInRange fails with the NOT NULL violation.
Indirect jackc modules (pgtype, pgproto3, pgio, chunkreader, puddle v1) drop
out of go.mod. .github/dependabot.yml's github.com/jackc/* group already
covers the v5/v2 paths.
Co-authored-by: Cursor <cursoragent@cursor.com>
…PRE_DEPLOY job (MARSOHS-1643) (#13) `tester serve` runs the tern migrations at startup, which means they run inside the web container's health-check window on App Platform. A slow migration (e.g. an index build on the 15 GB tests table) makes the deploy look unhealthy and roll back mid-migration; multiple web replicas serialize on the advisory lock; and the worker component rolls independently, so it can start against a not-yet-migrated schema. A PRE_DEPLOY job runs once, to completion, before any component of the new deployment starts. - `tester migrate`: connects with --pg-dsn / MIGRATE_PG_DSN (falling back to SERVE_PG_DSN so the job can reuse the web env), applies pending migrations, exits 0, non-zero on failure. - Every migration tern applies is now logged (Migrator.OnStart), for both `migrate` and `serve`. - New `serve --migrate-on-start` flag / SERVE_MIGRATE_ON_START env (default true, preserving current behaviour). When false, `serve` skips migrating and instead calls the new `(*db.PG).CheckSchema`, exiting non-zero with the current and expected versions if the schema is behind, so a misordered deploy fails fast rather than serving a stale schema. A newer schema is tolerated so the image can be rolled back. `serve` uses the concrete *db.PG, so the db.DB interface and mock are unchanged. A flag rather than a config.json field so the web component can be switched over with an app-spec env edit, without rebuilding the e2e image. - README: document `tester migrate`, `--migrate-on-start`, an App Platform PRE_DEPLOY job snippet and the rollout order. - Tests: TestPG_CheckSchema (current / behind / ahead) and TestServe_migrateOnStart (default / env / flag precedence). The app-spec change in digitalocean/e2e is a separate, manual step. Co-authored-by: Cursor <cursoragent@cursor.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Story: MARSOHS-1643.
Why
Semgrep (
semgrep-code-digitalocean[bot]) flags any new file importinggithub.com/jackc/pgx/v4— "Use github.com/jackc/pgx/v5 or newer instead of other postgres driver libraries" — which blocks #13 (cmd/tester/migrate.go). This moves the whole repo to pgx v5 and tern v2 rather than suppressing the finding. #13 will be stacked on this branch.API changes
github.com/jackc/pgx/v4→github.com/jackc/pgx/v5(pgxpool,pgconnlikewise; the standalonegithub.com/jackc/pgconnmodule becomesgithub.com/jackc/pgx/v5/pgconn).github.com/jackc/tern/migrate→github.com/jackc/tern/v2/migrate.NewMigrator,AppendMigration,Migrate,MigrateTo,GetCurrentVersionandOnStarthave the same shape (verified againsttern/v2@v2.4.3source); v2 just takes a v5*pgx.Conn.pgxpool.Connectis gone.pgxpool.Newconnects lazily, soserve(and thewithPGtest harness) nowPingright after constructing the pool so a bad DSN still fails at startup.github.com/lib/pqdropped:pq.Arraywas only used forruns.args(varchar(255)[]), which v5 encodes/scans as[]stringnatively. NULL ↔ nil slice behaviour is unchanged.nilslice/pointer passed for ajson/jsonbparameter as SQLNULL; v4 sent JSONnull.tests.result,tests.logsandruns.metaarejsonb NOT NULL, and the runner producesnilLogsfor a test with no output, so under v5 those inserts would fail withnull value in column "logs" ... violates not-null constraint. Those columns are now pre-marshalled withencoding/json(db.jsonbhelper) to keep the v4 wire behaviour.TestPG_AddTest_NoLogspins this; without the helperTestPG_ListRunSummariesInRangefails.jackc/*modules (pgtypev1,pgproto3/v2,pgio,chunkreader,puddlev1) drop out ofgo.mod..github/dependabot.yml'sgithub.com/jackc/*group already matches the v5/v2 paths, so no change there.No changes to
db.DBordb/db_mock.go.Verification
gofmt -l .,go vet ./...,go generate ./... && git diff --exit-code db/db_mock.go,PG_DSN=... go test -race -cover ./...all pass locally against a throwawaypostgres:16(db coverage 60.1%). Built binary:servewith a wrong password now exits 1 at startup with the SASL auth error (thePing); with a good DSN it migrates and serves.