perf(migrations): build the trgm and scope indexes concurrently

[zkrebbekx]

Closes #611.

The defect

Three released migrations build indexes on flexitype_attribute_value — always the largest table — with a plain CREATE INDEX, which takes a lock conflicting with every write and holds it for the whole build:

file	index
000004_outbox_search	idx_flexitype_attribute_value_trgm, ..._trgm_lower
000014_scoped_values	idx_flexitype_attribute_value_scope
000021_plan_indexes	the same trigram pair, rebuilt after 000018 dropped them

#595 established why "grandfathered" was the wrong answer: a deployment upgrading across the version still pays the stall, and bookkeeping records the version with no checksum, so correcting an applied migration is a no-op for those who have it and a fix for those who have not.

The tracker listed 000004 and 000014. 000021 belongs with them — it rebuilds the same pair, so it is the copy that survives a full upgrade chain.

Why none of them could be fixed in place

• 000004 creates tables and 000014 alters two. A no-transaction file may not do either, because it is replayed in full after an interruption.
• 000004 and 000021 must make the trigram build conditional: gin_trgm_ops does not exist where pg_trgm could not be installed. The only conditional form in SQL is a DO block, a DO block is a transaction, and CREATE INDEX CONCURRENTLY refuses to run inside one. 000021 says exactly this, at length, and takes the plain build as the lesser evil.

That last one is the real blocker, and it is not a SQL problem — it is a runner problem.

The fix

New file directive. -- +flexitype:requires-extension <name>. The runner checks pg_extension before applying, and where the extension is absent it skips the file without recording it. A database that installs the extension a week later applies the file on its next start; recording it as applied would mean the index is never built and nothing says so. The skip is announced on stderr, because a silently absent index reads as a planner that has simply got worse.

The name must be one bare identifier. A directive line carrying anything else would read as an extension nobody has, and skip the file for ever.

Two new migrations. 000041 builds both trigram indexes CONCURRENTLY, gated on pg_trgm. 000042 builds the scoped-value index CONCURRENTLY. Both are IF NOT EXISTS, so a database that already holds the index from 000004, 000014 or 000021 sees a no-op. The three originals keep their table DDL and their extension-creation attempt, and hand the builds over with a comment saying why.

000021's bespoke invalid-namesake reaper went with the builds it guarded. It existed because the runner's own reap covers CREATE INDEX CONCURRENTLY statements only, and those builds were not concurrent. Now they are, so the runner covers them — including the schema-qualification the in-file version had to hand-roll.

grandfatheredPlainIndexes drops from seven files to five.

Tests

TestExtensionGatedMigration (goconvey, Postgres):

• The directive parses alongside no-transaction.
• An installed extension (plpgsql, present everywhere) applies; an absent one skips rather than fails — failing would wedge every later migration on a managed provider.
• Three malformed directive lines are refused.
• On a fully migrated database, version 41 is recorded exactly when pg_trgm is installed, and both trigram indexes plus the scope index exist and are indisvalid.
• The upgrade shape: delete the 41/42 rows while the indexes exist, re-run — clean, indexes untouched.
• Down to 40 and back up: the revert drops all three, the re-apply rebuilds them. This is what catches a down file that still names an index its up file no longer creates.

TestEmbeddedMigrationDirectives now strips comments before looking for CONCURRENTLY — a file that merely names the migration its index moved to declares nothing to the runner.

Mutation-verified: recording a skipped file, accepting any directive text, and gutting the 000042 down file each fail 5 assertions.

Docs: docs/upgrades.md gains the directive, the reason it exists, and rule 7 — an index needing an optional extension goes in its own gated file rather than inside a DO block.

Green: full Go suite, golangci-lint clean.

🤖 Generated with Claude Code

https://claude.ai/code/session_01DaPmWmgaGYqLsjW8DCU3P1