Skip to content

Schema Migrations

Jump to bottom
Petrus Pradella edited this page Jul 15, 2026 · 6 revisions

Schema Migrations

What this page covers: evolving a backend's schema with the optional SchemaAwareStorage capability — register(...).migrate(), the forward-only model, the _schema_migrations ledger, and the backend base classes (SqlMigration, MongoMigration, LocalFileMigration, GroupedFileMigration, InMemoryMigration) plus getNativeClient for full control. It also explains how migrations relate to the automatic table creation SQL does on its own.

📌 Note — this is the DDL axis: it versions the collection's structure. Two other, orthogonal axes also say "version": Optimistic Locking versions one row to resolve concurrent writes, and Payload Schema Evolution versions one entity's field shape to upcast rows written by an older build. An entity can carry any combination of the three.

The 30-second version

Migrations are a capability, expressed as the interface schema.SchemaAwareStorage. Register your migrations and call migrate():

import br.com.finalcraft.everydatabase.*;
import br.com.finalcraft.everydatabase.schema.SchemaAwareStorage;
import br.com.finalcraft.everydatabase.modules.sql.SqlMigration;

// A migration: a version, a description, and what to do.
public final class V001_CreatePlayers extends SqlMigration {
    public static final V001_CreatePlayers INSTANCE = new V001_CreatePlayers();
    private V001_CreatePlayers() {}

    public String version()     { return "001"; }
    public String description() { return "Create players table"; }
    public String upScript() {
        return "CREATE TABLE IF NOT EXISTS players ("
             + "  uuid VARCHAR(36) NOT NULL,"
             + "  name VARCHAR(64) NOT NULL,"
             + "  PRIMARY KEY (uuid))";
    }
}

// Register + apply (idempotent: already-applied migrations are skipped).
storage.init().join();
if (storage instanceof SchemaAwareStorage schema) {
    schema.register(V001_CreatePlayers.INSTANCE, V002_AddLastSeen.INSTANCE)
          .migrate()
          .join();
}

migrate() applies pending migrations in version order, records each in a reserved _schema_migrations ledger, and skips anything already applied. Running it again is a no-op.

📌 Note — migrate() returns a CompletableFuture<Void>; the example uses .join() for brevity. If a migration fails, the future completes exceptionally and the sequence aborts — migrations after the failing one are not applied. See The Async API.

Capabilities are interfaces, not flags

There's no storage.supportsMigrations() boolean. A backend that can track and apply migrations implements SchemaAwareStorage; narrow to it with instanceof before calling register/migrate:

if (storage instanceof SchemaAwareStorage schema) {
    schema.register(V001.INSTANCE, V002.INSTANCE).migrate().join();
}

When you already hold a concrete type from a typed factory (e.g. Storages.createSQL(...) returns a SqlStorage, which is a SchemaAwareStorage), you can skip the check:

SqlStorage sql = Storages.createSQL(new SqlConfig("jdbc:mariadb://localhost/mc", "root", "pass"));
sql.init().join();
sql.register(V001_CreatePlayers.INSTANCE).migrate().join();

The interface surface:

public interface SchemaAwareStorage extends Storage {
    SchemaAwareStorage register(List<Migration> migrations);
    default SchemaAwareStorage register(Migration... migrations);   // varargs shorthand
    CompletableFuture<SchemaVersion>     currentVersion();          // lexicographically greatest applied, or none()
    CompletableFuture<List<Migration>>   pending();                 // registered-but-not-applied, in order
    CompletableFuture<Void>              migrate();                  // apply all pending, in order
}

📌 Note — register(...) must be called before migrate(), returns this for chaining, and accumulates across calls. Migrations are sorted by version() automatically, so registration order doesn't matter.

Forward-only, by design

There is intentionally no downScript() / rollback migration. Rollback-by-reverse-migration is an anti-pattern in production; to undo something, write a compensating forward migration (V005_DropDeprecatedColumn). Each migration applies exactly once and is recorded.

A Migration is identified by a lexicographically sortable version() string — "001", "002", or dates like "2024-01-15". It must be unique per storage, and ordering is natural string order, so zero-pad numeric versions ("010", not "10") — "10" sorts before "9". A duplicate version is rejected at register().

The _schema_migrations ledger

Each backend records which versions it has applied in its own reserved location:

Backend Where applied versions are tracked
MySQL / MariaDB · PostgreSQL · H2 a _schema_migrations table
MongoDB a _schema_migrations collection
LocalFile · GroupedFile a metadata file
InMemory an ephemeral in-memory list (dies with the instance)

This is what makes migrate() idempotent and safe to call on every startup: it consults the ledger, applies only the pending entries, and records each success. Query the state without applying anything:

SchemaVersion v = schema.currentVersion().join();   // e.g. SchemaVersion{002}, or none() == "0"
List<Migration> todo = schema.pending().join();     // empty == up to date

📌 Note — currentVersion() reports the lexicographically greatest applied version, not whichever was applied last. This holds on every backend, so it's stable even if migrations were applied out of version order (e.g. a lower version registered and run after a higher one). Zero-pad numeric versions so "greatest" matches your intended ordering.

⚠️ Gotcha — if a migration throws, the runner wraps the exception, aborts the sequence, and the future completes exceptionally. Migrations after the failing one are not applied; ones before it stay recorded as applied. Fix the cause and re-run migrate() — it resumes from the first unapplied version.

Writing a migration: the backend base classes

Implement the right base class for your backend; each unwraps the native client for you. For full control (multi-statement SQL, mixed operations, your own Mongo session), implement Migration.execute(MigrationContext) directly and pull the native client yourself. Besides the three below, the file/memory backends provide GroupedFileMigration and InMemoryMigration — both, like LocalFileMigration, hand you the Storage itself via executeOnStorage(...).

SQL — extend SqlMigration, implement upScript()

One DDL/DML statement, no trailing semicolon. The base class opens a Statement on the transaction's Connection and runs it:

public final class V002_AddLastSeen extends SqlMigration {
    public static final V002_AddLastSeen INSTANCE = new V002_AddLastSeen();
    private V002_AddLastSeen() {}

    public String version()     { return "002"; }
    public String description() { return "Add last_seen column"; }
    public String upScript() {
        return "ALTER TABLE players ADD COLUMN last_seen BIGINT";
    }
}

💡 Tip — for multiple statements or procedural logic, override execute(MigrationContext) and call context.getNativeClient(Connection.class) yourself rather than cramming several statements into one upScript().

MongoDB — extend MongoMigration, implement executeOnDatabase(MongoDatabase)

Mongo migrations are always code-based (no script concept) — you write Java against the MongoDatabase:

import com.mongodb.client.MongoDatabase;
import com.mongodb.client.model.Indexes;

public final class V002_AddNameIndex extends MongoMigration {
    public static final V002_AddNameIndex INSTANCE = new V002_AddNameIndex();
    private V002_AddNameIndex() {}

    public String version()     { return "002"; }
    public String description() { return "Add ascending index on name in player_data"; }

    protected void executeOnDatabase(MongoDatabase db) {
        db.getCollection("player_data")
          .createIndex(Indexes.ascending(COL_DATA + ".name"));   // COL_DATA == "storage_data"
    }
}

MongoMigration exposes the document field names the repository uses, so you can reach into stored blobs without the package-private class: COL_KEY ("_id" — Mongo stores the serialized entity key as the document _id) and COL_DATA ("storage_data", the JSON entity blob).

⚠️ Gotcha — executeOnDatabase runs outside a client session; the runner does not wrap migrations in a transaction. Multi-document transactions need a replica set anyway. For transactional safety inside a migration, override execute(MigrationContext) and start your own session.

LocalFile — extend LocalFileMigration, implement executeOnStorage(LocalFileStorage)

You get the LocalFileStorage itself — use storage.repository(descriptor) for high-level CRUD, or storage.baseDirectory() for raw file work:

import java.nio.file.*;

public final class V001_FixCorruptedFiles extends LocalFileMigration {
    public static final V001_FixCorruptedFiles INSTANCE = new V001_FixCorruptedFiles();
    private V001_FixCorruptedFiles() {}

    public String version()     { return "001"; }
    public String description() { return "Delete files with corrupted JSON"; }

    protected void executeOnStorage(LocalFileStorage storage) throws Exception {
        Path dir = storage.baseDirectory().resolve("playerdata");
        Files.walk(dir, 1)
             .filter(p -> p.toString().endsWith(".json") && isCorrupted(p))
             .forEach(p -> { try { Files.delete(p); } catch (IOException ignored) {} });
    }
}

Full control: MigrationContext.getNativeClient

Every base class is sugar over Migration.execute(MigrationContext). Implement it directly and pull the typed native handle when you need more than the base class offers:

<T> T getNativeClient(Class<T> type);
Connection    conn = context.getNativeClient(Connection.class);       // SQL
MongoDatabase db   = context.getNativeClient(MongoDatabase.class);     // Mongo
LocalFileStorage s = context.getNativeClient(LocalFileStorage.class);  // LocalFile

⚠️ Gotcha — requesting a type the backend doesn't provide throws IllegalArgumentException, which the runner surfaces as a migration failure. Ask each backend for its native type only.

Migrations vs. automatic table creation (SQL)

These are complementary, not competing. SqlStorage auto-creates the entity table on the first repository(descriptor) call (createTableIfAbsent — idempotent, also reconciles declared indexes), so for a brand-new collection you often need no migration at all. Migrations handle what auto-create can't infer: backfilling data, renaming columns, adding non-index constraints, custom DDL, data fixes.

🧭 Decision — let auto-create handle the initial table + declared indexes; reach for a migration when you must transform existing data or apply schema changes the descriptor doesn't express. The two run independently — migrate() doesn't replace auto-create, and auto-create doesn't record anything in _schema_migrations.

📌 Note — every persistent backend implements SchemaAwareStorage: SQL (all dialects), Mongo, LocalFile, and GroupedFile. In-Memory implements it too, but its ledger is ephemeral — it dies with the instance, so a fresh instance re-applies every migration (correct, since data and ledger reset together; the point on InMemory is data-seeding/transform migrations, DDL being meaningless). A reused instance behaves the same: close() resets the ephemeral ledger and the data together, so close() + init() re-applies the registered migrations from none() — the registrations survive; only the applied-ledger and data reset. See Choosing a Backend.

See also

EveryDatabase · Home · made by Petrus Pradella

EveryDatabase

🏠 Home

Getting Started

Core Concepts

Working with Data

Backends

Manager Module

Operations

Advanced

Reference

Contributing

Clone this wiki locally