Undocumented support for migrations (workaround)

[KristerV]

Problem

It's common to need to do migrations in the database and convert data between encrypted fields with the following conditions:

1. elixir is not installed (iex/mix not available)
2. migration is fully automated with rollback support
3. migration shouldn't start the whole runtime

Unfortunately this process is not documented and trying it myself resulted in the same error as danielberkompas/cloak#125 .

TLDR encrypt_existing_data.md solution is suboptimal.

Migration steps

I want to change a DateTime field to Date.

1. load records
2. decrypt field
3. use DateTime.to_date()
4. encrypt
5. update

Workaround

I did manage to read through a bunch of cloak's code and come up with a solution.

1. use MyApp.Vault.start_link() to start the Vault in Release (solves ETS error when running Vault without mix cloak#125).
2. use load() to decrypt your data.
3. use dump() to encrypt data.

Here's my full(ish) code, because I hate it when I find a solution online and a crucial piece is left out.

defmodule MyApp.Repo.Migrations.AlterTransactionDatetime do
  use Ecto.Migration
  import Ecto.Query, warn: false
  alias MyApp.Repo
  alias MyApp.Encryption.Types

  def up do
    dates =
      from(t in "table", select: {t.id, t.datetime})
      |> Repo.all()
      |> Enum.map(fn {id, datetime_encrypted} ->
        {:ok, datetime} = Types.DateTime.load(datetime_encrypted)
        {:ok, new_val} = datetime |> DateTime.to_date() |> Types.Date.dump()
        {id, new_val}
      end)

    alter table(:table) do
      remove :datetime
      add :date, :binary
    end

    flush()

    for {id, date} <- dates do
      from(t in "table", where: t.id == ^id, update: [set: [date: ^date]])
      |> Repo.update_all([])
    end
  end
end

defmodule MyApp.Release do
  @moduledoc """
  Used for executing DB release tasks when run in production without Mix
  installed.
  """
  @app :my_app

  def migrate do
    MyApp.Vault.start_link()
    load_app()

    for repo <- repos() do
      {:ok, _, _} = Ecto.Migrator.with_repo(repo, &Ecto.Migrator.run(&1, :up, all: true))
    end
  end

  [...]
end

Suggestions

1. There should be an actual migration guide, like the above (or better since i don't know if this is how the package is supposed to be used). and encrypt_existing_data.md imho should be deprecated in it's current form.
2. Document crucial functions like start_link, load and dump. Nothing fancy, for starters that they exist at all.
3. Ideally the package would check if Vault is running (or if the :ets key exists) so ETS error when running Vault without mix cloak#125 would have a nicer error.

[danielberkompas]

cloak version 1.1.4 (just released) will raise a helpful error if the vault hasn't been started, which will solve part of this.