Vandelay

The JMAP importer-exporter
_{One-shot account migration and backup across JMAP, IMAP, CalDAV, CardDAV, WebDAV, ManageSieve, Maildir, Google Takeout and Microsoft Exchange.}

Jerry: Well, what does he do?
George: He's an importer.
Jerry: Just imports, no exports?
George: He's an importer/exporter, okay?

About

Vandelay is a one-shot account-migration utility for JMAP, the JMAP analogue of imapsync generalised to every JMAP data type (mail, contacts, calendars, identities, sieve scripts, file storage). It imports an account from a wide range of source protocols into a local SQLite "archive", then exports that archive into a target JMAP server. Import and export never talk to each other, only to the SQLite archive. One archive holds exactly one account.

Because the archive is a self-contained SQLite file that fully describes one account, vandelay doubles as a per-account backup tool: run an import on a schedule to capture a fresh snapshot, keep the resulting SQLite file as your backup, and restore it later by running an export against a JMAP target.

Features

Many source protocols:
- JMAP
- IMAP
- CalDAV
- CardDAV
- WebDAV
- ManageSieve
- Maildir++
- Google Takeout
- Microsoft Exchange via EWS (experimental)
- Microsoft Exchange Online via Graph (experimental)
One target protocol: JMAP, with type-by-type stateless re-matching on every run.
Convergent: Re-running an interrupted import or export picks up where it left off without bookkeeping flags.
Multi-threaded, no async runtime: Blocking HTTP with per-server concurrency caps respected automatically.
Content-addressed blobs: Emails, sieve scripts and file-storage payloads are stored once by BLAKE3 hash and deduplicated across the archive.
Dry-run everywhere: Every command supports --dry-run to compute the full plan without writing.
Source-change protection: An archive remembers which account it was filled from; pointing it at a different one fails unless explicitly permitted.
Read-only inspection: A built-in inspect command dumps any object type from an archive for verification.

Install

# macOS / Linux
curl --proto '=https' --tlsv1.2 -LsSf \
  https://github.com/stalwartlabs/vandelay/releases/latest/download/vandelay-installer.sh | sh

# Homebrew
brew install stalwartlabs/tap/vandelay

# Windows
powershell -ExecutionPolicy Bypass -c "irm https://github.com/stalwartlabs/vandelay/releases/latest/download/vandelay-installer.ps1 | iex"

# npm
npm install -g @stalwartlabs/vandelay

# From source
cargo install --path .

A signed .msi is also published with each release.

Quick start

A typical run is two commands, one to capture a source account into a local SQLite archive and one to push that archive into a JMAP target.

# 1. Import an IMAP mailbox into a fresh archive.
export VANDELAY_PASSWORD='source-app-password'
vandelay import imap \
  --url imaps://imap.example.com \
  --auth-basic alice@example.com \
  alice.sqlite

# 2. Peek at what landed.
vandelay inspect alice.sqlite                 # per-type summary
vandelay inspect alice.sqlite mailbox         # mailbox tree
vandelay inspect alice.sqlite email --limit 5

# 3. Push the archive into a target JMAP server.
export VANDELAY_PASSWORD='target-password'
vandelay export \
  --url https://jmap.example.org \
  --auth-basic alice@example.org \
  --account-name alice@example.org \
  alice.sqlite

Both commands are convergent: rerun either to resume an interrupted run, or rerun import later to pick up new mail since the last snapshot. Use --dry-run on either side to compute the full plan without writing.

CLI quick reference

vandelay <import|export|inspect> [args...]

All actions accept a set of global flags (verbosity, worker pool size, retry policy, TLS handling) in addition to action-specific ones. The most useful are:

Flag	Purpose
`-j, --threads <N>`	Worker pool size (default: logical CPUs).
`--dry-run`	Compute the full plan; perform no writes.
`-v`, `-vv`, `-vvv`	Increase log verbosity.
`-q, --quiet`	Warnings and errors only.
`--max-retries <N>`	Max retries per request on transient failures (default 5).
`--allow-invalid-certs`	Accept self-signed / invalid TLS certs.

Credentials should be supplied via the VANDELAY_PASSWORD / VANDELAY_TOKEN / VANDELAY_EWS_CLIENT_SECRET / VANDELAY_GRAPH_TOKEN environment variables, or via an interactive prompt; passing them on the command line is supported but not recommended.

Import

vandelay import <source> [source-args...] <ARCHIVE>

Reads a source account into the local SQLite ARCHIVE (created if absent). Every importer accepts --allow-source-change to override the archive's source-identity guard.

JMAP

vandelay import jmap \
  --url <URL> \
  (--auth-basic <USER> [--auth-password <PASS>] | --auth-bearer [TOKEN]) \
  (--account-id <ID> | --account-name <NAME>) \
  [--objects <list>] \
  <ARCHIVE>

Imports a single JMAP account. --objects accepts a comma-separated list of object tokens (mailbox,email,calendar,calendarevent,addressbook,contactcard,identity,sievescript,participantidentity,filenode); default is everything the server advertises.

IMAP

vandelay import imap \
  --url imap(s)://host[:port] \
  (--auth-basic <USER> [--auth-password <PASS>] | --auth-bearer [TOKEN] --auth-user <USER>) \
  [--include <REGEX>...] [--exclude <REGEX>...] [--exclude-special <ROLE>...] \
  [--folder <NAME>...] [--subscribed-only] [--noautomap] \
  [--include-deleted] [--allow-cleartext] [--compress] \
  [--fetch-batch <N>] [--imap-connections <1..8>] \
  <ARCHIVE>

Imports mail (and only mail) from any IMAP server. Folder selection is via --include/--exclude regexes (mutually exclusive with the exact-match --folder); --exclude-special drops by SPECIAL-USE role.

CalDAV

vandelay import caldav \
  --url <http(s)://host[/path]> \
  (--auth-basic <USER> [--auth-password <PASS>] | --auth-bearer [TOKEN]) \
  [--allow-cleartext] [--dav-connections <1..8>] [--multiget-batch <N>] \
  <ARCHIVE>

Discovers the user's CalDAV principal (or accepts a URL pointing straight at a calendar-home or calendar), then imports calendars and events.

CardDAV

vandelay import carddav \
  --url <http(s)://host[/path]> \
  (--auth-basic <USER> [--auth-password <PASS>] | --auth-bearer [TOKEN]) \
  [--allow-cleartext] [--dav-connections <1..8>] [--multiget-batch <N>] \
  <ARCHIVE>

Same shape as caldav, but for address books and contacts.

WebDAV

vandelay import webdav \
  --url <http(s)://host[/path]> \
  (--auth-basic <USER> [--auth-password <PASS>] | --auth-bearer [TOKEN]) \
  [--allow-cleartext] [--dav-connections <1..8>] [--multiget-batch <N>] \
  <ARCHIVE>

Imports a plain WebDAV file collection as a JMAP FileNode tree.

ManageSieve

vandelay import managesieve \
  --url sieve(s)://host[:port] \
  (--auth-basic <USER> [--auth-password <PASS>] | --auth-bearer [TOKEN] --auth-user <USER>) \
  [--allow-cleartext] \
  <ARCHIVE>

Imports sieve scripts only. Each script is content-addressed in the blob table; the active script is recorded.

Maildir

vandelay import maildir <MAILDIR> <ARCHIVE> \
  [--include <REGEX>...] [--exclude <REGEX>...] [--folder <NAME>...] \
  [--noautomap] [--include-deleted]

Reads a local Maildir++ tree (a directory with cur/, new/, tmp/). No network. Folder selection mirrors the IMAP importer.

Google Takeout

vandelay import takeout <PATH> <ARCHIVE> [--noautomap]

Scans a directory tree recursively for .mbox, .ics and .vcf files and imports them. Tailored to Google Takeout layouts but works on any such tree; system-label role assignment can be disabled with --noautomap.

Microsoft Exchange (EWS)

vandelay import exchange-ews \
  [--url <EWS-ENDPOINT>] [--mailbox <SMTP>] \
  [--mailbox-kind primary|archive|public-folders] \
  (--auth-basic <USER> [--auth-password <PASS>] \
   | --auth-bearer [TOKEN] [--ews-tenant <T> --ews-client-id <ID> \
                            (--ews-device-code | --ews-client-secret <SECRET>)]) \
  [--ews-connections <1..8>] [--ews-getitem-batch <N>] [--ews-attachment-batch <N>] \
  [--ews-no-syncfolderitems] \
  <ARCHIVE>

Imports a mailbox via EWS, against either on-prem Exchange Server or Exchange Online. Autodiscover is used when --url is omitted (a --mailbox SMTP address is then required). Supports Basic, pre-acquired bearer, interactive device-code OAuth, and app-only client-credentials OAuth.

Microsoft Exchange (Graph)

vandelay import exchange-graph \
  (--client-id <UUID> [--tenant <ID>] | --access-token [TOKEN]) \
  [--user <UPN|UUID>] \
  [--mailbox-kind primary|archive] \
  [--objects mail,calendar,contacts] \
  [--event-body-format text|html] \
  [--graph-connections <1..16>] [--top <1..1000>] \
  <ARCHIVE>

Imports a mailbox from Exchange Online via Microsoft Graph. Without --access-token, the interactive device-code flow is used. public-folders is rejected here (use exchange-ews instead).

Export

vandelay export \
  --url <URL> \
  (--auth-basic <USER> [--auth-password <PASS>] | --auth-bearer [TOKEN]) \
  (--account-id <ID> | --account-name <NAME>) \
  [--objects <list>] [--prune [--yes]] \
  <ARCHIVE>

Stateless re-export of ARCHIVE into a target JMAP server account. The default behaviour is upsert-only: matched items are updated, unmatched local items are created, but pre-existing target items not covered by the archive are left alone.

--prune enables destructive reconciliation: target objects that do not match anything in the archive are deleted. The confirmation prompt can be skipped with --yes for automation. Export speaks JMAP only; no other target protocols are currently supported.

Inspect

vandelay inspect <ARCHIVE> [TYPE] [--failures|--failed-items] [--json] [--limit <N>] [--offset <N>]

Read-only dump of a local archive. This command never opens a network connection and never modifies archive contents (it does apply the schema on open, so an older archive gains any table added since it was written).

Omit TYPE for a per-type summary (counts of every object kind, blob storage stats, and the number of unresolved export failures).
Pass an object type to dump it: mailbox, email, identity, sievescript, addressbook, contactcard, calendar, calendarevent, participantidentity, filenode.
mailbox and filenode render as a tree (--limit/--offset are ignored); all other types use a paginated list and respect --limit and --offset.
--failures dumps the per-item export quarantine instead of an object type (the two cannot be combined). export writes one row per object the target refused, for every object type it exports and not just mail — archive-local id, creation id, size and full blake3 of the object's blob where it has one, message-id for a message, the blobId the target returned, its error type and detail, and the blob-retrievability verdict — so the items behind a failed=N summary can be identified without scraping warnings off stderr. (The failed counters also cover two things that are not per-item and so have no row: a type whose whole pass aborted, and a --prune destroy that the target rejected.) A row is replaced on the next attempt and removed once the item reaches the target, so what remains is exactly the unresolved set. An export also drops rows whose object is no longer in the archive at all — a re-import renumbers local ids, and a row naming a dead one describes an item nothing can identify any more — so a re-imported archive does not leave phantom failures behind. The archive is opened with an exclusive lock, so read it between runs rather than during one. --failed-items is an alias of --failures.
--json renders that dump as a single JSON object for a control plane instead of the columnar text an operator reads: {"failed_items":[{"type":"Email","id":"e499","message_id":"<a@example.com>","size":73932,"content_hash":"…","blob_id":"ebn0","error":"blobNotFound","detail":"BlobId ebn0 not found.","blob_probe":"orphaned_marker","failed_at":"…"}],"omitted":[]}. type, id and error are present and non-empty on every row; a field the archive has no value for is left out rather than emitted empty, so a reader never stores "" as if it were a recorded fact. failed_items is always present and always an array — an archive with nothing quarantined emits {"failed_items":[],"omitted":[]} and exits 0, never the human "(no export failures in archive)" notice and never an empty body, because a caller has to be able to tell "nothing failed" apart from "this binary does not implement the dump". type is the JMAP type name of an object that names a single transferable item. A refusal that is not one — a mailbox, an address book, a calendar, a file node — is withheld from failed_items and counted in omitted with its archive token and the reason, since it is already visible in the run's failed=N line and has no per-item identity a consumer could act on. --json requires the failure dump and is rejected on its own.

Testing

The default suite is hermetic (unit tests plus mockito-scripted JMAP/DAV/EWS/Graph behaviours) and needs no network or Docker:

cargo build
cargo clippy --all-targets
cargo test

Live and integration tests (Docker required)

Live integration tests against a Stalwart server and container-based tests against third-party servers (Dovecot, Cyrus, Radicale, Baikal, Apache mod_dav) are gated behind --ignored. They require a running Docker daemon: each test binary boots its own throwaway container via testcontainers (images are pulled automatically on first run), so Docker must be installed and docker info must succeed before invoking them.

Run them per binary, and always with --test-threads=1:

cargo test --test sync_jmap          -- --ignored --test-threads=1   # live JMAP import/export/convergence/prune
cargo test --test sync_imap          -- --ignored --test-threads=1
cargo test --test sync_managesieve   -- --ignored --test-threads=1
cargo test --test sync_maildir       -- --ignored --test-threads=1
cargo test --test sync_caldav        -- --ignored --test-threads=1
cargo test --test sync_carddav       -- --ignored --test-threads=1
cargo test --test sync_webdav        -- --ignored --test-threads=1
cargo test --test live_stalwart      -- --ignored --test-threads=1
cargo test --test seed_smoke         -- --ignored --test-threads=1
cargo test --test seed_only          -- --ignored --test-threads=1

# Third-party-server tests (one container each):
cargo test --test integration_radicale -- --ignored --test-threads=1
cargo test --test integration_baikal   -- --ignored --test-threads=1
cargo test --test integration_webdav   -- --ignored --test-threads=1
cargo test --test integration_dovecot  -- --ignored --test-threads=1
cargo test --test integration_cyrus    -- --ignored --test-threads=1

# Slow tests
cargo test --test mock_jmap -- --ignored

--test-threads=1 is mandatory, not just advisory: within a binary every test shares a single per-binary container, and each test provisions then tears down the same disposable vandelay.org domain (and opens the archive with SQLite EXCLUSIVE locking). Separate binaries are isolated (each boots its own container on dynamic host ports), so plain cargo test --test <name> invocations are safe to run one after another.

License

Licensed under either of

Apache License, Version 2.0 (LICENSE-APACHE or http://www.apache.org/licenses/LICENSE-2.0)
MIT license (LICENSE-MIT or http://opensource.org/licenses/MIT)

at your option.

Name		Name	Last commit message	Last commit date
Latest commit History 44 Commits
.github/workflows		.github/workflows
LICENSES		LICENSES
assets		assets
resources		resources
scripts		scripts
src		src
tests		tests
wix		wix
.gitignore		.gitignore
CHANGELOG.md		CHANGELOG.md
Cargo.lock		Cargo.lock
Cargo.toml		Cargo.toml
README.md		README.md
dist-workspace.toml		dist-workspace.toml
rust-toolchain.toml		rust-toolchain.toml

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Repository files navigation

Vandelay

The JMAP importer-exporter
_{One-shot account migration and backup across JMAP, IMAP, CalDAV, CardDAV, WebDAV, ManageSieve, Maildir, Google Takeout and Microsoft Exchange.}

About

Features

Install

Quick start

CLI quick reference

Import

JMAP

IMAP

CalDAV

CardDAV

WebDAV

ManageSieve

Maildir

Google Takeout

Microsoft Exchange (EWS)

Microsoft Exchange (Graph)

Export

Inspect

Testing

Live and integration tests (Docker required)

License

Copyright

About

Releases

Packages

Contributors

Languages

Folders and files

Latest commit

History

Repository files navigation

Vandelay

The JMAP importer-exporter One-shot account migration and backup across JMAP, IMAP, CalDAV, CardDAV, WebDAV, ManageSieve, Maildir, Google Takeout and Microsoft Exchange.

About

Features

Install

Quick start

CLI quick reference

Import

JMAP

IMAP

CalDAV

CardDAV

WebDAV

ManageSieve

Maildir

Google Takeout

Microsoft Exchange (EWS)

Microsoft Exchange (Graph)

Export

Inspect

Testing

Live and integration tests (Docker required)

License

Copyright

About

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages

The JMAP importer-exporter
_{One-shot account migration and backup across JMAP, IMAP, CalDAV, CardDAV, WebDAV, ManageSieve, Maildir, Google Takeout and Microsoft Exchange.}