imap-poller

A small Go library that keeps IMAP mailboxes synced: it authenticates, runs IDLE to receive pushed EXISTS notifications, and does incremental UID-based fetch of new messages, tracking a per-mailbox sync cursor so it never re-downloads what it has already seen. It is transport-agnostic and storage-agnostic — you bring the dialer, the token source, and the cursor storage; the library runs the protocol.

Extracted and generalized from a production mail-sync system.

Why

The naïve way to watch a mailbox is to poll SEARCH/FETCH on a timer, which is slow and wasteful. The right way is IMAP IDLE (the server pushes when new mail arrives) plus UID-based incremental fetch (you ask only for messages with a UID greater than the last one you stored). Doing that correctly means handling UIDVALIDITY resets, reconnect/backoff, OAuth token refresh mid-session, and making sure a handler error doesn't advance the cursor past an unprocessed message. This library handles those; you supply the small pieces that are specific to your app.

What you bring (the interfaces)

Everything app-specific is an interface, so the library depends on no database, no proxy, and no particular auth source:

Interface	You implement	Provided default
Dialer	how a TCP/TLS connection to the server is opened	TLSDialer (net.Dial + crypto/tls)
Authenticator	how the session authenticates	XOAUTH2Auth and PasswordAuth (SASL)
TokenProvider	supplies a fresh OAuth access token (+ expiry) per account	—
CursorStore	loads/saves the {UIDVALIDITY, LastUID} cursor per mailbox	—
AccountStore	lists accounts to watch and records liveness/status	—
MessageHandler	called with each new message	—

Because Dialer is an interface, routing connections through a proxy, a bastion, or a custom transport is just a different Dialer — the library neither knows nor cares.

Usage

The library exposes composable primitives — pick the layer you need. MessageHandler is func(ctx, imappoller.Message) error; returning an error leaves the cursor un-advanced so the message is retried.

One-shot / periodic incremental sweep of one account — Fetcher.PollOnce:

f := &imappoller.Fetcher{
    Address:  "imap.example.com:993",
    Dialer:   imappoller.TLSDialer{},                        // default when nil
    Auth:     imappoller.XOAUTH2Auth{Tokens: myTokenProvider}, // or PasswordAuth{Passwords: ...}
    Cursors:  myCursorStore,
    Accounts: myAccountStore,
    OnNewMessage: func(ctx context.Context, m imappoller.Message) error {
        log.Printf("[uid %d] %s — %v", m.UID, m.Subject, m.From)
        return nil
    },
}
n, err := f.PollOnce(ctx, accountID) // fetches only messages with UID > the stored cursor

Keep a fleet of accounts live over IDLE — Supervisor spawns a Session per mailbox and reacts to server pushes:

sup := &imappoller.Supervisor{
    Accounts:  myAccountStore,
    Mailboxes: []string{"INBOX"},
    NewSession: func(accountID int64, mailbox string) imappoller.Runner {
        return &imappoller.Session{
            AccountID: accountID, Mailbox: mailbox, Address: "imap.example.com:993",
            Auth:    imappoller.XOAUTH2Auth{Tokens: myTokenProvider},
            Cursors: myCursorStore, Accounts: myAccountStore,
            OnNewMessage: handle,
        }
    },
}
go sup.Run(ctx)   // reconciles the desired account set on a timer; sup.Poke() forces one now

Correctness details it handles

• Incremental sync by UID. Fetches UID > LastUID only; the cursor is the sole source of truth.
• UIDVALIDITY resets. If the server rotates UIDVALIDITY, the cursor is invalidated and the mailbox re-syncs from scratch rather than silently missing mail.
• Handler-safe cursor advance. The cursor only moves past a message after OnNewMessage returns nil, so a failed handler retries that message instead of losing it.
• IDLE lifecycle. Connect → auth → catch-up fetch → IDLE → on EXISTS, re-fetch → re-IDLE, with bounded teardown.
• Token refresh mid-session. Reconnects proactively before the OAuth token expires (expiry comes from TokenProvider; a zero expiry disables proactive refresh).
• Backoff + classification. Distinguishes auth failures from network failures and backs off appropriately; a Supervisor ramps and caps concurrency and has a staleness backstop.

Dependencies

• github.com/emersion/go-imap/v2 (+ go-sasl) for the IMAP protocol. Everything else is the standard library.

Testing

go test ./...          # 39 tests, run against an in-process TLS IMAP server
go test -race ./...

The tests spin up a real in-process TLS IMAP server that honors UID FETCH ranges, so incremental sync, UIDVALIDITY resets, IDLE/EXISTS round-trips, and token-refresh reconnects are exercised end to end.

License

MIT — see LICENSE.