Added support for XOAUTH2 authorization, and documented in base.pod

[desirider]

Hi John,

I've added and tested support for XOAUTH2 authorization protocol. Also added documentation, with Gmail as an example. The XOAUTH2 protocol requires an access token, and I am passing it via the -ap argument.

Tested it several times on my local machine.

Thanks,
Desirider.

[polarathene]

OAUTHBEARER is intended to replace XOAUTH2, but may take a while for that transition to be more widespread. Dovecot supports both.

• XOAUTH2 (Googles widely adopted implementation, which alpine mail client docs refer to as deprecated) and OAUTHBEARER (the newer variant standardized by RFC 7628 in 2015).
• Both are supported as different ways to verify authorization against the identity provider via the same OAuth Bearer Token (RFC 6750, 2012).

---

EDIT: Presently curl will work as shown below, but only when used within the v13.3 DMS container. Newer versions of curl have a bug where only OAUTHBEARER is used.

DMS recently added support for these in Dovecot with the v13.3 release.

• BATS OAuth2 test.
• IMAP AUTHENTICATE example commands used to test auth against Dovecot:
  • XOAUTH2
  • OAUTHBEARER
• Dovecot would then verify via bearer token HTTP Authorization header request to a mocked identity provider (DIY compose.yaml + Caddyfile usable without BATS, or equivalent setup in the related BATS test.

---

Offline test environment with Docker Compose + DMS

If you need a popular production instance to test against, DMS makes this simple (it already bundles the latest swaks from Github Releases):

# Normally this would provide TLS configured for both services, that's been omitted to keep the reproduction simple
services:
  # Quick and easy mailserver setup with Postfix + Dovecot for testing OAuth2 support
  dms:
    image: docker.io/mailserver/docker-mailserver:13.3
    container_name: dms-mail
    hostname: mail.example.test
    environment:
      # Enable the OAuth2 support in Dovecot and configure it for our mocked service (caddy):
      ENABLE_OAUTH2: 1
      OAUTH2_INTROSPECTION_URL: http://auth.example.test/userinfo/
    # Test authentication against these ports:
    ports:
      - "143:143" # IMAP STARTTLS (Dovecot)
      - "587:587" # SMTP STARTTLS (Postfix)
    configs:
      - source: dms-accounts
        target: /tmp/docker-mailserver/postfix-accounts.cf

  # This would normally be a proper auth service, this is sufficient to mock out the required behaviour for testing
  caddy-oauth2:
    image: caddy:2.7
    container_name: dms-oauth2
    # Leverage Docker's internal DNS for the private network bridge it creates between services:
    hostname: auth.example.test
    ports:
      - "80:80"
    configs:
      - source: mock-auth-service
        target: /etc/caddy/Caddyfile

# Using the Docker Compose `configs.content` feature instead of volume mounting separate files.
# NOTE: This feature requires Docker Compose v2.23.1 (Nov 2023) or newer:
# https://github.com/compose-spec/compose-spec/pull/446
configs:
  # Basic Caddyfile example, see a better documented equivalent at:
  # https://github.com/docker-mailserver/docker-mailserver/blob/v13.3.0/test/config/oauth2/Caddyfile
  mock-auth-service:
    content: |
      :80 {
        @auth header Authorization "Bearer DMS_YWNjZXNzX3Rva2Vu"

        handle @auth {
          respond `{ "email": "john.doe@example.test", "email_verified": true }`
        }
        # Otherwise fail when expected auth header and value were not matched:
        respond 401 {
          close
        }
      }
  # DMS expects an account to be configured to run, this config provides one
  # You can add new accounts with `docker compose exec dms setup email add user@example.test bad-password`
  # Login credentials:
  # user: "john.doe@example.test" password: "secret"
  # user: "jane.doe@example.test" password: "secret"
  dms-accounts:
    # NOTE: `$` needed to be repeated to escape it,
    # which opts out of the `compose.yaml` variable interpolation feature.
    content: |
      john.doe@example.test|{SHA512-CRYPT}$$6$$sbgFRCmQ.KWS5ryb$$EsWrlYosiadgdUOxCBHY0DQ3qFbeudDhNMqHs6jZt.8gmxUwiLVy738knqkHD4zj4amkb296HFqQ3yDq4UXt8.
      jane.doe@example.test|{SHA512-CRYPT}$$6$$o65y1ZXC4ooOPLwZ$$7TF1nYowEtNJpH6BwJBgdj2pPAxaCvhIKQA6ww5zdHm/AA7aemY9eoHC91DOgYNaKj1HLxSeWNDdvrp6mbtUY.

Commands

Generate the auth strings with base64 encoding via CLI if necessary:

These base64 encoded values will appear in the SMTP protocol exchange as part of SMTP `AUTH` / IMAP `AUTHENTICATE` commands

# In DMS Postfix delegates SMTP AUTH to Dovecot via SASL
# Dovecot expects to receive the XOAUTH2 / OAUTHBEARER auth string encoded as base64,
# which it decodes and verifies the bearer token at the configured OAuth2 service endpoint (eg: `/userinfo`)
# via an HTTP request with an Authorization header (of Bearer type).

# Encoding the XOAUTH2 auth string as base64:
$ echo -en 'user=john.doe@example.test\001auth=Bearer DMS_YWNjZXNzX3Rva2Vu\001\001' | base64 -w0; echo
dXNlcj1qb2huLmRvZUBleGFtcGxlLnRlc3QBYXV0aD1CZWFyZXIgRE1TX1lXTmpaWE56WDNSdmEyVnUBAQ==

# OAUTHBEARER equivalent:
$ echo -en 'n,a=john.doe@example.test,\001host=localhost\001port=143\001auth=Bearer DMS_YWNjZXNzX3Rva2Vu\001\001' | base64 -w0; echo
bixhPWpvaG4uZG9lQGV4YW1wbGUudGVzdCwBaG9zdD1sb2NhbGhvc3QBcG9ydD0xNDMBYXV0aD1CZWFyZXIgRE1TX1lXTmpaWE56WDNSdmEyVnUBAQ==

# Equivalent values for jane.doe@example.test with the same Access Token
# XOAUTH:
dXNlcj1qYW5lLmRvZUBleGFtcGxlLnRlc3QBYXV0aD1CZWFyZXIgRE1TX1lXTmpaWE56WDNSdmEyVnUBAQ==
# OAUTHBEARER:
bixhPWphbmUuZG9lQGV4YW1wbGUudGVzdCwBaG9zdD1sb2NhbGhvc3QBcG9ydD0xNDMBYXV0aD1CZWFyZXIgRE1TX1lXTmpaWE56WDNSdmEyVnUBAQ==

---

Test commands (run on the host system after a docker compose up --force-recreate):

# Testing against the mocked endpoint directly (which is what Dovecot is responsible for handling):
# - Within the DMS container this would be to http://auth.example.test/userinfo
# - Otherwise from the host system reach the caddy container via the published port on localhost
# In this case Dovecot is configured by default to validate successful auth by matching the
# returned `email` field  against the `user` field (provided via the auth string).
curl http://localhost:80/userinfo -H 'Authorization: Bearer DMS_YWNjZXNzX3Rva2Vu' -w '\n'
# Response: { "email": "john.doe@example.test", "email_verified": true }

# Testing through Postfix SMTP AUTH via the proposed swaks options of this PR:
# `swaks` is also available within the container (at `/usr/local/bin/swaks`),
# use it via `docker compose exec dms swaks ...`
swaks --server localhost:587 \
  --from john.doe@example.test \
  --to jane.doe@example.test \
  --auth XOAUTH2 \
  -au john.doe@example.test \
  -ap DMS_YWNjZXNzX3Rva2Vu

For comparison here is the equivalent with curl:

# Set `--login-options` to either 'AUTH=XOAUTH2' or 'AUTH=OAUTHBEARER'.
# Both are valid and the DMS container logs from Postfix will indicate the correct method like:
# `sasl_method=OAUTHBEARER, sasl_username=john.doe@example.test`
# The curl output itself also shows authentication success during the SMTP protocol.
#
# NOTE: If running on the host replace `mail.example.test` with `localhost`.
# NOTE: Technically `--upload-file` expects a proper input with RFC 5322 headers:
# https://everything.curl.dev/usingcurl/smtp
curl --silent --verbose \
  --url 'smtp://mail.example.test:587' \
  --user 'john.doe@example.test' \
  --login-options 'AUTH=XOAUTH2' \
  --oauth2-bearer 'DMS_YWNjZXNzX3Rva2Vu' \
  --mail-from 'john.doe@example.test' \
  --mail-rcpt 'jane.doe@example.test' \
  --upload-file <<< 'Hello Jane!'

Output

*   Trying 127.0.0.1:587...
* Connected to localhost (127.0.0.1) port 587 (#0)
< 220 mail.example.test ESMTP
> EHLO polarathene-laptop
< 250-mail.example.test
< 250-PIPELINING
< 250-SIZE 10240000
< 250-ETRN
< 250-AUTH PLAIN LOGIN OAUTHBEARER XOAUTH2
< 250-AUTH=PLAIN LOGIN OAUTHBEARER XOAUTH2
< 250-ENHANCEDSTATUSCODES
< 250-8BITMIME
< 250-DSN
< 250 CHUNKING
> AUTH OAUTHBEARER
< 334
> bixhPWpvaG4uZG9lQGV4YW1wbGUudGVzdCwBaG9zdD1sb2NhbGhvc3QBcG9ydD0xNDMBYXV0aD1CZWFyZXIgRE1TX1lXTmpaWE56WDNSdmEyVnUBAQ==
< 235 2.7.0 Authentication successful
> MAIL FROM:<john.doe@example.test>
< 250 2.1.0 Ok
> RCPT TO:<jane.doe@example.test>
< 250 2.1.5 Ok
> DATA
< 354 End data with <CR><LF>.<CR><LF>
} [12 bytes data]
< 250 2.0.0 Ok: queued as 11E3418A56
* Connection #0 to host localhost left intact

NOTE: Instead of < 250 2.0.0 Ok: queued as 11E3418A56 you may get < 521 5.5.2 mail.example.test Error: bare <LF> received which is due to a recent security feature that expects CRLF for SMTP and curl is not properly handling.

This security check can be ignored when performed within the DMS container and trust is given to mail clients running within that container by adding the ENV PERMIT_DOCKER=container. However that is not relevant to the auth test reproduction demonstrated

[felixmarch]

Wonder if there are any updates on this? 🤔

# dnf install postfix cyrus-sasl cyrus-sasl-plain sasl-xoauth2 swaks
# sudo -u postfix sasl-xoauth2-tool get-token outlook /etc/postfix/oauth_tokens/sender@example.com--client-id=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx --tenant=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx --use-device-flow
# swaks --server smtp.office365.com --port 587 --ehlo smtp.office365.com \
      --tls \
      --from 'sender@example.com' \
      --to 'recipient@example.com' \
      --auth XOAUTH2 \
      --auth-user 'sender@example.com' \
      --auth-password "`cat /etc/postfix/oauth_tokens/sender@example.com | jq -r .access_token`" \
      --header "Subject: $(date)" --body "This is a test mail"
*** XOAUTH2 is not a recognized auth type, skipping
*** No auth types supported

Is there new release planned after v20240103.0?

Cos Microsoft will completely disable the basic auth soon starting end of 2026 on new tenants, and mid 2027 for the rest.

https://techcommunity.microsoft.com/blog/exchange/updated-exchange-online-smtp-auth-basic-authentication-deprecation-timeline/4489835

[Neustradamus]

@jetmore: Any progress on this PR?

[minusdavid]

You could leverage Authen::SASL::Perl for both XOAUTH2 and OAUTHBEARER:

https://metacpan.org/release/EHUELS/Authen-SASL-2.2000/source/lib/Authen/SASL/Perl

[polarathene]

@minusdavid I think that isn't necessary, given the small part of this PR that implements the equivalent?

We are just waiting on @jetmore to eventually (hopefully) return to his project here with time to spare for review and publishing a new release with the feature.

[minusdavid]

@polarathene Fair enough!

I'd read somewhere that swaks uses Authen::SASL for the DIGETS-MD5 mechanism, and I know Authen::SASL::Perl comes with XOAUTH2 and OAUTHBEARER as well. But looking at do_smtp_auth_digest I suppose using Authen::SASL::Perl might not get you much. Plus, Authen::SASL defaults to Authen::SASL::XS anyways and for that you would need something like one of the plugins I mentioned on that nylas project.

[minusdavid]

That said, in #44 (comment) the suggestion is to install sasl-xoauth2 anyway, so it would be available to Authen::SASL if either Authen::SASL::XS or Authen::SASL::Cyrus was installed, and then you'd be able to leverage sasl-xoauth2 for fully.

But... yeah you're right. It's simple enough that you wouldn't need Authen::SASL.

[minusdavid]

Actually, that reminds me that XOAUTH2 and OAUTHBEARER weren't added to Authen::SASL::Perl until 2.1800 anyway, which is only in Debian Forky/Ubuntu 26.04 onwards...