Defer instead of delivering unsigned

[gurdiga]

Both mail containers start opendkim as an unsupervised background process and then exec postfix start-fg, so postfix is PID 1 and restart: always never fires if the milter dies. With milter_default_action=accept, postfix responds to an unreachable milter by delivering anyway — unsigned, with ordinary status=sent lines and no warning. Nothing in the stack would tell us: smtp-out has healthcheck: disable: true, and a signing outage shows up only as slow reputation decay.

That matters more than it looks, because feedsubscription.com publishes p=reject; adkim=s; aspf=s. Unsigned mail we hand to the recipient MX ourselves still passes DMARC on SPF alone, so direct delivery looks fine; mail that gets forwarded loses SPF, has no DKIM left to fall back on, and is rejected outright. Same for sandradodd.com through postilion.

So this switches both to milter_default_action=tempfail, which is Postfix’s own default and was explicitly overridden here. An unreachable milter now gets a 451 at MAIL FROM instead of a free pass. That is safe for the newsletter path because failed sends stay in the outbox — sendOutboxEmails does report.failed++; continue before postfixEmailMessage, so the item is never purged — and checkFeeds runs hourly and calls deliverItems whether or not the feed produced new items, so a stuck outbox drains on the next hour rather than the next daily send. The cost of a milter outage becomes an hour’s delay plus a "failed":N sending report. For postilion the clients are interactive submitters, so a 451 surfaces to whoever sent the message.

The second commit replaces sleep 0.5 with a real check that opendkim bound 127.0.0.1:8891. opendkim daemonizes, and the parent exits 0 whether or not the child reaches LISTEN, so set -euo pipefail does not catch a failed bind — postfix would start with nothing on 8891, which under tempfail means deferring everything instead of failing loudly at startup.

Worth being precise about what this does not change: a milter that is up but holds an unusable key already tempfails per message, from opendkim itself rather than from this setting. Verified by pointing KeyTable at a nonexistent key — 451 4.7.1 at end-of-data under both accept and tempfail. The gap was only ever the unreachable-milter case.

Verified against locally built images. With the milter up and a client on the compose network (172.18.0.3, non-loopback and inside mynetworks, as prod’s app is at 10.5.5.100), a normal transaction:

S: 220 feedsubscription.com ESMTP Postfix (Debian/GNU)
S: 250 CHUNKING
S: 250 2.1.0 Ok
S: 250 2.1.5 Ok

Then killing opendkim inside the container and repeating from the same non-loopback client:

S: 250 CHUNKING
S: 451 4.7.1 Service unavailable - try again later
S: 503 5.5.1 Error: need MAIL command

And the startup guard, with opendkim given a socket it cannot bind:

[smtp-out] Starting opendkim
[smtp-out] ERROR: opendkim did not bind 127.0.0.1:8891
--- exit code: 1 ---

postilion builds and starts the same way with the same settings.

One consequence to go in with eyes open: ssmtp on the host has mailhub=localhost:1587, so make watch-app alerts relay through smtp-out too. During a milter outage they get 451’d along with everything else, and the operator signal is the daily sending report going missing plus the error lines in the logs, not an alert in the inbox. Under accept those alerts would go out — but the signing failure itself generates no error to alert about, which is the whole problem.

The third commit exists because Codex pointed out, correctly, that tempfail on its own trades a silent failure for a stuck one: postfix is PID 1, so opendkim dying at runtime leaves the container up and every message 451ing forever, with restart: always never firing and the hourly retry unable to recover. supervise_milter polls the socket every 30s and stops postfix when it goes, which turns that into a restart. Verified end to end with --restart always: killing opendkim in a running container took it down and brought it back with the milter listening again.

restarts before: 0
restarts after: 1, state: running
milter back up? 1

Two edges of that guarantee, for whoever reads this next: the watchdog covers opendkim dying, not itself dying, since it is an ordinary background process with no supervision of its own; and stopping postfix this way exits 0 without a logged shutdown, so treat it as a stop rather than a drain — the queue is a bind mount, so anything in flight is on disk and gets retried after the restart.

A healthcheck on 8891 is still deliberately absent — Docker doesn’t restart unhealthy containers and nothing here monitors health, so it would add a signal nobody reads, where the watchdog actually recovers.

Deploying restarts both containers, so it wants the usual empty-queue window and a DKIM canary afterwards. Signing on the current smtp-out image was verified end to end today, so any difference after the swap is this change.