Switch to Mailman 3

[jhenstridge]

Mailman 2 relies on an end-of-life Python version and is effectively on life support. We should switch to Mailman 3.

This is a non-trivial change, since the architecture is quite different (a couple of Django web apps talking to a REST API server), and data (list information and archived emails) is stored in a database.

We'd need to be certain that we can reliably back up and restore such a system.

It would also be good to have a plan to keep the archive URLs working. This could go one of two ways:

1. Keep the old Pipermail archives around.
2. Determine the Message-ID for each pipermail URL, and to build a redirect map to equivalent HyperKitty archive URLs (which use a hash of the message ID).

[jhenstridge]

As a sketch of how the archive URLs could be rewritten if we wanted, consider the following two URLs that refer to the same message.

• https://mail.python.org/pipermail/python-list/2025-April/913216.html
• https://mail.python.org/archives/list/python-list@python.org/message/ODWB5HWMUA4ZRYW4NZCOLRAJICNUC7AD/

In the Pipermail page, we see the following in the HTML:

<LINK REL="made" HREF="mailto:python-list%40python.org?Subject=Re%3A%20Python%203.14.0a7%2C%203.13.3%2C%203.12.10%2C%203.11.12%2C%0A%203.10.17%20and%203.9.22%20are%20now%20available&In-Reply-To=%3CCANKMJBNhrcNS%2BgPG6DbinW0efwTg5xOans3-%2B3yO0Fksb9TE6g%40mail.gmail.com%3E">

If we URL decode the In-Reply-To parameter of the mailto URL, we get <CANKMJBNhrcNS+gPG6DbinW0efwTg5xOans3-+3yO0Fksb9TE6g@mail.gmail.com>. Drop the angle brackets and we've got the message ID.

The ID hyperkitty is using is a base32 encoded SHA1 hash of the message ID. If we apply that algorithm to our message ID, we get the slug at the end of the Hyperkitty URL:

>>> import hashlib, base64
>>> msg_id = b'CANKMJBNhrcNS+gPG6DbinW0efwTg5xOans3-+3yO0Fksb9TE6g@mail.gmail.com'
>>> base64.b32encode(hashlib.sha1(msg_id).digest())
b'ODWB5HWMUA4ZRYW4NZCOLRAJICNUC7AD'

It shouldn't be too difficult to write a script that extracts the message IDs from each Pipermail html page for a list archive, then constructs the equivalent Hyperkitty URL. This could then be used to generate a redirect map for the web server.

I guess one concern would be whether Nginx could efficiently handle that many redirects itself.