Summary
I maintain a fork at kno/amule that implements the
wire-level extensions an eMuleAI peer expects, so that aMule and eMuleAI
recognise each other's capabilities instead of falling back to the plain eD2k
subset. It is built on 3.0.1 and tracks this repository.
The code for everything described here, including the rendezvous framing
discussed in this thread, lives on the
integration/network-parity
branch. The fork's master mirrors this repository plus downstream tooling
only, so a branch cut from it starts clean.
Before preparing any pull request I would rather ask whether you want this at
all, and under what conditions — particularly the one in "Open question" below,
which decides whether two of the pieces are even offerable.
What it consists of
Seven pieces, each with its own tests, in dependency order:
| # |
Piece |
Depends on |
| 1 |
Peer capability recognition — the CT_MOD_MISCOPTIONS tag and its bits, plus demultiplexing of OP_UDPRESERVEDPROT2 frame types |
— |
| 2 |
Kademlia protocol 0x0a — version advertisement and the AICH gate |
— |
| 3 |
Address widening — a family-agnostic address type, so an IPv6 peer is not a peer with "no address" |
— |
| 4 |
IPv6 dual stack — one listening socket per family; reachability advertised only after inbound IPv6 is observed |
3 |
| 5 |
uTP transport — sharing the eD2k UDP port |
1, 3 |
| 6 |
NAT traversal — answering a relayed rendezvous, rate-limited, with bounded hole punches |
1, 5 |
| 7 |
QUIC NAT-T transport |
6 |
1 and 2 are small and self-contained. 3 is the one that unblocks the rest and
also the most invasive of the early ones.
Why the formats can be trusted
These are not guesses at eMuleAI's wire format. They were measured against a
running eMuleAI v1.6.0:
CT_MOD_MISCOPTIONS reads 0x0000001F from it — every bit of the mask this
work defines, matching bit for bit.
- The
EAQN1 proof magic and the ALPN string ed2k-ai-natt-quic-v1 both appear
verbatim in its binary.
- Its GnuTLS priority string offers the same four TLS 1.3 ciphers in the same
order, over a subset of the same groups.
- An eD2k handshake over native IPv6 completes between the two
implementations — a hand-built OP_HELLO to its port answered with
OP_HELLOANSWER.
Open question, and the reason for asking first
Piece 5 vendors libutp under src/extern/, as a byte-identical copy of
transmission/libutp at 490874c44a2e
(MIT), with no local patches and a provenance note. Pieces 5, 6 and 7 have no
meaning without a uTP implementation.
To be precise about what is and is not proposed: the bundled copy is the
default, not the only route. cmake/libutp.cmake offers both, following the
same bundled-or-system convention this project already uses for picojson
(USE_SYSTEM_PICOJSON):
USE_SYSTEM_LIBUTP=OFF (default) — the vendored copy.
USE_SYSTEM_LIBUTP=ON — a system-installed libutp; fails loudly if absent,
rather than silently downgrading to a green build with the transport missing at
runtime.
The default is the bundled copy because libutp is not packaged anywhere I can
find. Measured rather than assumed:
- Debian trixie: no
libutp-dev, no libutp, and apt-cache search returns
nothing; sources.debian.org reports no exact match.
- Homebrew: no formula (
brew search libutp offers only unrelated names).
So making the system route the default would break the build on most machines
with a FATAL_ERROR asking the user to install something that does not exist as
a package.
Vendoring is still a project-policy decision rather than an engineering one, which
is why I would rather hear your position than write a pull request that assumes
it. If it is unacceptable, uTP does not disappear — it becomes conditional on a
library your users would have to build by hand, and that changes what pieces 5–7
can honestly promise.
One property worth stating because it is what makes the copy maintainable: it is
byte-identical to the pinned upstream commit, with no local patches, and the
vendored CMakeLists.txt is not edited either — the build adapts to upstream's
target name instead. A patched third-party file turns every version bump into a
merge and makes the provenance note untrue.
What I would not offer yet
Being explicit so this is not discovered in review:
- QUIC (piece 7) is not ready. It compiles, negotiates TLS, classifies frames
and advertises its capability bit, but no QUIC connection has ever completed
with another implementation, and one decision inside its authentication proof
was made without evidence. I would not ask anyone to maintain that yet.
- NAT traversal (piece 6) answers but does not initiate. Choosing a relay is
deliberately gated off, because whether a candidate relay can reach a firewalled
peer is not knowable from the requesting side. IPv6 hole punches encode and parse
but are not sent.
- Nothing in the fork's own packaging, container images or translation catalogs is
proposed — those are downstream concerns and would conflict with your Weblate
pipeline.
What I am asking
- Is this direction something you want in aMule at all?
- What is your position on vendoring libutp under
src/extern/?
- If yes to (1): would you prefer one piece at a time in the order above,
starting with capability recognition, or a different shape?
Happy to adapt to whatever review process suits you, including splitting
pieces further.
Test plan
Each piece carries unit tests. The current fork tip is at 73/73 in both build
configurations, with CI green across Ubuntu, macOS and mingw-w64 (Debug and
Release), plus clang-tidy and clang-format. Per-piece test counts and evidence
would come with each pull request.
Summary
I maintain a fork at kno/amule that implements the
wire-level extensions an eMuleAI peer expects, so that aMule and eMuleAI
recognise each other's capabilities instead of falling back to the plain eD2k
subset. It is built on 3.0.1 and tracks this repository.
The code for everything described here, including the rendezvous framing
discussed in this thread, lives on the
integration/network-paritybranch. The fork's
mastermirrors this repository plus downstream toolingonly, so a branch cut from it starts clean.
Before preparing any pull request I would rather ask whether you want this at
all, and under what conditions — particularly the one in "Open question" below,
which decides whether two of the pieces are even offerable.
What it consists of
Seven pieces, each with its own tests, in dependency order:
CT_MOD_MISCOPTIONStag and its bits, plus demultiplexing ofOP_UDPRESERVEDPROT2frame types0x0a— version advertisement and the AICH gate1 and 2 are small and self-contained. 3 is the one that unblocks the rest and
also the most invasive of the early ones.
Why the formats can be trusted
These are not guesses at eMuleAI's wire format. They were measured against a
running eMuleAI v1.6.0:
CT_MOD_MISCOPTIONSreads0x0000001Ffrom it — every bit of the mask thiswork defines, matching bit for bit.
EAQN1proof magic and the ALPN stringed2k-ai-natt-quic-v1both appearverbatim in its binary.
order, over a subset of the same groups.
implementations — a hand-built
OP_HELLOto its port answered withOP_HELLOANSWER.Open question, and the reason for asking first
Piece 5 vendors libutp under
src/extern/, as a byte-identical copy oftransmission/libutp at
490874c44a2e(MIT), with no local patches and a provenance note. Pieces 5, 6 and 7 have no
meaning without a uTP implementation.
To be precise about what is and is not proposed: the bundled copy is the
default, not the only route.
cmake/libutp.cmakeoffers both, following thesame bundled-or-system convention this project already uses for picojson
(
USE_SYSTEM_PICOJSON):USE_SYSTEM_LIBUTP=OFF(default) — the vendored copy.USE_SYSTEM_LIBUTP=ON— a system-installed libutp; fails loudly if absent,rather than silently downgrading to a green build with the transport missing at
runtime.
The default is the bundled copy because libutp is not packaged anywhere I can
find. Measured rather than assumed:
libutp-dev, nolibutp, andapt-cache searchreturnsnothing;
sources.debian.orgreports no exact match.brew search libutpoffers only unrelated names).So making the system route the default would break the build on most machines
with a
FATAL_ERRORasking the user to install something that does not exist asa package.
Vendoring is still a project-policy decision rather than an engineering one, which
is why I would rather hear your position than write a pull request that assumes
it. If it is unacceptable, uTP does not disappear — it becomes conditional on a
library your users would have to build by hand, and that changes what pieces 5–7
can honestly promise.
One property worth stating because it is what makes the copy maintainable: it is
byte-identical to the pinned upstream commit, with no local patches, and the
vendored
CMakeLists.txtis not edited either — the build adapts to upstream'starget name instead. A patched third-party file turns every version bump into a
merge and makes the provenance note untrue.
What I would not offer yet
Being explicit so this is not discovered in review:
and advertises its capability bit, but no QUIC connection has ever completed
with another implementation, and one decision inside its authentication proof
was made without evidence. I would not ask anyone to maintain that yet.
deliberately gated off, because whether a candidate relay can reach a firewalled
peer is not knowable from the requesting side. IPv6 hole punches encode and parse
but are not sent.
proposed — those are downstream concerns and would conflict with your Weblate
pipeline.
What I am asking
src/extern/?starting with capability recognition, or a different shape?
Happy to adapt to whatever review process suits you, including splitting
pieces further.
Test plan
Each piece carries unit tests. The current fork tip is at 73/73 in both build
configurations, with CI green across Ubuntu, macOS and mingw-w64 (Debug and
Release), plus clang-tidy and clang-format. Per-piece test counts and evidence
would come with each pull request.