Add HPKE test vectors by sgmenda · Pull Request #273 · C2SP/wycheproof

sgmenda · 2026-08-20T19:04:52Z

HPKE (RFC 9180) base-mode test vectors for DHKEM(X25519, HKDF-SHA256) and ML-KEM-512/768/1024 (draft-ietf-hpke-pq-05). See doc/hpke.md.

For each KEM, under testvectors_v1/: hpke_{KEM}_encap_test.json (SetupBaseS on valid and invalid recipient keys), hpke_{KEM}_decrypt_test.json (SetupBaseR + Open), and hpke_{KEM}_sequence_test.json (ordered Open of several messages on one context). Each has its own schema.

Several cases come from bugs found in other implementations; each flag's notes entry links the advisory or fix.

Generated with AWS-LC, with cases it can't produce sealed by a separate RFC 9180 implementation. Every file is also checked by at least one other implementation (BoringSSL, Bouncy Castle, Go crypto/hpke, pyca).

Not included: X-Wing and Exporter (I punted them to follow-up PR), PSK/auth modes, other KEMs and hybrids, DeriveKeyPair vectors, and MessageLimitReachedError.

Wycheproof test vectors for HPKE (RFC 9180) base mode, covering DHKEM(X25519, HKDF-SHA256) and ML-KEM-512/768/1024 (draft-ietf-hpke-pq). See doc/hpke.md for detailed documentation. **For reviewers:** - we mark X25519 all-zero-DH keys as *invalid* per RFC 9180 §5.1.1, which is stricter than RFC 7748 and than `x25519_test.json`, which marks them **acceptable**. - we generated these vectors using aws-lc and checked them against pyca/cryptography, Go crypto/hpke, and BoringSSL everywhere they overlap. - one particular gap is HPKE with ML-KEM-512 which only aws-lc implements.

cpu · 2026-08-21T14:10:32Z

Thank you! Adding HPKE coverage is something I've talked about with Filippo but neither of us had gotten to it yet. I won't be able to review this until the ~end of next week but I'm very supportive.

As a side note I think the upstream hpkewg is interested in finding a home for their vectors (hpkewg/hpke#72). Did you want to reach out and see if they're interested in joining forces and using c2sp/wycheproof for that home? They also recently added some new corner cases (hpkewg/hpke#70), and if you haven't already considered those they might be helpful additions.

sgmenda · 2026-08-24T01:24:15Z

@cpu thanks for the pointers, I will reach out.

My initial instinct was those vectors should go in CCTV. But from the perspective of a crypto library, it's easier to track one external test vector library than two (Wycheproof and CCTV). @FiloSottile I'm curious for your thoughts?

cpu · 2026-08-31T20:18:32Z

My initial instinct was those vectors should go in CCTV. But from the perspective of a crypto library, it's easier to track one external test vector library than two (Wycheproof and CCTV)

Yeah I think the boundary of which belongs where is a little fuzzy. Beyond the benefit of one-source-of-truth Wycheproof 's JSON schemas make life easier for downstreams 🤷‍♂️ I'd be happy with either.

The encap schema is shared by the ML-KEM and DHKEM(X25519) files, so its skRm and ikmE descriptions must cover both: skRm is a 64-byte d || z seed for ML-KEM but a 32-byte scalar for DHKEM(X25519). Also drop a stray hyphen in doc/hpke.md that broke a "deterministic-encapsulation" line.

The doc claimed only AWS-LC implements ML-KEM in HPKE. That is wrong: BoringSSL implements ML-KEM-768 and 1024 (not 512), and Bouncy Castle implements all three. Verified by running the vectors against Bouncy Castle (116/116 ML-KEM cases agree) and by inspecting BoringSSL's HPKE KEM set. Reword to name the implementations and describe the uneven support instead of the inaccurate single-implementation claim.

Two additions inspired by the hpkewg edge-case vectors (hpkewg/hpke#70), staying within base mode and the existing X25519 / ML-KEM suites. Sequence vectors: new hpke_{x25519,mlkem512,mlkem768,mlkem1024}_sequence_test files (group type HpkeSequenceTest) exercise HPKE's stateful, ordered Open. A case runs SetupBaseR once and opens an ordered `decryptions` list, so the AEAD nonce evolves as base_nonce XOR sequenceNumber. Besides the in-order sequence, the negatives cover out-of-order, replayed, and skipped ciphertexts, which single-shot cases cannot reach. The decrypt schema now accepts either group type (one per file, as the linter requires). Embedded zero bytes: the decryption files gain valid cases whose info, aad, or pt contain interior NUL bytes (distinct from the empty-value cases), catching consumers that mishandle embedded zeros. Produced deterministically with AWS-LC and independently verified: the single-shot cases with pyca/cryptography, the sequence cases with Bouncy Castle's stateful HPKE context (which, unlike pyca, implements ML-KEM-512).

Three fixes found proofreading the PR: - doc said "two test vector files" per KEM; there are now three (add the sequence file to the list). - the ML-KEM-512 decrypt header still called the file an "only AWS-LC" single-implementation canary; Bouncy Castle also implements ML-KEM-512 in HPKE (and now cross-checks these), matching the corrected doc. Reword. - align the HpkeSequenceTestGroup kem enum with HpkeTestGroup (the same forward-looking KEM list) instead of only the four shipped KEMs.

sgmenda · 2026-09-18T21:52:53Z

I have pushed a major revision.

@jakemas thanks for the suggestions. now, every vector in this PR has been checked by at least two implementations.

@cpu thanks for the pointer to the hpkewg edge-case vectors. now, this PR includes two new classes of vectors:

vectors which test that the stateful Open correctly accepts valid sequences and rejects invalid ones.
vectors which test that zero-bytes in info/aad/pt are handled correctly

I have not imported the hpkewg vectors, since they would have expanded the scope significantly to include P-256, PSK, and exporting. Instead, I took inspiration from their edge case vectors to ensure that our coverage is a superset (for the schemes in scope).

cpu

Thanks! I think these are almost ready to go, I just had a few items of feedback to consider first.

I was able to write a downstream test using this branch for the Go HPKE impl of ML-KEM-768, ML-KEM-1024, & X25519 (we don't have ML-KEM-512) and everything produced the expected results. 👍

cpu · 2026-09-22T13:56:57Z

+      "source": {
+        "name": "wycheproof",
+        "version": "0.1"
+      },


Can you make the source name more specific throughout? I think something like github/sgmenda/hpke-wycheproof would be better if you're open to it.

github/sgmenda/hpke-wycheproof sounds like a reference to the non-existent github.com/sgmenda/hpke-wycheproof repo.

How about github/C2SP/wycheproof/pull/273?

cpu · 2026-09-22T14:16:15Z

+              "ikmE": {
+                "type": "string",
+                "format": "HexBytes",
+                "description": "Optional. The 32-byte seed that fixes the sender's ephemeral key. For ML-KEM it is the encapsulation randomness (ikmE); for DHKEM(X25519) it is the sender's ephemeral private key skEm, i.e. the seed a deterministic-encapsulation test hook consumes, not RFC 9180 DeriveKeyPair ikm. When present, deterministic encapsulation from this seed reproduces enc and ct exactly, so an implementation with such a hook can also check the sender direction. Absent for negative cases, where enc/ct are mutated and are not reproducible."


This feels a little bit awkward to me because it results in an ikmE with semantics that differ based on the algorithm. For the Go stdlib HPKE the ML-KEM cases were OK, but for the X25519 cases I couldn't just feed the value through our derandomization helper because it calls DeriveKeyPair and that's already been done, instead I have to switch on the algorithm and directly use NewPrivateKey for the bytes for X25519.

ISTM that putting the RFC 9180 ikmE value here across the board is more general and BoringSSL-family consumers can do the derivation to get skEm themselves (and/or we could have an optional field alongside the ikmE that has the expected skEM for them).

WDYT?

I agree.

ikmE is now the RFC 9180 value everywhere.

But I chose to keep skRm/skEm unclamped. This is consistent with RFC 9180 Appendix A and draft-ietf-hpke-hpke-05 7.1.2. But, it differs from Go's PrivateKey.Bytes() which returns different, clamped bytes. doc/hpke.md discuss this in detail.

cpu · 2026-09-22T14:17:04Z

+      "description": "A valid HPKE base-mode ciphertext with deterministic encapsulation from ikmE (the sender's ephemeral private key)."
+    },
+    "SmallOrderEnc": {
+      "bugType": "CAN_OF_WORMS",


The counterpart ZeroSharedSecret note on the encap tests use DEFINED for this. Should we be consistent here?

When I looked at it again, I realized these were actual bugs: SmallOrderEnc is AUTH_BYPASS (anyone can forge a message to the recipient) and ZeroSharedSecret is CONFIDENTIALITY (the sender encrypts under a publicly computable key). I also now link to RUSTSEC-2026-0072.

cpu · 2026-10-12T12:34:57Z

@sgmenda Gentle ping on this branch. I'd like to use these tests downstream for the Go stdlib HPKE and it'd be easier if it was merged :-) Do you think you'll be able to take a pass at the review feedback?

sgmenda · 2026-10-23T17:28:18Z

@cpu oops, I missed this ping. addressing feedback now.

Fix the issues cpu raised in review, plus others found while going over the vectors again. Fixes: - Correct RFC 9180 citations, error-type claims, flags and bugTypes. - Use RFC 9180 ikmE semantics for X25519 encapsulation and add skEm. - Use a source name that links to the originating PR. - Document that X25519 skRm and skEm are unclamped (as in RFC 9180 Appendix A and draft-ietf-hpke-hpke-05) and how implementations differ when serializing them. New vectors: - Sequence vectors get their own schema, with retry-after-failed-Open cases and an X25519 HKDF-SHA256/ChaCha20Poly1305 group. - X25519 cases for bugs found in other implementations. - ML-KEM implicit-rejection, Strcmp, MalleableCiphertext and modulus-overflow cases.

sgmenda · 2026-10-26T18:56:41Z

@cpu thanks for the detailed feedback. It took me down many rabbit holes, but I think I addressed them.

cpu · 2026-10-26T19:57:30Z

Thanks, I'll try to give this another pass soon but I'm very busy this week and have some travel obligations so there might be a longer than usual delay.

sgmenda · 2026-10-26T21:44:47Z

@cpu no worries at all.

jakemas reviewed Aug 24, 2026

View reviewed changes

Comment thread doc/hpke.md Resolved

Comment thread doc/hpke.md Resolved

Comment thread schemas/hpke_encap_test_schema.json Resolved

sgmenda mentioned this pull request Sep 1, 2026

Add ML-KEM support to HPKE (draft-ietf-hpke-pq-05) aws/aws-lc#3277

Merged

sgmenda added 5 commits September 18, 2026 12:41

Merge remote-tracking branch 'upstream/main' into hpke-mlkem-vectors

ce527d6

cpu reviewed Sep 22, 2026

View reviewed changes

cpu mentioned this pull request Oct 7, 2026

Advance the HPKE sequence only after open succeeds rustls/rustls#3340

Merged

Merge remote-tracking branch 'upstream/main' into hpke-mlkem-vectors

55ac5c2

sgmenda added 2 commits October 26, 2026 10:52

Merge branch 'main' into hpke-mlkem-vectors

9c9fd48

sgmenda added 2 commits October 26, 2026 12:07

fix docs

b7fb00c

more doc edits

11238a4

Conversation

sgmenda commented Aug 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cpu commented Aug 21, 2026

Uh oh!

sgmenda commented Aug 24, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

cpu commented Aug 31, 2026

Uh oh!

sgmenda commented Sep 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cpu left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

cpu Sep 22, 2026

Choose a reason for hiding this comment

Uh oh!

sgmenda Oct 26, 2026

Choose a reason for hiding this comment

Uh oh!

cpu Oct 26, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

cpu Sep 22, 2026

Choose a reason for hiding this comment

Uh oh!

sgmenda Oct 26, 2026

Choose a reason for hiding this comment

Uh oh!

cpu Sep 22, 2026

Choose a reason for hiding this comment

Uh oh!

sgmenda Oct 26, 2026

Choose a reason for hiding this comment

Uh oh!

cpu commented Oct 12, 2026

Uh oh!

sgmenda commented Oct 23, 2026

Uh oh!

sgmenda commented Oct 26, 2026

Uh oh!

cpu commented Oct 26, 2026

Uh oh!

sgmenda commented Oct 26, 2026

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

sgmenda commented Aug 20, 2026 •

edited

Loading

sgmenda commented Sep 18, 2026 •

edited

Loading