Skip to content

Fix broken AVX2 cmov that skips FO implicit rejection (key recovery) - #119

Closed
007bsd wants to merge 1 commit into
Argyle-Software:masterfrom
007bsd:fix-avx2-cmov-fo-rejection
Closed

007bsd wants to merge 1 commit into
Argyle-Software:masterfrom
007bsd:fix-avx2-cmov-fo-rejection

Conversation

@007bsd

@007bsd 007bsd commented Aug 16, 2026

Copy link
Copy Markdown

Problem

The AVX2 cmov in src/avx2/verify.rs builds its blend mask with _mm256_set1_epi64x(b as i64). For b == 1 each 64-bit lane is 0x00000000_00000001, so every byte's high bit is 0 and _mm256_blendv_epi8 copies nothing. The vector loop is a no-op for any b.

That cmov is the Fujisaki-Okamoto implicit-rejection step in crypto_kem_dec (called with len == KYBER_SYMBYTES == 32, entirely through the vector loop, no scalar tail). On an invalid ciphertext the rejection secret is never applied, so the returned shared secret depends on the decrypted plaintext of the attacker-chosen ciphertext. That is a chosen-ciphertext plaintext-checking oracle and allows recovery of a reused static secret key. Verified end to end on ML-KEM-768 (4,272 decapsulation queries, full key recovered). Only the opt-in avx2 feature is affected; the reference backend is correct.

Fix

Negate b once to 0x00/0xFF and broadcast per byte with _mm256_set1_epi8, matching the pq-crystals reference AVX2 code. The scalar tail then reuses the already-negated b, so its previous b = b.wrapping_neg() is removed to avoid a double negation.

Note: the crate appears unmaintained (last release 0.7.1, August 2023). Filing this fix for the public record and as the upstream reference for a RustSec advisory.

The AVX2 cmov built its blend mask with _mm256_set1_epi64x(b as i64), so for
b == 1 every byte's high bit is 0 and _mm256_blendv_epi8 copies nothing. cmov
is the Fujisaki-Okamoto implicit-rejection step in decapsulation (called with
len == 32, entirely through the vector loop), so on an invalid ciphertext the
rejection value was never applied, turning decapsulation into a chosen-
ciphertext plaintext-checking oracle and enabling static key recovery.

Fix: negate b once to 0x00/0xFF and broadcast per byte with _mm256_set1_epi8,
matching the pq-crystals reference; drop the now-redundant scalar-tail negation.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant