ML-DSA: Refactor sample ops dispatch to use a vtable

[mdcornu]

PR to address feedback from upstream OpenSSL review: openssl#30654

The initial implementation dispatched matrix_expand_A, vector_expand_S, and vector_expand_mask via global function pointers initialized with CRYPTO_ONCE.

Changes:

• Replace the CRYPTO_ONCE based global dispatch with an OSSL_ML_DSA_SAMPLE_OPS vtable. Callers obtain the right implementation via ossl_ml_dsa_sample_ops(), which returns either the generic scalar ops or the x86-64 AVX-512VL multi-buffer ops depending on the build and CPU capabilities.
• Move the x86-64 multi-buffer implementation into ml_dsa_sample_x86_64.inc, included at the bottom of ml_dsa_sample.c only when KECCAK1600_ASM and x86_64 are defined.

No functional change.

Updated perf is very similar (if not actually slightly better than the run_once implementaion):

CRYPTO_ONCE:

                            keygen     signs    verify keygens/s    sign/s  verify/s
               ML-DSA-44 0.000068s 0.000270s 0.000055s   14694.4    3705.4   18338.8
               ML-DSA-65 0.000142s 0.000458s 0.000085s    7034.2    2184.4   11752.5
               ML-DSA-87 0.000163s 0.000531s 0.000135s    6144.3    1882.4    7400.3

After VTABLE update:

                               keygen     signs    verify keygens/s    sign/s  verify/s
                  ML-DSA-44 0.000068s 0.000270s 0.000054s   14728.0    3703.5   18534.5
                  ML-DSA-65 0.000141s 0.000452s 0.000085s    7080.4    2212.5   11790.0
                  ML-DSA-87 0.000162s 0.000531s 0.000134s    6187.9    1883.5    7441.0

[slontis]

I dont quite understand why we do this in assembler?

[mdcornu]

This is because we can easily check the assembler version in the perl module.

If the assembler version supports AVX512VL extensions, the SHAKE x4 functions are generated and SHA3_avx512vl_capable() implements the OPENSSL_ia32cap_P table lookup.

If not, then only stubs are generated and the SHA3_avx512vl_capable() function is hardcoded to return 0, to guard against the use of the SHAKE x4 functions when they are not available.

Stub code can be seen here:

openssl/crypto/sha/asm/keccak1600x4-avx512vl.pl

Line 2137 in f5f4110

SHA3_avx512vl_capable:

[slontis]

The approach here looks reasonable, if we need to refactor common bits out a a later point then that is ok.

[mdcornu]

The approach here looks reasonable, if we need to refactor common bits out a a later point then that is ok.

Great! Thanks for looking into it. I'll push to version to the OpenSSL PR in case there is any more feedback.