diff --git a/README.md b/README.md index 56c564ba4..ab4cc5464 100644 --- a/README.md +++ b/README.md @@ -406,7 +406,7 @@ bash scripts/run/run_sdc_campaign.sh --selftest-classify # 解析 > `mce_check` 是真实 EDAC 后端测试(统计 `/proc/interrupts` 的 EDAC `ce/ue_count`),实测 `exit: pass`,非 placeholder。 -**能力门控的诚实跳过**(skip 不是只有"未实现"一种):17 个多流 HMAC 的 ipsec SVE 用例(`*_hmac_sha*` / `*_sha2_*`)每条 HMAC 流占一个 SVE lane(sha1/224/256 8×32-bit 需 `svcntw()>=8`,384/512 4×64-bit 需 `svcntd()>=4`,均为 VL≥256 bit),在 `test_init` 探测 `HWCAP_SVE` 后实测 `svcnt*`:短 VL 硬件(如 Neoverse N2,VL=128)返回 `CpuNotSupported` 跳过并在 skip-reason 报告实际 VL,绝不进入对未写入 lane 的比较(lane 超出 VL 的输出永不写入,比较只会读到未初始化内存——那不是检测而是噪声);无 SVE 的硬件在框架级 `test compiled with sve` 守卫处跳过。目标硅片(鲲鹏 930,VL=256)正常执行,内核经独立标量参考实现 + gem5 VL=256 仿真双重位级验证。 +**能力门控的诚实跳过**(skip 不是只有"未实现"一种):17 个多流 HMAC 的 ipsec SVE 用例(`*_hmac_sha*` / `*_sha2_*`)每条 HMAC 流占一个 SVE lane,**流数 = 运行时 lane 数(all-VL Phase 2 P4:sha1/224/256 为 `svcntw()` 4..32 流、384/512 为 `svcntd()` 2..16 流,128..1024-bit 全档自适应)**,经框架层 `arm64_require_sve_vl` 门控(HWCAP_SVE 能力 + 范围检查一行);W 调度表按 1024-bit 上限定维(修掉了旧固定 8/4 维度在 VL≥512 硬件上的行越界隐患),`data_size` knob 上限钳制到 HMAC 组装缓冲容量。无 SVE 的硬件在框架级 `test compiled with sve` 守卫处跳过。内核流数参数化形态经 gem5-fi `sve_sha_dual` 四档 16-run 实证(128-bit 档 4/2 流全绿即"短 VL 可跑"的命题实证;K/IV 常量表同源复制,FIPS 标量参考逐流字节级比对)。 ### `memcpy_rewr`:MPSC memcpy 一致性压测(使用选项与推荐参数) diff --git a/docs/research/progress-log.md b/docs/research/progress-log.md index 16225181b..45105d52a 100644 --- a/docs/research/progress-log.md +++ b/docs/research/progress-log.md @@ -687,6 +687,37 @@ M4 未尽项(假通过清单 ARM64 重推导、耗时尾部基线)在 v5 §1 fracture_loop_count 一致性问题记账未动。 - **承接**:P4(ipsec HMAC×17)计划已就绪(docs/superpowers/plans/ 2026-10-24-sve-sme-all-vl2-p4.md)。 +## 2026-10-24(all-VL Phase 2 P4:ipsec HMAC×17 流数参数化——两个既有缺陷随阶段修复) + +- **既有缺陷修复**(评估代理实证, 随 P4 落地): ①门控无上界——旧 + `svcntw()<8||svcntd()<4` 只查下界, VL≥512 硬件上 svst1 向 8 宽 W 行写 + 16/32 lane(越界栈写+串行污染); 修复=STREAMS_MAX_32/64 定维+require + 范围门+nstreams 填充三重防线。②data_size knob[1024,16MB] vs HMAC + 组装栈缓冲 2048B/流——>1975/1903 即溢出; 修复=hmac 层容量守卫 + (void→int 返回值, 17×3 调用点检查)+测试 knob 上限钳 1900/1830。 +- **重构**(feat/ipsec-allvl, 21 commit): 内核/HMAC 头参数化(内部 + 推导 n=svcntw()/svcntd()——向量通路按 VL 动作是自然律, 调用点最小 + 化; 向量代码零改动, 只动 C 层数组维度/循环边界)+17 测试五段式+ + verify.cpp 运行时化+meson 注释同步。docblock "Eight lanes"(4 流文件 + 实为 4 lane 的复制残留失实)一并修正。 +- **gem5 四档实证**(feat/sha-dual @ 955f766d, 16 run=四档×{fixed, + random×3}, 隔离代理执行): 流数逐档 4/8/16/32(32-bit 族)与 2/4/8/16 + (64-bit 族); 全部 SDCSHIELD_FAILS=0; **"VL=128 可跑"命题实证**(4/2 + 流全绿); W 定宽 32/16 在 512/1024 档无越界。**两轮验证**: 首轮 + 16/16 FAIL——验证代理抓住 kernel 标量 golden 两缺陷(&sw[0][s] 把 + [word][stream] 布局当连续状态, 行 1-4 恒 IV; mini-HMAC 两独立块被 + xN 连续读越界), 依规止步未提交; 修复后同矩阵全绿。findings.md 两轮 + 完整入档——"零告警构建≠正确"的纪律实录。 +- **验证模式**: 16-run 与 20-commit 实施均由隔离 subagent 执行; 每文件 + TU 零告警+skip 干净(框架编译特性门)+终态全量构建零告警+zstd19 pass。 +- **诚实边界**: verify.cpp 的 5200 向量 OpenSSL 重验需 SVE 硬件(本机 + 仅编译验证, commit 正文记录); hmac 容量守卫的 init 侧失败类别已由 + 20f73730 修正(compute_golden 经返回值传播, test_init 返回 + EXIT_FAILURE → 干净 Failed, 非 Skipped)——本条曾按旧形态记账, 评审 + 二轮勘误同步。 +- **承接**:P5(sleef/eigen×8)Eigen 侧已裁自 pin 回退(spike 证据: VLA + 语言级不可编译); sleef 半侧待 spike; P6 存量迁移(含本阶段 6 个 + 遗留手探文件)待做。 ## 2026-10-25(PR #280 检视处置 R2:rebase 契约统一 + asm 加固 + FNV 重锚) - **rebase 到 main(#277 已并入)**:11 条当前 head 意见中 6 条 @@ -717,3 +748,34 @@ M4 未尽项(假通过清单 ARM64 重推导、耗时尾部基线)在 v5 §1 真实运行 pass;selftest_arm64_vl_require_sve 走 helper log_skip 原文 skip;zstd19 -t 2000 -n 1 回归 pass;x86-64 零触及 (全部变更位于 #elif __aarch64__/arm64 专属文件)。 +## 2026-10-25(all-VL Phase 2 P4 评审二轮:移交项落地 + 真同源 gem5 实证 + 守卫/文档精度) + +- **移交项落地(PR #282 处置确认)**: sve_hmac.h 五入口 msg_len 前置 + 上界守卫——body=64/128+msg_len 与 padded 均为 size_t 运算先于检查, + msg_len 邻近 SIZE_MAX 时回绕回 2048 以内骗过 padded 后置单检, + memcpy 巨量溢出; 前置钳 SVE_HMAC_MSG_MAX_32=1975/_64=1903 + (1f3b4869)。 +- **真同源 gem5 实证**(评审二轮 Important 采纳: 原证据载体 sve_sha_dual + 是镜像副本, 非同源): 新 harness **sve_hmac_dual** 直接 #include 仓库 + 实际 sve_hmac.h(+sve_sha_kernels.h)——被测机器码与 17 测试同源。 + 四档 16-run(128/256/512/1024-bit × {fixed, random×3})全绿: 流数逐档 + 4/8/16/32 与 2/4/8/16 精确; RFC 2202/4231 六已知答案向量双侧(oracle+ + SVE)锚定公开值(hex 经 python3 hmac 与 openssl 双独立核验); 8 长度× + 逐流字节精确(含 1900/1830 knob 上限与 1975/1903 精确边界); 守卫探测 + msg 21+14/key 6+4 全部 -1; 16 checksum 互异。**阴性对照**: 还原无 + 守卫形态, 同 harness 在 msg_len=SIZE_MAX 探针处页表错误崩溃 + (0x7fffff0000, gem5 exit 134)——回绕缺口与守卫负载性正反双向实证 + (gem5-fi feat/pr284-hmac-dual @ 805838f6, findings.md P4 二轮条目)。 +- **守卫/文档精度**(评审二轮 Minor 三条采纳): ①17 文件流数上界字面量 + 32/16 → SVE_SHA_STREAMS_MAX_32/64 宏, 解除数组定维/检查界限失同步 + 耦合(e8ccec8e); ②init/run VL 漂移守卫 report_fail_msg 硬失败改 + log_skip(CpuNotSupported)+EXIT_SKIP——部署侧 per-thread VL pinting + 形态下按环境不支持跳过而非污染 SDC 失败信号, fail-closed 核心语义 + 保留(3aeb7aa3); ③meson+17 docblock 验证宣称口径修正——OpenSSL 位级 + 验证属原固定 8/4 流形态(5200+1360 向量, VL=256 主机), 参数化形态由 + 真同源 dual 承载, OpenSSL 重验需 SVE 主机(b155ab02)。 +- **诚实边界(更新)**: 17 测试文件的框架胶水(fork/TEST_LOOP/ + arm64_require_sve_vl 的 log_skip 类别机制)仍为宿主编译+SKIP 验证—— + 完整 sdcshield 二进制(fork+96 线程+OpenSSL)不可入 gem5 SE 单进程 + 配置; 胶水依赖的算术不变式(lanes 换算 svl/4==svcntw()、[4,32]/ + [2,16] 范围、漂移比较操作数)已由同源 harness 逐档执行。 diff --git a/scripts/sve-sha-verify/verify.cpp b/scripts/sve-sha-verify/verify.cpp index a1ac7f529..8a3ee805d 100644 --- a/scripts/sve-sha-verify/verify.cpp +++ b/scripts/sve-sha-verify/verify.cpp @@ -4,6 +4,11 @@ * must match OpenSSL bit-for-bit on a large vector set before being used * as ipsec SVE test golden bases. * + * All-VL Phase 2 P4: the stream count is the runtime lane count + * (svcntw()/svcntd(), one message per lane, 128..1024-bit VL), matching + * the parameterized kernels — the vector-set size per round scales with + * the host VL (5200 vectors at VL=256, more at wider VL). + * * Build: g++ -O2 -march=armv8.2-a+sve -I \ * this_file.c -o verifier -L -l:libcrypto.a */ @@ -44,25 +49,37 @@ static std::vector pad_sha512(const uint8_t *msg, size_t len) { int main() { int fails = 0, total = 0; - /* ---------- SHA-256 x8 ---------- */ + /* All-VL gate (adversarial review Important-1): arrays are sized to + the 1024-bit cap; a wider runtime VL would overflow them — mirror + the gem5 kernel's exit-3 guard. */ + if ((int)svcntw() < 4 || (int)svcntw() > SVE_SHA_STREAMS_MAX_32 || + (int)svcntd() < 2 || (int)svcntd() > SVE_SHA_STREAMS_MAX_64) { + fprintf(stderr, "unsupported VL (svcntw=%d svcntd=%d; arrays " + "sized for 128..1024-bit)\n", + (int)svcntw(), (int)svcntd()); + return 3; + } + + /* ---------- SHA-256 (streams = svcntw()) ---------- */ for (int round = 0; round < 200; ++round) { + const int n = (int)svcntw(); /* all-VL: runtime lane count */ size_t len = lcg() % 512; /* 0..511 字节 */ size_t nblocks = ((len + 9 + 63) / 64); - uint8_t msgs[8][512]; - for (int s = 0; s < 8; ++s) + uint8_t msgs[SVE_SHA_STREAMS_MAX_32][512]; + for (int s = 0; s < n; ++s) for (size_t i = 0; i < len; ++i) msgs[s][i] = (uint8_t)lcg(); - const uint8_t *mp[8]; - std::vector padded[8]; - for (int s = 0; s < 8; ++s) { + const uint8_t *mp[SVE_SHA_STREAMS_MAX_32]; + std::vector padded[SVE_SHA_STREAMS_MAX_32]; + for (int s = 0; s < n; ++s) { padded[s] = pad_sha256(msgs[s], len); mp[s] = padded[s].data(); } - uint32_t st[8][8]; + uint32_t st[8][SVE_SHA_STREAMS_MAX_32]; sha256_compress_x8(st, mp, (uint32_t)nblocks, sha256_iv); - for (int s = 0; s < 8; ++s) { + for (int s = 0; s < n; ++s) { uint8_t digest[32]; unsigned dlen = 0; EVP_Digest(msgs[s], len, digest, &dlen, EVP_sha256(), nullptr); @@ -82,23 +99,24 @@ int main() { } } - /* ---------- SHA-224 (IV 不同 + 截断 7 字) ---------- */ + /* ---------- SHA-224 (IV 不同 + 截断 7 字; streams = svcntw()) ---------- */ for (int round = 0; round < 100; ++round) { + const int n = (int)svcntw(); size_t len = lcg() % 256; size_t nblocks = ((len + 9 + 63) / 64); - uint8_t msgs[8][256]; - for (int s = 0; s < 8; ++s) + uint8_t msgs[SVE_SHA_STREAMS_MAX_32][256]; + for (int s = 0; s < n; ++s) for (size_t i = 0; i < len; ++i) msgs[s][i] = (uint8_t)lcg(); - const uint8_t *mp[8]; - std::vector padded[8]; - for (int s = 0; s < 8; ++s) { + const uint8_t *mp[SVE_SHA_STREAMS_MAX_32]; + std::vector padded[SVE_SHA_STREAMS_MAX_32]; + for (int s = 0; s < n; ++s) { padded[s] = pad_sha256(msgs[s], len); mp[s] = padded[s].data(); } - uint32_t st[8][8]; + uint32_t st[8][SVE_SHA_STREAMS_MAX_32]; sha256_compress_x8(st, mp, (uint32_t)nblocks, sha224_iv); - for (int s = 0; s < 8; ++s) { + for (int s = 0; s < n; ++s) { uint8_t digest[28]; unsigned dlen = 0; EVP_Digest(msgs[s], len, digest, &dlen, EVP_sha224(), nullptr); @@ -115,23 +133,24 @@ int main() { } } - /* ---------- SHA-512 x4 ---------- */ + /* ---------- SHA-512 (streams = svcntd()) ---------- */ for (int round = 0; round < 200; ++round) { + const int n = (int)svcntd(); size_t len = lcg() % 1024; size_t nblocks = ((len + 17 + 127) / 128); - uint8_t msgs[4][1024]; - for (int s = 0; s < 4; ++s) + uint8_t msgs[SVE_SHA_STREAMS_MAX_64][1024]; + for (int s = 0; s < n; ++s) for (size_t i = 0; i < len; ++i) msgs[s][i] = (uint8_t)lcg(); - const uint8_t *mp[4]; - std::vector padded[4]; - for (int s = 0; s < 4; ++s) { + const uint8_t *mp[SVE_SHA_STREAMS_MAX_64]; + std::vector padded[SVE_SHA_STREAMS_MAX_64]; + for (int s = 0; s < n; ++s) { padded[s] = pad_sha512(msgs[s], len); mp[s] = padded[s].data(); } - uint64_t st[8][4]; + uint64_t st[8][SVE_SHA_STREAMS_MAX_64]; sha512_compress_x4(st, mp, (uint32_t)nblocks, sha512_iv); - for (int s = 0; s < 4; ++s) { + for (int s = 0; s < n; ++s) { uint8_t digest[64]; unsigned dlen = 0; EVP_Digest(msgs[s], len, digest, &dlen, EVP_sha512(), nullptr); @@ -145,23 +164,24 @@ int main() { } } - /* ---------- SHA-384 ---------- */ + /* ---------- SHA-384 (streams = svcntd()) ---------- */ for (int round = 0; round < 100; ++round) { + const int n = (int)svcntd(); size_t len = lcg() % 512; size_t nblocks = ((len + 17 + 127) / 128); - uint8_t msgs[4][512]; - for (int s = 0; s < 4; ++s) + uint8_t msgs[SVE_SHA_STREAMS_MAX_64][512]; + for (int s = 0; s < n; ++s) for (size_t i = 0; i < len; ++i) msgs[s][i] = (uint8_t)lcg(); - const uint8_t *mp[4]; - std::vector padded[4]; - for (int s = 0; s < 4; ++s) { + const uint8_t *mp[SVE_SHA_STREAMS_MAX_64]; + std::vector padded[SVE_SHA_STREAMS_MAX_64]; + for (int s = 0; s < n; ++s) { padded[s] = pad_sha512(msgs[s], len); mp[s] = padded[s].data(); } - uint64_t st[8][4]; + uint64_t st[8][SVE_SHA_STREAMS_MAX_64]; sha512_compress_x4(st, mp, (uint32_t)nblocks, sha384_iv); - for (int s = 0; s < 4; ++s) { + for (int s = 0; s < n; ++s) { uint8_t digest[48]; unsigned dlen = 0; EVP_Digest(msgs[s], len, digest, &dlen, EVP_sha384(), nullptr); @@ -175,23 +195,24 @@ int main() { } } - /* ---------- SHA-1 x8 ---------- */ + /* ---------- SHA-1 (streams = svcntw()) ---------- */ for (int round = 0; round < 200; ++round) { + const int n = (int)svcntw(); size_t len = lcg() % 512; size_t nblocks = ((len + 9 + 63) / 64); - uint8_t msgs[8][512]; - for (int s = 0; s < 8; ++s) + uint8_t msgs[SVE_SHA_STREAMS_MAX_32][512]; + for (int s = 0; s < n; ++s) for (size_t i = 0; i < len; ++i) msgs[s][i] = (uint8_t)lcg(); - const uint8_t *mp[8]; - std::vector padded[8]; - for (int s = 0; s < 8; ++s) { + const uint8_t *mp[SVE_SHA_STREAMS_MAX_32]; + std::vector padded[SVE_SHA_STREAMS_MAX_32]; + for (int s = 0; s < n; ++s) { padded[s] = pad_sha256(msgs[s], len); mp[s] = padded[s].data(); } - uint32_t st[5][8]; + uint32_t st[5][SVE_SHA_STREAMS_MAX_32]; sha1_compress_x8(st, mp, (uint32_t)nblocks); - for (int s = 0; s < 8; ++s) { + for (int s = 0; s < n; ++s) { uint8_t digest[20]; unsigned dlen = 0; EVP_Digest(msgs[s], len, digest, &dlen, EVP_sha1(), nullptr); @@ -208,7 +229,8 @@ int main() { } } - printf("total=%d fails=%d → %s\n", total, fails, + printf("total=%d fails=%d (svcntw=%zu svcntd=%zu) → %s\n", total, fails, + (size_t)svcntw(), (size_t)svcntd(), fails == 0 ? "ALL MATCH ✓ (kernels verified against OpenSSL)" : "MISMATCH ✗"); return fails != 0; } diff --git a/tests/cpu/sve/ipsec/ipsec_3des_docsis_hmac_sha1_sve_wide.cpp b/tests/cpu/sve/ipsec/ipsec_3des_docsis_hmac_sha1_sve_wide.cpp index 3ce7377f0..8fc11c4b0 100644 --- a/tests/cpu/sve/ipsec/ipsec_3des_docsis_hmac_sha1_sve_wide.cpp +++ b/tests/cpu/sve/ipsec/ipsec_3des_docsis_hmac_sha1_sve_wide.cpp @@ -7,12 +7,17 @@ * @parblock * SVE port of ipsec_3des_docsis_hmac_sha1_avx512: AES/3DES encryption stays on OpenSSL EVP while * the HMAC-SHA1-96 MAC runs on the multi-stream SVE HMAC implementation - * (verified bit-for-bit against OpenSSL; see scripts/sve-sha-verify). - * Eight independent HMAC lanes run in parallel on SVE vector lanes: lane 0 + * (HMAC construction validated same-source in gem5-fi across 128..1024-bit against an + * independent scalar oracle and RFC 2202/4231 vectors; the underlying SHA + * kernels were verified bit-for-bit against OpenSSL in the original + * fixed-stream form — see scripts/sve-sha-verify and + * docs/research/progress-log.md). + * N = svcntw() (32-bit family) independent HMAC lanes run in parallel on + * SVE vector lanes (all-VL 128..1024-bit, one message per 32-bit lane): lane 0 * carries the original ciphertext message (MAC semantics identical to the - * original test), lanes 1-7 carry deterministic key-stream variant + * original test), lanes 1..N-1 carry deterministic key-stream variant * messages that fill the remaining lanes — same computation family, - * golden values precomputed for all 8. Ciphertext, MACs and decrypted + * golden values precomputed for all N. Ciphertext, MACs and decrypted * data are compared against golden values on every iteration to detect * silent data corruption. Iterations alternate between a re-rolled input * with a same-core recomputed golden and a frozen replay of the init-time @@ -28,19 +33,15 @@ #include "sandstone_ssl.h" #include "sve_hmac.h" #include "sdc_stress_config.h" -#include +#include #include #include -#ifndef HWCAP_SVE -#define HWCAP_SVE (1 << 22) -#endif - #define HMAC_KEY_SIZE_V (20) #define HMAC_96_DIGEST_SIZE (12) #define AES_KEY_SIZE_V (24) #define AES_IV_SIZE_V (8) -#define NSTREAM (8) +#define NSTREAM (SVE_SHA_STREAMS_MAX_32) /* All-VL: 数组按 1024-bit 上限定维; 运行时流数 = svcntw() */ struct ipsec_3des_docsis_hmac_sha1_sve_wide_data { uint8_t aes_key[AES_KEY_SIZE_V]; @@ -49,20 +50,21 @@ struct ipsec_3des_docsis_hmac_sha1_sve_wide_data { uint8_t *plaintext; uint8_t *golden_ciphertext; uint8_t golden_mac[NSTREAM][20]; /* 内部全摘要 20B, 比较取前 12 */ - /* 7 条变体消息: 确定性派生(异或字节掩码), 填 SVE lane */ + /* N-1 条变体消息: 确定性派生(异或字节掩码), 填 SVE lane */ uint8_t *variant_msg[NSTREAM-1]; size_t data_size; + int nstreams; /* All-VL: init 期流数 = svl/4 (svcntw()), run 漂移守卫用 */ }; static size_t ipsec_data_size(struct test *test) { - int64_t v = sdc_knob_int_clamped(test, "data_size", 0, 1024, 16 << 20); + int64_t v = sdc_knob_int_clamped(test, "data_size", 0, 1024, 1900); if (v == 0) - sdc_env_knob_int_clamped("ipsec_3des_docsis_hmac_sha1_sve_wide", "data_size", &v, 1024, 16 << 20); + sdc_env_knob_int_clamped("ipsec_3des_docsis_hmac_sha1_sve_wide", "data_size", &v, 1024, 1900); if (v > 0) return (size_t)v; return (size_t)sdc_auto_linear_count(1, 4, 1.5, 600, thread_count(), - 1024, 16 << 20, test->id, "data_size"); + 1024, 1900, test->id, "data_size"); } static void des3_encrypt(const uint8_t *key, const uint8_t *iv, const uint8_t *in, uint8_t *out, size_t len) { @@ -86,54 +88,56 @@ static void des3_decrypt(const uint8_t *key, const uint8_t *iv, const uint8_t *i s_EVP_CIPHER_CTX_free(ctx); } -static void ipsec_3des_docsis_hmac_sha1_sve_wide_compute_golden(struct ipsec_3des_docsis_hmac_sha1_sve_wide_data *d) { +static int ipsec_3des_docsis_hmac_sha1_sve_wide_compute_golden(struct test *test, struct ipsec_3des_docsis_hmac_sha1_sve_wide_data *d) { des3_encrypt(d->aes_key, d->aes_iv, d->plaintext, d->golden_ciphertext, d->data_size); - /* SVE 多流 HMAC: lane0=密文(主语义), lane1..7=变体 */ + /* SVE 多流 HMAC: lane0=密文(主语义), lane1..N-1=变体 */ + const int n = (int)svcntw(); /* 与 kernel 内部推导一致 */ const uint8_t *keys[NSTREAM], *msgs[NSTREAM]; uint8_t variant_keys[NSTREAM][20]; - for (int s = 0; s < NSTREAM; ++s) { + for (int s = 0; s < n; ++s) { memcpy(variant_keys[s], d->hmac_key, 20); if (s > 0) variant_keys[s][0] ^= (uint8_t)s; /* 变体密钥: 确定性微扰 */ keys[s] = variant_keys[s]; } msgs[0] = d->golden_ciphertext; - for (int s = 1; s < NSTREAM; ++s) { + for (int s = 1; s < n; ++s) { /* 变体消息: 密文异或流掩码(确定性) */ for (size_t i = 0; i < d->data_size; ++i) d->variant_msg[s-1][i] = d->golden_ciphertext[i] ^ (uint8_t)(s * 0x5A + (i & 0x0F)); msgs[s] = d->variant_msg[s-1]; } - hmac_sha1_x8(d->golden_mac, keys, 20, msgs, d->data_size); + if (hmac_sha1_x8(d->golden_mac, keys, 20, msgs, d->data_size) != 0) { + log_error("%s: HMAC capacity overflow (data_size too large)", test->id); + return -1; + } + return 0; } static int ipsec_3des_docsis_hmac_sha1_sve_wide_init(struct test *test) { if (s_EVP_CIPHER_CTX_new && s_EVP_EncryptInit_ex && s_EVP_DecryptInit_ex && s_EVP_CIPHER_CTX_new && s_EVP_des_ede3_cbc()) { - unsigned long hwcap = getauxval(AT_HWCAP); - if ((hwcap & HWCAP_SVE) == 0) { - log_skip(CpuNotSupportedSkipCategory, - "to be implemented (placeholder): ARM SVE required for ipsec_3des_docsis_hmac_sha1_sve_wide"); - return EXIT_SKIP; - } - /* The multi-stream SVE HMAC kernels give one stream per SVE - * lane: 8x32-bit (sha1/224/256) needs svcntw()>=8 and - * 4x64-bit (384/512) needs svcntd()>=4 -- both mean VL>=256 - * bit. On shorter-VL SVE hardware (e.g. Neoverse N2, VL=128) - * lanes beyond VL are never written, so skip cleanly instead - * of comparing unwritten memory (target silicon: VL=256, see - * tests/cpu/sve/meson.build). */ - if (svcntw() < 8 || svcntd() < 4) { - log_skip(CpuNotSupportedSkipCategory, - "ARM SVE vector length %zu bytes < 32 required by the multi-stream HMAC kernels (svcntw=%zu svcntd=%zu)", - svcntb(), (size_t)svcntw(), (size_t)svcntd()); + /* All-VL Phase 2 P4: capability + 128..1024-bit range check in + * one framework-layer call (replaces the old lower-bound-only + * svcntw()>=8/svcntd()>=4 gate -- the 8.2 defect: at VL>=512 the + * fixed 8/4-wide W rows overflowed; the STREAMS_MAX-sized rows + + * this range gate close both ends). arm64_require_sve_vl returns + * the VL in BYTES. */ + const long svl = arm64_require_sve_vl(test, 128, 1024); + if (svl < 0) return EXIT_SKIP; + const int nstreams = (int)svl / 4; /* one stream per 32-bit lane (svcntw) */ + if (nstreams < 4 || nstreams > SVE_SHA_STREAMS_MAX_32) { /* [4, SVE_SHA_STREAMS_MAX_32], release-effective, NDEBUG-proof */ + log_error("%s: streams %d outside range — unit drift", test->id, nstreams); + return EXIT_FAILURE; } struct ipsec_3des_docsis_hmac_sha1_sve_wide_data *d = (struct ipsec_3des_docsis_hmac_sha1_sve_wide_data *)malloc(sizeof(*d)); if (!d) return EXIT_SKIP; + d->nstreams = nstreams; /* 工作集字节数: -O ipsec_3des_docsis_hmac_sha1_sve_wide.data_size=N / * SDC_IPSEC_3DES_DOCSIS_HMAC_SHA1_SVE_WIDE_DATA_SIZE 环境变量 / 自适应: - * 每线程明文+密文+golden 密文+解密缓冲+变体消息 ~= (3+NSTREAM-1)x, - * 占物理内存 60%, clamp [1024, 16M]。线性缓冲用 sdc_auto_linear_count(sqrt 缩放会欠缩放压力)。历史硬编码 1024。3DES-CBC + * 每线程明文+密文+golden 密文+解密缓冲+变体消息 ~= (3+nstreams-1)x, + * 占物理内存 60%, clamp [1024, 1900]——上限闭环 8.4: HMAC 组装 + * 缓冲 full[N][2048], 2048-64-9=1975B 安全边界。线性缓冲用 sdc_auto_linear_count(sqrt 缩放会欠缩放压力)。历史硬编码 1024。3DES-CBC * 无填充分组 8B, 统一 16B 对齐(天然满足 8B), 对齐后 < 1024 取 1024。 */ d->data_size = ipsec_data_size(test); d->data_size &= ~(size_t)15; @@ -147,7 +151,7 @@ static int ipsec_3des_docsis_hmac_sha1_sve_wide_init(struct test *test) { free(d); return EXIT_SKIP; } - for (int s = 0; s < NSTREAM - 1; ++s) { + for (int s = 0; s < d->nstreams - 1; ++s) { d->variant_msg[s] = (uint8_t *)malloc(d->data_size); if (!d->variant_msg[s]) { free(d->plaintext); @@ -162,7 +166,20 @@ static int ipsec_3des_docsis_hmac_sha1_sve_wide_init(struct test *test) { memset_random(d->aes_iv, AES_IV_SIZE_V); memset_random(d->hmac_key, 20); memset_random(d->plaintext, d->data_size); - ipsec_3des_docsis_hmac_sha1_sve_wide_compute_golden(d); + if (ipsec_3des_docsis_hmac_sha1_sve_wide_compute_golden(test, d) != 0) { + /* P2a (review Minor): init-side failure propagates via the + * return value so the framework reports a clean Failed + * (sandstone_run.cpp: ret > 0 -> TestResult::Failed). + * report_fail_msg is a noreturn thread-cancel that only + * belongs in the test_run worker context — in test_init it + * would surface as Skipped("Unexpected OS error"). */ + free(d->plaintext); + free(d->golden_ciphertext); + for (int s = 0; s < d->nstreams - 1; ++s) + free(d->variant_msg[s]); + free(d); + return EXIT_FAILURE; + } test->data = d; return EXIT_SUCCESS; } else { @@ -174,10 +191,22 @@ static int ipsec_3des_docsis_hmac_sha1_sve_wide_init(struct test *test) { static int ipsec_3des_docsis_hmac_sha1_sve_wide_run(struct test *test, int cpu) { (void)cpu; struct ipsec_3des_docsis_hmac_sha1_sve_wide_data *d = (struct ipsec_3des_docsis_hmac_sha1_sve_wide_data *)test->data; + /* All-VL policy #3: never compare goldens across an init/run VL + * mismatch. Drift means this deployment's per-thread VL differs from + * the init thread's (e.g. a VL-pinning launcher changing the + * thread-inheritance chain) — an environment mismatch, which skips + * with CpuNotSupported (same category as the init-side range gate in + * arm64_require_sve_vl) instead of polluting the SDC failure signal. */ + if ((int)svcntw() != d->nstreams) { + log_skip(CpuNotSupportedSkipCategory, + "%s: SVE VL drifted between init (%d streams) and run (%d streams) — environment mismatch, not SDC", + test->id, d->nstreams, (int)svcntw()); + return EXIT_SKIP; + } uint8_t *ciphertext = (uint8_t *)malloc(d->data_size); uint8_t *decrypted = (uint8_t *)malloc(d->data_size); uint8_t mac[NSTREAM][20]; - uint8_t *variant_msg = (uint8_t *)malloc((NSTREAM-1) * d->data_size); + uint8_t *variant_msg = (uint8_t *)malloc((d->nstreams-1) * d->data_size); /* R2b (B2-003): per-thread plaintext + golden, re-rolled every * iteration (thread-safe: shared d is only read after init; the @@ -201,7 +230,7 @@ static int ipsec_3des_docsis_hmac_sha1_sve_wide_run(struct test *test, int cpu) "out of memory for per-thread buffers"); return EXIT_SKIP; } - for (int s = 0; s < NSTREAM - 1; ++s) { + for (int s = 0; s < d->nstreams - 1; ++s) { local_d.variant_msg[s] = (uint8_t *)malloc(d->data_size); if (!local_d.variant_msg[s]) { free(local_d.plaintext); @@ -234,27 +263,32 @@ static int ipsec_3des_docsis_hmac_sha1_sve_wide_run(struct test *test, int cpu) /* SVE 多流 HMAC (主消息 + 变体) */ { const uint8_t *keys[NSTREAM], *msgs[NSTREAM]; + const int n = (int)svcntw(); /* 与 kernel 内部推导一致 */ uint8_t variant_keys[NSTREAM][20]; - for (int s = 0; s < NSTREAM; ++s) { + for (int s = 0; s < n; ++s) { memcpy(variant_keys[s], d->hmac_key, 20); if (s > 0) variant_keys[s][0] ^= (uint8_t)s; keys[s] = variant_keys[s]; } msgs[0] = ciphertext; - for (int s = 1; s < NSTREAM; ++s) { + for (int s = 1; s < n; ++s) { for (size_t i = 0; i < d->data_size; ++i) variant_msg[(s-1)*d->data_size + i] = ciphertext[i] ^ (uint8_t)(s * 0x5A + (i & 0x0F)); msgs[s] = variant_msg + (s-1)*d->data_size; } - hmac_sha1_x8(mac, keys, 20, msgs, d->data_size); + if (hmac_sha1_x8(mac, keys, 20, msgs, d->data_size) != 0) { + log_error("%s: HMAC capacity overflow (data_size too large)", test->id); + report_fail_msg("%s: hmac assembly over capacity", test->id); + } } - for (int s = 0; s < NSTREAM; ++s) + for (int s = 0; s < d->nstreams; ++s) memcmp_or_fail(mac[s], d->golden_mac[s], 12, "HMAC-SHA1-96 digest mismatch (frozen replay)"); } else { /* re-roll: fresh plaintext + same-iteration golden (EVP encrypt + * SVE HMAC recomputed for this thread's data) */ memset_random(local_d.plaintext, d->data_size); - ipsec_3des_docsis_hmac_sha1_sve_wide_compute_golden(&local_d); + if (ipsec_3des_docsis_hmac_sha1_sve_wide_compute_golden(test, &local_d) != 0) + report_fail_msg("%s: hmac assembly over capacity", test->id); des3_encrypt(local_d.aes_key, local_d.aes_iv, local_d.plaintext, ciphertext, d->data_size); memcmp_or_fail(ciphertext, local_d.golden_ciphertext, d->data_size, "ciphertext mismatch (SVE)"); des3_decrypt(local_d.aes_key, local_d.aes_iv, ciphertext, decrypted, d->data_size); @@ -263,21 +297,25 @@ static int ipsec_3des_docsis_hmac_sha1_sve_wide_run(struct test *test, int cpu) /* SVE 多流 HMAC (主消息 + 变体) */ { const uint8_t *keys[NSTREAM], *msgs[NSTREAM]; + const int n = (int)svcntw(); /* 与 kernel 内部推导一致 */ uint8_t variant_keys[NSTREAM][20]; - for (int s = 0; s < NSTREAM; ++s) { + for (int s = 0; s < n; ++s) { memcpy(variant_keys[s], local_d.hmac_key, 20); if (s > 0) variant_keys[s][0] ^= (uint8_t)s; keys[s] = variant_keys[s]; } msgs[0] = ciphertext; - for (int s = 1; s < NSTREAM; ++s) { + for (int s = 1; s < n; ++s) { for (size_t i = 0; i < d->data_size; ++i) variant_msg[(s-1)*d->data_size + i] = ciphertext[i] ^ (uint8_t)(s * 0x5A + (i & 0x0F)); msgs[s] = variant_msg + (s-1)*d->data_size; } - hmac_sha1_x8(mac, keys, 20, msgs, d->data_size); + if (hmac_sha1_x8(mac, keys, 20, msgs, d->data_size) != 0) { + log_error("%s: HMAC capacity overflow (data_size too large)", test->id); + report_fail_msg("%s: hmac assembly over capacity", test->id); + } } - for (int s = 0; s < NSTREAM; ++s) + for (int s = 0; s < d->nstreams; ++s) memcmp_or_fail(mac[s], local_d.golden_mac[s], 12, "HMAC-SHA1-96 digest mismatch (SVE)"); } ++iter; @@ -285,7 +323,7 @@ static int ipsec_3des_docsis_hmac_sha1_sve_wide_run(struct test *test, int cpu) free(ciphertext); free(decrypted); free(variant_msg); - for (int s = 0; s < NSTREAM - 1; ++s) + for (int s = 0; s < d->nstreams - 1; ++s) free(local_d.variant_msg[s]); free(local_d.plaintext); free(local_d.golden_ciphertext); @@ -297,7 +335,7 @@ static int ipsec_3des_docsis_hmac_sha1_sve_wide_cleanup(struct test *test) { if (d) { free(d->plaintext); free(d->golden_ciphertext); - for (int s = 0; s < NSTREAM - 1; ++s) + for (int s = 0; s < d->nstreams - 1; ++s) free(d->variant_msg[s]); } free(d); diff --git a/tests/cpu/sve/ipsec/ipsec_3des_docsis_hmac_sha224_sve_wide.cpp b/tests/cpu/sve/ipsec/ipsec_3des_docsis_hmac_sha224_sve_wide.cpp index 7f6bafb0d..08ecbb28d 100644 --- a/tests/cpu/sve/ipsec/ipsec_3des_docsis_hmac_sha224_sve_wide.cpp +++ b/tests/cpu/sve/ipsec/ipsec_3des_docsis_hmac_sha224_sve_wide.cpp @@ -7,12 +7,17 @@ * @parblock * SVE port of ipsec_3des_docsis_hmac_sha224_avx512: AES/3DES encryption stays on OpenSSL EVP while * the HMAC-SHA224 MAC runs on the multi-stream SVE HMAC implementation - * (verified bit-for-bit against OpenSSL; see scripts/sve-sha-verify). - * Eight independent HMAC lanes run in parallel on SVE vector lanes: lane 0 + * (HMAC construction validated same-source in gem5-fi across 128..1024-bit against an + * independent scalar oracle and RFC 2202/4231 vectors; the underlying SHA + * kernels were verified bit-for-bit against OpenSSL in the original + * fixed-stream form — see scripts/sve-sha-verify and + * docs/research/progress-log.md). + * N = svcntw() (32-bit family) independent HMAC lanes run in parallel on + * SVE vector lanes (all-VL 128..1024-bit, one message per 32-bit lane): lane 0 * carries the original ciphertext message (MAC semantics identical to the - * original test), lanes 1-7 carry deterministic key-stream variant + * original test), lanes 1..N-1 carry deterministic key-stream variant * messages that fill the remaining lanes — same computation family, - * golden values precomputed for all 8. Ciphertext, MACs and decrypted + * golden values precomputed for all N. Ciphertext, MACs and decrypted * data are compared against golden values on every iteration to detect * silent data corruption. Iterations alternate between a re-rolled input * with a same-core recomputed golden and a frozen replay of the init-time @@ -28,19 +33,15 @@ #include "sandstone_ssl.h" #include "sve_hmac.h" #include "sdc_stress_config.h" -#include +#include #include #include -#ifndef HWCAP_SVE -#define HWCAP_SVE (1 << 22) -#endif - #define HMAC_KEY_SIZE_V (28) #define SHA224_DIGEST_SIZE (28) #define AES_KEY_SIZE_V (24) #define AES_IV_SIZE_V (8) -#define NSTREAM (8) +#define NSTREAM (SVE_SHA_STREAMS_MAX_32) /* All-VL: 数组按 1024-bit 上限定维; 运行时流数 = svcntw() */ struct ipsec_3des_docsis_hmac_sha224_sve_wide_data { uint8_t aes_key[AES_KEY_SIZE_V]; @@ -49,20 +50,21 @@ struct ipsec_3des_docsis_hmac_sha224_sve_wide_data { uint8_t *plaintext; uint8_t *golden_ciphertext; uint8_t golden_mac[NSTREAM][28]; - /* 7 条变体消息: 确定性派生(异或字节掩码), 填 SVE lane */ + /* N-1 条变体消息: 确定性派生(异或字节掩码), 填 SVE lane */ uint8_t *variant_msg[NSTREAM-1]; size_t data_size; + int nstreams; /* All-VL: init 期流数 = svl/4 (svcntw()), run 漂移守卫用 */ }; static size_t ipsec_data_size(struct test *test) { - int64_t v = sdc_knob_int_clamped(test, "data_size", 0, 1024, 16 << 20); + int64_t v = sdc_knob_int_clamped(test, "data_size", 0, 1024, 1900); if (v == 0) - sdc_env_knob_int_clamped("ipsec_3des_docsis_hmac_sha224_sve_wide", "data_size", &v, 1024, 16 << 20); + sdc_env_knob_int_clamped("ipsec_3des_docsis_hmac_sha224_sve_wide", "data_size", &v, 1024, 1900); if (v > 0) return (size_t)v; return (size_t)sdc_auto_linear_count(1, 4, 1.5, 600, thread_count(), - 1024, 16 << 20, test->id, "data_size"); + 1024, 1900, test->id, "data_size"); } static void des3_encrypt(const uint8_t *key, const uint8_t *iv, const uint8_t *in, uint8_t *out, size_t len) { @@ -86,54 +88,56 @@ static void des3_decrypt(const uint8_t *key, const uint8_t *iv, const uint8_t *i s_EVP_CIPHER_CTX_free(ctx); } -static void ipsec_3des_docsis_hmac_sha224_sve_wide_compute_golden(struct ipsec_3des_docsis_hmac_sha224_sve_wide_data *d) { +static int ipsec_3des_docsis_hmac_sha224_sve_wide_compute_golden(struct test *test, struct ipsec_3des_docsis_hmac_sha224_sve_wide_data *d) { des3_encrypt(d->aes_key, d->aes_iv, d->plaintext, d->golden_ciphertext, d->data_size); - /* SVE 多流 HMAC: lane0=密文(主语义), lane1..7=变体 */ + /* SVE 多流 HMAC: lane0=密文(主语义), lane1..N-1=变体 */ + const int n = (int)svcntw(); /* 与 kernel 内部推导一致 */ const uint8_t *keys[NSTREAM], *msgs[NSTREAM]; uint8_t variant_keys[NSTREAM][28]; - for (int s = 0; s < NSTREAM; ++s) { + for (int s = 0; s < n; ++s) { memcpy(variant_keys[s], d->hmac_key, 28); if (s > 0) variant_keys[s][0] ^= (uint8_t)s; /* 变体密钥: 确定性微扰 */ keys[s] = variant_keys[s]; } msgs[0] = d->golden_ciphertext; - for (int s = 1; s < NSTREAM; ++s) { + for (int s = 1; s < n; ++s) { /* 变体消息: 密文异或流掩码(确定性) */ for (size_t i = 0; i < d->data_size; ++i) d->variant_msg[s-1][i] = d->golden_ciphertext[i] ^ (uint8_t)(s * 0x5A + (i & 0x0F)); msgs[s] = d->variant_msg[s-1]; } - hmac_sha224_x8(d->golden_mac, keys, 28, msgs, d->data_size); + if (hmac_sha224_x8(d->golden_mac, keys, 28, msgs, d->data_size) != 0) { + log_error("%s: HMAC capacity overflow (data_size too large)", test->id); + return -1; + } + return 0; } static int ipsec_3des_docsis_hmac_sha224_sve_wide_init(struct test *test) { if (s_EVP_CIPHER_CTX_new && s_EVP_EncryptInit_ex && s_EVP_DecryptInit_ex && s_EVP_CIPHER_CTX_new && s_EVP_des_ede3_cbc()) { - unsigned long hwcap = getauxval(AT_HWCAP); - if ((hwcap & HWCAP_SVE) == 0) { - log_skip(CpuNotSupportedSkipCategory, - "to be implemented (placeholder): ARM SVE required for ipsec_3des_docsis_hmac_sha224_sve_wide"); - return EXIT_SKIP; - } - /* The multi-stream SVE HMAC kernels give one stream per SVE - * lane: 8x32-bit (sha1/224/256) needs svcntw()>=8 and - * 4x64-bit (384/512) needs svcntd()>=4 -- both mean VL>=256 - * bit. On shorter-VL SVE hardware (e.g. Neoverse N2, VL=128) - * lanes beyond VL are never written, so skip cleanly instead - * of comparing unwritten memory (target silicon: VL=256, see - * tests/cpu/sve/meson.build). */ - if (svcntw() < 8 || svcntd() < 4) { - log_skip(CpuNotSupportedSkipCategory, - "ARM SVE vector length %zu bytes < 32 required by the multi-stream HMAC kernels (svcntw=%zu svcntd=%zu)", - svcntb(), (size_t)svcntw(), (size_t)svcntd()); + /* All-VL Phase 2 P4: capability + 128..1024-bit range check in + * one framework-layer call (replaces the old lower-bound-only + * svcntw()>=8/svcntd()>=4 gate -- the 8.2 defect: at VL>=512 the + * fixed 8/4-wide W rows overflowed; the STREAMS_MAX-sized rows + + * this range gate close both ends). arm64_require_sve_vl returns + * the VL in BYTES. */ + const long svl = arm64_require_sve_vl(test, 128, 1024); + if (svl < 0) return EXIT_SKIP; + const int nstreams = (int)svl / 4; /* one stream per 32-bit lane (svcntw) */ + if (nstreams < 4 || nstreams > SVE_SHA_STREAMS_MAX_32) { /* [4, SVE_SHA_STREAMS_MAX_32], release-effective, NDEBUG-proof */ + log_error("%s: streams %d outside range — unit drift", test->id, nstreams); + return EXIT_FAILURE; } struct ipsec_3des_docsis_hmac_sha224_sve_wide_data *d = (struct ipsec_3des_docsis_hmac_sha224_sve_wide_data *)malloc(sizeof(*d)); if (!d) return EXIT_SKIP; + d->nstreams = nstreams; /* 工作集字节数: -O ipsec_3des_docsis_hmac_sha224_sve_wide.data_size=N / * SDC_IPSEC_3DES_DOCSIS_HMAC_SHA224_SVE_WIDE_DATA_SIZE 环境变量 / 自适应: - * 每线程明文+密文+golden 密文+解密缓冲+变体消息 ~= (3+NSTREAM-1)x, - * 占物理内存 60%, clamp [1024, 16M]。线性缓冲用 sdc_auto_linear_count(sqrt 缩放会欠缩放压力)。历史硬编码 1024。3DES-CBC + * 每线程明文+密文+golden 密文+解密缓冲+变体消息 ~= (3+nstreams-1)x, + * 占物理内存 60%, clamp [1024, 1900]——上限闭环 8.4: HMAC 组装 + * 缓冲 full[N][2048], 2048-64-9=1975B 安全边界。线性缓冲用 sdc_auto_linear_count(sqrt 缩放会欠缩放压力)。历史硬编码 1024。3DES-CBC * 无填充分组 8B, 统一 16B 对齐(天然满足 8B), 对齐后 < 1024 取 1024。 */ d->data_size = ipsec_data_size(test); d->data_size &= ~(size_t)15; @@ -147,7 +151,7 @@ static int ipsec_3des_docsis_hmac_sha224_sve_wide_init(struct test *test) { free(d); return EXIT_SKIP; } - for (int s = 0; s < NSTREAM - 1; ++s) { + for (int s = 0; s < d->nstreams - 1; ++s) { d->variant_msg[s] = (uint8_t *)malloc(d->data_size); if (!d->variant_msg[s]) { free(d->plaintext); @@ -162,7 +166,20 @@ static int ipsec_3des_docsis_hmac_sha224_sve_wide_init(struct test *test) { memset_random(d->aes_iv, AES_IV_SIZE_V); memset_random(d->hmac_key, 28); memset_random(d->plaintext, d->data_size); - ipsec_3des_docsis_hmac_sha224_sve_wide_compute_golden(d); + if (ipsec_3des_docsis_hmac_sha224_sve_wide_compute_golden(test, d) != 0) { + /* P2a (review Minor): init-side failure propagates via the + * return value so the framework reports a clean Failed + * (sandstone_run.cpp: ret > 0 -> TestResult::Failed). + * report_fail_msg is a noreturn thread-cancel that only + * belongs in the test_run worker context — in test_init it + * would surface as Skipped("Unexpected OS error"). */ + free(d->plaintext); + free(d->golden_ciphertext); + for (int s = 0; s < d->nstreams - 1; ++s) + free(d->variant_msg[s]); + free(d); + return EXIT_FAILURE; + } test->data = d; return EXIT_SUCCESS; } else { @@ -174,10 +191,22 @@ static int ipsec_3des_docsis_hmac_sha224_sve_wide_init(struct test *test) { static int ipsec_3des_docsis_hmac_sha224_sve_wide_run(struct test *test, int cpu) { (void)cpu; struct ipsec_3des_docsis_hmac_sha224_sve_wide_data *d = (struct ipsec_3des_docsis_hmac_sha224_sve_wide_data *)test->data; + /* All-VL policy #3: never compare goldens across an init/run VL + * mismatch. Drift means this deployment's per-thread VL differs from + * the init thread's (e.g. a VL-pinning launcher changing the + * thread-inheritance chain) — an environment mismatch, which skips + * with CpuNotSupported (same category as the init-side range gate in + * arm64_require_sve_vl) instead of polluting the SDC failure signal. */ + if ((int)svcntw() != d->nstreams) { + log_skip(CpuNotSupportedSkipCategory, + "%s: SVE VL drifted between init (%d streams) and run (%d streams) — environment mismatch, not SDC", + test->id, d->nstreams, (int)svcntw()); + return EXIT_SKIP; + } uint8_t *ciphertext = (uint8_t *)malloc(d->data_size); uint8_t *decrypted = (uint8_t *)malloc(d->data_size); uint8_t mac[NSTREAM][28]; - uint8_t *variant_msg = (uint8_t *)malloc((NSTREAM-1) * d->data_size); + uint8_t *variant_msg = (uint8_t *)malloc((d->nstreams-1) * d->data_size); /* R2b (B2-003): per-thread plaintext + golden, re-rolled every * iteration (thread-safe: shared d is only read after init; the @@ -201,7 +230,7 @@ static int ipsec_3des_docsis_hmac_sha224_sve_wide_run(struct test *test, int cpu "out of memory for per-thread buffers"); return EXIT_SKIP; } - for (int s = 0; s < NSTREAM - 1; ++s) { + for (int s = 0; s < d->nstreams - 1; ++s) { local_d.variant_msg[s] = (uint8_t *)malloc(d->data_size); if (!local_d.variant_msg[s]) { free(local_d.plaintext); @@ -234,27 +263,32 @@ static int ipsec_3des_docsis_hmac_sha224_sve_wide_run(struct test *test, int cpu /* SVE 多流 HMAC (主消息 + 变体) */ { const uint8_t *keys[NSTREAM], *msgs[NSTREAM]; + const int n = (int)svcntw(); /* 与 kernel 内部推导一致 */ uint8_t variant_keys[NSTREAM][28]; - for (int s = 0; s < NSTREAM; ++s) { + for (int s = 0; s < n; ++s) { memcpy(variant_keys[s], d->hmac_key, 28); if (s > 0) variant_keys[s][0] ^= (uint8_t)s; keys[s] = variant_keys[s]; } msgs[0] = ciphertext; - for (int s = 1; s < NSTREAM; ++s) { + for (int s = 1; s < n; ++s) { for (size_t i = 0; i < d->data_size; ++i) variant_msg[(s-1)*d->data_size + i] = ciphertext[i] ^ (uint8_t)(s * 0x5A + (i & 0x0F)); msgs[s] = variant_msg + (s-1)*d->data_size; } - hmac_sha224_x8(mac, keys, 28, msgs, d->data_size); + if (hmac_sha224_x8(mac, keys, 28, msgs, d->data_size) != 0) { + log_error("%s: HMAC capacity overflow (data_size too large)", test->id); + report_fail_msg("%s: hmac assembly over capacity", test->id); + } } - for (int s = 0; s < NSTREAM; ++s) + for (int s = 0; s < d->nstreams; ++s) memcmp_or_fail(mac[s], d->golden_mac[s], 28, "HMAC-SHA224 digest mismatch (frozen replay)"); } else { /* re-roll: fresh plaintext + same-iteration golden (EVP encrypt + * SVE HMAC recomputed for this thread's data) */ memset_random(local_d.plaintext, d->data_size); - ipsec_3des_docsis_hmac_sha224_sve_wide_compute_golden(&local_d); + if (ipsec_3des_docsis_hmac_sha224_sve_wide_compute_golden(test, &local_d) != 0) + report_fail_msg("%s: hmac assembly over capacity", test->id); des3_encrypt(local_d.aes_key, local_d.aes_iv, local_d.plaintext, ciphertext, d->data_size); memcmp_or_fail(ciphertext, local_d.golden_ciphertext, d->data_size, "ciphertext mismatch (SVE)"); des3_decrypt(local_d.aes_key, local_d.aes_iv, ciphertext, decrypted, d->data_size); @@ -263,21 +297,25 @@ static int ipsec_3des_docsis_hmac_sha224_sve_wide_run(struct test *test, int cpu /* SVE 多流 HMAC (主消息 + 变体) */ { const uint8_t *keys[NSTREAM], *msgs[NSTREAM]; + const int n = (int)svcntw(); /* 与 kernel 内部推导一致 */ uint8_t variant_keys[NSTREAM][28]; - for (int s = 0; s < NSTREAM; ++s) { + for (int s = 0; s < n; ++s) { memcpy(variant_keys[s], local_d.hmac_key, 28); if (s > 0) variant_keys[s][0] ^= (uint8_t)s; keys[s] = variant_keys[s]; } msgs[0] = ciphertext; - for (int s = 1; s < NSTREAM; ++s) { + for (int s = 1; s < n; ++s) { for (size_t i = 0; i < d->data_size; ++i) variant_msg[(s-1)*d->data_size + i] = ciphertext[i] ^ (uint8_t)(s * 0x5A + (i & 0x0F)); msgs[s] = variant_msg + (s-1)*d->data_size; } - hmac_sha224_x8(mac, keys, 28, msgs, d->data_size); + if (hmac_sha224_x8(mac, keys, 28, msgs, d->data_size) != 0) { + log_error("%s: HMAC capacity overflow (data_size too large)", test->id); + report_fail_msg("%s: hmac assembly over capacity", test->id); + } } - for (int s = 0; s < NSTREAM; ++s) + for (int s = 0; s < d->nstreams; ++s) memcmp_or_fail(mac[s], local_d.golden_mac[s], 28, "HMAC-SHA224 digest mismatch (SVE)"); } ++iter; @@ -285,7 +323,7 @@ static int ipsec_3des_docsis_hmac_sha224_sve_wide_run(struct test *test, int cpu free(ciphertext); free(decrypted); free(variant_msg); - for (int s = 0; s < NSTREAM - 1; ++s) + for (int s = 0; s < d->nstreams - 1; ++s) free(local_d.variant_msg[s]); free(local_d.plaintext); free(local_d.golden_ciphertext); @@ -297,7 +335,7 @@ static int ipsec_3des_docsis_hmac_sha224_sve_wide_cleanup(struct test *test) { if (d) { free(d->plaintext); free(d->golden_ciphertext); - for (int s = 0; s < NSTREAM - 1; ++s) + for (int s = 0; s < d->nstreams - 1; ++s) free(d->variant_msg[s]); } free(d); diff --git a/tests/cpu/sve/ipsec/ipsec_3des_docsis_hmac_sha256_sve_wide.cpp b/tests/cpu/sve/ipsec/ipsec_3des_docsis_hmac_sha256_sve_wide.cpp index d07ba4cab..bd0e94cd8 100644 --- a/tests/cpu/sve/ipsec/ipsec_3des_docsis_hmac_sha256_sve_wide.cpp +++ b/tests/cpu/sve/ipsec/ipsec_3des_docsis_hmac_sha256_sve_wide.cpp @@ -7,12 +7,17 @@ * @parblock * SVE port of ipsec_3des_docsis_hmac_sha256_avx512: AES/3DES encryption stays on OpenSSL EVP while * the HMAC-SHA256 MAC runs on the multi-stream SVE HMAC implementation - * (verified bit-for-bit against OpenSSL; see scripts/sve-sha-verify). - * Eight independent HMAC lanes run in parallel on SVE vector lanes: lane 0 + * (HMAC construction validated same-source in gem5-fi across 128..1024-bit against an + * independent scalar oracle and RFC 2202/4231 vectors; the underlying SHA + * kernels were verified bit-for-bit against OpenSSL in the original + * fixed-stream form — see scripts/sve-sha-verify and + * docs/research/progress-log.md). + * N = svcntw() (32-bit family) independent HMAC lanes run in parallel on + * SVE vector lanes (all-VL 128..1024-bit, one message per 32-bit lane): lane 0 * carries the original ciphertext message (MAC semantics identical to the - * original test), lanes 1-7 carry deterministic key-stream variant + * original test), lanes 1..N-1 carry deterministic key-stream variant * messages that fill the remaining lanes — same computation family, - * golden values precomputed for all 8. Ciphertext, MACs and decrypted + * golden values precomputed for all N. Ciphertext, MACs and decrypted * data are compared against golden values on every iteration to detect * silent data corruption. Iterations alternate between a re-rolled input * with a same-core recomputed golden and a frozen replay of the init-time @@ -28,19 +33,15 @@ #include "sandstone_ssl.h" #include "sve_hmac.h" #include "sdc_stress_config.h" -#include +#include #include #include -#ifndef HWCAP_SVE -#define HWCAP_SVE (1 << 22) -#endif - #define HMAC_KEY_SIZE_V (32) #define SHA256_DIGEST_SIZE (32) #define AES_KEY_SIZE_V (24) #define AES_IV_SIZE_V (8) -#define NSTREAM (8) +#define NSTREAM (SVE_SHA_STREAMS_MAX_32) /* All-VL: 数组按 1024-bit 上限定维; 运行时流数 = svcntw() */ struct ipsec_3des_docsis_hmac_sha256_sve_wide_data { uint8_t aes_key[AES_KEY_SIZE_V]; @@ -49,20 +50,21 @@ struct ipsec_3des_docsis_hmac_sha256_sve_wide_data { uint8_t *plaintext; uint8_t *golden_ciphertext; uint8_t golden_mac[NSTREAM][32]; - /* 7 条变体消息: 确定性派生(异或字节掩码), 填 SVE lane */ + /* N-1 条变体消息: 确定性派生(异或字节掩码), 填 SVE lane */ uint8_t *variant_msg[NSTREAM-1]; size_t data_size; + int nstreams; /* All-VL: init 期流数 = svl/4 (svcntw()), run 漂移守卫用 */ }; static size_t ipsec_data_size(struct test *test) { - int64_t v = sdc_knob_int_clamped(test, "data_size", 0, 1024, 16 << 20); + int64_t v = sdc_knob_int_clamped(test, "data_size", 0, 1024, 1900); if (v == 0) - sdc_env_knob_int_clamped("ipsec_3des_docsis_hmac_sha256_sve_wide", "data_size", &v, 1024, 16 << 20); + sdc_env_knob_int_clamped("ipsec_3des_docsis_hmac_sha256_sve_wide", "data_size", &v, 1024, 1900); if (v > 0) return (size_t)v; return (size_t)sdc_auto_linear_count(1, 4, 1.5, 600, thread_count(), - 1024, 16 << 20, test->id, "data_size"); + 1024, 1900, test->id, "data_size"); } static void des3_encrypt(const uint8_t *key, const uint8_t *iv, const uint8_t *in, uint8_t *out, size_t len) { @@ -86,54 +88,56 @@ static void des3_decrypt(const uint8_t *key, const uint8_t *iv, const uint8_t *i s_EVP_CIPHER_CTX_free(ctx); } -static void ipsec_3des_docsis_hmac_sha256_sve_wide_compute_golden(struct ipsec_3des_docsis_hmac_sha256_sve_wide_data *d) { +static int ipsec_3des_docsis_hmac_sha256_sve_wide_compute_golden(struct test *test, struct ipsec_3des_docsis_hmac_sha256_sve_wide_data *d) { des3_encrypt(d->aes_key, d->aes_iv, d->plaintext, d->golden_ciphertext, d->data_size); - /* SVE 多流 HMAC: lane0=密文(主语义), lane1..7=变体 */ + /* SVE 多流 HMAC: lane0=密文(主语义), lane1..N-1=变体 */ + const int n = (int)svcntw(); /* 与 kernel 内部推导一致 */ const uint8_t *keys[NSTREAM], *msgs[NSTREAM]; uint8_t variant_keys[NSTREAM][32]; - for (int s = 0; s < NSTREAM; ++s) { + for (int s = 0; s < n; ++s) { memcpy(variant_keys[s], d->hmac_key, 32); if (s > 0) variant_keys[s][0] ^= (uint8_t)s; /* 变体密钥: 确定性微扰 */ keys[s] = variant_keys[s]; } msgs[0] = d->golden_ciphertext; - for (int s = 1; s < NSTREAM; ++s) { + for (int s = 1; s < n; ++s) { /* 变体消息: 密文异或流掩码(确定性) */ for (size_t i = 0; i < d->data_size; ++i) d->variant_msg[s-1][i] = d->golden_ciphertext[i] ^ (uint8_t)(s * 0x5A + (i & 0x0F)); msgs[s] = d->variant_msg[s-1]; } - hmac_sha256_x8(d->golden_mac, keys, 32, msgs, d->data_size); + if (hmac_sha256_x8(d->golden_mac, keys, 32, msgs, d->data_size) != 0) { + log_error("%s: HMAC capacity overflow (data_size too large)", test->id); + return -1; + } + return 0; } static int ipsec_3des_docsis_hmac_sha256_sve_wide_init(struct test *test) { if (s_EVP_CIPHER_CTX_new && s_EVP_EncryptInit_ex && s_EVP_DecryptInit_ex && s_EVP_CIPHER_CTX_new && s_EVP_des_ede3_cbc()) { - unsigned long hwcap = getauxval(AT_HWCAP); - if ((hwcap & HWCAP_SVE) == 0) { - log_skip(CpuNotSupportedSkipCategory, - "to be implemented (placeholder): ARM SVE required for ipsec_3des_docsis_hmac_sha256_sve_wide"); - return EXIT_SKIP; - } - /* The multi-stream SVE HMAC kernels give one stream per SVE - * lane: 8x32-bit (sha1/224/256) needs svcntw()>=8 and - * 4x64-bit (384/512) needs svcntd()>=4 -- both mean VL>=256 - * bit. On shorter-VL SVE hardware (e.g. Neoverse N2, VL=128) - * lanes beyond VL are never written, so skip cleanly instead - * of comparing unwritten memory (target silicon: VL=256, see - * tests/cpu/sve/meson.build). */ - if (svcntw() < 8 || svcntd() < 4) { - log_skip(CpuNotSupportedSkipCategory, - "ARM SVE vector length %zu bytes < 32 required by the multi-stream HMAC kernels (svcntw=%zu svcntd=%zu)", - svcntb(), (size_t)svcntw(), (size_t)svcntd()); + /* All-VL Phase 2 P4: capability + 128..1024-bit range check in + * one framework-layer call (replaces the old lower-bound-only + * svcntw()>=8/svcntd()>=4 gate -- the 8.2 defect: at VL>=512 the + * fixed 8/4-wide W rows overflowed; the STREAMS_MAX-sized rows + + * this range gate close both ends). arm64_require_sve_vl returns + * the VL in BYTES. */ + const long svl = arm64_require_sve_vl(test, 128, 1024); + if (svl < 0) return EXIT_SKIP; + const int nstreams = (int)svl / 4; /* one stream per 32-bit lane (svcntw) */ + if (nstreams < 4 || nstreams > SVE_SHA_STREAMS_MAX_32) { /* [4, SVE_SHA_STREAMS_MAX_32], release-effective, NDEBUG-proof */ + log_error("%s: streams %d outside range — unit drift", test->id, nstreams); + return EXIT_FAILURE; } struct ipsec_3des_docsis_hmac_sha256_sve_wide_data *d = (struct ipsec_3des_docsis_hmac_sha256_sve_wide_data *)malloc(sizeof(*d)); if (!d) return EXIT_SKIP; + d->nstreams = nstreams; /* 工作集字节数: -O ipsec_3des_docsis_hmac_sha256_sve_wide.data_size=N / * SDC_IPSEC_3DES_DOCSIS_HMAC_SHA256_SVE_WIDE_DATA_SIZE 环境变量 / 自适应: - * 每线程明文+密文+golden 密文+解密缓冲+变体消息 ~= (3+NSTREAM-1)x, - * 占物理内存 60%, clamp [1024, 16M]。线性缓冲用 sdc_auto_linear_count(sqrt 缩放会欠缩放压力)。历史硬编码 1024。3DES-CBC + * 每线程明文+密文+golden 密文+解密缓冲+变体消息 ~= (3+nstreams-1)x, + * 占物理内存 60%, clamp [1024, 1900]——上限闭环 8.4: HMAC 组装 + * 缓冲 full[N][2048], 2048-64-9=1975B 安全边界。线性缓冲用 sdc_auto_linear_count(sqrt 缩放会欠缩放压力)。历史硬编码 1024。3DES-CBC * 无填充分组 8B, 统一 16B 对齐(天然满足 8B), 对齐后 < 1024 取 1024。 */ d->data_size = ipsec_data_size(test); d->data_size &= ~(size_t)15; @@ -147,7 +151,7 @@ static int ipsec_3des_docsis_hmac_sha256_sve_wide_init(struct test *test) { free(d); return EXIT_SKIP; } - for (int s = 0; s < NSTREAM - 1; ++s) { + for (int s = 0; s < d->nstreams - 1; ++s) { d->variant_msg[s] = (uint8_t *)malloc(d->data_size); if (!d->variant_msg[s]) { free(d->plaintext); @@ -162,7 +166,20 @@ static int ipsec_3des_docsis_hmac_sha256_sve_wide_init(struct test *test) { memset_random(d->aes_iv, AES_IV_SIZE_V); memset_random(d->hmac_key, 32); memset_random(d->plaintext, d->data_size); - ipsec_3des_docsis_hmac_sha256_sve_wide_compute_golden(d); + if (ipsec_3des_docsis_hmac_sha256_sve_wide_compute_golden(test, d) != 0) { + /* P2a (review Minor): init-side failure propagates via the + * return value so the framework reports a clean Failed + * (sandstone_run.cpp: ret > 0 -> TestResult::Failed). + * report_fail_msg is a noreturn thread-cancel that only + * belongs in the test_run worker context — in test_init it + * would surface as Skipped("Unexpected OS error"). */ + free(d->plaintext); + free(d->golden_ciphertext); + for (int s = 0; s < d->nstreams - 1; ++s) + free(d->variant_msg[s]); + free(d); + return EXIT_FAILURE; + } test->data = d; return EXIT_SUCCESS; } else { @@ -174,10 +191,22 @@ static int ipsec_3des_docsis_hmac_sha256_sve_wide_init(struct test *test) { static int ipsec_3des_docsis_hmac_sha256_sve_wide_run(struct test *test, int cpu) { (void)cpu; struct ipsec_3des_docsis_hmac_sha256_sve_wide_data *d = (struct ipsec_3des_docsis_hmac_sha256_sve_wide_data *)test->data; + /* All-VL policy #3: never compare goldens across an init/run VL + * mismatch. Drift means this deployment's per-thread VL differs from + * the init thread's (e.g. a VL-pinning launcher changing the + * thread-inheritance chain) — an environment mismatch, which skips + * with CpuNotSupported (same category as the init-side range gate in + * arm64_require_sve_vl) instead of polluting the SDC failure signal. */ + if ((int)svcntw() != d->nstreams) { + log_skip(CpuNotSupportedSkipCategory, + "%s: SVE VL drifted between init (%d streams) and run (%d streams) — environment mismatch, not SDC", + test->id, d->nstreams, (int)svcntw()); + return EXIT_SKIP; + } uint8_t *ciphertext = (uint8_t *)malloc(d->data_size); uint8_t *decrypted = (uint8_t *)malloc(d->data_size); uint8_t mac[NSTREAM][32]; - uint8_t *variant_msg = (uint8_t *)malloc((NSTREAM-1) * d->data_size); + uint8_t *variant_msg = (uint8_t *)malloc((d->nstreams-1) * d->data_size); /* R2b (B2-003): per-thread plaintext + golden, re-rolled every * iteration (thread-safe: shared d is only read after init; the @@ -201,7 +230,7 @@ static int ipsec_3des_docsis_hmac_sha256_sve_wide_run(struct test *test, int cpu "out of memory for per-thread buffers"); return EXIT_SKIP; } - for (int s = 0; s < NSTREAM - 1; ++s) { + for (int s = 0; s < d->nstreams - 1; ++s) { local_d.variant_msg[s] = (uint8_t *)malloc(d->data_size); if (!local_d.variant_msg[s]) { free(local_d.plaintext); @@ -234,27 +263,32 @@ static int ipsec_3des_docsis_hmac_sha256_sve_wide_run(struct test *test, int cpu /* SVE 多流 HMAC (主消息 + 变体) */ { const uint8_t *keys[NSTREAM], *msgs[NSTREAM]; + const int n = (int)svcntw(); /* 与 kernel 内部推导一致 */ uint8_t variant_keys[NSTREAM][32]; - for (int s = 0; s < NSTREAM; ++s) { + for (int s = 0; s < n; ++s) { memcpy(variant_keys[s], d->hmac_key, 32); if (s > 0) variant_keys[s][0] ^= (uint8_t)s; keys[s] = variant_keys[s]; } msgs[0] = ciphertext; - for (int s = 1; s < NSTREAM; ++s) { + for (int s = 1; s < n; ++s) { for (size_t i = 0; i < d->data_size; ++i) variant_msg[(s-1)*d->data_size + i] = ciphertext[i] ^ (uint8_t)(s * 0x5A + (i & 0x0F)); msgs[s] = variant_msg + (s-1)*d->data_size; } - hmac_sha256_x8(mac, keys, 32, msgs, d->data_size); + if (hmac_sha256_x8(mac, keys, 32, msgs, d->data_size) != 0) { + log_error("%s: HMAC capacity overflow (data_size too large)", test->id); + report_fail_msg("%s: hmac assembly over capacity", test->id); + } } - for (int s = 0; s < NSTREAM; ++s) + for (int s = 0; s < d->nstreams; ++s) memcmp_or_fail(mac[s], d->golden_mac[s], 32, "HMAC-SHA256 digest mismatch (frozen replay)"); } else { /* re-roll: fresh plaintext + same-iteration golden (EVP encrypt + * SVE HMAC recomputed for this thread's data) */ memset_random(local_d.plaintext, d->data_size); - ipsec_3des_docsis_hmac_sha256_sve_wide_compute_golden(&local_d); + if (ipsec_3des_docsis_hmac_sha256_sve_wide_compute_golden(test, &local_d) != 0) + report_fail_msg("%s: hmac assembly over capacity", test->id); des3_encrypt(local_d.aes_key, local_d.aes_iv, local_d.plaintext, ciphertext, d->data_size); memcmp_or_fail(ciphertext, local_d.golden_ciphertext, d->data_size, "ciphertext mismatch (SVE)"); des3_decrypt(local_d.aes_key, local_d.aes_iv, ciphertext, decrypted, d->data_size); @@ -263,21 +297,25 @@ static int ipsec_3des_docsis_hmac_sha256_sve_wide_run(struct test *test, int cpu /* SVE 多流 HMAC (主消息 + 变体) */ { const uint8_t *keys[NSTREAM], *msgs[NSTREAM]; + const int n = (int)svcntw(); /* 与 kernel 内部推导一致 */ uint8_t variant_keys[NSTREAM][32]; - for (int s = 0; s < NSTREAM; ++s) { + for (int s = 0; s < n; ++s) { memcpy(variant_keys[s], local_d.hmac_key, 32); if (s > 0) variant_keys[s][0] ^= (uint8_t)s; keys[s] = variant_keys[s]; } msgs[0] = ciphertext; - for (int s = 1; s < NSTREAM; ++s) { + for (int s = 1; s < n; ++s) { for (size_t i = 0; i < d->data_size; ++i) variant_msg[(s-1)*d->data_size + i] = ciphertext[i] ^ (uint8_t)(s * 0x5A + (i & 0x0F)); msgs[s] = variant_msg + (s-1)*d->data_size; } - hmac_sha256_x8(mac, keys, 32, msgs, d->data_size); + if (hmac_sha256_x8(mac, keys, 32, msgs, d->data_size) != 0) { + log_error("%s: HMAC capacity overflow (data_size too large)", test->id); + report_fail_msg("%s: hmac assembly over capacity", test->id); + } } - for (int s = 0; s < NSTREAM; ++s) + for (int s = 0; s < d->nstreams; ++s) memcmp_or_fail(mac[s], local_d.golden_mac[s], 32, "HMAC-SHA256 digest mismatch (SVE)"); } ++iter; @@ -285,7 +323,7 @@ static int ipsec_3des_docsis_hmac_sha256_sve_wide_run(struct test *test, int cpu free(ciphertext); free(decrypted); free(variant_msg); - for (int s = 0; s < NSTREAM - 1; ++s) + for (int s = 0; s < d->nstreams - 1; ++s) free(local_d.variant_msg[s]); free(local_d.plaintext); free(local_d.golden_ciphertext); @@ -297,7 +335,7 @@ static int ipsec_3des_docsis_hmac_sha256_sve_wide_cleanup(struct test *test) { if (d) { free(d->plaintext); free(d->golden_ciphertext); - for (int s = 0; s < NSTREAM - 1; ++s) + for (int s = 0; s < d->nstreams - 1; ++s) free(d->variant_msg[s]); } free(d); diff --git a/tests/cpu/sve/ipsec/ipsec_3des_docsis_hmac_sha384_sve_wide.cpp b/tests/cpu/sve/ipsec/ipsec_3des_docsis_hmac_sha384_sve_wide.cpp index b8e763bf4..c601daa03 100644 --- a/tests/cpu/sve/ipsec/ipsec_3des_docsis_hmac_sha384_sve_wide.cpp +++ b/tests/cpu/sve/ipsec/ipsec_3des_docsis_hmac_sha384_sve_wide.cpp @@ -7,12 +7,17 @@ * @parblock * SVE port of ipsec_3des_docsis_hmac_sha384_avx512: AES/3DES encryption stays on OpenSSL EVP while * the HMAC-SHA384 MAC runs on the multi-stream SVE HMAC implementation - * (verified bit-for-bit against OpenSSL; see scripts/sve-sha-verify). - * Eight independent HMAC lanes run in parallel on SVE vector lanes: lane 0 + * (HMAC construction validated same-source in gem5-fi across 128..1024-bit against an + * independent scalar oracle and RFC 2202/4231 vectors; the underlying SHA + * kernels were verified bit-for-bit against OpenSSL in the original + * fixed-stream form — see scripts/sve-sha-verify and + * docs/research/progress-log.md). + * N = svcntd() (64-bit family) independent HMAC lanes run in parallel on + * SVE vector lanes (all-VL 128..1024-bit, one message per 64-bit lane): lane 0 * carries the original ciphertext message (MAC semantics identical to the - * original test), lanes 1-7 carry deterministic key-stream variant + * original test), lanes 1..N-1 carry deterministic key-stream variant * messages that fill the remaining lanes — same computation family, - * golden values precomputed for all 8. Ciphertext, MACs and decrypted + * golden values precomputed for all N. Ciphertext, MACs and decrypted * data are compared against golden values on every iteration to detect * silent data corruption. Iterations alternate between a re-rolled input * with a same-core recomputed golden and a frozen replay of the init-time @@ -28,19 +33,15 @@ #include "sandstone_ssl.h" #include "sve_hmac.h" #include "sdc_stress_config.h" -#include +#include #include #include -#ifndef HWCAP_SVE -#define HWCAP_SVE (1 << 22) -#endif - #define HMAC_KEY_SIZE_V (48) #define SHA384_DIGEST_SIZE (48) #define AES_KEY_SIZE_V (24) #define AES_IV_SIZE_V (8) -#define NSTREAM (4) +#define NSTREAM (SVE_SHA_STREAMS_MAX_64) /* All-VL: 数组按 1024-bit 上限定维; 运行时流数 = svcntd() */ struct ipsec_3des_docsis_hmac_sha384_sve_wide_data { uint8_t aes_key[AES_KEY_SIZE_V]; @@ -49,20 +50,21 @@ struct ipsec_3des_docsis_hmac_sha384_sve_wide_data { uint8_t *plaintext; uint8_t *golden_ciphertext; uint8_t golden_mac[NSTREAM][48]; - /* 7 条变体消息: 确定性派生(异或字节掩码), 填 SVE lane */ + /* N-1 条变体消息: 确定性派生(异或字节掩码), 填 SVE lane */ uint8_t *variant_msg[NSTREAM-1]; size_t data_size; + int nstreams; /* All-VL: init 期流数 = svl/8 (svcntd()), run 漂移守卫用 */ }; static size_t ipsec_data_size(struct test *test) { - int64_t v = sdc_knob_int_clamped(test, "data_size", 0, 1024, 16 << 20); + int64_t v = sdc_knob_int_clamped(test, "data_size", 0, 1024, 1830); if (v == 0) - sdc_env_knob_int_clamped("ipsec_3des_docsis_hmac_sha384_sve_wide", "data_size", &v, 1024, 16 << 20); + sdc_env_knob_int_clamped("ipsec_3des_docsis_hmac_sha384_sve_wide", "data_size", &v, 1024, 1830); if (v > 0) return (size_t)v; return (size_t)sdc_auto_linear_count(1, 4, 1.5, 600, thread_count(), - 1024, 16 << 20, test->id, "data_size"); + 1024, 1830, test->id, "data_size"); } static void des3_encrypt(const uint8_t *key, const uint8_t *iv, const uint8_t *in, uint8_t *out, size_t len) { @@ -86,54 +88,56 @@ static void des3_decrypt(const uint8_t *key, const uint8_t *iv, const uint8_t *i s_EVP_CIPHER_CTX_free(ctx); } -static void ipsec_3des_docsis_hmac_sha384_sve_wide_compute_golden(struct ipsec_3des_docsis_hmac_sha384_sve_wide_data *d) { +static int ipsec_3des_docsis_hmac_sha384_sve_wide_compute_golden(struct test *test, struct ipsec_3des_docsis_hmac_sha384_sve_wide_data *d) { des3_encrypt(d->aes_key, d->aes_iv, d->plaintext, d->golden_ciphertext, d->data_size); - /* SVE 多流 HMAC: lane0=密文(主语义), lane1..7=变体 */ + /* SVE 多流 HMAC: lane0=密文(主语义), lane1..N-1=变体 */ + const int n = (int)svcntd(); /* 与 kernel 内部推导一致 */ const uint8_t *keys[NSTREAM], *msgs[NSTREAM]; uint8_t variant_keys[NSTREAM][48]; - for (int s = 0; s < NSTREAM; ++s) { + for (int s = 0; s < n; ++s) { memcpy(variant_keys[s], d->hmac_key, 48); if (s > 0) variant_keys[s][0] ^= (uint8_t)s; /* 变体密钥: 确定性微扰 */ keys[s] = variant_keys[s]; } msgs[0] = d->golden_ciphertext; - for (int s = 1; s < NSTREAM; ++s) { + for (int s = 1; s < n; ++s) { /* 变体消息: 密文异或流掩码(确定性) */ for (size_t i = 0; i < d->data_size; ++i) d->variant_msg[s-1][i] = d->golden_ciphertext[i] ^ (uint8_t)(s * 0x5A + (i & 0x0F)); msgs[s] = d->variant_msg[s-1]; } - hmac_sha384_x4(d->golden_mac, keys, 48, msgs, d->data_size); + if (hmac_sha384_x4(d->golden_mac, keys, 48, msgs, d->data_size) != 0) { + log_error("%s: HMAC capacity overflow (data_size too large)", test->id); + return -1; + } + return 0; } static int ipsec_3des_docsis_hmac_sha384_sve_wide_init(struct test *test) { if (s_EVP_CIPHER_CTX_new && s_EVP_EncryptInit_ex && s_EVP_DecryptInit_ex && s_EVP_CIPHER_CTX_new && s_EVP_des_ede3_cbc()) { - unsigned long hwcap = getauxval(AT_HWCAP); - if ((hwcap & HWCAP_SVE) == 0) { - log_skip(CpuNotSupportedSkipCategory, - "to be implemented (placeholder): ARM SVE required for ipsec_3des_docsis_hmac_sha384_sve_wide"); - return EXIT_SKIP; - } - /* The multi-stream SVE HMAC kernels give one stream per SVE - * lane: 8x32-bit (sha1/224/256) needs svcntw()>=8 and - * 4x64-bit (384/512) needs svcntd()>=4 -- both mean VL>=256 - * bit. On shorter-VL SVE hardware (e.g. Neoverse N2, VL=128) - * lanes beyond VL are never written, so skip cleanly instead - * of comparing unwritten memory (target silicon: VL=256, see - * tests/cpu/sve/meson.build). */ - if (svcntw() < 8 || svcntd() < 4) { - log_skip(CpuNotSupportedSkipCategory, - "ARM SVE vector length %zu bytes < 32 required by the multi-stream HMAC kernels (svcntw=%zu svcntd=%zu)", - svcntb(), (size_t)svcntw(), (size_t)svcntd()); + /* All-VL Phase 2 P4: capability + 128..1024-bit range check in + * one framework-layer call (replaces the old lower-bound-only + * svcntw()>=8/svcntd()>=4 gate -- the 8.2 defect: at VL>=512 the + * fixed 8/4-wide W rows overflowed; the STREAMS_MAX-sized rows + + * this range gate close both ends). arm64_require_sve_vl returns + * the VL in BYTES. */ + const long svl = arm64_require_sve_vl(test, 128, 1024); + if (svl < 0) return EXIT_SKIP; + const int nstreams = (int)svl / 8; /* one stream per 64-bit lane (svcntd) */ + if (nstreams < 2 || nstreams > SVE_SHA_STREAMS_MAX_64) { /* [2, SVE_SHA_STREAMS_MAX_64], release-effective, NDEBUG-proof */ + log_error("%s: streams %d outside range — unit drift", test->id, nstreams); + return EXIT_FAILURE; } struct ipsec_3des_docsis_hmac_sha384_sve_wide_data *d = (struct ipsec_3des_docsis_hmac_sha384_sve_wide_data *)malloc(sizeof(*d)); if (!d) return EXIT_SKIP; + d->nstreams = nstreams; /* 工作集字节数: -O ipsec_3des_docsis_hmac_sha384_sve_wide.data_size=N / * SDC_IPSEC_3DES_DOCSIS_HMAC_SHA384_SVE_WIDE_DATA_SIZE 环境变量 / 自适应: - * 每线程明文+密文+golden 密文+解密缓冲+变体消息 ~= (3+NSTREAM-1)x, - * 占物理内存 60%, clamp [1024, 16M]。线性缓冲用 sdc_auto_linear_count(sqrt 缩放会欠缩放压力)。历史硬编码 1024。3DES-CBC + * 每线程明文+密文+golden 密文+解密缓冲+变体消息 ~= (3+nstreams-1)x, + * 占物理内存 60%, clamp [1024, 1830]——上限闭环 8.4: HMAC 组装 + * 缓冲 full[N][2048], 2048-128-17=1903B 安全边界。线性缓冲用 sdc_auto_linear_count(sqrt 缩放会欠缩放压力)。历史硬编码 1024。3DES-CBC * 无填充分组 8B, 统一 16B 对齐(天然满足 8B), 对齐后 < 1024 取 1024。 */ d->data_size = ipsec_data_size(test); d->data_size &= ~(size_t)15; @@ -147,7 +151,7 @@ static int ipsec_3des_docsis_hmac_sha384_sve_wide_init(struct test *test) { free(d); return EXIT_SKIP; } - for (int s = 0; s < NSTREAM - 1; ++s) { + for (int s = 0; s < d->nstreams - 1; ++s) { d->variant_msg[s] = (uint8_t *)malloc(d->data_size); if (!d->variant_msg[s]) { free(d->plaintext); @@ -162,7 +166,20 @@ static int ipsec_3des_docsis_hmac_sha384_sve_wide_init(struct test *test) { memset_random(d->aes_iv, AES_IV_SIZE_V); memset_random(d->hmac_key, 48); memset_random(d->plaintext, d->data_size); - ipsec_3des_docsis_hmac_sha384_sve_wide_compute_golden(d); + if (ipsec_3des_docsis_hmac_sha384_sve_wide_compute_golden(test, d) != 0) { + /* P2a (review Minor): init-side failure propagates via the + * return value so the framework reports a clean Failed + * (sandstone_run.cpp: ret > 0 -> TestResult::Failed). + * report_fail_msg is a noreturn thread-cancel that only + * belongs in the test_run worker context — in test_init it + * would surface as Skipped("Unexpected OS error"). */ + free(d->plaintext); + free(d->golden_ciphertext); + for (int s = 0; s < d->nstreams - 1; ++s) + free(d->variant_msg[s]); + free(d); + return EXIT_FAILURE; + } test->data = d; return EXIT_SUCCESS; } else { @@ -174,10 +191,22 @@ static int ipsec_3des_docsis_hmac_sha384_sve_wide_init(struct test *test) { static int ipsec_3des_docsis_hmac_sha384_sve_wide_run(struct test *test, int cpu) { (void)cpu; struct ipsec_3des_docsis_hmac_sha384_sve_wide_data *d = (struct ipsec_3des_docsis_hmac_sha384_sve_wide_data *)test->data; + /* All-VL policy #3: never compare goldens across an init/run VL + * mismatch. Drift means this deployment's per-thread VL differs from + * the init thread's (e.g. a VL-pinning launcher changing the + * thread-inheritance chain) — an environment mismatch, which skips + * with CpuNotSupported (same category as the init-side range gate in + * arm64_require_sve_vl) instead of polluting the SDC failure signal. */ + if ((int)svcntd() != d->nstreams) { + log_skip(CpuNotSupportedSkipCategory, + "%s: SVE VL drifted between init (%d streams) and run (%d streams) — environment mismatch, not SDC", + test->id, d->nstreams, (int)svcntd()); + return EXIT_SKIP; + } uint8_t *ciphertext = (uint8_t *)malloc(d->data_size); uint8_t *decrypted = (uint8_t *)malloc(d->data_size); uint8_t mac[NSTREAM][48]; - uint8_t *variant_msg = (uint8_t *)malloc((NSTREAM-1) * d->data_size); + uint8_t *variant_msg = (uint8_t *)malloc((d->nstreams-1) * d->data_size); /* R2b (B2-003): per-thread plaintext + golden, re-rolled every * iteration (thread-safe: shared d is only read after init; the @@ -201,7 +230,7 @@ static int ipsec_3des_docsis_hmac_sha384_sve_wide_run(struct test *test, int cpu "out of memory for per-thread buffers"); return EXIT_SKIP; } - for (int s = 0; s < NSTREAM - 1; ++s) { + for (int s = 0; s < d->nstreams - 1; ++s) { local_d.variant_msg[s] = (uint8_t *)malloc(d->data_size); if (!local_d.variant_msg[s]) { free(local_d.plaintext); @@ -234,27 +263,32 @@ static int ipsec_3des_docsis_hmac_sha384_sve_wide_run(struct test *test, int cpu /* SVE 多流 HMAC (主消息 + 变体) */ { const uint8_t *keys[NSTREAM], *msgs[NSTREAM]; + const int n = (int)svcntd(); /* 与 kernel 内部推导一致 */ uint8_t variant_keys[NSTREAM][48]; - for (int s = 0; s < NSTREAM; ++s) { + for (int s = 0; s < n; ++s) { memcpy(variant_keys[s], d->hmac_key, 48); if (s > 0) variant_keys[s][0] ^= (uint8_t)s; keys[s] = variant_keys[s]; } msgs[0] = ciphertext; - for (int s = 1; s < NSTREAM; ++s) { + for (int s = 1; s < n; ++s) { for (size_t i = 0; i < d->data_size; ++i) variant_msg[(s-1)*d->data_size + i] = ciphertext[i] ^ (uint8_t)(s * 0x5A + (i & 0x0F)); msgs[s] = variant_msg + (s-1)*d->data_size; } - hmac_sha384_x4(mac, keys, 48, msgs, d->data_size); + if (hmac_sha384_x4(mac, keys, 48, msgs, d->data_size) != 0) { + log_error("%s: HMAC capacity overflow (data_size too large)", test->id); + report_fail_msg("%s: hmac assembly over capacity", test->id); + } } - for (int s = 0; s < NSTREAM; ++s) + for (int s = 0; s < d->nstreams; ++s) memcmp_or_fail(mac[s], d->golden_mac[s], 48, "HMAC-SHA384 digest mismatch (frozen replay)"); } else { /* re-roll: fresh plaintext + same-iteration golden (EVP encrypt + * SVE HMAC recomputed for this thread's data) */ memset_random(local_d.plaintext, d->data_size); - ipsec_3des_docsis_hmac_sha384_sve_wide_compute_golden(&local_d); + if (ipsec_3des_docsis_hmac_sha384_sve_wide_compute_golden(test, &local_d) != 0) + report_fail_msg("%s: hmac assembly over capacity", test->id); des3_encrypt(local_d.aes_key, local_d.aes_iv, local_d.plaintext, ciphertext, d->data_size); memcmp_or_fail(ciphertext, local_d.golden_ciphertext, d->data_size, "ciphertext mismatch (SVE)"); des3_decrypt(local_d.aes_key, local_d.aes_iv, ciphertext, decrypted, d->data_size); @@ -263,21 +297,25 @@ static int ipsec_3des_docsis_hmac_sha384_sve_wide_run(struct test *test, int cpu /* SVE 多流 HMAC (主消息 + 变体) */ { const uint8_t *keys[NSTREAM], *msgs[NSTREAM]; + const int n = (int)svcntd(); /* 与 kernel 内部推导一致 */ uint8_t variant_keys[NSTREAM][48]; - for (int s = 0; s < NSTREAM; ++s) { + for (int s = 0; s < n; ++s) { memcpy(variant_keys[s], local_d.hmac_key, 48); if (s > 0) variant_keys[s][0] ^= (uint8_t)s; keys[s] = variant_keys[s]; } msgs[0] = ciphertext; - for (int s = 1; s < NSTREAM; ++s) { + for (int s = 1; s < n; ++s) { for (size_t i = 0; i < d->data_size; ++i) variant_msg[(s-1)*d->data_size + i] = ciphertext[i] ^ (uint8_t)(s * 0x5A + (i & 0x0F)); msgs[s] = variant_msg + (s-1)*d->data_size; } - hmac_sha384_x4(mac, keys, 48, msgs, d->data_size); + if (hmac_sha384_x4(mac, keys, 48, msgs, d->data_size) != 0) { + log_error("%s: HMAC capacity overflow (data_size too large)", test->id); + report_fail_msg("%s: hmac assembly over capacity", test->id); + } } - for (int s = 0; s < NSTREAM; ++s) + for (int s = 0; s < d->nstreams; ++s) memcmp_or_fail(mac[s], local_d.golden_mac[s], 48, "HMAC-SHA384 digest mismatch (SVE)"); } ++iter; @@ -285,7 +323,7 @@ static int ipsec_3des_docsis_hmac_sha384_sve_wide_run(struct test *test, int cpu free(ciphertext); free(decrypted); free(variant_msg); - for (int s = 0; s < NSTREAM - 1; ++s) + for (int s = 0; s < d->nstreams - 1; ++s) free(local_d.variant_msg[s]); free(local_d.plaintext); free(local_d.golden_ciphertext); @@ -297,7 +335,7 @@ static int ipsec_3des_docsis_hmac_sha384_sve_wide_cleanup(struct test *test) { if (d) { free(d->plaintext); free(d->golden_ciphertext); - for (int s = 0; s < NSTREAM - 1; ++s) + for (int s = 0; s < d->nstreams - 1; ++s) free(d->variant_msg[s]); } free(d); diff --git a/tests/cpu/sve/ipsec/ipsec_3des_docsis_hmac_sha512_sve_wide.cpp b/tests/cpu/sve/ipsec/ipsec_3des_docsis_hmac_sha512_sve_wide.cpp index 06ed3ae98..dd4a312ad 100644 --- a/tests/cpu/sve/ipsec/ipsec_3des_docsis_hmac_sha512_sve_wide.cpp +++ b/tests/cpu/sve/ipsec/ipsec_3des_docsis_hmac_sha512_sve_wide.cpp @@ -7,12 +7,17 @@ * @parblock * SVE port of ipsec_3des_docsis_hmac_sha512_avx512: AES/3DES encryption stays on OpenSSL EVP while * the HMAC-SHA512 MAC runs on the multi-stream SVE HMAC implementation - * (verified bit-for-bit against OpenSSL; see scripts/sve-sha-verify). - * Eight independent HMAC lanes run in parallel on SVE vector lanes: lane 0 + * (HMAC construction validated same-source in gem5-fi across 128..1024-bit against an + * independent scalar oracle and RFC 2202/4231 vectors; the underlying SHA + * kernels were verified bit-for-bit against OpenSSL in the original + * fixed-stream form — see scripts/sve-sha-verify and + * docs/research/progress-log.md). + * N = svcntd() (64-bit family) independent HMAC lanes run in parallel on + * SVE vector lanes (all-VL 128..1024-bit, one message per 64-bit lane): lane 0 * carries the original ciphertext message (MAC semantics identical to the - * original test), lanes 1-7 carry deterministic key-stream variant + * original test), lanes 1..N-1 carry deterministic key-stream variant * messages that fill the remaining lanes — same computation family, - * golden values precomputed for all 8. Ciphertext, MACs and decrypted + * golden values precomputed for all N. Ciphertext, MACs and decrypted * data are compared against golden values on every iteration to detect * silent data corruption. Iterations alternate between a re-rolled input * with a same-core recomputed golden and a frozen replay of the init-time @@ -28,19 +33,15 @@ #include "sandstone_ssl.h" #include "sve_hmac.h" #include "sdc_stress_config.h" -#include +#include #include #include -#ifndef HWCAP_SVE -#define HWCAP_SVE (1 << 22) -#endif - #define HMAC_KEY_SIZE_V (64) #define SHA512_DIGEST_SIZE (64) #define AES_KEY_SIZE_V (24) #define AES_IV_SIZE_V (8) -#define NSTREAM (4) +#define NSTREAM (SVE_SHA_STREAMS_MAX_64) /* All-VL: 数组按 1024-bit 上限定维; 运行时流数 = svcntd() */ struct ipsec_3des_docsis_hmac_sha512_sve_wide_data { uint8_t aes_key[AES_KEY_SIZE_V]; @@ -49,20 +50,21 @@ struct ipsec_3des_docsis_hmac_sha512_sve_wide_data { uint8_t *plaintext; uint8_t *golden_ciphertext; uint8_t golden_mac[NSTREAM][64]; - /* 7 条变体消息: 确定性派生(异或字节掩码), 填 SVE lane */ + /* N-1 条变体消息: 确定性派生(异或字节掩码), 填 SVE lane */ uint8_t *variant_msg[NSTREAM-1]; size_t data_size; + int nstreams; /* All-VL: init 期流数 = svl/8 (svcntd()), run 漂移守卫用 */ }; static size_t ipsec_data_size(struct test *test) { - int64_t v = sdc_knob_int_clamped(test, "data_size", 0, 1024, 16 << 20); + int64_t v = sdc_knob_int_clamped(test, "data_size", 0, 1024, 1830); if (v == 0) - sdc_env_knob_int_clamped("ipsec_3des_docsis_hmac_sha512_sve_wide", "data_size", &v, 1024, 16 << 20); + sdc_env_knob_int_clamped("ipsec_3des_docsis_hmac_sha512_sve_wide", "data_size", &v, 1024, 1830); if (v > 0) return (size_t)v; return (size_t)sdc_auto_linear_count(1, 4, 1.5, 600, thread_count(), - 1024, 16 << 20, test->id, "data_size"); + 1024, 1830, test->id, "data_size"); } static void des3_encrypt(const uint8_t *key, const uint8_t *iv, const uint8_t *in, uint8_t *out, size_t len) { @@ -86,54 +88,56 @@ static void des3_decrypt(const uint8_t *key, const uint8_t *iv, const uint8_t *i s_EVP_CIPHER_CTX_free(ctx); } -static void ipsec_3des_docsis_hmac_sha512_sve_wide_compute_golden(struct ipsec_3des_docsis_hmac_sha512_sve_wide_data *d) { +static int ipsec_3des_docsis_hmac_sha512_sve_wide_compute_golden(struct test *test, struct ipsec_3des_docsis_hmac_sha512_sve_wide_data *d) { des3_encrypt(d->aes_key, d->aes_iv, d->plaintext, d->golden_ciphertext, d->data_size); - /* SVE 多流 HMAC: lane0=密文(主语义), lane1..7=变体 */ + /* SVE 多流 HMAC: lane0=密文(主语义), lane1..N-1=变体 */ + const int n = (int)svcntd(); /* 与 kernel 内部推导一致 */ const uint8_t *keys[NSTREAM], *msgs[NSTREAM]; uint8_t variant_keys[NSTREAM][64]; - for (int s = 0; s < NSTREAM; ++s) { + for (int s = 0; s < n; ++s) { memcpy(variant_keys[s], d->hmac_key, 64); if (s > 0) variant_keys[s][0] ^= (uint8_t)s; /* 变体密钥: 确定性微扰 */ keys[s] = variant_keys[s]; } msgs[0] = d->golden_ciphertext; - for (int s = 1; s < NSTREAM; ++s) { + for (int s = 1; s < n; ++s) { /* 变体消息: 密文异或流掩码(确定性) */ for (size_t i = 0; i < d->data_size; ++i) d->variant_msg[s-1][i] = d->golden_ciphertext[i] ^ (uint8_t)(s * 0x5A + (i & 0x0F)); msgs[s] = d->variant_msg[s-1]; } - hmac_sha512_x4(d->golden_mac, keys, 64, msgs, d->data_size); + if (hmac_sha512_x4(d->golden_mac, keys, 64, msgs, d->data_size) != 0) { + log_error("%s: HMAC capacity overflow (data_size too large)", test->id); + return -1; + } + return 0; } static int ipsec_3des_docsis_hmac_sha512_sve_wide_init(struct test *test) { if (s_EVP_CIPHER_CTX_new && s_EVP_EncryptInit_ex && s_EVP_DecryptInit_ex && s_EVP_CIPHER_CTX_new && s_EVP_des_ede3_cbc()) { - unsigned long hwcap = getauxval(AT_HWCAP); - if ((hwcap & HWCAP_SVE) == 0) { - log_skip(CpuNotSupportedSkipCategory, - "to be implemented (placeholder): ARM SVE required for ipsec_3des_docsis_hmac_sha512_sve_wide"); - return EXIT_SKIP; - } - /* The multi-stream SVE HMAC kernels give one stream per SVE - * lane: 8x32-bit (sha1/224/256) needs svcntw()>=8 and - * 4x64-bit (384/512) needs svcntd()>=4 -- both mean VL>=256 - * bit. On shorter-VL SVE hardware (e.g. Neoverse N2, VL=128) - * lanes beyond VL are never written, so skip cleanly instead - * of comparing unwritten memory (target silicon: VL=256, see - * tests/cpu/sve/meson.build). */ - if (svcntw() < 8 || svcntd() < 4) { - log_skip(CpuNotSupportedSkipCategory, - "ARM SVE vector length %zu bytes < 32 required by the multi-stream HMAC kernels (svcntw=%zu svcntd=%zu)", - svcntb(), (size_t)svcntw(), (size_t)svcntd()); + /* All-VL Phase 2 P4: capability + 128..1024-bit range check in + * one framework-layer call (replaces the old lower-bound-only + * svcntw()>=8/svcntd()>=4 gate -- the 8.2 defect: at VL>=512 the + * fixed 8/4-wide W rows overflowed; the STREAMS_MAX-sized rows + + * this range gate close both ends). arm64_require_sve_vl returns + * the VL in BYTES. */ + const long svl = arm64_require_sve_vl(test, 128, 1024); + if (svl < 0) return EXIT_SKIP; + const int nstreams = (int)svl / 8; /* one stream per 64-bit lane (svcntd) */ + if (nstreams < 2 || nstreams > SVE_SHA_STREAMS_MAX_64) { /* [2, SVE_SHA_STREAMS_MAX_64], release-effective, NDEBUG-proof */ + log_error("%s: streams %d outside range — unit drift", test->id, nstreams); + return EXIT_FAILURE; } struct ipsec_3des_docsis_hmac_sha512_sve_wide_data *d = (struct ipsec_3des_docsis_hmac_sha512_sve_wide_data *)malloc(sizeof(*d)); if (!d) return EXIT_SKIP; + d->nstreams = nstreams; /* 工作集字节数: -O ipsec_3des_docsis_hmac_sha512_sve_wide.data_size=N / * SDC_IPSEC_3DES_DOCSIS_HMAC_SHA512_SVE_WIDE_DATA_SIZE 环境变量 / 自适应: - * 每线程明文+密文+golden 密文+解密缓冲+变体消息 ~= (3+NSTREAM-1)x, - * 占物理内存 60%, clamp [1024, 16M]。线性缓冲用 sdc_auto_linear_count(sqrt 缩放会欠缩放压力)。历史硬编码 1024。3DES-CBC + * 每线程明文+密文+golden 密文+解密缓冲+变体消息 ~= (3+nstreams-1)x, + * 占物理内存 60%, clamp [1024, 1830]——上限闭环 8.4: HMAC 组装 + * 缓冲 full[N][2048], 2048-128-17=1903B 安全边界。线性缓冲用 sdc_auto_linear_count(sqrt 缩放会欠缩放压力)。历史硬编码 1024。3DES-CBC * 无填充分组 8B, 统一 16B 对齐(天然满足 8B), 对齐后 < 1024 取 1024。 */ d->data_size = ipsec_data_size(test); d->data_size &= ~(size_t)15; @@ -147,7 +151,7 @@ static int ipsec_3des_docsis_hmac_sha512_sve_wide_init(struct test *test) { free(d); return EXIT_SKIP; } - for (int s = 0; s < NSTREAM - 1; ++s) { + for (int s = 0; s < d->nstreams - 1; ++s) { d->variant_msg[s] = (uint8_t *)malloc(d->data_size); if (!d->variant_msg[s]) { free(d->plaintext); @@ -162,7 +166,20 @@ static int ipsec_3des_docsis_hmac_sha512_sve_wide_init(struct test *test) { memset_random(d->aes_iv, AES_IV_SIZE_V); memset_random(d->hmac_key, 64); memset_random(d->plaintext, d->data_size); - ipsec_3des_docsis_hmac_sha512_sve_wide_compute_golden(d); + if (ipsec_3des_docsis_hmac_sha512_sve_wide_compute_golden(test, d) != 0) { + /* P2a (review Minor): init-side failure propagates via the + * return value so the framework reports a clean Failed + * (sandstone_run.cpp: ret > 0 -> TestResult::Failed). + * report_fail_msg is a noreturn thread-cancel that only + * belongs in the test_run worker context — in test_init it + * would surface as Skipped("Unexpected OS error"). */ + free(d->plaintext); + free(d->golden_ciphertext); + for (int s = 0; s < d->nstreams - 1; ++s) + free(d->variant_msg[s]); + free(d); + return EXIT_FAILURE; + } test->data = d; return EXIT_SUCCESS; } else { @@ -174,10 +191,22 @@ static int ipsec_3des_docsis_hmac_sha512_sve_wide_init(struct test *test) { static int ipsec_3des_docsis_hmac_sha512_sve_wide_run(struct test *test, int cpu) { (void)cpu; struct ipsec_3des_docsis_hmac_sha512_sve_wide_data *d = (struct ipsec_3des_docsis_hmac_sha512_sve_wide_data *)test->data; + /* All-VL policy #3: never compare goldens across an init/run VL + * mismatch. Drift means this deployment's per-thread VL differs from + * the init thread's (e.g. a VL-pinning launcher changing the + * thread-inheritance chain) — an environment mismatch, which skips + * with CpuNotSupported (same category as the init-side range gate in + * arm64_require_sve_vl) instead of polluting the SDC failure signal. */ + if ((int)svcntd() != d->nstreams) { + log_skip(CpuNotSupportedSkipCategory, + "%s: SVE VL drifted between init (%d streams) and run (%d streams) — environment mismatch, not SDC", + test->id, d->nstreams, (int)svcntd()); + return EXIT_SKIP; + } uint8_t *ciphertext = (uint8_t *)malloc(d->data_size); uint8_t *decrypted = (uint8_t *)malloc(d->data_size); uint8_t mac[NSTREAM][64]; - uint8_t *variant_msg = (uint8_t *)malloc((NSTREAM-1) * d->data_size); + uint8_t *variant_msg = (uint8_t *)malloc((d->nstreams-1) * d->data_size); /* R2b (B2-003): per-thread plaintext + golden, re-rolled every * iteration (thread-safe: shared d is only read after init; the @@ -201,7 +230,7 @@ static int ipsec_3des_docsis_hmac_sha512_sve_wide_run(struct test *test, int cpu "out of memory for per-thread buffers"); return EXIT_SKIP; } - for (int s = 0; s < NSTREAM - 1; ++s) { + for (int s = 0; s < d->nstreams - 1; ++s) { local_d.variant_msg[s] = (uint8_t *)malloc(d->data_size); if (!local_d.variant_msg[s]) { free(local_d.plaintext); @@ -234,27 +263,32 @@ static int ipsec_3des_docsis_hmac_sha512_sve_wide_run(struct test *test, int cpu /* SVE 多流 HMAC (主消息 + 变体) */ { const uint8_t *keys[NSTREAM], *msgs[NSTREAM]; + const int n = (int)svcntd(); /* 与 kernel 内部推导一致 */ uint8_t variant_keys[NSTREAM][64]; - for (int s = 0; s < NSTREAM; ++s) { + for (int s = 0; s < n; ++s) { memcpy(variant_keys[s], d->hmac_key, 64); if (s > 0) variant_keys[s][0] ^= (uint8_t)s; keys[s] = variant_keys[s]; } msgs[0] = ciphertext; - for (int s = 1; s < NSTREAM; ++s) { + for (int s = 1; s < n; ++s) { for (size_t i = 0; i < d->data_size; ++i) variant_msg[(s-1)*d->data_size + i] = ciphertext[i] ^ (uint8_t)(s * 0x5A + (i & 0x0F)); msgs[s] = variant_msg + (s-1)*d->data_size; } - hmac_sha512_x4(mac, keys, 64, msgs, d->data_size); + if (hmac_sha512_x4(mac, keys, 64, msgs, d->data_size) != 0) { + log_error("%s: HMAC capacity overflow (data_size too large)", test->id); + report_fail_msg("%s: hmac assembly over capacity", test->id); + } } - for (int s = 0; s < NSTREAM; ++s) + for (int s = 0; s < d->nstreams; ++s) memcmp_or_fail(mac[s], d->golden_mac[s], 64, "HMAC-SHA512 digest mismatch (frozen replay)"); } else { /* re-roll: fresh plaintext + same-iteration golden (EVP encrypt + * SVE HMAC recomputed for this thread's data) */ memset_random(local_d.plaintext, d->data_size); - ipsec_3des_docsis_hmac_sha512_sve_wide_compute_golden(&local_d); + if (ipsec_3des_docsis_hmac_sha512_sve_wide_compute_golden(test, &local_d) != 0) + report_fail_msg("%s: hmac assembly over capacity", test->id); des3_encrypt(local_d.aes_key, local_d.aes_iv, local_d.plaintext, ciphertext, d->data_size); memcmp_or_fail(ciphertext, local_d.golden_ciphertext, d->data_size, "ciphertext mismatch (SVE)"); des3_decrypt(local_d.aes_key, local_d.aes_iv, ciphertext, decrypted, d->data_size); @@ -263,21 +297,25 @@ static int ipsec_3des_docsis_hmac_sha512_sve_wide_run(struct test *test, int cpu /* SVE 多流 HMAC (主消息 + 变体) */ { const uint8_t *keys[NSTREAM], *msgs[NSTREAM]; + const int n = (int)svcntd(); /* 与 kernel 内部推导一致 */ uint8_t variant_keys[NSTREAM][64]; - for (int s = 0; s < NSTREAM; ++s) { + for (int s = 0; s < n; ++s) { memcpy(variant_keys[s], local_d.hmac_key, 64); if (s > 0) variant_keys[s][0] ^= (uint8_t)s; keys[s] = variant_keys[s]; } msgs[0] = ciphertext; - for (int s = 1; s < NSTREAM; ++s) { + for (int s = 1; s < n; ++s) { for (size_t i = 0; i < d->data_size; ++i) variant_msg[(s-1)*d->data_size + i] = ciphertext[i] ^ (uint8_t)(s * 0x5A + (i & 0x0F)); msgs[s] = variant_msg + (s-1)*d->data_size; } - hmac_sha512_x4(mac, keys, 64, msgs, d->data_size); + if (hmac_sha512_x4(mac, keys, 64, msgs, d->data_size) != 0) { + log_error("%s: HMAC capacity overflow (data_size too large)", test->id); + report_fail_msg("%s: hmac assembly over capacity", test->id); + } } - for (int s = 0; s < NSTREAM; ++s) + for (int s = 0; s < d->nstreams; ++s) memcmp_or_fail(mac[s], local_d.golden_mac[s], 64, "HMAC-SHA512 digest mismatch (SVE)"); } ++iter; @@ -285,7 +323,7 @@ static int ipsec_3des_docsis_hmac_sha512_sve_wide_run(struct test *test, int cpu free(ciphertext); free(decrypted); free(variant_msg); - for (int s = 0; s < NSTREAM - 1; ++s) + for (int s = 0; s < d->nstreams - 1; ++s) free(local_d.variant_msg[s]); free(local_d.plaintext); free(local_d.golden_ciphertext); @@ -297,7 +335,7 @@ static int ipsec_3des_docsis_hmac_sha512_sve_wide_cleanup(struct test *test) { if (d) { free(d->plaintext); free(d->golden_ciphertext); - for (int s = 0; s < NSTREAM - 1; ++s) + for (int s = 0; s < d->nstreams - 1; ++s) free(d->variant_msg[s]); } free(d); diff --git a/tests/cpu/sve/ipsec/ipsec_aes128_cbc_hmac_sha1_sve.cpp b/tests/cpu/sve/ipsec/ipsec_aes128_cbc_hmac_sha1_sve.cpp index 44f930a83..8d7fcb199 100644 --- a/tests/cpu/sve/ipsec/ipsec_aes128_cbc_hmac_sha1_sve.cpp +++ b/tests/cpu/sve/ipsec/ipsec_aes128_cbc_hmac_sha1_sve.cpp @@ -7,12 +7,17 @@ * @parblock * SVE port of ipsec_aes128_cbc_hmac_sha1_avx: AES/3DES encryption stays on OpenSSL EVP while * the HMAC-SHA1-96 MAC runs on the multi-stream SVE HMAC implementation - * (verified bit-for-bit against OpenSSL; see scripts/sve-sha-verify). - * Eight independent HMAC lanes run in parallel on SVE vector lanes: lane 0 + * (HMAC construction validated same-source in gem5-fi across 128..1024-bit against an + * independent scalar oracle and RFC 2202/4231 vectors; the underlying SHA + * kernels were verified bit-for-bit against OpenSSL in the original + * fixed-stream form — see scripts/sve-sha-verify and + * docs/research/progress-log.md). + * N = svcntw() (32-bit family) independent HMAC lanes run in parallel on + * SVE vector lanes (all-VL 128..1024-bit, one message per 32-bit lane): lane 0 * carries the original ciphertext message (MAC semantics identical to the - * original test), lanes 1-7 carry deterministic key-stream variant + * original test), lanes 1..N-1 carry deterministic key-stream variant * messages that fill the remaining lanes — same computation family, - * golden values precomputed for all 8. Ciphertext, MACs and decrypted + * golden values precomputed for all N. Ciphertext, MACs and decrypted * data are compared against golden values on every iteration to detect * silent data corruption. Iterations alternate between a re-rolled input * with a same-core recomputed golden and a frozen replay of the init-time @@ -28,19 +33,15 @@ #include "sandstone_ssl.h" #include "sve_hmac.h" #include "sdc_stress_config.h" -#include +#include #include #include -#ifndef HWCAP_SVE -#define HWCAP_SVE (1 << 22) -#endif - #define HMAC_KEY_SIZE_V (20) #define HMAC_96_DIGEST_SIZE (12) #define AES_KEY_SIZE_V (16) #define AES_IV_SIZE_V (16) -#define NSTREAM (8) +#define NSTREAM (SVE_SHA_STREAMS_MAX_32) /* All-VL: 数组按 1024-bit 上限定维; 运行时流数 = svcntw() */ struct aes128_cbc_hmac_sha1_sve_data { uint8_t aes_key[AES_KEY_SIZE_V]; @@ -49,20 +50,21 @@ struct aes128_cbc_hmac_sha1_sve_data { uint8_t *plaintext; uint8_t *golden_ciphertext; uint8_t golden_mac[NSTREAM][20]; /* 内部全摘要 20B, 比较取前 12 */ - /* 7 条变体消息: 确定性派生(异或字节掩码), 填 SVE lane */ + /* N-1 条变体消息: 确定性派生(异或字节掩码), 填 SVE lane */ uint8_t *variant_msg[NSTREAM-1]; size_t data_size; + int nstreams; /* All-VL: init 期流数 = svl/4 (svcntw()), run 漂移守卫用 */ }; static size_t ipsec_data_size(struct test *test) { - int64_t v = sdc_knob_int_clamped(test, "data_size", 0, 1024, 16 << 20); + int64_t v = sdc_knob_int_clamped(test, "data_size", 0, 1024, 1900); if (v == 0) - sdc_env_knob_int_clamped("ipsec_aes128_cbc_hmac_sha1_sve", "data_size", &v, 1024, 16 << 20); + sdc_env_knob_int_clamped("ipsec_aes128_cbc_hmac_sha1_sve", "data_size", &v, 1024, 1900); if (v > 0) return (size_t)v; return (size_t)sdc_auto_linear_count(1, 4, 1.5, 600, thread_count(), - 1024, 16 << 20, test->id, "data_size"); + 1024, 1900, test->id, "data_size"); } static void aes_encrypt(const uint8_t *key, const uint8_t *iv, const uint8_t *in, uint8_t *out, size_t len) { @@ -86,54 +88,56 @@ static void aes_decrypt(const uint8_t *key, const uint8_t *iv, const uint8_t *in s_EVP_CIPHER_CTX_free(ctx); } -static void aes128_cbc_hmac_sha1_sve_compute_golden(struct aes128_cbc_hmac_sha1_sve_data *d) { +static int aes128_cbc_hmac_sha1_sve_compute_golden(struct test *test, struct aes128_cbc_hmac_sha1_sve_data *d) { aes_encrypt(d->aes_key, d->aes_iv, d->plaintext, d->golden_ciphertext, d->data_size); - /* SVE 多流 HMAC: lane0=密文(主语义), lane1..7=变体 */ + /* SVE 多流 HMAC: lane0=密文(主语义), lane1..N-1=变体 */ + const int n = (int)svcntw(); /* 与 kernel 内部推导一致 */ const uint8_t *keys[NSTREAM], *msgs[NSTREAM]; uint8_t variant_keys[NSTREAM][20]; - for (int s = 0; s < NSTREAM; ++s) { + for (int s = 0; s < n; ++s) { memcpy(variant_keys[s], d->hmac_key, 20); if (s > 0) variant_keys[s][0] ^= (uint8_t)s; /* 变体密钥: 确定性微扰 */ keys[s] = variant_keys[s]; } msgs[0] = d->golden_ciphertext; - for (int s = 1; s < NSTREAM; ++s) { + for (int s = 1; s < n; ++s) { /* 变体消息: 密文异或流掩码(确定性) */ for (size_t i = 0; i < d->data_size; ++i) d->variant_msg[s-1][i] = d->golden_ciphertext[i] ^ (uint8_t)(s * 0x5A + (i & 0x0F)); msgs[s] = d->variant_msg[s-1]; } - hmac_sha1_x8(d->golden_mac, keys, 20, msgs, d->data_size); + if (hmac_sha1_x8(d->golden_mac, keys, 20, msgs, d->data_size) != 0) { + log_error("%s: HMAC capacity overflow (data_size too large)", test->id); + return -1; + } + return 0; } static int aes128_cbc_hmac_sha1_sve_init(struct test *test) { if (s_EVP_CIPHER_CTX_new && s_EVP_EncryptInit_ex && s_EVP_DecryptInit_ex && s_EVP_CIPHER_CTX_new && s_EVP_aes_128_cbc()) { - unsigned long hwcap = getauxval(AT_HWCAP); - if ((hwcap & HWCAP_SVE) == 0) { - log_skip(CpuNotSupportedSkipCategory, - "to be implemented (placeholder): ARM SVE required for ipsec_aes128_cbc_hmac_sha1_sve"); - return EXIT_SKIP; - } - /* The multi-stream SVE HMAC kernels give one stream per SVE - * lane: 8x32-bit (sha1/224/256) needs svcntw()>=8 and - * 4x64-bit (384/512) needs svcntd()>=4 -- both mean VL>=256 - * bit. On shorter-VL SVE hardware (e.g. Neoverse N2, VL=128) - * lanes beyond VL are never written, so skip cleanly instead - * of comparing unwritten memory (target silicon: VL=256, see - * tests/cpu/sve/meson.build). */ - if (svcntw() < 8 || svcntd() < 4) { - log_skip(CpuNotSupportedSkipCategory, - "ARM SVE vector length %zu bytes < 32 required by the multi-stream HMAC kernels (svcntw=%zu svcntd=%zu)", - svcntb(), (size_t)svcntw(), (size_t)svcntd()); + /* All-VL Phase 2 P4: capability + 128..1024-bit range check in + * one framework-layer call (replaces the old lower-bound-only + * svcntw()>=8/svcntd()>=4 gate -- the 8.2 defect: at VL>=512 the + * fixed 8/4-wide W rows overflowed; the STREAMS_MAX-sized rows + + * this range gate close both ends). arm64_require_sve_vl returns + * the VL in BYTES. */ + const long svl = arm64_require_sve_vl(test, 128, 1024); + if (svl < 0) return EXIT_SKIP; + const int nstreams = (int)svl / 4; /* one stream per 32-bit lane (svcntw) */ + if (nstreams < 4 || nstreams > SVE_SHA_STREAMS_MAX_32) { /* [4, SVE_SHA_STREAMS_MAX_32], release-effective, NDEBUG-proof */ + log_error("%s: streams %d outside range — unit drift", test->id, nstreams); + return EXIT_FAILURE; } struct aes128_cbc_hmac_sha1_sve_data *d = (struct aes128_cbc_hmac_sha1_sve_data *)malloc(sizeof(*d)); if (!d) return EXIT_SKIP; + d->nstreams = nstreams; /* 工作集字节数: -O ipsec_aes128_cbc_hmac_sha1_sve.data_size=N / * SDC_IPSEC_AES128_CBC_HMAC_SHA1_SVE_DATA_SIZE 环境变量 / 自适应: - * 每线程明文+密文+golden 密文+解密缓冲+变体消息 ~= (3+NSTREAM-1)x, - * 占物理内存 60%, clamp [1024, 16M]。线性缓冲用 sdc_auto_linear_count(sqrt 缩放会欠缩放压力)。历史硬编码 1024。CBC 无填充 + * 每线程明文+密文+golden 密文+解密缓冲+变体消息 ~= (3+nstreams-1)x, + * 占物理内存 60%, clamp [1024, 1900]——上限闭环 8.4: HMAC 组装 + * 缓冲 full[N][2048], 2048-64-9=1975B 安全边界。线性缓冲用 sdc_auto_linear_count(sqrt 缩放会欠缩放压力)。历史硬编码 1024。CBC 无填充 * 场景尺寸必须对齐到 AES 分组 16B, 对齐后 < 1024 取 1024。 */ d->data_size = ipsec_data_size(test); d->data_size &= ~(size_t)15; @@ -147,7 +151,7 @@ static int aes128_cbc_hmac_sha1_sve_init(struct test *test) { free(d); return EXIT_SKIP; } - for (int s = 0; s < NSTREAM - 1; ++s) { + for (int s = 0; s < d->nstreams - 1; ++s) { d->variant_msg[s] = (uint8_t *)malloc(d->data_size); if (!d->variant_msg[s]) { free(d->plaintext); @@ -162,7 +166,20 @@ static int aes128_cbc_hmac_sha1_sve_init(struct test *test) { memset_random(d->aes_iv, AES_IV_SIZE_V); memset_random(d->hmac_key, 20); memset_random(d->plaintext, d->data_size); - aes128_cbc_hmac_sha1_sve_compute_golden(d); + if (aes128_cbc_hmac_sha1_sve_compute_golden(test, d) != 0) { + /* P2a (review Minor): init-side failure propagates via the + * return value so the framework reports a clean Failed + * (sandstone_run.cpp: ret > 0 -> TestResult::Failed). + * report_fail_msg is a noreturn thread-cancel that only + * belongs in the test_run worker context — in test_init it + * would surface as Skipped("Unexpected OS error"). */ + free(d->plaintext); + free(d->golden_ciphertext); + for (int s = 0; s < d->nstreams - 1; ++s) + free(d->variant_msg[s]); + free(d); + return EXIT_FAILURE; + } test->data = d; return EXIT_SUCCESS; } else { @@ -174,10 +191,22 @@ static int aes128_cbc_hmac_sha1_sve_init(struct test *test) { static int aes128_cbc_hmac_sha1_sve_run(struct test *test, int cpu) { (void)cpu; struct aes128_cbc_hmac_sha1_sve_data *d = (struct aes128_cbc_hmac_sha1_sve_data *)test->data; + /* All-VL policy #3: never compare goldens across an init/run VL + * mismatch. Drift means this deployment's per-thread VL differs from + * the init thread's (e.g. a VL-pinning launcher changing the + * thread-inheritance chain) — an environment mismatch, which skips + * with CpuNotSupported (same category as the init-side range gate in + * arm64_require_sve_vl) instead of polluting the SDC failure signal. */ + if ((int)svcntw() != d->nstreams) { + log_skip(CpuNotSupportedSkipCategory, + "%s: SVE VL drifted between init (%d streams) and run (%d streams) — environment mismatch, not SDC", + test->id, d->nstreams, (int)svcntw()); + return EXIT_SKIP; + } uint8_t *ciphertext = (uint8_t *)malloc(d->data_size); uint8_t *decrypted = (uint8_t *)malloc(d->data_size); uint8_t mac[NSTREAM][20]; - uint8_t *variant_msg = (uint8_t *)malloc((NSTREAM-1) * d->data_size); + uint8_t *variant_msg = (uint8_t *)malloc((d->nstreams-1) * d->data_size); /* R2b (B2-003): per-thread plaintext + golden, re-rolled every * iteration (thread-safe: shared d is only read after init; the @@ -201,7 +230,7 @@ static int aes128_cbc_hmac_sha1_sve_run(struct test *test, int cpu) { "out of memory for per-thread buffers"); return EXIT_SKIP; } - for (int s = 0; s < NSTREAM - 1; ++s) { + for (int s = 0; s < d->nstreams - 1; ++s) { local_d.variant_msg[s] = (uint8_t *)malloc(d->data_size); if (!local_d.variant_msg[s]) { free(local_d.plaintext); @@ -234,27 +263,32 @@ static int aes128_cbc_hmac_sha1_sve_run(struct test *test, int cpu) { /* SVE 多流 HMAC (主消息 + 变体) */ { const uint8_t *keys[NSTREAM], *msgs[NSTREAM]; + const int n = (int)svcntw(); /* 与 kernel 内部推导一致 */ uint8_t variant_keys[NSTREAM][20]; - for (int s = 0; s < NSTREAM; ++s) { + for (int s = 0; s < n; ++s) { memcpy(variant_keys[s], d->hmac_key, 20); if (s > 0) variant_keys[s][0] ^= (uint8_t)s; keys[s] = variant_keys[s]; } msgs[0] = ciphertext; - for (int s = 1; s < NSTREAM; ++s) { + for (int s = 1; s < n; ++s) { for (size_t i = 0; i < d->data_size; ++i) variant_msg[(s-1)*d->data_size + i] = ciphertext[i] ^ (uint8_t)(s * 0x5A + (i & 0x0F)); msgs[s] = variant_msg + (s-1)*d->data_size; } - hmac_sha1_x8(mac, keys, 20, msgs, d->data_size); + if (hmac_sha1_x8(mac, keys, 20, msgs, d->data_size) != 0) { + log_error("%s: HMAC capacity overflow (data_size too large)", test->id); + report_fail_msg("%s: hmac assembly over capacity", test->id); + } } - for (int s = 0; s < NSTREAM; ++s) + for (int s = 0; s < d->nstreams; ++s) memcmp_or_fail(mac[s], d->golden_mac[s], 12, "HMAC-SHA1-96 digest mismatch (frozen replay)"); } else { /* re-roll: fresh plaintext + same-iteration golden (EVP encrypt + * SVE HMAC recomputed for this thread's data) */ memset_random(local_d.plaintext, d->data_size); - aes128_cbc_hmac_sha1_sve_compute_golden(&local_d); + if (aes128_cbc_hmac_sha1_sve_compute_golden(test, &local_d) != 0) + report_fail_msg("%s: hmac assembly over capacity", test->id); aes_encrypt(local_d.aes_key, local_d.aes_iv, local_d.plaintext, ciphertext, d->data_size); memcmp_or_fail(ciphertext, local_d.golden_ciphertext, d->data_size, "ciphertext mismatch (SVE)"); aes_decrypt(local_d.aes_key, local_d.aes_iv, ciphertext, decrypted, d->data_size); @@ -263,21 +297,25 @@ static int aes128_cbc_hmac_sha1_sve_run(struct test *test, int cpu) { /* SVE 多流 HMAC (主消息 + 变体) */ { const uint8_t *keys[NSTREAM], *msgs[NSTREAM]; + const int n = (int)svcntw(); /* 与 kernel 内部推导一致 */ uint8_t variant_keys[NSTREAM][20]; - for (int s = 0; s < NSTREAM; ++s) { + for (int s = 0; s < n; ++s) { memcpy(variant_keys[s], local_d.hmac_key, 20); if (s > 0) variant_keys[s][0] ^= (uint8_t)s; keys[s] = variant_keys[s]; } msgs[0] = ciphertext; - for (int s = 1; s < NSTREAM; ++s) { + for (int s = 1; s < n; ++s) { for (size_t i = 0; i < d->data_size; ++i) variant_msg[(s-1)*d->data_size + i] = ciphertext[i] ^ (uint8_t)(s * 0x5A + (i & 0x0F)); msgs[s] = variant_msg + (s-1)*d->data_size; } - hmac_sha1_x8(mac, keys, 20, msgs, d->data_size); + if (hmac_sha1_x8(mac, keys, 20, msgs, d->data_size) != 0) { + log_error("%s: HMAC capacity overflow (data_size too large)", test->id); + report_fail_msg("%s: hmac assembly over capacity", test->id); + } } - for (int s = 0; s < NSTREAM; ++s) + for (int s = 0; s < d->nstreams; ++s) memcmp_or_fail(mac[s], local_d.golden_mac[s], 12, "HMAC-SHA1-96 digest mismatch (SVE)"); } ++iter; @@ -285,7 +323,7 @@ static int aes128_cbc_hmac_sha1_sve_run(struct test *test, int cpu) { free(ciphertext); free(decrypted); free(variant_msg); - for (int s = 0; s < NSTREAM - 1; ++s) + for (int s = 0; s < d->nstreams - 1; ++s) free(local_d.variant_msg[s]); free(local_d.plaintext); free(local_d.golden_ciphertext); @@ -297,7 +335,7 @@ static int aes128_cbc_hmac_sha1_sve_cleanup(struct test *test) { if (d) { free(d->plaintext); free(d->golden_ciphertext); - for (int s = 0; s < NSTREAM - 1; ++s) + for (int s = 0; s < d->nstreams - 1; ++s) free(d->variant_msg[s]); } free(d); diff --git a/tests/cpu/sve/ipsec/ipsec_aes128_cbc_sha2_224_sve.cpp b/tests/cpu/sve/ipsec/ipsec_aes128_cbc_sha2_224_sve.cpp index f936e6022..2585214bd 100644 --- a/tests/cpu/sve/ipsec/ipsec_aes128_cbc_sha2_224_sve.cpp +++ b/tests/cpu/sve/ipsec/ipsec_aes128_cbc_sha2_224_sve.cpp @@ -7,12 +7,17 @@ * @parblock * SVE port of ipsec_aes128_cbc_sha2_224_avx: AES/3DES encryption stays on OpenSSL EVP while * the HMAC-SHA224 MAC runs on the multi-stream SVE HMAC implementation - * (verified bit-for-bit against OpenSSL; see scripts/sve-sha-verify). - * Eight independent HMAC lanes run in parallel on SVE vector lanes: lane 0 + * (HMAC construction validated same-source in gem5-fi across 128..1024-bit against an + * independent scalar oracle and RFC 2202/4231 vectors; the underlying SHA + * kernels were verified bit-for-bit against OpenSSL in the original + * fixed-stream form — see scripts/sve-sha-verify and + * docs/research/progress-log.md). + * N = svcntw() (32-bit family) independent HMAC lanes run in parallel on + * SVE vector lanes (all-VL 128..1024-bit, one message per 32-bit lane): lane 0 * carries the original ciphertext message (MAC semantics identical to the - * original test), lanes 1-7 carry deterministic key-stream variant + * original test), lanes 1..N-1 carry deterministic key-stream variant * messages that fill the remaining lanes — same computation family, - * golden values precomputed for all 8. Ciphertext, MACs and decrypted + * golden values precomputed for all N. Ciphertext, MACs and decrypted * data are compared against golden values on every iteration to detect * silent data corruption. Iterations alternate between a re-rolled input * with a same-core recomputed golden and a frozen replay of the init-time @@ -28,19 +33,15 @@ #include "sandstone_ssl.h" #include "sve_hmac.h" #include "sdc_stress_config.h" -#include +#include #include #include -#ifndef HWCAP_SVE -#define HWCAP_SVE (1 << 22) -#endif - #define HMAC_KEY_SIZE_V (28) #define SHA224_DIGEST_SIZE (28) #define AES_KEY_SIZE_V (16) #define AES_IV_SIZE_V (16) -#define NSTREAM (8) +#define NSTREAM (SVE_SHA_STREAMS_MAX_32) /* All-VL: 数组按 1024-bit 上限定维; 运行时流数 = svcntw() */ struct aes128_cbc_sha2_224_sve_data { uint8_t aes_key[AES_KEY_SIZE_V]; @@ -49,20 +50,21 @@ struct aes128_cbc_sha2_224_sve_data { uint8_t *plaintext; uint8_t *golden_ciphertext; uint8_t golden_mac[NSTREAM][28]; - /* 7 条变体消息: 确定性派生(异或字节掩码), 填 SVE lane */ + /* N-1 条变体消息: 确定性派生(异或字节掩码), 填 SVE lane */ uint8_t *variant_msg[NSTREAM-1]; size_t data_size; + int nstreams; /* All-VL: init 期流数 = svl/4 (svcntw()), run 漂移守卫用 */ }; static size_t ipsec_data_size(struct test *test) { - int64_t v = sdc_knob_int_clamped(test, "data_size", 0, 1024, 16 << 20); + int64_t v = sdc_knob_int_clamped(test, "data_size", 0, 1024, 1900); if (v == 0) - sdc_env_knob_int_clamped("ipsec_aes128_cbc_sha2_224_sve", "data_size", &v, 1024, 16 << 20); + sdc_env_knob_int_clamped("ipsec_aes128_cbc_sha2_224_sve", "data_size", &v, 1024, 1900); if (v > 0) return (size_t)v; return (size_t)sdc_auto_linear_count(1, 4, 1.5, 600, thread_count(), - 1024, 16 << 20, test->id, "data_size"); + 1024, 1900, test->id, "data_size"); } static void aes_encrypt(const uint8_t *key, const uint8_t *iv, const uint8_t *in, uint8_t *out, size_t len) { @@ -86,54 +88,56 @@ static void aes_decrypt(const uint8_t *key, const uint8_t *iv, const uint8_t *in s_EVP_CIPHER_CTX_free(ctx); } -static void aes128_cbc_sha2_224_sve_compute_golden(struct aes128_cbc_sha2_224_sve_data *d) { +static int aes128_cbc_sha2_224_sve_compute_golden(struct test *test, struct aes128_cbc_sha2_224_sve_data *d) { aes_encrypt(d->aes_key, d->aes_iv, d->plaintext, d->golden_ciphertext, d->data_size); - /* SVE 多流 HMAC: lane0=密文(主语义), lane1..7=变体 */ + /* SVE 多流 HMAC: lane0=密文(主语义), lane1..N-1=变体 */ + const int n = (int)svcntw(); /* 与 kernel 内部推导一致 */ const uint8_t *keys[NSTREAM], *msgs[NSTREAM]; uint8_t variant_keys[NSTREAM][28]; - for (int s = 0; s < NSTREAM; ++s) { + for (int s = 0; s < n; ++s) { memcpy(variant_keys[s], d->hmac_key, 28); if (s > 0) variant_keys[s][0] ^= (uint8_t)s; /* 变体密钥: 确定性微扰 */ keys[s] = variant_keys[s]; } msgs[0] = d->golden_ciphertext; - for (int s = 1; s < NSTREAM; ++s) { + for (int s = 1; s < n; ++s) { /* 变体消息: 密文异或流掩码(确定性) */ for (size_t i = 0; i < d->data_size; ++i) d->variant_msg[s-1][i] = d->golden_ciphertext[i] ^ (uint8_t)(s * 0x5A + (i & 0x0F)); msgs[s] = d->variant_msg[s-1]; } - hmac_sha224_x8(d->golden_mac, keys, 28, msgs, d->data_size); + if (hmac_sha224_x8(d->golden_mac, keys, 28, msgs, d->data_size) != 0) { + log_error("%s: HMAC capacity overflow (data_size too large)", test->id); + return -1; + } + return 0; } static int aes128_cbc_sha2_224_sve_init(struct test *test) { if (s_EVP_CIPHER_CTX_new && s_EVP_EncryptInit_ex && s_EVP_DecryptInit_ex && s_EVP_CIPHER_CTX_new && s_EVP_aes_128_cbc()) { - unsigned long hwcap = getauxval(AT_HWCAP); - if ((hwcap & HWCAP_SVE) == 0) { - log_skip(CpuNotSupportedSkipCategory, - "to be implemented (placeholder): ARM SVE required for ipsec_aes128_cbc_sha2_224_sve"); - return EXIT_SKIP; - } - /* The multi-stream SVE HMAC kernels give one stream per SVE - * lane: 8x32-bit (sha1/224/256) needs svcntw()>=8 and - * 4x64-bit (384/512) needs svcntd()>=4 -- both mean VL>=256 - * bit. On shorter-VL SVE hardware (e.g. Neoverse N2, VL=128) - * lanes beyond VL are never written, so skip cleanly instead - * of comparing unwritten memory (target silicon: VL=256, see - * tests/cpu/sve/meson.build). */ - if (svcntw() < 8 || svcntd() < 4) { - log_skip(CpuNotSupportedSkipCategory, - "ARM SVE vector length %zu bytes < 32 required by the multi-stream HMAC kernels (svcntw=%zu svcntd=%zu)", - svcntb(), (size_t)svcntw(), (size_t)svcntd()); + /* All-VL Phase 2 P4: capability + 128..1024-bit range check in + * one framework-layer call (replaces the old lower-bound-only + * svcntw()>=8/svcntd()>=4 gate -- the 8.2 defect: at VL>=512 the + * fixed 8/4-wide W rows overflowed; the STREAMS_MAX-sized rows + + * this range gate close both ends). arm64_require_sve_vl returns + * the VL in BYTES. */ + const long svl = arm64_require_sve_vl(test, 128, 1024); + if (svl < 0) return EXIT_SKIP; + const int nstreams = (int)svl / 4; /* one stream per 32-bit lane (svcntw) */ + if (nstreams < 4 || nstreams > SVE_SHA_STREAMS_MAX_32) { /* [4, SVE_SHA_STREAMS_MAX_32], release-effective, NDEBUG-proof */ + log_error("%s: streams %d outside range — unit drift", test->id, nstreams); + return EXIT_FAILURE; } struct aes128_cbc_sha2_224_sve_data *d = (struct aes128_cbc_sha2_224_sve_data *)malloc(sizeof(*d)); if (!d) return EXIT_SKIP; + d->nstreams = nstreams; /* 工作集字节数: -O ipsec_aes128_cbc_sha2_224_sve.data_size=N / * SDC_IPSEC_AES128_CBC_SHA2_224_SVE_DATA_SIZE 环境变量 / 自适应: - * 每线程明文+密文+golden 密文+解密缓冲+变体消息 ~= (3+NSTREAM-1)x, - * 占物理内存 60%, clamp [1024, 16M]。线性缓冲用 sdc_auto_linear_count(sqrt 缩放会欠缩放压力)。历史硬编码 1024。CBC 无填充 + * 每线程明文+密文+golden 密文+解密缓冲+变体消息 ~= (3+nstreams-1)x, + * 占物理内存 60%, clamp [1024, 1900]——上限闭环 8.4: HMAC 组装 + * 缓冲 full[N][2048], 2048-64-9=1975B 安全边界。线性缓冲用 sdc_auto_linear_count(sqrt 缩放会欠缩放压力)。历史硬编码 1024。CBC 无填充 * 场景尺寸必须对齐到 AES 分组 16B, 对齐后 < 1024 取 1024。 */ d->data_size = ipsec_data_size(test); d->data_size &= ~(size_t)15; @@ -147,7 +151,7 @@ static int aes128_cbc_sha2_224_sve_init(struct test *test) { free(d); return EXIT_SKIP; } - for (int s = 0; s < NSTREAM - 1; ++s) { + for (int s = 0; s < d->nstreams - 1; ++s) { d->variant_msg[s] = (uint8_t *)malloc(d->data_size); if (!d->variant_msg[s]) { free(d->plaintext); @@ -162,7 +166,20 @@ static int aes128_cbc_sha2_224_sve_init(struct test *test) { memset_random(d->aes_iv, AES_IV_SIZE_V); memset_random(d->hmac_key, 28); memset_random(d->plaintext, d->data_size); - aes128_cbc_sha2_224_sve_compute_golden(d); + if (aes128_cbc_sha2_224_sve_compute_golden(test, d) != 0) { + /* P2a (review Minor): init-side failure propagates via the + * return value so the framework reports a clean Failed + * (sandstone_run.cpp: ret > 0 -> TestResult::Failed). + * report_fail_msg is a noreturn thread-cancel that only + * belongs in the test_run worker context — in test_init it + * would surface as Skipped("Unexpected OS error"). */ + free(d->plaintext); + free(d->golden_ciphertext); + for (int s = 0; s < d->nstreams - 1; ++s) + free(d->variant_msg[s]); + free(d); + return EXIT_FAILURE; + } test->data = d; return EXIT_SUCCESS; } else { @@ -174,10 +191,22 @@ static int aes128_cbc_sha2_224_sve_init(struct test *test) { static int aes128_cbc_sha2_224_sve_run(struct test *test, int cpu) { (void)cpu; struct aes128_cbc_sha2_224_sve_data *d = (struct aes128_cbc_sha2_224_sve_data *)test->data; + /* All-VL policy #3: never compare goldens across an init/run VL + * mismatch. Drift means this deployment's per-thread VL differs from + * the init thread's (e.g. a VL-pinning launcher changing the + * thread-inheritance chain) — an environment mismatch, which skips + * with CpuNotSupported (same category as the init-side range gate in + * arm64_require_sve_vl) instead of polluting the SDC failure signal. */ + if ((int)svcntw() != d->nstreams) { + log_skip(CpuNotSupportedSkipCategory, + "%s: SVE VL drifted between init (%d streams) and run (%d streams) — environment mismatch, not SDC", + test->id, d->nstreams, (int)svcntw()); + return EXIT_SKIP; + } uint8_t *ciphertext = (uint8_t *)malloc(d->data_size); uint8_t *decrypted = (uint8_t *)malloc(d->data_size); uint8_t mac[NSTREAM][28]; - uint8_t *variant_msg = (uint8_t *)malloc((NSTREAM-1) * d->data_size); + uint8_t *variant_msg = (uint8_t *)malloc((d->nstreams-1) * d->data_size); /* R2b (B2-003): per-thread plaintext + golden, re-rolled every * iteration (thread-safe: shared d is only read after init; the @@ -201,7 +230,7 @@ static int aes128_cbc_sha2_224_sve_run(struct test *test, int cpu) { "out of memory for per-thread buffers"); return EXIT_SKIP; } - for (int s = 0; s < NSTREAM - 1; ++s) { + for (int s = 0; s < d->nstreams - 1; ++s) { local_d.variant_msg[s] = (uint8_t *)malloc(d->data_size); if (!local_d.variant_msg[s]) { free(local_d.plaintext); @@ -234,27 +263,32 @@ static int aes128_cbc_sha2_224_sve_run(struct test *test, int cpu) { /* SVE 多流 HMAC (主消息 + 变体) */ { const uint8_t *keys[NSTREAM], *msgs[NSTREAM]; + const int n = (int)svcntw(); /* 与 kernel 内部推导一致 */ uint8_t variant_keys[NSTREAM][28]; - for (int s = 0; s < NSTREAM; ++s) { + for (int s = 0; s < n; ++s) { memcpy(variant_keys[s], d->hmac_key, 28); if (s > 0) variant_keys[s][0] ^= (uint8_t)s; keys[s] = variant_keys[s]; } msgs[0] = ciphertext; - for (int s = 1; s < NSTREAM; ++s) { + for (int s = 1; s < n; ++s) { for (size_t i = 0; i < d->data_size; ++i) variant_msg[(s-1)*d->data_size + i] = ciphertext[i] ^ (uint8_t)(s * 0x5A + (i & 0x0F)); msgs[s] = variant_msg + (s-1)*d->data_size; } - hmac_sha224_x8(mac, keys, 28, msgs, d->data_size); + if (hmac_sha224_x8(mac, keys, 28, msgs, d->data_size) != 0) { + log_error("%s: HMAC capacity overflow (data_size too large)", test->id); + report_fail_msg("%s: hmac assembly over capacity", test->id); + } } - for (int s = 0; s < NSTREAM; ++s) + for (int s = 0; s < d->nstreams; ++s) memcmp_or_fail(mac[s], d->golden_mac[s], 28, "HMAC-SHA224 digest mismatch (frozen replay)"); } else { /* re-roll: fresh plaintext + same-iteration golden (EVP encrypt + * SVE HMAC recomputed for this thread's data) */ memset_random(local_d.plaintext, d->data_size); - aes128_cbc_sha2_224_sve_compute_golden(&local_d); + if (aes128_cbc_sha2_224_sve_compute_golden(test, &local_d) != 0) + report_fail_msg("%s: hmac assembly over capacity", test->id); aes_encrypt(local_d.aes_key, local_d.aes_iv, local_d.plaintext, ciphertext, d->data_size); memcmp_or_fail(ciphertext, local_d.golden_ciphertext, d->data_size, "ciphertext mismatch (SVE)"); aes_decrypt(local_d.aes_key, local_d.aes_iv, ciphertext, decrypted, d->data_size); @@ -263,21 +297,25 @@ static int aes128_cbc_sha2_224_sve_run(struct test *test, int cpu) { /* SVE 多流 HMAC (主消息 + 变体) */ { const uint8_t *keys[NSTREAM], *msgs[NSTREAM]; + const int n = (int)svcntw(); /* 与 kernel 内部推导一致 */ uint8_t variant_keys[NSTREAM][28]; - for (int s = 0; s < NSTREAM; ++s) { + for (int s = 0; s < n; ++s) { memcpy(variant_keys[s], local_d.hmac_key, 28); if (s > 0) variant_keys[s][0] ^= (uint8_t)s; keys[s] = variant_keys[s]; } msgs[0] = ciphertext; - for (int s = 1; s < NSTREAM; ++s) { + for (int s = 1; s < n; ++s) { for (size_t i = 0; i < d->data_size; ++i) variant_msg[(s-1)*d->data_size + i] = ciphertext[i] ^ (uint8_t)(s * 0x5A + (i & 0x0F)); msgs[s] = variant_msg + (s-1)*d->data_size; } - hmac_sha224_x8(mac, keys, 28, msgs, d->data_size); + if (hmac_sha224_x8(mac, keys, 28, msgs, d->data_size) != 0) { + log_error("%s: HMAC capacity overflow (data_size too large)", test->id); + report_fail_msg("%s: hmac assembly over capacity", test->id); + } } - for (int s = 0; s < NSTREAM; ++s) + for (int s = 0; s < d->nstreams; ++s) memcmp_or_fail(mac[s], local_d.golden_mac[s], 28, "HMAC-SHA224 digest mismatch (SVE)"); } ++iter; @@ -285,7 +323,7 @@ static int aes128_cbc_sha2_224_sve_run(struct test *test, int cpu) { free(ciphertext); free(decrypted); free(variant_msg); - for (int s = 0; s < NSTREAM - 1; ++s) + for (int s = 0; s < d->nstreams - 1; ++s) free(local_d.variant_msg[s]); free(local_d.plaintext); free(local_d.golden_ciphertext); @@ -297,7 +335,7 @@ static int aes128_cbc_sha2_224_sve_cleanup(struct test *test) { if (d) { free(d->plaintext); free(d->golden_ciphertext); - for (int s = 0; s < NSTREAM - 1; ++s) + for (int s = 0; s < d->nstreams - 1; ++s) free(d->variant_msg[s]); } free(d); diff --git a/tests/cpu/sve/ipsec/ipsec_aes192_cbc_sha2_384_sve.cpp b/tests/cpu/sve/ipsec/ipsec_aes192_cbc_sha2_384_sve.cpp index 7884e0173..9ee392507 100644 --- a/tests/cpu/sve/ipsec/ipsec_aes192_cbc_sha2_384_sve.cpp +++ b/tests/cpu/sve/ipsec/ipsec_aes192_cbc_sha2_384_sve.cpp @@ -7,12 +7,17 @@ * @parblock * SVE port of ipsec_aes192_cbc_sha2_384_avx: AES/3DES encryption stays on OpenSSL EVP while * the HMAC-SHA384 MAC runs on the multi-stream SVE HMAC implementation - * (verified bit-for-bit against OpenSSL; see scripts/sve-sha-verify). - * Eight independent HMAC lanes run in parallel on SVE vector lanes: lane 0 + * (HMAC construction validated same-source in gem5-fi across 128..1024-bit against an + * independent scalar oracle and RFC 2202/4231 vectors; the underlying SHA + * kernels were verified bit-for-bit against OpenSSL in the original + * fixed-stream form — see scripts/sve-sha-verify and + * docs/research/progress-log.md). + * N = svcntd() (64-bit family) independent HMAC lanes run in parallel on + * SVE vector lanes (all-VL 128..1024-bit, one message per 64-bit lane): lane 0 * carries the original ciphertext message (MAC semantics identical to the - * original test), lanes 1-7 carry deterministic key-stream variant + * original test), lanes 1..N-1 carry deterministic key-stream variant * messages that fill the remaining lanes — same computation family, - * golden values precomputed for all 8. Ciphertext, MACs and decrypted + * golden values precomputed for all N. Ciphertext, MACs and decrypted * data are compared against golden values on every iteration to detect * silent data corruption. Iterations alternate between a re-rolled input * with a same-core recomputed golden and a frozen replay of the init-time @@ -28,19 +33,15 @@ #include "sandstone_ssl.h" #include "sve_hmac.h" #include "sdc_stress_config.h" -#include +#include #include #include -#ifndef HWCAP_SVE -#define HWCAP_SVE (1 << 22) -#endif - #define HMAC_KEY_SIZE_V (48) #define SHA384_DIGEST_SIZE (48) #define AES_KEY_SIZE_V (24) #define AES_IV_SIZE_V (16) -#define NSTREAM (4) +#define NSTREAM (SVE_SHA_STREAMS_MAX_64) /* All-VL: 数组按 1024-bit 上限定维; 运行时流数 = svcntd() */ struct aes192_cbc_sha2_384_sve_data { uint8_t aes_key[AES_KEY_SIZE_V]; @@ -49,20 +50,21 @@ struct aes192_cbc_sha2_384_sve_data { uint8_t *plaintext; uint8_t *golden_ciphertext; uint8_t golden_mac[NSTREAM][48]; - /* 7 条变体消息: 确定性派生(异或字节掩码), 填 SVE lane */ + /* N-1 条变体消息: 确定性派生(异或字节掩码), 填 SVE lane */ uint8_t *variant_msg[NSTREAM-1]; size_t data_size; + int nstreams; /* All-VL: init 期流数 = svl/8 (svcntd()), run 漂移守卫用 */ }; static size_t ipsec_data_size(struct test *test) { - int64_t v = sdc_knob_int_clamped(test, "data_size", 0, 1024, 16 << 20); + int64_t v = sdc_knob_int_clamped(test, "data_size", 0, 1024, 1830); if (v == 0) - sdc_env_knob_int_clamped("ipsec_aes192_cbc_sha2_384_sve", "data_size", &v, 1024, 16 << 20); + sdc_env_knob_int_clamped("ipsec_aes192_cbc_sha2_384_sve", "data_size", &v, 1024, 1830); if (v > 0) return (size_t)v; return (size_t)sdc_auto_linear_count(1, 4, 1.5, 600, thread_count(), - 1024, 16 << 20, test->id, "data_size"); + 1024, 1830, test->id, "data_size"); } static void aes_encrypt(const uint8_t *key, const uint8_t *iv, const uint8_t *in, uint8_t *out, size_t len) { @@ -86,54 +88,56 @@ static void aes_decrypt(const uint8_t *key, const uint8_t *iv, const uint8_t *in s_EVP_CIPHER_CTX_free(ctx); } -static void aes192_cbc_sha2_384_sve_compute_golden(struct aes192_cbc_sha2_384_sve_data *d) { +static int aes192_cbc_sha2_384_sve_compute_golden(struct test *test, struct aes192_cbc_sha2_384_sve_data *d) { aes_encrypt(d->aes_key, d->aes_iv, d->plaintext, d->golden_ciphertext, d->data_size); - /* SVE 多流 HMAC: lane0=密文(主语义), lane1..7=变体 */ + /* SVE 多流 HMAC: lane0=密文(主语义), lane1..N-1=变体 */ + const int n = (int)svcntd(); /* 与 kernel 内部推导一致 */ const uint8_t *keys[NSTREAM], *msgs[NSTREAM]; uint8_t variant_keys[NSTREAM][48]; - for (int s = 0; s < NSTREAM; ++s) { + for (int s = 0; s < n; ++s) { memcpy(variant_keys[s], d->hmac_key, 48); if (s > 0) variant_keys[s][0] ^= (uint8_t)s; /* 变体密钥: 确定性微扰 */ keys[s] = variant_keys[s]; } msgs[0] = d->golden_ciphertext; - for (int s = 1; s < NSTREAM; ++s) { + for (int s = 1; s < n; ++s) { /* 变体消息: 密文异或流掩码(确定性) */ for (size_t i = 0; i < d->data_size; ++i) d->variant_msg[s-1][i] = d->golden_ciphertext[i] ^ (uint8_t)(s * 0x5A + (i & 0x0F)); msgs[s] = d->variant_msg[s-1]; } - hmac_sha384_x4(d->golden_mac, keys, 48, msgs, d->data_size); + if (hmac_sha384_x4(d->golden_mac, keys, 48, msgs, d->data_size) != 0) { + log_error("%s: HMAC capacity overflow (data_size too large)", test->id); + return -1; + } + return 0; } static int aes192_cbc_sha2_384_sve_init(struct test *test) { if (s_EVP_CIPHER_CTX_new && s_EVP_EncryptInit_ex && s_EVP_DecryptInit_ex && s_EVP_CIPHER_CTX_new && s_EVP_aes_192_cbc()) { - unsigned long hwcap = getauxval(AT_HWCAP); - if ((hwcap & HWCAP_SVE) == 0) { - log_skip(CpuNotSupportedSkipCategory, - "to be implemented (placeholder): ARM SVE required for ipsec_aes192_cbc_sha2_384_sve"); - return EXIT_SKIP; - } - /* The multi-stream SVE HMAC kernels give one stream per SVE - * lane: 8x32-bit (sha1/224/256) needs svcntw()>=8 and - * 4x64-bit (384/512) needs svcntd()>=4 -- both mean VL>=256 - * bit. On shorter-VL SVE hardware (e.g. Neoverse N2, VL=128) - * lanes beyond VL are never written, so skip cleanly instead - * of comparing unwritten memory (target silicon: VL=256, see - * tests/cpu/sve/meson.build). */ - if (svcntw() < 8 || svcntd() < 4) { - log_skip(CpuNotSupportedSkipCategory, - "ARM SVE vector length %zu bytes < 32 required by the multi-stream HMAC kernels (svcntw=%zu svcntd=%zu)", - svcntb(), (size_t)svcntw(), (size_t)svcntd()); + /* All-VL Phase 2 P4: capability + 128..1024-bit range check in + * one framework-layer call (replaces the old lower-bound-only + * svcntw()>=8/svcntd()>=4 gate -- the 8.2 defect: at VL>=512 the + * fixed 8/4-wide W rows overflowed; the STREAMS_MAX-sized rows + + * this range gate close both ends). arm64_require_sve_vl returns + * the VL in BYTES. */ + const long svl = arm64_require_sve_vl(test, 128, 1024); + if (svl < 0) return EXIT_SKIP; + const int nstreams = (int)svl / 8; /* one stream per 64-bit lane (svcntd) */ + if (nstreams < 2 || nstreams > SVE_SHA_STREAMS_MAX_64) { /* [2, SVE_SHA_STREAMS_MAX_64], release-effective, NDEBUG-proof */ + log_error("%s: streams %d outside range — unit drift", test->id, nstreams); + return EXIT_FAILURE; } struct aes192_cbc_sha2_384_sve_data *d = (struct aes192_cbc_sha2_384_sve_data *)malloc(sizeof(*d)); if (!d) return EXIT_SKIP; + d->nstreams = nstreams; /* 工作集字节数: -O ipsec_aes192_cbc_sha2_384_sve.data_size=N / * SDC_IPSEC_AES192_CBC_SHA2_384_SVE_DATA_SIZE 环境变量 / 自适应: - * 每线程明文+密文+golden 密文+解密缓冲+变体消息 ~= (3+NSTREAM-1)x, - * 占物理内存 60%, clamp [1024, 16M]。线性缓冲用 sdc_auto_linear_count(sqrt 缩放会欠缩放压力)。历史硬编码 1024。CBC 无填充 + * 每线程明文+密文+golden 密文+解密缓冲+变体消息 ~= (3+nstreams-1)x, + * 占物理内存 60%, clamp [1024, 1830]——上限闭环 8.4: HMAC 组装 + * 缓冲 full[N][2048], 2048-128-17=1903B 安全边界。线性缓冲用 sdc_auto_linear_count(sqrt 缩放会欠缩放压力)。历史硬编码 1024。CBC 无填充 * 场景尺寸必须对齐到 AES 分组 16B, 对齐后 < 1024 取 1024。 */ d->data_size = ipsec_data_size(test); d->data_size &= ~(size_t)15; @@ -147,7 +151,7 @@ static int aes192_cbc_sha2_384_sve_init(struct test *test) { free(d); return EXIT_SKIP; } - for (int s = 0; s < NSTREAM - 1; ++s) { + for (int s = 0; s < d->nstreams - 1; ++s) { d->variant_msg[s] = (uint8_t *)malloc(d->data_size); if (!d->variant_msg[s]) { free(d->plaintext); @@ -162,7 +166,20 @@ static int aes192_cbc_sha2_384_sve_init(struct test *test) { memset_random(d->aes_iv, AES_IV_SIZE_V); memset_random(d->hmac_key, 48); memset_random(d->plaintext, d->data_size); - aes192_cbc_sha2_384_sve_compute_golden(d); + if (aes192_cbc_sha2_384_sve_compute_golden(test, d) != 0) { + /* P2a (review Minor): init-side failure propagates via the + * return value so the framework reports a clean Failed + * (sandstone_run.cpp: ret > 0 -> TestResult::Failed). + * report_fail_msg is a noreturn thread-cancel that only + * belongs in the test_run worker context — in test_init it + * would surface as Skipped("Unexpected OS error"). */ + free(d->plaintext); + free(d->golden_ciphertext); + for (int s = 0; s < d->nstreams - 1; ++s) + free(d->variant_msg[s]); + free(d); + return EXIT_FAILURE; + } test->data = d; return EXIT_SUCCESS; } else { @@ -174,10 +191,22 @@ static int aes192_cbc_sha2_384_sve_init(struct test *test) { static int aes192_cbc_sha2_384_sve_run(struct test *test, int cpu) { (void)cpu; struct aes192_cbc_sha2_384_sve_data *d = (struct aes192_cbc_sha2_384_sve_data *)test->data; + /* All-VL policy #3: never compare goldens across an init/run VL + * mismatch. Drift means this deployment's per-thread VL differs from + * the init thread's (e.g. a VL-pinning launcher changing the + * thread-inheritance chain) — an environment mismatch, which skips + * with CpuNotSupported (same category as the init-side range gate in + * arm64_require_sve_vl) instead of polluting the SDC failure signal. */ + if ((int)svcntd() != d->nstreams) { + log_skip(CpuNotSupportedSkipCategory, + "%s: SVE VL drifted between init (%d streams) and run (%d streams) — environment mismatch, not SDC", + test->id, d->nstreams, (int)svcntd()); + return EXIT_SKIP; + } uint8_t *ciphertext = (uint8_t *)malloc(d->data_size); uint8_t *decrypted = (uint8_t *)malloc(d->data_size); uint8_t mac[NSTREAM][48]; - uint8_t *variant_msg = (uint8_t *)malloc((NSTREAM-1) * d->data_size); + uint8_t *variant_msg = (uint8_t *)malloc((d->nstreams-1) * d->data_size); /* R2b (B2-003): per-thread plaintext + golden, re-rolled every * iteration (thread-safe: shared d is only read after init; the @@ -201,7 +230,7 @@ static int aes192_cbc_sha2_384_sve_run(struct test *test, int cpu) { "out of memory for per-thread buffers"); return EXIT_SKIP; } - for (int s = 0; s < NSTREAM - 1; ++s) { + for (int s = 0; s < d->nstreams - 1; ++s) { local_d.variant_msg[s] = (uint8_t *)malloc(d->data_size); if (!local_d.variant_msg[s]) { free(local_d.plaintext); @@ -234,27 +263,32 @@ static int aes192_cbc_sha2_384_sve_run(struct test *test, int cpu) { /* SVE 多流 HMAC (主消息 + 变体) */ { const uint8_t *keys[NSTREAM], *msgs[NSTREAM]; + const int n = (int)svcntd(); /* 与 kernel 内部推导一致 */ uint8_t variant_keys[NSTREAM][48]; - for (int s = 0; s < NSTREAM; ++s) { + for (int s = 0; s < n; ++s) { memcpy(variant_keys[s], d->hmac_key, 48); if (s > 0) variant_keys[s][0] ^= (uint8_t)s; keys[s] = variant_keys[s]; } msgs[0] = ciphertext; - for (int s = 1; s < NSTREAM; ++s) { + for (int s = 1; s < n; ++s) { for (size_t i = 0; i < d->data_size; ++i) variant_msg[(s-1)*d->data_size + i] = ciphertext[i] ^ (uint8_t)(s * 0x5A + (i & 0x0F)); msgs[s] = variant_msg + (s-1)*d->data_size; } - hmac_sha384_x4(mac, keys, 48, msgs, d->data_size); + if (hmac_sha384_x4(mac, keys, 48, msgs, d->data_size) != 0) { + log_error("%s: HMAC capacity overflow (data_size too large)", test->id); + report_fail_msg("%s: hmac assembly over capacity", test->id); + } } - for (int s = 0; s < NSTREAM; ++s) + for (int s = 0; s < d->nstreams; ++s) memcmp_or_fail(mac[s], d->golden_mac[s], 48, "HMAC-SHA384 digest mismatch (frozen replay)"); } else { /* re-roll: fresh plaintext + same-iteration golden (EVP encrypt + * SVE HMAC recomputed for this thread's data) */ memset_random(local_d.plaintext, d->data_size); - aes192_cbc_sha2_384_sve_compute_golden(&local_d); + if (aes192_cbc_sha2_384_sve_compute_golden(test, &local_d) != 0) + report_fail_msg("%s: hmac assembly over capacity", test->id); aes_encrypt(local_d.aes_key, local_d.aes_iv, local_d.plaintext, ciphertext, d->data_size); memcmp_or_fail(ciphertext, local_d.golden_ciphertext, d->data_size, "ciphertext mismatch (SVE)"); aes_decrypt(local_d.aes_key, local_d.aes_iv, ciphertext, decrypted, d->data_size); @@ -263,21 +297,25 @@ static int aes192_cbc_sha2_384_sve_run(struct test *test, int cpu) { /* SVE 多流 HMAC (主消息 + 变体) */ { const uint8_t *keys[NSTREAM], *msgs[NSTREAM]; + const int n = (int)svcntd(); /* 与 kernel 内部推导一致 */ uint8_t variant_keys[NSTREAM][48]; - for (int s = 0; s < NSTREAM; ++s) { + for (int s = 0; s < n; ++s) { memcpy(variant_keys[s], local_d.hmac_key, 48); if (s > 0) variant_keys[s][0] ^= (uint8_t)s; keys[s] = variant_keys[s]; } msgs[0] = ciphertext; - for (int s = 1; s < NSTREAM; ++s) { + for (int s = 1; s < n; ++s) { for (size_t i = 0; i < d->data_size; ++i) variant_msg[(s-1)*d->data_size + i] = ciphertext[i] ^ (uint8_t)(s * 0x5A + (i & 0x0F)); msgs[s] = variant_msg + (s-1)*d->data_size; } - hmac_sha384_x4(mac, keys, 48, msgs, d->data_size); + if (hmac_sha384_x4(mac, keys, 48, msgs, d->data_size) != 0) { + log_error("%s: HMAC capacity overflow (data_size too large)", test->id); + report_fail_msg("%s: hmac assembly over capacity", test->id); + } } - for (int s = 0; s < NSTREAM; ++s) + for (int s = 0; s < d->nstreams; ++s) memcmp_or_fail(mac[s], local_d.golden_mac[s], 48, "HMAC-SHA384 digest mismatch (SVE)"); } ++iter; @@ -285,7 +323,7 @@ static int aes192_cbc_sha2_384_sve_run(struct test *test, int cpu) { free(ciphertext); free(decrypted); free(variant_msg); - for (int s = 0; s < NSTREAM - 1; ++s) + for (int s = 0; s < d->nstreams - 1; ++s) free(local_d.variant_msg[s]); free(local_d.plaintext); free(local_d.golden_ciphertext); @@ -297,7 +335,7 @@ static int aes192_cbc_sha2_384_sve_cleanup(struct test *test) { if (d) { free(d->plaintext); free(d->golden_ciphertext); - for (int s = 0; s < NSTREAM - 1; ++s) + for (int s = 0; s < d->nstreams - 1; ++s) free(d->variant_msg[s]); } free(d); diff --git a/tests/cpu/sve/ipsec/ipsec_aes192_cbc_sha2_512_sve.cpp b/tests/cpu/sve/ipsec/ipsec_aes192_cbc_sha2_512_sve.cpp index ee3bc7be9..ce257277e 100644 --- a/tests/cpu/sve/ipsec/ipsec_aes192_cbc_sha2_512_sve.cpp +++ b/tests/cpu/sve/ipsec/ipsec_aes192_cbc_sha2_512_sve.cpp @@ -7,12 +7,17 @@ * @parblock * SVE port of ipsec_aes192_cbc_sha2_512_avx: AES/3DES encryption stays on OpenSSL EVP while * the HMAC-SHA512 MAC runs on the multi-stream SVE HMAC implementation - * (verified bit-for-bit against OpenSSL; see scripts/sve-sha-verify). - * Eight independent HMAC lanes run in parallel on SVE vector lanes: lane 0 + * (HMAC construction validated same-source in gem5-fi across 128..1024-bit against an + * independent scalar oracle and RFC 2202/4231 vectors; the underlying SHA + * kernels were verified bit-for-bit against OpenSSL in the original + * fixed-stream form — see scripts/sve-sha-verify and + * docs/research/progress-log.md). + * N = svcntd() (64-bit family) independent HMAC lanes run in parallel on + * SVE vector lanes (all-VL 128..1024-bit, one message per 64-bit lane): lane 0 * carries the original ciphertext message (MAC semantics identical to the - * original test), lanes 1-7 carry deterministic key-stream variant + * original test), lanes 1..N-1 carry deterministic key-stream variant * messages that fill the remaining lanes — same computation family, - * golden values precomputed for all 8. Ciphertext, MACs and decrypted + * golden values precomputed for all N. Ciphertext, MACs and decrypted * data are compared against golden values on every iteration to detect * silent data corruption. Iterations alternate between a re-rolled input * with a same-core recomputed golden and a frozen replay of the init-time @@ -28,19 +33,15 @@ #include "sandstone_ssl.h" #include "sve_hmac.h" #include "sdc_stress_config.h" -#include +#include #include #include -#ifndef HWCAP_SVE -#define HWCAP_SVE (1 << 22) -#endif - #define HMAC_KEY_SIZE_V (64) #define SHA512_DIGEST_SIZE (64) #define AES_KEY_SIZE_V (24) #define AES_IV_SIZE_V (16) -#define NSTREAM (4) +#define NSTREAM (SVE_SHA_STREAMS_MAX_64) /* All-VL: 数组按 1024-bit 上限定维; 运行时流数 = svcntd() */ struct aes192_cbc_sha2_512_sve_data { uint8_t aes_key[AES_KEY_SIZE_V]; @@ -49,20 +50,21 @@ struct aes192_cbc_sha2_512_sve_data { uint8_t *plaintext; uint8_t *golden_ciphertext; uint8_t golden_mac[NSTREAM][64]; - /* 7 条变体消息: 确定性派生(异或字节掩码), 填 SVE lane */ + /* N-1 条变体消息: 确定性派生(异或字节掩码), 填 SVE lane */ uint8_t *variant_msg[NSTREAM-1]; size_t data_size; + int nstreams; /* All-VL: init 期流数 = svl/8 (svcntd()), run 漂移守卫用 */ }; static size_t ipsec_data_size(struct test *test) { - int64_t v = sdc_knob_int_clamped(test, "data_size", 0, 1024, 16 << 20); + int64_t v = sdc_knob_int_clamped(test, "data_size", 0, 1024, 1830); if (v == 0) - sdc_env_knob_int_clamped("ipsec_aes192_cbc_sha2_512_sve", "data_size", &v, 1024, 16 << 20); + sdc_env_knob_int_clamped("ipsec_aes192_cbc_sha2_512_sve", "data_size", &v, 1024, 1830); if (v > 0) return (size_t)v; return (size_t)sdc_auto_linear_count(1, 4, 1.5, 600, thread_count(), - 1024, 16 << 20, test->id, "data_size"); + 1024, 1830, test->id, "data_size"); } static void aes_encrypt(const uint8_t *key, const uint8_t *iv, const uint8_t *in, uint8_t *out, size_t len) { @@ -86,54 +88,56 @@ static void aes_decrypt(const uint8_t *key, const uint8_t *iv, const uint8_t *in s_EVP_CIPHER_CTX_free(ctx); } -static void aes192_cbc_sha2_512_sve_compute_golden(struct aes192_cbc_sha2_512_sve_data *d) { +static int aes192_cbc_sha2_512_sve_compute_golden(struct test *test, struct aes192_cbc_sha2_512_sve_data *d) { aes_encrypt(d->aes_key, d->aes_iv, d->plaintext, d->golden_ciphertext, d->data_size); - /* SVE 多流 HMAC: lane0=密文(主语义), lane1..7=变体 */ + /* SVE 多流 HMAC: lane0=密文(主语义), lane1..N-1=变体 */ + const int n = (int)svcntd(); /* 与 kernel 内部推导一致 */ const uint8_t *keys[NSTREAM], *msgs[NSTREAM]; uint8_t variant_keys[NSTREAM][64]; - for (int s = 0; s < NSTREAM; ++s) { + for (int s = 0; s < n; ++s) { memcpy(variant_keys[s], d->hmac_key, 64); if (s > 0) variant_keys[s][0] ^= (uint8_t)s; /* 变体密钥: 确定性微扰 */ keys[s] = variant_keys[s]; } msgs[0] = d->golden_ciphertext; - for (int s = 1; s < NSTREAM; ++s) { + for (int s = 1; s < n; ++s) { /* 变体消息: 密文异或流掩码(确定性) */ for (size_t i = 0; i < d->data_size; ++i) d->variant_msg[s-1][i] = d->golden_ciphertext[i] ^ (uint8_t)(s * 0x5A + (i & 0x0F)); msgs[s] = d->variant_msg[s-1]; } - hmac_sha512_x4(d->golden_mac, keys, 64, msgs, d->data_size); + if (hmac_sha512_x4(d->golden_mac, keys, 64, msgs, d->data_size) != 0) { + log_error("%s: HMAC capacity overflow (data_size too large)", test->id); + return -1; + } + return 0; } static int aes192_cbc_sha2_512_sve_init(struct test *test) { if (s_EVP_CIPHER_CTX_new && s_EVP_EncryptInit_ex && s_EVP_DecryptInit_ex && s_EVP_CIPHER_CTX_new && s_EVP_aes_192_cbc()) { - unsigned long hwcap = getauxval(AT_HWCAP); - if ((hwcap & HWCAP_SVE) == 0) { - log_skip(CpuNotSupportedSkipCategory, - "to be implemented (placeholder): ARM SVE required for ipsec_aes192_cbc_sha2_512_sve"); - return EXIT_SKIP; - } - /* The multi-stream SVE HMAC kernels give one stream per SVE - * lane: 8x32-bit (sha1/224/256) needs svcntw()>=8 and - * 4x64-bit (384/512) needs svcntd()>=4 -- both mean VL>=256 - * bit. On shorter-VL SVE hardware (e.g. Neoverse N2, VL=128) - * lanes beyond VL are never written, so skip cleanly instead - * of comparing unwritten memory (target silicon: VL=256, see - * tests/cpu/sve/meson.build). */ - if (svcntw() < 8 || svcntd() < 4) { - log_skip(CpuNotSupportedSkipCategory, - "ARM SVE vector length %zu bytes < 32 required by the multi-stream HMAC kernels (svcntw=%zu svcntd=%zu)", - svcntb(), (size_t)svcntw(), (size_t)svcntd()); + /* All-VL Phase 2 P4: capability + 128..1024-bit range check in + * one framework-layer call (replaces the old lower-bound-only + * svcntw()>=8/svcntd()>=4 gate -- the 8.2 defect: at VL>=512 the + * fixed 8/4-wide W rows overflowed; the STREAMS_MAX-sized rows + + * this range gate close both ends). arm64_require_sve_vl returns + * the VL in BYTES. */ + const long svl = arm64_require_sve_vl(test, 128, 1024); + if (svl < 0) return EXIT_SKIP; + const int nstreams = (int)svl / 8; /* one stream per 64-bit lane (svcntd) */ + if (nstreams < 2 || nstreams > SVE_SHA_STREAMS_MAX_64) { /* [2, SVE_SHA_STREAMS_MAX_64], release-effective, NDEBUG-proof */ + log_error("%s: streams %d outside range — unit drift", test->id, nstreams); + return EXIT_FAILURE; } struct aes192_cbc_sha2_512_sve_data *d = (struct aes192_cbc_sha2_512_sve_data *)malloc(sizeof(*d)); if (!d) return EXIT_SKIP; + d->nstreams = nstreams; /* 工作集字节数: -O ipsec_aes192_cbc_sha2_512_sve.data_size=N / * SDC_IPSEC_AES192_CBC_SHA2_512_SVE_DATA_SIZE 环境变量 / 自适应: - * 每线程明文+密文+golden 密文+解密缓冲+变体消息 ~= (3+NSTREAM-1)x, - * 占物理内存 60%, clamp [1024, 16M]。线性缓冲用 sdc_auto_linear_count(sqrt 缩放会欠缩放压力)。历史硬编码 1024。CBC 无填充 + * 每线程明文+密文+golden 密文+解密缓冲+变体消息 ~= (3+nstreams-1)x, + * 占物理内存 60%, clamp [1024, 1830]——上限闭环 8.4: HMAC 组装 + * 缓冲 full[N][2048], 2048-128-17=1903B 安全边界。线性缓冲用 sdc_auto_linear_count(sqrt 缩放会欠缩放压力)。历史硬编码 1024。CBC 无填充 * 场景尺寸必须对齐到 AES 分组 16B, 对齐后 < 1024 取 1024。 */ d->data_size = ipsec_data_size(test); d->data_size &= ~(size_t)15; @@ -147,7 +151,7 @@ static int aes192_cbc_sha2_512_sve_init(struct test *test) { free(d); return EXIT_SKIP; } - for (int s = 0; s < NSTREAM - 1; ++s) { + for (int s = 0; s < d->nstreams - 1; ++s) { d->variant_msg[s] = (uint8_t *)malloc(d->data_size); if (!d->variant_msg[s]) { free(d->plaintext); @@ -162,7 +166,20 @@ static int aes192_cbc_sha2_512_sve_init(struct test *test) { memset_random(d->aes_iv, AES_IV_SIZE_V); memset_random(d->hmac_key, 64); memset_random(d->plaintext, d->data_size); - aes192_cbc_sha2_512_sve_compute_golden(d); + if (aes192_cbc_sha2_512_sve_compute_golden(test, d) != 0) { + /* P2a (review Minor): init-side failure propagates via the + * return value so the framework reports a clean Failed + * (sandstone_run.cpp: ret > 0 -> TestResult::Failed). + * report_fail_msg is a noreturn thread-cancel that only + * belongs in the test_run worker context — in test_init it + * would surface as Skipped("Unexpected OS error"). */ + free(d->plaintext); + free(d->golden_ciphertext); + for (int s = 0; s < d->nstreams - 1; ++s) + free(d->variant_msg[s]); + free(d); + return EXIT_FAILURE; + } test->data = d; return EXIT_SUCCESS; } else { @@ -174,10 +191,22 @@ static int aes192_cbc_sha2_512_sve_init(struct test *test) { static int aes192_cbc_sha2_512_sve_run(struct test *test, int cpu) { (void)cpu; struct aes192_cbc_sha2_512_sve_data *d = (struct aes192_cbc_sha2_512_sve_data *)test->data; + /* All-VL policy #3: never compare goldens across an init/run VL + * mismatch. Drift means this deployment's per-thread VL differs from + * the init thread's (e.g. a VL-pinning launcher changing the + * thread-inheritance chain) — an environment mismatch, which skips + * with CpuNotSupported (same category as the init-side range gate in + * arm64_require_sve_vl) instead of polluting the SDC failure signal. */ + if ((int)svcntd() != d->nstreams) { + log_skip(CpuNotSupportedSkipCategory, + "%s: SVE VL drifted between init (%d streams) and run (%d streams) — environment mismatch, not SDC", + test->id, d->nstreams, (int)svcntd()); + return EXIT_SKIP; + } uint8_t *ciphertext = (uint8_t *)malloc(d->data_size); uint8_t *decrypted = (uint8_t *)malloc(d->data_size); uint8_t mac[NSTREAM][64]; - uint8_t *variant_msg = (uint8_t *)malloc((NSTREAM-1) * d->data_size); + uint8_t *variant_msg = (uint8_t *)malloc((d->nstreams-1) * d->data_size); /* R2b (B2-003): per-thread plaintext + golden, re-rolled every * iteration (thread-safe: shared d is only read after init; the @@ -201,7 +230,7 @@ static int aes192_cbc_sha2_512_sve_run(struct test *test, int cpu) { "out of memory for per-thread buffers"); return EXIT_SKIP; } - for (int s = 0; s < NSTREAM - 1; ++s) { + for (int s = 0; s < d->nstreams - 1; ++s) { local_d.variant_msg[s] = (uint8_t *)malloc(d->data_size); if (!local_d.variant_msg[s]) { free(local_d.plaintext); @@ -234,27 +263,32 @@ static int aes192_cbc_sha2_512_sve_run(struct test *test, int cpu) { /* SVE 多流 HMAC (主消息 + 变体) */ { const uint8_t *keys[NSTREAM], *msgs[NSTREAM]; + const int n = (int)svcntd(); /* 与 kernel 内部推导一致 */ uint8_t variant_keys[NSTREAM][64]; - for (int s = 0; s < NSTREAM; ++s) { + for (int s = 0; s < n; ++s) { memcpy(variant_keys[s], d->hmac_key, 64); if (s > 0) variant_keys[s][0] ^= (uint8_t)s; keys[s] = variant_keys[s]; } msgs[0] = ciphertext; - for (int s = 1; s < NSTREAM; ++s) { + for (int s = 1; s < n; ++s) { for (size_t i = 0; i < d->data_size; ++i) variant_msg[(s-1)*d->data_size + i] = ciphertext[i] ^ (uint8_t)(s * 0x5A + (i & 0x0F)); msgs[s] = variant_msg + (s-1)*d->data_size; } - hmac_sha512_x4(mac, keys, 64, msgs, d->data_size); + if (hmac_sha512_x4(mac, keys, 64, msgs, d->data_size) != 0) { + log_error("%s: HMAC capacity overflow (data_size too large)", test->id); + report_fail_msg("%s: hmac assembly over capacity", test->id); + } } - for (int s = 0; s < NSTREAM; ++s) + for (int s = 0; s < d->nstreams; ++s) memcmp_or_fail(mac[s], d->golden_mac[s], 64, "HMAC-SHA512 digest mismatch (frozen replay)"); } else { /* re-roll: fresh plaintext + same-iteration golden (EVP encrypt + * SVE HMAC recomputed for this thread's data) */ memset_random(local_d.plaintext, d->data_size); - aes192_cbc_sha2_512_sve_compute_golden(&local_d); + if (aes192_cbc_sha2_512_sve_compute_golden(test, &local_d) != 0) + report_fail_msg("%s: hmac assembly over capacity", test->id); aes_encrypt(local_d.aes_key, local_d.aes_iv, local_d.plaintext, ciphertext, d->data_size); memcmp_or_fail(ciphertext, local_d.golden_ciphertext, d->data_size, "ciphertext mismatch (SVE)"); aes_decrypt(local_d.aes_key, local_d.aes_iv, ciphertext, decrypted, d->data_size); @@ -263,21 +297,25 @@ static int aes192_cbc_sha2_512_sve_run(struct test *test, int cpu) { /* SVE 多流 HMAC (主消息 + 变体) */ { const uint8_t *keys[NSTREAM], *msgs[NSTREAM]; + const int n = (int)svcntd(); /* 与 kernel 内部推导一致 */ uint8_t variant_keys[NSTREAM][64]; - for (int s = 0; s < NSTREAM; ++s) { + for (int s = 0; s < n; ++s) { memcpy(variant_keys[s], local_d.hmac_key, 64); if (s > 0) variant_keys[s][0] ^= (uint8_t)s; keys[s] = variant_keys[s]; } msgs[0] = ciphertext; - for (int s = 1; s < NSTREAM; ++s) { + for (int s = 1; s < n; ++s) { for (size_t i = 0; i < d->data_size; ++i) variant_msg[(s-1)*d->data_size + i] = ciphertext[i] ^ (uint8_t)(s * 0x5A + (i & 0x0F)); msgs[s] = variant_msg + (s-1)*d->data_size; } - hmac_sha512_x4(mac, keys, 64, msgs, d->data_size); + if (hmac_sha512_x4(mac, keys, 64, msgs, d->data_size) != 0) { + log_error("%s: HMAC capacity overflow (data_size too large)", test->id); + report_fail_msg("%s: hmac assembly over capacity", test->id); + } } - for (int s = 0; s < NSTREAM; ++s) + for (int s = 0; s < d->nstreams; ++s) memcmp_or_fail(mac[s], local_d.golden_mac[s], 64, "HMAC-SHA512 digest mismatch (SVE)"); } ++iter; @@ -285,7 +323,7 @@ static int aes192_cbc_sha2_512_sve_run(struct test *test, int cpu) { free(ciphertext); free(decrypted); free(variant_msg); - for (int s = 0; s < NSTREAM - 1; ++s) + for (int s = 0; s < d->nstreams - 1; ++s) free(local_d.variant_msg[s]); free(local_d.plaintext); free(local_d.golden_ciphertext); @@ -297,7 +335,7 @@ static int aes192_cbc_sha2_512_sve_cleanup(struct test *test) { if (d) { free(d->plaintext); free(d->golden_ciphertext); - for (int s = 0; s < NSTREAM - 1; ++s) + for (int s = 0; s < d->nstreams - 1; ++s) free(d->variant_msg[s]); } free(d); diff --git a/tests/cpu/sve/ipsec/ipsec_aes192_ctr_hmac_sha1_sve.cpp b/tests/cpu/sve/ipsec/ipsec_aes192_ctr_hmac_sha1_sve.cpp index 54a4a1403..609a2e8e0 100644 --- a/tests/cpu/sve/ipsec/ipsec_aes192_ctr_hmac_sha1_sve.cpp +++ b/tests/cpu/sve/ipsec/ipsec_aes192_ctr_hmac_sha1_sve.cpp @@ -7,12 +7,17 @@ * @parblock * SVE port of ipsec_aes192_ctr_hmac_sha1_avx: AES/3DES encryption stays on OpenSSL EVP while * the HMAC-SHA1-96 MAC runs on the multi-stream SVE HMAC implementation - * (verified bit-for-bit against OpenSSL; see scripts/sve-sha-verify). - * Eight independent HMAC lanes run in parallel on SVE vector lanes: lane 0 + * (HMAC construction validated same-source in gem5-fi across 128..1024-bit against an + * independent scalar oracle and RFC 2202/4231 vectors; the underlying SHA + * kernels were verified bit-for-bit against OpenSSL in the original + * fixed-stream form — see scripts/sve-sha-verify and + * docs/research/progress-log.md). + * N = svcntw() (32-bit family) independent HMAC lanes run in parallel on + * SVE vector lanes (all-VL 128..1024-bit, one message per 32-bit lane): lane 0 * carries the original ciphertext message (MAC semantics identical to the - * original test), lanes 1-7 carry deterministic key-stream variant + * original test), lanes 1..N-1 carry deterministic key-stream variant * messages that fill the remaining lanes — same computation family, - * golden values precomputed for all 8. Ciphertext, MACs and decrypted + * golden values precomputed for all N. Ciphertext, MACs and decrypted * data are compared against golden values on every iteration to detect * silent data corruption. Iterations alternate between a re-rolled input * with a same-core recomputed golden and a frozen replay of the init-time @@ -28,19 +33,15 @@ #include "sandstone_ssl.h" #include "sve_hmac.h" #include "sdc_stress_config.h" -#include +#include #include #include -#ifndef HWCAP_SVE -#define HWCAP_SVE (1 << 22) -#endif - #define HMAC_KEY_SIZE_V (20) #define HMAC_96_DIGEST_SIZE (12) #define AES_KEY_SIZE_V (24) #define AES_IV_SIZE_V (16) -#define NSTREAM (8) +#define NSTREAM (SVE_SHA_STREAMS_MAX_32) /* All-VL: 数组按 1024-bit 上限定维; 运行时流数 = svcntw() */ struct aes192_ctr_hmac_sha1_sve_data { uint8_t aes_key[AES_KEY_SIZE_V]; @@ -49,20 +50,21 @@ struct aes192_ctr_hmac_sha1_sve_data { uint8_t *plaintext; uint8_t *golden_ciphertext; uint8_t golden_mac[NSTREAM][20]; /* 内部全摘要 20B, 比较取前 12 */ - /* 7 条变体消息: 确定性派生(异或字节掩码), 填 SVE lane */ + /* N-1 条变体消息: 确定性派生(异或字节掩码), 填 SVE lane */ uint8_t *variant_msg[NSTREAM-1]; size_t data_size; + int nstreams; /* All-VL: init 期流数 = svl/4 (svcntw()), run 漂移守卫用 */ }; static size_t ipsec_data_size(struct test *test) { - int64_t v = sdc_knob_int_clamped(test, "data_size", 0, 1024, 16 << 20); + int64_t v = sdc_knob_int_clamped(test, "data_size", 0, 1024, 1900); if (v == 0) - sdc_env_knob_int_clamped("ipsec_aes192_ctr_hmac_sha1_sve", "data_size", &v, 1024, 16 << 20); + sdc_env_knob_int_clamped("ipsec_aes192_ctr_hmac_sha1_sve", "data_size", &v, 1024, 1900); if (v > 0) return (size_t)v; return (size_t)sdc_auto_linear_count(1, 4, 1.5, 600, thread_count(), - 1024, 16 << 20, test->id, "data_size"); + 1024, 1900, test->id, "data_size"); } static void aes_ctr_encrypt(const uint8_t *key, const uint8_t *iv, const uint8_t *in, uint8_t *out, size_t len) { @@ -84,54 +86,56 @@ static void aes_ctr_decrypt(const uint8_t *key, const uint8_t *iv, const uint8_t s_EVP_CIPHER_CTX_free(ctx); } -static void aes192_ctr_hmac_sha1_sve_compute_golden(struct aes192_ctr_hmac_sha1_sve_data *d) { +static int aes192_ctr_hmac_sha1_sve_compute_golden(struct test *test, struct aes192_ctr_hmac_sha1_sve_data *d) { aes_ctr_encrypt(d->aes_key, d->aes_iv, d->plaintext, d->golden_ciphertext, d->data_size); - /* SVE 多流 HMAC: lane0=密文(主语义), lane1..7=变体 */ + /* SVE 多流 HMAC: lane0=密文(主语义), lane1..N-1=变体 */ + const int n = (int)svcntw(); /* 与 kernel 内部推导一致 */ const uint8_t *keys[NSTREAM], *msgs[NSTREAM]; uint8_t variant_keys[NSTREAM][20]; - for (int s = 0; s < NSTREAM; ++s) { + for (int s = 0; s < n; ++s) { memcpy(variant_keys[s], d->hmac_key, 20); if (s > 0) variant_keys[s][0] ^= (uint8_t)s; /* 变体密钥: 确定性微扰 */ keys[s] = variant_keys[s]; } msgs[0] = d->golden_ciphertext; - for (int s = 1; s < NSTREAM; ++s) { + for (int s = 1; s < n; ++s) { /* 变体消息: 密文异或流掩码(确定性) */ for (size_t i = 0; i < d->data_size; ++i) d->variant_msg[s-1][i] = d->golden_ciphertext[i] ^ (uint8_t)(s * 0x5A + (i & 0x0F)); msgs[s] = d->variant_msg[s-1]; } - hmac_sha1_x8(d->golden_mac, keys, 20, msgs, d->data_size); + if (hmac_sha1_x8(d->golden_mac, keys, 20, msgs, d->data_size) != 0) { + log_error("%s: HMAC capacity overflow (data_size too large)", test->id); + return -1; + } + return 0; } static int aes192_ctr_hmac_sha1_sve_init(struct test *test) { if (s_EVP_CIPHER_CTX_new && s_EVP_EncryptInit_ex && s_EVP_DecryptInit_ex && s_EVP_CIPHER_CTX_new && s_EVP_aes_192_ctr()) { - unsigned long hwcap = getauxval(AT_HWCAP); - if ((hwcap & HWCAP_SVE) == 0) { - log_skip(CpuNotSupportedSkipCategory, - "to be implemented (placeholder): ARM SVE required for ipsec_aes192_ctr_hmac_sha1_sve"); - return EXIT_SKIP; - } - /* The multi-stream SVE HMAC kernels give one stream per SVE - * lane: 8x32-bit (sha1/224/256) needs svcntw()>=8 and - * 4x64-bit (384/512) needs svcntd()>=4 -- both mean VL>=256 - * bit. On shorter-VL SVE hardware (e.g. Neoverse N2, VL=128) - * lanes beyond VL are never written, so skip cleanly instead - * of comparing unwritten memory (target silicon: VL=256, see - * tests/cpu/sve/meson.build). */ - if (svcntw() < 8 || svcntd() < 4) { - log_skip(CpuNotSupportedSkipCategory, - "ARM SVE vector length %zu bytes < 32 required by the multi-stream HMAC kernels (svcntw=%zu svcntd=%zu)", - svcntb(), (size_t)svcntw(), (size_t)svcntd()); + /* All-VL Phase 2 P4: capability + 128..1024-bit range check in + * one framework-layer call (replaces the old lower-bound-only + * svcntw()>=8/svcntd()>=4 gate -- the 8.2 defect: at VL>=512 the + * fixed 8/4-wide W rows overflowed; the STREAMS_MAX-sized rows + + * this range gate close both ends). arm64_require_sve_vl returns + * the VL in BYTES. */ + const long svl = arm64_require_sve_vl(test, 128, 1024); + if (svl < 0) return EXIT_SKIP; + const int nstreams = (int)svl / 4; /* one stream per 32-bit lane (svcntw) */ + if (nstreams < 4 || nstreams > SVE_SHA_STREAMS_MAX_32) { /* [4, SVE_SHA_STREAMS_MAX_32], release-effective, NDEBUG-proof */ + log_error("%s: streams %d outside range — unit drift", test->id, nstreams); + return EXIT_FAILURE; } struct aes192_ctr_hmac_sha1_sve_data *d = (struct aes192_ctr_hmac_sha1_sve_data *)malloc(sizeof(*d)); if (!d) return EXIT_SKIP; + d->nstreams = nstreams; /* 工作集字节数: -O ipsec_aes192_ctr_hmac_sha1_sve.data_size=N / * SDC_IPSEC_AES192_CTR_HMAC_SHA1_SVE_DATA_SIZE 环境变量 / 自适应: - * 每线程明文+密文+golden 密文+解密缓冲+变体消息 ~= (3+NSTREAM-1)x, - * 占物理内存 60%, clamp [1024, 16M]。线性缓冲用 sdc_auto_linear_count(sqrt 缩放会欠缩放压力)。历史硬编码 1024。CTR 流式 + * 每线程明文+密文+golden 密文+解密缓冲+变体消息 ~= (3+nstreams-1)x, + * 占物理内存 60%, clamp [1024, 1900]——上限闭环 8.4: HMAC 组装 + * 缓冲 full[N][2048], 2048-64-9=1975B 安全边界。线性缓冲用 sdc_auto_linear_count(sqrt 缩放会欠缩放压力)。历史硬编码 1024。CTR 流式 * 无需分组对齐, 统一 16B 对齐无害, 对齐后 < 1024 取 1024。 */ d->data_size = ipsec_data_size(test); d->data_size &= ~(size_t)15; @@ -145,7 +149,7 @@ static int aes192_ctr_hmac_sha1_sve_init(struct test *test) { free(d); return EXIT_SKIP; } - for (int s = 0; s < NSTREAM - 1; ++s) { + for (int s = 0; s < d->nstreams - 1; ++s) { d->variant_msg[s] = (uint8_t *)malloc(d->data_size); if (!d->variant_msg[s]) { free(d->plaintext); @@ -160,7 +164,20 @@ static int aes192_ctr_hmac_sha1_sve_init(struct test *test) { memset_random(d->aes_iv, AES_IV_SIZE_V); memset_random(d->hmac_key, 20); memset_random(d->plaintext, d->data_size); - aes192_ctr_hmac_sha1_sve_compute_golden(d); + if (aes192_ctr_hmac_sha1_sve_compute_golden(test, d) != 0) { + /* P2a (review Minor): init-side failure propagates via the + * return value so the framework reports a clean Failed + * (sandstone_run.cpp: ret > 0 -> TestResult::Failed). + * report_fail_msg is a noreturn thread-cancel that only + * belongs in the test_run worker context — in test_init it + * would surface as Skipped("Unexpected OS error"). */ + free(d->plaintext); + free(d->golden_ciphertext); + for (int s = 0; s < d->nstreams - 1; ++s) + free(d->variant_msg[s]); + free(d); + return EXIT_FAILURE; + } test->data = d; return EXIT_SUCCESS; } else { @@ -172,10 +189,22 @@ static int aes192_ctr_hmac_sha1_sve_init(struct test *test) { static int aes192_ctr_hmac_sha1_sve_run(struct test *test, int cpu) { (void)cpu; struct aes192_ctr_hmac_sha1_sve_data *d = (struct aes192_ctr_hmac_sha1_sve_data *)test->data; + /* All-VL policy #3: never compare goldens across an init/run VL + * mismatch. Drift means this deployment's per-thread VL differs from + * the init thread's (e.g. a VL-pinning launcher changing the + * thread-inheritance chain) — an environment mismatch, which skips + * with CpuNotSupported (same category as the init-side range gate in + * arm64_require_sve_vl) instead of polluting the SDC failure signal. */ + if ((int)svcntw() != d->nstreams) { + log_skip(CpuNotSupportedSkipCategory, + "%s: SVE VL drifted between init (%d streams) and run (%d streams) — environment mismatch, not SDC", + test->id, d->nstreams, (int)svcntw()); + return EXIT_SKIP; + } uint8_t *ciphertext = (uint8_t *)malloc(d->data_size); uint8_t *decrypted = (uint8_t *)malloc(d->data_size); uint8_t mac[NSTREAM][20]; - uint8_t *variant_msg = (uint8_t *)malloc((NSTREAM-1) * d->data_size); + uint8_t *variant_msg = (uint8_t *)malloc((d->nstreams-1) * d->data_size); /* R2b (B2-003): per-thread plaintext + golden, re-rolled every * iteration (thread-safe: shared d is only read after init; the @@ -199,7 +228,7 @@ static int aes192_ctr_hmac_sha1_sve_run(struct test *test, int cpu) { "out of memory for per-thread buffers"); return EXIT_SKIP; } - for (int s = 0; s < NSTREAM - 1; ++s) { + for (int s = 0; s < d->nstreams - 1; ++s) { local_d.variant_msg[s] = (uint8_t *)malloc(d->data_size); if (!local_d.variant_msg[s]) { free(local_d.plaintext); @@ -232,27 +261,32 @@ static int aes192_ctr_hmac_sha1_sve_run(struct test *test, int cpu) { /* SVE 多流 HMAC (主消息 + 变体) */ { const uint8_t *keys[NSTREAM], *msgs[NSTREAM]; + const int n = (int)svcntw(); /* 与 kernel 内部推导一致 */ uint8_t variant_keys[NSTREAM][20]; - for (int s = 0; s < NSTREAM; ++s) { + for (int s = 0; s < n; ++s) { memcpy(variant_keys[s], d->hmac_key, 20); if (s > 0) variant_keys[s][0] ^= (uint8_t)s; keys[s] = variant_keys[s]; } msgs[0] = ciphertext; - for (int s = 1; s < NSTREAM; ++s) { + for (int s = 1; s < n; ++s) { for (size_t i = 0; i < d->data_size; ++i) variant_msg[(s-1)*d->data_size + i] = ciphertext[i] ^ (uint8_t)(s * 0x5A + (i & 0x0F)); msgs[s] = variant_msg + (s-1)*d->data_size; } - hmac_sha1_x8(mac, keys, 20, msgs, d->data_size); + if (hmac_sha1_x8(mac, keys, 20, msgs, d->data_size) != 0) { + log_error("%s: HMAC capacity overflow (data_size too large)", test->id); + report_fail_msg("%s: hmac assembly over capacity", test->id); + } } - for (int s = 0; s < NSTREAM; ++s) + for (int s = 0; s < d->nstreams; ++s) memcmp_or_fail(mac[s], d->golden_mac[s], 12, "HMAC-SHA1-96 digest mismatch (frozen replay)"); } else { /* re-roll: fresh plaintext + same-iteration golden (EVP encrypt + * SVE HMAC recomputed for this thread's data) */ memset_random(local_d.plaintext, d->data_size); - aes192_ctr_hmac_sha1_sve_compute_golden(&local_d); + if (aes192_ctr_hmac_sha1_sve_compute_golden(test, &local_d) != 0) + report_fail_msg("%s: hmac assembly over capacity", test->id); aes_ctr_encrypt(local_d.aes_key, local_d.aes_iv, local_d.plaintext, ciphertext, d->data_size); memcmp_or_fail(ciphertext, local_d.golden_ciphertext, d->data_size, "ciphertext mismatch (SVE)"); aes_ctr_decrypt(local_d.aes_key, local_d.aes_iv, ciphertext, decrypted, d->data_size); @@ -261,21 +295,25 @@ static int aes192_ctr_hmac_sha1_sve_run(struct test *test, int cpu) { /* SVE 多流 HMAC (主消息 + 变体) */ { const uint8_t *keys[NSTREAM], *msgs[NSTREAM]; + const int n = (int)svcntw(); /* 与 kernel 内部推导一致 */ uint8_t variant_keys[NSTREAM][20]; - for (int s = 0; s < NSTREAM; ++s) { + for (int s = 0; s < n; ++s) { memcpy(variant_keys[s], local_d.hmac_key, 20); if (s > 0) variant_keys[s][0] ^= (uint8_t)s; keys[s] = variant_keys[s]; } msgs[0] = ciphertext; - for (int s = 1; s < NSTREAM; ++s) { + for (int s = 1; s < n; ++s) { for (size_t i = 0; i < d->data_size; ++i) variant_msg[(s-1)*d->data_size + i] = ciphertext[i] ^ (uint8_t)(s * 0x5A + (i & 0x0F)); msgs[s] = variant_msg + (s-1)*d->data_size; } - hmac_sha1_x8(mac, keys, 20, msgs, d->data_size); + if (hmac_sha1_x8(mac, keys, 20, msgs, d->data_size) != 0) { + log_error("%s: HMAC capacity overflow (data_size too large)", test->id); + report_fail_msg("%s: hmac assembly over capacity", test->id); + } } - for (int s = 0; s < NSTREAM; ++s) + for (int s = 0; s < d->nstreams; ++s) memcmp_or_fail(mac[s], local_d.golden_mac[s], 12, "HMAC-SHA1-96 digest mismatch (SVE)"); } ++iter; @@ -283,7 +321,7 @@ static int aes192_ctr_hmac_sha1_sve_run(struct test *test, int cpu) { free(ciphertext); free(decrypted); free(variant_msg); - for (int s = 0; s < NSTREAM - 1; ++s) + for (int s = 0; s < d->nstreams - 1; ++s) free(local_d.variant_msg[s]); free(local_d.plaintext); free(local_d.golden_ciphertext); @@ -295,7 +333,7 @@ static int aes192_ctr_hmac_sha1_sve_cleanup(struct test *test) { if (d) { free(d->plaintext); free(d->golden_ciphertext); - for (int s = 0; s < NSTREAM - 1; ++s) + for (int s = 0; s < d->nstreams - 1; ++s) free(d->variant_msg[s]); } free(d); diff --git a/tests/cpu/sve/ipsec/ipsec_aes192_ctr_hmac_sha224_sve.cpp b/tests/cpu/sve/ipsec/ipsec_aes192_ctr_hmac_sha224_sve.cpp index be08567a1..c7c25d798 100644 --- a/tests/cpu/sve/ipsec/ipsec_aes192_ctr_hmac_sha224_sve.cpp +++ b/tests/cpu/sve/ipsec/ipsec_aes192_ctr_hmac_sha224_sve.cpp @@ -7,12 +7,17 @@ * @parblock * SVE port of ipsec_aes192_ctr_hmac_sha224_avx: AES/3DES encryption stays on OpenSSL EVP while * the HMAC-SHA224 MAC runs on the multi-stream SVE HMAC implementation - * (verified bit-for-bit against OpenSSL; see scripts/sve-sha-verify). - * Eight independent HMAC lanes run in parallel on SVE vector lanes: lane 0 + * (HMAC construction validated same-source in gem5-fi across 128..1024-bit against an + * independent scalar oracle and RFC 2202/4231 vectors; the underlying SHA + * kernels were verified bit-for-bit against OpenSSL in the original + * fixed-stream form — see scripts/sve-sha-verify and + * docs/research/progress-log.md). + * N = svcntw() (32-bit family) independent HMAC lanes run in parallel on + * SVE vector lanes (all-VL 128..1024-bit, one message per 32-bit lane): lane 0 * carries the original ciphertext message (MAC semantics identical to the - * original test), lanes 1-7 carry deterministic key-stream variant + * original test), lanes 1..N-1 carry deterministic key-stream variant * messages that fill the remaining lanes — same computation family, - * golden values precomputed for all 8. Ciphertext, MACs and decrypted + * golden values precomputed for all N. Ciphertext, MACs and decrypted * data are compared against golden values on every iteration to detect * silent data corruption. Iterations alternate between a re-rolled input * with a same-core recomputed golden and a frozen replay of the init-time @@ -28,19 +33,15 @@ #include "sandstone_ssl.h" #include "sve_hmac.h" #include "sdc_stress_config.h" -#include +#include #include #include -#ifndef HWCAP_SVE -#define HWCAP_SVE (1 << 22) -#endif - #define HMAC_KEY_SIZE_V (28) #define SHA224_DIGEST_SIZE (28) #define AES_KEY_SIZE_V (24) #define AES_IV_SIZE_V (16) -#define NSTREAM (8) +#define NSTREAM (SVE_SHA_STREAMS_MAX_32) /* All-VL: 数组按 1024-bit 上限定维; 运行时流数 = svcntw() */ struct aes192_ctr_hmac_sha224_sve_data { uint8_t aes_key[AES_KEY_SIZE_V]; @@ -49,20 +50,21 @@ struct aes192_ctr_hmac_sha224_sve_data { uint8_t *plaintext; uint8_t *golden_ciphertext; uint8_t golden_mac[NSTREAM][28]; - /* 7 条变体消息: 确定性派生(异或字节掩码), 填 SVE lane */ + /* N-1 条变体消息: 确定性派生(异或字节掩码), 填 SVE lane */ uint8_t *variant_msg[NSTREAM-1]; size_t data_size; + int nstreams; /* All-VL: init 期流数 = svl/4 (svcntw()), run 漂移守卫用 */ }; static size_t ipsec_data_size(struct test *test) { - int64_t v = sdc_knob_int_clamped(test, "data_size", 0, 1024, 16 << 20); + int64_t v = sdc_knob_int_clamped(test, "data_size", 0, 1024, 1900); if (v == 0) - sdc_env_knob_int_clamped("ipsec_aes192_ctr_hmac_sha224_sve", "data_size", &v, 1024, 16 << 20); + sdc_env_knob_int_clamped("ipsec_aes192_ctr_hmac_sha224_sve", "data_size", &v, 1024, 1900); if (v > 0) return (size_t)v; return (size_t)sdc_auto_linear_count(1, 4, 1.5, 600, thread_count(), - 1024, 16 << 20, test->id, "data_size"); + 1024, 1900, test->id, "data_size"); } static void aes_ctr_encrypt(const uint8_t *key, const uint8_t *iv, const uint8_t *in, uint8_t *out, size_t len) { @@ -84,54 +86,56 @@ static void aes_ctr_decrypt(const uint8_t *key, const uint8_t *iv, const uint8_t s_EVP_CIPHER_CTX_free(ctx); } -static void aes192_ctr_hmac_sha224_sve_compute_golden(struct aes192_ctr_hmac_sha224_sve_data *d) { +static int aes192_ctr_hmac_sha224_sve_compute_golden(struct test *test, struct aes192_ctr_hmac_sha224_sve_data *d) { aes_ctr_encrypt(d->aes_key, d->aes_iv, d->plaintext, d->golden_ciphertext, d->data_size); - /* SVE 多流 HMAC: lane0=密文(主语义), lane1..7=变体 */ + /* SVE 多流 HMAC: lane0=密文(主语义), lane1..N-1=变体 */ + const int n = (int)svcntw(); /* 与 kernel 内部推导一致 */ const uint8_t *keys[NSTREAM], *msgs[NSTREAM]; uint8_t variant_keys[NSTREAM][28]; - for (int s = 0; s < NSTREAM; ++s) { + for (int s = 0; s < n; ++s) { memcpy(variant_keys[s], d->hmac_key, 28); if (s > 0) variant_keys[s][0] ^= (uint8_t)s; /* 变体密钥: 确定性微扰 */ keys[s] = variant_keys[s]; } msgs[0] = d->golden_ciphertext; - for (int s = 1; s < NSTREAM; ++s) { + for (int s = 1; s < n; ++s) { /* 变体消息: 密文异或流掩码(确定性) */ for (size_t i = 0; i < d->data_size; ++i) d->variant_msg[s-1][i] = d->golden_ciphertext[i] ^ (uint8_t)(s * 0x5A + (i & 0x0F)); msgs[s] = d->variant_msg[s-1]; } - hmac_sha224_x8(d->golden_mac, keys, 28, msgs, d->data_size); + if (hmac_sha224_x8(d->golden_mac, keys, 28, msgs, d->data_size) != 0) { + log_error("%s: HMAC capacity overflow (data_size too large)", test->id); + return -1; + } + return 0; } static int aes192_ctr_hmac_sha224_sve_init(struct test *test) { if (s_EVP_CIPHER_CTX_new && s_EVP_EncryptInit_ex && s_EVP_DecryptInit_ex && s_EVP_CIPHER_CTX_new && s_EVP_aes_192_ctr()) { - unsigned long hwcap = getauxval(AT_HWCAP); - if ((hwcap & HWCAP_SVE) == 0) { - log_skip(CpuNotSupportedSkipCategory, - "to be implemented (placeholder): ARM SVE required for ipsec_aes192_ctr_hmac_sha224_sve"); - return EXIT_SKIP; - } - /* The multi-stream SVE HMAC kernels give one stream per SVE - * lane: 8x32-bit (sha1/224/256) needs svcntw()>=8 and - * 4x64-bit (384/512) needs svcntd()>=4 -- both mean VL>=256 - * bit. On shorter-VL SVE hardware (e.g. Neoverse N2, VL=128) - * lanes beyond VL are never written, so skip cleanly instead - * of comparing unwritten memory (target silicon: VL=256, see - * tests/cpu/sve/meson.build). */ - if (svcntw() < 8 || svcntd() < 4) { - log_skip(CpuNotSupportedSkipCategory, - "ARM SVE vector length %zu bytes < 32 required by the multi-stream HMAC kernels (svcntw=%zu svcntd=%zu)", - svcntb(), (size_t)svcntw(), (size_t)svcntd()); + /* All-VL Phase 2 P4: capability + 128..1024-bit range check in + * one framework-layer call (replaces the old lower-bound-only + * svcntw()>=8/svcntd()>=4 gate -- the 8.2 defect: at VL>=512 the + * fixed 8/4-wide W rows overflowed; the STREAMS_MAX-sized rows + + * this range gate close both ends). arm64_require_sve_vl returns + * the VL in BYTES. */ + const long svl = arm64_require_sve_vl(test, 128, 1024); + if (svl < 0) return EXIT_SKIP; + const int nstreams = (int)svl / 4; /* one stream per 32-bit lane (svcntw) */ + if (nstreams < 4 || nstreams > SVE_SHA_STREAMS_MAX_32) { /* [4, SVE_SHA_STREAMS_MAX_32], release-effective, NDEBUG-proof */ + log_error("%s: streams %d outside range — unit drift", test->id, nstreams); + return EXIT_FAILURE; } struct aes192_ctr_hmac_sha224_sve_data *d = (struct aes192_ctr_hmac_sha224_sve_data *)malloc(sizeof(*d)); if (!d) return EXIT_SKIP; + d->nstreams = nstreams; /* 工作集字节数: -O ipsec_aes192_ctr_hmac_sha224_sve.data_size=N / * SDC_IPSEC_AES192_CTR_HMAC_SHA224_SVE_DATA_SIZE 环境变量 / 自适应: - * 每线程明文+密文+golden 密文+解密缓冲+变体消息 ~= (3+NSTREAM-1)x, - * 占物理内存 60%, clamp [1024, 16M]。线性缓冲用 sdc_auto_linear_count(sqrt 缩放会欠缩放压力)。历史硬编码 1024。CTR 流式 + * 每线程明文+密文+golden 密文+解密缓冲+变体消息 ~= (3+nstreams-1)x, + * 占物理内存 60%, clamp [1024, 1900]——上限闭环 8.4: HMAC 组装 + * 缓冲 full[N][2048], 2048-64-9=1975B 安全边界。线性缓冲用 sdc_auto_linear_count(sqrt 缩放会欠缩放压力)。历史硬编码 1024。CTR 流式 * 无需分组对齐, 统一 16B 对齐无害, 对齐后 < 1024 取 1024。 */ d->data_size = ipsec_data_size(test); d->data_size &= ~(size_t)15; @@ -145,7 +149,7 @@ static int aes192_ctr_hmac_sha224_sve_init(struct test *test) { free(d); return EXIT_SKIP; } - for (int s = 0; s < NSTREAM - 1; ++s) { + for (int s = 0; s < d->nstreams - 1; ++s) { d->variant_msg[s] = (uint8_t *)malloc(d->data_size); if (!d->variant_msg[s]) { free(d->plaintext); @@ -160,7 +164,20 @@ static int aes192_ctr_hmac_sha224_sve_init(struct test *test) { memset_random(d->aes_iv, AES_IV_SIZE_V); memset_random(d->hmac_key, 28); memset_random(d->plaintext, d->data_size); - aes192_ctr_hmac_sha224_sve_compute_golden(d); + if (aes192_ctr_hmac_sha224_sve_compute_golden(test, d) != 0) { + /* P2a (review Minor): init-side failure propagates via the + * return value so the framework reports a clean Failed + * (sandstone_run.cpp: ret > 0 -> TestResult::Failed). + * report_fail_msg is a noreturn thread-cancel that only + * belongs in the test_run worker context — in test_init it + * would surface as Skipped("Unexpected OS error"). */ + free(d->plaintext); + free(d->golden_ciphertext); + for (int s = 0; s < d->nstreams - 1; ++s) + free(d->variant_msg[s]); + free(d); + return EXIT_FAILURE; + } test->data = d; return EXIT_SUCCESS; } else { @@ -172,10 +189,22 @@ static int aes192_ctr_hmac_sha224_sve_init(struct test *test) { static int aes192_ctr_hmac_sha224_sve_run(struct test *test, int cpu) { (void)cpu; struct aes192_ctr_hmac_sha224_sve_data *d = (struct aes192_ctr_hmac_sha224_sve_data *)test->data; + /* All-VL policy #3: never compare goldens across an init/run VL + * mismatch. Drift means this deployment's per-thread VL differs from + * the init thread's (e.g. a VL-pinning launcher changing the + * thread-inheritance chain) — an environment mismatch, which skips + * with CpuNotSupported (same category as the init-side range gate in + * arm64_require_sve_vl) instead of polluting the SDC failure signal. */ + if ((int)svcntw() != d->nstreams) { + log_skip(CpuNotSupportedSkipCategory, + "%s: SVE VL drifted between init (%d streams) and run (%d streams) — environment mismatch, not SDC", + test->id, d->nstreams, (int)svcntw()); + return EXIT_SKIP; + } uint8_t *ciphertext = (uint8_t *)malloc(d->data_size); uint8_t *decrypted = (uint8_t *)malloc(d->data_size); uint8_t mac[NSTREAM][28]; - uint8_t *variant_msg = (uint8_t *)malloc((NSTREAM-1) * d->data_size); + uint8_t *variant_msg = (uint8_t *)malloc((d->nstreams-1) * d->data_size); /* R2b (B2-003): per-thread plaintext + golden, re-rolled every * iteration (thread-safe: shared d is only read after init; the @@ -199,7 +228,7 @@ static int aes192_ctr_hmac_sha224_sve_run(struct test *test, int cpu) { "out of memory for per-thread buffers"); return EXIT_SKIP; } - for (int s = 0; s < NSTREAM - 1; ++s) { + for (int s = 0; s < d->nstreams - 1; ++s) { local_d.variant_msg[s] = (uint8_t *)malloc(d->data_size); if (!local_d.variant_msg[s]) { free(local_d.plaintext); @@ -232,27 +261,32 @@ static int aes192_ctr_hmac_sha224_sve_run(struct test *test, int cpu) { /* SVE 多流 HMAC (主消息 + 变体) */ { const uint8_t *keys[NSTREAM], *msgs[NSTREAM]; + const int n = (int)svcntw(); /* 与 kernel 内部推导一致 */ uint8_t variant_keys[NSTREAM][28]; - for (int s = 0; s < NSTREAM; ++s) { + for (int s = 0; s < n; ++s) { memcpy(variant_keys[s], d->hmac_key, 28); if (s > 0) variant_keys[s][0] ^= (uint8_t)s; keys[s] = variant_keys[s]; } msgs[0] = ciphertext; - for (int s = 1; s < NSTREAM; ++s) { + for (int s = 1; s < n; ++s) { for (size_t i = 0; i < d->data_size; ++i) variant_msg[(s-1)*d->data_size + i] = ciphertext[i] ^ (uint8_t)(s * 0x5A + (i & 0x0F)); msgs[s] = variant_msg + (s-1)*d->data_size; } - hmac_sha224_x8(mac, keys, 28, msgs, d->data_size); + if (hmac_sha224_x8(mac, keys, 28, msgs, d->data_size) != 0) { + log_error("%s: HMAC capacity overflow (data_size too large)", test->id); + report_fail_msg("%s: hmac assembly over capacity", test->id); + } } - for (int s = 0; s < NSTREAM; ++s) + for (int s = 0; s < d->nstreams; ++s) memcmp_or_fail(mac[s], d->golden_mac[s], 28, "HMAC-SHA224 digest mismatch (frozen replay)"); } else { /* re-roll: fresh plaintext + same-iteration golden (EVP encrypt + * SVE HMAC recomputed for this thread's data) */ memset_random(local_d.plaintext, d->data_size); - aes192_ctr_hmac_sha224_sve_compute_golden(&local_d); + if (aes192_ctr_hmac_sha224_sve_compute_golden(test, &local_d) != 0) + report_fail_msg("%s: hmac assembly over capacity", test->id); aes_ctr_encrypt(local_d.aes_key, local_d.aes_iv, local_d.plaintext, ciphertext, d->data_size); memcmp_or_fail(ciphertext, local_d.golden_ciphertext, d->data_size, "ciphertext mismatch (SVE)"); aes_ctr_decrypt(local_d.aes_key, local_d.aes_iv, ciphertext, decrypted, d->data_size); @@ -261,21 +295,25 @@ static int aes192_ctr_hmac_sha224_sve_run(struct test *test, int cpu) { /* SVE 多流 HMAC (主消息 + 变体) */ { const uint8_t *keys[NSTREAM], *msgs[NSTREAM]; + const int n = (int)svcntw(); /* 与 kernel 内部推导一致 */ uint8_t variant_keys[NSTREAM][28]; - for (int s = 0; s < NSTREAM; ++s) { + for (int s = 0; s < n; ++s) { memcpy(variant_keys[s], local_d.hmac_key, 28); if (s > 0) variant_keys[s][0] ^= (uint8_t)s; keys[s] = variant_keys[s]; } msgs[0] = ciphertext; - for (int s = 1; s < NSTREAM; ++s) { + for (int s = 1; s < n; ++s) { for (size_t i = 0; i < d->data_size; ++i) variant_msg[(s-1)*d->data_size + i] = ciphertext[i] ^ (uint8_t)(s * 0x5A + (i & 0x0F)); msgs[s] = variant_msg + (s-1)*d->data_size; } - hmac_sha224_x8(mac, keys, 28, msgs, d->data_size); + if (hmac_sha224_x8(mac, keys, 28, msgs, d->data_size) != 0) { + log_error("%s: HMAC capacity overflow (data_size too large)", test->id); + report_fail_msg("%s: hmac assembly over capacity", test->id); + } } - for (int s = 0; s < NSTREAM; ++s) + for (int s = 0; s < d->nstreams; ++s) memcmp_or_fail(mac[s], local_d.golden_mac[s], 28, "HMAC-SHA224 digest mismatch (SVE)"); } ++iter; @@ -283,7 +321,7 @@ static int aes192_ctr_hmac_sha224_sve_run(struct test *test, int cpu) { free(ciphertext); free(decrypted); free(variant_msg); - for (int s = 0; s < NSTREAM - 1; ++s) + for (int s = 0; s < d->nstreams - 1; ++s) free(local_d.variant_msg[s]); free(local_d.plaintext); free(local_d.golden_ciphertext); @@ -295,7 +333,7 @@ static int aes192_ctr_hmac_sha224_sve_cleanup(struct test *test) { if (d) { free(d->plaintext); free(d->golden_ciphertext); - for (int s = 0; s < NSTREAM - 1; ++s) + for (int s = 0; s < d->nstreams - 1; ++s) free(d->variant_msg[s]); } free(d); diff --git a/tests/cpu/sve/ipsec/ipsec_aes192_ctr_hmac_sha256_sve.cpp b/tests/cpu/sve/ipsec/ipsec_aes192_ctr_hmac_sha256_sve.cpp index d3f9ccb85..40a7f0d32 100644 --- a/tests/cpu/sve/ipsec/ipsec_aes192_ctr_hmac_sha256_sve.cpp +++ b/tests/cpu/sve/ipsec/ipsec_aes192_ctr_hmac_sha256_sve.cpp @@ -7,12 +7,17 @@ * @parblock * SVE port of ipsec_aes192_ctr_hmac_sha256_avx: AES/3DES encryption stays on OpenSSL EVP while * the HMAC-SHA256 MAC runs on the multi-stream SVE HMAC implementation - * (verified bit-for-bit against OpenSSL; see scripts/sve-sha-verify). - * Eight independent HMAC lanes run in parallel on SVE vector lanes: lane 0 + * (HMAC construction validated same-source in gem5-fi across 128..1024-bit against an + * independent scalar oracle and RFC 2202/4231 vectors; the underlying SHA + * kernels were verified bit-for-bit against OpenSSL in the original + * fixed-stream form — see scripts/sve-sha-verify and + * docs/research/progress-log.md). + * N = svcntw() (32-bit family) independent HMAC lanes run in parallel on + * SVE vector lanes (all-VL 128..1024-bit, one message per 32-bit lane): lane 0 * carries the original ciphertext message (MAC semantics identical to the - * original test), lanes 1-7 carry deterministic key-stream variant + * original test), lanes 1..N-1 carry deterministic key-stream variant * messages that fill the remaining lanes — same computation family, - * golden values precomputed for all 8. Ciphertext, MACs and decrypted + * golden values precomputed for all N. Ciphertext, MACs and decrypted * data are compared against golden values on every iteration to detect * silent data corruption. Iterations alternate between a re-rolled input * with a same-core recomputed golden and a frozen replay of the init-time @@ -28,19 +33,15 @@ #include "sandstone_ssl.h" #include "sve_hmac.h" #include "sdc_stress_config.h" -#include +#include #include #include -#ifndef HWCAP_SVE -#define HWCAP_SVE (1 << 22) -#endif - #define HMAC_KEY_SIZE_V (32) #define SHA256_DIGEST_SIZE (32) #define AES_KEY_SIZE_V (24) #define AES_IV_SIZE_V (16) -#define NSTREAM (8) +#define NSTREAM (SVE_SHA_STREAMS_MAX_32) /* All-VL: 数组按 1024-bit 上限定维; 运行时流数 = svcntw() */ struct aes192_ctr_hmac_sha256_sve_data { uint8_t aes_key[AES_KEY_SIZE_V]; @@ -49,20 +50,21 @@ struct aes192_ctr_hmac_sha256_sve_data { uint8_t *plaintext; uint8_t *golden_ciphertext; uint8_t golden_mac[NSTREAM][32]; - /* 7 条变体消息: 确定性派生(异或字节掩码), 填 SVE lane */ + /* N-1 条变体消息: 确定性派生(异或字节掩码), 填 SVE lane */ uint8_t *variant_msg[NSTREAM-1]; size_t data_size; + int nstreams; /* All-VL: init 期流数 = svl/4 (svcntw()), run 漂移守卫用 */ }; static size_t ipsec_data_size(struct test *test) { - int64_t v = sdc_knob_int_clamped(test, "data_size", 0, 1024, 16 << 20); + int64_t v = sdc_knob_int_clamped(test, "data_size", 0, 1024, 1900); if (v == 0) - sdc_env_knob_int_clamped("ipsec_aes192_ctr_hmac_sha256_sve", "data_size", &v, 1024, 16 << 20); + sdc_env_knob_int_clamped("ipsec_aes192_ctr_hmac_sha256_sve", "data_size", &v, 1024, 1900); if (v > 0) return (size_t)v; return (size_t)sdc_auto_linear_count(1, 4, 1.5, 600, thread_count(), - 1024, 16 << 20, test->id, "data_size"); + 1024, 1900, test->id, "data_size"); } static void aes_ctr_encrypt(const uint8_t *key, const uint8_t *iv, const uint8_t *in, uint8_t *out, size_t len) { @@ -84,54 +86,56 @@ static void aes_ctr_decrypt(const uint8_t *key, const uint8_t *iv, const uint8_t s_EVP_CIPHER_CTX_free(ctx); } -static void aes192_ctr_hmac_sha256_sve_compute_golden(struct aes192_ctr_hmac_sha256_sve_data *d) { +static int aes192_ctr_hmac_sha256_sve_compute_golden(struct test *test, struct aes192_ctr_hmac_sha256_sve_data *d) { aes_ctr_encrypt(d->aes_key, d->aes_iv, d->plaintext, d->golden_ciphertext, d->data_size); - /* SVE 多流 HMAC: lane0=密文(主语义), lane1..7=变体 */ + /* SVE 多流 HMAC: lane0=密文(主语义), lane1..N-1=变体 */ + const int n = (int)svcntw(); /* 与 kernel 内部推导一致 */ const uint8_t *keys[NSTREAM], *msgs[NSTREAM]; uint8_t variant_keys[NSTREAM][32]; - for (int s = 0; s < NSTREAM; ++s) { + for (int s = 0; s < n; ++s) { memcpy(variant_keys[s], d->hmac_key, 32); if (s > 0) variant_keys[s][0] ^= (uint8_t)s; /* 变体密钥: 确定性微扰 */ keys[s] = variant_keys[s]; } msgs[0] = d->golden_ciphertext; - for (int s = 1; s < NSTREAM; ++s) { + for (int s = 1; s < n; ++s) { /* 变体消息: 密文异或流掩码(确定性) */ for (size_t i = 0; i < d->data_size; ++i) d->variant_msg[s-1][i] = d->golden_ciphertext[i] ^ (uint8_t)(s * 0x5A + (i & 0x0F)); msgs[s] = d->variant_msg[s-1]; } - hmac_sha256_x8(d->golden_mac, keys, 32, msgs, d->data_size); + if (hmac_sha256_x8(d->golden_mac, keys, 32, msgs, d->data_size) != 0) { + log_error("%s: HMAC capacity overflow (data_size too large)", test->id); + return -1; + } + return 0; } static int aes192_ctr_hmac_sha256_sve_init(struct test *test) { if (s_EVP_CIPHER_CTX_new && s_EVP_EncryptInit_ex && s_EVP_DecryptInit_ex && s_EVP_CIPHER_CTX_new && s_EVP_aes_192_ctr()) { - unsigned long hwcap = getauxval(AT_HWCAP); - if ((hwcap & HWCAP_SVE) == 0) { - log_skip(CpuNotSupportedSkipCategory, - "to be implemented (placeholder): ARM SVE required for ipsec_aes192_ctr_hmac_sha256_sve"); - return EXIT_SKIP; - } - /* The multi-stream SVE HMAC kernels give one stream per SVE - * lane: 8x32-bit (sha1/224/256) needs svcntw()>=8 and - * 4x64-bit (384/512) needs svcntd()>=4 -- both mean VL>=256 - * bit. On shorter-VL SVE hardware (e.g. Neoverse N2, VL=128) - * lanes beyond VL are never written, so skip cleanly instead - * of comparing unwritten memory (target silicon: VL=256, see - * tests/cpu/sve/meson.build). */ - if (svcntw() < 8 || svcntd() < 4) { - log_skip(CpuNotSupportedSkipCategory, - "ARM SVE vector length %zu bytes < 32 required by the multi-stream HMAC kernels (svcntw=%zu svcntd=%zu)", - svcntb(), (size_t)svcntw(), (size_t)svcntd()); + /* All-VL Phase 2 P4: capability + 128..1024-bit range check in + * one framework-layer call (replaces the old lower-bound-only + * svcntw()>=8/svcntd()>=4 gate -- the 8.2 defect: at VL>=512 the + * fixed 8/4-wide W rows overflowed; the STREAMS_MAX-sized rows + + * this range gate close both ends). arm64_require_sve_vl returns + * the VL in BYTES. */ + const long svl = arm64_require_sve_vl(test, 128, 1024); + if (svl < 0) return EXIT_SKIP; + const int nstreams = (int)svl / 4; /* one stream per 32-bit lane (svcntw) */ + if (nstreams < 4 || nstreams > SVE_SHA_STREAMS_MAX_32) { /* [4, SVE_SHA_STREAMS_MAX_32], release-effective, NDEBUG-proof */ + log_error("%s: streams %d outside range — unit drift", test->id, nstreams); + return EXIT_FAILURE; } struct aes192_ctr_hmac_sha256_sve_data *d = (struct aes192_ctr_hmac_sha256_sve_data *)malloc(sizeof(*d)); if (!d) return EXIT_SKIP; + d->nstreams = nstreams; /* 工作集字节数: -O ipsec_aes192_ctr_hmac_sha256_sve.data_size=N / * SDC_IPSEC_AES192_CTR_HMAC_SHA256_SVE_DATA_SIZE 环境变量 / 自适应: - * 每线程明文+密文+golden 密文+解密缓冲+变体消息 ~= (3+NSTREAM-1)x, - * 占物理内存 60%, clamp [1024, 16M]。线性缓冲用 sdc_auto_linear_count(sqrt 缩放会欠缩放压力)。历史硬编码 1024。CTR 流式 + * 每线程明文+密文+golden 密文+解密缓冲+变体消息 ~= (3+nstreams-1)x, + * 占物理内存 60%, clamp [1024, 1900]——上限闭环 8.4: HMAC 组装 + * 缓冲 full[N][2048], 2048-64-9=1975B 安全边界。线性缓冲用 sdc_auto_linear_count(sqrt 缩放会欠缩放压力)。历史硬编码 1024。CTR 流式 * 无需分组对齐, 统一 16B 对齐无害, 对齐后 < 1024 取 1024。 */ d->data_size = ipsec_data_size(test); d->data_size &= ~(size_t)15; @@ -145,7 +149,7 @@ static int aes192_ctr_hmac_sha256_sve_init(struct test *test) { free(d); return EXIT_SKIP; } - for (int s = 0; s < NSTREAM - 1; ++s) { + for (int s = 0; s < d->nstreams - 1; ++s) { d->variant_msg[s] = (uint8_t *)malloc(d->data_size); if (!d->variant_msg[s]) { free(d->plaintext); @@ -160,7 +164,20 @@ static int aes192_ctr_hmac_sha256_sve_init(struct test *test) { memset_random(d->aes_iv, AES_IV_SIZE_V); memset_random(d->hmac_key, 32); memset_random(d->plaintext, d->data_size); - aes192_ctr_hmac_sha256_sve_compute_golden(d); + if (aes192_ctr_hmac_sha256_sve_compute_golden(test, d) != 0) { + /* P2a (review Minor): init-side failure propagates via the + * return value so the framework reports a clean Failed + * (sandstone_run.cpp: ret > 0 -> TestResult::Failed). + * report_fail_msg is a noreturn thread-cancel that only + * belongs in the test_run worker context — in test_init it + * would surface as Skipped("Unexpected OS error"). */ + free(d->plaintext); + free(d->golden_ciphertext); + for (int s = 0; s < d->nstreams - 1; ++s) + free(d->variant_msg[s]); + free(d); + return EXIT_FAILURE; + } test->data = d; return EXIT_SUCCESS; } else { @@ -172,10 +189,22 @@ static int aes192_ctr_hmac_sha256_sve_init(struct test *test) { static int aes192_ctr_hmac_sha256_sve_run(struct test *test, int cpu) { (void)cpu; struct aes192_ctr_hmac_sha256_sve_data *d = (struct aes192_ctr_hmac_sha256_sve_data *)test->data; + /* All-VL policy #3: never compare goldens across an init/run VL + * mismatch. Drift means this deployment's per-thread VL differs from + * the init thread's (e.g. a VL-pinning launcher changing the + * thread-inheritance chain) — an environment mismatch, which skips + * with CpuNotSupported (same category as the init-side range gate in + * arm64_require_sve_vl) instead of polluting the SDC failure signal. */ + if ((int)svcntw() != d->nstreams) { + log_skip(CpuNotSupportedSkipCategory, + "%s: SVE VL drifted between init (%d streams) and run (%d streams) — environment mismatch, not SDC", + test->id, d->nstreams, (int)svcntw()); + return EXIT_SKIP; + } uint8_t *ciphertext = (uint8_t *)malloc(d->data_size); uint8_t *decrypted = (uint8_t *)malloc(d->data_size); uint8_t mac[NSTREAM][32]; - uint8_t *variant_msg = (uint8_t *)malloc((NSTREAM-1) * d->data_size); + uint8_t *variant_msg = (uint8_t *)malloc((d->nstreams-1) * d->data_size); /* R2b (B2-003): per-thread plaintext + golden, re-rolled every * iteration (thread-safe: shared d is only read after init; the @@ -199,7 +228,7 @@ static int aes192_ctr_hmac_sha256_sve_run(struct test *test, int cpu) { "out of memory for per-thread buffers"); return EXIT_SKIP; } - for (int s = 0; s < NSTREAM - 1; ++s) { + for (int s = 0; s < d->nstreams - 1; ++s) { local_d.variant_msg[s] = (uint8_t *)malloc(d->data_size); if (!local_d.variant_msg[s]) { free(local_d.plaintext); @@ -232,27 +261,32 @@ static int aes192_ctr_hmac_sha256_sve_run(struct test *test, int cpu) { /* SVE 多流 HMAC (主消息 + 变体) */ { const uint8_t *keys[NSTREAM], *msgs[NSTREAM]; + const int n = (int)svcntw(); /* 与 kernel 内部推导一致 */ uint8_t variant_keys[NSTREAM][32]; - for (int s = 0; s < NSTREAM; ++s) { + for (int s = 0; s < n; ++s) { memcpy(variant_keys[s], d->hmac_key, 32); if (s > 0) variant_keys[s][0] ^= (uint8_t)s; keys[s] = variant_keys[s]; } msgs[0] = ciphertext; - for (int s = 1; s < NSTREAM; ++s) { + for (int s = 1; s < n; ++s) { for (size_t i = 0; i < d->data_size; ++i) variant_msg[(s-1)*d->data_size + i] = ciphertext[i] ^ (uint8_t)(s * 0x5A + (i & 0x0F)); msgs[s] = variant_msg + (s-1)*d->data_size; } - hmac_sha256_x8(mac, keys, 32, msgs, d->data_size); + if (hmac_sha256_x8(mac, keys, 32, msgs, d->data_size) != 0) { + log_error("%s: HMAC capacity overflow (data_size too large)", test->id); + report_fail_msg("%s: hmac assembly over capacity", test->id); + } } - for (int s = 0; s < NSTREAM; ++s) + for (int s = 0; s < d->nstreams; ++s) memcmp_or_fail(mac[s], d->golden_mac[s], 32, "HMAC-SHA256 digest mismatch (frozen replay)"); } else { /* re-roll: fresh plaintext + same-iteration golden (EVP encrypt + * SVE HMAC recomputed for this thread's data) */ memset_random(local_d.plaintext, d->data_size); - aes192_ctr_hmac_sha256_sve_compute_golden(&local_d); + if (aes192_ctr_hmac_sha256_sve_compute_golden(test, &local_d) != 0) + report_fail_msg("%s: hmac assembly over capacity", test->id); aes_ctr_encrypt(local_d.aes_key, local_d.aes_iv, local_d.plaintext, ciphertext, d->data_size); memcmp_or_fail(ciphertext, local_d.golden_ciphertext, d->data_size, "ciphertext mismatch (SVE)"); aes_ctr_decrypt(local_d.aes_key, local_d.aes_iv, ciphertext, decrypted, d->data_size); @@ -261,21 +295,25 @@ static int aes192_ctr_hmac_sha256_sve_run(struct test *test, int cpu) { /* SVE 多流 HMAC (主消息 + 变体) */ { const uint8_t *keys[NSTREAM], *msgs[NSTREAM]; + const int n = (int)svcntw(); /* 与 kernel 内部推导一致 */ uint8_t variant_keys[NSTREAM][32]; - for (int s = 0; s < NSTREAM; ++s) { + for (int s = 0; s < n; ++s) { memcpy(variant_keys[s], local_d.hmac_key, 32); if (s > 0) variant_keys[s][0] ^= (uint8_t)s; keys[s] = variant_keys[s]; } msgs[0] = ciphertext; - for (int s = 1; s < NSTREAM; ++s) { + for (int s = 1; s < n; ++s) { for (size_t i = 0; i < d->data_size; ++i) variant_msg[(s-1)*d->data_size + i] = ciphertext[i] ^ (uint8_t)(s * 0x5A + (i & 0x0F)); msgs[s] = variant_msg + (s-1)*d->data_size; } - hmac_sha256_x8(mac, keys, 32, msgs, d->data_size); + if (hmac_sha256_x8(mac, keys, 32, msgs, d->data_size) != 0) { + log_error("%s: HMAC capacity overflow (data_size too large)", test->id); + report_fail_msg("%s: hmac assembly over capacity", test->id); + } } - for (int s = 0; s < NSTREAM; ++s) + for (int s = 0; s < d->nstreams; ++s) memcmp_or_fail(mac[s], local_d.golden_mac[s], 32, "HMAC-SHA256 digest mismatch (SVE)"); } ++iter; @@ -283,7 +321,7 @@ static int aes192_ctr_hmac_sha256_sve_run(struct test *test, int cpu) { free(ciphertext); free(decrypted); free(variant_msg); - for (int s = 0; s < NSTREAM - 1; ++s) + for (int s = 0; s < d->nstreams - 1; ++s) free(local_d.variant_msg[s]); free(local_d.plaintext); free(local_d.golden_ciphertext); @@ -295,7 +333,7 @@ static int aes192_ctr_hmac_sha256_sve_cleanup(struct test *test) { if (d) { free(d->plaintext); free(d->golden_ciphertext); - for (int s = 0; s < NSTREAM - 1; ++s) + for (int s = 0; s < d->nstreams - 1; ++s) free(d->variant_msg[s]); } free(d); diff --git a/tests/cpu/sve/ipsec/ipsec_aes192_ctr_hmac_sha384_sve.cpp b/tests/cpu/sve/ipsec/ipsec_aes192_ctr_hmac_sha384_sve.cpp index 13284dbc0..17d2dd796 100644 --- a/tests/cpu/sve/ipsec/ipsec_aes192_ctr_hmac_sha384_sve.cpp +++ b/tests/cpu/sve/ipsec/ipsec_aes192_ctr_hmac_sha384_sve.cpp @@ -7,12 +7,17 @@ * @parblock * SVE port of ipsec_aes192_ctr_hmac_sha384_avx: AES/3DES encryption stays on OpenSSL EVP while * the HMAC-SHA384 MAC runs on the multi-stream SVE HMAC implementation - * (verified bit-for-bit against OpenSSL; see scripts/sve-sha-verify). - * Eight independent HMAC lanes run in parallel on SVE vector lanes: lane 0 + * (HMAC construction validated same-source in gem5-fi across 128..1024-bit against an + * independent scalar oracle and RFC 2202/4231 vectors; the underlying SHA + * kernels were verified bit-for-bit against OpenSSL in the original + * fixed-stream form — see scripts/sve-sha-verify and + * docs/research/progress-log.md). + * N = svcntd() (64-bit family) independent HMAC lanes run in parallel on + * SVE vector lanes (all-VL 128..1024-bit, one message per 64-bit lane): lane 0 * carries the original ciphertext message (MAC semantics identical to the - * original test), lanes 1-7 carry deterministic key-stream variant + * original test), lanes 1..N-1 carry deterministic key-stream variant * messages that fill the remaining lanes — same computation family, - * golden values precomputed for all 8. Ciphertext, MACs and decrypted + * golden values precomputed for all N. Ciphertext, MACs and decrypted * data are compared against golden values on every iteration to detect * silent data corruption. Iterations alternate between a re-rolled input * with a same-core recomputed golden and a frozen replay of the init-time @@ -28,19 +33,15 @@ #include "sandstone_ssl.h" #include "sve_hmac.h" #include "sdc_stress_config.h" -#include +#include #include #include -#ifndef HWCAP_SVE -#define HWCAP_SVE (1 << 22) -#endif - #define HMAC_KEY_SIZE_V (48) #define SHA384_DIGEST_SIZE (48) #define AES_KEY_SIZE_V (24) #define AES_IV_SIZE_V (16) -#define NSTREAM (4) +#define NSTREAM (SVE_SHA_STREAMS_MAX_64) /* All-VL: 数组按 1024-bit 上限定维; 运行时流数 = svcntd() */ struct aes192_ctr_hmac_sha384_sve_data { uint8_t aes_key[AES_KEY_SIZE_V]; @@ -49,20 +50,21 @@ struct aes192_ctr_hmac_sha384_sve_data { uint8_t *plaintext; uint8_t *golden_ciphertext; uint8_t golden_mac[NSTREAM][48]; - /* 7 条变体消息: 确定性派生(异或字节掩码), 填 SVE lane */ + /* N-1 条变体消息: 确定性派生(异或字节掩码), 填 SVE lane */ uint8_t *variant_msg[NSTREAM-1]; size_t data_size; + int nstreams; /* All-VL: init 期流数 = svl/8 (svcntd()), run 漂移守卫用 */ }; static size_t ipsec_data_size(struct test *test) { - int64_t v = sdc_knob_int_clamped(test, "data_size", 0, 1024, 16 << 20); + int64_t v = sdc_knob_int_clamped(test, "data_size", 0, 1024, 1830); if (v == 0) - sdc_env_knob_int_clamped("ipsec_aes192_ctr_hmac_sha384_sve", "data_size", &v, 1024, 16 << 20); + sdc_env_knob_int_clamped("ipsec_aes192_ctr_hmac_sha384_sve", "data_size", &v, 1024, 1830); if (v > 0) return (size_t)v; return (size_t)sdc_auto_linear_count(1, 4, 1.5, 600, thread_count(), - 1024, 16 << 20, test->id, "data_size"); + 1024, 1830, test->id, "data_size"); } static void aes_ctr_encrypt(const uint8_t *key, const uint8_t *iv, const uint8_t *in, uint8_t *out, size_t len) { @@ -84,54 +86,56 @@ static void aes_ctr_decrypt(const uint8_t *key, const uint8_t *iv, const uint8_t s_EVP_CIPHER_CTX_free(ctx); } -static void aes192_ctr_hmac_sha384_sve_compute_golden(struct aes192_ctr_hmac_sha384_sve_data *d) { +static int aes192_ctr_hmac_sha384_sve_compute_golden(struct test *test, struct aes192_ctr_hmac_sha384_sve_data *d) { aes_ctr_encrypt(d->aes_key, d->aes_iv, d->plaintext, d->golden_ciphertext, d->data_size); - /* SVE 多流 HMAC: lane0=密文(主语义), lane1..7=变体 */ + /* SVE 多流 HMAC: lane0=密文(主语义), lane1..N-1=变体 */ + const int n = (int)svcntd(); /* 与 kernel 内部推导一致 */ const uint8_t *keys[NSTREAM], *msgs[NSTREAM]; uint8_t variant_keys[NSTREAM][48]; - for (int s = 0; s < NSTREAM; ++s) { + for (int s = 0; s < n; ++s) { memcpy(variant_keys[s], d->hmac_key, 48); if (s > 0) variant_keys[s][0] ^= (uint8_t)s; /* 变体密钥: 确定性微扰 */ keys[s] = variant_keys[s]; } msgs[0] = d->golden_ciphertext; - for (int s = 1; s < NSTREAM; ++s) { + for (int s = 1; s < n; ++s) { /* 变体消息: 密文异或流掩码(确定性) */ for (size_t i = 0; i < d->data_size; ++i) d->variant_msg[s-1][i] = d->golden_ciphertext[i] ^ (uint8_t)(s * 0x5A + (i & 0x0F)); msgs[s] = d->variant_msg[s-1]; } - hmac_sha384_x4(d->golden_mac, keys, 48, msgs, d->data_size); + if (hmac_sha384_x4(d->golden_mac, keys, 48, msgs, d->data_size) != 0) { + log_error("%s: HMAC capacity overflow (data_size too large)", test->id); + return -1; + } + return 0; } static int aes192_ctr_hmac_sha384_sve_init(struct test *test) { if (s_EVP_CIPHER_CTX_new && s_EVP_EncryptInit_ex && s_EVP_DecryptInit_ex && s_EVP_CIPHER_CTX_new && s_EVP_aes_192_ctr()) { - unsigned long hwcap = getauxval(AT_HWCAP); - if ((hwcap & HWCAP_SVE) == 0) { - log_skip(CpuNotSupportedSkipCategory, - "to be implemented (placeholder): ARM SVE required for ipsec_aes192_ctr_hmac_sha384_sve"); - return EXIT_SKIP; - } - /* The multi-stream SVE HMAC kernels give one stream per SVE - * lane: 8x32-bit (sha1/224/256) needs svcntw()>=8 and - * 4x64-bit (384/512) needs svcntd()>=4 -- both mean VL>=256 - * bit. On shorter-VL SVE hardware (e.g. Neoverse N2, VL=128) - * lanes beyond VL are never written, so skip cleanly instead - * of comparing unwritten memory (target silicon: VL=256, see - * tests/cpu/sve/meson.build). */ - if (svcntw() < 8 || svcntd() < 4) { - log_skip(CpuNotSupportedSkipCategory, - "ARM SVE vector length %zu bytes < 32 required by the multi-stream HMAC kernels (svcntw=%zu svcntd=%zu)", - svcntb(), (size_t)svcntw(), (size_t)svcntd()); + /* All-VL Phase 2 P4: capability + 128..1024-bit range check in + * one framework-layer call (replaces the old lower-bound-only + * svcntw()>=8/svcntd()>=4 gate -- the 8.2 defect: at VL>=512 the + * fixed 8/4-wide W rows overflowed; the STREAMS_MAX-sized rows + + * this range gate close both ends). arm64_require_sve_vl returns + * the VL in BYTES. */ + const long svl = arm64_require_sve_vl(test, 128, 1024); + if (svl < 0) return EXIT_SKIP; + const int nstreams = (int)svl / 8; /* one stream per 64-bit lane (svcntd) */ + if (nstreams < 2 || nstreams > SVE_SHA_STREAMS_MAX_64) { /* [2, SVE_SHA_STREAMS_MAX_64], release-effective, NDEBUG-proof */ + log_error("%s: streams %d outside range — unit drift", test->id, nstreams); + return EXIT_FAILURE; } struct aes192_ctr_hmac_sha384_sve_data *d = (struct aes192_ctr_hmac_sha384_sve_data *)malloc(sizeof(*d)); if (!d) return EXIT_SKIP; + d->nstreams = nstreams; /* 工作集字节数: -O ipsec_aes192_ctr_hmac_sha384_sve.data_size=N / * SDC_IPSEC_AES192_CTR_HMAC_SHA384_SVE_DATA_SIZE 环境变量 / 自适应: - * 每线程明文+密文+golden 密文+解密缓冲+变体消息 ~= (3+NSTREAM-1)x, - * 占物理内存 60%, clamp [1024, 16M]。线性缓冲用 sdc_auto_linear_count(sqrt 缩放会欠缩放压力)。历史硬编码 1024。CTR 流式 + * 每线程明文+密文+golden 密文+解密缓冲+变体消息 ~= (3+nstreams-1)x, + * 占物理内存 60%, clamp [1024, 1830]——上限闭环 8.4: HMAC 组装 + * 缓冲 full[N][2048], 2048-128-17=1903B 安全边界。线性缓冲用 sdc_auto_linear_count(sqrt 缩放会欠缩放压力)。历史硬编码 1024。CTR 流式 * 无需分组对齐, 统一 16B 对齐无害, 对齐后 < 1024 取 1024。 */ d->data_size = ipsec_data_size(test); d->data_size &= ~(size_t)15; @@ -145,7 +149,7 @@ static int aes192_ctr_hmac_sha384_sve_init(struct test *test) { free(d); return EXIT_SKIP; } - for (int s = 0; s < NSTREAM - 1; ++s) { + for (int s = 0; s < d->nstreams - 1; ++s) { d->variant_msg[s] = (uint8_t *)malloc(d->data_size); if (!d->variant_msg[s]) { free(d->plaintext); @@ -160,7 +164,20 @@ static int aes192_ctr_hmac_sha384_sve_init(struct test *test) { memset_random(d->aes_iv, AES_IV_SIZE_V); memset_random(d->hmac_key, 48); memset_random(d->plaintext, d->data_size); - aes192_ctr_hmac_sha384_sve_compute_golden(d); + if (aes192_ctr_hmac_sha384_sve_compute_golden(test, d) != 0) { + /* P2a (review Minor): init-side failure propagates via the + * return value so the framework reports a clean Failed + * (sandstone_run.cpp: ret > 0 -> TestResult::Failed). + * report_fail_msg is a noreturn thread-cancel that only + * belongs in the test_run worker context — in test_init it + * would surface as Skipped("Unexpected OS error"). */ + free(d->plaintext); + free(d->golden_ciphertext); + for (int s = 0; s < d->nstreams - 1; ++s) + free(d->variant_msg[s]); + free(d); + return EXIT_FAILURE; + } test->data = d; return EXIT_SUCCESS; } else { @@ -172,10 +189,22 @@ static int aes192_ctr_hmac_sha384_sve_init(struct test *test) { static int aes192_ctr_hmac_sha384_sve_run(struct test *test, int cpu) { (void)cpu; struct aes192_ctr_hmac_sha384_sve_data *d = (struct aes192_ctr_hmac_sha384_sve_data *)test->data; + /* All-VL policy #3: never compare goldens across an init/run VL + * mismatch. Drift means this deployment's per-thread VL differs from + * the init thread's (e.g. a VL-pinning launcher changing the + * thread-inheritance chain) — an environment mismatch, which skips + * with CpuNotSupported (same category as the init-side range gate in + * arm64_require_sve_vl) instead of polluting the SDC failure signal. */ + if ((int)svcntd() != d->nstreams) { + log_skip(CpuNotSupportedSkipCategory, + "%s: SVE VL drifted between init (%d streams) and run (%d streams) — environment mismatch, not SDC", + test->id, d->nstreams, (int)svcntd()); + return EXIT_SKIP; + } uint8_t *ciphertext = (uint8_t *)malloc(d->data_size); uint8_t *decrypted = (uint8_t *)malloc(d->data_size); uint8_t mac[NSTREAM][48]; - uint8_t *variant_msg = (uint8_t *)malloc((NSTREAM-1) * d->data_size); + uint8_t *variant_msg = (uint8_t *)malloc((d->nstreams-1) * d->data_size); /* R2b (B2-003): per-thread plaintext + golden, re-rolled every * iteration (thread-safe: shared d is only read after init; the @@ -199,7 +228,7 @@ static int aes192_ctr_hmac_sha384_sve_run(struct test *test, int cpu) { "out of memory for per-thread buffers"); return EXIT_SKIP; } - for (int s = 0; s < NSTREAM - 1; ++s) { + for (int s = 0; s < d->nstreams - 1; ++s) { local_d.variant_msg[s] = (uint8_t *)malloc(d->data_size); if (!local_d.variant_msg[s]) { free(local_d.plaintext); @@ -232,27 +261,32 @@ static int aes192_ctr_hmac_sha384_sve_run(struct test *test, int cpu) { /* SVE 多流 HMAC (主消息 + 变体) */ { const uint8_t *keys[NSTREAM], *msgs[NSTREAM]; + const int n = (int)svcntd(); /* 与 kernel 内部推导一致 */ uint8_t variant_keys[NSTREAM][48]; - for (int s = 0; s < NSTREAM; ++s) { + for (int s = 0; s < n; ++s) { memcpy(variant_keys[s], d->hmac_key, 48); if (s > 0) variant_keys[s][0] ^= (uint8_t)s; keys[s] = variant_keys[s]; } msgs[0] = ciphertext; - for (int s = 1; s < NSTREAM; ++s) { + for (int s = 1; s < n; ++s) { for (size_t i = 0; i < d->data_size; ++i) variant_msg[(s-1)*d->data_size + i] = ciphertext[i] ^ (uint8_t)(s * 0x5A + (i & 0x0F)); msgs[s] = variant_msg + (s-1)*d->data_size; } - hmac_sha384_x4(mac, keys, 48, msgs, d->data_size); + if (hmac_sha384_x4(mac, keys, 48, msgs, d->data_size) != 0) { + log_error("%s: HMAC capacity overflow (data_size too large)", test->id); + report_fail_msg("%s: hmac assembly over capacity", test->id); + } } - for (int s = 0; s < NSTREAM; ++s) + for (int s = 0; s < d->nstreams; ++s) memcmp_or_fail(mac[s], d->golden_mac[s], 48, "HMAC-SHA384 digest mismatch (frozen replay)"); } else { /* re-roll: fresh plaintext + same-iteration golden (EVP encrypt + * SVE HMAC recomputed for this thread's data) */ memset_random(local_d.plaintext, d->data_size); - aes192_ctr_hmac_sha384_sve_compute_golden(&local_d); + if (aes192_ctr_hmac_sha384_sve_compute_golden(test, &local_d) != 0) + report_fail_msg("%s: hmac assembly over capacity", test->id); aes_ctr_encrypt(local_d.aes_key, local_d.aes_iv, local_d.plaintext, ciphertext, d->data_size); memcmp_or_fail(ciphertext, local_d.golden_ciphertext, d->data_size, "ciphertext mismatch (SVE)"); aes_ctr_decrypt(local_d.aes_key, local_d.aes_iv, ciphertext, decrypted, d->data_size); @@ -261,21 +295,25 @@ static int aes192_ctr_hmac_sha384_sve_run(struct test *test, int cpu) { /* SVE 多流 HMAC (主消息 + 变体) */ { const uint8_t *keys[NSTREAM], *msgs[NSTREAM]; + const int n = (int)svcntd(); /* 与 kernel 内部推导一致 */ uint8_t variant_keys[NSTREAM][48]; - for (int s = 0; s < NSTREAM; ++s) { + for (int s = 0; s < n; ++s) { memcpy(variant_keys[s], local_d.hmac_key, 48); if (s > 0) variant_keys[s][0] ^= (uint8_t)s; keys[s] = variant_keys[s]; } msgs[0] = ciphertext; - for (int s = 1; s < NSTREAM; ++s) { + for (int s = 1; s < n; ++s) { for (size_t i = 0; i < d->data_size; ++i) variant_msg[(s-1)*d->data_size + i] = ciphertext[i] ^ (uint8_t)(s * 0x5A + (i & 0x0F)); msgs[s] = variant_msg + (s-1)*d->data_size; } - hmac_sha384_x4(mac, keys, 48, msgs, d->data_size); + if (hmac_sha384_x4(mac, keys, 48, msgs, d->data_size) != 0) { + log_error("%s: HMAC capacity overflow (data_size too large)", test->id); + report_fail_msg("%s: hmac assembly over capacity", test->id); + } } - for (int s = 0; s < NSTREAM; ++s) + for (int s = 0; s < d->nstreams; ++s) memcmp_or_fail(mac[s], local_d.golden_mac[s], 48, "HMAC-SHA384 digest mismatch (SVE)"); } ++iter; @@ -283,7 +321,7 @@ static int aes192_ctr_hmac_sha384_sve_run(struct test *test, int cpu) { free(ciphertext); free(decrypted); free(variant_msg); - for (int s = 0; s < NSTREAM - 1; ++s) + for (int s = 0; s < d->nstreams - 1; ++s) free(local_d.variant_msg[s]); free(local_d.plaintext); free(local_d.golden_ciphertext); @@ -295,7 +333,7 @@ static int aes192_ctr_hmac_sha384_sve_cleanup(struct test *test) { if (d) { free(d->plaintext); free(d->golden_ciphertext); - for (int s = 0; s < NSTREAM - 1; ++s) + for (int s = 0; s < d->nstreams - 1; ++s) free(d->variant_msg[s]); } free(d); diff --git a/tests/cpu/sve/ipsec/ipsec_aes192_ctr_hmac_sha512_sve.cpp b/tests/cpu/sve/ipsec/ipsec_aes192_ctr_hmac_sha512_sve.cpp index ed5a93408..0f2a77388 100644 --- a/tests/cpu/sve/ipsec/ipsec_aes192_ctr_hmac_sha512_sve.cpp +++ b/tests/cpu/sve/ipsec/ipsec_aes192_ctr_hmac_sha512_sve.cpp @@ -7,12 +7,17 @@ * @parblock * SVE port of ipsec_aes192_ctr_hmac_sha512_avx: AES/3DES encryption stays on OpenSSL EVP while * the HMAC-SHA512 MAC runs on the multi-stream SVE HMAC implementation - * (verified bit-for-bit against OpenSSL; see scripts/sve-sha-verify). - * Eight independent HMAC lanes run in parallel on SVE vector lanes: lane 0 + * (HMAC construction validated same-source in gem5-fi across 128..1024-bit against an + * independent scalar oracle and RFC 2202/4231 vectors; the underlying SHA + * kernels were verified bit-for-bit against OpenSSL in the original + * fixed-stream form — see scripts/sve-sha-verify and + * docs/research/progress-log.md). + * N = svcntd() (64-bit family) independent HMAC lanes run in parallel on + * SVE vector lanes (all-VL 128..1024-bit, one message per 64-bit lane): lane 0 * carries the original ciphertext message (MAC semantics identical to the - * original test), lanes 1-7 carry deterministic key-stream variant + * original test), lanes 1..N-1 carry deterministic key-stream variant * messages that fill the remaining lanes — same computation family, - * golden values precomputed for all 8. Ciphertext, MACs and decrypted + * golden values precomputed for all N. Ciphertext, MACs and decrypted * data are compared against golden values on every iteration to detect * silent data corruption. Iterations alternate between a re-rolled input * with a same-core recomputed golden and a frozen replay of the init-time @@ -28,19 +33,15 @@ #include "sandstone_ssl.h" #include "sve_hmac.h" #include "sdc_stress_config.h" -#include +#include #include #include -#ifndef HWCAP_SVE -#define HWCAP_SVE (1 << 22) -#endif - #define HMAC_KEY_SIZE_V (64) #define SHA512_DIGEST_SIZE (64) #define AES_KEY_SIZE_V (24) #define AES_IV_SIZE_V (16) -#define NSTREAM (4) +#define NSTREAM (SVE_SHA_STREAMS_MAX_64) /* All-VL: 数组按 1024-bit 上限定维; 运行时流数 = svcntd() */ struct aes192_ctr_hmac_sha512_sve_data { uint8_t aes_key[AES_KEY_SIZE_V]; @@ -49,20 +50,21 @@ struct aes192_ctr_hmac_sha512_sve_data { uint8_t *plaintext; uint8_t *golden_ciphertext; uint8_t golden_mac[NSTREAM][64]; - /* 7 条变体消息: 确定性派生(异或字节掩码), 填 SVE lane */ + /* N-1 条变体消息: 确定性派生(异或字节掩码), 填 SVE lane */ uint8_t *variant_msg[NSTREAM-1]; size_t data_size; + int nstreams; /* All-VL: init 期流数 = svl/8 (svcntd()), run 漂移守卫用 */ }; static size_t ipsec_data_size(struct test *test) { - int64_t v = sdc_knob_int_clamped(test, "data_size", 0, 1024, 16 << 20); + int64_t v = sdc_knob_int_clamped(test, "data_size", 0, 1024, 1830); if (v == 0) - sdc_env_knob_int_clamped("ipsec_aes192_ctr_hmac_sha512_sve", "data_size", &v, 1024, 16 << 20); + sdc_env_knob_int_clamped("ipsec_aes192_ctr_hmac_sha512_sve", "data_size", &v, 1024, 1830); if (v > 0) return (size_t)v; return (size_t)sdc_auto_linear_count(1, 4, 1.5, 600, thread_count(), - 1024, 16 << 20, test->id, "data_size"); + 1024, 1830, test->id, "data_size"); } static void aes_ctr_encrypt(const uint8_t *key, const uint8_t *iv, const uint8_t *in, uint8_t *out, size_t len) { @@ -84,54 +86,56 @@ static void aes_ctr_decrypt(const uint8_t *key, const uint8_t *iv, const uint8_t s_EVP_CIPHER_CTX_free(ctx); } -static void aes192_ctr_hmac_sha512_sve_compute_golden(struct aes192_ctr_hmac_sha512_sve_data *d) { +static int aes192_ctr_hmac_sha512_sve_compute_golden(struct test *test, struct aes192_ctr_hmac_sha512_sve_data *d) { aes_ctr_encrypt(d->aes_key, d->aes_iv, d->plaintext, d->golden_ciphertext, d->data_size); - /* SVE 多流 HMAC: lane0=密文(主语义), lane1..7=变体 */ + /* SVE 多流 HMAC: lane0=密文(主语义), lane1..N-1=变体 */ + const int n = (int)svcntd(); /* 与 kernel 内部推导一致 */ const uint8_t *keys[NSTREAM], *msgs[NSTREAM]; uint8_t variant_keys[NSTREAM][64]; - for (int s = 0; s < NSTREAM; ++s) { + for (int s = 0; s < n; ++s) { memcpy(variant_keys[s], d->hmac_key, 64); if (s > 0) variant_keys[s][0] ^= (uint8_t)s; /* 变体密钥: 确定性微扰 */ keys[s] = variant_keys[s]; } msgs[0] = d->golden_ciphertext; - for (int s = 1; s < NSTREAM; ++s) { + for (int s = 1; s < n; ++s) { /* 变体消息: 密文异或流掩码(确定性) */ for (size_t i = 0; i < d->data_size; ++i) d->variant_msg[s-1][i] = d->golden_ciphertext[i] ^ (uint8_t)(s * 0x5A + (i & 0x0F)); msgs[s] = d->variant_msg[s-1]; } - hmac_sha512_x4(d->golden_mac, keys, 64, msgs, d->data_size); + if (hmac_sha512_x4(d->golden_mac, keys, 64, msgs, d->data_size) != 0) { + log_error("%s: HMAC capacity overflow (data_size too large)", test->id); + return -1; + } + return 0; } static int aes192_ctr_hmac_sha512_sve_init(struct test *test) { if (s_EVP_CIPHER_CTX_new && s_EVP_EncryptInit_ex && s_EVP_DecryptInit_ex && s_EVP_CIPHER_CTX_new && s_EVP_aes_192_ctr()) { - unsigned long hwcap = getauxval(AT_HWCAP); - if ((hwcap & HWCAP_SVE) == 0) { - log_skip(CpuNotSupportedSkipCategory, - "to be implemented (placeholder): ARM SVE required for ipsec_aes192_ctr_hmac_sha512_sve"); - return EXIT_SKIP; - } - /* The multi-stream SVE HMAC kernels give one stream per SVE - * lane: 8x32-bit (sha1/224/256) needs svcntw()>=8 and - * 4x64-bit (384/512) needs svcntd()>=4 -- both mean VL>=256 - * bit. On shorter-VL SVE hardware (e.g. Neoverse N2, VL=128) - * lanes beyond VL are never written, so skip cleanly instead - * of comparing unwritten memory (target silicon: VL=256, see - * tests/cpu/sve/meson.build). */ - if (svcntw() < 8 || svcntd() < 4) { - log_skip(CpuNotSupportedSkipCategory, - "ARM SVE vector length %zu bytes < 32 required by the multi-stream HMAC kernels (svcntw=%zu svcntd=%zu)", - svcntb(), (size_t)svcntw(), (size_t)svcntd()); + /* All-VL Phase 2 P4: capability + 128..1024-bit range check in + * one framework-layer call (replaces the old lower-bound-only + * svcntw()>=8/svcntd()>=4 gate -- the 8.2 defect: at VL>=512 the + * fixed 8/4-wide W rows overflowed; the STREAMS_MAX-sized rows + + * this range gate close both ends). arm64_require_sve_vl returns + * the VL in BYTES. */ + const long svl = arm64_require_sve_vl(test, 128, 1024); + if (svl < 0) return EXIT_SKIP; + const int nstreams = (int)svl / 8; /* one stream per 64-bit lane (svcntd) */ + if (nstreams < 2 || nstreams > SVE_SHA_STREAMS_MAX_64) { /* [2, SVE_SHA_STREAMS_MAX_64], release-effective, NDEBUG-proof */ + log_error("%s: streams %d outside range — unit drift", test->id, nstreams); + return EXIT_FAILURE; } struct aes192_ctr_hmac_sha512_sve_data *d = (struct aes192_ctr_hmac_sha512_sve_data *)malloc(sizeof(*d)); if (!d) return EXIT_SKIP; + d->nstreams = nstreams; /* 工作集字节数: -O ipsec_aes192_ctr_hmac_sha512_sve.data_size=N / * SDC_IPSEC_AES192_CTR_HMAC_SHA512_SVE_DATA_SIZE 环境变量 / 自适应: - * 每线程明文+密文+golden 密文+解密缓冲+变体消息 ~= (3+NSTREAM-1)x, - * 占物理内存 60%, clamp [1024, 16M]。线性缓冲用 sdc_auto_linear_count(sqrt 缩放会欠缩放压力)。历史硬编码 1024。CTR 流式 + * 每线程明文+密文+golden 密文+解密缓冲+变体消息 ~= (3+nstreams-1)x, + * 占物理内存 60%, clamp [1024, 1830]——上限闭环 8.4: HMAC 组装 + * 缓冲 full[N][2048], 2048-128-17=1903B 安全边界。线性缓冲用 sdc_auto_linear_count(sqrt 缩放会欠缩放压力)。历史硬编码 1024。CTR 流式 * 无需分组对齐, 统一 16B 对齐无害, 对齐后 < 1024 取 1024。 */ d->data_size = ipsec_data_size(test); d->data_size &= ~(size_t)15; @@ -145,7 +149,7 @@ static int aes192_ctr_hmac_sha512_sve_init(struct test *test) { free(d); return EXIT_SKIP; } - for (int s = 0; s < NSTREAM - 1; ++s) { + for (int s = 0; s < d->nstreams - 1; ++s) { d->variant_msg[s] = (uint8_t *)malloc(d->data_size); if (!d->variant_msg[s]) { free(d->plaintext); @@ -160,7 +164,20 @@ static int aes192_ctr_hmac_sha512_sve_init(struct test *test) { memset_random(d->aes_iv, AES_IV_SIZE_V); memset_random(d->hmac_key, 64); memset_random(d->plaintext, d->data_size); - aes192_ctr_hmac_sha512_sve_compute_golden(d); + if (aes192_ctr_hmac_sha512_sve_compute_golden(test, d) != 0) { + /* P2a (review Minor): init-side failure propagates via the + * return value so the framework reports a clean Failed + * (sandstone_run.cpp: ret > 0 -> TestResult::Failed). + * report_fail_msg is a noreturn thread-cancel that only + * belongs in the test_run worker context — in test_init it + * would surface as Skipped("Unexpected OS error"). */ + free(d->plaintext); + free(d->golden_ciphertext); + for (int s = 0; s < d->nstreams - 1; ++s) + free(d->variant_msg[s]); + free(d); + return EXIT_FAILURE; + } test->data = d; return EXIT_SUCCESS; } else { @@ -172,10 +189,22 @@ static int aes192_ctr_hmac_sha512_sve_init(struct test *test) { static int aes192_ctr_hmac_sha512_sve_run(struct test *test, int cpu) { (void)cpu; struct aes192_ctr_hmac_sha512_sve_data *d = (struct aes192_ctr_hmac_sha512_sve_data *)test->data; + /* All-VL policy #3: never compare goldens across an init/run VL + * mismatch. Drift means this deployment's per-thread VL differs from + * the init thread's (e.g. a VL-pinning launcher changing the + * thread-inheritance chain) — an environment mismatch, which skips + * with CpuNotSupported (same category as the init-side range gate in + * arm64_require_sve_vl) instead of polluting the SDC failure signal. */ + if ((int)svcntd() != d->nstreams) { + log_skip(CpuNotSupportedSkipCategory, + "%s: SVE VL drifted between init (%d streams) and run (%d streams) — environment mismatch, not SDC", + test->id, d->nstreams, (int)svcntd()); + return EXIT_SKIP; + } uint8_t *ciphertext = (uint8_t *)malloc(d->data_size); uint8_t *decrypted = (uint8_t *)malloc(d->data_size); uint8_t mac[NSTREAM][64]; - uint8_t *variant_msg = (uint8_t *)malloc((NSTREAM-1) * d->data_size); + uint8_t *variant_msg = (uint8_t *)malloc((d->nstreams-1) * d->data_size); /* R2b (B2-003): per-thread plaintext + golden, re-rolled every * iteration (thread-safe: shared d is only read after init; the @@ -199,7 +228,7 @@ static int aes192_ctr_hmac_sha512_sve_run(struct test *test, int cpu) { "out of memory for per-thread buffers"); return EXIT_SKIP; } - for (int s = 0; s < NSTREAM - 1; ++s) { + for (int s = 0; s < d->nstreams - 1; ++s) { local_d.variant_msg[s] = (uint8_t *)malloc(d->data_size); if (!local_d.variant_msg[s]) { free(local_d.plaintext); @@ -232,27 +261,32 @@ static int aes192_ctr_hmac_sha512_sve_run(struct test *test, int cpu) { /* SVE 多流 HMAC (主消息 + 变体) */ { const uint8_t *keys[NSTREAM], *msgs[NSTREAM]; + const int n = (int)svcntd(); /* 与 kernel 内部推导一致 */ uint8_t variant_keys[NSTREAM][64]; - for (int s = 0; s < NSTREAM; ++s) { + for (int s = 0; s < n; ++s) { memcpy(variant_keys[s], d->hmac_key, 64); if (s > 0) variant_keys[s][0] ^= (uint8_t)s; keys[s] = variant_keys[s]; } msgs[0] = ciphertext; - for (int s = 1; s < NSTREAM; ++s) { + for (int s = 1; s < n; ++s) { for (size_t i = 0; i < d->data_size; ++i) variant_msg[(s-1)*d->data_size + i] = ciphertext[i] ^ (uint8_t)(s * 0x5A + (i & 0x0F)); msgs[s] = variant_msg + (s-1)*d->data_size; } - hmac_sha512_x4(mac, keys, 64, msgs, d->data_size); + if (hmac_sha512_x4(mac, keys, 64, msgs, d->data_size) != 0) { + log_error("%s: HMAC capacity overflow (data_size too large)", test->id); + report_fail_msg("%s: hmac assembly over capacity", test->id); + } } - for (int s = 0; s < NSTREAM; ++s) + for (int s = 0; s < d->nstreams; ++s) memcmp_or_fail(mac[s], d->golden_mac[s], 64, "HMAC-SHA512 digest mismatch (frozen replay)"); } else { /* re-roll: fresh plaintext + same-iteration golden (EVP encrypt + * SVE HMAC recomputed for this thread's data) */ memset_random(local_d.plaintext, d->data_size); - aes192_ctr_hmac_sha512_sve_compute_golden(&local_d); + if (aes192_ctr_hmac_sha512_sve_compute_golden(test, &local_d) != 0) + report_fail_msg("%s: hmac assembly over capacity", test->id); aes_ctr_encrypt(local_d.aes_key, local_d.aes_iv, local_d.plaintext, ciphertext, d->data_size); memcmp_or_fail(ciphertext, local_d.golden_ciphertext, d->data_size, "ciphertext mismatch (SVE)"); aes_ctr_decrypt(local_d.aes_key, local_d.aes_iv, ciphertext, decrypted, d->data_size); @@ -261,21 +295,25 @@ static int aes192_ctr_hmac_sha512_sve_run(struct test *test, int cpu) { /* SVE 多流 HMAC (主消息 + 变体) */ { const uint8_t *keys[NSTREAM], *msgs[NSTREAM]; + const int n = (int)svcntd(); /* 与 kernel 内部推导一致 */ uint8_t variant_keys[NSTREAM][64]; - for (int s = 0; s < NSTREAM; ++s) { + for (int s = 0; s < n; ++s) { memcpy(variant_keys[s], local_d.hmac_key, 64); if (s > 0) variant_keys[s][0] ^= (uint8_t)s; keys[s] = variant_keys[s]; } msgs[0] = ciphertext; - for (int s = 1; s < NSTREAM; ++s) { + for (int s = 1; s < n; ++s) { for (size_t i = 0; i < d->data_size; ++i) variant_msg[(s-1)*d->data_size + i] = ciphertext[i] ^ (uint8_t)(s * 0x5A + (i & 0x0F)); msgs[s] = variant_msg + (s-1)*d->data_size; } - hmac_sha512_x4(mac, keys, 64, msgs, d->data_size); + if (hmac_sha512_x4(mac, keys, 64, msgs, d->data_size) != 0) { + log_error("%s: HMAC capacity overflow (data_size too large)", test->id); + report_fail_msg("%s: hmac assembly over capacity", test->id); + } } - for (int s = 0; s < NSTREAM; ++s) + for (int s = 0; s < d->nstreams; ++s) memcmp_or_fail(mac[s], local_d.golden_mac[s], 64, "HMAC-SHA512 digest mismatch (SVE)"); } ++iter; @@ -283,7 +321,7 @@ static int aes192_ctr_hmac_sha512_sve_run(struct test *test, int cpu) { free(ciphertext); free(decrypted); free(variant_msg); - for (int s = 0; s < NSTREAM - 1; ++s) + for (int s = 0; s < d->nstreams - 1; ++s) free(local_d.variant_msg[s]); free(local_d.plaintext); free(local_d.golden_ciphertext); @@ -295,7 +333,7 @@ static int aes192_ctr_hmac_sha512_sve_cleanup(struct test *test) { if (d) { free(d->plaintext); free(d->golden_ciphertext); - for (int s = 0; s < NSTREAM - 1; ++s) + for (int s = 0; s < d->nstreams - 1; ++s) free(d->variant_msg[s]); } free(d); diff --git a/tests/cpu/sve/ipsec/ipsec_aes256_cbc_hmac_sha1_sve.cpp b/tests/cpu/sve/ipsec/ipsec_aes256_cbc_hmac_sha1_sve.cpp index afa5b1b4e..fa90cbc91 100644 --- a/tests/cpu/sve/ipsec/ipsec_aes256_cbc_hmac_sha1_sve.cpp +++ b/tests/cpu/sve/ipsec/ipsec_aes256_cbc_hmac_sha1_sve.cpp @@ -7,12 +7,17 @@ * @parblock * SVE port of ipsec_aes256_cbc_hmac_sha1_avx: AES/3DES encryption stays on OpenSSL EVP while * the HMAC-SHA1-96 MAC runs on the multi-stream SVE HMAC implementation - * (verified bit-for-bit against OpenSSL; see scripts/sve-sha-verify). - * Eight independent HMAC lanes run in parallel on SVE vector lanes: lane 0 + * (HMAC construction validated same-source in gem5-fi across 128..1024-bit against an + * independent scalar oracle and RFC 2202/4231 vectors; the underlying SHA + * kernels were verified bit-for-bit against OpenSSL in the original + * fixed-stream form — see scripts/sve-sha-verify and + * docs/research/progress-log.md). + * N = svcntw() (32-bit family) independent HMAC lanes run in parallel on + * SVE vector lanes (all-VL 128..1024-bit, one message per 32-bit lane): lane 0 * carries the original ciphertext message (MAC semantics identical to the - * original test), lanes 1-7 carry deterministic key-stream variant + * original test), lanes 1..N-1 carry deterministic key-stream variant * messages that fill the remaining lanes — same computation family, - * golden values precomputed for all 8. Ciphertext, MACs and decrypted + * golden values precomputed for all N. Ciphertext, MACs and decrypted * data are compared against golden values on every iteration to detect * silent data corruption. Iterations alternate between a re-rolled input * with a same-core recomputed golden and a frozen replay of the init-time @@ -28,19 +33,15 @@ #include "sandstone_ssl.h" #include "sve_hmac.h" #include "sdc_stress_config.h" -#include +#include #include #include -#ifndef HWCAP_SVE -#define HWCAP_SVE (1 << 22) -#endif - #define HMAC_KEY_SIZE_V (20) #define HMAC_96_DIGEST_SIZE (12) #define AES_KEY_SIZE_V (32) #define AES_IV_SIZE_V (16) -#define NSTREAM (8) +#define NSTREAM (SVE_SHA_STREAMS_MAX_32) /* All-VL: 数组按 1024-bit 上限定维; 运行时流数 = svcntw() */ struct aes256_cbc_hmac_sha1_sve_data { uint8_t aes_key[AES_KEY_SIZE_V]; @@ -49,20 +50,21 @@ struct aes256_cbc_hmac_sha1_sve_data { uint8_t *plaintext; uint8_t *golden_ciphertext; uint8_t golden_mac[NSTREAM][20]; /* 内部全摘要 20B, 比较取前 12 */ - /* 7 条变体消息: 确定性派生(异或字节掩码), 填 SVE lane */ + /* N-1 条变体消息: 确定性派生(异或字节掩码), 填 SVE lane */ uint8_t *variant_msg[NSTREAM-1]; size_t data_size; + int nstreams; /* All-VL: init 期流数 = svl/4 (svcntw()), run 漂移守卫用 */ }; static size_t ipsec_data_size(struct test *test) { - int64_t v = sdc_knob_int_clamped(test, "data_size", 0, 1024, 16 << 20); + int64_t v = sdc_knob_int_clamped(test, "data_size", 0, 1024, 1900); if (v == 0) - sdc_env_knob_int_clamped("ipsec_aes256_cbc_hmac_sha1_sve", "data_size", &v, 1024, 16 << 20); + sdc_env_knob_int_clamped("ipsec_aes256_cbc_hmac_sha1_sve", "data_size", &v, 1024, 1900); if (v > 0) return (size_t)v; return (size_t)sdc_auto_linear_count(1, 4, 1.5, 600, thread_count(), - 1024, 16 << 20, test->id, "data_size"); + 1024, 1900, test->id, "data_size"); } static void aes_encrypt(const uint8_t *key, const uint8_t *iv, const uint8_t *in, uint8_t *out, size_t len) { @@ -86,54 +88,56 @@ static void aes_decrypt(const uint8_t *key, const uint8_t *iv, const uint8_t *in s_EVP_CIPHER_CTX_free(ctx); } -static void aes256_cbc_hmac_sha1_sve_compute_golden(struct aes256_cbc_hmac_sha1_sve_data *d) { +static int aes256_cbc_hmac_sha1_sve_compute_golden(struct test *test, struct aes256_cbc_hmac_sha1_sve_data *d) { aes_encrypt(d->aes_key, d->aes_iv, d->plaintext, d->golden_ciphertext, d->data_size); - /* SVE 多流 HMAC: lane0=密文(主语义), lane1..7=变体 */ + /* SVE 多流 HMAC: lane0=密文(主语义), lane1..N-1=变体 */ + const int n = (int)svcntw(); /* 与 kernel 内部推导一致 */ const uint8_t *keys[NSTREAM], *msgs[NSTREAM]; uint8_t variant_keys[NSTREAM][20]; - for (int s = 0; s < NSTREAM; ++s) { + for (int s = 0; s < n; ++s) { memcpy(variant_keys[s], d->hmac_key, 20); if (s > 0) variant_keys[s][0] ^= (uint8_t)s; /* 变体密钥: 确定性微扰 */ keys[s] = variant_keys[s]; } msgs[0] = d->golden_ciphertext; - for (int s = 1; s < NSTREAM; ++s) { + for (int s = 1; s < n; ++s) { /* 变体消息: 密文异或流掩码(确定性) */ for (size_t i = 0; i < d->data_size; ++i) d->variant_msg[s-1][i] = d->golden_ciphertext[i] ^ (uint8_t)(s * 0x5A + (i & 0x0F)); msgs[s] = d->variant_msg[s-1]; } - hmac_sha1_x8(d->golden_mac, keys, 20, msgs, d->data_size); + if (hmac_sha1_x8(d->golden_mac, keys, 20, msgs, d->data_size) != 0) { + log_error("%s: HMAC capacity overflow (data_size too large)", test->id); + return -1; + } + return 0; } static int aes256_cbc_hmac_sha1_sve_init(struct test *test) { if (s_EVP_CIPHER_CTX_new && s_EVP_EncryptInit_ex && s_EVP_DecryptInit_ex && s_EVP_CIPHER_CTX_new && s_EVP_aes_256_cbc()) { - unsigned long hwcap = getauxval(AT_HWCAP); - if ((hwcap & HWCAP_SVE) == 0) { - log_skip(CpuNotSupportedSkipCategory, - "to be implemented (placeholder): ARM SVE required for ipsec_aes256_cbc_hmac_sha1_sve"); - return EXIT_SKIP; - } - /* The multi-stream SVE HMAC kernels give one stream per SVE - * lane: 8x32-bit (sha1/224/256) needs svcntw()>=8 and - * 4x64-bit (384/512) needs svcntd()>=4 -- both mean VL>=256 - * bit. On shorter-VL SVE hardware (e.g. Neoverse N2, VL=128) - * lanes beyond VL are never written, so skip cleanly instead - * of comparing unwritten memory (target silicon: VL=256, see - * tests/cpu/sve/meson.build). */ - if (svcntw() < 8 || svcntd() < 4) { - log_skip(CpuNotSupportedSkipCategory, - "ARM SVE vector length %zu bytes < 32 required by the multi-stream HMAC kernels (svcntw=%zu svcntd=%zu)", - svcntb(), (size_t)svcntw(), (size_t)svcntd()); + /* All-VL Phase 2 P4: capability + 128..1024-bit range check in + * one framework-layer call (replaces the old lower-bound-only + * svcntw()>=8/svcntd()>=4 gate -- the 8.2 defect: at VL>=512 the + * fixed 8/4-wide W rows overflowed; the STREAMS_MAX-sized rows + + * this range gate close both ends). arm64_require_sve_vl returns + * the VL in BYTES. */ + const long svl = arm64_require_sve_vl(test, 128, 1024); + if (svl < 0) return EXIT_SKIP; + const int nstreams = (int)svl / 4; /* one stream per 32-bit lane (svcntw) */ + if (nstreams < 4 || nstreams > SVE_SHA_STREAMS_MAX_32) { /* [4, SVE_SHA_STREAMS_MAX_32], release-effective, NDEBUG-proof */ + log_error("%s: streams %d outside range — unit drift", test->id, nstreams); + return EXIT_FAILURE; } struct aes256_cbc_hmac_sha1_sve_data *d = (struct aes256_cbc_hmac_sha1_sve_data *)malloc(sizeof(*d)); if (!d) return EXIT_SKIP; + d->nstreams = nstreams; /* 工作集字节数: -O ipsec_aes256_cbc_hmac_sha1_sve.data_size=N / * SDC_IPSEC_AES256_CBC_HMAC_SHA1_SVE_DATA_SIZE 环境变量 / 自适应: - * 每线程明文+密文+golden 密文+解密缓冲+变体消息 ~= (3+NSTREAM-1)x, - * 占物理内存 60%, clamp [1024, 16M]。线性缓冲用 sdc_auto_linear_count(sqrt 缩放会欠缩放压力)。历史硬编码 1024。CBC 无填充 + * 每线程明文+密文+golden 密文+解密缓冲+变体消息 ~= (3+nstreams-1)x, + * 占物理内存 60%, clamp [1024, 1900]——上限闭环 8.4: HMAC 组装 + * 缓冲 full[N][2048], 2048-64-9=1975B 安全边界。线性缓冲用 sdc_auto_linear_count(sqrt 缩放会欠缩放压力)。历史硬编码 1024。CBC 无填充 * 场景尺寸必须对齐到 AES 分组 16B, 对齐后 < 1024 取 1024。 */ d->data_size = ipsec_data_size(test); d->data_size &= ~(size_t)15; @@ -147,7 +151,7 @@ static int aes256_cbc_hmac_sha1_sve_init(struct test *test) { free(d); return EXIT_SKIP; } - for (int s = 0; s < NSTREAM - 1; ++s) { + for (int s = 0; s < d->nstreams - 1; ++s) { d->variant_msg[s] = (uint8_t *)malloc(d->data_size); if (!d->variant_msg[s]) { free(d->plaintext); @@ -162,7 +166,20 @@ static int aes256_cbc_hmac_sha1_sve_init(struct test *test) { memset_random(d->aes_iv, AES_IV_SIZE_V); memset_random(d->hmac_key, 20); memset_random(d->plaintext, d->data_size); - aes256_cbc_hmac_sha1_sve_compute_golden(d); + if (aes256_cbc_hmac_sha1_sve_compute_golden(test, d) != 0) { + /* P2a (review Minor): init-side failure propagates via the + * return value so the framework reports a clean Failed + * (sandstone_run.cpp: ret > 0 -> TestResult::Failed). + * report_fail_msg is a noreturn thread-cancel that only + * belongs in the test_run worker context — in test_init it + * would surface as Skipped("Unexpected OS error"). */ + free(d->plaintext); + free(d->golden_ciphertext); + for (int s = 0; s < d->nstreams - 1; ++s) + free(d->variant_msg[s]); + free(d); + return EXIT_FAILURE; + } test->data = d; return EXIT_SUCCESS; } else { @@ -174,10 +191,22 @@ static int aes256_cbc_hmac_sha1_sve_init(struct test *test) { static int aes256_cbc_hmac_sha1_sve_run(struct test *test, int cpu) { (void)cpu; struct aes256_cbc_hmac_sha1_sve_data *d = (struct aes256_cbc_hmac_sha1_sve_data *)test->data; + /* All-VL policy #3: never compare goldens across an init/run VL + * mismatch. Drift means this deployment's per-thread VL differs from + * the init thread's (e.g. a VL-pinning launcher changing the + * thread-inheritance chain) — an environment mismatch, which skips + * with CpuNotSupported (same category as the init-side range gate in + * arm64_require_sve_vl) instead of polluting the SDC failure signal. */ + if ((int)svcntw() != d->nstreams) { + log_skip(CpuNotSupportedSkipCategory, + "%s: SVE VL drifted between init (%d streams) and run (%d streams) — environment mismatch, not SDC", + test->id, d->nstreams, (int)svcntw()); + return EXIT_SKIP; + } uint8_t *ciphertext = (uint8_t *)malloc(d->data_size); uint8_t *decrypted = (uint8_t *)malloc(d->data_size); uint8_t mac[NSTREAM][20]; - uint8_t *variant_msg = (uint8_t *)malloc((NSTREAM-1) * d->data_size); + uint8_t *variant_msg = (uint8_t *)malloc((d->nstreams-1) * d->data_size); /* R2b (B2-003): per-thread plaintext + golden, re-rolled every * iteration (thread-safe: shared d is only read after init; the @@ -201,7 +230,7 @@ static int aes256_cbc_hmac_sha1_sve_run(struct test *test, int cpu) { "out of memory for per-thread buffers"); return EXIT_SKIP; } - for (int s = 0; s < NSTREAM - 1; ++s) { + for (int s = 0; s < d->nstreams - 1; ++s) { local_d.variant_msg[s] = (uint8_t *)malloc(d->data_size); if (!local_d.variant_msg[s]) { free(local_d.plaintext); @@ -234,27 +263,32 @@ static int aes256_cbc_hmac_sha1_sve_run(struct test *test, int cpu) { /* SVE 多流 HMAC (主消息 + 变体) */ { const uint8_t *keys[NSTREAM], *msgs[NSTREAM]; + const int n = (int)svcntw(); /* 与 kernel 内部推导一致 */ uint8_t variant_keys[NSTREAM][20]; - for (int s = 0; s < NSTREAM; ++s) { + for (int s = 0; s < n; ++s) { memcpy(variant_keys[s], d->hmac_key, 20); if (s > 0) variant_keys[s][0] ^= (uint8_t)s; keys[s] = variant_keys[s]; } msgs[0] = ciphertext; - for (int s = 1; s < NSTREAM; ++s) { + for (int s = 1; s < n; ++s) { for (size_t i = 0; i < d->data_size; ++i) variant_msg[(s-1)*d->data_size + i] = ciphertext[i] ^ (uint8_t)(s * 0x5A + (i & 0x0F)); msgs[s] = variant_msg + (s-1)*d->data_size; } - hmac_sha1_x8(mac, keys, 20, msgs, d->data_size); + if (hmac_sha1_x8(mac, keys, 20, msgs, d->data_size) != 0) { + log_error("%s: HMAC capacity overflow (data_size too large)", test->id); + report_fail_msg("%s: hmac assembly over capacity", test->id); + } } - for (int s = 0; s < NSTREAM; ++s) + for (int s = 0; s < d->nstreams; ++s) memcmp_or_fail(mac[s], d->golden_mac[s], 12, "HMAC-SHA1-96 digest mismatch (frozen replay)"); } else { /* re-roll: fresh plaintext + same-iteration golden (EVP encrypt + * SVE HMAC recomputed for this thread's data) */ memset_random(local_d.plaintext, d->data_size); - aes256_cbc_hmac_sha1_sve_compute_golden(&local_d); + if (aes256_cbc_hmac_sha1_sve_compute_golden(test, &local_d) != 0) + report_fail_msg("%s: hmac assembly over capacity", test->id); aes_encrypt(local_d.aes_key, local_d.aes_iv, local_d.plaintext, ciphertext, d->data_size); memcmp_or_fail(ciphertext, local_d.golden_ciphertext, d->data_size, "ciphertext mismatch (SVE)"); aes_decrypt(local_d.aes_key, local_d.aes_iv, ciphertext, decrypted, d->data_size); @@ -263,21 +297,25 @@ static int aes256_cbc_hmac_sha1_sve_run(struct test *test, int cpu) { /* SVE 多流 HMAC (主消息 + 变体) */ { const uint8_t *keys[NSTREAM], *msgs[NSTREAM]; + const int n = (int)svcntw(); /* 与 kernel 内部推导一致 */ uint8_t variant_keys[NSTREAM][20]; - for (int s = 0; s < NSTREAM; ++s) { + for (int s = 0; s < n; ++s) { memcpy(variant_keys[s], local_d.hmac_key, 20); if (s > 0) variant_keys[s][0] ^= (uint8_t)s; keys[s] = variant_keys[s]; } msgs[0] = ciphertext; - for (int s = 1; s < NSTREAM; ++s) { + for (int s = 1; s < n; ++s) { for (size_t i = 0; i < d->data_size; ++i) variant_msg[(s-1)*d->data_size + i] = ciphertext[i] ^ (uint8_t)(s * 0x5A + (i & 0x0F)); msgs[s] = variant_msg + (s-1)*d->data_size; } - hmac_sha1_x8(mac, keys, 20, msgs, d->data_size); + if (hmac_sha1_x8(mac, keys, 20, msgs, d->data_size) != 0) { + log_error("%s: HMAC capacity overflow (data_size too large)", test->id); + report_fail_msg("%s: hmac assembly over capacity", test->id); + } } - for (int s = 0; s < NSTREAM; ++s) + for (int s = 0; s < d->nstreams; ++s) memcmp_or_fail(mac[s], local_d.golden_mac[s], 12, "HMAC-SHA1-96 digest mismatch (SVE)"); } ++iter; @@ -285,7 +323,7 @@ static int aes256_cbc_hmac_sha1_sve_run(struct test *test, int cpu) { free(ciphertext); free(decrypted); free(variant_msg); - for (int s = 0; s < NSTREAM - 1; ++s) + for (int s = 0; s < d->nstreams - 1; ++s) free(local_d.variant_msg[s]); free(local_d.plaintext); free(local_d.golden_ciphertext); @@ -297,7 +335,7 @@ static int aes256_cbc_hmac_sha1_sve_cleanup(struct test *test) { if (d) { free(d->plaintext); free(d->golden_ciphertext); - for (int s = 0; s < NSTREAM - 1; ++s) + for (int s = 0; s < d->nstreams - 1; ++s) free(d->variant_msg[s]); } free(d); diff --git a/tests/cpu/sve/ipsec/ipsec_aes256_cbc_sha2_224_sve.cpp b/tests/cpu/sve/ipsec/ipsec_aes256_cbc_sha2_224_sve.cpp index 17f53d841..eac53c62b 100644 --- a/tests/cpu/sve/ipsec/ipsec_aes256_cbc_sha2_224_sve.cpp +++ b/tests/cpu/sve/ipsec/ipsec_aes256_cbc_sha2_224_sve.cpp @@ -7,12 +7,17 @@ * @parblock * SVE port of ipsec_aes256_cbc_sha2_224_avx: AES/3DES encryption stays on OpenSSL EVP while * the HMAC-SHA224 MAC runs on the multi-stream SVE HMAC implementation - * (verified bit-for-bit against OpenSSL; see scripts/sve-sha-verify). - * Eight independent HMAC lanes run in parallel on SVE vector lanes: lane 0 + * (HMAC construction validated same-source in gem5-fi across 128..1024-bit against an + * independent scalar oracle and RFC 2202/4231 vectors; the underlying SHA + * kernels were verified bit-for-bit against OpenSSL in the original + * fixed-stream form — see scripts/sve-sha-verify and + * docs/research/progress-log.md). + * N = svcntw() (32-bit family) independent HMAC lanes run in parallel on + * SVE vector lanes (all-VL 128..1024-bit, one message per 32-bit lane): lane 0 * carries the original ciphertext message (MAC semantics identical to the - * original test), lanes 1-7 carry deterministic key-stream variant + * original test), lanes 1..N-1 carry deterministic key-stream variant * messages that fill the remaining lanes — same computation family, - * golden values precomputed for all 8. Ciphertext, MACs and decrypted + * golden values precomputed for all N. Ciphertext, MACs and decrypted * data are compared against golden values on every iteration to detect * silent data corruption. Iterations alternate between a re-rolled input * with a same-core recomputed golden and a frozen replay of the init-time @@ -28,19 +33,15 @@ #include "sandstone_ssl.h" #include "sve_hmac.h" #include "sdc_stress_config.h" -#include +#include #include #include -#ifndef HWCAP_SVE -#define HWCAP_SVE (1 << 22) -#endif - #define HMAC_KEY_SIZE_V (28) #define SHA224_DIGEST_SIZE (28) #define AES_KEY_SIZE_V (32) #define AES_IV_SIZE_V (16) -#define NSTREAM (8) +#define NSTREAM (SVE_SHA_STREAMS_MAX_32) /* All-VL: 数组按 1024-bit 上限定维; 运行时流数 = svcntw() */ struct aes256_cbc_sha2_224_sve_data { uint8_t aes_key[AES_KEY_SIZE_V]; @@ -49,20 +50,21 @@ struct aes256_cbc_sha2_224_sve_data { uint8_t *plaintext; uint8_t *golden_ciphertext; uint8_t golden_mac[NSTREAM][28]; - /* 7 条变体消息: 确定性派生(异或字节掩码), 填 SVE lane */ + /* N-1 条变体消息: 确定性派生(异或字节掩码), 填 SVE lane */ uint8_t *variant_msg[NSTREAM-1]; size_t data_size; + int nstreams; /* All-VL: init 期流数 = svl/4 (svcntw()), run 漂移守卫用 */ }; static size_t ipsec_data_size(struct test *test) { - int64_t v = sdc_knob_int_clamped(test, "data_size", 0, 1024, 16 << 20); + int64_t v = sdc_knob_int_clamped(test, "data_size", 0, 1024, 1900); if (v == 0) - sdc_env_knob_int_clamped("ipsec_aes256_cbc_sha2_224_sve", "data_size", &v, 1024, 16 << 20); + sdc_env_knob_int_clamped("ipsec_aes256_cbc_sha2_224_sve", "data_size", &v, 1024, 1900); if (v > 0) return (size_t)v; return (size_t)sdc_auto_linear_count(1, 4, 1.5, 600, thread_count(), - 1024, 16 << 20, test->id, "data_size"); + 1024, 1900, test->id, "data_size"); } static void aes_encrypt(const uint8_t *key, const uint8_t *iv, const uint8_t *in, uint8_t *out, size_t len) { @@ -86,54 +88,56 @@ static void aes_decrypt(const uint8_t *key, const uint8_t *iv, const uint8_t *in s_EVP_CIPHER_CTX_free(ctx); } -static void aes256_cbc_sha2_224_sve_compute_golden(struct aes256_cbc_sha2_224_sve_data *d) { +static int aes256_cbc_sha2_224_sve_compute_golden(struct test *test, struct aes256_cbc_sha2_224_sve_data *d) { aes_encrypt(d->aes_key, d->aes_iv, d->plaintext, d->golden_ciphertext, d->data_size); - /* SVE 多流 HMAC: lane0=密文(主语义), lane1..7=变体 */ + /* SVE 多流 HMAC: lane0=密文(主语义), lane1..N-1=变体 */ + const int n = (int)svcntw(); /* 与 kernel 内部推导一致 */ const uint8_t *keys[NSTREAM], *msgs[NSTREAM]; uint8_t variant_keys[NSTREAM][28]; - for (int s = 0; s < NSTREAM; ++s) { + for (int s = 0; s < n; ++s) { memcpy(variant_keys[s], d->hmac_key, 28); if (s > 0) variant_keys[s][0] ^= (uint8_t)s; /* 变体密钥: 确定性微扰 */ keys[s] = variant_keys[s]; } msgs[0] = d->golden_ciphertext; - for (int s = 1; s < NSTREAM; ++s) { + for (int s = 1; s < n; ++s) { /* 变体消息: 密文异或流掩码(确定性) */ for (size_t i = 0; i < d->data_size; ++i) d->variant_msg[s-1][i] = d->golden_ciphertext[i] ^ (uint8_t)(s * 0x5A + (i & 0x0F)); msgs[s] = d->variant_msg[s-1]; } - hmac_sha224_x8(d->golden_mac, keys, 28, msgs, d->data_size); + if (hmac_sha224_x8(d->golden_mac, keys, 28, msgs, d->data_size) != 0) { + log_error("%s: HMAC capacity overflow (data_size too large)", test->id); + return -1; + } + return 0; } static int aes256_cbc_sha2_224_sve_init(struct test *test) { if (s_EVP_CIPHER_CTX_new && s_EVP_EncryptInit_ex && s_EVP_DecryptInit_ex && s_EVP_CIPHER_CTX_new && s_EVP_aes_256_cbc()) { - unsigned long hwcap = getauxval(AT_HWCAP); - if ((hwcap & HWCAP_SVE) == 0) { - log_skip(CpuNotSupportedSkipCategory, - "to be implemented (placeholder): ARM SVE required for ipsec_aes256_cbc_sha2_224_sve"); - return EXIT_SKIP; - } - /* The multi-stream SVE HMAC kernels give one stream per SVE - * lane: 8x32-bit (sha1/224/256) needs svcntw()>=8 and - * 4x64-bit (384/512) needs svcntd()>=4 -- both mean VL>=256 - * bit. On shorter-VL SVE hardware (e.g. Neoverse N2, VL=128) - * lanes beyond VL are never written, so skip cleanly instead - * of comparing unwritten memory (target silicon: VL=256, see - * tests/cpu/sve/meson.build). */ - if (svcntw() < 8 || svcntd() < 4) { - log_skip(CpuNotSupportedSkipCategory, - "ARM SVE vector length %zu bytes < 32 required by the multi-stream HMAC kernels (svcntw=%zu svcntd=%zu)", - svcntb(), (size_t)svcntw(), (size_t)svcntd()); + /* All-VL Phase 2 P4: capability + 128..1024-bit range check in + * one framework-layer call (replaces the old lower-bound-only + * svcntw()>=8/svcntd()>=4 gate -- the 8.2 defect: at VL>=512 the + * fixed 8/4-wide W rows overflowed; the STREAMS_MAX-sized rows + + * this range gate close both ends). arm64_require_sve_vl returns + * the VL in BYTES. */ + const long svl = arm64_require_sve_vl(test, 128, 1024); + if (svl < 0) return EXIT_SKIP; + const int nstreams = (int)svl / 4; /* one stream per 32-bit lane (svcntw) */ + if (nstreams < 4 || nstreams > SVE_SHA_STREAMS_MAX_32) { /* [4, SVE_SHA_STREAMS_MAX_32], release-effective, NDEBUG-proof */ + log_error("%s: streams %d outside range — unit drift", test->id, nstreams); + return EXIT_FAILURE; } struct aes256_cbc_sha2_224_sve_data *d = (struct aes256_cbc_sha2_224_sve_data *)malloc(sizeof(*d)); if (!d) return EXIT_SKIP; + d->nstreams = nstreams; /* 工作集字节数: -O ipsec_aes256_cbc_sha2_224_sve.data_size=N / * SDC_IPSEC_AES256_CBC_SHA2_224_SVE_DATA_SIZE 环境变量 / 自适应: - * 每线程明文+密文+golden 密文+解密缓冲+变体消息 ~= (3+NSTREAM-1)x, - * 占物理内存 60%, clamp [1024, 16M]。线性缓冲用 sdc_auto_linear_count(sqrt 缩放会欠缩放压力)。历史硬编码 1024。CBC 无填充 + * 每线程明文+密文+golden 密文+解密缓冲+变体消息 ~= (3+nstreams-1)x, + * 占物理内存 60%, clamp [1024, 1900]——上限闭环 8.4: HMAC 组装 + * 缓冲 full[N][2048], 2048-64-9=1975B 安全边界。线性缓冲用 sdc_auto_linear_count(sqrt 缩放会欠缩放压力)。历史硬编码 1024。CBC 无填充 * 场景尺寸必须对齐到 AES 分组 16B, 对齐后 < 1024 取 1024。 */ d->data_size = ipsec_data_size(test); d->data_size &= ~(size_t)15; @@ -147,7 +151,7 @@ static int aes256_cbc_sha2_224_sve_init(struct test *test) { free(d); return EXIT_SKIP; } - for (int s = 0; s < NSTREAM - 1; ++s) { + for (int s = 0; s < d->nstreams - 1; ++s) { d->variant_msg[s] = (uint8_t *)malloc(d->data_size); if (!d->variant_msg[s]) { free(d->plaintext); @@ -162,7 +166,20 @@ static int aes256_cbc_sha2_224_sve_init(struct test *test) { memset_random(d->aes_iv, AES_IV_SIZE_V); memset_random(d->hmac_key, 28); memset_random(d->plaintext, d->data_size); - aes256_cbc_sha2_224_sve_compute_golden(d); + if (aes256_cbc_sha2_224_sve_compute_golden(test, d) != 0) { + /* P2a (review Minor): init-side failure propagates via the + * return value so the framework reports a clean Failed + * (sandstone_run.cpp: ret > 0 -> TestResult::Failed). + * report_fail_msg is a noreturn thread-cancel that only + * belongs in the test_run worker context — in test_init it + * would surface as Skipped("Unexpected OS error"). */ + free(d->plaintext); + free(d->golden_ciphertext); + for (int s = 0; s < d->nstreams - 1; ++s) + free(d->variant_msg[s]); + free(d); + return EXIT_FAILURE; + } test->data = d; return EXIT_SUCCESS; } else { @@ -174,10 +191,22 @@ static int aes256_cbc_sha2_224_sve_init(struct test *test) { static int aes256_cbc_sha2_224_sve_run(struct test *test, int cpu) { (void)cpu; struct aes256_cbc_sha2_224_sve_data *d = (struct aes256_cbc_sha2_224_sve_data *)test->data; + /* All-VL policy #3: never compare goldens across an init/run VL + * mismatch. Drift means this deployment's per-thread VL differs from + * the init thread's (e.g. a VL-pinning launcher changing the + * thread-inheritance chain) — an environment mismatch, which skips + * with CpuNotSupported (same category as the init-side range gate in + * arm64_require_sve_vl) instead of polluting the SDC failure signal. */ + if ((int)svcntw() != d->nstreams) { + log_skip(CpuNotSupportedSkipCategory, + "%s: SVE VL drifted between init (%d streams) and run (%d streams) — environment mismatch, not SDC", + test->id, d->nstreams, (int)svcntw()); + return EXIT_SKIP; + } uint8_t *ciphertext = (uint8_t *)malloc(d->data_size); uint8_t *decrypted = (uint8_t *)malloc(d->data_size); uint8_t mac[NSTREAM][28]; - uint8_t *variant_msg = (uint8_t *)malloc((NSTREAM-1) * d->data_size); + uint8_t *variant_msg = (uint8_t *)malloc((d->nstreams-1) * d->data_size); /* R2b (B2-003): per-thread plaintext + golden, re-rolled every * iteration (thread-safe: shared d is only read after init; the @@ -201,7 +230,7 @@ static int aes256_cbc_sha2_224_sve_run(struct test *test, int cpu) { "out of memory for per-thread buffers"); return EXIT_SKIP; } - for (int s = 0; s < NSTREAM - 1; ++s) { + for (int s = 0; s < d->nstreams - 1; ++s) { local_d.variant_msg[s] = (uint8_t *)malloc(d->data_size); if (!local_d.variant_msg[s]) { free(local_d.plaintext); @@ -234,27 +263,32 @@ static int aes256_cbc_sha2_224_sve_run(struct test *test, int cpu) { /* SVE 多流 HMAC (主消息 + 变体) */ { const uint8_t *keys[NSTREAM], *msgs[NSTREAM]; + const int n = (int)svcntw(); /* 与 kernel 内部推导一致 */ uint8_t variant_keys[NSTREAM][28]; - for (int s = 0; s < NSTREAM; ++s) { + for (int s = 0; s < n; ++s) { memcpy(variant_keys[s], d->hmac_key, 28); if (s > 0) variant_keys[s][0] ^= (uint8_t)s; keys[s] = variant_keys[s]; } msgs[0] = ciphertext; - for (int s = 1; s < NSTREAM; ++s) { + for (int s = 1; s < n; ++s) { for (size_t i = 0; i < d->data_size; ++i) variant_msg[(s-1)*d->data_size + i] = ciphertext[i] ^ (uint8_t)(s * 0x5A + (i & 0x0F)); msgs[s] = variant_msg + (s-1)*d->data_size; } - hmac_sha224_x8(mac, keys, 28, msgs, d->data_size); + if (hmac_sha224_x8(mac, keys, 28, msgs, d->data_size) != 0) { + log_error("%s: HMAC capacity overflow (data_size too large)", test->id); + report_fail_msg("%s: hmac assembly over capacity", test->id); + } } - for (int s = 0; s < NSTREAM; ++s) + for (int s = 0; s < d->nstreams; ++s) memcmp_or_fail(mac[s], d->golden_mac[s], 28, "HMAC-SHA224 digest mismatch (frozen replay)"); } else { /* re-roll: fresh plaintext + same-iteration golden (EVP encrypt + * SVE HMAC recomputed for this thread's data) */ memset_random(local_d.plaintext, d->data_size); - aes256_cbc_sha2_224_sve_compute_golden(&local_d); + if (aes256_cbc_sha2_224_sve_compute_golden(test, &local_d) != 0) + report_fail_msg("%s: hmac assembly over capacity", test->id); aes_encrypt(local_d.aes_key, local_d.aes_iv, local_d.plaintext, ciphertext, d->data_size); memcmp_or_fail(ciphertext, local_d.golden_ciphertext, d->data_size, "ciphertext mismatch (SVE)"); aes_decrypt(local_d.aes_key, local_d.aes_iv, ciphertext, decrypted, d->data_size); @@ -263,21 +297,25 @@ static int aes256_cbc_sha2_224_sve_run(struct test *test, int cpu) { /* SVE 多流 HMAC (主消息 + 变体) */ { const uint8_t *keys[NSTREAM], *msgs[NSTREAM]; + const int n = (int)svcntw(); /* 与 kernel 内部推导一致 */ uint8_t variant_keys[NSTREAM][28]; - for (int s = 0; s < NSTREAM; ++s) { + for (int s = 0; s < n; ++s) { memcpy(variant_keys[s], local_d.hmac_key, 28); if (s > 0) variant_keys[s][0] ^= (uint8_t)s; keys[s] = variant_keys[s]; } msgs[0] = ciphertext; - for (int s = 1; s < NSTREAM; ++s) { + for (int s = 1; s < n; ++s) { for (size_t i = 0; i < d->data_size; ++i) variant_msg[(s-1)*d->data_size + i] = ciphertext[i] ^ (uint8_t)(s * 0x5A + (i & 0x0F)); msgs[s] = variant_msg + (s-1)*d->data_size; } - hmac_sha224_x8(mac, keys, 28, msgs, d->data_size); + if (hmac_sha224_x8(mac, keys, 28, msgs, d->data_size) != 0) { + log_error("%s: HMAC capacity overflow (data_size too large)", test->id); + report_fail_msg("%s: hmac assembly over capacity", test->id); + } } - for (int s = 0; s < NSTREAM; ++s) + for (int s = 0; s < d->nstreams; ++s) memcmp_or_fail(mac[s], local_d.golden_mac[s], 28, "HMAC-SHA224 digest mismatch (SVE)"); } ++iter; @@ -285,7 +323,7 @@ static int aes256_cbc_sha2_224_sve_run(struct test *test, int cpu) { free(ciphertext); free(decrypted); free(variant_msg); - for (int s = 0; s < NSTREAM - 1; ++s) + for (int s = 0; s < d->nstreams - 1; ++s) free(local_d.variant_msg[s]); free(local_d.plaintext); free(local_d.golden_ciphertext); @@ -297,7 +335,7 @@ static int aes256_cbc_sha2_224_sve_cleanup(struct test *test) { if (d) { free(d->plaintext); free(d->golden_ciphertext); - for (int s = 0; s < NSTREAM - 1; ++s) + for (int s = 0; s < d->nstreams - 1; ++s) free(d->variant_msg[s]); } free(d); diff --git a/tests/cpu/sve/ipsec/ipsec_aes256_cbc_sha2_256_sve.cpp b/tests/cpu/sve/ipsec/ipsec_aes256_cbc_sha2_256_sve.cpp index dbc5322c6..a36caccfa 100644 --- a/tests/cpu/sve/ipsec/ipsec_aes256_cbc_sha2_256_sve.cpp +++ b/tests/cpu/sve/ipsec/ipsec_aes256_cbc_sha2_256_sve.cpp @@ -7,12 +7,17 @@ * @parblock * SVE port of ipsec_aes256_cbc_sha2_256_avx: AES/3DES encryption stays on OpenSSL EVP while * the HMAC-SHA256 MAC runs on the multi-stream SVE HMAC implementation - * (verified bit-for-bit against OpenSSL; see scripts/sve-sha-verify). - * Eight independent HMAC lanes run in parallel on SVE vector lanes: lane 0 + * (HMAC construction validated same-source in gem5-fi across 128..1024-bit against an + * independent scalar oracle and RFC 2202/4231 vectors; the underlying SHA + * kernels were verified bit-for-bit against OpenSSL in the original + * fixed-stream form — see scripts/sve-sha-verify and + * docs/research/progress-log.md). + * N = svcntw() (32-bit family) independent HMAC lanes run in parallel on + * SVE vector lanes (all-VL 128..1024-bit, one message per 32-bit lane): lane 0 * carries the original ciphertext message (MAC semantics identical to the - * original test), lanes 1-7 carry deterministic key-stream variant + * original test), lanes 1..N-1 carry deterministic key-stream variant * messages that fill the remaining lanes — same computation family, - * golden values precomputed for all 8. Ciphertext, MACs and decrypted + * golden values precomputed for all N. Ciphertext, MACs and decrypted * data are compared against golden values on every iteration to detect * silent data corruption. Iterations alternate between a re-rolled input * with a same-core recomputed golden and a frozen replay of the init-time @@ -28,19 +33,15 @@ #include "sandstone_ssl.h" #include "sve_hmac.h" #include "sdc_stress_config.h" -#include +#include #include #include -#ifndef HWCAP_SVE -#define HWCAP_SVE (1 << 22) -#endif - #define HMAC_KEY_SIZE_V (32) #define SHA256_DIGEST_SIZE (32) #define AES_KEY_SIZE_V (32) #define AES_IV_SIZE_V (16) -#define NSTREAM (8) +#define NSTREAM (SVE_SHA_STREAMS_MAX_32) /* All-VL: 数组按 1024-bit 上限定维; 运行时流数 = svcntw() */ struct aes256_cbc_sha2_256_sve_data { uint8_t aes_key[AES_KEY_SIZE_V]; @@ -49,20 +50,21 @@ struct aes256_cbc_sha2_256_sve_data { uint8_t *plaintext; uint8_t *golden_ciphertext; uint8_t golden_mac[NSTREAM][32]; - /* 7 条变体消息: 确定性派生(异或字节掩码), 填 SVE lane */ + /* N-1 条变体消息: 确定性派生(异或字节掩码), 填 SVE lane */ uint8_t *variant_msg[NSTREAM-1]; size_t data_size; + int nstreams; /* All-VL: init 期流数 = svl/4 (svcntw()), run 漂移守卫用 */ }; static size_t ipsec_data_size(struct test *test) { - int64_t v = sdc_knob_int_clamped(test, "data_size", 0, 1024, 16 << 20); + int64_t v = sdc_knob_int_clamped(test, "data_size", 0, 1024, 1900); if (v == 0) - sdc_env_knob_int_clamped("ipsec_aes256_cbc_sha2_256_sve", "data_size", &v, 1024, 16 << 20); + sdc_env_knob_int_clamped("ipsec_aes256_cbc_sha2_256_sve", "data_size", &v, 1024, 1900); if (v > 0) return (size_t)v; return (size_t)sdc_auto_linear_count(1, 4, 1.5, 600, thread_count(), - 1024, 16 << 20, test->id, "data_size"); + 1024, 1900, test->id, "data_size"); } static void aes_encrypt(const uint8_t *key, const uint8_t *iv, const uint8_t *in, uint8_t *out, size_t len) { @@ -86,54 +88,56 @@ static void aes_decrypt(const uint8_t *key, const uint8_t *iv, const uint8_t *in s_EVP_CIPHER_CTX_free(ctx); } -static void aes256_cbc_sha2_256_sve_compute_golden(struct aes256_cbc_sha2_256_sve_data *d) { +static int aes256_cbc_sha2_256_sve_compute_golden(struct test *test, struct aes256_cbc_sha2_256_sve_data *d) { aes_encrypt(d->aes_key, d->aes_iv, d->plaintext, d->golden_ciphertext, d->data_size); - /* SVE 多流 HMAC: lane0=密文(主语义), lane1..7=变体 */ + /* SVE 多流 HMAC: lane0=密文(主语义), lane1..N-1=变体 */ + const int n = (int)svcntw(); /* 与 kernel 内部推导一致 */ const uint8_t *keys[NSTREAM], *msgs[NSTREAM]; uint8_t variant_keys[NSTREAM][32]; - for (int s = 0; s < NSTREAM; ++s) { + for (int s = 0; s < n; ++s) { memcpy(variant_keys[s], d->hmac_key, 32); if (s > 0) variant_keys[s][0] ^= (uint8_t)s; /* 变体密钥: 确定性微扰 */ keys[s] = variant_keys[s]; } msgs[0] = d->golden_ciphertext; - for (int s = 1; s < NSTREAM; ++s) { + for (int s = 1; s < n; ++s) { /* 变体消息: 密文异或流掩码(确定性) */ for (size_t i = 0; i < d->data_size; ++i) d->variant_msg[s-1][i] = d->golden_ciphertext[i] ^ (uint8_t)(s * 0x5A + (i & 0x0F)); msgs[s] = d->variant_msg[s-1]; } - hmac_sha256_x8(d->golden_mac, keys, 32, msgs, d->data_size); + if (hmac_sha256_x8(d->golden_mac, keys, 32, msgs, d->data_size) != 0) { + log_error("%s: HMAC capacity overflow (data_size too large)", test->id); + return -1; + } + return 0; } static int aes256_cbc_sha2_256_sve_init(struct test *test) { if (s_EVP_CIPHER_CTX_new && s_EVP_EncryptInit_ex && s_EVP_DecryptInit_ex && s_EVP_CIPHER_CTX_new && s_EVP_aes_256_cbc()) { - unsigned long hwcap = getauxval(AT_HWCAP); - if ((hwcap & HWCAP_SVE) == 0) { - log_skip(CpuNotSupportedSkipCategory, - "to be implemented (placeholder): ARM SVE required for ipsec_aes256_cbc_sha2_256_sve"); - return EXIT_SKIP; - } - /* The multi-stream SVE HMAC kernels give one stream per SVE - * lane: 8x32-bit (sha1/224/256) needs svcntw()>=8 and - * 4x64-bit (384/512) needs svcntd()>=4 -- both mean VL>=256 - * bit. On shorter-VL SVE hardware (e.g. Neoverse N2, VL=128) - * lanes beyond VL are never written, so skip cleanly instead - * of comparing unwritten memory (target silicon: VL=256, see - * tests/cpu/sve/meson.build). */ - if (svcntw() < 8 || svcntd() < 4) { - log_skip(CpuNotSupportedSkipCategory, - "ARM SVE vector length %zu bytes < 32 required by the multi-stream HMAC kernels (svcntw=%zu svcntd=%zu)", - svcntb(), (size_t)svcntw(), (size_t)svcntd()); + /* All-VL Phase 2 P4: capability + 128..1024-bit range check in + * one framework-layer call (replaces the old lower-bound-only + * svcntw()>=8/svcntd()>=4 gate -- the 8.2 defect: at VL>=512 the + * fixed 8/4-wide W rows overflowed; the STREAMS_MAX-sized rows + + * this range gate close both ends). arm64_require_sve_vl returns + * the VL in BYTES. */ + const long svl = arm64_require_sve_vl(test, 128, 1024); + if (svl < 0) return EXIT_SKIP; + const int nstreams = (int)svl / 4; /* one stream per 32-bit lane (svcntw) */ + if (nstreams < 4 || nstreams > SVE_SHA_STREAMS_MAX_32) { /* [4, SVE_SHA_STREAMS_MAX_32], release-effective, NDEBUG-proof */ + log_error("%s: streams %d outside range — unit drift", test->id, nstreams); + return EXIT_FAILURE; } struct aes256_cbc_sha2_256_sve_data *d = (struct aes256_cbc_sha2_256_sve_data *)malloc(sizeof(*d)); if (!d) return EXIT_SKIP; + d->nstreams = nstreams; /* 工作集字节数: -O ipsec_aes256_cbc_sha2_256_sve.data_size=N / * SDC_IPSEC_AES256_CBC_SHA2_256_SVE_DATA_SIZE 环境变量 / 自适应: - * 每线程明文+密文+golden 密文+解密缓冲+变体消息 ~= (3+NSTREAM-1)x, - * 占物理内存 60%, clamp [1024, 16M]。线性缓冲用 sdc_auto_linear_count(sqrt 缩放会欠缩放压力)。历史硬编码 1024。CBC 无填充 + * 每线程明文+密文+golden 密文+解密缓冲+变体消息 ~= (3+nstreams-1)x, + * 占物理内存 60%, clamp [1024, 1900]——上限闭环 8.4: HMAC 组装 + * 缓冲 full[N][2048], 2048-64-9=1975B 安全边界。线性缓冲用 sdc_auto_linear_count(sqrt 缩放会欠缩放压力)。历史硬编码 1024。CBC 无填充 * 场景尺寸必须对齐到 AES 分组 16B, 对齐后 < 1024 取 1024。 */ d->data_size = ipsec_data_size(test); d->data_size &= ~(size_t)15; @@ -147,7 +151,7 @@ static int aes256_cbc_sha2_256_sve_init(struct test *test) { free(d); return EXIT_SKIP; } - for (int s = 0; s < NSTREAM - 1; ++s) { + for (int s = 0; s < d->nstreams - 1; ++s) { d->variant_msg[s] = (uint8_t *)malloc(d->data_size); if (!d->variant_msg[s]) { free(d->plaintext); @@ -162,7 +166,20 @@ static int aes256_cbc_sha2_256_sve_init(struct test *test) { memset_random(d->aes_iv, AES_IV_SIZE_V); memset_random(d->hmac_key, 32); memset_random(d->plaintext, d->data_size); - aes256_cbc_sha2_256_sve_compute_golden(d); + if (aes256_cbc_sha2_256_sve_compute_golden(test, d) != 0) { + /* P2a (review Minor): init-side failure propagates via the + * return value so the framework reports a clean Failed + * (sandstone_run.cpp: ret > 0 -> TestResult::Failed). + * report_fail_msg is a noreturn thread-cancel that only + * belongs in the test_run worker context — in test_init it + * would surface as Skipped("Unexpected OS error"). */ + free(d->plaintext); + free(d->golden_ciphertext); + for (int s = 0; s < d->nstreams - 1; ++s) + free(d->variant_msg[s]); + free(d); + return EXIT_FAILURE; + } test->data = d; return EXIT_SUCCESS; } else { @@ -174,10 +191,22 @@ static int aes256_cbc_sha2_256_sve_init(struct test *test) { static int aes256_cbc_sha2_256_sve_run(struct test *test, int cpu) { (void)cpu; struct aes256_cbc_sha2_256_sve_data *d = (struct aes256_cbc_sha2_256_sve_data *)test->data; + /* All-VL policy #3: never compare goldens across an init/run VL + * mismatch. Drift means this deployment's per-thread VL differs from + * the init thread's (e.g. a VL-pinning launcher changing the + * thread-inheritance chain) — an environment mismatch, which skips + * with CpuNotSupported (same category as the init-side range gate in + * arm64_require_sve_vl) instead of polluting the SDC failure signal. */ + if ((int)svcntw() != d->nstreams) { + log_skip(CpuNotSupportedSkipCategory, + "%s: SVE VL drifted between init (%d streams) and run (%d streams) — environment mismatch, not SDC", + test->id, d->nstreams, (int)svcntw()); + return EXIT_SKIP; + } uint8_t *ciphertext = (uint8_t *)malloc(d->data_size); uint8_t *decrypted = (uint8_t *)malloc(d->data_size); uint8_t mac[NSTREAM][32]; - uint8_t *variant_msg = (uint8_t *)malloc((NSTREAM-1) * d->data_size); + uint8_t *variant_msg = (uint8_t *)malloc((d->nstreams-1) * d->data_size); /* R2b (B2-003): per-thread plaintext + golden, re-rolled every * iteration (thread-safe: shared d is only read after init; the @@ -201,7 +230,7 @@ static int aes256_cbc_sha2_256_sve_run(struct test *test, int cpu) { "out of memory for per-thread buffers"); return EXIT_SKIP; } - for (int s = 0; s < NSTREAM - 1; ++s) { + for (int s = 0; s < d->nstreams - 1; ++s) { local_d.variant_msg[s] = (uint8_t *)malloc(d->data_size); if (!local_d.variant_msg[s]) { free(local_d.plaintext); @@ -234,27 +263,32 @@ static int aes256_cbc_sha2_256_sve_run(struct test *test, int cpu) { /* SVE 多流 HMAC (主消息 + 变体) */ { const uint8_t *keys[NSTREAM], *msgs[NSTREAM]; + const int n = (int)svcntw(); /* 与 kernel 内部推导一致 */ uint8_t variant_keys[NSTREAM][32]; - for (int s = 0; s < NSTREAM; ++s) { + for (int s = 0; s < n; ++s) { memcpy(variant_keys[s], d->hmac_key, 32); if (s > 0) variant_keys[s][0] ^= (uint8_t)s; keys[s] = variant_keys[s]; } msgs[0] = ciphertext; - for (int s = 1; s < NSTREAM; ++s) { + for (int s = 1; s < n; ++s) { for (size_t i = 0; i < d->data_size; ++i) variant_msg[(s-1)*d->data_size + i] = ciphertext[i] ^ (uint8_t)(s * 0x5A + (i & 0x0F)); msgs[s] = variant_msg + (s-1)*d->data_size; } - hmac_sha256_x8(mac, keys, 32, msgs, d->data_size); + if (hmac_sha256_x8(mac, keys, 32, msgs, d->data_size) != 0) { + log_error("%s: HMAC capacity overflow (data_size too large)", test->id); + report_fail_msg("%s: hmac assembly over capacity", test->id); + } } - for (int s = 0; s < NSTREAM; ++s) + for (int s = 0; s < d->nstreams; ++s) memcmp_or_fail(mac[s], d->golden_mac[s], 32, "HMAC-SHA256 digest mismatch (frozen replay)"); } else { /* re-roll: fresh plaintext + same-iteration golden (EVP encrypt + * SVE HMAC recomputed for this thread's data) */ memset_random(local_d.plaintext, d->data_size); - aes256_cbc_sha2_256_sve_compute_golden(&local_d); + if (aes256_cbc_sha2_256_sve_compute_golden(test, &local_d) != 0) + report_fail_msg("%s: hmac assembly over capacity", test->id); aes_encrypt(local_d.aes_key, local_d.aes_iv, local_d.plaintext, ciphertext, d->data_size); memcmp_or_fail(ciphertext, local_d.golden_ciphertext, d->data_size, "ciphertext mismatch (SVE)"); aes_decrypt(local_d.aes_key, local_d.aes_iv, ciphertext, decrypted, d->data_size); @@ -263,21 +297,25 @@ static int aes256_cbc_sha2_256_sve_run(struct test *test, int cpu) { /* SVE 多流 HMAC (主消息 + 变体) */ { const uint8_t *keys[NSTREAM], *msgs[NSTREAM]; + const int n = (int)svcntw(); /* 与 kernel 内部推导一致 */ uint8_t variant_keys[NSTREAM][32]; - for (int s = 0; s < NSTREAM; ++s) { + for (int s = 0; s < n; ++s) { memcpy(variant_keys[s], local_d.hmac_key, 32); if (s > 0) variant_keys[s][0] ^= (uint8_t)s; keys[s] = variant_keys[s]; } msgs[0] = ciphertext; - for (int s = 1; s < NSTREAM; ++s) { + for (int s = 1; s < n; ++s) { for (size_t i = 0; i < d->data_size; ++i) variant_msg[(s-1)*d->data_size + i] = ciphertext[i] ^ (uint8_t)(s * 0x5A + (i & 0x0F)); msgs[s] = variant_msg + (s-1)*d->data_size; } - hmac_sha256_x8(mac, keys, 32, msgs, d->data_size); + if (hmac_sha256_x8(mac, keys, 32, msgs, d->data_size) != 0) { + log_error("%s: HMAC capacity overflow (data_size too large)", test->id); + report_fail_msg("%s: hmac assembly over capacity", test->id); + } } - for (int s = 0; s < NSTREAM; ++s) + for (int s = 0; s < d->nstreams; ++s) memcmp_or_fail(mac[s], local_d.golden_mac[s], 32, "HMAC-SHA256 digest mismatch (SVE)"); } ++iter; @@ -285,7 +323,7 @@ static int aes256_cbc_sha2_256_sve_run(struct test *test, int cpu) { free(ciphertext); free(decrypted); free(variant_msg); - for (int s = 0; s < NSTREAM - 1; ++s) + for (int s = 0; s < d->nstreams - 1; ++s) free(local_d.variant_msg[s]); free(local_d.plaintext); free(local_d.golden_ciphertext); @@ -297,7 +335,7 @@ static int aes256_cbc_sha2_256_sve_cleanup(struct test *test) { if (d) { free(d->plaintext); free(d->golden_ciphertext); - for (int s = 0; s < NSTREAM - 1; ++s) + for (int s = 0; s < d->nstreams - 1; ++s) free(d->variant_msg[s]); } free(d); diff --git a/tests/cpu/sve/ipsec/sve_hmac.h b/tests/cpu/sve/ipsec/sve_hmac.h index 6aa7bc0a8..aa744f397 100644 --- a/tests/cpu/sve/ipsec/sve_hmac.h +++ b/tests/cpu/sve/ipsec/sve_hmac.h @@ -10,8 +10,13 @@ * message is [K^ipad block][msg blocks] for the inner hash and * [K^opad block][inner digest block] for the outer hash. The first block * is identical across streams but the chaining state evolves per stream, - * so 8 HMACs run in parallel on SVE lanes with identical results to - * serial HMAC (verified against OpenSSL in the standalone verifier). + * so N HMACs run in parallel on SVE lanes (N = svcntw() for the 32-bit + * family / svcntd() for the 64-bit family, all-VL Phase 2 P4) with + * identical results to serial HMAC (the underlying SHA kernels are + * verified against OpenSSL in the standalone verifier, + * scripts/sve-sha-verify/verify.cpp — five SHA sections, no HMAC + * section; the HMAC construction itself follows RFC 2104 and is + * cross-checked init-vs-run inside the ipsec tests). */ #ifndef SVE_HMAC_H #define SVE_HMAC_H @@ -19,17 +24,44 @@ #include "sve_sha_kernels.h" #include -/* SHA-256 家族 HMAC: 8 条流并行。 - * keys: 8 条流的 HMAC 密钥(每条 HMAC_KEY_SIZE 字节, 可各不相同)。 - * msgs: 8 条消息(各 msg_len 字节, msg_len < 2^32/8)。 - * out: out[s] 收到第 s 条流的 32 字节摘要(大端)。 */ -static inline void hmac_sha256_x8(uint8_t out[8][32], - const uint8_t *const keys[8], size_t key_len, - const uint8_t *const msgs[8], size_t msg_len) +/* msg_len 前置上界 (PR #282 处置移交, P4 落地): 下面各入口的 + * body = 块 + msg_len 与 padded = ((body + 9/17 + 63/127) / 块) * 块 都是 + * size_t 运算且先于任何上界检查执行 —— msg_len 接近 SIZE_MAX 时 + * body/padded 回绕回 2048 以内, 后置的 padded 单检通过, 随后 + * memcpy(full[s] + 块, msgs[s], msg_len) 巨量溢出。前置钳制 msg_len: + * padded <= 2048 (full[] 行宽) 等价于 32-bit 族 64 + msg_len + 9 <= 2048 + * (msg_len <= 1975)、64-bit 族 128 + msg_len + 17 <= 2048 (msg_len <= 1903)。 */ +#define SVE_HMAC_MSG_MAX_32 (2048 - 64 - 9) /* 1975 */ +#define SVE_HMAC_MSG_MAX_64 (2048 - 128 - 17) /* 1903 */ + +/* SHA-256 家族 HMAC: N 条流并行 (N = svcntw(), 内部推导)。 + * keys: N 条流的 HMAC 密钥(每条 HMAC_KEY_SIZE 字节, 可各不相同)。 + * msgs: N 条消息(各 msg_len 字节, msg_len <= SVE_HMAC_MSG_MAX_32)。 + * out: out[s] 收到第 s 条流的 32 字节摘要(大端)。 + * 返回 0 成功; -1 容量溢出(流数超 1024-bit 定维上限, key_len 超密钥块, + * 或 msg_len 超 SVE_HMAC_MSG_MAX_32 装配界)。 */ +static inline int hmac_sha256_x8(uint8_t out[SVE_SHA_STREAMS_MAX_32][32], + const uint8_t *const keys[SVE_SHA_STREAMS_MAX_32], size_t key_len, + const uint8_t *const msgs[SVE_SHA_STREAMS_MAX_32], size_t msg_len) { + const int n = (int)svcntw(); + /* Stream-count capacity guard (review P1): all per-stream arrays below + are dimensioned to SVE_SHA_STREAMS_MAX_32 (the 1024-bit cap) — a + 2048-bit host (svcntw() == 64) would overflow them. Callers gate VL + to 128..1024-bit via arm64_require_sve_vl(); this is the shared-layer + backstop (same-type gate as verify.cpp's exit-3). */ + if (n > SVE_SHA_STREAMS_MAX_32) + return -1; /* HMAC 密钥块 = 64 字节: key 补零后与 ipad(0x36)/opad(0x5c) 异或 */ - uint8_t ipad_blk[8][64], opad_blk[8][64]; - for (int s = 0; s < 8; ++s) { + uint8_t ipad_blk[SVE_SHA_STREAMS_MAX_32][64], opad_blk[SVE_SHA_STREAMS_MAX_32][64]; + /* Capacity guard, key side (adversarial review Minor-1): the ipad/opad + XOR loops index keys[s][i] unbounded — key_len beyond the 64-byte + (32-bit family) block overflows the stack. RFC 2104 long-key hashing + is out of scope for these callers (all pass <= 64B). */ + if (key_len > 64) + return -1; + + for (int s = 0; s < n; ++s) { memset(ipad_blk[s], 0x36, 64); memset(opad_blk[s], 0x5c, 64); for (size_t i = 0; i < key_len; ++i) { @@ -40,13 +72,23 @@ static inline void hmac_sha256_x8(uint8_t out[8][32], /* ---- 内层: inner[s] = SHA256(K^ipad[s] || msgs[s]) ---- * 每流消息 = ipad 块(64B) + msg(msg_len) → padding 后总块数 */ + /* Capacity guard, message side (PR #282 handover): bound msg_len BEFORE + the body/padded arithmetic — a near-SIZE_MAX msg_len wraps both back + under 2048 and slips past the padded check below. */ + if (msg_len > SVE_HMAC_MSG_MAX_32) + return -1; size_t body = 64 + msg_len; size_t padded = ((body + 9 + 63) / 64) * 64; uint32_t nblocks = (uint32_t)(padded / 64); - /* 组装 8 条完整消息(含 padding+长度) */ - uint8_t full[8][ /* padded */ 64 * 32]; /* 支持到 2048B 消息; 测试用 1024 */ - for (int s = 0; s < 8; ++s) { + /* Capacity guard (all-VL Phase 2 P4, fixes the data_size knob vs + 2048B buffer overflow): padded must fit full[]'s 2048 bytes. */ + if (padded > 2048) + return -1; + + /* 组装 N 条完整消息(含 padding+长度); 32 流上限 × 2048B = 64KB 栈, 可接受 */ + uint8_t full[SVE_SHA_STREAMS_MAX_32][2048]; + for (int s = 0; s < n; ++s) { memcpy(full[s], ipad_blk[s], 64); memcpy(full[s] + 64, msgs[s], msg_len); memset(full[s] + body, 0, padded - body); @@ -56,14 +98,14 @@ static inline void hmac_sha256_x8(uint8_t out[8][32], full[s][padded - 1 - i] = (uint8_t)(bits >> (8 * i)); } - const uint8_t *mp[8]; - for (int s = 0; s < 8; ++s) mp[s] = full[s]; - uint32_t st[8][8]; + const uint8_t *mp[SVE_SHA_STREAMS_MAX_32]; + for (int s = 0; s < n; ++s) mp[s] = full[s]; + uint32_t st[8][SVE_SHA_STREAMS_MAX_32]; sha256_compress_x8(st, mp, nblocks, sha256_iv); /* ---- 外层: out[s] = SHA256(K^opad[s] || inner_digest[s](32B)) ---- * 消息 = opad 块(64) + 32 字节摘要 → 2 块 */ - for (int s = 0; s < 8; ++s) { + for (int s = 0; s < n; ++s) { memcpy(full[s], opad_blk[s], 64); for (int w = 0; w < 8; ++w) { uint32_t v = st[w][s]; @@ -79,10 +121,10 @@ static inline void hmac_sha256_x8(uint8_t out[8][32], for (int i = 0; i < 8; ++i) full[s][127 - i] = (uint8_t)(bits >> (8 * i)); } - uint32_t st2[8][8]; + uint32_t st2[8][SVE_SHA_STREAMS_MAX_32]; sha256_compress_x8(st2, mp, 2, sha256_iv); - for (int s = 0; s < 8; ++s) + for (int s = 0; s < n; ++s) for (int w = 0; w < 8; ++w) { uint32_t v = st2[w][s]; out[s][w*4 + 0] = (uint8_t)(v >> 24); @@ -90,15 +132,29 @@ static inline void hmac_sha256_x8(uint8_t out[8][32], out[s][w*4 + 2] = (uint8_t)(v >> 8); out[s][w*4 + 3] = (uint8_t)(v); } + return 0; } -/* SHA-512 家族 HMAC: 4 条流并行 (128 字节密钥块, 64 字节摘要) */ -static inline void hmac_sha512_x4(uint8_t out[4][64], - const uint8_t *const keys[4], size_t key_len, - const uint8_t *const msgs[4], size_t msg_len) +/* SHA-512 家族 HMAC: N 条流并行 (N = svcntd(), 128 字节密钥块, 64 字节摘要) */ +static inline int hmac_sha512_x4(uint8_t out[SVE_SHA_STREAMS_MAX_64][64], + const uint8_t *const keys[SVE_SHA_STREAMS_MAX_64], size_t key_len, + const uint8_t *const msgs[SVE_SHA_STREAMS_MAX_64], size_t msg_len) { - uint8_t ipad_blk[4][128], opad_blk[4][128]; - for (int s = 0; s < 4; ++s) { + const int n = (int)svcntd(); + /* Stream-count capacity guard (review P1): all per-stream arrays below + are dimensioned to SVE_SHA_STREAMS_MAX_64 (the 1024-bit cap) — a + 2048-bit host (svcntd() == 32) would overflow them. Callers gate VL + to 128..1024-bit via arm64_require_sve_vl(); this is the shared-layer + backstop (same-type gate as verify.cpp's exit-3). */ + if (n > SVE_SHA_STREAMS_MAX_64) + return -1; + uint8_t ipad_blk[SVE_SHA_STREAMS_MAX_64][128], opad_blk[SVE_SHA_STREAMS_MAX_64][128]; + /* Capacity guard, key side (see the 32-bit family note): the block is + 128 bytes in the SHA-512 family. */ + if (key_len > 128) + return -1; + + for (int s = 0; s < n; ++s) { memset(ipad_blk[s], 0x36, 128); memset(opad_blk[s], 0x5c, 128); for (size_t i = 0; i < key_len; ++i) { @@ -107,12 +163,22 @@ static inline void hmac_sha512_x4(uint8_t out[4][64], } } + /* Capacity guard, message side (PR #282 handover): bound msg_len BEFORE + the body/padded arithmetic — a near-SIZE_MAX msg_len wraps both back + under 2048 and slips past the padded check below. */ + if (msg_len > SVE_HMAC_MSG_MAX_64) + return -1; size_t body = 128 + msg_len; size_t padded = ((body + 17 + 127) / 128) * 128; uint32_t nblocks = (uint32_t)(padded / 128); - uint8_t full[4][128 * 16]; /* 支持到 2048B 消息 */ - for (int s = 0; s < 4; ++s) { + /* Capacity guard (all-VL Phase 2 P4, fixes the data_size knob vs + 2048B buffer overflow): padded must fit full[]'s 2048 bytes. */ + if (padded > 2048) + return -1; + + uint8_t full[SVE_SHA_STREAMS_MAX_64][2048]; /* 16 流上限 × 2048B = 32KB 栈 */ + for (int s = 0; s < n; ++s) { memcpy(full[s], ipad_blk[s], 128); memcpy(full[s] + 128, msgs[s], msg_len); memset(full[s] + body, 0, padded - body); @@ -122,12 +188,12 @@ static inline void hmac_sha512_x4(uint8_t out[4][64], for (int i = 0; i < 8; ++i) full[s][padded - 1 - i] = (uint8_t)(bits >> (8 * i)); } - const uint8_t *mp[4]; - for (int s = 0; s < 4; ++s) mp[s] = full[s]; - uint64_t st[8][4]; + const uint8_t *mp[SVE_SHA_STREAMS_MAX_64]; + for (int s = 0; s < n; ++s) mp[s] = full[s]; + uint64_t st[8][SVE_SHA_STREAMS_MAX_64]; sha512_compress_x4(st, mp, nblocks, sha512_iv); - for (int s = 0; s < 4; ++s) { + for (int s = 0; s < n; ++s) { memcpy(full[s], opad_blk[s], 128); for (int w = 0; w < 8; ++w) for (int k = 0; k < 8; ++k) @@ -138,22 +204,38 @@ static inline void hmac_sha512_x4(uint8_t out[4][64], uint64_t bits = 192 * 8; for (int i = 0; i < 8; ++i) full[s][255 - i] = (uint8_t)(bits >> (8 * i)); } - uint64_t st2[8][4]; + uint64_t st2[8][SVE_SHA_STREAMS_MAX_64]; sha512_compress_x4(st2, mp, 2, sha512_iv); - for (int s = 0; s < 4; ++s) + for (int s = 0; s < n; ++s) for (int w = 0; w < 8; ++w) for (int k = 0; k < 8; ++k) out[s][w*8 + k] = (uint8_t)(st2[w][s] >> (56 - 8*k)); + return 0; } -/* SHA-1 HMAC: 8 条流并行 (64B 密钥块, 20 字节摘要) */ -static inline void hmac_sha1_x8(uint8_t out[8][20], - const uint8_t *const keys[8], size_t key_len, - const uint8_t *const msgs[8], size_t msg_len) +/* SHA-1 HMAC: N 条流并行 (N = svcntw(), 64B 密钥块, 20 字节摘要) */ +static inline int hmac_sha1_x8(uint8_t out[SVE_SHA_STREAMS_MAX_32][20], + const uint8_t *const keys[SVE_SHA_STREAMS_MAX_32], size_t key_len, + const uint8_t *const msgs[SVE_SHA_STREAMS_MAX_32], size_t msg_len) { - uint8_t ipad_blk[8][64], opad_blk[8][64]; - for (int s = 0; s < 8; ++s) { + const int n = (int)svcntw(); + /* Stream-count capacity guard (review P1): all per-stream arrays below + are dimensioned to SVE_SHA_STREAMS_MAX_32 (the 1024-bit cap) — a + 2048-bit host (svcntw() == 64) would overflow them. Callers gate VL + to 128..1024-bit via arm64_require_sve_vl(); this is the shared-layer + backstop (same-type gate as verify.cpp's exit-3). */ + if (n > SVE_SHA_STREAMS_MAX_32) + return -1; + uint8_t ipad_blk[SVE_SHA_STREAMS_MAX_32][64], opad_blk[SVE_SHA_STREAMS_MAX_32][64]; + /* Capacity guard, key side (adversarial review Minor-1): the ipad/opad + XOR loops index keys[s][i] unbounded — key_len beyond the 64-byte + (32-bit family) block overflows the stack. RFC 2104 long-key hashing + is out of scope for these callers (all pass <= 64B). */ + if (key_len > 64) + return -1; + + for (int s = 0; s < n; ++s) { memset(ipad_blk[s], 0x36, 64); memset(opad_blk[s], 0x5c, 64); for (size_t i = 0; i < key_len; ++i) { @@ -162,12 +244,22 @@ static inline void hmac_sha1_x8(uint8_t out[8][20], } } + /* Capacity guard, message side (PR #282 handover): bound msg_len BEFORE + the body/padded arithmetic — a near-SIZE_MAX msg_len wraps both back + under 2048 and slips past the padded check below. */ + if (msg_len > SVE_HMAC_MSG_MAX_32) + return -1; size_t body = 64 + msg_len; size_t padded = ((body + 9 + 63) / 64) * 64; uint32_t nblocks = (uint32_t)(padded / 64); - uint8_t full[8][64 * 32]; - for (int s = 0; s < 8; ++s) { + /* Capacity guard (all-VL Phase 2 P4, fixes the data_size knob vs + 2048B buffer overflow): padded must fit full[]'s 2048 bytes. */ + if (padded > 2048) + return -1; + + uint8_t full[SVE_SHA_STREAMS_MAX_32][2048]; + for (int s = 0; s < n; ++s) { memcpy(full[s], ipad_blk[s], 64); memcpy(full[s] + 64, msgs[s], msg_len); memset(full[s] + body, 0, padded - body); @@ -177,12 +269,12 @@ static inline void hmac_sha1_x8(uint8_t out[8][20], full[s][padded - 1 - i] = (uint8_t)(bits >> (8 * i)); } - const uint8_t *mp[8]; - for (int s = 0; s < 8; ++s) mp[s] = full[s]; - uint32_t st[5][8]; + const uint8_t *mp[SVE_SHA_STREAMS_MAX_32]; + for (int s = 0; s < n; ++s) mp[s] = full[s]; + uint32_t st[5][SVE_SHA_STREAMS_MAX_32]; sha1_compress_x8(st, mp, nblocks); - for (int s = 0; s < 8; ++s) { + for (int s = 0; s < n; ++s) { memcpy(full[s], opad_blk[s], 64); for (int w = 0; w < 5; ++w) { uint32_t v = st[w][s]; @@ -198,10 +290,10 @@ static inline void hmac_sha1_x8(uint8_t out[8][20], for (int i = 0; i < 8; ++i) full[s][127 - i] = (uint8_t)(bits >> (8 * i)); } - uint32_t st2[5][8]; + uint32_t st2[5][SVE_SHA_STREAMS_MAX_32]; sha1_compress_x8(st2, mp, 2); - for (int s = 0; s < 8; ++s) + for (int s = 0; s < n; ++s) for (int w = 0; w < 5; ++w) { uint32_t v = st2[w][s]; out[s][w*4 + 0] = (uint8_t)(v >> 24); @@ -209,16 +301,32 @@ static inline void hmac_sha1_x8(uint8_t out[8][20], out[s][w*4 + 2] = (uint8_t)(v >> 8); out[s][w*4 + 3] = (uint8_t)(v); } + return 0; } /* SHA-224 HMAC: 与 SHA-256 HMAC 同构但用 sha224_iv, 摘要 7 字(28 字节)。 */ -static inline void hmac_sha224_x8(uint8_t out[8][28], - const uint8_t *const keys[8], size_t key_len, - const uint8_t *const msgs[8], size_t msg_len) +static inline int hmac_sha224_x8(uint8_t out[SVE_SHA_STREAMS_MAX_32][28], + const uint8_t *const keys[SVE_SHA_STREAMS_MAX_32], size_t key_len, + const uint8_t *const msgs[SVE_SHA_STREAMS_MAX_32], size_t msg_len) { - uint8_t ipad_blk[8][64], opad_blk[8][64]; - for (int s = 0; s < 8; ++s) { + const int n = (int)svcntw(); + /* Stream-count capacity guard (review P1): all per-stream arrays below + are dimensioned to SVE_SHA_STREAMS_MAX_32 (the 1024-bit cap) — a + 2048-bit host (svcntw() == 64) would overflow them. Callers gate VL + to 128..1024-bit via arm64_require_sve_vl(); this is the shared-layer + backstop (same-type gate as verify.cpp's exit-3). */ + if (n > SVE_SHA_STREAMS_MAX_32) + return -1; + uint8_t ipad_blk[SVE_SHA_STREAMS_MAX_32][64], opad_blk[SVE_SHA_STREAMS_MAX_32][64]; + /* Capacity guard, key side (adversarial review Minor-1): the ipad/opad + XOR loops index keys[s][i] unbounded — key_len beyond the 64-byte + (32-bit family) block overflows the stack. RFC 2104 long-key hashing + is out of scope for these callers (all pass <= 64B). */ + if (key_len > 64) + return -1; + + for (int s = 0; s < n; ++s) { memset(ipad_blk[s], 0x36, 64); memset(opad_blk[s], 0x5c, 64); for (size_t i = 0; i < key_len; ++i) { @@ -227,12 +335,22 @@ static inline void hmac_sha224_x8(uint8_t out[8][28], } } + /* Capacity guard, message side (PR #282 handover): bound msg_len BEFORE + the body/padded arithmetic — a near-SIZE_MAX msg_len wraps both back + under 2048 and slips past the padded check below. */ + if (msg_len > SVE_HMAC_MSG_MAX_32) + return -1; size_t body = 64 + msg_len; size_t padded = ((body + 9 + 63) / 64) * 64; uint32_t nblocks = (uint32_t)(padded / 64); - uint8_t full[8][64 * 32]; - for (int s = 0; s < 8; ++s) { + /* Capacity guard (all-VL Phase 2 P4, fixes the data_size knob vs + 2048B buffer overflow): padded must fit full[]'s 2048 bytes. */ + if (padded > 2048) + return -1; + + uint8_t full[SVE_SHA_STREAMS_MAX_32][2048]; + for (int s = 0; s < n; ++s) { memcpy(full[s], ipad_blk[s], 64); memcpy(full[s] + 64, msgs[s], msg_len); memset(full[s] + body, 0, padded - body); @@ -242,12 +360,12 @@ static inline void hmac_sha224_x8(uint8_t out[8][28], full[s][padded - 1 - i] = (uint8_t)(bits >> (8 * i)); } - const uint8_t *mp[8]; - for (int s = 0; s < 8; ++s) mp[s] = full[s]; - uint32_t st[8][8]; + const uint8_t *mp[SVE_SHA_STREAMS_MAX_32]; + for (int s = 0; s < n; ++s) mp[s] = full[s]; + uint32_t st[8][SVE_SHA_STREAMS_MAX_32]; sha256_compress_x8(st, mp, nblocks, sha224_iv); - for (int s = 0; s < 8; ++s) { + for (int s = 0; s < n; ++s) { memcpy(full[s], opad_blk[s], 64); /* SHA-224 摘要是 7 字(28B) —— 外层消息 = 64+28 = 92 字节, 不是 96! */ for (int w = 0; w < 7; ++w) { @@ -263,10 +381,10 @@ static inline void hmac_sha224_x8(uint8_t out[8][28], for (int i = 0; i < 8; ++i) full[s][127 - i] = (uint8_t)(bits >> (8 * i)); } - uint32_t st2[8][8]; + uint32_t st2[8][SVE_SHA_STREAMS_MAX_32]; sha256_compress_x8(st2, mp, 2, sha224_iv); - for (int s = 0; s < 8; ++s) + for (int s = 0; s < n; ++s) for (int w = 0; w < 7; ++w) { uint32_t v = st2[w][s]; out[s][w*4 + 0] = (uint8_t)(v >> 24); @@ -274,15 +392,29 @@ static inline void hmac_sha224_x8(uint8_t out[8][28], out[s][w*4 + 2] = (uint8_t)(v >> 8); out[s][w*4 + 3] = (uint8_t)(v); } + return 0; } /* SHA-384 HMAC: SHA-512 内核 + sha384_iv, 摘要 6 字(48 字节)。 */ -static inline void hmac_sha384_x4(uint8_t out[4][48], - const uint8_t *const keys[4], size_t key_len, - const uint8_t *const msgs[4], size_t msg_len) +static inline int hmac_sha384_x4(uint8_t out[SVE_SHA_STREAMS_MAX_64][48], + const uint8_t *const keys[SVE_SHA_STREAMS_MAX_64], size_t key_len, + const uint8_t *const msgs[SVE_SHA_STREAMS_MAX_64], size_t msg_len) { - uint8_t ipad_blk[4][128], opad_blk[4][128]; - for (int s = 0; s < 4; ++s) { + const int n = (int)svcntd(); + /* Stream-count capacity guard (review P1): all per-stream arrays below + are dimensioned to SVE_SHA_STREAMS_MAX_64 (the 1024-bit cap) — a + 2048-bit host (svcntd() == 32) would overflow them. Callers gate VL + to 128..1024-bit via arm64_require_sve_vl(); this is the shared-layer + backstop (same-type gate as verify.cpp's exit-3). */ + if (n > SVE_SHA_STREAMS_MAX_64) + return -1; + uint8_t ipad_blk[SVE_SHA_STREAMS_MAX_64][128], opad_blk[SVE_SHA_STREAMS_MAX_64][128]; + /* Capacity guard, key side (see the 32-bit family note): the block is + 128 bytes in the SHA-512 family. */ + if (key_len > 128) + return -1; + + for (int s = 0; s < n; ++s) { memset(ipad_blk[s], 0x36, 128); memset(opad_blk[s], 0x5c, 128); for (size_t i = 0; i < key_len; ++i) { @@ -291,12 +423,22 @@ static inline void hmac_sha384_x4(uint8_t out[4][48], } } + /* Capacity guard, message side (PR #282 handover): bound msg_len BEFORE + the body/padded arithmetic — a near-SIZE_MAX msg_len wraps both back + under 2048 and slips past the padded check below. */ + if (msg_len > SVE_HMAC_MSG_MAX_64) + return -1; size_t body = 128 + msg_len; size_t padded = ((body + 17 + 127) / 128) * 128; uint32_t nblocks = (uint32_t)(padded / 128); - uint8_t full[4][128 * 16]; - for (int s = 0; s < 4; ++s) { + /* Capacity guard (all-VL Phase 2 P4, fixes the data_size knob vs + 2048B buffer overflow): padded must fit full[]'s 2048 bytes. */ + if (padded > 2048) + return -1; + + uint8_t full[SVE_SHA_STREAMS_MAX_64][2048]; + for (int s = 0; s < n; ++s) { memcpy(full[s], ipad_blk[s], 128); memcpy(full[s] + 128, msgs[s], msg_len); memset(full[s] + body, 0, padded - body); @@ -306,12 +448,12 @@ static inline void hmac_sha384_x4(uint8_t out[4][48], for (int i = 0; i < 8; ++i) full[s][padded - 1 - i] = (uint8_t)(bits >> (8 * i)); } - const uint8_t *mp[4]; - for (int s = 0; s < 4; ++s) mp[s] = full[s]; - uint64_t st[8][4]; + const uint8_t *mp[SVE_SHA_STREAMS_MAX_64]; + for (int s = 0; s < n; ++s) mp[s] = full[s]; + uint64_t st[8][SVE_SHA_STREAMS_MAX_64]; sha512_compress_x4(st, mp, nblocks, sha384_iv); - for (int s = 0; s < 4; ++s) { + for (int s = 0; s < n; ++s) { memcpy(full[s], opad_blk[s], 128); /* SHA-384 摘要是 6 字(48B) —— 外层消息 = 128+48 = 176 字节, 不是 192! */ for (int w = 0; w < 6; ++w) @@ -322,14 +464,14 @@ static inline void hmac_sha384_x4(uint8_t out[4][48], uint64_t bits = 176 * 8; for (int i = 0; i < 8; ++i) full[s][255 - i] = (uint8_t)(bits >> (8 * i)); } - uint64_t st2[8][4]; + uint64_t st2[8][SVE_SHA_STREAMS_MAX_64]; sha512_compress_x4(st2, mp, 2, sha384_iv); - for (int s = 0; s < 4; ++s) + for (int s = 0; s < n; ++s) for (int w = 0; w < 6; ++w) for (int k = 0; k < 8; ++k) out[s][w*8 + k] = (uint8_t)(st2[w][s] >> (56 - 8*k)); + return 0; } #endif /* SVE_HMAC_H */ - diff --git a/tests/cpu/sve/ipsec/sve_sha_kernels.h b/tests/cpu/sve/ipsec/sve_sha_kernels.h index 20d2bf5a9..b3e23370f 100644 --- a/tests/cpu/sve/ipsec/sve_sha_kernels.h +++ b/tests/cpu/sve/ipsec/sve_sha_kernels.h @@ -10,9 +10,10 @@ * * Design: SHA compression is serial *within* one message (block i+1 chains * on block i's state) but independent *across* messages. The kernels - * process 8 independent messages in parallel — one per 32-bit SVE lane - * (SHA-256 family) or 4 per 64-bit lane (SHA-512 family) — with message - * schedules and round state entirely in SVE registers. + * process N independent messages in parallel — one per 32-bit SVE lane + * (SHA-256 family) or one per 64-bit lane (SHA-512 family), N = + * svcntw()/svcntd() at the caller's runtime VL (all-VL Phase 2 P4) — with + * message schedules and round state entirely in SVE registers. * * Correctness gate: verified against OpenSSL EVP digests on a large * vector set (see the standalone verifier) before use as a golden base. @@ -33,6 +34,18 @@ #define SVE_ROTR64(pg, x, n) \ svorr_u64_x(pg, svlsr_n_u64_x(pg, (x), (n)), svlsl_n_u64_x(pg, (x), 64 - (n))) +/* All-VL Phase 2 P4: stream count = runtime lane count (one message per + lane); arrays sized to the 1024-bit cap. The vector code itself is + lane-count-agnostic (ptrue auto-scales; the schedule is per-column). + Precondition (review P1): callers must keep svcntw() <= + SVE_SHA_STREAMS_MAX_32 / svcntd() <= SVE_SHA_STREAMS_MAX_64 (i.e. + VL <= 1024-bit) — the W rows below and the callers' per-stream arrays + are dimensioned to these caps, so a 2048-bit host would overflow them. + Every entry point enforces this before touching the kernels: the + sve_hmac.h entries return -1, the standalone verifier exits 3. */ +#define SVE_SHA_STREAMS_MAX_32 32 /* svcntw() at VL=1024-bit */ +#define SVE_SHA_STREAMS_MAX_64 16 /* svcntd() at VL=1024-bit */ + /* ================= SHA-256 family (SHA-224/SHA-256), 8×32-bit lanes ================= */ static const uint32_t sha256_k[64] = { @@ -53,14 +66,17 @@ static const uint32_t sha224_iv[8] = { 0xc1059ed8,0x367cd507,0x3070dd17,0xf70e5939,0xffc00b31,0x68581511,0x64f98fa7,0xbefa4fa4 }; -/* 8 条等长消息并行压缩。msgs: 8 条消息指针(各 nblocks*64B, 已含 padding 与长度); - * out: out[s] 写第 s 条消息的 8 个状态字(小端 uint32)。 - * 要求 svcntw() >= 8 (VL>=256)。 */ -static inline void sha256_compress_x8(uint32_t out[8][8], /* [word][stream] */ - const uint8_t *const msgs[8], +/* N 条等长消息并行压缩 (N = svcntw(), 32-bit lane 每条一流)。msgs: N 条消息 + * 指针(各 nblocks*64B, 已含 padding 与长度); out: out[w][s] 写第 s 条消息的 + * 8 个状态字(小端 uint32)。streams = svcntw() at the caller's runtime VL + * (all-VL Phase 2: 调用方 128..1024-bit 门控, 此处 n∈[4,32] 由 VL 决定)。 */ +static inline void sha256_compress_x8(uint32_t out[8][SVE_SHA_STREAMS_MAX_32], /* [word][stream] */ + const uint8_t *const msgs[SVE_SHA_STREAMS_MAX_32], uint32_t nblocks, const uint32_t iv[8]) { + /* 流数 = 运行时 lane 数(内部推导): 向量通路按 VL 动作, n≡lane 数。 */ + const int n = (int)svcntw(); svbool_t pg = svptrue_b32(); svuint32_t a = svdup_u32(iv[0]), b = svdup_u32(iv[1]), @@ -73,11 +89,12 @@ static inline void sha256_compress_x8(uint32_t out[8][8], /* [word][stream] */ svuint32_t a0 = a, b0 = b, c0 = c, d0 = d, e0 = e, f0 = f, g0 = g, h0 = h; /* SVE 类型不能做数组元素(可变长度类型禁入聚合) —— 消息调度字表 - * 驻留内存 uint32_t[64][8] (lane=流), 每步用 svld1/svst1 往返。 - * 调度展开与轮函数仍在 SVE 8-lane 向量里算。 */ - uint32_t W[64][8]; + * 驻留内存 uint32_t[64][SVE_SHA_STREAMS_MAX_32] (lane=流, 行宽按 + * 1024-bit 上限定维——修复旧 [8] 定维在 VL>=512 时的越界栈写), + * 每步用 svld1/svst1 往返。调度展开与轮函数仍在 SVE 向量里算。 */ + uint32_t W[64][SVE_SHA_STREAMS_MAX_32]; for (int t = 0; t < 16; ++t) { - for (int s = 0; s < 8; ++s) { + for (int s = 0; s < n; ++s) { const uint8_t *p = msgs[s] + (size_t)blk * 64 + (size_t)t * 4; W[t][s] = ((uint32_t)p[0] << 24) | ((uint32_t)p[1] << 16) | ((uint32_t)p[2] << 8) | (uint32_t)p[3]; @@ -172,13 +189,17 @@ static const uint64_t sha384_iv[8] = { 0x67332667ffc00b31ULL,0x8eb44a8768581511ULL,0xdb0c2e0d64f98fa7ULL,0x47b5481dbefa4fa4ULL }; -/* 4 条等长消息并行压缩 (64-bit lane)。msgs: 4 条消息指针(各 nblocks*128B); - * out[word][stream]: 8 个字 × 4 条流。要求 svcntd() >= 4 (VL>=256)。 */ -static inline void sha512_compress_x4(uint64_t out[8][4], /* [word][stream] */ - const uint8_t *const msgs[4], +/* N 条等长消息并行压缩 (64-bit lane, N = svcntd())。msgs: N 条消息指针(各 + * nblocks*128B); out[word][stream]: 8 个字 × N 条流。streams = svcntd() at + * the caller's runtime VL (all-VL Phase 2: 调用方 128..1024-bit 门控, 此处 + * n∈[2,16] 由 VL 决定)。 */ +static inline void sha512_compress_x4(uint64_t out[8][SVE_SHA_STREAMS_MAX_64], /* [word][stream] */ + const uint8_t *const msgs[SVE_SHA_STREAMS_MAX_64], uint32_t nblocks, const uint64_t iv[8]) { + /* 流数 = 运行时 lane 数(内部推导): 向量通路按 VL 动作, n≡lane 数。 */ + const int n = (int)svcntd(); svbool_t pg = svptrue_b64(); svuint64_t a = svdup_u64(iv[0]), b = svdup_u64(iv[1]), @@ -189,9 +210,9 @@ static inline void sha512_compress_x4(uint64_t out[8][4], /* [word][stream] */ for (uint32_t blk = 0; blk < nblocks; ++blk) { svuint64_t a0 = a, b0 = b, c0 = c, d0 = d, e0 = e, f0 = f, g0 = g, h0 = h; - uint64_t W[80][4]; + uint64_t W[80][SVE_SHA_STREAMS_MAX_64]; for (int t = 0; t < 16; ++t) { - for (int s = 0; s < 4; ++s) { + for (int s = 0; s < n; ++s) { const uint8_t *p = msgs[s] + (size_t)blk * 128 + (size_t)t * 8; uint64_t w = 0; for (int k = 0; k < 8; ++k) @@ -254,15 +275,20 @@ static inline void sha512_compress_x4(uint64_t out[8][4], /* [word][stream] */ svst1_u64(pg, out[7], h); } -/* ================= SHA-1 (HMAC-SHA1-96 用), 8×32-bit lanes ================= */ +/* ================= SHA-1 (HMAC-SHA1-96 用), N×32-bit lanes ================= */ -/* 4 条等长消息并行 (64-bit lane)。msgs: 4 条消息指针(各 nblocks*128B); - * out[s]: 8 个 uint64 状态。要求 svcntd() >= 4 (VL>=256)。 */ +/* N 条等长消息并行压缩 (32-bit lane, N = svcntw())。msgs: N 条消息指针(各 + * nblocks*64B); out[w][s]: 5 个状态字 × N 条流。streams = svcntw() at the + * caller's runtime VL (all-VL Phase 2: 调用方 128..1024-bit 门控, 此处 + * n∈[4,32] 由 VL 决定)。(旧注释 "4 条 64-bit lane" 系 sha512 段复制残留, + * 实况一直是 32-bit lane。) */ -static inline void sha1_compress_x8(uint32_t out[5][8], /* [word][stream] */ - const uint8_t *const msgs[8], +static inline void sha1_compress_x8(uint32_t out[5][SVE_SHA_STREAMS_MAX_32], /* [word][stream] */ + const uint8_t *const msgs[SVE_SHA_STREAMS_MAX_32], uint32_t nblocks) { + /* 流数 = 运行时 lane 数(内部推导): 向量通路按 VL 动作, n≡lane 数。 */ + const int n = (int)svcntw(); static const uint32_t iv[5] = {0x67452301, 0xefcdab89, 0x98badcfe, 0x10325476, 0xc3d2e1f0}; svbool_t pg = svptrue_b32(); @@ -273,9 +299,9 @@ static inline void sha1_compress_x8(uint32_t out[5][8], /* [word][stream] */ for (uint32_t blk = 0; blk < nblocks; ++blk) { svuint32_t a0 = a, b0 = b, c0 = c, d0 = d, e0 = e; - uint32_t W[80][8]; + uint32_t W[80][SVE_SHA_STREAMS_MAX_32]; for (int t = 0; t < 16; ++t) { - for (int s = 0; s < 8; ++s) { + for (int s = 0; s < n; ++s) { const uint8_t *p = msgs[s] + (size_t)blk * 64 + (size_t)t * 4; W[t][s] = ((uint32_t)p[0] << 24) | ((uint32_t)p[1] << 16) | ((uint32_t)p[2] << 8) | (uint32_t)p[3]; diff --git a/tests/cpu/sve/meson.build b/tests/cpu/sve/meson.build index ed36c8fc9..66b571ec7 100644 --- a/tests/cpu/sve/meson.build +++ b/tests/cpu/sve/meson.build @@ -11,7 +11,8 @@ # Naming rule (user decision 2026-09-19): _avx/_avx2 -> _sve, _avx512 -> # _sve_wide. The _wide suffix reflects the wider-vector variant of the pair # (original avx2=256-bit vs avx512=512-bit), NOT the SVE2 instruction set — -# no SVE2 instructions are used (target silicon has SVE1 only, VL=256). +# no SVE2 instructions are used (target silicon Kunpeng 920: SVE1, VL=256; +# the tests themselves are all-VL 128..1024-bit since all-VL Phase 2). # # Build discipline mirrors tests/cpu/arm64/meson.build: the whole subdir is # entered only under the aarch64 guard (x86-64 untouched), and the SVE object @@ -297,12 +298,19 @@ if host_machine.cpu_family() == 'aarch64' # IPSec domain (24 ports). Two tiers: # - HMAC/SHA family (17): encryption stays on OpenSSL EVP, the MAC runs # on the multi-stream SVE HMAC layer (tests/cpu/sve/ipsec/sve_hmac.h - # over sve_sha_kernels.h — both verified bit-for-bit against OpenSSL, - # 5200+1360 vectors, see scripts/sve-sha-verify). NSTREAM lanes: lane 0 - # carries the original ciphertext message (MAC semantics identical to - # the avx-named original), lanes 1..N-1 carry deterministic key-stream - # variant messages that fill the remaining SVE lanes; goldens for all - # lanes precomputed in init. + # over sve_sha_kernels.h). The SHA kernels were verified bit-for-bit + # against OpenSSL in the original fixed 8/4-stream form (5200+1360 + # vectors, scripts/sve-sha-verify, VL=256 host); the all-VL + # parameterized form is validated same-source in gem5-fi at + # 128..1024-bit (independent scalar oracle + RFC 2202/4231 vectors, + # gem5-fi workloads/sdcshield/sve_hmac_dual — verification boundary in + # docs/research/progress-log.md; an OpenSSL re-run needs an SVE host). + # N = svcntw()/svcntd() + # lanes (all-VL 128..1024-bit, one message per lane; stream count is + # the runtime lane count): lane 0 carries the original ciphertext + # message (MAC semantics identical to the avx-named original), lanes + # 1..N-1 carry deterministic key-stream variant messages that fill the + # remaining SVE lanes; goldens for all lanes precomputed in init. # - EVP-only family (7: GCM x2, CMAC/XCBC x5): the workload has no SHA # component to move to SVE (CMAC/XCBC are AES-CBC-MAC chains inside # OpenSSL; upstream has no SVE AES path), so these are faithful