Where can I download llama3_1-8b-instruct-hf-torch-fp4.tar.gz for MLPerf v6.0 Llama3.1-8B?

[XindiG6]

Hi, I’m trying to reproduce the NVIDIA MLPerf Inference v6.0 Llama3.1-8B result using the Docker flow.

The README mentions this FP4 artifact:

/opt/fp4-quantized-modelopt/llama3_1-8b-instruct-hf-torch-fp4.tar.gz

However, after pulling the NGC image:

nvcr.io/nvidia/mlperf/mlperf-inference:tensorrt_llm_release-feat-1.2-mlpinf-b5ddff4_mlperf-main-f538816_jan28_x86

I cannot find this tarball inside the container:

find / -name "llama3_1-8b-instruct-hf-torch-fp4.tar.gz" 2>/dev/null

I then tried using the public Hugging Face FP4 model instead:

nvidia/Llama-3.1-8B-Instruct-NVFP4

and placed it at:

build/models/Llama3.1-8B/fp4-quantized-modelopt/llama3_1-8b-instruct-hf-torch-fp4

The benchmark runs successfully, but AccuracyOnly consistently fails only on ROUGE-L:

ROUGE1 PASS
ROUGE2 PASS
ROUGEL 24.22 < 24.250743 FAIL
ROUGELSUM PASS
GEN_LEN PASS

So I’m wondering whether the Hugging Face FP4 model is different from the exact MLPerf submission artifact.

Could you clarify where to download the exact:

llama3_1-8b-instruct-hf-torch-fp4.tar.gz

or whether nvidia/Llama-3.1-8B-Instruct-NVFP4 is intended to be the replacement?

Thanks!