FP8模型跑出来的视频为何都是花屏的？

[newstargo]

python inference_nava.py --config configs/nava.yaml --ckpt /NAVA-model/NAVA_fp8.safetensors --out_dir ./eval_results_sp --data_format json --data_file text.jsonl --t5_offload --frames 17 --fps 8 --steps 30 --gen_turn 1

JSONL: {"prompt": "女子在跳舞", "image_path": "E:/13.png"}

你好，我用4090的48G 加载FP8模型，无论宽高大小，帧数大小，FPS在8-24之间，I2V出来的视频除首帧外，后面都变形的，T2V也是人像模糊不成样，感觉是生图模块不清晰，合成出来就不行，请问是否这样，还是有什么参数没设好？

idx0_.-av-0.mp4

另外如果加载全量的NAVA模型，使用了--group_offload，GPU占28G，内存占85G时，还没推理提示错误如下：

E:\v311\Lib\site-packages_distutils_hack_init_.py:53: UserWarning: Reliance on distutils from stdlib is deprecated. Users must rely on setuptools to provide the distutils module. Avoid importing distutils or import setuptools first, and avoid setting SETUPTOOLS_USE_DISTUTILS=stdlib. Register concerns at https://github.com/pypa/setuptools/issues/new?template=distutils-deprecation.yml
warnings.warn(
Multiple distributions found for package optimum. Picked distribution: optimum
[Rewriter] Disabled (pass --rewrite to enable)
F:\NAVA\nava_src\models\nava\modules\vae2_2.py:906: UserWarning: expandable_segments not supported on this platform (Triggered internally at C:\actions-runner_work\pytorch\pytorch\pytorch\c10/cuda/CUDAAllocatorConfig.h:35.)
mean = torch.tensor(
Using cache found in C:\Users\nsg/.cache\torch\hub\IDRnD_ReDimNet_master
[AudioVAE] ReDimNet speaker model loaded successfully on cuda:0
temporal_scaling: 0.24 !!!!!
Score model (MMDIT) all parameters:6296995520
missing: [], unexpected: []
[Offload] T5 CPU offload enabled: encoder moves to GPU only during text encoding
Traceback (most recent call last):
File "F:\NAVA\inference_nava.py", line 810, in
main()
File "E:\v311\Lib\site-packages\torch\utils_contextlib.py", line 120, in decorate_context
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "F:\NAVA\inference_nava.py", line 504, in main
apply_group_offload(pipe.model.backbone, args.offload_group_size, device)
File "F:\NAVA\inference_nava.py", line 147, in apply_group_offload
d[name] = p.data.pin_memory()
^^^^^^^^^^^^^^^^^^^
torch.AcceleratorError: CUDA error: resource already mapped
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1
Compile with TORCH_USE_CUDA_DSA to enable device-side assertions.

这能怎么解决，请求解！

[robingg1]

The issue is mainly caused by the prompt being too short. In our experience, overly short prompts often lead to a distribution that is very close to the negative prompt, resulting in noticeably worse generation quality.

We highly recommend using the prompt format described in the README. The easiest option is to use the provided prompt rewriter. Alternatively, you can manually enrich the prompt with detailed descriptions of the dance movements, body actions, and audio elements.

Below is an example generated with the rewritten prompt on our side, which demonstrates the expected quality.
Prompt: 高质量时尚舞蹈视频，现代室内展览空间内，一位身穿金色亮片单肩礼服的长发女性站在画面中央，全程正面面对镜头。视频开始时她双手叉腰，自信微笑，身体随着音乐节奏轻轻摆动；随后她保持正面朝向，通过连续的左右摆胯、重心切换和踩点动作进行舞蹈，肩部和上身跟随节拍自然律动，长发轻微晃动；随着动作持续，摆动幅度逐渐增大，时而单手叉腰配合简单手势，时而恢复双手叉腰，全程不转身、不背对镜头；视频结束时她保持双手叉腰姿势定格并看向镜头。画面中没有人物对白，也没有任何旁白。镜头采用稳定单镜头拍摄，视频开始时为正面中近景，随后以缓慢连续的速度平滑拉远至全身景别，全程无切镜、无跳切、无突然变焦，重点捕捉人物正面舞蹈动作和表情变化。音频为节奏鲜明的流行舞曲，同时保留轻微脚步声和衣物摩擦声。整体听感轻快动感，呈现自信优雅的舞台表演氛围。
https://github.com/user-attachments/assets/69723e3c-f490-4878-8e87-013b8dff78a6
https://github.com/user-attachments/assets/5b4f78e8-0454-4f48-92b2-29d35c9a1685

[newstargo]

的确是提示词问题，谢谢。

[lanbomo]

@newstargo 请问一下您的机器配置是CUDA哪个版本呢？torch和flash-attn的版本呢？还有您4090的48G生成这个8秒视频花了多长时间？我也使用4090，24g和48g都试过，折腾了几天，硬是没弄成功，运行始终是报错。我想这可能是很少人关注的原因，因为让它成功运行起来就已经非常难了，不像ltx这些，comfyui装好就能直接用了。

[newstargo]

cuda 12.8, torch 2.8.0 flash-attn 2.8.2 , 生成时间2分钟左右，24G应跑不了，推理占15G合成时飚到33G，现在问题是增加--rewrite参数提示词重写，加载完qwen后在推理前就自动跳出，用deepseek手动重写的提示词，生成效果又是乱一团

[robingg1]

I'm not sure what's causing the blurriness, but there were indeed bugs in the rewrite path inside inference_nava
previously. The repo has been updated — we've added two end-to-end workflows under scripts/:

• scripts/inference_fp8_rewrite.sh — text-only rewrite path.
• scripts/inference_fp8_vl_rewrite.sh — auto-routes per sample: when an image_path is provided, the VL model
  captions the image and the caption is composed with your text prompt before rewrite; when only text is provided, it
  falls back to plain rewrite.

Please give them a try.

For the 33 GB peak during video synthesis — that's because VAE tiling wasn't enabled. Follow the flags in
scripts/inference_fp8.sh (--vae_tiling --vae_tile_size ... --vae_tile_stride ...); on our side peak stays under 24
GB with tiling on.

On environment setup — apologies, we've only been able to test on H/A-class GPUs due to limited hardware. Our
reference env is torch 2.8.0 + flash_attn 2.7.4.post1. I've just pushed a more robust install procedure — please
give it a try. If it still doesn't run, please open a separate issue tagged as environment-related with the
relevant error logs attached, and I'll follow up there.

[newstargo]

@robingg1 跑了几次中后段都会乱或一致性不一

idx0_.-av-0.mp4

idx0_.-av-01.mp4

[Captioner] IN image: E:/13.png
[Captioner] Done in 10.5s (84 tokens)
[Captioner] OUT (116 chars): 一位身着金色单肩亮片礼服的女子优雅站立，黑色长发垂落，佩戴珍珠项链与耳环，右手轻抚腰间，左 手叉腰，面带微笑直视镜头。背景是鲜红色墙面，悬挂一幅现代几何抽象画，光线柔和温暖，整体氛围奢华而静谧，仿佛在高端晚宴或艺术展览现场定格的瞬间。
[Captioner] Offloaded to CPU
[Compose] OUT (125 chars): 一位身着金色单肩亮片礼服的女子优雅站立，黑色长发垂落，佩戴珍珠项链与耳环，右手轻抚腰间，左手 叉腰，面带微笑直视镜头。背景是鲜红色墙面，悬挂一幅现代几何抽象画，光线柔和温暖，整体氛围奢华而静谧，仿佛在高端晚宴或艺术展览现场定格的瞬间。 女子对着镜头跳舞
[Rewriter] Onloaded to cuda:0
[Rewriter] IN (125 chars): 一位身着金色单肩亮片礼服的女子优雅站立，黑色长发垂落，佩戴珍珠项链与耳环，右手轻抚腰间，左手叉腰，面带微笑直视镜头。背景是鲜红色墙面，悬挂一幅现代几何抽象画，光线柔和温暖，整体氛围奢华而静谧，仿佛在高端晚宴或艺术展览现场定格的瞬间。 女子对着镜头跳舞
[Rewriter] target pairs: 0
[Rewriter] Generating attempt 1/5 (input tokens: 1100)...
[Rewriter] Done in 28.5s (241 tokens, =0)
[Rewriter] OUT (332 chars): 这是一段电影机质感的视频，画面中有一位身着金色单肩亮片礼服的女子优雅站立，黑色长发垂落，佩戴珍珠项链与耳环，右手轻抚腰间，左手叉腰，面带微笑直视镜头。随后她缓缓转身，以流畅的舞步开始轻盈旋转，身体随节奏微微起伏，右脚点地、左脚前移，动作舒展而富有韵律，双手自然摆动，指尖微抬如在空中划出弧线。背景为鲜红色墙面，悬挂一幅现代几何抽象画，光线柔和温暖，映照在她的衣料上泛起细微光泽。整个场景如同高端晚宴或艺术展览现场的定格瞬间，随着舞蹈推进，镜头采用稳定的中近景跟拍，从侧前方平移缓慢推进，聚焦于她的面部表情与肢体动作，捕捉微笑渐深、眼神流转的细节。画面中没有人物对白，也没有任何旁白，仅以环境的微光与脚步声衬托出奢华而静谧的氛围，整体节奏舒缓，充满优雅与沉静的张力。