perf(Vulkan): persistent VkPipelineCache (cold-start fix) - #9
Conversation
…tch)
Adds a disk-backed VkPipelineCache for the Vulkan backend - the Vulkan analog
of the Metal binary-archive PSO cache. On this machine (RTX 2070) a cold
first inference took 7-42s depending on weights; after the first run every
later launch loads precompiled PSOs (verified: 1.14 MB cache, 'loaded
1140086 bytes', warm 0.42s, same 33 notes), independent of NVIDIA/AMD driver
caches and suitable for shipping a pre-warmed cache in packages.
- cmake/patches/ggml-vulkan-pipeline-cache.{patch,md}: pure ggml-vulkan.cpp
patch (v0.19.0), verified clean via git apply --check on a fresh clone
- cmake/Dependencies.cmake: apply it after ggml FetchContent (idempotent)
- env: GGML_VK_PIPELINE_CACHE_PATH / GGML_VK_DISABLE_PIPELINE_CACHE /
GGML_VK_PIPELINE_CACHE_DEBUG; default <user-cache>/game_ggml_vk_pipeline.cache
- .gitattributes: force LF for *.patch (git apply requirement on Windows)
- README: document the new cold-start behavior
|
@coderabbitai full review |
|
Warning Review limit reached
Next review available in: 59 minutes Limit details: You’ve used all 1 included review currently available under your plan. You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (5)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
1 participant
动机
Vulkan 冷启动是 GPU 路径唯一硬伤：本机（RTX2070）首次推理冷启 7-42s（取决于权重集合），
依赖 NVIDIA 驱动级 shader 缓存才能暖。AMD/Intel/其它驱动与跨进程场景无保证。
改动
game.cpp 自有 ggml patch（对应 Metal binary-archive 思路）：
（device init 读入 → createComputePipeline 复用 → compute 完成后按脏标记写回）
本地实测（RTX2070 / Vulkan / Q8 n1 10s）
已验证