Working recipes and measured benchmarks for serving large LLMs on pre-Ampere NVIDIA GPUs — Tesla V100 (sm_70) and RTX 2080 Ti (sm_75). vLLM forks, llama.cpp tensor parallelism, AWQ/MoE gotchas.
cuda benchmarks turing volta awq tensor-parallelism llm gpu-server llama-cpp vllm local-llm llm-inference tesla-v100 rtx-2080ti
-
Updated
Jul 22, 2026