# llama-cpp — Build Configs

## Build Selection Guide

Two llama-server builds on this machine:

| Build | Binary | Purpose |
|-------|--------|---------|
| sm75 (b9341) | `/usr/local/bin/llama-server-sm75` | Qwen 3.5/3.6 + Qwopus with MTP-Turbo. Supports turbo cache types. `--n-cpu-moe 128` |
| upstream (ac4cdde) | `/usr/local/bin/llama-server` | Gemma 4 QAT models. `--spec-type draft-mtp` |

**Key difference:** turbo3/turbo4 cache types require the sm75 build. Upstream does NOT support turbo cache types.

## Build Recipe (sm75 with CUDA 12.6 + GCC 13)

Built with: `GGML_CUDA=ON`, `GGML_CUDA_FA=ON`, `GGML_CUDA_GRAPHS=ON`, `GGML_CPU=ON`, `GGML_OPENMP=ON`, `GGML_NATIVE=ON`, `GGML_LLAMAFILE=ON`

```bash
git clone https://github.com/EsmaeelNabil/llama.cpp-mtp-turbo-quant.git
cd llama.cpp-mtp-turbo-quant
git checkout feat/mtp-turboquant-kv-cache
mkdir build && cd build
cmake .. -DGGML_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES="75" -DGGML_CUDA_FA=ON \
  -DGGML_CUDA_GRAPHS=ON -DGGML_NATIVE=ON -DGGML_LLAMAFILE=ON \
  -DCMAKE_BUILD_TYPE=Release
make -j$(nproc)
```

## Benchmark Results (RTX 2080 Ti 11GB, MTP draft=4, turbo KV, flash-attn)

| Rank | Model | tok/s | Context | VRAM | Notes |
|------|-------|-------|---------|------|-------|
| 1 | gemma-12b | **53.7** | 256K | 8.4GB | Dense, no MTP |
| 2 | heretic-35b | 48.8 | 250K | 5.0GB | MoE, daily driver |
| 3 | qwen36-35b-mtp | 48.6 | 250K | 5.7GB | MoE+MTP |
| 4 | qwen35-9b-mtp | 45.7 | 250K | 7.4GB | Dense+MTP |
| 5 | qwopus-9b | 45.8 | 175K | 8.5GB | Coding specialist |
| 6 | qwen36-35b (no MTP) | 36.1 | 150K | ~4GB | Baseline MoE |
| 7 | ds-flash-9b | 33.0 | 200K | 8.6GB | Fast prompt eval |
| 8 | gemma-26b | **35.4** | 128K | 9.0GB | MoE, n-cpu-moe=16 |

**Key takeaways:**
- draft=4, threads=16, batch=2048/512 is winning combo for 35B MoE
- gemma-12b is fastest in fleet (53.7 t/s) at 256K
- 256K context fits all models on 11GB
- MTP speed varies by prompt type (coding 45-48, blog 34-42 t/s)

## Model Downloads
```bash
# Find exact filename
python3 -c "from huggingface_hub import list_repo_files; [print(f) for f in list_repo_files('USER/REPO') if 'Q4_K_M' in f]"

# Download
hf download USER/REPO filename.gguf --local-dir /models/downloads/
```

**Pitfall:** GGUF filenames may differ from expectations. Always list repo files first.