# 2080 Ti Fleet Benchmark Results (May 2026)

## Research Context

- **llama.cpp version:** b9341 (a578bb6b9) — EsmaeelNabil MTP-TurboQuant fork
- **GPU:** RTX 2080 Ti (11GB VRAM, SM75, 280W PL)
- **CPU:** 16C/32T Xeon, 62GB RAM
- **Test method:** llama-server via API, 80-token prompt eval + 200-token generation, non-streaming

## MTP KV Slot Boundary Bug (GitHub #23658)

Draft acceptance rate can collapse at specific context sizes due to KV slot alignment. Observed:
- 12032 ctx → 16% acceptance rate, 1.12x speedup
- 12288 ctx → 71% acceptance rate, 1.91x speedup
- Pattern recurs at larger context sizes

**Mitigation:** Test MTP explicitly at target context sizes. Adjust by ±1024 if speed seems wrong.

## All Configs Tested

### qwen36-35b-mtp (Qwen3.6-35B-A3B-MTP-Q4_K_M.gguf, 21.1 GB)

| Test | Context | Draft | KV Cache | Batch | Gen t/s | PE t/s | VRAM |
|------|---------|-------|----------|-------|---------|--------|------|
| Baseline | 150K | 3 | turbo3/turbo4 | 2048/512 | 33.5 | 70.4 | 8,980 |
| Draft=2 | 150K | 2 | turbo3/turbo4 | 2048/512 | 35.3 | 68.4 | — |
| q8_0 KV | 150K | 2 | q8_0/q8_0 | 2048/512 | 35.4 | 72.0 | 9,950 |
| **Small batch** | **150K** | 2 | **turbo3/turbo4** | **256/256** | **36.5** | **70.8** | **8,552** |
| 200K best | 200K | 2 | turbo3/turbo4 | 256/256 | 33.8 | 73.6 | 8,916 |
| 250K test | 250K | 2 | turbo3/turbo4 | 256/256 | 34.7 | 64.3 | 9,284 |

### qwen35-9b-mtp (Qwen3.5-9B-MTP-Q4_K_M.gguf, 5.5 GB)

| Test | Context | Draft | KV Cache | Batch | Gen t/s | PE t/s | VRAM |
|------|---------|-------|----------|-------|---------|--------|------|
| Best config | 150K | 2 | turbo3/turbo4 | 256/128 | 34.1 | 253.9 | 6,706 |
| Best config | 200K | 2 | turbo3/turbo4 | 256/128 | 33.6 | 252.1 | 6,870 |

### Existing Fleet (optimized flags)

| Model | Context | Draft | Batch | Gen t/s | PE t/s | VRAM |
|-------|---------|-------|-------|---------|--------|------|
| heretic-35b | 250K | 2 | 256/256 | 36.6 | 89.6 | 4,192 |
| heretic-35b (orig flags thr=32,d=3) | 250K | 3 | 256/256 | 37.0 | 80.7 | 4,192 |
| qwopus-9b | 150K | 2 | 256/128 | 37.4 | 285.6 | 8,088 |
| ds-flash-9b | 200K | 2 | 256/128 | 34.8 | 288.0 | 8,252 |
| gemma-26b | 131K | n/a | 2048/512 | 30.1 | 90.1 | 3,868 |

## Key Findings

1. **Draft=2 > Draft=3** — Lower draft count consistently produced equal or better generation speed on 11GB VRAM. Counterintuitive but repeatable across all MTP models tested.
2. **Small batches win** — Batch=256/256 for 35B MoE, batch=256/128 for 9B dense. Larger batches only help prompt eval but hurt generation on VRAM-constrained GPUs.
3. **turbo3/turbo4 = q8_0 speed, -1.4GB VRAM** — No reason to use q8_0 on 11GB cards.
4. **Gemma-26b improved 25→30 t/s** — Just by using turbo cache + optimal batch.