# Fleet Benchmarks — Jun 30, 2026

RTX 2080 Ti 11 GB, llama.cpp b9743 sm75, all measurements via llama-swap. 500-token generation, q8_0 KV cache.

## Current Fleet (6 models)

| Model | Quant | Ctx | Load | Gen t/s | Gen2 t/s | PP t/s | VRAM | Free | Flags notes |
|-------|-------|:---:|:----:|:-------:|:--------:|:------:|:----:|:----:|-------------|
| Qwythos-9B-MTP-Q6 | Q6_K | 131K | 0s | 95.2 | 81.9 | 567 | 10,558 | 445 | MTP n=4; 131K (up from 32K) dropped speed from 111→95 |
| Ornith-9B-Q5 | Q5_K_M | 64K | 15s | 69.2 | 70.2 | 236 | 7,186 | 3,817 | Q5 upgrade enabled 64K with on-GPU KV (was Q6+32K at 76 t/s) |
| Qwen3.6-35B-MTP | Q4_K_M | 230K | 60s | 60.0 | 59.1 | 65 | 10,936 | **67** | **Critically tight** — 67 MiB free, compute-buffer OOM risk |
| Gemma-26B-200K | Q4_K_XL | 230K | 36s | 56.7 | 57.9 | 109 | 10,502 | 501 | Stable; temp=0.2 should be 1.0 per Gemma docs |
| GLM-4.7-Flash | Q4_K_M | 200K | 47s | 30.4 | 30.8 | 72 | 10,332 | 671 | Non-REAP variant (18 GB file); slower than REAP's 53 t/s |
| Ornith-35B-Q6-MTP | Q6_K | 131K | 65s | 21.2 | 25.1 | 33 | 9,894 | 1,109 | MTP improves warm→cold (~19%); ncm=30 optimal |

## Key Findings

1. **Qwen36 at 67 MiB free** needs `--no-warmup` or lower `-fitt` — it will OOM on compute buffer spike
2. **Gemma temp mismatch** — 0.2 vs recommended 1.0 — likely suppressing quality
3. **GLM non-REAP** is 30 t/s vs REAP's 42-54 t/s — the 18 GB file vs 14 GB is the difference
4. **Qwythos-9B** at 131K is still the fastest model at 95 t/s, down from 111 at 32K (KV cache overhead)
5. **Ornith-35B** MTP shows warm-up benefit: 25.1 t/s warm vs 21.2 cold = +18%