# Fleet Audit — June 19, 2026

## Hardware
- GPU: RTX 2080 Ti 11GB (TU102, SM75)
- Driver: 550.163.01, CUDA 12.6
- RAM: 62GB DDR4
- CPU: Xeon E5-2697A v4
- llama.cpp builds: sm75 b9341 (MTP-Turbo), upstream ac4cdde (QAT)
- Proxy: llama-swap v226 on :9292

## Pre-Audit Fleet (7 models)
| Model | Role | VRAM | t/s | Ctx |
|-------|------|------|-----|-----|
| Qwen3.6-35B-A3B-MTP | Primary coder | 7.2 GB | 42-48.6 | 250K |
| Qwopus3.5-9B-Coder-MTP | 9B coder | ~8 GB | 45.8 | 175K |
| Qwen3.5-9B-MTP | Uncensored 9B | 7.5 GB | 45.7-126 | 250K |
| Gemma 4 12B Dense QAT | Fast generalist | 7.7 GB | 53.7 | 256K |
| Gemma 4 26B A4B MoE QAT | Big reasoning | 10.8 GB | 35.4 | 256K |
| Nex-N2-mini | Agentic 35B | 2.8 GB | 27 | 250K |
| Bonsai-8B Q1_0 | Edge classifier | ~2 GB | ~114 | 65K |

## Issues Found (P0)
1. **`--chat-template-kwargs` on ALL 7 models** — deprecated flag routing output to reasoning_content. Fixed to `--reasoning off`.
2. **`--min-p 0` on qwen35-9b-mtp** — disables min-p filtering entirely on uncensored model. Fixed to `0.05`.
3. **Missing `--min-p` on nex-n2-mini** — defaults to 0. Added `0.05`.
4. **Missing `--mlock` on bonsai-8b** — `--no-mmap` without `--mlock` allows weight swapping. Added.
5. **Missing `--top-k` on bonsai-8b** — defaults to 40. Added 20.
6. **Missing `--spec-draft-type-k/q` on gemma-12b and gemma-26b** — draft KV cache not explicitly set. Added.

## Redundancy Analysis
- **HIGH:** Qwopus 9B Coder → redundant to Qwen3.6-35B-MTP (primary coder)
- **HIGH:** Nex-N2-mini → redundant to Qwen3.6-35B-MTP (same MoE class, 27 vs 42 t/s)
- **MODERATE:** Gemma 26B → partial overlap with Qwen3.6 for reasoning
- **QUESTIONABLE:** Bonsai-8B → fork dependency (696 commits behind upstream), limited utility
- **UNIQUE:** Qwen3.5-9B (only uncensored), Gemma 12B (only dense big model)

## Post-Audit Fleet (3 models)
| Model | Role | File | VRAM | t/s | Ctx | Binary |
|-------|------|------|------|-----|-----|--------|
| qwen36-35b-mtp | Primary coder | 22 GB | 7.2 GB | 42-48.6 | 250K | sm75 |
| gemma-12b | Fast generalist | 6.3 GB | 7.7 GB | 53.7 | 256K | upstream |
| gemma-26b | Big reasoning | 14 GB | 10.8 GB | 35.4 | 256K | upstream |

**Disk freed:** 33.7 GB (Qwopus 7.1 + Qwen3.5 5.5 + Nex 20 + Bonsai 1.1)

## Critical Stability Notes
- All models use q8_0/q8_0 KV cache (not turbo3/turbo4) as permanent fix for Turing SM75 MTP hang bug (llama.cpp #9364, #9866). ~3% speed cost.
- Gemma 26B runs at 10.8/11 GB — only 200 MB headroom. Realistic max context is below 256K. Risk of OOM on concurrent requests.
- Gemma 4 uses `--spec-type draft-mtp` (separate draft model via `--model-draft`), NOT `--spec-type mtp` which crashes.