# 2080 Ti VRAM Reference (11 GB) — Tested Model Fit

Concrete VRAM measurements for every model on the fleet. Each figure was obtained by:
1. Starting the model via **llama-swap** with full production flags (including `--n-cpu-moe` for MoE models)
2. Reading `nvidia-smi --query-gpu=memory.used` after load completes
3. Running a short completion to verify the model works end-to-end

## Fit Table — All Models Tested June 14, 2026

| Model | File Size | VRAM Used | RAM Used | Speed | First Load | Status |
|---|---|---|---|---|---|---|
| **bonsai-8b** | 1.1 GB | ~2 GB | ~2 GB | ~114 t/s | ~5s | ✅ |
| **qwen35-9b-mtp** | 5.5 GB | 7.5 GB | ~6 GB | 126 t/s | ~10s | ✅ |
| **gemma-12b** | 6.3 GB | 7.7 GB | ~6 GB | 123 t/s | ~15s | ✅ |
| **qwopus-9b** | 7.1 GB | ~8 GB | ~6 GB | ~46 t/s | ~20s | ✅ |
| **gemma-26b** | 14 GB | **10.8 GB** | ~5 GB | 45 t/s | 43s | ⚠️ Tight |
| **nex-n2-mini** | 20 GB | **2.8 GB** | ~25 GB | 27 t/s | 67s | ✅ (`--n-cpu-moe 128`) |
| **qwen36-35b-mtp** | 22 GB | **7.2 GB** | ~22 GB | 42 t/s | 31s | ✅ (`--n-cpu-moe 128`) |

## Key Insights

**All 7 models fit on 11GB with `--n-cpu-moe`.** The two models previously listed as OOM (nex-n2-mini, qwen36-35b-mtp) work fine with MoE expert offloading. `--n-cpu-moe 128` keeps non-expert layers on GPU (~3 GB for MoE) while sending expert FC layers to CPU. This is the correct fix — NOT lowering `-ngl`.

**MoE VRAM breakdown (qwen36-35b-mtp):**
- Non-expert layers on GPU: ~3 GB (constant regardless of file size)
- KV cache (turbo, 2048 ctx): ~0.1 GB
- MTP draft model: ~0.5 GB (on-device)
- Total: ~7.2 GB — leaves 3.8 GB for context expansion

**gemma-26b at 10.8 GB is critically tight:** small completions OK, full 256K may OOM, concurrency=1 required.

**First request load times:** Small models 5-15s, medium 15-20s, large MoE 31-43s, largest (nex-n2) 67s. Subsequent requests within TTL (30-60 min) are instant.

## Preferred Local Model Order

1. **qwen35-9b-mtp** — 7.5 GB, 126 t/s, 250K ctx ✅ best balance
2. **gemma-12b** — 7.7 GB, 123 t/s, 256K ctx ✅ strong reasoning
3. **qwopus-9b** — 8 GB, Q6_K quality, 175K ctx ✅ coding specialist
4. **qwen36-35b-mtp** — 7.2 GB, 42 t/s, 250K ctx ⚠️ slower but strongest reasoning
5. **bonsai-8b** — 2 GB, 114 t/s, 65K ctx ✅ ultra-fast edge