# Ornith-1.0-35B Q8_0 n-cpu-moe Sweep

**Date:** 2026-06-26
**Binary:** `/usr/local/bin/llama-server-sm75` (b9743)
**GPU:** RTX 2080 Ti (11 GB)
**RAM:** 62 GB
**Model:** `/models/downloads/Ornith-1.0-35B-GGUF/ornith-1.0-35b-Q8_0.gguf` (35 GB, Q8_0)

## Flags (shared across all runs)
```
--jinja --reasoning off -ngl 99 --no-mmap --mlock --prio 2 --poll 30 --no-cont-batching
--timeout 60 --threads 16 --threads-batch 32 --parallel 1 -c 8192
--batch-size 2048 --ubatch-size 512 --cache-type-k q8_0 --cache-type-v q8_0
--flash-attn on --no-host -fitt 1024 --no-kv-offload
--temp 0.6 --top-p 0.95 --min-p 0.05 --top-k 20 --metrics
```

## Results

| n-cpu-moe | VRAM used | Avg TG (3 runs) | Status |
|-----------|-----------|-----------------|--------|
| 40 | 2,572 MiB | 17.00 t/s | ✅ |
| 38 | 4,194 MiB | 17.48 t/s | ✅ |
| 36 | 5,826 MiB | 16.19 t/s | ✅ |
| **32** | **9,090 MiB** | **17.95 t/s** | **✅ fastest** |
| 28 | — | — | ❌ OOM (needs 11.8 GB) |
| 24 | — | — | ❌ OOM (needs 15.1 GB) |

## Key Takeaways

- **Sweet spot: `--n-cpu-moe 32`** — lowest value that loads = most experts on GPU = fastest
- VRAM jumps sharply at ncm=32 (9.1 GB) → below that it OOMs
- All working values produce similar throughput (16-18 t/s) — the bottleneck is non-expert attention layers on GPU
- ncm=36 had a consistent dip to 16.2 t/s (possibly noise from layer boundary alignment)
- The model has ~40 MoE layers, so ncm >= 40 means all experts on CPU