# Ornith-1.0-35B Q6_K MTP — Benchmark (2026-06-27)

**Hardware:** RTX 2080 Ti 11 GB, Intel Xeon E5-2697A v4 @ 2.60 GHz, 64 GB RAM
**Binary:** llama-server-sm75 (b9743, CUDA sm_75)
**Model:** skinnyctax/Ornith-1.0-35B-Q6_K-Frankenstein-MTP-GGUF → `ornith-1.0-35b-Q6_K-MTP-final.gguf` (28 GB)
**Prompt:** "Write a Python function is_prime(n)." (4 new tokens after cache, generating ~140 tokens)
**Spec:** `--spec-type draft-mtp --spec-draft-type-k q8_0 --spec-draft-type-v q8_0`

## n-cpu-moe Sweep (spec-draft-n-max 4)

| n-cpu-moe | VRAM | Avg TG | Status |
|-----------|------|--------|--------|
| 48 | 3.6 GB | 27.9 t/s | ✅ All experts CPU |
| **30** | **9.9 GB** | **33.7 t/s** | **⭐ Best** — 10 expert layers on GPU |
| 29 | 10.5 GB | 32.9 t/s | ✅ Loads but slower (diminishing returns) |
| 28 | — | — | ❌ OOM (MTP context, 568 MiB) |
| 24 | — | — | ❌ OOM (model loader) |

**Optimal: `--n-cpu-moe 30`** — 10 of 40 expert layers on GPU. Higher (29) loads but slower due to VRAM pressure on compute buffers. Lower (32+) leaves speed on table.

## spec-draft-n-max Sweep (n-cpu-moe 30)

| n_max | Avg TG | Notes |
|-------|--------|-------|
| 1 | 31.4 t/s | Highest acceptance per position, but one token at a time |
| 2 | 33.0 t/s | Good balance |
| 3 | 32.1 t/s | |
| **4** | **33.7 t/s** | **⭐ Best** — 4-token drafts with ~95% acceptance |

## MTP Acceptance Stats (n-cpu-moe 30, n_max=4, steady state)

| Run | TG | MTP Accept Rate | Mean Accepted Len |
|-----|-----|-----------------|-------------------|
| 1 | 33.4 t/s | ~87% | 4.46 |
| 2 | 34.5 t/s | ~95% | 4.80 |
| 3 | 33.1 t/s | ~95% | 4.66 |

**Steady-state acceptance: ~95%, mean accepted length: ~4.7 tokens per draft step.**
Per-position rates (steady): (0.979, 0.907, 0.897, 0.887) — exceptionally high for all 4 positions.

## Comparison vs Q8

| Metric | Q8 (no MTP) | Q6 MTP | Δ |
|--------|-------------|--------|---|
| File size | 35 GB | 28 GB | -20% |
| VRAM | 9.1 GB | 9.9 GB | +9% |
| TG (generation) | 18.9 t/s | **33.7 t/s** | **+78%** |
| MTP acceptance | n/a | 95% | — |

## 32-thread Attempt (OOM)

`--threads 32 --threads-batch 32` with n-cpu-moe 30 caused CUDA OOM (needed 8546 MiB buffer). The different thread count changes the compute graph allocation pattern, requiring more contiguous VRAM. Stick with `--threads 16 --threads-batch 32` for this card.

## Ceiling Analysis

Hardware bottleneck: CPU↔GPU PCIe bandwidth for the 30 offloaded expert layers (75% of model). MTP self-spec decoding at 95% acceptance already captures most of the theoretical speedup from speculative decoding. To go faster: bigger GPU (more VRAM = more expert layers on-device) or faster system RAM (DDR5 vs DDR4-2400).

## Config

```yaml
cmd: "/usr/local/bin/llama-server-sm75 -m /models/downloads/Ornith-1.0-35B-GGUF/ornith-1.0-35b-Q6_K-MTP-final.gguf --port ${PORT} --host 127.0.0.1 --jinja --reasoning off -ngl 99 --no-mmap --mlock --prio 2 --poll 30 --no-cont-batching --timeout 300 --threads 16 --threads-batch 32 --parallel 1 -c 131072 --batch-size 2048 --ubatch-size 512 --cache-type-k q8_0 --cache-type-v q8_0 --flash-attn on --no-host -fitt 1024 --n-cpu-moe 30 --no-kv-offload --spec-type draft-mtp --spec-draft-n-max 4 --spec-draft-type-k q8_0 --spec-draft-type-v q8_0 --temp 0.6 --top-p 0.95 --min-p 0.05 --top-k 20 --metrics"
aliases: [ornith, ornith-35b, ornith-mtp]
```

## Source

- **Model:** https://huggingface.co/skinnyctax/Ornith-1.0-35B-Q6_K-Frankenstein-MTP-GGUF
- **Base:** https://huggingface.co/deepreinforce-ai/Ornith-1.0-35B
- **Graft method:** 20 MTP head tensors surgically grafted from Qwopus3.6 (same qwen35moe architecture), metadata patched (block_count 40→41, nextn_predict_layers=1)