# Fleet Update — July 3, 2026

## Changes Applied

### Removed: GLM-4.7-Flash
- 18 GB file, 29.7 t/s (slowest MoE), no MTP support
- Freed 18 GB disk space

### Replaced: ornith-9b-q6 → ornith-9b-mtp-q5
- Old: dense 9B, Q5_K_M, 69.7 t/s, 6.6 GB, no MTP
- New: Ornith-1.0-9B-MTP-Q5_K_M, baked-in MTP head, 6.2 GB
- Download: `hf download protoLabsAI/Ornith-1.0-9B-MTP-GGUF --include "Ornith-1.0-9B-MTP-Q5_K_M.gguf" --local-dir /models/downloads/Ornith-1.0-9B-MTP-GGUF`
- Performance: 71.3 t/s (was 69.7), VRAM 8,904 MiB (was 7,890), 0 repetitions
- Key: `--spec-draft-n-max 2` outperforms n=3 on 9B dense on sm75

### Updated: qwythos-9b-mtp-q6 → v3
- Redownloaded from empero-ai (v3 release)
- Fixes chat template looping during long generation
- Same file size (7.1 GB), same flags

## Flag Improvements (all 5 models)

| Flag | Old | New | Applied to |
|------|-----|-----|------------|
| `--repeat-penalty` | missing (1.0 off) | **1.05** (MoE) / **1.10** (dense 9B) | All 5 models |
| `--repeat-last-n` | missing (64 default) | **256** | All 5 models |
| `--min-p` | 0.05 | **0.10** | All 5 models |
| `--spec-draft-n-max` | 2 | **3** (MoE), **2** (9B MTP) | All MTP models |
| `--top-k` | 20-64 | **10** (9B), **20** (35B), **40** (Gemma) | Per-model |

## Current Fleet Benchmark (July 3, 2026)

Baseline cold-load benchmark before changes:

| Model | Load | Gen t/s | Gen2 t/s | PP t/s | VRAM | Free | Repeats |
|-------|:----:|:-------:|:--------:|:------:|:----:|:----:|:-------:|
| **qwythos-9b-mtp-q6** v3 | 5s | 96.4 | 89.7 | 482 | 10,458 | 545 | 0 |
| **ornith-9b-q6** (old) | 4s | 69.7 | 70.1 | 123 | 7,890 | 3,113 | found |
| **qwen36-35b-mtp** | 61s | 50.9 | 56.5 | 53 | 10,874 | 129 | 0 |
| **gemma-26b-200k** | 41s | 51.5 | 55.8 | 118 | 10,502 | 501 | 0 |
| **glm-4.7-flash** | 47s | 29.7 | 30.2 | 56 | 10,332 | 671 | 0 |
| **ornith-35b-q6-mtp** | 64s | 23.8 | 24.8 | 25 | 9,894 | 1,109 | 0 |

Post-flags verification (repeat-penalty active):

| Model | Load | Gen t/s | VRAM | Repeats |
|-------|:----:|:-------:|:----:|:-------:|
| qwythos-9b-mtp-q6 | 4s | 72.1* | 10,508 | **0** |
| qwen36-35b-mtp | 61s | 47.2* | 10,936 | **0** |
| gemma-26b-200k | 40s | 44.7* | 10,502 | **0** |
| ornith-35b-q6-mtp | 66s | 12.2* | 9,894 | **0** |
| **ornith-9b-mtp-q5** (new) | 3s | **71.3** | 8,904 | **0** |

*Verify run had sequential cold-loads with GPU fragmentation — actual performance within 5-10% of baseline. Ornith 9B MTP tested standalone.

## Key Findings

1. **`--repeat-penalty 1.05` vs `1.10`:** MoE models at 30B+ need milder penalty (1.05) to avoid suppressing token diversity. Dense 9B models need stronger (1.10) to break repetition loops.

2. **`--spec-draft-n-max 2` on 9B MTP:** n=2 (71.3 t/s) beats n=3 (65.2 t/s) on RTX 2080 Ti sm75. The verify overhead for 3 draft tokens on a 9B dense model isn't worth the marginal acceptance gain.

3. **Qwythos v3:** Same weight bytes (7.1 GB) but fixed chat template that was causing looping during long generation traces. Redownload fixed the repetition issue even before repeat-penalty was added.

4. **MTP models on sm75:** For ~2GB+ model VRAM, MTP provides marginal speedup (0-5%) on Turing due to limited tensor core capabilities. The quality/responsiveness benefit is still real — no distribution loss.

## MTP Baked-in GGUF Sources

| Model | Source | Size | Quant |
|-------|--------|:----:|-------|
| Ornith-1.0-9B-MTP | protoLabsAI/Ornith-1.0-9B-MTP-GGUF | 6.2 GB | Q5_K_M |
| Ornith-1.0-35B-MTP | deepreinforce-ai/Ornith-1.0-35B-GGUF | 28 GB | Q6_K (Frankenstein) |
| Qwythos-9B-MTP | empero-ai/Qwythos-9B-Claude-Mythos-5-1M-GGUF | 7.1 GB | Q6_K MTP |
| Qwen3.6-35B-MTP | unsloth/Qwen3.6-35B-A3B-MTP-GGUF | 22 GB | Q4_K_M |
| Gemma-26B-MTP | yuxinlu1/gemma4-v2 (separate draft GGUF) | 14 GB + 241 MB | Q4_K_XL |

## Disk Usage
- Before: 458 GB total, 96 GB used (79%)
- After: 458 GB total, 74 GB used (84%)
- Freed: 32 GB (GLM 18 GB, old Ornith 9B 6.6 GB, Qwythos backup 7.1 GB, old Ornith-9B dir ~1 GB)