# 2026-07-03 — APEX & UD Experiment Results on sm75

## What We Tried

### Qwen3.6-35B-A3B

| Quant | Format | Size | Gen t/s | VRAM | Verdict |
|---|---|---|---|---|---|
| **Q4_K_M** (original) | K-quant | 22 GB | **50.9** | 10,874 | ✅ **Fastest** — kept |
| **APEX I-Compact MTP** | I-quant | 17.3 GB | 46.8 | 9,710 | ❌ Slower, same quality |
| **UD-IQ4_NL** (Unsloth Dynamic 2.0) | Dynamic IQ4 | 18.5 GB | 38.7 | 10,036 | ❌ **32% slower** — worst |

### Ornith-35B

| Quant | Format | Size | Gen t/s | Load | Verdict |
|---|---|---|---|---|---|
| **Q6_K-MTP** (Frankenstein graft) | K-quant | 28 GB | 24.8 | 64s | ❌ Replaced |
| **APEX I-Compact MTP** ⭐ | I-quant | 17 GB | **28.5** | **12s** | ✅ **Kept** — 5.3x faster load, 262K ctx |

### Ornith-9B (dense, MTP variant)

| Quant | Format | Size | Gen t/s | Verdict |
|---|---|---|---|---|
| **Q5_K_M MTP** | K-quant | 6.2 GB | **71.3** | ✅ Fastest — restored after experiment |
| Q6_K MTP | K-quant | 7.6 GB | 65.6 | ❌ Slower, not worth 1 GB extra |

## Key Findings

1. **I-quants are slower on sm75 (Turing).** APEX I-Compact lost 5 t/s vs Q4_K_M despite being 5 GB smaller. UD-IQ4_NL lost 18 t/s (32%). The non-uniform bit layout of I-quants doesn't align with Turing's compute patterns.

2. **Exception: MoE models bottlenecked by expert swapping.** Ornith-35B at Q6 (28 GB) was so bottlenecked by CPU-GPU expert transfer that the smaller APEX I-Compact (17 GB) was faster despite the I-quant compute tax. The 11 GB saved in file size reduced PCIe pressure enough to overcome the format penalty.

3. **Decision rule:** If gen < 30 t/s and VRAM > 10,000 MiB → try smaller I-quant. If gen > 30 t/s → keep K-quant.

4. **APEX I-Compact quality is NOT better than Q4_K_M.** From the APEX technical report: I-Compact PPL 6.669 vs standard Q4_K_M ~6.610. The APEX advantage is smaller file size and MTP baked in, not quality. Don't claim "same or better quality" without citing the PPL numbers.

5. **Always benchmark claims.** The "APEX I-Compact ≈ Q4_K_M quality" claim from the marketing page needed real benchmark data to verify — which showed it's actually slightly worse. The user rightfully pushed back on unsubstantiated assertions.

## Sources

- APEX Technical Report: https://github.com/mudler/apex-quant/blob/main/paper/APEX_Technical_Report.md
- Unsloth Qwen3.5 GGUF Benchmarks: https://unsloth.ai/docs/models/qwen3.5/gguf-benchmarks
- mudler/Qwen3.6-35B-A3B-APEX-MTP-GGUF: https://huggingface.co/mudler/Qwen3.6-35B-A3B-APEX-MTP-GGUF
- unsloth/Qwen3.6-35B-A3B-MTP-GGUF: https://huggingface.co/unsloth/Qwen3.6-35B-A3B-MTP-GGUF