# Qwen-AgentWorld-35B-A3B-UD-Q6_K --n-cpu-moe Sweep

**Date:** 2026-06-27
**GPU:** NVIDIA RTX 2080 Ti 11GB
**System RAM:** 62 GB
**Binary:** `/usr/local/bin/llama-server-sm75` (b9743)
**File:** 28 GB, 35B-A3B MoE (Qwen3.5 base, UD-Q6_K)
**Tested values:** 32, 28, 24, 20, 16
**Prompt:** "Write a Python function is_prime(n)." — 100 tokens gen, 3 runs per value

## Results

| `--n-cpu-moe` | VRAM after load | TG (avg 3 runs) | PP (cached) | Status |
|---|---|---|---|---|
| 32 | 7,738 MiB | 19.91 t/s | 44.6 t/s | ✅ |
| **28** | **10,258 MiB** | **20.44 t/s** | **46.3 t/s** | **✅ sweet spot** |
| 24 | — | — | — | ❌ OOM on load |
| 20 | — | — | — | ❌ OOM on load |
| 16 | — | — | — | ❌ OOM on load |

## Interpretation

- **Sweet spot: `--n-cpu-moe 28`** — 2 layers of MoE experts on GPU (layers 28-39), rest on CPU. Fills 10.3 GB out of 11 GB, leaving ~700 MiB safety margin. Fastest generation at 20.44 t/s.
- **n-cpu-moe 32** — all 40 expert layers on CPU (0 on GPU). Slightly slower (19.91 t/s) due to more PCIe transfers. Uses less VRAM (7.7 GB).
- **n-cpu-moe 24** and below — OOM. The remaining non-expert weights (attention, shared expert, embeddings) plus compute buffers exceed 11 GB.
- Compared to Ornith-35B-Q8 (35 GB), AgentWorld-Q6 (28 GB) allows 2 more expert layers on GPU (n-cpu-moe 28 vs 32 for Ornith-Q8) due to the smaller file size freeing ~7 GB of combined RAM/VRAM pressure.

## Config Applied

Same pattern as Qwen3.6-35B and Ornith-35B:
- `-ngl 99 --no-mmap --mlock --no-kv-offload`
- `-c 131072 --batch-size 2048 --ubatch-size 512`
- `--cache-type-k q8_0 --cache-type-v q8_0 --flash-attn on`
- `-fitt 1024 --temp 0.6 --top-p 0.95 --min-p 0.05 --top-k 20`
- `--n-cpu-moe 28`