# GLM-4.7-Flash n-cpu-moe Sweep — June 2026

**Model:** GLM-4.7-Flash-Q4_K_M.gguf (18 GB, deepseek2 arch, 30B total / 4.7B active params)
**GPU:** RTX 2080 Ti 11GB (sm_75)
**Binary:** llama-server-sm75 b9743
**KV cache:** q8_0, `--no-kv-offload` (system RAM)
**Test prompt:** "Write a Python quicksort with comments. Return ONLY code." (17-40 tokens)
**max_tokens:** 256

## Results

| n-cpu-moe | VRAM | Gen t/s | Prompt t/s | Δ gen | Δ prompt | Notes |
|-----------|------|---------|------------|-------|----------|-------|
| 40 | 4,914 MiB | 25.6 | 26 | — | — | Stock config |
| 32 | 7,598 MiB | 29.3 | 85 | +14% | +227% | Modest gen gain, prompt jumps |
| 28 | 8,992 MiB | 28.8 | 86 | +13% | +231% | ~same as 32, more VRAM used |
| 24 | 10,334 MiB | 36.2 | 87 | +41% | +235% | **Sweet spot** — gen threshold |
| 20 | OOM | — | — | — | — | Crashes on model load |

## Interpretation

- **ncm=28→24 is a threshold** — gen jumps from 29 to 36 t/s, suggesting a critical batch of expert layers fits between these two values
- **ncm=32 and ncm=28 are nearly identical** — the extra 4 layers moved to GPU don't improve generation
- **ncm=24 leaves ~600MB headroom** on 11GB card, enough for compute graph buffers at 1024/256 batch/ubatch
- **KV cache in system RAM** (`--no-kv-offload`) decouples context length from VRAM — ncm=24 is safe at any context the model supports

## Current Config (llama-swap)

```yaml
glm-4.7-flash:
  cmd: "/usr/local/bin/llama-server-sm75 -m /models/downloads/GLM-4.7-Flash-Q4_K_M.gguf
    --port ${PORT} --host 127.0.0.1 --jinja --reasoning off
    -ngl 99 --no-mmap --timeout 300 --threads 16 --threads-batch 32
    --parallel 1 -c 200000 --batch-size 1024 --ubatch-size 256
    --flash-attn on --n-cpu-moe 24 --no-kv-offload
    --temp 1.0 --top-p 0.95 --min-p 0.01 --repeat-penalty 1.0 --top-k 40 --metrics"
```

## Test Method

Each ncm value tested by:
1. Killing any existing GLM server
2. Starting llama-server directly (not through llama-swap) with `terminal(background=true)`
3. Waiting for `/v1/chat/completions` endpoint to respond
4. Running 2 gen runs, recording timings from response `timings` dict
5. Recording VRAM via `nvidia-smi --query-gpu=memory.used`
6. Killing server, waiting 2s for VRAM release
7. Repeat with next ncm value

Total test time: ~15 minutes for 5 values.