# Gemma 4 26B-A4B — Vision MoE Setup Reference

Model from the May 2026 onboarding on Dell Precision 5810 (GTX 1080 Ti 11GB, Xeon E5-2697A v4, 62GB RAM).

## Architecture

- **Type:** MoE — 128 total experts, **8 active per token**
- **Total params:** 26B (4B active equivalent)
- **Layers:** 30
- **Native context:** 262,144 (256K)
- **Vision:** ✅ SigLIP encoder via mmproj

## VRAM at Q4_K_M (15.6 GB file)

| Component | Location | Size |
|-----------|----------|------|
| Attention weights | GPU | ~1.57 GB |
| Expert weights | **CPU** | ~14.4 GB |
| KV cache | CPU (--no-kv-offload) | ~100 MB |
| Vision encoder | CPU (--no-mmproj-offload) | ~1.1 GB |
| **Total GPU** | | **~10.3 GB / 11 GB** |

## Flags Unique to Gemma 4

Unlike dense models or Qwen 35B MoE, Gemma 4 needs:

```bash
--n-cpu-moe 128                  # 128 experts total
--chat-template-kwargs '{"enable_thinking":false}'  # suppress thinking tokens
--mmproj /path/to/mmproj-F16.gguf  # vision encoder
--no-mmproj-offload               # keep vision on CPU (won't fit GPU)
```

**`--n-cpu-moe`** — The number here equals TOTAL experts (128), not active ones (8). For Qwen 35B-A3B, this is 64.

**Thinking suppression** — Without `--chat-template-kwargs`, every response starts with `<think>...</think>`. Same key as Qwen3/3.5/3.6 models.

**Vision CPU offload** — `--no-mmproj-offload` is CRITICAL. Without it, llama-server crashes with OOM loading the vision encoder. The vision encoder runs on CPU with negligible slowdown for single-image inference.

## Download

```bash
# Model
wget -O /models/llm/gemma4/gemma-4-26B-A4B-it-Q4_K_M.gguf \
  "https://huggingface.co/lmstudio-community/Gemma-4-26B-A4B-it-GGUF/resolve/main/gemma-4-26B-A4B-it-Q4_K_M.gguf"

# Vision projector
wget -O /models/llm/gemma4/mmproj-F16.gguf \
  "https://huggingface.co/lmstudio-community/Gemma-4-26B-A4B-it-GGUF/resolve/main/mmproj-F16.gguf"
```

## Speed

| Metric | Value |
|--------|-------|
| Generation | ~31 t/s |
| Prompt processing | ~54 t/s |
| Vision + text | ~40 t/s (single image) |

## Comparison: Gemma 4 vs Qwen 35B MoE

| Aspect | Qwen3.6-35B-A3B | Gemma 4 26B-A4B |
|--------|-----------------|-----------------|
| Active/token | 3 experts | 8 experts |
| `--n-cpu-moe` | 64 | 128 |
| Speed | ~12 t/s | ~31 t/s |
| Vision | ❌ | ✅ |
| Context | 128K | 256K (we use 128K) |
| VRAM on GPU | ~10 GB | ~10.3 GB |
| System RAM | ~20 GB | ~16 GB |
| GGUF size | ~22 GB | ~15.6 GB |