# Gemma 4 12B Unified — Official Specs (from Hugging Face Model Card)

Source: https://huggingface.co/google/gemma-4-12B-it
Retrieved: 2026-06-04

## Architecture

| Property | Value |
|----------|-------|
| Parameters | 11.95B (dense) |
| Layers | 48 |
| Sliding Window | 1024 tokens |
| Context Length | **256K tokens** (official) |
| Vocabulary | 262K |
| Modalities | Text, Image, Audio |
| Attention | Hybrid — sliding window + global (final layer always global). Global layers use unified K+V + p-RoPE |
| Type | Encoder-free (unified) — no separate vision/audio encoders. Raw image patches and audio waveforms projected directly into embedding space via linear layers |

## Recommended Sampling Parameters (Google)

- `temperature = 1.0`
- `top_p = 0.95`
- `top_k = 64`

Note: For agent/tool-calling workflows, `temperature=0.2` is acceptable for determinism. The `top_k=64` recommendation should still be followed.

## Thinking Mode

- Native thinking via `<|think|>` token
- In llama.cpp: `--reasoning off` suppresses thinking trace (all output in `content`)
- Without `--reasoning off`: thinking trace in `reasoning_content`, answer in `content`

## Benchmark Comparison (Google Official)

| Benchmark | Gemma 4 12B | Gemma 3 27B (no think) |
|-----------|-------------|----------------------|
| MMLU Pro | 77.2% | 67.6% |
| AIME 2026 (no tools) | 77.5% | 20.8% |
| LiveCodeBench v6 | 72.0% | 29.1% |
| GPQA Diamond | 78.8% | 42.4% |
| MMMU Pro (Vision) | 69.1% | 49.7% |

## Long Context Performance (MRCR v2, 8-needle @ 128K)

Gemma 4 12B: **43.4%** (vs Gemma 3 27B: 13.5%)

## Context Size — Important

The GGUF may report conservative metadata (131072) but Google's official model card table explicitly lists "Context Length: 256K tokens" for 12B Unified. The discrepancy is a quantizer artifact — the model's architecture (p-RoPE + hybrid attention + unified K+V on global layers) supports 256K natively.

When adjusting context size in llama-swap, always:
1. Edit `~/.config/llama-swap/config.yaml`
2. **Restart llama-swap** (does NOT hot-reload config)
3. Verify flags with `ps aux | grep [l]lama-server`
4. Check VRAM with `nvidia-smi` after loading
5. If OOM, reduce in 32K increments until stable

## Confirmed Benchmarks (RTX 2080 Ti 11GB, Q4_K_M)

| Metric | 131K context | 200K context |
|--------|-------------|-------------|
| Generation speed | 53.6 tok/s | 53.3 tok/s |
| Prompt processing | 348 t/s | ~1300 t/s (2K prompt) |
| VRAM | 7.3 GiB | 8.1 GiB |
| Headroom | 3.7 GiB | 3.2 GiB |

200K context confirmed operational at 53.3 tok/s sustained. 256K expected to fit (~8.5-8.8 GiB estimate) but untested.