# GPT-OSS-20B Q6_K on 11GB: Auto-Fit OOM Diagnosis

## The Problem

GPT-OSS-20B (20.9B/3.6B active MoE, Q6_K, 12 GB GGUF) failed to load with:
```
E ggml_backend_cuda_buffer_type_alloc_buffer: allocating 6839.26 MiB on device 0: cudaMalloc failed: out of memory
E llama_model_load: error loading model: unable to allocate CUDA0 buffer
```

Root cause was the default config pattern used for smaller MoE models:
- `-ngl 99` — forces all layers to GPU
- `--n-cpu-moe 10` — tries to move first 10 layers' experts to CPU
- `--mlock` — locks memory pages

These flags together set `tensor_buft_overrides` which **disables auto-fit**. With auto-fit disabled and -ngl 99 overriding everything, the 12 GB file must fit entirely on 11 GB VRAM — impossible.

## Diagnostic Output

```
0.00.808.462 W common_fit_params: failed to fit params to free device memory: n_gpu_layers already set by user to 99, abort
0.01.465.339 W warning: failed to mlock 626388992-byte buffer: Cannot allocate memory
0.01.465.888 E ggml_backend_cuda_buffer_type_alloc_buffer: allocating 6839.26 MiB on device 0: cudaMalloc failed: out of memory
0.01.465.893 E alloc_tensor_range: failed to allocate CUDA0 buffer of size 7171482880
0.01.597.737 E llama_model_load: error loading model: unable to allocate CUDA0 buffer
```

Key tell: "n_gpu_layers already set by user to 99, abort" — auto-fit is running and bailing because -ngl is set.

## The Fix

Remove `-ngl`, `--n-cpu-moe`, and `--mlock`. Add `--no-kv-offload` to move KV cache to CPU. Let auto-fit handle the GPU/CPU split.

**Working config:**
```
/usr/local/bin/llama-server-sm75 \
  -m /models/downloads/gpt-oss-20b-Q6_K.gguf \
  --port ${PORT} --host 127.0.0.1 \
  --jinja --no-mmap \
  --prio 2 --poll 30 --no-cont-batching --timeout 300 \
  --threads 16 --threads-batch 32 --parallel 1 \
  -c 131072 \
  --batch-size 1024 --ubatch-size 256 \
  --cache-type-k q8_0 --cache-type-v q8_0 \
  --flash-attn on --no-host -fitt 1024 \
  --no-kv-offload \
  --temp 0.7 --top-p 0.95 --min-p 0.05 --top-k 40 \
  --metrics
```

**Result:** 9,872 MiB VRAM (1,130 MiB free). 131K context, q8_0 cache on CPU.

## General Rule

When a GGUF file is close to or exceeds VRAM capacity:
1. Do NOT set `-ngl` or `--n-cpu-moe` — these disable auto-fit
2. Do add `--no-kv-offload` to save 2-5 GB of GPU VRAM
3. Let auto-fit decide the GPU/CPU split automatically
4. This applies especially to models with large GGUF files (10+ GB on 11 GB cards)