# Cross-Provider Model Comparison & VRAM Fit Analysis

Compare GGUFs of the same model across different quant providers (Unsloth, Google official, LM Studio Community, bartowski, etc.) and determine VRAM fit for a specific GPU.

## When to use

- User asks "which version of model X is best?"
- Multiple providers have GGUFs of the same base model with different quant formats
- You need to know if a specific GGUF fits in a given GPU VRAM budget
- Comparing Unsloth UD-Q4_K_XL vs Google Q4_0 vs LM Studio Community variants

## Compare across providers

```bash
# Search for all repos offering a model
curl -s 'https://huggingface.co/api/models?search=gemma-4-12B-it-qat-gguf&sort=downloads&direction=-1'
```

Returns providers sorted by download count. Common providers for popular models:

| Provider | Quant style | Typical tag |
|----------|-------------|-------------|
| `unsloth/` | UD-Q4_K_XL (Unsloth Dynamic 2.0 mixed quant) | Best quality-per-byte |
| `google/` | Q4_0 (pure 4-bit, official) | Official baseline |
| `lmstudio-community/` | Q4_0 (repackaged) | Same as Google, different label |
| `bartowski/` | Multiple: Q2_K through Q8_0 | Wide quant choice |

## Get actual file sizes (xet storage)

Unsloth and many newer repos use **xet storage**. The `/api/models/<repo>/tree/main` endpoint (without `?recursive=true`) returns accurate `size` fields. The API response heading also shows xet usage.

```bash
# GET ACTUAL SIZES (works with xet)
curl -s 'https://huggingface.co/api/models/unsloth/gemma-4-26B-A4B-it-qat-GGUF/tree/main'

# Returns entries with path + size (bytes) — not redirect pointers.
# The non-recursive endpoint gives direct sizes; recursive=true may not.
```

**Filtering results (Python):**

```python
import json, subprocess

result = subprocess.run(
    ['curl', '-s', f'https://huggingface.co/api/models/{repo}/tree/main'],
    capture_output=True, text=True
)
data = json.loads(result.stdout)

# Separate main GGUFs from mmproj/mtp/auxiliary files
for f in data:
    p = f['path']
    size_gb = f.get('size', 0) / (1024**3)
    if p.endswith('.gguf'):
        if p.startswith('mmproj'):
            kind = 'mmproj'
        elif 'MTP' in p or 'mtp' in p:
            kind = 'mtp-drafter'
        else:
            kind = 'main'
        print(f'{size_gb:>8.2f} GB  [{kind}] {p}')
```

**Subdirectories:** Some providers (Unsloth) store MTP drafters in a `MTP/` subdirectory. Query them separately:

```bash
curl -s 'https://huggingface.co/api/models/unsloth/gemma-4-26B-A4B-it-qat-GGUF/tree/main/MTP'
```

## VRAM fit formulas

All values below are approximate. For a precise check, download and load the GGUF — reported VRAM from `nvidia-smi` is authoritative.

**GGUF file size ≈ model VRAM** (no KV cache overhead considered yet):

| GPU | VRAM | Max GGUF size with headroom |
|-----|------|----------------------------|
| RTX 2080 Ti | 11 GB | ~8.5-9 GB (leaves ~2 GB for KV cache/overhead) |
| GTX 1080 Ti | 11 GB | ~8.5-9 GB |
| RTX 3070 | 8 GB | ~6 GB |
| RTX 4090 | 24 GB | ~20 GB |

**KV cache overhead** (per 4K context):
- Dense model: ~model_dim * layers * 2 bytes * context / 1 GB
- MoE model: same formula per active layer (much less)
- Rule of thumb: add 1-2 GB for 32K context, 2-4 GB for 128K context

**MoE tip:** MoE models (like Gemma 4 26B-A4B or Qwen3.6-35B-A3B) can fit more parameters in the same VRAM because only active experts load at once. A 26B total / 4B active MoE at Q4_K_M (~10 GB) fits in 11 GB even though the raw param count suggests 13+ GB.

## Cross-provider comparison example

### Gemma 4 12B QAT

| Provider | File | Size | Quant style |
|----------|------|------|-------------|
| unsloth/ | gemma-4-12B-it-qat-UD-Q4_K_XL.gguf | 6.26 GB | UD (mixed) — best quality-per-byte |
| google/ | gemma-4-12b-it-qat-q4_0.gguf | 6.50 GB | Q4_0 (pure) |
| lmstudio-community/ | gemma-4-12B-it-QAT-Q4_0.gguf | 6.50 GB | Q4_0 (pure) |

### Gemma 4 26B MoE QAT

| Provider | File | Size | Quant style |
|----------|------|------|-------------|
| unsloth/ | gemma-4-26B-A4B-it-qat-UD-Q4_K_XL.gguf | 13.27 GB | UD (mixed) — best quality-per-byte |
| google/ | gemma-4-26B_q4_0-it.gguf | 13.45 GB | Q4_0 (pure) |
| lmstudio-community/ | gemma-4-26B-A4B-it-QAT-Q4_0.gguf | 13.45 GB | Q4_0 (pure) |

- Unsloth UD is SMALLER than pure Q4_0 AND higher quality per byte
- All three start from the same Google QAT checkpoint base
- Unsloth's additional MTP drafter is a separate ~0.24 GB file

## Publishing HF repo names as model IDs

When the user runs multiple GGUFs via **llama-swap** or **systemd services**, the same concept applies — but the model ID displayed to clients comes from `llama-server`'s `--alias` flag, not the file path. When you recommend a download, also record the future alias:

```yaml
# llama-swap config entry
- name: gemma-4-12b-qat           # alias
  model: /models/downloads/gemma-4-12B-it-qat-UD-Q4_K_XL.gguf
```

## Pitfalls

- **xet repos need the non-recursive tree API.** `?recursive=true` may return size=0 for xet-stored files. Use `/tree/main` (no recursive flag) for accurate byte sizes.
- **MMProj files are not the model.** A repo with a 13 GB GGUF and a 1 GB mmproj.gguf is 13 GB of model + 1 GB of vision encoder — the mmproj is only loaded when you need vision capabilities.
- **MTP drafters don't affect main model VRAM.** The 0.23 GB Q4_0 MTP drafter is tiny and doesn't change the ~2 GB KV cache headroom calculation.
- **Don't rely on `?local-app=llama.cpp` snippet for file sizes.** The local-app view shows quant labels but not sizes. Use the tree API for accurate byte counts.
- **MoE QAT models can be SMALLER than non-QAT.** Gemma 4 26B MoE at QAT UD-Q4_K_XL is 14 GB vs 16 GB non-QAT Q4_K_M. The QAT restructuring allows better compression alongside the quantization, not worse. Always check the actual file size before assuming.