# llama-swap — Troubleshooting

## Key Pitfalls

### `--listen` Flag in Systemd Unit Overrides Config
**Extremely subtle trap.** Changing `listen: 0.0.0.0:9292` in `config.yaml` and restarting via `systemctl` has NO effect if the systemd unit passes `--listen 127.0.0.1:9292` — the CLI flag takes precedence.

Always check BOTH files when troubleshooting bind address:
```bash
grep 'listen' ~/.config/llama-swap/config.yaml
grep 'listen' /etc/systemd/system/llama-swap.service
```
Fix the systemd unit with `sudo sed -i` + `sudo systemctl daemon-reload` + restart.

### Client Configs Drift from llama-swap Config
**Most common complaint.** llama-swap may serve 128K but Hermes config has `context_length: 65536`. Always run the cross-config sync check after any config change. Check `/running` for actual `-c` flag vs Hermes config.

### First Request After Switch = Cold Load (30-90s)
The model loads from disk into VRAM. Normal.

### Hermes Gateway Session Bloat Looks Like Slow Local Models
If direct `/v1/chat/completions` probes are fast but `hermes chat` takes minutes, check `~/.hermes/sessions/sessions.json` for high `last_prompt_tokens`. Fix with `/reset`.

### Model IDs Must Match
Between client requests and config keys. Old aliases like `qwen36-35b-ud` don't exist in llama-swap and silently cap context.

### TTL in Seconds
`1800` not `"30m"`.

### No Auth
UFW restricts LAN to CachyOS only.

### Docker Containers Blocked by UFW
When UFW is active with `deny (routed)`, Docker containers on compose bridge networks can't reach host services. Fix:
```bash
sudo ufw allow in on br-<network> to any port 9292 proto tcp
```

### Dense Models at High Context
Dense 14B Q4_K_M: ~7GB weights + ~5GB KV at 128K = OOM on 11GB. Must use `--no-kv-offload` or smaller context.

### llama.cpp Build Version Matters for VRAM
Newer builds (b1-3c7616b+) include strict `common_params_fit_impl` that aborts if projected VRAM > free. Fix: `--no-kv-offload`.

### MTP KV Slot Boundary Bug (GitHub #23658)
Draft acceptance rate can collapse at specific context sizes (256-token difference changed 16%→71%). Test target context sizes explicitly.

### Custom Fork Segfaults When Spawned by llama-swap
Two causes: missing LD_LIBRARY_PATH (fix via systemd drop-in) or compiler/runtime mismatch (reduce to minimal flags, or build from source on this machine).

### YAML `>` Folded Block Adds Trailing Newline
Use single-line `cmd:` for custom fork entries.

### Community GGUF May Strip MTP Metadata
Weight tensors survive but `nextn_predict_layers` key may be missing. Check HF `config.json` for MTP config before assuming GGUF is MTP-capable.

### `--flash-attn off` requires f16 KV cache
Error with "V cache quantization requires flash_attn" when combining `--cache-type-v q8_0` and `--flash-attn off`.

### turbo3/turbo4 Require sm75 Build
Upstream binary doesn't support turbo cache types. Only sm75/MTP-Turbo fork supports them.

### `env` in cmd Does NOT Work
Go's os/exec, not shell. Use systemd `Environment=` drop-in or wrapper script.

### Hermes Desktop Can't See Local Models
Add `custom:local` provider pointing to server's llama-swap URL in Desktop's config.yaml. Set `fallback_providers: [custom:local]`.

### Open WebUI Ollama Errors During Startup
The WebUI polls llama-swap every ~11s. Errors during cold load are harmless — resolve once model responds.

### host.docker.internal May Not Resolve
Use `extra_hosts: - "host.docker.internal:host-gateway"` or hardcode the bridge gateway IP.

## Common Fix Flow
1. `systemctl status llama-swap` — is it running?
2. `curl :9292/v1/models` — are models registered?
3. `curl :9292/running` — check actual process args
4. `nvidia-smi` — enough free VRAM?
5. `scripts/cross-ref-audit.py` — paths match?
6. Check `journalctl -u llama-swap` for errors

## Reference Files
- `references/ufw-docker.md` — Docker containers blocked by UFW
- `references/vram-breakdown-dense-14b.md` — Dense 14B VRAM numbers
- `references/mtp-model-downloads.md` — Unsloth MTP repos
- `references/fleet-benchmarks-2080ti-may2026.md` — Full benchmark results
- `references/bonsai-prismml-setup.md` — Bonsai-8B fork integration
- `references/hermes-migration-cleanup.md` — Archive layout for cutover