GTX 1080 Ti + Xeon E5-2697A v4 running Qwen3.6-35B via llama-server-turbo
| Component | Details |
|---|---|
| CPU | Intel Xeon E5-2697A v4 — 16c / 32t, 2.6 GHz, AVX2, 40 MB L3 |
| GPU | NVIDIA GTX 1080 Ti — 11 GB GDDR5X, Compute Capability 6.1 |
| RAM | 64 GB DDR4 ECC |
| Storage | 458 GB NVMe SSD (23 GB used) |
| OS | Debian 13 (bookworm), Linux 6.12.85, OpenClaw 2026.5.2 |
Quantized MoE variant of Qwen3.6, 35B total params with ~3B active per token. Q4_K_M quantization keeps quality high while fitting comfortably on 11 GB VRAM.
| Property | Value |
|---|---|
| Format | Gguf (GGUF) |
| Quantization | Q4_K_M |
| Context | 256,000 tokens |
| VRAM Usage | ~10 GB (with -ngl 99) |
| Active Params | ~3B (MoE routing) |
Baseline prefill was ~420 tok/s (~15.2 s for 6.4k tokens). Adding --batch-size 4096 and --ubatch-size 1024 pushed prefill to ~605 tok/s (~10.7 s) — a ~43% improvement in prompt processing time.
llama-server-turbo
-m /models/downloads/Qwen3.6-35B-A3B-UD-Q4_K_M.gguf
--port 8081 --host 0.0.0.0
--cache-type-k turbo4 --cache-type-v turbo3
--n-cpu-moe 30 -ngl 99 --no-mmap --mlock
--jinja --chat-template-kwargs '{"enable_thinking": false}'
--flash-attn auto --prio 2 --poll 80
-c 256000 --ctx-size 256000
--threads 16 --threads-batch 32 --parallel 1
--batch-size 4096 --ubatch-size 1024
| Flag | Why |
|---|---|
-ngl 99 | Offload all layers to GPU (11 GB VRAM handles it) |
--cache-type-k turbo4 --cache-type-v turbo3 | TurboQuant KV cache — ~43% faster prefill |
--flash-attn auto | Flash attention enabled for compute 6.1+ (auto = safe for 1080 Ti) |
--batch-size 4096 --ubatch-size 1024 | Larger batch = faster prefill, no generation impact |
--threads-batch 32 --parallel 1 | Optimal thread/parallel balance for this CPU |
--prio 2 --poll 80 | Higher scheduler priority, 80% CPU poll = lower latency |
--chat-template-kwargs {"enable_thinking": false} | Disables thinking output — required for OpenClaw/Telegram compatibility |
--no-mmap --mlock | Lock model in RAM for consistent latency |
Managed via systemd at /etc/systemd/system/llama-server.service
sudo systemctl enable --now llama-server.service
curl http://127.0.0.1:8081/health
# → {"status":"ok"}
API endpoint: http://127.0.0.1:8081/v1/chat/completions
When the conversation summary grows to ~60k+ tokens, the full prompt re-processing can take 2+ minutes on the 1080 Ti. OpenClaw's default LLM timeout (30s) was too short, causing FailoverError: LLM request timed out.
Fix applied: Set models.providers.local-qwen.timeoutSeconds = 480 in ~/.openclaw/openclaw.json — gives the server 8 minutes to process large prompts.