GTX 1080 Ti Local Model Fleet Report

Updated after lineup cleanup + 60k–100k context expansion. Removed from active lineup: DeepSeek-R1-8B, Ministral-3-8B-Reasoning, Gemma-3-12B.

Fastest decode tested

19.41 tok/s

Qwen3.6-35B-A3B @ 100k, batch 4096, ubatch 1024

Best dense high-context decode

9.21 tok/s

Gemma-4-26B @ 100k, batch 2048, ubatch 512

Best high-context fit rescue

Qwen3.5-9B

100k only became stable after shrinking to 1024/256

Main conclusion

MoE wins

Big MoE models beat several smaller dense models on this 11GB Pascal card.

Current lineup

Model	Size (GiB)	Class	Highest verified context
Qwen3.6-35B-A3B-UD	19.46	MoE	100k verified
Qwen3.5-9B-UD	5.24	Dense	100k verified @ smaller batch
Qwen3-8B	4.68	Dense	65k verified
Qwen3-14B	8.38	Dense	65k verified @ smaller batch
Qwen3-4B-Instruct-2507-UD	4.71	Dense	100k verified
Gemma-3-12B-QAT	6.43	Dense	100k verified
Gemma-4-26B	12.66	MoE-ish	100k verified

Rank by decode	Model	Best variant	Prompt tok/s	Gen tok/s
1	Qwen3.6-35B-A3B-UD	100k ctx \| turbo4/turbo4 \| batch 4096 ubatch 1024	310.10	19.41
2	Gemma-4-26B	100k ctx \| turbo4/turbo4 \| batch 2048 ubatch 512 \| n-cpu-moe 128	168.49	9.21
3	Qwen3.5-9B-UD	100k ctx \| turbo4/turbo4 \| batch 1024 ubatch 256 \| no-kv-offload	377.52	8.62
4	Gemma-3-12B-QAT	100k ctx \| turbo4/turbo4 \| batch 2048 ubatch 512 \| mmproj on CPU	173.28	5.35
5	Qwen3-8B	65k ctx \| YaRN scale 2 orig 32768 \| batch 2048 ubatch 512	177.39	3.35
6	Qwen3-4B-Instruct-2507-UD	100k ctx \| turbo4/turbo4 \| batch 2048 ubatch 512	181.78	3.16
7	Qwen3-14B	65k ctx \| YaRN scale 2 orig 32768 \| batch 1024 ubatch 256	123.39	2.73

Batch size mostly improves prompt/prefill throughput; decode barely moves unless the model architecture changes.
uBatch is the main fit/stability lever on 11GB VRAM. When a model OOMs or fails to load, reducing ubatch first is often the save.
turbo4 KV cache is a major enabler for 60k-100k tests on this card.
no-kv-offload remains the safer long-context default on this Pascal GPU for consistency/stability.
Flash-attn was already validated earlier on this box, but these long-context runs centered on batch/ubatch/YaRN/KV-cache tradeoffs.
For Qwen YaRN variants, 2.0/32768 and 1.6/40960 were effectively tied at 65k on Qwen3-8B.
MoE models are the big winners here: Qwen3.6-35B-A3B and Gemma-4 outperform many smaller dense models in real decode speed.

Size: 19.46 GiB · Class: MoE · Context: 100k verified

Recommended setup: 100k ctx | turbo4/turbo4 | batch 4096 ubatch 1024

Variant tested	Prompt toks	Prompt tok/s	Gen tok/s	Status	Notes
100k ctx \| turbo4/turbo4 \| batch 4096 ubatch 1024	95000	310.10	19.41	best prompt	Best overall high-context setup tested. Decode unchanged vs smaller batch.
100k ctx \| turbo4/turbo4 \| batch 2048 ubatch 512	95000	240.54	19.12	works	Lower prefill than 4096/1024; decode nearly same.

Size: 5.24 GiB · Class: Dense · Context: 100k verified @ smaller batch

Recommended setup: 100k ctx | turbo4/turbo4 | batch 1024 ubatch 256 | no-kv-offload

Variant tested	Prompt toks	Prompt tok/s	Gen tok/s	Status	Notes
100k ctx \| turbo4/turbo4 \| batch 1024 ubatch 256 \| no-kv-offload	60800	377.52	8.62	best fit	100k works only after shrinking batch/ubatch.
100k ctx \| turbo4/turbo4 \| batch 2048 ubatch 512 \| no-kv-offload	—	—	—	OOM	OOM during prompt processing.

Size: 4.68 GiB · Class: Dense · Context: 65k verified

Recommended setup: 65k ctx | YaRN scale 2 orig 32768 | batch 2048 ubatch 512

Variant tested	Prompt toks	Prompt tok/s	Gen tok/s	Status	Notes
65k ctx \| YaRN scale 2 orig 32768 \| batch 2048 ubatch 512	55500	177.39	3.35	best of two	Slightly faster than 1.6/40960, but basically tied.
65k ctx \| YaRN scale 1.6 orig 40960 \| batch 2048 ubatch 512	55500	177.18	3.27	works	Near-identical to YaRN 2.0/32768.

Size: 8.38 GiB · Class: Dense · Context: 65k verified @ smaller batch

Recommended setup: 65k ctx | YaRN scale 2 orig 32768 | batch 1024 ubatch 256

Variant tested	Prompt toks	Prompt tok/s	Gen tok/s	Status	Notes
65k ctx \| YaRN scale 2 orig 32768 \| batch 2048 ubatch 512	—	—	—	load fail	Too aggressive for 11GB VRAM.
65k ctx \| YaRN scale 2 orig 32768 \| batch 1024 ubatch 256	55500	123.39	2.73	best fit	Reduced batch fixed load issue.

Size: 4.71 GiB · Class: Dense · Context: 100k verified

Recommended setup: 100k ctx | turbo4/turbo4 | batch 2048 ubatch 512

Variant tested	Prompt toks	Prompt tok/s	Gen tok/s	Status	Notes
100k ctx \| turbo4/turbo4 \| batch 2048 ubatch 512	59200	181.78	3.16	works	100k works cleanly, but decode is slow for a 4B at this quant/context.

Size: 6.43 GiB · Class: Dense · Context: 100k verified

Recommended setup: 100k ctx | turbo4/turbo4 | batch 2048 ubatch 512 | mmproj on CPU

Variant tested	Prompt toks	Prompt tok/s	Gen tok/s	Status	Notes
100k ctx \| turbo4/turbo4 \| batch 2048 ubatch 512 \| mmproj on CPU	64001	173.28	5.35	works	Balanced long-context option; better decode than Qwen8B/14B dense runs.

Size: 12.66 GiB · Class: MoE-ish · Context: 100k verified

Recommended setup: 100k ctx | turbo4/turbo4 | batch 2048 ubatch 512 | n-cpu-moe 128

Variant tested	Prompt toks	Prompt tok/s	Gen tok/s	Status	Notes
100k ctx \| turbo4/turbo4 \| batch 2048 ubatch 512 \| n-cpu-moe 128	64001	168.49	9.21	best decode after 35B	Strong decode speed despite slower prefill.