# Flash Attention on GTX 1080 Ti / Pascal (CC 6.1)

Session finding: do **not** assume Pascal rejects `--flash-attn`.

## Tested environment

- GPU: NVIDIA GeForce GTX 1080 Ti
- Compute capability: 6.1
- Binary: `/usr/local/bin/llama-server`
- Build/version: `3c7616b`
- Validation signal: server log showed `llama_context: flash_attn = enabled` when launched with `--flash-attn on`

## Why this matters

Older notes said Pascal/CC 6.1 rejects `--fa`. That is not universally true. The right durable lesson is:

1. inspect the actual binary's help/version
2. launch a one-off server with `--flash-attn on`
3. verify the log line `flash_attn = enabled`
4. benchmark `on` vs `off` on the real model/context before changing service units

## Benchmark method

- Fresh server start for each condition (`on` then `off`)
- Single uncached prompt only
- Same flags except `--flash-attn on|off`
- Same prompt, same `n_predict=8`
- Measured from response JSON `timings` plus wall time

## Qwen3-4B-Instruct-2507-UD-Q8_K_XL

Flags of interest:
- `--cache-type-k turbo4 --cache-type-v turbo4`
- `-ngl 99`
- `--no-mmap --mlock --jinja --no-kv-offload`
- `--batch-size 2048 --ubatch-size 512`
- `-c 16384`

### `--flash-attn on`
- tokens_evaluated: 10156
- prompt_ms: 24729.96
- prompt tok/s: 410.68
- predicted_ms: 598.28
- generation tok/s: 13.37
- wall time: 25.371 s

### `--flash-attn off`
- tokens_evaluated: 10156
- prompt_ms: 24747.922
- prompt tok/s: 410.38
- predicted_ms: 601.909
- generation tok/s: 13.29
- wall time: 25.394 s

### Result
Essentially a wash. Tiny FA win, not meaningful enough to justify forcing `on`.

## Qwen3-8B-Q4_K_M

Flags of interest:
- `--cache-type-k turbo4 --cache-type-v turbo4`
- `-ngl 99`
- `--no-mmap --mlock --jinja --no-kv-offload`
- `--rope-scaling yarn --rope-scale 2 --yarn-orig-ctx 32768`
- `--batch-size 2048 --ubatch-size 512`
- `-c 32768`

### `--flash-attn on`
- tokens_evaluated: 10156
- prompt_ms: 28225.183
- prompt tok/s: 359.82
- predicted_ms: 620.294
- generation tok/s: 12.90
- wall time: 28.892 s

### `--flash-attn off`
- tokens_evaluated: 10156
- prompt_ms: 28190.539
- prompt tok/s: 360.26
- predicted_ms: 626.279
- generation tok/s: 12.77
- wall time: 28.861 s

### Result
Also a wash. Prompt slightly worse with FA, generation slightly better; overall difference is noise-level.

## Recommendation

For Pascal / GTX 1080 Ti on this class of build:

- `--flash-attn on` can work
- but it is **not automatically a win**
- prefer omitted / `auto` unless benchmarking on the exact binary+model+context shows a real gain
- do not mass-edit service units to force `on` based on generic llama.cpp advice alone

## Reuse pattern

When asked whether FA helps on an older NVIDIA card:

```bash
/usr/local/bin/llama-server --version
/usr/local/bin/llama-server --help | grep -Ei 'flash|fa '
```

Then run one fresh-start `on` and one fresh-start `off` benchmark with the real model and context. Reject cached second-hit numbers; compare uncached prompt timing only.