# GPU Architecture Upgrade

When physically swapping an NVIDIA GPU (e.g., GTX 1080 Ti → RTX 2080 Ti), here's what to check and update.

## Driver Auto-Detection

Modern NVIDIA drivers (550+) auto-detect new GPUs in the same family. After a physical swap:
```bash
nvidia-smi                    # Verify GPU name, VRAM, driver version
lspci -nn | grep -i nvidia   # Confirm PCI device ID
nvidia-smi --query-gpu=compute_cap --format=csv  # Confirm SM version
```

If the driver version supports both cards (550.x covers Pascal through Ada), **no driver reinstall is needed**. The kernel module loads the new GPU automatically.

## Backup Existing Binaries First

Before rebuilding, preserve the old SM-architecture binaries — they're the only working build for the old GPU:

```bash
mkdir -p ~/src/llama.cpp/backups-sm61
sudo cp /usr/local/bin/llama-server /usr/local/bin/llama-server.bak \
        /usr/local/bin/llama-server-turbo /usr/local/bin/llama-server-turbo.bak \
        ~/src/llama.cpp/backups-sm61/
```

## Fork Discovery (When Upgrading GPU Architecture)

A GPU upgrade is a good time to also switch to a newer or more feature-rich llama.cpp fork. To discover branches:

```bash
# List all branches of a fork
curl -sL "https://api.github.com/repos/<owner>/<repo>/branches?per_page=100" | \
  python3 -c "import json,sys; [print(b['name']) for b in json.load(sys.stdin)]" | \
  grep -iE 'mtp|turbo|spec|all.in.one|merge'

# Get commits on a specific branch
curl -sL "https://api.github.com/repos/<owner>/<repo>/commits?sha=<branch>&per_page=10" | \
  python3 -c "import json,sys; [print(c['sha'][:8], c['commit']['message'].split(chr(10))[0][:100]) for c in json.load(sys.stdin)]"

# Check repo metadata (default branch, last pushed)
curl -sL "https://api.github.com/repos/<owner>/<repo>" | \
  python3 -c "import json,sys; d=json.load(sys.stdin); print(f\"default_branch={d['default_branch']} pushed={d['pushed_at']} stars={d['stargazers_count']}\")"
```

Key things to check before cloning:
- Is there an `all-in-one` or `feat/mtp-turboquant` branch? (merged features, less conflict risk)
- How recent is the upstream base? (May 2026+ for Gemma 3/4, Qwen3.6 vision support)
- Does the merge commit message show conflict resolution for flash attention head-dim exclusions? (critical for wide models)

## llama.cpp Rebuild Required

The driver works, but **compiled CUDA binaries are architecture-locked**. An `llama-server` built for `sm_61` (GTX 1080 Ti / Pascal) will run on an RTX 2080 Ti (Turing / `sm_75`) via CUDA binary compatibility, but **tensor cores will sit idle** — no `sm_75`-specific kernel variants were compiled.

### Inspecting an Existing Build

**Find the build directory:**
```bash
find ~/src/llama.cpp -maxdepth 2 -name "CMakeCache.txt"
```

**Check what SM arch it was built for:**
```bash
grep CMAKE_CUDA_ARCHITECTURES build-*/CMakeCache.txt
# → CMAKE_CUDA_ARCHITECTURES:UNINITIALIZED=61   ← sm_61 only!
```

**Confirm arch in the binary itself:**
```bash
strings /usr/local/bin/llama-server | grep -E '^sm_[0-9]|\.target sm_'
# → .target sm_61
```

**Full build config audit:**
```bash
grep '^GGML\|^CMAKE_BUILD_TYPE\|^CMAKE_CUDA_ARCH' build-*/CMakeCache.txt | sort
```

**Compare binaries by size:**
```bash
size /usr/local/bin/llama-server /usr/local/bin/llama-server-turbo
# text    data    bss     dec     hex     filename
# 94360048 7163368 1075904 102599320 61d8a98 llama-server
# 131849672 7068976 1056688 139975336 857daa8 llama-server-turbo
```
Larger `.text` = more CUDA kernel instantiations (likely `GGML_CUDA_FA_ALL_QUANTS=ON`).

**Verify CUDA linkage:**
```bash
ldd /usr/local/bin/llama-server | grep -i cuda
# libcudart.so.12, libcublas.so.12, libcublasLt.so.12
```

**Check for local patches:**
```bash
cd ~/src/llama.cpp && git log --oneline -5 && git diff --stat HEAD
```

### Rebuilding for the New Arch

Change only `CMAKE_CUDA_ARCHITECTURES` — keep all other flags identical to the old build.

**Standard case (nvcc on PATH, GCC 13 host compiler):**
```bash
cd ~/src/llama.cpp
mkdir -p build-cuda-sm75 && cd build-cuda-sm75

CUDAHOSTCXX=/usr/bin/gcc-13 cmake .. \
  -DCMAKE_CUDA_ARCHITECTURES=75 \
  -DGGML_CUDA=ON \
  -DGGML_CUDA_FA=ON \
  -DGGML_CUDA_GRAPHS=ON \
  -DGGML_CUDA_COMPRESSION_MODE=size \
  -DGGML_CPU=ON \
  -DGGML_NATIVE=ON \
  -DGGML_OPENMP=ON \
  -DCMAKE_BUILD_TYPE=Release

cmake --build . --target llama-server -j$(nproc)
```

**CUDA 12.6 with nvcc not on PATH:** When the CUDA toolkit lives at `/usr/local/cuda-12.6` and `nvcc` isn't in `PATH`, cmake's `enable_language(CUDA)` fails during configure. Set all three CUDA compiler variables:

```bash
CUDAHOSTCXX=/usr/bin/g++-13 cmake .. \
  -DCMAKE_C_COMPILER=/usr/bin/gcc-13 \
  -DCMAKE_CXX_COMPILER=/usr/bin/g++-13 \
  -DCMAKE_CUDA_COMPILER=/usr/local/cuda-12.6/bin/nvcc \
  -DCMAKE_CUDA_HOST_COMPILER=/usr/bin/g++-13 \
  -DCMAKE_CUDA_ARCHITECTURES=75 \
  -DGGML_CUDA=ON \
  -DGGML_CUDA_FA=ON \
  -DGGML_CUDA_GRAPHS=ON \
  -DGGML_CPU=ON \
  -DGGML_OPENMP=ON \
  -DGGML_NATIVE=ON \
  -DGGML_LLAMAFILE=ON \
  ...

make -j$(nproc) llama-server
```

`CUDAHOSTCXX` alone only works if cmake can already find `nvcc`. When it can't, `CMAKE_CUDA_COMPILER` tells cmake WHERE nvcc is, and `CMAKE_CUDA_HOST_COMPILER` tells nvcc which host compiler to use (bypassing GCC 14).

Then backup and install:
```bash
sudo cp /usr/local/bin/llama-server /usr/local/bin/llama-server-sm61.bak
sudo cp bin/llama-server /usr/local/bin/llama-server
```

### SM Architecture Impacts

| Feature | sm_61 (Pascal) | sm_75 (Turing) |
|---|---|---|
| Tensor cores | ❌ None | ✅ 544 INT8 / 272 FP16 |
| Flash Attention | ✅ Works (CUDA cores) | ✅ Faster (tensor core ops) |
| FP16 throughput | Baseline | ~2x with tensor cores |
| Max clock | ~1800 MHz | ~2100 MHz |

After rebuilding for `sm_75`, `llama-server --version` should show `compute capability 7.5` in the CUDA init output and tensor-core-accelerated flash attention kernels will be selected.