# Build Recipe: llama.cpp with MTP + TurboQuant for RTX 2080 Ti

## Source
- **Fork:** `EsmaeelNabil/llama.cpp-mtp-turbo-quant`
- **Branch:** `feat/mtp-turboquant-kv-cache` (SHA: `a578bb6b`)
- **What it merges:** am17an MTP (multi-token prediction) + TheTom TurboQuant (turbo2/3/4 KV cache compression)
- Clone: `git clone https://github.com/EsmaeelNabil/llama.cpp-mtp-turbo-quant.git`

## Build Environment
- OS: Debian 13 (Trixie)
- Host GPU: RTX 2080 Ti 11GB (sm_75, Turing)
- CUDA Toolkit: 12.6 (at `/usr/local/cuda-12.6`)
- Driver: 550.163.01 (CUDA 12.4 runtime)
- Host compiler: GCC 13.3.0 (NOT GCC 14 — CUDA 12.6 doesn't support it)

## Pitfalls Hit & Resolved

### Pitfall 1: GCC 14 unsupported by CUDA 12.6
CUDA 12.6's nvcc supports GCC up to version 13. Debian 13 ships GCC 14 as default.
**Fix:** Install `gcc-13` and `g++-13` explicitly (`apt install gcc-13 g++-13`), then pass them to cmake.

### Pitfall 2: CUDA compiler auto-detection fails
Cmake's `enable_language(CUDA)` can't find nvcc when it's not on PATH. Even with `-DCMAKE_C_COMPILER=gcc-13`, the CUDA compiler ID test picks up the system GCC.
**Fix:** Set ALL of these explicitly:
```bash
-DCMAKE_CUDA_COMPILER=/usr/local/cuda-12.6/bin/nvcc
-DCMAKE_CUDA_HOST_COMPILER=/usr/bin/g++-13
```
Plus the environment variable `CUDAHOSTCXX=/usr/bin/g++-13` before cmake.

### Pitfall 3: Shared library install
Newer llama.cpp builds produce shared libraries (libggml-cuda.so, libggml.so, etc.) instead of a monolithic binary. The CUDA kernels live in `libggml-cuda.so.0.11.1` (~224MB).
**Fix:** After build, copy ALL shared libs to `/usr/local/lib/` and run `ldconfig`:
```bash
sudo cp libllama-common.so.0.0.* libmtmd.so.0.0.* libllama.so.0.0.* \
        libggml-base.so.0.* libggml.so.0.* libggml-cpu.so.0.* \
        libggml-cuda.so.0.* /usr/local/lib/
sudo ldconfig
```

### Pitfall 4: Binary is small (9MB) — not broken
The `llama-server` binary itself is only ~9MB because everything is in shared libs. The real CUDA weight is in `libggml-cuda.so` (224MB). This is normal for the new build system.

## Build Command

```bash
cd ~/src/llama.cpp-mtp-turbo
git checkout feat/mtp-turboquant-kv-cache

rm -rf build-cuda-sm75 && mkdir build-cuda-sm75 && cd build-cuda-sm75

CUDAHOSTCXX=/usr/bin/g++-13 cmake .. \
  -DCMAKE_BUILD_TYPE=Release \
  -DCMAKE_C_COMPILER=/usr/bin/gcc-13 \
  -DCMAKE_CXX_COMPILER=/usr/bin/g++-13 \
  -DCMAKE_CUDA_COMPILER=/usr/local/cuda-12.6/bin/nvcc \
  -DCMAKE_CUDA_HOST_COMPILER=/usr/bin/g++-13 \
  -DCMAKE_CUDA_ARCHITECTURES=75 \
  -DGGML_CUDA=ON \
  -DGGML_CUDA_FA=ON \
  -DGGML_CUDA_GRAPHS=ON \
  -DGGML_CPU=ON \
  -DGGML_OPENMP=ON \
  -DGGML_NATIVE=ON \
  -DGGML_LLAMAFILE=ON \
  -DGGML_BUILD_EXAMPLES=OFF \
  -DGGML_BUILD_TESTS=OFF \
  -DGGML_CCACHE=OFF

make -j$(nproc) llama-server
```

## Verification
```bash
# Check CUDA arch in the shared library
strings build-cuda-sm75/bin/libggml-cuda.so | grep '\.target sm_' | sort -u
# Should output: .target sm_75

# Check it detects the GPU
build-cuda-sm75/bin/llama-server --version
# Should show: RTX 2080 Ti, compute capability 7.5

# Check turbo cache types available
build-cuda-sm75/bin/llama-server --help | grep 'turbo'
# Should show: turbo2, turbo3, turbo4

# Check MTP available
build-cuda-sm75/bin/llama-server --help | grep 'mtp'
# Should show: --spec-type ... mtp ...
```

## Install
```bash
sudo cp build-cuda-sm75/bin/llama-server /usr/local/bin/llama-server-sm75
sudo cp build-cuda-sm75/bin/lib*.so.0.* /usr/local/lib/
sudo ldconfig
sudo ln -sf /usr/local/bin/llama-server-sm75 /usr/local/bin/llama-server
```

## Backing Up Old Build (sm_61 for 1080 Ti)
```bash
mkdir -p backups-sm61
cp /usr/local/bin/llama-server* backups-sm61/
# Old binaries preserved in ~/src/llama.cpp/backups-sm61/
```