# Performance Patch Auditing Methodology

When applying external performance patches (from forks, PRs, or research), **verify the code path activates** before concluding pass/fail. Surface-level benchmarks without internals confirmation are insufficient.

## Diagnostic Injection

If a patch has no logging at its activation point, inject `fprintf(stderr, ...)` before building:

```c
fprintf(stderr, "[DIAG] HIT key_param=%ld threshold=%ld size=%zu\n",
    (long)val, (long)threshold, ggml_nbytes(tensor));
```

- Guard with `static int count` (max 5 prints) to avoid log spam
- Print condition values to verify thresholds are met
- Rebuild (`cmake --build build -j --target llama-server`), reinstall, retest

## Benchmark Protocol

1. **Unique prompts every run** — generate random words. Prompt caching invalidates repeats.
2. **`cache_prompt: false`** in completion request
3. **`--ubatch-size == --batch-size`** — full-batch prefill, no micro-batching
4. **3+ runs** per configuration, average the middle runs
5. **Use `/completion` endpoint** (raw), not `/v1/completions` (lacks detailed timings)
6. **Spinning disk?** Skip first run — model pages may not be in cache

## Compute-to-Upload Ratio

For PCIe-bound optimizations (MoE prefetch, weight streaming, KV offload):

```
Prefetch helps when:  compute_time_per_layer > upload_time
                     (GPU works on layer N while PCIe feeds layer N+1)
```

| GPU | FP16 TFLOPS | PCIe | Upload 176 MiB | Compute/Layer | Ratio | Works? |
|-----|------------|------|----------------|---------------|-------|--------|
| RTX 3060 12GB | 12.7 | Gen4 ≈24 GB/s | ~7 ms | ~18 ms | 2.6:1 | Yes |
| RTX 2080 Ti 11GB | 26.9 | Gen3 ≈12 GB/s | ~14 ms | ~9 ms | 0.64:1 | No |

**Key insight:** A faster GPU on slower PCIe inverts the ratio — compute finishes before upload completes. Overlap becomes sequential + overhead.

## Sign-Off Checklist

1. Diagnostic confirms code path fires
2. Thresholds verified (tensor sizes, batch conditions met)
3. Multiple runs averaged
4. Flags (`--no-mmap`, `--n-cpu-moe`, etc.) accounted for
5. Hardware profile compared against original author's setup

---

For the fable5 MoE prefetch patches applied to our build: `references/fable5-moe-prefetch-patches.md`