# DSpark Speculative Decoding — Status (July 2026)

## What is DSpark?

DeepSeek + PKU speculative decoding framework layered on DFlash. Adds a semi-autoregressive Markov head that conditions each draft position on the previously sampled token, lifting acceptance rates over DFlash's independent-argmax approach.

Paper: [DeepSpec repo / DSpark_paper.pdf](https://github.com/deepseek-ai/DeepSpec/blob/main/DSpark_paper.pdf)

## GGUF Speculators Available

| Speculator | Target Model | GGUF Size | Source |
|---|---|---|---|
| Qwen3-8B DSpark block7 | Qwen3-8B | 1.53 GB (Q4_K_M) | `ankk98/dspark-qwen3-8b-block7-Q4_K_M-GGUF` |
| Gemma4-12B DSpark block7 | Gemma4-12B | ~1.9 GB (Q4_0) | `ankk98/dspark-gemma4-12b-block7-Q4_0-GGUF` |
| Qwen3-4B DSpark block7 | Qwen3-4B | safetensors only | `deepseek-ai/dspark_qwen3_4b_block7` |
| Qwen3-14B DSpark block7 | Qwen3-14B | safetensors only | `deepseek-ai/dspark_qwen3_14b_block7` |

## llama.cpp Status

- **PR #25173** (wjinxu, opened June 30, 2026) adds DSpark on top of the merged DFlash drafter
- Uses `--spec-type draft-dspark` with `-md <draft_gguf>`
- **NOT MERGED** as of b9743 (our build, June 20)
- Requires `llama-quantize` with `--target-model-dir` to convert safetensor draft to GGUF
- Performance (RTX 4090, Qwen3-8B bf16): 1.88× vs baseline, 1.21× vs DFlash

## Compatibility Constraints

- **DSpark speculators are target-model-specific.** The draft head shares the target's token embeddings and LM head and operates on the target's hidden states. A Qwen3-8B speculator will NOT work with Qwen3.5-9B or Ornith-9B (different hidden dimensions / layer structure).
- No DSpark speculators exist for Qwen3.5-9B, Qwen3.6-9B, Ornith variants, or any Qwen3.6 model as of July 2026.

## NVFP4/MXFP4 — Blackwell Only

- NVFP4 (GGML type 40, E2M1 + E4M3 scale, 16-element blocks) and MXFP4 (type 39, E2M1 + E8M0 scale, 32-element blocks) require Blackwell tensor cores (sm_120 / sm_121).
- **Incompatible with RTX 2080 Ti (sm_75).** These GGUFs will fail to load on Turing hardware.
- Repos to avoid: `s-batman/Ornith-1.0-35B-NVFP4-MTP-GGUF`, any GGUF with `MXFP4_MOE` or `NVFP4` ftype.

## Best Alternative on 11GB (sm_75)

For Ornith-1.0-35B on RTX 2080 Ti, the `LordNeel/Ornith-1.0-35B-GGUF-llamacpp-tp1` repo's integrated MTP graft (`ornith-1.0-35b-IQ4_XS-MTP-graft-headQ6.gguf`, 19.6 GB) achieves:
- KLD 0.0731 (between Q5_K_M at 0.0354 and Q4_K_M at 0.0864 — beats Q4_K_M)
- SWE-bench verified-mini 60% (within noise of Q5_K_M at 64%)
- MTP acceptance 82-96%, single-stream ~1.35× speedup
- Runs at ~29.5 t/s with `--n-cpu-moe 24` on 11GB

## When to Revisit

1. PR #25173 merges into llama.cpp master
2. Someone releases a DSpark speculator trained for a model in your fleet (Qwen3.6-35B, Ornith-35B)
3. Rebuild `llama-server-sm75` from source with DSpark support
4. Test with `--spec-type draft-dspark --spec-draft-n-max 7`