# s-batman/Ornith-1.0-35B-NVFP4-MTP-GGUF

MXFP4 + Multi-Token Prediction (MTP) heads grafted from
[Qwen3.6-35B-A3B](https://huggingface.co/Qwen/Qwen3.6-35B-A3B), packaged as a
single GGUF for llama.cpp.

Designed for NVIDIA Blackwell GPUs (sm\_120 / sm\_121) including the **RTX PRO**
**6000** and the **DGX Spark (GB10)**. MXFP4 is dequantized natively by
Blackwell tensor cores, and the grafted MTP heads enable `draft-mtp`
speculative decoding for ~2× decode throughput versus the body-only quant.

## Original Model

[Ornith-1.0-35B](https://huggingface.co/deepreinforce-ai/Ornith-1.0-35B) is a
self-improving agentic-coding model released by the
[DeepReinforce](https://deep-reinforce.com/) team, post-trained via RL on top
of Qwen3.5-35B-A3B. It emits a `reasoning_content` block before its final
answer and is competitive with Qwen3.6-35B-A3B and Gemma 4-31B on
Terminal-Bench 2.1, SWE-bench Verified/Pro, and Claw-eval.

- **Architecture:** Qwen3.5 MoE (`qwen3_5_moe`), 40 layers, 256 experts,
hidden\_size 2048
- **Parameters:** 35B total / ~3B active
- **Vocabulary:** 248,064 tokens (multimodal vocab preserved; vision tower not
included in this GGUF)
- **License:** MIT (inherited from upstream)
- **Citation:** see [Citation](https://huggingface.co/s-batman/Ornith-1.0-35B-NVFP4-MTP-GGUF#citation) below

## Quantization Details

This repository ships **two files** with the same trunk weights but different
expert quantizations:

### File 1 — `ornith-1.0-35b-MXFP4_MOE-MTP.gguf` (MXFP4 + MTP, the original)

| Property | Value |
| --- | --- |
| **Body weights — trunk 3D experts** (`blk.*.ffn_*_exps.weight`, 80 tensors) | **MXFP4** (GGML type 39, E2M1 + E8M0 scale, 32-element blocks, 4.25 bpw) |
| **Body weights — 2D linears** (`attn_qkv`, `attn_gate`, `ssm_*`, shared experts) | Q8\_0 |
| **MTP head — 3D experts** (`blk.40.nextn.ffn_*_exps`, 2 tensors) | MXFP4 |
| **MTP head — non-experts** (`blk.40.attn_*`, `nextn.*_norm`, `nextn.eh_proj`, `blk.40.ffn_down_exps`) | Q8\_0 (one Q5\_K outlier from the graft source: `blk.40.ffn_down_exps.weight`) |
| **Sensitive tensors** (`token_embd`, `output`) | Q8\_0 |
| **Norms, biases, router** (`*.norm.weight`, `ssm_dt.bias`, `ffn_gate_inp*`) | F32 / BF16 (default for MXFP4\_MOE) |
| **Total size** | **20.30 GB** (4.572 bpw average) |
| **Total tensors** | 713 |
| **Source ftype** | `MXFP4_MOE` (LLAMA\_FTYPE\_MOSTLY\_MXFP4\_MOE = 38) |
| **Source tool** | `llama-quantize MXFP4_MOE` from llama.cpp build 9590 (CUDA 13.1, sm\_120 Blackwell) |
| **MTP source** | MTP block grafted byte-for-byte from `unsloth/Qwen3.6-35B-A3B-MTP-GGUF` |

### File 2 — `ornith-1.0-35b-NVFP4-MTP.gguf` (true NVFP4 + MTP, NEW)

| Property | Value |
| --- | --- |
| **Body weights — trunk 3D experts** (`blk.*.ffn_*_exps.weight`, `blk.*.ffn_gate_up_exps.weight`, 80 tensors) | **NVFP4** (GGML type 40, E2M1 + E4M3 scale, 16-element blocks, 4.50 bpw) |
| **Body weights — 2D linears** (`attn_qkv`, `attn_gate`, `ssm_*`, shared experts) | Q8\_0 |
| **MTP head — 3D experts** (`blk.40.nextn.ffn_*_exps`, 2 tensors) | MXFP4 (from graft source, since the Qwen3.6 MTP source is MXFP4) |
| **MTP head — non-experts** (`blk.40.attn_*`, `nextn.*_norm`, `nextn.eh_proj`, `blk.40.ffn_down_exps`) | Q8\_0 (one Q5\_K outlier from the graft source: `blk.40.ffn_down_exps.weight`) |
| **Sensitive tensors** (`token_embd`, `output`) | Q8\_0 |
| **Norms, biases, router** (`*.norm.weight`, `ssm_dt.bias`, `ffn_gate_inp*`) | F32 / BF16 (default for NVFP4 routing) |
| **Total size** | **19.85 GiB (21.31 GB)** (~4.80 bpw average) |
| **Total tensors** | 713 |
| **Source ftype** | `NVFP4` (LLAMA\_FTYPE\_MOSTLY\_NVFP4 = 39) |
| **Source tool** | `llama-quantize NVFP4` from llama.cpp patched to (a) register NVFP4 in `QUANT_OPTIONS`, (b) route 3D MoE tensors to NVFP4 and 2D linears to Q8\_0 (parity with MXFP4\_MOE) |
| **MTP source** | MTP block grafted byte-for-byte from `unsloth/Qwen3.6-35B-A3B-MTP-GGUF` |

### What is MXFP4?

MXFP4 is the **OCP Microscaling Formats** 4-bit floating-point spec
(`OCP Microscaling Formats (MX) Specification v1.0`). It is an
**open, vendor-neutral standard** — supported on both NVIDIA Blackwell and
AMD Instinct MI355X. Each block of **32 contiguous elements** shares a
single 8-bit unsigned scale factor (`E8M0`) — a pure power-of-two with no
mantissa, giving an enormous dynamic range (2⁻¹²⁷ to 2¹²⁷) but fractional
precision only at the byte-block level.

### What is NVFP4, and how does it differ?

NVFP4 is NVIDIA's **proprietary** Blackwell-native variant. Same element
encoding (E2M1) and same 4-bit storage cost, but:

|  | MXFP4 (this file) | NVFP4 (second file, released) |
| --- | --- | --- |
| Element | E2M1 | E2M1 (same) |
| Block size | 32 elements | **16 elements** |
| Scale format | E8M0 (power-of-two only) | **FP8 E4M3** (fractional) |
| Second-level scale | none | **FP32 per tensor** |
| Effective bpw | 4.25 | **4.50** |
| Hardware | Blackwell + AMD MI355X | **Blackwell only** |
| Tensor cores used | Blackwell FP4 | Blackwell FP4 (same) |

NVFP4's finer 16-element blocks + fractional E4M3 scales + per-tensor FP32
shift fit each block's distribution more tightly, giving slightly lower
quantization error than MXFP4's power-of-two snapping. Empirically the gap
on 35B-class MoE models is **<1%** on standard benchmarks (MMLU, GPQA,
HumanEval); for most inference workloads the two formats are functionally
interchangeable on Blackwell.

### What is MTP and why graft it?

[Multi-Token Prediction](https://arxiv.org/abs/2004.14525) uses a small
auxiliary head to predict multiple tokens ahead. At inference, those
predictions become a draft that the main model verifies in a single forward
pass — speculative decoding with **zero quality loss** (output distribution
is identical to non-MTP) and 1.5-2× decode speedup when drafts are
accepted.

Ornith-1.0-35B is a Qwen3.5-35B-A3B post-trained variant. Its trunk weights
share the same parameter shapes and base tensor layout as Qwen3.6-35B-A3B,
including the same `qwen3_5_moe` MTP block position (20 tensors at
`blk.40.*`, ~488 MB). Because the MTP head operates on the same hidden
state and embedding space as the trunk, the grafted heads work without any
additional training — the MTP drafts the next token given the trunk's
hidden state, the trunk then verifies. **No re-tuning required.**

Acceptance rates measured on RTX PRO 6000:

| Sampling | Accept rate | Speedup vs body-only |
| --- | --- | --- |
| Greedy (temp=0) | **100%** (21/21, 80/80) | ~1.85× |
| temp=0.6, top-p=0.95 | 86–93% | ~1.7× |
| temp=0.9 (production) | 75–88% | ~1.5× |

## Performance

### RTX PRO 6000 (Blackwell, sm\_120, 97 GB GDDR7, ~1.79 TB/s)

Single-slot, 200k context, `temp=0.9`, `top_p=0.95`, `top_k=20`, `min_p=0.01`,
KV cache q8\_0:

| Mode | Decode (tok/s) | Prefill (tok/s) | Notes |
| --- | --- | --- | --- |
| MXFP4\_MOE (body only, no MTP) | 240 | ~6,000 | baseline |
| **MXFP4\_MOE + MTP (n-max=3)** | **310–320** | ~6,000 | production config |
| Q8\_K\_XL + MTP (Unsloth upstream) | ~165 | ~5,500 | reference |

The MTP head costs ~280 MiB of KV context; the trunk savings (4.57 BPW vs
8.5 BPW for Q8\_K) more than pay for it.

### DGX Spark (Blackwell GB10, sm\_121, 121 GB unified LPDDR5X, ~273 GB/s)

3 slots × 200k context each (`c=600000`, `parallel=3`), KV cache q8\_0.
Production sweep (8 configs, B×UB×n-max combinations, 4-run stability test):

| Config (B / UB / n-max) | Combined tok/s | Per-slot tok/s | Accept rate |
| --- | --- | --- | --- |
| **2048 / 2048 / 3** ⭐ (winner) | **93.0 ± 1.2** | ~31 each | 67–88% |
| 2048 / 1024 / 3 (baseline) | 92.0 | ~31 each | 58–83% |
| 4096 / 1024 / 3 | 92.4 | ~31 each | 60–91% |
| 1024 / 512 / 3 | 91.6 | ~31 each | 64–79% |
| 2048 / 1024 / 4 | 85.3 | ~29 each | 51–70% |
| 2048 / 1024 / 5 | 82.3 | ~29 each | 48–68% |
| 2048 / 1024 / 6 | 81.3 | ~30 each | 43–51% |

The ~41% throughput ratio vs the 6000 Pro (93 / 240) matches the GB10's
~273 GB/s vs the 6000 Pro's ~1.79 TB/s unified-memory bandwidth gap,
confirming the workload is bandwidth-bound. MTP-only (no ngram-mod) was
chosen because the MTP head already provides the draft stream and the
ngram-mod bookkeeping overhead exceeds its marginal gain.

## Provided Files

| File | Size | Tensor type | Notes |
| --- | --- | --- | --- |
| `ornith-1.0-35b-MXFP4_MOE-MTP.gguf` (this file — MXFP4 under the hood) | 20.30 GB | MXFP4 (type 39) for experts, Q8\_0 for sensitive, F32 for norms | Recommended for most Blackwell users — same throughput as NVFP4, smaller storage |
| `ornith-1.0-35b-NVFP4-MTP.gguf` ( **RELEASED** — true NVFP4, tensor type 40) | 21.31 GB | NVFP4 (type 40) for the 80 trunk 3D experts, Q8\_0 for 2D linears, F32 for norms | Slightly higher accuracy vs MXFP4 (E4M3 fractional scales vs E8M0 power-of-two). Use when MMLU-Pro / GPQA accuracy matters more than 1 GB of storage. |
| `chat_template.jinja` | 7.5 KB | — | Jinja chat template (inherited from upstream; emits `reasoning_content` block) |
| `config.json` | 3.3 KB | — | Original HF model config (text\_config subset) |
| `tokenizer.json` | 19 MB | — | HF fast tokenizer |
| `tokenizer_config.json` | 1.2 KB | — | Tokenizer metadata |
| `vocab.json` | 6.7 MB | — | BPE vocab |
| `preprocessor_config.json` | 390 B | — | Image preprocessor (inherited; unused by text-only inference) |
| `processor_config.json` | 1.2 KB | — | Multimodal processor (inherited; unused by text-only inference) |
| `generation_config.json` | 213 B | — | Generation defaults |
| `model.safetensors.index.json` | 3.3 MB | — | Weight map (informational) |
| `LICENSE` | 1 KB | — | MIT license text |
| `README.md` | — | — | This file |

The `video_preprocessor_config.json` file is also present but is
informational only.

## Usage with llama.cpp

### Requirements

- llama.cpp build **9590** or later (FP4 tensor core support, sm\_120 /
sm\_121 CUDA)
- CUDA toolkit with Blackwell support
- Build with `-DGGML_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES="120;121"` (set to
your GPU's compute capability)

### Server (recommended for production)

**RTX PRO 6000 / single-GPU Blackwell, single slot, 200k context:**

```bash
llama-server \
  -m ornith-1.0-35b-MXFP4_MOE-MTP.gguf \
  --host 0.0.0.0 --port 8080 --slots --metrics \
  -t 20 -cb --no-warmup --no-mmap --mlock \
  --jinja -fa on -ctk q8_0 -ctv q8_0 \
  --cache-reuse 256 -ctxcp 256 --checkpoint-every-n-tokens 4096 \
  -b 2048 -ub 2048 -c 1000000 -np 5 -ngl 99 \
  --chat-template-file chat_template.jinja \
  --spec-type draft-mtp,ngram-mod \
  --spec-draft-n-max 3 \
  --spec-ngram-mod-n-match 24 \
  --spec-ngram-mod-n-min 48 \
  --spec-ngram-mod-n-max 64 \
  --reasoning-budget 2048 \
  --reasoning-budget-message "I have thought enough. Let me give my answer now." \
  --temp 0.9 --top-p 0.95 --top-k 20 --min-p 0.01 --repeat-penalty 1.0
```

**DGX Spark / 3 slots × 200k context each:**

```bash
llama-server \
  -m ornith-1.0-35b-MXFP4_MOE-MTP.gguf \
  --host 0.0.0.0 --port 8080 --slots --metrics \
  -t 20 -cb --no-warmup --no-mmap --mlock \
  --jinja -fa on -ctk q8_0 -ctv q8_0 \
  --cache-reuse 256 -ctxcp 256 --checkpoint-every-n-tokens 4096 \
  -b 2048 -ub 2048 -c 600000 -np 3 -ngl 99 \
  --chat-template-file chat_template.jinja \
  --spec-type draft-mtp \
  --spec-draft-n-max 3 \
  --reasoning-budget 2048 \
  --reasoning-budget-message "I have thought enough. Let me give my answer now." \
  --temp 0.9 --top-p 0.95 --top-k 20 --min-p 0.01 --repeat-penalty 1.0
```

### CLI

```bash
llama-cli \
  -m ornith-1.0-35b-MXFP4_MOE-MTP.gguf \
  --chat-template-file chat_template.jinja \
  -p "Explain gradient descent in 3 sentences." \
  -ngl 99 \
  --temp 0.9 --top-p 0.95 --top-k 20 --min-p 0.01 \
  --spec-type draft-mtp --spec-draft-n-max 3
```

### Direct download with llama.cpp

```bash
llama-cli \
  --hf-repo s-batman/Ornith-1.0-35B-NVFP4-MTP-GGUF \
  --hf-file ornith-1.0-35b-NVFP4-MTP.gguf \
  -p "What is 17 times 23?"
```

(Substitute `ornith-1.0-35b-MXFP4_MOE-MTP.gguf` for the original MXFP4
variant.)

## Important Notes

- **Blackwell only.** MXFP4/NVFP4 are hardware-specific formats. They will
not run efficiently on non-Blackwell GPUs. For AMD, Intel, or pre-Blackwell
NVIDIA GPUs, use the upstream
[deepreinforce-ai/Ornith-1.0-35B-GGUF](https://huggingface.co/deepreinforce-ai/Ornith-1.0-35B-GGUF)
(Q4\_K\_M / Q8\_0 variants).
- **`--no-mmap` recommended** on unified-memory machines (DGX Spark) to
avoid mmap-related slowdowns.
- **KV cache type q8\_0 is the production setting.** F16 KV uses ~2× more
memory for negligible quality change on long context.
- **MTP is single-slot at peak speed** but works well in multi-slot mode —
the MTP context is shared across slots and adds minimal per-slot overhead.
On the DGX Spark, 3 slots at ~31 t/s each = ~93 t/s combined.
- **MXFP4 vs NVFP4** — pick `ornith-1.0-35b-MXFP4_MOE-MTP.gguf` (the MXFP4
file) for smallest storage and identical Blackwell throughput. Pick
`ornith-1.0-35b-NVFP4-MTP.gguf` (the NVFP4 file) if you need the slightly
tighter quantization error of E4M3 fractional scales.

## How the MTP graft was made

The MTP heads were transferred byte-for-byte from
[`unsloth/Qwen3.6-35B-A3B-MTP-GGUF`](https://huggingface.co/unsloth/Qwen3.6-35B-A3B-MTP-GGUF)
into the deepreinforce-ai Ornith-1.0-35B MXFP4 GGUF:

1. Source GGUF (Qwen3.6) has a 20-tensor MTP block at `blk.40.*` totaling
~488 MB.
2. Destination GGUF (Ornith-1.0-35B MXFP4) has the same `qwen3_5_moe`
architecture with `block_count=40` and no MTP tensors.
3. The graft script appends the MTP block immediately after the destination
data block, rewrites tensor offsets, and adds two new KV pairs:
`qwen35moe.nextn_predict_layers=1` and bumps
`qwen35moe.block_count=40→41`.

This is possible because:

- Ornith-1.0-35B is post-trained on top of Qwen3.5-35B-A3B (same
architecture, same shapes, same embeddings)
- The MTP head is small (~488 MB) and depends only on the trunk's hidden
state and the shared embedding/output projection
- Qwen3.5 and Qwen3.6 share the same `qwen3_5_moe` MTP architecture
(Qwen3.6 is Qwen3.5 post-trained)

**Acceptance at 100% on greedy decoding** confirms the graft is functionally
correct: when the trunk deterministically predicts token N, the MTP head's
prediction of token N+1 matches exactly.

The graft script is available on request.

## How to verify the tensor types

To confirm this file is MXFP4 (not NVFP4), inspect the GGUF header with the
official `gguf_dump.py` tool (shipped with llama.cpp):

```bash
python3 gguf-py/gguf/scripts/gguf_dump.py --no-tensors --markdown \
    ornith-1.0-35b-MXFP4_MOE-MTP.gguf | head -50
```

Look for:

- `general.file_type = 38` (means `LLAMA_FTYPE_MOSTLY_MXFP4_MOE`)
- Tensor types: 82× `MXFP4` (the 3D experts), 320× `Q8_0`, 308× `F32`,
2× `BF16`, 1× `Q5_K` (graft outlier)

If those numbers are present, you have the MXFP4 file. For true NVFP4
the `general.file_type` would be 39, the tensor count for type 40 (`NVFP4`)
would be 80 (40 trunk layers × 2 expert tensors per layer), and total file
size would be ~21.3 GB.

### Verifying the NVFP4 file

To confirm the second file in this repository is genuine NVFP4:

```bash
python3 gguf-py/gguf/scripts/gguf_dump.py --no-tensors --markdown \
    ornith-1.0-35b-NVFP4-MTP.gguf | head -60
```

Look for:

- `general.file_type = 39` (means `LLAMA_FTYPE_MOSTLY_NVFP4`)
- Tensor types in this file: **80× `NVFP4`** (type 40 — the 3D trunk experts
only: `blk.*.ffn_down_exps.weight` \+ `blk.*.ffn_gate_up_exps.weight` × 40
layers), **320× `Q8_0`** (2D linears: `attn_qkv`, `attn_gate`, `ssm_*`,
shared experts, sensitive tensors), **308× `F32`** (norms, biases, router),
**2× `MXFP4`** (the 2 large 3D expert tensors grafted from the Qwen MTP
source — type 39, not 40), 2× `I32`, 1× `Q5_K` (graft outlier)
- Total: 713 tensors, ~21.31 GB
- `qwen35moe.block_count = 41` and `qwen35moe.nextn_predict_layers = 1`
confirm the MTP head is grafted in

The NVFP4 file was produced by patching llama.cpp's
`tools/quantize/quantize.cpp` and `src/llama-quant.cpp` to:

1. Register `LLAMA_FTYPE_MOSTLY_NVFP4` (value 39) in `QUANT_OPTIONS` so the
CLI accepts `NVFP4` as an ftype argument (the upstream `QUANT_OPTIONS`
table omits NVFP4 even though `llama.h` defines the constant), AND
2. Route 3D MoE tensors (`tensor->ne[2] > 1`) to `GGML_TYPE_NVFP4` and 2D
tensors to `GGML_TYPE_Q8_0` — mirroring the routing block that MXFP4\_MOE
already has at `llama-quant.cpp:461`.

Without fix (2), the quantizer would default every tensor to NVFP4 (yielding
390 NVFP4 tensors), losing Q8\_0 accuracy on the 2D linears.

## Licensing

This model is licensed under **MIT**, the same license as the original
[deepreinforce-ai/Ornith-1.0-35B](https://huggingface.co/deepreinforce-ai/Ornith-1.0-35B).
See [LICENSE](https://huggingface.co/s-batman/Ornith-1.0-35B-NVFP4-MTP-GGUF/tree/main/LICENSE) for the full text.

The MTP heads originate from
[Qwen/Qwen3.6-35B-A3B](https://huggingface.co/Qwen/Qwen3.6-35B-A3B)
(Apache 2.0). The MTP block was sourced via the
[unsloth/Qwen3.6-35B-A3B-MTP-GGUF](https://huggingface.co/unsloth/Qwen3.6-35B-A3B-MTP-GGUF)
redistribution, which is permitted under the original Apache 2.0 terms.

## Citation

```bibtex
@misc{ornith-35b,
    title = {{Ornith-1.0-35B}: Agentic Coding, Open to All},
    url = {https://deep-reinforce.com/ornith_1_0.html},
    author = {{DeepReinforce Team}},
    year = {2026}
}

@misc{qwen3.5,
    title = {{Qwen3.5-35B-A3B}: Post-trained Mixture-of-Experts Language Model},
    url = {https://huggingface.co/Qwen/Qwen3.5-35B-A3B},
    author = {{Qwen Team}},
    year = {2026}
}

@misc{qwen3.6,
    title = {{Qwen3.6-35B-A3B}: Qwen3.5 Post-trained with MTP Heads},
    url = {https://huggingface.co/Qwen/Qwen3.6-35B-A3B},
    author = {{Qwen Team}},
    year = {2026}
}
```

## Acknowledgements

- **DeepReinforce Team** for training and releasing
[Ornith-1.0-35B](https://huggingface.co/deepreinforce-ai/Ornith-1.0-35B)
- **Alibaba Qwen Team** for the Qwen3.5 and Qwen3.6 base models
- **Unsloth** for the
[MTP-enabled GGUF redistribution](https://huggingface.co/unsloth/Qwen3.6-35B-A3B-MTP-GGUF)
that made the graft possible
- **llama.cpp** for FP4 tensor core support and the `draft-mtp`
speculative decoding implementation

Downloads last month28,600

GGUF

Model size

36B params

Architecture

qwen35moe

Chat template

Hardware compatibility

[Log In](https://huggingface.co/login?next=https%3A%2F%2Fhuggingface.co%2Fs-batman%2FOrnith-1.0-35B-NVFP4-MTP-GGUF) to add your hardware

4-bit

MTPMXFP4\_MOE

20.3 GBMTPNVFP4

21.3 GB

Inference Providers [NEW](https://huggingface.co/docs/inference-providers)

[Text Generation](https://huggingface.co/tasks/text-generation "Learn more about text-generation")

This model isn't deployed by any Inference Provider. [🙋Ask for provider support](https://huggingface.co/spaces/huggingface/InferenceSupport/discussions/new?title=s-batman/Ornith-1.0-35B-NVFP4-MTP-GGUF&description=React%20to%20this%20comment%20with%20an%20emoji%20to%20vote%20for%20%5Bs-batman%2FOrnith-1.0-35B-NVFP4-MTP-GGUF%5D(%2Fs-batman%2FOrnith-1.0-35B-NVFP4-MTP-GGUF)%20to%20be%20supported%20by%20Inference%20Providers.%0A%0A(optional)%20Which%20providers%20are%20you%20interested%20in%3F%20(Novita%2C%20Hyperbolic%2C%20Together%E2%80%A6)%0A)

## Model tree for s-batman/Ornith-1.0-35B-NVFP4-MTP-GGUF

Base model

[deepreinforce-ai/Ornith-1.0-35B](https://huggingface.co/deepreinforce-ai/Ornith-1.0-35B)

Quantized

( [109](https://huggingface.co/models?other=base_model:quantized:deepreinforce-ai/Ornith-1.0-35B))

this model

## Paper for s-batman/Ornith-1.0-35B-NVFP4-MTP-GGUF

[Paper • 2004.14525 •Published Apr 30, 2020](https://huggingface.co/papers/2004.14525)

StripeM-Inner

Built with Multi-Token Prediction (MTP) to speed up inference.

Built with Multi-Token Prediction (MTP) to speed up inference.

Inference providers allow you to run inference using different serverless providers.