# Tool Calling with Local Models (llama.cpp + --jinja)

## Discovery (July 2026)

Previous fleet skill stated local models "do NOT support tool calling" — this was incorrect for `--jinja`-enabled Qwen3-based models.

## Verified Working Setup

### Inference Engine
- llama.cpp b9743+ with `--jinja` flag
- No special tool-calling build required — standard CUDA build with `llama-server` target
- The `--jinja` flag enables chat template processing that converts OpenAI `tools` parameter ↔ model native tool tokens

### Models Tested (RTX 2080 Ti 11 GB)

| Model | Arch | Tool Calling | Notes |
|-------|------|:-----------:|-------|
| Qwythos-9B-MTP-Q6 | Qwen3.5 dense | ✅ | Best balance: 96 t/s, 131K ctx, reliable |
| Ornith-9B-MTP-Q5 | Qwen3.5 dense | ✅ | 71 t/s, 96K ctx |
| Ornith-35B-APEX-MTP | Qwen3.5 MoE | ✅ | 28 t/s, 262K ctx — slow load but powerful |
| Gemma-26B-200K | Gemma4 MoE | ✅ | Uses separate draft model, tool calling works |
| Qwen3.6-35B-MTP | Qwen3.6 MoE | ❌ | Hallucinates instead of calling tools |

### Test Procedure

Direct API call to llama-swap:
```bash
curl -s http://127.0.0.1:9292/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "<model-id>",
    "messages": [{"role":"user","content":"What is the weather in Paris? Use the get_weather tool if needed."}],
    "tools": [{"type":"function","function":{"name":"get_weather","description":"Get weather for a city","parameters":{"type":"object","properties":{"city":{"type":"string"}},"required":["city"]}}}],
    "stream": false,
    "max_tokens": 200
  }' | python3 -c "import sys,json; d=json.load(sys.stdin); m=d['choices'][0]['message']; print('tool_calls:', m.get('tool_calls','NONE'))"
```

## Implications
1. Hermes Agent Bridge tool works with 4 of 5 local models
2. Terminal auto-invoke works with tool-capable models
3. Agentic web search is possible with these models
4. No cloud API needed for basic tool use
5. Set `function_calling: 'native'` per-model, not globally