--- name: model-routing description: "Smart model routing, urgency classification, cost guards, and escalation protocols for Hermes Agent. Determines which model/provider to use based on task type, urgency, and cost sensitivity." version: 1.0.0 tags: [model-routing, urgency, cost-management, escalation, sql-fast-path] --- # Model Routing — Smart Model Selection for Hermes ⚡ > **⚠️ STATUS: Reference Architecture — Not Actively Wired In** > This skill was developed as a proof-of-concept for automatic model routing based on urgency, task type, and cost. The scripts are archived at `~/.hermes/archived-routing-scripts/` and are NOT wired into daily Hermes operation. The working cost-saving strategy is simpler: keep local as default, let `fallback_providers` handle failures, and switch models consciously via `/model`. See the "Lessons Learned" section before building on this. > > **Why it's archived:** The routing pipeline adds latency (~2-3 Python process launches per message) with no benefit for everyday chat, Q&A, and most tool-assisted work. The user's local models handle 95%+ of tasks, and DeepSeek V4 Flash is cheap enough ($0.03/session) that the cost guard machinery was over-engineering for the actual burn rate. Determines which model/provider to use based on task type, urgency, and cost sensitivity. Prevents wasted API calls, ensures fast responses for urgent work, and keeps costs low for routine tasks. ## Core Principle **SmolLM3 CPU is the traffic controller. Local coder models do daily coding. Qwen3.6-35B handles hard local thinking. ChatGPT Auth is the first premium lane. DeepSeek/OpenRouter paid models are backup only. Paid big models require approval. Urgent work can use faster premium paths. No-rush work stays local and cheap.** ## Routing Priority Order 1. **Local models first** — always, unless urgency is explicit 2. **ChatGPT Auth** — for urgent SQL/DAX/VBA or when user says "fast"/"urgent" 3. **OpenRouter/DeepSeek** — backup only 4. **Qwen3-235B, MiniMax M2.5/M2.7** — require explicit user approval ## Urgency Classification Classify the user's message into one of these urgency levels: | Signal Words | Mode | Action | |---|---|---| | "urgent", "ASAP", "fast", "quick", "need this now", "emergency", "critical" | **Instant** | ChatGPT Auth first for SQL/DAX/VBA; OpenRouter coder for coding | | "today", "tonight", "by EOD", "deadline", "due today" | **Today** | Local coder first (Qwen2.5-Coder-7B); escalate to ChatGPT if local too slow | | "this week", "no hurry", "low priority", "background" | **This Week** | Qwen3.6-35B or Qwen3-Coder-30B local | | "no rush", "save cost", "whenever", "cheap", "local only", "minimize cost" | **Savings** | Qwen3-8B or Qwen2.5-Coder-7B local only | **Detection order matters:** Check savings signals FIRST (most specific), then this week, then instant, then today. This prevents "need" from matching "instant" when the user says "no rush save cost". ## SQL/DAX/VBA Fast Path User does a lot of SQL, DAX, VBA, warehouse reporting, dashboards, and productivity queries. Rules: - If small SQL/DAX/VBA fix with implied urgency → **ChatGPT Auth first** - No urgency stated → **Qwen3-Coder-30B local first** - User says "no rush" or "save cost" → **Qwen2.5-Coder-7B local** - Always provide corrected full query/code - Avoid over-explaining - If risk is high, run a second review ## Cost Guard Rules 1. **Local-first default** — all tasks start on local models unless urgency is explicit 2. **ChatGPT Auth gate** — only use for SQL/DAX/VBA with urgency, or when user says "fast"/"urgent" 3. **OpenRouter/DeepSeek** — backup only. Never first choice. 4. **Approval required** — Qwen3-235B, MiniMax M2.5, MiniMax M2.7, or any model >$0.01/request 5. **No ChatGPT for tiny tasks** — if a local model can handle it (classification, routing, simple rewrite), don't burn API credits 6. **No rush = local only** — if user says "no rush", "save cost", or "whenever", stay local 7. **Context packet required** — before any paid model call, show the packet so user can approve or redirect ## Model Selection Guide | Task Type | Urgency | Model | Fallback | Reason | |---|---|---|---|---| | Quick SQL/DAX/VBA fix | Instant | ChatGPT Auth | DeepSeek | Speed matters, small scope | | Urgent coding (deadline today) | Instant | ChatGPT Auth | OpenRouter qwen3-coder | Fastest premium path | | Daily coding (snippets, refactors) | Today | Qwen2.5-Coder-7B (8085) | Qwen3-Coder-30B (8083) | Fits GPU, fast, specialized | | Complex reasoning / architecture | Today | Qwen3.6-35B (8081) | OpenRouter qwen3-coder | Best local brain | | Hard local thinking | Today | Qwen3.6-35B (8081) | DeepSeek | User's stated priority | | Routing / classification | Any | Llama-3.2-3B (8086) | SmolLM3-3B (8087) | Better instruction following | | Memory / entity extraction | Any | SmolLM3-3B (8087) | Llama-3.2-3B (8086) | Designed for tool-use | | Quick summary / rewrite | No rush | Qwen3-8B (8084) | Qwen2.5-Coder-7B (8085) | Fast local, no API cost | | Creative writing / chat | No rush | Qwen3.5-9B (8082) | Qwen3.6-35B (8081) | Abliterated, fun, fast | | Heavy multi-file refactor | This week | Qwen3-Coder-30B (8083) | ChatGPT Auth | Best local coder | ## Context Packets Before using ChatGPT Auth, DeepSeek, OpenRouter, or paid models, create the appropriate packet: ### Full Packet (for paid model escalation) ``` GOAL: URGENCY: DEADLINE: RISK LEVEL: TASK SIZE: COST PRIORITY: FILES: ERROR: WHAT I TRIED: CONSTRAINTS: EXPECTED OUTPUT: WHY ESCALATION IS NEEDED: MODEL RECOMMENDED: RECOMMENDED PATH: WHY THIS PATH: ``` ### Quick Packet (SQL/DAX/VBA) ``` GOAL: QUERY/CODE: ERROR OR ISSUE: EXPECTED OUTPUT: DEADLINE: MODEL PATH: ``` ## Scripts (Archived) Three helper scripts are archived at `~/.hermes/archived-routing-scripts/`: - **`approval-guard.py`** — Blocks expensive models (Qwen3-235B, MiniMax M2.5/M2.7) without approval. Returns exit 0 = allowed, exit 1 = blocked. - **`urgency-router.py`** — Classifies urgency and recommends model path. Outputs JSON. Usage: `python3 ~/.hermes/archived-routing-scripts/urgency-router.py "user message here"` - **`context-packet.py`** — Generates formatted context packets. Usage: `python3 ~/.hermes/archived-routing-scripts/context-packet.py full [options]` or `python3 ~/.hermes/archived-routing-scripts/context-packet.py sql [options]` These are kept as reference in case the user ever wants to revisit automated routing. They are NOT loaded or invoked during normal operation. ## Config Setup Required config.yaml settings: ```yaml fallback_providers: - openrouter fallback_model: provider: openrouter model: qwen/qwen3-coder ``` API keys in `~/.hermes/.env`: ```bash export OPENROUTER_API_KEY="your-key-here" export DEEPSEEK_API_KEY="your-key-here" ``` ### Verification — Confirm Real State vs. Summary Claims When a session summary or previous agent claims that config changes weren't applied or that keys are missing, **verify the actual system state** before repeating those claims. Session search results are LLM-generated summaries and can be misleading: - `***` masked terminal output gets misinterpreted as a literal placeholder - A failed write attempt (credential guard blocking overwrite) gets summarized as "keys not set" when the real .env was already configured - Creating `.env.example` looks like first-time key setup when it's just a template **Check API keys are real (not placeholders):** ```bash # Modification time reveals if keys existed before/after a session stat ~/.hermes/.env | grep Modify # Confirm keys are non-empty (empty value after = means truly missing) grep -E "^OPENROUTER_API_KEY=|^DEEPSEEK_API_KEY=" ~/.hermes/.env | grep -v "^.*=$" ``` **Check config changes took effect:** ```bash grep -n "fallback_providers\|fallback_model\|display.personality" ~/.hermes/config.yaml ``` **Check scripts exist and validate config:** ```bash ls -la ~/.hermes/scripts/{approval-guard,urgency-router,context-packet}* python3 -c "import yaml; yaml.safe_load(open('/home/rurouni/.hermes/config.yaml')); print('Config valid')" ``` ## Pitfalls **Savings signals must be checked before instant signals.** The word "need" appears in both "need this now" (instant) and "no rush save cost" (savings). If you check instant first, "need" always matches instant and savings is never detected. Check savings → this week → instant → today. **Don't force local-first for urgent SQL/DAX/VBA.** The user does a lot of SQL/DAX/VBA work and may need it fast. If urgency is implied, use ChatGPT Auth first. Don't make them wait for a slow local model. **Approval guard blocks but doesn't auto-approve.** The script returns exit 1 for blocked models. The agent must present the context packet and get user approval before proceeding. **Context packet is mandatory before paid models.** Never use a paid model without showing the appropriate packet first. This gives the user visibility and control over spending. **ChatGPT Auth is not just an API key.** It requires OAuth setup via `hermes login --provider openai-codex`. Without OAuth, the ChatGPT Auth lane is unavailable and the router falls back to OpenRouter/DeepSeek. **Don't trust session search summaries about system state blindly.** When a summary says "API keys not set" or "X wasn't applied," verify against actual files on disk. Session summaries are generated by LLMs that can misinterpret `***` masked output as placeholders, infer "failed" from credential-guard blocks, or conflate template creation with first-time setup. Run `stat`, `grep`, and `yaml.safe_load` to confirm real state before repeating claims. ## Lessons Learned (May 2026) ### 1. Research before building When the user asks for a complex system (model routing, cost guards, multi-agent orchestration), **research existing solutions first** before writing any code. Check: - The Hermes skills hub (`hermes skills search`) - GitHub for similar projects - Whether Hermes itself already solves the problem (e.g., `fallback_providers`, `credential_pool_strategies`, built-in approvals) Building custom scripts that duplicate or overlap with built-in features wastes time and adds maintenance burden. ### 2. Consider the everyday experience A complex routing pipeline makes sense for automated task delegation but **adds latency for normal conversations**. Every script launch per message adds seconds of overhead. Ask: "Will this code run on every message, or only on specific task types?" If the former, it needs to be near-zero overhead or it will degrade the user experience. ### 3. Know the real costs before building cost guards DeepSeek V4 Flash pricing makes it nearly free for normal use: - Cached input: $0.0028/1M tokens - Uncached input: $0.14/1M tokens - Output: $0.28/1M tokens - Real-world cost: ~$0.03 per heavy 39-call session with 80K+ contexts Complex approval gating and context-packet generators were over-engineering for a burn rate of $0.03/session. Always measure actual spend before building cost-control infrastructure. ### 4. Use `verify_actual_state` before trusting session search summaries Session search returns LLM-generated summaries that can be misleading. A `***` masked env var, a credential-guard-blocked write, or a clean `.env.example` can all be misinterpreted as "keys not set." Always `stat` modified timestamps and `grep` actual values before trusting summaries. ### 5. Prefer built-in over custom Hermes already has: - `fallback_providers` — built-in, no scripts needed - `credential_pool_strategies` — key rotation - `approvals.mode` — command gating - `auto-model-switch` — physical service swapping Use these first. Only build custom scripts when the built-in features genuinely don't cover the use case. ## Related Skills - `auto-model-switch` — Physical model service swapping (ports 8081-8087) - `hermes-agent` — Provider setup, OAuth, config reference - `kanban-orchestrator` — Multi-agent routing (different domain: task decomposition)