[Skip to content](https://github.com/NousResearch/hermes-agent/issues/4379#start-of-content)

You signed in with another tab or window. [Reload](https://github.com/NousResearch/hermes-agent/issues/4379) to refresh your session.You signed out in another tab or window. [Reload](https://github.com/NousResearch/hermes-agent/issues/4379) to refresh your session.You switched accounts on another tab or window. [Reload](https://github.com/NousResearch/hermes-agent/issues/4379) to refresh your session.Dismiss alert

{{ message }}

### Uh oh!

There was an error while loading. [Please reload this page](https://github.com/NousResearch/hermes-agent/issues/4379).

[NousResearch](https://github.com/NousResearch)/ **[hermes-agent](https://github.com/NousResearch/hermes-agent)** Public

- [Notifications](https://github.com/login?return_to=%2FNousResearch%2Fhermes-agent) You must be signed in to change notification settings
- [Fork\\
37.8k](https://github.com/login?return_to=%2FNousResearch%2Fhermes-agent)
- [Star\\
208k](https://github.com/login?return_to=%2FNousResearch%2Fhermes-agent)


# Token overhead analysis: 73% of each API call is fixed overhead (~13.9K tokens) — data + suggestions\#4379

[New issue](https://github.com/login?return_to=https://github.com/NousResearch/hermes-agent/issues/4379)

Copy link

[New issue](https://github.com/login?return_to=https://github.com/NousResearch/hermes-agent/issues/4379)

Copy link

Open

[#48622](https://github.com/NousResearch/hermes-agent/pull/48622)

Open

[Token overhead analysis: 73% of each API call is fixed overhead (~13.9K tokens) — data + suggestions](https://github.com/NousResearch/hermes-agent/issues/4379#top)#4379

[#48622](https://github.com/NousResearch/hermes-agent/pull/48622)

Copy link

Labels

[P2Medium — degraded but workaround exists](https://github.com/NousResearch/hermes-agent/issues?q=state%3Aopen%20label%3A%22P2%22) Medium — degraded but workaround exists [comp/agentCore agent runtime: loop, agent\_init, prompt builder, context-compression, responses endpoint](https://github.com/NousResearch/hermes-agent/issues?q=state%3Aopen%20label%3A%22comp%2Fagent%22) Core agent runtime: loop, agent\_init, prompt builder, context-compression, responses endpoint [comp/toolsTool registry, model\_tools, toolsets](https://github.com/NousResearch/hermes-agent/issues?q=state%3Aopen%20label%3A%22comp%2Ftools%22) Tool registry, model\_tools, toolsets [type/perfPerformance improvement or optimization](https://github.com/NousResearch/hermes-agent/issues?q=state%3Aopen%20label%3A%22type%2Fperf%22) Performance improvement or optimization

## Description

[![@Bichev](https://avatars.githubusercontent.com/u/1519477?v=4&size=48)](https://github.com/Bichev)

[Bichev](https://github.com/Bichev)

opened [on Mar 31on Apr 1, 2026](https://github.com/NousResearch/hermes-agent/issues/4379#issue-4183166839)

Issue body actions

## Summary

I built a [monitoring dashboard](https://github.com/Bichev/hermes-dashboard) to profile token consumption on a Hermes v0.6.0 deployment running Telegram + WhatsApp + Cron gateways. After analyzing 6 request dumps from `~/.hermes/sessions/`, I found that **73% of every API call is fixed overhead** that doesn't change between requests — regardless of which model or provider is used.

## Data

### Per-Request Token Breakdown (from request dumps)

| Component | Tokens | % of Avg Request |
| --- | --- | --- |
| Tool definitions (31 tools) | 8,759 | 46.1% |
| System prompt (SOUL.md + skills catalog) | 5,176 | 27.2% |
| Messages (conversation context) | 3,000–8,775 | 26.7% avg |
| **Total per request** | **~17,000–23,000** |  |

The fixed overhead (system prompt + tool definitions) is **~13,935 tokens per API call**, paid before any conversation content is processed.

### Tool Definition Costs

All 31 tools from `_HERMES_CORE_TOOLS` are loaded for every platform:

| Tool | Tokens |
| --- | --- |
| `cronjob` | 729 |
| `delegate_task` | 699 |
| `skill_manage` | 699 |
| `terminal` | 693 |
| `execute_code` | 629 |
| `session_search` | 563 |
| `memory` | 536 |
| `search_files` | 438 |
| `patch` | 375 |
| `todo` | 339 |
| 11x `browser_*` tools | 1,258 combined |

### Real-World Impact

In a single evening with 3 active gateway sessions:

| Session | Platform | Messages | Est. API Calls | Est. Input Tokens |
| --- | --- | --- | --- | --- |
| Chat session | Telegram | 168 | ~84 | ~1.6M |
| Group chat | WhatsApp | 122 | ~61 | ~1.2M |
| Group chat | WhatsApp | 64 | ~32 | ~574K |

**Total estimated input: ~3.9M tokens across 10 sessions (~207 API calls)**

For agentic coding tasks with hundreds of tool calls, the overhead compounds further:

| Scenario | API Calls | Fixed Overhead Alone | Est. Total Input |
| --- | --- | --- | --- |
| Feature implementation | 100 | 1.4M tokens | ~4M tokens |
| Large refactor | 500 | 7M tokens | ~25M tokens |
| Full project build | 1,000 | 14M tokens | ~60M tokens |

The dollar cost depends on the model/provider, but the token overhead is constant — these 13.9K tokens are sent on every call whether using Sonnet, Haiku, Llama, or any other model via OpenRouter.

## Root Causes

### 1\. All platforms share `_HERMES_CORE_TOOLS` (toolsets.py)

`hermes-telegram`, `hermes-whatsapp`, `hermes-discord`, `hermes-slack`, and `hermes-signal` all resolve to the same `_HERMES_CORE_TOOLS` list. This means a WhatsApp message in a group chat loads all 11 `browser_*` tools (1,258 tokens) even though browser automation isn't usable from a messaging platform.

The infrastructure for per-platform differentiation already exists — `platform_toolsets` in config.yaml maps platforms to toolset names, and `toolsets.py` has modular toolset definitions. But the actual toolset definitions (`hermes-telegram`, `hermes-whatsapp`, etc.) all point to the same shared list.

### 2\. Skills catalog injected into every system prompt

The skills index adds ~2,200 tokens to the system prompt on every request, regardless of whether the conversation needs any skills. Skills are already accessible on-demand via `skill_view` / `skills_list` tools.

### 3\. Compression threshold may be too conservative

Default `compression.threshold: 0.5` means context doesn't get compressed until it exceeds 50% of max tokens. Combined with `protect_last_n: 20`, long messaging sessions accumulate full message history for many turns before compression kicks in.

## Suggestions

These are ordered by impact-to-effort ratio:

### Quick win: Platform-aware tool filtering

Create lighter toolset variants for messaging platforms that exclude tools not usable in that context:

```
_HERMES_MESSAGING_TOOLS = [t for t in _HERMES_CORE_TOOLS if not t.startswith("browser_")]
```

This saves **~1,258 tokens per request** on all messaging platforms with a 1-line change. The existing `platform_toolsets` config already maps platforms to toolset names, so users could override this.

### Medium effort: Lazy skills loading

Don't inject the skills index into the system prompt. The agent already has `skills_list` and `skill_view` tools — it can discover skills on demand. This saves **~2,200 tokens per request**.

### Low effort: Document compression tuning

The current defaults (`threshold: 0.5`, `protect_last_n: 20`) are conservative for messaging use cases where conversations can be 100+ messages. Documenting recommended settings for high-volume messaging (e.g., `threshold: 0.3`, `protect_last_n: 10`) would help users reduce costs.

## Methodology

Built a [transparent API proxy + dashboard](https://github.com/Bichev/hermes-dashboard) that:

1. Intercepts API calls via a transparent proxy
2. Logs token usage, cost, latency, and model to SQLite
3. Reads `~/.hermes/state.db` for session metadata (platform, message counts, tool calls)
4. Analyzes `~/.hermes/sessions/request_dump_*.json` files for per-component token breakdown

The dashboard is open source and works with any Hermes installation, any model, any provider.

## Environment

- Hermes Agent v0.6.0
- Ubuntu 24.04, DigitalOcean VPS
- Model: `anthropic/claude-sonnet-4-5` via OpenRouter (findings apply to any model — overhead is in the prompt, not model-specific)
- Gateways: Telegram, WhatsApp, Cron
- Honcho: not active (zero overhead confirmed)

👀React with 👀1themeadery

## Activity

[Next](https://github.com/NousResearch/hermes-agent/issues/4379?timeline_page=1)

[![](https://avatars.githubusercontent.com/u/48355244?s=64&u=69258b3a128c635cca97f62938f20792c66f6cf3&v=4)fathah](https://github.com/fathah)

mentioned this [on Apr 1on Apr 1, 2026](https://github.com/NousResearch/hermes-agent/issues/4379#event-5595902573)

- [feat(skills): smart ranking, usage tracking, and lifecycle management #4406](https://github.com/NousResearch/hermes-agent/pull/4406)


### alexferrari88 commented on Apr 2on Apr 2, 2026

[![@alexferrari88](https://avatars.githubusercontent.com/u/49028826?u=e11ccd8958051181733cab463cc0ebabfa98fc26&v=4&size=48)](https://github.com/alexferrari88)

[alexferrari88](https://github.com/alexferrari88)

[on Apr 2on Apr 2, 2026](https://github.com/NousResearch/hermes-agent/issues/4379#issuecomment-4175799251)

Contributor

More actions

Thank you for your analysis. I have also notices that Hermes Agent consumes Codex tokens incredibly fast for the simplest request. I think we should introduce something like Claude Code's [Tool search tool](https://platform.claude.com/docs/en/agents-and-tools/tool-use/tool-search-tool).

I disagree on disabling the browser tools based on the messaging platform since I often use Telegram to spawn browser tools based jobs.

👍React with 👍1kamjin3086

### lbatalha commented on Apr 2on Apr 2, 2026

[![@lbatalha](https://avatars.githubusercontent.com/u/5883822?v=4&size=48)](https://github.com/lbatalha)

[lbatalha](https://github.com/lbatalha)

[on Apr 2on Apr 2, 2026](https://github.com/NousResearch/hermes-agent/issues/4379#issuecomment-4176766888)

Last edited by lbatalha

Contributor

More actions

FYI if you read the [docs](https://hermes-agent.nousresearch.com/docs/reference/toolsets-reference?_highlight=toolse) or look at the config file, you can define which tools are available to the agent _per platform_:

```
platform_toolsets:
  cli:
  - browser
  - clarify
  - code_execution
  - cronjob
[...]
  discord:
  - browser
  - clarify
  - code_execution
[...]
```

Also tool definitions dont change with every request, so they are cached by the provider:

[https://developers.openai.com/api/docs/guides/prompt-caching#what-can-be-cached](https://developers.openai.com/api/docs/guides/prompt-caching#what-can-be-cached)

(they are a separate field in the completions api)

You can see it here with llama.cpp:

![Image](https://private-user-images.githubusercontent.com/5883822/573035874-dae26d6f-d602-41ad-a54a-a77a404c2775.png?jwt=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3ODMwNDA0MjMsIm5iZiI6MTc4MzA0MDEyMywicGF0aCI6Ii81ODgzODIyLzU3MzAzNTg3NC1kYWUyNmQ2Zi1kNjAyLTQxYWQtYTU0YS1hNzdhNDA0YzI3NzUucG5nP1gtQW16LUFsZ29yaXRobT1BV1M0LUhNQUMtU0hBMjU2JlgtQW16LUNyZWRlbnRpYWw9QUtJQVZDT0RZTFNBNTNQUUs0WkElMkYyMDI2MDcwMyUyRnVzLWVhc3QtMSUyRnMzJTJGYXdzNF9yZXF1ZXN0JlgtQW16LURhdGU9MjAyNjA3MDNUMDA1NTIzWiZYLUFtei1FeHBpcmVzPTMwMCZYLUFtei1TaWduYXR1cmU9YjMwYmRmMGU4NDY5NmIyN2Y2ODk1ZjRlZjMzNmMzNzdkM2RjMjgzZDEzODk3ZDdjNjhlMWZjN2M0ZDZhYWJkMyZYLUFtei1TaWduZWRIZWFkZXJzPWhvc3QmcmVzcG9uc2UtY29udGVudC10eXBlPWltYWdlJTJGcG5nIn0.FVfqZ9pcKa4YPzA-r3TadoD6FH2k2Ww-F3rJ854MuOs)

All of those requests contain the the full tool definitions, but only tokens in the `prompt` and `generated` column are actually being processed

Are you finding that for cloud providers, you are not seeing this caching behavior work?

### alexferrari88 commented on Apr 2on Apr 2, 2026

[![@alexferrari88](https://avatars.githubusercontent.com/u/49028826?u=e11ccd8958051181733cab463cc0ebabfa98fc26&v=4&size=48)](https://github.com/alexferrari88)

[alexferrari88](https://github.com/alexferrari88)

[on Apr 2on Apr 2, 2026](https://github.com/NousResearch/hermes-agent/issues/4379#issuecomment-4176985846)

Last edited by alexferrari88

Contributor

More actions

[@lbatalha](https://github.com/lbatalha) thank you for this clarification. This solves the worry about costs but nevertheless doesn't serve the pollution of the context window. That is why Anthropic introduces the tool search tool that I linked before ( [here](https://platform.claude.com/docs/en/agents-and-tools/tool-use/tool-search-tool)).

👍React with 👍3lbatalha, kamjin3086 and MartinBalerio

### lbatalha commented on Apr 2on Apr 2, 2026

[![@lbatalha](https://avatars.githubusercontent.com/u/5883822?v=4&size=48)](https://github.com/lbatalha)

[lbatalha](https://github.com/lbatalha)

[on Apr 2on Apr 2, 2026](https://github.com/NousResearch/hermes-agent/issues/4379#issuecomment-4178687152)

Contributor

More actions

Tool search does seem like an easy win.

I think the current toolsets are pretty granular already, but OP seems to disagree. Maybe having the ability to define custom toolsets would be nice?

For the compression tuning, I think 0.3 is a pretty brutal default to set, 0.5 is already relatively aggressive (when compared to some other agent harnesses) and it seems to strike a good balance for various use cases.

Hermes has the profiles feature which does not share the same config, so you could tune compression to whatever you feel is adequate for your use case (ie group chats vs cli coding agent).

### alexferrari88 commented on Apr 3on Apr 3, 2026

[![@alexferrari88](https://avatars.githubusercontent.com/u/49028826?u=e11ccd8958051181733cab463cc0ebabfa98fc26&v=4&size=48)](https://github.com/alexferrari88)

[alexferrari88](https://github.com/alexferrari88)

[on Apr 3on Apr 3, 2026](https://github.com/NousResearch/hermes-agent/issues/4379#issuecomment-4182747250)

Contributor

More actions

I'd work on the tool search but with 800+ PRs I doubt it would be even checked. Not being passive aggressive here, seriously. I cannot even imagine how much pressure there is on this project' maintainers right now

[![](https://avatars.githubusercontent.com/u/178506647?s=64&v=4)JuanDragin](https://github.com/JuanDragin)

mentioned this [on Apr 6on Apr 6, 2026](https://github.com/NousResearch/hermes-agent/issues/4379#event-5811932471)

- [\[Critical UX\] Memory persistence, token waste from session replay, state.db corruption, and environment hallucination field report from heavy production use #5563](https://github.com/NousResearch/hermes-agent/issues/5563)


[![](https://avatars.githubusercontent.com/u/3543?s=64&v=4)jbarket](https://github.com/jbarket)

mentioned this in 2 pull requests [on Apr 8on Apr 8, 2026](https://github.com/NousResearch/hermes-agent/issues/4379#event-5872073170)

- [feat(tools): progressive tool loading via tool\_search #6318](https://github.com/NousResearch/hermes-agent/pull/6318)

- [feat(agent): context-aware tool result budgeting #6339](https://github.com/NousResearch/hermes-agent/pull/6339)


[![](https://avatars.githubusercontent.com/u/131039422?s=64&u=214d7aec56d554b8cff64943632ccf98c9c11c0c&v=4)SHL0MS](https://github.com/SHL0MS)

mentioned this in 2 issues [on Apr 11on Apr 11, 2026](https://github.com/NousResearch/hermes-agent/issues/4379#event-5919005353)

- [\[Tracking\] /compress display bugs #7955](https://github.com/NousResearch/hermes-agent/issues/7955)

- [\[Feature\] Per-profile tool selection — opt-in lightweight toolsets #7961](https://github.com/NousResearch/hermes-agent/issues/7961)


### fathah commented on Apr 16on Apr 16, 2026

[![@fathah](https://avatars.githubusercontent.com/u/48355244?u=69258b3a128c635cca97f62938f20792c66f6cf3&v=4&size=48)](https://github.com/fathah)

[fathah](https://github.com/fathah)

[on Apr 16on Apr 16, 2026](https://github.com/NousResearch/hermes-agent/issues/4379#issuecomment-4258258878)

More actions

Check this PR

[#4406](https://github.com/NousResearch/hermes-agent/pull/4406)

### 32 remaining items

Load more

Loading

[Sign up for free](https://github.com/signup?return_to=https://github.com/NousResearch/hermes-agent/issues/4379)**to join this conversation on GitHub.** Already have an account? [Sign in to comment](https://github.com/login?return_to=https://github.com/NousResearch/hermes-agent/issues/4379)

## Metadata

## Metadata

### Assignees

No one assigned

### Labels

[P2Medium — degraded but workaround exists](https://github.com/NousResearch/hermes-agent/issues?q=state%3Aopen%20label%3A%22P2%22) Medium — degraded but workaround exists [comp/agentCore agent runtime: loop, agent\_init, prompt builder, context-compression, responses endpoint](https://github.com/NousResearch/hermes-agent/issues?q=state%3Aopen%20label%3A%22comp%2Fagent%22) Core agent runtime: loop, agent\_init, prompt builder, context-compression, responses endpoint [comp/toolsTool registry, model\_tools, toolsets](https://github.com/NousResearch/hermes-agent/issues?q=state%3Aopen%20label%3A%22comp%2Ftools%22) Tool registry, model\_tools, toolsets [type/perfPerformance improvement or optimization](https://github.com/NousResearch/hermes-agent/issues?q=state%3Aopen%20label%3A%22type%2Fperf%22) Performance improvement or optimization

### Type

No type

### Fields

No fields configured for issues without a type.

### Projects

No projects

### Milestone

No milestone

### Relationships

None yet

### Development

- [feat(tools): add stub\_mode to reduce main-thread tool schema tokensNousResearch/hermes-agent](https://github.com/NousResearch/hermes-agent/pull/48622)

### Participants

[![@houleixx](https://avatars.githubusercontent.com/u/530146?s=64&u=8609abc86e2e4e19edc497774d133ccb31af2c41&v=4)](https://github.com/houleixx)[![@Bichev](https://avatars.githubusercontent.com/u/1519477?s=64&v=4)](https://github.com/Bichev)[![@lbatalha](https://avatars.githubusercontent.com/u/5883822?s=64&v=4)](https://github.com/lbatalha)[![@konsisumer](https://avatars.githubusercontent.com/u/11262660?s=64&u=77d61b7b820926232ea4256feb3fecae34622abe&v=4)](https://github.com/konsisumer)[![@fathah](https://avatars.githubusercontent.com/u/48355244?s=64&u=69258b3a128c635cca97f62938f20792c66f6cf3&v=4)](https://github.com/fathah)

+9

## Issue actions

- ![](https://github.githubassets.com/assets/github-copilot-app-light-7138e992c731a2bb.png)Open in GitHub Copilot app

You can’t perform that action at this time.