# Local model, Hindsight memory, and dashboard troubleshooting

Use this when a local GGUF model behaves strangely, Hindsight is expected to be the memory backend but `hermes memory status` says built-in only, or the web dashboard is enabled but not reachable.

## 1. Local GGUF model gives bizarre/repetitive answers

Do **not** start with apologies, reporting upstream, or blind model switching. First verify the live serving stack.

Checklist:

```bash
# Current configured runtime model/provider/base_url
python3 - <<'PY'
import yaml, json
cfg=yaml.safe_load(open('/home/rurouni/.hermes/config.yaml')) or {}
print(json.dumps(cfg.get('model', {}), indent=2))
print(json.dumps(cfg.get('custom_providers', []), indent=2)[:4000])
PY

# Services, ports, GPU
systemctl --no-pager --plain list-units 'llama-server*.service' --all
ss -ltnp | grep -E ':808[0-9]|:8090' || true
nvidia-smi --query-gpu=memory.used,memory.total,utilization.gpu --format=csv,noheader,nounits || true

# Recent switching/gateway errors
tail -200 ~/.hermes/logs/model-switch.log
grep -iE 'APIConnectionError|base_url|model=' ~/.hermes/logs/gateway.log | tail -80 || true
```

Directly probe the suspected model endpoint before blaming Hermes history/prompting:

```bash
python3 - <<'PY'
import json, time, urllib.request
port=8088
model='phi-4-Q4_K.gguf'
payload={
  'model': model,
  'messages': [{'role':'user','content':'hi'}],
  'max_tokens': 80,
  'temperature': 0.2,
  'stream': False,
}
req=urllib.request.Request(
  f'http://127.0.0.1:{port}/v1/chat/completions',
  data=json.dumps(payload).encode(),
  headers={'Content-Type':'application/json'},
)
t=time.time()
data=json.loads(urllib.request.urlopen(req, timeout=120).read())
print('latency_s', round(time.time()-t, 2))
print(data['choices'][0]['message']['content'])
PY
```

### Phi-4 specific pitfall

`phi-4-Q4_K.gguf` metadata reports native context `phi3.context_length = 16384`. Running it at `-c 65536 --rope-scale 8.0` can produce unrelated, repetitive hallucinations even for `hi`. TurboKV itself is not necessarily the culprit: `turbo4` at native 16K tested normally.

Known-good Phi service shape:

```ini
ExecStart=/usr/local/bin/llama-server \
  -m /models/llm/phi/phi-4-Q4_K.gguf \
  --port 8088 \
  --host 0.0.0.0 \
  --cache-type-k turbo4 \
  --cache-type-v turbo4 \
  -ngl 99 \
  --no-mmap \
  --mlock \
  --jinja \
  --no-kv-offload \
  -c 16384 \
  --ctx-size 16384 \
  --threads 16 \
  --threads-batch 32 \
  --parallel 1
```

Also update the local provider catalog entry/context to 16384 so the picker/UI does not advertise fake 64K context for Phi.

## 2. Duplicate model switch watchers

Duplicate lines in `~/.hermes/logs/model-switch.log` often mean both system and user path watchers are active. This can create races.

```bash
systemctl --user is-active hermes-config.path || true
systemctl is-active hermes-config.path || true
```

Prefer one watcher. In Ray's setup the **user** watcher is the intended active one:

```bash
sudo systemctl disable --now hermes-config.path hermes-config.service
systemctl --user enable --now hermes-config.path
```

## 3. Hindsight expected but `hermes memory status` says built-in only

Two separate configs matter:

1. Hermes activates provider plugins via `memory.provider` in `~/.hermes/config.yaml`.
2. The Hindsight plugin reads its provider config from `$HERMES_HOME/hindsight/config.json`, then `~/.hindsight/config.json`, then env vars. A top-level `hindsight:` block in `config.yaml` alone is not sufficient.

Fix/verify:

```bash
# Install local Hindsight deps in the Hermes venv if missing
cd ~/.hermes/hermes-agent
uv pip install --python ~/.hermes/hermes-agent/venv/bin/python --quiet --upgrade hindsight-all

# Set Hermes provider selector
python3 - <<'PY'
import yaml, tempfile, os
from pathlib import Path
p=Path('/home/rurouni/.hermes/config.yaml')
cfg=yaml.safe_load(p.read_text()) or {}
cfg.setdefault('memory', {})['provider']='hindsight'
fd,tmp=tempfile.mkstemp(prefix='.config.', suffix='.yaml', dir=str(p.parent))
with os.fdopen(fd,'w') as f:
    yaml.safe_dump(cfg, f, sort_keys=False, allow_unicode=True)
    f.flush(); os.fsync(f.fileno())
os.replace(tmp, p)
PY

# Provider-specific config for local embedded Hindsight using SmolLM3 on 8087
mkdir -p ~/.hermes/hindsight ~/.hindsight/profiles
cat > ~/.hermes/hindsight/config.json <<'JSON'
{
  "mode": "local_embedded",
  "llm_provider": "openai_compatible",
  "llm_base_url": "http://127.0.0.1:8087/v1",
  "llm_model": "HuggingFaceTB_SmolLM3-3B-Q4_K_M.gguf",
  "bank_id": "hermes",
  "recall_budget": "mid",
  "memory_mode": "hybrid",
  "auto_recall": true,
  "auto_retain": true,
  "retain_every_n_turns": 1,
  "retain_async": true,
  "timeout": 120,
  "idle_timeout": 300
}
JSON

cat > ~/.hindsight/profiles/hermes.env <<'ENV'
HINDSIGHT_API_LLM_PROVIDER=openai
HINDSIGHT_API_LLM_API_KEY=
HINDSIGHT_API_LLM_MODEL=HuggingFaceTB_SmolLM3-3B-Q4_K_M.gguf
HINDSIGHT_API_LOG_LEVEL=info
HINDSIGHT_API_LLM_BASE_URL=http://127.0.0.1:8087/v1
HINDSIGHT_EMBED_DAEMON_IDLE_TIMEOUT=300
ENV

# Start/verify SmolLM backend
sudo systemctl start llama-server-smol.service
curl -s http://127.0.0.1:8087/v1/models | head -c 200

# Verify Hindsight plugin availability
~/.hermes/hermes-agent/venv/bin/python - <<'PY'
from plugins.memory.hindsight import _check_local_runtime
print(_check_local_runtime())
PY
hermes memory status
```

Expected status:

```text
Provider:  hindsight
Plugin:    installed ✓
Status:    available ✓
```

Restart gateway or start a fresh session after changing memory provider config.

## 4. Dashboard enabled but not on

Check the user service directly; do not assume enabled means running:

```bash
systemctl --user is-enabled hermes-dashboard.service || true
systemctl --user is-active hermes-dashboard.service || true
systemctl --user --no-pager status hermes-dashboard.service | sed -n '1,80p'
ss -ltnp | grep -E ':9119' || true
```

Known service shape for Ray's Tailscale-gated dashboard:

```ini
[Service]
Type=simple
ExecStart=/home/rurouni/.hermes/hermes-agent/venv/bin/hermes dashboard --host 0.0.0.0 --port 9119 --no-open --insecure
Restart=on-failure
RestartSec=5
Environment=HERMES_HOME=/home/rurouni/.hermes
```

Start/verify:

```bash
systemctl --user start hermes-dashboard.service
systemctl --user is-active hermes-dashboard.service
ss -ltnp | grep ':9119'
curl -sS --max-time 3 http://127.0.0.1:9119/api/health || true
```

`{"detail":"Unauthorized"}` from the health/API endpoint is acceptable: it proves the server is up and enforcing auth.