# Hindsight Memory — Local Embedded Setup

Hindsight is a memory plugin for Hermes that provides a knowledge graph with entity resolution and multi-strategy retrieval. Local Embedded mode spins up a PostgreSQL daemon in the background and uses a local LLM for entity extraction and memory synthesis.

## Requirements

- **62GB+ RAM** recommended (daemon uses ~50-100MB idle, PostgreSQL + embedding model)
- **A small LLM endpoint** for background extraction (1-3B model on CPU works perfectly)
- **No external API key needed** — the extraction LLM can be a local OpenAI-compatible endpoint

## Architecture

```
Hermes session
    │
    ├─ Chat model (35B on GPU)  ← for actual conversation
    │
    └─ Hindsight daemon
         │
         ├─ PostgreSQL (local, auto-started)
         ├─ Built-in embedder (local, no API needed)
         └─ LLM for extraction (3B on CPU, separate port)
              ← SmolLM3-3B, Llama-3.2-3B, etc.
```

## Setup Steps

### 1. Start a small extraction model

Run a lightweight model on a separate port alongside your main model:

```bash
sudo systemctl start llama-server-smol.service   # e.g. SmolLM3-3B on port 8087
```

Verify it's responding:
```bash
curl http://127.0.0.1:8087/v1/models
```

### 2. Configure Hindsight

```bash
hermes config set memory.provider hindsight
hermes config set hindsight.mode local_embedded
hermes config set hindsight.llm_provider openai_compatible
hermes config set hindsight.llm_base_url http://127.0.0.1:8087/v1
hermes config set hindsight.llm_model HuggingFaceTB_SmolLM3-3B-Q6_K.gguf   # or whatever model you're using
hermes config set hindsight.auto_recall true
hermes config set hindsight.auto_retain true
```

### 3. Set the LLM API key

Local endpoints don't need auth, but Hindsight expects the env var:

```bash
echo "HINDSIGHT_LLM_API_KEY=not-needed" >> ~/.hermes/.env
```

### 4. Restart Hermes

Do `/reset` in the CLI or start a new session. The hindsight-client dependency auto-installs on first load. The daemon starts on first memory operation.

## Config Reference

| Key | Default | Notes |
|-----|---------|-------|
| `hindsight.mode` | `cloud` | `local_embedded` for fully local |
| `hindsight.llm_provider` | `openai` | Use `openai_compatible` for llama.cpp |
| `hindsight.llm_base_url` | — | Your local model endpoint |
| `hindsight.llm_model` | per-provider | The model name on the endpoint |
| `hindsight.auto_recall` | `true` | Auto-inject relevant memories each turn |
| `hindsight.auto_retain` | `true` | Save conversation to memory after each turn |
| `hindsight.recall_budget` | `mid` | `low` / `mid` / `high` — thoroughness of recall |
| `hindsight.recall_prefetch_method` | `recall` | `recall` (raw facts) or `reflect` (LLM synthesis) |

## Behavior

- **Daemon lifecycle:** Auto-starts on first use, auto-stops after 5 min idle
- **Startup logs:** `~/.hermes/logs/hindsight-embed.log`
- **Runtime logs:** `~/.hindsight/profiles/<profile>.log`
- **Hindsight web UI:** `hindsight-embed -p hermes ui start` (local embedded only)
- **Config file:** `~/.hermes/hindsight/config.json`

## Pitfalls

- **Extraction model must be running** before Hindsight starts. If the small model is down, extraction silently fails.
- **Don't use your main 35B** for extraction — it's overkill and wastes GPU memory. A 1-3B model on CPU handles this fine.
- **Quantize for CPU speed** — a 3B model at Q4_K_M (~1.8 GB) is about 30-40% faster on CPU than Q6_K (~2.4 GB) with negligible quality loss for entity extraction. The bottleneck is RAM bandwidth — less data to move = faster tokens.
- **Secondary models coexist** with the auto-model-switch system — the path watcher only stops the primary service, so your extraction model stays up.
- **Dummy API key** (`not-needed`) works for local llama.cpp endpoints since they don't authenticate.
- **Memory convergence** takes a few sessions — the knowledge graph gets smarter as more entities and relationships are extracted.