# Open WebUI Audit Checklist

Comprehensive end-to-end audit methodology for any Open WebUI deployment. Run these checks in order.

## 1. Installation & Version

- [ ] Check container: `docker inspect open-webui --format '{{.Config.Image}}'`
- [ ] Check build hash: `docker exec open-webui env | grep WEBUI_BUILD_VERSION`
- [ ] Compare against latest release: check `https://github.com/open-webui/open-webui/releases`
- [ ] Verify data volume is a bind mount (not named volume): check compose file `volumes:` section
- [ ] Verify health check: `curl -s -o /dev/null -w "%{http_code}" http://127.0.0.1:3081/health`
- [ ] Check DB migration status: `docker exec open-webui python3 -c "import sqlite3; c=sqlite3.connect('/app/backend/data/webui.db'); print(c.execute('SELECT * FROM alembic_version').fetchone())"`

## 2. Config Table Schema

**Check which schema version:**
```bash
docker exec open-webui python3 -c "
import sqlite3
conn = sqlite3.connect('/app/backend/data/webui.db')
# Try old schema
try:
    r = conn.execute('SELECT data FROM config LIMIT 1').fetchone()
    print('PRE-v0.10 schema (single JSON blob)')
except:
    print('v0.10+ schema (per-key rows)')
conn.close()
"
```

- [ ] If pre-v0.10, all DB scripts use `SELECT id, data FROM config LIMIT 1` pattern
- [ ] If v0.10+, DB scripts must use `WHERE key='openai.X'` pattern instead

## 3. Config Settings Audit

- [ ] `function_calling: false` (local llama.cpp models cannot use tool calling)
- [ ] `builtin_tools: false` (same reason)
- [ ] `web_search: true` (if SearXNG is connected)
- [ ] Model IDs match llama-swap fleet (no orphans)
- [ ] `WEBUI_SECRET_KEY` is set (session persistence)
- [ ] `CORS_ALLOW_ORIGIN=*` (LAN/Tailscale streaming)

**Check from DB:**
```python
# Pre-v0.10:
import sqlite3, json
conn = sqlite3.connect('webui.db')
data = json.loads(conn.execute('SELECT data FROM config LIMIT 1').fetchone()[0])
print(data['openai']['model_capabilities'])

# v0.10+:
cur = conn.execute("SELECT key, value FROM config WHERE key LIKE 'openai.model_capabilities.%'")
for k,v in cur: print(f"{k}: {v}")
```

## 4. Model Fleet Audit

- [ ] All llama-swap models present in OWUI model table
- [ ] No stale models in OWUI that aren't in llama-swap
- [ ] Model names match exactly (no suffix drift like `-q6` vs `-mtp-q5`)
- [ ] The `model` table, `config` key `openai.model_ids`, and `config` key `openai.api_configs["0"].model_ids` are all in sync

**Sync verification:**
```python
import sqlite3, json, urllib.request

# Get llama-swap models
r = urllib.request.urlopen('http://127.0.0.1:9292/v1/models')
fleet = [m['id'] for m in json.loads(r.read())['data']]

conn = sqlite3.connect('/app/backend/data/webui.db')

# Check v0.10+ config
if conn.execute("PRAGMA table_info(config)").fetchall()[0][1] == 'key':
    val = json.loads(conn.execute("SELECT value FROM config WHERE key='openai.model_ids'").fetchone()[0])
    print(f'Config model_ids: {val}')
    val2 = json.loads(conn.execute("SELECT value FROM config WHERE key='openai.api_configs'").fetchone()[0])
    api_models = val2.get('0', {}).get('model_ids', [])
    print(f'API configs model_ids: {api_models}')
else:
    data = json.loads(conn.execute('SELECT data FROM config LIMIT 1').fetchone()[0])
    print(f'Config model_ids: {data["openai"]["model_ids"]}')

# Check model table
cur = conn.execute('SELECT base_model_id FROM model')
db_models = [r[0] for r in cur]
print(f'Model table: {db_models}')

# Find drift
stale = set(db_models) - set(fleet)
missing = set(fleet) - set(db_models)
if stale: print(f'STALE in DB: {stale}')
if missing: print(f'MISSING from DB: {missing}')
```

## 5. Environment Variables

Check all env vars are properly set in the running container:
```bash
docker exec open-webui env | sort
```

**Critical vars to verify:**
| Variable | Expected | Impact if missing |
|----------|----------|-------------------|
| `WEBUI_SECRET_KEY` | Set (auto-gen or explicit) | Sessions invalidate on restart; MCP OAuth breaks |
| `BYPASS_WEB_SEARCH_EMBEDDING_AND_RETRIEVAL` | `true` | Web search results may not reach model |
| `WEB_SEARCH_RESULT_COUNT` | `10` (not 5) | Too few results for good context |
| `ENABLE_QUERIES_CACHE` | `true` | Redundant inference costs |
| `CORS_ALLOW_ORIGIN` | `*` on LAN | Streaming broken from non-localhost |
| `AIOHTTP_CLIENT_TIMEOUT` | `300` | Chat hangs forever on slow model |
| `AIOHTTP_POOL_CONNECTIONS_PER_HOST` | `2-4` | Request pileup |
| `FORWARDED_ALLOW_IPS` | `*` behind proxy | IP forwarding breaks |

## 6. Web Search

- [ ] SearXNG container running and healthy: `docker ps --filter name=searxng`
- [ ] Test SearXNG standalone: `curl -s "http://127.0.0.1:8080/search?q=test&format=json" | python3 -c "import sys,json; d=json.load(sys.stdin); print(f'Results: {len(d.get(\"results\",[]))}')"`
- [ ] Test OWUI search API: POST to `/api/v1/retrieval/process/web/search`
- [ ] Verify `BYPASS_WEB_SEARCH_EMBEDDING_AND_RETRIEVAL=true` reaches model

## 7. Tool/Function Calling

- [ ] If using local llama.cpp models: `function_calling=false` in DB config
- [ ] If using cloud models with tool support: `function_calling` set appropriately per model
- [ ] Test tool calling: send message with a tool-capable model and check response for `tool_calls`

## 8. Task Model

- [ ] A fast, small model is configured as Task Model (Local)
- [ ] Task Model is NOT one of the main chat models (prevents resource contention)

## 9. Security

- [ ] `ENABLE_SIGNUP=false` (disabled after admin created)
- [ ] `ENABLE_COMMUNITY_SHARING=false`
- [ ] `SCARF_NO_ANALYTICS=true`
- [ ] `DO_NOT_TRACK=true`
- [ ] `ANONYMIZED_TELEMETRY=false`

## 10. v0.10+ New Features Check

If on v0.10+, verify these are configured:
- [ ] `CONTEXT_COMPACTION_ENABLED=true` — critical for long coding sessions
- [ ] `CONTEXT_COMPACTION_THRESHOLD=0.8`
- [ ] `ENABLE_CODE_INTERPRETER=true` — Pyodide sandbox
- [ ] Explorer MCP connections (Admin → Settings → External Tools)
- [ ] Knowledge bases created (Workspace → Knowledge)
- [ ] Event system configured (Admin → Settings → Events)