# Session 2026-06-05: Complete Server Audit

## System Profile
- **Host:** debian-ai (192.168.1.50, Tailscale 100.126.244.3)
- **CPU:** Intel Xeon E5-2697A v4 @ 2.60GHz (32 cores)
- **RAM:** 64 GB
- **GPU:** RTX 2080 Ti 11GB (compute 7.5, driver 550.163.01)
- **OS:** Debian 13, kernel 6.12.85+deb13-amd64
- **NVMe:** 238.5 GB (221 GB usable, 74G used / 136G avail after cleanup)
- **RAID0 /data:** 1.5 TB (sda + sdc, hot non-critical)
- **HDD /backup:** 1.8 TB (sdb1)
- **HDD /models:** 465.8 GB (sdd1)
- **HDD /archive:** 1.8 TB (sde1)

## Disk Cleanup Results (executed)
- Docker dangling images: 1.7 GB freed
- Docker unused images (nexstream:v2, golang:1.22): ~7.0 GB freed
- Docker build cache: 2.45 GB freed
- Apt autoremove (old kernel + libs): 111 MB freed
- Cursor sandbox cache (/data/tmp): ~15 GB freed
- /tmp junk (jadx, apk, androidx): ~0.6 GB freed
- **Total root NVMe freed: ~11 GB** (85G → 74G, 41% → 36%)
- **Swappiness:** set to 10 (was default 60), persisted in /etc/sysctl.conf

## Security Findings (not fixed — user to decide)
1. **No firewall** — 6 ports exposed on 0.0.0.0: 80, 22, 3091, 8000, 9119, 18789
2. **Open-terminal API key in plaintext** — visible via `ps aux`
3. **CORS_ALLOW_ORIGIN=*** on Open WebUI
4. **X11Forwarding yes** on SSH
5. **Fail2ban has no active jails**
6. **No Docker resource limits** on any container
7. **No Docker log rotation** — json-file with no size limits
8. **nexstream-scraper unhealthy** for 2+ days, no auto-recovery
9. **No backups** — /backup is empty, no cron jobs, no rsync
10. **PostgreSQL shared_buffers = 128MB** on 64GB RAM server
11. **Redis no maxmemory** limit
12. **Open WebUI dangerous features enabled** — code execution, code interpreter
13. **3 crash reboots in 10 days** (May 25, 26, then stable)

## Docker Container Memory (measured via cgroup/proc)
| Container | RSS | Notes |
|-----------|-----|-------|
| open-webui | 844 MB | uvicorn worker |
| nexstream-scraper | 35 MB | Node.js |
| searxng-core | 72 MB | Python worker |
| open-terminal | 44 MB | ttyd |
| searxng-valkey | ~16 MB | Redis-compatible |

## Host RAM Breakdown
| Process | RSS | Notes |
|---------|-----|-------|
| llama-server | ~28 GB | RTX 2080 Ti, 7.6 GB VRAM + 20 GB host RAM |
| Open WebUI | ~0.8 GB | uvicorn |
| llama-swap | ~0.2 GB | Router |
| Hermes gateway | ~0.3 GB | |
| Hermes dashboard | ~0.2 GB | |
| Redis | ~0.005 GB | |
| Docker daemon + shims | ~0.05 GB | |
| OS overhead | ~1 GB | |
| **Host total** | **~30.4 GB** | |
| **Available for containers** | **~33.6 GB** | 64 GB total |

## User Constraints
- **RAID0 on /data: DO NOT TOUCH** — user explicitly said "don't touch my raid 0"
- **Docker limits: leave alone** — user wants full GPU utilization for llama-cpp
- Backup on slow HDDs was discussed but not implemented

## Key Commands for Future Audits
```bash
# Container memory (cgroup-based)
find /sys/fs/cgroup -name "memory.current" -path "*<name>*" 2>/dev/null | head -1
cat <path>

# Host PID VmRSS
pid=$(docker inspect <container> --format '{{.State.Pid}}')
cat /proc/$pid/status | grep VmRSS

# Docker volumes (needs sudo)
sudo du -sh /var/lib/docker/volumes/*/_data 2>/dev/null | sort -rh

# Crash history
last reboot | head -5
journalctl --since "7 days ago" -p err --no-pager -n 20

# All listening ports
ss -tlnp

# Firewall
ufw status verbose 2>/dev/null || echo "no ufw"
iptables -L -n -v 2>/dev/null | head -20
```