# Docker Resource Limits — Research & Calculation Method

## Measuring Actual Container Memory

`docker stats --no-stream` may return empty or hang. Use these alternatives:

### Method 1: Host PID VmRSS (for containers with host PIDs)
```bash
pid=$(docker inspect <container> --format '{{.State.Pid}}' 2>/dev/null)
cat /proc/$pid/status 2>/dev/null | grep VmRSS
```

### Method 2: Cgroup memory.current (for systemd-managed containers)
```bash
find /sys/fs/cgroup -name "memory.current" -path "*<container-name>*" 2>/dev/null | head -1
cat <path>  # returns bytes
```

### Method 3: Containerd shim PIDs
```bash
ps aux | grep containerd-shim | grep -v grep
# Each shim manages one container; check /proc/<shim_pid>/status
```

## Calculating Available RAM for Containers

**Key principle:** Host processes (llama-server, Redis, etc.) are NOT in Docker and always get priority. Containers share remaining RAM.

```
Total RAM: 64 GB
- llama-server (host): ~28 GB (model weights + KV cache)
- Open WebUI (host): ~0.8 GB
- llama-swap (host): ~0.2 GB
- Hermes gateway (host): ~0.3 GB
- Hermes dashboard (host): ~0.2 GB
- Redis (host): ~0.005 GB
- Docker daemon + shims: ~0.05 GB
- OS overhead: ~1 GB
= ~30.4 GB used by host
= ~33.6 GB available for containers
```

**llama-server VRAM is separate:** The RTX 2080 Ti's 11 GB VRAM is managed by NVIDIA driver/CUDA, NOT Docker cgroups. Docker memory limits only apply to host RAM.

## Recommended Limits (per-container)

| Container | Observed RSS | --memory | --memory-reservation | --memory-swap | Rationale |
|-----------|-------------|----------|---------------------|---------------|-----------|
| open-webui | 844 MB | 2g | 1g | 2g | Python + embeddings + vector DB. 2x headroom for model loading spikes. |
| nexstream-scraper | 35 MB | 1g | 512m | 1g | Node.js scraper. Spikes during scraping bursts. |
| searxng-core | 72 MB | 512m | 256m | 512m | Python web search. Worker uses 72 MB. 7x headroom. |
| open-terminal | 44 MB | 512m | 256m | 512m | ttyd terminal server. 44 MB observed. |
| searxng-valkey | ~16 MB | 256m | 128m | 256m | Redis-compatible cache. Tiny footprint. |

**Total container memory: ~4.5 GB out of 33.6 GB available (13%).**

## Docker Run Flags

```bash
--memory=<hard_limit> --memory-reservation=<soft_limit> --memory-swap=<hard_limit>
--log-opt max-size=10m --log-opt max-file=3
--restart=unless-stopped
--cpus=<limit>  # optional, prevents CPU starvation of host processes
```

Setting `--memory-swap` equal to `--memory` prevents swap usage entirely.

## Why These Numbers (Research)

- **Docker memory limits are soft by default:** When a container hits its limit, the kernel tries to reclaim memory first before OOM-killing. `--memory-reservation` at 50% of `--memory` gives the kernel an early pressure hint.
- **Docker memory limits DON'T restrict GPU VRAM:** The NVIDIA driver manages GPU memory independently. Your llama-server's 28 GB RSS is host RAM for model weights (Q4_K_M 35B MoE = ~22 GB) + KV cache (q8_0 = ~6 GB) + CUDA pinned memory overhead.
- **Without memory limits, a single container can consume all host RAM** and OOM-kill the llama-server or host processes. This is the primary risk.

## Docker Logging Rotation

Default json-file driver has no size limits. Without rotation, a busy container can fill your NVMe in days.

```bash
# Per-container
docker run --log-opt max-size=10m --log-opt max-file=3 ...

# Global (in /etc/docker/daemon.json)
{
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "10m",
    "max-file": "3"
  }
}
```

## Docker Restart Policy

Without `--restart=unless-stopped`, containers that crash or become unhealthy stay down. Your nexstream-scraper was unhealthy for 2+ days with no recovery.

```bash
--restart=unless-stopped
```

## Docker Network Cleanup

Unused Docker networks accumulate over time. Each creates iptables rules and consumes kernel resources.

```bash
docker network prune
```

Check which networks are unused:
```bash
docker network ls --filter type=custom
for net in $(docker network ls --filter type=custom --format "{{.Name}}"); do
  containers=$(docker network inspect "$net" --format '{{range .Containers}}{{.Name}} {{end}}' 2>/dev/null)
  echo "$net -> $containers"
done
```

## Docker Volume Cleanup

Dangling volumes waste space:
```bash
docker volume ls --filter dangling=true
docker volume prune
```

Check volume sizes:
```bash
sudo du -sh /var/lib/docker/volumes/*/_data 2>/dev/null | sort -rh
```