---
name: docker-security
description: Specialized workflows for auditing, securing, and troubleshooting Docker containers on Linux hosts.
---

# Docker & Container Security

Specialized workflows for auditing, securing, and troubleshooting Docker containers on Linux hosts.

## Triggers
- Container is reported as "unhealthy"
- Container is exposed to the public internet unnecessarily
- Container logs show high volumes of 404/403 errors (bot scanning)
- Need to restrict service access to Tailscale/LAN

## Workflows

### 1. Investigating Unhealthy Containers
When a container is marked `unhealthy`:
1.  **Inspect Health Status:** Use `docker inspect <name> --format '{{json .State.Health}}'` to find the failing command and error output.
2.  **Check Logs:** Use `docker logs <name> --tail 100` to see application-level errors.
3.  **Verify Connectivity:** Attempt to reach the service from *inside* the container using `docker exec <name> <tool> <url>` (e.g., `curl` or `wget`).
    - *Pitfall:* Many minimal containers (like Alpine-based ones) do not have `curl` installed. Use `wget` as a fallback.
4.  **Check Resource Exhaustion:** Check `ps aux` on the host to see if the container process is consuming excessive CPU/RAM, which can cause health check timeouts.

### 2. Hardening Network Exposure
To prevent bot scanning and brute-force attacks:
- **Avoid `0.0.0.0` for sensitive services:** Never bind management interfaces (Dashboards, APIs, Scrapers) to `0.0.0.0` unless they are behind a firewall or proxy.
- **Bind to Localhost:** Use `-p 127.0.0.1:PORT:PORT` to restrict access to the host only.
- **Bind to Tailscale:** Use `-p 100.x.x.x:PORT:PORT` to restrict access to the Tailscale network.
- **Audit Ports:** Regularly run `docker ps` to verify which ports are mapped to `0.0.0.0`.

### 3. Full Port Exposure Audit

When a user reports performance issues, bot traffic, or you suspect unnecessary exposure, run this systematic audit:

**Step 1 — List ALL listening ports:**
```bash
ss -tlnp4 | grep LISTEN
```

**Step 2 — Identify services on 0.0.0.0 vs 127.0.0.1:**
```bash
echo "=== EXTERNALLY REACHABLE (0.0.0.0) ==="
ss -tlnp4 | grep "0.0.0.0:" | grep -v "127.0.0.1"
echo "=== SAFE (127.0.0.1) ==="
ss -tlnp4 | grep "127.0.0.1:"
```

**Step 3 — Cross-reference with Docker:**
```bash
docker ps --format "table {{.Names}}\t{{.Image}}\t{{.Ports}}\t{{.Status}}"
```

**Step 4 — Check Tailscale exposure:**
```bash
tailscale serve status 2>/dev/null
tailscale funnel status 2>/dev/null
```

**Step 5 — Verify iptables doesn't mask 0.0.0.0 binding:**
```bash
sudo iptables -L ts-input -n 2>/dev/null | head -10
# If you see: ACCEPT all -- 0.0.0.0/0 0.0.0.0/0
# Then ALL 0.0.0.0 services ARE reachable from the internet
```

**Step 6 — Systematically move each exposed service to 127.0.0.1:**

| Service Type | How to fix | Example |
|---|---|---|
| **Docker container (host network)** | Switch to bridge network + `extra_hosts` + `host.docker.internal` | Change `network_mode: host` → `ports: ["127.0.0.1:PORT:PORT"]`, update internal URLs to `host.docker.internal` |
| **Docker container (bridge with 0.0.0.0)** | Add `127.0.0.1:` prefix to port mapping | Change `"PORT:PORT"` → `"127.0.0.1:PORT:PORT"` |
| **Standalone process** | Kill process, restart with `--host 127.0.0.1` flag | `hermes dashboard --host 127.0.0.1 --port 9119` |
| **Systemd service** | Edit `ExecStart` in unit file, add `--listen 127.0.0.1:PORT` | `systemctl edit llama-swap --full`, change `--listen :PORT` → `--listen 127.0.0.1:PORT`, then `daemon-reload && restart` |
| **Config file** | Edit the `listen` directive | Change `listen: :9292` → `listen: 127.0.0.1:9292` |

## Pitfalls
- **The "Bot Loop":** High volumes of automated scanning (404/403 errors) can overwhelm a Node.js event loop, causing the application to become unresponsive and fail its internal health checks.
- **Permission Denied on `lsof`:** Running `lsof` on container processes from the host may require `sudo` or result in `Permission denied` for certain overlay filesystem paths.
- **Docker health check using `localhost` fails on IPv6:** Inside a Docker container, `localhost` resolves to `::1` (IPv6 loopback) first. If the app binds to `0.0.0.0` (IPv4 only), `wget http://localhost:PORT/health` fails with "Connection refused." **Fix:** Use `127.0.0.1` instead of `localhost` in the health check URL. Verify with `docker exec <name> wget -qO- http://127.0.0.1:PORT/health` before updating the compose file.
- **Minimal containers lack `curl`:** Many Alpine-based containers ship `wget` but not `curl`. Always try `wget` as the health check tool before assuming the endpoint is down.
- **Tailscale's `ts-input` iptables chain accepts ALL traffic:** The chain has an `ACCEPT all -- 0.0.0.0/0 0.0.0.0/0` rule. This makes the INPUT chain's DROP policy effectively meaningless — any service bound to `0.0.0.0` is reachable from the internet. Always verify with `ss -tlnp4` and compare against `ts-input`.
- **`network_mode: host` exposes the container to all host interfaces:** A container using host networking binds its internal ports to `0.0.0.0` on the host, making them reachable from the internet. Switch to bridge networking with explicit `127.0.0.1` port mapping to isolate it. When doing so, update internal service URLs from `127.0.0.1` to `host.docker.internal` (with `extra_hosts: - "host.docker.internal:host-gateway"`).

## References
- [Session: Nexstream Scraper Bot Attack](references/nexstream-bot-attack.md)
- [Session: Full Server Port Exposure Audit](references/server-port-exposure-audit.md)