---
name: local-ai-web-apps
description: "Deploy self-hosted AI web applications (GPU-backed + Next.js/FastAPI) accessible from LAN and Tailscale devices (phone, laptop)"
version: 1.0.0
author: Hermes Agent
license: MIT
platforms: [linux]
metadata:
  hermes:
    tags: [deployment, web-apps, networking, gpu, nextjs]
    related_skills: [system-administration, tailscale-funnel-setup, open-webui]
---

# Local AI Web Apps

## Overview

Deploying self-hosted AI web applications (image generation, chatbots, inference UIs) that are accessible from mobile devices over LAN or Tailscale. Covers the recurring pattern of: start with a stock FastAPI + Next.js app → reconfigure for network access → handle port conflicts → verify from external IP.

## When to Use

- User asks to set up an AI web app (image gen, inference UI, etc.) that comes with a Next.js frontend and Python/FastAPI backend
- User wants to access the app from their phone or another device on the same LAN or Tailscale network
- User hits "can't connect" errors from external IPs
- User reports the web UI loads but buttons/controls don't work from phone

## Security Model

This server runs on a **trusted LAN** (192.168.1.0/24) with **Tailscale** (100.x.x.x). Services bind to `0.0.0.0` by default, which is appropriate here because:
- Access is restricted by network perimeter (LAN firewall, Tailscale ACLs)
- No port is exposed to the public internet (except `:3091` via Tailscale Funnel)
- All existing services (Open WebUI :3081, Open Terminal :8000, SearXNG :8080) follow the same pattern

**Do not use `0.0.0.0` on a cloud VM or any machine exposed to the public internet** without a reverse proxy with auth.

**Proactive explain when binding to 0.0.0.0:** The user may ask "isn't 0.0.0.0 bad for security?" — explain upfront that it's the same risk profile as every other service on the server, and the trust boundary is the LAN/Tailscale network perimeter. Offer to harden (specific IP bind, reverse proxy auth, or nginx) but don't alarm. The user is security-conscious but pragmatic.

## Common Pitfalls (Check These First)

### 1. Port Conflicts

Always check the target ports before starting. Many services compete for common ports:

```bash
ss -tlnp | grep -E ':8000|:3000|:8080|:5173'
```

Common port assignments on this server:
- `:8000` — Open Web Terminal (`open-terminal`)
- `:8080` — Open WebUI
- `:3000` — Next.js frontends (one at a time)
- `:9292` — llama-swap
- `:3091` — nexstream-scraper

**Resolution:** Use a different port via env var (`BACKEND_PORT=8001`, `FRONTEND_PORT=3001`).

### 2. Backend Binding to localhost

Most FastAPI/uvicorn apps default to `127.0.0.1` (localhost only). From another device, this is unreachable.

**Fix:** Start uvicorn with `--host 0.0.0.0`:
```bash
uvicorn app:app --host 0.0.0.0 --port 8001
```

### 3. Next.js Dev Mode Blocks Cross-Origin

Next.js dev mode (next dev / npm run dev) blocks HMR WebSocket connections from non-localhost IPs by default. From a mobile browser over LAN/Tailscale, the page loads but buttons, form submission, and other interactive elements won't work because the HMR client fails.

**Fix — production build (recommended):**
```bash
NEXT_PUBLIC_BACKEND_URL="http://192.168.1.50:8001" npm run build
PORT=3000 npm start
```

**Fix — dev mode with allowed origins (for iterative development):**
Add `allowedDevOrigins` to `next.config.ts`:
```typescript
const nextConfig: NextConfig = {
  allowedDevOrigins: ["127.0.0.1", "192.168.1.50", "100.126.244.3", "localhost"],
};
```

### 5. VRAM Exhaustion at Higher Resolutions (GPU-backed Apps)

When the app uses GPU (image generation, video gen, etc.), higher resolutions can OOM. On an RTX 2080 Ti (11 GB) with Bonsai Image 4B:

- **512×512** — works fine (~5.9 GiB peak)
- **1024×1024** — OOM at VAE decode phase (needs ~4.5 GiB temp allocation on top of ~6 GiB baseline)

The diffusion/transformer step fits at both resolutions — the OOM occurs specifically during **VAE decode** which allocates a large contiguous tensor.

**Diagnosis:** Look for `torch.OutOfMemoryError: CUDA out of memory` in backend logs. Check `nvidia-smi` — if VRAM is >10.5 GiB used and only one process is listed, it's resolution-bound, not a multi-process conflict.

**Fixes:**
- Reduce resolution from 1024→512 (biggest impact)
- Reduce step count (2-3 steps instead of 4)
- Set `PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True` to reduce fragmentation (might buy ~200-300 MB)
- Check if other GPU processes are running (`nvidia-smi`)

### 6. API URL Baked for Localhost

Next.js frontends use `NEXT_PUBLIC_BACKEND_URL` to make API calls to the backend. When this is set to `http://127.0.0.1:8001`, the phone's browser tries to call `127.0.0.1:8001` on the phone itself — which doesn't exist.

**Fix:** Set `NEXT_PUBLIC_BACKEND_URL` to the server's LAN IP:
```
NEXT_PUBLIC_BACKEND_URL="http://192.168.1.50:8001"
```
Also use Tailscale IP (`http://100.126.244.3:8001`) if the client connects via Tailscale.

This must be set at build time (it's compiled into the JS bundle) or at dev-server start time (read from env).

## Procedure

### Step 1: Scan for Port Conflicts

```bash
ss -tlnp | grep -E ':8000|:3000'
```

If taken, note the process name and choose alternative ports.

### Step 2: Start Backend on 0.0.0.0

Start the backend with the correct env vars for the GPU pipeline:

```bash
cd /path/to/app
.venv/bin/uvicorn scripts.local_backend:app \
  --host 0.0.0.0 --port 8001
```

For GPU-backed apps (like image generation), set required env vars:
- `MFLUX_STUDIO_GPU_DEFAULT_BACKEND`
- `MFLUX_STUDIO_GPU_TERNARY_TRANSFORMER_PATH` (or equivalent)
- `MFLUX_STUDIO_GPU_TEXT_ENCODER_PATH`
- `MFLUX_STUDIO_GPU_VAE_PATH`
- `MFLUX_STUDIO_GPU_TOKENIZER_PATH`
- `BONSAI_SUPPORTED_FAMILIES`

### Step 3: Build Frontend for Production

Build the Next.js frontend with the correct backend URL:

```bash
cd /path/to/app/vendor/image-studio/frontend
PATH=../../.venv/bin:$PATH \
  NEXT_PUBLIC_BACKEND_URL="http://192.168.1.50:8001" \
  npm run build
```

### Step 4: Start Frontend in Production Mode

```bash
PATH=../../.venv/bin:$PATH \
  PORT=3000 \
  NEXT_PUBLIC_BACKEND_URL="http://192.168.1.50:8001" \
  npm start
```

### Step 5: Verify from External IP

```bash
# From the server itself
curl -s http://192.168.1.50:8001/backends
curl -s http://192.168.1.50:3000/ | head -5

# Also verify the frontend's API proxy
curl -s http://192.168.1.50:3000/api/backends
```

### Step 6: Verify from Phone

Open on phone:
- `http://192.168.1.50:3000` — LAN
- `http://100.126.244.3:3000` — Tailscale

**Do not skip this step.** The user accesses services primarily from their phone. A curl test from the server is not sufficient — Next.js dev mode blocks cross-origin HMR, and baked-in localhost API URLs break on external devices.

**Checklist for phone access:**
- Backend bound to `0.0.0.0` (not `127.0.0.1`)
- Frontend in production mode (`npm start`, not `npm run dev`) OR `allowedDevOrigins` configured
- `NEXT_PUBLIC_BACKEND_URL` set to the server's LAN/Tailscale IP (not `127.0.0.1` or `localhost`)
- If buttons are non-functional, re-check the backend URL in the frontend build

### Step 7: Verify GPU Consumption

If the app uses GPU, check that VRAM is available and the process consumes what you expect:

```bash
nvidia-smi
```

List all GPU processes. If another service (llama-swap, another AI app) is consuming VRAM, either wait for TTL timeout or stop it before starting the new app. Don't assume the GPU is free — check first.

## Bonsai Image Generation (PrismML Bonsai Image 4B)

### Quick Start

```bash
# Clone and install
git clone https://github.com/PrismML-Eng/Bonsai-Image-Demo.git ~/Bonsai-Image-Demo
cd ~/Bonsai-Image-Demo
./setup.sh                                           # installs deps, downloads ternary-gemlite model

# One-shot generation (cold-start each run)
bash scripts/generate.sh -p "your prompt" --size 1024x1024

# Or start the web studio (recommended for repeated use)
BACKEND_PORT=8001 bash scripts/serve.sh               # port 8000 is often taken
# Open http://localhost:3000
```

### Quick Reference Table

| Item | Value |
|------|-------|
| Repo | `~/Bonsai-Image-Demo` |
| Working override | `BACKEND_PORT=8001 FRONTEND_PORT=3000 bash scripts/serve.sh` |
| Default variant | `ternary` → `bonsai-ternary-gemlite` (1.21 GB) |
| Binary variant | `bonsai-binary-gemlite` (0.93 GB, for ultra-low VRAM) |
| Idle VRAM | ~10.7 GB (on 11 GB RTX 2080 Ti) |
| Model size | ~1.21 GB transformer, ~6.8 GiB VRAM at 1024×1024 |

### Variant Switching

```bash
BONSAI_VARIANT=binary ./setup.sh
BONSAI_VARIANT=binary bash scripts/serve.sh
```

### Endpoints

- `http://localhost:3000` — Web UI
- `http://localhost:<backend-port>/docs` — FastAPI OpenAPI docs
- `http://localhost:<backend-port>/backends` — Health check / model status

### Teardown

```bash
pkill -f "Bonsai-Image-Demo|scripts/serve.sh|scripts.local_backend|next dev|next start" 2>/dev/null || true
fuser -k 3000/tcp 2>/dev/null || true
fuser -k <backend-port>/tcp 2>/dev/null || true
nvidia-smi  # Verify GPU is freed (~1 MiB)
```

## Reference Files

- `references/bonsai-image-setup.md` — Session-specific setup details for Bonsai Image 4B ternary model
- `references/bonsai-image-generation.md` — Full Bonsai model setup details (gemlite, HQQ, diffusers stack)
- `references/bonsai-port-conflict-log.md` — Exact commands, error signatures from a real host with port 8000 occupied

## Pitfalls

- **Don't start the backend without the required GPU env vars** — The Bonsai GPU backend crashes at startup if `MFLUX_STUDIO_GPU_TERNARY_TRANSFORMER_PATH` (or equivalent) is unset. The serve.sh script normally sets these; when starting manually, you must set them all.
- **Don't use `npm run dev` for remote access without allowedDevOrigins** — The HMR WebSocket block makes the UI non-interactive.
- **Don't skip the build step** — In dev mode, `NEXT_PUBLIC_BACKEND_URL` is read from env at server start. In production mode, it's compiled into the JS and won't update if you change the env after the build. Rebuild if the IP changes.
- **Frontend and backend logs are in different places** — Backend logs go to stderr (visible in the process output). Check `.serve-logs/backend.log` and `.serve-logs/frontend.log` if using the serve.sh script.
- **Production build may fail on first run** — Missing node_modules, stale .next cache, or TypeScript errors can block the build. Run `npm install` first, then `rm -rf .next` and retry.
- **Multiple frontend processes on same port** — `next dev` has a cleanup trap, but `npm start` doesn't. If you get a port conflict, kill all node processes: `pkill -f "next start\\|next dev"` then retry.
- **VRAM exhaustion at higher resolutions** — Even if the model fits at startup, generating at high resolutions (1024×1024) can OOM during VAE decode. Test at 512×512 first, then scale up.
- **Don't skip `nvidia-smi` check before starting a GPU-backed app** — If llama-swap loaded a model, the GPU has zero free memory. Either wait for the model to unload (TTL timeout) or stop llama-swap if needed for testing.