# Bonsai Image 4B — Setup Reference

## Model

- **Name:** PrismML Bonsai Image 4B (ternary gemlite-2bit variant)
- **Transformer:** 1.58-bit (ternary), ~1.21 GB
- **Text encoder:** HQQ 4-bit
- **VRAM peak:** ~5.9 GiB at 512×512, ~6.8 GiB at 1024×1024
- **GPU:** RTX 2080 Ti (11 GB) — fits comfortably

## Repo

- https://github.com/PrismML-Eng/Bonsai-Image-Demo
- Cloned to: `/home/rurouni/Bonsai-Image-Demo/`

## Directory Structure

```
Bonsai-Image-Demo/
├── models/
│   └── bonsai-image-4B-ternary-gemlite/
│       ├── transformer-gemlite-int2/    # Ternary weights (1.58-bit)
│       ├── text_encoder-hqq-4bit/       # Text encoder + tokenizer
│       └── vae/                         # VAE decoder
├── vendor/image-studio/frontend/        # Next.js frontend
├── scripts/
│   ├── local_backend.py                 # FastAPI backend wrapper (394 lines)
│   ├── serve.sh                         # Orchestrator script (420 lines)
│   ├── generate.sh                      # One-shot generation
│   ├── generate.py                      # Python generation engine (464 lines)
│   └── download_model.sh                # Model downloader
├── .serve-logs/                         # Runtime logs
├── outputs/                             # Generated images + .gemlite_cache
└── .venv/                               # Python virtual env (gemlite, hqq, etc.)
```

## Three Backend Ports on This Server

| Port | Service | Purpose |
|---|---|---|
| 8000 | Open Terminal | Web-based terminal — DO NOT USE |
| 8001 | Bonsai Image Backend | FastAPI GPU inference |
| 3000 | Bonsai Image Frontend | Next.js web UI |

## Environment Variables (GPU Backend)

Required when starting the backend manually:

```bash
MFLUX_STUDIO_GPU_DEFAULT_BACKEND="bonsai-ternary-gemlite"
BONSAI_SUPPORTED_FAMILIES="bonsai-ternary"
MFLUX_STUDIO_GPU_TERNARY_TRANSFORMER_PATH="/home/rurouni/Bonsai-Image-Demo/models/bonsai-image-4B-ternary-gemlite/transformer-gemlite-int2"
MFLUX_STUDIO_GPU_TEXT_ENCODER_PATH="/home/rurouni/Bonsai-Image-Demo/models/bonsai-image-4B-ternary-gemlite/text_encoder-hqq-4bit"
MFLUX_STUDIO_GPU_VAE_PATH="/home/rurouni/Bonsai-Image-Demo/models/bonsai-image-4B-ternary-gemlite/vae"
MFLUX_STUDIO_GPU_TOKENIZER_PATH="/home/rurouni/Bonsai-Image-Demo/models/bonsai-image-4B-ternary-gemlite/text_encoder-hqq-4bit/tokenizer"
```

The `MFLUX_STUDIO_GPU_BINARY_TRANSFORMER_PATH` env var must be set but can be empty string if binary variant not downloaded.

## Start Sequence

### Backend (manual, background)

```bash
cd ~/Bonsai-Image-Demo
.venv/bin/uvicorn scripts.local_backend:app \
  --host 0.0.0.0 --port 8001
```
Takes ~8s to load and warm up.

### Frontend (production build, background)

```bash
cd ~/Bonsai-Image-Demo/vendor/image-studio/frontend

# Build (one-time, or after IP changes)
PATH=~/.venv/bin:$PATH \
  NEXT_PUBLIC_BACKEND_URL="http://192.168.1.50:8001" \
  npm run build

# Start production server
PATH=~/.venv/bin:$PATH \
  PORT=3000 \
  NEXT_PUBLIC_BACKEND_URL="http://192.168.1.50:8001" \
  npm start
```

### Frontend (dev mode, for iteration)

```bash
cd ~/Bonsai-Image-Demo/vendor/image-studio/frontend
PATH=~/.venv/bin:$PATH \
  PORT=3000 \
  NEXT_PUBLIC_BACKEND_URL="http://192.168.1.50:8001" \
  npm run dev
```

One-time: add to `next.config.ts`:
```typescript
allowedDevOrigins: ["127.0.0.1", "192.168.1.50", "100.126.244.3", "localhost"]
```

### Via serve.sh (original script, auto-mode)

```bash
cd ~/Bonsai-Image-Demo
BACKEND_PORT=8001 bash scripts/serve.sh
```
Note: serve.sh sets all env vars automatically BUT insists on `BACKEND_PORT` being free. Kill any existing process on that port first.

## Health Check

```bash
curl -s http://localhost:8001/backends
# {"kind":"gemlite","supported_families":["bonsai-ternary"],"default_family":"bonsai-ternary","healthy":true,"reason":null}
```

```bash
curl -s http://localhost:3000/ | head -3
# Returns HTML of the Bonsai studio page
```

```bash
curl -s http://localhost:3000/api/backends
# Same as /backends but proxied through Next.js
```

## Known Issues

- **Cold start is slow** — First request after idle triggers a prewarm that takes ~8s (transformer load + text encoder + VAE). Subsequent requests are fast.
- **No swap between variants** — Only ternary-gemlite is downloaded. The frontend dropdown shows Binary as an option but it will 500 if selected.
- **No warmup at start** — The `BONSAI_WARMUP_SHAPES` env is empty, so the first actual generation triggers JIT compilation in triton. Takes ~2-3s longer than subsequent calls.
- **Logs to stderr** — Backend logs go to stderr, not a file, when started manually. Use `2>&1 | tee -a backend.log` to capture.
- **1024×1024 OOM on 2080 Ti** — The model loads ~6 GiB baseline (transformer + text encoder + VAE). PyTorch reserves ~5 GiB more. VAE decode for 1024×1024 needs a 4.5 GiB spike → OOM at 10.7/10.74 GiB. Stick to 512×512 or smaller. Setting `PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True` may help fragmentation but doesn't resolve the 4.5 GiB gap.