# Go Scraper Proxy — Docker + Nginx Deployment

## Overview

Go scraper using Fiber + Colly. Embeds in Docker behind nginx reverse proxy.

## Required Files

```
~/scraper-proxy/
├── main.go           # Go Fiber server + Colly scrapers
├── go.mod            # Go module
├── go.sum            # Dependencies
├── Dockerfile        # Multi-stage build
└── compose.yml       # Docker Compose
```

## Dockerfile

```dockerfile
FROM golang:1.23-alpine AS builder
WORKDIR /build
RUN apk add --no-cache git ca-certificates
COPY go.mod go.sum ./
RUN go mod download
COPY main.go .
RUN CGO_ENABLED=0 GOOS=linux go build -ldflags="-s -w" -o scraper-proxy .
FROM gcr.io/distroless/static-debian12:nonroot
COPY --from=builder /build/scraper-proxy /app/scraper-proxy
EXPOSE 8099
ENTRYPOINT ["/app/scraper-proxy"]
```

Final image ~25MB, runtime <25MB RAM.

## Docker Compose

```yaml
services:
  scraper-proxy:
    image: unspooled/scraper-proxy:latest
    container_name: scraper-proxy
    restart: unless-stopped
    ports:
      - "127.0.0.1:8099:8099"
    environment:
      - PORT=8099
    mem_limit: 64m
    cpus: 0.5
```

## Nginx Config

```nginx
upstream scraper_proxy {
    server 127.0.0.1:8099;
    keepalive 64;
}

location ~ ^/api/scraper/(.*)$ {
    proxy_pass http://scraper_proxy/$1$is_args$args;
    proxy_http_version 1.1;
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_connect_timeout 10s;
    proxy_send_timeout 30s;
    proxy_read_timeout 30s;
}
```

Strips `/api/scraper/` prefix.

## API Design

```
GET /health                → {"status":"ok","service":"unspooled-scraper-proxy"}
GET /links/movie/{id}      → {"sources":[...],"cached":bool}
GET /links/series/{id}/{s}/{e} → {"sources":[...],"cached":bool}
```

## Colly Scraper Pattern

```go
c := colly.NewCollector(
    colly.AllowedDomains("allmovieland.you"),
    colly.UserAgent("Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36"),
    colly.Async(true),
)
c.SetRequestTimeout(10 * time.Second)
c.Limit(&colly.LimitRule{
    DomainGlob:  "*",
    Parallelism: 2,
    Delay:       500 * time.Millisecond,
})
c.OnResponse(func(r *colly.Response) {
    // 1. Try JSON unmarshal
    // 2. Regex fallback for file/url fields
    // 3. Plain URL extraction for mp4/m3u8/mkv
})
c.Visit(url)
c.Wait()
```

## Cache Strategy

- In-memory (Fiber storage/memory)
- 5-min TTL, GC interval 60s
- Key: `{scrapeID}:{mediaType}:{season}:{episode}`
- Cache hit ~70µs, first scrape ~2-3s

## Android Integration

```kotlin
private const val SCRAPER_PROXY_BASE = "http://192.168.1.50/api/scraper"

fun buildProxyEndpoint(id: String, isSeries: Boolean): String {
    val parts = id.split(":")
    val baseId = parts[0]
    val season = parts.getOrNull(1) ?: "1"
    val episode = parts.getOrNull(2) ?: "1"
    return if (isSeries) "$SCRAPER_PROXY_BASE/links/series/$baseId/$season/$episode"
    else "$SCRAPER_PROXY_BASE/links/movie/$baseId"
}
```

## Server Move

```bash
docker save unspooled/scraper-proxy:latest | gzip > scraper-proxy.tar.gz
# Load on target
docker load < scraper-proxy.tar.gz
docker run -d --name scraper-proxy --restart unless-stopped \
  -p 127.0.0.1:8099:8099 --memory 64m --cpus 0.5 \
  unspooled/scraper-proxy:latest
```

Update `SCRAPER_PROXY_BASE` in `DirectLinkProvider.kt`, rebuild APK.

## Cloudflare Restriction

Do NOT deploy on Workers free tier for scraper proxies (ToS violation). Use a VPS.