# Self-Hosted Stremio Scraper Pattern

## Architecture

A Node.js Express server implementing the Stremio addon protocol that aggregates streams from multiple upstream scrapers and checks debrid cache availability.

### Server Layout

```
services/nexstream-scraper/
  src/index.js          — Express on port 3091, all routes
  src/addon.js          — Stremio manifest builder
  src/routes/stream.js  — orchestrator: parallel scrape + debrid check
  src/scrapers/         — one file per upstream scraper
  src/debrid/           — one file per debrid provider
  src/cache.js          — L1 memory + L2 SQLite
  src/config.js         — .env loader with defaults
  src/health.js         — /health + /manifest.json endpoints
```

### Key Patterns

**Parallel scraping with Promise.allSettled:**
```javascript
const results = await Promise.allSettled(
  scrapers.map(s => s.search(imdbId, type, season, episode))
);
const allStreams = results
  .filter(r => r.status === 'fulfilled')
  .flatMap(r => r.value);
```

**Debrid-first cache check:**
```javascript
// After scraping: batch-check all unique hashes against all debrid providers
const cached = await Promise.all([
  rd.checkCache(hashes),
  ad.checkCache(hashes),
  pm.checkCache(hashes),
  tb.checkCache(hashes),
]);
// Return cached URLs first, then uncached magnets
```

**Two-tier cache:**
- L1: node-cache (in-memory, 30min TTL, 10K entries, <5ms response)
- L2: better-sqlite3 (persistent, 24h TTL, WAL mode, auto-vacuum)
- Cache key: `stream:{type}:{imdbId}`

**Rate limiting per debrid provider:**
```javascript
class RateLimiter {
  constructor(perMinute) { /* token bucket */ }
  async acquire() { /* wait if needed */ }
}
```

### Supported Providers

**Scrapers (5):** Torrentio, Comet, MediaFusion, Jackett (Torznab), Zilean (DMM)
**Debrid (4):** Real-Debrid, AllDebrid, Premiumize, TorBox

### Integration with Android TV app

The scraper is a Stremio-compatible addon. Add it as a system seed in `SystemAddonCatalog`:
```kotlin
SeedDef(
    id = "nexstream_scraper",
    manifestUrl = "${BuildConfig.NEXSTREAM_SCRAPER_URL}/manifest.json",
    category = UnifiedAddonCategory.OPTIONAL_DEBRID_ADDONS,
    resources = listOf("stream"),
    enabledByDefault = true,
    priority = 15,
)
```

The existing Stremio addon client in the app consumes it automatically — no new client code needed.