# APK-native torrent scraper migration

Use this when moving scraper capability into the Unspooled Android TV APK instead of depending on a Node/Express scraper service.

## Core lesson

Do **not** bundle Node/Express or make `/api/scrape` the primary app path. Keep scraper orchestration APK-native and reserve helper/backend services for Cloudflare/headless-browser extraction.

## Working architecture

```text
ProviderRegistry
  -> ScraperProvider
    -> EmbeddedTorrentScraperEngine
      -> EmbeddedTorrentProvider[]
        -> TPB / BitSearch / Nyaa / 1337x best-effort
      -> TorrentSearchResult
      -> ScraperLink
      -> DebridRouter
      -> DebridService
      -> Media3 playback
```

## Implementation pattern

1. Add an `app/src/main/java/.../data/scraper/embedded/` package.
2. Define a small APK-native contract:
   - `ScrapeContext(type, id, imdbId, tmdbId, title, year, season, episode, aliases)`
   - `TorrentSearchResult(infoHash, title, source, seeders, sizeBytes, magnet)`
   - `EmbeddedTorrentProvider.search(context)`
3. Resolve title/year from existing Android TMDB infrastructure before searching.
4. Convert provider output to existing `ScraperLink`; do not create a parallel playback model.
5. Feed `ScraperLink` into existing `DebridRouter`; keep Real-Debrid on the Android `addMagnet + poll + selectFiles + unrestrict` flow.
6. Keep resolution bounded (e.g. top 12 ranked hashes). `DebridRouter` can add/poll uncached magnets, so resolving dozens at once can make TV playback feel hung.
7. Use `jsoup` for HTML providers. Avoid regex-only HTML parsing in APK code.

## Provider split

| Source | APK-native role | Notes |
|---|---|---|
| TPB / ApiBay | Primary embedded torrent source | Low friction JSON; good first provider. |
| BitSearch | Embedded with jsoup | Parse search/detail pages and magnets. |
| Nyaa | Embedded with jsoup | Useful anime source; table/magnet parsing is straightforward. |
| 1337x | Best-effort embedded + helper fallback | Direct HTML often blocks; fallback may call helper `/api/scrape`. |
| FlareSolverr / Cloudflare hosts | Helper/backend only | Not sane to embed in Android TV APK. |
| FileMoon / StreamTape / VOE / RabbitStream | Helper/backend or WebView-only experiment | JS/challenge-heavy; APK should play final direct/HLS URL only. |

## Wiring pitfall

`DefaultAddonScrapers.ALL` may be empty. If `ScraperProvider.isAvailable()` only checks that list, the native scraper is silently disabled. `ScraperProvider` must return available when `EmbeddedTorrentScraperEngine.isAvailable()` is true and merge native streams with any configured addon scraper output.

## Verification

Run at minimum:

```bash
cd ~/Unspooled
./gradlew :app:compileDebugKotlin :app:testDebugUnitTest
./gradlew :app:assembleDebug
```

If a device is attached, install/run the debug APK and test a known title such as `tt0133093`. Without an attached device, report that hardware playback was not verified.

## Bad approaches

- Do not bundle a Node runtime or Express server in the APK.
- Do not make LAN `/api/scrape` mandatory for ordinary torrent results.
- Do not reintroduce Real-Debrid `instantAvailability` as the playback-critical path.
- Do not auto-resolve every uncached result; cap candidates or resolve on selection.
- Do not split torrent/embed sources into separate UI sections when the product goal is one unified dropdown.