# Ultimate Scraper — APK Transition Notes

Session lesson: Ray clarified that Ultimate Scraper is intended for use in a streaming app APK, not as a permanently server-side scraper. Treat Express routes and Node services as compatibility/test harnesses unless the user explicitly asks for a backend.

## Durable guidance

When working on `~/ultimate-scraper/` or adjacent scraper code for the Android streaming app:

1. **Ask/assume target shape correctly**
   - Default to APK-embeddable scraper core when the user mentions the streaming app APK.
   - Do not keep adding server-only Express endpoints as the final architecture unless they are needed as a local test harness or LAN helper.

2. **Separate core pipeline from transport**
   - Core: providers, metadata matching, release parsing, scoring, normalization.
   - Transport: Express/Stremio endpoints, `/api/resolve`, health routes.
   - For APK use, design a provider interface that can be ported to Kotlin/Android or called through an embedded JS engine/WebView.

3. **Audit server dependencies before claiming APK readiness**
   - `express`, `axios`, `cheerio`, `better-sqlite3`, `dotenv`, Node `fs/path/url` APIs, and local ports do not directly embed in a native APK.
   - SQLite cache can map to Android Room/SQLite.
   - HTTP client maps to OkHttp/Ktor.
   - Cheerio parsing maps to Jsoup or embedded JS parser.
   - Express routes become app service methods, repository calls, or optional LAN helper endpoints.

4. **Live server checks are still useful**
   - Keep using Node service tests to verify provider behavior quickly.
   - Label them as backend harness checks, not proof of APK integration.

## Known July 2026 fix set from re-audit

A good server-harness baseline had these fixes applied:

- `.env` exists and TMDB is configured for IMDB/TMDB title lookup.
- Runtime version uses config/package version (`4.1.0` in that session), not stale hardcoded `2.0.0`.
- `/api/resolve?url=` proxies to local free-scraper (`FREE_SCRAPER_URL=http://127.0.0.1:3092`) for compatibility/testing.
- RD cache pre-check avoids brittle false-negative behavior from the instant cache endpoint; resolution still uses addMagnet/selectFiles/info/unrestrict flow.
- BitSearch parser uses Cheerio/detail pages/magnet extraction instead of one raw HTML regex.
- 1337x fallback uses direct search/detail-page scraping and optional `FLARESOLVERR_URL`, not only the Render proxy.

## Output framing for Ray

If the user asks to “fix it” and then clarifies APK target, report both:

- What was fixed in the current server harness.
- What still needs to be changed for APK embedding.

Do not present server route availability as the final destination if the app is APK-first.