# Android App Scraping Architecture

How Cinema HD, BeeTV, TeaTV, and similar apps scrape free content and integrate Real-Debrid.

## Source: Decompiled APK Analysis

This document captures patterns extracted from reverse-engineering DebridStream, NetFly, Strmr, and other Android TV streaming APKs found under `~/apk-analysis/` and `~/debridstream-analysis/`.

## Architecture Overview

```
┌─────────────────────────────────────────────────────┐
│                  Android App (Kotlin/Java)           │
│                                                      │
│  ┌────────────────┐    ┌──────────────────────────┐ │
│  │ Torrent Site    │    │ Real-Debrid API Client   │ │
│  │ Scrapers        │    │                          │ │
│  │  • 1337x        │    │  /torrents/instantAvail  │ │
│  │  • YTS          │───▶│  /torrents/addMagnet     │ │
│  │  • TPB          │    │  /torrents/selectFiles   │ │
│  │  • EZTV         │    │  /torrents/info          │ │
│  │  • TorrentGalaxy│    │  /unrestrict/link        │ │
│  │  • Kickass      │    └──────────┬───────────────┘ │
│  │  • LimeTorrents │               │                  │
│  └────────────────┘               │                  │
│                                    ▼                  │
│  ┌───────────────────────────────────────────────┐   │
│  │  ExoPlayer / Media3                           │   │
│  │  Plays direct HTTP URL from RD CDN            │   │
│  └───────────────────────────────────────────────┘   │
└─────────────────────────────────────────────────────┘
```

## Real-Debrid API Integration Sequence

### Legacy (deprecated June 2024)
```
POST /torrents/instantAvailability
Body: { "hashes": { "infoHash": [...] } }
Response: { "infoHash": { "rd": [{ "video": 1 }] } }
```
This returned immediately if cached. Deprecated with error 37.

### Current Flow (addMagnet + poll)
```
1. POST /torrents/addMagnet
   Body: magnet=...
   Response: { "id": "ABC123", "hash": "...", "filename": "..." }

2. POST /torrents/selectFiles/{id}
   Body: files=all
   Response: 200 OK

3. GET /torrents/info/{id}  (poll every 1.5s, up to 15 times)
   Response: { "status": "downloaded", "links": ["https://..."], "bytes": 12345 }

4. POST /unrestrict/link
   Body: link=https://...
   Response: { "download": "https://...", "filename": "movie.mp4", "size": 12345 }
```

### Rate Limiting
- 60 requests/minute per API key
- Use separate rate limiters per provider (RD, AD, PM, TB)

## Built-in Scraper Patterns

### Torrent Site Scrapers
Each scraper is a class/function that:
1. Searches the site for a title + year
2. Parses HTML for magnet links or info hashes
3. Extracts: infoHash, filename, seeders, leechers, size, quality
4. Returns a list of `ScraperLink` objects

```kotlin
data class ScraperLink(
    val infoHash: String,
    val name: String,
    val seeders: Int,
    val leechers: Int,
    val size: Long,
    val quality: String,  // "4K", "1080p", "720p", "SD"
    val source: String    // "1337x", "YTS", etc.
)
```

### Direct HTTP Extractors (Non-Debrid)
For free streaming without debrid, apps use WebView-based extraction:
1. Load embed URL (StreamTape, DoodStream, FileMoon, MixDrop) in hidden WebView
2. Set `WebViewClient.shouldInterceptRequest()` to monitor all network requests
3. Capture `.m3u8` and `.mp4` URLs from intercepted requests
4. Stop WebView and play captured URL in native player

```kotlin
// Pseudocode for WebView intercept
webView.webViewClient = object : WebViewClient() {
    override fun shouldInterceptRequest(
        view: WebView, request: WebResourceRequest
    ): WebResourceResponse? {
        val url = request.url.toString()
        if (url.contains(".m3u8") || url.contains("/master")) {
            capturedUrl = url
        }
        return null // Let WebView load normally
    }
}
```

### Key Difference from Server-Side
| Approach | Cloudflare | Rate Limited | Requires IP Reputation |
|----------|-----------|-------------|----------------------|
| **WebView on device** | ✅ Passes | ❌ No | ❌ No |
| **Playwright server** | 🟡 Sometimes | ✅ Yes | ✅ Yes |
| **HTTP requests** | ❌ Blocked | ✅ Yes | ✅ Yes |

The WebView approach works because:
- Runs on the user's actual home IP
- Uses a real browser engine (System WebView / Chrome)
- Can execute JavaScript naturally
- Cloudflare sees a legitimate user, not a datacenter IP
- No special stealth plugins needed — the real browser IS the stealth

## Files Found in Decompiled APKs

### DebridStream (`~/debridstream-analysis/`)
- `ScraperResultsViewModel.java` — Main scraper coordination, OkHttp client with connection pooling
- `CloudManagerViewModel.java` — RD cloud management
- `PlayerViewModel.java` — ExoPlayer integration with RD links
- All obfuscated (class names like `C0671h3`, `C0656e3`)

### NetFly (`~/apk-analysis/netfly-jadx/`)
- Flutter-based app with custom CMS backend
- API endpoints: `api_collection/list`, `api_video/random`, `api_video/tags`
- Config-driven: JSON files define Trakt lists for catalog rows
- No built-in scrapers — relies on backend API

### Strmr/Mr. TV (`~/apk-analysis/strmr-jadx/`)
- Trakt-based catalog (trending, popular, user lists)
- JSON configs define home screen rows
- Similar architecture to NetFly — backend API, not local scrapers

## Best Practices from These Apps

1. **Separate scraper from resolver** — scrapers find hashes, resolvers check availability
2. **Prioritize cached sources** — check RD instant availability first, only addMagnet if not cached
3. **Parallel scraping** — fire all torrent site scrapers simultaneously, aggregate results
4. **Rate limiting per provider** — separate rate limiters for RD, AD, PM, TB
5. **Connection pooling** — reuse HTTP connections across scrapers (OkHttp connection pool)
6. **WebView fallback** — when HTTP scraping fails, fall back to WebView on user's device
7. **Client-side extraction** — embed URLs resolve better on the client than the server