# Magnetio Audit — July 2026

Audit of [peterdsp/Magnetio](https://github.com/peterdsp/Magnetio) (v1.1.5, 4 months old, 19 stars, 22 providers).
Cloned to `/home/rurouni/Magnetio/`.

## Architecture

- **Separate services**: Scraper (port 8080) + Addon (port 7000) + Redis (6379)
- **Scraper**: Express.js, cheerio HTML parsing, axios HTTP, bottleneck rate limiting, keyv+redis caching
- **Addon**: Stremio addon protocol, 8 debrid providers, configuration UI, subtitle sync pipeline, Prometheus metrics

## Provider Domains — Verified July 4, 2026 (Batch Tested from Debian 13 Server)

All domains tested via `httpClient.js` (Bottleneck-limited, 5s timeout, browser-like User-Agent).

### Confirmed Working (respond with content)

| Provider | Working Domain(s) | Response | Issue |
|----------|-------------------|----------|-------|
| **TPB** | `apibay.org` | ✅ ~28KB JSON | Perfect — 100 results/query |
| **TheRarBG** | `therarbg.com` | ✅ Slow | 2-pass: search list → detail pages. 5-8 results/movie |
| **BitSearch** | `bitsearch.to` | ✅ 69KB | Legacy provider works. ~5 results/movie |
| **Nyaa** | `nyaa.si` | ✅ 115KB | Legacy provider works. Results are anime (mostly music for "Inception") |
| **TorrentGalaxy** | `torrentgalaxy.one` | ✅ 1260KB | Site loads but **search parameter is ignored** — returns 190 latest torrents, not query-specific |
| **EZTV** | `eztv.re`, `eztvx.to` | ✅ 619KB each | Sites load. **Search URL gets 403** — can't search for specific content |
| **KickassTorrents** | `kick4ss.com`, `kickasstorrents.bz`, `kickasst.net`, `thekat.cc`, `kickass2.fun`, `kickass.one` | ✅ 20-28KB each | Sites load. **Cheerio selectors don't match** — Magnetio scrapers use `tr.odd, tr.even` but these domains use different HTML. Need selector updates |
| **GloTorrents** | `glodls.to`, `gtso.cc` | ✅ 5-10KB | Sites respond but Magnetio scrapers find no results |
| **TorLock** | `torlock2.com` | ✅ 624KB (slow, ~4s) | Site loads, slow response. Magnetio scrapers may timeout before it responds |
| **LimeTorrents** | `www.limetorrents.fun`, `www.limetorrents.lol`, `limetorrents.info` | ✅ 8KB each | Legacy provider works |
| **1337x** | `1337x.bz` | ✅ 13KB | Site loads, search endpoint may not work. 1337x.to/.st/.eu/.ws all 403 |
| **BTDig** | `btdig.com` | ✅ 13KB | Site responds |
| **YTS proxy** | `yts.bypassed.wtf`, `yts.homes` | ✅ 5-90KB | Proxy sites load but **API returns 0 movies** — YTS API is effectively dead |

### Dead / Blocked (July 2026)

| Provider | Domains Tried | Status |
|----------|--------------|--------|
| YTS API | yts.mx (ENOTFOUND), yts.do (0 results), yts.unblockit.date (proxy page, no API) | ❌ ALL DEAD |
| 1337x originals | 1337x.to (.st .gd .eu .ws) | ❌ ALL 403 |
| EZTV search | eztv.re/ag/x.to/ch search URLs | ❌ ALL 403 |
| TorrentDownloads | torrentdownload.info (timeout), torrentdownloads.pro (timeout) | ❌ ALL DEAD |
| BT4G | bt4gprx.com | ❌ 403 |
| SubsPlease | subsplease.org | ⚠️ API returns nothing for non-anime queries |
| AnimeTosho | feed.animetosho.org | ⚠️ RSS feed returns empty for most queries |
| Torznab | — | ❌ Requires self-hosted Jackett |

### Critical Pattern

**Only 2 providers consistently produce real results: TPB + TheRarBG.** All others either:
- Return HTML but the cheerio selectors don't match (KAT, TG, BTDig, Glo)
- Return HTML but the search parameter is ignored (TG returns latest, not query-specific)
- Have functional sites but get 403 on search URLs (EZTV, 1337x)
- Are too slow and timeout before responding (TorLock at 4s+)
- Are dead/blocked (YTS API, TorrentDownloads, BT4G)

## Infrastructure Borrowed (MIT Licensed)

### httpClient.js
`~/ultimate-scraper/src/providers/lib/httpClient.js`
- Bottleneck per-provider rate limiters (minTime: 500ms, maxConcurrent: 2)
- Retries on 5xx with exponential backoff (1s, 2s)
- Default browser-like headers (User-Agent, Accept-Language)
- Does NOT retry 4xx client errors (403, 404, 429)

### domainRotation.js
`~/ultimate-scraper/src/providers/lib/domainRotation.js`
- `tryDomains(domains, requestFn, providerName)` — tries each domain in order
- Failed domains marked with cooldown: 5min for errors, 10min for 429
- If ALL domains are failed, tries all as last resort
- `classifyError(status)`: 429=ratelimited, 403=blocked, 5xx=server, null=non-retryable
- `PROVIDER_DOMAINS` map — **UPDATED July 2026** with verified working domains (KAT → kick4ss.com, TG → torrentgalaxy.one, 1337x → 1337x.bz, etc.)

### titleHelper.js
`~/ultimate-scraper/src/providers/lib/titleHelper.js`
- `parseTitle(title)`: extracts quality, codec, source, languages, HDR, bitdepth
- `buildSearchQuery(meta)`: "Movie Name 2024" or "Show Name S01E02"
- `extractInfoHash(magnet)`: regex-based BTIH extraction
- `parseSize(str)`: human-readable → bytes (handles GB, MB, KB, TB, ГБ, МБ)

### contentFilter.js
`~/ultimate-scraper/src/providers/lib/contentFilter.js`
- `filterByContent(results, meta)`: drops results whose normalized title doesn't contain ≥70% of search words
- `deduplicateByHash(results)`: keeps highest-seeders per infoHash

## 11 Providers Copied to Ultimate Scraper

All adapted from Magnetio source to `~/ultimate-scraper/src/providers/torrent/`:

| File | Magnetio Source | Status |
|------|----------------|--------|
| `yts.js` | `scraper/providers/yts.js` | ❌ YTS API dead — kept for when it returns |
| `torrentgalaxy.js` | `scraper/providers/torrentgalaxy.js` | ⚠️ Site loads, search ignored |
| `kickasstorrents.js` | `scraper/providers/kickasstorrents.js` | ⚠️ Sites load, selectors wrong |
| `bt4g.js` | `scraper/providers/bt4g.js` | ❌ 403 |
| `btdig.js` | `scraper/providers/btdig.js` | ⚠️ Site responds |
| `glotorrents.js` | `scraper/providers/glotorrents.js` | ⚠️ Site responds |
| `torlock.js` | `scraper/providers/torlock.js` | ⚠️ Very slow (~4s) |
| `torrentdownloads.js` | `scraper/providers/torrentdownloads.js` | ❌ Both domains dead |
| `therarbg.js` | `scraper/providers/therarbg.js` | ✅ Working in pipeline (5-8 results) |
| `subsplease.js` | `scraper/providers/subsplease.js` | ❌ Anime only, no hits for movie queries |
| `animetosho.js` | `scraper/providers/animetosho.js` | ❌ Anime only |

**3 bugs in copied code fixed**:
1. `animetosho.js` regex had `mapping:` instead of `magnet:` (transcription error)
2. All providers use `export const id/name/scrape` (named exports) — imported via `import * as` not default
3. `domainRotation.js` dead UNBLOCKIT references removed, working domains substituted

## Key Lessons

1. **Most Magnetio domains are 3-4 months stale** — verify before depending on them. The `unblockit.download` proxy pattern was dead by July 2026 (all 403).
2. **Domain rotation is critical but fallback domains die just as fast.** Refresh domain lists every 2-3 months.
3. **Proxy domains have different HTML structure than originals.** A KAT scraper that uses `tr.odd, tr.even` selectors on `katcr.to` won't work on `kick4ss.com`. Cheerio selectors must be updated per domain.
4. **The per-provider rate limiting (Bottleneck) + domain cooldown is the real value** — prevents hammering failing domains.
5. **TPB via apibay.org is the only consistently working free source as of July 2026.** BitSearch and TheRarBG are secondary.
6. **Content filtering by title is essential** — raw scraped results include unrelated content (e.g., TorrentGalaxy returns 190 latest torrents regardless of search query).
7. **`if (cached) return cached` is buggy**: `[]` is truthy in JS — cached empty arrays from dead-domain runs permanently disable the pipeline. Always check `cached && cached.length > 0`.
8. **`isTorrentMetadata(/^\d/)` kills single-word movies**: The pattern excludes "2010" as metadata, making "Inception" impossible to match. Don't exclude year numbers.
9. **Content filter min word count**: `Math.max(2, ceil(n * 0.7))` fails for single-word titles. Use `Math.max(1, ...)`.
10. **Magnetio uses named exports**: `export const id`, `export const name`, `export async function scrape`. Import via `import * as prov from './prov.js'`, not default import.