# Archive Project Research (2026-05-31)

Analysis of all projects in `/home/rurouni/Unspooled-archive/` for features to adopt.

## YARR! Stremio (`yarr-stremio-main/`)
### Top features
- **9 config presets**: Torrentio Replacement, 4K HDR Enthusiast, Speed Demon, Quality Over Quantity, Maximum Coverage, Mobile Optimized, Anime, Spanish, Portuguese — pre-curated full configs
- **RTN ranking**: Scored by quality(0-100) + source(0-100) + codec(0-12) + audio(0-15) + HDR(0-15) + group trust(0-10) + seeders(log) + size penalties
- **10 formatter styles**: prism, pro, torrentio, google, google-light, compact, ultra-clean, detailed, minimal, yarr
- **Smart dedup**: 4 strategies (infohash, smart-hash, tracker+name, filename) with OR logic
- **Provider health monitor**: 3-state (healthy→degraded→down), EMA tracking, auto-retry after 30min
- **ML recommendation engine**: Learns user preferences over time
- **Trakt-based auto-config**: Generates optimal setup from watch history
- **External addon aggregation**: Fetches from 3 user-configured addons
- **52 torrent scrapers + 310 IPTV sources**

## Comet (`comet-main/`)
### Top features
- **Cache state machine**: FRESH/STALE/EMPTY/FIRST_SEARCH states — intelligent cache+scrape decisions
- **Context-aware scraping**: `live` vs `background` modes per scraper — user-facing vs pre-caching
- **Sharded parse cache**: LRU split across shards with in-flight dedup via threading.Event
- **Multi-instance scraping**: JSON array of URLs treated as separate scraper instances
- **Scored file selection**: +1000 exact S/E, +500 multi-ep, +200 episode match, +100 filename match, +50 title match, size tiebreaker
- **DMM hashlist ingester**: Offline DB of known content from Debrid Media Manager
- **CometNet P2P**: WebSocket gossip protocol for metadata sharing between instances
- **Distributed DB locks**: PostgreSQL `ON CONFLICT DO UPDATE` for multi-instance coordination
- **Digital release filter**: TMDB release date check blocks unreleased content
- **Smart language detection**: Trakt country aliases → language mapping
- **18 scraper sources** including: Torrentio, Jackett, Prowlarr, Zilean, MediaFusion, AIOStreams, Nyaa, AnimeTosho, SeaDex, BitMagnet, DMM

## MediaFusion (`MediaFusion-main/`)
### Top features
- **290+ env vars** for every configurable parameter
- **PostgreSQL job queue** with LISTEN/NOTIFY: background scrapers, DMM ingester, metadata import, cache warmup
- **10 debrid providers**: RD, AD, DL, PM, TB, OC, ED, PikPak, Seedr, StremThru
- **15+ public HTML scraper indexers** with CSS selector configs
- **Usenet support**: 7 providers (Easynews, TorBox Usenet, Debrider, SABnzbd, NzbGet, NzbDAV, NNTP)
- **Live TV/AceStream/Telegram scrapers** via MTProto
- **Redis stampede prevention**: SETNX lock, 300s TTL, prevents concurrent scrapes of same content
- **PTT torrent name parsing**: Rust-native parse-torrent-title port
- **IMDb dataset import**: Periodic downloads from IMDb non-commercial datasets
- **Catalog system**: 50+ catalog sources with PostgreSQL-backed search (GIN + trigram indexes)
- **React config UI**: Vite + TailwindCSS 4 + Radix UI

## Sootio (`sootio-stremio-addon-main/`)
### Top features
- **Tiered debrid cache processor**: Quota system — REMUX=2, BluRay=2, WEB-DL=2, WEBRip=1, Audio=1, Other=10 per resolution
- **Early return system**: Configurable per-service timeout (RD=2500ms early return)
- **Multi-layer caching**: SQLite → return immediately → background refresh → Torz API integration
- **Search coordinator**: Singleton caches scraper results (10min TTL), shares across debrid services
- **Codec diversification**: Max 2 H.265 + 2 H.264 per resolution
- **Adaptive timeouts**: Based on historical P95 response times
- **HTTP streaming providers**: 13 direct HTTP sources with lazy URL resolution pattern
- **FlareSolverr integration**: Cloudflare bypass with circuit breaker
- **Home media server support**: Direct streaming from personal Plex/Jellyfin/Emby
- **462-line .env.example**: Extremely detailed configuration

## Scraper-Proxy (`scraper-proxy/`)
- Simple Go Fiber service, 2 files, 1083 lines
- Embed resolver with iframe chain following (up to 2 levels deep)
- Remote control endpoint for phone-to-TV text input
- Not relevant to our Node.js stack — skip

## Implemented in nexstream-scraper v2

From this research session:
1. Config presets (YARR!)
2. Per-resolution caps (Comet)
3. Adaptive timeouts (Sootio)
4. Provider health monitor (YARR!/Comet)
5. Request queue / inflight dedup (YARR!)
6. Speed tiers (YARR!)

Not yet implemented (high value):
- RTN ranking with group trust scores
- Formatter styles (10 display styles)
- Cache state machine (FRESH/STALE/EMPTY)
- Early return system
- DMM hashlist ingester
- Background cache warmer with demand priority
- Distributed lock system for multi-instance
- FlareSolverr integration
- Search coordinator (share results across debrid providers)