# apt-finder Implementation Plan

> **For Hermes:** Execute tasks sequentially. Tasks are dependency-linked (each builds on prior). Do NOT use subagent-driven-development for this — tasks share files and form a dependency chain. Direct execution with targeted `patch()` is faster.

**Goal:** A self-hosted CLI tool that scrapes Craigslist, Zillow, Apartments.com, and Redfin for rental listings, deduplicates across runs, outputs comparison tables, optionally pushes to Telegram, and logs contact attempts.

**Architecture:** Python CLI with plugin-style extractor registry (from free-scraper). Each site = one scraper module registered by domain. Craigslist via aiohttp+BS4 (easy), Zillow via HTML extraction, Apartments.com via Playwright, Redfin via aiohttp. SQLite for dedup. Rich for output tables.

**Tech Stack:** Python 3.11+, aiohttp, BeautifulSoup4, Playwright (chromium), Rich, SQLite3 (stdlib)

**Location:** `~/apt-finder/`

---

## Audit: What we take from each repo

### Taken into apt-finder:

| Repo | What we take | File |
|------|------------|------|
| **mory91/rent-finder** | Pydantic model schemas (ApartmentListing, SearchCriteria), async scraper base class, CLI entry point pattern | `models.py`, `scrapers/base.py`, `cli.py` |
| **hanzili/hanzi-browse apt-finder** | 3-phase workflow (Search→Compare→Contact), scam detection rules, safety checks, comparison table format, contact logging workflow | `safety.py`, `contact.py`, `output.py` |
| **flathunters/flathunter** | SQLite dedup pattern (store seen listing URLs, skip on re-scrape), Telegram notifier config, cron scheduling | `db.py`, `notify.py` |
| **mmynk/apartments-scraper** | Selenium scoring/ranking system (weighted score based on price, beds, distance), Apartments.com scraper approach | `scrapers/apartments.py` |
| **VikParuchuri/apartment-finder** | SQLite schema (listings table with dedup), Craigslist scraper approach | `db.py` |
| **free-scraper (your existing)** | Extractor registry pattern (register → findExtractor by domain → priority-based fallback chain) | `scrapers/registry.py` |
| **foxhound/scrapling (your existing)** | YAML config pattern for per-site settings, stealth browser headers | `config.yaml`, `scrapers/base.py` |
| **FlareSolverr (your existing)** | Cloudflare bypass proxy — if Playwright gets blocked, proxy through local FlareSolverr | `scrapers/base.py` |

### NOT taken (reasons):

- **rent-finder's LLM agent loop** — requires paid OpenAI/Anthropic API keys
- **rent-finder's evaluation framework** — reward functions, best-of-N selection, model comparison (useless for a real tool)
- **flathunter's captcha solvers** — 2Captcha/Capmonster/Imagetyperz integration (paid services)
- **flathunter's German-specific scrapers** — ImmoScout24, WG-Gesucht, Immowelt (irrelevant for US)
- **hanzi-browse's browser-extension dependency** — requires Chrome extension, can't cron
- **mmynk's Google Maps integration** — requires Google API key (paid)
- **thepalbi/deptos-scraper** — WIP, Argentina-specific, 2022 abandoned

### What Zillow confirmed:
Zillow search results page (`/homes/for_rent/{city}-{state}/`) renders listing data in server-side HTML. Price, address, beds/baths, sqft, amenities all available via BS4 extraction. No Playwright needed for initial results. Verified by web_extract on Chicago IL page.

---

## Project structure

```
~/apt-finder/
├── pyproject.toml
├── config.yaml
├── README.md
├── apt_finder/
│   ├── __init__.py
│   ├── __main__.py          # python -m apt_finder entry
│   ├── cli.py               # Click CLI with subcommands
│   ├── config.py            # Load config.yaml
│   ├── models.py            # Pydantic schemas
│   ├── db.py                # SQLite dedup + history
│   ├── output.py            # Rich comparison tables
│   ├── safety.py            # Scam detection rules
│   ├── contact.py           # Contact/logging workflow
│   ├── notify.py            # Telegram push
│   └── scrapers/
│       ├── __init__.py
│       ├── registry.py      # Extractor registry (from free-scraper)
│       ├── base.py          # AsyncHTTPScraper base class + FlareSolverr proxy
│       ├── craigslist.py    # aiohttp+BS4
│       ├── zillow.py        # aiohttp+BS4 (HTML extraction)
│       ├── apartments.py    # Playwright (heavy JS)
│       └── redfin.py        # aiohttp+BS4 (light JS)
└── tests/
    ├── __init__.py
    ├── test_models.py
    ├── test_db.py
    ├── test_safety.py
    └── test_scrapers/
        ├── __init__.py
        ├── test_craigslist.py
        ├── test_zillow.py
        └── test_redfin.py
```

---

## Tasks

### Task 1: Project scaffold + config + models

**Objective:** Create project skeleton, pyproject.toml, config.yaml loading, and Pydantic models

**Files:**
- Create: `~/apt-finder/pyproject.toml`
- Create: `~/apt-finder/config.yaml`
- Create: `~/apt-finder/apt_finder/__init__.py`
- Create: `~/apt-finder/apt_finder/__main__.py`
- Create: `~/apt-finder/apt_finder/config.py`
- Create: `~/apt-finder/apt_finder/models.py`
- Create: `~/apt-finder/tests/__init__.py`

**Step 1: Create project scaffold and models**

pyproject.toml:
```toml
[project]
name = "apt-finder"
version = "0.1.0"
description = "Self-hosted rental apartment finder CLI — scrapes Craigslist, Zillow, Apartments.com, Redfin"
requires-python = ">=3.11"
dependencies = [
    "click>=8.1.0",
    "aiohttp>=3.9.0",
    "beautifulsoup4>=4.12.0",
    "playwright>=1.40.0",
    "rich>=13.7.0",
    "pyyaml>=6.0",
    "pydantic>=2.5.0",
]

[project.scripts]
apt-finder = "apt_finder.cli:main"

[build-system]
requires = ["setuptools>=61.0"]
build-backend = "setuptools.build_meta"
```

config.yaml:
```yaml
# ~/apt-finder/config.yaml
defaults:
  min_price: 0
  max_price: 3000
  min_bedrooms: 1
  max_bedrooms: 3
  location: ""

scrapers:
  craigslist:
    enabled: true
    city_mapping:
      # Map city names to Craigslist subdomains
      boston: "boston"
      new-york: "newyork"
      san-francisco: "sfbay"
      chicago: "chicago"
      los-angeles: "losangeles"
      seattle: "seattle"
      austin: "austin"
      denver: "denver"
      portland: "portland"
      washington-dc: "washingtondc"

  zillow:
    enabled: true

  apartments:
    enabled: true

  redfin:
    enabled: true

database:
  path: "~/.apt-finder/listings.db"

telegram:
  enabled: false
  bot_token: ""
  chat_id: 0

output:
  format: "table"   # table, json, csv
  max_listings: 20
```

models.py:
```python
from datetime import datetime
from typing import Optional
from pydantic import BaseModel, Field
from enum import Enum

class Platform(str, Enum):
    CRAIGSLIST = "craigslist"
    ZILLOW = "zillow"
    APARTMENTS = "apartments"
    REDFIN = "redfin"

class Listing(BaseModel):
    platform: Platform
    title: str
    price: float
    address: str
    url: str
    bedrooms: Optional[float] = None
    bathrooms: Optional[float] = None
    sqft: Optional[int] = None
    available_date: Optional[str] = None
    amenities: list[str] = Field(default_factory=list)
    description: Optional[str] = None
    image_url: Optional[str] = None
    scraped_at: datetime = Field(default_factory=datetime.utcnow)
    listing_id: str = ""

    def dedup_key(self) -> str:
        """Unique key for dedup: URL wins, fallback to address+price"""
        if self.url:
            return self.url
        return f"{self.platform}|{self.address}|{self.price}"

class SearchCriteria(BaseModel):
    location: str = ""
    min_price: float = 0
    max_price: float = 3000
    min_bedrooms: float = 0
    max_bedrooms: float = 5
    platforms: list[Platform] = Field(default_factory=lambda: list(Platform))

class ScrapeResult(BaseModel):
    platform: Platform
    listings: list[Listing]
    error: Optional[str] = None
    total_found: int = 0
```

**Step 2: Create CLI entry point**

`cli.py`:
```python
import click

@click.group()
def main():
    """apt-finder: Self-hosted rental apartment finder."""
    pass

@main.command()
@click.option("--location", "-l", help="City or neighborhood to search")
@click.option("--max-price", "-p", type=int, default=3000, help="Maximum monthly rent")
@click.option("--min-bedrooms", "-b", type=float, default=1, help="Minimum bedrooms")
@click.option("--sites", help="Comma-separated sites (craigslist,zillow,apartments,redfin)")
def search(location, max_price, min_bedrooms, sites):
    """Search for apartments across all configured platforms."""
    click.echo(f"Searching {location}...")

@main.command()
def compare():
    """Show comparison table of last search results."""
    pass

@main.command()
def contact():
    """Log a contact attempt for a listing."""
    pass

@main.command()
def check():
    """Run scheduled check and push new listings via Telegram."""
    pass

if __name__ == "__main__":
    main()
```

**Verification:**
```bash
cd ~/apt-finder
python -m venv .venv
source .venv/bin/activate
pip install -e .
apt-finder --help
# Expected: shows search, compare, contact, check subcommands
```
---

### Task 2: SQLite database layer

**Objective:** Implement SQLite dedup + history storage

**Files:**
- Create: `~/apt-finder/apt_finder/db.py`
- Create: `~/apt-finder/tests/test_db.py`

**Step 1: Write test**
```python
import pytest
import tempfile
from pathlib import Path
from apt_finder.db import ListingDB
from apt_finder.models import Listing, Platform

@pytest.fixture
def db():
    with tempfile.NamedTemporaryFile(suffix='.db', delete=False) as f:
        db = ListingDB(Path(f.name))
        yield db
        db.close()
        Path(f.name).unlink()

def test_insert_and_dedup(db):
    listing = Listing(
        platform=Platform.CRAIGSLIST,
        title="Test Apartment",
        price=1500.0,
        address="123 Test St",
        url="https://test.com/1",
        bedrooms=1,
        bathrooms=1,
    )
    # First insert should succeed
    assert db.insert(listing) == True
    # Same URL again should be deduped
    assert db.insert(listing) == False

def test_get_new_listings(db):
    existing = Listing(
        platform=Platform.CRAIGSLIST,
        title="Existing", price=1000.0,
        address="1 Old St", url="https://old.com/1"
    )
    db.insert(existing)

    new_listing = Listing(
        platform=Platform.ZILLOW,
        title="New", price=2000.0,
        address="2 New St", url="https://new.com/1"
    )
    new_only = db.get_new([new_listing])
    assert len(new_only) == 1
    assert new_only[0].url == "https://new.com/1"

def test_get_all_listings(db):
    for i in range(3):
        db.insert(Listing(
            platform=Platform.CRAIGSLIST,
            title=f"Apt {i}", price=1000+i,
            address=f"{i} St", url=f"https://test.com/{i}"
        ))
    all_l = db.get_all()
    assert len(all_l) == 3
```

**Step 2: Implement db.py**
```python
import sqlite3
import json
from pathlib import Path
from datetime import datetime
from typing import Optional
from apt_finder.models import Listing, Platform

SCHEMA = """
CREATE TABLE IF NOT EXISTS listings (
    id INTEGER PRIMARY KEY AUTOINCREMENT,
    dedup_key TEXT UNIQUE NOT NULL,
    platform TEXT NOT NULL,
    title TEXT,
    price REAL,
    address TEXT,
    url TEXT,
    bedrooms REAL,
    bathrooms REAL,
    sqft INTEGER,
    available_date TEXT,
    amenities TEXT,
    description TEXT,
    image_url TEXT,
    scraped_at TEXT,
    contacted INTEGER DEFAULT 0,
    contacted_at TEXT,
    notes TEXT
);
CREATE INDEX IF NOT EXISTS idx_listings_platform ON listings(platform);
CREATE INDEX IF NOT EXISTS idx_listings_scraped ON listings(scraped_at);
"""

class ListingDB:
    def __init__(self, db_path: Path = Path("~/.apt-finder/listings.db").expanduser()):
        db_path.parent.mkdir(parents=True, exist_ok=True)
        self.conn = sqlite3.connect(str(db_path))
        self.conn.row_factory = sqlite3.Row
        self.conn.executescript(SCHEMA)

    def insert(self, listing: Listing) -> bool:
        """Insert a listing. Returns True if new, False if duplicate."""
        try:
            self.conn.execute(
                """INSERT OR IGNORE INTO listings
                (dedup_key, platform, title, price, address, url, bedrooms, bathrooms,
                 sqft, available_date, amenities, description, image_url, scraped_at)
                VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)""",
                (
                    listing.dedup_key(),
                    listing.platform.value,
                    listing.title,
                    listing.price,
                    listing.address,
                    listing.url,
                    listing.bedrooms,
                    listing.bathrooms,
                    listing.sqft,
                    listing.available_date,
                    json.dumps(listing.amenities),
                    listing.description,
                    listing.image_url,
                    listing.scraped_at.isoformat(),
                )
            )
            self.conn.commit()
            return self.conn.total_changes > 0
        except sqlite3.IntegrityError:
            return False

    def get_new(self, listings: list[Listing]) -> list[Listing]:
        """Filter to only listings not yet in DB."""
        existing = set()
        for row in self.conn.execute("SELECT dedup_key FROM listings"):
            existing.add(row["dedup_key"])
        return [l for l in listings if l.dedup_key() not in existing]

    def get_all(self, limit: int = 100) -> list[Listing]:
        """Get all stored listings."""
        rows = self.conn.execute(
            "SELECT * FROM listings ORDER BY scraped_at DESC LIMIT ?", (limit,)
        ).fetchall()
        return [self._row_to_listing(r) for r in rows]

    def mark_contacted(self, dedup_key: str, notes: str = "") -> None:
        self.conn.execute(
            "UPDATE listings SET contacted = 1, contacted_at = ?, notes = ? WHERE dedup_key = ?",
            (datetime.utcnow().isoformat(), notes, dedup_key)
        )
        self.conn.commit()

    def _row_to_listing(self, row) -> Listing:
        return Listing(
            platform=Platform(row["platform"]),
            title=row["title"] or "",
            price=row["price"] or 0,
            address=row["address"] or "",
            url=row["url"] or "",
            bedrooms=row["bedrooms"],
            bathrooms=row["bathrooms"],
            sqft=row["sqft"],
            available_date=row["available_date"],
            amenities=json.loads(row["amenities"]) if row["amenities"] else [],
            description=row["description"],
            image_url=row["image_url"],
        )

    def close(self):
        self.conn.close()
```

**Verification:**
```bash
pytest tests/test_db.py -v
# 3 tests PASS
```
---

### Task 3: Scraper registry + base class

**Objective:** Create the extractor registry pattern (from free-scraper) and base scraper class with FlareSolverr proxy support

**Files:**
- Create: `~/apt-finder/apt_finder/scrapers/__init__.py`
- Create: `~/apt-finder/apt_finder/scrapers/registry.py`
- Create: `~/apt-finder/apt_finder/scrapers/base.py`

**Step 1: Implement registry.py**
```python
"""Extractor registry — priority-based plugin system adapted from free-scraper."""

from typing import Protocol, Optional
from apt_finder.models import Listing, SearchCriteria

class Scraper(Protocol):
    name: str
    domains: list[str]
    priority: int

    async def search(self, criteria: SearchCriteria) -> list[Listing]:
        ...

_registry: list[Scraper] = []

def register(scraper: Scraper) -> None:
    """Register a scraper."""
    _registry.append(scraper)
    _registry.sort(key=lambda s: s.priority, reverse=True)

def find_by_domain(domain: str) -> Optional[Scraper]:
    """Find highest-priority scraper for a domain."""
    for s in _registry:
        if any(d in domain or domain == d for d in s.domains):
            return s
    return None

def get_enabled(platforms: list[str]) -> list[Scraper]:
    """Get all scrapers for the given platforms."""
    return [s for s in _registry if s.name in platforms]

def get_all() -> list[Scraper]:
    """Get all registered scrapers."""
    return list(_registry)
```

**Step 2: Implement base.py**
```python
"""Base scraper class with aiohttp session management and FlareSolverr fallback."""

import aiohttp
from typing import Optional
from apt_finder.models import Listing, SearchCriteria

class AsyncScraper:
    """Base class for all scrapers. Provides aiohttp session + FlareSolverr proxy."""

    name: str = ""
    domains: list[str] = []
    priority: int = 10

    def __init__(self, flare_url: Optional[str] = None, proxy: Optional[str] = None):
        self.flare_url = flare_url
        self.proxy = proxy
        self._session: Optional[aiohttp.ClientSession] = None

    async def get_session(self) -> aiohttp.ClientSession:
        if self._session is None or self._session.closed:
            timeout = aiohttp.ClientTimeout(total=30)
            headers = {
                "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) "
                              "AppleWebKit/537.36 Chrome/120 Safari/537.36",
                "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8",
                "Accept-Language": "en-US,en;q=0.5",
            }
            self._session = aiohttp.ClientSession(headers=headers, timeout=timeout)
        return self._session

    async def close(self):
        if self._session and not self._session.closed:
            await self._session.close()

    async def fetch(self, url: str) -> str:
        """Fetch a page via aiohttp. If FlareSolverr is configured, proxy through it."""
        session = await self.get_session()
        kwargs = {}
        if self.proxy:
            kwargs["proxy"] = self.proxy
        async with session.get(url, **kwargs) as resp:
            resp.raise_for_status()
            return await resp.text()

    async def search(self, criteria: SearchCriteria) -> list[Listing]:
        """Override in subclass."""
        raise NotImplementedError

    def __repr__(self) -> str:
        return f"<Scraper {self.name} (priority={self.priority})>"
```

**Verification:**
```bash
python -c "from apt_finder.scrapers.registry import get_all; print(get_all())"
# Expected: []
```
---

### Task 4: Craigslist scraper

**Objective:** Scrape Craigslist rental listings via aiohttp+BS4 on their search results page + internal JSON API

**Files:**
- Create: `~/apt-finder/apt_finder/scrapers/craigslist.py`
- Create: `~/apt-finder/tests/test_scrapers/test_craigslist.py`

**Implementation:**

**craigslist.py:**
```python
"""Craigslist rental scraper — uses aiohttp+BS4 on the search results page.
Also supports the internal JSON API endpoint for structured data."""

import re
import json
from bs4 import BeautifulSoup
from typing import Optional
from apt_finder.models import Listing, SearchCriteria, Platform
from apt_finder.scrapers.registry import register
from apt_finder.scrapers.base import AsyncScraper

CITY_MAPPINGS = {
    "boston": "boston", "new-york": "newyork", "new york": "newyork",
    "san-francisco": "sfbay", "san francisco": "sfbay", "sf": "sfbay",
    "chicago": "chicago", "los-angeles": "losangeles", "la": "losangeles",
    "seattle": "seattle", "austin": "austin", "denver": "denver",
    "portland": "portland", "washington-dc": "washingtondc", "dc": "washingtondc",
}

class CraigslistScraper(AsyncScraper):
    name = "craigslist"
    domains = ["craigslist.org"]
    priority = 20  # Highest — Craigslist has best data

    def __init__(self, city_mapping: Optional[dict] = None, **kwargs):
        super().__init__(**kwargs)
        self.city_mapping = city_mapping or CITY_MAPPINGS

    async def search(self, criteria: SearchCriteria) -> list[Listing]:
        city_slug = self._city_to_slug(criteria.location)
        if not city_slug:
            return []

        url = self._build_url(city_slug, criteria)
        html = await self.fetch(url)
        return self._parse_html(html, city_slug)

    def _city_to_slug(self, location: str) -> Optional[str]:
        """Convert a location string to a Craigslist subdomain."""
        if not location:
            return None
        key = location.lower().strip()
        if key in self.city_mapping:
            return self.city_mapping[key]
        # Try direct match (city name without spaces)
        return re.sub(r'[^a-z]', '', key)

    def _build_url(self, city: str, criteria: SearchCriteria) -> str:
        params = []
        if criteria.max_price > 0:
            params.append(f"max_price={int(criteria.max_price)}")
        if criteria.min_price > 0:
            params.append(f"min_price={int(criteria.min_price)}")
        if criteria.min_bedrooms > 0:
            params.append(f"min_bedrooms={int(criteria.min_bedrooms)}")
        if criteria.max_bedrooms < 5:
            params.append(f"max_bedrooms={int(criteria.max_bedrooms)}")
        params.append("availabilityMode=0")
        params.append("sale_date=all+dates")
        params.append("sort=date")
        base = f"https://{city}.craigslist.org/search/apa"
        return f"{base}?{'&'.join(params)}" if params else base

    def _parse_html(self, html: str, city: str) -> list[Listing]:
        """Parse Craigslist search results page HTML."""
        soup = BeautifulSoup(html, 'html.parser')
        listings = []

        # Try JSON-embedded data first (postings-data)
        json_ld = soup.find('script', type='application/ld+json')
        if json_ld:
            try:
                data = json.loads(json_ld.string)
                return self._parse_json_ld(data, city)
            except (json.JSONDecodeError, TypeError):
                pass

        # Fallback: HTML parsing
        for item in soup.select('.cl-static-search-result'):
            link = item.find('a')
            if not link:
                continue
            url = link.get('href', '')
            title_el = item.find('div', class_='title')
            price_el = item.find('div', class_='price')
            detail_el = item.find('div', class_='details')
            location_el = item.find('div', class_='location')

            title = title_el.get_text(strip=True) if title_el else "Unknown"
            price_text = price_el.get_text(strip=True) if price_el else "$0"
            price = float(re.sub(r'[^0-9.]', '', price_text) or 0)
            address = location_el.get_text(strip=True) if location_el else city

            # Parse bedrooms from details
            bedrooms = None
            if detail_el:
                detail_text = detail_el.get_text()
                bd_match = re.search(r'(\d+)\s*br', detail_text, re.I)
                if bd_match:
                    bedrooms = float(bd_match.group(1))

            listings.append(Listing(
                platform=Platform.CRAIGSLIST,
                title=title,
                price=price,
                address=address,
                url=url,
                bedrooms=bedrooms,
            ))

        return listings

    def _parse_json_ld(self, data: dict, city: str) -> list[Listing]:
        """Fallback JSON-LD parser if structured data is available."""
        listings = []
        items = data if isinstance(data, list) else [data]
        for item in items:
            if not isinstance(item, dict):
                continue
            listings.append(Listing(
                platform=Platform.CRAIGSLIST,
                title=item.get('name', 'Unknown'),
                price=float(item.get('price', '0').replace('$', '').replace(',', '') or 0),
                address=item.get('address', {}).get('addressLocality', city),
                url=item.get('url', ''),
                bedrooms=None,
            ))
        return listings

register(CraigslistScraper())
```

**Verification:**
```bash
python -c "
import asyncio
from apt_finder.models import SearchCriteria
from apt_finder.scrapers.craigslist import CraigslistScraper
s = CraigslistScraper()
results = asyncio.run(s.search(SearchCriteria(location='boston', max_price=2500, min_bedrooms=1)))
print(f'Found {len(results)} listings')
for r in results[:3]:
    print(f'  {r.price} - {r.title[:40]} - {r.url}')
"
# Expected: 5-10 listings from Boston Craigslist
```
---

### Task 5: Zillow scraper

**Objective:** Scrape Zillow rental listings via aiohttp+BS4 HTML extraction

**Files:**
- Create: `~/apt-finder/apt_finder/scrapers/zillow.py`
- Create: `~/apt-finder/tests/test_scrapers/test_zillow.py`

**Implementation:**

```python
"""Zillow rental scraper — extracts listing data from server-rendered HTML.
Zillow renders listing cards with price, address, beds/baths, sqft in the initial HTML."""

import re
from bs4 import BeautifulSoup
from typing import Optional
from urllib.parse import quote
from apt_finder.models import Listing, SearchCriteria, Platform
from apt_finder.scrapers.registry import register
from apt_finder.scrapers.base import AsyncScraper

class ZillowScraper(AsyncScraper):
    name = "zillow"
    domains = ["zillow.com"]
    priority = 15

    def __init__(self, **kwargs):
        super().__init__(**kwargs)

    async def search(self, criteria: SearchCriteria) -> list[Listing]:
        url = self._build_url(criteria)
        html = await self.fetch(url)
        return self._parse_listings(html)

    def _build_url(self, criteria: SearchCriteria) -> str:
        location = quote(criteria.location.strip()) if criteria.location else ""
        base = f"https://www.zillow.com/homes/for_rent/{location}/"
        params = []
        if criteria.max_price > 0:
            params.append(f"price={criteria.min_price}-{criteria.max_price}")
        if criteria.min_bedrooms > 0:
            params.append(f"beds={int(criteria.min_bedrooms)}-{int(criteria.max_bedrooms)}")
        params.append("searchQueryState=...")
        return f"{base}?{'&'.join(params)}" if params else base

    def _parse_listings(self, html: str) -> list[Listing]:
        """Extract listing cards from Zillow's search results HTML."""
        soup = BeautifulSoup(html, 'html.parser')
        listings = []
        seen_urls = set()

        # Zillow renders listing cards as anchor tags with specific data attributes
        for card in soup.select('article, [data-test="property-card"], a[href*="/apartments/"]'):
            url = card.get('href', '')
            if isinstance(url, list):
                url = url[0] if url else ''
            if not url or not url.startswith('/'):
                continue
            full_url = f"https://www.zillow.com{url}"
            if full_url in seen_urls:
                continue
            seen_urls.add(full_url)

            # Price
            price_el = card.select_one('[data-test="property-card-price"], span:has(> $)')
            price = 0
            if price_el:
                price_text = price_el.get_text(strip=True)
                price_match = re.search(r'\$?([0-9,]+)', price_text.replace(',', ''))
                if price_match:
                    price = float(price_match.group(1))

            # Address
            address_el = card.select_one('[data-test="property-card-addr"], address')
            address = address_el.get_text(strip=True) if address_el else ""

            # Title
            title_el = card.select_one('[data-test="property-card-title"], h2, h3')
            title = title_el.get_text(strip=True) if title_el else address

            # Beds/baths/size
            details_el = card.select_one('[data-test="property-card-details"]')
            bedrooms = None
            bathrooms = None
            sqft = None
            if details_el:
                detail_text = details_el.get_text().lower()
                bd = re.search(r'(\d+)\s*bd', detail_text)
                if bd: bedrooms = float(bd.group(1))
                ba = re.search(r'(\d+)\s*ba', detail_text)
                if ba: bathrooms = float(ba.group(1))
                sf = re.search(r'(\d+,?\d*)\s*sqft', detail_text.replace(',', ''))
                if sf: sqft = int(sf.group(1).replace(',', ''))

            if price == 0 and not address:
                continue  # Skip non-listing elements

            listings.append(Listing(
                platform=Platform.ZILLOW,
                title=title or address,
                price=price,
                address=address,
                url=full_url,
                bedrooms=bedrooms,
                bathrooms=bathrooms,
                sqft=sqft,
            ))

        return listings

register(ZillowScraper())
```

**Verification:**
```bash
python -c "
import asyncio
from apt_finder.models import SearchCriteria
from apt_finder.scrapers.zillow import ZillowScraper
s = ZillowScraper()
results = asyncio.run(s.search(SearchCriteria(location='boston', max_price=2500)))
print(f'Found {len(results)} listings')
for r in results[:3]:
    print(f'  \${r.price} - {r.address[:30]} - {r.bedrooms}bd/{r.bathrooms}ba')
"
# Expected: ~20 listings from Zillow Boston
```
---

### Task 6: Apartments.com scraper (Playwright)

**Objective:** Scrape Apartments.com listings via Playwright (site is heavy JS-rendered)

**Files:**
- Create: `~/apt-finder/apt_finder/scrapers/apartments.py`

**Implementation:**

```python
"""Apartments.com scraper — uses Playwright for JS-rendered listings.
This site is heavily JS-dependent. Playwright is required."""

import re
from typing import Optional
from urllib.parse import quote
from apt_finder.models import Listing, SearchCriteria, Platform
from apt_finder.scrapers.registry import register
from apt_finder.scrapers.base import AsyncScraper

class ApartmentsScraper(AsyncScraper):
    name = "apartments"
    domains = ["apartments.com"]
    priority = 10

    def __init__(self, headless: bool = True, **kwargs):
        super().__init__(**kwargs)
        self.headless = headless

    async def search(self, criteria: SearchCriteria) -> list[Listing]:
        """Use Playwright to load Apartments.com and extract listings."""
        try:
            from playwright.async_api import async_playwright
        except ImportError:
            return [Listing(
                platform=Platform.APARTMENTS,
                title="Playwright not installed",
                price=0, address="", url="",
            )]

        url = self._build_url(criteria)
        listings = []

        async with async_playwright() as p:
            browser = await p.chromium.launch(headless=self.headless)
            page = await browser.new_page()
            await page.goto(url, wait_until="networkidle", timeout=30000)

            # Wait for listing cards to render
            await page.wait_for_selector('[data-page="search"] article, .placard, .rental-listing', timeout=10000)

            cards = await page.query_selector_all('article, .placard')
            for card in cards[:25]:  # Limit to 25 listings
                try:
                    title_el = await card.query_selector('[data-mark="listing-title"] span, h2, .property-title')
                    title = await title_el.inner_text() if title_el else ""

                    price_el = await card.query_selector('[data-mark="listing-price"], .price, .rent-amount')
                    price_text = await price_el.inner_text() if price_el else "$0"
                    price_match = re.search(r'\$?([0-9,]+)', price_text.replace(',', ''))
                    price = float(price_match.group(1)) if price_match else 0

                    addr_el = await card.query_selector('[data-mark="listing-address"], address, .property-address')
                    address = await addr_el.inner_text() if addr_el else ""

                    link_el = await card.query_selector('a[href*="/apartments/"]')
                    url_part = await link_el.get_attribute('href') if link_el else ""
                    full_url = f"https://www.apartments.com{url_part}" if url_part else url

                    beds_el = await card.query_selector('[data-mark="listing-bedroom-count"], .bedroom-count')
                    beds_text = await beds_el.inner_text() if beds_el else ""
                    bedrooms = float(re.search(r'(\d+)', beds_text).group(1)) if re.search(r'(\d+)', beds_text) else None

                    if price > 0:
                        listings.append(Listing(
                            platform=Platform.APARTMENTS,
                            title=title or address,
                            price=price,
                            address=address,
                            url=full_url,
                            bedrooms=bedrooms,
                        ))
                except Exception:
                    continue

            await browser.close()

        return listings

    def _build_url(self, criteria: SearchCriteria) -> str:
        location = quote(criteria.location.strip()) if criteria.location else ""
        base = f"https://www.apartments.com/{location}/"
        params = []
        if criteria.max_price > 0:
            params.append(f"price={int(criteria.max_price)}")
        if criteria.min_bedrooms > 0:
            params.append(f"beds={int(criteria.min_bedrooms)}-{int(criteria.max_bedrooms)}")
        return f"{base}?{'&'.join(params)}" if params else base

register(ApartmentsScraper())
```

**Verification:**
```bash
playwright install chromium
python -c "
import asyncio
from apt_finder.models import SearchCriteria
from apt_finder.scrapers.apartments import ApartmentsScraper
s = ApartmentsScraper()
results = asyncio.run(s.search(SearchCriteria(location='boston-ma', max_price=2500)))
print(f'Found {len(results)} listings')
for r in results[:3]:
    print(f'  \${r.price} - {r.title[:40]}')
"
# Expected: ~10-25 listings
```
---

### Task 7: Redfin scraper

**Objective:** Scrape Redfin rental listings via aiohttp+BS4

**Files:**
- Create: `~/apt-finder/apt_finder/scrapers/redfin.py`
- Create: `~/apt-finder/tests/test_scrapers/test_redfin.py`

**Implementation:**

```python
"""Redfin rental scraper — aiohttp+BS4 for rental search results."""

import re
import json
from bs4 import BeautifulSoup
from urllib.parse import quote
from typing import Optional
from apt_finder.models import Listing, SearchCriteria, Platform
from apt_finder.scrapers.registry import register
from apt_finder.scrapers.base import AsyncScraper

class RedfinScraper(AsyncScraper):
    name = "redfin"
    domains = ["redfin.com"]
    priority = 10

    def __init__(self, **kwargs):
        super().__init__(**kwargs)

    async def search(self, criteria: SearchCriteria) -> list[Listing]:
        url = self._build_url(criteria)
        html = await self.fetch(url)
        return self._parse_listings(html)

    def _build_url(self, criteria: SearchCriteria) -> str:
        location = quote(criteria.location.strip()) if criteria.location else ""
        return f"https://www.redfin.com/apartments-for-rent/{location}"

    def _parse_listings(self, html: str) -> list[Listing]:
        """Extract listing data from Redfin search results."""
        soup = BeautifulSoup(html, 'html.parser')
        listings = []
        seen_urls = set()

        # Try embedded JSON data first
        for script in soup.find_all('script', type='application/ld+json'):
            try:
                data = json.loads(script.string)
                items = data if isinstance(data, list) else [data]
                for item in items:
                    if not isinstance(item, dict):
                        continue
                    url = item.get('url', '')
                    if url in seen_urls:
                        continue
                    seen_urls.add(url)
                    price_str = str(item.get('price', '0')).replace('$', '').replace(',', '')
                    try:
                        price = float(price_str)
                    except ValueError:
                        price = 0

                    address = item.get('address', {})
                    if isinstance(address, dict):
                        addr = address.get('streetAddress', '') or ''
                    else:
                        addr = str(address)

                    listings.append(Listing(
                        platform=Platform.REDFIN,
                        title=item.get('name', addr),
                        price=price,
                        address=addr,
                        url=url,
                    ))
                if listings:
                    return listings
            except (json.JSONDecodeError, TypeError):
                continue

        # Fallback: HTML parsing
        for card in soup.select('[data-rf-test-id="property-card"], .property-card'):
            link = card.find('a', href=re.compile(r'/apartments/'))
            if not link:
                continue
            url = link.get('href', '')
            if url in seen_urls:
                continue
            seen_urls.add(url)
            full_url = f"https://www.redfin.com{url}" if url.startswith('/') else url

            price_el = card.select_one('.price, [data-rf-test-id="property-card-price"]')
            price_text = price_el.get_text(strip=True) if price_el else "$0"
            price = float(re.sub(r'[^0-9.]', '', price_text) or 0)

            addr_el = card.select_one('.address, [data-rf-test-id="property-card-address"]')
            address = addr_el.get_text(strip=True) if addr_el else ""

            if price > 0:
                listings.append(Listing(
                    platform=Platform.REDFIN,
                    title=address,
                    price=price,
                    address=address,
                    url=full_url,
                ))

        return listings

register(RedfinScraper())
```

**Verification:**
```bash
python -c "
import asyncio
from apt_finder.models import SearchCriteria
from apt_finder.scrapers.redfin import RedfinScraper
s = RedfinScraper()
results = asyncio.run(s.search(SearchCriteria(location='boston-ma', max_price=2500)))
print(f'Found {len(results)} listings')
for r in results[:3]:
    print(f'  \${r.price} - {r.address[:30]}')
"
# Expected: ~5-15 listings (Redfin is stricter about blocking)
```
---

### Task 8: Output — Rich comparison tables

**Objective:** Generate formatted comparison tables using Rich library

**Files:**
- Create: `~/apt-finder/apt_finder/output.py`

**Implementation:**

```python
"""Output formatting — Rich comparison tables + JSON/CSV export."""

from typing import Optional
from rich.console import Console
from rich.table import Table
from rich.text import Text
from rich import box
import json
import csv
import io
from apt_finder.models import Listing

console = Console()

def render_table(listings: list[Listing], title: str = "Apartment Listings") -> str:
    """Render listings as a formatted Rich table. Returns the string output."""
    table = Table(title=title, box=box.ROUNDED, header_style="bold cyan")
    table.add_column("#", style="dim", width=3)
    table.add_column("Price", style="green", width=10)
    table.add_column("Bed/Bath", style="yellow", width=10)
    table.add_column("Address", style="white", width=35, no_wrap=False)
    table.add_column("Platform", style="blue", width=12)
    table.add_column("Link", style="blue", width=50)

    for i, listing in enumerate(listings, 1):
        beds = f"{listing.bedrooms or '?'}bd"
        baths = f"/{listing.bathrooms or '?'}ba" if listing.bathrooms else ""
        bed_bath = f"{beds}{baths}" if listing.bedrooms else "—"

        price_color = "green" if listing.price < 1500 else "yellow" if listing.price < 2500 else "red"
        price_str = f"${int(listing.price):,}" if listing.price else "—"

        addr = listing.address or listing.title or ""
        if len(addr) > 35:
            addr = addr[:32] + "..."

        table.add_row(
            str(i),
            Text(price_str, style=price_color),
            bed_bath,
            Text(addr, style="white"),
            listing.platform.value,
            listing.url or "",
        )

    with console.capture() as capture:
        console.print(table)
    return capture.get()

def render_summary(listings: list[Listing]) -> str:
    """Generate a short text summary of findings."""
    if not listings:
        return "No listings found."

    prices = [l.price for l in listings if l.price > 0]
    avg_price = sum(prices) / len(prices) if prices else 0

    lines = [
        f"**Total listings:** {len(listings)}",
        f"**Price range:** ${int(min(prices or [0])):,} — ${int(max(prices or [0])):,}",
        f"**Average price:** ${int(avg_price):,}",
        f"**Platforms:** {', '.join(set(l.platform.value for l in listings))}",
        "",
        "**Cheapest:**",
    ]
    cheapest = sorted(listings, key=lambda l: l.price)[:3]
    for l in cheapest:
        lines.append(f"  ${int(l.price):,} — {l.title[:40]} ({l.platform.value})")

    return "\n".join(lines)

def to_json(listings: list[Listing], indent: int = 2) -> str:
    """Export listings as JSON."""
    return json.dumps([l.model_dump() for l in listings], indent=indent, default=str)

def to_csv(listings: list[Listing]) -> str:
    """Export listings as CSV string."""
    output = io.StringIO()
    writer = csv.writer(output)
    writer.writerow(["price", "address", "bedrooms", "bathrooms", "sqft", "platform", "url", "title"])
    for l in listings:
        writer.writerow([l.price, l.address, l.bedrooms, l.bathrooms, l.sqft, l.platform.value, l.url, l.title])
    return output.getvalue()
```

**Verification:**
```bash
python -c "
from apt_finder.models import Listing, Platform
from apt_finder.output import render_table
l = [Listing(platform=Platform.CRAIGSLIST, title='Test', price=1500, address='123 Main St', url='https://x.com/1', bedrooms=1, bathrooms=1)]
print(render_table(l))
"
# Expected: Formatted table with price, address, bed/bath, link
```
---

### Task 9: Scam detection (safety)

**Objective:** Implement scam detection rules from hanzi-browse apt-finder skill

**Files:**
- Create: `~/apt-finder/apt_finder/safety.py`
- Create: `~/apt-finder/tests/test_safety.py`

**Implementation:**

```python
"""Scam detection rules adapted from hanzili/hanzi-browse apartment-finder SKILL.md."""

import re
from typing import Optional
from apt_finder.models import Listing

SUSPICIOUS_PATTERNS = [
    r"owner\s+is\s+(overseas|abroad|out\s+of\s+country)",
    r"(send|wire|transfer)\s+(deposit|money|payment).*?(hold|reserve|secure)",
    r"(can'?t\s+)?(show|view|see)\s+(the\s+)?(unit|apartment|place)",
    r"(western\s+union|moneygram|gift\s+card|cash\s+app|venmo\s+deposit)",
    r"no\s+(credit\s+)?check.*?(move|move-in|rent)",
    r"below\s+market.*?(price|rent)",
]

CONTACT_RED_FLAGS = [
    r"@gmail\.com$",
    r"@yahoo\.com$",
    r"@outlook\.com$",
    r"@hotmail\.com$",
]

class SafetyReport:
    def __init__(self, listing: Listing, avg_price: float = 0):
        self.listing = listing
        self.avg_price = avg_price
        self.flags: list[str] = []
        self._check()

    def _check(self):
        description = (self.listing.description or "").lower()
        title = self.listing.title.lower()

        # Check for suspicious text patterns
        for pattern in SUSPICIOUS_PATTERNS:
            if re.search(pattern, description, re.I) or re.search(pattern, title, re.I):
                self.flags.append(f"Suspicious text pattern: '{pattern}'")

        # Check for price significantly below market
        if self.avg_price > 0 and self.listing.price > 0:
            ratio = self.listing.price / self.avg_price
            if ratio < 0.7:
                self.flags.append(
                    f"Price (${int(self.listing.price):,}) is {int((1-ratio)*100)}% "
                    f"below market average (${int(self.avg_price):,})"
                )

        # Check for personal email in description
        email_match = re.search(r'[\w.]+@[\w.]+\.\w+', description or "")
        if email_match:
            email = email_match.group()
            for pattern in CONTACT_RED_FLAGS:
                if re.search(pattern, email, re.I):
                    self.flags.append(f"Personal email used for contact: {email}")

        # Check for no photos (empty image_url)
        if not self.listing.image_url:
            self.flags.append("No photos provided")

    @property
    def is_suspicious(self) -> bool:
        return len(self.flags) >= 2

    @property
    def risk_level(self) -> str:
        if len(self.flags) >= 3:
            return "HIGH"
        elif len(self.flags) >= 1:
            return "MEDIUM"
        return "LOW"

    def summary(self) -> str:
        if not self.flags:
            return "✅ No red flags detected"
        lines = [f"⚠️  Risk level: {self.risk_level}"]
        for f in self.flags:
            lines.append(f"  • {f}")
        return "\n".join(lines)
```

**Verification:**
```bash
python -c "
from apt_finder.models import Listing, Platform
from apt_finder.safety import SafetyReport

# Safe listing
safe = Listing(platform=Platform.CRAIGSLIST, title='Nice 1BR', price=1500, address='123 Main', url='x', description='Great location near transit')
print('Safe:', SafetyReport(safe).summary())

# Suspicious listing
bad = Listing(platform=Platform.CRAIGSLIST, title='Too good to be true', price=500, address='456 Fake', url='y', description='Owner is overseas, send deposit via Western Union to hold the unit. Email: scammer@gmail.com')
print('Suspicious:', SafetyReport(bad, avg_price=1500).summary())
"
```
---

### Task 10: Contact workflow + logging

**Objective:** Contact drafting/message logging workflow

**Files:**
- Create: `~/apt-finder/apt_finder/contact.py`

**Implementation:**

```python
"""Contact workflow — draft messages, log contact attempts."""

from datetime import datetime
from pathlib import Path
from typing import Optional
from apt_finder.models import Listing

CONTACTS_FILE = Path("~/.apt-finder/contacts.log").expanduser()

def draft_inquiry(listing: Listing, name: str = "", email: str = "") -> str:
    """Draft an inquiry message for a listing. User must approve before sending."""
    return f"""To: {listing.title} at {listing.address}
Re: Rental listing — ${int(listing.price):,}/mo

Hi,

I'm interested in the unit at {listing.address} listed at ${int(listing.price):,}/month.
I'm looking to move in soon and would love to schedule a viewing or get more information.

Could you let me know about availability, application requirements, and any additional fees?

Thank you,
{name or "[Your Name]"}
{email or "[Your Email]"}
"""

def log_contact(listing: Listing, method: str = "website", status: str = "sent", notes: str = "") -> None:
    """Log a contact attempt to the contacts log file."""
    CONTACTS_FILE.parent.mkdir(parents=True, exist_ok=True)
    timestamp = datetime.utcnow().isoformat()
    line = (
        f"{timestamp} | {listing.platform.value} | {listing.address} | "
        f"${int(listing.price):,} | {method} | {status}"
    )
    if notes:
        line += f" | {notes}"
    with open(CONTACTS_FILE, "a") as f:
        f.write(line + "\n")

def get_contact_log(limit: int = 20) -> list[str]:
    """Read recent contact log entries."""
    if not CONTACTS_FILE.exists():
        return []
    with open(CONTACTS_FILE) as f:
        lines = f.readlines()
    return [l.strip() for l in lines[-limit:]]
```
---

### Task 11: Telegram notifier

**Objective:** Send new listing results via Telegram bot

**Files:**
- Create: `~/apt-finder/apt_finder/notify.py`

**Implementation:**

```python
"""Telegram notification — send new listings to a Telegram chat."""

from typing import Optional
from apt_finder.models import Listing

async def send_telegram(
    listings: list[Listing],
    bot_token: str,
    chat_id: int,
    api_url: str = "https://api.telegram.org",
) -> bool:
    """Send new listings to a Telegram chat."""
    import aiohttp

    if not bot_token or not chat_id:
        return False

    messages = []
    for listing in listings[:5]:  # Max 5 per message to avoid length limits
        beds = f"{listing.bedrooms}br" if listing.bedrooms else "?"
        msg = (
            f"🏠 *{listing.title[:50]}*\n"
            f"💰 ${int(listing.price):,}/mo | 🛏 {beds}\n"
            f"📍 {listing.address[:60]}\n"
            f"🔗 {listing.url}"
        )
        messages.append(msg)

    if not messages:
        return False

    full_msg = "🔍 *New Apartment Listings*\n\n" + "\n\n".join(messages)
    url = f"{api_url}/bot{bot_token}/sendMessage"

    async with aiohttp.ClientSession() as session:
        async with session.post(url, json={
            "chat_id": chat_id,
            "text": full_msg,
            "parse_mode": "Markdown",
            "disable_web_page_preview": True,
        }) as resp:
            return resp.status == 200
```
---

### Task 12: CLI — wire everything together

**Objective:** Complete CLI with search, compare, contact, check commands

**Files:**
- Modify: `~/apt-finder/apt_finder/cli.py`
- Modify: `~/apt-finder/apt_finder/__main__.py`

**Implementation — cli.py (full):**

```python
"""CLI entry point — search, compare, contact, check."""

import asyncio
import sys
from pathlib import Path
from typing import Optional

import click
from rich.console import Console
from rich.prompt import Prompt
from rich.table import Table

from apt_finder.config import load_config
from apt_finder.models import Listing, SearchCriteria, Platform
from apt_finder.scrapers.registry import get_enabled
from apt_finder.db import ListingDB
from apt_finder.output import render_table, render_summary, to_json, to_csv
from apt_finder.safety import SafetyReport
from apt_finder.contact import draft_inquiry, log_contact

console = Console()

@click.group()
@click.option("--config", "-c", default="~/.apt-finder/config.yaml", help="Config file path")
@click.pass_context
def main(ctx, config):
    """apt-finder: Self-hosted rental apartment finder."""
    ctx.ensure_object(dict)
    ctx.obj["config_path"] = Path(config).expanduser()

@main.command()
@click.option("--location", "-l", default="", help="City to search (e.g. boston, chicago)")
@click.option("--max-price", "-p", type=int, default=0, help="Maximum monthly rent")
@click.option("--min-price", type=int, default=0, help="Minimum monthly rent")
@click.option("--min-bedrooms", "-b", type=float, default=0, help="Minimum bedrooms")
@click.option("--max-bedrooms", type=float, default=5, help="Maximum bedrooms")
@click.option("--sites", help="Comma-separated sites (craigslist,zillow,apartments,redfin)")
@click.option("--output", type=click.Choice(["table", "json", "csv"]), default="table")
@click.option("--save/--no-save", default=True, help="Save to database")
@click.pass_context
def search(ctx, location, max_price, min_price, min_bedrooms, max_bedrooms, sites, output, save):
    """Search for apartments across all configured platforms."""
    config = load_config(ctx.obj["config_path"])
    cfg_defaults = config.get("defaults", {})

    criteria = SearchCriteria(
        location=location or cfg_defaults.get("location", ""),
        min_price=min_price or cfg_defaults.get("min_price", 0),
        max_price=max_price or cfg_defaults.get("max_price", 3000),
        min_bedrooms=min_bedrooms or cfg_defaults.get("min_bedrooms", 1),
        max_bedrooms=max_bedrooms or cfg_defaults.get("max_bedrooms", 3),
        platforms=[],
    )

    # Determine which platforms to search
    if sites:
        platform_names = [s.strip() for s in sites.split(",")]
    else:
        cfg_sites = config.get("scrapers", {})
        platform_names = [k for k, v in cfg_sites.items() if v.get("enabled", True)]
    criteria.platforms = [Platform(p) for p in platform_names if p in Platform._value2member_map_]

    if not criteria.location:
        criteria.location = click.prompt("Location (city)")
        if not criteria.location:
            console.print("[red]Location is required[/red]")
            sys.exit(1)

    console.print(f"[bold]Searching {criteria.location} for rentals up to ${criteria.max_price:,}...[/bold]")

    scrapers = get_enabled(platform_names)
    if not scrapers:
        console.print("[red]No scrapers found for configured platforms[/red]")
        sys.exit(1)

    all_listings = []
    errors = []

    async def run_searches():
        nonlocal all_listings, errors
        for scraper in scrapers:
            try:
                results = await scraper.search(criteria)
                console.print(f"  [green]✓[/green] {scraper.name}: {len(results)} listings")
                all_listings.extend(results)
            except Exception as e:
                console.print(f"  [red]✗[/red] {scraper.name}: {e}")
                errors.append(f"{scraper.name}: {e}")

    asyncio.run(run_searches())

    if not all_listings:
        console.print("\n[yellow]No listings found. Try a different location or broader criteria.[/yellow]")
        sys.exit(0)

    # Dedup and save
    if save:
        db = ListingDB()
        new_listings = db.get_new(all_listings)
        for l in new_listings:
            db.insert(l)
        db.close()
        console.print(f"\nSaved {len(new_listings)} new listings to database")
        display_listings = all_listings
    else:
        display_listings = all_listings

    # Output
    if output == "json":
        console.print(to_json(display_listings))
    elif output == "csv":
        console.print(to_csv(display_listings))
    else:
        console.print(render_table(display_listings[:config.get("output", {}).get("max_listings", 20)]))
        console.print(render_summary(display_listings))

    if errors:
        console.print(f"\n[yellow]Errors: {len(errors)}[/yellow]")

@main.command()
@click.option("--limit", type=int, default=20)
@click.option("--output", type=click.Choice(["table", "json", "csv"]), default="table")
@click.pass_context
def compare(ctx, limit, output):
    """Show comparison table of saved listings."""
    db = ListingDB()
    listings = db.get_all(limit)
    db.close()

    if not listings:
        console.print("[yellow]No saved listings. Run 'search' first.[/yellow]")
        return

    if output == "json":
        console.print(to_json(listings))
    elif output == "csv":
        console.print(to_csv(listings))
    else:
        console.print(render_table(listings, title="Saved Listings"))

@main.command()
@click.argument("listing_idx", type=int, required=False)
@click.pass_context
def contact(ctx, listing_idx):
    """Draft a contact message for a listing and log it."""
    db = ListingDB()
    listings = db.get_all(50)
    db.close()

    if not listings:
        console.print("[yellow]No listings saved. Run 'search' first.[/yellow]")
        return

    if not listing_idx:
        console.print(render_table(listings[:20], title="Select a listing"))
        listing_idx = click.prompt("Enter listing number", type=int)

    if listing_idx < 1 or listing_idx > len(listings):
        console.print("[red]Invalid listing number[/red]")
        return

    listing = listings[listing_idx - 1]
    name = click.prompt("Your name", default="")
    email = click.prompt("Your email", default="")

    msg = draft_inquiry(listing, name, email)
    console.print("\n[bold]Draft message:[/bold]")
    console.print(msg)

    if click.confirm("Send this inquiry?"):
        log_contact(listing)
        console.print("[green]✓ Contact logged[/green]")
    else:
        console.print("[yellow]Cancelled[/yellow]")

@main.command()
@click.pass_context
def check(ctx):
    """Run scheduled check and push new listings via Telegram."""
    config = load_config(ctx.obj["config_path"])
    telegram_cfg = config.get("telegram", {})
    if not telegram_cfg.get("enabled"):
        console.print("[yellow]Telegram not configured. Edit config.yaml to enable.[/yellow]")

    # Run search with saved criteria from last run
    # For now, just check DB for listings not yet notified
    console.print("[bold]Scheduled check complete[/bold]")

if __name__ == "__main__":
    main()
```

**__main__.py:**
```python
from apt_finder.cli import main
main()
```

**Verification:**
```bash
# Full integration test
apt-finder search --location boston --max-price 2500 --min-bedrooms 1 --output table
# Expected: searches all enabled sites, prints comparison table
```
---

### Task 13: Config loader

**Objective:** Load config.yaml with sensible defaults

**Files:**
- Create: `~/apt-finder/apt_finder/config.py`

**Implementation:**

```python
"""Configuration loader — reads config.yaml with defaults."""

import yaml
from pathlib import Path
from typing import Any

DEFAULT_CONFIG = {
    "defaults": {
        "min_price": 0,
        "max_price": 3000,
        "min_bedrooms": 1,
        "max_bedrooms": 3,
        "location": "",
    },
    "scrapers": {
        "craigslist": {"enabled": True},
        "zillow": {"enabled": True},
        "apartments": {"enabled": True},
        "redfin": {"enabled": True},
    },
    "output": {
        "format": "table",
        "max_listings": 20,
    },
    "database": {
        "path": "~/.apt-finder/listings.db",
    },
    "telegram": {
        "enabled": False,
        "bot_token": "",
        "chat_id": 0,
    },
}

def load_config(path: Path) -> dict[str, Any]:
    """Load config from YAML file, merging with defaults."""
    config = DEFAULT_CONFIG.copy()
    if path.exists():
        with open(path) as f:
            user_config = yaml.safe_load(f) or {}
        _deep_merge(config, user_config)
    return config

def _deep_merge(base: dict, override: dict) -> None:
    """Deep merge override into base dict."""
    for key, value in override.items():
        if key in base and isinstance(base[key], dict) and isinstance(value, dict):
            _deep_merge(base[key], value)
        else:
            base[key] = value
```
---

## Dependencies

After all tasks, install and verify:
```bash
cd ~/apt-finder
pip install -e .
playwright install chromium

# Smoke test each scraper
python -c "
import asyncio
from apt_finder.scrapers.registry import get_all
for s in get_all():
    print(f'Registered: {s.name} [{s.domains}] priority={s.priority}')
"
```

---

## Edge cases & pitfalls

| Issue | Mitigation |
|-------|-----------|
| Zillow blocks plain HTTP | Fall back to Playwright mode or proxy through FlareSolverr |
| Apartments.com Playwright timeout | Increase timeout, add retry logic in base class |
| Craigslist changes HTML structure | JSON-LD fallback parser in craigslist.py |
| Rate limiting (too many requests) | Add random delays between requests (1-3s) in base.fetch() |
| No listings found for criteria | Clear error message suggesting broader search |
| DB file locked (concurrent runs) | WAL mode in SQLite |
| Telegram not configured | Graceful skip, info message |
| Missing Playwright for apartments.com | Install prompt on first run |
```