docs(plan): add Phase 1, 2, 3 implementation specs

docs/phase-1/task-httpx-singleton.md

@@ -0,0 +1,157 @@
+# Task: httpx singleton with connection pool
+
+## Description
+
+The current `scraper.fetch_ads()` creates a **new** `httpx.AsyncClient` on every call:
+
+```python
+async def fetch_ads(keyword: str):
+    async with httpx.AsyncClient(timeout=30.0) as client:
+        resp = await client.get(_API_URL, ...)
+```
+
+This means each scrape cycle incurs the full cost of TCP handshake + TLS negotiation (≈100-300ms per call on a cold connection). For keywords scraped every 5 minutes with multiple pages, this overhead adds up to **seconds of unnecessary latency per cycle**.
+
+This task replaces the per-call client with a module-level singleton that reuses connections via keepalive.
+
+## Architecture
+
+```
+Current:
+  Cycle 1: create AsyncClient → fetch → close → ~300ms overhead
+  Cycle 2: create AsyncClient → fetch → close → ~300ms overhead
+  Cycle N: ... (repeated forever)
+
+Target:
+  Module load: create AsyncClient (singleton, keepalive pool)
+  Cycle 1: use client → fetch → ~50ms (warm connection)
+  Cycle 2: use client → fetch → ~50ms (warm connection)
+  Cycle N: ... 
+  Shutdown: close client gracefully
+```
+
+### Key design decisions
+
+- **Module-level singleton** (`_client = None`, lazy init). Simpler than dependency injection and works with the existing async context.
+- **Keepalive connections**: Default `max_keepalive_connections=5` handles concurrent keyword scrapes efficiently.
+- **Client recreation on error**: If the client is closed or encounters a fatal transport error, it's recreated on the next call. This prevents stale connection issues.
+
+## Implementation Details
+
+### 1. Add singleton getter to `scraper.py`
+
+```python
+import os
+
+_client: httpx.AsyncClient | None = None
+
+
+async def get_client() -> httpx.AsyncClient:
+    """Return a shared AsyncClient with keepalive connection pool."""
+    global _client
+    
+    if _client is None or _client.is_closed:
+        max_conns = int(os.getenv("HTTP_MAX_CONNECTIONS", "10"))
+        max_keepalive = int(os.getenv("HTTP_KEEPALIVE_CONNECTIONS", "5"))
+        
+        _client = httpx.AsyncClient(
+            timeout=float(os.getenv("HTTP_TIMEOUT_S", "30.0")),
+            limits=httpx.Limits(
+                max_connections=max_conns,
+                max_keepalive_connections=max_keepalive,
+                keepalive_expiry=60,  # seconds
+            ),
+        )
+        logger.info(
+            "Created httpx client: max_conns=%d, keepalive=%d",
+            max_conns, max_keepalive,
+        )
+    
+    return _client
+
+
+async def close_client() -> None:
+    """Close the shared AsyncClient. Call during shutdown."""
+    global _client
+    if _client and not _client.is_closed:
+        await _client.aclose()
+        logger.info("Closed httpx client")
+        _client = None
+```
+
+### 2. Update `fetch_ads()` to use the singleton
+
+**Replace:**
+```python
+async def fetch_ads(keyword: str):
+    params = {...}
+    async with httpx.AsyncClient(timeout=30.0) as client:
+        for attempt in range(1, 4):
+            try:
+                resp = await client.get(_API_URL, headers=_HEADERS, params=params)
+```
+
+**With:**
+```python
+async def fetch_ads(keyword: str):
+    params = {...}
+    client = await get_client()
+    
+    for attempt in range(1, 4):
+        try:
+            resp = await client.get(_API_URL, headers=_HEADERS, params=params)
+            resp.raise_for_status()
+            data = resp.json()
+            break
+        except httpx.ConnectError as exc:
+            # Transport error — recreate client on next attempt
+            logger.warning("Transport error on attempt %d: %s", attempt, exc)
+            await close_client()  # force recreation
+            if attempt < 3:
+                await asyncio.sleep(2 ** attempt)
+                continue
+            raise
+        except Exception as exc:
+            logger.warning("fetch_ads attempt %d failed: %s", attempt, exc)
+            if attempt < 3:
+                await asyncio.sleep(2 ** attempt)
+                continue
+            raise
+    
+    # ... rest unchanged (extract ads_raw, total_hits)
+```
+
+### 3. Call `close_client()` during shutdown in `main.py`
+
+Add to the cleanup function (from Phase 0 task-graceful-shutdown):
+
+```python
+async def cleanup(app: Application) -> None:
+    logger.info("Shutting down...")
+    
+    # ... existing cleanup steps ...
+    
+    # Close HTTP client
+    from scraper import close_client
+    await close_client()
+    
+    # ... rest of cleanup (close DB pool, etc.)
+```
+
+### 4. Update `.env.example` with new config options
+
+```bash
+# HTTP Client Configuration
+HTTP_MAX_CONNECTIONS=10          # Max concurrent connections to willhaben API
+HTTP_KEEPALIVE_CONNECTIONS=5     # Connections kept alive in the pool
+HTTP_TIMEOUT_S=30.0              # Request timeout in seconds
+```
+
+## Acceptance Criteria
+
+- [ ] Only one `httpx.AsyncClient` is created per process lifetime (logged once at startup)
+- [ ] Subsequent calls to `fetch_ads()` reuse the existing client (no "Created httpx client" log)
+- [ ] After calling `close_client()`, a new call to `get_client()` creates a fresh client
+- [ ] Connection keepalive reduces latency for sequential API calls (verifiable via timing in logs)
+- [ ] Fatal transport errors trigger client recreation without crashing the scheduler
+- [ ] The client is properly closed during graceful shutdown (no resource warnings)