# Task: httpx singleton with connection pool

## Description

The current `scraper.fetch_ads()` creates a **new** `httpx.AsyncClient` on every call:

```python
async def fetch_ads(keyword: str):
    async with httpx.AsyncClient(timeout=30.0) as client:
        resp = await client.get(_API_URL, ...)
```

This means each scrape cycle incurs the full cost of TCP handshake + TLS negotiation (≈100-300ms per call on a cold connection). For keywords scraped every 5 minutes with multiple pages, this overhead adds up to **seconds of unnecessary latency per cycle**.

This task replaces the per-call client with a module-level singleton that reuses connections via keepalive.

## Architecture

```
Current:
  Cycle 1: create AsyncClient → fetch → close → ~300ms overhead
  Cycle 2: create AsyncClient → fetch → close → ~300ms overhead
  Cycle N: ... (repeated forever)

Target:
  Module load: create AsyncClient (singleton, keepalive pool)
  Cycle 1: use client → fetch → ~50ms (warm connection)
  Cycle 2: use client → fetch → ~50ms (warm connection)
  Cycle N: ... 
  Shutdown: close client gracefully
```

### Key design decisions

- **Module-level singleton** (`_client = None`, lazy init). Simpler than dependency injection and works with the existing async context.
- **Keepalive connections**: Default `max_keepalive_connections=5` handles concurrent keyword scrapes efficiently.
- **Client recreation on error**: If the client is closed or encounters a fatal transport error, it's recreated on the next call. This prevents stale connection issues.

## Implementation Details

### 1. Add singleton getter to `scraper.py`

```python
import os

_client: httpx.AsyncClient | None = None


async def get_client() -> httpx.AsyncClient:
    """Return a shared AsyncClient with keepalive connection pool."""
    global _client
    
    if _client is None or _client.is_closed:
        max_conns = int(os.getenv("HTTP_MAX_CONNECTIONS", "10"))
        max_keepalive = int(os.getenv("HTTP_KEEPALIVE_CONNECTIONS", "5"))
        
        _client = httpx.AsyncClient(
            timeout=float(os.getenv("HTTP_TIMEOUT_S", "30.0")),
            limits=httpx.Limits(
                max_connections=max_conns,
                max_keepalive_connections=max_keepalive,
                keepalive_expiry=60,  # seconds
            ),
        )
        logger.info(
            "Created httpx client: max_conns=%d, keepalive=%d",
            max_conns, max_keepalive,
        )
    
    return _client


async def close_client() -> None:
    """Close the shared AsyncClient. Call during shutdown."""
    global _client
    if _client and not _client.is_closed:
        await _client.aclose()
        logger.info("Closed httpx client")
        _client = None
```

### 2. Update `fetch_ads()` to use the singleton

**Replace:**
```python
async def fetch_ads(keyword: str):
    params = {...}
    async with httpx.AsyncClient(timeout=30.0) as client:
        for attempt in range(1, 4):
            try:
                resp = await client.get(_API_URL, headers=_HEADERS, params=params)
```

**With:**
```python
async def fetch_ads(keyword: str):
    params = {...}
    client = await get_client()
    
    for attempt in range(1, 4):
        try:
            resp = await client.get(_API_URL, headers=_HEADERS, params=params)
            resp.raise_for_status()
            data = resp.json()
            break
        except httpx.ConnectError as exc:
            # Transport error — recreate client on next attempt
            logger.warning("Transport error on attempt %d: %s", attempt, exc)
            await close_client()  # force recreation
            if attempt < 3:
                await asyncio.sleep(2 ** attempt)
                continue
            raise
        except Exception as exc:
            logger.warning("fetch_ads attempt %d failed: %s", attempt, exc)
            if attempt < 3:
                await asyncio.sleep(2 ** attempt)
                continue
            raise
    
    # ... rest unchanged (extract ads_raw, total_hits)
```

### 3. Call `close_client()` during shutdown in `main.py`

Add to the cleanup function (from Phase 0 task-graceful-shutdown):

```python
async def cleanup(app: Application) -> None:
    logger.info("Shutting down...")
    
    # ... existing cleanup steps ...
    
    # Close HTTP client
    from scraper import close_client
    await close_client()
    
    # ... rest of cleanup (close DB pool, etc.)
```

### 4. Update `.env.example` with new config options

```bash
# HTTP Client Configuration
HTTP_MAX_CONNECTIONS=10          # Max concurrent connections to willhaben API
HTTP_KEEPALIVE_CONNECTIONS=5     # Connections kept alive in the pool
HTTP_TIMEOUT_S=30.0              # Request timeout in seconds
```

## Acceptance Criteria

- [ ] Only one `httpx.AsyncClient` is created per process lifetime (logged once at startup)
- [ ] Subsequent calls to `fetch_ads()` reuse the existing client (no "Created httpx client" log)
- [ ] After calling `close_client()`, a new call to `get_client()` creates a fresh client
- [ ] Connection keepalive reduces latency for sequential API calls (verifiable via timing in logs)
- [ ] Fatal transport errors trigger client recreation without crashing the scheduler
- [ ] The client is properly closed during graceful shutdown (no resource warnings)