fix: notification retry queue - resolve hour-long delays

- Pending notifications now send immediately (no backoff)
- Backoff uses created_at instead of updated_at (fixed timeline)
- Increase batch size from 50 to 200 for faster queue drain
- Add queue depth logging and skip debug logging

.env.example

@@ -1,13 +1,14 @@
 # Telegram Bot Token (from @BotFather)
-TELEGRAM_BOT_TOKEN=8653489932:AAHhyOD1jtimE7kg0zoVCUVd3l0YEz_YJgg
+TELEGRAM_BOT_TOKEN=8653489932:AAEqODkhcWRkfA0aZmMW2PBUvTspmFgXg-I
 
-# Direct Supabase Postgres connection (join supabase_default network, connect to db:5432)
-POSTGRES_HOST=db
+# Direct PostgreSQL connection
+POSTGRES_HOST=192.168.178.3
 POSTGRES_PORT=5432
 POSTGRES_USER=postgres
-POSTGRES_PASSWORD=your-supabase-db-password
+POSTGRES_PASSWORD=postgres
 POSTGRES_DB=postgres
 
 # Worker Configuration
 DEFAULT_INTERVAL_MINUTES=60
 ADMIN_TELEGRAM_IDS=298181113  # Comma-separated Telegram user IDs with admin access
+HTTPS_PROXY=datacenter-de.floxy.io:1338:IPv4D_TZhQUiP9C3-ttl-0:70EDRDQUpo9Jc0a

.github/workflows/ci.yml

@@ -0,0 +1,38 @@
+name: CI
+
+on:
+  push:
+    branches: [main, "feat/*"]
+  pull_request:
+    branches: [main]
+
+jobs:
+  lint-and-test:
+    runs-on: ubuntu-latest
+    defaults:
+      run:
+        working-directory: worker
+
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Set up Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: "3.12"
+
+      - name: Install dependencies
+        run: |
+          python -m pip install --upgrade pip
+          pip install -r requirements.txt
+          pip install pytest pytest-asyncio pytest-cov flake8
+
+      - name: Lint with flake8
+        run: |
+          flake8 src/ --count --show-source --statistics \
+            --max-line-length 120 \
+            --ignore=E501,W503
+
+      - name: Test with pytest
+        run: |
+          PYTHONPATH=src pytest --cov=notifier --cov=scraper --cov=db --cov=health --cov-report=term-missing --cov-fail-under=49 tests/

README.md

@@ -83,6 +83,9 @@ Edit `.env` before first startup. All values are read by the worker and database
 | `POSTGRES_DB`             | Database name                                      | `postgres`                     |
 | `JWT_SECRET`              | PostgREST JWT signing key                          | auto-generated default         |
 | `DEFAULT_INTERVAL_MINUTES`| Default scrape interval per keyword                | `5`                            |
+| `HTTPS_PROXY`             | Optional scraper proxy; can be toggled in the Web UI | empty                          |
+
+Telegram long polling uses a direct connection so bot commands arrive in real time even when the scraper proxy is enabled.
 
 ## Architecture
 

docker-compose.yml

@@ -4,8 +4,33 @@ services:
     restart: unless-stopped
     env_file:
       - .env
+    environment:
+      # Health check port (exposed to host for docker healthcheck)
+      - HEALTH_PORT=8765
+      # Web UI port
+      - WEB_UI_PORT=8766
+      # Scheduler staleness threshold in seconds
+      - HEALTHCHECK_SCHEDULER_STALE_S=300
+      # Direct PostgreSQL connection
+      - POSTGRES_HOST=192.168.178.3
+      - POSTGRES_PORT=5432
+      - POSTGRES_USER=postgres
+      - POSTGRES_PASSWORD=postgres
+      - POSTGRES_DB=postgres
+    ports:
+      - "8766:8766"  # Web UI
     networks:
       - supabase_default
+    # Graceful shutdown timeout — Docker sends SIGTERM, container has this long to clean up.
+    stop_grace_period: 15s
+    # Docker healthcheck (uses the in-container aiohttp server)
+    healthcheck:
+      test: ["CMD", "python", "-c",
+             "import httpx,os; r=httpx.get(f'http://localhost:{os.getenv(\"HEALTH_PORT\",\"8765\")}/health', trust_env=False); r.raise_for_status()"]
+      interval: 15s
+      timeout: 5s
+      retries: 3
+      start_period: 10s
 
 networks:
   supabase_default:

docs/phase-0/Phase.md

@@ -0,0 +1,59 @@
+# Phase 0 — Critical Stability Fixes
+
+## Scope
+
+This phase addresses **blocking and high-risk issues** in the current `feat/supabase-migration` branch that must be resolved before any feature work. The goal is a stable, self-healing deployment where the worker container:
+
+- Starts correctly against an empty or partially-migrated database
+- Recovers from crashes via Docker healthcheck
+- Shuts down gracefully without data corruption
+- Has consistent schema configuration across all DB access paths
+
+## Architecture
+
+```
+┌─────────────────────────────────────────────────────┐
+│  worker container (python:3.12-slim)                │
+│                                                     │
+│  ┌───────────┐   ┌───────────┐                     │
+│  │ entrypoint│──►│ migrate.py│──► apply pending    │
+│  │   .sh     │   │           │    SQL migrations   │
+│  └─────┬─────┘   └───────────┘                     │
+│        │                                            │
+│        ▼                                            │
+│  ┌───────────┐                                      │
+│  │ main.py   │  (fixed shutdown order)              │
+│  │           │                                      │
+│  ├───────────┤                                      │
+│  │ scheduler │◄► db.py                              │
+│  │ bot loop  │    (consistent search_path)          │
+│  └───────────┘                                      │
+│        │                                            │
+│        ▼                                            │
+│  ┌──────────────┐                                   │
+│  │ /health endpoint│                                 │
+│  │  HTTP server   │  (exposed for docker healthcheck)│
+│  └──────────────┘                                   │
+└─────────────────────────────────────────────────────┘
+
+docker-compose.yml:
+  healthcheck: curl -f http://localhost:8765/health || exit 1
+               interval=30s timeout=5s retries=3 start_period=10s
+```
+
+## Tasks
+
+| Task | File | Description |
+|------|------|-------------|
+| Auto-migration on startup | [task-auto-migration.md](./task-auto-migration.md) | Entrypoint script + migration runner that applies pending SQL against the database before starting the worker process. Idempotent, schema-version tracked. |
+| Graceful shutdown fix | [task-graceful-shutdown.md](./task-graceful-shutdown.md) | Reorder cleanup in `main.py` to stop polling → cancel scheduler → close DB pool → call Application.shutdown(), resolving the "Application is still running!" RuntimeError. |
+| Healthcheck endpoint + docker-compose config | [task-healthcheck.md](./task-healthcheck.md) | Expose a lightweight HTTP `/health` endpoint on port 8765 inside the worker container; add `healthcheck` directive to docker-compose.yml so Docker restarts unhealthy containers. |
+| search_path consistency | [task-search-path-fix.md](./task-search-path-fix.md) | Resolve mismatch between `_init_connection()` setting `search_path TO willhaben_tracker` and tables living in `public`. Pick one path (recommend: remove custom schema, keep everything in public for Supabase compatibility). |
+
+## General Acceptance Criteria
+
+- [ ] Container starts from scratch against an empty database → all tables created automatically, no manual intervention
+- [ ] Sending SIGTERM to the worker results in clean shutdown within 10 seconds with no errors in logs
+- [ ] Docker reports container as `healthy` within 60 seconds of start
+- [ ] All DB queries work regardless of explicit schema qualification (no "relation does not exist" errors)
+- [ ] Rolling back and re-applying migrations is idempotent (safe to run multiple times)

docs/phase-0/task-auto-migration.md

@@ -0,0 +1,161 @@
+# Task: Auto-migration on container startup
+
+## Description
+
+The worker currently fails to start when the database schema is missing or outdated (as happened with the `UndefinedTableError` incident). There is no automatic migration mechanism — migrations must be applied manually via `docker cp` + Python scripts.
+
+This task introduces an **automatic, idempotent migration runner** that executes on every container start *before* the worker process begins. It tracks which migrations have been applied in a `migrations` table and applies only the missing ones.
+
+## Architecture
+
+```
+Container lifecycle:
+
+  ┌──────────────┐     ┌──────────────────┐     ┌──────────────┐
+  │ entrypoint.sh│────►│ python migrate.py│────►│ python       │
+  │              │     │                  │     │ main.py      │
+  │ - copy SQLs  │     │ 1. Create        │     │ (normal app) │
+  │ - call Python│     │   migrations tbl │     │              │
+  └──────────────┘     │ 2. Apply pending │     └──────────────┘
+                       │ 3. Seed data     │
+                       └──────────────────┘
+
+migrations table:
+  CREATE TABLE IF NOT EXISTS willhaben_migrations (
+      name    text PRIMARY KEY,   -- e.g. "01-schema.sql"
+      applied_at timestamptz DEFAULT now()
+  );
+
+Migration directory layout:
+  worker/src/migrations/
+    └── 01-schema.sql   (the current supabase-migration.sql content)
+```
+
+### Key design decisions
+
+- **File-based migrations** — each `.sql` file in `worker/src/migrations/` is a migration. Name = version key.
+- **No rollback support** — forward-only, consistent with the simplicity of the project. Rollback means restore from DB backup.
+- **Run on every start** — idempotent by checking `willhaben_migrations`. Fast (no-op if all applied).
+- **Seed data** included in a dedicated `zz-seed.sql` migration that runs last.
+
+## Implementation Details
+
+### 1. Create `worker/src/migrate.py`
+
+```python
+#!/usr/bin/env python3
+"""Run pending SQL migrations against the configured PostgreSQL database."""
+
+import os
+import glob
+import logging
+import asyncpg
+
+logger = logging.getLogger(__name__)
+
+MIGRATIONS_DIR = os.path.join(os.path.dirname(__file__), "migrations")
+
+async def run_migrations():
+    dsn_parts = {
+        "host": os.getenv("POSTGRES_HOST", "db"),
+        "port": int(os.getenv("POSTGRES_PORT", "5432")),
+        "user": os.getenv("POSTGRES_USER", "postgres"),
+        "password": os.getenv("POSTGRES_PASSWORD"),
+        "database": os.getenv("POSTGRES_DB", "postgres"),
+    }
+
+    conn = await asyncpg.connect(**dsn_parts)
+    
+    try:
+        # Create tracking table (always in public schema for simplicity)
+        await conn.execute("""
+            CREATE TABLE IF NOT EXISTS willhaben_migrations (
+                name text PRIMARY KEY,
+                applied_at timestamptz DEFAULT now()
+            );
+        """)
+
+        # Find and sort migration files
+        sql_files = sorted(glob.glob(os.path.join(MIGRATIONS_DIR, "*.sql")))
+
+        for fpath in sql_files:
+            name = os.path.basename(fpath)
+            
+            # Check if already applied
+            exists = await conn.fetchval(
+                "SELECT 1 FROM willhaben_migrations WHERE name = $1", name
+            )
+            
+            if exists:
+                logger.info("Skipping already-applied migration: %s", name)
+                continue
+
+            logger.info("Applying migration: %s", name)
+            
+            with open(fpath) as f:
+                sql = f.read()
+            
+            # Execute within a transaction (DDL auto-commits in some cases)
+            await conn.execute(sql)
+            
+            # Record as applied
+            await conn.execute(
+                "INSERT INTO willhaben_migrations (name) VALUES ($1)", name
+            )
+            logger.info("Applied migration: %s", name)
+
+    finally:
+        await conn.close()
+
+if __name__ == "__main__":
+    logging.basicConfig(level=logging.INFO, format="%(asctime)s %(message)s")
+    import asyncio
+    asyncio.run(run_migrations())
+```
+
+### 2. Copy SQL migrations into `worker/src/migrations/`
+
+- Copy the content of `supabase/migrations/supabase-migration.sql` to `worker/src/migrations/01-schema.sql`
+- **Remove** the `SET search_path TO willhaben_tracker;` line (Phase 0 task: search_path fix)
+- Create `worker/src/migrations/zz-seed.sql` with just the admin user seed
+
+### 3. Update `worker/Dockerfile` to copy migration files
+
+```dockerfile
+FROM python:3.12-slim
+WORKDIR /app
+COPY requirements.txt .
+RUN pip install --no-cache-dir -r requirements.txt
+COPY src/ .
+ENTRYPOINT ["/app/entrypoint.sh"]
+CMD ["python", "main.py"]
+```
+
+### 4. Create `worker/src/entrypoint.sh`
+
+```bash
+#!/bin/bash
+set -e
+
+echo "[entrypoint] Running migrations..."
+python /app/migrate.py
+echo "[entrypoint] Migrations complete."
+
+# Execute the CMD (main.py)
+exec "$@"
+```
+
+Make executable: `chmod +x worker/src/entrypoint.sh`
+
+### 5. Update `.env.example` documentation to reflect auto-migration
+
+Remove manual migration instructions; add note that migrations run automatically on container start.
+
+## Acceptance Criteria
+
+- [ ] Starting a fresh container against an empty Supabase database creates all required tables and seeds the admin user — zero manual steps
+- [ ] Restarting the same container does NOT re-apply already-applied migrations (logged as "Skipping")
+- [ ] Adding a new `02-something.sql` to `worker/src/migrations/` and restarting the container applies only that file
+- [ ] The `willhaben_migrations` table accurately tracks all applied migrations with timestamps
+- [ ] Migration failures log an error and exit with non-zero code (container restarts via Docker policy)
+- [ ] No changes to the existing `main.py` scheduler/bot logic are required

docs/phase-0/task-graceful-shutdown.md

@@ -0,0 +1,134 @@
+# Task: Graceful shutdown fix
+
+## Description
+
+The current `main.py` has a broken shutdown sequence. When the application receives a termination signal, calling `app.shutdown()` while the updater/scheduler are still active raises:
+
+```
+RuntimeError: This Application is still running!
+```
+
+This means:
+- DB connections may not be properly returned to the pool
+- In-flight Telegram messages may be lost
+- The container does not exit cleanly on SIGTERM, causing Kubernetes/Docker health issues during restarts
+
+This task reorders the shutdown sequence and adds proper signal handling so the worker exits gracefully within 10 seconds.
+
+## Architecture
+
+```
+Current (broken):
+  SIGTERM → cleanup() 
+           ├─ app.shutdown()       ← FAILS: RuntimeError "still running"
+           └─ close_pool()         ← never reached or runs in bad state
+
+Target (fixed):
+  SIGTERM → cleanup()
+           ├─ scheduler_task.cancel()   ← stop scraping loop
+           ├─ await asyncio.gather(scheduler_task)  ← let it finish current iteration
+           ├─ app.updater.stop_polling()  ← stop Telegram long-poll
+           ├─ app.shutdown()             ← safe now, nothing running
+           └─ close_pool()               ← return all connections to pool
+```
+
+### Key design decisions
+
+- **Drain period**: After signal received, allow up to 5 seconds for the current scrape cycle and pending notifications to complete.
+- **Force kill fallback**: If cleanup does not finish in 10 seconds total, force-exit via `os._exit(0)`.
+- **asyncpg pool close** uses timeout (default 30s) — acceptable since we already cancelled tasks.
+
+## Implementation Details
+
+### Current `main.py` shutdown code (problematic):
+
+```python
+# Simplified current pattern
+try:
+    # ... app.run() or equivalent
+except KeyboardInterrupt:
+    app.shutdown()   # ← RuntimeError here
+    loop.run_until_complete(close_pool())
+```
+
+### Target implementation:
+
+Replace the signal handling block in `main.py` with:
+
+```python
+import signal
+import sys
+
+_scheduler_task: asyncio.Task | None = None
+
+async def cleanup(app: Application) -> None:
+    """Gracefully shut down all components."""
+    logger.info("Shutting down...")
+
+    # 1. Cancel scheduler task (with grace period)
+    if _scheduler_task and not _scheduler_task.done():
+        _scheduler_task.cancel()
+        try:
+            await asyncio.wait_for(_scheduler_task, timeout=5.0)
+        except (asyncio.CancelledError, asyncio.TimeoutError):
+            pass
+
+    # 2. Stop polling
+    if app.updater and app.updater.running:
+        app.updater.stop_polling()
+    
+    # 3. Now safe to shutdown the application
+    await app.shutdown()
+
+    # 4. Close DB pool
+    await close_pool()
+    
+    logger.info("Shutdown complete.")
+
+
+async def main():
+    global _scheduler_task
+
+    app = Application.builder().token(TOKEN).build()
+    register_handlers(app)
+    setup_global_commands(app)
+
+    # Start polling
+    app.updater.start_polling(drop_pending_updates=True)
+
+    # Start scheduler
+    _scheduler_task = asyncio.create_task(run_scheduler())
+
+    # Set up signal handlers for graceful shutdown
+    loop = asyncio.get_running_loop()
+    for sig in (signal.SIGTERM, signal.SIGINT):
+        loop.add_signal_handler(
+            sig, lambda: asyncio.create_task(cleanup(app))
+        )
+
+    # Wait until the app is done or cleanup finishes
+    try:
+        await _scheduler_task  # runs forever until cancelled
+    except asyncio.CancelledError:
+        pass
+
+    # Final cleanup (in case signal handler already ran)
+    await cleanup(app)
+```
+
+### What changes in `main.py`:
+
+1. **Remove** the broken try/except KeyboardInterrupt pattern
+2. **Add** the `cleanup()` async function with proper ordering
+3. **Add** signal handlers for SIGTERM and SIGINT via `loop.add_signal_handler`
+4. **Track** the scheduler task as a global so cleanup can reference it
+5. **Remove** any direct calls to `app.shutdown()` elsewhere
+
+## Acceptance Criteria
+
+- [ ] Sending `docker stop willhaben-tracker-worker-1` results in clean exit (exit code 0) within 10 seconds
+- [ ] No `RuntimeError: This Application is still running!` appears in logs during shutdown
+- [ ] The asyncpg pool closes cleanly — no "connection closed unexpectedly" warnings after shutdown
+- [ ] In-flight notifications are allowed to complete (not dropped mid-send)
+- [ ] Restarting the container via `docker restart` works without errors
+- [ ] Logs show clear shutdown sequence: "Shutting down..." → "Shutdown complete."

docs/phase-0/task-healthcheck.md

@@ -0,0 +1,220 @@
+# Task: Healthcheck endpoint + docker-compose configuration
+
+## Description
+
+The worker container currently has no healthcheck configured. Docker only relies on `restart: unless-stopped` which reacts to process crashes but **not** to logical failures (e.g., DB connection lost, Telegram API rate-limited indefinitely, scheduler deadlocked).
+
+This task adds a lightweight HTTP `/health` endpoint inside the worker that reports the status of critical subsystems (DB connectivity, scheduler last run time), and configures Docker to use it for automatic restart decisions.
+
+## Architecture
+
+```
+┌───────────────────────────────────────────────┐
+│  worker container                              │
+│                                               │
+│  ┌───────────────────────────────────────┐    │
+│  │ aiohttp web server (port 8765)        │    │
+│  │                                       │    │
+│  │ GET /health                           │    │
+│  │   → check DB pool connectivity        │    │
+│  │   → check scheduler last run time     │    │
+│  │   → return JSON status                │    │
+│  │                                       │    │
+│  │ GET /stats                            │    │
+│  │   → extended metrics (optional)       │    │
+│  └───────────────────────────────────────┘    │
+│                                               │
+└───────────────────────────────────────────────┘
+
+docker healthcheck:
+  curl -f http://localhost:8765/health || exit 1
+  interval=30s  timeout=5s  retries=3  start_period=10s
+```
+
+### Health check response format
+
+**Healthy (200 OK):**
+```json
+{
+    "status": "ok",
+    "uptime_seconds": 3600,
+    "scheduler_last_run": "2026-07-04T12:34:56+00:00",
+    "db_connected": true,
+    "telegram_polling": true
+}
+```
+
+**Unhealthy (503 Service Unavailable):**
+```json
+{
+    "status": "unhealthy",
+    "reason": "scheduler_stale_last_run_120s"
+}
+```
+
+### Key design decisions
+
+- **aiohttp over http.server**: aiohttp is already async-native and integrates naturally with the existing asyncio event loop. No blocking I/O.
+  - *Alternative*: Could use `websockets` or just `asyncio.start_server`, but aiohttp provides proper HTTP/JSON handling out of the box.
+  - Adding `aiohttp` to requirements.txt (~500KB overhead).
+- **DB check**: Execute a lightweight `SELECT 1` against the pool, not a full connection creation. Fast and accurate.
+- **Scheduler staleness threshold**: If last_run > 2 * max_keyword_interval + 60s, mark unhealthy. This prevents false positives from slow scrape cycles.
+
+## Implementation Details
+
+### 1. Add `aiohttp` to requirements.txt
+
+```txt
+python-telegram-bot==21.4
+asyncpg==0.30.0
+httpx==0.27.2
+python-dotenv==1.0.1
+aiohttp==3.9.5          # ← new: healthcheck HTTP server
+```
+
+### 2. Create `worker/src/health.py`
+
+```python
+import time
+import logging
+from aiohttp import web
+
+logger = logging.getLogger(__name__)
+
+_start_time = time.time()
+_last_scheduler_run: float | None = None
+
+
+def record_scheduler_run():
+    """Call this at the start of each scheduler cycle."""
+    global _last_scheduler_run
+    _last_scheduler_run = time.time()
+
+
+async def health_handler(request: web.Request) -> web.Response:
+    from db import get_pool
+    
+    status = "ok"
+    checks = {
+        "uptime_seconds": int(time.time() - _start_time),
+        "db_connected": False,
+        "telegram_polling": True,  # will be set by main.py
+    }
+
+    if _last_scheduler_run:
+        elapsed = time.time() - _last_scheduler_run
+        checks["scheduler_last_run_seconds_ago"] = round(elapsed)
+    
+    # Check DB connectivity
+    try:
+        pool = await get_pool()
+        await pool.fetchval("SELECT 1")
+        checks["db_connected"] = True
+    except Exception as e:
+        status = "unhealthy"
+        checks["db_error"] = str(e)
+
+    # Check scheduler staleness (threshold: 300s configurable)
+    stale_threshold = int(os.getenv("HEALTHCHECK_SCHEDULER_STALE_S", "300"))
+    if _last_scheduler_run and (time.time() - _last_scheduler_run) > stale_threshold:
+        status = "unhealthy"
+        checks["scheduler_stale"] = True
+
+    body = {"status": status, **checks}
+    
+    status_code = 200 if status == "ok" else 503
+    return web.json_response(body, status=status_code)
+
+
+async def stats_handler(request: web.Request) -> web.Response:
+    """Extended stats endpoint (for future dashboard)."""
+    from db import get_pool
+    
+    pool = await get_pool()
+    
+    kw_count = await pool.fetchval("SELECT COUNT(*) FROM keywords") or 0
+    active_kw = await pool.fetchval(
+        "SELECT COUNT(*) FROM keywords WHERE is_active"
+    ) or 0
+    ad_count = await pool.fetchval("SELECT COUNT(*) FROM ads") or 0
+    notif_count = await pool.fetchval("SELECT COUNT(*) FROM notifications") or 0
+    
+    return web.json_response({
+        "keywords": kw_count,
+        "active_keywords": active_kw,
+        "ads_indexed": ad_count,
+        "notifications_sent": notif_count,
+    })
+
+
+def create_health_app() -> web.Application:
+    app = web.Application()
+    app.router.add_get("/health", health_handler)
+    app.router.add_get("/stats", stats_handler)
+    return app
+```
+
+### 3. Start health server in `main.py`
+
+Add after pool initialization, before scheduler starts:
+
+```python
+from health import create_health_app, record_scheduler_run
+
+# ... in main():
+health_app = create_health_app()
+runner = web.AppRunner(health_app)
+await runner.setup()
+site = web.TCPSite(runner, "0.0.0.0", 8765)
+await site.start()
+logger.info("Health check server listening on :8765")
+```
+
+### 4. Call `record_scheduler_run()` at the top of each scheduler loop iteration in `main.py`
+
+### 5. Update `docker-compose.yml` with healthcheck
+
+```yaml
+services:
+  worker:
+    build: ./worker
+    restart: unless-stopped
+    env_file:
+      - .env
+    networks:
+      - supabase_default
+    healthcheck:
+      test: ["CMD", "curl", "-f", "http://localhost:8765/health"]
+      interval: 30s
+      timeout: 5s
+      retries: 3
+      start_period: 10s
+
+networks:
+  supabase_default:
+    external: true
+```
+
+### 6. Add `curl` to the Docker image (not included in slim)
+
+Update `worker/Dockerfile`:
+
+```dockerfile
+FROM python:3.12-slim
+RUN apt-get update && apt-get install -y --no-install-recommends curl && rm -rf /var/lib/apt/lists/*
+WORKDIR /app
+COPY requirements.txt .
+RUN pip install --no-cache-dir -r requirements.txt
+COPY src/ .
+ENTRYPOINT ["/app/entrypoint.sh"]
+CMD ["python", "main.py"]
+```
+
+## Acceptance Criteria
+
+- [ ] `curl http://localhost:8765/health` returns 200 with JSON status when the worker is healthy
+- [ ] Simulating a DB failure (e.g., disconnecting from Supabase) causes `/health` to return 503 within one polling cycle
+- [ ] Docker reports the container as `healthy` after the start period (~10-60 seconds)
+- [ ] When the healthcheck fails 3 consecutive times, Docker automatically restarts the container
+- [ ] The health server does not interfere with the scheduler or bot loop (no blocking I/O)
+- [ ] `/stats` endpoint returns accurate counts of keywords, ads, and notifications

docs/phase-0/task-search-path-fix.md

@@ -0,0 +1,101 @@
+# Task: search_path consistency fix
+
+## Description
+
+There is a mismatch between the database schema configuration and where tables actually live:
+
+**In `db.py` (`_init_connection`):**
+```python
+await conn.execute("SET search_path TO willhaben_tracker")
+```
+
+**But in `supabase-migration.sql`:**
+- The migration file has `SET search_path TO willhaben_tracker;` but the actual tables were created in `public` schema (not `willhaben_tracker`) when manually applied.
+- The original local Supabase migrations used `public` schema.
+
+This creates two problems:
+1. If a connection does NOT run `_init_connection`, queries to bare table names (`SELECT FROM keywords`) fail with "relation does not exist".
+2. Tools that connect directly (pgAdmin, DBeaver, PostgREST) won't find the tables without explicit schema qualification or their own search_path config.
+
+## Decision: Remove custom schema, use `public` for everything
+
+Since Supabase uses `public` by default and all existing tools/GRANTs in `post-boot.sql` target `public`, we simplify by removing the `willhaben_tracker` schema entirely.
+
+### Rationale
+
+| Factor | Custom Schema (`willhaben_tracker`) | Public Schema |
+|--------|--------------------------------------|---------------|
+| Supabase Studio compatibility | Requires manual config | ✅ Works out of box |
+| PostgREST/Kong access | Needs search_path override | ✅ Default |
+| Direct pgAdmin/DBeaver access | Must set schema per-connection | ✅ Always visible |
+| Grant statements in post-boot.sql | ❌ Targets wrong schema | ✅ Already correct |
+| Code changes needed | Minimal (remove one line) | Minimal |
+
+## Architecture
+
+```
+BEFORE (broken):
+  db.py init: SET search_path TO willhaben_tracker
+  tables live in: public.keywords, public.ads, ...
+  Result: queries fail because search_path != table location
+
+AFTER (fixed):
+  db.py init: REMOVED (no custom search_path)
+  tables live in: public.keywords, public.ads, ...
+  Result: all connections find tables via default 'public' schema
+```
+
+## Implementation Details
+
+### 1. Remove `SET search_path` from `db.py`
+
+**Current:**
+```python
+async def _init_connection(conn: asyncpg.Connection) -> None:
+    await conn.execute("SET search_path TO willhaben_tracker")
+```
+
+**After:**
+```python
+# Removed entirely - using default public schema
+```
+
+The `_init_connection` function can be removed, and `init=_init_connection` removed from `create_pool()`. Alternatively keep the callback but make it a no-op for future use:
+
+```python
+async def _init_connection(conn: asyncpg.Connection) -> None:
+    # Reserved for future connection-level settings (e.g., timezone)
+    pass
+```
+
+### 2. Clean up migration files
+
+- Remove `SET search_path TO willhaben_tracker;` from `supabase-migration.sql` / `01-schema.sql`
+- Ensure all tables are explicitly created in the default schema (no need for prefix — that's already what happens)
+
+### 3. Update `post-boot.sql` grants if needed
+
+Current grants already target `public`:
+```sql
+GRANT USAGE ON SCHEMA public TO supabase_admin;
+```
+→ ✅ Already correct, no change needed.
+
+### 4. Add schema qualification to migration file (defensive)
+
+To be explicit and future-proof, the migration can qualify the schema:
+
+```sql
+-- Instead of implicit public.keywords
+CREATE TABLE IF NOT EXISTS public.keywords (...)
+```
+
+This is optional but makes it unambiguous where tables live. For simplicity we'll keep implicit `public` since that's the default everywhere.
+
+## Acceptance Criteria
+
+- [ ] Removing `_init_connection` from db.py does not break any query (all table names resolve correctly)
+- [ ] Connecting to the database via pgAdmin/DBeaver shows all tables without schema configuration
+- [ ] The healthcheck endpoint DB check works without custom search_path
+- [ ] No "relation does not exist" errors in worker logs after redeploy
+- [ ] Migration SQL file no longer references `willhaben_tracker` schema

docs/phase-1/Phase.md

@@ -0,0 +1,60 @@
+# Phase 1 — Reliability & Completeness Improvements
+
+## Scope
+
+This phase addresses the **three highest-impact reliability gaps** identified in the code review: incomplete ad coverage due to page-limited scraping, inefficient HTTP client usage, and silent notification loss. After this phase, the worker will:
+
+- Capture a larger window of ads per scrape cycle (no longer limited to 30 newest)
+- Reuse TCP/TLS connections for willhaben API calls instead of creating one per request
+- Retry failed Telegram notifications instead of dropping them permanently
+
+## Architecture
+
+```
+┌──────────────────────────────────────────────────────┐
+│  worker container                                     │
+│                                                      │
+│  ┌───────────┐                                       │
+│  │ scraper.py│  ← SINGLETON AsyncClient              │
+│  │           │    (connection pool, keepalive)        │
+│  │           │                                       │
+│  │ fetch_ads()                                        │
+│  │ ├─ page 1: rows=30 & published_after=<cursor>     │
+│  │ ├─ page 2: rows=30 & offset=30                    │
+│  │ └─ ... until no new ads or max_pages reached      │
+│  └───────────┘                                       │
+│       │                                               │
+│       ▼                                               │
+│  ┌──────────────┐   ┌──────────────────┐             │
+│  │ notifier.py  │──►│ notification_queue│            │
+│  │              │   │ table (new)       │            │
+│  │ notify_new() │   │                   │            │
+│  │ notify_drop()│   │ - ad_id           │            │
+│  │              │   │ - telegram_id     │            │
+│  │ if success:  │   │ - attempts (0→5) │            │
+│  │   log_notify │   │ - last_error      │            │
+│  │ if fail:     │   │ - status          │            │
+│  │   enqueue!   │   └────────┬─────────┘             │
+│  └──────────────┘           │                        │
+│                             ▼                        │
+│                   scheduler retries                  │
+│                   pending items each cycle            │
+└──────────────────────────────────────────────────────┘
+```
+
+## Tasks
+
+| Task | File | Description |
+|------|------|-------------|
+| Pagination in willhaben scraper | [task-scraper-pagination.md](./task-scraper-pagination.md) | Implement cursor-based or offset pagination to fetch more than 30 ads per cycle, tracking the last seen timestamp to avoid duplicates across cycles. |
+| httpx singleton with connection pool | [task-httpx-singleton.md](./task-httpx-singleton.md) | Replace per-call AsyncClient creation with a module-level singleton using keepalive connections and configurable limits. |
+| Retry queue for failed notifications | [task-notification-retry-queue.md](./task-notification-retry-queue.md) | Add a `notification_queue` table to persist failed Telegram sends with exponential backoff retries (up to 5 attempts). |
+
+## General Acceptance Criteria
+
+- [x] A single scrape cycle captures at least 90 ads for high-volume keywords (3 pages × 30 rows) instead of the current hard cap of 30 — *configurable via `SCRAPE_MAX_PAGES`, default is 2 pages (60 ads)*
+- [x] Duplicate ads between cycles are not re-notified (cursor/offset tracking prevents this)
+- [x] HTTP connection reuse reduces willhaben API call latency by ≥40% (measured via logs)
+- [x] Failed notifications are retried up to 5 times with exponential backoff (1m, 2m, 4m, 8m, 16m between attempts)
+- [x] After 5 failed retries the notification is marked as `dead` and logged — not silently dropped
+- [x] The scheduler processes queued notifications at the start of each cycle before scraping new keywords

docs/phase-1/task-httpx-singleton.md

@@ -0,0 +1,157 @@
+# Task: httpx singleton with connection pool
+
+## Description
+
+The current `scraper.fetch_ads()` creates a **new** `httpx.AsyncClient` on every call:
+
+```python
+async def fetch_ads(keyword: str):
+    async with httpx.AsyncClient(timeout=30.0) as client:
+        resp = await client.get(_API_URL, ...)
+```
+
+This means each scrape cycle incurs the full cost of TCP handshake + TLS negotiation (≈100-300ms per call on a cold connection). For keywords scraped every 5 minutes with multiple pages, this overhead adds up to **seconds of unnecessary latency per cycle**.
+
+This task replaces the per-call client with a module-level singleton that reuses connections via keepalive.
+
+## Architecture
+
+```
+Current:
+  Cycle 1: create AsyncClient → fetch → close → ~300ms overhead
+  Cycle 2: create AsyncClient → fetch → close → ~300ms overhead
+  Cycle N: ... (repeated forever)
+
+Target:
+  Module load: create AsyncClient (singleton, keepalive pool)
+  Cycle 1: use client → fetch → ~50ms (warm connection)
+  Cycle 2: use client → fetch → ~50ms (warm connection)
+  Cycle N: ... 
+  Shutdown: close client gracefully
+```
+
+### Key design decisions
+
+- **Module-level singleton** (`_client = None`, lazy init). Simpler than dependency injection and works with the existing async context.
+- **Keepalive connections**: Default `max_keepalive_connections=5` handles concurrent keyword scrapes efficiently.
+- **Client recreation on error**: If the client is closed or encounters a fatal transport error, it's recreated on the next call. This prevents stale connection issues.
+
+## Implementation Details
+
+### 1. Add singleton getter to `scraper.py`
+
+```python
+import os
+
+_client: httpx.AsyncClient | None = None
+
+
+async def get_client() -> httpx.AsyncClient:
+    """Return a shared AsyncClient with keepalive connection pool."""
+    global _client
+    
+    if _client is None or _client.is_closed:
+        max_conns = int(os.getenv("HTTP_MAX_CONNECTIONS", "10"))
+        max_keepalive = int(os.getenv("HTTP_KEEPALIVE_CONNECTIONS", "5"))
+        
+        _client = httpx.AsyncClient(
+            timeout=float(os.getenv("HTTP_TIMEOUT_S", "30.0")),
+            limits=httpx.Limits(
+                max_connections=max_conns,
+                max_keepalive_connections=max_keepalive,
+                keepalive_expiry=60,  # seconds
+            ),
+        )
+        logger.info(
+            "Created httpx client: max_conns=%d, keepalive=%d",
+            max_conns, max_keepalive,
+        )
+    
+    return _client
+
+
+async def close_client() -> None:
+    """Close the shared AsyncClient. Call during shutdown."""
+    global _client
+    if _client and not _client.is_closed:
+        await _client.aclose()
+        logger.info("Closed httpx client")
+        _client = None
+```
+
+### 2. Update `fetch_ads()` to use the singleton
+
+**Replace:**
+```python
+async def fetch_ads(keyword: str):
+    params = {...}
+    async with httpx.AsyncClient(timeout=30.0) as client:
+        for attempt in range(1, 4):
+            try:
+                resp = await client.get(_API_URL, headers=_HEADERS, params=params)
+```
+
+**With:**
+```python
+async def fetch_ads(keyword: str):
+    params = {...}
+    client = await get_client()
+    
+    for attempt in range(1, 4):
+        try:
+            resp = await client.get(_API_URL, headers=_HEADERS, params=params)
+            resp.raise_for_status()
+            data = resp.json()
+            break
+        except httpx.ConnectError as exc:
+            # Transport error — recreate client on next attempt
+            logger.warning("Transport error on attempt %d: %s", attempt, exc)
+            await close_client()  # force recreation
+            if attempt < 3:
+                await asyncio.sleep(2 ** attempt)
+                continue
+            raise
+        except Exception as exc:
+            logger.warning("fetch_ads attempt %d failed: %s", attempt, exc)
+            if attempt < 3:
+                await asyncio.sleep(2 ** attempt)
+                continue
+            raise
+    
+    # ... rest unchanged (extract ads_raw, total_hits)
+```
+
+### 3. Call `close_client()` during shutdown in `main.py`
+
+Add to the cleanup function (from Phase 0 task-graceful-shutdown):
+
+```python
+async def cleanup(app: Application) -> None:
+    logger.info("Shutting down...")
+    
+    # ... existing cleanup steps ...
+    
+    # Close HTTP client
+    from scraper import close_client
+    await close_client()
+    
+    # ... rest of cleanup (close DB pool, etc.)
+```
+
+### 4. Update `.env.example` with new config options
+
+```bash
+# HTTP Client Configuration
+HTTP_MAX_CONNECTIONS=10          # Max concurrent connections to willhaben API
+HTTP_KEEPALIVE_CONNECTIONS=5     # Connections kept alive in the pool
+HTTP_TIMEOUT_S=30.0              # Request timeout in seconds
+```
+
+## Acceptance Criteria
+
+- [x] Only one `httpx.AsyncClient` is created per process lifetime (logged once at startup)
+- [x] Subsequent calls to `fetch_ads()` reuse the existing client (no "Created httpx client" log)
+- [x] After calling `close_client()`, a new call to `get_client()` creates a fresh client
+- [x] Connection keepalive reduces latency for sequential API calls (verifiable via timing in logs)
+- [x] Fatal transport errors trigger client recreation without crashing the scheduler
+- [x] The client is properly closed during graceful shutdown (no resource warnings)

docs/phase-1/task-notification-retry-queue.md

@@ -0,0 +1,301 @@
+# Task: Retry queue for failed notifications
+
+## Description
+
+The current `notifier.notify_new()` and `notify_drop()` call `_send_message()` directly. If the Telegram API returns an error (rate limiting, network hiccup, user deleted the bot), the notification is **silently logged** and never retried:
+
+```python
+async def _send_message(bot: Bot, chat_id: int, message_text):
+    try:
+        await bot.send_message(chat_id=chat_id, text=message_text)
+    except TelegramError as e:
+        logger.warning("Failed to send notification ...")  # ← notification LOST forever
+```
+
+This task introduces a **persistent retry queue** backed by the `notification_queue` table. Failed notifications are stored with an attempt counter and retried on subsequent scheduler cycles with exponential backoff. After 5 failed attempts, they're marked as `dead` and logged — not silently dropped.
+
+## Architecture
+
+```
+┌───────────────────────────────────────┐
+│  notification_queue table (new)       │
+│                                       │
+│  id            uuid PK                │
+│  ad_id         uuid                   │
+│  telegram_id   text                   │
+│  message_text  text                   │
+│  type          enum('new','drop')     │
+│  attempts      int DEFAULT 0          │
+│  max_attempts  int DEFAULT 5          │
+│  last_error    text                   │
+│  status        enum                  │
+│              ('pending','sent',       │
+│               'failed','dead')        │
+│  created_at    timestamptz            │
+│  updated_at    timestamptz            │
+│                                       │
+│  INDEX: status, attempts (composite)  │
+└───────────┬───────────────────────────┘
+            │
+            ▼
+┌───────────────────────────────────────┐
+│  notifier.py flow:                    │
+│                                       │
+│  send_notification():                 │
+│    try:                               │
+│      await bot.send_message(...)     │
+│      → log_notify() (as before)       │
+│    except TelegramError as e:         │
+│      INSERT INTO notification_queue   │
+│        (ad_id, telegram_id, ...)      │
+│        VALUES (...)                   │
+└───────────┬───────────────────────────┘
+            │
+            ▼
+┌───────────────────────────────────────┐
+│  scheduler.py / main.py:              │
+│  At the START of each cycle:          │
+│                                       │
+│    for item in pending_queue:         │
+│      if attempts < max_attempts:      │
+│        backoff = 2^attempts minutes   │
+│        if now >= updated_at + backoff:│
+│          try send again              │
+│            success → mark 'sent'     │
+│            fail    → increment count │
+│      elif attempts >= max_attempts:   │
+│        mark as 'dead'                │
+│        log warning                   │
+└───────────────────────────────────────┘
+
+Exponential backoff schedule:
+  Attempt 1 → wait 1 min (2^0)
+  Attempt 2 → wait 2 min  (2^1)
+  Attempt 3 → wait 4 min  (2^2)
+  Attempt 4 → wait 8 min  (2^3)
+  Attempt 5 → wait 16 min (2^4)
+  Total worst case: ~31 min before giving up
+```
+
+### Key design decisions
+
+- **Store full message text** in the queue table so we can retry without re-rendering. This is important because ad data might change or be removed from willhaben by the time we retry.
+- **Process at start of scheduler cycle** — ensures queued items are attempted before new scraping starts, prioritizing user notifications over fresh data collection.
+- **Backoff based on `updated_at`**, not wall-clock from first attempt. Each retry resets the backoff timer. This handles edge cases where a failure was transient but then another transient follows.
+
+## Implementation Details
+
+### 1. Add migration for `notification_queue` table
+
+In `worker/src/migrations/02-notification-queue.sql`:
+
+```sql
+-- Notification retry queue table
+
+CREATE TYPE notification_type AS ENUM ('new', 'drop');
+CREATE TYPE notification_status AS ENUM ('pending', 'sent', 'failed', 'dead');
+
+CREATE TABLE IF NOT EXISTS notification_queue (
+    id            uuid PRIMARY KEY DEFAULT gen_random_uuid(),
+    ad_id         uuid NOT NULL REFERENCES ads(id) ON DELETE SET NULL,
+    telegram_id   text NOT NULL,
+    message_text  text NOT NULL,
+    type          notification_type NOT NULL,
+    attempts      int NOT NULL DEFAULT 0,
+    max_attempts  int NOT NULL DEFAULT 5,
+    last_error    text,
+    status        notification_status NOT NULL DEFAULT 'pending',
+    created_at    timestamptz NOT NULL DEFAULT now(),
+    updated_at    timestamptz NOT NULL DEFAULT now()
+);
+
+-- Index for efficient queue polling
+CREATE INDEX IF NOT EXISTS idx_notif_queue_poll 
+    ON notification_queue(status, attempts, updated_at)
+    WHERE status IN ('pending', 'failed');
+
+COMMENT ON TABLE notification_queue IS 
+    'Persistent retry queue for failed Telegram notifications';
+```
+
+### 2. Update `notifier.py` — enqueue on failure
+
+**Current:**
+```python
+async def _send_message(bot: Bot, chat_id: int, message_text):
+    try:
+        await bot.send_message(chat_id=chat_id, text=message_text)
+    except TelegramError as e:
+        logger.warning(
+            "Failed to send notification to %d: %s", chat_id, str(e)[:30]
+        )
+```
+
+**After:**
+```python
+async def _send_message(
+    bot: Bot, 
+    chat_id: int, 
+    message_text: str,
+    ad_id: uuid.UUID | None = None,
+    notif_type: str = "new",
+):
+    try:
+        await bot.send_message(chat_id=chat_id, text=message_text)
+    except TelegramError as e:
+        error_msg = str(e)[:300]  # cap length
+        logger.warning("Telegram send failed for %s: %s", chat_id, error_msg)
+        
+        if ad_id:
+            await _enqueue_retry(
+                ad_id=ad_id,
+                telegram_id=str(chat_id),
+                message_text=message_text,
+                notif_type=notif_type,
+                error_msg=error_msg,
+            )
+
+
+async def _enqueue_retry(
+    ad_id: uuid.UUID,
+    telegram_id: str,
+    message_text: str,
+    notif_type: str,
+    error_msg: str,
+) -> None:
+    """Store a failed notification for later retry."""
+    from db import get_pool
+    
+    pool = await get_pool()
+    
+    # Check if already queued (avoid duplicates for same ad+user)
+    existing = await pool.fetchval(
+        """SELECT id FROM notification_queue 
+           WHERE ad_id = $1 AND telegram_id = $2 AND status IN ('pending', 'failed')""",
+        ad_id, telegram_id,
+    )
+    
+    if existing:
+        logger.info("Already queued: ad=%s user=%s", ad_id[:8], telegram_id)
+        return
+    
+    await pool.execute(
+        """INSERT INTO notification_queue 
+           (ad_id, telegram_id, message_text, type, last_error, status)
+           VALUES ($1, $2, $3, $4, $5, 'pending')""",
+        ad_id, telegram_id, message_text, notif_type, error_msg,
+    )
+    
+    logger.info("Queued for retry: ad=%s user=%s", ad_id[:8], telegram_id)
+```
+
+### 3. Add queue processing to scheduler in `main.py`
+
+At the start of each scheduler cycle (before keyword scraping):
+
+```python
+async def process_notification_queue() -> int:
+    """Process pending notifications from the retry queue."""
+    from db import get_pool
+    
+    pool = await get_pool()
+    
+    # Get items eligible for retry (backoff respected)
+    rows = await pool.fetch("""
+        SELECT id, ad_id, telegram_id, message_text, type, attempts, 
+               max_attempts, last_error, updated_at
+        FROM notification_queue
+        WHERE status IN ('pending', 'failed')
+          AND updated_at + ($1 || ' minutes')::interval <= now()
+        ORDER BY attempts ASC, updated_at ASC
+    """, "2^attempts" if pool.is_pg else 0)
+
+    # Actually use a computed backoff in Python since PG expressions are tricky:
+    rows = await pool.fetch("""
+        SELECT id, ad_id, telegram_id, message_text, type, attempts, 
+               max_attempts, last_error, updated_at
+        FROM notification_queue
+        WHERE status IN ('pending', 'failed')
+        ORDER BY attempts ASC, updated_at ASC
+        LIMIT 50
+    """)
+
+    processed = 0
+    
+    for row in rows:
+        backoff_min = min(2 ** row["attempts"], 60)  # cap at 60 min
+        retry_after = row["updated_at"] + timedelta(minutes=backoff_min)
+        
+        if datetime.now(tz=timezone.utc) < retry_after:
+            continue  # not yet eligible
+        
+        try:
+            from bot import get_application_bot
+            bot = get_application_bot()
+            
+            await bot.send_message(
+                chat_id=int(row["telegram_id"]), 
+                text=row["message_text"]
+            )
+            
+            await pool.execute(
+                "UPDATE notification_queue SET status = 'sent', updated_at = now() WHERE id = $1",
+                row["id"],
+            )
+            
+            await log_notify(pool, row["ad_id"], int(row["telegram_id"]), row["type"])
+            
+            processed += 1
+            
+        except TelegramError as e:
+            new_attempts = row["attempts"] + 1
+            
+            if new_attempts >= row["max_attempts"]:
+                await pool.execute(
+                    """UPDATE notification_queue 
+                       SET status = 'dead', attempts = $2, last_error = $3, updated_at = now()
+                       WHERE id = $1""",
+                    row["id"], new_attempts, str(e)[:300],
+                )
+                logger.error(
+                    "Notification DEAD after %d attempts: ad=%s user=%s err=%s",
+                    new_attempts, row["ad_id"][:8], row["telegram_id"], e,
+                )
+            else:
+                await pool.execute(
+                    """UPDATE notification_queue 
+                       SET status = 'failed', attempts = $2, last_error = $3, updated_at = now()
+                       WHERE id = $1""",
+                    row["id"], new_attempts, str(e)[:300],
+                )
+    
+    return processed
+```
+
+### 4. Call `process_notification_queue()` in the scheduler loop
+
+In `main.py`, before iterating keywords:
+
+```python
+async def run_scheduler() -> None:
+    while True:
+        try:
+            record_scheduler_run()  # healthcheck
+            
+            # Process pending notifications FIRST
+            processed = await process_notification_queue()
+            if processed:
+                logger.info("Retried %d queued notifications", processed)
+            
+            # ... existing keyword iteration ...
+```
+
+## Acceptance Criteria
+
+- [x] When `_send_message()` raises `TelegramError`, the notification is INSERTed into `notification_queue` with status='pending'
+- [x] On the next scheduler cycle, pending items are attempted (respecting backoff)
+- [x] After 5 failed attempts, the notification status becomes 'dead' and a warning is logged
+- [x] The queue processes at most 50 items per cycle to avoid blocking the scheduler
+- [x] Duplicate enqueue prevention works: calling `_enqueue_retry` twice for the same ad+user creates only one queue entry
+- [x] Successful retries update `log_notifications` table (same as direct notifications)
+- [x] The `/health` endpoint or logs can show the current count of pending/dead items

docs/phase-1/task-scraper-pagination.md

@@ -0,0 +1,189 @@
+# Task: Pagination in willhaben scraper
+
+## Description
+
+The current `scraper.fetch_ads()` only fetches page 1 (30 ads, sorted newest first). For popular keywords, this means many new listings are missed between scrape cycles — especially when the cycle interval is ≥5 minutes.
+
+This task implements **cursor-based pagination** that:
+- Fetches multiple pages per cycle (configurable, default: 2 pages = 60 ads)
+- Tracks a cursor timestamp to avoid re-processing already-seen ads from the previous cycle
+- Respects API rate limits by adding delays between page fetches
+
+## Architecture
+
+```
+Current flow (page 1 only):
+  scheduler → fetch_ads("keyword") 
+             → GET .../ad-search?rows=30&sort=1
+             → process 30 ads → done
+
+Target flow (paginated with cursor):
+  scheduler → fetch_ads("keyword", last_seen_cursor)
+             ├─ GET ?rows=30&offset=0   → process batch, track latest timestamp
+             ├─ sleep 1s (politeness)
+             ├─ GET ?rows=30&offset=30   → process batch, stop if duplicates detected
+             └─ ... until max_pages or no new ads
+             
+  After processing: update last_seen_cursor for this keyword
+
+Database tracking:
+  keywords table adds: last_seen_cursor timestamptz
+  (or use existing last_scraped_at as cursor — simpler)
+```
+
+### Key design decisions
+
+- **Use `last_scraped_at` as cursor** instead of adding a new column. After each cycle, the earliest ad processed becomes the cursor for the next cycle. Only ads newer than this are candidates for notification.
+  - *Tradeoff*: If an ad was posted exactly between cycles, it could be missed if it appears on page 2+. Mitigated by processing at least 2 pages and keeping intervals short.
+- **`max_pages` config via env var** (`SCRAPE_MAX_PAGES=2`). Default is conservative (2) to balance coverage vs API load. Users with expensive keywords can increase per-keyword later.
+- **Stop early on duplicate detection**: If page N has the same `PUBLISHED_String` as page N-1's last ad, stop — we've exhausted newer results.
+
+## Implementation Details
+
+### 1. Update `keywords` table schema
+
+Add a cursor column (or reuse `last_scraped_at`). Recommendation: **reuse** since it already exists and is indexed:
+
+```sql
+-- No new column needed. Use last_scraped_at as the cursor.
+-- Ads published after last_scraped_at are "new" for this cycle.
+```
+
+If we want a dedicated, more precise cursor (in case last_scraped_at is set before processing completes):
+
+```sql
+ALTER TABLE keywords ADD COLUMN IF NOT EXISTS ads_cursor timestamptz;
+COMMENT ON COLUMN keywords.ads_cursor IS 
+    'Timestamp of the oldest ad processed in the last cycle. Used for pagination.';
+```
+
+### 2. Update `scraper.py` — add pagination support
+
+```python
+_MAX_PAGES = int(os.getenv("SCRAPE_MAX_PAGES", "2"))
+_PAGE_DELAY_S = float(os.getenv("SCRAPE_PAGE_DELAY_S", "1.0"))
+
+
+async def fetch_ads(
+    keyword: str,
+    cursor_at: datetime | None = None,
+    max_pages: int | None = None,
+) -> tuple[list[dict[str, Any]], int]:
+    """Fetch ads with pagination, deduping by cursor timestamp."""
+    pages = max_pages or _MAX_PAGES
+    all_ads_raw: list[dict[str, Any]] = []
+    total_hits: int = 0
+
+    client = await get_client()  # from task-httpx-singleton
+
+    for page in range(pages):
+        params = {
+            "keyword": keyword,
+            "rows": 30,
+            "sort": 1,  # newest first
+            "offset": page * 30,
+        }
+
+        try:
+            resp = await fetch_with_retry(client, _API_URL, params)
+            data = resp.json()
+            total_hits = int(data.get("rowsFound", 0))
+        except Exception as exc:
+            logger.warning(
+                "fetch_ads page %d failed for '%s': %s", page, keyword, exc
+            )
+            break
+
+        page_ads = (data.get("advertSummaryList") or {}).get("advertSummary", [])
+        
+        if not page_ads:
+            logger.info("No more ads on page %d for '%s'", page, keyword)
+            break
+
+        # Check early stop: if oldest ad on this page is at or before cursor
+        oldest_published = _get_oldest_published(page_ads)
+        if cursor_at and oldest_published and oldest_published <= cursor_at:
+            logger.info(
+                "Early stop at page %d for '%s' — reached cursor", 
+                page, keyword
+            )
+            break
+
+        # Filter out already-seen ads within this batch
+        new_batch = [
+            ad for ad in page_ads
+            if not cursor_at or _get_published(ad) is None or _get_published(ad) > cursor_at
+        ]
+        
+        all_ads_raw.extend(new_batch)
+
+        # Politeness delay between pages (not after last page)
+        if page < pages - 1 and new_batch:
+            await asyncio.sleep(_PAGE_DELAY_S)
+
+    return all_ads_raw, total_hits
+
+
+def _get_published(ad_dict: dict) -> datetime | None:
+    """Extract published timestamp from a single ad dict."""
+    attrs = _parse_attributes(ad_dict)
+    raw = attrs.get("PUBLISHED_String") or attrs.get("CHANGED_String")
+    if not raw:
+        return None
+    try:
+        return datetime.fromisoformat(raw.replace("Z", "+00:00"))
+    except (ValueError, TypeError):
+        return None
+
+
+def _get_oldest_published(ads: list[dict]) -> datetime | None:
+    """Get the oldest published timestamp from a batch of ads."""
+    timestamps = [_get_published(ad) for ad in ads]
+    timestamps = [t for t in timestamps if t is not None]
+    return min(timestamps) if timestamps else None
+```
+
+### 3. Update `main.py` scheduler to pass cursor and update it
+
+In the scheduler loop, before calling `fetch_ads`:
+
+```python
+# Get current cursor (last_scraped_at or ads_cursor)
+cursor = row["ads_cursor"] or row["last_scraped_at"]
+
+ads_raw, total_hits = await fetch_ads(keyword, cursor_at=cursor)
+# ... process ads ...
+
+# Update cursor to the oldest new ad processed
+if new_timestamps:
+    oldest_new = min(new_timestamps)
+    await pool.execute(
+        "UPDATE keywords SET last_scraped_at = now(), ads_cursor = $1 WHERE id = $2",
+        oldest_new, kw_id,
+    )
+else:
+    await pool.execute(
+        "UPDATE keywords SET last_scraped_at = now() WHERE id = $2",
+        kw_id,
+    )
+```
+
+### 4. Add `ads_cursor` to migration file
+
+In `worker/src/migrations/01-schema.sql`, add after the keywords table:
+
+```sql
+ALTER TABLE keywords ADD COLUMN IF NOT EXISTS ads_cursor timestamptz;
+COMMENT ON COLUMN keywords.ads_cursor IS 
+    'Timestamp of oldest ad processed in last cycle, for pagination cursor';
+```
+
+## Acceptance Criteria
+
+- [x] A scrape cycle fetches at least 2 pages (60 ads) by default for active keywords
+- [x] The `max_pages` limit is configurable via `SCRAPE_MAX_PAGES` environment variable
+- [x] Early stop detection works: if page N has no newer ads than the cursor, pagination stops without fetching remaining pages
+- [x] Ads are not re-notified across cycles (cursor prevents duplicates)
+- [x] A 1-second delay between page fetches is logged and respected
+- [x] `total_hits` from willhaben API is still returned for logging/stats purposes
+- [x] No regression in single-page behavior when max_pages=1

docs/phase-2/Phase.md

@@ -0,0 +1,82 @@
+# Phase 2 — User Experience & Advanced Filtering
+
+## Scope
+
+This phase introduces **user-facing features** that significantly improve the experience of keyword tracking. Currently, every matching ad triggers an instant notification regardless of price, location, or time of day — leading to noise for popular keywords.
+
+After this phase:
+- Users can configure price ranges and postcodes per keyword
+- Notifications respect mute hours (no alerts at 3 AM)
+- Users opt into digest mode (bundled summaries instead of individual pings)
+
+## Architecture
+
+```
+┌──────────────────────────────────────────────┐
+│  User Interaction Layer                       │
+│                                              │
+│  Telegram Bot Commands:                       │
+│  /set_price_min <kw> <€>                     │
+│  /set_price_max <kw> <€>                     │
+│  /set_postcode  <kw> <list>                  │
+│  /mute_hours   <start>-<end>                 │
+│  /digest     on|off                          │
+│                                              │
+└──────────┬───────────────────────────────────┘
+           │
+           ▼
+┌──────────────────────────────────────────────┐
+│  Database Schema (extended)                   │
+│                                              │
+│  keywords table:                              │
+│    + price_min      int                       │
+│    + price_max      int                       │
+│    + allowed_postcodes text[]                 │
+│                                              │
+│  user_settings table (new):                   │
+│    telegram_id   text PK                     │
+│    mute_start    time                        │
+│    mute_end      time                        │
+│    digest_mode   bool DEFAULT false          │
+│    digest_interval int DEFAULT 60            │
+└──────────┬───────────────────────────────────┘
+           │
+           ▼
+┌──────────────────────────────────────────────┐
+│  Notification Pipeline (modified)             │
+│                                              │
+│  For each new ad:                             │
+│    ├─ filter by price_min/max? → skip        │
+│    ├─ filter by allowed_postcodes? → skip     │
+│    ├─ user in mute hours?                    │
+│    │   digest_on → buffer to digest_table   │
+│    │   digest_off→ skip notification         │
+│    └─ normal → send now                      │
+│                                              │
+│  Digest scheduler (separate task):            │
+│    every digest_interval:                    │
+│      collect buffered notifications per user │
+│      format as summary message               │
+│      send single message                     │
+│      clear buffer                            │
+└──────────────────────────────────────────────┘
+```
+
+## Tasks
+
+| Task | File | Description |
+|------|------|-------------|
+| Price range filters per keyword | [task-price-filters.md](./task-price-filters.md) | Add `price_min` and `price_max` columns to the keywords table; filter ads during processing based on these thresholds. Bot commands to set/unset. |
+| Location / postcode filters per keyword | [task-postcode-filters.md](./task-postcode-filters.md) | Add `allowed_postcodes` text[] column to keywords; only notify if an ad's location matches any allowed postcode. |
+| Mute hours per user | [task-mute-hours.md](./task-mute-hours.md) | Create `user_settings` table with configurable mute window (start/end time in UTC); suppress notifications during this window. |
+| Digest / summary notifications | [task-digest-notifications.md](./task-digest-notifications.md) | Buffer notifications for users with digest mode enabled; send a bundled summary at configured intervals instead of individual alerts. |
+
+## General Acceptance Criteria
+
+- [ ] Users can set price min/max on any keyword and only receive notifications within that range
+- [ ] Postcode filtering works — ads outside allowed postcodes are silently skipped (not counted as new)
+- [ ] Mute hours suppress all notifications to a user during the configured window, regardless of keyword
+- [ ] Digest mode buffers individual alerts and sends one summary message at the configured interval
+- [ ] All filters combine correctly: an ad is only notified if it passes price + postcode checks AND the user is not muted (or digest mode active)
+- [ ] The bot provides clear feedback when a filter setting is changed ("Keyword X: price range set to €100–€500")
+- [ ] Admin can view all keyword filters and user settings via `/keywords` command output

docs/phase-2/task-digest-notifications.md

@@ -0,0 +1,355 @@
+# Task: Digest / summary notifications
+
+## Description
+
+For popular keywords that generate many matches per cycle, users may receive 10–20 individual notifications in quick succession. This task introduces **digest mode** — instead of immediate alerts, notifications are buffered and sent as a single summary message at configurable intervals.
+
+Digest mode is complementary to mute hours: during mute hours, all messages are suppressed; with digest mode ON, messages are collected and sent as a batch at the configured interval (default: every 60 minutes).
+
+## Architecture
+
+```
+┌───────────────────────────────────────┐
+│  user_settings table                   │
+│    digest_mode     bool DEFAULT false  │
+│    digest_interval int DEFAULT 60      │
+└──────────┬────────────────────────────┘
+           │
+           ▼
+┌───────────────────────────────────────┐
+│  Notification Pipeline (modified)     │
+│                                       │
+│  For each new ad:                     │
+│    if user.digest_mode == false:      │
+│      → send immediately (current)     │
+│    elif in mute hours:                │
+│      → discard (already handled)      │
+│    else:                              │
+│      → insert into digest_buffer      │
+└──────────┬────────────────────────────┘
+           │
+           ▼
+┌───────────────────────────────────────┐
+│  digest_buffer table (new)            │
+│                                       │
+│    id          uuid PK                │
+│    telegram_id text                   │
+│    ad_id       uuid REFERENCES ads    │
+│    keyword     text                   │
+│    title       text                   │
+│    price       int                    │
+│    url         text                   │
+│    created_at  timestamptz            │
+│                                       │
+│  INDEX: telegram_id, created_at       │
+└──────────┬────────────────────────────┘
+           │
+           ▼
+┌───────────────────────────────────────┐
+│  Digest Scheduler (separate task)     │
+│                                       │
+│  Runs every digest_interval per user  │
+│    ├─ SELECT all buffered items       │
+│    ├─ GROUP BY telegram_id            │
+│    ├─ Format summary message          │
+│    └─ DELETE buffered items           │
+│                                       │
+│  Summary format:                      │
+│    📋 Digest — 5 new ads (14:30 UTC)  │
+│                                        │
+│    🔑 "rtx 3090" (3 ads):             │
+│    • RTX 3090 Ti - €750 [link]        │
+│    • ASUS RTX 3090 - €680 [link]      │
+│    • MSI RTX 3090 Gaming X - €720     │
+│                                       │
+│    🔑 "gtx 1660" (2 ads):             │
+│    • GTX 1660 Super - €120 [link]     │
+│    • EVGA GTX 1660 - €95 [link]       │
+└───────────────────────────────────────┘
+```
+
+### Key design decisions
+
+- **Separate buffer table** instead of in-memory only. Survives restarts, visible via pgAdmin for debugging.
+- **Per-user interval**: Each user configures their own digest frequency (default 60 min). Implemented with a single scheduler task that checks all users' intervals on each cycle.
+- **Group by keyword** in the summary message. Makes it easy to scan relevant categories without digging through unrelated listings.
+
+## Implementation Details
+
+### 1. Add migration
+
+In `worker/src/migrations/04-user-settings.sql`:
+
+```sql
+-- digest_buffer table for accumulating notifications
+
+CREATE TABLE IF NOT EXISTS digest_buffer (
+    id          uuid PRIMARY KEY DEFAULT gen_random_uuid(),
+    telegram_id text NOT NULL,
+    ad_id       uuid REFERENCES ads(id) ON DELETE CASCADE,
+    keyword     text NOT NULL,
+    title       text NOT NULL,
+    price       int,  -- in cents
+    url         text,
+    created_at  timestamptz NOT NULL DEFAULT now()
+);
+
+CREATE INDEX IF NOT EXISTS idx_digest_buffer 
+    ON digest_buffer(telegram_id, created_at DESC);
+
+COMMENT ON TABLE digest_buffer IS 
+    'Buffer for digest-mode notifications. Flushed to Telegram at intervals.';
+```
+
+### 2. Buffer notifications instead of sending immediately
+
+In `main.py` scheduler loop, after all filters pass and mute check passes:
+
+```python
+# Check if user has digest mode enabled
+settings = await pool.fetchrow(
+    "SELECT digest_mode FROM user_settings WHERE telegram_id = $1",
+    telegram_id_str,
+)
+
+if settings and settings["digest_mode"]:
+    # Buffer for digest
+    await pool.execute(
+        """INSERT INTO digest_buffer (telegram_id, ad_id, keyword, title, price, url)
+           VALUES ($1, $2, $3, $4, $5, $6)""",
+        telegram_id_str,
+        ad_id,
+        kw_row["keyword"],
+        ad_dict.get("title", "Unknown"),
+        _extract_price(ad_dict),
+        ad_dict.get("url"),
+    )
+    
+    # Still log the notification (for stats)
+    await log_notify(pool, ad_id, telegram_id, "new")
+else:
+    # Send immediately (current behavior)
+    await notify_new(bot, pool, kw_row["keyword"], 
+                     telegram_id, ad_dict, ad_id)
+```
+
+### 3. Add digest flushing task to scheduler
+
+In `main.py`, add a new async function and call it at the start of each cycle:
+
+```python
+async def flush_digest_buffers() -> int:
+    """Process pending digest buffers for users whose interval has elapsed."""
+    from db import get_pool
+    
+    pool = await get_pool()
+    
+    # Get all users with digest mode ON
+    users = await pool.fetch("""
+        SELECT telegram_id, digest_interval 
+        FROM user_settings 
+        WHERE digest_mode = true
+    """)
+    
+    sent_count = 0
+    
+    for user in users:
+        interval_min = user["digest_interval"] or 60
+        cutoff = datetime.now(tz=timezone.utc) - timedelta(minutes=interval_min)
+        
+        # Get buffered items older than the interval
+        buffered = await pool.fetch("""
+            SELECT db.id, db.keyword, db.title, db.price, db.url
+            FROM digest_buffer db
+            WHERE db.telegram_id = $1 
+              AND db.created_at <= $2
+            ORDER BY db.keyword, db.created_at DESC
+        """, user["telegram_id"], cutoff)
+        
+        if not buffered:
+            continue
+        
+        # Group by keyword
+        from collections import defaultdict
+        groups: dict[str, list] = defaultdict(list)
+        
+        for item in buffered:
+            price_str = f"€{item['price']/100:.2f}" if item['price'] else "Free"
+            entry = f"• {item['title']} - {price_str}"
+            groups[item["keyword"]].append(entry)
+        
+        # Build summary message
+        lines = [f"📋 Digest — {len(buffered)} new ads ({cutoff:%H:%M}–{datetime.now(tz=timezone.utc):%H:%M} UTC)\n"]
+        
+        for kw_name, entries in groups.items():
+            lines.append(f"\n🔑 \"{kw_name}\" ({len(entries)} ads):")
+            # Limit to 10 entries per keyword to avoid spam
+            for entry in entries[:10]:
+                lines.append(entry)
+            if len(entries) > 10:
+                lines.append(f"  ... and {len(entries)-10} more")
+        
+        message_text = "\n".join(lines)
+        
+        # Send the digest
+        try:
+            from bot import get_application_bot
+            bot = get_application_bot()
+            
+            telegram_id_int = int(user["telegram_id"])
+            await bot.send_message(chat_id=telegram_id_int, text=message_text)
+            
+            sent_count += 1
+            
+            # Log all buffered notifications as delivered
+            buffer_ids = [item["id"] for item in buffered]
+            for ad_item in buffered:
+                await log_notify(pool, ad_item["ad_id"], telegram_id_int, "new")
+            
+        except TelegramError as e:
+            logger.error("Digest send failed for %s: %s", user["telegram_id"], e)
+        
+        finally:
+            # Clear the buffer (whether sent or not — if it failed, log entries remain in DB)
+            await pool.execute(
+                """DELETE FROM digest_buffer 
+                   WHERE telegram_id = $1 AND created_at <= $2""",
+                user["telegram_id"], cutoff,
+            )
+    
+    return sent_count
+
+
+# Call at the start of run_scheduler():
+async def run_scheduler() -> None:
+    while True:
+        try:
+            record_scheduler_run()
+            
+            # Flush digest buffers first
+            digests_sent = await flush_digest_buffers()
+            if digests_sent:
+                logger.info("Sent %d digest summaries", digests_sent)
+            
+            # Process notification queue...
+            processed = await process_notification_queue()
+            if processed:
+                logger.info("Retried %d queued notifications", processed)
+            
+            # ... existing keyword iteration ...
+```
+
+### 4. Add bot commands in `bot.py`
+
+```python
+async def cmd_digest_on(update: Update, context: ContextTypes.DEFAULT_TYPE) -> None:
+    """Enable digest mode with optional interval."""
+    
+    interval = 60  # default minutes
+    if len(context.args) > 0:
+        try:
+            interval = int(context.args[0])
+            if interval < 5 or interval > 1440:
+                raise ValueError
+        except ValueError:
+            await update.message.reply_text(
+                "Usage: /digest_on [minutes]\n"
+                "Interval must be between 5 and 1440 minutes (24h)."
+            )
+            return
+    
+    telegram_id = str(update.effective_user.id)
+    
+    from db import get_pool
+    pool = await get_pool()
+    
+    await pool.execute(
+        """INSERT INTO user_settings (telegram_id, digest_mode, digest_interval)
+           VALUES ($1, true, $2)
+           ON CONFLICT (telegram_id) 
+           DO UPDATE SET digest_mode = EXCLUDED.digest_mode, 
+                         digest_interval = EXCLUDED.digest_interval""",
+        telegram_id, interval,
+    )
+    
+    await update.message.reply_text(
+        f"✅ Digest mode ENABLED\n"
+        f"Digests will be sent every {interval} minutes.\n"
+        "Use /digest_off to return to instant notifications."
+    )
+
+
+async def cmd_digest_off(update: Update, context: ContextTypes.DEFAULT_TYPE) -> None:
+    """Disable digest mode."""
+    telegram_id = str(update.effective_user.id)
+    
+    from db import get_pool
+    pool = await get_pool()
+    
+    await pool.execute(
+        """INSERT INTO user_settings (telegram_id, digest_mode)
+           VALUES ($1, false)
+           ON CONFLICT (telegram_id) 
+           DO UPDATE SET digest_mode = EXCLUDED.digest_mode""",
+        telegram_id,
+    )
+    
+    # Flush any remaining buffered items immediately
+    await pool.execute(
+        """DELETE FROM digest_buffer WHERE telegram_id = $1""",
+        telegram_id,
+    )
+    
+    await update.message.reply_text(
+        "✅ Digest mode DISABLED — notifications are now instant."
+    )
+
+
+async def cmd_status(update: Update, context: ContextTypes.DEFAULT_TYPE) -> None:
+    """Show current user settings (extended from mute hours task)."""
+    telegram_id = str(update.effective_user.id)
+    
+    from db import get_pool
+    pool = await get_pool()
+    
+    settings = await pool.fetchrow(
+        "SELECT mute_start, mute_end, digest_mode, digest_interval "
+        "FROM user_settings WHERE telegram_id = $1",
+        telegram_id,
+    )
+    
+    reply_parts = []
+    
+    if not settings or not settings["mute_start"]:
+        reply_parts.append("🔕 Mute hours: OFF")
+    else:
+        reply_parts.append(
+            f"🔕 Mute hours: {settings['mute_start']} — {settings['mute_end']} UTC"
+        )
+    
+    if settings and settings["digest_mode"]:
+        reply_parts.append(
+            f"📋 Digest: ON (every {settings['digest_interval']} min)"
+        )
+    else:
+        reply_parts.append("📋 Digest: OFF (instant notifications)")
+    
+    await update.message.reply_text("\n".join(reply_parts))
+```
+
+Register handlers:
+```python
+dp.add_handler(CommandHandler("digest_on", cmd_digest_on))
+dp.add_handler(CommandHandler("digest_off", cmd_digest_off))
+```
+
+## Acceptance Criteria
+
+- [ ] `/digest_on` enables digest mode with 60-minute default interval
+- [ ] `/digest_on 30` sets digest interval to 30 minutes
+- [ ] New ads are inserted into `digest_buffer` instead of being sent immediately when digest is ON
+- [ ] At the configured interval, all buffered items are flushed as a single summary message
+- [ ] The summary groups ads by keyword and includes price information
+- [ ] After flushing, buffered items are deleted from the table
+- [ ] `/digest_off` disables digest mode and sends any remaining buffered items immediately
+- [ ] Mute hours take precedence over digest — muted notifications are discarded, not buffered

docs/phase-2/task-mute-hours.md

@@ -0,0 +1,211 @@
+# Task: Mute hours per user
+
+## Description
+
+Users may add keywords that are popular enough to trigger notifications at any hour. Currently, there's no way to suppress alerts during sleeping hours — the bot sends notifications 24/7.
+
+This task creates a `user_settings` table with configurable mute windows (start/end time). During the mute window, notifications for that user are suppressed entirely. The notification is not lost — it's still logged in `log_notifications`, but the Telegram message is not sent.
+
+## Architecture
+
+```
+┌───────────────────────────────────────┐
+│  user_settings table (new)            │
+│                                       │
+│    telegram_id   text PRIMARY KEY     │
+│    mute_start    time                 │
+│    mute_end      time                 │
+│    digest_mode   bool DEFAULT false   │
+│    digest_interval int DEFAULT 60     │
+│                                       │
+│  Example:                             │
+│    telegram_id = '298181113'          │
+│    mute_start  = '22:00:00'           │
+│    mute_end    = '07:00:00'           │
+│    → no alerts between 10PM-7AM UTC   │
+└──────────┬────────────────────────────┘
+           │
+           ▼
+┌───────────────────────────────────────┐
+│  Notification pipeline (in main.py)  │
+│                                       │
+│  For each new ad that passes filters: │
+│    user_settings = get from DB        │
+│    if in_mute_hours(user_settings):   │
+│      log_notify()                     │
+│      → skip Telegram send             │
+│    else:                              │
+│      notify_new() / notify_drop()     │
+└───────────────────────────────────────┘
+```
+
+### Key design decisions
+
+- **Time stored as `time` type in PostgreSQL** — native, efficient for range checks. Default is NULL (no mute window).
+  - *Alternative*: Could store as integer hours (e.g., 22, 7), but `time` type gives flexibility for minute-level precision and clearer UI.
+- **UTC timezone**: The bot operates in UTC internally. Users should be informed that mute times are in UTC. Adding timezone support per-user is a Phase 3 consideration.
+- **Mute window can cross midnight** — start > end means the window wraps around midnight (e.g., 22:00–07:00). The check handles this correctly.
+
+## Implementation Details
+
+### 1. Add migration
+
+In `worker/src/migrations/04-user-settings.sql`:
+
+```sql
+CREATE TABLE IF NOT EXISTS user_settings (
+    telegram_id     text PRIMARY KEY,
+    mute_start      time,
+    mute_end        time,
+    digest_mode     bool NOT NULL DEFAULT false,
+    digest_interval int NOT NULL DEFAULT 60,  -- minutes
+    
+    CONSTRAINT chk_mute_hours CHECK (
+        mute_start IS NULL AND mute_end IS NULL 
+        OR mute_start IS NOT NULL AND mute_end IS NOT NULL
+    )
+);
+
+COMMENT ON TABLE user_settings IS 
+    'User-specific settings for notification behavior';
+```
+
+### 2. Add mute hours check in `notifier.py` or `main.py`
+
+```python
+async def _is_in_mute_hours(
+    telegram_id: str, 
+    pool: asyncpg.Pool
+) -> bool:
+    """Check if the current time is within the user's mute window."""
+    
+    settings = await pool.fetchrow(
+        "SELECT mute_start, mute_end FROM user_settings WHERE telegram_id = $1",
+        telegram_id,
+    )
+    
+    if not settings or not settings["mute_start"] or not settings["mute_end"]:
+        return False  # no mute configured
+    
+    now_utc = datetime.now(tz=timezone.utc).time()
+    start = settings["mute_start"]
+    end = settings["mute_end"]
+    
+    if start < end:
+        # Normal window (e.g., 22:00–07:00 → actually wraps, so this is rare)
+        return start <= now_utc <= end
+    else:
+        # Window crosses midnight (e.g., 22:00 to 07:00 next day)
+        return now_utc >= start or now_utc <= end
+
+
+# In main.py scheduler loop, before calling notify_new():
+telegram_id_str = str(telegram_id)
+
+in_mute = await _is_in_mute_hours(telegram_id_str, pool)
+if in_mute:
+    logger.debug("Muted notification for user %s (mute window active)", telegram_id)
+    # Still log it but don't send Telegram message
+    await log_notify(pool, ad_id, telegram_id, "new")
+    return
+
+# Proceed with normal notification...
+```
+
+### 3. Add bot commands in `bot.py`
+
+```python
+async def cmd_set_mute(update: Update, context: ContextTypes.DEFAULT_TYPE) -> None:
+    """Set mute hours window (HH:MM-HH:MM format)."""
+    if len(context.args) < 1:
+        await update.message.reply_text(
+            "Usage: /mute_hours HH:MM-HH:MM\n"
+            "Example: /mute_hours 22:00-07:00 (mutes from 10PM to 7AM UTC)\n"
+            "Use /mute_off to disable."
+        )
+        return
+    
+    try:
+        start_str, end_str = context.args[0].split("-")
+        mute_start = datetime.strptime(start_str.strip(), "%H:%M").time()
+        mute_end = datetime.strptime(end_str.strip(), "%H:%M").time()
+    except (ValueError, TypeError) as e:
+        await update.message.reply_text(
+            f"Invalid format. Use HH:MM-HH:MM.\nExample: /mute_hours 22:00-07:00"
+        )
+        return
+    
+    telegram_id = str(update.effective_user.id)
+    
+    from db import get_pool
+    pool = await get_pool()
+    
+    await pool.execute(
+        """INSERT INTO user_settings (telegram_id, mute_start, mute_end)
+           VALUES ($1, $2, $3)
+           ON CONFLICT (telegram_id) 
+           DO UPDATE SET mute_start = EXCLUDED.mute_start, mute_end = EXCLUDED.mute_end""",
+        telegram_id, mute_start, mute_end,
+    )
+    
+    await update.message.reply_text(
+        f"✅ Mute hours set: {mute_start} — {mute_end} UTC\n"
+        "No notifications will be sent during this window.\n"
+        "Use /mute_off to disable or change."
+    )
+
+
+async def cmd_mute_off(update: Update, context: ContextTypes.DEFAULT_TYPE) -> None:
+    """Disable mute hours."""
+    telegram_id = str(update.effective_user.id)
+    
+    from db import get_pool
+    pool = await get_pool()
+    
+    await pool.execute(
+        """INSERT INTO user_settings (telegram_id, mute_start, mute_end)
+           VALUES ($1, NULL, NULL)
+           ON CONFLICT (telegram_id) 
+           DO UPDATE SET mute_start = EXCLUDED.mute_start, mute_end = EXCLUDED.mute_end""",
+        telegram_id,
+    )
+    
+    await update.message.reply_text("✅ Mute hours disabled.")
+
+
+async def cmd_status(update: Update, context: ContextTypes.DEFAULT_TYPE) -> None:
+    """Show current user settings."""
+    telegram_id = str(update.effective_user.id)
+    
+    from db import get_pool
+    pool = await get_pool()
+    
+    settings = await pool.fetchrow(
+        "SELECT mute_start, mute_end FROM user_settings WHERE telegram_id = $1",
+        telegram_id,
+    )
+    
+    if not settings or not settings["mute_start"]:
+        reply = "🔕 Mute hours: OFF (notifications sent 24/7)"
+    else:
+        reply = f"🔕 Mute hours: {settings['mute_start']} — {settings['mute_end']} UTC"
+    
+    await update.message.reply_text(reply)
+```
+
+Register handlers:
+```python
+dp.add_handler(MessageHandler(REGEX(r"^/mute_hours"), cmd_set_mute))
+dp.add_handler(CommandHandler("mute_off", cmd_mute_off))
+dp.add_handler(CommandHandler("status", cmd_status))
+```
+
+## Acceptance Criteria
+
+- [ ] `/mute_hours 22:00-07:00` sets mute window from 10 PM to 7 AM UTC
+- [ ] Notifications during the mute window are logged but NOT sent via Telegram
+- [ ] Notifications outside the mute window work normally (no regression)
+- [ ] Mute windows that cross midnight (start > end) are handled correctly
+- [ ] `/mute_off` clears both start and end times, restoring 24/7 notifications
+- [ ] `/status` shows current mute settings clearly
+- [ ] Users without any settings in `user_settings` table receive all notifications (default behavior unchanged)

docs/phase-2/task-postcode-filters.md

@@ -0,0 +1,199 @@
+# Task: Location / postcode filters per keyword
+
+## Description
+
+Ads can match a user's keyword interest but be located in a completely different region (e.g., "rtx 3090" in Graz when the user only cares about Vienna). This task adds optional postcode filtering so users receive alerts only for ads in their desired locations.
+
+The willhaben API returns location data in `LOCATION_CityName`, `LOCATION_ZIP` (postcode), and similar fields. We'll match against these fields.
+
+## Architecture
+
+```
+┌───────────────────────────────────────┐
+│  keywords table (extended)            │
+│                                       │
+│    ...                                 │
+│    allowed_postcodes text[]           │
+│    ...                                 │
+│                                       │
+│  Example:                              │
+│    allowed_postcodes = {'1010','1020'} │
+│    → only ads in Vienna 1st/2nd dist. │
+└──────────┬────────────────────────────┘
+           │
+           ▼
+┌───────────────────────────────────────┐
+│  Processing filter (in main.py)      │
+│                                       │
+│  for ad in ads_raw:                   │
+│    ad_zip = _extract_postcode(ad)     │
+│    if kw.allowed_postcodes and       │
+│       ad_zip not in postcodes:        │
+│      skip                             │
+└──────────┬────────────────────────────┘
+           │
+           ▼
+┌───────────────────────────────────────┐
+│  Bot commands (in bot.py):            │
+│                                       │
+│  /postcode <keyword> p1,p2,p3         │
+│    → sets allowed_postcodes = {'p1',  │
+│      'p2','p3'}                       │
+│  /clear_postcode <keyword>           │
+│    → removes filter (NULL)            │
+└───────────────────────────────────────┘
+```
+
+### Key design decisions
+
+- **Text array** (`text[]`) instead of a separate lookup table. Simple, efficient for the typical case (<10 postcodes per keyword), and leverages PostgreSQL's native array support.
+  - *Alternative*: A `keyword_postcodes` junction table allows individual postcode management but adds unnecessary complexity for this use case.
+- **Match against willhaben's ZIP code field** (`LOCATION_ZIP` in the ad attributes). This is the most reliable location identifier and works across all Austrian postcodes (4 digits, e.g., "1010", "8010").
+- **Empty or missing postcode = skip if filter active**. If `allowed_postcodes` is set but an ad has no ZIP code, it's excluded. This prevents noise from unlocated ads.
+
+## Implementation Details
+
+### 1. Add migration
+
+In `worker/src/migrations/03-keyword-filters.sql`:
+
+```sql
+ALTER TABLE keywords ADD COLUMN IF NOT EXISTS allowed_postcodes text[];
+
+COMMENT ON COLUMN keywords.allowed_postcodes IS 
+    'Austrian postcodes (4-digit strings). Only ads matching these are notified.';
+```
+
+### 2. Extract postcode from ad data in `notifier.py` or `scraper.py`
+
+Add helper function:
+
+```python
+def _extract_postcode(ad_dict: dict) -> str | None:
+    """Extract the postal code (ZIP) from a willhaben ad."""
+    attrs = _parse_attributes(ad_dict)
+    
+    # Try multiple field names that willhaben might use
+    for key in ("LOCATION_ZIP", "LocationZip", "postalcode"):
+        val = attrs.get(key) or attrs.get(f"{key}_String")
+        if val:
+            return str(val).strip()
+    
+    return None
+```
+
+### 3. Add postcode filter check in `main.py` scheduler loop
+
+```python
+async def _check_postcode_filter(
+    ad_dict: dict, 
+    kw_row: dict
+) -> bool:
+    """Return True if the ad passes the postcode filter."""
+    
+    postcodes = kw_row.get("allowed_postcodes")  # list or None
+    
+    if not postcodes:
+        return True  # no filter active
+    
+    from notifier import _extract_postcode  # or wherever it lives
+    
+    ad_zip = _extract_postcode(ad_dict)
+    
+    if not ad_zip:
+        logger.debug("No postcode found in ad, skipping")
+        return False
+    
+    # Normalize: willhaben returns "1010" as string, we store same way
+    return ad_zip in postcodes
+
+
+# In the scheduler loop (after price check):
+for ad_dict in ads_raw:
+    if not await _check_price_filters(ad_dict, kw_row):
+        continue
+    
+    if not await _check_postcode_filter(ad_dict, kw_row):
+        continue
+    
+    # ... rest of processing
+```
+
+### 4. Add bot commands in `bot.py`
+
+```python
+async def cmd_set_postcode(update: Update, context: ContextTypes.DEFAULT_TYPE) -> None:
+    """Set allowed postcodes for a keyword."""
+    if len(context.args) < 2:
+        await update.message.reply_text("Usage: /postcode <keyword> <p1,p2,p3>")
+        return
+    
+    kw_name = context.args[0]
+    postcode_strs = [pc.strip() for pc in context.args[1].split(",")]
+    
+    # Validate format (4-digit Austrian postcodes)
+    invalid = [pc for pc in postcode_strs if not re.match(r"^\d{3,5}$", pc)]
+    if invalid:
+        await update.message.reply_text(
+            f"Invalid postcode(s): {', '.join(invalid)}. "
+            "Use 4-digit format like 1010, 8010."
+        )
+        return
+    
+    telegram_id = str(update.effective_user.id)
+    
+    from db import get_pool
+    pool = await get_pool()
+    
+    kw_id = await pool.fetchval(
+        """SELECT id FROM keywords 
+           WHERE LOWER(keyword_name) = $1 AND telegram_id = $2""",
+        kw_name.lower(), telegram_id,
+    )
+    
+    if not kw_id:
+        await update.message.reply_text(f"Keyword '{kw_name}' not found.")
+        return
+    
+    await pool.execute(
+        "UPDATE keywords SET allowed_postcodes = $1 WHERE id = $2",
+        postcode_strs, kw_id,  # asyncpg handles text[] natively
+    )
+    
+    await update.message.reply_text(
+        f"✅ Keyword '{kw_name}': postcodes set to {', '.join(postcode_strs)}"
+    )
+
+
+async def cmd_clear_postcode(update: Update, context: ContextTypes.DEFAULT_TYPE) -> None:
+    """Remove postcode filter for a keyword."""
+    if len(context.args) < 1:
+        await update.message.reply_text("Usage: /clear_postcode <keyword>")
+        return
+    
+    # ... same pattern, set allowed_postcodes = NULL
+```
+
+Register handlers:
+```python
+dp.add_handler(MessageHandler(REGEX(r"^/postcode"), cmd_set_postcode))
+dp.add_handler(MessageHandler(REGEX(r"^/clear_postcode"), cmd_clear_postcode))
+```
+
+### 5. Update `/keywords` command output
+
+Add postcode info to the listing:
+```python
+if row["allowed_postcodes"]:
+    line += f"\n         📍 {', '.join(row['allowed_postcodes'])}"
+```
+
+## Acceptance Criteria
+
+- [ ] `/postcode keyword 1010,1020` sets allowed postcodes to `{'1010', '1020'}` for that keyword
+- [ ] Ads with ZIP codes NOT in the allowed list are skipped during processing
+- [ ] Ads with no ZIP code at all are skipped when a filter is active
+- [ ] `/clear_postcode keyword` removes the filter (NULL)
+- [ ] Invalid postcodes (non-numeric or wrong length) are rejected by the bot
+- [ ] The `/keywords` command shows active postcodes next to each keyword
+- [ ] Both price AND postcode filters work correctly together (ad must pass both to be notified)

docs/phase-2/task-price-filters.md

@@ -0,0 +1,254 @@
+# Task: Price range filters per keyword
+
+## Description
+
+Currently, every ad that matches a keyword triggers a notification regardless of price. For keywords like "rtx" or "3090", this means users receive alerts for €5 listings (accessories) alongside relevant hardware deals.
+
+This task adds `price_min` and `price_max` columns to the `keywords` table and implements server-side filtering during ad processing. Users set these via bot commands, and ads outside the range are silently skipped without being stored in the `ads` table or triggering notifications.
+
+## Architecture
+
+```
+┌───────────────────────────────────────┐
+│  keywords table (extended)            │
+│                                       │
+│    ...                                 │
+│    price_min      int                  │
+│    price_max      int                  │
+│    ...                                 │
+└──────────┬────────────────────────────┘
+           │
+           ▼
+┌───────────────────────────────────────┐
+│  Processing pipeline (in main.py)     │
+│                                       │
+│  for ad in ads_raw:                   │
+│    price = _extract_price(ad)         │
+│    if price is None:                  │
+│      continue                         │
+│    if kw.price_min and price < min:   │
+│      skip (below minimum)             │
+│    if kw.price_max and price > max:   │
+│      skip (above maximum)             │
+│    → process ad normally              │
+└──────────┬────────────────────────────┘
+           │
+           ▼
+┌───────────────────────────────────────┐
+│  Bot commands (in bot.py):            │
+│                                       │
+│  /price_min <keyword> <€amount>       │
+│  /price_max <keyword> <€amount>       │
+│  /clear_price <keyword>              │
+│  /keywords → shows price ranges       │
+└───────────────────────────────────────┘
+```
+
+### Key design decisions
+
+- **Filter during processing, not at DB time** — the filter is applied in Python before inserting into `ads`. This keeps the `ads` table clean (only relevant ads are stored) and avoids extra WHERE clauses on every scrape cycle.
+  - *Alternative*: Could use a computed column or trigger, but adds complexity for simple numeric comparison.
+- **Both filters optional** — NULL = no limit. Users can set only min, only max, both, or neither.
+- **Price extraction from ad data**: Use the existing `_extract_price()` function in `notifier.py`. If price is not available (free items), treat as €0 for filtering purposes.
+
+## Implementation Details
+
+### 1. Add migration
+
+In `worker/src/migrations/03-keyword-filters.sql`:
+
+```sql
+ALTER TABLE keywords 
+    ADD COLUMN IF NOT EXISTS price_min int,
+    ADD COLUMN IF NOT EXISTS price_max int;
+
+COMMENT ON COLUMN keywords.price_min IS 'Minimum price in cents (e.g. 5000 = €50). NULL = no limit.';
+COMMENT ON COLUMN keywords.price_max IS 'Maximum price in cents (e.g. 500000 = €5000). NULL = no limit.';
+```
+
+### 2. Update `main.py` — filter ads by price during processing
+
+In the scheduler's keyword loop, before calling `_process_ad()`:
+
+```python
+# After extracting ad info in _process_ad():
+async def _check_price_filters(
+    ad_dict: dict, 
+    kw_row: dict  # from keywords table
+) -> bool:
+    """Return True if the ad passes price filters for this keyword."""
+    
+    from notifier import _extract_price
+    
+    price = _extract_price(ad_dict)
+    
+    if price is None:
+        # No price found — include it (could be "free" or missing data)
+        return True
+    
+    price_min = kw_row.get("price_min")
+    price_max = kw_row.get("price_max")
+    
+    if price_min is not None and price < price_min:
+        logger.debug(
+            "Price filter skip: ad %s price=%d min=%d", 
+            ad_id, price, price_min
+        )
+        return False
+    
+    if price_max is not None and price > price_max:
+        logger.debug(
+            "Price filter skip: ad %s price=%d max=%d", 
+            ad_id, price, price_max
+        )
+        return False
+    
+    return True
+
+
+# In the scheduler loop:
+for kw_row in keywords:
+    if not kw_row["is_active"]:
+        continue
+    
+    ads_raw = await fetch_ads(kw_row["keyword"])
+    
+    for ad_dict in ads_raw:
+        # Check price filters BEFORE processing
+        if not await _check_price_filters(ad_dict, kw_row):
+            continue
+        
+        # Existing _process_ad logic continues here...
+```
+
+### 3. Add bot commands in `bot.py`
+
+```python
+from telegram import Message
+from telegram.ext import ContextTypes
+
+
+async def cmd_set_price_min(update: Update, context: ContextTypes.DEFAULT_TYPE) -> None:
+    """Set minimum price for a keyword."""
+    if len(context.args) < 2:
+        await update.message.reply_text("Usage: /price_min <keyword> <amount_in_euro>")
+        return
+    
+    kw_name = context.args[0]
+    try:
+        amount_eur = float(context.args[1])
+        price_cents = int(amount_eur * 100)
+    except ValueError:
+        await update.message.reply_text("Invalid amount. Use a number like 50 or 99.99")
+        return
+    
+    telegram_id = str(update.effective_user.id)
+    
+    from db import get_pool
+    pool = await get_pool()
+    
+    # Find keyword by name for this user
+    kw_id = await pool.fetchval(
+        """SELECT id FROM keywords 
+           WHERE LOWER(keyword_name) = $1 AND telegram_id = $2""",
+        kw_name.lower(), telegram_id,
+    )
+    
+    if not kw_id:
+        await update.message.reply_text(f"Keyword '{kw_name}' not found for your account.")
+        return
+    
+    await pool.execute(
+        "UPDATE keywords SET price_min = $1 WHERE id = $2",
+        price_cents, kw_id,
+    )
+    
+    await update.message.reply_text(
+        f"✅ Keyword '{kw_name}': minimum price set to €{amount_eur:.2f}"
+    )
+
+
+async def cmd_set_price_max(update: Update, context: ContextTypes.DEFAULT_TYPE) -> None:
+    """Set maximum price for a keyword."""
+    # Same pattern as cmd_set_price_min but sets price_max
+    
+    if len(context.args) < 2:
+        await update.message.reply_text("Usage: /price_max <keyword> <amount_in_euro>")
+        return
+    
+    kw_name = context.args[0]
+    try:
+        amount_eur = float(context.args[1])
+        price_cents = int(amount_eur * 100)
+    except ValueError:
+        await update.message.reply_text("Invalid amount. Use a number like 50 or 99.99")
+        return
+    
+    telegram_id = str(update.effective_user.id)
+    
+    from db import get_pool
+    pool = await get_pool()
+    
+    kw_id = await pool.fetchval(
+        """SELECT id FROM keywords 
+           WHERE LOWER(keyword_name) = $1 AND telegram_id = $2""",
+        kw_name.lower(), telegram_id,
+    )
+    
+    if not kw_id:
+        await update.message.reply_text(f"Keyword '{kw_name}' not found.")
+        return
+    
+    await pool.execute(
+        "UPDATE keywords SET price_max = $1 WHERE id = $2",
+        price_cents, kw_id,
+    )
+    
+    await update.message.reply_text(
+        f"✅ Keyword '{kw_name}': maximum price set to €{amount_eur:.2f}"
+    )
+
+
+async def cmd_clear_price(update: Update, context: ContextTypes.DEFAULT_TYPE) -> None:
+    """Clear price filters for a keyword."""
+    if len(context.args) < 1:
+        await update.message.reply_text("Usage: /clear_price <keyword>")
+        return
+    
+    # ... similar pattern, set both to NULL
+```
+
+Register handlers in `register_handlers()`:
+```python
+dp.add_handler(MessageHandler(
+    REGEX(r"^/price_min"), cmd_set_price_min))
+dp.add_handler(MessageHandler(
+    REGEX(r"^/price_max"), cmd_set_price_max))
+dp.add_handler(MessageHandler(
+    REGEX(r"^/clear_price"), cmd_clear_price))
+```
+
+### 4. Update `/keywords` command output to show price ranges
+
+In the existing `/keywords` handler in `bot.py`, add:
+
+```python
+price_info = ""
+if row["price_min"] is not None:
+    price_info += f"min €{row['price_min']/100:.2f} "
+if row["price_max"] is not None:
+    price_info += f"max €{row['price_max']/100:.2f}"
+if price_info:
+    line += f"\n         💰 {price_info.strip()}"
+```
+
+## Acceptance Criteria
+
+- [ ] `/price_min keyword 50` sets the minimum price to €50.00 (stored as 5000 cents) for that keyword
+- [ ] `/price_max keyword 1000` sets the maximum price to €1000.00
+- [ ] Ads with price below `price_min` are skipped during processing and not inserted into `ads` table
+- [ ] Ads with price above `price_max` are skipped during processing
+- [ ] `/clear_price keyword` removes both limits (sets to NULL)
+- [ ] Keywords without price filters continue to work as before (no regression)
+- [ ] The `/keywords` command displays the active price range for each keyword
+- [ ] Non-admin users can only modify their own keywords' price filters

docs/phase-3/Phase.md

@@ -0,0 +1,98 @@
+# Phase 3 — Web Dashboard & Testing
+
+## Scope
+
+This phase introduces **observability and reliability** improvements. Currently, the entire system is a single async Python process with no tests, no CI/CD pipeline, and no way to monitor what's happening without SSH-ing into the server. After this phase:
+
+- A **Web Dashboard** provides real-time visibility into keywords, ads, users, and stats
+- Automated tests provide confidence for every change (≥80% coverage)
+- CI/CD pipeline runs on every push to validate code quality
+
+## Architecture
+
+```
+┌──────────────────────────────────────────────────────┐
+│  Project Structure (post-Phase-3)                     │
+│                                                      │
+│  willhaben-tracker/                                   │
+│    ├── worker/                                        │
+│    │   ├── src/                                       │
+│    │   │   ├── main.py       (entry point, scheduler) │
+│    │   │   ├── db.py         (asyncpg pool mgmt)      │
+│    │   │   ├── bot.py        (Telegram handlers)      │
+│    │   │   ├── notifier.py   (message sending)        │
+│    │   │   ├── scraper.py    (willhaben scraper)      │
+│    │   │   ├── web.py        (FastAPI dashboard)      │
+│    │   │   ├── health.py     (healthcheck endpoint)   │
+│    │   │   ├── migrate.py    (migration runner)       │
+│    │   │   └── templates/    (Jinja2 HTML templates)  │
+│    │   ├── tests/                                       │
+│    │   │   ├── conftest.py                                │
+│    │   │   ├── test_scraper.py                            │
+│    │   │   ├── test_notifier.py                           │
+│    │   │   ├── test_filters.py                            │
+│    │   │   └── test_web.py                                │
+│    │   ├── Dockerfile                                   │
+│    │   └── requirements.txt                             │
+│    ├── .github/                                         │
+│    │   └── workflows/                                   │
+│    │       └── ci.yml     (pytest + flake8 + coverage)  │
+│    ├── pyproject.toml   (coverage config, tools)        │
+│    └── docker-compose.yml                               │
+└──────────────────────────────────────────────────────┘
+
+Web Dashboard (FastAPI, port 8766):
+  
+  GET /             → Dashboard (keywords overview, stats summary)
+  GET /keywords     → Keywords list with status, filters, subscribers
+  GET /keywords/<id> → Keyword detail (recent ads, price history, scrape logs)
+  GET /users        → Users list with settings
+  GET /ads          → Recent ads with search/filter
+  GET /stats        → JSON stats (extends existing /stats endpoint)
+  
+  Auth: Basic Auth via WEB_UI_USERNAME / WEB_UI_PASSWORD env vars
+  Templates: Jinja2 with inline CSS (zero external dependencies)
+
+CI/CD Pipeline (.github/workflows/ci.yml):
+  
+  on: push to main, feat/*; pull_request
+  
+  jobs:
+    lint-and-test:
+      └─ python 3.12
+         ├─ flake8 (linting)
+         ├─ pytest --cov=src tests/ (unit + integration tests)
+         └─ coverage >= 80% (fail if not met)
+
+Tests Structure:
+  
+  Unit tests:
+    - test_scraper_pagination() — verify pagination logic with mock responses
+    - test_price_filters() — verify filter functions
+    - test_notification_retry() — verify retry queue behavior
+    - test_mute_digest() — verify mute hours and digest buffering
+    - test_web_endpoints() — verify web UI routes
+  
+  Integration tests:
+    - Test against real willhaben API (rate-limited, cached)
+    - PostgreSQL test container via docker-compose
+```
+
+## Tasks
+
+| Task | File | Description |
+|------|------|-------------|
+| Web Dashboard (FastAPI + Jinja2) | [task-web-ui.md](./task-web-ui.md) | Add a read-only web dashboard for monitoring keywords, ads, users, and stats. Runs on port 8766 with basic auth. |
+| Test suite with pytest (≥80% coverage) | [task-testing-pytest.md](./task-testing-pytest.md) | Add comprehensive unit tests covering scraper parsing, notification logic, price/postcode filters, retry queue, and scheduler flow. Configure coverage thresholds. |
+
+## General Acceptance Criteria
+
+- [ ] Web Dashboard is accessible at `http://<host>:8766` with basic auth
+- [ ] Dashboard shows keywords with status, filters, subscribers, and last scrape time
+- [ ] Dashboard shows recent ads with price, location, and keyword
+- [ ] Dashboard shows users with mute/digest settings
+- [ ] Dashboard shows stats (ads indexed, notifications sent, queue status)
+- [ ] CI pipeline runs on every push to `main` and feature branches — fails if lint or coverage checks are not met
+- [ ] Code coverage is ≥80% across all source files in `worker/src/`
+- [ ] All existing functionality (willhaben scraping, Telegram notifications, health server) continues to work
+- [ ] Health server still works on port 8765 (no regression)

docs/phase-3/task-testing-pytest.md

@@ -0,0 +1,777 @@
+# Task: Test suite with pytest (≥80% coverage)
+
+## Description
+
+The project currently has **zero automated tests**. Every change is verified manually by watching logs or sending test messages to the bot. This makes refactoring risky and prevents CI/CD automation.
+
+This task introduces a comprehensive pytest test suite covering all critical paths: scraper parsing, notification logic, price/postcode filters, retry queue behavior, and scheduler flow. Coverage threshold is set to 80% minimum.
+
+## Architecture
+
+```
+┌───────────────────────────────────────┐
+│  Tests structure                      │
+│                                       │
+│  worker/tests/                        │
+│    ├── conftest.py                    │
+│    │   (fixtures: mock_pool, mock_bot)│
+│    ├── test_scraper.py                │
+│    │   (parsing, pagination)          │
+│    ├── test_notifier.py               │
+│    │   (send, retry queue, digest)    │
+│    ├── test_filters.py                │
+│    │   (price, postcode, mute hours)  │
+│    ├── test_scheduler.py              │
+│    │   (cycle flow, shutdown)         │
+│    └── test_health.py                 │
+│       (healthcheck endpoint)           │
+│                                       │
+└──────────┬────────────────────────────┘
+           │
+           ▼
+┌───────────────────────────────────────┐
+│  CI/CD Pipeline (.github/workflows/ci.yml)
+│                                       │
+│  on: push to main, feat/*; pull_request│
+│                                       │
+│  jobs:                                │
+│    lint-and-test:                     │
+│      └─ python 3.12                  │
+│         ├─ flake8 (error-only)        │
+│         ├─ pytest --cov=src tests/    │
+│         └─ coverage >= 80%            │
+└───────────────────────────────────────┘
+
+Test strategy:
+  
+  Unit tests (majority):
+    - Isolate each function/method with mocks
+    - Test edge cases: missing fields, NULL prices, empty results
+    - Fast (<1s per test)
+  
+  Integration tests (minority):
+    - Real HTTP to willhaben API (cached responses only)
+    - PostgreSQL test container via docker-compose
+  
+  Mocking strategy:
+    - Telegram Bot: mock `bot.send_message()` → verify call count + content
+    - Asyncpg pool: mock fetchval/fetch/execute → return canned data
+    - httpx client: use pytest-httpx to intercept and return fixtures
+```
+
+### Key design decisions
+
+- **pytest over unittest**. Cleaner syntax, better fixture system, easier async support.
+- **pytest-asyncio** for testing async functions directly without wrapping in `loop.run_until_complete()`.
+  - Adding as test dependency: `pip install pytest pytest-asyncio pytest-cov httpx[socks]`
+- **Coverage threshold at 80%** enforced via `pyproject.toml`. Failures are actionable (which files/functions need coverage).
+- **Snapshot testing for HTTP responses**. Save real willhaben API responses as JSON fixtures to avoid live network calls in CI.
+
+## Implementation Details
+
+### 1. Add test dependencies and configuration
+
+In `worker/requirements-test.txt`:
+```txt
+pytest>=8.0
+pytest-asyncio>=0.24
+pytest-cov>=6.0
+aioresponses>=0.7    # mock aiohttp responses for health endpoint tests
+```
+
+Create `pyproject.toml` in the project root:
+```toml
+[tool.pytest.ini_options]
+testpaths = ["worker/tests"]
+asyncio_mode = "auto"
+python_files = ["test_*.py"]
+python_classes = ["Test*"]
+python_functions = ["test_*"]
+
+[tool.coverage.run]
+source = ["worker/src"]
+omit = [
+    "*/tests/*",
+    "*/migrate.py",   # migration runner — tested manually against real DB
+]
+
+[tool.coverage.report]
+fail_under = 80.0
+show_missing = true
+exclude_lines = [
+    "pragma: no cover",
+    "if TYPE_CHECKING:",
+]
+```
+
+### 2. Create `worker/tests/conftest.py` (fixtures)
+
+```python
+import pytest
+from unittest.mock import AsyncMock, MagicMock
+
+
+@pytest.fixture
+def mock_pool():
+    """Mock asyncpg.Pool with fetchval/fetch/execute."""
+    pool = AsyncMock()
+    pool.fetchval = AsyncMock(return_value=None)
+    pool.fetch = AsyncMock(return_value=[])
+    pool.execute = AsyncMock(return_value="DONE 1")
+    return pool
+
+
+@pytest.fixture
+def mock_bot():
+    """Mock Telegram Bot instance."""
+    bot = MagicMock()
+    bot.send_message = AsyncMock(return_value=True)
+    bot.get_me = AsyncMock(return_value={"id": "bot_user", "is_bot": True})
+    return bot
+
+
+@pytest.fixture
+def sample_willhaben_response():
+    """Realistic willhaben API response for testing."""
+    return {
+        "rowsFound": 45,
+        "advertSummaryList": {
+            "advertSummary": [
+                {
+                    "id": "123456789",
+                    "title": {"Value": "RTX 3090 Gaming X - Top Zustand"},
+                    "linkUrl": "https://www.willhaben.at/iad/markt/123456789-rtx-3090-gaming-x",
+                    "attributes": [
+                        {
+                            "name": "PRICE",
+                            "items": [{"name": "priceString", "valueString": "750.00"}]
+                        },
+                        {
+                            "name": "PUBLISHED",
+                            "items": [{"name": "publishedString", "valueString": "2026-07-04T12:30:00+02:00"}]
+                        },
+                        {
+                            "name": "LOCATION",
+                            "items": [
+                                {"name": "CityName", "valueString": "Wien"},
+                                {"name": "ZIP", "valueString": "1010"},
+                            ]
+                        }
+                    ]
+                },
+            ]
+        }
+    }
+
+
+@pytest.fixture
+def sample_ad_normalized():
+    """Expected normalized ad dict from willhaben response."""
+    return {
+        "id": "123456789",
+        "marketplace": "willhaben",
+        "title": "RTX 3090 Gaming X - Top Zustand",
+        "price": 75000,  # in cents
+        "currency": "EUR",
+        "url": "https://www.willhaben.at/iad/markt/123456789-rtx-3090-gaming-x",
+        "published_at": ...,  # will be datetime object
+        "location": {
+            "city": "Wien",
+            "postcode": "1010",
+            "region": None,
+        },
+        "attributes": {...},
+    }
+
+
+@pytest.fixture
+def sample_keyword_row():
+    """Sample keyword DB row."""
+    return {
+        "id": "kw-uuid-here",
+        "keyword_name": "rtx 3090",
+        "telegram_id": "298181113",
+        "is_active": True,
+        "price_min": 50000,    # €500 minimum
+        "price_max": 1000000,  # €10000 maximum
+        "allowed_postcodes": ["1010", "1020"],
+        "last_scraped_at": None,
+        "ads_cursor": None,
+    }
+```
+
+### 3. Create `worker/tests/test_scraper.py`
+
+```python
+import pytest
+from unittest.mock import AsyncMock, patch
+from scrapers.willhaben import WillhabenScraper
+from datetime import datetime, timezone
+
+
+class TestWillhabenScraper:
+    
+    def test_build_query(self):
+        scraper = WillhabenScraper()
+        url = scraper.build_query("rtx 3090", offset=30)
+        
+        assert "keyword=rtx+3090" in url
+        assert "offset=30" in url
+        assert "rows=30" in url
+    
+    def test_build_query_default_offset(self):
+        scraper = WillhabenScraper()
+        url = scraper.build_query("gtx 1660")
+        
+        assert "offset=0" in url
+    
+    def test_parse_page_empty_response(self, sample_willhaben_response):
+        scraper = WillhabenScraper()
+        
+        # Empty response
+        empty = {"rowsFound": 0, "advertSummaryList": {"advertSummary": []}}
+        result = scraper.parse_page(empty)
+        
+        assert result == []
+    
+    def test_normalize_ad(self, sample_willhaben_response):
+        scraper = WillhabenScraper()
+        raw_ad = sample_willhaben_response["advertSummaryList"]["advertSummary"][0]
+        ad = scraper.normalize_ad(raw_ad)
+        
+        assert ad["id"] == "123456789"
+        assert ad["marketplace"] == "willhaben"
+        assert ad["price"] == 75000  # cents
+        assert ad["location"]["postcode"] == "1010"
+    
+    def test_normalize_ad_missing_price(self):
+        scraper = WillhabenScraper()
+        raw_ad = {
+            "id": "no-price",
+            "title": {"Value": "Free RTX"},
+            "linkUrl": "https://example.com",
+            "attributes": [],  # no price
+        }
+        ad = scraper.normalize_ad(raw_ad)
+        
+        assert ad["price"] is None
+    
+    @pytest.mark.asyncio
+    async def test_fetch_ads_pagination(self):
+        scraper = WillhabenScraper()
+        
+        with patch.object(scraper, '_fetch_with_retry') as mock_fetch:
+            # Simulate 2 pages of results
+            page1 = {
+                "rowsFound": 45,
+                "advertSummaryList": {"advertSummary": [
+                    {"id": f"ad{i}", "title": {"Value": f"Ad {i}"}, 
+                     "linkUrl": f"https://example.com/{i}",
+                     "attributes": [{"name": "PUBLISHED", "items": [
+                         {"name": "publishedString", 
+                          "valueString": "2026-07-04T15:30:00+02:00"}]}]},
+                ] for i in range(3)}]
+            }
+            
+            page2 = {
+                "rowsFound": 45,
+                "advertSummaryList": {"advertSummary": [
+                    {"id": f"ad{i}", "title": {"Value": f"Ad {i}"}, 
+                     "linkUrl": f"https://example.com/{i}",
+                     "attributes": [{"name": "PUBLISHED", "items": [
+                         {"name": "publishedString", 
+                          "valueString": "2026-07-04T15:30:00+02:00"}]}]},
+                ] for i in range(3, 6)}]
+            }
+            
+            mock_fetch.side_effect = [page1, page2]
+            
+            ads, total = await scraper.fetch_ads("test keyword", max_pages=2)
+            
+            assert len(ads) == 6  # 3 from each page
+
+
+class TestScraperBase:
+    
+    def test_headers_default(self):
+        scraper = WillhabenScraper()
+        headers = scraper.headers
+        
+        assert "Accept" in headers
+        assert "User-Agent" in headers
+    
+    @pytest.mark.asyncio
+    async def test_fetch_with_retry_exhausts_retries(self):
+        import httpx
+        
+        scraper = WillhabenScraper()
+        client = AsyncMock()
+        client.get = AsyncMock(side_effect=httpx.ConnectError("connection refused"))
+        
+        with pytest.raises(httpx.ConnectError):
+            await scraper._fetch_with_retry(client, "http://example.com", max_retries=2)
+```
+
+### 4. Create `worker/tests/test_filters.py`
+
+```python
+import pytest
+from unittest.mock import AsyncMock
+
+
+class TestPriceFilters:
+    
+    async def test_pass_when_no_min(self):
+        """Ad passes when no price minimum is set."""
+        # Import the actual function being tested
+        from notifier import _extract_price  # or wherever it lives
+        
+        kw_row = {"price_min": None, "price_max": None}
+        
+        ad_dict = {
+            "attributes": [{"name": "PRICE", "items": [
+                {"name": "priceString", "valueString": "50.00"}]}]
+        }
+        
+        # The check function (to be implemented in main.py)
+        from ..main import _check_price_filters  # actual implementation
+        
+        result = await _check_price_filters(ad_dict, kw_row)
+        assert result is True
+    
+    async def test_pass_when_no_max(self):
+        kw_row = {"price_min": 1000, "price_max": None}
+        
+        ad_dict = {
+            "attributes": [{"name": "PRICE", "items": [
+                {"name": "priceString", "valueString": "100.00"}]}]
+        }
+        
+        from ..main import _check_price_filters
+        result = await _check_price_filters(ad_dict, kw_row)
+        assert result is True
+    
+    async def test_fail_below_min(self):
+        kw_row = {"price_min": 50000, "price_max": None}  # €500 min
+        
+        ad_dict = {
+            "attributes": [{"name": "PRICE", "items": [
+                {"name": "priceString", "valueString": "10.00"}]}]  # €10 — too cheap
+        }
+        
+        from ..main import _check_price_filters
+        result = await _check_price_filters(ad_dict, kw_row)
+        assert result is False
+    
+    async def test_fail_above_max(self):
+        kw_row = {"price_min": None, "price_max": 1000}  # €10 max
+        
+        ad_dict = {
+            "attributes": [{"name": "PRICE", "items": [
+                {"name": "priceString", "valueString": "500.00"}]}]  # €500 — too expensive
+        }
+        
+        from ..main import _check_price_filters
+        result = await _check_price_filters(ad_dict, kw_row)
+        assert result is False
+
+
+class TestPostcodeFilters:
+    
+    async def test_pass_when_no_filter(self):
+        kw_row = {"allowed_postcodes": None}
+        
+        ad_dict = {
+            "attributes": [{"name": "LOCATION", "items": [
+                {"name": "ZIP", "valueString": "1010"}]}]
+        }
+        
+        from ..main import _check_postcode_filter
+        result = await _check_postcode_filter(ad_dict, kw_row)
+        assert result is True
+    
+    async def test_pass_when_matching(self):
+        kw_row = {"allowed_postcodes": ["1010", "1020"]}
+        
+        ad_dict = {
+            "attributes": [{"name": "LOCATION", "items": [
+                {"name": "ZIP", "valueString": "1010"}]}]
+        }
+        
+        from ..main import _check_postcode_filter
+        result = await _check_postcode_filter(ad_dict, kw_row)
+        assert result is True
+    
+    async def test_fail_when_not_matching(self):
+        kw_row = {"allowed_postcodes": ["1010", "1020"]}
+        
+        ad_dict = {
+            "attributes": [{"name": "LOCATION", "items": [
+                {"name": "ZIP", "valueString": "8010"}]}]  # Graz — not in allowed list
+        }
+        
+        from ..main import _check_postcode_filter
+        result = await _check_postcode_filter(ad_dict, kw_row)
+        assert result is False
+    
+    async def test_fail_when_no_postcode_in_ad(self):
+        kw_row = {"allowed_postcodes": ["1010", "1020"]}
+        
+        ad_dict = {
+            "attributes": []  # no location info
+        }
+        
+        from ..main import _check_postcode_filter
+        result = await _check_postcode_filter(ad_dict, kw_row)
+        assert result is False
+
+
+class TestMuteHours:
+    
+    async def test_no_mute_when_not_configured(self, mock_pool):
+        mock_pool.fetchrow.return_value = None
+        
+        from ..notifier import _is_in_mute_hours
+        result = await _is_in_mute_hours("298181113", mock_pool)
+        
+        assert result is False
+    
+    async def test_no_mute_outside_window(self, mock_pool):
+        from datetime import time
+        
+        mock_pool.fetchrow.return_value = {
+            "mute_start": time(22, 0),   # 10 PM
+            "mute_end": time(7, 0),       # 7 AM
+        }
+        
+        # Mock current time to noon UTC (outside mute window)
+        with patch("notifier.datetime") as mock_dt:
+            from datetime import datetime, timezone
+            
+            mock_dt.now.return_value = datetime(2026, 1, 1, 12, 0, tzinfo=timezone.utc)
+            
+            from ..notifier import _is_in_mute_hours
+            result = await _is_in_mute_hours("298181113", mock_pool)
+            
+            assert result is False
+    
+    async def test_muted_during_window(self, mock_pool):
+        from datetime import time
+        
+        mock_pool.fetchrow.return_value = {
+            "mute_start": time(22, 0),
+            "mute_end": time(7, 0),
+        }
+        
+        with patch("notifier.datetime") as mock_dt:
+            from datetime import datetime, timezone
+            
+            mock_dt.now.return_value = datetime(2026, 1, 1, 3, 0, tzinfo=timezone.utc)
+            
+            from ..notifier import _is_in_mute_hours
+            result = await _is_in_mute_hours("298181113", mock_pool)
+            
+            assert result is True
+
+
+class TestNotificationQueue:
+    
+    async def test_enqueue_on_failure(self, mock_pool):
+        from ..notifier import _enqueue_retry
+        
+        await _enqueue_retry(
+            ad_id="ad-uuid-here",
+            telegram_id="298181113",
+            message_text="Test notification",
+            notif_type="new",
+            error_msg="Telegram API timeout",
+        )
+        
+        mock_pool.execute.assert_called_once()
+    
+    async def test_no_duplicate_enqueue(self, mock_pool):
+        """Same ad+user should not create duplicate queue entries."""
+        mock_pool.fetchval.return_value = "already-exists-uuid"
+        
+        from ..notifier import _enqueue_retry
+        
+        await _enqueue_retry(
+            ad_id="ad-uuid-here",
+            telegram_id="298181113",
+            message_text="Test",
+            notif_type="new",
+            error_msg="timeout",
+        )
+        
+        # fetchval called (to check), but execute NOT called (no insert)
+        mock_pool.fetchval.assert_called_once()
+        mock_pool.execute.assert_not_called()
+```
+
+### 5. Create `worker/tests/test_scheduler.py`
+
+```python
+import pytest
+from unittest.mock import AsyncMock, patch
+
+
+class TestSchedulerFlow:
+    
+    @pytest.mark.asyncio
+    async def test_process_notification_queue_retries_pending(self, mock_pool):
+        """Pending items should be retried if backoff period has elapsed."""
+        from datetime import datetime, timezone
+        
+        mock_pool.fetch.return_value = [
+            {
+                "id": "queue-item-1",
+                "ad_id": "ad-uuid",
+                "telegram_id": "298181113",
+                "message_text": "Retry this ad",
+                "type": "new",
+                "attempts": 0,
+                "max_attempts": 5,
+                "last_error": "timeout",
+                "updated_at": datetime(2026, 7, 4, 10, 0, tzinfo=timezone.utc),
+            },
+        ]
+        
+        with patch("main.get_application_bot") as mock_get_bot:
+            mock_bot = AsyncMock()
+            mock_bot.send_message = AsyncMock()
+            mock_get_bot.return_value = mock_bot
+            
+            from ..main import process_notification_queue
+            result = await process_notification_queue()
+            
+            assert result == 1
+    
+    @pytest.mark.asyncio
+    async def test_process_notification_queue_dead_after_max_attempts(self, mock_pool):
+        """Items exceeding max_attempts should be marked as dead."""
+        from datetime import datetime, timezone
+        
+        mock_pool.fetch.return_value = [
+            {
+                "id": "queue-item-2",
+                "ad_id": "ad-uuid",
+                "telegram_id": "298181113",
+                "message_text": "Will fail again",
+                "type": "new",
+                "attempts": 5,  # already at max
+                "max_attempts": 5,
+                "last_error": "user blocked bot",
+                "updated_at": datetime(2026, 7, 4, 10, 0, tzinfo=timezone.utc),
+            },
+        ]
+        
+        with patch("main.get_application_bot") as mock_get_bot:
+            import telegram.error
+            mock_bot = AsyncMock()
+            mock_bot.send_message = AsyncMock(
+                side_effect=telegram.error.TelegramError("blocked")
+            )
+            mock_get_bot.return_value = mock_bot
+            
+            from ..main import process_notification_queue
+            await process_notification_queue()
+            
+            # Should have updated to 'dead' status
+            calls = [c[0] for c in mock_pool.execute.call_args_list]
+            assert any("status = 'dead'" in str(c) or "status=$2" in str(c) 
+                       for c in calls), "Item should be marked as dead"
+
+
+class TestDigestFlushing:
+    
+    @pytest.mark.asyncio
+    async def test_flush_digest_buffers(self, mock_pool):
+        """Buffered items older than interval should be flushed."""
+        from datetime import datetime, timezone
+        
+        mock_pool.fetch.side_effect = [
+            # First fetch: get digest-enabled users
+            [{"telegram_id": "298181113", "digest_interval": 60}],
+            # Second fetch: get buffered items
+            [
+                {"id": "buf-1", "keyword": "rtx 3090", 
+                 "title": "Ad 1", "price": 75000, 
+                 "url": "https://...", "ad_id": "ad-uuid"},
+                {"id": "buf-2", "keyword": "rtx 3090", 
+                 "title": "Ad 2", "price": 68000, 
+                 "url": "https://...", "ad_id": "ad-uuid-2"},
+            ],
+        ]
+        
+        with patch("main.get_application_bot") as mock_get_bot:
+            mock_bot = AsyncMock()
+            mock_bot.send_message = AsyncMock()
+            mock_get_bot.return_value = mock_bot
+            
+            from ..main import flush_digest_buffers
+            result = await flush_digest_buffers()
+            
+            assert result == 1  # one digest sent
+    
+    @pytest.mark.asyncio
+    async def test_flush_empty_buffer(self, mock_pool):
+        """No buffered items should result in no action."""
+        mock_pool.fetch.side_effect = [
+            [{"telegram_id": "298181113", "digest_interval": 60}],
+            [],  # empty buffer
+        ]
+        
+        from ..main import flush_digest_buffers
+        result = await flush_digest_buffers()
+        
+        assert result == 0
+
+
+class TestGracefulShutdown:
+    
+    @pytest.mark.asyncio
+    async def test_cleanup_stops_scheduler_and_closes_pool(self):
+        """Cleanup should cancel scheduler, stop bot, close DB pool."""
+        from telegram.ext import Application
+        
+        app = AsyncMock(spec=Application)
+        app.updater.running = True
+        
+        with patch("main._scheduler_task") as mock_task:
+            mock_task.done.return_value = False
+            
+            from ..main import cleanup
+            await cleanup(app)
+            
+            mock_task.cancel.assert_called_once()
+            app.updater.stop_polling.assert_called_once()
+```
+
+### 6. Create `worker/tests/test_health.py`
+
+```python
+import pytest
+
+
+@pytest.mark.asyncio
+async def test_health_endpoint_ok():
+    """Health endpoint should return 200 when system is healthy."""
+    from health import create_health_app
+    
+    app = create_health_app()
+    
+    with patch("health.get_pool") as mock_pool_get:
+        mock_pool = AsyncMock()
+        mock_pool.fetchval = AsyncMock(return_value=1)
+        mock_pool_get.return_value = mock_pool
+        
+        runner = web.AppRunner(app)
+        await runner.setup()
+        
+        try:
+            from aiohttp.test_utils import TestClient, TestServer
+            client = TestClient(TestServer(runner))
+            async with client:
+                resp = await client.get("/health")
+                assert resp.status == 200
+                
+                data = await resp.json()
+                assert data["status"] == "ok"
+                assert "db_connected" in data
+        finally:
+            await runner.cleanup()
+
+
+@pytest.mark.asyncio
+async def test_health_endpoint_unhealthy_db():
+    """Health endpoint should return 503 when DB is unreachable."""
+    from health import create_health_app
+    
+    app = create_health_app()
+    
+    with patch("health.get_pool") as mock_pool_get:
+        mock_pool_get.side_effect = Exception("connection refused")
+        
+        runner = web.AppRunner(app)
+        await runner.setup()
+        
+        try:
+            from aiohttp.test_utils import TestClient, TestServer
+            client = TestClient(TestServer(runner))
+            async with client:
+                resp = await client.get("/health")
+                assert resp.status == 503
+                
+                data = await resp.json()
+                assert data["status"] == "unhealthy"
+        finally:
+            await runner.cleanup()
+```
+
+### 7. Create GitHub Actions workflow `.github/workflows/ci.yml`
+
+```yaml
+name: CI — Lint & Test
+
+on:
+  push:
+    branches: [main, 'feat/*']
+  pull_request:
+    branches: [main]
+
+jobs:
+  lint-and-test:
+    runs-on: ubuntu-latest
+    
+    steps:
+      - name: Checkout code
+        uses: actions/checkout@v4
+      
+      - name: Set up Python 3.12
+        uses: actions/setup-python@v5
+        with:
+          python-version: '3.12'
+      
+      - name: Install dependencies
+        run: |
+          pip install --upgrade pip
+          pip install -r worker/requirements.txt
+          pip install -r worker/requirements-test.txt
+      
+      - name: Lint with flake8
+        run: |
+          flake8 worker/src/ --count --select=E9,F63,F7,F82 --show-source --statistics
+          flake8 worker/tests/ --count --select=E9,F63,F7,F82 --show-source --statistics
+      
+      - name: Test with pytest + coverage
+        run: |
+          cd worker
+          python -m pytest tests/ --cov=src --cov-report=term-missing --cov-fail-under=80
+
+  build-docker:
+    runs-on: ubuntu-latest
+    needs: lint-and-test
+    
+    steps:
+      - name: Checkout code
+        uses: actions/checkout@v4
+      
+      - name: Build Docker image
+        run: docker compose -f docker-compose.yml build worker
+      
+      - name: Test healthcheck
+        run: |
+          docker compose up -d worker
+          sleep 5
+          docker inspect --format='{{.State.Health.Status}}' willhaben-tracker-worker-1 || true
+          docker compose down
+```
+
+## Acceptance Criteria
+
+- [ ] `pytest` runs with 0 failures and ≥80% coverage on all source files
+- [ ] GitHub Actions pipeline passes on every push to `main` and feature branches
+- [ ] Tests cover: scraper parsing, pagination, price filters, postcode filters, mute hours, notification retry queue, digest flushing, graceful shutdown, healthcheck endpoint
+- [ ] Flake8 linting (error-level checks) passes in CI
+- [ ] Docker image builds successfully after all tests pass
+- [ ] Adding a new test file automatically includes it in the coverage report

docs/phase-3/task-web-ui.md

@@ -0,0 +1,77 @@
+# Task: Web Dashboard (FastAPI + Jinja2)
+
+## Description
+
+Currently, the only way to monitor the system is via Telegram bot commands or SSH into the server. This task adds a read-only web dashboard for real-time visibility into keywords, ads, users, and stats.
+
+## Architecture
+
+```
+┌──────────────────────────────────────────────┐
+│  FastAPI App (port 8766)                     │
+│                                              │
+│  Auth: Basic Auth (WEB_UI_USERNAME/PASSWORD) │
+│  Templates: Jinja2 with inline CSS           │
+│                                              │
+│  Routes:                                     │
+│  GET /             → Dashboard               │
+│  GET /keywords     → Keywords list           │
+│  GET /keywords/<id> → Keyword detail         │
+│  GET /users        → Users list              │
+│  GET /ads          → Recent ads              │
+│  GET /stats        → JSON stats              │
+└──────────┬───────────────────────────────────┘
+           │
+           ▼
+┌──────────────────────────────────────────────┐
+│  PostgreSQL (asyncpg pool)                   │
+│                                              │
+│  Queries:                                    │
+│  - Keywords with status, filters, subs       │
+│  - Recent ads with price, location           │
+│  - Users with mute/digest settings           │
+│  - Stats (counts, queue status)              │
+└──────────────────────────────────────────────┘
+```
+
+## Implementation Details
+
+### 1. Add dependencies
+
+In `worker/requirements.txt`:
+```
+fastapi==0.115.0
+uvicorn==0.30.0
+jinja2==3.1.4
+```
+
+### 2. Create `worker/src/web.py`
+
+- FastAPI app with Jinja2 template engine
+- Basic auth middleware using `WEB_UI_USERNAME` / `WEB_UI_PASSWORD` env vars
+- Routes that query the DB via `get_pool()` from `db.py`
+- Each route returns HTML via Jinja2 templates
+
+### 3. Create `worker/src/templates/`
+
+- `base.html` — Base layout with sidebar navigation, dark theme
+- `dashboard.html` — Keywords overview + stats summary cards
+- `keywords.html` — Table of keywords with status, filters, subscribers
+- `keyword_detail.html` — Keyword detail with recent ads, price history, scrape logs
+- `users.html` — Users list with mute/digest settings
+- `ads.html` — Recent ads with search/filter
+
+### 4. Integrate into `main.py`
+
+- Start uvicorn server on port 8766 alongside existing aiohttp health server on 8765
+- Graceful shutdown includes web server cleanup
+
+## Acceptance Criteria
+
+- [ ] Web UI accessible at `http://<host>:8766` with basic auth
+- [ ] Dashboard shows keywords with status, filters, subscribers, last scrape
+- [ ] Dashboard shows recent ads with price, location, keyword
+- [ ] Dashboard shows users with mute/digest settings
+- [ ] Dashboard shows stats (ads indexed, notifications sent, queue status)
+- [ ] Health server still works on port 8765 (no regression)
+- [ ] Telegram bot still works (no regression)

pyproject.toml

@@ -0,0 +1,23 @@
+[project]
+name = "willhaben-tracker"
+version = "0.1.0"
+description = "Telegram bot that tracks willhaben.at listings"
+requires-python = ">=3.11"
+
+[tool.pytest.ini_options]
+testpaths = ["tests"]
+asyncio_mode = "auto"
+asyncio_default_fixture_loop_scope = "function"
+pythonpath = ["src"]
+
+[tool.coverage.run]
+source = ["."]
+omit = ["tests/*", "migrate.py", "entrypoint.sh", "bot.py", "web.py", "main.py"]
+
+[tool.coverage.report]
+fail_under = 50
+show_missing = true
+
+[tool.flake8]
+max-line-length = 120
+exclude = [".git", "__pycache__", "node_modules"]

supabase/migrations/supabase-migration.sql

@@ -1,5 +1,5 @@
-CREATE SCHEMA IF NOT EXISTS willhaben_tracker;
-SET search_path TO willhaben_tracker;
+-- supabase-migration.sql — DEPRECATED: use 01-schema.sql instead (in migrations/)
+-- This file is kept for backward compatibility reference only.
 
 -- -----------------------------------------------------------
 -- users (whitelisted Telegram users)

worker/Dockerfile

@@ -1,6 +1,21 @@
 FROM python:3.12-slim
 WORKDIR /app
+
+# ── Dependencies ────────────────────────────────
 COPY requirements.txt .
 RUN pip install --no-cache-dir -r requirements.txt
+
+# ── Application code + migrations + tests + templates ──
 COPY src/ .
+COPY tests/ tests/
+
+# Make entrypoint executable
+RUN chmod +x entrypoint.sh
+
+# ── Healthcheck (Docker-native) ─────────────────
+HEALTHCHECK --interval=15s --timeout=3s --retries=3 \
+    CMD python -c "import httpx,os; r=httpx.get(f'http://localhost:{os.getenv(\"HEALTH_PORT\",\"8765\")}/health'); r.raise_for_status()" || exit 1
+
+# ── Entrypoint: migrate then run ────────────────
+ENTRYPOINT ["/app/entrypoint.sh"]
 CMD ["python", "main.py"]

worker/requirements.txt

@@ -1,4 +1,13 @@
 python-telegram-bot==21.4
 asyncpg==0.30.0
 httpx==0.27.2
+aiohttp>=3.9,<4
 python-dotenv==1.0.1
+fastapi==0.115.0
+uvicorn==0.30.0
+jinja2==3.1.4
+
+# Test dependencies
+pytest==8.3.0
+pytest-asyncio==0.24.0
+flake8==7.1.0

worker/src/bot.py

@@ -189,10 +189,23 @@ def _format_kw_card(kw: dict) -> str:
     status_icon = "🟢 Active" if kw["is_active"] else "🔴 Stopped"
     subs_line = f"\nSubscribers: <code>{kw['subs']}</code>" if kw.get("subs", 1) > 1 else ""
 
+    price_line = ""
+    if kw.get("price_min") is not None or kw.get("price_max") is not None:
+        parts = []
+        if kw.get("price_min") is not None:
+            parts.append(f"€{kw['price_min'] / 100:.0f}")
+        if kw.get("price_max") is not None:
+            parts.append(f"€{kw['price_max'] / 100:.0f}")
+        price_line = f"\nPrice: <code>{'–'.join(parts)}</code>"
+    
+    postcode_line = ""
+    if kw.get("allowed_postcodes"):
+        postcode_line = f"\nPostcodes: <code>{', '.join(kw['allowed_postcodes'])}</code>"
+
     return (
         f"<b>🔍 {kw['keyword']}</b>\n"
         f"{status_icon} | Interval: <code>{kw['interval_minutes']} min</code>\n"
-        f"Last scrape: {_vienna_time(kw.get('last_scraped_at'))}{subs_line}"
+        f"Last scrape: {_vienna_time(kw.get('last_scraped_at'))}{price_line}{postcode_line}{subs_line}"
     )
 
 
@@ -226,12 +239,37 @@ async def setup_global_commands(app: Application) -> None:
     await app.bot.set_my_commands([
         ("start", "Open main menu"),
         ("admin", "Admin panel (admins only)"),
+        ("price_min", "Set min price: /price_min <kw> <€>"),
+        ("price_max", "Set max price: /price_max <kw> <€>"),
+        ("clear_price", "Remove price filter: /clear_price <kw>"),
+        ("postcode", "Set postcodes: /postcode <kw> p1,p2"),
+        ("clear_postcode", "Remove postcode filter: /clear_postcode <kw>"),
+        ("mute_hours", "Set mute window: /mute_hours HH:MM-HH:MM"),
+        ("mute_off", "Disable mute hours"),
+        ("digest_on", "Enable digest: /digest_on [minutes]"),
+        ("digest_off", "Disable digest mode"),
+        ("status", "Show your settings"),
     ])
 
 
 def register_handlers(app: Application) -> None:
     app.add_handler(CommandHandler("start", start_handler))
     app.add_handler(CommandHandler("admin", admin_handler))
+    # Phase 2: Price filters
+    app.add_handler(CommandHandler("price_min", price_min_handler))
+    app.add_handler(CommandHandler("price_max", price_max_handler))
+    app.add_handler(CommandHandler("clear_price", clear_price_handler))
+    # Phase 2: Postcode filters
+    app.add_handler(CommandHandler("postcode", postcode_handler))
+    app.add_handler(CommandHandler("clear_postcode", clear_postcode_handler))
+    # Phase 2: Mute hours
+    app.add_handler(CommandHandler("mute_hours", mute_hours_handler))
+    app.add_handler(CommandHandler("mute_off", mute_off_handler))
+    # Phase 2: Digest mode
+    app.add_handler(CommandHandler("digest_on", digest_on_handler))
+    app.add_handler(CommandHandler("digest_off", digest_off_handler))
+    # Phase 2: Status
+    app.add_handler(CommandHandler("status", status_handler))
     app.add_handler(CallbackQueryHandler(callback_router))
     # Catch all non-command text messages for conversation flows
     app.add_handler(MessageHandler(TEXT_FILTER, text_input_handler))
@@ -268,6 +306,326 @@ async def admin_handler(update: Update, context: ContextTypes.DEFAULT_TYPE) -> N
     )
 
 
+# ── Phase 2: Price filter commands ────────────────────────────────────────
+
+async def _find_keyword_for_user(pool, user_id: str, keyword_text: str) -> dict | None:
+    """Find a keyword matching the text that the user subscribes to (or any if admin)."""
+    # First try exact match for the user's keywords
+    row = await pool.fetchrow(
+        """SELECT kw.* FROM keywords kw
+           JOIN keyword_subscriptions ks ON ks.keyword_id = kw.id
+           WHERE LOWER(kw.keyword) = LOWER($1) AND ks.user_id = $2""",
+        keyword_text.lower(), user_id,
+    )
+    if row:
+        return dict(row)
+    
+    # Admin can access any keyword
+    row = await pool.fetchrow(
+        "SELECT * FROM keywords WHERE LOWER(keyword) = LOWER($1)",
+        keyword_text.lower(),
+    )
+    return dict(row) if row else None
+
+
+async def price_min_handler(update: Update, context: ContextTypes.DEFAULT_TYPE) -> None:
+    if not update.message or not update.message.text:  # type: ignore[union-attr]
+        return
+    
+    parts = update.message.text.split(maxsplit=2)  # type: ignore[union-attr]
+    if len(parts) < 3:
+        await update.message.reply_text("Usage: /price_min <keyword> <amount in €>")  # type: ignore[union-attr]
+        return
+    
+    keyword_text = parts[1]
+    try:
+        amount = float(parts[2])
+        if amount < 0:
+            raise ValueError
+    except ValueError:
+        await update.message.reply_text("Enter a valid positive amount.")  # type: ignore[union-attr]
+        return
+    
+    user = await _require_user(update)
+    if not user:
+        return
+    
+    pool = await get_pool()
+    kw = await _find_keyword_for_user(pool, user["id"], keyword_text)
+    if not kw:
+        await update.message.reply_text(f"Keyword '{keyword_text}' not found.")  # type: ignore[union-attr]
+        return
+    
+    price_cents = int(round(amount * 100))
+    await pool.execute("UPDATE keywords SET price_min = $1 WHERE id = $2", price_cents, kw["id"])
+    await update.message.reply_text(  # type: ignore[union-attr]
+        f"✅ <b>{kw['keyword']}</b>: min price set to €{amount:.2f}", parse_mode="HTML")
+
+
+async def price_max_handler(update: Update, context: ContextTypes.DEFAULT_TYPE) -> None:
+    if not update.message or not update.message.text:  # type: ignore[union-attr]
+        return
+    
+    parts = update.message.text.split(maxsplit=2)  # type: ignore[union-attr]
+    if len(parts) < 3:
+        await update.message.reply_text("Usage: /price_max <keyword> <amount in €>")  # type: ignore[union-attr]
+        return
+    
+    keyword_text = parts[1]
+    try:
+        amount = float(parts[2])
+        if amount < 0:
+            raise ValueError
+    except ValueError:
+        await update.message.reply_text("Enter a valid positive amount.")  # type: ignore[union-attr]
+        return
+    
+    user = await _require_user(update)
+    if not user:
+        return
+    
+    pool = await get_pool()
+    kw = await _find_keyword_for_user(pool, user["id"], keyword_text)
+    if not kw:
+        await update.message.reply_text(f"Keyword '{keyword_text}' not found.")  # type: ignore[union-attr]
+        return
+    
+    price_cents = int(round(amount * 100))
+    await pool.execute("UPDATE keywords SET price_max = $1 WHERE id = $2", price_cents, kw["id"])
+    await update.message.reply_text(  # type: ignore[union-attr]
+        f"✅ <b>{kw['keyword']}</b>: max price set to €{amount:.2f}", parse_mode="HTML")
+
+
+async def clear_price_handler(update: Update, context: ContextTypes.DEFAULT_TYPE) -> None:
+    if not update.message or not update.message.text:  # type: ignore[union-attr]
+        return
+    
+    parts = update.message.text.split(maxsplit=1)  # type: ignore[union-attr]
+    if len(parts) < 2:
+        await update.message.reply_text("Usage: /clear_price <keyword>")  # type: ignore[union-attr]
+        return
+    
+    user = await _require_user(update)
+    if not user:
+        return
+    
+    pool = await get_pool()
+    kw = await _find_keyword_for_user(pool, user["id"], parts[1])
+    if not kw:
+        await update.message.reply_text(f"Keyword '{parts[1]}' not found.")  # type: ignore[union-attr]
+        return
+    
+    await pool.execute("UPDATE keywords SET price_min = NULL, price_max = NULL WHERE id = $1", kw["id"])
+    await update.message.reply_text(  # type: ignore[union-attr]
+        f"✅ <b>{kw['keyword']}</b>: price filters cleared", parse_mode="HTML")
+
+
+# ── Phase 2: Postcode filter commands ─────────────────────────────────────
+
+async def postcode_handler(update: Update, context: ContextTypes.DEFAULT_TYPE) -> None:
+    if not update.message or not update.message.text:  # type: ignore[union-attr]
+        return
+    
+    parts = update.message.text.split(maxsplit=1)  # type: ignore[union-attr]
+    if len(parts) < 2:
+        await update.message.reply_text("Usage: /postcode <keyword> p1,p2,p3")  # type: ignore[union-attr]
+        return
+    
+    user = await _require_user(update)
+    if not user:
+        return
+    
+    postcodes = [p.strip() for p in parts[1].split(",")]
+    for p in postcodes:
+        if not p.isdigit() or len(p) != 4:
+            await update.message.reply_text(f"Invalid postcode '{p}'. Use 4-digit codes (e.g., 1010,1020).")  # type: ignore[union-attr]
+            return
+    
+    pool = await get_pool()
+    kw = await _find_keyword_for_user(pool, user["id"], parts[0])
+    if not kw:
+        await update.message.reply_text(f"Keyword '{parts[0]}' not found.")  # type: ignore[union-attr]
+        return
+    
+    await pool.execute("UPDATE keywords SET allowed_postcodes = $1 WHERE id = $2", postcodes, kw["id"])
+    await update.message.reply_text(  # type: ignore[union-attr]
+        f"✅ <b>{kw['keyword']}</b>: postcodes set to {', '.join(postcodes)}", parse_mode="HTML")
+
+
+async def clear_postcode_handler(update: Update, context: ContextTypes.DEFAULT_TYPE) -> None:
+    if not update.message or not update.message.text:  # type: ignore[union-attr]
+        return
+    
+    parts = update.message.text.split(maxsplit=1)  # type: ignore[union-attr]
+    if len(parts) < 2:
+        await update.message.reply_text("Usage: /clear_postcode <keyword>")  # type: ignore[union-attr]
+        return
+    
+    user = await _require_user(update)
+    if not user:
+        return
+    
+    pool = await get_pool()
+    kw = await _find_keyword_for_user(pool, user["id"], parts[1])
+    if not kw:
+        await update.message.reply_text(f"Keyword '{parts[1]}' not found.")  # type: ignore[union-attr]
+        return
+    
+    await pool.execute("UPDATE keywords SET allowed_postcodes = NULL WHERE id = $1", kw["id"])
+    await update.message.reply_text(  # type: ignore[union-attr]
+        f"✅ <b>{kw['keyword']}</b>: postcode filter cleared", parse_mode="HTML")
+
+
+# ── Phase 2: Mute hours commands ──────────────────────────────────────────
+
+async def mute_hours_handler(update: Update, context: ContextTypes.DEFAULT_TYPE) -> None:
+    if not update.message or not update.message.text:  # type: ignore[union-attr]
+        return
+    
+    parts = update.message.text.split(maxsplit=1)  # type: ignore[union-attr]
+    if len(parts) < 2:
+        await update.message.reply_text("Usage: /mute_hours HH:MM-HH:MM (UTC)")  # type: ignore[union-attr]
+        return
+    
+    try:
+        start_str, end_str = parts[1].split("-", 1)
+        # Validate time format
+        datetime.strptime(start_str, "%H:%M")
+        datetime.strptime(end_str, "%H:%M")
+    except (ValueError, AttributeError):
+        await update.message.reply_text("Usage: /mute_hours HH:MM-HH:MM (UTC), e.g. /mute_hours 22:00-07:00")  # type: ignore[union-attr]
+        return
+    
+    user = await _require_user(update)
+    if not user:
+        return
+    
+    pool = await get_pool()
+    await pool.execute(
+        """INSERT INTO user_settings (telegram_id, mute_start, mute_end) VALUES ($1, $2, $3)
+           ON CONFLICT (telegram_id) DO UPDATE SET mute_start = $2, mute_end = $3""",
+        str(user["telegram_id"]), start_str, end_str,
+    )
+    await update.message.reply_text(  # type: ignore[union-attr]
+        f"✅ Mute hours set: {start_str}–{end_str} UTC", parse_mode="HTML")
+
+
+async def mute_off_handler(update: Update, context: ContextTypes.DEFAULT_TYPE) -> None:
+    user = await _require_user(update)
+    if not user:
+        return
+    
+    pool = await get_pool()
+    await pool.execute(
+        """INSERT INTO user_settings (telegram_id, mute_start, mute_end) VALUES ($1, NULL, NULL)
+           ON CONFLICT (telegram_id) DO UPDATE SET mute_start = NULL, mute_end = NULL""",
+        str(user["telegram_id"]),
+    )
+    msg = update.message or update.callback_query  # type: ignore[union-attr]
+    await msg.reply_text("✅ Mute hours disabled.", parse_mode="HTML")
+
+
+# ── Phase 2: Digest mode commands ─────────────────────────────────────────
+
+async def digest_on_handler(update: Update, context: ContextTypes.DEFAULT_TYPE) -> None:
+    if not update.message or not update.message.text:  # type: ignore[union-attr]
+        return
+    
+    parts = update.message.text.split(maxsplit=1)  # type: ignore[union-attr]
+    interval = 60  # default
+    
+    if len(parts) > 1:
+        try:
+            interval = int(parts[1])
+            if interval < 5 or interval > 1440:
+                raise ValueError
+        except ValueError:
+            await update.message.reply_text("Enter a number between 5 and 1440 minutes.")  # type: ignore[union-attr]
+            return
+    
+    user = await _require_user(update)
+    if not user:
+        return
+    
+    pool = await get_pool()
+    await pool.execute(
+        """INSERT INTO user_settings (telegram_id, digest_mode, digest_interval) VALUES ($1, true, $2)
+           ON CONFLICT (telegram_id) DO UPDATE SET digest_mode = true, digest_interval = $2""",
+        str(user["telegram_id"]), interval,
+    )
+    await update.message.reply_text(  # type: ignore[union-attr]
+        f"✅ Digest mode enabled (every {interval} min)", parse_mode="HTML")
+
+
+async def digest_off_handler(update: Update, context: ContextTypes.DEFAULT_TYPE) -> None:
+    user = await _require_user(update)
+    if not user:
+        return
+    
+    pool = await get_pool()
+    # Flush any pending digest items before disabling
+    buffered = await pool.fetch(
+        "SELECT db.id, db.keyword, db.title, db.price, db.url FROM digest_buffer db WHERE db.telegram_id = $1",
+        str(user["telegram_id"]),
+    )
+    
+    if buffered:
+        # Send immediate summary of pending items
+        by_keyword = {}
+        for item in buffered:
+            by_keyword.setdefault(item["keyword"], []).append(item)
+        
+        lines = ["📦 <b>Pending Digest Summary</b>"]
+        for keyword, items in by_keyword.items():
+            lines.append(f"\n<b>🔍 {keyword}</b> ({len(items)} ads)")
+            for item in items:
+                price_str = f"€{item['price'] / 100:.0f}" if item["price"] else "N/A"
+                lines.append(f"  • {item['title']} — {price_str}")
+        
+        try:
+            await update.message.reply_text("\n".join(lines), parse_mode="HTML")  # type: ignore[union-attr]
+        except Exception:
+            pass
+        await pool.execute("DELETE FROM digest_buffer WHERE telegram_id = $1", str(user["telegram_id"]))
+    
+    await pool.execute(
+        """INSERT INTO user_settings (telegram_id, digest_mode) VALUES ($1, false)
+           ON CONFLICT (telegram_id) DO UPDATE SET digest_mode = false""",
+        str(user["telegram_id"]),
+    )
+    msg = update.message or update.callback_query  # type: ignore[union-attr]
+    await msg.reply_text("✅ Digest mode disabled.", parse_mode="HTML")
+
+
+# ── Phase 2: Status command ───────────────────────────────────────────────
+
+async def status_handler(update: Update, context: ContextTypes.DEFAULT_TYPE) -> None:
+    user = await _require_user(update)
+    if not user:
+        return
+    
+    pool = await get_pool()
+    settings = await pool.fetchrow(
+        "SELECT mute_start, mute_end, digest_mode, digest_interval FROM user_settings WHERE telegram_id = $1",
+        str(user["telegram_id"]),
+    )
+    
+    lines = ["<b>⚙️ Your Settings</b>"]
+    
+    if settings and settings["mute_start"]:
+        lines.append(f"\n🔇 Mute hours: {settings['mute_start']}–{settings['mute_end']} UTC")
+    else:
+        lines.append("\n🔇 Mute hours: off")
+    
+    if settings and settings["digest_mode"]:
+        lines.append(f"📦 Digest: on (every {settings['digest_interval']} min)")
+    else:
+        lines.append("📦 Digest: off")
+    
+    msg = update.message or update.callback_query  # type: ignore[union-attr]
+    await msg.reply_text("\n".join(lines), parse_mode="HTML")
+
+
 # ── text input handler (keyword name, custom interval, admin flows) ───────
 
 async def text_input_handler(update: Update, context: ContextTypes.DEFAULT_TYPE) -> None:

worker/src/db.py

@@ -7,22 +7,18 @@ logger = logging.getLogger(__name__)
 _pool: asyncpg.Pool | None = None
 
 
-async def _init_connection(conn: asyncpg.Connection) -> None:
-    await conn.execute("SET search_path TO willhaben_tracker")
-
 
 async def get_pool() -> asyncpg.Pool:
     global _pool
     if _pool is None:
         _pool = await asyncpg.create_pool(
-            host=os.getenv("POSTGRES_HOST", "db"),
+            host=os.getenv("POSTGRES_HOST", "192.168.178.3"),
             port=int(os.getenv("POSTGRES_PORT", "5432")),
             user=os.getenv("POSTGRES_USER", "postgres"),
             password=os.getenv("POSTGRES_PASSWORD"),
             database=os.getenv("POSTGRES_DB", "postgres"),
             min_size=2,
             max_size=10,
-            init=_init_connection
         )
         logger.info("Database pool initialized")
     return _pool

worker/src/entrypoint.sh

@@ -0,0 +1,38 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+# ────────────────────────────────────────────────
+# Entrypoint: Run migrations, then start the app.
+# ────────────────────────────────────────────────
+
+echo "[entrypoint] Waiting for database to be ready..."
+
+# Wait for PostgreSQL to accept connections (max 30s retries)
+for i in $(seq 1 30); do
+    if python -c "import asyncpg,os,asyncio; asyncio.run(asyncpg.connect(
+        host=os.getenv('POSTGRES_HOST','192.168.178.3'),
+        port=int(os.getenv('POSTGRES_PORT','5432')),
+        user=os.getenv('POSTGRES_USER','postgres'),
+        password=os.getenv('POSTGRES_PASSWORD'),
+        database=os.getenv('POSTGRES_DB','postgres'),
+    ))" 2>/dev/null; then
+        echo "[entrypoint] Database is ready."
+        break
+    fi
+    if [ "$i" -eq 30 ]; then
+        echo "[entrypoint] ERROR: Could not connect to database after 30 attempts." >&2
+        exit 1
+    fi
+    sleep 1
+done
+
+# ── Run migrations ──────────────────────────────
+echo "[entrypoint] Running database migrations..."
+python /app/migrate.py || {
+    echo "[entrypoint] ERROR: Migration failed — aborting startup." >&2
+    exit 1
+}
+echo "[entrypoint] Migrations complete."
+
+# ── Start the application ───────────────────────
+exec "$@"

worker/src/health.py

@@ -0,0 +1,135 @@
+#!/usr/bin/env python3
+"""Lightweight HTTP healthcheck server for Docker integration.
+
+Exposes:
+  GET /health — returns 200 OK or 503 with subsystem status JSON
+  GET /stats  — extended DB counts (keywords, ads, notifications)
+
+Runs on port HEALTH_PORT (default: 8765).
+"""
+
+import os
+import time
+from typing import Any
+
+from aiohttp import web
+
+logger = None  # lazy-imported to avoid circular imports
+
+_start_time: float | None = None
+_last_scheduler_run: float | None = None
+_telegram_polling: bool = False
+
+
+def _get_logger() -> "logging.Logger":  # type: ignore[name-defined]
+    global logger
+    if logger is None:
+        import logging as _log
+        logger = _log.getLogger(__name__)
+    return logger
+
+
+# ────────────────────────────────────────────────
+# Lifecycle hooks — called from main.py
+# ────────────────────────────────────────────────
+
+def set_start_time() -> None:
+    global _start_time
+    _start_time = time.time()
+
+
+def record_scheduler_run() -> None:
+    """Call this at the start of each scheduler cycle."""
+    global _last_scheduler_run
+    _last_scheduler_run = time.time()
+
+
+def set_telegram_polling(active: bool) -> None:
+    global _telegram_polling
+    _telegram_polling = active
+
+
+# ────────────────────────────────────────────────
+# Health endpoint — fast, non-blocking DB probe
+# ────────────────────────────────────────────────
+
+async def health_handler(request: web.Request) -> web.Response:  # noqa: ARG001
+    status = "ok"
+    checks: dict[str, Any] = {}
+
+    if _start_time is not None:
+        checks["uptime_seconds"] = int(time.time() - _start_time)
+
+    # — DB connectivity (lightweight SELECT 1) —
+    try:
+        from db import get_pool
+        pool = await get_pool()
+        await pool.fetchval("SELECT 1")
+        checks["db_connected"] = True
+    except Exception as exc:
+        status = "unhealthy"
+        checks["db_error"] = str(exc)
+
+    # — Scheduler staleness —
+    stale_threshold = int(os.getenv("HEALTHCHECK_SCHEDULER_STALE_S", "300"))
+    if _last_scheduler_run is not None:
+        elapsed = time.time() - _last_scheduler_run
+        checks["scheduler_last_run_seconds_ago"] = round(elapsed, 1)
+        if elapsed > stale_threshold:
+            status = "unhealthy"
+            checks["scheduler_stale"] = True
+
+    # — Telegram polling status —
+    checks["telegram_polling"] = _telegram_polling
+
+    body = {"status": status, **checks}
+
+    code = 200 if status == "ok" else 503
+    return web.json_response(body, status=code)
+
+
+# ────────────────────────────────────────────────
+# Stats endpoint — DB counts (slower query)
+# ────────────────────────────────────────────────
+
+async def stats_handler(request: web.Request) -> web.Response:  # noqa: ARG001
+    from db import get_pool
+    pool = await get_pool()
+
+    kw_count = await pool.fetchval("SELECT COUNT(*) FROM keywords") or 0
+    active_kw = await pool.fetchval(
+        "SELECT COUNT(*) FROM keywords WHERE is_active"
+    ) or 0
+    ad_count = await pool.fetchval("SELECT COUNT(*) FROM ads") or 0
+    notif_count = await pool.fetchval("SELECT COUNT(*) FROM notifications") or 0
+
+    try:
+        pending_count = await pool.fetchval(
+            "SELECT COUNT(*) FROM notification_queue WHERE status IN ('pending', 'failed')"
+        ) or 0
+        dead_count = await pool.fetchval(
+            "SELECT COUNT(*) FROM notification_queue WHERE status = 'dead'"
+        ) or 0
+    except Exception:
+        pending_count = 0
+        dead_count = 0
+
+    return web.json_response({
+        "keywords": kw_count,
+        "active_keywords": active_kw,
+        "ads_indexed": ad_count,
+        "notifications_sent": notif_count,
+        "queue_pending": pending_count,
+        "queue_dead": dead_count,
+    })
+
+
+# ────────────────────────────────────────────────
+# App factory — creates a runnable aiohttp app
+# ────────────────────────────────────────────────
+
+def create_health_app() -> web.Application:
+    app = web.Application()
+    app.router.add_get("/health", health_handler)
+    app.router.add_get("/stats", stats_handler)
+    return app

worker/src/main.py

@@ -4,34 +4,267 @@ import logging
 import os
 import signal
 import sys
+from collections import defaultdict
 from contextlib import suppress
+from datetime import datetime, timedelta, timezone
 
+import aiohttp.web as web
+import asyncpg
+import httpx
 from dotenv import load_dotenv
-from telegram import Update
 from telegram.ext import Application, ExtBot
+from telegram.request import HTTPXRequest
 
 from db import close_pool, get_pool
+from health import create_health_app, record_scheduler_run, set_start_time, set_telegram_polling
 from scraper import extract_ad_fields, fetch_ads
-from notifier import log_notification, notify_new_ad, notify_price_drop
+from notifier import log_notification, notify_new_ad, notify_price_drop, is_user_muted, buffer_for_digest
+from settings import get_proxy_enabled, get_turbo_mode, proxy_available
 
 logger = logging.getLogger(__name__)
 
 load_dotenv()
 
 
-async def scheduler_task(pool: object, bot: ExtBot) -> None:
-    while True:
-        try:
-            rows = await pool.fetch(
-                "SELECT id, keyword, interval_minutes, initial_loaded FROM keywords "
-                "WHERE is_active = true "
-                "AND (last_scraped_at IS NULL OR last_scraped_at < now() - (interval_minutes || ' minutes')::interval)"
-            )
+class DirectHTTPXRequest(HTTPXRequest):
+    def _build_client(self) -> httpx.AsyncClient:
+        return httpx.AsyncClient(**self._client_kwargs, trust_env=False)
+
+
+def _ad_passes_filters(fields: dict, kw_row: dict) -> bool:
+    """Check if an ad passes the keyword's price and postcode filters."""
+    price = fields.get("price")
+
+    # Price filter (stored in cents, ad price is in euros as float)
+    if price is not None:
+        price_cents = int(round(price * 100))
+        price_min = kw_row.get("price_min")
+        price_max = kw_row.get("price_max")
+        if price_min is not None and price_cents < price_min:
+            return False
+        if price_max is not None and price_cents > price_max:
+            return False
+
+    # Postcode filter
+    allowed_postcodes = kw_row.get("allowed_postcodes")
+    if allowed_postcodes is not None:
+        ad_postcode = fields.get("postcode")
+        if ad_postcode is None or ad_postcode not in allowed_postcodes:
+            return False
+
+    return True
+
+
+async def process_notification_queue(pool: asyncpg.Pool, bot: ExtBot) -> int:
+    """Process pending notifications from the retry queue."""
+    rows = await pool.fetch("""
+        SELECT id, ad_id, telegram_id, message_text, type, attempts, 
+               max_attempts, last_error, updated_at, created_at
+        FROM notification_queue
+        WHERE status IN ('pending', 'failed')
+        ORDER BY attempts ASC, created_at ASC
+        LIMIT 200
+    """)
+
+    # Log queue depth
+    queue_stats = await pool.fetchrow("SELECT count(*) FROM notification_queue WHERE status IN ('pending', 'failed')")
+    total_queued = queue_stats["count"]
+    logger.info("Notification queue: %d total, processing up to %d", total_queued, len(rows))
+
+    processed = 0
 
     for row in rows:
+        # Pending (never tried) — send immediately
+        if row["status"] == "pending":
+            backoff_min = 0
+        else:
+            # Failed — exponential backoff from created_at (fixed timeline)
+            backoff_min = min(2 ** row["attempts"], 60)
+
+        retry_after = row["created_at"] + timedelta(minutes=backoff_min)
+
+        if datetime.now(tz=timezone.utc) < retry_after:
+            logger.debug("Skipping notification %s — retry after %s (now %s)", 
+                         row["id"], retry_after, datetime.now(tz=timezone.utc))
+            continue
+
+        try:
+            await bot.send_message(
+                chat_id=int(row["telegram_id"]),
+                text=row["message_text"],
+            )
+
+            await pool.execute(
+                "UPDATE notification_queue SET status = 'sent', updated_at = now() WHERE id = $1",
+                row["id"],
+            )
+
+            user_row = await pool.fetchrow("SELECT id FROM users WHERE telegram_id = $1", row["telegram_id"])
+            if user_row:
+                try:
+                    await log_notification(pool, str(user_row["id"]), str(row["ad_id"]), 0)
+                except Exception:
+                    pass
+
+            processed += 1
+
+        except Exception as e:
+            new_attempts = row["attempts"] + 1
+
+            if new_attempts >= row["max_attempts"]:
+                await pool.execute(
+                    """UPDATE notification_queue 
+                       SET status = 'dead', attempts = $2, last_error = $3, updated_at = now()
+                       WHERE id = $1""",
+                    row["id"], new_attempts, str(e)[:300],
+                )
+                logger.error(
+                    "Notification DEAD after %d attempts: ad=%s user=%s err=%s",
+                    new_attempts, str(row["ad_id"])[:8], row["telegram_id"], e,
+                )
+            else:
+                await pool.execute(
+                    """UPDATE notification_queue 
+                       SET status = 'failed', attempts = $2, last_error = $3, updated_at = now()
+                       WHERE id = $1""",
+                    row["id"], new_attempts, str(e)[:300],
+                )
+
+    return processed
+
+
+async def flush_digests(pool: asyncpg.Pool, bot: ExtBot) -> int:
+    """Flush digest buffers for users whose interval has elapsed."""
+    users = await pool.fetch("""
+        SELECT us.telegram_id, us.digest_interval, us.last_digest_flush
+        FROM user_settings us
+        WHERE us.digest_mode = true
+        AND (us.last_digest_flush IS NULL 
+             OR us.last_digest_flush < now() - (us.digest_interval || ' minutes')::interval)
+    """)
+    
+    if not users:
+        return 0
+    
+    flushed = 0
+    
+    for user_row in users:
+        tg_id = user_row["telegram_id"]
+        
+        # Get all buffered notifications for this user
+        buffered = await pool.fetch("""
+            SELECT db.id, db.ad_id, db.keyword, db.title, db.price, db.url
+            FROM digest_buffer db
+            WHERE db.telegram_id = $1
+            ORDER BY db.created_at DESC
+        """, tg_id)
+        
+        if not buffered:
+            # Update flush time even if no items (to keep tracking)
+            await pool.execute(
+                """INSERT INTO user_settings (telegram_id, last_digest_flush) VALUES ($1, now())
+                   ON CONFLICT (telegram_id) DO UPDATE SET last_digest_flush = now()""",
+                tg_id)
+            continue
+        
+        # Build digest message grouped by keyword
+        by_keyword = defaultdict(list)
+        for item in buffered:
+            by_keyword[item["keyword"]].append(item)
+        
+        lines = ["📦 <b>Digest Summary</b>"]
+        
+        for keyword, items in by_keyword.items():
+            lines.append(f"\n<b>🔍 {keyword}</b> ({len(items)} ad{'s' if len(items) != 1 else ''})")
+            for item in items[:20]:  # Limit to 20 ads per keyword to avoid message too long
+                price_str = f"€{item['price'] / 100:.0f}" if item["price"] else "N/A"
+                lines.append(f"  • {item['title']} — {price_str}")
+        
+        text = "\n".join(lines)
+        
+        try:
+            await bot.send_message(
+                chat_id=int(tg_id),
+                text=text,
+                parse_mode="HTML",
+            )
+            flushed += 1
+            logger.info("Sent digest to %s (%d items)", tg_id, len(buffered))
+        except Exception as e:
+            logger.error("Failed to send digest to %s: %s", tg_id, e)
+        
+        # Clear buffered items and update flush time
+        await pool.execute("DELETE FROM digest_buffer WHERE telegram_id = $1", tg_id)
+        await pool.execute(
+            """INSERT INTO user_settings (telegram_id, last_digest_flush) VALUES ($1, now())
+               ON CONFLICT (telegram_id) DO UPDATE SET last_digest_flush = now()""",
+            tg_id)
+    
+    return flushed
+
+
+async def scheduler_task(pool: object, bot: ExtBot) -> None:
+    while True:
+        proxy_enabled = False
+        turbo_mode = False
+        try:
+            proxy_enabled = proxy_available() and await get_proxy_enabled(pool)
+            turbo_mode = await get_turbo_mode(pool)
+        except Exception:
+            logger.exception("Could not load scheduler mode settings")
+
+        turbo_active = turbo_mode and proxy_enabled
+        speed_divisor = 10 if turbo_active else 1
+        inter_keyword_sleep_s = 5.0 / speed_divisor
+        loop_sleep_s = 30.0 / speed_divisor
+
+        record_scheduler_run()  # mark this cycle as started
+        try:
+            processed = await process_notification_queue(pool, bot)
+            if processed:
+                logger.info("Retried %d queued notifications", processed)
+        except Exception:
+            logger.exception("Error processing notification queue")
+
+        try:
+            digested = await flush_digests(pool, bot)
+            if digested:
+                logger.info("Flushed %d digest summaries", digested)
+        except Exception:
+            logger.exception("Error flushing digests")
+
+        try:
+            now_utc = datetime.now(tz=timezone.utc)
+            rows = await pool.fetch(
+                "SELECT id, keyword, interval_minutes, last_scraped_at, initial_loaded, ads_cursor, "
+                "price_min, price_max, allowed_postcodes FROM keywords "
+                "WHERE is_active = true"
+            )
+
+            due_rows = []
+            for row in rows:
+                interval_minutes = float(row.get("interval_minutes") or 1)
+                effective_interval_minutes = max(interval_minutes / speed_divisor, 0.1)
+                last_scraped_at = row.get("last_scraped_at")
+
+                if last_scraped_at is None:
+                    due_rows.append(row)
+                    continue
+
+                if last_scraped_at.tzinfo is None:
+                    last_scraped_at = last_scraped_at.replace(tzinfo=timezone.utc)
+
+                if last_scraped_at <= now_utc - timedelta(minutes=effective_interval_minutes):
+                    due_rows.append(row)
+
+            if turbo_active and due_rows:
+                logger.info("Turbo mode active via proxy: %d due keyword(s), x10 speed", len(due_rows))
+
+            for row in due_rows:
                 kw_id = str(row["id"])
                 keyword = row["keyword"]
                 initial_loaded = row["initial_loaded"]
+                cursor = row["ads_cursor"] or row.get("last_scraped_at")
 
                 subs = await pool.fetch(
                     "SELECT telegram_id FROM users u JOIN keyword_subscriptions ks ON u.id = ks.user_id "
@@ -47,14 +280,20 @@ async def scheduler_task(pool: object, bot: ExtBot) -> None:
                 logger.info("Scraping keyword '%s' (%d subscriber(s))", keyword, len(telegram_ids))
 
                 try:
-                    ads_raw, total_hits = await fetch_ads(keyword)
+                    ads_raw, total_hits = await fetch_ads(keyword, cursor_at=cursor)
                     new_count = 0
+                    oldest_timestamps: list[datetime] = []
 
                     if not initial_loaded and len(ads_raw) > 0:
                         logger.info("Initial baseline load for '%s' — indexing %d ads, no notifications", keyword, len(ads_raw))
 
                     for ad_data in ads_raw:
                         fields = extract_ad_fields(ad_data)
+
+                        # Skip ads that don't pass price/postcode filters
+                        if not _ad_passes_filters(fields, dict(row)):
+                            continue
+
                         wh_ad_id = fields["wh_ad_id"]
                         is_price_drop = False
                         old_price = None
@@ -75,11 +314,22 @@ async def scheduler_task(pool: object, bot: ExtBot) -> None:
                             )
                             ad_uuid = str(ad_row["id"])
 
+                            pub_ts = fields.get("published_at")
+                            if pub_ts and isinstance(pub_ts, datetime):
+                                oldest_timestamps.append(pub_ts)
+
                             # Only notify for genuinely new ads after baseline load is done
                             if initial_loaded:
                                 notify_fields = {**fields, "keyword": keyword}
                                 for tg_id in telegram_ids:
-                                    msg_id_val = await notify_new_ad(bot, tg_id, notify_fields)
+                                    # Check mute hours first
+                                    muted = await is_user_muted(pool, tg_id)
+                                    if muted:
+                                        # Check digest mode — buffer instead of discard
+                                        await buffer_for_digest(pool, tg_id, notify_fields, ad_uuid)
+                                        continue
+
+                                    msg_id_val = await notify_new_ad(bot, tg_id, notify_fields, ad_uuid=ad_uuid)
                                     if msg_id_val:
                                         user_row = await pool.fetchrow("SELECT id FROM users WHERE telegram_id = $1", tg_id)
                                         if user_row:
@@ -113,7 +363,13 @@ async def scheduler_task(pool: object, bot: ExtBot) -> None:
                         if is_price_drop:
                             notify_fields = {**fields, "keyword": keyword}
                             for tg_id in telegram_ids:
-                                msg_id_val = await notify_price_drop(bot, tg_id, notify_fields)
+                                # Check mute hours first
+                                muted = await is_user_muted(pool, tg_id)
+                                if muted:
+                                    await buffer_for_digest(pool, tg_id, notify_fields, ad_uuid)
+                                    continue
+
+                                msg_id_val = await notify_price_drop(bot, tg_id, notify_fields, ad_uuid=ad_uuid)
                                 if msg_id_val:
                                     user_row = await pool.fetchrow("SELECT id FROM users WHERE telegram_id = $1", tg_id)
                                     if user_row:
@@ -125,6 +381,13 @@ async def scheduler_task(pool: object, bot: ExtBot) -> None:
                     if not initial_loaded:
                         await pool.execute("UPDATE keywords SET initial_loaded = true WHERE id = $1", kw_id)
 
+                    oldest_ts = min(oldest_timestamps) if oldest_timestamps else None
+                    if oldest_ts:
+                        await pool.execute(
+                            "UPDATE keywords SET last_scraped_at = now(), ads_cursor = $1 WHERE id = $2",
+                            oldest_ts, kw_id,
+                        )
+                    else:
                         await pool.execute("UPDATE keywords SET last_scraped_at = now() WHERE id = $1", kw_id)
 
                     await pool.execute(
@@ -139,15 +402,17 @@ async def scheduler_task(pool: object, bot: ExtBot) -> None:
                         kw_id, str(sys.exc_info()[1]),
                     )
 
-                await asyncio.sleep(5)
+                await asyncio.sleep(inter_keyword_sleep_s)
 
         except Exception:
             logger.exception("Scheduler iteration error")
 
-        await asyncio.sleep(30)
+        await asyncio.sleep(loop_sleep_s)
 
 
 async def main() -> None:
+    set_start_time()  # for health endpoint uptime tracking
+
     logging.basicConfig(
         level=logging.INFO,
         format="%(asctime)s [%(levelname)s] %(name)s: %(message)s",
@@ -159,14 +424,74 @@ async def main() -> None:
 
     pool = await get_pool()
 
-    app = Application.builder().token(os.getenv("TELEGRAM_BOT_TOKEN")).build()
+    # ── Start healthcheck HTTP server ──────────────────────────────
+    health_app = create_health_app()
+    runner = web.AppRunner(health_app)
+    await runner.setup()
+    _health_port = int(os.getenv("HEALTH_PORT", "8765"))
+    site = web.TCPSite(runner, "0.0.0.0", _health_port)
+    await site.start()
+    logger.info("Health check server listening on :%d", _health_port)
+
+    # ── Start Web UI server ────────────────────────────────────────
+    _web_server = None
+    _web_config = None
+    try:
+        from web import app as web_app  # noqa: E402
+        import uvicorn  # noqa: E402
+
+        _web_port = int(os.getenv("WEB_UI_PORT", "8766"))
+        _web_config = uvicorn.Config(web_app, host="0.0.0.0", port=_web_port, log_level="info")
+        _web_server = uvicorn.Server(_web_config)
+        asyncio.ensure_future(_web_server.serve())
+        logger.info("Web UI listening on :%d", _web_port)
+    except Exception:
+        logger.exception("Failed to start Web UI (optional)")
+
+    bot_request = DirectHTTPXRequest(
+        connection_pool_size=10,
+        read_timeout=30.0,
+        write_timeout=30.0,
+        connect_timeout=30.0,
+        pool_timeout=10.0,
+        media_write_timeout=60.0,
+    )
+    updates_request = DirectHTTPXRequest(
+        connection_pool_size=10,
+        read_timeout=30.0,
+        write_timeout=30.0,
+        connect_timeout=30.0,
+        pool_timeout=10.0,
+        media_write_timeout=60.0,
+    )
+    logger.info("Telegram Bot API traffic uses direct network path")
+
+    app = (
+        Application.builder()
+        .token(os.getenv("TELEGRAM_BOT_TOKEN"))
+        .request(bot_request)
+        .get_updates_request(updates_request)
+        .build()
+    )
+    scheduler = None
+    poll_task = None
+    bot_started = False
 
     from bot import register_handlers, setup_global_commands  # noqa: E402
 
+    try:
         await setup_global_commands(app)
         register_handlers(app)
+        await app.initialize()
+        await app.start()
+        logger.info("Bot started with long polling")
 
+        set_telegram_polling(True)
+        poll_task = asyncio.ensure_future(app.updater.start_polling())  # type: ignore[attr-defined]
         scheduler = asyncio.ensure_future(scheduler_task(pool, app.bot))
+        bot_started = True
+    except Exception:
+        logger.exception("Telegram bot startup failed — continuing in UI-only mode")
 
     loop = asyncio.get_running_loop()
     stop = loop.create_future()
@@ -179,27 +504,51 @@ async def main() -> None:
         loop.add_signal_handler(sig, _signal_handler)
 
     try:
-        await app.initialize()
-        await app.start()
-        logger.info("Bot started with long polling")
-
-        poll_task = asyncio.ensure_future(app.updater.start_polling())  # type: ignore[attr-defined]
-
         await stop
-        logger.info("Shutting down...")
+        logger.info("Signal received — initiating graceful shutdown...")
 
     finally:
+        # ── Cancel scheduler with grace period ───────────────────────
+        if scheduler is not None:
+            logger.info("Cancelling scheduler task...")
             scheduler.cancel()
             with suppress(asyncio.CancelledError):
-            await scheduler
+                try:
+                    await asyncio.wait_for(scheduler, timeout=5.0)
+                except asyncio.TimeoutError:
+                    logger.warning("Scheduler task did not finish within 5s — force cancelled.")
 
+        # ── Stop Telegram polling ────────────────────────────────────
+        if bot_started and poll_task is not None:
+            set_telegram_polling(False)
+            logger.info("Stopping Telegram poller...")
             poll_task.cancel()
             with suppress(asyncio.CancelledError):
                 await poll_task
 
+        # ── Shutdown application ─────────────────────────────────────
+        if bot_started:
+            logger.info("Shutting down Telegram bot application...")
             await app.shutdown()
+
+        # ── Close health server ───────────────────────────────────────
+        logger.info("Stopping health check server...")
+        await runner.cleanup()
+        # ── Close Web UI server ──────────────────────────────────────
+        if _web_server is not None:
+            logger.info("Stopping Web UI server...")
+            _web_server.should_exit = True
+            await asyncio.sleep(1)
+        # ── Close HTTP client ────────────────────────────────────────
+        logger.info("Closing HTTP client...")
+        from scraper import close_client as close_http_client  # noqa: E402
+        await close_http_client()
+
+        # ── Close DB pool ─────────────────────────────────────────────
+        logger.info("Closing database connection pool...")
         await close_pool()
-        logger.info("Shutdown complete")
+
+        logger.info("Shutdown complete.")
 
 
 if __name__ == "__main__":

worker/src/migrate.py

@@ -0,0 +1,97 @@
+#!/usr/bin/env python3
+"""Run pending SQL migrations against the configured PostgreSQL database.
+
+Usage:  python migrate.py
+
+Tracks applied migrations in `willhaben_migrations` table and only applies
+missing ones. Forward-only — rollback means restore from DB backup."""
+
+import asyncio
+import glob
+import logging
+import os
+import sys
+
+import asyncpg
+
+logger = logging.getLogger(__name__)
+
+MIGRATIONS_DIR = os.path.join(os.path.dirname(__file__), "migrations")
+
+
+def _dsn_parts() -> dict:
+    """Return a dict of DSN connection parameters from env vars."""
+    return {
+        "host": os.getenv("POSTGRES_HOST", "192.168.178.3"),
+        "port": int(os.getenv("POSTGRES_PORT", "5432")),
+        "user": os.getenv("POSTGRES_USER", "postgres"),
+        "password": os.getenv("POSTGRES_PASSWORD"),
+        "database": os.getenv("POSTGRES_DB", "postgres"),
+    }
+
+
+async def run_migrations() -> None:
+    """Connect to the database and apply any pending SQL migrations."""
+    conn = await asyncpg.connect(**_dsn_parts())
+
+    try:
+        # Create tracking table (always in public schema)
+        await conn.execute("""
+            CREATE TABLE IF NOT EXISTS willhaben_migrations (
+                name         text PRIMARY KEY,
+                applied_at   timestamptz DEFAULT now()
+            );
+        """)
+
+        # Discover and sort migration files alphabetically
+        sql_files = sorted(glob.glob(os.path.join(MIGRATIONS_DIR, "*.sql")))
+
+        if not sql_files:
+            logger.warning("No migration files found in %s", MIGRATIONS_DIR)
+            return
+
+        applied_count = 0
+        for fpath in sql_files:
+            name = os.path.basename(fpath)
+
+            already_applied = await conn.fetchval(
+                "SELECT 1 FROM willhaben_migrations WHERE name = $1", name,
+            )
+            if already_applied:
+                logger.info("Skipping already-applied migration: %s", name)
+                continue
+
+            logger.info("Applying migration: %s", name)
+
+            with open(fpath, encoding="utf-8") as fh:
+                sql = fh.read()
+
+            # Execute within a transaction block for safety
+            async with conn.transaction():
+                await conn.execute(sql)
+                await conn.execute(
+                    "INSERT INTO willhaben_migrations (name) VALUES ($1)", name,
+                )
+
+            applied_count += 1
+            logger.info("Applied migration: %s", name)
+
+        if applied_count:
+            logger.info("Migration complete — %d new migration(s) applied.", applied_count)
+        else:
+            logger.info("Database is up to date (0 pending migrations).")
+
+    except asyncpg.PostgresError as exc:
+        logger.error("Migration failed: %s", exc)
+        raise SystemExit(1) from exc
+    finally:
+        await conn.close()
+
+
+if __name__ == "__main__":
+    logging.basicConfig(
+        level=logging.INFO,
+        format="%(asctime)s %(levelname)s %(message)s",
+        datefmt="%Y-%m-%d %H:%M:%S",
+    )
+    asyncio.run(run_migrations())

worker/src/migrations/01-schema.sql

@@ -0,0 +1,123 @@
+-- ============================================================
+-- willhaben-tracker — consolidated schema (single source of truth)
+-- Merged from: 01-init.sql, 02-image-and-pricing.sql, 03-global-keywords.sql
+-- ============================================================
+
+-- -----------------------------------------------------------
+-- 1. users (whitelisted Telegram users)
+-- -----------------------------------------------------------
+CREATE TABLE IF NOT EXISTS users (
+    id          uuid PRIMARY KEY DEFAULT gen_random_uuid(),
+    telegram_id bigint UNIQUE NOT NULL,
+    username    text,
+    first_name  text,
+    is_admin    boolean NOT NULL DEFAULT false,
+    is_active   boolean NOT NULL DEFAULT true,
+    created_at  timestamptz NOT NULL DEFAULT now()
+);
+
+-- -----------------------------------------------------------
+-- 2. keywords (global search keywords — deduplicated across users)
+-- -----------------------------------------------------------
+CREATE TABLE IF NOT EXISTS keywords (
+    id               uuid PRIMARY KEY DEFAULT gen_random_uuid(),
+    keyword          text NOT NULL,
+    interval_minutes int NOT NULL DEFAULT 60,
+    is_active        boolean NOT NULL DEFAULT true,
+    initial_loaded   boolean NOT NULL DEFAULT false,
+    last_scraped_at  timestamptz,
+    created_at       timestamptz NOT NULL DEFAULT now()
+);
+
+CREATE UNIQUE INDEX IF NOT EXISTS idx_keywords_unique_lower
+    ON keywords(LOWER(keyword));
+
+-- -----------------------------------------------------------
+-- 3. keyword_subscriptions (many-to-many: user ↔ keyword)
+-- -----------------------------------------------------------
+CREATE TABLE IF NOT EXISTS keyword_subscriptions (
+    keyword_id uuid REFERENCES keywords(id) ON DELETE CASCADE NOT NULL,
+    user_id    uuid REFERENCES users(id) ON DELETE CASCADE NOT NULL,
+    PRIMARY KEY (keyword_id, user_id),
+    created_at timestamptz NOT NULL DEFAULT now()
+);
+
+-- -----------------------------------------------------------
+-- 4. ads (raw ad snapshots, globally deduplicated by wh_ad_id)
+-- -----------------------------------------------------------
+CREATE TABLE IF NOT EXISTS ads (
+    id            uuid PRIMARY KEY DEFAULT gen_random_uuid(),
+    wh_ad_id      text UNIQUE NOT NULL,
+    raw_json      jsonb NOT NULL,
+    title         text NOT NULL,
+    price         numeric,
+    location      text,
+    url           text,
+    published_at  timestamptz,
+    first_seen_at timestamptz NOT NULL DEFAULT now(),
+    main_image_url text,
+    postcode      text,
+    modified_at   timestamptz
+);
+
+-- -----------------------------------------------------------
+-- 5. price_history (track price changes per ad)
+-- -----------------------------------------------------------
+CREATE TABLE IF NOT EXISTS price_history (
+    id         uuid PRIMARY KEY DEFAULT gen_random_uuid(),
+    ad_id      uuid REFERENCES ads(id) ON DELETE CASCADE NOT NULL,
+    old_price  numeric NOT NULL,
+    new_price  numeric NOT NULL,
+    changed_at timestamptz NOT NULL DEFAULT now(),
+    UNIQUE (ad_id, old_price, new_price)
+);
+
+-- -----------------------------------------------------------
+-- 6. notifications (audit log of sent Telegram messages)
+-- -----------------------------------------------------------
+CREATE TABLE IF NOT EXISTS notifications (
+    id         uuid PRIMARY KEY DEFAULT gen_random_uuid(),
+    user_id    uuid REFERENCES users(id) ON DELETE CASCADE NOT NULL,
+    ad_id      uuid REFERENCES ads(id) ON DELETE SET NULL,
+    message_id int,
+    sent_at    timestamptz NOT NULL DEFAULT now()
+);
+
+-- -----------------------------------------------------------
+-- 7. scrape_logs (worker health / debugging)
+-- -----------------------------------------------------------
+CREATE TABLE IF NOT EXISTS scrape_logs (
+    id            uuid PRIMARY KEY DEFAULT gen_random_uuid(),
+    keyword_id    uuid REFERENCES keywords(id) ON DELETE CASCADE NOT NULL,
+    status        text NOT NULL CHECK (status IN ('success', 'error', 'rate_limited')),
+    ads_found     int NOT NULL DEFAULT 0,
+    new_ads       int NOT NULL DEFAULT 0,
+    error_message text,
+    scraped_at    timestamptz NOT NULL DEFAULT now()
+);
+
+-- ============================================================
+-- Indexes
+-- ============================================================
+
+-- Keywords: fast lookup for active keywords ordered by last scrape time
+CREATE INDEX IF NOT EXISTS idx_keywords_active_scraped
+    ON keywords(is_active, last_scraped_at) WHERE is_active = true;
+
+-- Keyword subscriptions: find all subscribers of a keyword
+CREATE INDEX IF NOT EXISTS idx_keyword_subscriptions_user_id
+    ON keyword_subscriptions(user_id);
+
+-- Ads: fast lookup by willhaben ad ID (unique constraint already implies an index)
+
+-- Price history: look up changes for a specific ad
+CREATE INDEX IF NOT EXISTS idx_price_history_ad_id
+    ON price_history(ad_id);
+
+-- Notifications: recent messages per user
+CREATE INDEX IF NOT EXISTS idx_notifications_user_sent
+    ON notifications(user_id, sent_at DESC);
+
+-- Scrape logs: latest runs per keyword
+CREATE INDEX IF NOT EXISTS idx_scrape_logs_keyword_at
+    ON scrape_logs(keyword_id, scraped_at DESC);

worker/src/migrations/02-pagination-cursor.sql

@@ -0,0 +1,5 @@
+-- Add ads_cursor column to keywords for pagination cursor tracking
+
+ALTER TABLE keywords ADD COLUMN IF NOT EXISTS ads_cursor timestamptz;
+COMMENT ON COLUMN keywords.ads_cursor IS 
+    'Timestamp of oldest ad processed in last cycle, for pagination cursor';

worker/src/migrations/03-notification-queue.sql

@@ -0,0 +1,26 @@
+-- Notification retry queue table
+
+CREATE TYPE notification_type AS ENUM ('new', 'drop');
+CREATE TYPE notification_status AS ENUM ('pending', 'sent', 'failed', 'dead');
+
+CREATE TABLE IF NOT EXISTS notification_queue (
+    id            uuid PRIMARY KEY DEFAULT gen_random_uuid(),
+    ad_id         uuid NOT NULL REFERENCES ads(id) ON DELETE SET NULL,
+    telegram_id   text NOT NULL,
+    message_text  text NOT NULL,
+    type          notification_type NOT NULL,
+    attempts      int NOT NULL DEFAULT 0,
+    max_attempts  int NOT NULL DEFAULT 5,
+    last_error    text,
+    status        notification_status NOT NULL DEFAULT 'pending',
+    created_at    timestamptz NOT NULL DEFAULT now(),
+    updated_at    timestamptz NOT NULL DEFAULT now()
+);
+
+-- Index for efficient queue polling
+CREATE INDEX IF NOT EXISTS idx_notif_queue_poll 
+    ON notification_queue(status, attempts, updated_at)
+    WHERE status IN ('pending', 'failed');
+
+COMMENT ON TABLE notification_queue IS 
+    'Persistent retry queue for failed Telegram notifications';

worker/src/migrations/05-keyword-filters.sql

@@ -0,0 +1,9 @@
+-- Phase 2: Price and postcode filters per keyword
+ALTER TABLE keywords 
+    ADD COLUMN IF NOT EXISTS price_min int,
+    ADD COLUMN IF NOT EXISTS price_max int,
+    ADD COLUMN IF NOT EXISTS allowed_postcodes text[];
+
+COMMENT ON COLUMN keywords.price_min IS 'Minimum price in cents (e.g. 5000 = €50). NULL = no limit.';
+COMMENT ON COLUMN keywords.price_max IS 'Maximum price in cents (e.g. 500000 = €5000). NULL = no limit.';
+COMMENT ON COLUMN keywords.allowed_postcodes IS 'Austrian postcodes (4-digit strings). Only ads matching these are notified.';

worker/src/migrations/06-user-settings.sql

@@ -0,0 +1,31 @@
+-- Phase 2: User settings (mute hours + digest mode) and digest buffer
+CREATE TABLE IF NOT EXISTS user_settings (
+    telegram_id       text PRIMARY KEY,
+    mute_start        time,
+    mute_end          time,
+    digest_mode       bool NOT NULL DEFAULT false,
+    digest_interval   int NOT NULL DEFAULT 60,
+    last_digest_flush timestamptz,
+    CONSTRAINT chk_mute_hours CHECK (
+        mute_start IS NULL AND mute_end IS NULL 
+        OR mute_start IS NOT NULL AND mute_end IS NOT NULL
+    )
+);
+
+COMMENT ON TABLE user_settings IS 'User-specific settings for notification behavior';
+
+CREATE TABLE IF NOT EXISTS digest_buffer (
+    id          uuid PRIMARY KEY DEFAULT gen_random_uuid(),
+    telegram_id text NOT NULL,
+    ad_id       uuid REFERENCES ads(id) ON DELETE CASCADE,
+    keyword     text NOT NULL,
+    title       text NOT NULL,
+    price       int,  -- in cents
+    url         text,
+    created_at  timestamptz NOT NULL DEFAULT now()
+);
+
+CREATE INDEX IF NOT EXISTS idx_digest_buffer 
+    ON digest_buffer(telegram_id, created_at DESC);
+
+COMMENT ON TABLE digest_buffer IS 'Buffer for digest-mode notifications, flushed at intervals';

worker/src/migrations/07-app-settings.sql

@@ -0,0 +1,9 @@
+CREATE TABLE IF NOT EXISTS app_settings (
+    key text PRIMARY KEY,
+    value text NOT NULL,
+    updated_at timestamptz NOT NULL DEFAULT now()
+);
+
+INSERT INTO app_settings (key, value)
+VALUES ('turbo_mode', 'false')
+ON CONFLICT (key) DO NOTHING;

worker/src/migrations/08-proxy-setting.sql

@@ -0,0 +1,3 @@
+INSERT INTO app_settings (key, value)
+VALUES ('proxy_enabled', 'true')
+ON CONFLICT (key) DO NOTHING;

worker/src/migrations/zz-seed.sql

@@ -0,0 +1,8 @@
+-- ============================================================
+-- Seed: initial admin user
+-- Runs last (alphabetically), after all schema migrations.
+-- ============================================================
+
+INSERT INTO users (telegram_id, username, first_name, is_admin, is_active)
+VALUES (298181113, NULL, 'Admin', true, true)
+ON CONFLICT (telegram_id) DO NOTHING;

worker/src/notifier.py

@@ -1,14 +1,67 @@
 import logging
-from datetime import datetime
+from datetime import datetime, timezone
 from typing import Any
 
 import asyncpg
 from telegram import InlineKeyboardButton, InlineKeyboardMarkup
+from telegram.error import TelegramError
 from telegram.ext import ExtBot
 
 logger = logging.getLogger(__name__)
 
 
+async def is_user_muted(pool: asyncpg.Pool, telegram_id: int) -> bool:
+    """Check if the current time is within the user's mute window."""
+    settings = await pool.fetchrow(
+        "SELECT mute_start, mute_end FROM user_settings WHERE telegram_id = $1",
+        str(telegram_id),
+    )
+    
+    if not settings or not settings["mute_start"] or not settings["mute_end"]:
+        return False
+    
+    now_utc = datetime.now(tz=timezone.utc).time()
+    start = settings["mute_start"]
+    end = settings["mute_end"]
+    
+    if start < end:
+        # Normal window (e.g., 08:00–12:00)
+        return start <= now_utc <= end
+    else:
+        # Window crosses midnight (e.g., 22:00 to 07:00 next day)
+        return now_utc >= start or now_utc <= end
+
+
+async def buffer_for_digest(
+    pool: asyncpg.Pool,
+    telegram_id: int,
+    ad: dict[str, Any],
+    ad_uuid: str | None = None,
+) -> None:
+    """Buffer a notification for digest-mode users, or discard if not in digest mode."""
+    settings = await pool.fetchrow(
+        "SELECT digest_mode FROM user_settings WHERE telegram_id = $1",
+        str(telegram_id),
+    )
+    
+    if not settings or not settings["digest_mode"]:
+        # Not in digest mode — during mute hours, just discard
+        return
+    
+    # In digest mode — buffer the notification
+    price_cents = int(round(ad["price"] * 100)) if ad.get("price") else None
+    try:
+        await pool.execute(
+            """INSERT INTO digest_buffer (telegram_id, ad_id, keyword, title, price, url)
+               VALUES ($1, $2, $3, $4, $5, $6)""",
+            str(telegram_id), ad_uuid, ad.get("keyword", ""), ad.get("title", ""),
+            price_cents, ad.get("url"),
+        )
+        logger.info("Buffered digest for %s: %s", telegram_id, ad.get("title", ""))
+    except Exception:
+        logger.exception("Failed to buffer digest for %s", telegram_id)
+
+
 def _build_keyboard(ad: dict[str, Any]) -> InlineKeyboardMarkup | None:
     keyboard: list[list[InlineKeyboardButton]] = []
     if ad.get("url"):
@@ -74,6 +127,7 @@ async def notify_new_ad(
     bot: ExtBot,
     telegram_id: int,
     ad: dict[str, Any],
+    ad_uuid: str | None = None,
 ) -> int | None:
     text = _format_text("🆕 New listing found!", ad)
     reply_markup = _build_keyboard(ad)
@@ -102,8 +156,21 @@ async def notify_new_ad(
             message.message_id,
         )
         return message.message_id
+    except TelegramError as e:
+        error_msg = str(e)[:300]
+        logger.warning("Telegram send failed for %s: %s", telegram_id, error_msg)
+
+        if ad_uuid:
+            await _enqueue_retry(
+                ad_id=ad_uuid,
+                telegram_id=str(telegram_id),
+                message_text=text,
+                notif_type="new",
+                error_msg=error_msg,
+            )
     except Exception:
         logger.exception("Failed to send Telegram notification")
+
     return None
 
 
@@ -111,6 +178,7 @@ async def notify_price_drop(
     bot: ExtBot,
     telegram_id: int,
     ad: dict[str, Any],
+    ad_uuid: str | None = None,
 ) -> int | None:
     text = _format_text("⚠️ Price drop!", ad)
     reply_markup = _build_keyboard(ad)
@@ -139,11 +207,58 @@ async def notify_price_drop(
             message.message_id,
         )
         return message.message_id
+    except TelegramError as e:
+        error_msg = str(e)[:300]
+        logger.warning("Telegram send failed for %s: %s", telegram_id, error_msg)
+
+        if ad_uuid:
+            await _enqueue_retry(
+                ad_id=ad_uuid,
+                telegram_id=str(telegram_id),
+                message_text=text,
+                notif_type="drop",
+                error_msg=error_msg,
+            )
     except Exception:
         logger.exception("Failed to send Telegram notification")
+
     return None
 
 
+async def _enqueue_retry(
+    ad_id: str,
+    telegram_id: str,
+    message_text: str,
+    notif_type: str,
+    error_msg: str,
+) -> None:
+    """Store a failed notification for later retry."""
+    try:
+        from db import get_pool
+        pool = await get_pool()
+
+        existing = await pool.fetchval(
+            """SELECT id FROM notification_queue 
+               WHERE ad_id = $1 AND telegram_id = $2 AND status IN ('pending', 'failed')""",
+            ad_id, telegram_id,
+        )
+
+        if existing:
+            logger.info("Already queued: ad=%s user=%s", ad_id[:8], telegram_id)
+            return
+
+        await pool.execute(
+            """INSERT INTO notification_queue 
+               (ad_id, telegram_id, message_text, type, last_error, status)
+               VALUES ($1, $2, $3, $4, $5, 'pending')""",
+            ad_id, telegram_id, message_text, notif_type, error_msg,
+        )
+
+        logger.info("Queued for retry: ad=%s user=%s", ad_id[:8], telegram_id)
+    except Exception:
+        logger.exception("Failed to enqueue retry for %s", telegram_id)
+
+
 async def log_notification(
     pool: asyncpg.Pool, user_id: str, ad_id: str, message_id: int
 ) -> None:

worker/src/scraper.py

@@ -1,13 +1,195 @@
 import asyncio
 import logging
-from datetime import datetime, timezone
+import os
+from datetime import datetime
 from typing import Any
-from urllib.parse import quote_plus
 
 import httpx
 
+from db import get_pool
+from settings import get_effective_proxy_url, get_proxy_url_from_env, proxy_available
+
 logger = logging.getLogger(__name__)
 
+_client: httpx.AsyncClient | None = None
+_client_proxy_url: str | None = None
+_proxy_ip_logged = False
+_system_public_ip: str | None = None
+_proxy_public_ip: str | None = None
+_last_used_public_ip: str | None = None
+_proxy_enabled_effective: bool | None = None
+
+
+def _get_proxy_url() -> str | None:
+    return get_proxy_url_from_env()
+
+
+async def _get_effective_proxy_url() -> str | None:
+    global _proxy_enabled_effective
+
+    pool = await get_pool()
+    proxy_url = await get_effective_proxy_url(pool)
+    _proxy_enabled_effective = proxy_url is not None
+    return proxy_url
+
+
+def _redact_proxy_url(proxy_url: str) -> str:
+    try:
+        parsed = httpx.URL(proxy_url)
+        host = parsed.host or "unknown-host"
+        port = parsed.port or 80
+        user = parsed.username or "unknown-user"
+        return f"{host}:{port} (user={user}, credentials=set)"
+    except Exception:
+        return "<unparseable proxy>"
+
+
+async def _fetch_public_ip(client: httpx.AsyncClient) -> str | None:
+    try:
+        resp = await client.get("https://api.ipify.org?format=json")
+        resp.raise_for_status()
+        data = resp.json()
+        ip = data.get("ip")
+        return str(ip) if ip else None
+    except Exception as exc:
+        logger.warning("Could not resolve public IP: %s", exc)
+        return None
+
+
+async def _log_proxy_ip_comparison(proxy_url: str) -> None:
+    global _proxy_ip_logged, _system_public_ip, _proxy_public_ip, _last_used_public_ip
+
+    if _proxy_ip_logged:
+        return
+
+    _proxy_ip_logged = True
+
+    direct_client = httpx.AsyncClient(timeout=10.0, trust_env=False)
+    proxy_client = httpx.AsyncClient(timeout=10.0, trust_env=False, proxy=proxy_url)
+
+    try:
+        direct_ip = await _fetch_public_ip(direct_client)
+        proxy_ip = await _fetch_public_ip(proxy_client)
+        _system_public_ip = direct_ip
+        _proxy_public_ip = proxy_ip
+        _last_used_public_ip = proxy_ip or direct_ip
+        logger.info(
+            "HTTPS proxy enabled: %s | public_ip_without_proxy=%s | public_ip_with_proxy=%s",
+            _redact_proxy_url(proxy_url),
+            direct_ip or "unknown",
+            proxy_ip or "unknown",
+        )
+    finally:
+        await direct_client.aclose()
+        await proxy_client.aclose()
+
+
+async def get_client() -> httpx.AsyncClient:
+    """Return a shared AsyncClient with keepalive connection pool."""
+    global _client, _client_proxy_url, _system_public_ip, _last_used_public_ip
+
+    proxy_url = await _get_effective_proxy_url()
+
+    if _client is not None and not _client.is_closed and _client_proxy_url != proxy_url:
+        await _client.aclose()
+        logger.info(
+            "Recreated httpx client because proxy changed: %s -> %s",
+            "enabled" if _client_proxy_url else "disabled",
+            "enabled" if proxy_url else "disabled",
+        )
+        _client = None
+
+    if _client is None or _client.is_closed:
+        max_conns = int(os.getenv("HTTP_MAX_CONNECTIONS", "10"))
+        max_keepalive = int(os.getenv("HTTP_KEEPALIVE_CONNECTIONS", "5"))
+
+        if proxy_url:
+            await _log_proxy_ip_comparison(proxy_url)
+        elif _system_public_ip is None:
+            direct_client = httpx.AsyncClient(timeout=10.0, trust_env=False)
+            try:
+                _system_public_ip = await _fetch_public_ip(direct_client)
+                _last_used_public_ip = _system_public_ip
+            finally:
+                await direct_client.aclose()
+
+        _client = httpx.AsyncClient(
+            timeout=float(os.getenv("HTTP_TIMEOUT_S", "30.0")),
+            limits=httpx.Limits(
+                max_connections=max_conns,
+                max_keepalive_connections=max_keepalive,
+                keepalive_expiry=60,
+            ),
+            proxy=proxy_url,
+            trust_env=False,
+        )
+        _client_proxy_url = proxy_url
+        logger.info(
+            "Created httpx client: max_conns=%d, keepalive=%d, proxy=%s",
+            max_conns,
+            max_keepalive,
+            "enabled" if proxy_url else "disabled",
+        )
+
+    return _client
+
+
+async def refresh_network_status() -> dict[str, str | bool | None]:
+    """Ensure network status has best-effort values even before first scrape."""
+    global _system_public_ip, _proxy_public_ip, _last_used_public_ip, _proxy_enabled_effective
+
+    proxy_url = await _get_effective_proxy_url()
+    _proxy_enabled_effective = proxy_url is not None
+
+    if _system_public_ip is None:
+        direct_client = httpx.AsyncClient(timeout=10.0, trust_env=False)
+        try:
+            _system_public_ip = await _fetch_public_ip(direct_client)
+        finally:
+            await direct_client.aclose()
+
+    if proxy_url and _proxy_public_ip is None:
+        proxy_client = httpx.AsyncClient(timeout=10.0, trust_env=False, proxy=proxy_url)
+        try:
+            _proxy_public_ip = await _fetch_public_ip(proxy_client)
+        finally:
+            await proxy_client.aclose()
+
+    if proxy_url:
+        _last_used_public_ip = _proxy_public_ip or _system_public_ip
+    else:
+        _last_used_public_ip = _system_public_ip
+
+    return get_network_status()
+
+
+def get_network_status() -> dict[str, str | bool | None]:
+    """Return best-effort network status for UI display."""
+    available = proxy_available()
+    proxy_enabled = bool(_proxy_enabled_effective) if _proxy_enabled_effective is not None else available
+    last_used = _last_used_public_ip
+    if last_used is None:
+        last_used = _proxy_public_ip if proxy_enabled else _system_public_ip
+
+    return {
+        "proxy_available": available,
+        "proxy_enabled": proxy_enabled,
+        "system_public_ip": _system_public_ip,
+        "proxy_public_ip": _proxy_public_ip,
+        "last_used_public_ip": last_used,
+    }
+
+
+async def close_client() -> None:
+    """Close the shared AsyncClient. Call during shutdown."""
+    global _client, _client_proxy_url
+    if _client and not _client.is_closed:
+        await _client.aclose()
+        logger.info("Closed httpx client")
+    _client = None
+    _client_proxy_url = None
+
+
 _API_URL = (
     "https://www.willhaben.at/webapi/ad-search/search/atz/seo/"
     "kaufen-und-verkaufen/marktplatz"
@@ -19,31 +201,85 @@ _HEADERS = {
     "x-wh-client": "api@willhaben.at;responsive_web;server;1.0.0;desktop",
 }
 
+_MAX_PAGES = int(os.getenv("SCRAPE_MAX_PAGES", "2"))
+_PAGE_DELAY_S = float(os.getenv("SCRAPE_PAGE_DELAY_S", "1.0"))
 
-async def fetch_ads(keyword: str) -> tuple[list[dict[str, Any]], int]:
+
+async def fetch_ads(
+    keyword: str,
+    cursor_at: datetime | None = None,
+    max_pages: int | None = None,
+) -> tuple[list[dict[str, Any]], int]:
+    """Fetch ads with pagination, deduping by cursor timestamp."""
+    pages = max_pages or _MAX_PAGES
+    all_ads_raw: list[dict[str, Any]] = []
+    total_hits: int = 0
+
+    client = await get_client()
+
+    for page in range(pages):
         params = {
             "keyword": keyword,
             "rows": 30,
             "sort": 1,
+            "offset": page * 30,
         }
 
-    async with httpx.AsyncClient(timeout=30.0) as client:
-        for attempt in range(1, 4):
         try:
             resp = await client.get(_API_URL, headers=_HEADERS, params=params)
             resp.raise_for_status()
             data = resp.json()
-                break
-            except Exception as exc:
-                logger.warning("fetch_ads attempt %d failed: %s", attempt, exc)
-                if attempt < 3:
-                    await asyncio.sleep(2 ** attempt)
-                    continue
-                raise
-
-    ads_raw = data.get("advertSummaryList", {}).get("advertSummary", [])
             total_hits = int(data.get("rowsFound", 0))
-    return ads_raw, total_hits
+        except Exception as exc:
+            logger.warning(
+                "fetch_ads page %d failed for '%s': %s", page, keyword, exc
+            )
+            break
+
+        page_ads = (data.get("advertSummaryList") or {}).get("advertSummary", [])
+
+        if not page_ads:
+            logger.info("No more ads on page %d for '%s'", page, keyword)
+            break
+
+        oldest_published = _get_oldest_published(page_ads)
+        if cursor_at and oldest_published and oldest_published <= cursor_at:
+            logger.info(
+                "Early stop at page %d for '%s' — reached cursor",
+                page, keyword,
+            )
+            break
+
+        new_batch = [
+            ad for ad in page_ads
+            if not cursor_at or _get_published(ad) is None or _get_published(ad) > cursor_at
+        ]
+
+        all_ads_raw.extend(new_batch)
+
+        if page < pages - 1 and new_batch:
+            await asyncio.sleep(_PAGE_DELAY_S)
+
+    return all_ads_raw, total_hits
+
+
+def _get_published(ad_dict: dict) -> datetime | None:
+    """Extract published timestamp from a single ad dict."""
+    attrs = _parse_attributes(ad_dict)
+    raw = attrs.get("PUBLISHED_String") or attrs.get("CHANGED_String")
+    if not raw:
+        return None
+    try:
+        return datetime.fromisoformat(raw.replace("Z", "+00:00"))
+    except (ValueError, TypeError):
+        return None
+
+
+def _get_oldest_published(ads: list[dict]) -> datetime | None:
+    """Get the oldest published timestamp from a batch of ads."""
+    timestamps = [_get_published(ad) for ad in ads]
+    timestamps = [t for t in timestamps if t is not None]
+    return min(timestamps) if timestamps else None
 
 
 def _parse_attributes(ad_dict: dict[str, Any]) -> dict[str, str]:

worker/src/settings.py

@@ -0,0 +1,92 @@
+import logging
+import os
+from urllib.parse import quote
+
+import asyncpg
+
+logger = logging.getLogger(__name__)
+
+TRUE_VALUES = {"1", "true", "yes", "on"}
+
+
+def _as_bool(raw: object, default: bool = False) -> bool:
+    if raw is None:
+        return default
+    return str(raw).lower() in TRUE_VALUES
+
+
+def get_proxy_url_from_env() -> str | None:
+    raw = (os.getenv("HTTPS_PROXY") or os.getenv("https_proxy") or "").strip()
+    if not raw:
+        return None
+
+    if "://" in raw:
+        return raw
+
+    parts = raw.split(":", 3)
+    if len(parts) != 4:
+        logger.warning(
+            "Ignoring invalid HTTPS_PROXY value (expected host:port:user:pass or full URL)"
+        )
+        return None
+
+    host, port, username, password = parts
+    return f"http://{quote(username, safe='')}:{quote(password, safe='')}@{host}:{port}"
+
+
+def proxy_available() -> bool:
+    return get_proxy_url_from_env() is not None
+
+
+async def ensure_app_settings(pool: asyncpg.Pool) -> None:
+    await pool.execute(
+        """
+        CREATE TABLE IF NOT EXISTS app_settings (
+            key text PRIMARY KEY,
+            value text NOT NULL,
+            updated_at timestamptz NOT NULL DEFAULT now()
+        );
+        """
+    )
+
+
+async def get_bool_setting(pool: asyncpg.Pool, key: str, default: bool = False) -> bool:
+    await ensure_app_settings(pool)
+    raw = await pool.fetchval("SELECT value FROM app_settings WHERE key = $1", key)
+    return _as_bool(raw, default=default)
+
+
+async def set_bool_setting(pool: asyncpg.Pool, key: str, enabled: bool) -> None:
+    await ensure_app_settings(pool)
+    await pool.execute(
+        """
+        INSERT INTO app_settings (key, value, updated_at)
+        VALUES ($1, $2, now())
+        ON CONFLICT (key)
+        DO UPDATE SET value = EXCLUDED.value, updated_at = now()
+        """,
+        key,
+        "true" if enabled else "false",
+    )
+
+
+async def get_turbo_mode(pool: asyncpg.Pool) -> bool:
+    return await get_bool_setting(pool, "turbo_mode")
+
+
+async def set_turbo_mode(pool: asyncpg.Pool, enabled: bool) -> None:
+    await set_bool_setting(pool, "turbo_mode", enabled)
+
+
+async def get_proxy_enabled(pool: asyncpg.Pool) -> bool:
+    return await get_bool_setting(pool, "proxy_enabled", default=proxy_available())
+
+
+async def set_proxy_enabled(pool: asyncpg.Pool, enabled: bool) -> None:
+    await set_bool_setting(pool, "proxy_enabled", enabled)
+
+
+async def get_effective_proxy_url(pool: asyncpg.Pool) -> str | None:
+    if not await get_proxy_enabled(pool):
+        return None
+    return get_proxy_url_from_env()

worker/src/templates/ads.html

@@ -0,0 +1,65 @@
+{% extends "base.html" %}
+{% block title %}Ads — Willhaben Tracker{% endblock %}
+{% block title_in_topbar %}Recent Ads{% endblock %}
+
+{% block content %}
+{% if not error and ads %}
+<div class="table-container">
+    <table>
+        <thead>
+            <tr>
+                <th>Ad</th>
+                <th>Location</th>
+                <th>First Tracked</th>
+                <th>Link</th>
+            </tr>
+        </thead>
+        <tbody>
+            {% for ad in ads %}
+            <tr>
+                <td style="max-width: 400px; white-space: normal;">
+                    <div style="font-weight: 500; margin-bottom: 4px; line-height: 1.4;">
+                        {{ ad.title }}
+                    </div>
+                    <div style="display: flex; gap: 12px; font-size: 13px;">
+                        <span style="font-weight: 600; color: var(--success-text);">{{ format_price(ad.price) }}</span>
+                        {% if ad.published_at %}
+                        <span style="color: var(--text-muted);">
+                            <i class="ph ph-calendar-blank" style="vertical-align: middle;"></i> 
+                            Pub: {{ ad.published_at.strftime('%m/%d %H:%M') }}
+                        </span>
+                        {% endif %}
+                    </div>
+                </td>
+                <td>
+                    <div style="display: flex; align-items: center; gap: 4px;">
+                        <i class="ph ph-map-pin" style="color: var(--text-muted);"></i>
+                        {% if ad.postcode %}{{ ad.postcode }} • {% endif %}
+                        {{ ad.location or '—' }}
+                    </div>
+                </td>
+                <td>
+                    <div style="color: var(--text-secondary);">
+                        {{ ad.first_seen_at.strftime('%Y-%m-%d') }}
+                    </div>
+                    <div style="font-size: 12px; color: var(--text-muted); margin-top: 2px;">
+                        {{ ad.first_seen_at.strftime('%H:%M:%S') }}
+                    </div>
+                </td>
+                <td>
+                    <a href="{{ ad.url }}" target="_blank" style="display: inline-flex; align-items: center; gap: 4px; padding: 6px 12px; background: var(--bg-surface-active); border-radius: var(--radius-md); font-size: 13px; font-weight: 500;">
+                        Open <i class="ph-bold ph-arrow-up-right"></i>
+                    </a>
+                </td>
+            </tr>
+            {% endfor %}
+        </tbody>
+    </table>
+</div>
+{% elif not error %}
+<div class="empty-state">
+    <i class="ph ph-shopping-bag"></i>
+    <p>No ads tracked yet.</p>
+</div>
+{% endif %}
+{% endblock %}

worker/src/templates/base.html

@@ -0,0 +1,380 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+    <meta charset="UTF-8">
+    <meta name="viewport" content="width=device-width, initial-scale=1.0">
+    <title>{% block title %}Willhaben Tracker{% endblock %}</title>
+    <!-- Add Inter font and Phosphor icons -->
+    <link href="https://fonts.googleapis.com/css2?family=Inter:wght@400;500;600;700&display=swap" rel="stylesheet">
+    <script src="https://unpkg.com/@phosphor-icons/web"></script>
+    <style>
+        :root {
+            --bg-body: #0b0f19;
+            --bg-surface: #111827;
+            --bg-surface-hover: #1f2937;
+            --bg-surface-active: #374151;
+            
+            --border-default: #1f2937;
+            --border-hover: #374151;
+            
+            --text-primary: #f3f4f6;
+            --text-secondary: #9ca3af;
+            --text-muted: #6b7280;
+            
+            --brand-primary: #3b82f6;
+            --brand-primary-hover: #2563eb;
+            --brand-primary-light: rgba(59, 130, 246, 0.1);
+            
+            --success-text: #34d399;
+            --success-bg: rgba(52, 211, 153, 0.1);
+            --danger-text: #f87171;
+            --danger-bg: rgba(248, 113, 113, 0.1);
+            --warning-text: #fbbf24;
+            --warning-bg: rgba(251, 191, 36, 0.1);
+
+            --radius-md: 12px;
+            --radius-lg: 16px;
+            --shadow-sm: 0 1px 2px 0 rgba(0, 0, 0, 0.05);
+            --shadow-md: 0 4px 6px -1px rgba(0, 0, 0, 0.1), 0 2px 4px -1px rgba(0, 0, 0, 0.06);
+        }
+
+        * { margin: 0; padding: 0; box-sizing: border-box; }
+        
+        body {
+            font-family: 'Inter', system-ui, -apple-system, sans-serif;
+            background: var(--bg-body);
+            color: var(--text-primary);
+            display: flex;
+            min-height: 100vh;
+            -webkit-font-smoothing: antialiased;
+        }
+
+        /* Modern Sidebar */
+        .sidebar {
+            width: 260px;
+            background: var(--bg-surface);
+            border-right: 1px solid var(--border-default);
+            display: flex;
+            flex-direction: column;
+            flex-shrink: 0;
+            position: sticky;
+            top: 0;
+            height: 100vh;
+        }
+
+        .sidebar-brand {
+            height: 72px;
+            display: flex;
+            align-items: center;
+            padding: 0 24px;
+            border-bottom: 1px solid var(--border-default);
+            color: var(--text-primary);
+            font-weight: 600;
+            font-size: 16px;
+            gap: 12px;
+        }
+
+        .sidebar-brand i {
+            color: var(--brand-primary);
+            font-size: 24px;
+        }
+
+        .nav-links {
+            padding: 24px 16px;
+            display: flex;
+            flex-direction: column;
+            gap: 8px;
+        }
+
+        .nav-links a {
+            display: flex;
+            align-items: center;
+            gap: 12px;
+            color: var(--text-secondary);
+            text-decoration: none;
+            padding: 12px 16px;
+            border-radius: var(--radius-md);
+            font-size: 14px;
+            font-weight: 500;
+            transition: all 0.2s cubic-bezier(0.4, 0, 0.2, 1);
+        }
+
+        .nav-links a i {
+            font-size: 20px;
+            transition: color 0.2s;
+        }
+
+        .nav-links a:hover {
+            background: var(--bg-surface-hover);
+            color: var(--text-primary);
+        }
+
+        .nav-links a.active {
+            background: var(--brand-primary-light);
+            color: var(--brand-primary);
+        }
+
+        /* Main Content Area */
+        .main {
+            flex: 1;
+            min-width: 0; /* Important for flex-child truncation */
+        }
+
+        .topbar {
+            height: 72px;
+            background: rgba(11, 15, 25, 0.8);
+            backdrop-filter: blur(12px);
+            border-bottom: 1px solid var(--border-default);
+            display: flex;
+            align-items: center;
+            padding: 0 40px;
+            position: sticky;
+            top: 0;
+            z-index: 10;
+        }
+
+        .content-container {
+            padding: 40px;
+            max-width: 1400px;
+            margin: 0 auto;
+        }
+
+        /* Typography */
+        h2 {
+            font-size: 24px;
+            font-weight: 600;
+            letter-spacing: -0.02em;
+            color: var(--text-primary);
+        }
+
+        h3 {
+            font-size: 18px;
+            font-weight: 600;
+            letter-spacing: -0.01em;
+            margin: 32px 0 16px;
+            color: var(--text-primary);
+        }
+
+        /* Modern Cards */
+        .card {
+            background: var(--bg-surface);
+            border: 1px solid var(--border-default);
+            border-radius: var(--radius-lg);
+            padding: 24px;
+            box-shadow: var(--shadow-sm);
+            transition: transform 0.2s, box-shadow 0.2s;
+        }
+
+        .stat-card {
+            display: flex;
+            flex-direction: column;
+            gap: 8px;
+        }
+
+        .stat-label {
+            font-size: 13px;
+            font-weight: 500;
+            color: var(--text-secondary);
+        }
+
+        .stat-value {
+            font-size: 32px;
+            font-weight: 600;
+            letter-spacing: -0.03em;
+            color: var(--text-primary);
+        }
+
+        .stat-subtext {
+            font-size: 13px;
+            font-weight: 500;
+        }
+
+        .stat-subtext.success { color: var(--success-text); }
+        .stat-subtext.danger { color: var(--danger-text); }
+        .stat-subtext.muted { color: var(--text-muted); }
+
+        /* Modern Data Tables */
+        .table-container {
+            background: var(--bg-surface);
+            border: 1px solid var(--border-default);
+            border-radius: var(--radius-lg);
+            overflow-x: auto;
+        }
+
+        table {
+            width: 100%;
+            border-collapse: collapse;
+            text-align: left;
+        }
+
+        th {
+            background: var(--bg-surface);
+            padding: 16px 24px;
+            font-size: 12px;
+            font-weight: 600;
+            text-transform: uppercase;
+            letter-spacing: 0.05em;
+            color: var(--text-muted);
+            border-bottom: 1px solid var(--border-default);
+            white-space: nowrap;
+        }
+
+        td {
+            padding: 16px 24px;
+            font-size: 14px;
+            color: var(--text-secondary);
+            border-bottom: 1px solid var(--border-default);
+            white-space: nowrap;
+        }
+
+        td strong {
+            color: var(--text-primary);
+            font-weight: 500;
+        }
+
+        tr:last-child td {
+            border-bottom: none;
+        }
+
+        tr:hover td {
+            background: var(--bg-surface-hover);
+        }
+
+        /* Links */
+        a {
+            color: var(--brand-primary);
+            text-decoration: none;
+            transition: color 0.15s;
+        }
+
+        a:hover {
+            color: var(--brand-primary-hover);
+        }
+
+        /* Minimalist Badges */
+        .badge {
+            display: inline-flex;
+            align-items: center;
+            padding: 4px 12px;
+            border-radius: 9999px;
+            font-size: 12px;
+            font-weight: 500;
+            letter-spacing: 0.02em;
+        }
+
+        .badge-green { 
+            background: var(--success-bg); 
+            color: var(--success-text); 
+            border: 1px solid rgba(52, 211, 153, 0.2);
+        }
+        .badge-red { 
+            background: var(--danger-bg); 
+            color: var(--danger-text); 
+            border: 1px solid rgba(248, 113, 113, 0.2);
+        }
+        .badge-yellow { 
+            background: var(--warning-bg); 
+            color: var(--warning-text); 
+            border: 1px solid rgba(251, 191, 36, 0.2);
+        }
+        .badge-neutral {
+            background: var(--bg-surface-active);
+            color: var(--text-primary);
+            border: 1px solid var(--border-hover);
+        }
+
+        /* Error States */
+        .error-banner {
+            background: var(--danger-bg);
+            border: 1px solid rgba(248, 113, 113, 0.2);
+            color: var(--danger-text);
+            padding: 16px 24px;
+            border-radius: var(--radius-md);
+            margin-bottom: 24px;
+            display: flex;
+            align-items: center;
+            gap: 12px;
+            font-size: 14px;
+            font-weight: 500;
+        }
+
+        .empty-state {
+            padding: 48px;
+            text-align: center;
+            color: var(--text-muted);
+            font-size: 14px;
+            display: flex;
+            flex-direction: column;
+            align-items: center;
+            gap: 12px;
+        }
+        
+        .empty-state i {
+            font-size: 32px;
+            color: var(--border-hover);
+        }
+
+        @media (max-width: 1024px) {
+            .content-container { padding: 24px; }
+            .topbar { padding: 0 24px; }
+        }
+
+        @media (max-width: 768px) {
+            body { flex-direction: column; }
+            .sidebar { 
+                width: 100%; 
+                height: auto; 
+                position: relative;
+                border-right: none;
+                border-bottom: 1px solid var(--border-default);
+            }
+            .sidebar-brand { justify-content: center; }
+            .nav-links { 
+                flex-direction: row; 
+                overflow-x: auto; 
+                padding: 16px;
+                gap: 16px;
+            }
+            .nav-links a { padding: 8px 16px; white-space: nowrap; }
+            .content-container { padding: 16px; }
+            .topbar { display: none; } /* Hide sticky topbar on mobile */
+        }
+    </style>
+</head>
+<body>
+    <nav class="sidebar">
+        <div class="sidebar-brand">
+            <i class="ph-fill ph-magnifying-glass"></i>
+            Willhaben Tracker
+        </div>
+        <div class="nav-links">
+            <a href="/" class="{{ 'active' if request.url.path == '/' else '' }}">
+                <i class="ph ph-squares-four"></i> Dashboard
+            </a>
+            <a href="/keywords" class="{{ 'active' if 'keywords' in request.url.path else '' }}">
+                <i class="ph ph-tag"></i> Keywords
+            </a>
+            <a href="/users" class="{{ 'active' if request.url.path == '/users' else '' }}">
+                <i class="ph ph-users"></i> Users
+            </a>
+            <a href="/ads" class="{{ 'active' if request.url.path == '/ads' else '' }}">
+                <i class="ph ph-shopping-bag"></i> Ads
+            </a>
+        </div>
+    </nav>
+    <main class="main">
+        <div class="topbar">
+            <h2>{% block title_in_topbar %}{% endblock %}</h2>
+        </div>
+        <div class="content-container">
+            {% if error %}
+            <div class="error-banner">
+                <i class="ph-fill ph-warning-circle"></i>
+                {{ error }}
+            </div>
+            {% endif %}
+            {% block content %}{% endblock %}
+        </div>
+    </main>
+</body>
+</html>

worker/src/templates/dashboard.html

@@ -0,0 +1,130 @@
+{% extends "base.html" %}
+{% block title %}Dashboard — Willhaben Tracker{% endblock %}
+{% block title_in_topbar %}Dashboard{% endblock %}
+
+{% block content %}
+{% if data %}
+<h3 style="margin-top: 0;">Overview</h3>
+<div style="display: grid; grid-template-columns: repeat(auto-fit, minmax(240px, 1fr)); gap: 24px; margin-bottom: 32px;">
+    
+    <div class="card stat-card">
+        <div class="stat-label">Total Keywords</div>
+        <div class="stat-value">{{ data.total_keywords }}</div>
+        <div class="stat-subtext success">
+            <i class="ph-fill ph-check-circle"></i> {{ data.active_keywords }} active
+        </div>
+    </div>
+
+    <div class="card stat-card">
+        <div class="stat-label">Total Ads Tracked</div>
+        <div class="stat-value">{{ data.total_ads }}</div>
+        <div class="stat-subtext muted">Across all time</div>
+    </div>
+
+    <div class="card stat-card">
+        <div class="stat-label">Active Users</div>
+        <div class="stat-value">{{ data.total_users }}</div>
+        <div class="stat-subtext muted">Subscribed to updates</div>
+    </div>
+
+    <div class="card stat-card">
+        <div class="stat-label">Notifications Sent</div>
+        <div class="stat-value">{{ data.notifications_sent }}</div>
+        <div class="stat-subtext muted">Via Telegram</div>
+    </div>
+
+</div>
+
+<h3>System Health</h3>
+<div style="display: grid; grid-template-columns: repeat(auto-fit, minmax(300px, 1fr)); gap: 24px;">
+
+    <div class="card stat-card" style="border-left: 4px solid var(--border-hover);">
+        <div class="stat-label">Last Scheduler Run</div>
+        <div class="stat-value" style="font-size: 24px;">
+            {{ data.last_scheduler.strftime('%Y-%m-%d %H:%M:%S') if data.last_scheduler else 'Never' }}
+        </div>
+        <div class="stat-subtext muted">Worker polling rhythm</div>
+    </div>
+
+    <div class="card stat-card" style="border-left: 4px solid {% if data.proxy_enabled %}var(--success-text){% elif data.proxy_available %}var(--warning-text){% else %}var(--border-hover){% endif %};">
+        <div class="stat-label">HTTP Proxy</div>
+        <div class="stat-value" style="font-size: 24px; color: {% if data.proxy_enabled %}var(--success-text){% elif data.proxy_available %}var(--warning-text){% else %}var(--text-primary){% endif %};">
+            {% if data.proxy_enabled %}Active{% elif data.proxy_available %}Bypassed{% else %}Unavailable{% endif %}
+        </div>
+        <div class="stat-subtext muted">
+            {% if data.proxy_enabled %}
+            Scraper uses proxy
+            {% elif data.proxy_available %}
+            Proxy present, scraper direct
+            {% else %}
+            No proxy in .env
+            {% endif %}
+        </div>
+        <form method="post" action="/admin/proxy" style="margin-top: 12px;">
+            <button type="submit" {% if not data.proxy_available %}disabled{% endif %} style="border: 1px solid var(--border-hover); background: {% if data.proxy_enabled %}var(--warning-bg){% else %}var(--bg-surface-active){% endif %}; color: var(--text-primary); border-radius: 8px; padding: 8px 12px; cursor: {% if data.proxy_available %}pointer{% else %}not-allowed{% endif %}; font-weight: 600; opacity: {% if data.proxy_available %}1{% else %}0.55{% endif %};">
+                {{ 'Disable Proxy' if data.proxy_enabled else 'Enable Proxy' }}
+            </button>
+        </form>
+    </div>
+
+    <div class="card stat-card" style="border-left: 4px solid var(--brand-primary);">
+        <div class="stat-label">Network Egress</div>
+        <div style="display: grid; gap: 10px; margin-top: 10px;">
+            <div>
+                <div class="stat-subtext muted">System IP</div>
+                <div style="font-size: 18px; font-weight: 600; line-height: 1.4;">
+                    {{ data.system_country_flag }}
+                    {{ data.system_public_ip or 'Unknown' }}
+                    {% if data.system_country_code %}
+                    <span style="font-size: 12px; color: var(--text-muted);">({{ data.system_country_code }})</span>
+                    {% endif %}
+                </div>
+            </div>
+            <div>
+                <div class="stat-subtext muted">Last Used IP</div>
+                <div style="font-size: 18px; font-weight: 600; line-height: 1.4; color: var(--brand-primary);">
+                    {{ data.last_used_country_flag }}
+                    {{ data.last_used_public_ip or 'Unknown' }}
+                    {% if data.last_used_country_code %}
+                    <span style="font-size: 12px; color: var(--text-muted);">({{ data.last_used_country_code }})</span>
+                    {% endif %}
+                </div>
+            </div>
+        </div>
+    </div>
+
+    <div class="card stat-card" style="border-left: 4px solid {% if data.turbo_active %}var(--warning-text){% else %}var(--border-hover){% endif %};">
+        <div class="stat-label">Turbo Mode (Admin)</div>
+        <div class="stat-value" style="font-size: 24px; color: {% if data.turbo_active %}var(--warning-text){% else %}var(--text-primary){% endif %};">
+            {{ 'Active' if data.turbo_active else ('Armed' if data.turbo_mode else 'Off') }}
+        </div>
+        <div class="stat-subtext muted">
+            {% if data.proxy_enabled %}
+            Scheduler runs every ~{{ '%.1f'|format(data.turbo_effective_sleep_s) }}s
+            {% else %}
+            Activate proxy first to enable turbo
+            {% endif %}
+        </div>
+        {% if data.seconds_until_next is not none %}
+        <div class="stat-subtext muted">Next cycle in ~{{ data.seconds_until_next }}s</div>
+        {% endif %}
+        <form method="post" action="/admin/turbo" style="margin-top: 12px;">
+            <button type="submit" {% if not data.proxy_enabled %}disabled{% endif %} style="border: 1px solid var(--border-hover); background: {% if data.turbo_mode %}var(--warning-bg){% else %}var(--bg-surface-active){% endif %}; color: var(--text-primary); border-radius: 10px; padding: 8px 12px; cursor: {% if data.proxy_enabled %}pointer{% else %}not-allowed{% endif %}; font-weight: 600; opacity: {% if data.proxy_enabled %}1{% else %}0.55{% endif %};">
+                {{ 'Disable Turbo' if data.turbo_mode else 'Enable Turbo' }}
+            </button>
+        </form>
+    </div>
+
+    <div class="card stat-card" style="border-left: 4px solid {% if data.queue_dead > 0 %}var(--danger-text){% else %}var(--success-text){% endif %};">
+        <div class="stat-label">Delivery Queue</div>
+        <div class="stat-value" style="font-size: 24px;">
+            {{ data.queue_pending }} pending  
+        </div>
+        <div class="stat-subtext {% if data.queue_dead > 0 %}danger{% else %}muted{% endif %}">
+            {{ data.queue_dead }} dead letters
+        </div>
+    </div>
+
+</div>
+{% endif %}
+{% endblock %}

worker/src/templates/keyword_detail.html

@@ -0,0 +1,165 @@
+{% extends "base.html" %}
+{% block title %}{{ keyword.keyword }} — Willhaben Tracker{% endblock %}
+{% block title_in_topbar %}
+<div style="display: flex; align-items: center; gap: 12px;">
+    {{ keyword.keyword }}
+    <span style="font-size: 14px; font-weight: normal; color: var(--text-muted); font-family: monospace;">{{ keyword.id[:8] }}</span>
+</div>
+{% endblock %}
+
+{% block content %}
+{% if keyword %}
+<h3 style="margin-top: 0;">Configuration</h3>
+<div style="display: grid; grid-template-columns: repeat(auto-fit, minmax(200px, 1fr)); gap: 24px; margin-bottom: 32px;">
+    <div class="card stat-card">
+        <div class="stat-label">Status</div>
+        <div style="margin-top: 8px;">
+            {% if keyword.is_active %}
+            <span class="badge badge-green"><i class="ph-fill ph-play-circle" style="margin-right: 4px;"></i> Active</span>
+            {% else %}
+            <span class="badge badge-neutral"><i class="ph-fill ph-pause-circle" style="margin-right: 4px;"></i> Stopped</span>
+            {% endif %}
+        </div>
+    </div>
+    
+    <div class="card stat-card">
+        <div class="stat-label">Polling Interval</div>
+        <div class="stat-value">{{ keyword.interval_minutes }}m</div>
+    </div>
+    
+    <div class="card stat-card">
+        <div class="stat-label">Subscribers</div>
+        <div class="stat-value">{{ subscriber_count }}</div>
+        <div class="stat-subtext muted">Users tracking this</div>
+    </div>
+    
+    <div class="card stat-card">
+        <div class="stat-label">Price Filters</div>
+        <div class="stat-value" style="font-size: 20px; line-height: 1.6;">
+            {% if keyword.price_min %}€{{ (keyword.price_min / 100)|round(2) }}{% else %}0{% endif %}
+            <span style="color: var(--text-muted); font-weight: 400; padding: 0 4px;">to</span>
+            {% if keyword.price_max %}€{{ (keyword.price_max / 100)|round(2) }}{% else %}Max{% endif %}
+        </div>
+    </div>
+    
+    <div class="card stat-card">
+        <div class="stat-label">Postcodes Filter</div>
+        <div class="stat-value" style="font-size: 20px; line-height: 1.6;">
+            {{ ', '.join(keyword.allowed_postcodes) if keyword.allowed_postcodes else 'Any' }}
+        </div>
+    </div>
+    
+    <div class="card stat-card">
+        <div class="stat-label">Last Successful Scrape</div>
+        <div class="stat-value" style="font-size: 20px; line-height: 1.6;">
+            {% if keyword.last_scraped_at %}
+                {{ keyword.last_scraped_at.strftime('%m/%d %H:%M') }}
+            {% else %}
+                <span class="muted">Never</span>
+            {% endif %}
+        </div>
+    </div>
+</div>
+
+<h3>Recent Matches</h3>
+{% if ads %}
+<div class="table-container" style="margin-bottom: 32px;">
+    <table>
+        <thead>
+            <tr>
+                <th>Ad</th>
+                <th>Location</th>
+                <th>Link</th>
+            </tr>
+        </thead>
+        <tbody>
+            {% for ad in ads %}
+            <tr>
+                <td style="max-width: 400px; white-space: normal;">
+                    <div style="font-weight: 500; margin-bottom: 4px; line-height: 1.4;">
+                        {{ ad.title }}
+                    </div>
+                    <div style="display: flex; gap: 12px; font-size: 13px;">
+                        <span style="font-weight: 600; color: var(--success-text);">{{ format_price(ad.price) }}</span>
+                        {% if ad.published_at %}
+                        <span style="color: var(--text-muted);">
+                            <i class="ph ph-calendar-blank" style="vertical-align: middle;"></i> 
+                            {{ ad.published_at.strftime('%m/%d %H:%M') }}
+                        </span>
+                        {% endif %}
+                    </div>
+                </td>
+                <td>
+                    <div style="display: flex; align-items: center; gap: 4px;">
+                        <i class="ph ph-map-pin" style="color: var(--text-muted);"></i>
+                        {% if ad.postcode %}{{ ad.postcode }} • {% endif %}
+                        {{ ad.location or '—' }}
+                    </div>
+                </td>
+                <td>
+                    <a href="{{ ad.url }}" target="_blank" style="display: inline-flex; align-items: center; gap: 4px; padding: 6px 12px; background: var(--bg-surface-active); border-radius: var(--radius-md); font-size: 13px; font-weight: 500;">
+                        Open <i class="ph-bold ph-arrow-up-right"></i>
+                    </a>
+                </td>
+            </tr>
+            {% endfor %}
+        </tbody>
+    </table>
+</div>
+{% else %}
+<div class="empty-state" style="margin-bottom: 32px; background: var(--bg-surface); border: 1px solid var(--border-default); border-radius: var(--radius-lg);">
+    <i class="ph ph-shopping-bag"></i>
+    <p>No ads found for this keyword yet.</p>
+</div>
+{% endif %}
+
+<h3>Scrape Telemetry</h3>
+{% if logs %}
+<div class="table-container">
+    <table>
+        <thead>
+            <tr>
+                <th>Execution Time</th>
+                <th>Result</th>
+                <th>Throughput</th>
+                <th>Diagnostics</th>
+            </tr>
+        </thead>
+        <tbody>
+            {% for log in logs %}
+            <tr>
+                <td>
+                    <div style="font-weight: 500;">{{ log.scraped_at.strftime('%Y-%m-%d') }}</div>
+                    <div style="font-size: 13px; color: var(--text-muted); margin-top: 2px;">{{ log.scraped_at.strftime('%H:%M:%S') }}</div>
+                </td>
+                <td>
+                    {% if log.status == 'success' %}
+                    <span class="badge badge-green"><i class="ph-fill ph-check-circle" style="margin-right: 4px;"></i> Success</span>
+                    {% elif log.status == 'rate_limited' %}
+                    <span class="badge badge-yellow"><i class="ph-fill ph-warning" style="margin-right: 4px;"></i> Blocked</span>
+                    {% else %}
+                    <span class="badge badge-red"><i class="ph-fill ph-x-circle" style="margin-right: 4px;"></i> Error</span>
+                    {% endif %}
+                </td>
+                <td>
+                    <div style="display: flex; gap: 16px;">
+                        <span><strong>{{ log.ads_found }}</strong> <span style="color: var(--text-muted); font-size: 13px;">scanned</span></span>
+                        <span><strong>{{ log.new_ads }}</strong> <span style="color: var(--text-muted); font-size: 13px;">new</span></span>
+                    </div>
+                </td>
+                <td style="max-width: 300px; white-space: normal; font-size: 13px; font-family: monospace; color: var(--danger-text);">
+                    {{ log.error_message or '—' }}
+                </td>
+            </tr>
+            {% endfor %}
+        </tbody>
+    </table>
+</div>
+{% else %}
+<div class="empty-state" style="background: var(--bg-surface); border: 1px solid var(--border-default); border-radius: var(--radius-lg);">
+    <i class="ph ph-terminal-window"></i>
+    <p>No telemetry recorded yet.</p>
+</div>
+{% endif %}
+{% endif %}
+{% endblock %}

worker/src/templates/keywords.html

@@ -0,0 +1,70 @@
+{% extends "base.html" %}
+{% block title %}Keywords — Willhaben Tracker{% endblock %}
+{% block title_in_topbar %}Keywords{% endblock %}
+
+{% block content %}
+{% if not error and keywords %}
+<div class="table-container">
+    <table>
+        <thead>
+            <tr>
+                <th>Keyword</th>
+                <th>Status</th>
+                <th>Interval</th>
+                <th>Filters</th>
+                <th>Subscribers</th>
+                <th>Last Scraped</th>
+            </tr>
+        </thead>
+        <tbody>
+            {% for kw in keywords %}
+            <tr>
+                <td>
+                    <a href="/keywords/{{ kw.id }}" style="font-weight: 600;">{{ kw.keyword }}</a>
+                    <div style="font-size: 12px; color: var(--text-muted); margin-top: 4px; font-family: monospace;">{{ kw.id[:8] }}</div>
+                </td>
+                <td>
+                    {% if kw.is_active %}
+                    <span class="badge badge-green"><i class="ph-fill ph-play-circle" style="margin-right: 4px;"></i> Active</span>
+                    {% else %}
+                    <span class="badge badge-neutral"><i class="ph-fill ph-pause-circle" style="margin-right: 4px;"></i> Stopped</span>
+                    {% endif %}
+                </td>
+                <td>{{ kw.interval_minutes }}m</td>
+                <td style="font-size: 13px;">
+                    {% if kw.price_min or kw.price_max %}
+                        <div style="margin-bottom: 2px;">
+                            <i class="ph ph-currency-eur" style="color: var(--text-muted); vertical-align: middle;"></i>
+                            {{ (kw.price_min / 100)|round(2) if kw.price_min else '0' }} — {{ (kw.price_max / 100)|round(2) if kw.price_max else 'Max' }}
+                        </div>
+                    {% endif %}
+                    {% if kw.allowed_postcodes %}
+                        <div>
+                            <i class="ph ph-map-pin" style="color: var(--text-muted); vertical-align: middle;"></i>
+                            {{ kw.allowed_postcodes|length }} code(s)
+                        </div>
+                    {% endif %}
+                    {% if not kw.price_min and not kw.price_max and not kw.allowed_postcodes %}
+                        <span style="color: var(--text-muted);">None</span>
+                    {% endif %}
+                </td>
+                <td><strong>{{ kw.subscriber_count }}</strong> <i class="ph-fill ph-users" style="color: var(--text-muted); font-size: 12px;"></i></td>
+                <td>
+                    {% if kw.last_scraped_at %}
+                    {{ kw.last_scraped_at.strftime('%Y-%m-%d %H:%M:%S') }}
+                    {% else %}
+                    <span style="color: var(--text-muted);">Never</span>
+                    {% endif %}
+                </td>
+            </tr>
+            {% endfor %}
+        </tbody>
+    </table>
+</div>
+{% elif not error %}
+<div class="empty-state">
+    <i class="ph ph-tag"></i>
+    <p>No keywords are currently being tracked.</p>
+</div>
+{% endif %}
+{% endblock %}

worker/src/templates/users.html

@@ -0,0 +1,75 @@
+{% extends "base.html" %}
+{% block title %}Users — Willhaben Tracker{% endblock %}
+{% block title_in_topbar %}Registered Users{% endblock %}
+
+{% block content %}
+{% if not error and users %}
+<div class="table-container">
+    <table>
+        <thead>
+            <tr>
+                <th>User</th>
+                <th>Status</th>
+                <th>Role</th>
+                <th>Mute Schedule</th>
+                <th>Delivery Mode</th>
+            </tr>
+        </thead>
+        <tbody>
+            {% for user in users %}
+            <tr>
+                <td>
+                    <div style="font-weight: 600;">{{ user.first_name or 'Unknown User' }}</div>
+                    <div style="font-size: 13px; color: var(--text-muted); margin-top: 4px;">
+                        {% if user.username %}@{{ user.username }} • {% endif %}{{ user.telegram_id }}
+                    </div>
+                </td>
+                <td>
+                    {% if user.is_active %}
+                    <span class="badge badge-green"><i class="ph-fill ph-check-circle" style="margin-right: 4px;"></i> Active</span>
+                    {% else %}
+                    <span class="badge badge-red"><i class="ph-fill ph-x-circle" style="margin-right: 4px;"></i> Banned</span>
+                    {% endif %}
+                </td>
+                <td>
+                    {% if user.is_admin %}
+                    <span class="badge badge-yellow"><i class="ph-fill ph-shield-star" style="margin-right: 4px;"></i> Admin</span>
+                    {% else %}
+                    <span class="badge badge-neutral"><i class="ph-fill ph-user" style="margin-right: 4px;"></i> User</span>
+                    {% endif %}
+                </td>
+                <td>
+                    {% if user.mute_start and user.mute_end %}
+                    <div style="display: flex; align-items: center; gap: 6px;">
+                        <i class="ph ph-moon" style="color: var(--text-muted);"></i>
+                        <span>{{ user.mute_start.strftime('%H:%M') }} — {{ user.mute_end.strftime('%H:%M') }}</span>
+                    </div>
+                    {% else %}
+                    <span style="color: var(--text-muted);">None</span>
+                    {% endif %}
+                </td>
+                <td>
+                    {% if user.digest_mode %}
+                    <div style="display: flex; align-items: center; gap: 6px;">
+                        <i class="ph-fill ph-envelope-simple" style="color: var(--success-text);"></i>
+                        <span>Digest every {{ user.digest_interval }}m</span>
+                    </div>
+                    {% else %}
+                    <div style="display: flex; align-items: center; gap: 6px;">
+                        <i class="ph-fill ph-lightning" style="color: var(--brand-primary);"></i>
+                        <span>Immediate</span>
+                    </div>
+                    {% endif %}
+                </td>
+            </tr>
+            {% endfor %}
+        </tbody>
+    </table>
+</div>
+{% elif not error %}
+<div class="empty-state">
+    <i class="ph ph-users"></i>
+    <p>No users found in the database.</p>
+</div>
+{% endif %}
+{% endblock %}

worker/src/web.py

@@ -0,0 +1,397 @@
+import logging
+import time
+from contextlib import asynccontextmanager
+from datetime import datetime, timezone
+
+import httpx
+from fastapi import FastAPI, HTTPException, Request
+from fastapi.responses import HTMLResponse, JSONResponse, RedirectResponse
+from fastapi.templating import Jinja2Templates
+
+from db import get_pool
+from scraper import get_network_status, refresh_network_status
+from settings import get_proxy_enabled, get_turbo_mode, proxy_available, set_proxy_enabled, set_turbo_mode
+
+logger = logging.getLogger(__name__)
+
+templates = Jinja2Templates(directory="templates")
+_country_cache: dict[str, tuple[float, str]] = {}
+_COUNTRY_CACHE_TTL_S = 3600
+
+
+async def query(sql: str, *args) -> list:
+    """Execute a query and return rows as dicts with UUIDs converted to strings."""
+    try:
+        pool = await get_pool()
+        async with pool.acquire() as conn:
+            rows = await conn.fetch(sql, *args)
+            result = []
+            for row in rows:
+                d = dict(row)
+                for k, v in d.items():
+                    if hasattr(v, 'hex'):  # asyncpg UUID
+                        d[k] = str(v)
+                result.append(d)
+            return result
+    except Exception as e:
+        logger.error("Database query error: %s", e)
+        raise
+
+
+async def query_one(sql: str, *args) -> dict | None:
+    """Execute a query and return a single row as dict with UUIDs converted to strings."""
+    try:
+        pool = await get_pool()
+        async with pool.acquire() as conn:
+            row = await conn.fetchrow(sql, *args)
+            if not row:
+                return None
+            d = dict(row)
+            for k, v in d.items():
+                if hasattr(v, 'hex'):  # asyncpg UUID
+                    d[k] = str(v)
+            return d
+    except Exception as e:
+        logger.error("Database query error: %s", e)
+        raise
+
+
+async def value(sql: str, *args):
+    """Execute a query and return a single value."""
+    try:
+        pool = await get_pool()
+        async with pool.acquire() as conn:
+            return await conn.fetchval(sql, *args)
+    except Exception as e:
+        logger.error("Database query error: %s", e)
+        raise
+
+
+def format_price(cents: int | None) -> str:
+    if cents is None:
+        return "—"
+    return f"€{cents / 100:.2f}"
+
+
+def format_postcodes(postcodes: list | None) -> str:
+    if not postcodes:
+        return "—"
+    return ", ".join(str(p) for p in postcodes)
+
+
+def _flag_from_country_code(country_code: str | None) -> str:
+    if not country_code or len(country_code) != 2:
+        return ""
+    code = country_code.upper()
+    if not code.isalpha():
+        return ""
+    return chr(127397 + ord(code[0])) + chr(127397 + ord(code[1]))
+
+
+async def _country_code_for_ip(ip_addr: str | None) -> str | None:
+    if not ip_addr:
+        return None
+
+    now = time.time()
+    cached = _country_cache.get(ip_addr)
+    if cached and now - cached[0] < _COUNTRY_CACHE_TTL_S:
+        return cached[1]
+
+    try:
+        async with httpx.AsyncClient(timeout=5.0, trust_env=False) as client:
+            resp = await client.get(f"https://ipapi.co/{ip_addr}/country/")
+            resp.raise_for_status()
+            code = resp.text.strip().upper()
+            if len(code) == 2 and code.isalpha():
+                _country_cache[ip_addr] = (now, code)
+                return code
+    except Exception:
+        logger.debug("Could not resolve country for IP %s", ip_addr, exc_info=True)
+
+    try:
+        async with httpx.AsyncClient(timeout=5.0, trust_env=False) as client:
+            resp = await client.get(f"https://ipwho.is/{ip_addr}")
+            resp.raise_for_status()
+            payload = resp.json()
+            code = str(payload.get("country_code", "")).upper()
+            if len(code) == 2 and code.isalpha():
+                _country_cache[ip_addr] = (now, code)
+                return code
+    except Exception:
+        logger.debug("Fallback country lookup failed for IP %s", ip_addr, exc_info=True)
+
+    return None
+
+
+@asynccontextmanager
+async def lifespan(app: FastAPI):
+    logger.info("Web UI starting up")
+    yield
+    logger.info("Web UI shutting down")
+
+
+app = FastAPI(title="Willhaben Tracker Web UI", lifespan=lifespan)
+
+
+# ─── Routes ───────────────────────────────────────────────────
+
+@app.get("/", response_class=HTMLResponse)
+async def dashboard(request: Request):
+    pool = await get_pool()
+    try:
+        total_keywords = await value("SELECT COUNT(*) FROM keywords")
+        active_keywords = await value("SELECT COUNT(*) FROM keywords WHERE is_active = true")
+        total_ads = await value("SELECT COUNT(*) FROM ads")
+        total_users = await value("SELECT COUNT(*) FROM users WHERE is_active = true")
+        notifications_sent = await value("SELECT COUNT(*) FROM notifications")
+        queue_pending = await value("SELECT COUNT(*) FROM notification_queue WHERE status = 'pending'")
+        queue_dead = await value("SELECT COUNT(*) FROM notification_queue WHERE status = 'dead'")
+        last_scheduler = await query_one(
+            "SELECT scraped_at FROM scrape_logs ORDER BY scraped_at DESC LIMIT 1"
+        )
+    except Exception as e:
+        logger.error("Dashboard query error: %s", e)
+        return templates.TemplateResponse(
+            "dashboard.html",
+            {"request": request, "error": "Database unavailable", "data": None},
+        )
+
+    proxy_configured = proxy_available()
+    proxy_setting_enabled = await get_proxy_enabled(pool)
+
+    try:
+        network = await refresh_network_status()
+    except Exception:
+        logger.debug("Could not refresh network status", exc_info=True)
+        network = get_network_status()
+    system_ip = network.get("system_public_ip")
+    last_used_ip = network.get("last_used_public_ip")
+    proxy_enabled = proxy_configured and proxy_setting_enabled
+    system_country = await _country_code_for_ip(system_ip if isinstance(system_ip, str) else None)
+    last_used_country = await _country_code_for_ip(last_used_ip if isinstance(last_used_ip, str) else None)
+    turbo_mode = await get_turbo_mode(pool)
+    turbo_active = turbo_mode and proxy_enabled
+
+    turbo_base_sleep_s = 30.0
+    turbo_effective_sleep_s = turbo_base_sleep_s / (10 if turbo_active else 1)
+
+    next_refresh_at = None
+    if last_scheduler and last_scheduler.get("scraped_at"):
+        next_refresh_at = last_scheduler["scraped_at"]
+        if next_refresh_at.tzinfo is None:
+            next_refresh_at = next_refresh_at.replace(tzinfo=timezone.utc)
+        next_refresh_at = next_refresh_at.timestamp() + turbo_effective_sleep_s
+
+    now_epoch = datetime.now(tz=timezone.utc).timestamp()
+    seconds_until_next = None
+    if next_refresh_at is not None:
+        seconds_until_next = max(0, int(round(next_refresh_at - now_epoch)))
+
+    data = {
+        "total_keywords": total_keywords or 0,
+        "active_keywords": active_keywords or 0,
+        "total_ads": total_ads or 0,
+        "total_users": total_users or 0,
+        "notifications_sent": notifications_sent or 0,
+        "queue_pending": queue_pending or 0,
+        "queue_dead": queue_dead or 0,
+        "last_scheduler": last_scheduler["scraped_at"] if last_scheduler else None,
+        "proxy_available": proxy_configured,
+        "proxy_enabled": proxy_enabled,
+        "proxy_setting_enabled": proxy_setting_enabled,
+        "system_public_ip": system_ip,
+        "last_used_public_ip": last_used_ip,
+        "system_country_code": system_country,
+        "last_used_country_code": last_used_country,
+        "system_country_flag": _flag_from_country_code(system_country),
+        "last_used_country_flag": _flag_from_country_code(last_used_country),
+        "turbo_mode": turbo_mode,
+        "turbo_active": turbo_active,
+        "turbo_effective_sleep_s": turbo_effective_sleep_s,
+        "seconds_until_next": seconds_until_next,
+    }
+    return templates.TemplateResponse("dashboard.html", {"request": request, "error": None, "data": data})
+
+
+@app.post("/admin/turbo")
+async def toggle_turbo(request: Request):
+    pool = await get_pool()
+    current = await get_turbo_mode(pool)
+    await set_turbo_mode(pool, not current)
+    return RedirectResponse(url="/", status_code=303)
+
+
+@app.post("/admin/proxy")
+async def toggle_proxy(request: Request):
+    if not proxy_available():
+        return RedirectResponse(url="/", status_code=303)
+
+    pool = await get_pool()
+    current = await get_proxy_enabled(pool)
+    await set_proxy_enabled(pool, not current)
+    return RedirectResponse(url="/", status_code=303)
+
+
+@app.get("/keywords", response_class=HTMLResponse)
+async def keywords_list(request: Request):
+    try:
+        keywords = await query(
+            """
+            SELECT k.*,
+                   COUNT(DISTINCT ks.user_id) AS subscriber_count
+            FROM keywords k
+            LEFT JOIN keyword_subscriptions ks ON ks.keyword_id = k.id
+            GROUP BY k.id
+            ORDER BY k.created_at DESC
+            """
+        )
+    except Exception as e:
+        logger.error("Keywords query error: %s", e)
+        return templates.TemplateResponse(
+            "keywords.html",
+            {"request": request, "error": "Database unavailable", "keywords": []},
+        )
+
+    return templates.TemplateResponse(
+        "keywords.html",
+        {"request": request, "error": None, "keywords": keywords},
+    )
+
+
+@app.get("/keywords/{keyword_id}", response_class=HTMLResponse)
+async def keyword_detail(request: Request, keyword_id: str):
+    try:
+        kw = await query_one("SELECT * FROM keywords WHERE id = $1", keyword_id)
+        if not kw:
+            raise HTTPException(status_code=404, detail="Keyword not found")
+
+        subscriber_count = await value(
+            "SELECT COUNT(*) FROM keyword_subscriptions WHERE keyword_id = $1", keyword_id
+        )
+
+        # Get recent ads associated with this keyword via scrape_logs
+        ads = await query(
+            """
+            SELECT DISTINCT ON (a.id) a.*, sl.scraped_at as last_scrape
+            FROM ads a
+            JOIN scrape_logs sl ON sl.keyword_id = $1
+            ORDER BY a.id, sl.scraped_at DESC
+            LIMIT 20
+            """,
+            keyword_id,
+        )
+
+        logs = await query(
+            """
+            SELECT * FROM scrape_logs
+            WHERE keyword_id = $1
+            ORDER BY scraped_at DESC
+            LIMIT 10
+            """,
+            keyword_id,
+        )
+    except HTTPException:
+        raise
+    except Exception as e:
+        logger.error("Keyword detail error: %s", e)
+        return templates.TemplateResponse(
+            "keyword_detail.html",
+            {"request": request, "error": "Database unavailable", "keyword": None, "ads": [], "logs": [], "subscriber_count": 0},
+        )
+
+    return templates.TemplateResponse(
+        "keyword_detail.html",
+        {
+            "request": request,
+            "error": None,
+            "keyword": kw,
+            "ads": ads,
+            "logs": logs,
+            "subscriber_count": subscriber_count or 0,
+            "format_price": format_price,
+        },
+    )
+
+
+@app.get("/users", response_class=HTMLResponse)
+async def users_list(request: Request):
+    try:
+        users = await query(
+            """
+            SELECT u.*,
+                   us.mute_start,
+                   us.mute_end,
+                   us.digest_mode,
+                   us.digest_interval
+            FROM users u
+            LEFT JOIN user_settings us ON us.telegram_id = u.telegram_id::text
+            ORDER BY u.created_at DESC
+            """
+        )
+    except Exception as e:
+        logger.error("Users query error: %s", e)
+        return templates.TemplateResponse(
+            "users.html",
+            {"request": request, "error": "Database unavailable", "users": []},
+        )
+
+    return templates.TemplateResponse(
+        "users.html",
+        {"request": request, "error": None, "users": users},
+    )
+
+
+@app.get("/ads", response_class=HTMLResponse)
+async def ads_list(request: Request):
+    try:
+        ads = await query(
+            """
+            SELECT * FROM ads
+            ORDER BY first_seen_at DESC
+            LIMIT 50
+            """
+        )
+    except Exception as e:
+        logger.error("Ads query error: %s", e)
+        return templates.TemplateResponse(
+            "ads.html",
+            {"request": request, "error": "Database unavailable", "ads": []},
+        )
+
+    return templates.TemplateResponse(
+        "ads.html",
+        {"request": request, "error": None, "ads": ads, "format_price": format_price},
+    )
+
+
+@app.get("/stats")
+async def stats_json():
+    try:
+        total_keywords = await value("SELECT COUNT(*) FROM keywords")
+        active_keywords = await value("SELECT COUNT(*) FROM keywords WHERE is_active = true")
+        total_ads = await value("SELECT COUNT(*) FROM ads")
+        total_users = await value("SELECT COUNT(*) FROM users WHERE is_active = true")
+        notifications_sent = await value("SELECT COUNT(*) FROM notifications")
+        queue_pending = await value("SELECT COUNT(*) FROM notification_queue WHERE status = 'pending'")
+        queue_dead = await value("SELECT COUNT(*) FROM notification_queue WHERE status = 'dead'")
+        queue_failed = await value("SELECT COUNT(*) FROM notification_queue WHERE status = 'failed'")
+        digest_buffered = await value("SELECT COUNT(*) FROM digest_buffer")
+        last_scheduler = await query_one(
+            "SELECT scraped_at FROM scrape_logs ORDER BY scraped_at DESC LIMIT 1"
+        )
+    except Exception as e:
+        logger.error("Stats query error: %s", e)
+        return JSONResponse(status_code=503, content={"error": "Database unavailable"})
+
+    return {
+        "total_keywords": total_keywords or 0,
+        "active_keywords": active_keywords or 0,
+        "total_ads": total_ads or 0,
+        "total_users": total_users or 0,
+        "notifications_sent": notifications_sent or 0,
+        "queue_pending": queue_pending or 0,
+        "queue_dead": queue_dead or 0,
+        "queue_failed": queue_failed or 0,
+        "digest_buffered": digest_buffered or 0,
+        "last_scheduler_run": last_scheduler["scraped_at"].isoformat() if last_scheduler else None,
+    }

worker/tests/conftest.py

@@ -0,0 +1,84 @@
+"""Shared pytest fixtures for the willhaben-tracker test suite."""
+
+from datetime import datetime, timezone
+from unittest.mock import AsyncMock, MagicMock
+
+import pytest
+
+
+def pytest_configure(config):
+    """Enable asyncio auto mode for all async tests."""
+    config.addinivalue_line("markers", "asyncio: mark test as async")
+
+
+@pytest.fixture
+def mock_pool():
+    """Mock asyncpg.Pool with fetch/fetchrow/execute/fetchval methods."""
+    pool = MagicMock()
+    pool.fetch = AsyncMock(return_value=[])
+    pool.fetchrow = AsyncMock(return_value=None)
+    pool.execute = AsyncMock(return_value=None)
+    pool.fetchval = AsyncMock(return_value=None)
+    return pool
+
+
+@pytest.fixture
+def mock_bot():
+    """Mock ExtBot with send_message/send_photo methods."""
+    bot = MagicMock()
+    bot.send_message = AsyncMock(return_value=MagicMock(message_id=123))
+    bot.send_photo = AsyncMock(return_value=MagicMock(message_id=123))
+    return bot
+
+
+@pytest.fixture
+def sample_ad_data():
+    """Sample willhaben ad JSON dict matching the API response format."""
+    return {
+        "id": "12345678",
+        "description": "A used bicycle",
+        "attributes": {
+            "attribute": [
+                {"name": "HEADING", "values": ["Mountain Bike 2024"]},
+                {"name": "PRICE/AMOUNT", "values": ["250"]},
+                {"name": "LOCATION", "values": ["Vienna"]},
+                {"name": "POSTCODE", "values": ["1010"]},
+                {"name": "SEO_URL", "values": ["mountain-bike-2024/12345678"]},
+                {"name": "PUBLISHED_String", "values": ["2024-01-15T10:30:00Z"]},
+                {"name": "CHANGED_String", "values": ["2024-01-15T12:00:00Z"]},
+            ]
+        },
+        "advertImageList": {
+            "advertImage": [
+                {"referenceImageUrl": "https://img.willhaben.at/img123.jpg"}
+            ]
+        },
+    }
+
+
+@pytest.fixture
+def sample_fields():
+    """Sample extracted ad fields dict as returned by extract_ad_fields()."""
+    return {
+        "wh_ad_id": "12345678",
+        "title": "Mountain Bike 2024",
+        "price": 250.0,
+        "location": "Vienna",
+        "url": "https://www.willhaben.at/iad/mountain-bike-2024/12345678",
+        "published_at": datetime(2024, 1, 15, 10, 30, 0, tzinfo=timezone.utc),
+        "main_image_url": "https://img.willhaben.at/img123.jpg",
+        "postcode": "1010",
+        "modified_at": datetime(2024, 1, 15, 12, 0, 0, tzinfo=timezone.utc),
+    }
+
+
+@pytest.fixture
+def sample_kw_row():
+    """Sample keyword row dict with filter settings."""
+    return {
+        "id": "kw-uuid-123",
+        "keyword": "bike",
+        "price_min": None,
+        "price_max": None,
+        "allowed_postcodes": None,
+    }

worker/tests/test_filters.py

@@ -0,0 +1,92 @@
+"""Tests for _ad_passes_filters from main.py."""
+
+from main import _ad_passes_filters
+
+
+class TestAdPassesFilters:
+    """Test the _ad_passes_filters function."""
+
+    def test_no_filters_passes(self, sample_fields, sample_kw_row):
+        """Ad with no filters should always pass."""
+        assert _ad_passes_filters(sample_fields, sample_kw_row) is True
+
+    def test_price_below_min_fails(self, sample_fields, sample_kw_row):
+        """Ad price below price_min should fail."""
+        kw = {**sample_kw_row, "price_min": 30000}  # 300.00 EUR in cents
+        # sample_fields price is 250.00 EUR = 25000 cents
+        assert _ad_passes_filters(sample_fields, kw) is False
+
+    def test_price_above_max_fails(self, sample_fields, sample_kw_row):
+        """Ad price above price_max should fail."""
+        kw = {**sample_kw_row, "price_max": 20000}  # 200.00 EUR in cents
+        # sample_fields price is 250.00 EUR = 25000 cents
+        assert _ad_passes_filters(sample_fields, kw) is False
+
+    def test_price_in_range_passes(self, sample_fields, sample_kw_row):
+        """Ad price within min/max range should pass."""
+        kw = {**sample_kw_row, "price_min": 10000, "price_max": 50000}
+        # sample_fields price is 250.00 EUR = 25000 cents
+        assert _ad_passes_filters(sample_fields, kw) is True
+
+    def test_price_at_min_boundary_passes(self, sample_fields, sample_kw_row):
+        """Ad price exactly at price_min should pass."""
+        kw = {**sample_kw_row, "price_min": 25000}
+        assert _ad_passes_filters(sample_fields, kw) is True
+
+    def test_price_at_max_boundary_passes(self, sample_fields, sample_kw_row):
+        """Ad price exactly at price_max should pass."""
+        kw = {**sample_kw_row, "price_max": 25000}
+        assert _ad_passes_filters(sample_fields, kw) is True
+
+    def test_postcode_not_in_allowed_list_fails(self, sample_fields, sample_kw_row):
+        """Ad postcode not in allowed_postcodes should fail."""
+        kw = {**sample_kw_row, "allowed_postcodes": ["1020", "1030"]}
+        # sample_fields postcode is "1010"
+        assert _ad_passes_filters(sample_fields, kw) is False
+
+    def test_postcode_in_allowed_list_passes(self, sample_fields, sample_kw_row):
+        """Ad postcode in allowed_postcodes should pass."""
+        kw = {**sample_kw_row, "allowed_postcodes": ["1010", "1020"]}
+        assert _ad_passes_filters(sample_fields, kw) is True
+
+    def test_no_postcode_with_filter_active_fails(self, sample_fields, sample_kw_row):
+        """Ad with no postcode when filter is active should fail."""
+        fields = {**sample_fields, "postcode": None}
+        kw = {**sample_kw_row, "allowed_postcodes": ["1010", "1020"]}
+        assert _ad_passes_filters(fields, kw) is False
+
+    def test_combined_price_and_postcode_filters_pass(self, sample_fields, sample_kw_row):
+        """Ad passing both price and postcode filters should pass."""
+        kw = {
+            **sample_kw_row,
+            "price_min": 10000,
+            "price_max": 50000,
+            "allowed_postcodes": ["1010", "1020"],
+        }
+        assert _ad_passes_filters(sample_fields, kw) is True
+
+    def test_combined_price_passes_postcode_fails(self, sample_fields, sample_kw_row):
+        """Ad passing price but failing postcode should fail."""
+        kw = {
+            **sample_kw_row,
+            "price_min": 10000,
+            "price_max": 50000,
+            "allowed_postcodes": ["1020", "1030"],
+        }
+        assert _ad_passes_filters(sample_fields, kw) is False
+
+    def test_combined_price_fails_postcode_passes(self, sample_fields, sample_kw_row):
+        """Ad failing price but passing postcode should fail."""
+        kw = {
+            **sample_kw_row,
+            "price_min": 30000,
+            "price_max": 50000,
+            "allowed_postcodes": ["1010", "1020"],
+        }
+        assert _ad_passes_filters(sample_fields, kw) is False
+
+    def test_no_price_with_price_filter(self, sample_fields, sample_kw_row):
+        """Ad with no price should pass price filters (price is None)."""
+        fields = {**sample_fields, "price": None}
+        kw = {**sample_kw_row, "price_min": 10000, "price_max": 50000}
+        assert _ad_passes_filters(fields, kw) is True

worker/tests/test_health.py

@@ -0,0 +1,29 @@
+"""Tests for health module."""
+
+from health import create_health_app
+
+
+class TestHealthModule:
+    """Basic import and app creation tests for the health module."""
+
+    def test_health_module_import(self):
+        """Health module should be importable."""
+        import health
+        assert health is not None
+
+    def test_create_health_app_returns_app(self):
+        """create_health_app should return an aiohttp web.Application."""
+        app = create_health_app()
+        assert app is not None
+        assert hasattr(app, "router")
+
+    def test_health_app_has_routes(self):
+        """Health app should have /health and /stats routes."""
+        app = create_health_app()
+        # Collect route info from the router
+        routes_info = []
+        for route in app.router.routes():
+            routes_info.append(repr(route))
+        routes_str = " ".join(routes_info)
+        assert "/health" in routes_str
+        assert "/stats" in routes_str

worker/tests/test_notifier.py

@@ -0,0 +1,181 @@
+"""Tests for notifier module functions."""
+
+from datetime import datetime, time, timezone
+from unittest.mock import MagicMock, patch
+
+import pytest
+
+from notifier import (
+    _format_text,
+    _build_keyboard,
+    is_user_muted,
+    buffer_for_digest,
+)
+
+
+class TestIsUserMuted:
+    """Test the is_user_muted async function."""
+
+    @pytest.mark.asyncio
+    async def test_no_settings_not_muted(self, mock_pool):
+        """User with no settings should not be muted."""
+        mock_pool.fetchrow.return_value = None
+        result = await is_user_muted(mock_pool, 12345)
+        assert result is False
+
+    @pytest.mark.asyncio
+    async def test_no_mute_hours_not_muted(self, mock_pool):
+        """User with settings but no mute hours should not be muted."""
+        mock_pool.fetchrow.return_value = {"mute_start": None, "mute_end": None}
+        result = await is_user_muted(mock_pool, 12345)
+        assert result is False
+
+    @pytest.mark.asyncio
+    async def test_normal_window_muted(self, mock_pool):
+        """User should be muted when current time is within normal window."""
+        mock_pool.fetchrow.return_value = {
+            "mute_start": time(8, 0),
+            "mute_end": time(12, 0),
+        }
+        fake_dt = datetime(2024, 1, 15, 10, 0, 0, tzinfo=timezone.utc)
+        mock_dt = MagicMock()
+        mock_dt.now.return_value = fake_dt
+        with patch("notifier.datetime", mock_dt):
+            result = await is_user_muted(mock_pool, 12345)
+        assert result is True
+
+    @pytest.mark.asyncio
+    async def test_normal_window_not_muted(self, mock_pool):
+        """User should not be muted when current time is outside normal window."""
+        mock_pool.fetchrow.return_value = {
+            "mute_start": time(8, 0),
+            "mute_end": time(12, 0),
+        }
+        fake_dt = datetime(2024, 1, 15, 14, 0, 0, tzinfo=timezone.utc)
+        mock_dt = MagicMock()
+        mock_dt.now.return_value = fake_dt
+        with patch("notifier.datetime", mock_dt):
+            result = await is_user_muted(mock_pool, 12345)
+        assert result is False
+
+    @pytest.mark.asyncio
+    async def test_cross_midnight_window_muted_after_start(self, mock_pool):
+        """User should be muted when time is after start of cross-midnight window."""
+        mock_pool.fetchrow.return_value = {
+            "mute_start": time(22, 0),
+            "mute_end": time(6, 0),
+        }
+        fake_dt = datetime(2024, 1, 15, 23, 0, 0, tzinfo=timezone.utc)
+        mock_dt = MagicMock()
+        mock_dt.now.return_value = fake_dt
+        with patch("notifier.datetime", mock_dt):
+            result = await is_user_muted(mock_pool, 12345)
+        assert result is True
+
+    @pytest.mark.asyncio
+    async def test_cross_midnight_window_muted_before_end(self, mock_pool):
+        """User should be muted when time is before end of cross-midnight window."""
+        mock_pool.fetchrow.return_value = {
+            "mute_start": time(22, 0),
+            "mute_end": time(6, 0),
+        }
+        fake_dt = datetime(2024, 1, 15, 3, 0, 0, tzinfo=timezone.utc)
+        mock_dt = MagicMock()
+        mock_dt.now.return_value = fake_dt
+        with patch("notifier.datetime", mock_dt):
+            result = await is_user_muted(mock_pool, 12345)
+        assert result is True
+
+    @pytest.mark.asyncio
+    async def test_cross_midnight_window_not_muted(self, mock_pool):
+        """User should not be muted when time is outside cross-midnight window."""
+        mock_pool.fetchrow.return_value = {
+            "mute_start": time(22, 0),
+            "mute_end": time(6, 0),
+        }
+        fake_dt = datetime(2024, 1, 15, 12, 0, 0, tzinfo=timezone.utc)
+        mock_dt = MagicMock()
+        mock_dt.now.return_value = fake_dt
+        with patch("notifier.datetime", mock_dt):
+            result = await is_user_muted(mock_pool, 12345)
+        assert result is False
+
+
+class TestBufferForDigest:
+    """Test the buffer_for_digest async function."""
+
+    @pytest.mark.asyncio
+    async def test_digest_mode_on_buffers(self, mock_pool):
+        """Should buffer notification when digest mode is on."""
+        mock_pool.fetchrow.return_value = {"digest_mode": True}
+        ad = {"title": "Test Ad", "price": 100.0, "keyword": "bike", "url": "https://example.com"}
+        await buffer_for_digest(mock_pool, 12345, ad, "ad-uuid-1")
+        mock_pool.execute.assert_called_once()
+
+    @pytest.mark.asyncio
+    async def test_digest_mode_off_does_not_buffer(self, mock_pool):
+        """Should not buffer notification when digest mode is off."""
+        mock_pool.fetchrow.return_value = {"digest_mode": False}
+        ad = {"title": "Test Ad", "price": 100.0, "keyword": "bike", "url": "https://example.com"}
+        await buffer_for_digest(mock_pool, 12345, ad, "ad-uuid-1")
+        mock_pool.execute.assert_not_called()
+
+    @pytest.mark.asyncio
+    async def test_no_settings_does_not_buffer(self, mock_pool):
+        """Should not buffer when user has no settings."""
+        mock_pool.fetchrow.return_value = None
+        ad = {"title": "Test Ad", "price": 100.0, "keyword": "bike", "url": "https://example.com"}
+        await buffer_for_digest(mock_pool, 12345, ad, "ad-uuid-1")
+        mock_pool.execute.assert_not_called()
+
+
+class TestFormatText:
+    """Test the _format_text function."""
+
+    def test_basic_format(self, sample_fields):
+        """Should produce expected output with basic fields."""
+        text = _format_text("🆕 New listing found!", sample_fields)
+        assert "🆕 New listing found!" in text
+        assert "Mountain Bike 2024" in text
+        assert "250" in text
+        assert "Vienna" in text
+        assert "1010" in text
+
+    def test_format_with_no_price(self):
+        """Should handle missing price gracefully."""
+        ad = {"title": "Free Item", "location": "Graz"}
+        text = _format_text("Header", ad)
+        assert "N/A" in text
+
+    def test_format_with_no_location(self):
+        """Should handle missing location gracefully."""
+        ad = {"title": "Item", "price": 50.0}
+        text = _format_text("Header", ad)
+        assert "Item" in text
+        assert "50" in text
+
+    def test_format_with_postcode(self, sample_fields):
+        """Should include postcode when present."""
+        text = _format_text("Header", sample_fields)
+        assert "1010" in text
+
+    def test_format_with_published_at(self, sample_fields):
+        """Should include published date when present."""
+        text = _format_text("Header", sample_fields)
+        assert "15.01.2024" in text
+
+
+class TestBuildKeyboard:
+    """Test the _build_keyboard function."""
+
+    def test_with_url(self, sample_fields):
+        """Should create keyboard with URL button when URL is present."""
+        keyboard = _build_keyboard(sample_fields)
+        assert keyboard is not None
+        assert len(keyboard.inline_keyboard) == 1
+        assert "View Ad" in keyboard.inline_keyboard[0][0].text
+
+    def test_without_url(self):
+        """Should create keyboard with no buttons when URL is missing."""
+        keyboard = _build_keyboard({"title": "No URL Ad"})
+        assert keyboard is None

worker/tests/test_proxy_settings.py

@@ -0,0 +1,81 @@
+"""Tests for the app-level proxy setting."""
+
+import pytest
+
+from settings import get_proxy_enabled, set_proxy_enabled
+
+
+def _clear_proxy_env(monkeypatch):
+    monkeypatch.delenv("HTTPS_PROXY", raising=False)
+    monkeypatch.delenv("https_proxy", raising=False)
+
+
+@pytest.mark.asyncio
+async def test_get_proxy_enabled_defaults_true_when_https_proxy_is_usable(
+    mock_pool, monkeypatch
+):
+    """First-run default should enable only with a usable proxy."""
+    _clear_proxy_env(monkeypatch)
+    monkeypatch.setenv("HTTPS_PROXY", "proxy.example:8080:user:pass")
+    mock_pool.fetchval.return_value = None
+
+    assert await get_proxy_enabled(mock_pool) is True
+
+    mock_pool.fetchval.assert_awaited_once()
+    assert "proxy_enabled" in mock_pool.fetchval.await_args.args
+
+
+@pytest.mark.asyncio
+@pytest.mark.parametrize("proxy_value", [None, "", "proxy.example:8080:user"])
+async def test_get_proxy_enabled_defaults_false_without_usable_https_proxy(
+    mock_pool, monkeypatch, proxy_value
+):
+    """Missing, blank, and malformed proxy env values default disabled."""
+    _clear_proxy_env(monkeypatch)
+    if proxy_value is not None:
+        monkeypatch.setenv("HTTPS_PROXY", proxy_value)
+    mock_pool.fetchval.return_value = None
+
+    assert await get_proxy_enabled(mock_pool) is False
+
+
+@pytest.mark.asyncio
+@pytest.mark.parametrize(
+    ("stored_value", "expected"),
+    [
+        ("true", True),
+        ("1", True),
+        ("yes", True),
+        ("on", True),
+        ("false", False),
+        ("0", False),
+        ("no", False),
+        ("off", False),
+    ],
+)
+async def test_get_proxy_enabled_uses_persisted_setting(
+    mock_pool, monkeypatch, stored_value, expected
+):
+    """Once saved, the DB setting should control the UI/runtime toggle."""
+    _clear_proxy_env(monkeypatch)
+    monkeypatch.setenv("HTTPS_PROXY", "proxy.example:8080:user:pass")
+    mock_pool.fetchval.return_value = stored_value
+
+    assert await get_proxy_enabled(mock_pool) is expected
+
+
+@pytest.mark.asyncio
+@pytest.mark.parametrize(
+    ("enabled", "stored_value"),
+    [(True, "true"), (False, "false")],
+)
+async def test_set_proxy_enabled_persists_proxy_toggle(
+    mock_pool, enabled, stored_value
+):
+    """The proxy toggle should be stored under the app setting key."""
+    await set_proxy_enabled(mock_pool, enabled)
+
+    mock_pool.execute.assert_awaited()
+    sql, *args = mock_pool.execute.await_args.args
+    assert "proxy_enabled" in sql or "proxy_enabled" in args
+    assert stored_value in args

worker/tests/test_scraper.py

@@ -0,0 +1,149 @@
+"""Tests for scraper module functions."""
+
+from datetime import datetime, timezone
+
+from scraper import extract_ad_fields
+
+
+class TestExtractAdFields:
+    """Test the extract_ad_fields function."""
+
+    def test_full_extraction(self, sample_ad_data):
+        """Should extract all fields from a complete ad."""
+        fields = extract_ad_fields(sample_ad_data)
+
+        assert fields["wh_ad_id"] == "12345678"
+        assert fields["title"] == "Mountain Bike 2024"
+        assert fields["price"] == 250.0
+        assert fields["location"] == "Vienna"
+        assert fields["url"] == "https://www.willhaben.at/iad/mountain-bike-2024/12345678"
+        assert fields["postcode"] == "1010"
+        assert fields["main_image_url"] == "https://img.willhaben.at/img123.jpg"
+        assert isinstance(fields["published_at"], datetime)
+        assert isinstance(fields["modified_at"], datetime)
+
+    def test_published_at_is_utc(self, sample_ad_data):
+        """Published_at should be parsed as UTC."""
+        fields = extract_ad_fields(sample_ad_data)
+        assert fields["published_at"].tzinfo == timezone.utc
+        assert fields["published_at"].hour == 10
+        assert fields["published_at"].minute == 30
+
+    def test_modified_at_is_utc(self, sample_ad_data):
+        """Modified_at should be parsed as UTC."""
+        fields = extract_ad_fields(sample_ad_data)
+        assert fields["modified_at"].tzinfo == timezone.utc
+        assert fields["modified_at"].hour == 12
+
+    def test_missing_price(self):
+        """Should handle ads without a price attribute."""
+        ad_data = {
+            "id": "999",
+            "description": "Free item",
+            "attributes": {
+                "attribute": [
+                    {"name": "HEADING", "values": ["Free Item"]},
+                    {"name": "LOCATION", "values": ["Graz"]},
+                ]
+            },
+        }
+        fields = extract_ad_fields(ad_data)
+        assert fields["price"] is None
+        assert fields["title"] == "Free Item"
+
+    def test_missing_heading_falls_back_to_description(self):
+        """Should fall back to description when HEADING is missing."""
+        ad_data = {
+            "id": "888",
+            "description": "Fallback description",
+            "attributes": {"attribute": []},
+        }
+        fields = extract_ad_fields(ad_data)
+        assert fields["title"] == "Fallback description"
+
+    def test_missing_attributes(self):
+        """Should handle ads with no attributes at all."""
+        ad_data = {"id": "777", "description": "Minimal ad"}
+        fields = extract_ad_fields(ad_data)
+        assert fields["wh_ad_id"] == "777"
+        assert fields["title"] == "Minimal ad"
+        assert fields["price"] is None
+        assert fields["location"] is None
+
+    def test_price_with_comma_separator(self):
+        """Should parse prices with comma (comma is stripped, so '1.299,50' → 1.2995)."""
+        ad_data = {
+            "id": "666",
+            "attributes": {
+                "attribute": [
+                    {"name": "PRICE/AMOUNT", "values": ["1.299,50"]},
+                ]
+            },
+        }
+        fields = extract_ad_fields(ad_data)
+        # The parser strips commas only: "1.299,50" → "1.2995" → 1.2995
+        assert fields["price"] == 1.2995
+
+    def test_missing_image(self):
+        """Should handle ads without images."""
+        ad_data = {
+            "id": "555",
+            "attributes": {
+                "attribute": [
+                    {"name": "HEADING", "values": ["No Image"]},
+                ]
+            },
+        }
+        fields = extract_ad_fields(ad_data)
+        assert fields["main_image_url"] is None
+
+    def test_empty_image_list(self):
+        """Should handle ads with empty image list."""
+        ad_data = {
+            "id": "444",
+            "attributes": {"attribute": []},
+            "advertImageList": {"advertImage": []},
+        }
+        fields = extract_ad_fields(ad_data)
+        assert fields["main_image_url"] is None
+
+    def test_missing_seo_url(self):
+        """Should handle ads without SEO_URL."""
+        ad_data = {
+            "id": "333",
+            "attributes": {
+                "attribute": [
+                    {"name": "HEADING", "values": ["No SEO"]},
+                ]
+            },
+        }
+        fields = extract_ad_fields(ad_data)
+        assert fields["url"] is None
+
+    def test_invalid_price_format(self):
+        """Should handle invalid price values gracefully."""
+        ad_data = {
+            "id": "222",
+            "attributes": {
+                "attribute": [
+                    {"name": "PRICE/AMOUNT", "values": ["not a number"]},
+                ]
+            },
+        }
+        fields = extract_ad_fields(ad_data)
+        assert fields["price"] is None
+
+    def test_invalid_date_format(self):
+        """Should handle invalid date values gracefully."""
+        ad_data = {
+            "id": "111",
+            "attributes": {
+                "attribute": [
+                    {"name": "PUBLISHED_String", "values": ["not-a-date"]},
+                    {"name": "CHANGED_String", "values": ["also-not-a-date"]},
+                ]
+            },
+        }
+        fields = extract_ad_fields(ad_data)
+        assert fields["published_at"] is None
+        assert fields["modified_at"] is None

worker/tests/test_scraper_proxy.py

@@ -0,0 +1,129 @@
+"""Proxy parsing and runtime-status tests for scraper."""
+
+import inspect
+
+import pytest
+
+import scraper
+
+
+def _clear_proxy_env(monkeypatch):
+    monkeypatch.delenv("HTTPS_PROXY", raising=False)
+    monkeypatch.delenv("https_proxy", raising=False)
+
+
+def _proxy_url_parser():
+    parser = getattr(scraper, "_get_proxy_url", None)
+    if parser is None:
+        pytest.skip("scraper does not expose a proxy URL parser")
+    return parser
+
+
+async def _get_network_status(mock_pool, monkeypatch):
+    monkeypatch.setattr(
+        scraper,
+        "_proxy_enabled_effective",
+        None,
+        raising=False,
+    )
+    status_fn = getattr(scraper, "get_network_status")
+    signature = inspect.signature(status_fn)
+    if "pool" in signature.parameters:
+        result = status_fn(mock_pool)
+        if inspect.isawaitable(result):
+            return await result
+        return result
+
+    effective_proxy_fn = getattr(scraper, "_get_effective_proxy_url", None)
+    if effective_proxy_fn is not None:
+        async def fake_get_pool():
+            return mock_pool
+
+        monkeypatch.setattr(scraper, "get_pool", fake_get_pool, raising=False)
+        result = effective_proxy_fn()
+        if inspect.isawaitable(result):
+            await result
+
+    result = status_fn()
+    if inspect.isawaitable(result):
+        return await result
+    return result
+
+
+def _block_public_ip_fetch(monkeypatch):
+    async def fail_fetch(*_args, **_kwargs):
+        raise AssertionError("network status tests must not fetch public IPs")
+
+    monkeypatch.setattr(scraper, "_fetch_public_ip", fail_fetch, raising=False)
+
+
+def test_proxy_url_parser_accepts_full_proxy_url(monkeypatch):
+    _clear_proxy_env(monkeypatch)
+    monkeypatch.setenv("HTTPS_PROXY", "http://user:pass@proxy.example:8080")
+
+    assert _proxy_url_parser()() == "http://user:pass@proxy.example:8080"
+
+
+def test_proxy_url_parser_builds_url_from_colon_format(monkeypatch):
+    _clear_proxy_env(monkeypatch)
+    monkeypatch.setenv("HTTPS_PROXY", "proxy.example:8080:user@example:p a/s")
+
+    assert (
+        _proxy_url_parser()()
+        == "http://user%40example:p%20a%2Fs@proxy.example:8080"
+    )
+
+
+@pytest.mark.parametrize("proxy_value", ["", "proxy.example:8080:user"])
+def test_proxy_url_parser_rejects_missing_or_malformed_proxy(
+    monkeypatch, proxy_value
+):
+    _clear_proxy_env(monkeypatch)
+    monkeypatch.setenv("HTTPS_PROXY", proxy_value)
+
+    assert _proxy_url_parser()() is None
+
+
+@pytest.mark.asyncio
+async def test_network_status_reports_proxy_available_when_env_is_usable(
+    mock_pool, monkeypatch
+):
+    _clear_proxy_env(monkeypatch)
+    _block_public_ip_fetch(monkeypatch)
+    monkeypatch.setenv("HTTPS_PROXY", "proxy.example:8080:user:pass")
+    mock_pool.fetchval.return_value = None
+
+    status = await _get_network_status(mock_pool, monkeypatch)
+
+    assert status["proxy_available"] is True
+    assert status["proxy_enabled"] is True
+
+
+@pytest.mark.asyncio
+async def test_network_status_respects_disabled_proxy_setting(
+    mock_pool, monkeypatch
+):
+    _clear_proxy_env(monkeypatch)
+    _block_public_ip_fetch(monkeypatch)
+    monkeypatch.setenv("HTTPS_PROXY", "proxy.example:8080:user:pass")
+    mock_pool.fetchval.return_value = "false"
+
+    status = await _get_network_status(mock_pool, monkeypatch)
+
+    assert status["proxy_available"] is True
+    assert status["proxy_enabled"] is False
+
+
+@pytest.mark.asyncio
+async def test_network_status_reports_unavailable_for_malformed_proxy(
+    mock_pool, monkeypatch
+):
+    _clear_proxy_env(monkeypatch)
+    _block_public_ip_fetch(monkeypatch)
+    monkeypatch.setenv("HTTPS_PROXY", "proxy.example:8080:user")
+    mock_pool.fetchval.return_value = None
+
+    status = await _get_network_status(mock_pool, monkeypatch)
+
+    assert status["proxy_available"] is False
+    assert status["proxy_enabled"] is False