# Task: Graceful shutdown fix

## Description

The current `main.py` has a broken shutdown sequence. When the application receives a termination signal, calling `app.shutdown()` while the updater/scheduler are still active raises:

```
RuntimeError: This Application is still running!
```

This means:
- DB connections may not be properly returned to the pool
- In-flight Telegram messages may be lost
- The container does not exit cleanly on SIGTERM, causing Kubernetes/Docker health issues during restarts

This task reorders the shutdown sequence and adds proper signal handling so the worker exits gracefully within 10 seconds.

## Architecture

```
Current (broken):
  SIGTERM → cleanup() 
           ├─ app.shutdown()       ← FAILS: RuntimeError "still running"
           └─ close_pool()         ← never reached or runs in bad state

Target (fixed):
  SIGTERM → cleanup()
           ├─ scheduler_task.cancel()   ← stop scraping loop
           ├─ await asyncio.gather(scheduler_task)  ← let it finish current iteration
           ├─ app.updater.stop_polling()  ← stop Telegram long-poll
           ├─ app.shutdown()             ← safe now, nothing running
           └─ close_pool()               ← return all connections to pool
```

### Key design decisions

- **Drain period**: After signal received, allow up to 5 seconds for the current scrape cycle and pending notifications to complete.
- **Force kill fallback**: If cleanup does not finish in 10 seconds total, force-exit via `os._exit(0)`.
- **asyncpg pool close** uses timeout (default 30s) — acceptable since we already cancelled tasks.

## Implementation Details

### Current `main.py` shutdown code (problematic):

```python
# Simplified current pattern
try:
    # ... app.run() or equivalent
except KeyboardInterrupt:
    app.shutdown()   # ← RuntimeError here
    loop.run_until_complete(close_pool())
```

### Target implementation:

Replace the signal handling block in `main.py` with:

```python
import signal
import sys

_scheduler_task: asyncio.Task | None = None

async def cleanup(app: Application) -> None:
    """Gracefully shut down all components."""
    logger.info("Shutting down...")

    # 1. Cancel scheduler task (with grace period)
    if _scheduler_task and not _scheduler_task.done():
        _scheduler_task.cancel()
        try:
            await asyncio.wait_for(_scheduler_task, timeout=5.0)
        except (asyncio.CancelledError, asyncio.TimeoutError):
            pass

    # 2. Stop polling
    if app.updater and app.updater.running:
        app.updater.stop_polling()
    
    # 3. Now safe to shutdown the application
    await app.shutdown()

    # 4. Close DB pool
    await close_pool()
    
    logger.info("Shutdown complete.")


async def main():
    global _scheduler_task

    app = Application.builder().token(TOKEN).build()
    register_handlers(app)
    setup_global_commands(app)

    # Start polling
    app.updater.start_polling(drop_pending_updates=True)

    # Start scheduler
    _scheduler_task = asyncio.create_task(run_scheduler())

    # Set up signal handlers for graceful shutdown
    loop = asyncio.get_running_loop()
    for sig in (signal.SIGTERM, signal.SIGINT):
        loop.add_signal_handler(
            sig, lambda: asyncio.create_task(cleanup(app))
        )

    # Wait until the app is done or cleanup finishes
    try:
        await _scheduler_task  # runs forever until cancelled
    except asyncio.CancelledError:
        pass

    # Final cleanup (in case signal handler already ran)
    await cleanup(app)
```

### What changes in `main.py`:

1. **Remove** the broken try/except KeyboardInterrupt pattern
2. **Add** the `cleanup()` async function with proper ordering
3. **Add** signal handlers for SIGTERM and SIGINT via `loop.add_signal_handler`
4. **Track** the scheduler task as a global so cleanup can reference it
5. **Remove** any direct calls to `app.shutdown()` elsewhere

## Acceptance Criteria

- [ ] Sending `docker stop willhaben-tracker-worker-1` results in clean exit (exit code 0) within 10 seconds
- [ ] No `RuntimeError: This Application is still running!` appears in logs during shutdown
- [ ] The asyncpg pool closes cleanly — no "connection closed unexpectedly" warnings after shutdown
- [ ] In-flight notifications are allowed to complete (not dropped mid-send)
- [ ] Restarting the container via `docker restart` works without errors
- [ ] Logs show clear shutdown sequence: "Shutting down..." → "Shutdown complete."