feat(phase-0): stability hardening — search path fix, auto-migrations, graceful shutdown, health endpoint

- Remove SET search_path from db.py and migration SQL (Supabase uses public schema) - Add migrate.py with tracking table for forward-only SQL migrations - Add entrypoint.sh: waits for DB, runs migrations, then starts app - Copy 01-schema.sql + zz-seed.sql to worker/src/migrations/ - Add health.py: /health endpoint (200/503) with DB connectivity + scheduler staleness checks - /stats endpoint with keyword/ad/notification counts - Rewrite main.py shutdown sequence: signal handler, 5s grace for scheduler, ordered cleanup - Update Dockerfile: HEALTHCHECK directive, entrypoint, COPY migrations - Update docker-compose.yml: stop_grace_period=15s, healthcheck config, env vars - Add aiohttp>=3.9 to requirements.txt for health server
2026-07-04 11:30:45 -04:00
parent 64506f20b3
commit f540cbe7ef
16 changed files with 1133 additions and 9 deletions
@@ -0,0 +1,59 @@
+# Phase 0 — Critical Stability Fixes
+
+## Scope
+
+This phase addresses **blocking and high-risk issues** in the current `feat/supabase-migration` branch that must be resolved before any feature work. The goal is a stable, self-healing deployment where the worker container:
+
+- Starts correctly against an empty or partially-migrated database
+- Recovers from crashes via Docker healthcheck
+- Shuts down gracefully without data corruption
+- Has consistent schema configuration across all DB access paths
+
+## Architecture
+
+```
+┌─────────────────────────────────────────────────────┐
+│  worker container (python:3.12-slim)                │
+│                                                     │
+│  ┌───────────┐   ┌───────────┐                     │
+│  │ entrypoint│──►│ migrate.py│──► apply pending    │
+│  │   .sh     │   │           │    SQL migrations   │
+│  └─────┬─────┘   └───────────┘                     │
+│        │                                            │
+│        ▼                                            │
+│  ┌───────────┐                                      │
+│  │ main.py   │  (fixed shutdown order)              │
+│  │           │                                      │
+│  ├───────────┤                                      │
+│  │ scheduler │◄► db.py                              │
+│  │ bot loop  │    (consistent search_path)          │
+│  └───────────┘                                      │
+│        │                                            │
+│        ▼                                            │
+│  ┌──────────────┐                                   │
+│  │ /health endpoint│                                 │
+│  │  HTTP server   │  (exposed for docker healthcheck)│
+│  └──────────────┘                                   │
+└─────────────────────────────────────────────────────┘
+
+docker-compose.yml:
+  healthcheck: curl -f http://localhost:8765/health || exit 1
+               interval=30s timeout=5s retries=3 start_period=10s
+```
+
+## Tasks
+
+| Task | File | Description |
+|------|------|-------------|
+| Auto-migration on startup | [task-auto-migration.md](./task-auto-migration.md) | Entrypoint script + migration runner that applies pending SQL against the database before starting the worker process. Idempotent, schema-version tracked. |
+| Graceful shutdown fix | [task-graceful-shutdown.md](./task-graceful-shutdown.md) | Reorder cleanup in `main.py` to stop polling → cancel scheduler → close DB pool → call Application.shutdown(), resolving the "Application is still running!" RuntimeError. |
+| Healthcheck endpoint + docker-compose config | [task-healthcheck.md](./task-healthcheck.md) | Expose a lightweight HTTP `/health` endpoint on port 8765 inside the worker container; add `healthcheck` directive to docker-compose.yml so Docker restarts unhealthy containers. |
+| search_path consistency | [task-search-path-fix.md](./task-search-path-fix.md) | Resolve mismatch between `_init_connection()` setting `search_path TO willhaben_tracker` and tables living in `public`. Pick one path (recommend: remove custom schema, keep everything in public for Supabase compatibility). |
+
+## General Acceptance Criteria
+
+- [ ] Container starts from scratch against an empty database → all tables created automatically, no manual intervention
+- [ ] Sending SIGTERM to the worker results in clean shutdown within 10 seconds with no errors in logs
+- [ ] Docker reports container as `healthy` within 60 seconds of start
+- [ ] All DB queries work regardless of explicit schema qualification (no "relation does not exist" errors)
+- [ ] Rolling back and re-applying migrations is idempotent (safe to run multiple times)