hermes
f540cbe7ef
- Remove SET search_path from db.py and migration SQL (Supabase uses public schema) - Add migrate.py with tracking table for forward-only SQL migrations - Add entrypoint.sh: waits for DB, runs migrations, then starts app - Copy 01-schema.sql + zz-seed.sql to worker/src/migrations/ - Add health.py: /health endpoint (200/503) with DB connectivity + scheduler staleness checks - /stats endpoint with keyword/ad/notification counts - Rewrite main.py shutdown sequence: signal handler, 5s grace for scheduler, ordered cleanup - Update Dockerfile: HEALTHCHECK directive, entrypoint, COPY migrations - Update docker-compose.yml: stop_grace_period=15s, healthcheck config, env vars - Add aiohttp>=3.9 to requirements.txt for health server
4.2 KiB
4.2 KiB
Phase 0 — Critical Stability Fixes
Scope
This phase addresses blocking and high-risk issues in the current feat/supabase-migration branch that must be resolved before any feature work. The goal is a stable, self-healing deployment where the worker container:
- Starts correctly against an empty or partially-migrated database
- Recovers from crashes via Docker healthcheck
- Shuts down gracefully without data corruption
- Has consistent schema configuration across all DB access paths
Architecture
┌─────────────────────────────────────────────────────┐
│ worker container (python:3.12-slim) │
│ │
│ ┌───────────┐ ┌───────────┐ │
│ │ entrypoint│──►│ migrate.py│──► apply pending │
│ │ .sh │ │ │ SQL migrations │
│ └─────┬─────┘ └───────────┘ │
│ │ │
│ ▼ │
│ ┌───────────┐ │
│ │ main.py │ (fixed shutdown order) │
│ │ │ │
│ ├───────────┤ │
│ │ scheduler │◄► db.py │
│ │ bot loop │ (consistent search_path) │
│ └───────────┘ │
│ │ │
│ ▼ │
│ ┌──────────────┐ │
│ │ /health endpoint│ │
│ │ HTTP server │ (exposed for docker healthcheck)│
│ └──────────────┘ │
└─────────────────────────────────────────────────────┘
docker-compose.yml:
healthcheck: curl -f http://localhost:8765/health || exit 1
interval=30s timeout=5s retries=3 start_period=10s
Tasks
| Task | File | Description |
|---|---|---|
| Auto-migration on startup | task-auto-migration.md | Entrypoint script + migration runner that applies pending SQL against the database before starting the worker process. Idempotent, schema-version tracked. |
| Graceful shutdown fix | task-graceful-shutdown.md | Reorder cleanup in main.py to stop polling → cancel scheduler → close DB pool → call Application.shutdown(), resolving the "Application is still running!" RuntimeError. |
| Healthcheck endpoint + docker-compose config | task-healthcheck.md | Expose a lightweight HTTP /health endpoint on port 8765 inside the worker container; add healthcheck directive to docker-compose.yml so Docker restarts unhealthy containers. |
| search_path consistency | task-search-path-fix.md | Resolve mismatch between _init_connection() setting search_path TO willhaben_tracker and tables living in public. Pick one path (recommend: remove custom schema, keep everything in public for Supabase compatibility). |
General Acceptance Criteria
- Container starts from scratch against an empty database → all tables created automatically, no manual intervention
- Sending SIGTERM to the worker results in clean shutdown within 10 seconds with no errors in logs
- Docker reports container as
healthywithin 60 seconds of start - All DB queries work regardless of explicit schema qualification (no "relation does not exist" errors)
- Rolling back and re-applying migrations is idempotent (safe to run multiple times)