feat(phase-0): stability hardening — search path fix, auto-migrations, graceful shutdown, health endpoint
- Remove SET search_path from db.py and migration SQL (Supabase uses public schema) - Add migrate.py with tracking table for forward-only SQL migrations - Add entrypoint.sh: waits for DB, runs migrations, then starts app - Copy 01-schema.sql + zz-seed.sql to worker/src/migrations/ - Add health.py: /health endpoint (200/503) with DB connectivity + scheduler staleness checks - /stats endpoint with keyword/ad/notification counts - Rewrite main.py shutdown sequence: signal handler, 5s grace for scheduler, ordered cleanup - Update Dockerfile: HEALTHCHECK directive, entrypoint, COPY migrations - Update docker-compose.yml: stop_grace_period=15s, healthcheck config, env vars - Add aiohttp>=3.9 to requirements.txt for health server
This commit is contained in:
@@ -0,0 +1,101 @@
|
||||
# Task: search_path consistency fix
|
||||
|
||||
## Description
|
||||
|
||||
There is a mismatch between the database schema configuration and where tables actually live:
|
||||
|
||||
**In `db.py` (`_init_connection`):**
|
||||
```python
|
||||
await conn.execute("SET search_path TO willhaben_tracker")
|
||||
```
|
||||
|
||||
**But in `supabase-migration.sql`:**
|
||||
- The migration file has `SET search_path TO willhaben_tracker;` but the actual tables were created in `public` schema (not `willhaben_tracker`) when manually applied.
|
||||
- The original local Supabase migrations used `public` schema.
|
||||
|
||||
This creates two problems:
|
||||
1. If a connection does NOT run `_init_connection`, queries to bare table names (`SELECT FROM keywords`) fail with "relation does not exist".
|
||||
2. Tools that connect directly (pgAdmin, DBeaver, PostgREST) won't find the tables without explicit schema qualification or their own search_path config.
|
||||
|
||||
## Decision: Remove custom schema, use `public` for everything
|
||||
|
||||
Since Supabase uses `public` by default and all existing tools/GRANTs in `post-boot.sql` target `public`, we simplify by removing the `willhaben_tracker` schema entirely.
|
||||
|
||||
### Rationale
|
||||
|
||||
| Factor | Custom Schema (`willhaben_tracker`) | Public Schema |
|
||||
|--------|--------------------------------------|---------------|
|
||||
| Supabase Studio compatibility | Requires manual config | ✅ Works out of box |
|
||||
| PostgREST/Kong access | Needs search_path override | ✅ Default |
|
||||
| Direct pgAdmin/DBeaver access | Must set schema per-connection | ✅ Always visible |
|
||||
| Grant statements in post-boot.sql | ❌ Targets wrong schema | ✅ Already correct |
|
||||
| Code changes needed | Minimal (remove one line) | Minimal |
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
BEFORE (broken):
|
||||
db.py init: SET search_path TO willhaben_tracker
|
||||
tables live in: public.keywords, public.ads, ...
|
||||
Result: queries fail because search_path != table location
|
||||
|
||||
AFTER (fixed):
|
||||
db.py init: REMOVED (no custom search_path)
|
||||
tables live in: public.keywords, public.ads, ...
|
||||
Result: all connections find tables via default 'public' schema
|
||||
```
|
||||
|
||||
## Implementation Details
|
||||
|
||||
### 1. Remove `SET search_path` from `db.py`
|
||||
|
||||
**Current:**
|
||||
```python
|
||||
async def _init_connection(conn: asyncpg.Connection) -> None:
|
||||
await conn.execute("SET search_path TO willhaben_tracker")
|
||||
```
|
||||
|
||||
**After:**
|
||||
```python
|
||||
# Removed entirely - using default public schema
|
||||
```
|
||||
|
||||
The `_init_connection` function can be removed, and `init=_init_connection` removed from `create_pool()`. Alternatively keep the callback but make it a no-op for future use:
|
||||
|
||||
```python
|
||||
async def _init_connection(conn: asyncpg.Connection) -> None:
|
||||
# Reserved for future connection-level settings (e.g., timezone)
|
||||
pass
|
||||
```
|
||||
|
||||
### 2. Clean up migration files
|
||||
|
||||
- Remove `SET search_path TO willhaben_tracker;` from `supabase-migration.sql` / `01-schema.sql`
|
||||
- Ensure all tables are explicitly created in the default schema (no need for prefix — that's already what happens)
|
||||
|
||||
### 3. Update `post-boot.sql` grants if needed
|
||||
|
||||
Current grants already target `public`:
|
||||
```sql
|
||||
GRANT USAGE ON SCHEMA public TO supabase_admin;
|
||||
```
|
||||
→ ✅ Already correct, no change needed.
|
||||
|
||||
### 4. Add schema qualification to migration file (defensive)
|
||||
|
||||
To be explicit and future-proof, the migration can qualify the schema:
|
||||
|
||||
```sql
|
||||
-- Instead of implicit public.keywords
|
||||
CREATE TABLE IF NOT EXISTS public.keywords (...)
|
||||
```
|
||||
|
||||
This is optional but makes it unambiguous where tables live. For simplicity we'll keep implicit `public` since that's the default everywhere.
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- [ ] Removing `_init_connection` from db.py does not break any query (all table names resolve correctly)
|
||||
- [ ] Connecting to the database via pgAdmin/DBeaver shows all tables without schema configuration
|
||||
- [ ] The healthcheck endpoint DB check works without custom search_path
|
||||
- [ ] No "relation does not exist" errors in worker logs after redeploy
|
||||
- [ ] Migration SQL file no longer references `willhaben_tracker` schema
|
||||
Reference in New Issue
Block a user