hermes
f540cbe7ef
- Remove SET search_path from db.py and migration SQL (Supabase uses public schema) - Add migrate.py with tracking table for forward-only SQL migrations - Add entrypoint.sh: waits for DB, runs migrations, then starts app - Copy 01-schema.sql + zz-seed.sql to worker/src/migrations/ - Add health.py: /health endpoint (200/503) with DB connectivity + scheduler staleness checks - /stats endpoint with keyword/ad/notification counts - Rewrite main.py shutdown sequence: signal handler, 5s grace for scheduler, ordered cleanup - Update Dockerfile: HEALTHCHECK directive, entrypoint, COPY migrations - Update docker-compose.yml: stop_grace_period=15s, healthcheck config, env vars - Add aiohttp>=3.9 to requirements.txt for health server
102 lines
3.8 KiB
Markdown
102 lines
3.8 KiB
Markdown
# Task: search_path consistency fix
|
|
|
|
## Description
|
|
|
|
There is a mismatch between the database schema configuration and where tables actually live:
|
|
|
|
**In `db.py` (`_init_connection`):**
|
|
```python
|
|
await conn.execute("SET search_path TO willhaben_tracker")
|
|
```
|
|
|
|
**But in `supabase-migration.sql`:**
|
|
- The migration file has `SET search_path TO willhaben_tracker;` but the actual tables were created in `public` schema (not `willhaben_tracker`) when manually applied.
|
|
- The original local Supabase migrations used `public` schema.
|
|
|
|
This creates two problems:
|
|
1. If a connection does NOT run `_init_connection`, queries to bare table names (`SELECT FROM keywords`) fail with "relation does not exist".
|
|
2. Tools that connect directly (pgAdmin, DBeaver, PostgREST) won't find the tables without explicit schema qualification or their own search_path config.
|
|
|
|
## Decision: Remove custom schema, use `public` for everything
|
|
|
|
Since Supabase uses `public` by default and all existing tools/GRANTs in `post-boot.sql` target `public`, we simplify by removing the `willhaben_tracker` schema entirely.
|
|
|
|
### Rationale
|
|
|
|
| Factor | Custom Schema (`willhaben_tracker`) | Public Schema |
|
|
|--------|--------------------------------------|---------------|
|
|
| Supabase Studio compatibility | Requires manual config | ✅ Works out of box |
|
|
| PostgREST/Kong access | Needs search_path override | ✅ Default |
|
|
| Direct pgAdmin/DBeaver access | Must set schema per-connection | ✅ Always visible |
|
|
| Grant statements in post-boot.sql | ❌ Targets wrong schema | ✅ Already correct |
|
|
| Code changes needed | Minimal (remove one line) | Minimal |
|
|
|
|
## Architecture
|
|
|
|
```
|
|
BEFORE (broken):
|
|
db.py init: SET search_path TO willhaben_tracker
|
|
tables live in: public.keywords, public.ads, ...
|
|
Result: queries fail because search_path != table location
|
|
|
|
AFTER (fixed):
|
|
db.py init: REMOVED (no custom search_path)
|
|
tables live in: public.keywords, public.ads, ...
|
|
Result: all connections find tables via default 'public' schema
|
|
```
|
|
|
|
## Implementation Details
|
|
|
|
### 1. Remove `SET search_path` from `db.py`
|
|
|
|
**Current:**
|
|
```python
|
|
async def _init_connection(conn: asyncpg.Connection) -> None:
|
|
await conn.execute("SET search_path TO willhaben_tracker")
|
|
```
|
|
|
|
**After:**
|
|
```python
|
|
# Removed entirely - using default public schema
|
|
```
|
|
|
|
The `_init_connection` function can be removed, and `init=_init_connection` removed from `create_pool()`. Alternatively keep the callback but make it a no-op for future use:
|
|
|
|
```python
|
|
async def _init_connection(conn: asyncpg.Connection) -> None:
|
|
# Reserved for future connection-level settings (e.g., timezone)
|
|
pass
|
|
```
|
|
|
|
### 2. Clean up migration files
|
|
|
|
- Remove `SET search_path TO willhaben_tracker;` from `supabase-migration.sql` / `01-schema.sql`
|
|
- Ensure all tables are explicitly created in the default schema (no need for prefix — that's already what happens)
|
|
|
|
### 3. Update `post-boot.sql` grants if needed
|
|
|
|
Current grants already target `public`:
|
|
```sql
|
|
GRANT USAGE ON SCHEMA public TO supabase_admin;
|
|
```
|
|
→ ✅ Already correct, no change needed.
|
|
|
|
### 4. Add schema qualification to migration file (defensive)
|
|
|
|
To be explicit and future-proof, the migration can qualify the schema:
|
|
|
|
```sql
|
|
-- Instead of implicit public.keywords
|
|
CREATE TABLE IF NOT EXISTS public.keywords (...)
|
|
```
|
|
|
|
This is optional but makes it unambiguous where tables live. For simplicity we'll keep implicit `public` since that's the default everywhere.
|
|
|
|
## Acceptance Criteria
|
|
|
|
- [ ] Removing `_init_connection` from db.py does not break any query (all table names resolve correctly)
|
|
- [ ] Connecting to the database via pgAdmin/DBeaver shows all tables without schema configuration
|
|
- [ ] The healthcheck endpoint DB check works without custom search_path
|
|
- [ ] No "relation does not exist" errors in worker logs after redeploy
|
|
- [ ] Migration SQL file no longer references `willhaben_tracker` schema
|