cim_summary/.planning/milestones/v1.0-phases/01-data-foundation/01-CONTEXT.md

Phase 1: Data Foundation - Context

Gathered: 2026-02-24 Status: Ready for planning

## Phase Boundary

Create database schema (tables, indexes, migrations) and model layer for the monitoring system. Requirements: INFR-01 (tables with indexes), INFR-04 (use existing Supabase connection). No services, no API routes, no frontend work.

## Implementation Decisions

Migration approach

• Use the existing DatabaseMigrator class in backend/src/models/migrate.ts
• New .sql files go in src/models/migrations/, run with npm run db:migrate
• The migrator tracks applied migrations in a migrations table — handles idempotency
• Forward-only migrations (no rollback/down scripts). If something needs fixing, write a new migration.
• Migrations execute via supabase.rpc('exec_sql', { sql }) — works with cloud Supabase from any environment including Firebase

Schema details

• Status fields use TEXT with CHECK constraints (e.g., CHECK (status IN ('healthy','degraded','down'))) — easy to extend, no enum type management
• Table names are descriptive, matching existing style: service_health_checks, alert_events (like processing_jobs, document_chunks)
• Include JSONB probe_details / details columns for flexible metadata per service (response codes, error specifics) without future schema changes
• All tables get indexes on created_at (required for 30-day retention queries and dashboard time-range filters)
• Enable Row Level Security on new tables — admin-only access, matching existing security patterns

Model layer pattern

• One model file per table: HealthCheckModel.ts, AlertEventModel.ts
• Static methods on model classes (e.g., AlertEventModel.create(), AlertEventModel.findActive()) — matches DocumentModel.ts pattern
• Use getSupabaseServiceClient() (PostgREST) for all monitoring reads/writes — monitoring is not on the critical processing path, so no need for direct PostgreSQL pool
• Input validation in the model layer before writing (defense in depth alongside DB CHECK constraints)

Claude's Discretion

• Exact column types for non-status fields (INTEGER vs BIGINT for latency_ms, etc.)
• Whether to create a shared base model or keep models independent
• Index strategy beyond created_at (e.g., composite indexes on service_name + created_at)
• Winston logging patterns within model methods

## Specific Ideas

• The existing performance_metrics table already exists but nothing writes to it — verify its schema before building on it
• Research found that uploadMonitoringService.ts stores data in-memory only — the new persistent tables replace this pattern
• The ProcessingJobModel.ts uses direct PostgreSQL for critical writes as a pattern reference, but monitoring tables don't need this

## Deferred Ideas

None — discussion stayed within phase scope

---

Phase: 01-data-foundation Context gathered: 2026-02-24