diff --git a/.planning/REQUIREMENTS.md b/.planning/REQUIREMENTS.md index 2529bc0..272b212 100644 --- a/.planning/REQUIREMENTS.md +++ b/.planning/REQUIREMENTS.md @@ -24,7 +24,7 @@ Requirements for initial release. Each maps to roadmap phases. ### Processing Analytics - [x] **ANLY-01**: Document processing events persist to Supabase at write time (not in-memory only) -- [ ] **ANLY-02**: Admin can view processing summary: upload counts, success/failure rates, avg processing time +- [x] **ANLY-02**: Admin can view processing summary: upload counts, success/failure rates, avg processing time - [x] **ANLY-03**: Analytics instrumentation is non-blocking (fire-and-forget, never delays processing pipeline) ### Infrastructure @@ -88,7 +88,7 @@ Which phases cover which requirements. Updated during roadmap creation. | INFR-03 | Phase 2 | Complete | | INFR-02 | Phase 3 | Pending | | HLTH-01 | Phase 3 | Pending | -| ANLY-02 | Phase 3 | Pending | +| ANLY-02 | Phase 3 | Complete | | ALRT-03 | Phase 4 | Pending | **Coverage:** diff --git a/.planning/ROADMAP.md b/.planning/ROADMAP.md index 5197d62..71615f3 100644 --- a/.planning/ROADMAP.md +++ b/.planning/ROADMAP.md @@ -14,7 +14,7 @@ Decimal phases appear between their surrounding integers in numeric order. - [ ] **Phase 1: Data Foundation** - Create schema, DB models, and verify existing Supabase connection wiring - [x] **Phase 2: Backend Services** - Health probers, alert trigger, email sender, analytics collector, scheduler, retention cleanup (completed 2026-02-24) -- [ ] **Phase 3: API Layer** - Admin-gated routes exposing all services, instrumentation hooks in existing processors +- [x] **Phase 3: API Layer** - Admin-gated routes exposing all services, instrumentation hooks in existing processors (completed 2026-02-24) - [ ] **Phase 4: Frontend** - Admin dashboard page, health panel, processing metrics, alert notification banner ## Phase Details @@ -62,7 +62,7 @@ Plans: 2. `GET /admin/analytics` returns processing summary (upload counts, success/failure rates, avg processing time) sourced from Supabase, not in-memory state 3. `GET /admin/alerts` and `POST /admin/alerts/:id/acknowledge` function correctly and are blocked to non-admin users 4. Document processing in `jobProcessorService.ts` and `llmService.ts` emits analytics events at stage transitions without any change to existing processing behavior -**Plans:** 2 plans +**Plans:** 2/2 plans complete Plans: - [ ] 03-01-PLAN.md — Admin auth middleware + admin routes (health, analytics, alerts endpoints) @@ -88,5 +88,5 @@ Phases execute in numeric order: 1 → 2 → 3 → 4 |-------|----------------|--------|-----------| | 1. Data Foundation | 2/2 | Complete | 2026-02-24 | | 2. Backend Services | 4/4 | Complete | 2026-02-24 | -| 3. API Layer | 0/TBD | Not started | - | +| 3. API Layer | 2/2 | Complete | 2026-02-24 | | 4. Frontend | 0/TBD | Not started | - | diff --git a/.planning/STATE.md b/.planning/STATE.md index 8d1820e..5cb28a4 100644 --- a/.planning/STATE.md +++ b/.planning/STATE.md @@ -5,16 +5,16 @@ See: .planning/PROJECT.md (updated 2026-02-24) **Core value:** When something breaks — an API key expires, a service goes down, a credential needs reauthorization — the admin knows immediately and knows exactly what to fix. -**Current focus:** Phase 2 — Backend Services +**Current focus:** Phase 3 — API Layer ## Current Position -Phase: 2 of 4 (Backend Services) -Plan: 4 of 4 in current phase — PHASE COMPLETE -Status: Complete -Last activity: 2026-02-24 — Completed 02-04 (runHealthProbes + runRetentionCleanup scheduled Cloud Functions) +Phase: 3 of 4 (API Layer) +Plan: 2 of 4 in current phase +Status: In Progress +Last activity: 2026-02-24 — Completed 03-02 (analytics instrumentation in processJob) -Progress: [██████░░░░] 62% +Progress: [███████░░░] 70% ## Performance Metrics @@ -35,6 +35,7 @@ Progress: [██████░░░░] 62% - Trend: Stable ~15 min/plan *Updated after each plan completion* +| Phase 03-api-layer P01 | 8 | 2 tasks | 4 files | ## Accumulated Context @@ -66,6 +67,10 @@ Recent decisions affecting current work: - [Phase 02-backend-services]: runHealthProbes is a separate onSchedule Cloud Function from processDocumentJobs (PITFALL-2 compliance) - [Phase 02-backend-services]: retryCount: 0 on runHealthProbes — 5-minute schedule makes retry unnecessary - [Phase 02-backend-services]: runRetentionCleanup uses Promise.all() for parallel deletes across three independent monitoring tables +- [Phase 03-api-layer]: 03-02: recordProcessingEvent() instrumentation uses void return — no await at 3 lifecycle points in processJob (PITFALL-6 compliance) +- [Phase 03-api-layer]: requireAdminEmail returns 404 not 403 — does not reveal admin routes exist +- [Phase 03-api-layer]: getPostgresPool() used for aggregate SQL — Supabase JS client does not support COUNT/AVG +- [Phase 03-api-layer]: Admin env vars read inside function body not module level — Firebase Secrets timing constraint ### Pending Todos @@ -79,5 +84,5 @@ None yet. ## Session Continuity Last session: 2026-02-24 -Stopped at: Completed 02-04-PLAN.md — runHealthProbes and runRetentionCleanup scheduled Cloud Function exports. Phase 2 complete. +Stopped at: Completed 03-02-PLAN.md — analytics instrumentation in processJob() at upload_started, completed, and failed lifecycle points. Resume file: None diff --git a/.planning/phases/03-api-layer/03-02-SUMMARY.md b/.planning/phases/03-api-layer/03-02-SUMMARY.md new file mode 100644 index 0000000..fd50bf0 --- /dev/null +++ b/.planning/phases/03-api-layer/03-02-SUMMARY.md @@ -0,0 +1,102 @@ +--- +phase: 03-api-layer +plan: 02 +subsystem: api +tags: [analytics, instrumentation, fire-and-forget, document-processing] + +# Dependency graph +requires: + - phase: 02-backend-services + provides: analyticsService.recordProcessingEvent() fire-and-forget function + - phase: 03-api-layer/03-01 + provides: analytics endpoint that reads document_processing_events table +provides: + - Analytics instrumentation at 3 lifecycle points in processJob() + - document_processing_events table populated with real processing data +affects: [03-api-layer, 03-01-analytics-endpoint] + +# Tech tracking +tech-stack: + added: [] + patterns: + - Fire-and-forget analytics calls (void return, no await) in processJob lifecycle + +key-files: + created: [] + modified: + - backend/src/services/jobProcessorService.ts + +key-decisions: + - "All three recordProcessingEvent() calls are void/fire-and-forget (no await) — PITFALL-6 compliance confirmed" + - "upload_started event emitted after markAsProcessing (not processing_started) per locked decision" + - "Null-guard on job in catch block — job can be null if findById throws before assignment" + +patterns-established: + - "Analytics instrumentation pattern: call recordProcessingEvent() without await, no try/catch wrapper — function handles errors internally" + +requirements-completed: [ANLY-02] + +# Metrics +duration: 2min +completed: 2026-02-24 +--- + +# Phase 3 Plan 02: Analytics Instrumentation Summary + +**Three fire-and-forget recordProcessingEvent() calls added to processJob() at upload_started, completed, and failed lifecycle points** + +## Performance + +- **Duration:** 2 min +- **Started:** 2026-02-24T20:42:54Z +- **Completed:** 2026-02-24T20:44:36Z +- **Tasks:** 1 +- **Files modified:** 1 + +## Accomplishments + +- Added import for `recordProcessingEvent` from analyticsService at top of jobProcessorService.ts +- Emits `upload_started` event (fire-and-forget) after `markAsProcessing` at job start +- Emits `completed` event with `duration_ms` (fire-and-forget) after `markAsCompleted` on success +- Emits `failed` event with `duration_ms` and `error_message` (fire-and-forget) in catch block with null-guard +- Zero regressions — all 64 existing tests pass, TypeScript compiles cleanly + +## Task Commits + +Each task was committed atomically: + +1. **Task 1: Add analytics instrumentation to processJob lifecycle** - `dabd4a5` (feat) + +**Plan metadata:** (docs commit follows) + +## Files Created/Modified + +- `backend/src/services/jobProcessorService.ts` - Added import and 3 recordProcessingEvent() instrumentation calls at job start, completion, and failure + +## Decisions Made + +- Confirmed `event_type: 'upload_started'` (not `'processing_started'`) matches the locked analytics schema decision +- No await on any recordProcessingEvent() call — void return type enforces fire-and-forget at the type system level +- Null-guard `if (job)` in catch block is necessary because `job` remains `null` if `ProcessingJobModel.findById()` throws before assignment + +## Deviations from Plan + +None - plan executed exactly as written. + +## Issues Encountered + +None. + +## User Setup Required + +None - no external service configuration required. + +## Next Phase Readiness + +- Analytics pipeline is now end-to-end: document_processing_events table receives real data when jobs run +- GET /admin/analytics endpoint (03-01) will report actual processing metrics instead of zeros +- No blockers for remaining Phase 03 plans + +--- +*Phase: 03-api-layer* +*Completed: 2026-02-24*