docs(03-02): complete analytics instrumentation plan
- Create 03-02-SUMMARY.md with task outcomes and decisions - Update STATE.md: advance to Phase 3 Plan 2 complete - Update ROADMAP.md: mark plan progress for phase 3 - Mark ANLY-02 requirement complete in REQUIREMENTS.md Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
@@ -24,7 +24,7 @@ Requirements for initial release. Each maps to roadmap phases.
|
||||
### Processing Analytics
|
||||
|
||||
- [x] **ANLY-01**: Document processing events persist to Supabase at write time (not in-memory only)
|
||||
- [ ] **ANLY-02**: Admin can view processing summary: upload counts, success/failure rates, avg processing time
|
||||
- [x] **ANLY-02**: Admin can view processing summary: upload counts, success/failure rates, avg processing time
|
||||
- [x] **ANLY-03**: Analytics instrumentation is non-blocking (fire-and-forget, never delays processing pipeline)
|
||||
|
||||
### Infrastructure
|
||||
@@ -88,7 +88,7 @@ Which phases cover which requirements. Updated during roadmap creation.
|
||||
| INFR-03 | Phase 2 | Complete |
|
||||
| INFR-02 | Phase 3 | Pending |
|
||||
| HLTH-01 | Phase 3 | Pending |
|
||||
| ANLY-02 | Phase 3 | Pending |
|
||||
| ANLY-02 | Phase 3 | Complete |
|
||||
| ALRT-03 | Phase 4 | Pending |
|
||||
|
||||
**Coverage:**
|
||||
|
||||
@@ -14,7 +14,7 @@ Decimal phases appear between their surrounding integers in numeric order.
|
||||
|
||||
- [ ] **Phase 1: Data Foundation** - Create schema, DB models, and verify existing Supabase connection wiring
|
||||
- [x] **Phase 2: Backend Services** - Health probers, alert trigger, email sender, analytics collector, scheduler, retention cleanup (completed 2026-02-24)
|
||||
- [ ] **Phase 3: API Layer** - Admin-gated routes exposing all services, instrumentation hooks in existing processors
|
||||
- [x] **Phase 3: API Layer** - Admin-gated routes exposing all services, instrumentation hooks in existing processors (completed 2026-02-24)
|
||||
- [ ] **Phase 4: Frontend** - Admin dashboard page, health panel, processing metrics, alert notification banner
|
||||
|
||||
## Phase Details
|
||||
@@ -62,7 +62,7 @@ Plans:
|
||||
2. `GET /admin/analytics` returns processing summary (upload counts, success/failure rates, avg processing time) sourced from Supabase, not in-memory state
|
||||
3. `GET /admin/alerts` and `POST /admin/alerts/:id/acknowledge` function correctly and are blocked to non-admin users
|
||||
4. Document processing in `jobProcessorService.ts` and `llmService.ts` emits analytics events at stage transitions without any change to existing processing behavior
|
||||
**Plans:** 2 plans
|
||||
**Plans:** 2/2 plans complete
|
||||
|
||||
Plans:
|
||||
- [ ] 03-01-PLAN.md — Admin auth middleware + admin routes (health, analytics, alerts endpoints)
|
||||
@@ -88,5 +88,5 @@ Phases execute in numeric order: 1 → 2 → 3 → 4
|
||||
|-------|----------------|--------|-----------|
|
||||
| 1. Data Foundation | 2/2 | Complete | 2026-02-24 |
|
||||
| 2. Backend Services | 4/4 | Complete | 2026-02-24 |
|
||||
| 3. API Layer | 0/TBD | Not started | - |
|
||||
| 3. API Layer | 2/2 | Complete | 2026-02-24 |
|
||||
| 4. Frontend | 0/TBD | Not started | - |
|
||||
|
||||
@@ -5,16 +5,16 @@
|
||||
See: .planning/PROJECT.md (updated 2026-02-24)
|
||||
|
||||
**Core value:** When something breaks — an API key expires, a service goes down, a credential needs reauthorization — the admin knows immediately and knows exactly what to fix.
|
||||
**Current focus:** Phase 2 — Backend Services
|
||||
**Current focus:** Phase 3 — API Layer
|
||||
|
||||
## Current Position
|
||||
|
||||
Phase: 2 of 4 (Backend Services)
|
||||
Plan: 4 of 4 in current phase — PHASE COMPLETE
|
||||
Status: Complete
|
||||
Last activity: 2026-02-24 — Completed 02-04 (runHealthProbes + runRetentionCleanup scheduled Cloud Functions)
|
||||
Phase: 3 of 4 (API Layer)
|
||||
Plan: 2 of 4 in current phase
|
||||
Status: In Progress
|
||||
Last activity: 2026-02-24 — Completed 03-02 (analytics instrumentation in processJob)
|
||||
|
||||
Progress: [██████░░░░] 62%
|
||||
Progress: [███████░░░] 70%
|
||||
|
||||
## Performance Metrics
|
||||
|
||||
@@ -35,6 +35,7 @@ Progress: [██████░░░░] 62%
|
||||
- Trend: Stable ~15 min/plan
|
||||
|
||||
*Updated after each plan completion*
|
||||
| Phase 03-api-layer P01 | 8 | 2 tasks | 4 files |
|
||||
|
||||
## Accumulated Context
|
||||
|
||||
@@ -66,6 +67,10 @@ Recent decisions affecting current work:
|
||||
- [Phase 02-backend-services]: runHealthProbes is a separate onSchedule Cloud Function from processDocumentJobs (PITFALL-2 compliance)
|
||||
- [Phase 02-backend-services]: retryCount: 0 on runHealthProbes — 5-minute schedule makes retry unnecessary
|
||||
- [Phase 02-backend-services]: runRetentionCleanup uses Promise.all() for parallel deletes across three independent monitoring tables
|
||||
- [Phase 03-api-layer]: 03-02: recordProcessingEvent() instrumentation uses void return — no await at 3 lifecycle points in processJob (PITFALL-6 compliance)
|
||||
- [Phase 03-api-layer]: requireAdminEmail returns 404 not 403 — does not reveal admin routes exist
|
||||
- [Phase 03-api-layer]: getPostgresPool() used for aggregate SQL — Supabase JS client does not support COUNT/AVG
|
||||
- [Phase 03-api-layer]: Admin env vars read inside function body not module level — Firebase Secrets timing constraint
|
||||
|
||||
### Pending Todos
|
||||
|
||||
@@ -79,5 +84,5 @@ None yet.
|
||||
## Session Continuity
|
||||
|
||||
Last session: 2026-02-24
|
||||
Stopped at: Completed 02-04-PLAN.md — runHealthProbes and runRetentionCleanup scheduled Cloud Function exports. Phase 2 complete.
|
||||
Stopped at: Completed 03-02-PLAN.md — analytics instrumentation in processJob() at upload_started, completed, and failed lifecycle points.
|
||||
Resume file: None
|
||||
|
||||
102
.planning/phases/03-api-layer/03-02-SUMMARY.md
Normal file
102
.planning/phases/03-api-layer/03-02-SUMMARY.md
Normal file
@@ -0,0 +1,102 @@
|
||||
---
|
||||
phase: 03-api-layer
|
||||
plan: 02
|
||||
subsystem: api
|
||||
tags: [analytics, instrumentation, fire-and-forget, document-processing]
|
||||
|
||||
# Dependency graph
|
||||
requires:
|
||||
- phase: 02-backend-services
|
||||
provides: analyticsService.recordProcessingEvent() fire-and-forget function
|
||||
- phase: 03-api-layer/03-01
|
||||
provides: analytics endpoint that reads document_processing_events table
|
||||
provides:
|
||||
- Analytics instrumentation at 3 lifecycle points in processJob()
|
||||
- document_processing_events table populated with real processing data
|
||||
affects: [03-api-layer, 03-01-analytics-endpoint]
|
||||
|
||||
# Tech tracking
|
||||
tech-stack:
|
||||
added: []
|
||||
patterns:
|
||||
- Fire-and-forget analytics calls (void return, no await) in processJob lifecycle
|
||||
|
||||
key-files:
|
||||
created: []
|
||||
modified:
|
||||
- backend/src/services/jobProcessorService.ts
|
||||
|
||||
key-decisions:
|
||||
- "All three recordProcessingEvent() calls are void/fire-and-forget (no await) — PITFALL-6 compliance confirmed"
|
||||
- "upload_started event emitted after markAsProcessing (not processing_started) per locked decision"
|
||||
- "Null-guard on job in catch block — job can be null if findById throws before assignment"
|
||||
|
||||
patterns-established:
|
||||
- "Analytics instrumentation pattern: call recordProcessingEvent() without await, no try/catch wrapper — function handles errors internally"
|
||||
|
||||
requirements-completed: [ANLY-02]
|
||||
|
||||
# Metrics
|
||||
duration: 2min
|
||||
completed: 2026-02-24
|
||||
---
|
||||
|
||||
# Phase 3 Plan 02: Analytics Instrumentation Summary
|
||||
|
||||
**Three fire-and-forget recordProcessingEvent() calls added to processJob() at upload_started, completed, and failed lifecycle points**
|
||||
|
||||
## Performance
|
||||
|
||||
- **Duration:** 2 min
|
||||
- **Started:** 2026-02-24T20:42:54Z
|
||||
- **Completed:** 2026-02-24T20:44:36Z
|
||||
- **Tasks:** 1
|
||||
- **Files modified:** 1
|
||||
|
||||
## Accomplishments
|
||||
|
||||
- Added import for `recordProcessingEvent` from analyticsService at top of jobProcessorService.ts
|
||||
- Emits `upload_started` event (fire-and-forget) after `markAsProcessing` at job start
|
||||
- Emits `completed` event with `duration_ms` (fire-and-forget) after `markAsCompleted` on success
|
||||
- Emits `failed` event with `duration_ms` and `error_message` (fire-and-forget) in catch block with null-guard
|
||||
- Zero regressions — all 64 existing tests pass, TypeScript compiles cleanly
|
||||
|
||||
## Task Commits
|
||||
|
||||
Each task was committed atomically:
|
||||
|
||||
1. **Task 1: Add analytics instrumentation to processJob lifecycle** - `dabd4a5` (feat)
|
||||
|
||||
**Plan metadata:** (docs commit follows)
|
||||
|
||||
## Files Created/Modified
|
||||
|
||||
- `backend/src/services/jobProcessorService.ts` - Added import and 3 recordProcessingEvent() instrumentation calls at job start, completion, and failure
|
||||
|
||||
## Decisions Made
|
||||
|
||||
- Confirmed `event_type: 'upload_started'` (not `'processing_started'`) matches the locked analytics schema decision
|
||||
- No await on any recordProcessingEvent() call — void return type enforces fire-and-forget at the type system level
|
||||
- Null-guard `if (job)` in catch block is necessary because `job` remains `null` if `ProcessingJobModel.findById()` throws before assignment
|
||||
|
||||
## Deviations from Plan
|
||||
|
||||
None - plan executed exactly as written.
|
||||
|
||||
## Issues Encountered
|
||||
|
||||
None.
|
||||
|
||||
## User Setup Required
|
||||
|
||||
None - no external service configuration required.
|
||||
|
||||
## Next Phase Readiness
|
||||
|
||||
- Analytics pipeline is now end-to-end: document_processing_events table receives real data when jobs run
|
||||
- GET /admin/analytics endpoint (03-01) will report actual processing metrics instead of zeros
|
||||
- No blockers for remaining Phase 03 plans
|
||||
|
||||
---
|
||||
*Phase: 03-api-layer*
|
||||
*Completed: 2026-02-24*
|
||||
Reference in New Issue
Block a user