9c916d12f4
Major release with significant performance improvements and new processing strategy. ## Core Changes - Implemented simple_full_document processing strategy (default) - Full document → LLM approach: 1-2 passes, ~5-6 minutes processing time - Achieved 100% completeness with 2 API calls (down from 5+) - Removed redundant Document AI passes for faster processing ## Financial Data Extraction - Enhanced deterministic financial table parser - Improved FY3/FY2/FY1/LTM identification from varying CIM formats - Automatic merging of parser results with LLM extraction ## Code Quality & Infrastructure - Cleaned up debug logging (removed emoji markers from production code) - Fixed Firebase Secrets configuration (using modern defineSecret approach) - Updated OpenAI API key - Resolved deployment conflicts (secrets vs environment variables) - Added .env files to Firebase ignore list ## Deployment - Firebase Functions v2 deployment successful - All 7 required secrets verified and configured - Function URL: https://api-y56ccs6wva-uc.a.run.app ## Performance Improvements - Processing time: ~5-6 minutes (down from 23+ minutes) - API calls: 1-2 (down from 5+) - Completeness: 100% achievable - LLM Model: claude-3-7-sonnet-latest ## Breaking Changes - Default processing strategy changed to 'simple_full_document' - RAG processor available as alternative strategy 'document_ai_agentic_rag' ## Files Changed - 36 files changed, 5642 insertions(+), 4451 deletions(-) - Removed deprecated documentation files - Cleaned up unused services and models This release represents a major refactoring focused on speed, accuracy, and maintainability.
51 lines
1.2 KiB
SQL
51 lines
1.2 KiB
SQL
-- Verify document exists at database level (bypassing all RLS and views)
|
|
|
|
-- Step 1: Check if documents is a table or view
|
|
SELECT
|
|
table_schema,
|
|
table_name,
|
|
table_type
|
|
FROM information_schema.tables
|
|
WHERE table_name = 'documents'
|
|
AND table_schema = 'public';
|
|
|
|
-- Step 2: Check document with superuser privileges (bypasses everything)
|
|
-- This will show if document actually exists in base table
|
|
SET ROLE postgres;
|
|
|
|
SELECT
|
|
id,
|
|
user_id,
|
|
status,
|
|
original_file_name,
|
|
created_at
|
|
FROM public.documents
|
|
WHERE id = '78359b58-762c-4a68-a8e4-17ce38580a8d'::uuid;
|
|
|
|
-- If no rows returned, document doesn't exist in base table
|
|
-- If rows returned, document exists but FK constraint still can't see it
|
|
|
|
RESET ROLE;
|
|
|
|
-- Step 3: Check all schemas for documents table
|
|
SELECT
|
|
schemaname,
|
|
tablename,
|
|
tableowner
|
|
FROM pg_tables
|
|
WHERE tablename = 'documents';
|
|
|
|
-- Step 4: Check if there are any views named documents
|
|
SELECT
|
|
schemaname,
|
|
viewname
|
|
FROM pg_views
|
|
WHERE viewname = 'documents';
|
|
|
|
-- Step 5: Count total documents in base table
|
|
SET ROLE postgres;
|
|
SELECT COUNT(*) as total_documents FROM public.documents;
|
|
SELECT COUNT(*) as processing_llm_documents FROM public.documents WHERE status = 'processing_llm';
|
|
RESET ROLE;
|
|
|