Files

admin 9c916d12f4 feat: Production release v2.0.0 - Simple Document Processor

Major release with significant performance improvements and new processing strategy.

## Core Changes
- Implemented simple_full_document processing strategy (default)
- Full document → LLM approach: 1-2 passes, ~5-6 minutes processing time
- Achieved 100% completeness with 2 API calls (down from 5+)
- Removed redundant Document AI passes for faster processing

## Financial Data Extraction
- Enhanced deterministic financial table parser
- Improved FY3/FY2/FY1/LTM identification from varying CIM formats
- Automatic merging of parser results with LLM extraction

## Code Quality & Infrastructure
- Cleaned up debug logging (removed emoji markers from production code)
- Fixed Firebase Secrets configuration (using modern defineSecret approach)
- Updated OpenAI API key
- Resolved deployment conflicts (secrets vs environment variables)
- Added .env files to Firebase ignore list

## Deployment
- Firebase Functions v2 deployment successful
- All 7 required secrets verified and configured
- Function URL: https://api-y56ccs6wva-uc.a.run.app

## Performance Improvements
- Processing time: ~5-6 minutes (down from 23+ minutes)
- API calls: 1-2 (down from 5+)
- Completeness: 100% achievable
- LLM Model: claude-3-7-sonnet-latest

## Breaking Changes
- Default processing strategy changed to 'simple_full_document'
- RAG processor available as alternative strategy 'document_ai_agentic_rag'

## Files Changed
- 36 files changed, 5642 insertions(+), 4451 deletions(-)
- Removed deprecated documentation files
- Cleaned up unused services and models

This release represents a major refactoring focused on speed, accuracy, and maintainability.

2025-11-09 21:07:22 -05:00

3.1 KiB

Raw Blame History

Quick Fix Implementation Summary

Problem

List fields (keyAttractions, potentialRisks, valueCreationLevers, criticalQuestions, missingInformation) were not consistently generating 5-8 numbered items, causing test failures.

Solution Implemented (Phase 1: Quick Fix)

Files Modified

backend/src/services/llmService.ts
- Added generateText() method for simple text completion tasks
- Line 105-121: New public method wrapping callLLM for quick repairs
backend/src/services/optimizedAgenticRAGProcessor.ts
- Line 1299-1320: Added list field validation call before returning results
- Line 2136-2307: Added 3 new methods:
  - validateAndRepairListFields() - Validates all list fields have 5-8 items
  - repairListField() - Uses LLM to fix lists with wrong item count
  - getNestedField() / setNestedField() - Utility methods for nested object access

How It Works

After multi-pass extraction completes, the code now validates each list field
If a list has < 5 or > 8 items, it automatically repairs it:
- For lists < 5 items: Asks LLM to expand to 6 items
- For lists > 8 items: Asks LLM to consolidate to 7 items
Uses document context to ensure new items are relevant
Lower temperature (0.3) for more consistent output
Tracks repair API calls separately

Test Status

✅ Build successful
🔄 Running pipeline test to validate fix
Expected: All tests should pass with list validation

Next Steps (Phase 2: Proper Fix - This Week)

Implement Tool Use API (Proper Solution)

Create /backend/src/services/llmStructuredExtraction.ts:

Use Anthropic's tool use API with JSON schema
Define strict schemas with minItems/maxItems constraints
Claude will internally retry until schema compliance
More reliable than post-processing repair

Benefits:

100% schema compliance (Claude retries internally)
No post-processing repair needed
Lower overall API costs (fewer retry attempts)
Better architectural pattern

Timeline:

Phase 1 (Quick Fix): ✅ Complete (2 hours)
Phase 2 (Tool Use): 📅 Implement this week (6 hours)
Total investment: 8 hours

Additional Improvements for Later

1. Semantic Chunking (Week 2)

Replace fixed 4000-char chunks with semantic chunking
Respect document structure (don't break tables/sections)
Use 800-char chunks with 200-char overlap
Expected improvement: 12-30% better retrieval accuracy

2. Hybrid Retrieval (Week 3)

Add BM25/keyword search alongside vector similarity
Implement cross-encoder reranking
Consider HyDE (Hypothetical Document Embeddings)
Expected improvement: 15-25% better retrieval accuracy

3. Fix RAG Search Issue

Current logs show avgSimilarity: 0
Implement HyDE or improve query embedding strategy
Problem: Query embeddings don't match document embeddings well

References

Claude Tool Use: https://docs.claude.com/en/docs/agents-and-tools/tool-use
RAG Chunking: https://community.databricks.com/t5/technical-blog/the-ultimate-guide-to-chunking-strategies
Structured Output: https://dev.to/heuperman/how-to-get-consistent-structured-output-from-claude-20o5

3.1 KiB Raw Blame History