feat: Implement optimized agentic RAG processor with vector embeddings and LLM analysis

- Add LLM analysis integration to optimized agentic RAG processor
- Fix strategy routing in job queue service to use configured processing strategy
- Update ProcessingResult interface to include LLM analysis results
- Integrate vector database operations with semantic chunking
- Add comprehensive CIM review generation with proper error handling
- Fix TypeScript errors and improve type safety
- Ensure complete pipeline from upload to final analysis output

The optimized agentic RAG processor now:
- Creates intelligent semantic chunks with metadata enrichment
- Generates vector embeddings for all chunks
- Stores chunks in pgvector database with optimized batching
- Runs LLM analysis to generate comprehensive CIM reviews
- Provides complete integration from upload to final output

Tested successfully with STAX CIM document processing.
This commit is contained in:
Jon
2025-07-28 20:11:32 -04:00
parent 7cca54445d
commit adb33154cc
8 changed files with 166 additions and 125 deletions

View File

@@ -1,5 +1,6 @@
import { EventEmitter } from 'events';
import { logger } from '../utils/logger';
import { config } from '../config/env';
import { ProcessingOptions } from './documentProcessingService';
import { unifiedDocumentProcessor } from './unifiedDocumentProcessor';
@@ -209,8 +210,8 @@ class JobQueueService extends EventEmitter {
await this.updateJobStatus(job.id, 'processing');
// Use unified processor for strategy-aware processing
const strategy = options?.strategy || 'chunking';
logger.info('Processing document job with strategy', { documentId, strategy, jobId: job.id });
const strategy = options?.strategy || config.processingStrategy;
logger.info('Processing document job with strategy', { documentId, strategy, jobId: job.id, configStrategy: config.processingStrategy });
const result = await unifiedDocumentProcessor.processDocument(
documentId,