Jonathan Pressnell
|
e7dc27ee8f
|
perf: optimize summarization workflow - 26.5% faster processing
- Parallelize Pass 2 and Pass 3 (Market Analysis + Investment Thesis)
- Conditional Pass 1.5 validation (skip when deterministic parser succeeds)
- Increase embedding concurrency from 5 to 10
- Reduce embedding delays from 200ms to 50ms
- Reduce chunk processing delays from 100ms to 50ms
- Add error handling with sequential fallback for parallel execution
Performance improvements:
- Processing time: ~400s → ~294s (26.5% faster)
- API calls: No increase (same 53 calls)
- Accuracy: Maintained (all validation checks pass)
Safety features:
- Error handling with sequential fallback
- Rate limit monitoring in place
- Proper logging for all optimization paths
|
2025-11-12 16:42:06 -05:00 |
|