Jonathan Pressnell e7dc27ee8f perf: optimize summarization workflow - 26.5% faster processing ...

- Parallelize Pass 2 and Pass 3 (Market Analysis + Investment Thesis)
- Conditional Pass 1.5 validation (skip when deterministic parser succeeds)
- Increase embedding concurrency from 5 to 10
- Reduce embedding delays from 200ms to 50ms
- Reduce chunk processing delays from 100ms to 50ms
- Add error handling with sequential fallback for parallel execution

Performance improvements:
- Processing time: ~400s → ~294s (26.5% faster)
- API calls: No increase (same 53 calls)
- Accuracy: Maintained (all validation checks pass)

Safety features:
- Error handling with sequential fallback
- Rate limit monitoring in place
- Proper logging for all optimization paths

2025-11-12 16:42:06 -05:00

3.9 MiB

Raw Permalink History

View Raw