# Project Cleanup Plan

## Files Found for Cleanup

### 🗑️ Category 1: SAFE TO DELETE (Backups & Temp Files)

**Backup Files:**
- `backend/.env.backup` (4.1K, Nov 4)
- `backend/.env.backup.20251031_221937` (4.1K, Oct 31)
- `backend/diagnostic-report.json` (1.9K, Oct 31)

**Total Space:** ~10KB

**Action:** DELETE - These are temporary diagnostic/backup files

---

### 📄 Category 2: REDUNDANT DOCUMENTATION (Consider Deleting)

**Analysis Reports (Already in Git History):**
- `CLEANUP_ANALYSIS_REPORT.md` (staged for deletion)
- `CLEANUP_COMPLETION_REPORT.md` (staged for deletion)
- `DOCUMENTATION_AUDIT_REPORT.md` (staged for deletion)
- `DOCUMENTATION_COMPLETION_REPORT.md` (staged for deletion)
- `FRONTEND_DOCUMENTATION_SUMMARY.md` (staged for deletion)
- `LLM_DOCUMENTATION_SUMMARY.md` (staged for deletion)
- `OPERATIONAL_DOCUMENTATION_SUMMARY.md` (staged for deletion)

**Action:** ALREADY STAGED FOR DELETION - Git will handle

**Duplicate/Outdated Guides:**
- `BETTER_APPROACHES.md` (untracked)
- `DEPLOYMENT_INSTRUCTIONS.md` (untracked) - Duplicate of `DEPLOYMENT_GUIDE.md`?
- `IMPLEMENTATION_GUIDE.md` (untracked)
- `LLM_ANALYSIS.md` (untracked)

**Action:** REVIEW THEN DELETE if redundant with other docs

---

### 🛠️ Category 3: DIAGNOSTIC SCRIPTS (28 total)

**Keep These (Core Utilities):**
- `check-database-failures.ts` ✅ (used in troubleshooting)
- `check-current-processing.ts` ✅ (monitoring)
- `test-openrouter-simple.ts` ✅ (testing)
- `test-full-llm-pipeline.ts` ✅ (testing)
- `setup-database.ts` ✅ (setup)

**Consider Deleting (One-Time Use):**
- `check-current-job.ts` (redundant with check-current-processing)
- `check-table-schema.ts` (one-time diagnostic)
- `check-third-party-services.ts` (one-time diagnostic)
- `comprehensive-diagnostic.ts` (one-time diagnostic)
- `create-job-direct.ts` (testing helper)
- `create-job-for-stuck-document.ts` (one-time fix)
- `create-test-job.ts` (testing helper)
- `diagnose-processing-issues.ts` (one-time diagnostic)
- `diagnose-upload-issues.ts` (one-time diagnostic)
- `fix-table-schema.ts` (one-time fix)
- `mark-stuck-as-failed.ts` (one-time fix)
- `monitor-document-processing.ts` (redundant)
- `monitor-system.ts` (redundant)
- `setup-gcs-permissions.ts` (one-time setup)
- `setup-processing-jobs-table.ts` (one-time setup)
- `test-gcs-integration.ts` (one-time test)
- `test-job-creation.ts` (testing helper)
- `test-linkage.ts` (one-time test)
- `test-llm-processing-offline.ts` (testing)
- `test-openrouter-quick.ts` (redundant with simple)
- `test-postgres-connection.ts` (one-time test)
- `test-production-upload.ts` (one-time test)
- `test-staging-environment.ts` (one-time test)

**Action:** ARCHIVE or DELETE ~18-20 scripts

---

### 📁 Category 4: SHELL SCRIPTS & SQL

**Shell Scripts:**
- `backend/scripts/check-document-status.sh` (shell version, have TS version)
- `backend/scripts/sync-firebase-config.sh` (one-time use)
- `backend/scripts/sync-firebase-config.ts` (one-time use)
- `backend/scripts/run-sql-file.js` (utility, keep?)
- `backend/scripts/verify-schema.js` (one-time use)

**SQL Directory:**
- `backend/sql/` (contains migration scripts?)

**Action:** REVIEW - Keep utilities, delete one-time scripts

---

### 📝 Category 5: DOCUMENTATION TO KEEP

**Essential Docs:**
- `README.md` ✅
- `QUICK_START.md` ✅
- `backend/TROUBLESHOOTING_PLAN.md` ✅ (just created)
- `DEPLOYMENT_GUIDE.md` ✅
- `CONFIGURATION_GUIDE.md` ✅
- `DATABASE_SCHEMA_DOCUMENTATION.md` ✅
- `BPCP CIM REVIEW TEMPLATE.md` ✅

**Consider Consolidating:**
- Multiple service `.md` files in `backend/src/services/`
- Multiple component `.md` files in `frontend/src/`

---

## Recommended Action Plan

### Phase 1: Safe Cleanup (No Risk)
```bash
# Delete backup files
rm backend/.env.backup*
rm backend/diagnostic-report.json

# Clear old logs (keep last 7 days)
find backend/logs -name "*.log" -mtime +7 -delete
```

### Phase 2: Remove One-Time Diagnostic Scripts
```bash
cd backend/src/scripts

# Delete one-time diagnostics
rm check-table-schema.ts
rm check-third-party-services.ts
rm comprehensive-diagnostic.ts
rm create-job-direct.ts
rm create-job-for-stuck-document.ts
rm create-test-job.ts
rm diagnose-processing-issues.ts
rm diagnose-upload-issues.ts
rm fix-table-schema.ts
rm mark-stuck-as-failed.ts
rm setup-gcs-permissions.ts
rm setup-processing-jobs-table.ts
rm test-gcs-integration.ts
rm test-job-creation.ts
rm test-linkage.ts
rm test-openrouter-quick.ts
rm test-postgres-connection.ts
rm test-production-upload.ts
rm test-staging-environment.ts
```

### Phase 3: Remove Redundant Documentation
```bash
cd /home/jonathan/Coding/cim_summary

# Delete untracked redundant docs
rm BETTER_APPROACHES.md
rm LLM_ANALYSIS.md
rm IMPLEMENTATION_GUIDE.md

# If DEPLOYMENT_INSTRUCTIONS.md is duplicate:
# rm DEPLOYMENT_INSTRUCTIONS.md
```

### Phase 4: Consolidate Service Documentation
Move inline documentation comments instead of separate `.md` files

---

## Estimated Space Saved

- Backup files: ~10KB
- Diagnostic scripts: ~50-100KB
- Documentation: ~50KB
- Old logs: Variable (could be 100s of KB)

**Total:** ~200-300KB (not huge, but cleaner project)

---

## Recommendation

**Execute Phase 1 immediately** (safe, no risk)
**Execute Phase 2 after review** (can always recreate scripts)
**Hold Phase 3** until you confirm docs are redundant
**Hold Phase 4** for later refactoring

Would you like me to execute the cleanup?