admin/HomeAudit
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
705a2757c1d7777df15717ef598cb96ed216f67f
HomeAudit/dev_documentation/infrastructure/SERVICE_ANALYSIS_AND_CADDYFILE.md
admin 705a2757c1 Major infrastructure migration and Vaultwarden PostgreSQL troubleshooting 
COMPREHENSIVE CHANGES:

INFRASTRUCTURE MIGRATION:
- Migrated services to Docker Swarm on OMV800 (192.168.50.229)
- Deployed PostgreSQL database for Vaultwarden migration
- Updated all stack configurations for Docker Swarm compatibility
- Added comprehensive monitoring stack (Prometheus, Grafana, Blackbox)
- Implemented proper secret management for all services

VAULTWARDEN POSTGRESQL MIGRATION:
- Attempted migration from SQLite to PostgreSQL for NFS compatibility
- Created PostgreSQL stack with proper user/password configuration
- Built custom Vaultwarden image with PostgreSQL support
- Troubleshot persistent SQLite fallback issue despite PostgreSQL config
- Identified known issue where Vaultwarden silently falls back to SQLite
- Added ENABLE_DB_WAL=false to prevent filesystem compatibility issues
- Current status: Old Vaultwarden on lenovo410 still working, new one has config issues

PAPERLESS SERVICES:
- Successfully deployed Paperless-NGX and Paperless-AI on OMV800
- Both services running on ports 8000 and 3000 respectively
- Caddy configuration updated for external access
- Services accessible via paperless.pressmess.duckdns.org and paperless-ai.pressmess.duckdns.org

CADDY CONFIGURATION:
- Updated Caddyfile on Surface (192.168.50.254) for new service locations
- Fixed Vaultwarden reverse proxy to point to new Docker Swarm service
- Removed old notification hub reference that was causing conflicts
- All services properly configured for external access via DuckDNS

BACKUP AND DISCOVERY:
- Created comprehensive backup system for all hosts
- Generated detailed discovery reports for infrastructure analysis
- Implemented automated backup validation scripts
- Created migration progress tracking and verification reports

MONITORING STACK:
- Deployed Prometheus, Grafana, and Blackbox monitoring
- Created infrastructure and system overview dashboards
- Added proper service discovery and alerting configuration
- Implemented performance monitoring for all critical services

DOCUMENTATION:
- Reorganized documentation into logical structure
- Created comprehensive migration playbook and troubleshooting guides
- Added hardware specifications and optimization recommendations
- Documented all configuration changes and service dependencies

CURRENT STATUS:
- Paperless services: ✅ Working and accessible externally
- Vaultwarden: ❌ PostgreSQL configuration issues, old instance still working
- Monitoring: ✅ Deployed and operational
- Caddy: ✅ Updated and working for external access
- PostgreSQL: ✅ Database running, connection issues with Vaultwarden

NEXT STEPS:
- Continue troubleshooting Vaultwarden PostgreSQL configuration
- Consider alternative approaches for Vaultwarden migration
- Validate all external service access
- Complete final migration validation

TECHNICAL NOTES:
- Used Docker Swarm for orchestration on OMV800
- Implemented proper secret management for sensitive data
- Added comprehensive logging and monitoring
- Created automated backup and validation scripts
2025-08-30 20:18:44 -04:00

4.9 KiB
Raw Blame History

SERVICE ANALYSIS AND CADDYFILE - COMPLETE

Generated: 2025-08-29
Status: COMPLETE - Caddy deployed, Docker Swarm ready

🎯 EXECUTIVE SUMMARY

Completed comprehensive service analysis and Caddyfile deployment. All services are now properly routed through Caddy with SSL certificates. Docker Swarm is fully configured with all nodes joined and labeled.

📊 CURRENT STATUS

✅ COMPLETED TASKS

  • Service Analysis: All services identified and mapped
  • Caddyfile Deployment: Deployed on surface (192.168.50.254)
  • Security Hardening: Removed high-risk services from external access
  • Docker Swarm: All 6 nodes joined and labeled
  • Network Setup: swarm-public overlay network created
  • Paperless Services: Both NGX and AI now running on OMV800 with updated Caddyfile

🔧 INFRASTRUCTURE OVERVIEW

  • Reverse Proxy: Caddy (surface: 192.168.50.254)
  • Container Orchestration: Docker Swarm (OMV800 as manager)
  • Storage: OMV800 with mergerfs pools
  • Monitoring: Uptime Kuma (audrey: 192.168.50.145)

🏗️ DOCKER SWARM ARCHITECTURE

Node Configuration:

OMV800 (Manager)     - role=storage, cpu=high, memory=high, gpu=false
fedora               - role=compute, cpu=medium, memory=medium, gpu=false  
lenovo410            - role=compute, cpu=medium, memory=medium, gpu=false
audrey               - role=compute, cpu=medium, memory=medium, gpu=false
surface              - role=compute, cpu=medium, memory=medium, gpu=false
lenovo420            - role=ai-ml, cpu=high, memory=high, gpu=true

Networks:

  • swarm-public: Overlay network for service communication
  • database-network: For database services
  • monitoring-network: For monitoring services
  • ingress: For ingress traffic

🌐 SERVICE ROUTING (CADDY)

Active Services:

nextcloud.pressmess.duckdns.org → 192.168.50.229:8080 (OMV800)
jellyfin.pressmess.duckdns.org → 192.168.50.229:8096 (OMV800)
immich.pressmess.duckdns.org → 192.168.50.229:2283 (OMV800)
gitea.pressmess.duckdns.org → 192.168.50.229:3001 (OMV800)
joplin.pressmess.duckdns.org → 192.168.50.229:22300 (OMV800)
vikunja.pressmess.duckdns.org → 192.168.50.229:3456 (OMV800)
n8npressmess.duckdns.org → 192.168.50.181:5678 (lenovo410)
portainer.pressmess.duckdns.org → 192.168.50.181:9000 (lenovo410)
homeassistant.pressmess.duckdns.org → 192.168.50.181:8123 (lenovo410)
music-assistant.pressmess.duckdns.org → 192.168.50.181:8095 (lenovo410)
esphome.pressmess.duckdns.org → 192.168.50.181:6052 (lenovo410)
paperless-ai.pressmess.duckdns.org → 192.168.50.229:3000 (OMV800)
paperless.pressmess.duckdns.org → 192.168.50.229:8000 (OMV800)
zwave.pressmess.duckdns.org → 192.168.50.181:8091 (lenovo410)
vaultwarden.pressmess.duckdns.org → 192.168.50.181:8088 (lenovo410)
omnitools.pressmess.duckdns.org → 192.168.50.66:9080 (lenovo420)
appflowy-server.pressmess.duckdns.org → 192.168.50.254:8080 (surface)
dashboard.pressmess.duckdns.org → 192.168.50.254:8090 (surface)
uptime-kuma.pressmess.duckdns.org → 192.168.50.145:3001 (audrey)

Security-Restricted Services (Local Access Only):

  • OMV/OMV Backup: System management interfaces
  • Portainer Agent: Docker daemon access
  • Code-Server: Full IDE access
  • Dozzle: Docker logs viewer
  • AdGuard Home: DNS filtering

🔒 SECURITY DECISIONS

External Access (via Caddy):

  • ✅ User Services: Nextcloud, Jellyfin, Immich, etc.
  • ✅ Monitoring: Uptime Kuma
  • ✅ Development: Gitea, n8n
  • ✅ IoT: Home Assistant, ESPHome

Local Access Only:

  • 🔒 System Management: OMV, OMV Backup
  • 🔒 Container Management: Portainer Agent
  • 🔒 Development Tools: Code-Server, Dozzle
  • 🔒 Network Security: AdGuard Home

🎯 NEXT STEPS

Ready for Service Migration:

  1. Deploy Database Services (PostgreSQL, MariaDB)
  2. Migrate Services to Swarm (one by one)
  3. Optimize Service Distribution (move n8n to fedora)
  4. Deploy Basic Monitoring (Grafana + Netdata)
  5. Configure GPU Acceleration (for Jellyfin/Immich)

Infrastructure Status:

  • ✅ Docker Swarm: Complete
  • ✅ Caddy: Deployed and secured
  • ✅ Storage: Configured and working
  • ✅ Network: Overlay networks ready
  • ✅ Node Labels: Applied for service placement

📋 DEPLOYMENT CHECKLIST

✅ COMPLETED:

  • Service analysis and mapping
  • Caddyfile deployment and security hardening
  • Docker Swarm setup (all nodes joined)
  • Node labeling for service placement
  • Overlay network creation
  • SSL certificate generation
  • Service conflict resolution

🔄 NEXT:

  • Deploy database services
  • Migrate services to Docker Swarm
  • Optimize service distribution
  • Deploy monitoring stack
  • Configure GPU acceleration

Status: READY FOR SERVICE MIGRATION 🚀

Reference in New Issue View Git Blame Copy Permalink