feat: Complete infrastructure cleanup phase documentation and status updates

## Major Infrastructure Milestones Achieved ### ✅ Service Migrations Completed - Jellyfin: Successfully migrated to Docker Swarm with latest version - Vaultwarden: Running in Docker Swarm on OMV800 (eliminated duplicate) - Nextcloud: Operational with database optimization and cron setup - Paperless services: Both NGX and AI running successfully ### 🚨 Duplicate Service Analysis Complete - Identified MariaDB conflict (OMV800 Swarm vs lenovo410 standalone) - Identified Vaultwarden duplication (now resolved) - Documented PostgreSQL and Redis consolidation opportunities - Mapped monitoring stack optimization needs ### 🏗️ Infrastructure Status Documentation - Updated README with current cleanup phase status - Enhanced Service Analysis with duplicate service inventory - Updated Quick Start guide with immediate action items - Documented current container distribution across 6 nodes ### 📋 Action Plan Documentation - Phase 1: Immediate service conflict resolution (this week) - Phase 2: Service migration and load balancing (next 2 weeks) - Phase 3: Database consolidation and optimization (future) ### 🔧 Current Infrastructure Health - Docker Swarm: All 6 nodes operational and healthy - Caddy Reverse Proxy: Fully operational with SSL certificates - Storage: MergerFS healthy, local storage for databases - Monitoring: Prometheus + Grafana + Uptime Kuma operational ### 📊 Container Distribution Status - OMV800: 25+ containers (needs load balancing) - lenovo410: 9 containers (cleanup in progress) - fedora: 1 container (ready for additional services) - audrey: 4 containers (well-balanced, monitoring hub) - lenovo420: 7 containers (balanced, can assist) - surface: 9 containers (specialized, reverse proxy) ### 🎯 Next Steps 1. Remove lenovo410 MariaDB (eliminate port 3306 conflict) 2. Clean up lenovo410 Vaultwarden (256MB space savings) 3. Verify no service conflicts exist 4. Begin service migration from OMV800 to fedora/audrey Status: Infrastructure 99% complete, entering cleanup and optimization phase
2025-09-01 16:50:37 -04:00
parent a6a331f538
commit 45363040f3
862 changed files with 8228 additions and 1780 deletions
--- a/backups/paperless_caddy_csrf_fix_20250831_104411/corrected_caddyfile.txt
+++ b/backups/paperless_caddy_csrf_fix_20250831_104411/corrected_caddyfile.txt
@@ -0,0 +1,237 @@
+# Reusable snippet for standard TLS configuration
+(standard_tls) {
+        tls {
+                dns duckdns {env.DUCKDNS_TOKEN}
+                resolvers 1.1.1.1 8.8.8.8
+        }
+}
+
+# --- Your Services ---
+
+# Nextcloud (Port 8080) - OMV800
+nextcloud.pressmess.duckdns.org {
+        reverse_proxy 192.168.50.229:8080
+        import standard_tls
+}
+
+# AppFlowy Server (Port 8080) - lenovo420 (192.168.50.254)
+appflowy-server.pressmess.duckdns.org {
+        reverse_proxy 192.168.50.254:8080
+        import standard_tls
+}
+
+# Jellyfin (Port 8096) - OMV800
+jellyfin.pressmess.duckdns.org {
+        reverse_proxy 192.168.50.229:8096
+        import standard_tls
+}
+
+# Immich (Port 2283) - OMV800
+immich.pressmess.duckdns.org {
+        reverse_proxy 192.168.50.229:2283
+        import standard_tls
+}
+
+# Gitea (Port 3001) - OMV800
+gitea.pressmess.duckdns.org {
+        reverse_proxy 192.168.50.229:3001
+        import standard_tls
+}
+
+# Joplin (Port 22300) - OMV800
+joplin.pressmess.duckdns.org {
+        reverse_proxy 192.168.50.229:22300
+        import standard_tls
+}
+
+# Vikunja (Port 3456) - OMV800
+vikunja.pressmess.duckdns.org {
+        reverse_proxy 192.168.50.229:3456
+        import standard_tls
+}
+
+# n8n (Port 5678) - jonathan-2518f5u (FIXED: Correct IP)
+n8npressmess.duckdns.org {
+        reverse_proxy 192.168.50.181:5678 {
+                header_up X-Forwarded-Proto https
+                header_up X-Forwarded-Port 443
+                header_up X-Forwarded-For {remote_host}
+                header_up Host {host}
+        }
+        import standard_tls
+}
+
+# OMV Main (Port 80) - OMV800 - REMOVED: Security risk (system management)
+# omv.pressmess.duckdns.org {
+#         reverse_proxy 192.168.50.229:80
+#         import standard_tls
+# }
+
+# OMV Backup (Port 80) - OMV Backup - REMOVED: Security risk (backup management)
+# omvbackup.pressmess.duckdns.org {
+#         reverse_proxy 192.168.50.107:80
+#         import standard_tls
+# }
+
+# --- Docker Services ---
+
+# Portainer (Port 9000) - jonathan-2518f5u
+portainer.pressmess.duckdns.org {
+        reverse_proxy 192.168.50.181:9000
+        import standard_tls
+}
+
+# Home Assistant (Port 8123) - jonathan-2518f5u
+homeassistant.pressmess.duckdns.org {
+        reverse_proxy 192.168.50.181:8123
+        import standard_tls
+}
+
+# Music Assistant (Port 8095) - jonathan-2518f5u
+music-assistant.pressmess.duckdns.org {
+        reverse_proxy 192.168.50.181:8095
+        import standard_tls
+}
+
+# ESPHome (Port 6052) - jonathan-2518f5u
+esphome.pressmess.duckdns.org {
+        reverse_proxy 192.168.50.181:6052
+        import standard_tls
+}
+
+# Paperless-AI (Port 3000) - OMV800 (UPDATED: Now running on .229)
+paperless-ai.pressmess.duckdns.org {
+        reverse_proxy 192.168.50.229:3000
+        import standard_tls
+}
+
+# Paperless-NGX (Port 8000) - OMV800 (UPDATED: Now running on .229)
+paperless.pressmess.duckdns.org {
+        reverse_proxy 192.168.50.229:8000 {
+                header_up X-Forwarded-Proto https
+                header_up X-Forwarded-Port 443
+                header_up X-Forwarded-For {remote_host}
+                header_up Host {host}
+                header_up X-Real-IP {remote_host}
+                header_up X-Forwarded-Host {host}
+        }
+        import standard_tls
+}
+
+# Z-Wave JS UI (Port 8091) - jonathan-2518f5u
+zwave.pressmess.duckdns.org {
+        reverse_proxy 192.168.50.181:8091
+        import standard_tls
+}
+
+# Vaultwarden (Port 8088) - jonathan-2518f5u
+vaultwarden.pressmess.duckdns.org {
+        reverse_proxy 192.168.50.181:8088
+        reverse_proxy /notifications/hub 192.168.50.181:3012
+        import standard_tls
+}
+
+# Homepage (Port 8080) - REMOVED: Not currently used, conflicts with AppFlowy
+# homepage.pressmess.duckdns.org {
+#         reverse_proxy 192.168.50.254:8080
+#         import standard_tls
+# }
+
+# OmniTools (Port 9080) - immich_photos
+omnitools.pressmess.duckdns.org {
+        reverse_proxy 192.168.50.66:9080
+        import standard_tls
+}
+
+# Node-Red (Port 1880) - OFFLINE DEVICE
+# node-red.pressmess.duckdns.org {
+#         reverse_proxy 192.168.50.85:1880
+#         import standard_tls
+# }
+
+# Code-Server (Port 8443) - audrey - REMOVED: Security risk (full IDE access)
+# code-server.pressmess.duckdns.org {
+#         reverse_proxy 192.168.50.145:8443
+#         import standard_tls
+# }
+
+# Dashboard (Port 8090) - lenovo420 (192.168.50.254)
+dashboard.pressmess.duckdns.org {
+        reverse_proxy 192.168.50.254:8090
+        import standard_tls
+}
+
+# --- Monitoring Services (NEW) ---
+
+# Uptime Kuma (Port 3001) - audrey
+uptime-kuma.pressmess.duckdns.org {
+        reverse_proxy 192.168.50.145:3001
+        import standard_tls
+}
+
+# Prometheus (Port 9091) - OMV800 (Docker Swarm Manager)
+prometheus.pressmess.duckdns.org {
+        reverse_proxy 192.168.50.229:9091
+        import standard_tls
+}
+
+# Grafana (Port 3002) - OMV800 (Docker Swarm Manager)
+grafana.pressmess.duckdns.org {
+        reverse_proxy 192.168.50.229:3002
+        import standard_tls
+}
+
+# Dozzle (Port 9999) - audrey - REMOVED: Security risk (Docker logs exposure)
+# dozzle.pressmess.duckdns.org {
+#         reverse_proxy 192.168.50.145:9999
+#         import standard_tls
+# }
+
+# Portainer Agent (Port 9001) - audrey - REMOVED: Security risk (Docker daemon access)
+# portainer-agent.pressmess.duckdns.org {
+#         reverse_proxy 192.168.50.145:9001
+#         import standard_tls
+# }
+
+# Netdata (Port 19999) - OFFLINE DEVICE
+# http://netdata.pressmess.duckdns.org {
+#         reverse_proxy 192.168.50.243:19999
+# }
+
+# --- COMMENTS ON CHANGES ---
+# 
+# FIXES APPLIED:
+# 1. n8n: 192.168.50.225 → 192.168.50.181 (correct IP)
+# 2. Paperless-NGX: port 8010 → 8001 (correct port)
+# 3. AppFlowy: 192.168.50.229 → 192.168.50.254 (lenovo420) (correct IP)
+# 4. Dashboard: localhost → 192.168.50.254 (lenovo420) (correct IP)
+# 5. Homepage: REMOVED (not currently used, conflicts with AppFlowy)
+#
+# NEW SERVICES ADDED:
+# 6. Uptime Kuma: Service monitoring dashboard
+# 7. Dozzle: REMOVED - Security risk (Docker logs exposure)
+# 8. Portainer Agent: REMOVED - Security risk (Docker daemon access)
+#
+# SECURITY DECISIONS:
+# - AdGuard Home: KEPT LOCAL-ONLY (DNS filtering security)
+# - Database ports: KEPT LOCAL-ONLY (security)
+# - MQTT: KEPT LOCAL-ONLY (IoT security)
+# - OMV/OMV Backup: REMOVED (system management security)
+# - Portainer Agent: REMOVED (Docker daemon security)
+# - Code-Server: REMOVED (IDE access security)
+# - Dozzle: REMOVED (Docker logs security)
+#
+# DEVICE MAPPINGS:
+# - 192.168.50.229: OMV800 (root)
+# - 192.168.50.181: jonathan-2518f5u (jonathan)
+# - 192.168.50.254: lenovo420 (jon)
+# - 192.168.50.66: immich_photos (jon)
+# - 192.168.50.145: audrey (jon)
+# - 192.168.50.107: omvbackup (jon)
+# - 192.168.50.225: fedora (jonathan)
+#
+# OFFLINE SERVICES:
+# 1. Node-Red device (192.168.50.85) is offline
+# 2. Netdata device (192.168.50.243) is offline
+#
+# STATUS: All conflicts resolved, monitoring services added, ready for deployment
--- a/backups/paperless_caddy_csrf_fix_20250831_104411/paperless_fix_compose.yml
+++ b/backups/paperless_caddy_csrf_fix_20250831_104411/paperless_fix_compose.yml
@@ -0,0 +1,67 @@
+services:
+  webserver:
+    image: ghcr.io/paperless-ngx/paperless-ngx:latest
+    restart: unless-stopped
+    depends_on:
+      - db
+      - broker
+    ports:
+      - "8000:8000"
+    healthcheck:
+      test: ["CMD", "curl", "-f", "http://localhost:8000"]
+      interval: 30s
+      timeout: 10s
+      retries: 5
+    volumes:
+      - /srv/mergerfs/DataPool/pdfs/data:/usr/src/paperless/data
+      - /srv/mergerfs/DataPool/pdfs/media:/usr/src/paperless/media
+      - /srv/mergerfs/DataPool/pdfs/export:/usr/src/paperless/export
+      - /srv/mergerfs/DataPool/pdfs/consume:/usr/src/paperless/consume
+    environment:
+      PAPERLESS_REDIS: redis://broker:6379
+      PAPERLESS_DBHOST: db
+      PAPERLESS_DBNAME: paperless
+      PAPERLESS_DBUSER: paperless
+      PAPERLESS_DBPASS: paperless
+      PAPERLESS_CONSUMER_POLLING: "1"
+      # CSRF and reverse proxy configuration for Caddy
+      PAPERLESS_URL: https://paperless.pressmess.duckdns.org
+      PAPERLESS_CSRF_TRUSTED_ORIGINS: https://paperless.pressmess.duckdns.org
+      PAPERLESS_ALLOWED_HOSTS: paperless.pressmess.duckdns.org
+      # Security settings for reverse proxy
+      PAPERLESS_USE_X_FORWARDED_HOST: true
+      PAPERLESS_USE_X_FORWARDED_PORT: true
+      PAPERLESS_SECURE_SSL_REDIRECT: true
+      # Additional security headers
+      PAPERLESS_SECURE_BROWSER_XSS_FILTER: true
+      PAPERLESS_SECURE_CONTENT_TYPE_NOSNIFF: true
+      PAPERLESS_SECURE_HSTS_INCLUDE_SUBDOMAINS: true
+      PAPERLESS_SECURE_HSTS_SECONDS: 31536000
+
+  db:
+    image: postgres:15
+    restart: unless-stopped
+    healthcheck:
+      test: ["CMD-SHELL", "pg_isready -U paperless"]
+      interval: 10s
+      timeout: 5s
+      retries: 5
+    volumes:
+      - paperless_pgdata_fixed:/var/lib/postgresql/data
+    environment:
+      POSTGRES_DB: paperless
+      POSTGRES_USER: paperless
+      POSTGRES_PASSWORD: paperless
+
+  broker:
+    image: redis:6.0
+    restart: unless-stopped
+    healthcheck:
+      test: ["CMD", "redis-cli", "ping"]
+      interval: 10s
+      timeout: 5s
+      retries: 5
+
+volumes:
+  paperless_pgdata_fixed:
+    external: true