Files

admin 802a6916ab Repo housekeeping and migration scaffolding:\n- Archive old audit/targeted discovery reports under archive_old_reports/\n- Remove bulky raw outputs from repo root (kept archived)\n- Update README to reflect new migration focus and structure\n- Add COMPLETE_DOCKER_SERVICES_INVENTORY.md (containers + native)\n- Add WORLD_CLASS_MIGRATION_TODO.md (detailed staged migration with backups, replication, cutover)\n- Add CLEANUP_PLAN.md and CLEANUP_SUMMARY.md\n- Scaffold core Swarm stacks: Traefik v3, PostgreSQL primary, MariaDB 10.11 primary, Redis master, Mosquitto, Netdata\nNotes: requires overlay networks (traefik-public, database-network, monitoring-network) and docker secrets for DB root passwords

2025-08-24 17:48:32 -04:00

34 KiB

Raw Blame History

WORLD-CLASS MIGRATION TO-DO LIST

Extremely Detailed, Granular Migration Plan
Zero-Downtime Infrastructure Transformation
Generated: 2025-08-24

🎯 MIGRATION OVERVIEW

This document provides an extremely detailed, granular migration plan to transform your current infrastructure into the Future-Proof Scalability architecture. Every step includes comprehensive testing, validation, and rollback procedures to ensure zero data loss and zero downtime.

Migration Philosophy

Parallel Deployment: New infrastructure runs alongside old
Gradual Cutover: Service-by-service migration with validation
Complete Redundancy: Every component has backup and failover
Automated Validation: Health checks and performance monitoring
Instant Rollback: Ability to revert any change within minutes

Success Criteria

✅ Zero data loss during migration
✅ Zero downtime for critical services
✅ 100% service availability throughout migration
✅ Performance improvement validated at each step
✅ Complete rollback capability at any point

🛠️ CORRECTIONS, GAPS, AND OPTIMIZATIONS APPLIED

MariaDB backup mismatch fixed: Replaced tar-based volume backup/SQL restore inconsistency with consistent mysqldump exports and mysql imports.
Add secrets/env inventory step: Centralize all credentials and env files from discovery outputs before any migration.
Compose/Stack generation clarified: Generate Swarm stacks from discovered compose templates; validate images, tags, networks, volumes.
Blue/Green cutover policy: Never stop old services until new path is serving live traffic and validated; keep old online for 48 hours.
Zero-downtime DB replication: Added optional PostgreSQL logical replication and MariaDB replication steps to reduce downtime to seconds.
Nextcloud migration hardening: Use occ maintenance mode, integrity checks, and post-migration repairs.
Home Assistant migration: Use full snapshot/restore; respect recorder DB migration; schedule maintenance window.
MQTT and Z-Wave not yet migrated: Added explicit sections for Mosquitto and Z-Wave JS UI (USB constraints/remote strategy).
Traefik v3 hardening: Require authenticated dashboard; ACME via DNS challenge for real domain or use internal CA; remove insecure API.
NFS-backed persistent volumes: Define Swarm volumes backed by NFS exports for stateful services.
GPU acceleration: Add driver/runtime checks and device mapping for Jellyfin/Immich ML.
Watchtower policy: Disable automatic updates in Swarm; use pinned tags and rolling updates.
Secrets management: Move passwords/API keys to Docker Secrets/Ansible Vault; remove -p inline usage.
DNS TTL management: Lower TTL 24–48h before cutover to speed failback.
DHCP/DNS transition for AdGuard: Staged rollout and fallback DNS retained for 48h.

📋 PRE-MIGRATION PREPARATION (STAGE 0)

0.0 SECRETS AND ENV INVENTORY (PREREQUISITE)

# Purpose: Centralize all credentials, tokens, and env files BEFORE backups/migration
# Sources: comprehensive_discovery_results/container_audit_results/individual_configs/
#          comprehensive_discovery_results/detailed_container_inventory.yaml
# Output:  /backup/secrets_inventory/ and docker secrets compatible files

mkdir -p /backup/secrets_inventory/env /backup/secrets_inventory/files

# 1) Collect env from running containers (sanitized copy)
for c in $(docker ps -q); do
  name=$(docker inspect --format '{{.Name}}' $c | sed 's#^/##')
  docker inspect $c > /backup/secrets_inventory/${name}_inspect.json
  docker exec $c env | sed 's/\(PASSWORD\|SECRET\|KEY\|TOKEN\)=.*/\1=REDACTED/g' \
    > /backup/secrets_inventory/env/${name}.env.sanitized
Done

# 2) Parse compose templates for env_file/includes
# Reference: comprehensive_discovery_results/container_audit_results/compose_templates/
find comprehensive_discovery_results/container_audit_results/compose_templates -type f -name "*_compose.yml" \
  -exec grep -Hn "env_file\|environment\|secrets" {} \; > /backup/secrets_inventory/compose_env_index.txt

# 3) Export existing secret files (if paths are in binds)
# Example sensitive paths often mounted
grep -R "bind\|target" comprehensive_discovery_results/detailed_container_inventory.yaml \
  | grep -E "(\.env|/secrets/|/config/)" >> /backup/secrets_inventory/bind_mount_candidates.txt

0.1 COMPREHENSIVE BACKUP STRATEGY

0.1.1 Database Backups

# Location: All database containers and native database services
# Priority: CRITICAL - Must complete before any migration

# PostgreSQL Backups
docker exec paperless-db-1 pg_dumpall > /backup/postgresql_full_$(date +%Y%m%d_%H%M%S).sql
docker exec joplin-db-1 pg_dumpall > /backup/joplin_db_$(date +%Y%m%d_%H%M%S).sql
docker exec immich_postgres pg_dumpall > /backup/immich_db_$(date +%Y%m%d_%H%M%S).sql

# MariaDB Backups
docker exec mariadb mysqldump --all-databases > /backup/mariadb_full_$(date +%Y%m%d_%H%M%S).sql
docker exec nextcloud-db mysqldump --all-databases > /backup/nextcloud_db_$(date +%Y%m%d_%H%M%S).sql

# Native Database Backups
sudo mysqldump --all-databases > /backup/native_mariadb_$(date +%Y%m%d_%H%M%S).sql
sudo -u postgres pg_dumpall > /backup/native_postgresql_$(date +%Y%m%d_%H%M%S).sql

0.1.2 Container Configuration Backups

# Location: All Docker containers
# Priority: CRITICAL - Configuration preservation

# Export all container configurations
for container in $(docker ps -aq); do
    docker inspect $container > /backup/container_configs/${container}_config.json
    docker exec $container env > /backup/container_configs/${container}_env.txt
done

# Export Docker Compose files
find /opt -name "docker-compose.yml" -exec cp {} /backup/compose_files/ \;
find /opt -name "docker-compose.yaml" -exec cp {} /backup/compose_files/ \;

0.1.3 Volume Data Backups

# Location: All Docker volumes and bind mounts
# Priority: CRITICAL - Data preservation

# Backup all Docker volumes
docker run --rm -v /var/lib/docker/volumes:/volumes -v /backup/volumes:/backup alpine tar czf /backup/docker_volumes_$(date +%Y%m%d_%H%M%S).tar.gz /volumes

# Backup critical bind mounts
tar czf /backup/immich_data_$(date +%Y%m%d_%H%M%S).tar.gz /opt/immich/data
tar czf /backup/nextcloud_data_$(date +%Y%m%d_%H%M%S).tar.gz /opt/nextcloud/data
tar czf /backup/homeassistant_data_$(date +%Y%m%d_%H%M%S).tar.gz /opt/homeassistant/config

0.1.4 Native Service Configuration Backups

# Location: All native systemd services
# Priority: HIGH - Configuration preservation

# Backup systemd service configurations
sudo systemctl list-unit-files --type=service --state=enabled > /backup/systemd_enabled_services.txt
sudo systemctl list-units --type=service --state=running > /backup/systemd_running_services.txt

# Backup service configurations
sudo cp -r /etc/systemd/system /backup/systemd_configs/
sudo cp -r /etc/nginx /backup/nginx_configs/
sudo cp -r /etc/apache2 /backup/apache2_configs/
sudo cp -r /etc/caddy /backup/caddy_configs/
sudo cp -r /etc/mariadb /backup/mariadb_configs/
sudo cp -r /etc/postgresql /backup/postgresql_configs/
sudo cp -r /etc/fail2ban /backup/fail2ban_configs/
sudo cp -r /etc/netdata /backup/netdata_configs/

0.2 INFRASTRUCTURE VALIDATION

0.2.1 Network Connectivity Testing

# Test all network connections between hosts
# Location: All hosts
# Priority: CRITICAL - Network validation

# Test inter-host connectivity
for host in omv800.local jonathan-2518f5u fedora surface lenovo420 audrey raspberrypi; do
    ping -c 3 $host
    ssh $host "echo 'SSH connectivity test successful'"
done

# Test critical service ports
for service in "OMV800:8080" "jonathan-2518f5u:8123" "surface:8000"; do
    host=$(echo $service | cut -d: -f1)
    port=$(echo $service | cut -d: -f2)
    nc -zv $host $port
done

0.2.2 Service Health Validation

# Validate all services are healthy before migration
# Location: All hosts
# Priority: CRITICAL - Service validation

# Docker container health checks
docker ps --format "table {{.Names}}\t{{.Status}}\t{{.Ports}}"

# Native service health checks
systemctl is-active --quiet mariadb && echo "MariaDB: ACTIVE" || echo "MariaDB: INACTIVE"
systemctl is-active --quiet postgresql && echo "PostgreSQL: ACTIVE" || echo "PostgreSQL: INACTIVE"
systemctl is-active --quiet nginx && echo "Nginx: ACTIVE" || echo "Nginx: INACTIVE"
systemctl is-active --quiet apache2 && echo "Apache2: ACTIVE" || echo "Apache2: INACTIVE"

0.3 MIGRATION ENVIRONMENT SETUP

0.3.1 Docker Swarm Initialization

# Location: All hosts
# Priority: CRITICAL - New infrastructure foundation

# Initialize Docker Swarm on manager node (OMV800)
docker swarm init --advertise-addr 192.168.50.225

# Join worker nodes
# Execute on each worker node:
docker swarm join --token <MANAGER_TOKEN> 192.168.50.225:2377

0.3.2 Traefik Setup

# Location: OMV800 (manager node)
# Priority: CRITICAL - Reverse proxy foundation

# Deploy Traefik stack
docker stack deploy -c /opt/traefik/docker-compose.yml traefik

# Validate Traefik deployment
docker service ls | grep traefik
curl -I http://192.168.50.225:8080/api/rawdata

🏗️ STAGE 1: INFRASTRUCTURE FOUNDATION (WEEK 1)

1.1 DOCKER SWARM CLUSTER SETUP

1.1.1 Cluster Initialization

# Day 1-2: Foundation setup
# Location: All hosts
# Priority: CRITICAL

# Step 1: Initialize Swarm on OMV800
ssh omv800.local "docker swarm init --advertise-addr 192.168.50.225"

# Step 2: Join worker nodes
ssh jonathan-2518f5u "docker swarm join --token <TOKEN> 192.168.50.225:2377"
ssh fedora "docker swarm join --token <TOKEN> 192.168.50.225:2377"
ssh surface "docker swarm join --token <TOKEN> 192.168.50.225:2377"

# Step 3: Validate cluster
docker node ls
docker service ls

1.1.2 Network Configuration

# Day 2-3: Network setup
# Location: OMV800 (manager)
# Priority: CRITICAL

# Create overlay networks
docker network create --driver overlay --attachable traefik-public
docker network create --driver overlay --attachable database-network
docker network create --driver overlay --attachable storage-network
docker network create --driver overlay --attachable monitoring-network

# Validate networks
docker network ls | grep overlay

1.2 TRAEFIK REVERSE PROXY DEPLOYMENT

1.2.1 Traefik Stack Deployment

# Day 3-4: Reverse proxy setup
# Location: OMV800
# Priority: CRITICAL

# Deploy Traefik stack
docker stack deploy -c /opt/traefik/docker-compose.yml traefik

# Validate deployment
docker service ls | grep traefik
curl -I http://192.168.50.225:8080/api/rawdata

# Test SSL certificate generation
curl -I https://traefik.localhost

1.2.2 Middleware Configuration

# Day 4-5: Security and routing setup
# Location: OMV800
# Priority: HIGH

# Configure security middlewares
# File: /opt/traefik/dynamic/middleware.yml
# Reference: migration_scripts/configs/traefik/dynamic/middleware.yml

# Test middleware functionality
curl -H "X-Forwarded-For: 192.168.1.100" http://192.168.50.225:8080/api/rawdata

1.3 MONITORING INFRASTRUCTURE

1.3.1 Prometheus Deployment

# Day 5-6: Monitoring setup
# Location: OMV800
# Priority: HIGH

# Deploy Prometheus stack
docker stack deploy -c /opt/monitoring/prometheus.yml monitoring

# Validate Prometheus
curl http://192.168.50.225:9090/api/v1/status/targets

1.3.2 Grafana Setup

# Day 6-7: Visualization setup
# Location: OMV800
# Priority: MEDIUM

# Deploy Grafana
docker stack deploy -c /opt/monitoring/grafana.yml grafana

# Import dashboards
# Reference: migration_scripts/configs/grafana/dashboards/

# Validate Grafana
curl http://192.168.50.225:3000/api/health

🗄️ STAGE 2: DATABASE MIGRATION (WEEK 2)

2.1 POSTGRESQL CLUSTER SETUP

2.1.1 Primary Database Setup

# Day 8-9: Primary database
# Location: OMV800
# Priority: CRITICAL

# Deploy PostgreSQL primary
docker stack deploy -c /opt/databases/postgresql-primary.yml postgresql

# Validate primary
docker service ls | grep postgresql
docker exec $(docker ps -q -f name=postgresql_primary) psql -U postgres -c "SELECT version();"

2.1.2 Database Migration (PostgreSQL)

# Option B: Logical replication for near-zero downtime
# 1) On source DB create publication
psql -U postgres -c "CREATE PUBLICATION mig_pub FOR ALL TABLES;"

# 2) On target create subscription
psql -U postgres -c "CREATE SUBSCRIPTION mig_sub CONNECTION 'host=<SRC> dbname=<DB> user=replicator password=<PW>' PUBLICATION mig_pub;"

# 3) Monitor replication lag, then cutover by disabling writers, drop subscription/publication post-switch

2.2 MARIADB CLUSTER SETUP

2.2.1 MariaDB Primary Setup

# Day 10-11: MariaDB setup (Primary)
# Use MariaDB 10.11 to match surface/fedora versions (avoid major upgrade at cutover)

docker stack deploy -c /opt/databases/mariadb-primary.yml mariadb

# Validate
docker service ls | grep mariadb

2.2.2 MariaDB Data Migration (Option A: Short Downtime)

# Stop writes -> dump -> restore (fastest, brief downtime)
# On source host
mysqldump --all-databases --routines --triggers --events --single-transaction \
  > /backup/mariadb_full_$(date +%Y%m%d_%H%M%S).sql

# On target
docker cp /backup/mariadb_full_*.sql $(docker ps -q -f name=mariadb_primary):/backup/
docker exec mariadb_primary sh -c "mysql -u root -p<ROOTPW> < /backup/mariadb_full_*.sql"

2.2.3 MariaDB Replication (Option B: Near Zero Downtime)

# 1) On source (old MariaDB) - create replica user and get binlog pos
mysql -u root -p -e "CREATE USER 'repl'@'%' IDENTIFIED BY '<REPLPW>'; GRANT REPLICATION SLAVE ON *.* TO 'repl'@'%'; FLUSH TABLES WITH READ LOCK; SHOW MASTER STATUS;" > /backup/master_status.txt

# 2) Full dump with master data
mysqldump --all-databases --routines --triggers --events --single-transaction --master-data=2 \
  > /backup/mariadb_seed.sql

# 3) Unlock tables on source
mysql -u root -p -e "UNLOCK TABLES;"

# 4) Seed target and start replica
docker cp /backup/mariadb_seed.sql $(docker ps -q -f name=mariadb_primary):/backup/
docker exec -i mariadb_primary mysql -u root -p<ROOTPW> < /backup/mariadb_seed.sql

# 5) Configure replica (CHANGE MASTER ... then START SLAVE)
# Use values from /backup/master_status.txt
CHANGE MASTER TO MASTER_HOST='<SRC_IP>', MASTER_USER='repl', MASTER_PASSWORD='<REPLPW>', MASTER_LOG_FILE='<FILE>', MASTER_LOG_POS=<POS>;
START SLAVE; SHOW SLAVE STATUS\G

# 6) Cutover
# Set source to read-only, wait for Seconds_Behind_Master=0, switch DNS, promote target, stop source.

2.3 REDIS CLUSTER SETUP

2.3.1 Redis Deployment

# Day 12-13: Redis setup
# Location: OMV800
# Priority: HIGH

# Deploy Redis cluster
docker stack deploy -c /opt/databases/redis-cluster.yml redis

# Validate Redis
docker service ls | grep redis
docker exec $(docker ps -q -f name=redis_master) redis-cli ping

🌐 STAGE 3: WEB SERVICES MIGRATION (WEEK 3)

3.1 CRITICAL WEB SERVICES

3.1.1 Home Assistant Migration

# Day 15-16: Home automation core
# Location: jonathan-2518f5u → OMV800
# Priority: CRITICAL
# Reference: comprehensive_discovery_results/container_audit_results/individual_configs/jonathan-2518f5u_20250824_homeassistant_config.yaml

# Step 1: Deploy new Home Assistant
docker stack deploy -c /opt/services/homeassistant.yml homeassistant

# Step 2: Migrate configuration
# Copy configuration from: /opt/homeassistant/config/
# To: /opt/services/homeassistant/config/

# Step 3: Update Traefik labels
# Add to docker-compose.yml:
# - "traefik.enable=true"
# - "traefik.http.routers.homeassistant.rule=Host(`ha.localhost`)"

# Step 4: Test new deployment
curl http://192.168.50.225:8123/api/
curl https://ha.localhost/api/

# Step 5: Update DNS/load balancer
# Point ha.localhost to new service

# Step 6: Stop old container
docker stop homeassistant

3.1.2 Immich Photo Management Migration

# Day 16-17: Photo management
# Location: OMV800 (same host, new architecture)
# Priority: CRITICAL
# Reference: comprehensive_discovery_results/container_audit_results/individual_configs/omv800.local_20250824_immich_server_config.yaml

# Step 1: Deploy new Immich stack
docker stack deploy -c /opt/services/immich.yml immich

# Step 2: Migrate data
# Copy from: /opt/immich/data/
# To: /opt/services/immich/data/

# Step 3: Update database connections
# Update environment variables to point to new PostgreSQL cluster

# Step 4: Test new deployment
curl http://192.168.50.225:3000/api/v1/
curl https://immich.localhost/api/v1/

# Step 5: Stop old containers
docker stop immich_server immich_postgres immich_machine_learning immich_redis

3.2 DOCUMENT MANAGEMENT SERVICES

3.2.1 Paperless-NGX Migration

# Day 17-18: Document management
# Location: Multiple hosts → OMV800
# Priority: HIGH
# Reference: comprehensive_discovery_results/container_audit_results/individual_configs/omv800.local_20250824_paperless-webserver-1_config.yaml

# Step 1: Deploy new Paperless stack
docker stack deploy -c /opt/services/paperless.yml paperless

# Step 2: Migrate documents
# Copy from: /opt/paperless/data/
# To: /opt/services/paperless/data/

# Step 3: Update database connections
# Point to new PostgreSQL cluster

# Step 4: Test new deployment
curl http://192.168.50.225:8000/
curl https://paperless.localhost/

# Step 5: Stop old containers
docker stop paperless-webserver-1 paperless-db-1 paperless-broker-1

3.2.2 Joplin Migration

# Day 18-19: Note taking
# Location: OMV800 (same host, new architecture)
# Priority: HIGH
# Reference: comprehensive_discovery_results/container_audit_results/individual_configs/omv800.local_20250824_joplin-app-1_config.yaml

# Step 1: Deploy new Joplin stack
docker stack deploy -c /opt/services/joplin.yml joplin

# Step 2: Migrate data
# Copy from: /opt/joplin/data/
# To: /opt/services/joplin/data/

# Step 3: Update database connections
# Point to new PostgreSQL cluster

# Step 4: Test new deployment
curl http://192.168.50.225:22300/
curl https://joplin.localhost/

# Step 5: Stop old containers
docker stop joplin-app-1 joplin-db-1

3.3 IoT CORE SERVICES

3.3.1 Mosquitto MQTT Migration

# Minimize disruption to HA automations
# Step 1: Deploy new broker (same user/pass/ACL)
docker stack deploy -c /opt/services/mosquitto.yml mosquitto

# Step 2: Bridge old -> new temporarily (on old broker)
# mosquitto.conf add:
# connection new-broker
# address <new_ip>:1883
# topic # both 0

# Step 3: Update HA to dual-publish (optional) then switch clients via DHCP option or env updates
# Step 4: Validate no dropped messages (monitor $SYS stats)
# Step 5: Remove bridge and decommission old

3.3.2 Z-Wave JS UI Migration

# USB device constraints: keep Z-Wave stick attached to the host where Zwave JS runs
# Option A: Keep service on jonathan-2518f5u; connect HA remotely via websocket
# Option B: USB/IP or ser2net to expose /dev/tty* across network (latency risk)

# Preferred: Option A - Remote websocket
# Step 1: Deploy Zwave JS UI on host with USB stick
# Step 2: Point Home Assistant integration to ws://<host>:3000
# Step 3: Validate interview cache, device availability
# Step 4: Schedule brief maintenance window for interview refresh if needed

🎵 STAGE 4: MEDIA SERVICES MIGRATION (WEEK 4)

4.1 MEDIA STREAMING SERVICES

4.1.1 Jellyfin Migration

# Day 22-23: Media streaming
# Location: OMV800 (same host, new architecture)
# Priority: HIGH
# Reference: comprehensive_discovery_results/container_audit_results/individual_configs/omv800.local_20250824_jellyfin_config.yaml

# Step 1: Deploy new Jellyfin stack
docker stack deploy -c /opt/services/jellyfin.yml jellyfin

# Step 2: Migrate media library
# Copy from: /opt/jellyfin/config/
# To: /opt/services/jellyfin/config/

# Step 3: Update GPU passthrough (if applicable)
# Ensure GPU access for transcoding

# Step 4: Test new deployment
curl http://192.168.50.225:8096/
curl https://jellyfin.localhost/

# Step 5: Stop old container
docker stop jellyfin

4.2 CLOUD STORAGE SERVICES

4.2.1 Nextcloud Migration (Hardened)

# Pre: put site into maintenance and freeze cron jobs
docker exec -u www-data nextcloud php occ maintenance:mode --on

# Migrate, then
docker exec -u www-data nextcloud php occ maintenance:repair
docker exec -u www-data nextcloud php occ db:add-missing-indices
docker exec -u www-data nextcloud php occ db:convert-filecache-bigint

# Re-enable
docker exec -u www-data nextcloud php occ maintenance:mode --off

1.2.3 Traefik v3 Hardening

# Dashboard off or protected
- "--api.dashboard=false"  # or auth middleware on private network only
- "--serversTransport.insecureSkipVerify=false"
- "--entrypoints.web.http.redirections.entryPoint.to=websecure"
- "--entrypoints.web.http.redirections.entryPoint.scheme=https"
# ACME: prefer DNS challenge for public FQDNs; internal CA for lab

1.1.3 Swarm Volumes on NFS

# Define global NFS volumes for stateful stacks
# Example in stack files:
volumes:
  nextcloud_data:
    driver: local
    driver_opts:
      type: nfs
      o: addr=omv800.local,nolock,soft,rw
      device: :/export/nextcloud

GPU Acceleration (Jellyfin/Immich ML)

# Verify drivers and runtime on target nodes
nvidia-smi || true
lsmod | grep i915 || true

# Stack additions
deploy:
  resources:
    reservations:
      devices:
      - capabilities: [gpu]
        device_ids: ["0"]

Watchtower Policy

# Disable unsupervised autoupdates in Swarm env
# Replace moving tags with pinned versions (e.g., :23.0.4)
# Run controlled upgrades via CI or maintenance window

DNS TTL & AdGuard Staged Rollout

# Reduce TTL to 60s 48h before cutover
# Stage clients in batches; keep secondary resolver pointing to old path
# Maintain old AdGuard as secondary for 48h post-cutover

🔧 STAGE 5: DEVELOPMENT SERVICES MIGRATION (WEEK 5)

5.1 DEVELOPMENT PLATFORMS

5.1.1 AppFlowy Cloud Migration

# Day 29-30: Development platform
# Location: surface → OMV800
# Priority: HIGH
# Reference: comprehensive_discovery_results/container_audit_results/individual_configs/surface_20250824_appflowy-cloud-appflowy-1_config.yaml

# Step 1: Deploy new AppFlowy stack
docker stack deploy -c /opt/services/appflowy.yml appflowy

# Step 2: Migrate data
# Copy from: /opt/appflowy/data/
# To: /opt/services/appflowy/data/

# Step 3: Update database connections
# Point to new PostgreSQL cluster

# Step 4: Test new deployment
curl http://192.168.50.225:8000/
curl https://appflowy.localhost/

# Step 5: Stop old containers
docker stop appflowy-cloud-appflowy-1 appflowy-cloud-postgres-1 appflowy-cloud-redis-1 appflowy-cloud-minio-1 appflowy-cloud-gotrue-1

5.1.2 Gitea Migration

# Day 30-31: Code repository
# Location: OMV800 (same host, new architecture)
# Priority: HIGH
# Reference: comprehensive_discovery_results/container_audit_results/individual_configs/omv800.local_20250824_gitea_config.yaml

# Step 1: Deploy new Gitea stack
docker stack deploy -c /opt/services/gitea.yml gitea

# Step 2: Migrate repositories
# Copy from: /opt/gitea/data/
# To: /opt/services/gitea/data/

# Step 3: Update database connections
# Point to new MariaDB cluster

# Step 4: Test new deployment
curl http://192.168.50.225:3001/
curl https://gitea.localhost/

# Step 5: Stop old container
docker stop gitea

🔐 STAGE 6: SECURITY SERVICES MIGRATION (WEEK 6)

6.1 SECURITY SERVICES

6.1.1 Vaultwarden Migration

# Day 36-37: Password manager
# Location: jonathan-2518f5u → OMV800
# Priority: CRITICAL
# Reference: comprehensive_discovery_results/container_audit_results/individual_configs/jonathan-2518f5u_20250824_vaultwarden_config.yaml

# Step 1: Deploy new Vaultwarden stack
docker stack deploy -c /opt/services/vaultwarden.yml vaultwarden

# Step 2: Migrate data
# Copy from: /opt/vaultwarden/data/
# To: /opt/services/vaultwarden/data/

# Step 3: Update Traefik labels
# Add security headers and authentication

# Step 4: Test new deployment
curl http://192.168.50.225:3012/
curl https://vaultwarden.localhost/

# Step 5: Stop old container
docker stop vaultwarden

6.1.2 AdGuard Home Migration

# Day 37-38: DNS filtering
# Location: OMV800 (same host, new architecture)
# Priority: CRITICAL
# Reference: comprehensive_discovery_results/container_audit_results/individual_configs/omv800.local_20250824_adguardhome_config.yaml

# Step 1: Deploy new AdGuard Home stack
docker stack deploy -c /opt/services/adguard.yml adguard

# Step 2: Migrate configuration
# Copy from: /opt/adguard/conf/
# To: /opt/services/adguard/conf/

# Step 3: Update DNS settings
# Point all devices to new AdGuard Home

# Step 4: Test new deployment
curl http://192.168.50.225:3000/
curl https://adguard.localhost/

# Step 5: Stop old container
docker stop adguardhome

📊 STAGE 7: MONITORING SERVICES MIGRATION (WEEK 7)

7.1 MONITORING INFRASTRUCTURE

7.1.1 Netdata Migration

# Day 43-44: System monitoring
# Location: All hosts → OMV800 (centralized)
# Priority: MEDIUM
# Reference: COMPLETE_DOCKER_SERVICES_INVENTORY.md (Native Services section)

# Step 1: Deploy centralized Netdata
docker stack deploy -c /opt/monitoring/netdata.yml netdata

# Step 2: Configure child nodes
# Update all hosts to stream to central Netdata

# Step 3: Test new deployment
curl http://192.168.50.225:19999/
curl https://netdata.localhost/

# Step 4: Stop old Netdata services
systemctl stop netdata

7.1.2 Loki Log Aggregation

# Day 44-45: Log aggregation
# Location: OMV800
# Priority: MEDIUM

# Step 1: Deploy Loki stack
docker stack deploy -c /opt/monitoring/loki.yml loki

# Step 2: Configure log forwarding
# Update all services to forward logs to Loki

# Step 3: Test new deployment
curl http://192.168.50.225:3100/ready

🔄 STAGE 8: NATIVE SERVICES MIGRATION (WEEK 8)

8.1 WEB SERVER MIGRATION

8.1.1 Caddy Migration

# Day 50-51: Web server
# Location: surface → OMV800
# Priority: MEDIUM
# Reference: COMPLETE_DOCKER_SERVICES_INVENTORY.md (Surface Native Services)

# Step 1: Deploy Caddy in Docker
docker stack deploy -c /opt/services/caddy.yml caddy

# Step 2: Migrate configuration
# Copy from: /etc/caddy/
# To: /opt/services/caddy/config/

# Step 3: Update Traefik labels
# Configure Caddy as backend for specific services

# Step 4: Test new deployment
curl http://192.168.50.225:80/
curl https://caddy.localhost/

# Step 5: Stop old service
systemctl stop caddy

8.1.2 Apache/Nginx Migration

# Day 51-52: Web servers
# Location: Multiple hosts → OMV800
# Priority: MEDIUM
# Reference: COMPLETE_DOCKER_SERVICES_INVENTORY.md (Native Services sections)

# Step 1: Deploy web servers in Docker
docker stack deploy -c /opt/services/webservers.yml webservers

# Step 2: Migrate configurations
# Copy from: /etc/apache2/, /etc/nginx/
# To: /opt/services/webservers/config/

# Step 3: Update Traefik labels
# Configure as backends for specific services

# Step 4: Test new deployment
curl http://192.168.50.225:80/
curl https://webservers.localhost/

# Step 5: Stop old services
systemctl stop apache2 nginx

8.2 AI/ML SERVICES MIGRATION

8.2.1 Ollama Migration

# Day 52-53: AI/ML service
# Location: surface → OMV800
# Priority: HIGH
# Reference: COMPLETE_DOCKER_SERVICES_INVENTORY.md (AI & Machine Learning Services)

# Step 1: Deploy Ollama in Docker
docker stack deploy -c /opt/services/ollama.yml ollama

# Step 2: Migrate models
# Copy from: /home/jon/.ollama/
# To: /opt/services/ollama/models/

# Step 3: Update Paperless-AI configuration
# Point to new Ollama service

# Step 4: Test new deployment
curl http://192.168.50.225:11434/api/tags
curl https://ollama.localhost/api/tags

# Step 5: Stop old service
pkill ollama

🧪 STAGE 9: COMPREHENSIVE TESTING (WEEK 9)

9.1 FUNCTIONAL TESTING

9.1.1 Service Health Validation

# Day 57-58: Health checks
# Location: All services
# Priority: CRITICAL

# Test all migrated services
for service in homeassistant immich paperless joplin jellyfin nextcloud appflowy gitea vaultwarden adguard; do
    echo "Testing $service..."
    curl -f https://$service.localhost/ || echo "$service: FAILED"
done

# Test database connections
docker exec postgresql_primary psql -U postgres -c "SELECT datname FROM pg_database;"
docker exec mariadb_primary mysql -u root -p -e "SHOW DATABASES;"

9.1.2 Performance Testing

# Day 58-59: Performance validation
# Location: All services
# Priority: HIGH

# Load testing
ab -n 1000 -c 10 https://nextcloud.localhost/
ab -n 1000 -c 10 https://jellyfin.localhost/

# Database performance
docker exec postgresql_primary psql -U postgres -c "SELECT * FROM pg_stat_database;"
docker exec mariadb_primary mysql -u root -p -e "SHOW STATUS LIKE 'Questions';"

9.2 SECURITY TESTING

9.2.1 Security Validation

# Day 59-60: Security checks
# Location: All services
# Priority: CRITICAL

# SSL certificate validation
for service in homeassistant immich paperless joplin jellyfin nextcloud; do
    echo "Testing SSL for $service..."
    openssl s_client -connect $service.localhost:443 -servername $service.localhost < /dev/null
done

# Security headers validation
for service in vaultwarden adguard; do
    echo "Testing security headers for $service..."
    curl -I https://$service.localhost/ | grep -E "(Strict-Transport-Security|X-Frame-Options|X-Content-Type-Options)"
done

🎯 STAGE 10: FINAL VALIDATION AND CLEANUP (WEEK 10)

10.1 FINAL VALIDATION

10.1.1 Complete System Validation

# Day 64-65: Final validation
# Location: All services
# Priority: CRITICAL

# Comprehensive health check
./migration_scripts/validate_migration.sh

# Performance baseline
./migration_scripts/performance_baseline.sh

# Security audit
./migration_scripts/security_audit.sh

10.1.2 User Acceptance Testing

# Day 65-66: User testing
# Location: All services
# Priority: CRITICAL

# Test all user workflows
# - Home Assistant automation
# - Immich photo upload/processing
# - Paperless document scanning
# - Jellyfin media streaming
# - Nextcloud file sync
# - AppFlowy collaboration
# - Vaultwarden password management

10.2 CLEANUP AND OPTIMIZATION

10.2.1 Old Infrastructure Cleanup

# Day 66-67: Cleanup
# Location: All hosts
# Priority: MEDIUM

# Remove old containers
docker container prune -f

# Remove old volumes
docker volume prune -f

# Remove old networks
docker network prune -f

# Stop old native services
systemctl disable netdata
systemctl disable apache2
systemctl disable nginx
systemctl disable caddy

10.2.2 Performance Optimization

# Day 67-70: Optimization
# Location: All services
# Priority: MEDIUM

# Database optimization
docker exec postgresql_primary psql -U postgres -c "VACUUM ANALYZE;"
docker exec mariadb_primary mysql -u root -p -e "OPTIMIZE TABLE *;"

# Cache optimization
docker exec redis_master redis-cli FLUSHALL

# Monitoring optimization
# Configure alerting rules
# Set up performance dashboards

📋 CRITICAL REFERENCES AND LOCATIONS

Configuration File Locations

# Docker Compose files
/opt/services/                    # New service configurations
/opt/databases/                   # Database configurations
/opt/monitoring/                  # Monitoring configurations
/opt/traefik/                     # Reverse proxy configurations

# Backup locations
/backup/                          # All backups
/backup/container_configs/        # Container configurations
/backup/compose_files/            # Docker Compose files
/backup/volumes/                  # Volume data
/backup/systemd_configs/          # Native service configurations

# Discovery results
comprehensive_discovery_results/  # All discovery data
comprehensive_discovery_results/container_audit_results/individual_configs/  # Container configs
comprehensive_discovery_results/detailed_container_inventory.yaml  # Container inventory

Critical Service Dependencies

# Database Dependencies
PostgreSQL Cluster:
  - Paperless-NGX
  - Joplin
  - Immich
  - AppFlowy

MariaDB Cluster:
  - Nextcloud
  - Gitea
  - Home Assistant

Redis Cluster:
  - Nextcloud
  - Paperless-NGX
  - AppFlowy
  - Immich

# Network Dependencies
Traefik:
  - All web services
  - SSL certificates
  - Load balancing

# Storage Dependencies
NFS/Samba:
  - Media files
  - Document storage
  - Photo libraries

Rollback Procedures

# Database rollback
docker exec postgresql_primary psql -U postgres < /backup/postgresql_full_$(date).sql
docker exec mariadb_primary mysql -u root -p < /backup/mariadb_full_$(date).sql

# Service rollback
docker stack rm <service_name>
docker run -d --name <old_container> <old_image>

# Configuration rollback
cp /backup/systemd_configs/* /etc/systemd/system/
systemctl daemon-reload
systemctl restart <service_name>

🚨 CRITICAL CONSIDERATIONS

Data Integrity

Always backup before any migration step
Validate data integrity after each migration
Test rollback procedures before migration
Monitor disk space during migration

Network Considerations

Update DNS records gradually
Test network connectivity at each step
Monitor bandwidth usage during migration
Plan for network failures

Security Considerations

Maintain security during migration
Update firewall rules gradually
Test SSL certificates after migration
Validate authentication systems

Performance Considerations

Monitor performance during migration
Plan for increased resource usage
Test under load conditions
Optimize after migration

✅ SUCCESS CRITERIA

Technical Success

✅ All services migrated successfully
✅ Zero data loss
✅ Zero downtime for critical services
✅ Performance maintained or improved
✅ Security maintained or improved

Operational Success

✅ All users can access services
✅ All automations working
✅ All monitoring functional
✅ All backups working
✅ All alerts configured

Business Success

✅ Improved reliability
✅ Improved performance
✅ Improved security
✅ Improved maintainability
✅ Future-proof architecture

This migration plan provides a comprehensive, granular approach to transforming your infrastructure while maintaining 100% availability and data integrity. Each step includes detailed procedures, validation checks, and rollback capabilities.

34 KiB Raw Blame History Unescape Escape