# WORLD-CLASS MIGRATION TO-DO LIST
**Extremely Detailed, Granular Migration Plan**  
**Zero-Downtime Infrastructure Transformation**  
**Generated:** 2025-08-24

---

## 🎯 MIGRATION OVERVIEW

This document provides an **extremely detailed, granular migration plan** to transform your current infrastructure into the Future-Proof Scalability architecture. Every step includes comprehensive testing, validation, and rollback procedures to ensure **zero data loss** and **zero downtime**.

### **Migration Philosophy**
- **Parallel Deployment**: New infrastructure runs alongside old
- **Gradual Cutover**: Service-by-service migration with validation
- **Complete Redundancy**: Every component has backup and failover
- **Automated Validation**: Health checks and performance monitoring
- **Instant Rollback**: Ability to revert any change within minutes

### **Success Criteria**
- ✅ **Zero data loss** during migration
- ✅ **Zero downtime** for critical services
- ✅ **100% service availability** throughout migration
- ✅ **Performance improvement** validated at each step
- ✅ **Complete rollback capability** at any point

---

## 🛠️ CORRECTIONS, GAPS, AND OPTIMIZATIONS APPLIED

- **MariaDB backup mismatch fixed**: Replaced tar-based volume backup/SQL restore inconsistency with consistent `mysqldump` exports and `mysql` imports.
- **Add secrets/env inventory step**: Centralize all credentials and env files from discovery outputs before any migration.
- **Compose/Stack generation clarified**: Generate Swarm stacks from discovered compose templates; validate images, tags, networks, volumes.
- **Blue/Green cutover policy**: Never stop old services until new path is serving live traffic and validated; keep old online for 48 hours.
- **Zero-downtime DB replication**: Added optional PostgreSQL logical replication and MariaDB replication steps to reduce downtime to seconds.
- **Nextcloud migration hardening**: Use `occ` maintenance mode, integrity checks, and post-migration repairs.
- **Home Assistant migration**: Use full snapshot/restore; respect recorder DB migration; schedule maintenance window.
- **MQTT and Z-Wave not yet migrated**: Added explicit sections for Mosquitto and Z-Wave JS UI (USB constraints/remote strategy).
- **Traefik v3 hardening**: Require authenticated dashboard; ACME via DNS challenge for real domain or use internal CA; remove insecure API.
- **NFS-backed persistent volumes**: Define Swarm volumes backed by NFS exports for stateful services.
- **GPU acceleration**: Add driver/runtime checks and device mapping for Jellyfin/Immich ML.
- **Watchtower policy**: Disable automatic updates in Swarm; use pinned tags and rolling updates.
- **Secrets management**: Move passwords/API keys to Docker Secrets/Ansible Vault; remove `-p` inline usage.
- **DNS TTL management**: Lower TTL 24–48h before cutover to speed failback.
- **DHCP/DNS transition for AdGuard**: Staged rollout and fallback DNS retained for 48h.

## 📋 PRE-MIGRATION PREPARATION (STAGE 0)

### **0.0 SECRETS AND ENV INVENTORY (PREREQUISITE)**
```bash
# Purpose: Centralize all credentials, tokens, and env files BEFORE backups/migration
# Sources: comprehensive_discovery_results/container_audit_results/individual_configs/
#          comprehensive_discovery_results/detailed_container_inventory.yaml
# Output:  /backup/secrets_inventory/ and docker secrets compatible files

mkdir -p /backup/secrets_inventory/env /backup/secrets_inventory/files

# 1) Collect env from running containers (sanitized copy)
for c in $(docker ps -q); do
  name=$(docker inspect --format '{{.Name}}' $c | sed 's#^/##')
  docker inspect $c > /backup/secrets_inventory/${name}_inspect.json
  docker exec $c env | sed 's/\(PASSWORD\|SECRET\|KEY\|TOKEN\)=.*/\1=REDACTED/g' \
    > /backup/secrets_inventory/env/${name}.env.sanitized
Done

# 2) Parse compose templates for env_file/includes
# Reference: comprehensive_discovery_results/container_audit_results/compose_templates/
find comprehensive_discovery_results/container_audit_results/compose_templates -type f -name "*_compose.yml" \
  -exec grep -Hn "env_file\|environment\|secrets" {} \; > /backup/secrets_inventory/compose_env_index.txt

# 3) Export existing secret files (if paths are in binds)
# Example sensitive paths often mounted
grep -R "bind\|target" comprehensive_discovery_results/detailed_container_inventory.yaml \
  | grep -E "(\.env|/secrets/|/config/)" >> /backup/secrets_inventory/bind_mount_candidates.txt
```

### **0.1 COMPREHENSIVE BACKUP STRATEGY**

#### **0.1.1 Database Backups**
```bash
# Location: All database containers and native database services
# Priority: CRITICAL - Must complete before any migration

# PostgreSQL Backups
docker exec paperless-db-1 pg_dumpall > /backup/postgresql_full_$(date +%Y%m%d_%H%M%S).sql
docker exec joplin-db-1 pg_dumpall > /backup/joplin_db_$(date +%Y%m%d_%H%M%S).sql
docker exec immich_postgres pg_dumpall > /backup/immich_db_$(date +%Y%m%d_%H%M%S).sql

# MariaDB Backups
docker exec mariadb mysqldump --all-databases > /backup/mariadb_full_$(date +%Y%m%d_%H%M%S).sql
docker exec nextcloud-db mysqldump --all-databases > /backup/nextcloud_db_$(date +%Y%m%d_%H%M%S).sql

# Native Database Backups
sudo mysqldump --all-databases > /backup/native_mariadb_$(date +%Y%m%d_%H%M%S).sql
sudo -u postgres pg_dumpall > /backup/native_postgresql_$(date +%Y%m%d_%H%M%S).sql
```

#### **0.1.2 Container Configuration Backups**
```bash
# Location: All Docker containers
# Priority: CRITICAL - Configuration preservation

# Export all container configurations
for container in $(docker ps -aq); do
    docker inspect $container > /backup/container_configs/${container}_config.json
    docker exec $container env > /backup/container_configs/${container}_env.txt
done

# Export Docker Compose files
find /opt -name "docker-compose.yml" -exec cp {} /backup/compose_files/ \;
find /opt -name "docker-compose.yaml" -exec cp {} /backup/compose_files/ \;
```

#### **0.1.3 Volume Data Backups**
```bash
# Location: All Docker volumes and bind mounts
# Priority: CRITICAL - Data preservation

# Backup all Docker volumes
docker run --rm -v /var/lib/docker/volumes:/volumes -v /backup/volumes:/backup alpine tar czf /backup/docker_volumes_$(date +%Y%m%d_%H%M%S).tar.gz /volumes

# Backup critical bind mounts
tar czf /backup/immich_data_$(date +%Y%m%d_%H%M%S).tar.gz /opt/immich/data
tar czf /backup/nextcloud_data_$(date +%Y%m%d_%H%M%S).tar.gz /opt/nextcloud/data
tar czf /backup/homeassistant_data_$(date +%Y%m%d_%H%M%S).tar.gz /opt/homeassistant/config
```

#### **0.1.4 Native Service Configuration Backups**
```bash
# Location: All native systemd services
# Priority: HIGH - Configuration preservation

# Backup systemd service configurations
sudo systemctl list-unit-files --type=service --state=enabled > /backup/systemd_enabled_services.txt
sudo systemctl list-units --type=service --state=running > /backup/systemd_running_services.txt

# Backup service configurations
sudo cp -r /etc/systemd/system /backup/systemd_configs/
sudo cp -r /etc/nginx /backup/nginx_configs/
sudo cp -r /etc/apache2 /backup/apache2_configs/
sudo cp -r /etc/caddy /backup/caddy_configs/
sudo cp -r /etc/mariadb /backup/mariadb_configs/
sudo cp -r /etc/postgresql /backup/postgresql_configs/
sudo cp -r /etc/fail2ban /backup/fail2ban_configs/
sudo cp -r /etc/netdata /backup/netdata_configs/
```

### **0.2 INFRASTRUCTURE VALIDATION**

#### **0.2.1 Network Connectivity Testing**
```bash
# Test all network connections between hosts
# Location: All hosts
# Priority: CRITICAL - Network validation

# Test inter-host connectivity
for host in omv800.local jonathan-2518f5u fedora surface lenovo420 audrey raspberrypi; do
    ping -c 3 $host
    ssh $host "echo 'SSH connectivity test successful'"
done

# Test critical service ports
for service in "OMV800:8080" "jonathan-2518f5u:8123" "surface:8000"; do
    host=$(echo $service | cut -d: -f1)
    port=$(echo $service | cut -d: -f2)
    nc -zv $host $port
done
```

#### **0.2.2 Service Health Validation**
```bash
# Validate all services are healthy before migration
# Location: All hosts
# Priority: CRITICAL - Service validation

# Docker container health checks
docker ps --format "table {{.Names}}\t{{.Status}}\t{{.Ports}}"

# Native service health checks
systemctl is-active --quiet mariadb && echo "MariaDB: ACTIVE" || echo "MariaDB: INACTIVE"
systemctl is-active --quiet postgresql && echo "PostgreSQL: ACTIVE" || echo "PostgreSQL: INACTIVE"
systemctl is-active --quiet nginx && echo "Nginx: ACTIVE" || echo "Nginx: INACTIVE"
systemctl is-active --quiet apache2 && echo "Apache2: ACTIVE" || echo "Apache2: INACTIVE"
```

### **0.3 MIGRATION ENVIRONMENT SETUP**

#### **0.3.1 Docker Swarm Initialization**
```bash
# Location: All hosts
# Priority: CRITICAL - New infrastructure foundation

# Initialize Docker Swarm on manager node (OMV800)
docker swarm init --advertise-addr 192.168.50.225

# Join worker nodes
# Execute on each worker node:
docker swarm join --token <MANAGER_TOKEN> 192.168.50.225:2377
```

#### **0.3.2 Traefik Setup**
```bash
# Location: OMV800 (manager node)
# Priority: CRITICAL - Reverse proxy foundation

# Deploy Traefik stack
docker stack deploy -c /opt/traefik/docker-compose.yml traefik

# Validate Traefik deployment
docker service ls | grep traefik
curl -I http://192.168.50.225:8080/api/rawdata
```

---

## 🏗️ STAGE 1: INFRASTRUCTURE FOUNDATION (WEEK 1)

### **1.1 DOCKER SWARM CLUSTER SETUP**

#### **1.1.1 Cluster Initialization**
```bash
# Day 1-2: Foundation setup
# Location: All hosts
# Priority: CRITICAL

# Step 1: Initialize Swarm on OMV800
ssh omv800.local "docker swarm init --advertise-addr 192.168.50.225"

# Step 2: Join worker nodes
ssh jonathan-2518f5u "docker swarm join --token <TOKEN> 192.168.50.225:2377"
ssh fedora "docker swarm join --token <TOKEN> 192.168.50.225:2377"
ssh surface "docker swarm join --token <TOKEN> 192.168.50.225:2377"

# Step 3: Validate cluster
docker node ls
docker service ls
```

#### **1.1.2 Network Configuration**
```bash
# Day 2-3: Network setup
# Location: OMV800 (manager)
# Priority: CRITICAL

# Create overlay networks
docker network create --driver overlay --attachable traefik-public
docker network create --driver overlay --attachable database-network
docker network create --driver overlay --attachable storage-network
docker network create --driver overlay --attachable monitoring-network

# Validate networks
docker network ls | grep overlay
```

### **1.2 TRAEFIK REVERSE PROXY DEPLOYMENT**

#### **1.2.1 Traefik Stack Deployment**
```bash
# Day 3-4: Reverse proxy setup
# Location: OMV800
# Priority: CRITICAL

# Deploy Traefik stack
docker stack deploy -c /opt/traefik/docker-compose.yml traefik

# Validate deployment
docker service ls | grep traefik
curl -I http://192.168.50.225:8080/api/rawdata

# Test SSL certificate generation
curl -I https://traefik.localhost
```

#### **1.2.2 Middleware Configuration**
```bash
# Day 4-5: Security and routing setup
# Location: OMV800
# Priority: HIGH

# Configure security middlewares
# File: /opt/traefik/dynamic/middleware.yml
# Reference: migration_scripts/configs/traefik/dynamic/middleware.yml

# Test middleware functionality
curl -H "X-Forwarded-For: 192.168.1.100" http://192.168.50.225:8080/api/rawdata
```

### **1.3 MONITORING INFRASTRUCTURE**

#### **1.3.1 Prometheus Deployment**
```bash
# Day 5-6: Monitoring setup
# Location: OMV800
# Priority: HIGH

# Deploy Prometheus stack
docker stack deploy -c /opt/monitoring/prometheus.yml monitoring

# Validate Prometheus
curl http://192.168.50.225:9090/api/v1/status/targets
```

#### **1.3.2 Grafana Setup**
```bash
# Day 6-7: Visualization setup
# Location: OMV800
# Priority: MEDIUM

# Deploy Grafana
docker stack deploy -c /opt/monitoring/grafana.yml grafana

# Import dashboards
# Reference: migration_scripts/configs/grafana/dashboards/

# Validate Grafana
curl http://192.168.50.225:3000/api/health
```

---

## 🗄️ STAGE 2: DATABASE MIGRATION (WEEK 2)

### **2.1 POSTGRESQL CLUSTER SETUP**

#### **2.1.1 Primary Database Setup**
```bash
# Day 8-9: Primary database
# Location: OMV800
# Priority: CRITICAL

# Deploy PostgreSQL primary
docker stack deploy -c /opt/databases/postgresql-primary.yml postgresql

# Validate primary
docker service ls | grep postgresql
docker exec $(docker ps -q -f name=postgresql_primary) psql -U postgres -c "SELECT version();"
```

#### **2.1.2 Database Migration (PostgreSQL)**
```bash
# Option B: Logical replication for near-zero downtime
# 1) On source DB create publication
psql -U postgres -c "CREATE PUBLICATION mig_pub FOR ALL TABLES;"

# 2) On target create subscription
psql -U postgres -c "CREATE SUBSCRIPTION mig_sub CONNECTION 'host=<SRC> dbname=<DB> user=replicator password=<PW>' PUBLICATION mig_pub;"

# 3) Monitor replication lag, then cutover by disabling writers, drop subscription/publication post-switch
```

### **2.2 MARIADB CLUSTER SETUP**

#### **2.2.1 MariaDB Primary Setup**
```bash
# Day 10-11: MariaDB setup (Primary)
# Use MariaDB 10.11 to match surface/fedora versions (avoid major upgrade at cutover)

docker stack deploy -c /opt/databases/mariadb-primary.yml mariadb

# Validate
docker service ls | grep mariadb
```

#### **2.2.2 MariaDB Data Migration (Option A: Short Downtime)**
```bash
# Stop writes -> dump -> restore (fastest, brief downtime)
# On source host
mysqldump --all-databases --routines --triggers --events --single-transaction \
  > /backup/mariadb_full_$(date +%Y%m%d_%H%M%S).sql

# On target
docker cp /backup/mariadb_full_*.sql $(docker ps -q -f name=mariadb_primary):/backup/
docker exec mariadb_primary sh -c "mysql -u root -p<ROOTPW> < /backup/mariadb_full_*.sql"
```

#### **2.2.3 MariaDB Replication (Option B: Near Zero Downtime)**
```bash
# 1) On source (old MariaDB) - create replica user and get binlog pos
mysql -u root -p -e "CREATE USER 'repl'@'%' IDENTIFIED BY '<REPLPW>'; GRANT REPLICATION SLAVE ON *.* TO 'repl'@'%'; FLUSH TABLES WITH READ LOCK; SHOW MASTER STATUS;" > /backup/master_status.txt

# 2) Full dump with master data
mysqldump --all-databases --routines --triggers --events --single-transaction --master-data=2 \
  > /backup/mariadb_seed.sql

# 3) Unlock tables on source
mysql -u root -p -e "UNLOCK TABLES;"

# 4) Seed target and start replica
docker cp /backup/mariadb_seed.sql $(docker ps -q -f name=mariadb_primary):/backup/
docker exec -i mariadb_primary mysql -u root -p<ROOTPW> < /backup/mariadb_seed.sql

# 5) Configure replica (CHANGE MASTER ... then START SLAVE)
# Use values from /backup/master_status.txt
CHANGE MASTER TO MASTER_HOST='<SRC_IP>', MASTER_USER='repl', MASTER_PASSWORD='<REPLPW>', MASTER_LOG_FILE='<FILE>', MASTER_LOG_POS=<POS>;
START SLAVE; SHOW SLAVE STATUS\G

# 6) Cutover
# Set source to read-only, wait for Seconds_Behind_Master=0, switch DNS, promote target, stop source.
```

### **2.3 REDIS CLUSTER SETUP**

#### **2.3.1 Redis Deployment**
```bash
# Day 12-13: Redis setup
# Location: OMV800
# Priority: HIGH

# Deploy Redis cluster
docker stack deploy -c /opt/databases/redis-cluster.yml redis

# Validate Redis
docker service ls | grep redis
docker exec $(docker ps -q -f name=redis_master) redis-cli ping
```

---

## 🌐 STAGE 3: WEB SERVICES MIGRATION (WEEK 3)

### **3.1 CRITICAL WEB SERVICES**

#### **3.1.1 Home Assistant Migration**
```bash
# Day 15-16: Home automation core
# Location: jonathan-2518f5u → OMV800
# Priority: CRITICAL
# Reference: comprehensive_discovery_results/container_audit_results/individual_configs/jonathan-2518f5u_20250824_homeassistant_config.yaml

# Step 1: Deploy new Home Assistant
docker stack deploy -c /opt/services/homeassistant.yml homeassistant

# Step 2: Migrate configuration
# Copy configuration from: /opt/homeassistant/config/
# To: /opt/services/homeassistant/config/

# Step 3: Update Traefik labels
# Add to docker-compose.yml:
# - "traefik.enable=true"
# - "traefik.http.routers.homeassistant.rule=Host(`ha.localhost`)"

# Step 4: Test new deployment
curl http://192.168.50.225:8123/api/
curl https://ha.localhost/api/

# Step 5: Update DNS/load balancer
# Point ha.localhost to new service

# Step 6: Stop old container
docker stop homeassistant
```

#### **3.1.2 Immich Photo Management Migration**
```bash
# Day 16-17: Photo management
# Location: OMV800 (same host, new architecture)
# Priority: CRITICAL
# Reference: comprehensive_discovery_results/container_audit_results/individual_configs/omv800.local_20250824_immich_server_config.yaml

# Step 1: Deploy new Immich stack
docker stack deploy -c /opt/services/immich.yml immich

# Step 2: Migrate data
# Copy from: /opt/immich/data/
# To: /opt/services/immich/data/

# Step 3: Update database connections
# Update environment variables to point to new PostgreSQL cluster

# Step 4: Test new deployment
curl http://192.168.50.225:3000/api/v1/
curl https://immich.localhost/api/v1/

# Step 5: Stop old containers
docker stop immich_server immich_postgres immich_machine_learning immich_redis
```

### **3.2 DOCUMENT MANAGEMENT SERVICES**

#### **3.2.1 Paperless-NGX Migration**
```bash
# Day 17-18: Document management
# Location: Multiple hosts → OMV800
# Priority: HIGH
# Reference: comprehensive_discovery_results/container_audit_results/individual_configs/omv800.local_20250824_paperless-webserver-1_config.yaml

# Step 1: Deploy new Paperless stack
docker stack deploy -c /opt/services/paperless.yml paperless

# Step 2: Migrate documents
# Copy from: /opt/paperless/data/
# To: /opt/services/paperless/data/

# Step 3: Update database connections
# Point to new PostgreSQL cluster

# Step 4: Test new deployment
curl http://192.168.50.225:8000/
curl https://paperless.localhost/

# Step 5: Stop old containers
docker stop paperless-webserver-1 paperless-db-1 paperless-broker-1
```

#### **3.2.2 Joplin Migration**
```bash
# Day 18-19: Note taking
# Location: OMV800 (same host, new architecture)
# Priority: HIGH
# Reference: comprehensive_discovery_results/container_audit_results/individual_configs/omv800.local_20250824_joplin-app-1_config.yaml

# Step 1: Deploy new Joplin stack
docker stack deploy -c /opt/services/joplin.yml joplin

# Step 2: Migrate data
# Copy from: /opt/joplin/data/
# To: /opt/services/joplin/data/

# Step 3: Update database connections
# Point to new PostgreSQL cluster

# Step 4: Test new deployment
curl http://192.168.50.225:22300/
curl https://joplin.localhost/

# Step 5: Stop old containers
docker stop joplin-app-1 joplin-db-1
```

### **3.3 IoT CORE SERVICES**

#### **3.3.1 Mosquitto MQTT Migration**
```bash
# Minimize disruption to HA automations
# Step 1: Deploy new broker (same user/pass/ACL)
docker stack deploy -c /opt/services/mosquitto.yml mosquitto

# Step 2: Bridge old -> new temporarily (on old broker)
# mosquitto.conf add:
# connection new-broker
# address <new_ip>:1883
# topic # both 0

# Step 3: Update HA to dual-publish (optional) then switch clients via DHCP option or env updates
# Step 4: Validate no dropped messages (monitor $SYS stats)
# Step 5: Remove bridge and decommission old
```

#### **3.3.2 Z-Wave JS UI Migration**
```bash
# USB device constraints: keep Z-Wave stick attached to the host where Zwave JS runs
# Option A: Keep service on jonathan-2518f5u; connect HA remotely via websocket
# Option B: USB/IP or ser2net to expose /dev/tty* across network (latency risk)

# Preferred: Option A - Remote websocket
# Step 1: Deploy Zwave JS UI on host with USB stick
# Step 2: Point Home Assistant integration to ws://<host>:3000
# Step 3: Validate interview cache, device availability
# Step 4: Schedule brief maintenance window for interview refresh if needed
```

---

## 🎵 STAGE 4: MEDIA SERVICES MIGRATION (WEEK 4)

### **4.1 MEDIA STREAMING SERVICES**

#### **4.1.1 Jellyfin Migration**
```bash
# Day 22-23: Media streaming
# Location: OMV800 (same host, new architecture)
# Priority: HIGH
# Reference: comprehensive_discovery_results/container_audit_results/individual_configs/omv800.local_20250824_jellyfin_config.yaml

# Step 1: Deploy new Jellyfin stack
docker stack deploy -c /opt/services/jellyfin.yml jellyfin

# Step 2: Migrate media library
# Copy from: /opt/jellyfin/config/
# To: /opt/services/jellyfin/config/

# Step 3: Update GPU passthrough (if applicable)
# Ensure GPU access for transcoding

# Step 4: Test new deployment
curl http://192.168.50.225:8096/
curl https://jellyfin.localhost/

# Step 5: Stop old container
docker stop jellyfin
```

### **4.2 CLOUD STORAGE SERVICES**

#### **4.2.1 Nextcloud Migration (Hardened)**
```bash
# Pre: put site into maintenance and freeze cron jobs
docker exec -u www-data nextcloud php occ maintenance:mode --on

# Migrate, then
docker exec -u www-data nextcloud php occ maintenance:repair
docker exec -u www-data nextcloud php occ db:add-missing-indices
docker exec -u www-data nextcloud php occ db:convert-filecache-bigint

# Re-enable
docker exec -u www-data nextcloud php occ maintenance:mode --off
```

### **1.2.3 Traefik v3 Hardening**
```bash
# Dashboard off or protected
- "--api.dashboard=false"  # or auth middleware on private network only
- "--serversTransport.insecureSkipVerify=false"
- "--entrypoints.web.http.redirections.entryPoint.to=websecure"
- "--entrypoints.web.http.redirections.entryPoint.scheme=https"
# ACME: prefer DNS challenge for public FQDNs; internal CA for lab
```

### **1.1.3 Swarm Volumes on NFS**
```bash
# Define global NFS volumes for stateful stacks
# Example in stack files:
volumes:
  nextcloud_data:
    driver: local
    driver_opts:
      type: nfs
      o: addr=omv800.local,nolock,soft,rw
      device: :/export/nextcloud
```

### **GPU Acceleration (Jellyfin/Immich ML)**
```bash
# Verify drivers and runtime on target nodes
nvidia-smi || true
lsmod | grep i915 || true

# Stack additions
deploy:
  resources:
    reservations:
      devices:
      - capabilities: [gpu]
        device_ids: ["0"]
```

### **Watchtower Policy**
```bash
# Disable unsupervised autoupdates in Swarm env
# Replace moving tags with pinned versions (e.g., :23.0.4)
# Run controlled upgrades via CI or maintenance window
```

### **DNS TTL & AdGuard Staged Rollout**
```bash
# Reduce TTL to 60s 48h before cutover
# Stage clients in batches; keep secondary resolver pointing to old path
# Maintain old AdGuard as secondary for 48h post-cutover
```

---

## 🔧 STAGE 5: DEVELOPMENT SERVICES MIGRATION (WEEK 5)

### **5.1 DEVELOPMENT PLATFORMS**

#### **5.1.1 AppFlowy Cloud Migration**
```bash
# Day 29-30: Development platform
# Location: surface → OMV800
# Priority: HIGH
# Reference: comprehensive_discovery_results/container_audit_results/individual_configs/surface_20250824_appflowy-cloud-appflowy-1_config.yaml

# Step 1: Deploy new AppFlowy stack
docker stack deploy -c /opt/services/appflowy.yml appflowy

# Step 2: Migrate data
# Copy from: /opt/appflowy/data/
# To: /opt/services/appflowy/data/

# Step 3: Update database connections
# Point to new PostgreSQL cluster

# Step 4: Test new deployment
curl http://192.168.50.225:8000/
curl https://appflowy.localhost/

# Step 5: Stop old containers
docker stop appflowy-cloud-appflowy-1 appflowy-cloud-postgres-1 appflowy-cloud-redis-1 appflowy-cloud-minio-1 appflowy-cloud-gotrue-1
```

#### **5.1.2 Gitea Migration**
```bash
# Day 30-31: Code repository
# Location: OMV800 (same host, new architecture)
# Priority: HIGH
# Reference: comprehensive_discovery_results/container_audit_results/individual_configs/omv800.local_20250824_gitea_config.yaml

# Step 1: Deploy new Gitea stack
docker stack deploy -c /opt/services/gitea.yml gitea

# Step 2: Migrate repositories
# Copy from: /opt/gitea/data/
# To: /opt/services/gitea/data/

# Step 3: Update database connections
# Point to new MariaDB cluster

# Step 4: Test new deployment
curl http://192.168.50.225:3001/
curl https://gitea.localhost/

# Step 5: Stop old container
docker stop gitea
```

---

## 🔐 STAGE 6: SECURITY SERVICES MIGRATION (WEEK 6)

### **6.1 SECURITY SERVICES**

#### **6.1.1 Vaultwarden Migration**
```bash
# Day 36-37: Password manager
# Location: jonathan-2518f5u → OMV800
# Priority: CRITICAL
# Reference: comprehensive_discovery_results/container_audit_results/individual_configs/jonathan-2518f5u_20250824_vaultwarden_config.yaml

# Step 1: Deploy new Vaultwarden stack
docker stack deploy -c /opt/services/vaultwarden.yml vaultwarden

# Step 2: Migrate data
# Copy from: /opt/vaultwarden/data/
# To: /opt/services/vaultwarden/data/

# Step 3: Update Traefik labels
# Add security headers and authentication

# Step 4: Test new deployment
curl http://192.168.50.225:3012/
curl https://vaultwarden.localhost/

# Step 5: Stop old container
docker stop vaultwarden
```

#### **6.1.2 AdGuard Home Migration**
```bash
# Day 37-38: DNS filtering
# Location: OMV800 (same host, new architecture)
# Priority: CRITICAL
# Reference: comprehensive_discovery_results/container_audit_results/individual_configs/omv800.local_20250824_adguardhome_config.yaml

# Step 1: Deploy new AdGuard Home stack
docker stack deploy -c /opt/services/adguard.yml adguard

# Step 2: Migrate configuration
# Copy from: /opt/adguard/conf/
# To: /opt/services/adguard/conf/

# Step 3: Update DNS settings
# Point all devices to new AdGuard Home

# Step 4: Test new deployment
curl http://192.168.50.225:3000/
curl https://adguard.localhost/

# Step 5: Stop old container
docker stop adguardhome
```

---

## 📊 STAGE 7: MONITORING SERVICES MIGRATION (WEEK 7)

### **7.1 MONITORING INFRASTRUCTURE**

#### **7.1.1 Netdata Migration**
```bash
# Day 43-44: System monitoring
# Location: All hosts → OMV800 (centralized)
# Priority: MEDIUM
# Reference: COMPLETE_DOCKER_SERVICES_INVENTORY.md (Native Services section)

# Step 1: Deploy centralized Netdata
docker stack deploy -c /opt/monitoring/netdata.yml netdata

# Step 2: Configure child nodes
# Update all hosts to stream to central Netdata

# Step 3: Test new deployment
curl http://192.168.50.225:19999/
curl https://netdata.localhost/

# Step 4: Stop old Netdata services
systemctl stop netdata
```

#### **7.1.2 Loki Log Aggregation**
```bash
# Day 44-45: Log aggregation
# Location: OMV800
# Priority: MEDIUM

# Step 1: Deploy Loki stack
docker stack deploy -c /opt/monitoring/loki.yml loki

# Step 2: Configure log forwarding
# Update all services to forward logs to Loki

# Step 3: Test new deployment
curl http://192.168.50.225:3100/ready
```

---

## 🔄 STAGE 8: NATIVE SERVICES MIGRATION (WEEK 8)

### **8.1 WEB SERVER MIGRATION**

#### **8.1.1 Caddy Migration**
```bash
# Day 50-51: Web server
# Location: surface → OMV800
# Priority: MEDIUM
# Reference: COMPLETE_DOCKER_SERVICES_INVENTORY.md (Surface Native Services)

# Step 1: Deploy Caddy in Docker
docker stack deploy -c /opt/services/caddy.yml caddy

# Step 2: Migrate configuration
# Copy from: /etc/caddy/
# To: /opt/services/caddy/config/

# Step 3: Update Traefik labels
# Configure Caddy as backend for specific services

# Step 4: Test new deployment
curl http://192.168.50.225:80/
curl https://caddy.localhost/

# Step 5: Stop old service
systemctl stop caddy
```

#### **8.1.2 Apache/Nginx Migration**
```bash
# Day 51-52: Web servers
# Location: Multiple hosts → OMV800
# Priority: MEDIUM
# Reference: COMPLETE_DOCKER_SERVICES_INVENTORY.md (Native Services sections)

# Step 1: Deploy web servers in Docker
docker stack deploy -c /opt/services/webservers.yml webservers

# Step 2: Migrate configurations
# Copy from: /etc/apache2/, /etc/nginx/
# To: /opt/services/webservers/config/

# Step 3: Update Traefik labels
# Configure as backends for specific services

# Step 4: Test new deployment
curl http://192.168.50.225:80/
curl https://webservers.localhost/

# Step 5: Stop old services
systemctl stop apache2 nginx
```

### **8.2 AI/ML SERVICES MIGRATION**

#### **8.2.1 Ollama Migration**
```bash
# Day 52-53: AI/ML service
# Location: surface → OMV800
# Priority: HIGH
# Reference: COMPLETE_DOCKER_SERVICES_INVENTORY.md (AI & Machine Learning Services)

# Step 1: Deploy Ollama in Docker
docker stack deploy -c /opt/services/ollama.yml ollama

# Step 2: Migrate models
# Copy from: /home/jon/.ollama/
# To: /opt/services/ollama/models/

# Step 3: Update Paperless-AI configuration
# Point to new Ollama service

# Step 4: Test new deployment
curl http://192.168.50.225:11434/api/tags
curl https://ollama.localhost/api/tags

# Step 5: Stop old service
pkill ollama
```

---

## 🧪 STAGE 9: COMPREHENSIVE TESTING (WEEK 9)

### **9.1 FUNCTIONAL TESTING**

#### **9.1.1 Service Health Validation**
```bash
# Day 57-58: Health checks
# Location: All services
# Priority: CRITICAL

# Test all migrated services
for service in homeassistant immich paperless joplin jellyfin nextcloud appflowy gitea vaultwarden adguard; do
    echo "Testing $service..."
    curl -f https://$service.localhost/ || echo "$service: FAILED"
done

# Test database connections
docker exec postgresql_primary psql -U postgres -c "SELECT datname FROM pg_database;"
docker exec mariadb_primary mysql -u root -p -e "SHOW DATABASES;"
```

#### **9.1.2 Performance Testing**
```bash
# Day 58-59: Performance validation
# Location: All services
# Priority: HIGH

# Load testing
ab -n 1000 -c 10 https://nextcloud.localhost/
ab -n 1000 -c 10 https://jellyfin.localhost/

# Database performance
docker exec postgresql_primary psql -U postgres -c "SELECT * FROM pg_stat_database;"
docker exec mariadb_primary mysql -u root -p -e "SHOW STATUS LIKE 'Questions';"
```

### **9.2 SECURITY TESTING**

#### **9.2.1 Security Validation**
```bash
# Day 59-60: Security checks
# Location: All services
# Priority: CRITICAL

# SSL certificate validation
for service in homeassistant immich paperless joplin jellyfin nextcloud; do
    echo "Testing SSL for $service..."
    openssl s_client -connect $service.localhost:443 -servername $service.localhost < /dev/null
done

# Security headers validation
for service in vaultwarden adguard; do
    echo "Testing security headers for $service..."
    curl -I https://$service.localhost/ | grep -E "(Strict-Transport-Security|X-Frame-Options|X-Content-Type-Options)"
done
```

---

## 🎯 STAGE 10: FINAL VALIDATION AND CLEANUP (WEEK 10)

### **10.1 FINAL VALIDATION**

#### **10.1.1 Complete System Validation**
```bash
# Day 64-65: Final validation
# Location: All services
# Priority: CRITICAL

# Comprehensive health check
./migration_scripts/validate_migration.sh

# Performance baseline
./migration_scripts/performance_baseline.sh

# Security audit
./migration_scripts/security_audit.sh
```

#### **10.1.2 User Acceptance Testing**
```bash
# Day 65-66: User testing
# Location: All services
# Priority: CRITICAL

# Test all user workflows
# - Home Assistant automation
# - Immich photo upload/processing
# - Paperless document scanning
# - Jellyfin media streaming
# - Nextcloud file sync
# - AppFlowy collaboration
# - Vaultwarden password management
```

### **10.2 CLEANUP AND OPTIMIZATION**

#### **10.2.1 Old Infrastructure Cleanup**
```bash
# Day 66-67: Cleanup
# Location: All hosts
# Priority: MEDIUM

# Remove old containers
docker container prune -f

# Remove old volumes
docker volume prune -f

# Remove old networks
docker network prune -f

# Stop old native services
systemctl disable netdata
systemctl disable apache2
systemctl disable nginx
systemctl disable caddy
```

#### **10.2.2 Performance Optimization**
```bash
# Day 67-70: Optimization
# Location: All services
# Priority: MEDIUM

# Database optimization
docker exec postgresql_primary psql -U postgres -c "VACUUM ANALYZE;"
docker exec mariadb_primary mysql -u root -p -e "OPTIMIZE TABLE *;"

# Cache optimization
docker exec redis_master redis-cli FLUSHALL

# Monitoring optimization
# Configure alerting rules
# Set up performance dashboards
```

---

## 📋 CRITICAL REFERENCES AND LOCATIONS

### **Configuration File Locations**
```bash
# Docker Compose files
/opt/services/                    # New service configurations
/opt/databases/                   # Database configurations
/opt/monitoring/                  # Monitoring configurations
/opt/traefik/                     # Reverse proxy configurations

# Backup locations
/backup/                          # All backups
/backup/container_configs/        # Container configurations
/backup/compose_files/            # Docker Compose files
/backup/volumes/                  # Volume data
/backup/systemd_configs/          # Native service configurations

# Discovery results
comprehensive_discovery_results/  # All discovery data
comprehensive_discovery_results/container_audit_results/individual_configs/  # Container configs
comprehensive_discovery_results/detailed_container_inventory.yaml  # Container inventory
```

### **Critical Service Dependencies**
```yaml
# Database Dependencies
PostgreSQL Cluster:
  - Paperless-NGX
  - Joplin
  - Immich
  - AppFlowy

MariaDB Cluster:
  - Nextcloud
  - Gitea
  - Home Assistant

Redis Cluster:
  - Nextcloud
  - Paperless-NGX
  - AppFlowy
  - Immich

# Network Dependencies
Traefik:
  - All web services
  - SSL certificates
  - Load balancing

# Storage Dependencies
NFS/Samba:
  - Media files
  - Document storage
  - Photo libraries
```

### **Rollback Procedures**
```bash
# Database rollback
docker exec postgresql_primary psql -U postgres < /backup/postgresql_full_$(date).sql
docker exec mariadb_primary mysql -u root -p < /backup/mariadb_full_$(date).sql

# Service rollback
docker stack rm <service_name>
docker run -d --name <old_container> <old_image>

# Configuration rollback
cp /backup/systemd_configs/* /etc/systemd/system/
systemctl daemon-reload
systemctl restart <service_name>
```

---

## 🚨 CRITICAL CONSIDERATIONS

### **Data Integrity**
- **Always backup before any migration step**
- **Validate data integrity after each migration**
- **Test rollback procedures before migration**
- **Monitor disk space during migration**

### **Network Considerations**
- **Update DNS records gradually**
- **Test network connectivity at each step**
- **Monitor bandwidth usage during migration**
- **Plan for network failures**

### **Security Considerations**
- **Maintain security during migration**
- **Update firewall rules gradually**
- **Test SSL certificates after migration**
- **Validate authentication systems**

### **Performance Considerations**
- **Monitor performance during migration**
- **Plan for increased resource usage**
- **Test under load conditions**
- **Optimize after migration**

---

## ✅ SUCCESS CRITERIA

### **Technical Success**
- ✅ All services migrated successfully
- ✅ Zero data loss
- ✅ Zero downtime for critical services
- ✅ Performance maintained or improved
- ✅ Security maintained or improved

### **Operational Success**
- ✅ All users can access services
- ✅ All automations working
- ✅ All monitoring functional
- ✅ All backups working
- ✅ All alerts configured

### **Business Success**
- ✅ Improved reliability
- ✅ Improved performance
- ✅ Improved security
- ✅ Improved maintainability
- ✅ Future-proof architecture

---

**This migration plan provides a comprehensive, granular approach to transforming your infrastructure while maintaining 100% availability and data integrity. Each step includes detailed procedures, validation checks, and rollback capabilities.**
