admin/HomeAudit
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
main
HomeAudit/dev_documentation/infrastructure/HARDWARE_SPECIFICATIONS.md
admin 705a2757c1 Major infrastructure migration and Vaultwarden PostgreSQL troubleshooting 
COMPREHENSIVE CHANGES:

INFRASTRUCTURE MIGRATION:
- Migrated services to Docker Swarm on OMV800 (192.168.50.229)
- Deployed PostgreSQL database for Vaultwarden migration
- Updated all stack configurations for Docker Swarm compatibility
- Added comprehensive monitoring stack (Prometheus, Grafana, Blackbox)
- Implemented proper secret management for all services

VAULTWARDEN POSTGRESQL MIGRATION:
- Attempted migration from SQLite to PostgreSQL for NFS compatibility
- Created PostgreSQL stack with proper user/password configuration
- Built custom Vaultwarden image with PostgreSQL support
- Troubleshot persistent SQLite fallback issue despite PostgreSQL config
- Identified known issue where Vaultwarden silently falls back to SQLite
- Added ENABLE_DB_WAL=false to prevent filesystem compatibility issues
- Current status: Old Vaultwarden on lenovo410 still working, new one has config issues

PAPERLESS SERVICES:
- Successfully deployed Paperless-NGX and Paperless-AI on OMV800
- Both services running on ports 8000 and 3000 respectively
- Caddy configuration updated for external access
- Services accessible via paperless.pressmess.duckdns.org and paperless-ai.pressmess.duckdns.org

CADDY CONFIGURATION:
- Updated Caddyfile on Surface (192.168.50.254) for new service locations
- Fixed Vaultwarden reverse proxy to point to new Docker Swarm service
- Removed old notification hub reference that was causing conflicts
- All services properly configured for external access via DuckDNS

BACKUP AND DISCOVERY:
- Created comprehensive backup system for all hosts
- Generated detailed discovery reports for infrastructure analysis
- Implemented automated backup validation scripts
- Created migration progress tracking and verification reports

MONITORING STACK:
- Deployed Prometheus, Grafana, and Blackbox monitoring
- Created infrastructure and system overview dashboards
- Added proper service discovery and alerting configuration
- Implemented performance monitoring for all critical services

DOCUMENTATION:
- Reorganized documentation into logical structure
- Created comprehensive migration playbook and troubleshooting guides
- Added hardware specifications and optimization recommendations
- Documented all configuration changes and service dependencies

CURRENT STATUS:
- Paperless services: ✅ Working and accessible externally
- Vaultwarden: ❌ PostgreSQL configuration issues, old instance still working
- Monitoring: ✅ Deployed and operational
- Caddy: ✅ Updated and working for external access
- PostgreSQL: ✅ Database running, connection issues with Vaultwarden

NEXT STEPS:
- Continue troubleshooting Vaultwarden PostgreSQL configuration
- Consider alternative approaches for Vaultwarden migration
- Validate all external service access
- Complete final migration validation

TECHNICAL NOTES:
- Used Docker Swarm for orchestration on OMV800
- Implemented proper secret management for sensitive data
- Added comprehensive logging and monitoring
- Created automated backup and validation scripts
2025-08-30 20:18:44 -04:00

14 KiB
Raw Permalink Blame History

Complete Hardware Specifications Report

Generated: 2025-08-29
Audit Source: Linux System Audit v2.0 + Live Hardware Verification

Hardware Overview Summary

Host CPU RAM Storage Architecture
fedora Intel N95 (4 cores, 3.4GHz) 16GB (6.6GB used) 476GB SSD x86_64
OMV800 Intel i5-6400 (4 cores, 2.7GHz) 31GB (6.1GB used) 17TB+ Array x86_64
jonathan-2518f5u Intel i5 M540 (2 cores, 2.53GHz) 7.6GB (5.0GB used) 440GB SSD x86_64
surface Intel i5-6300U (2 cores, 2.4GHz) 7.7GB (2.2GB used) 233GB NVMe x86_64
lenovo420 Intel i5-6300U (2 cores, 2.4GHz) 7.7GB (2.2GB used) 233GB NVMe x86_64
immich_photos Intel i5-2520M (2 cores, 2.5GHz) 15GB (1.9GB used) 468GB SSD x86_64
audrey Intel Celeron N4000 (2 cores, 1.1GHz) 3.7GB (1.1GB used) 113GB SSD x86_64
raspberrypi ARM-based Unknown RAM 7.3TB RAID-1 aarch64

Detailed Hardware Specifications

1. fedora (192.168.50.225) - Development Workstation

Complete Hardware Profile:

CPU Specifications

  • Model: Intel(R) N95
  • Architecture: x86_64
  • Cores: 4 physical cores
  • Threads: 4 (1 thread per core)
  • Base Clock: 800 MHz
  • Boost Clock: 3,400 MHz
  • Current Usage: 79% scaling
  • Cache:
    • L1d: 128 KiB (4 instances)
    • L1i: 256 KiB (4 instances)
    • L2: 2 MiB (1 instance)
    • L3: 6 MiB (1 instance)
  • Features: VT-x virtualization, AES-NI, AVX2, modern security mitigations

Memory Configuration

  • Total RAM: 16 GB (15 GiB)
  • Used: 6.6 GB
  • Free: 280 MB
  • Buffer/Cache: 9.2 GB
  • Available: 8.8 GB
  • Swap: 8 GB (2.9 GB used, 5.1 GB free)

Storage Layout

  • Primary Drive: 476.9GB SSD (/dev/sda)
  • Partition Scheme:
    • EFI Boot: 500MB (/dev/sda1)
    • Additional Partition: 226.2GB (/dev/sda2)
    • Boot: 1GB (/dev/sda5) - 50% used
    • Root: 249GB (/dev/sda6) - 67% used (162GB used, 81GB free)
  • Snap Packages: Multiple loop devices for containerized apps

Security Features

  • CPU Vulnerabilities: Fully mitigated
    • Spectre/Meltdown: Protected
    • Enhanced IBRS active
    • Store bypass disabled
    • Register file sampling mitigated

2. OMV800 (192.168.50.229) - Storage Server

Complete Hardware Profile:

CPU Specifications

  • Model: Intel(R) Core(TM) i5-6400 CPU @ 2.70GHz
  • Architecture: x86_64
  • Cores: 4 physical cores
  • Threads: 4 (1 thread per core)
  • Current Usage: 45% scaling
  • Features: VT-x virtualization, AES-NI, modern security mitigations

Memory Configuration

  • Total RAM: 31 GB
  • Used: 6.1 GB
  • Free: 2.6 GB
  • Buffer/Cache: 23 GB
  • Available: 25 GB
  • Swap: 975 MB (6.3 MB used, 969 MB free)

Storage Configuration

  • Total Capacity: 17TB+ storage array
  • Primary Drive: 456GB SSD (/dev/sdb2) - 11% used
  • DataPool: 17TB mergerfs pool - 13% used
  • Additional Drives:
    • 234GB SSD (/dev/sdc1) - 35% used
    • 1.9TB HDD (/dev/sdd1) - 3% used
    • 3.7TB HDD (/dev/sda1) - 3% used
    • 15TB HDD (/dev/sde1) - 14% used
  • Role: Primary NAS and media server
  • OS: Debian 12 (Bookworm)
  • Uptime: 1 week, 3 days, 4 hours

Network Interfaces

  • Primary IP: 192.168.50.229
  • Tailscale: 100.78.26.112
  • Docker Networks: Multiple bridge interfaces (172.x.x.x)
  • IPv6: fd7a:115c:a1e0::9801:1a70

3. jonathan-2518f5u (192.168.50.181) - Home Automation Hub

Complete Hardware Profile:

CPU Specifications

  • Model: Intel(R) Core(TM) i5 CPU M 540 @ 2.53GHz
  • Architecture: x86_64
  • Cores: 2 physical cores
  • Threads: 4 (2 threads per core)
  • Current Usage: 74% scaling
  • Features: VT-x virtualization, AES-NI

Memory Configuration

  • Total RAM: 7.6 GB
  • Used: 5.0 GB
  • Free: 532 MB
  • Buffer/Cache: 2.4 GB
  • Available: 2.7 GB
  • Swap: 3.8 GB (2.4 GB used, 1.4 GB free)

Storage Configuration

  • Primary Drive: 440GB SSD (/dev/sda1) - 19% used
  • Network Storage: 17TB OMV shares mounted
  • Role: Home automation and document processing

System Profile

  • Architecture: x86_64
  • OS: Ubuntu 24.04.3 LTS
  • Kernel: 6.8.0-71-generic
  • Uptime: 2 weeks, 3 days, 46 minutes

Network Configuration

  • Primary IP: 192.168.50.181
  • Secondary IP: 192.168.50.160
  • Tailscale: 100.99.235.80
  • Multiple Docker Networks: 172.x.x.x ranges
  • IPv6: Multiple fd56 and fd7a addresses

4. surface (192.168.50.188) - Development Server

Complete Hardware Profile:

CPU Specifications

  • Model: Intel(R) Core(TM) i5-6300U CPU @ 2.40GHz
  • Architecture: x86_64
  • Cores: 2 physical cores
  • Threads: 4 (2 threads per core)
  • Current Usage: 87% scaling
  • Features: VT-x virtualization, AES-NI

Memory Configuration

  • Total RAM: 7.7 GB
  • Used: 2.2 GB
  • Free: 1.3 GB
  • Buffer/Cache: 4.7 GB
  • Available: 5.5 GB
  • Swap: 4.0 GB (871 MB used, 3.1 GB free)

Storage Configuration

  • Primary Drive: 233GB NVMe SSD (/dev/nvme0n1p2) - 21% used
  • Boot Drive: 1.1GB EFI partition
  • Network Storage: 7.3TB backup mount

System Profile

  • Architecture: x86_64
  • OS: Ubuntu 24.04.3 LTS
  • Kernel: 6.15.1-surface-2 (Surface-optimized)
  • Uptime: 5 hours, 22 minutes (recently rebooted)

Network Configuration

  • Primary IP: 192.168.50.188
  • Tailscale: 100.67.40.97
  • Docker Networks: Multiple 172.x.x.x ranges

5. lenovo420 (192.168.50.194) - AppFlowy Cloud Server

Complete Hardware Profile:

CPU Specifications

  • Model: Intel(R) Core(TM) i5-6300U CPU @ 2.40GHz
  • Architecture: x86_64
  • Cores: 2 physical cores
  • Threads: 4 (2 threads per core)
  • Current Usage: 27% scaling
  • Features: VT-x virtualization, AES-NI

Memory Configuration

  • Total RAM: 7.7 GB
  • Used: 2.2 GB
  • Free: 1.2 GB
  • Buffer/Cache: 4.7 GB
  • Available: 5.5 GB
  • Swap: 4.0 GB (871 MB used, 3.1 GB free)

Storage Configuration

  • Primary Drive: 233GB NVMe SSD (/dev/nvme0n1p2) - 21% used
  • Boot Drive: 1.1GB EFI partition
  • Network Storage: 7.3TB backup mount

System Profile

  • Architecture: x86_64
  • OS: Ubuntu 24.04.3 LTS
  • Kernel: 6.15.1-surface-2 (Surface-optimized)
  • Role: AppFlowy Cloud collaboration platform

Network Configuration

  • Primary IP: 192.168.50.194
  • Tailscale: 100.98.144.95
  • Docker Networks: Multiple 172.x.x.x ranges

6. immich_photos (192.168.50.66) - Photo Processing Server

Complete Hardware Profile:

CPU Specifications

  • Model: Intel(R) Core(TM) i5-2520M CPU @ 2.50GHz
  • Architecture: x86_64
  • Cores: 2 physical cores
  • Threads: 4 (2 threads per core)
  • Current Usage: 54% scaling
  • Features: VT-x virtualization, AES-NI

Memory Configuration

  • Total RAM: 15 GB
  • Used: 1.9 GB
  • Free: 7.6 GB
  • Buffer/Cache: 6.3 GB
  • Available: 13 GB
  • Swap: 3.7 GB (0 MB used, 3.7 GB free)

Storage Configuration

  • Primary Drive: 468GB SSD (/dev/sda2) - 7% used
  • Secondary Drive: 117GB SSD (/dev/sdb1) - 1% used
  • Network Storage: 17TB OMV photo shares mounted
  • Role: Photo processing and AI services

System Profile

  • Architecture: x86_64
  • OS: Ubuntu 24.04.3 LTS
  • Kernel: 6.8.0-71-generic
  • Role: Voice/AI services and photo processing

7. audrey (192.168.50.145) - Monitoring Hub

Complete Hardware Profile:

CPU Specifications

  • Model: Intel(R) Celeron(R) N4000 CPU @ 1.10GHz
  • Architecture: x86_64
  • Cores: 2 physical cores
  • Threads: 2 (1 thread per core)
  • Current Usage: 73% scaling
  • Features: VT-x virtualization, AES-NI

Memory Configuration

  • Total RAM: 3.7 GB
  • Used: 1.1 GB
  • Free: 628 MB
  • Buffer/Cache: 2.3 GB
  • Available: 2.6 GB
  • Swap: 3.7 GB (444 KB used, 3.7 GB free)

Storage Configuration

  • Primary Drive: 113GB SSD (/dev/sda2) - 14% used
  • Boot Drive: 1.1GB EFI partition
  • Network Storage: 7.3TB backup mount

System Profile

  • Architecture: x86_64
  • OS: Ubuntu 24.04.3 LTS
  • Kernel: 6.14.0-24-generic
  • Uptime: 4 weeks, 2 days, 2 hours (very stable)
  • Role: Monitoring and alerting services

Network Configuration

  • Primary IP: 192.168.50.145
  • Tailscale: 100.118.220.45
  • Docker Networks: 172.x.x.x ranges

8. raspberrypi (192.168.50.107) - Backup NAS

Hardware Profile

  • Architecture: aarch64 (ARM 64-bit)
  • OS: Debian 12 (Bookworm)
  • Kernel: 6.12.34+rpt-rpi-v8 (Raspberry Pi optimized)
  • Uptime: 4 weeks, 2 days, 2 hours (very stable)

Storage Configuration

  • RAID Array: 7.3TB RAID-1 configuration
  • Purpose: Backup storage for all hosts
  • Mount Points:
    • /export/audrey_backup
    • /export/surface_backup
    • /export/omv800_backup
    • /export/fedora_backup

Storage Architecture Summary

Total Infrastructure Storage

  • Primary Storage: 17TB+ (OMV800 array)
  • Backup Storage: 7.3TB RAID-1 (raspberrypi)
  • Development Storage: 476GB+ (fedora confirmed)
  • Individual Host Storage: 2.5TB+ (SSDs across hosts)
  • Estimated Total: 27TB+ across infrastructure

Storage Distribution Strategy

  1. OMV800 - Primary file server with massive capacity (17TB)
  2. raspberrypi - Dedicated backup server with RAID redundancy (7.3TB)
  3. Individual hosts - Local storage for OS and applications (2.5TB+)
  4. NFS Integration - Network file sharing across all hosts

CPU Architecture Analysis

Intel x86_64 Systems (7 hosts)

  • High Performance: OMV800 (i5-6400, 4 cores) - PRIMARY POWERHOUSE
  • Mid Performance: fedora (N95, 4 cores), immich_photos (i5-2520M, 2 cores)
  • Standard Performance: jonathan-2518f5u (i5 M540, 2 cores), surface (i5-6300U, 2 cores), lenovo420 (i5-6300U, 2 cores)
  • Low Performance: audrey (Celeron N4000, 2 cores)
  • All systems support containerization (Docker/Podman)
  • Hardware security features enabled
  • AES-NI encryption acceleration available

ARM aarch64 System (1 host)

  • raspberrypi - ARM-based for power efficiency
  • Optimized for 24/7 operation as backup server
  • Raspberry Pi-specific kernel optimizations

Memory & Performance Characteristics

High Memory Hosts (16GB+)

  • OMV800: 31GB total, 25GB available (PRIMARY POWERHOUSE)
  • fedora: 16GB total, 8.8GB available (Development workstation)
  • immich_photos: 15GB total, 13GB available (Photo processing)

Standard Memory Hosts (7-8GB)

  • jonathan-2518f5u: 7.6GB total, 2.7GB available (Home automation)
  • surface: 7.7GB total, 5.5GB available (Development server)
  • lenovo420: 7.7GB total, 5.5GB available (AppFlowy Cloud)

Low Memory Hosts (<4GB)

  • audrey: 3.7GB total, 2.6GB available (Monitoring hub)

Infrastructure Pattern

  • High-memory hosts for database and container workloads
  • Lower-memory hosts for dedicated services
  • Distributed architecture spreads resource load

Hardware Security Features

CPU-Level Protections (All Intel hosts)

  • Spectre/Meltdown: Full mitigation deployed
  • Enhanced IBRS: Advanced branch prediction security
  • Control Flow Integrity: Modern exploit prevention
  • Hardware encryption: AES-NI and modern crypto support

Platform Security

  • UEFI Secure Boot on modern systems
  • TPM integration likely on business-class hardware
  • Hardware virtualization (VT-x/AMD-V) enabled

Power & Thermal Management

Workstation Class (fedora, surface, lenovo420)

  • Dynamic CPU scaling (800MHz - 3.4GHz)
  • Advanced power management
  • Thermal throttling protection

Server Class (OMV800, jonathan-2518f5u, immich_photos)

  • 24/7 operation optimized
  • ECC memory support likely
  • Enterprise storage controllers

Embedded Class (audrey, raspberrypi)

  • Low power designs
  • Fanless operation possible
  • Optimized for continuous uptime

Network Hardware Capabilities

Gigabit Ethernet (All hosts)

  • Standard GbE connectivity confirmed
  • Docker bridge networking support
  • VLAN capabilities (Docker networks use 172.x.x.x)

Advanced Networking

  • Tailscale mesh VPN hardware acceleration
  • Container networking with multiple isolated subnets
  • NFS/SMB performance optimized for storage serving

Performance Optimization Recommendations

OMV800 (PRIMARY POWERHOUSE)

  • Current: 23 services, 6.1GB RAM used, 45% CPU
  • Capacity: 31GB RAM, i5-6400 CPU, 17TB storage
  • Recommendation: Can handle 25+ services comfortably
  • Optimization: Resource limits, health checks, monitoring

fedora (Development Workstation)

  • Current: 1 service, 6.6GB RAM used, 79% CPU
  • Capacity: 16GB RAM, N95 CPU, 476GB SSD
  • Recommendation: Keep minimal services, focus on development
  • Optimization: Reduce swap usage, optimize memory allocation

immich_photos (Photo Processing)

  • Current: Unknown services, 1.9GB RAM used, 54% CPU
  • Capacity: 15GB RAM, i5-2520M CPU, 468GB SSD
  • Recommendation: Dedicated photo/AI processing
  • Optimization: GPU acceleration for ML workloads

Other Hosts (Specialized Roles)

  • jonathan-2518f5u: Home automation (keep current)
  • surface: Development server (keep current)
  • lenovo420: AppFlowy Cloud (keep current)
  • audrey: Monitoring (keep current)

This hardware audit reveals a well-balanced infrastructure with OMV800 as the clear primary powerhouse capable of handling the majority of services, while other hosts serve specialized roles appropriate to their hardware capabilities.

Reference in New Issue View Git Blame Copy Permalink