Files
HomeAudit/scripts/setup_paperless_ai_integration.sh
admin 45363040f3 feat: Complete infrastructure cleanup phase documentation and status updates
## Major Infrastructure Milestones Achieved

###  Service Migrations Completed
- Jellyfin: Successfully migrated to Docker Swarm with latest version
- Vaultwarden: Running in Docker Swarm on OMV800 (eliminated duplicate)
- Nextcloud: Operational with database optimization and cron setup
- Paperless services: Both NGX and AI running successfully

### 🚨 Duplicate Service Analysis Complete
- Identified MariaDB conflict (OMV800 Swarm vs lenovo410 standalone)
- Identified Vaultwarden duplication (now resolved)
- Documented PostgreSQL and Redis consolidation opportunities
- Mapped monitoring stack optimization needs

### 🏗️ Infrastructure Status Documentation
- Updated README with current cleanup phase status
- Enhanced Service Analysis with duplicate service inventory
- Updated Quick Start guide with immediate action items
- Documented current container distribution across 6 nodes

### 📋 Action Plan Documentation
- Phase 1: Immediate service conflict resolution (this week)
- Phase 2: Service migration and load balancing (next 2 weeks)
- Phase 3: Database consolidation and optimization (future)

### 🔧 Current Infrastructure Health
- Docker Swarm: All 6 nodes operational and healthy
- Caddy Reverse Proxy: Fully operational with SSL certificates
- Storage: MergerFS healthy, local storage for databases
- Monitoring: Prometheus + Grafana + Uptime Kuma operational

### 📊 Container Distribution Status
- OMV800: 25+ containers (needs load balancing)
- lenovo410: 9 containers (cleanup in progress)
- fedora: 1 container (ready for additional services)
- audrey: 4 containers (well-balanced, monitoring hub)
- lenovo420: 7 containers (balanced, can assist)
- surface: 9 containers (specialized, reverse proxy)

### 🎯 Next Steps
1. Remove lenovo410 MariaDB (eliminate port 3306 conflict)
2. Clean up lenovo410 Vaultwarden (256MB space savings)
3. Verify no service conflicts exist
4. Begin service migration from OMV800 to fedora/audrey

Status: Infrastructure 99% complete, entering cleanup and optimization phase
2025-09-01 16:50:37 -04:00

261 lines
7.5 KiB
Bash
Executable File

#!/bin/bash
# Paperless AI Integration Setup Script
# This script helps configure Paperless AI to work with your existing Paperless-ngx setup
set -e
echo "🔧 Setting up Paperless AI Integration with Paperless-ngx"
echo "========================================================"
# Colors for output
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
BLUE='\033[0;34m'
NC='\033[0m' # No Color
# Function to print colored output
print_status() {
echo -e "${BLUE}[INFO]${NC} $1"
}
print_success() {
echo -e "${GREEN}[SUCCESS]${NC} $1"
}
print_warning() {
echo -e "${YELLOW}[WARNING]${NC} $1"
}
print_error() {
echo -e "${RED}[ERROR]${NC} $1"
}
# Check if running as root
if [[ $EUID -eq 0 ]]; then
print_error "This script should not be run as root"
exit 1
fi
# Check if Docker is running
if ! docker info > /dev/null 2>&1; then
print_error "Docker is not running. Please start Docker first."
exit 1
fi
# Check if Paperless-ngx is running
print_status "Checking if Paperless-ngx is running..."
if ! docker ps | grep -q "paperless"; then
print_warning "Paperless-ngx container not found. Please start it first."
read -p "Do you want to continue anyway? (y/N): " -n 1 -r
echo
if [[ ! $REPLY =~ ^[Yy]$ ]]; then
exit 1
fi
fi
# Create environment file for Paperless AI
ENV_FILE="stacks/ai/.env"
print_status "Creating environment file: $ENV_FILE"
mkdir -p "$(dirname "$ENV_FILE")"
# Check if .env file already exists
if [[ -f "$ENV_FILE" ]]; then
print_warning "Environment file already exists. Backing up..."
cp "$ENV_FILE" "${ENV_FILE}.backup.$(date +%Y%m%d_%H%M%S)"
fi
# Get Paperless-ngx admin credentials
print_status "Setting up Paperless-ngx connection..."
echo "# Paperless AI Environment Configuration" > "$ENV_FILE"
echo "# Generated on $(date)" >> "$ENV_FILE"
echo "" >> "$ENV_FILE"
# Paperless-ngx connection settings
echo "# Paperless-ngx Connection Settings" >> "$ENV_FILE"
echo "PAPERLESS_URL=https://paperless.pressmess.duckdns.org" >> "$ENV_FILE"
# Get admin credentials
read -p "Enter Paperless-ngx admin username (default: admin): " PAPERLESS_USERNAME
PAPERLESS_USERNAME=${PAPERLESS_USERNAME:-admin}
echo "PAPERLESS_USERNAME=$PAPERLESS_USERNAME" >> "$ENV_FILE"
read -s -p "Enter Paperless-ngx admin password: " PAPERLESS_PASSWORD
echo
echo "PAPERLESS_PASSWORD=$PAPERLESS_PASSWORD" >> "$ENV_FILE"
# API Token setup
print_status "Setting up API Token..."
echo "" >> "$ENV_FILE"
echo "# API Token (optional - will be generated if not provided)" >> "$ENV_FILE"
read -p "Do you have an existing API token? (y/N): " -n 1 -r
echo
if [[ $REPLY =~ ^[Yy]$ ]]; then
read -s -p "Enter your API token: " PAPERLESS_API_TOKEN
echo
echo "PAPERLESS_API_TOKEN=$PAPERLESS_API_TOKEN" >> "$ENV_FILE"
else
print_status "API token will be generated automatically when Paperless AI starts"
echo "PAPERLESS_API_TOKEN=" >> "$ENV_FILE"
fi
# AI Provider Configuration
echo "" >> "$ENV_FILE"
echo "# AI Provider Configuration" >> "$ENV_FILE"
echo "# At least one AI provider must be configured" >> "$ENV_FILE"
# OpenAI Configuration
read -p "Do you want to configure OpenAI? (y/N): " -n 1 -r
echo
if [[ $REPLY =~ ^[Yy]$ ]]; then
read -s -p "Enter your OpenAI API key: " OPENAI_API_KEY
echo
echo "OPENAI_API_KEY=$OPENAI_API_KEY" >> "$ENV_FILE"
fi
# Ollama Configuration
read -p "Do you want to configure Ollama (local AI)? (y/N): " -n 1 -r
echo
if [[ $REPLY =~ ^[Yy]$ ]]; then
read -p "Enter Ollama base URL (default: http://ollama:11434): " OLLAMA_BASE_URL
OLLAMA_BASE_URL=${OLLAMA_BASE_URL:-http://ollama:11434}
echo "OLLAMA_BASE_URL=$OLLAMA_BASE_URL" >> "$ENV_FILE"
fi
# DeepSeek Configuration
read -p "Do you want to configure DeepSeek? (y/N): " -n 1 -r
echo
if [[ $REPLY =~ ^[Yy]$ ]]; then
read -s -p "Enter your DeepSeek API key: " DEEPSEEK_API_KEY
echo
echo "DEEPSEEK_API_KEY=$DEEPSEEK_API_KEY" >> "$ENV_FILE"
fi
# Azure OpenAI Configuration
read -p "Do you want to configure Azure OpenAI? (y/N): " -n 1 -r
echo
if [[ $REPLY =~ ^[Yy]$ ]]; then
read -s -p "Enter your Azure OpenAI API key: " AZURE_OPENAI_API_KEY
echo
read -p "Enter your Azure OpenAI endpoint: " AZURE_OPENAI_ENDPOINT
echo "AZURE_OPENAI_API_KEY=$AZURE_OPENAI_API_KEY" >> "$ENV_FILE"
echo "AZURE_OPENAI_ENDPOINT=$AZURE_OPENAI_ENDPOINT" >> "$ENV_FILE"
fi
# Processing Configuration
echo "" >> "$ENV_FILE"
echo "# Processing Configuration" >> "$ENV_FILE"
echo "PAPERLESS_AI_AUTO_TAG=true" >> "$ENV_FILE"
echo "PAPERLESS_AI_AUTO_TITLE=true" >> "$ENV_FILE"
echo "PAPERLESS_AI_CONFIDENCE_THRESHOLD=0.7" >> "$ENV_FILE"
echo "PAPERLESS_AI_PROCESSING_INTERVAL=300" >> "$ENV_FILE"
print_success "Environment file created: $ENV_FILE"
# Create deployment script
DEPLOY_SCRIPT="scripts/deploy_paperless_ai.sh"
print_status "Creating deployment script: $DEPLOY_SCRIPT"
cat > "$DEPLOY_SCRIPT" << 'EOF'
#!/bin/bash
# Deploy Paperless AI with proper integration
set -e
echo "🚀 Deploying Paperless AI with Paperless-ngx integration..."
# Check if environment file exists
if [[ ! -f "stacks/ai/.env" ]]; then
echo "❌ Environment file not found. Please run setup_paperless_ai_integration.sh first."
exit 1
fi
# Load environment variables
set -a
source stacks/ai/.env
set +a
# Deploy Paperless AI
cd stacks/ai
docker-compose -f paperless-ai.yml --env-file .env up -d
echo "✅ Paperless AI deployed successfully!"
echo "🌐 Access Paperless AI at: https://paperless-ai.pressmess.duckdns.org"
echo "📊 Check logs with: docker-compose -f stacks/ai/paperless-ai.yml logs -f"
EOF
chmod +x "$DEPLOY_SCRIPT"
# Create verification script
VERIFY_SCRIPT="scripts/verify_paperless_ai.sh"
print_status "Creating verification script: $VERIFY_SCRIPT"
cat > "$VERIFY_SCRIPT" << 'EOF'
#!/bin/bash
# Verify Paperless AI integration
set -e
echo "🔍 Verifying Paperless AI integration..."
# Check if containers are running
echo "Checking container status..."
if docker ps | grep -q "paperless-ai"; then
echo "✅ Paperless AI container is running"
else
echo "❌ Paperless AI container is not running"
exit 1
fi
if docker ps | grep -q "paperless"; then
echo "✅ Paperless-ngx container is running"
else
echo "❌ Paperless-ngx container is not running"
fi
# Check database connectivity
echo "Checking database connectivity..."
if docker exec paperless-ai pg_isready -h postgresql_postgresql_primary -U postgres > /dev/null 2>&1; then
echo "✅ Database connection successful"
else
echo "❌ Database connection failed"
fi
# Check API connectivity
echo "Checking API connectivity..."
if curl -f -s "https://paperless.pressmess.duckdns.org/api/" > /dev/null; then
echo "✅ Paperless-ngx API accessible"
else
echo "❌ Paperless-ngx API not accessible"
fi
if curl -f -s "http://localhost:3000/health" > /dev/null; then
echo "✅ Paperless AI health check passed"
else
echo "❌ Paperless AI health check failed"
fi
echo "🎉 Verification complete!"
EOF
chmod +x "$VERIFY_SCRIPT"
print_success "Setup complete!"
echo ""
echo "📋 Next steps:"
echo "1. Review the environment file: $ENV_FILE"
echo "2. Deploy Paperless AI: ./$DEPLOY_SCRIPT"
echo "3. Verify integration: ./$VERIFY_SCRIPT"
echo ""
echo "🔧 Manual configuration may be required:"
echo "- Ensure Paperless-ngx has API access enabled"
echo "- Configure AI provider API keys in the environment file"
echo "- Set up proper network connectivity between containers"
echo ""
echo "📚 Documentation:"
echo "- Paperless AI: https://github.com/clusterzx/paperless-ai"
echo "- Paperless-ngx API: https://docs.paperless-ngx.com/api/"