Initial: backup/restore scripts + README for DO server

- backup.sh: weekly cron collecting scripts, systemd, WG, OLS vhosts, cron, mysql creds - restore.sh: 8-phase interactive disaster recovery wizard - README.md: full rebuild guide, credentials, architecture notes Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-30 17:50:59 -05:00 · 2026-06-09 03:53:09 +00:00
commit 5b1f83b1ea
4 changed files with 690 additions and 0 deletions
@@ -0,0 +1,4 @@
+*.log
+*.tmp
+__pycache__/
+*.pyc
@@ -0,0 +1,172 @@
+# DO Server Config Backup & Restore
+
+Weekly backup of the DigitalOcean production server (`orbis`, 165.22.1.228).  
+CyberPanel/OpenLiteSpeed, Ubuntu 24.04, 48GB disk.
+
+---
+
+## What's Backed Up
+
+| Directory | Source | Contents |
+|-----------|--------|----------|
+| `scripts/` | `/usr/local/bin/` | jarvis-deploy, jarvis-watchdog, jarvis-backup, jarvis-agent, ttg-backup |
+| `systemd/` | `/etc/systemd/system/` | jarvis-agent.service, fastapi_ssh_server.service |
+| `wireguard/` | `/etc/wireguard/` | wg0.conf (VPN hub for home network), other WG profiles |
+| `network/` | `/etc/netplan/` | 50-cloud-init.yaml, hosts, hostname |
+| `cron/` | `crontab -l` | root_custom (JARVIS lines), root_full (everything) |
+| `ssh/` | `/root/.ssh/` | authorized_keys, id_rsa.pub (no private keys) |
+| `ols-vhosts/` | `/usr/local/lsws/conf/vhosts/` | All 8 site OLS vhost configs |
+| `mysql/` | `/root/.my.cnf` | MySQL root credentials, database list |
+| `infra/` | `/opt/infra/` | DO infra repo snapshot |
+| `smtp-docs/` | `/opt/smtp-for-websites/` | CyberMail SMTP config docs |
+
+**Backup schedule:** Every Sunday at 4:00 AM  
+**Log:** `/var/log/do-server-backup.log`  
+**Manual trigger:** `/usr/local/bin/do-server-backup`
+
+---
+
+## Sites on This Server
+
+| Domain | GitHub Repo | DB |
+|--------|-------------|-----|
+| jarvis.orbishosting.com | myronblair/jarvis | jarvis_db |
+| tomsjavajive.com | myronblair/tomsjavajive | toms_tjj_db |
+| epictravelexpeditions.com | myronblair/epictravelexpeditions | epic_epic_db |
+| parkerslingshot.epictravelexpeditions.com | myronblair/parkerslingshot | epic_parkersling |
+| parkerslingshotrentals.com | myronblair/parkerslingshotrentals | parker_db |
+| orbishosting.com | myronblair/orbishosting | — |
+| orbis.orbishosting.com | myronblair/orbis-hosting-portal | — |
+| tomtomgames.com | myronblair/tomtomgames | tomt_ttg_db |
+
+GitHub PAT: `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` (expires ~2026-08-20)
+
+---
+
+## What Is NOT Backed Up Here
+
+| Item | Location / Notes |
+|------|-----------------|
+| Website files | All in GitHub repos — `git pull` on restore |
+| Databases | Backed up daily by `jarvis-backup.sh` → `/var/backups/jarvis/` |
+| SSL certs | `/etc/letsencrypt/` — re-issue via CyberPanel after restore |
+| Gitignored configs | `api/config.php` files — must recreate manually (see DB creds below) |
+| SSH private key | `/root/.ssh/id_rsa` — regenerate or restore from secure storage |
+| CyberPanel itself | Reinstall via `sh <(curl https://cyberpanel.net/install.sh)` |
+| composer binary | Reinstall: `curl -sS https://getcomposer.org/installer | php -- --install-dir=/usr/local/bin` |
+
+---
+
+## Disaster Recovery — Full Server Rebuild
+
+**Estimated time: 60–90 minutes**
+
+### Step 1 — Create new DigitalOcean Droplet
+
+- Region: Same as original (NYC3 / SFO3 — check DO console)
+- Image: Ubuntu 24.04 LTS
+- Size: At minimum 4GB RAM, 80GB disk (original was 4vCPU/8GB/160GB)
+- Add SSH key from your local machine
+
+### Step 2 — Clone this repo
+
+```bash
+apt update && apt install -y git
+git clone https://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@github.com/myronblair/do-server-config.git /opt/do-server-config
+bash /opt/do-server-config/restore.sh
+```
+
+The restore script is interactive — it walks you through each phase and asks before applying each section.
+
+### Step 3 — DNS Update (if IP changed)
+
+If the new droplet has a different IP, update DNS before or immediately after:
+- Cloudflare → DNS → Update A records for all domains
+- Also update GitHub webhook URLs if they use IP directly (they don't — they use jarvis.orbishosting.com which is behind Cloudflare)
+
+### Step 4 — Database Restore
+
+Databases are in daily backups at `/var/backups/jarvis/`. If restoring to a new server:
+
+```bash
+# Copy the latest backup from old server (if still accessible)
+scp root@165.22.1.228:/var/backups/jarvis/jarvis_backup_YYYY-MM-DD_*.tar.gz /tmp/
+
+# Extract and restore
+cd /tmp && tar -xzf jarvis_backup_*.tar.gz
+# Then for each database:
+mysql -u root -p jarvis_db     < sql/jarvis_db.sql
+mysql -u root -p toms_tjj_db   < sql/toms_tjj_db.sql
+mysql -u root -p tomt_ttg_db   < sql/tomt_ttg_db.sql
+mysql -u root -p epic_epic_db  < sql/epic_epic_db.sql
+mysql -u root -p epic_parkersling < sql/epic_parkersling.sql
+mysql -u root -p parker_db     < sql/parker_db.sql
+```
+
+You'll need to create the DB users first:
+```sql
+CREATE USER 'jarvis_user'@'localhost' IDENTIFIED BY 'J4rv1s_Pr0t0c0l_2026!';
+GRANT ALL ON jarvis_db.* TO 'jarvis_user'@'localhost';
+-- repeat for each DB user
+```
+
+### Step 5 — Recreate Gitignored Configs
+
+After pulling website code, create these manually:
+
+**`/home/jarvis.orbishosting.com/api/config.php`** — Contains all JARVIS credentials (Groq API key, Ollama URL, GitHub PAT, JARVIS DB credentials, ElevenLabs key, etc.)
+
+**`/home/tomsjavajive.com/config/database.php`** — TJJ DB credentials
+
+**`/home/epictravelexpeditions.com/api/config.php`** — Epic/Parker credentials
+
+Reference: `api/config.php.example` or `config/database.php.example` in each repo.
+
+---
+
+## Key Credentials Quick Reference
+
+| Service | User | Password |
+|---------|------|----------|
+| Root SSH | root | Gonewalk1974!@# |
+| CyberPanel admin | admin | (set during reinstall) |
+| MySQL root | root | Z9Of4NVs6ji74x (see mysql/my.cnf) |
+| jarvis_db | jarvis_user | J4rv1s_Pr0t0c0l_2026! |
+| toms_tjj_db | toms_tjj_user | +60wlPc+55e@gFq4 |
+| tomt_ttg_db | tomt_ttg_user | q#q+mrOcozsa7I6J |
+| epic_parkersling | epic_parkersling | Joker1974!!! |
+
+---
+
+## WireGuard VPN (wg0)
+
+The DO server is the VPN hub for the home network tunnel:
+- `wg0` on DO: `10.201.0.1/24`, ListenPort 51820
+- Peer: Alpine LXC CT110 on home network → `10.201.0.2/32`
+- Traffic flows: CT110 → DO for MediaStack VPN exit
+
+Config is in `wireguard/wg0.conf`. Keys are included (this is a private repo).
+
+---
+
+## Architecture Notes
+
+- **CyberPanel/OLS** manages PHP-FPM, virtual hosts, SSL, DNS (PowerDNS), mail (Postfix)
+- **Cloudflare** is in front of all public sites — DNS proxy mode, Rocket Loader ON (breaks inline JS — `Cache-Control: no-transform` header is the fix)
+- **JARVIS webhooks** at `/home/jarvis.orbishosting.com/public_html/webhook.php` handle auto-deploys for all 8+ GitHub repos
+- **Deploy queue** `/tmp/jarvis-deploy-queue.txt` processed by `jarvis-deploy.sh` every minute
+- **Watchdog** `jarvis-watchdog.sh` runs every 5 min — restarts lsws/mysql/redis if down, inserts alerts to DB
+- **Daily DB backup** `jarvis-backup.sh` runs at 2am → `/var/backups/jarvis/` (7-day retention)
+- **JARVIS agent** registers this server with JARVIS monitoring dashboard
+
+---
+
+## After Restore — Verify Services
+
+```bash
+systemctl status lsws mysql redis jarvis-agent
+crontab -l
+wg show
+curl -sk https://jarvis.orbishosting.com/api.php?action=ping
+tail -f /home/jarvis.orbishosting.com/logs/watchdog.log
+```
@@ -0,0 +1,160 @@
+#!/usr/bin/env bash
+# =============================================================================
+# DO Server Config Backup — runs on orbis (165.22.1.228)
+# Backs up all critical configs/scripts to GitHub weekly
+# Install: /usr/local/bin/do-server-backup
+# Cron:    0 4 * * 0 /usr/local/bin/do-server-backup >> /var/log/do-server-backup.log 2>&1
+# =============================================================================
+set -euo pipefail
+
+PAT="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"
+REPO_URL="https://${PAT}@github.com/myronblair/do-server-config.git"
+REPO_DIR="/opt/do-server-config"
+LOG_PREFIX="[$(date '+%Y-%m-%d %H:%M:%S')] [orbis]"
+
+log() { echo "$LOG_PREFIX $*"; }
+
+# ---------------------------------------------------------------------------
+# 1. Clone or update repo
+# ---------------------------------------------------------------------------
+if [[ -d "$REPO_DIR/.git" ]]; then
+    log "Pulling latest from GitHub"
+    cd "$REPO_DIR"
+    git config user.email "backup@orbishosting.com"
+    git config user.name  "DO Server Backup"
+    git pull --rebase origin main -q || true
+else
+    log "Cloning repo to $REPO_DIR"
+    git clone "$REPO_URL" "$REPO_DIR"
+    cd "$REPO_DIR"
+    git config user.email "backup@orbishosting.com"
+    git config user.name  "DO Server Backup"
+fi
+
+cd "$REPO_DIR"
+mkdir -p scripts systemd wireguard network cron ssh ols-vhosts mysql infra
+
+# ---------------------------------------------------------------------------
+# 2. Custom scripts from /usr/local/bin (text only — skip large binaries)
+# ---------------------------------------------------------------------------
+log "Backing up custom scripts"
+for f in /usr/local/bin/jarvis-*.sh \
+          /usr/local/bin/jarvis-*.py \
+          /usr/local/bin/ttg-backup.sh \
+          /usr/local/bin/do-server-backup; do
+    [[ -f "$f" ]] || continue
+    size=$(stat -c%s "$f" 2>/dev/null || echo 0)
+    [[ $size -lt 524288 ]] && cp "$f" scripts/ || log "  SKIP (too large): $f"
+done
+# composer is a stock PHP tool — skip it
+
+# ---------------------------------------------------------------------------
+# 3. Custom systemd service units (skip stock DO/system units)
+# ---------------------------------------------------------------------------
+log "Backing up custom systemd units"
+CUSTOM_UNITS="jarvis-agent.service fastapi_ssh_server.service"
+for unit in $CUSTOM_UNITS; do
+    src="/etc/systemd/system/$unit"
+    [[ -f "$src" ]] && cp "$src" systemd/ || true
+done
+
+# ---------------------------------------------------------------------------
+# 4. WireGuard configs (includes private keys — repo is private)
+# ---------------------------------------------------------------------------
+log "Backing up WireGuard configs"
+for f in /etc/wireguard/*.conf; do
+    [[ -f "$f" ]] && cp "$f" wireguard/ || true
+done
+
+# ---------------------------------------------------------------------------
+# 5. Network / netplan
+# ---------------------------------------------------------------------------
+log "Backing up netplan"
+for f in /etc/netplan/*.yaml; do
+    [[ -f "$f" ]] && cp "$f" network/ || true
+done
+cp /etc/hosts network/hosts 2>/dev/null || true
+cp /etc/hostname network/hostname 2>/dev/null || true
+
+# ---------------------------------------------------------------------------
+# 6. Root crontab — custom entries only (strip CyberPanel boilerplate)
+# ---------------------------------------------------------------------------
+log "Backing up crontab"
+crontab -l 2>/dev/null | grep -v "^#\|CyberCP\|acme.sh\|cleansessions\|run_scheduled_scans\|pdnsHealthCheck\|findBWUsage\|postfixSenderPolicy\|upgradeCritical\|renew\.py\|IncScheduler\|e2scrub\|imunify\|sessionclean\|lsws\b" \
+    | sed '/^[[:space:]]*$/d' > cron/root_custom
+# Also keep the full crontab for reference
+crontab -l 2>/dev/null > cron/root_full || echo "# no crontab" > cron/root_full
+
+# ---------------------------------------------------------------------------
+# 7. SSH authorized_keys
+# ---------------------------------------------------------------------------
+log "Backing up SSH keys"
+[[ -f /root/.ssh/authorized_keys ]] && cp /root/.ssh/authorized_keys ssh/ || true
+[[ -f /root/.ssh/id_rsa.pub ]]      && cp /root/.ssh/id_rsa.pub ssh/      || true
+
+# ---------------------------------------------------------------------------
+# 8. OpenLiteSpeed vhost configs (CyberPanel-managed)
+# ---------------------------------------------------------------------------
+log "Backing up OLS vhost configs"
+for vdir in /usr/local/lsws/conf/vhosts/*/; do
+    vname=$(basename "$vdir")
+    [[ "$vname" == "Example" ]] && continue
+    mkdir -p "ols-vhosts/$vname"
+    for conf in "$vdir"*.conf; do
+        [[ -f "$conf" ]] && cp "$conf" "ols-vhosts/$vname/" || true
+    done
+done
+# OLS main listener/vhost mapping
+grep -E "^\s*(listener|virtualHost|address |map |vhRoot|vhDomain|configFile)" \
+    /usr/local/lsws/conf/httpd_config.conf 2>/dev/null > ols-vhosts/httpd_vhosts_summary.txt || true
+
+# ---------------------------------------------------------------------------
+# 9. MySQL root credentials file
+# ---------------------------------------------------------------------------
+log "Backing up MySQL credentials"
+[[ -f /root/.my.cnf ]] && cp /root/.my.cnf mysql/my.cnf || true
+# Document all databases
+mysql -e "SHOW DATABASES;" 2>/dev/null | grep -v "^Database\|information_schema\|performance_schema\|sys" > mysql/databases.txt || true
+
+# ---------------------------------------------------------------------------
+# 10. /opt/infra snapshot (already a separate git repo — copy contents)
+# ---------------------------------------------------------------------------
+log "Backing up /opt/infra snapshot"
+if [[ -d /opt/infra ]]; then
+    rsync -a --exclude='.git' /opt/infra/ infra/
+fi
+
+# SMTP config docs
+if [[ -d /opt/smtp-for-websites ]]; then
+    mkdir -p smtp-docs
+    rsync -a --exclude='.git' /opt/smtp-for-websites/ smtp-docs/
+fi
+
+# ---------------------------------------------------------------------------
+# 11. CyberPanel website list (for documentation)
+# ---------------------------------------------------------------------------
+log "Documenting website list"
+{
+    echo "# Websites on DO server — $(date '+%Y-%m-%d')"
+    echo ""
+    for d in /home/*/public_html; do
+        site=$(echo "$d" | sed 's|/home/||;s|/public_html||')
+        diskuse=$(du -sh "$d" 2>/dev/null | cut -f1)
+        echo "- $site  ($diskuse)"
+    done
+} > ols-vhosts/site-list.txt
+
+# ---------------------------------------------------------------------------
+# 12. Commit and push
+# ---------------------------------------------------------------------------
+log "Committing changes"
+git add -A
+if git diff --cached --quiet; then
+    log "No changes to commit"
+else
+    CHANGES=$(git diff --cached --stat | tail -1)
+    git commit -m "[orbis] Weekly backup $(date '+%Y-%m-%d') — $CHANGES"
+    log "Pushing to GitHub"
+    git push origin main
+    log "Backup complete"
+fi
@@ -0,0 +1,354 @@
+#!/usr/bin/env bash
+# =============================================================================
+# DO Server Restore — run on a fresh DigitalOcean Ubuntu 24.04 droplet
+#
+# Usage:
+#   bash restore.sh
+#
+# Prerequisites:
+#   - Fresh Ubuntu 24.04 droplet at 165.22.1.228
+#   - Root SSH access
+#   - Run this script BEFORE installing CyberPanel (network/SSH steps)
+#     OR after CyberPanel install (full restore)
+# =============================================================================
+
+REPO_DIR="$(cd "$(dirname "$0")" && pwd)"
+
+RED='\033[0;31m'; YELLOW='\033[1;33m'; GREEN='\033[0;32m'; CYAN='\033[0;36m'; BOLD='\033[1m'; NC='\033[0m'
+
+header()  { echo -e "\n${CYAN}══════════════════════════════════════════════════${NC}"; echo -e "${CYAN}${BOLD}  $*${NC}"; echo -e "${CYAN}══════════════════════════════════════════════════${NC}"; }
+success() { echo -e "${GREEN}  ✓ $*${NC}"; }
+warn()    { echo -e "${YELLOW}  ⚠ $*${NC}"; }
+info()    { echo -e "  → $*"; }
+step()    { echo -e "\n${BOLD}  $*${NC}"; }
+die()     { echo -e "${RED}  ✗ $*${NC}" >&2; exit 1; }
+
+confirm() {
+    echo -e "\n${YELLOW}  $1${NC}"
+    read -rp "  Apply? [Y/n] " ans
+    [[ "${ans:-Y}" =~ ^[Yy]$ ]]
+}
+
+[[ $(id -u) -eq 0 ]] || die "Must run as root"
+
+# ---------------------------------------------------------------------------
+# Welcome
+# ---------------------------------------------------------------------------
+clear
+echo -e "${CYAN}"
+cat << 'BANNER'
+  ╔══════════════════════════════════════════════════════╗
+  ║     DO SERVER RESTORE — orbis.orbishosting.com       ║
+  ║     165.22.1.228  |  Ubuntu 24.04  |  CyberPanel     ║
+  ╚══════════════════════════════════════════════════════╝
+BANNER
+echo -e "${NC}"
+echo "  Source : $REPO_DIR"
+echo "  Date   : $(date)"
+echo ""
+warn "This script restores a fresh Ubuntu 24.04 droplet to full production."
+warn "It is interactive — confirm each step as you go."
+echo ""
+read -rp "  Type 'yes' to continue: " confirm_start
+[[ "$confirm_start" == "yes" ]] || { echo "Aborted."; exit 0; }
+
+APPLIED=(); SKIPPED=()
+
+# ---------------------------------------------------------------------------
+# PHASE 1: SSH & NETWORK
+# ---------------------------------------------------------------------------
+header "PHASE 1 — SSH & Network"
+
+step "1.1 SSH authorized_keys"
+if [[ -f "$REPO_DIR/ssh/authorized_keys" ]]; then
+    if confirm "Restore SSH authorized_keys?"; then
+        mkdir -p /root/.ssh && chmod 700 /root/.ssh
+        cp "$REPO_DIR/ssh/authorized_keys" /root/.ssh/authorized_keys
+        chmod 600 /root/.ssh/authorized_keys
+        success "authorized_keys restored"
+        APPLIED+=("ssh")
+    else; SKIPPED+=("ssh"); fi
+fi
+
+step "1.2 Hostname"
+if confirm "Set hostname to 'orbis'?"; then
+    hostnamectl set-hostname orbis
+    success "Hostname set"
+    APPLIED+=("hostname")
+else; SKIPPED+=("hostname"); fi
+
+step "1.3 WireGuard VPN"
+if [[ -d "$REPO_DIR/wireguard" ]] && confirm "Restore WireGuard configs and enable wg0?"; then
+    apt install -y wireguard -q 2>/dev/null
+    cp "$REPO_DIR/wireguard/"*.conf /etc/wireguard/ 2>/dev/null || true
+    chmod 600 /etc/wireguard/*.conf
+    systemctl enable wg-quick@wg0
+    systemctl start  wg-quick@wg0 2>/dev/null || warn "wg0 start failed — check /etc/wireguard/wg0.conf peers"
+    success "WireGuard configs restored"
+    APPLIED+=("wireguard")
+else; SKIPPED+=("wireguard"); fi
+
+# ---------------------------------------------------------------------------
+# PHASE 2: CyberPanel (manual step — cannot automate)
+# ---------------------------------------------------------------------------
+header "PHASE 2 — CyberPanel Install"
+
+echo ""
+warn "CyberPanel must be installed manually before continuing with web setup."
+warn "Run these commands in a separate terminal, then come back here."
+echo ""
+info "Install CyberPanel (OpenLiteSpeed, PHP, MySQL, Postfix, PowerDNS):"
+echo ""
+echo -e "  ${BOLD}sh <(curl https://cyberpanel.net/install.sh)${NC}"
+echo ""
+info "During install, choose:"
+info "  • OpenLiteSpeed (not LiteSpeed Enterprise)"
+info "  • Install Full service (PowerDNS, Postfix, Pure-FTPd)"
+info "  • Set admin password when prompted"
+info "  • PHP versions to install: 8.1, 8.2, 8.3, 8.4, 8.5"
+echo ""
+read -rp "  Press ENTER once CyberPanel is fully installed... "
+
+# ---------------------------------------------------------------------------
+# PHASE 3: MySQL
+# ---------------------------------------------------------------------------
+header "PHASE 3 — MySQL Credentials"
+
+step "3.1 MySQL root credentials"
+if [[ -f "$REPO_DIR/mysql/my.cnf" ]]; then
+    if confirm "Restore /root/.my.cnf (MySQL root credentials)?"; then
+        cp "$REPO_DIR/mysql/my.cnf" /root/.my.cnf
+        chmod 600 /root/.my.cnf
+        success "MySQL credentials restored"
+        APPLIED+=("mysql-creds")
+    else; SKIPPED+=("mysql-creds"); fi
+fi
+
+step "3.2 Restore databases"
+info "Databases to restore: $(cat $REPO_DIR/mysql/databases.txt 2>/dev/null | tr '\n' ' ')"
+echo ""
+warn "Databases are restored from the JARVIS daily backup (not this repo)."
+info "Latest backup is at: /var/backups/jarvis/ (if migrating from old server)"
+info "  OR download from the old server:"
+info "    scp root@165.22.1.228:/var/backups/jarvis/jarvis_backup_latest.tar.gz /tmp/"
+echo ""
+info "To restore manually after getting the backup file:"
+info "  tar -xzf jarvis_backup_*.tar.gz"
+info "  mysql -u root -p jarvis_db < sql/jarvis_db.sql"
+info "  mysql -u root -p toms_tjj_db < sql/toms_tjj_db.sql"
+info "  # (repeat for each database in sql/)"
+echo ""
+info "DB users/passwords are in each site's gitignored config file:"
+info "  jarvis_db      → jarvis_user / J4rv1s_Pr0t0c0l_2026!"
+info "  toms_tjj_db    → toms_tjj_user / +60wlPc+55e@gFq4"
+info "  tomt_ttg_db    → tomt_ttg_user / q#q+mrOcozsa7I6J"
+info "  epic_epic_db   → epic_epic_user / (see epictravelexpeditions api/config.php)"
+info "  epic_parkersling → epic_parkersling / Joker1974!!!"
+info "  parker_db      → parker_user / (see parkerslingshotrentals db config)"
+read -rp "  Press ENTER to continue... "
+
+# ---------------------------------------------------------------------------
+# PHASE 4: Websites
+# ---------------------------------------------------------------------------
+header "PHASE 4 — Websites (GitHub → server)"
+
+step "4.1 Create websites in CyberPanel"
+warn "Each site must be created in CyberPanel before pulling code."
+echo ""
+info "Go to CyberPanel → Websites → Create Website for each:"
+echo ""
+printf "  %-45s %s\n" "Domain" "PHP"
+printf "  %-45s %s\n" "------" "---"
+printf "  %-45s %s\n" "jarvis.orbishosting.com" "8.5"
+printf "  %-45s %s\n" "tomsjavajive.com" "8.5"
+printf "  %-45s %s\n" "epictravelexpeditions.com" "8.5"
+printf "  %-45s %s\n" "parkerslingshotrentals.com" "8.5"
+printf "  %-45s %s\n" "orbishosting.com" "8.5"
+printf "  %-45s %s\n" "orbis.orbishosting.com" "8.5"
+printf "  %-45s %s\n" "tomtomgames.com" "8.5"
+echo ""
+read -rp "  Press ENTER once all sites are created in CyberPanel... "
+
+step "4.2 Pull website code from GitHub"
+PAT="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"
+declare -A SITE_REPOS=(
+    ["jarvis.orbishosting.com"]="jarvis"
+    ["tomsjavajive.com"]="tomsjavajive"
+    ["epictravelexpeditions.com"]="epictravelexpeditions"
+    ["parkerslingshotrentals.com"]="parkerslingshotrentals"
+    ["orbishosting.com"]="orbishosting"
+    ["orbis.orbishosting.com"]="orbis-hosting-portal"
+    ["tomtomgames.com"]="tomtomgames"
+)
+
+if confirm "Pull all site repos from GitHub into /home/*/public_html/?"; then
+    for domain in "${!SITE_REPOS[@]}"; do
+        repo="${SITE_REPOS[$domain]}"
+        target="/home/$domain/public_html"
+        if [[ -d "$target" ]]; then
+            info "Cloning $repo → $target"
+            # Preserve existing CyberPanel-created files, then overlay git
+            tmpdir=$(mktemp -d)
+            git clone "https://${PAT}@github.com/myronblair/$repo.git" "$tmpdir" -q 2>&1
+            rsync -a --exclude='.git' "$tmpdir/" "$target/"
+            rm -rf "$tmpdir"
+            success "$domain pulled"
+        else
+            warn "  $target doesn't exist — create the site in CyberPanel first"
+        fi
+    done
+    # Parker Slingshot subfolder
+    if [[ -d /home/epictravelexpeditions.com ]]; then
+        info "Cloning parkerslingshot subfolder"
+        git clone "https://${PAT}@github.com/myronblair/parkerslingshot.git" \
+            /home/epictravelexpeditions.com/parkerslingshot -q 2>&1 && \
+            success "parkerslingshot cloned" || warn "parkerslingshot clone failed"
+    fi
+    APPLIED+=("websites")
+else; SKIPPED+=("websites"); fi
+
+step "4.3 Restore gitignored config files"
+warn "These files contain credentials and are NOT in GitHub."
+warn "They must be manually recreated or copied from backup."
+echo ""
+info "Required config files:"
+info "  /home/jarvis.orbishosting.com/api/config.php  (all JARVIS credentials)"
+info "  /home/tomsjavajive.com/config/database.php    (TJJ database)"
+info "  /home/epictravelexpeditions.com/api/config.php"
+info "  /home/parkerslingshotrentals.com/admin/config.php (if separate)"
+echo ""
+info "Reference templates are in each repo's api/config.php.example or similar."
+info "DB creds are documented above in step 3.2."
+read -rp "  Press ENTER once config files are restored... "
+
+# ---------------------------------------------------------------------------
+# PHASE 5: Custom Scripts & Services
+# ---------------------------------------------------------------------------
+header "PHASE 5 — Custom Scripts, Systemd, Cron"
+
+step "5.1 Custom scripts to /usr/local/bin"
+if [[ -d "$REPO_DIR/scripts" ]] && confirm "Restore custom scripts?"; then
+    cp "$REPO_DIR/scripts/"* /usr/local/bin/ 2>/dev/null || true
+    chmod +x /usr/local/bin/jarvis-*.sh /usr/local/bin/jarvis-*.py \
+             /usr/local/bin/ttg-backup.sh /usr/local/bin/do-server-backup 2>/dev/null || true
+    success "Scripts restored"
+    APPLIED+=("scripts")
+else; SKIPPED+=("scripts"); fi
+
+step "5.2 Systemd service units"
+if [[ -d "$REPO_DIR/systemd" ]] && confirm "Restore and enable custom systemd units?"; then
+    for unit in "$REPO_DIR/systemd/"*.service; do
+        [[ -f "$unit" ]] || continue
+        bname=$(basename "$unit")
+        cp "$unit" /etc/systemd/system/
+        systemctl enable "$bname" 2>/dev/null || true
+        info "  Enabled: $bname"
+    done
+    systemctl daemon-reload
+    success "Systemd units restored and enabled"
+    APPLIED+=("systemd")
+    warn "jarvis-agent will fail until JARVIS DB is running — start it after DB restore"
+else; SKIPPED+=("systemd"); fi
+
+step "5.3 Root crontab (custom entries)"
+if [[ -f "$REPO_DIR/cron/root_custom" ]] && confirm "Restore custom crontab entries?"; then
+    # Merge with existing CyberPanel crontab
+    CURRENT=$(crontab -l 2>/dev/null || true)
+    CUSTOM=$(cat "$REPO_DIR/cron/root_custom")
+    # Deduplicate
+    { echo "$CURRENT"; echo ""; echo "# --- Restored custom entries ---"; echo "$CUSTOM"; } | \
+        sort -u | crontab -
+    success "Custom crontab entries restored"
+    info "Review with: crontab -l"
+    APPLIED+=("crontab")
+else; SKIPPED+=("crontab"); fi
+
+# ---------------------------------------------------------------------------
+# PHASE 6: OLS Vhost Configs
+# ---------------------------------------------------------------------------
+header "PHASE 6 — OpenLiteSpeed Vhost Configs"
+
+if [[ -d "$REPO_DIR/ols-vhosts" ]]; then
+    step "6.1 Restore OLS vhost config files"
+    warn "CyberPanel manages these files — overwriting may cause conflicts."
+    warn "Only do this if CyberPanel's auto-generated configs are wrong."
+    echo ""
+    info "Vhost configs backed up:"
+    ls "$REPO_DIR/ols-vhosts/" | grep -v "^httpd\|^site-list" | sed 's/^/    /'
+    echo ""
+    if confirm "Restore OLS vhost .conf files to /usr/local/lsws/conf/vhosts/?"; then
+        for vdir in "$REPO_DIR/ols-vhosts/"/*/; do
+            vname=$(basename "$vdir")
+            [[ -d "/usr/local/lsws/conf/vhosts/$vname" ]] || mkdir -p "/usr/local/lsws/conf/vhosts/$vname"
+            cp "$vdir"*.conf "/usr/local/lsws/conf/vhosts/$vname/" 2>/dev/null || true
+            info "  Restored: $vname"
+        done
+        systemctl reload lsws 2>/dev/null || systemctl restart lsws 2>/dev/null
+        success "OLS vhost configs restored and OLS reloaded"
+        APPLIED+=("ols-vhosts")
+    else
+        SKIPPED+=("ols-vhosts")
+        info "Skipped — CyberPanel will auto-generate vhost configs when sites are created"
+    fi
+fi
+
+# ---------------------------------------------------------------------------
+# PHASE 7: SSL / HTTPS
+# ---------------------------------------------------------------------------
+header "PHASE 7 — SSL Certificates"
+
+echo ""
+info "SSL certs are NOT backed up here (they're in /etc/letsencrypt/)."
+info "Re-issue via CyberPanel → SSL → Issue SSL for each domain."
+info "  OR run: certbot certonly --webroot -w /home/<domain>/public_html -d <domain>"
+echo ""
+info "After SSL is issued, each site's OLS vhost will auto-update."
+read -rp "  Press ENTER to continue... "
+
+# ---------------------------------------------------------------------------
+# PHASE 8: JARVIS-Specific
+# ---------------------------------------------------------------------------
+header "PHASE 8 — JARVIS Services"
+
+step "8.1 JARVIS deploy + watchdog scripts"
+info "jarvis-deploy.sh and jarvis-watchdog.sh are already restored via Step 5.1"
+info "Verify cron entries are active: crontab -l | grep jarvis"
+
+step "8.2 JARVIS agent"
+info "Start JARVIS agent service:"
+info "  systemctl start jarvis-agent"
+info "  journalctl -u jarvis-agent -f"
+
+step "8.3 Backup script"
+info "Install this backup script as /usr/local/bin/do-server-backup:"
+if confirm "Install do-server-backup cron (weekly Sunday 4am)?"; then
+    cp "$REPO_DIR/backup.sh" /usr/local/bin/do-server-backup
+    chmod +x /usr/local/bin/do-server-backup
+    (crontab -l 2>/dev/null | grep -v do-server-backup; \
+     echo "0 4 * * 0 /usr/local/bin/do-server-backup >> /var/log/do-server-backup.log 2>&1") | crontab -
+    success "Backup cron installed"
+    APPLIED+=("backup-cron")
+else; SKIPPED+=("backup-cron"); fi
+
+# ---------------------------------------------------------------------------
+# Summary
+# ---------------------------------------------------------------------------
+header "Restore Complete"
+echo ""
+[[ ${#APPLIED[@]} -gt 0 ]] && success "Applied: ${APPLIED[*]}"
+[[ ${#SKIPPED[@]} -gt 0 ]] && warn    "Skipped: ${SKIPPED[*]}"
+echo ""
+echo -e "${YELLOW}  Final checklist:${NC}"
+echo "  • Test https://jarvis.orbishosting.com"
+echo "  • Test https://tomsjavajive.com"
+echo "  • Check OLS status: systemctl status lsws"
+echo "  • Check MySQL: systemctl status mysql"
+echo "  • Check JARVIS agent: systemctl status jarvis-agent"
+echo "  • Verify webhook: curl -s https://jarvis.orbishosting.com/webhook.php"
+echo "  • Check WireGuard: wg show"
+echo "  • Confirm crons: crontab -l"
+echo "  • Point GitHub webhooks to new server IP (if IP changed)"
+echo ""
+echo "  CyberPanel admin: https://165.22.1.228:8090"
+echo "  JARVIS: https://jarvis.orbishosting.com"
+echo ""