commit 04fc9941ff5cb77a995fc574af26d5e276fa25d8 Author: Myron Blair Date: Tue Jun 9 03:42:39 2026 +0000 Initial: backup/restore scripts, README - backup.sh: weekly cron script for PVE1+PVE2 - restore.sh: interactive disaster recovery wizard - README.md: step-by-step recovery guide for both nodes Co-Authored-By: Claude Sonnet 4.6 diff --git a/.gitignore b/.gitignore new file mode 100644 index 0000000..69eff5a --- /dev/null +++ b/.gitignore @@ -0,0 +1,9 @@ +# Never store private keys or secrets +*.key +priv/ +id_rsa +*.pem +*.pfx + +# Logs +*.log diff --git a/README.md b/README.md new file mode 100644 index 0000000..e4dd02e --- /dev/null +++ b/README.md @@ -0,0 +1,176 @@ +# Proxmox Config Backup & Restore + +Automated weekly backup of PVE1 and PVE2 configuration to GitHub. +In a host failure, this gets you back up in 15–30 minutes. + +--- + +## Repository Structure + +``` +proxmox-config/ + backup.sh ← runs on each node weekly, commits changes + restore.sh ← interactive restore on a fresh Proxmox install + README.md ← this file + shared/ ← cluster-wide configs (backed up from PVE1) + storage.cfg — storage definitions (local-lvm, GoFlex, ProxBackups) + datacenter.cfg — cluster settings + user.cfg — users and API tokens + corosync.conf — cluster membership + jobs.cfg — backup job schedules + vzdump.cron — vzdump cron config + firewall/ — cluster firewall rules + ha/ — HA group/resource configs + nodes/ + pve/ + qemu-server/ — PVE1 VM .conf files (100–120) + lxc/ — PVE1 LXC container configs (110) + pve2/ + qemu-server/ — PVE2 VM .conf files (106, 302) + pve/ ← PVE1-specific (hostname: pve, 10.48.200.90) + network/ — /etc/network/interfaces + cron/ — root crontab + scripts/ — custom shell/python scripts + systemd/ — custom service units + jarvis-agent/ — JARVIS monitoring agent config + ssh/ — authorized_keys (no private keys) + pve2/ ← PVE2-specific (hostname: pve2, 10.48.200.91) + (same structure) +``` + +--- + +## Backup Schedule + +Both nodes run `backup.sh` via cron every **Sunday at 3:00 AM**. + +``` +0 3 * * 0 /usr/local/bin/proxmox-backup >> /var/log/proxmox-backup.log 2>&1 +``` + +The script: +- Pulls latest from GitHub (handles both nodes committing) +- Collects all config files listed above +- Skips large binaries (ollama, filebrowser — see reinstall notes below) +- Skips private keys (`/etc/pve/priv/`, SSH private keys) +- Commits with a timestamp and pushes + +--- + +## Disaster Recovery — Host Failure + +### Scenario A: PVE1 (primary) fails + +PVE1 hosts most VMs (100–113, 118, 120). Storage for those VMs is on `local-lvm` (PVE1's LVM thin pool) or `GoFlex` NAS. VMs on local-lvm **are at risk** if the disk fails — restore from PBS. + +**Steps:** + +1. **Install fresh Proxmox** on replacement hardware. + - Set IP to `10.48.200.90` during install, or set after. + - Set hostname to `pve`. + +2. **Clone this repo:** + ```bash + apt install -y git + git clone https://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@github.com/myronblair/proxmox-config.git /opt/proxmox-config + ``` + +3. **Run the restore script:** + ```bash + bash /opt/proxmox-config/restore.sh pve1 + ``` + The script is interactive — confirm each section as it goes. + +4. **Reboot** to apply network config: + ```bash + reboot + ``` + +5. **Restore VMs from PBS** (Proxmox Backup Server at `10.48.200.85`): + - Log in to PVE web UI: `https://10.48.200.90:8006` + - Datacenter → Storage → Add → Proxmox Backup Server + - Server: `10.48.200.85` + - Datastore: `PBSBackup` + - Username: `root@pam` + - Fingerprint: see `shared/pbs_fingerprint.txt` + - For each VM: select the backup → Restore + +6. **Reinstall large binaries** (not in git — too large): + ```bash + # Ollama (local LLM on VM 210 inside this host) + curl -fsSL https://ollama.ai/install.sh | sh + ollama pull llama3.2 + + # File Browser + curl -fsSL https://raw.githubusercontent.com/filebrowser/get/master/get.sh | bash + # Config is at /root/.filebrowser.json or wherever it was set + ``` + +7. **Start services:** + ```bash + systemctl start jarvis-agent filebrowser + ``` + +--- + +### Scenario B: PVE2 (secondary) fails + +PVE2 hosts VM 106 (unknown) and VM 302 (NetworkBackup). PVE2 uses `local-lvm` for storage. + +**Steps:** + +1. **Install fresh Proxmox**, set IP `10.48.200.91`, hostname `pve2`. + +2. **Clone repo and run restore:** + ```bash + apt install -y git + git clone https://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@github.com/myronblair/proxmox-config.git /opt/proxmox-config + bash /opt/proxmox-config/restore.sh pve2 + ``` + +3. **Join the cluster from PVE1:** + ```bash + # On PVE1: + pvecm add 10.48.200.91 + # This auto-syncs all cluster state to PVE2 + ``` + +4. **Restore VMs 106 and 302 from PBS.** + +5. **Reboot PVE2**, then verify cluster: `pvecm status` + +--- + +## What Is NOT Backed Up Here + +| Item | Where to find it | +|------|-----------------| +| VM disk images | Proxmox Backup Server (10.48.200.85) — runs nightly | +| /etc/pve/priv/ | Private CA key, auth keys — DO NOT store in git; regenerated by Proxmox on fresh install | +| SSH private keys | /root/.ssh/id_rsa — regenerate or restore from secure storage | +| ollama binary | Reinstall via install script (see above) | +| filebrowser binary | Reinstall via get.sh (see above) | +| blockalign, urbackupclientctl | Reinstall from their respective sources if needed | + +--- + +## Nodes Quick Reference + +| Node | Hostname | IP | Port | Role | +|------|----------|----|------|------| +| PVE1 | pve | 10.48.200.90 | 8006 | Primary — most VMs | +| PVE2 | pve2 | 10.48.200.91 | 8006 | Secondary — VM 106, 302 | +| PBS | — | 10.48.200.85 | 8007 | Backup server | + +Proxmox web login: `root` / `Joker1974!!!` +FortiGate DDNS (PVE1 from internet): `orbisne.fortiddns.com` + +--- + +## Manual Backup Trigger + +To run a backup immediately on either node: +```bash +/usr/local/bin/proxmox-backup +tail -f /var/log/proxmox-backup.log +``` diff --git a/backup.sh b/backup.sh new file mode 100644 index 0000000..837da17 --- /dev/null +++ b/backup.sh @@ -0,0 +1,161 @@ +#!/usr/bin/env bash +# ============================================================================= +# Proxmox Config Backup — runs on PVE1 or PVE2 +# Backs up all critical configs to GitHub weekly +# Install: /usr/local/bin/proxmox-backup +# Cron: 0 3 * * 0 /usr/local/bin/proxmox-backup >> /var/log/proxmox-backup.log 2>&1 +# ============================================================================= +set -euo pipefail + +PAT="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" +REPO_URL="https://${PAT}@github.com/myronblair/proxmox-config.git" +REPO_DIR="/opt/proxmox-config" +NODE=$(hostname) +LOG_PREFIX="[$(date '+%Y-%m-%d %H:%M:%S')] [$NODE]" + +log() { echo "$LOG_PREFIX $*"; } +die() { echo "$LOG_PREFIX ERROR: $*" >&2; exit 1; } + +# --------------------------------------------------------------------------- +# 1. Clone or update local repo +# --------------------------------------------------------------------------- +if [[ -d "$REPO_DIR/.git" ]]; then + log "Pulling latest from GitHub" + cd "$REPO_DIR" + git config user.email "proxmox-backup@orbishosting.com" + git config user.name "Proxmox Backup" + git pull --rebase origin main -q || { log "Pull failed — continuing with local state"; true; } +else + log "Cloning repo to $REPO_DIR" + git clone "$REPO_URL" "$REPO_DIR" + cd "$REPO_DIR" + git config user.email "proxmox-backup@orbishosting.com" + git config user.name "Proxmox Backup" +fi + +cd "$REPO_DIR" + +# --------------------------------------------------------------------------- +# 2. Node-specific directories +# --------------------------------------------------------------------------- +NODE_DIR="$REPO_DIR/$NODE" +mkdir -p \ + "$NODE_DIR/network" \ + "$NODE_DIR/cron" \ + "$NODE_DIR/scripts" \ + "$NODE_DIR/systemd" \ + "$NODE_DIR/jarvis-agent" + +# --------------------------------------------------------------------------- +# 3. Shared PVE cluster configs (only back up from primary node to avoid +# conflicts — PVE1/pve is the primary; pve2 skips this section) +# --------------------------------------------------------------------------- +if [[ "$NODE" == "pve" ]]; then + log "Backing up shared PVE cluster configs" + SHARED_DIR="$REPO_DIR/shared" + mkdir -p \ + "$SHARED_DIR/firewall" \ + "$SHARED_DIR/ha" \ + "$SHARED_DIR/sdn" \ + "$SHARED_DIR/nodes/pve/qemu-server" \ + "$SHARED_DIR/nodes/pve/lxc" \ + "$SHARED_DIR/nodes/pve2/qemu-server" \ + "$SHARED_DIR/nodes/pve2/lxc" + + # Top-level cluster configs + for f in storage.cfg datacenter.cfg user.cfg corosync.conf vzdump.cron jobs.cfg ceph.conf replication.cfg; do + [[ -f "/etc/pve/$f" ]] && cp "/etc/pve/$f" "$SHARED_DIR/$f" || true + done + + # Firewall + [[ -d /etc/pve/firewall ]] && rsync -a --delete /etc/pve/firewall/ "$SHARED_DIR/firewall/" 2>/dev/null || true + + # HA + [[ -d /etc/pve/ha ]] && rsync -a --delete /etc/pve/ha/ "$SHARED_DIR/ha/" 2>/dev/null || true + + # SDN + [[ -d /etc/pve/sdn ]] && rsync -a --delete /etc/pve/sdn/ "$SHARED_DIR/sdn/" 2>/dev/null || true + + # VM/LXC configs (per-node) + for PVENODE in pve pve2; do + [[ -d "/etc/pve/nodes/$PVENODE/qemu-server" ]] && \ + rsync -a --delete /etc/pve/nodes/$PVENODE/qemu-server/ "$SHARED_DIR/nodes/$PVENODE/qemu-server/" 2>/dev/null || true + [[ -d "/etc/pve/nodes/$PVENODE/lxc" ]] && \ + rsync -a --delete /etc/pve/nodes/$PVENODE/lxc/ "$SHARED_DIR/nodes/$PVENODE/lxc/" 2>/dev/null || true + done + + # PBS fingerprint (needed for restore) + grep -A5 "pbs: ProxBackups" /etc/pve/storage.cfg > "$SHARED_DIR/pbs_fingerprint.txt" 2>/dev/null || true +fi + +# --------------------------------------------------------------------------- +# 4. Network config +# --------------------------------------------------------------------------- +log "Backing up network config" +cp /etc/network/interfaces "$NODE_DIR/network/interfaces" +[[ -d /etc/network/interfaces.d ]] && \ + rsync -a --delete /etc/network/interfaces.d/ "$NODE_DIR/network/interfaces.d/" 2>/dev/null || true +cp /etc/hosts "$NODE_DIR/network/hosts" +cp /etc/hostname "$NODE_DIR/network/hostname" +[[ -f /etc/resolv.conf ]] && cp /etc/resolv.conf "$NODE_DIR/network/resolv.conf" || true + +# --------------------------------------------------------------------------- +# 5. Root crontab +# --------------------------------------------------------------------------- +log "Backing up crontab" +crontab -l 2>/dev/null > "$NODE_DIR/cron/root" || echo "# no crontab" > "$NODE_DIR/cron/root" +[[ -f /etc/vzdump.conf ]] && cp /etc/vzdump.conf "$NODE_DIR/vzdump.conf" || true + +# --------------------------------------------------------------------------- +# 6. Custom scripts (text only — skip large binaries) +# --------------------------------------------------------------------------- +log "Backing up custom scripts" +for f in /usr/local/bin/*.sh /usr/local/bin/*.py /usr/local/bin/*.pl; do + [[ -f "$f" ]] || continue + size=$(stat -c%s "$f" 2>/dev/null || echo 0) + if [[ $size -lt 524288 ]]; then # skip files > 512KB (binaries) + cp "$f" "$NODE_DIR/scripts/" + fi +done + +# --------------------------------------------------------------------------- +# 7. Custom systemd units (only non-stock units) +# --------------------------------------------------------------------------- +log "Backing up custom systemd units" +STOCK_UNITS="chronyd.service iscsi.service pve-manager.service smartd.service sshd.service zed.service" +for f in /etc/systemd/system/*.service; do + [[ -f "$f" ]] || continue + bname=$(basename "$f") + if ! echo "$STOCK_UNITS" | grep -qw "$bname"; then + cp "$f" "$NODE_DIR/systemd/" + fi +done + +# --------------------------------------------------------------------------- +# 8. JARVIS agent config +# --------------------------------------------------------------------------- +log "Backing up JARVIS agent config" +[[ -f /opt/jarvis-agent/config.json ]] && \ + cp /opt/jarvis-agent/config.json "$NODE_DIR/jarvis-agent/config.json" || true + +# --------------------------------------------------------------------------- +# 9. SSH authorized_keys (for access restoration — no private keys) +# --------------------------------------------------------------------------- +mkdir -p "$NODE_DIR/ssh" +[[ -f /root/.ssh/authorized_keys ]] && cp /root/.ssh/authorized_keys "$NODE_DIR/ssh/authorized_keys" || true +[[ -f /root/.ssh/id_rsa.pub ]] && cp /root/.ssh/id_rsa.pub "$NODE_DIR/ssh/id_rsa.pub" || true + +# --------------------------------------------------------------------------- +# 10. Commit and push +# --------------------------------------------------------------------------- +log "Committing changes" +git add -A +if git diff --cached --quiet; then + log "No changes to commit" +else + CHANGES=$(git diff --cached --stat | tail -1) + git commit -m "[$NODE] Weekly backup $(date '+%Y-%m-%d') — $CHANGES" + log "Pushing to GitHub" + git push origin main + log "Backup complete" +fi diff --git a/restore.sh b/restore.sh new file mode 100644 index 0000000..08877ff --- /dev/null +++ b/restore.sh @@ -0,0 +1,303 @@ +#!/usr/bin/env bash +# ============================================================================= +# Proxmox Config Restore — run on a freshly installed Proxmox node +# +# Usage: +# bash restore.sh pve1 — restore PVE1 (primary, hostname "pve") +# bash restore.sh pve2 — restore PVE2 (secondary, hostname "pve2") +# +# Prerequisites on fresh Proxmox install: +# apt install -y git +# git clone https://@github.com/myronblair/proxmox-config.git /opt/proxmox-config +# bash /opt/proxmox-config/restore.sh pve1 +# ============================================================================= + +TARGET="${1:-}" +REPO_DIR="$(cd "$(dirname "$0")" && pwd)" + +RED='\033[0;31m'; YELLOW='\033[1;33m'; GREEN='\033[0;32m'; CYAN='\033[0;36m'; NC='\033[0m' + +header() { echo -e "\n${CYAN}══════════════════════════════════════${NC}"; echo -e "${CYAN} $*${NC}"; echo -e "${CYAN}══════════════════════════════════════${NC}"; } +success() { echo -e "${GREEN} ✓ $*${NC}"; } +warn() { echo -e "${YELLOW} ⚠ $*${NC}"; } +info() { echo -e " → $*"; } +die() { echo -e "${RED} ✗ $*${NC}" >&2; exit 1; } + +confirm() { + local msg="$1" + echo -e "\n${YELLOW} $msg${NC}" + read -rp " Apply this section? [Y/n] " ans + [[ "${ans:-Y}" =~ ^[Yy]$ ]] +} + +# --------------------------------------------------------------------------- +# Validate arguments +# --------------------------------------------------------------------------- +if [[ -z "$TARGET" ]]; then + echo "Usage: $0 pve1|pve2" + echo "" + echo " pve1 — restore primary node (hostname: pve, IP: 10.48.200.90)" + echo " pve2 — restore secondary node (hostname: pve2, IP: 10.48.200.91)" + exit 1 +fi + +if [[ "$TARGET" == "pve1" ]]; then + NODE_DIR="$REPO_DIR/pve" + EXPECTED_HOSTNAME="pve" + EXPECTED_IP="10.48.200.90" +elif [[ "$TARGET" == "pve2" ]]; then + NODE_DIR="$REPO_DIR/pve2" + EXPECTED_HOSTNAME="pve2" + EXPECTED_IP="10.48.200.91" +else + die "Unknown target: $TARGET. Use pve1 or pve2." +fi + +SHARED_DIR="$REPO_DIR/shared" + +[[ -d "$NODE_DIR" ]] || die "Node directory not found: $NODE_DIR" +[[ $(id -u) -eq 0 ]] || die "Must run as root" + +# --------------------------------------------------------------------------- +# Welcome banner +# --------------------------------------------------------------------------- +clear +echo -e "${CYAN}" +cat << 'BANNER' + ╔═══════════════════════════════════════════════════════╗ + ║ PROXMOX CONFIG RESTORE — OrbiShosting ║ + ╚═══════════════════════════════════════════════════════╝ +BANNER +echo -e "${NC}" +echo " Target : $TARGET ($EXPECTED_HOSTNAME — $EXPECTED_IP)" +echo " Source : $REPO_DIR" +echo " Date : $(date)" +echo "" +warn "This script restores configs to a FRESH Proxmox install." +warn "Running on an existing node may overwrite running config." +echo "" +read -rp " Type 'yes' to continue: " confirm_start +[[ "$confirm_start" == "yes" ]] || { echo "Aborted."; exit 0; } + +APPLIED=() +SKIPPED=() + +# --------------------------------------------------------------------------- +# SECTION 1: Hostname +# --------------------------------------------------------------------------- +header "1 / 8 — Hostname" +info "Will set hostname to: $EXPECTED_HOSTNAME" +[[ -f "$NODE_DIR/network/hostname" ]] && info "Backed-up hostname: $(cat $NODE_DIR/network/hostname)" +if confirm "Set hostname to $EXPECTED_HOSTNAME?"; then + echo "$EXPECTED_HOSTNAME" > /etc/hostname + hostname "$EXPECTED_HOSTNAME" + success "Hostname set to $EXPECTED_HOSTNAME" + APPLIED+=("hostname") +else + SKIPPED+=("hostname") +fi + +# --------------------------------------------------------------------------- +# SECTION 2: Network interfaces +# --------------------------------------------------------------------------- +header "2 / 8 — Network Configuration" +if [[ -f "$NODE_DIR/network/interfaces" ]]; then + info "Current /etc/network/interfaces:" + cat /etc/network/interfaces | sed 's/^/ /' + info "" + info "Backed-up interfaces:" + cat "$NODE_DIR/network/interfaces" | sed 's/^/ /' + warn "Applying network config requires a reboot to take effect." + if confirm "Restore network interfaces?"; then + cp /etc/network/interfaces "/etc/network/interfaces.bak.$(date +%s)" + cp "$NODE_DIR/network/interfaces" /etc/network/interfaces + [[ -d "$NODE_DIR/network/interfaces.d" ]] && \ + rsync -a "$NODE_DIR/network/interfaces.d/" /etc/network/interfaces.d/ + success "Network interfaces restored (reboot required)" + APPLIED+=("network") + else + SKIPPED+=("network") + fi +else + warn "No network interfaces backup found — skipping" + SKIPPED+=("network") +fi + +# --------------------------------------------------------------------------- +# SECTION 3: /etc/hosts +# --------------------------------------------------------------------------- +header "3 / 8 — /etc/hosts" +if [[ -f "$NODE_DIR/network/hosts" ]]; then + if confirm "Restore /etc/hosts?"; then + cp /etc/hosts "/etc/hosts.bak.$(date +%s)" + cp "$NODE_DIR/network/hosts" /etc/hosts + success "/etc/hosts restored" + APPLIED+=("hosts") + else + SKIPPED+=("hosts") + fi +fi + +# --------------------------------------------------------------------------- +# SECTION 4: Custom scripts +# --------------------------------------------------------------------------- +header "4 / 8 — Custom Scripts (/usr/local/bin)" +if [[ -d "$NODE_DIR/scripts" ]] && [[ -n "$(ls -A $NODE_DIR/scripts 2>/dev/null)" ]]; then + info "Scripts to restore:" + ls "$NODE_DIR/scripts" | sed 's/^/ /' + if confirm "Restore custom scripts to /usr/local/bin/?"; then + cp "$NODE_DIR/scripts/"* /usr/local/bin/ 2>/dev/null || true + chmod +x /usr/local/bin/*.sh /usr/local/bin/*.py 2>/dev/null || true + success "Scripts restored" + APPLIED+=("scripts") + if [[ "$TARGET" == "pve1" ]]; then + warn "Large binaries NOT restored (ollama, filebrowser, etc.)" + warn "Reinstall manually:" + warn " ollama : curl -fsSL https://ollama.ai/install.sh | sh" + warn " filebrowser: curl -fsSL https://raw.githubusercontent.com/filebrowser/get/master/get.sh | bash" + fi + else + SKIPPED+=("scripts") + fi +else + info "No custom scripts backed up" +fi + +# --------------------------------------------------------------------------- +# SECTION 5: Systemd units +# --------------------------------------------------------------------------- +header "5 / 8 — Systemd Service Units" +if [[ -d "$NODE_DIR/systemd" ]] && [[ -n "$(ls -A $NODE_DIR/systemd 2>/dev/null)" ]]; then + info "Units to restore:" + ls "$NODE_DIR/systemd" | sed 's/^/ /' + if confirm "Restore and enable systemd units?"; then + for unit in "$NODE_DIR/systemd/"*.service; do + [[ -f "$unit" ]] || continue + bname=$(basename "$unit") + cp "$unit" /etc/systemd/system/ + systemctl enable "$bname" 2>/dev/null || true + info " Enabled: $bname" + done + systemctl daemon-reload + success "Systemd units restored and enabled" + APPLIED+=("systemd") + warn "Units are ENABLED but not started — start manually after verifying prerequisites" + warn " e.g.: systemctl start jarvis-agent filebrowser" + else + SKIPPED+=("systemd") + fi +fi + +# --------------------------------------------------------------------------- +# SECTION 6: JARVIS agent +# --------------------------------------------------------------------------- +header "6 / 8 — JARVIS Monitoring Agent" +if [[ -f "$NODE_DIR/jarvis-agent/config.json" ]]; then + info "Agent config found:" + cat "$NODE_DIR/jarvis-agent/config.json" | sed 's/^/ /' + if confirm "Restore JARVIS agent config?"; then + mkdir -p /opt/jarvis-agent + cp "$NODE_DIR/jarvis-agent/config.json" /opt/jarvis-agent/config.json + + # Install agent if not present + if [[ ! -f /opt/jarvis-agent/jarvis-agent.py ]]; then + info "Downloading JARVIS agent..." + curl -sk https://jarvis.orbishosting.com/install-agent.sh | bash -s "$(hostname)" proxmox || \ + warn "Auto-install failed — manually install: curl -sk https://jarvis.orbishosting.com/install-agent.sh | bash -s $(hostname) proxmox" + fi + success "JARVIS agent restored" + APPLIED+=("jarvis-agent") + else + SKIPPED+=("jarvis-agent") + fi +fi + +# --------------------------------------------------------------------------- +# SECTION 7: Root crontab +# --------------------------------------------------------------------------- +header "7 / 8 — Root Crontab" +if [[ -f "$NODE_DIR/cron/root" ]] && ! grep -q "^# no crontab" "$NODE_DIR/cron/root"; then + info "Cron entries to restore:" + cat "$NODE_DIR/cron/root" | sed 's/^/ /' + if confirm "Restore root crontab?"; then + crontab "$NODE_DIR/cron/root" + success "Crontab restored" + APPLIED+=("crontab") + else + SKIPPED+=("crontab") + fi +else + info "No crontab entries backed up — skipping" +fi + +# --------------------------------------------------------------------------- +# SECTION 8: PVE cluster configs (pve1 only — pve2 joins cluster instead) +# --------------------------------------------------------------------------- +header "8 / 8 — PVE Cluster / VM Configs" +if [[ "$TARGET" == "pve1" ]] && [[ -d "$SHARED_DIR" ]]; then + echo "" + warn "PVE cluster configs (storage, VM configs, users, firewall) are" + warn "managed by Proxmox's cluster filesystem (/etc/pve/)." + warn "" + warn "IMPORTANT: Do NOT copy /etc/pve/ files directly on a fresh install." + warn "Proxmox will auto-populate these when the node is online." + warn "" + info "Instead, after Proxmox is installed and running:" + info " 1. Restore VMs from Proxmox Backup Server (PBS) at 10.48.200.85" + info " → In PVE web UI: Datacenter → Storage → ProxBackups" + info " → PBS fingerprint: $(grep 'fingerprint' $SHARED_DIR/storage.cfg 2>/dev/null | head -1 | tr -d ' \t' || echo 'see shared/storage.cfg')" + info "" + info " 2. Re-add storage manually if PBS isn't auto-discovered:" + info " → Datacenter → Storage → Add → Proxmox Backup Server" + info "" + info " 3. VM configs for reference (not to copy directly):" + [[ -d "$SHARED_DIR/nodes/pve/qemu-server" ]] && ls "$SHARED_DIR/nodes/pve/qemu-server/" | sed 's/^/ /' + echo "" + if confirm "Copy VM .conf files to /etc/pve/nodes/pve/qemu-server/ (only do this if cluster is NOT active)?"; then + mkdir -p /etc/pve/nodes/pve/qemu-server /etc/pve/nodes/pve/lxc + cp "$SHARED_DIR/nodes/pve/qemu-server/"*.conf /etc/pve/nodes/pve/qemu-server/ 2>/dev/null || true + cp "$SHARED_DIR/nodes/pve/lxc/"*.conf /etc/pve/nodes/pve/lxc/ 2>/dev/null || true + cp "$SHARED_DIR/storage.cfg" /etc/pve/ 2>/dev/null || true + cp "$SHARED_DIR/datacenter.cfg" /etc/pve/ 2>/dev/null || true + success "PVE configs copied — restart pve-cluster to apply" + systemctl restart pve-cluster 2>/dev/null || true + APPLIED+=("pve-configs") + else + SKIPPED+=("pve-configs") + info "Skipped — restore VMs from PBS backup instead (recommended)" + fi + +elif [[ "$TARGET" == "pve2" ]]; then + warn "PVE2 does not hold the primary cluster config." + info "To restore PVE2 as a cluster member:" + info " 1. Fresh Proxmox install with hostname 'pve2' and IP 10.48.200.91" + info " 2. On PVE1: pvecm add 10.48.200.91" + info " 3. PVE2 will auto-sync all VM configs and cluster state from PVE1" + info " 4. VM 106 and 302 will reappear automatically" + SKIPPED+=("pve-configs") +fi + +# --------------------------------------------------------------------------- +# Summary +# --------------------------------------------------------------------------- +header "Restore Complete" +echo "" +if [[ ${#APPLIED[@]} -gt 0 ]]; then + success "Applied sections: ${APPLIED[*]}" +fi +if [[ ${#SKIPPED[@]} -gt 0 ]]; then + warn "Skipped sections: ${SKIPPED[*]}" +fi +echo "" +echo -e "${YELLOW} Next steps:${NC}" +if [[ " ${APPLIED[*]} " =~ " network " ]]; then + echo " • REBOOT to apply network changes: reboot" +fi +echo " • Verify Proxmox web UI at https://$EXPECTED_IP:8006" +echo " • Restore VMs from PBS backup server at https://10.48.200.85:8007" +echo " • Start JARVIS agent: systemctl start jarvis-agent" +if [[ "$TARGET" == "pve1" ]]; then + echo " • Reinstall ollama if needed: curl -fsSL https://ollama.ai/install.sh | sh" + echo " • Reinstall filebrowser: curl -fsSL https://raw.githubusercontent.com/filebrowser/get/master/get.sh | bash" +fi +echo ""