mirror of
https://github.com/myronblair/proxmox-config
synced 2026-06-30 15:59:57 -05:00
Initial: backup/restore scripts, README
- backup.sh: weekly cron script for PVE1+PVE2 - restore.sh: interactive disaster recovery wizard - README.md: step-by-step recovery guide for both nodes Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,9 @@
|
||||
# Never store private keys or secrets
|
||||
*.key
|
||||
priv/
|
||||
id_rsa
|
||||
*.pem
|
||||
*.pfx
|
||||
|
||||
# Logs
|
||||
*.log
|
||||
@@ -0,0 +1,176 @@
|
||||
# Proxmox Config Backup & Restore
|
||||
|
||||
Automated weekly backup of PVE1 and PVE2 configuration to GitHub.
|
||||
In a host failure, this gets you back up in 15–30 minutes.
|
||||
|
||||
---
|
||||
|
||||
## Repository Structure
|
||||
|
||||
```
|
||||
proxmox-config/
|
||||
backup.sh ← runs on each node weekly, commits changes
|
||||
restore.sh ← interactive restore on a fresh Proxmox install
|
||||
README.md ← this file
|
||||
shared/ ← cluster-wide configs (backed up from PVE1)
|
||||
storage.cfg — storage definitions (local-lvm, GoFlex, ProxBackups)
|
||||
datacenter.cfg — cluster settings
|
||||
user.cfg — users and API tokens
|
||||
corosync.conf — cluster membership
|
||||
jobs.cfg — backup job schedules
|
||||
vzdump.cron — vzdump cron config
|
||||
firewall/ — cluster firewall rules
|
||||
ha/ — HA group/resource configs
|
||||
nodes/
|
||||
pve/
|
||||
qemu-server/ — PVE1 VM .conf files (100–120)
|
||||
lxc/ — PVE1 LXC container configs (110)
|
||||
pve2/
|
||||
qemu-server/ — PVE2 VM .conf files (106, 302)
|
||||
pve/ ← PVE1-specific (hostname: pve, 10.48.200.90)
|
||||
network/ — /etc/network/interfaces
|
||||
cron/ — root crontab
|
||||
scripts/ — custom shell/python scripts
|
||||
systemd/ — custom service units
|
||||
jarvis-agent/ — JARVIS monitoring agent config
|
||||
ssh/ — authorized_keys (no private keys)
|
||||
pve2/ ← PVE2-specific (hostname: pve2, 10.48.200.91)
|
||||
(same structure)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Backup Schedule
|
||||
|
||||
Both nodes run `backup.sh` via cron every **Sunday at 3:00 AM**.
|
||||
|
||||
```
|
||||
0 3 * * 0 /usr/local/bin/proxmox-backup >> /var/log/proxmox-backup.log 2>&1
|
||||
```
|
||||
|
||||
The script:
|
||||
- Pulls latest from GitHub (handles both nodes committing)
|
||||
- Collects all config files listed above
|
||||
- Skips large binaries (ollama, filebrowser — see reinstall notes below)
|
||||
- Skips private keys (`/etc/pve/priv/`, SSH private keys)
|
||||
- Commits with a timestamp and pushes
|
||||
|
||||
---
|
||||
|
||||
## Disaster Recovery — Host Failure
|
||||
|
||||
### Scenario A: PVE1 (primary) fails
|
||||
|
||||
PVE1 hosts most VMs (100–113, 118, 120). Storage for those VMs is on `local-lvm` (PVE1's LVM thin pool) or `GoFlex` NAS. VMs on local-lvm **are at risk** if the disk fails — restore from PBS.
|
||||
|
||||
**Steps:**
|
||||
|
||||
1. **Install fresh Proxmox** on replacement hardware.
|
||||
- Set IP to `10.48.200.90` during install, or set after.
|
||||
- Set hostname to `pve`.
|
||||
|
||||
2. **Clone this repo:**
|
||||
```bash
|
||||
apt install -y git
|
||||
git clone https://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@github.com/myronblair/proxmox-config.git /opt/proxmox-config
|
||||
```
|
||||
|
||||
3. **Run the restore script:**
|
||||
```bash
|
||||
bash /opt/proxmox-config/restore.sh pve1
|
||||
```
|
||||
The script is interactive — confirm each section as it goes.
|
||||
|
||||
4. **Reboot** to apply network config:
|
||||
```bash
|
||||
reboot
|
||||
```
|
||||
|
||||
5. **Restore VMs from PBS** (Proxmox Backup Server at `10.48.200.85`):
|
||||
- Log in to PVE web UI: `https://10.48.200.90:8006`
|
||||
- Datacenter → Storage → Add → Proxmox Backup Server
|
||||
- Server: `10.48.200.85`
|
||||
- Datastore: `PBSBackup`
|
||||
- Username: `root@pam`
|
||||
- Fingerprint: see `shared/pbs_fingerprint.txt`
|
||||
- For each VM: select the backup → Restore
|
||||
|
||||
6. **Reinstall large binaries** (not in git — too large):
|
||||
```bash
|
||||
# Ollama (local LLM on VM 210 inside this host)
|
||||
curl -fsSL https://ollama.ai/install.sh | sh
|
||||
ollama pull llama3.2
|
||||
|
||||
# File Browser
|
||||
curl -fsSL https://raw.githubusercontent.com/filebrowser/get/master/get.sh | bash
|
||||
# Config is at /root/.filebrowser.json or wherever it was set
|
||||
```
|
||||
|
||||
7. **Start services:**
|
||||
```bash
|
||||
systemctl start jarvis-agent filebrowser
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Scenario B: PVE2 (secondary) fails
|
||||
|
||||
PVE2 hosts VM 106 (unknown) and VM 302 (NetworkBackup). PVE2 uses `local-lvm` for storage.
|
||||
|
||||
**Steps:**
|
||||
|
||||
1. **Install fresh Proxmox**, set IP `10.48.200.91`, hostname `pve2`.
|
||||
|
||||
2. **Clone repo and run restore:**
|
||||
```bash
|
||||
apt install -y git
|
||||
git clone https://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@github.com/myronblair/proxmox-config.git /opt/proxmox-config
|
||||
bash /opt/proxmox-config/restore.sh pve2
|
||||
```
|
||||
|
||||
3. **Join the cluster from PVE1:**
|
||||
```bash
|
||||
# On PVE1:
|
||||
pvecm add 10.48.200.91
|
||||
# This auto-syncs all cluster state to PVE2
|
||||
```
|
||||
|
||||
4. **Restore VMs 106 and 302 from PBS.**
|
||||
|
||||
5. **Reboot PVE2**, then verify cluster: `pvecm status`
|
||||
|
||||
---
|
||||
|
||||
## What Is NOT Backed Up Here
|
||||
|
||||
| Item | Where to find it |
|
||||
|------|-----------------|
|
||||
| VM disk images | Proxmox Backup Server (10.48.200.85) — runs nightly |
|
||||
| /etc/pve/priv/ | Private CA key, auth keys — DO NOT store in git; regenerated by Proxmox on fresh install |
|
||||
| SSH private keys | /root/.ssh/id_rsa — regenerate or restore from secure storage |
|
||||
| ollama binary | Reinstall via install script (see above) |
|
||||
| filebrowser binary | Reinstall via get.sh (see above) |
|
||||
| blockalign, urbackupclientctl | Reinstall from their respective sources if needed |
|
||||
|
||||
---
|
||||
|
||||
## Nodes Quick Reference
|
||||
|
||||
| Node | Hostname | IP | Port | Role |
|
||||
|------|----------|----|------|------|
|
||||
| PVE1 | pve | 10.48.200.90 | 8006 | Primary — most VMs |
|
||||
| PVE2 | pve2 | 10.48.200.91 | 8006 | Secondary — VM 106, 302 |
|
||||
| PBS | — | 10.48.200.85 | 8007 | Backup server |
|
||||
|
||||
Proxmox web login: `root` / `Joker1974!!!`
|
||||
FortiGate DDNS (PVE1 from internet): `orbisne.fortiddns.com`
|
||||
|
||||
---
|
||||
|
||||
## Manual Backup Trigger
|
||||
|
||||
To run a backup immediately on either node:
|
||||
```bash
|
||||
/usr/local/bin/proxmox-backup
|
||||
tail -f /var/log/proxmox-backup.log
|
||||
```
|
||||
@@ -0,0 +1,161 @@
|
||||
#!/usr/bin/env bash
|
||||
# =============================================================================
|
||||
# Proxmox Config Backup — runs on PVE1 or PVE2
|
||||
# Backs up all critical configs to GitHub weekly
|
||||
# Install: /usr/local/bin/proxmox-backup
|
||||
# Cron: 0 3 * * 0 /usr/local/bin/proxmox-backup >> /var/log/proxmox-backup.log 2>&1
|
||||
# =============================================================================
|
||||
set -euo pipefail
|
||||
|
||||
PAT="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"
|
||||
REPO_URL="https://${PAT}@github.com/myronblair/proxmox-config.git"
|
||||
REPO_DIR="/opt/proxmox-config"
|
||||
NODE=$(hostname)
|
||||
LOG_PREFIX="[$(date '+%Y-%m-%d %H:%M:%S')] [$NODE]"
|
||||
|
||||
log() { echo "$LOG_PREFIX $*"; }
|
||||
die() { echo "$LOG_PREFIX ERROR: $*" >&2; exit 1; }
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 1. Clone or update local repo
|
||||
# ---------------------------------------------------------------------------
|
||||
if [[ -d "$REPO_DIR/.git" ]]; then
|
||||
log "Pulling latest from GitHub"
|
||||
cd "$REPO_DIR"
|
||||
git config user.email "proxmox-backup@orbishosting.com"
|
||||
git config user.name "Proxmox Backup"
|
||||
git pull --rebase origin main -q || { log "Pull failed — continuing with local state"; true; }
|
||||
else
|
||||
log "Cloning repo to $REPO_DIR"
|
||||
git clone "$REPO_URL" "$REPO_DIR"
|
||||
cd "$REPO_DIR"
|
||||
git config user.email "proxmox-backup@orbishosting.com"
|
||||
git config user.name "Proxmox Backup"
|
||||
fi
|
||||
|
||||
cd "$REPO_DIR"
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 2. Node-specific directories
|
||||
# ---------------------------------------------------------------------------
|
||||
NODE_DIR="$REPO_DIR/$NODE"
|
||||
mkdir -p \
|
||||
"$NODE_DIR/network" \
|
||||
"$NODE_DIR/cron" \
|
||||
"$NODE_DIR/scripts" \
|
||||
"$NODE_DIR/systemd" \
|
||||
"$NODE_DIR/jarvis-agent"
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 3. Shared PVE cluster configs (only back up from primary node to avoid
|
||||
# conflicts — PVE1/pve is the primary; pve2 skips this section)
|
||||
# ---------------------------------------------------------------------------
|
||||
if [[ "$NODE" == "pve" ]]; then
|
||||
log "Backing up shared PVE cluster configs"
|
||||
SHARED_DIR="$REPO_DIR/shared"
|
||||
mkdir -p \
|
||||
"$SHARED_DIR/firewall" \
|
||||
"$SHARED_DIR/ha" \
|
||||
"$SHARED_DIR/sdn" \
|
||||
"$SHARED_DIR/nodes/pve/qemu-server" \
|
||||
"$SHARED_DIR/nodes/pve/lxc" \
|
||||
"$SHARED_DIR/nodes/pve2/qemu-server" \
|
||||
"$SHARED_DIR/nodes/pve2/lxc"
|
||||
|
||||
# Top-level cluster configs
|
||||
for f in storage.cfg datacenter.cfg user.cfg corosync.conf vzdump.cron jobs.cfg ceph.conf replication.cfg; do
|
||||
[[ -f "/etc/pve/$f" ]] && cp "/etc/pve/$f" "$SHARED_DIR/$f" || true
|
||||
done
|
||||
|
||||
# Firewall
|
||||
[[ -d /etc/pve/firewall ]] && rsync -a --delete /etc/pve/firewall/ "$SHARED_DIR/firewall/" 2>/dev/null || true
|
||||
|
||||
# HA
|
||||
[[ -d /etc/pve/ha ]] && rsync -a --delete /etc/pve/ha/ "$SHARED_DIR/ha/" 2>/dev/null || true
|
||||
|
||||
# SDN
|
||||
[[ -d /etc/pve/sdn ]] && rsync -a --delete /etc/pve/sdn/ "$SHARED_DIR/sdn/" 2>/dev/null || true
|
||||
|
||||
# VM/LXC configs (per-node)
|
||||
for PVENODE in pve pve2; do
|
||||
[[ -d "/etc/pve/nodes/$PVENODE/qemu-server" ]] && \
|
||||
rsync -a --delete /etc/pve/nodes/$PVENODE/qemu-server/ "$SHARED_DIR/nodes/$PVENODE/qemu-server/" 2>/dev/null || true
|
||||
[[ -d "/etc/pve/nodes/$PVENODE/lxc" ]] && \
|
||||
rsync -a --delete /etc/pve/nodes/$PVENODE/lxc/ "$SHARED_DIR/nodes/$PVENODE/lxc/" 2>/dev/null || true
|
||||
done
|
||||
|
||||
# PBS fingerprint (needed for restore)
|
||||
grep -A5 "pbs: ProxBackups" /etc/pve/storage.cfg > "$SHARED_DIR/pbs_fingerprint.txt" 2>/dev/null || true
|
||||
fi
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 4. Network config
|
||||
# ---------------------------------------------------------------------------
|
||||
log "Backing up network config"
|
||||
cp /etc/network/interfaces "$NODE_DIR/network/interfaces"
|
||||
[[ -d /etc/network/interfaces.d ]] && \
|
||||
rsync -a --delete /etc/network/interfaces.d/ "$NODE_DIR/network/interfaces.d/" 2>/dev/null || true
|
||||
cp /etc/hosts "$NODE_DIR/network/hosts"
|
||||
cp /etc/hostname "$NODE_DIR/network/hostname"
|
||||
[[ -f /etc/resolv.conf ]] && cp /etc/resolv.conf "$NODE_DIR/network/resolv.conf" || true
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 5. Root crontab
|
||||
# ---------------------------------------------------------------------------
|
||||
log "Backing up crontab"
|
||||
crontab -l 2>/dev/null > "$NODE_DIR/cron/root" || echo "# no crontab" > "$NODE_DIR/cron/root"
|
||||
[[ -f /etc/vzdump.conf ]] && cp /etc/vzdump.conf "$NODE_DIR/vzdump.conf" || true
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 6. Custom scripts (text only — skip large binaries)
|
||||
# ---------------------------------------------------------------------------
|
||||
log "Backing up custom scripts"
|
||||
for f in /usr/local/bin/*.sh /usr/local/bin/*.py /usr/local/bin/*.pl; do
|
||||
[[ -f "$f" ]] || continue
|
||||
size=$(stat -c%s "$f" 2>/dev/null || echo 0)
|
||||
if [[ $size -lt 524288 ]]; then # skip files > 512KB (binaries)
|
||||
cp "$f" "$NODE_DIR/scripts/"
|
||||
fi
|
||||
done
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 7. Custom systemd units (only non-stock units)
|
||||
# ---------------------------------------------------------------------------
|
||||
log "Backing up custom systemd units"
|
||||
STOCK_UNITS="chronyd.service iscsi.service pve-manager.service smartd.service sshd.service zed.service"
|
||||
for f in /etc/systemd/system/*.service; do
|
||||
[[ -f "$f" ]] || continue
|
||||
bname=$(basename "$f")
|
||||
if ! echo "$STOCK_UNITS" | grep -qw "$bname"; then
|
||||
cp "$f" "$NODE_DIR/systemd/"
|
||||
fi
|
||||
done
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 8. JARVIS agent config
|
||||
# ---------------------------------------------------------------------------
|
||||
log "Backing up JARVIS agent config"
|
||||
[[ -f /opt/jarvis-agent/config.json ]] && \
|
||||
cp /opt/jarvis-agent/config.json "$NODE_DIR/jarvis-agent/config.json" || true
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 9. SSH authorized_keys (for access restoration — no private keys)
|
||||
# ---------------------------------------------------------------------------
|
||||
mkdir -p "$NODE_DIR/ssh"
|
||||
[[ -f /root/.ssh/authorized_keys ]] && cp /root/.ssh/authorized_keys "$NODE_DIR/ssh/authorized_keys" || true
|
||||
[[ -f /root/.ssh/id_rsa.pub ]] && cp /root/.ssh/id_rsa.pub "$NODE_DIR/ssh/id_rsa.pub" || true
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 10. Commit and push
|
||||
# ---------------------------------------------------------------------------
|
||||
log "Committing changes"
|
||||
git add -A
|
||||
if git diff --cached --quiet; then
|
||||
log "No changes to commit"
|
||||
else
|
||||
CHANGES=$(git diff --cached --stat | tail -1)
|
||||
git commit -m "[$NODE] Weekly backup $(date '+%Y-%m-%d') — $CHANGES"
|
||||
log "Pushing to GitHub"
|
||||
git push origin main
|
||||
log "Backup complete"
|
||||
fi
|
||||
+303
@@ -0,0 +1,303 @@
|
||||
#!/usr/bin/env bash
|
||||
# =============================================================================
|
||||
# Proxmox Config Restore — run on a freshly installed Proxmox node
|
||||
#
|
||||
# Usage:
|
||||
# bash restore.sh pve1 — restore PVE1 (primary, hostname "pve")
|
||||
# bash restore.sh pve2 — restore PVE2 (secondary, hostname "pve2")
|
||||
#
|
||||
# Prerequisites on fresh Proxmox install:
|
||||
# apt install -y git
|
||||
# git clone https://<PAT>@github.com/myronblair/proxmox-config.git /opt/proxmox-config
|
||||
# bash /opt/proxmox-config/restore.sh pve1
|
||||
# =============================================================================
|
||||
|
||||
TARGET="${1:-}"
|
||||
REPO_DIR="$(cd "$(dirname "$0")" && pwd)"
|
||||
|
||||
RED='\033[0;31m'; YELLOW='\033[1;33m'; GREEN='\033[0;32m'; CYAN='\033[0;36m'; NC='\033[0m'
|
||||
|
||||
header() { echo -e "\n${CYAN}══════════════════════════════════════${NC}"; echo -e "${CYAN} $*${NC}"; echo -e "${CYAN}══════════════════════════════════════${NC}"; }
|
||||
success() { echo -e "${GREEN} ✓ $*${NC}"; }
|
||||
warn() { echo -e "${YELLOW} ⚠ $*${NC}"; }
|
||||
info() { echo -e " → $*"; }
|
||||
die() { echo -e "${RED} ✗ $*${NC}" >&2; exit 1; }
|
||||
|
||||
confirm() {
|
||||
local msg="$1"
|
||||
echo -e "\n${YELLOW} $msg${NC}"
|
||||
read -rp " Apply this section? [Y/n] " ans
|
||||
[[ "${ans:-Y}" =~ ^[Yy]$ ]]
|
||||
}
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Validate arguments
|
||||
# ---------------------------------------------------------------------------
|
||||
if [[ -z "$TARGET" ]]; then
|
||||
echo "Usage: $0 pve1|pve2"
|
||||
echo ""
|
||||
echo " pve1 — restore primary node (hostname: pve, IP: 10.48.200.90)"
|
||||
echo " pve2 — restore secondary node (hostname: pve2, IP: 10.48.200.91)"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if [[ "$TARGET" == "pve1" ]]; then
|
||||
NODE_DIR="$REPO_DIR/pve"
|
||||
EXPECTED_HOSTNAME="pve"
|
||||
EXPECTED_IP="10.48.200.90"
|
||||
elif [[ "$TARGET" == "pve2" ]]; then
|
||||
NODE_DIR="$REPO_DIR/pve2"
|
||||
EXPECTED_HOSTNAME="pve2"
|
||||
EXPECTED_IP="10.48.200.91"
|
||||
else
|
||||
die "Unknown target: $TARGET. Use pve1 or pve2."
|
||||
fi
|
||||
|
||||
SHARED_DIR="$REPO_DIR/shared"
|
||||
|
||||
[[ -d "$NODE_DIR" ]] || die "Node directory not found: $NODE_DIR"
|
||||
[[ $(id -u) -eq 0 ]] || die "Must run as root"
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Welcome banner
|
||||
# ---------------------------------------------------------------------------
|
||||
clear
|
||||
echo -e "${CYAN}"
|
||||
cat << 'BANNER'
|
||||
╔═══════════════════════════════════════════════════════╗
|
||||
║ PROXMOX CONFIG RESTORE — OrbiShosting ║
|
||||
╚═══════════════════════════════════════════════════════╝
|
||||
BANNER
|
||||
echo -e "${NC}"
|
||||
echo " Target : $TARGET ($EXPECTED_HOSTNAME — $EXPECTED_IP)"
|
||||
echo " Source : $REPO_DIR"
|
||||
echo " Date : $(date)"
|
||||
echo ""
|
||||
warn "This script restores configs to a FRESH Proxmox install."
|
||||
warn "Running on an existing node may overwrite running config."
|
||||
echo ""
|
||||
read -rp " Type 'yes' to continue: " confirm_start
|
||||
[[ "$confirm_start" == "yes" ]] || { echo "Aborted."; exit 0; }
|
||||
|
||||
APPLIED=()
|
||||
SKIPPED=()
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# SECTION 1: Hostname
|
||||
# ---------------------------------------------------------------------------
|
||||
header "1 / 8 — Hostname"
|
||||
info "Will set hostname to: $EXPECTED_HOSTNAME"
|
||||
[[ -f "$NODE_DIR/network/hostname" ]] && info "Backed-up hostname: $(cat $NODE_DIR/network/hostname)"
|
||||
if confirm "Set hostname to $EXPECTED_HOSTNAME?"; then
|
||||
echo "$EXPECTED_HOSTNAME" > /etc/hostname
|
||||
hostname "$EXPECTED_HOSTNAME"
|
||||
success "Hostname set to $EXPECTED_HOSTNAME"
|
||||
APPLIED+=("hostname")
|
||||
else
|
||||
SKIPPED+=("hostname")
|
||||
fi
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# SECTION 2: Network interfaces
|
||||
# ---------------------------------------------------------------------------
|
||||
header "2 / 8 — Network Configuration"
|
||||
if [[ -f "$NODE_DIR/network/interfaces" ]]; then
|
||||
info "Current /etc/network/interfaces:"
|
||||
cat /etc/network/interfaces | sed 's/^/ /'
|
||||
info ""
|
||||
info "Backed-up interfaces:"
|
||||
cat "$NODE_DIR/network/interfaces" | sed 's/^/ /'
|
||||
warn "Applying network config requires a reboot to take effect."
|
||||
if confirm "Restore network interfaces?"; then
|
||||
cp /etc/network/interfaces "/etc/network/interfaces.bak.$(date +%s)"
|
||||
cp "$NODE_DIR/network/interfaces" /etc/network/interfaces
|
||||
[[ -d "$NODE_DIR/network/interfaces.d" ]] && \
|
||||
rsync -a "$NODE_DIR/network/interfaces.d/" /etc/network/interfaces.d/
|
||||
success "Network interfaces restored (reboot required)"
|
||||
APPLIED+=("network")
|
||||
else
|
||||
SKIPPED+=("network")
|
||||
fi
|
||||
else
|
||||
warn "No network interfaces backup found — skipping"
|
||||
SKIPPED+=("network")
|
||||
fi
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# SECTION 3: /etc/hosts
|
||||
# ---------------------------------------------------------------------------
|
||||
header "3 / 8 — /etc/hosts"
|
||||
if [[ -f "$NODE_DIR/network/hosts" ]]; then
|
||||
if confirm "Restore /etc/hosts?"; then
|
||||
cp /etc/hosts "/etc/hosts.bak.$(date +%s)"
|
||||
cp "$NODE_DIR/network/hosts" /etc/hosts
|
||||
success "/etc/hosts restored"
|
||||
APPLIED+=("hosts")
|
||||
else
|
||||
SKIPPED+=("hosts")
|
||||
fi
|
||||
fi
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# SECTION 4: Custom scripts
|
||||
# ---------------------------------------------------------------------------
|
||||
header "4 / 8 — Custom Scripts (/usr/local/bin)"
|
||||
if [[ -d "$NODE_DIR/scripts" ]] && [[ -n "$(ls -A $NODE_DIR/scripts 2>/dev/null)" ]]; then
|
||||
info "Scripts to restore:"
|
||||
ls "$NODE_DIR/scripts" | sed 's/^/ /'
|
||||
if confirm "Restore custom scripts to /usr/local/bin/?"; then
|
||||
cp "$NODE_DIR/scripts/"* /usr/local/bin/ 2>/dev/null || true
|
||||
chmod +x /usr/local/bin/*.sh /usr/local/bin/*.py 2>/dev/null || true
|
||||
success "Scripts restored"
|
||||
APPLIED+=("scripts")
|
||||
if [[ "$TARGET" == "pve1" ]]; then
|
||||
warn "Large binaries NOT restored (ollama, filebrowser, etc.)"
|
||||
warn "Reinstall manually:"
|
||||
warn " ollama : curl -fsSL https://ollama.ai/install.sh | sh"
|
||||
warn " filebrowser: curl -fsSL https://raw.githubusercontent.com/filebrowser/get/master/get.sh | bash"
|
||||
fi
|
||||
else
|
||||
SKIPPED+=("scripts")
|
||||
fi
|
||||
else
|
||||
info "No custom scripts backed up"
|
||||
fi
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# SECTION 5: Systemd units
|
||||
# ---------------------------------------------------------------------------
|
||||
header "5 / 8 — Systemd Service Units"
|
||||
if [[ -d "$NODE_DIR/systemd" ]] && [[ -n "$(ls -A $NODE_DIR/systemd 2>/dev/null)" ]]; then
|
||||
info "Units to restore:"
|
||||
ls "$NODE_DIR/systemd" | sed 's/^/ /'
|
||||
if confirm "Restore and enable systemd units?"; then
|
||||
for unit in "$NODE_DIR/systemd/"*.service; do
|
||||
[[ -f "$unit" ]] || continue
|
||||
bname=$(basename "$unit")
|
||||
cp "$unit" /etc/systemd/system/
|
||||
systemctl enable "$bname" 2>/dev/null || true
|
||||
info " Enabled: $bname"
|
||||
done
|
||||
systemctl daemon-reload
|
||||
success "Systemd units restored and enabled"
|
||||
APPLIED+=("systemd")
|
||||
warn "Units are ENABLED but not started — start manually after verifying prerequisites"
|
||||
warn " e.g.: systemctl start jarvis-agent filebrowser"
|
||||
else
|
||||
SKIPPED+=("systemd")
|
||||
fi
|
||||
fi
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# SECTION 6: JARVIS agent
|
||||
# ---------------------------------------------------------------------------
|
||||
header "6 / 8 — JARVIS Monitoring Agent"
|
||||
if [[ -f "$NODE_DIR/jarvis-agent/config.json" ]]; then
|
||||
info "Agent config found:"
|
||||
cat "$NODE_DIR/jarvis-agent/config.json" | sed 's/^/ /'
|
||||
if confirm "Restore JARVIS agent config?"; then
|
||||
mkdir -p /opt/jarvis-agent
|
||||
cp "$NODE_DIR/jarvis-agent/config.json" /opt/jarvis-agent/config.json
|
||||
|
||||
# Install agent if not present
|
||||
if [[ ! -f /opt/jarvis-agent/jarvis-agent.py ]]; then
|
||||
info "Downloading JARVIS agent..."
|
||||
curl -sk https://jarvis.orbishosting.com/install-agent.sh | bash -s "$(hostname)" proxmox || \
|
||||
warn "Auto-install failed — manually install: curl -sk https://jarvis.orbishosting.com/install-agent.sh | bash -s $(hostname) proxmox"
|
||||
fi
|
||||
success "JARVIS agent restored"
|
||||
APPLIED+=("jarvis-agent")
|
||||
else
|
||||
SKIPPED+=("jarvis-agent")
|
||||
fi
|
||||
fi
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# SECTION 7: Root crontab
|
||||
# ---------------------------------------------------------------------------
|
||||
header "7 / 8 — Root Crontab"
|
||||
if [[ -f "$NODE_DIR/cron/root" ]] && ! grep -q "^# no crontab" "$NODE_DIR/cron/root"; then
|
||||
info "Cron entries to restore:"
|
||||
cat "$NODE_DIR/cron/root" | sed 's/^/ /'
|
||||
if confirm "Restore root crontab?"; then
|
||||
crontab "$NODE_DIR/cron/root"
|
||||
success "Crontab restored"
|
||||
APPLIED+=("crontab")
|
||||
else
|
||||
SKIPPED+=("crontab")
|
||||
fi
|
||||
else
|
||||
info "No crontab entries backed up — skipping"
|
||||
fi
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# SECTION 8: PVE cluster configs (pve1 only — pve2 joins cluster instead)
|
||||
# ---------------------------------------------------------------------------
|
||||
header "8 / 8 — PVE Cluster / VM Configs"
|
||||
if [[ "$TARGET" == "pve1" ]] && [[ -d "$SHARED_DIR" ]]; then
|
||||
echo ""
|
||||
warn "PVE cluster configs (storage, VM configs, users, firewall) are"
|
||||
warn "managed by Proxmox's cluster filesystem (/etc/pve/)."
|
||||
warn ""
|
||||
warn "IMPORTANT: Do NOT copy /etc/pve/ files directly on a fresh install."
|
||||
warn "Proxmox will auto-populate these when the node is online."
|
||||
warn ""
|
||||
info "Instead, after Proxmox is installed and running:"
|
||||
info " 1. Restore VMs from Proxmox Backup Server (PBS) at 10.48.200.85"
|
||||
info " → In PVE web UI: Datacenter → Storage → ProxBackups"
|
||||
info " → PBS fingerprint: $(grep 'fingerprint' $SHARED_DIR/storage.cfg 2>/dev/null | head -1 | tr -d ' \t' || echo 'see shared/storage.cfg')"
|
||||
info ""
|
||||
info " 2. Re-add storage manually if PBS isn't auto-discovered:"
|
||||
info " → Datacenter → Storage → Add → Proxmox Backup Server"
|
||||
info ""
|
||||
info " 3. VM configs for reference (not to copy directly):"
|
||||
[[ -d "$SHARED_DIR/nodes/pve/qemu-server" ]] && ls "$SHARED_DIR/nodes/pve/qemu-server/" | sed 's/^/ /'
|
||||
echo ""
|
||||
if confirm "Copy VM .conf files to /etc/pve/nodes/pve/qemu-server/ (only do this if cluster is NOT active)?"; then
|
||||
mkdir -p /etc/pve/nodes/pve/qemu-server /etc/pve/nodes/pve/lxc
|
||||
cp "$SHARED_DIR/nodes/pve/qemu-server/"*.conf /etc/pve/nodes/pve/qemu-server/ 2>/dev/null || true
|
||||
cp "$SHARED_DIR/nodes/pve/lxc/"*.conf /etc/pve/nodes/pve/lxc/ 2>/dev/null || true
|
||||
cp "$SHARED_DIR/storage.cfg" /etc/pve/ 2>/dev/null || true
|
||||
cp "$SHARED_DIR/datacenter.cfg" /etc/pve/ 2>/dev/null || true
|
||||
success "PVE configs copied — restart pve-cluster to apply"
|
||||
systemctl restart pve-cluster 2>/dev/null || true
|
||||
APPLIED+=("pve-configs")
|
||||
else
|
||||
SKIPPED+=("pve-configs")
|
||||
info "Skipped — restore VMs from PBS backup instead (recommended)"
|
||||
fi
|
||||
|
||||
elif [[ "$TARGET" == "pve2" ]]; then
|
||||
warn "PVE2 does not hold the primary cluster config."
|
||||
info "To restore PVE2 as a cluster member:"
|
||||
info " 1. Fresh Proxmox install with hostname 'pve2' and IP 10.48.200.91"
|
||||
info " 2. On PVE1: pvecm add 10.48.200.91"
|
||||
info " 3. PVE2 will auto-sync all VM configs and cluster state from PVE1"
|
||||
info " 4. VM 106 and 302 will reappear automatically"
|
||||
SKIPPED+=("pve-configs")
|
||||
fi
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Summary
|
||||
# ---------------------------------------------------------------------------
|
||||
header "Restore Complete"
|
||||
echo ""
|
||||
if [[ ${#APPLIED[@]} -gt 0 ]]; then
|
||||
success "Applied sections: ${APPLIED[*]}"
|
||||
fi
|
||||
if [[ ${#SKIPPED[@]} -gt 0 ]]; then
|
||||
warn "Skipped sections: ${SKIPPED[*]}"
|
||||
fi
|
||||
echo ""
|
||||
echo -e "${YELLOW} Next steps:${NC}"
|
||||
if [[ " ${APPLIED[*]} " =~ " network " ]]; then
|
||||
echo " • REBOOT to apply network changes: reboot"
|
||||
fi
|
||||
echo " • Verify Proxmox web UI at https://$EXPECTED_IP:8006"
|
||||
echo " • Restore VMs from PBS backup server at https://10.48.200.85:8007"
|
||||
echo " • Start JARVIS agent: systemctl start jarvis-agent"
|
||||
if [[ "$TARGET" == "pve1" ]]; then
|
||||
echo " • Reinstall ollama if needed: curl -fsSL https://ollama.ai/install.sh | sh"
|
||||
echo " • Reinstall filebrowser: curl -fsSL https://raw.githubusercontent.com/filebrowser/get/master/get.sh | bash"
|
||||
fi
|
||||
echo ""
|
||||
Reference in New Issue
Block a user