[orbis] Weekly backup 2026-06-21 — 8 files changed, 218 insertions(+), 302 deletions(-)

This commit is contained in:
DO Server Backup
2026-06-21 04:00:08 +00:00
parent 91852514c8
commit 8185a4fd93
8 changed files with 218 additions and 302 deletions
+139 -23
View File
@@ -6,18 +6,44 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
This is a home-lab / managed-hosting environment. There is no local codebase to build or test — work consists of editing PHP/JS files on remote servers via SSH and managing infrastructure across several machines. All tool calls use `sshpass` with password auth.
## Tailscale Network
All key hosts are on Tailscale (myronblair@gmail.com). Use Tailscale IPs for SSH — no relay or port forwarding needed.
| Host | Tailscale IP | LAN IP | Password |
|------|-------------|--------|----------|
| Claude VM (this) | 100.69.120.58 | 10.48.200.29 | — |
| PVE1 | 100.80.188.8 | 10.48.200.90 | `Joker1974!!!` |
| PVE2 | 100.87.186.12 | 10.48.200.91 | `Joker1974!!!` |
| DO server (orbis) | 100.121.13.34 | 165.22.1.228 | `Gonewalk1974!@#` |
| FusionPBX | 100.74.46.120 | 134.209.72.226 | `Joker1974!@#` |
| JARVIS VM | 100.77.178.42 | 10.48.200.211 | `Joker1974!!!` |
| NPM VM | 100.110.239.71 | 10.48.200.201 | `Joker1974!!!` |
| Ollama VM | 100.96.100.113 | 10.48.200.210 | `Joker1974!!!` |
| NovaCPX (hostpanel-110) | 100.86.51.18 | 10.48.200.110 | `Joker1974!!!` |
| HomeBridge | 100.124.182.18 | 10.48.200.18 | — |
| WireGuard CT | 100.122.55.10 | 10.48.200.19 | — |
| Synology NAS | 100.118.175.5 | 10.48.200.249 | — |
| mini-it12 (Windows) | 100.98.151.120 | 10.48.200.87 | — |
**DNS note:** FortiGate blocks outbound port 53. All PVE1 VMs must use `10.48.200.90` (PVE1 dnsmasq → 100.100.100.100) as their DNS server, not 8.8.8.8 directly.
## Server Map
| Host | IP | SSH | Purpose |
|------|-----|-----|---------|
| DO (main) | 165.22.1.228 | `root / Gonewalk1974!@#` | CyberPanel/OLS — all websites + JARVIS |
| FusionPBX | 134.209.72.226 | `root / Joker1974!@#` | FreeSWITCH PBX |
| PVE1 (Proxmox) | orbisne.fortiddns.com (10.48.200.90) | `root / Joker1974!!!` | Primary hypervisor — FortiGate DDNS, auto-updates if IP changes |
| DO (main) | 165.22.1.228 | `root / Gonewalk1974!@#` | CyberPanel/OLS — all websites (not JARVIS after migration) |
| FusionPBX | 134.209.72.226 | `root / Joker1974!@#` (via Tailscale 100.74.46.120) | FreeSWITCH PBX |
| PVE1 (Proxmox) | orbisne.fortiddns.com (10.48.200.90) | `root / Joker1974!!!` (via Tailscale 100.80.188.8) | Primary hypervisor |
| PVE2 (Proxmox) | 10.48.200.91 | `root / Joker1974!!!` | Secondary hypervisor |
| Ollama VM | 10.48.200.95 | `myron → sudo` | llama3.2 local LLM (PVE1 VM 210) |
| JARVIS VM | 10.48.200.211 | `root / Joker1974!!!` (via Tailscale 100.77.178.42) | JARVIS dashboard — PVE1 VM 211, 8c/16GB |
| NPM VM | 10.48.200.201 | `root / Joker1974!!!` (via Tailscale 100.110.239.71) | Nginx Proxy Manager — PVE1 VM 200 (LAN IP .201 despite VM ID 200) |
| Ollama VM | 10.48.200.210 | `root / Joker1974!!!` (via Tailscale 100.96.100.113) | Local LLM — PVE1 VM 210, 4c/8GB |
| Home Assistant | 10.48.200.97 | `myron → sudo` | HA VM 101 |
| NetworkBackup | 10.48.200.99 | `myron → sudo` | Backup VM (PVE2 VM 302) |
| MediaStack | 10.48.200.35 | `root via PVE1 key` | Sonarr/Radarr/Prowlarr/qBittorrent (PVE1 VM 113) |
| NovaCPX | 10.48.200.110 | `root / Joker1974!!!` (direct SSH — Tailscale 100.86.51.18 needs re-auth) | Custom hosting control panel (PVE1 VM 120) |
| NPM | 10.48.200.201 | `root / Joker1974!!!` (via Tailscale 100.110.239.71) | Nginx Proxy Manager — PVE1 VM 200 · NPM API: `POST http://localhost:81/api/tokens` identity=myronblair@outlook.com |
**SSH password order** (try in sequence if first fails): `Joker1974!@#``Joker1974!!!``Joker1974!`
@@ -34,7 +60,7 @@ sshpass -p 'Joker1974!!!' ssh -o StrictHostKeyChecking=no root@orbisne.fortiddns
For commands inside VMs on PVE1:
```bash
sshpass -p 'Joker1974!!!' ssh -o StrictHostKeyChecking=no root@orbisne.fortiddns.com \
'qm guest exec 210 -- bash -c "commands here"'
'qm guest exec <VMID> -- bash -c "commands here"'
```
## Websites on DO (165.22.1.228)
@@ -43,7 +69,7 @@ All sites live at `/home/<domain>/public_html/` on DO. CyberPanel/OpenLiteSpeed
| Site | Path | GitHub |
|------|------|--------|
| jarvis.orbishosting.com | /home/jarvis.orbishosting.com/ | myronblair/jarvis |
| ~~jarvis.orbishosting.com~~ | ~~removed from DO~~ | myronblair/jarvis (now on PVE1 VM 211) |
| tomsjavajive.com | /home/tomsjavajive.com/public_html/ | myronblair/tomsjavajive |
| epictravelexpeditions.com | /home/epictravelexpeditions.com/public_html/ | myronblair/epictravelexpeditions |
| parkerslingshotrentals.com | /home/parkerslingshotrentals.com/public_html/ | myronblair/parkerslingshotrentals |
@@ -61,12 +87,18 @@ All sites live at `/home/<domain>/public_html/` on DO. CyberPanel/OpenLiteSpeed
**Auto-deploy is active.** Push to `main` on any site repo → GitHub webhook → server pulls automatically within 1 minute. PHP syntax is validated before deploy; bad commits are auto-reverted.
Webhook handler: `https://jarvis.orbishosting.com/webhook.php` (HMAC secret: `4c8805f0285214ff0a0602b5880270b935f36a896946c7f1`)
Deploy queue: `/tmp/jarvis-deploy-queue.txt` | Runner: `/usr/local/bin/jarvis-deploy.sh` (cron every min)
Deploy log: `/home/jarvis.orbishosting.com/logs/deploy.log`
**Two separate webhook handlers:**
- **JARVIS repo** → `http://jarvis.orbishosting.com:1972/webhook.php` — deploys to JARVIS VM (`/var/www/jarvis/`). Deploy log: `/var/www/jarvis/logs/deploy.log`
- **All 6 website repos → `https://tomtomgames.com/webhook.php` on DO — deploys to `/home/<site>/public_html/` on DO. Deploy log: `/home/tomtomgames.com/logs/deploy.log`. Deploy log: `/home/<site>/logs/deploy.log`
HMAC secret (both handlers): `4c8805f0285214ff0a0602b5880270b935f36a896946c7f1`
Deploy queue: `/tmp/jarvis-deploy-queue.txt` | Runner: `/usr/local/bin/jarvis-deploy.sh` (cron every min, on both DO and JARVIS VM)
For hotfixes that can't wait 1 min, SCP directly:
```bash
# JARVIS VM
scp -o StrictHostKeyChecking=no /tmp/changed.php root@100.77.178.42:/var/www/jarvis/public_html/changed.php
# DO websites
sshpass -p 'Gonewalk1974!@#' scp -o StrictHostKeyChecking=no /tmp/changed.php \
root@165.22.1.228:/home/site.com/public_html/changed.php
```
@@ -80,18 +112,33 @@ Gitignored credentials (never in GitHub): `api/config.php` (JARVIS, epictravelex
## JARVIS System
Iron Man-style AI dashboard at `https://jarvis.orbishosting.com`. All files on DO at `/home/jarvis.orbishosting.com/`.
Iron Man-style AI dashboard at `http://jarvis.orbishosting.com:1972`. **Migrated from DO to PVE1 VM 211 (2026-06-18).** All files on JARVIS VM at `/var/www/jarvis/`.
**Access:**
- Dashboard: `http://jarvis.orbishosting.com:1972`
- Admin: `http://jarvis.orbishosting.com:1972/admin`
- Internal (LAN): `http://10.48.200.211` or via Tailscale `http://100.77.178.42`
- FortiGate VIP: external port `1972``10.48.200.211:80`
- Cloudflare: DNS only (grey cloud) — no CF proxy, no SSL overhead on origin
**Stack on JARVIS VM:**
- nginx + PHP 8.3-FPM (replaces OLS/lsphp on DO)
- MariaDB (jarvis_db local, `jarvis_user / J4rv1s_Pr0t0c0l_2026!`)
- Redis (`redis-server`)
- Python 3 + Arc Reactor daemon
**Architecture:**
- `public_html/api.php` — API router; has `session_write_close()` guard (must skip for `auth` endpoint to prevent LSAPI session deadlock)
- `public_html/api.php` — API router; `session_start()` skipped only for machine-agent sub-actions (heartbeat/metrics/ha_state/command_result/register); browser-facing agent routes (list/status/myip) need session. Has `session_write_close()` guard (must skip for `auth` endpoint to prevent LSAPI session deadlock).
- `api/config.php` — all credentials/constants (gitignored)
- `api/endpoints/chat.php` — 4-tier AI: KB intent → Ollama (10.48.200.95:11434) → Groq (`compound-beta-mini`) → Claude API; includes Tier 0.7 planner intents (tasks/appointments/briefing)
- `api/endpoints/agent.php` — push-based agent registration/heartbeat/metrics
- `api/endpoints/chat.php` — 4-tier AI: KB intent → Groq (`compound-beta-mini`) → Claude API; includes Tier 0.7 planner intents (tasks/appointments/briefing). Ollama at `http://10.48.200.210:11434`.
- `api/endpoints/agent.php` — push-based agent registration/heartbeat/metrics; browser actions (list/status/myip) auth via `$_SESSION`, machine actions auth via `X-Agent-Key` header
- `api/endpoints/alerts.php` — auto-generates alerts (CPU >85%, RAM >85%, disk >88%, offline agents, site down); dispatches restart commands to agents when their services fail
- `api/endpoints/facts_collector.php` — runs every 3 min via cron; collects agent metrics, KB facts, Proxmox/HA/Ollama status, and all 7 site HTTP health checks
- `api/endpoints/stats_cache.php` — runs every 5 min via cron; weather/news/Proxmox cache refresh
- `api/endpoints/facts_collector.php` — runs every 3 min via cron (php8.3); collects agent metrics, KB facts, Proxmox/HA status, and all 7 site HTTP health checks. Site checks use external URLs (JARVIS VM is NOT the web host). `$fresh()` queries `WHERE category=?` (not `fact_category`).
- `api/endpoints/stats_cache.php` — runs every 5 min via cron; weather/news/Proxmox cache refresh. Proxmox API at `https://10.48.200.90:8006` (direct LAN).
- `api/endpoints/do_server.php` — reads `/proc` for JARVIS VM stats; also includes DO server agent metrics (`do_server` key from jarvis-do agent via Tailscale).
- `api/endpoints/planner.php` — tasks & appointments CRUD; routes: `planner/tasks`, `planner/appointments`, `planner/today`, `planner/done`
- `api/endpoints/ha.php` — HA entity list reads from `ha_entities` table (real-time agent push); service calls go direct to HA_URL (`http://orbisne.fortiddns.com:8123`)
- `api/endpoints/ha.php` — HA entity list reads from `ha_entities` table (real-time agent push); service calls go direct to HA_URL (`http://10.48.200.97:8123`)
- `api/lib/kb_engine.php``storeFact()` ON DUPLICATE KEY UPDATE always sets `updated_at=NOW()` explicitly; without this, unchanged values don't bump the timestamp and freshness checks break.
**Voice system (index.html):**
- Continuous SpeechRecognition; mic stays open always (mute toggle button)
@@ -107,14 +154,34 @@ Iron Man-style AI dashboard at `https://jarvis.orbishosting.com`. All files on D
- Home page: small top-bar badge "N TASKS · N APPTS" when items due today (no panel added)
- Admin CRUD at `/admin` under PLANNER section → TASKS and APPOINTMENTS tabs
**Agent system:** Agents phone home every 10s (heartbeat) / 30s (metrics) to `https://165.22.1.228` with `Host: jarvis.orbishosting.com` header (bypasses Cloudflare). Config at `/opt/jarvis-agent/config.json` on each Linux agent.
Agent installer (one-liner for any Linux host): `curl -sk https://jarvis.orbishosting.com/install-agent.sh | bash -s <hostname> <linux|homeassistant|proxmox>`
**Agent system:** Agents phone home every 10s (heartbeat) / 30s (metrics) to `http://10.48.200.211` (direct LAN — no Cloudflare). Config at `/etc/jarvis-agent/config.json` or `/opt/jarvis-agent/config.json` on each Linux agent.
Agent installer (one-liner for any Linux host): `curl -sk http://10.48.200.211/install-agent.sh | bash -s <hostname> <linux|homeassistant|proxmox>`
DO server agent (jarvis-do) uses Tailscale: `jarvis_url: http://100.77.178.42`
**Self-healing:** `/usr/local/bin/jarvis-watchdog.sh` runs every 5 min (root cron). Restarts lsws/mysql/redis if down, restarts offline Proxmox VM agents via `qm guest exec`, inserts alerts to DB, rotates logs.
Watchdog log: `/home/jarvis.orbishosting.com/logs/watchdog.log`
**Agent file paths by host** (for manual updates — push to correct path then restart service):
- Most Linux hosts: `/opt/jarvis-agent/jarvis-agent.py` · service: `systemctl restart jarvis-agent`
- WireGuard CT (10.48.200.19, Alpine): `/opt/jarvis-agent/agent.py` · service: `rc-service jarvis-agent restart`
- `public_html/agent/jarvis-agent.py` is the self-update URL — must be kept in sync with `agent/jarvis-agent.py` (both are tracked in git; auto-deploy keeps them in sync after 2026-06-17)
**JARVIS DB:** `jarvis_db` on localhost. User: `jarvis_user / J4rv1s_Pr0t0c0l_2026!`. phpMyAdmin at `/phpmyadmin` (myron / Joker1974!!!).
Tables (18 total): agent_commands, agent_metrics, alerts, api_cache, appointments, conversations, ha_entities, kb_facts, kb_intents, kb_ollama_models, kb_preferences, known_commands, metrics_history, network_devices, registered_agents, tasks, users + (see schema for full column list)
**Self-healing:** `/usr/local/bin/jarvis-watchdog.sh` runs every 5 min (root cron on DO). Restarts lsws/mysql/redis on DO if down. Log: `/usr/local/lsws/logs/watchdog.log` on DO.
**JARVIS DB:** `jarvis_db` on JARVIS VM localhost (MariaDB). User: `jarvis_user / J4rv1s_Pr0t0c0l_2026!`. phpMyAdmin at `/phpmyadmin` on JARVIS VM (myron / Joker1974!!!).
Core tables: agent_commands, agent_metrics, alerts, api_cache, appointments, conversations, ha_entities, kb_facts, kb_intents, kb_ollama_models, kb_preferences, known_commands, metrics_history, network_devices, registered_agents, tasks, users. Arc Reactor adds: arc_jobs, guardian_events, guardian_config, agent_screenshots.
`kb_facts` schema: `(id, category, fact_key, fact_value, host, expires_at, updated_at)` — column is `category` not `fact_category`.
**Arc Reactor daemon:** Python service at `/opt/jarvis-arc/reactor.py` on JARVIS VM, port 7474, managed by `systemctl restart jarvis-arc`. Deploy source: `deploy/reactor.py` in the jarvis repo. After pushing to GitHub, auto-deploy pulls to `/var/www/jarvis/deploy/reactor.py` — then manually `cp /var/www/jarvis/deploy/reactor.py /opt/jarvis-arc/reactor.py && systemctl restart jarvis-arc`. Log: `/var/www/jarvis/logs/arc_reactor.log`.
**Arc Reactor AI routing:**
| Feature | Provider | Model |
|---------|----------|-------|
| Guardian anomaly alerts | Groq | `llama-3.3-70b-versatile` |
| SITREP | Groq | `llama-3.3-70b-versatile` |
| Vision: text-only sysinfo snapshot | Groq | `llama-3.3-70b-versatile` |
| Vision: actual screenshot image | Claude | `claude-opus-4-8-20251101` |
| Email drafting, research, tool_loop | Claude | `claude-sonnet-4-6` |
`llm_call(messages, provider)` cascades: groq → ollama on failure. Pass `"groq"` or `"claude"` as provider.
**Groq API note:** Use model name `compound-beta-mini` directly — NOT `groq/compound-beta-mini` (that's OpenAI router syntax and will 404 on api.groq.com).
@@ -147,6 +214,35 @@ Automated media server on PVE1 VM 113. All traffic routes through WireGuard VPN
**GitHub:** `myronblair/mediastack` (private) — config files, systemd units, README with full setup notes.
**JARVIS agent quirks:** needs `jarvis_url`, `registration_key` (`f846a9aaf7ce9a61742c63c87c4186052a71d2a580c65518`), `ssl_verify: false` in config.
## NovaCPX Panel
Custom web hosting control panel (PVE1 VM 120, 10.48.200.110). Root SSH: `sshpass -p 'Joker1974!!!' ssh -o StrictHostKeyChecking=no root@10.48.200.110` (direct LAN — use this, Tailscale 100.86.51.18 requires re-auth periodically).
**Public URLs (via NPM → FortiGate VIP port 443 → 10.48.200.201):**
- Admin: `https://admin.novacpx.orbishosting.com` (→ port 8882) · `admin / Admin2026!` or `myron / Joker1974!!!`
- Reseller: `https://reseller.novacpx.orbishosting.com` (→ port 8881)
- User: `https://panel.novacpx.orbishosting.com` or `https://novacpx.orbishosting.com` (→ port 8880)
- Webmail: port 8883 (Roundcube) — no public NPM proxy yet
- `https://web.orbishosting.com` → port 80 (placeholder for a new hosted website)
**Ports:** 8880 (user) · 8881 (reseller) · 8882 (admin) · 8883 (Roundcube webmail)
**Paths:** Panel web root `/srv/novacpx/public/` · Git repo `/opt/novacpx-src/` · DB `/var/lib/novacpx/panel.db` (SQLite) · Config `/etc/novacpx/config.ini`
**Config notes:** `/etc/novacpx/config.ini` must have `server = nginx` (not apache) — VhostManager checks this to write the correct vhost format.
**GitHub:** `myronblair/novacpx` (private). Auto-deploy active: push to `main` (stable) or `beta` → webhook → VM pulls. GitHub Actions auto-bumps VERSION: main→PATCH, beta→-beta.N suffix. Current version: 1.0.40.
**Update channels:** `stable` tracks `origin/main`, `beta` tracks `origin/beta`. Set in Admin → Settings → Update Channel.
**Local clone:** `/tmp/novacpx/` on this machine. All edits go here first, then `git push origin main`. The deploy runner syncs `panel/``/srv/novacpx/public/` and `panel/lib/``/srv/novacpx/public/lib/`. For immediate changes use SCP to `root@10.48.200.110:/srv/novacpx/public/`.
**PHP-FPM:** Per-account pools in `/etc/php/8.3/fpm/pool.d/`. If php8.3-fpm fails to start, check for orphaned pool configs referencing deleted Linux users — remove them and `systemctl start php8.3-fpm`.
**JARVIS agent:** Installed, online. Agent ID: `novacpx_e3b07264`.
**SQLite quirk:** Never use MySQL syntax (ON DUPLICATE KEY, NOW(), DATE_ADD, etc.). DB.php has translate() layer but endpoints must also use SQLite syntax directly.
## Parker Slingshot Rentals
Admin portal at `/admin/index.php` uses HMAC-signed cookie auth (not PHP sessions — sessions were unreliable under LiteSpeed caching). Admin: `admin / Parker2026!`. DB: `epic_parkersling / epic_parkersling / Joker1974!!!`.
@@ -155,7 +251,11 @@ Admin portal at `/admin/index.php` uses HMAC-signed cookie auth (not PHP session
Production at 134.209.72.226. Web: `https://fusion.orbishosting.com` (admin / fY7XP5swgtpbzrYLhkeVYkA4744). SIP profiles served via Lua XML handler — config changes require deleting `/var/cache/fusionpbx/FusionPBX.configuration.sofia.conf` to force reload. Extension 1000 (Yealink T48S at 10.48.200.43) registered on production server via port 5080 with `aggressive-nat-detection=true` to bypass FortiGate SIP ALG.
**SSH access:** Port 22 firewalled from internet — only from 107.178.2.130 / 97.154.109.245. Relay all SSH through DO:
**SSH access:** Direct via Tailscale (preferred):
```bash
sshpass -p 'Joker1974!@#' ssh -o StrictHostKeyChecking=no root@100.74.46.120
```
Fallback if Tailscale down — relay through DO:
```bash
sshpass -p 'Gonewalk1974!@#' ssh -o StrictHostKeyChecking=no root@165.22.1.228 \
'sshpass -p "Joker1974!@#" ssh -o StrictHostKeyChecking=no root@134.209.72.226 "command"'
@@ -182,6 +282,20 @@ CyberPanel uses `lsphp85`. Run PHP scripts directly with:
For PHP syntax checking use `php8.3 -l file.php` — lsphp segfaults on `-l` flag.
When a PHP endpoint uses `ob_start()` + `header.php` pattern, add `ob_end_clean()` before any CSV/JSON response output.
**Cloudflare Rocket Loader:** JARVIS uses `data-cfasync="false"` on every `<script>` tag in `index.html` (including CDN scripts like face-api.js). One untagged script is enough to trigger Rocket Loader's bootstrap, which injects `mainScript.js` and causes `SyntaxError: Identifier 'mainScriptFlag' has already been declared`. `Cache-Control: no-store, no-cache, must-revalidate, no-transform` is set in `index.php` but tag every new script with `data-cfasync="false"` anyway.
**Cloudflare auto-deploy cache problem:** After pushing JS fixes, Cloudflare CDN serves stale cached files even on hard refresh. Bump the `?v=YYYYMMDD` query param on script tags in `index.html` to force a cache miss. Current version param: `?v=20260617b`.
**JS file structure (as of 2026-06-17):**
- `assets/js/jarvis-effects.js` — canvas particle effects, sparklines
- `assets/js/jarvis-overlays.js` — sleep overlay, network map canvas
- `assets/js/jarvis-app.js` — globals, init, chat, voice, system/network/HA/alerts/weather panels
- `assets/js/panels/jarvis-arc.js` — Arc Reactor, Intel Protocol, Comms Protocol, Guardian Mode
- `assets/js/panels/jarvis-agents.js` — Missions, Directives, Memory, Clearance, Agents tab, Sites, Vision Protocol
- `assets/js/panels/jarvis-assistant.js` — Chat History, Command Palette, Suggestions, Mobile switcher, Topology map
A SyntaxError in any panels/ file breaks only that group — other panels stay functional. `escHtml()` is defined in jarvis-arc.js (loads first) and is global to all subsequent files.
## GitHub Repos
| Repo | Site | DB Schema |
@@ -198,6 +312,8 @@ When a PHP endpoint uses `ob_start()` + `header.php` pattern, add `ob_end_clean(
| myronblair/mediastack | MediaStack VM 113 | config/, systemd units, wg0.conf, README |
| myronblair/do-server-config | DO server backup | scripts, systemd, WG, OLS vhosts, restore.sh |
| myronblair/proxmox-config | PVE1+PVE2 backup | shared cluster configs + per-node, restore.sh |
| myronblair/novacpx | admin.novacpx.orbishosting.com | db/schema.sql (SQLite, 19+ tables) |
| myronblair/web-dashboard | web.orbishosting.com | — |
| myronblair/fusionpbx-config | FusionPBX backup | PostgreSQL dump (gzip) + FS configs, restore.sh |
All repos are private. Each has `config/vhost/` with OLS vhost config. The jarvis repo also has `deploy/` (watchdog, deploy runner, systemd units) and `agent/jarvis-agent.py`.