ai-context/gotchas.md

Critical Gotchas — Read Before Running Commands

PHP / OLS

• NEVER use lsphp -l for syntax check — it segfaults. Use php8.3 -l file.php
• Run CLI scripts with: /usr/local/lsws/lsphp85/bin/lsphp /path/script.php
• If endpoint uses ob_start() + header.php pattern → add ob_end_clean() before CSV/JSON output
• MySQL charset: always utf8mb4_unicode_ci — mixing with general_ci breaks JOINs (error 1267)

SSH / Networking

• DO server (165.22.1.228) cannot reach local network (10.48.200.x) directly
• To reach local VMs from DO: use agent commands (shell type) or FortiGate DDNS
• PVE1 SSH via DDNS works: root@orbisne.fortiddns.com (Joker1974!!!)
• PVE2 has no external port forward — only reachable locally or via cluster API through PVE1
• Proxmox API port 8006 IS forwarded: orbisne.fortiddns.com:8006 works from DO

JARVIS Agents

• Agent config: /etc/jarvis-agent/config.json | Runtime state: /var/lib/jarvis-agent/state.json
• 401 "Invalid agent key" → state.json has stale key. Fix: overwrite state.json with correct agent_id + api_key from registered_agents table, then systemctl restart jarvis-agent
• Agent heartbeat uses X-Agent-Key header (NOT body field)
• shell command type requires {"command":"...","allowed":true} in command_data
• Metrics stored as JSON in metric_data column — use JSON_EXTRACT(metric_data,'$.cpu_percent') NOT direct columns

Groq AI

• Model name: compound-beta-mini — NOT groq/compound-beta-mini (that's OpenAI router syntax, 404s)

Proxmox

• stats_cache.php uses orbisne.fortiddns.com:8006 NOT PROXMOX_HOST (local IP unreachable from DO)
• --nameserver in Proxmox must be space-separated: "8.8.8.8 1.1.1.1" (comma causes netplan bug)
• Run commands in VMs: qm guest exec <VMID> -- bash -c 'cmd' (requires guest agent installed)

Deploy

• Always git add + commit + push after editing files on server — webhook auto-deploys within 1 min
• PHP syntax validated before deploy — bad commits auto-reverted
• LSAPI session deadlock: session_write_close() must be called in api.php after auth check

API Endpoint Auth

• Netscan endpoint (/api/netscan) bypasses main auth — uses X-Registration-Key header
• Admin portal uses separate PHP session name (jarvis_admin) — different from main JARVIS session
• Cloudflare real IP: use $_SERVER['HTTP_CF_CONNECTING_IP'] not REMOTE_ADDR

Network Scan

• The JARVIS "RUN NETWORK SCAN" button does NOT scan from DO (can't reach local network)
• It queues a shell command to PVE1 agent → PVE1 runs nmap → pushes results to /api/netscan
• Results appear ~40 seconds after clicking (10s for agent pickup + 30s nmap)
• Chat "scan network" intent returns real DB data — never hallucinated

FusionPBX

• SIP config changes need cache delete before they take effect: rm /var/cache/fusionpbx/FusionPBX.configuration.sofia.conf
• mod_presence is NOT installed on this server

Backup Agent State Fix (Common Issue)

# If an agent shows "Invalid agent key" after reinstall:
# 1. Get correct values from DB
mysql -u jarvis_user -pJ4rv1s_Pr0t0c0l_2026! jarvis_db -e \
  "SELECT agent_id, api_key FROM registered_agents WHERE hostname='<hostname>';"
# 2. Overwrite state on the agent machine
cat > /var/lib/jarvis-agent/state.json << EOF
{"api_key": "<api_key_from_db>", "agent_id": "<agent_id_from_db>"}
EOF
systemctl restart jarvis-agent