CIS490/scripts/install-msfrpcd.sh
max 1b6c7b2f4a Collectors 2/4/5 + fleet runner + sample manifest + Tier-3 setup scripts
This is the chunk that makes "real data" actually flow on multiple
hosts in parallel. End-to-end pipe was up at 613c6fa / 2579683; now
the lab-host side has the diversity + concurrency it needs.

Collectors landed:
  collectors/qmp.py          — source 2 (oracle). Tiny synchronous QMP
                               client + row builder + run loop. Tolerates
                               older qemu without query-stats.
  collectors/guest_agent.py  — source 5 (deployable). Reads the
                               virtio-serial host-side socket, parses
                               agent JSON-lines, re-stamps to the host
                               monotonic clock, persists.
  collectors/pcap.py         — source 4 (deployable). tcpdump capture
                               + pure-Python pcap reader + 100 ms
                               netflow.jsonl bucketizer. Decodes
                               Ethernet/IPv4/TCP/UDP enough for the
                               schema in docs/data-model.md.

In-guest agent:
  vm/guest-agent/cis490_agent.py — stdlib-only Python agent. Reads
    /proc/{stat,meminfo,loadavg,net/dev,net/tcp*}, top-N RSS procs,
    thermal. Writes JSON-lines to /dev/virtio-ports/cis490.guest.agent.
  tools/build_cidata.py — embeds the agent + an OpenRC service into
    user-data so first boot of the Alpine cidata image auto-starts it.

Launchers:
  vm/launch_demo.sh / launch_target.sh — second virtio-serial port for
    the agent socket; SLOT env support so multiple VMs run without
    socket / port collisions; PORT_BASE on launch_target so multiple
    target VMs hostfwd different host ports.
  vm/setup_bridge.sh — creates host-only br-malware (10.200.0.1/24,
    no NAT). Idempotent.

Fleet:
  orchestrator/fleet.py — capacity detector (cores / RAM / load
    headroom) + concurrent-slot runner. Per-slot ENV selects the
    sample. FleetCapacity dataclass round-trips into meta.json so
    "this episode ran with 6 concurrent VMs" is auditable post-hoc.
  tools/run_fleet.py — CLI: --capacity report; --waves N runs N
    waves of (max_concurrent) episodes each, every slot with a
    different sample.
  etc/cis490-orchestrator.service — now drives the fleet runner with
    Restart=always so each invocation runs one wave and respawns,
    giving a continuous stream.

Samples:
  samples/manifest.toml — six profiles spanning the five major
    behaviour shapes. Each entry is real OR mimic (sha256 distinguishes).
  samples/manifest.py — strict TOML loader (rejects dups, unknown
    categories) + deterministic select(host_id, slot, episode_index)
    so different hosts on the network walk the catalog in different
    orders without any coordinator.

EpisodeRunner:
  orchestrator/episode.py — optional qmp_socket + guest_agent_socket
    fields on EpisodeConfig; when set, additional collector threads
    run alongside proc_qemu. EpisodeResult now carries rows_qmp +
    rows_guest counters.

Tier-3 setup automation:
  scripts/install-msfrpcd.sh — installs metasploit-framework where
    the package manager has it, generates a strong password into
    /etc/cis490/msfrpc.env, drops a hardened systemd unit bound to
    127.0.0.1:55553. After this, run_tier3_demo.py works zero-touch
    once MSFRPC_PASSWORD is sourced.
  scripts/fetch-metasploitable2.sh — accepts IMAGE_URL + IMAGE_SHA256
    from the operator (Rapid7 download is registration-walled), pulls,
    verifies, converts vmdk → qcow2, lands at vm/images/.

Tests: 82 pass (was 51). New suites:
  tests/test_qmp.py       — fake QMP server, capability handshake,
                            blockstats, async-event interleaving,
                            5-failure backoff
  tests/test_guest_agent.py — fake virtio socket, JSON-lines read +
                              re-stamp, malformed-line tolerance
  tests/test_pcap.py      — synthetic pcap with TCP/UDP/ARP frames,
                            bucketize correctness across windows
  tests/test_fleet.py     — capacity math (8-core idle / low-RAM /
                            high-load / Pi5 / 1-core box), manifest
                            selection determinism + diversity

What's queued for the next commit (already discussed in convo):
  - MSFExploitDriver v2: map sample.profile → distinct in-session
    workload so Tier-3 episodes don't all produce the same yes-loop
    envelope. Critical for ML to learn varied malware shapes.
  - Real-sample fetch from MalwareBazaar by sha256.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-30 00:02:27 -05:00

124 lines
4.5 KiB
Bash
Executable file

#!/usr/bin/env bash
# Install + configure ``msfrpcd`` for the Tier-3 exploit driver.
#
# Idempotent: re-running on a host that already has msfrpcd refreshes
# the systemd unit and credentials but doesn't reinstall the framework.
#
# Steps:
# 1. Install metasploit-framework via the host package manager (or
# report the right one-liner for that distro). Big download —
# ~1 GiB and several minutes.
# 2. Generate a strong password and store at /etc/cis490/msfrpc.env
# (mode 0640, owner root:cis490).
# 3. Drop /etc/systemd/system/cis490-msfrpcd.service that runs
# msfrpcd bound to 127.0.0.1:55553 with the generated password.
# 4. Enable + start.
#
# After this runs, ``MSFRPC_PASSWORD=$(. /etc/cis490/msfrpc.env;
# echo $MSFRPC_PASSWORD)`` makes tools/run_tier3_demo.py work zero-touch.
set -euo pipefail
ETC_ROOT="/etc/cis490"
ENV_FILE="$ETC_ROOT/msfrpc.env"
UNIT="/etc/systemd/system/cis490-msfrpcd.service"
PORT="${MSFRPC_PORT:-55553}"
USER_NAME="${MSFRPC_USER:-msf}"
log() { printf '[install-msfrpcd] %s\n' "$*" >&2; }
die() { log "FATAL: $*"; exit 1; }
[[ $EUID -eq 0 ]] || die "must run as root"
command -v systemctl >/dev/null || die "systemd not found"
# --- 1. install metasploit-framework -----------------------------------
if ! command -v msfrpcd >/dev/null; then
log "msfrpcd not found; installing metasploit-framework"
if command -v apt-get >/dev/null; then
# The Debian/Ubuntu metasploit-framework package isn't in
# the default repos for most distros. Use Rapid7's official
# nightly installer when available.
if [[ ! -x /opt/metasploit-framework/bin/msfrpcd ]]; then
log "fetching Rapid7 nightly installer"
curl -fsSL https://raw.githubusercontent.com/rapid7/metasploit-omnibus/master/config/templates/metasploit-framework-wrappers/msfupdate.erb \
-o /tmp/msfinstall.sh || true
log "automated install not available — install manually:"
log " https://docs.metasploit.com/docs/using-metasploit/getting-started/nightly-installers.html"
die "rerun once msfrpcd is on PATH"
fi
# Symlink the wrapper so ``msfrpcd`` is on PATH.
ln -sf /opt/metasploit-framework/bin/msfrpcd /usr/local/bin/msfrpcd
elif command -v pacman >/dev/null; then
log "pacman -S metasploit"
pacman -Sy --noconfirm metasploit
elif command -v dnf >/dev/null; then
die "Fedora/RHEL: install metasploit-framework manually, then re-run"
else
die "unknown package manager — install metasploit-framework manually"
fi
fi
command -v msfrpcd >/dev/null || die "msfrpcd still missing after install attempt"
# --- 2. generate password ----------------------------------------------
install -d -m 0755 -o root -g root "$ETC_ROOT"
if ! id -u cis490 >/dev/null 2>&1; then
useradd --system --no-create-home --shell /usr/sbin/nologin cis490
fi
if [[ ! -f "$ENV_FILE" ]]; then
log "generating msfrpc password"
PW="$(openssl rand -base64 24 | tr -d '/+=' | head -c 32)"
install -m 0640 -o root -g cis490 /dev/stdin "$ENV_FILE" <<EOF
# Auto-generated by install-msfrpcd.sh — do not edit.
MSFRPC_HOST=127.0.0.1
MSFRPC_PORT=$PORT
MSFRPC_USER=$USER_NAME
MSFRPC_PASSWORD=$PW
EOF
else
log "$ENV_FILE exists; preserving existing password"
fi
# --- 3. systemd unit ----------------------------------------------------
log "installing systemd unit"
cat > "$UNIT" <<EOF
[Unit]
Description=CIS490 — Metasploit RPC daemon (loopback only)
Documentation=https://maxgit.wg/spectral/CIS490
After=network-online.target
Wants=network-online.target
[Service]
Type=simple
EnvironmentFile=$ENV_FILE
# msfrpcd flags:
# -P <pw> password
# -U <user> username
# -a <ip> bind address (loopback only — Tier-3 driver runs locally)
# -p <port> port
# -f foreground (no daemonization, so systemd manages PID)
ExecStart=/usr/bin/env msfrpcd -P \${MSFRPC_PASSWORD} -U \${MSFRPC_USER} -a 127.0.0.1 -p \${MSFRPC_PORT} -f
Restart=on-failure
RestartSec=5
NoNewPrivileges=true
PrivateTmp=true
ProtectSystem=full
ProtectHome=true
[Install]
WantedBy=multi-user.target
EOF
systemctl daemon-reload
systemctl enable --now cis490-msfrpcd
# --- 4. final smoke -----------------------------------------------------
sleep 2
if ! ss -ltn 2>/dev/null | grep -q ":$PORT"; then
log "WARN: nothing listening on 127.0.0.1:$PORT yet — check"
log " journalctl -u cis490-msfrpcd"
fi
log "done. To run a Tier-3 episode:"
log " set -a; . $ENV_FILE; set +a"
log " python tools/run_tier3_demo.py --module vsftpd_234_backdoor"