This is the chunk that makes "real data" actually flow on multiple
hosts in parallel. End-to-end pipe was up at 613c6fa / 2579683; now
the lab-host side has the diversity + concurrency it needs.
Collectors landed:
collectors/qmp.py — source 2 (oracle). Tiny synchronous QMP
client + row builder + run loop. Tolerates
older qemu without query-stats.
collectors/guest_agent.py — source 5 (deployable). Reads the
virtio-serial host-side socket, parses
agent JSON-lines, re-stamps to the host
monotonic clock, persists.
collectors/pcap.py — source 4 (deployable). tcpdump capture
+ pure-Python pcap reader + 100 ms
netflow.jsonl bucketizer. Decodes
Ethernet/IPv4/TCP/UDP enough for the
schema in docs/data-model.md.
In-guest agent:
vm/guest-agent/cis490_agent.py — stdlib-only Python agent. Reads
/proc/{stat,meminfo,loadavg,net/dev,net/tcp*}, top-N RSS procs,
thermal. Writes JSON-lines to /dev/virtio-ports/cis490.guest.agent.
tools/build_cidata.py — embeds the agent + an OpenRC service into
user-data so first boot of the Alpine cidata image auto-starts it.
Launchers:
vm/launch_demo.sh / launch_target.sh — second virtio-serial port for
the agent socket; SLOT env support so multiple VMs run without
socket / port collisions; PORT_BASE on launch_target so multiple
target VMs hostfwd different host ports.
vm/setup_bridge.sh — creates host-only br-malware (10.200.0.1/24,
no NAT). Idempotent.
Fleet:
orchestrator/fleet.py — capacity detector (cores / RAM / load
headroom) + concurrent-slot runner. Per-slot ENV selects the
sample. FleetCapacity dataclass round-trips into meta.json so
"this episode ran with 6 concurrent VMs" is auditable post-hoc.
tools/run_fleet.py — CLI: --capacity report; --waves N runs N
waves of (max_concurrent) episodes each, every slot with a
different sample.
etc/cis490-orchestrator.service — now drives the fleet runner with
Restart=always so each invocation runs one wave and respawns,
giving a continuous stream.
Samples:
samples/manifest.toml — six profiles spanning the five major
behaviour shapes. Each entry is real OR mimic (sha256 distinguishes).
samples/manifest.py — strict TOML loader (rejects dups, unknown
categories) + deterministic select(host_id, slot, episode_index)
so different hosts on the network walk the catalog in different
orders without any coordinator.
EpisodeRunner:
orchestrator/episode.py — optional qmp_socket + guest_agent_socket
fields on EpisodeConfig; when set, additional collector threads
run alongside proc_qemu. EpisodeResult now carries rows_qmp +
rows_guest counters.
Tier-3 setup automation:
scripts/install-msfrpcd.sh — installs metasploit-framework where
the package manager has it, generates a strong password into
/etc/cis490/msfrpc.env, drops a hardened systemd unit bound to
127.0.0.1:55553. After this, run_tier3_demo.py works zero-touch
once MSFRPC_PASSWORD is sourced.
scripts/fetch-metasploitable2.sh — accepts IMAGE_URL + IMAGE_SHA256
from the operator (Rapid7 download is registration-walled), pulls,
verifies, converts vmdk → qcow2, lands at vm/images/.
Tests: 82 pass (was 51). New suites:
tests/test_qmp.py — fake QMP server, capability handshake,
blockstats, async-event interleaving,
5-failure backoff
tests/test_guest_agent.py — fake virtio socket, JSON-lines read +
re-stamp, malformed-line tolerance
tests/test_pcap.py — synthetic pcap with TCP/UDP/ARP frames,
bucketize correctness across windows
tests/test_fleet.py — capacity math (8-core idle / low-RAM /
high-load / Pi5 / 1-core box), manifest
selection determinism + diversity
What's queued for the next commit (already discussed in convo):
- MSFExploitDriver v2: map sample.profile → distinct in-session
workload so Tier-3 episodes don't all produce the same yes-loop
envelope. Critical for ML to learn varied malware shapes.
- Real-sample fetch from MalwareBazaar by sha256.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
124 lines
4.5 KiB
Bash
Executable file
124 lines
4.5 KiB
Bash
Executable file
#!/usr/bin/env bash
|
|
# Install + configure ``msfrpcd`` for the Tier-3 exploit driver.
|
|
#
|
|
# Idempotent: re-running on a host that already has msfrpcd refreshes
|
|
# the systemd unit and credentials but doesn't reinstall the framework.
|
|
#
|
|
# Steps:
|
|
# 1. Install metasploit-framework via the host package manager (or
|
|
# report the right one-liner for that distro). Big download —
|
|
# ~1 GiB and several minutes.
|
|
# 2. Generate a strong password and store at /etc/cis490/msfrpc.env
|
|
# (mode 0640, owner root:cis490).
|
|
# 3. Drop /etc/systemd/system/cis490-msfrpcd.service that runs
|
|
# msfrpcd bound to 127.0.0.1:55553 with the generated password.
|
|
# 4. Enable + start.
|
|
#
|
|
# After this runs, ``MSFRPC_PASSWORD=$(. /etc/cis490/msfrpc.env;
|
|
# echo $MSFRPC_PASSWORD)`` makes tools/run_tier3_demo.py work zero-touch.
|
|
|
|
set -euo pipefail
|
|
|
|
ETC_ROOT="/etc/cis490"
|
|
ENV_FILE="$ETC_ROOT/msfrpc.env"
|
|
UNIT="/etc/systemd/system/cis490-msfrpcd.service"
|
|
PORT="${MSFRPC_PORT:-55553}"
|
|
USER_NAME="${MSFRPC_USER:-msf}"
|
|
|
|
log() { printf '[install-msfrpcd] %s\n' "$*" >&2; }
|
|
die() { log "FATAL: $*"; exit 1; }
|
|
|
|
[[ $EUID -eq 0 ]] || die "must run as root"
|
|
command -v systemctl >/dev/null || die "systemd not found"
|
|
|
|
# --- 1. install metasploit-framework -----------------------------------
|
|
if ! command -v msfrpcd >/dev/null; then
|
|
log "msfrpcd not found; installing metasploit-framework"
|
|
if command -v apt-get >/dev/null; then
|
|
# The Debian/Ubuntu metasploit-framework package isn't in
|
|
# the default repos for most distros. Use Rapid7's official
|
|
# nightly installer when available.
|
|
if [[ ! -x /opt/metasploit-framework/bin/msfrpcd ]]; then
|
|
log "fetching Rapid7 nightly installer"
|
|
curl -fsSL https://raw.githubusercontent.com/rapid7/metasploit-omnibus/master/config/templates/metasploit-framework-wrappers/msfupdate.erb \
|
|
-o /tmp/msfinstall.sh || true
|
|
log "automated install not available — install manually:"
|
|
log " https://docs.metasploit.com/docs/using-metasploit/getting-started/nightly-installers.html"
|
|
die "rerun once msfrpcd is on PATH"
|
|
fi
|
|
# Symlink the wrapper so ``msfrpcd`` is on PATH.
|
|
ln -sf /opt/metasploit-framework/bin/msfrpcd /usr/local/bin/msfrpcd
|
|
elif command -v pacman >/dev/null; then
|
|
log "pacman -S metasploit"
|
|
pacman -Sy --noconfirm metasploit
|
|
elif command -v dnf >/dev/null; then
|
|
die "Fedora/RHEL: install metasploit-framework manually, then re-run"
|
|
else
|
|
die "unknown package manager — install metasploit-framework manually"
|
|
fi
|
|
fi
|
|
|
|
command -v msfrpcd >/dev/null || die "msfrpcd still missing after install attempt"
|
|
|
|
# --- 2. generate password ----------------------------------------------
|
|
install -d -m 0755 -o root -g root "$ETC_ROOT"
|
|
if ! id -u cis490 >/dev/null 2>&1; then
|
|
useradd --system --no-create-home --shell /usr/sbin/nologin cis490
|
|
fi
|
|
if [[ ! -f "$ENV_FILE" ]]; then
|
|
log "generating msfrpc password"
|
|
PW="$(openssl rand -base64 24 | tr -d '/+=' | head -c 32)"
|
|
install -m 0640 -o root -g cis490 /dev/stdin "$ENV_FILE" <<EOF
|
|
# Auto-generated by install-msfrpcd.sh — do not edit.
|
|
MSFRPC_HOST=127.0.0.1
|
|
MSFRPC_PORT=$PORT
|
|
MSFRPC_USER=$USER_NAME
|
|
MSFRPC_PASSWORD=$PW
|
|
EOF
|
|
else
|
|
log "$ENV_FILE exists; preserving existing password"
|
|
fi
|
|
|
|
# --- 3. systemd unit ----------------------------------------------------
|
|
log "installing systemd unit"
|
|
cat > "$UNIT" <<EOF
|
|
[Unit]
|
|
Description=CIS490 — Metasploit RPC daemon (loopback only)
|
|
Documentation=https://maxgit.wg/spectral/CIS490
|
|
After=network-online.target
|
|
Wants=network-online.target
|
|
|
|
[Service]
|
|
Type=simple
|
|
EnvironmentFile=$ENV_FILE
|
|
# msfrpcd flags:
|
|
# -P <pw> password
|
|
# -U <user> username
|
|
# -a <ip> bind address (loopback only — Tier-3 driver runs locally)
|
|
# -p <port> port
|
|
# -f foreground (no daemonization, so systemd manages PID)
|
|
ExecStart=/usr/bin/env msfrpcd -P \${MSFRPC_PASSWORD} -U \${MSFRPC_USER} -a 127.0.0.1 -p \${MSFRPC_PORT} -f
|
|
Restart=on-failure
|
|
RestartSec=5
|
|
NoNewPrivileges=true
|
|
PrivateTmp=true
|
|
ProtectSystem=full
|
|
ProtectHome=true
|
|
|
|
[Install]
|
|
WantedBy=multi-user.target
|
|
EOF
|
|
|
|
systemctl daemon-reload
|
|
systemctl enable --now cis490-msfrpcd
|
|
|
|
# --- 4. final smoke -----------------------------------------------------
|
|
sleep 2
|
|
if ! ss -ltn 2>/dev/null | grep -q ":$PORT"; then
|
|
log "WARN: nothing listening on 127.0.0.1:$PORT yet — check"
|
|
log " journalctl -u cis490-msfrpcd"
|
|
fi
|
|
|
|
log "done. To run a Tier-3 episode:"
|
|
log " set -a; . $ENV_FILE; set +a"
|
|
log " python tools/run_tier3_demo.py --module vsftpd_234_backdoor"
|