bolyai/CIS490
3
0
Fork 0
Code Issues 5 Pull requests Projects 1 Releases Packages Wiki Activity Actions

Deploy Tier 3 + Tier 4 on k-gamingcom (msfrpcd, Metasploitable2, optional bridge) #17

New issue
Open
opened 2026-04-30 22:57:53 -05:00 by max · 5 comments
max commented 2026-04-30 22:57:53 -05:00
Owner
Copy link

Context

This lab host has shipped Tier-2 episodes only — meta.exploit.module_name
and meta.sample.sha256 are null in 100% of episodes from this host
(verified on the receiver Pi). The orchestrator and fleet runner have
no bug; both auto-detect Tier-3 readiness via the _msfrpcd_available()
probe and switch automatically. What's missing is operator-side
deployment of the prerequisites.

Steps (one-time, ~20 min)

Follow AGENTS.md → "Tier 3 + Tier 4 deploy". The short version:

  1. sudo /opt/cis490/scripts/install-msfrpcd.sh — installs
    metasploit-framework + drops cis490-msfrpcd.service. ~1 GiB
    download the first time.
  2. sudo systemctl enable --now cis490-msfrpcd.service and confirm
    systemctl is-active cis490-msfrpcd.service prints active.
  3. Get a Metasploitable2 download URL + sha256 from your Rapid7
    registration (the URL is registration-walled; the script supports
    any working mirror as long as the sha256 matches). Then: 
    IMAGE_URL=<url> IMAGE_SHA256=<sha> sudo OUT_DIR=/var/lib/cis490/vm/images \
    /opt/cis490/scripts/fetch-metasploitable2.sh
  4. (Optional but unlocks 3 of 5 modules.) sudo /opt/cis490/scripts/setup_bridge.sh.

Verify

sudo -u cis490 \
    MSFRPC_PASSWORD="$(. /etc/cis490/msfrpc.env; echo $MSFRPC_PASSWORD)" \
    /opt/cis490/.venv/bin/python \
    /opt/cis490/tools/run_tier3_demo.py \
    --module vsftpd_234_backdoor \
    --target-port 21 --target-boot-timeout 240

If this prints module loaded: vsftpd_234_backdoor (exploit/unix/ftp/vsftpd_234_backdoor)
and episode_id = 01... at the end, Tier 3 is live on k-gamingcom.

Tier 4 (optional, follow-up)

Once Tier 3 is up, sign up for a free MalwareBazaar API key
(https://bazaar.abuse.ch/), pick one sample sha256 per family, run
fetch_sample.py, and add the sha256 + source + url fields to
samples/manifest.toml. Restart cis490-orchestrator. The
deterministic selector will start picking real binaries automatically.

Reference

  • AGENTS.md → "Tier 3 + Tier 4 deploy"
  • Issue #8 (elliott-lab — same gap, originally filed there)
  • scripts/install-msfrpcd.sh, scripts/fetch-metasploitable2.sh
  • tools/run_tier3_demo.py, tools/fetch_sample.py

Don't shortcut

  • DO NOT install metasploit-framework manually outside the script —
    the orchestrator probe expects the systemd unit + the password file
    at /etc/cis490/msfrpc.env.
  • DO NOT push your MalwareBazaar API key. samples/.bazaar.token is
    gitignored.
  • DO NOT add unverified sha256s to manifest.toml — fetch_sample.py
    hash-verifies on download.
## Context This lab host has shipped Tier-2 episodes only — `meta.exploit.module_name` and `meta.sample.sha256` are null in 100% of episodes from this host (verified on the receiver Pi). The orchestrator and fleet runner have no bug; both auto-detect Tier-3 readiness via the `_msfrpcd_available()` probe and switch automatically. What's missing is operator-side deployment of the prerequisites. ## Steps (one-time, ~20 min) Follow AGENTS.md → "Tier 3 + Tier 4 deploy". The short version: 1. `sudo /opt/cis490/scripts/install-msfrpcd.sh` — installs metasploit-framework + drops cis490-msfrpcd.service. ~1 GiB download the first time. 2. `sudo systemctl enable --now cis490-msfrpcd.service` and confirm `systemctl is-active cis490-msfrpcd.service` prints `active`. 3. Get a Metasploitable2 download URL + sha256 from your Rapid7 registration (the URL is registration-walled; the script supports any working mirror as long as the sha256 matches). Then: ``` IMAGE_URL=<url> IMAGE_SHA256=<sha> sudo OUT_DIR=/var/lib/cis490/vm/images \ /opt/cis490/scripts/fetch-metasploitable2.sh ``` 4. (Optional but unlocks 3 of 5 modules.) `sudo /opt/cis490/scripts/setup_bridge.sh`. ## Verify ``` sudo -u cis490 \ MSFRPC_PASSWORD="$(. /etc/cis490/msfrpc.env; echo $MSFRPC_PASSWORD)" \ /opt/cis490/.venv/bin/python \ /opt/cis490/tools/run_tier3_demo.py \ --module vsftpd_234_backdoor \ --target-port 21 --target-boot-timeout 240 ``` If this prints `module loaded: vsftpd_234_backdoor (exploit/unix/ftp/vsftpd_234_backdoor)` and `episode_id = 01...` at the end, Tier 3 is live on k-gamingcom. ## Tier 4 (optional, follow-up) Once Tier 3 is up, sign up for a free MalwareBazaar API key (https://bazaar.abuse.ch/), pick one sample sha256 per family, run `fetch_sample.py`, and add the sha256 + source + url fields to `samples/manifest.toml`. Restart cis490-orchestrator. The deterministic selector will start picking real binaries automatically. ## Reference - AGENTS.md → "Tier 3 + Tier 4 deploy" - Issue #8 (elliott-lab — same gap, originally filed there) - `scripts/install-msfrpcd.sh`, `scripts/fetch-metasploitable2.sh` - `tools/run_tier3_demo.py`, `tools/fetch_sample.py` ## Don't shortcut - DO NOT install metasploit-framework manually outside the script — the orchestrator probe expects the systemd unit + the password file at /etc/cis490/msfrpc.env. - DO NOT push your MalwareBazaar API key. samples/.bazaar.token is gitignored. - DO NOT add unverified sha256s to manifest.toml — fetch_sample.py hash-verifies on download.
max commented 2026-04-30 23:12:22 -05:00
Author
Owner
Copy link

Updated in 683bfe9 — fully automated. The deploy is now ONE command:

sudo /opt/cis490/scripts/install-tier-3-4.sh

No operator URL/sha needed for Metasploitable2 (SourceForge mirror baked in, TOFU sha256 pinning), no manual metasploit-framework install (Rapid7 omnibus auto-runs), no manual manifest edits for Tier 4 (auto_fetch_samples.py walks each family if MALWAREBAZAAR_API_KEY is set). install-lab-host.sh now calls this automatically after the mTLS cert lands, so a fresh host gets Tier 3 zero-touch from the second pass.

If you've already done the install-lab-host.sh dance on this host, just pull main + run:

cd /opt/cis490 && sudo -u cis490 git pull origin main
sudo /opt/cis490/scripts/install-tier-3-4.sh
# optional Tier 4:
#   MALWAREBAZAAR_API_KEY=<key> sudo /opt/cis490/scripts/install-tier-3-4.sh
sudo systemctl restart cis490-orchestrator

The script ends with a status block; paste that here and we're done.

Updated in 683bfe9 — fully automated. The deploy is now ONE command: ``` sudo /opt/cis490/scripts/install-tier-3-4.sh ``` No operator URL/sha needed for Metasploitable2 (SourceForge mirror baked in, TOFU sha256 pinning), no manual metasploit-framework install (Rapid7 omnibus auto-runs), no manual manifest edits for Tier 4 (auto_fetch_samples.py walks each family if MALWAREBAZAAR_API_KEY is set). install-lab-host.sh now calls this automatically after the mTLS cert lands, so a fresh host gets Tier 3 zero-touch from the second pass. If you've already done the install-lab-host.sh dance on this host, just pull main + run: ``` cd /opt/cis490 && sudo -u cis490 git pull origin main sudo /opt/cis490/scripts/install-tier-3-4.sh # optional Tier 4: # MALWAREBAZAAR_API_KEY=<key> sudo /opt/cis490/scripts/install-tier-3-4.sh sudo systemctl restart cis490-orchestrator ``` The script ends with a status block; paste that here and we're done.
max commented 2026-05-01 00:44:57 -05:00
Author
Owner
Copy link

Tier 4 is now mandatory (5d0e8e3). install-tier-3-4.sh hard-fails if zero real binaries land. The MB API key is auto-distributed from the Pi's bootstrap.wg endpoint — operator runs scripts/set-malwarebazaar-key.sh ONCE on the Pi, lab hosts pull the key automatically.

When ready on this host:

cd /opt/cis490 && sudo -u cis490 git pull origin main
sudo /opt/cis490/scripts/install-tier-3-4.sh
sudo systemctl restart cis490-orchestrator

If the Pi-side key isn't provisioned yet, the script will tell you. Don't try to bypass with SKIP_TIER4 — that defeats the point of the project.

Tier 4 is now mandatory (5d0e8e3). install-tier-3-4.sh hard-fails if zero real binaries land. The MB API key is auto-distributed from the Pi's bootstrap.wg endpoint — operator runs scripts/set-malwarebazaar-key.sh ONCE on the Pi, lab hosts pull the key automatically. When ready on this host: ``` cd /opt/cis490 && sudo -u cis490 git pull origin main sudo /opt/cis490/scripts/install-tier-3-4.sh sudo systemctl restart cis490-orchestrator ``` If the Pi-side key isn't provisioned yet, the script will tell you. Don't try to bypass with SKIP_TIER4 — that defeats the point of the project.
max commented 2026-05-01 01:18:05 -05:00
Author
Owner
Copy link

Updated again (265f3ad). Tier 4 now uses theZoo — public security-research repo, no auth, no signup, no operator step. Zero-touch end-to-end:

cd /opt/cis490 && sudo -u cis490 git pull origin main
sudo /opt/cis490/scripts/install-tier-3-4.sh
sudo systemctl restart cis490-orchestrator

The deploy clones theZoo (~500MB, one-time per host, cached at /var/lib/cis490/theZoo), walks each manifest family, extracts the password-protected zip ('infected'), stages real binaries at /opt/cis490/samples/store/, and rewrites manifest.toml. Hard-fails if zero binaries land — no silent mimic-only fallback.

Safety note: theZoo contains live malware. Lab hosts run it inside the VM only (chunked upload via msfrpcd → in-session execute on Metasploitable2 → snapshot revert at episode end). It NEVER touches the lab host directly.

Ignore the earlier MalwareBazaar / bootstrap.wg secret instructions — that whole code path was removed.

Updated again (265f3ad). Tier 4 now uses theZoo — public security-research repo, no auth, no signup, no operator step. Zero-touch end-to-end: ``` cd /opt/cis490 && sudo -u cis490 git pull origin main sudo /opt/cis490/scripts/install-tier-3-4.sh sudo systemctl restart cis490-orchestrator ``` The deploy clones theZoo (~500MB, one-time per host, cached at /var/lib/cis490/theZoo), walks each manifest family, extracts the password-protected zip ('infected'), stages real binaries at /opt/cis490/samples/store/<sha256>, and rewrites manifest.toml. Hard-fails if zero binaries land — no silent mimic-only fallback. Safety note: theZoo contains live malware. Lab hosts run it inside the VM only (chunked upload via msfrpcd → in-session execute on Metasploitable2 → snapshot revert at episode end). It NEVER touches the lab host directly. Ignore the earlier MalwareBazaar / bootstrap.wg secret instructions — that whole code path was removed.
max commented 2026-05-01 01:43:07 -05:00
Author
Owner
Copy link

Receiver now enforces a code-commit allow-list (cc0c969). Every PUT must carry X-Cis490-Code-Commit and that commit must be in the receiver's window of recent commits to spectral/CIS490@main. The allow-list auto-refreshes from Forgejo every 5 seconds — when new code is pushed, the new hash is acceptable instantly with no service restart.

What this means for this lab host:

  • pull main on this host before generating new data: cd /opt/cis490 && sudo -u cis490 git pull origin main
  • re-run install-lab-host.sh so VERSION gets restamped
  • restart the orchestrator + shipper
  • next ship carries X-Cis490-Code-Commit and lands cleanly

If you ship under stale code, the receiver returns 412 with a remediation block printed verbatim into your shipper journal — exact commands to run. Don't try to bypass; the gate exists so buggy pre-fix episodes don't pollute the training set.

The 1067 episodes already collected (without code_version stamps) are unaffected — the gate only runs on new PUTs. Existing index rows stay valid; new rows gain a 'commit' field for trainer-side filtering.

Receiver now enforces a code-commit allow-list (cc0c969). Every PUT must carry X-Cis490-Code-Commit and that commit must be in the receiver's window of recent commits to spectral/CIS490@main. The allow-list auto-refreshes from Forgejo every 5 seconds — when new code is pushed, the new hash is acceptable instantly with no service restart. What this means for this lab host: - pull main on this host before generating new data: `cd /opt/cis490 && sudo -u cis490 git pull origin main` - re-run install-lab-host.sh so VERSION gets restamped - restart the orchestrator + shipper - next ship carries X-Cis490-Code-Commit and lands cleanly If you ship under stale code, the receiver returns 412 with a remediation block printed verbatim into your shipper journal — exact commands to run. Don't try to bypass; the gate exists so buggy pre-fix episodes don't pollute the training set. The 1067 episodes already collected (without code_version stamps) are unaffected — the gate only runs on new PUTs. Existing index rows stay valid; new rows gain a 'commit' field for trainer-side filtering.
max commented 2026-05-01 11:54:09 -05:00
Author
Owner
Copy link

The deploy steps in this issue are out of date — install-lab-host.sh and scripts/install-tier-3-4.sh now do everything end-to-end with zero operator interaction, including no Rapid7 registration:

  • msfrpcd: auto-installed via the Rapid7 omnibus installer (no signup).
  • Metasploitable2: auto-fetched from the public SourceForge mirror with TOFU sha256 pinning (no Rapid7 download form).
  • Tier 4 real-malware samples: auto-fetched from theZoo (no API key).
  • Bridge br-malware: auto-comes-up if the kernel supports it.
  • Live verify fire (vsftpd_234_backdoor) runs and the script returns non-zero if no episode landed.

Just pull and re-run the installer:

cd /opt/cis490
sudo -u cis490 git pull origin main
sudo /opt/cis490/scripts/install-lab-host.sh

That now also (a) drains pre-stamp episodes via tools/quarantine_unstamped.py, and (b) restarts cis490-shipper + cis490-orchestrator so the new code reaches the live daemon. See AGENTS.md → "Tier 3 + Tier 4 deploy" and "Shipper says 400/412".

I'll close this issue once Tier-3 episodes start landing in the receiver index.

The deploy steps in this issue are out of date — `install-lab-host.sh` and `scripts/install-tier-3-4.sh` now do everything end-to-end with **zero operator interaction**, including no Rapid7 registration: - **msfrpcd**: auto-installed via the Rapid7 omnibus installer (no signup). - **Metasploitable2**: auto-fetched from the public SourceForge mirror with TOFU sha256 pinning (no Rapid7 download form). - **Tier 4 real-malware samples**: auto-fetched from theZoo (no API key). - **Bridge `br-malware`**: auto-comes-up if the kernel supports it. - **Live verify fire** (`vsftpd_234_backdoor`) runs and the script returns non-zero if no episode landed. Just pull and re-run the installer: ```sh cd /opt/cis490 sudo -u cis490 git pull origin main sudo /opt/cis490/scripts/install-lab-host.sh ``` That now also (a) drains pre-stamp episodes via `tools/quarantine_unstamped.py`, and (b) restarts `cis490-shipper` + `cis490-orchestrator` so the new code reaches the live daemon. See AGENTS.md → "Tier 3 + Tier 4 deploy" and "Shipper says 400/412". I'll close this issue once Tier-3 episodes start landing in the receiver index.
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
Dev_REL3_050126
Dev_REL2_043026
Dev_REL1_043026
No results found.
Labels
Clear labels
No items
No labels
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: bolyai/CIS490#17
No description provided.
Delete branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?