Elliott Kolden
86fdd03de4
Full bring-up of this host from a clean clone: installed uv/perf/tcpdump, downloaded Alpine 3.21 cloud image, built cidata ISO, took baseline-v1 snapshot. Validated single-episode demo (853 rows, 8 phases) and 2-episode campaign loop (campaign_done.marker written). Cherry-picked campaign runner from Dev_REL1_042926. Fixed .gitignore to cover campaign output files. Issue report at reports/Dev_REL1_043026.md covers ISS-001 through ISS-007, with ISS-005 (missing install-lab-host.sh) remaining open. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
212 lines
6.4 KiB
Markdown
212 lines
6.4 KiB
Markdown
# Setup & Integration Report — Dev_REL1_043026
|
|
|
|
**Date:** 2026-04-30
|
|
**Branch:** Dev_REL1_043026 (day branch off Dev_REL1)
|
|
**Host:** elliott-ThinkPad (Arch Linux, kernel 6.19.14-arch1-1)
|
|
**Scope:** Full lab-host bring-up from a clean `git clone` — install prereqs,
|
|
build VM image, run single episode, validate campaign loop.
|
|
|
|
---
|
|
|
|
## Environment Baseline
|
|
|
|
| Item | Expected | Found | Status |
|
|
|---|---|---|---|
|
|
| `/dev/kvm` | exists | `crw-rw-rw-` (world-writable) | OK |
|
|
| kvm group membership | required | user NOT in group | OK — world perm bypasses group |
|
|
| QEMU | ≥ 8.0 | 11.0.0 | OK |
|
|
| Python | ≥ 3.11 | 3.14.4 | OK |
|
|
| `uv` | installed | missing | Fixed |
|
|
| `perf` | installed | missing | Fixed |
|
|
| `tcpdump` | installed | missing | Fixed |
|
|
| `zstd` | installed | 1.5.7 | OK |
|
|
| `vm/images/` | populated | empty directory missing | Fixed |
|
|
| `scripts/install-lab-host.sh` | exists | missing | **Open — ISS-005** |
|
|
|
|
---
|
|
|
|
## Issues Found and Resolved
|
|
|
|
### ISS-001 — `uv` not installed
|
|
|
|
**Symptom:** `uv: command not found` — unable to install Python deps or run
|
|
any `uv run` command.
|
|
|
|
**Root cause:** `deploy.md` and `lab-setup.md` both reference
|
|
`scripts/install-lab-host.sh` as the mechanism to install `uv`, but that
|
|
script does not exist in the repository.
|
|
|
|
**Fix applied:** Installed system package via pacman (after mirror sync):
|
|
```
|
|
sudo pacman -Sy && sudo pacman -S uv
|
|
```
|
|
Fallback (no sudo): `curl -LsSf https://astral.sh/uv/install.sh | sh`
|
|
installs to `~/.local/bin/uv`.
|
|
|
|
**Residual:** See ISS-005 for the missing installer script.
|
|
|
|
---
|
|
|
|
### ISS-002 — `perf` and `tcpdump` not installed
|
|
|
|
**Symptom:** `perf: command not found`, `tcpdump: command not found`.
|
|
Both are listed as host prerequisites in `docs/lab-setup.md`.
|
|
|
|
**Fix applied:**
|
|
```
|
|
sudo pacman -S perf tcpdump
|
|
```
|
|
|
|
---
|
|
|
|
### ISS-003 — Pacman mirror 404 errors on linux-tools group
|
|
|
|
**Symptom:** `sudo pacman -S linux-tools` failed with 404 errors across all
|
|
configured mirrors. Mirrors held linux-tools 7.0.1 while the database
|
|
referenced 7.0.2.
|
|
|
|
**Root cause:** Local pacman databases were stale.
|
|
|
|
**Fix applied:** `sudo pacman -Sy` refreshed the databases; subsequent
|
|
install of individual packages (`perf`, `tcpdump`, `uv`) succeeded.
|
|
|
|
**Recommendation:** Add `pacman -Sy` as the first step in
|
|
`install-lab-host.sh`.
|
|
|
|
---
|
|
|
|
### ISS-004 — VM images missing (`alpine-baseline.qcow2`, `cidata.iso`)
|
|
|
|
**Symptom:** `launch_demo.sh` exits immediately:
|
|
```
|
|
no image at .../vm/images/alpine-baseline.qcow2
|
|
```
|
|
|
|
**Root cause:** `vm/images/` is correctly gitignored, but the bring-up
|
|
procedure for first-time hosts is undocumented outside of
|
|
`docs/lab-setup.md` (which references the non-existent install script).
|
|
|
|
**Fix applied (manual steps):**
|
|
```bash
|
|
# 1. Create images directory
|
|
mkdir -p vm/images
|
|
|
|
# 2. Download Alpine 3.21 cloud image
|
|
curl -L -o vm/images/alpine-baseline.qcow2 \
|
|
https://dl-cdn.alpinelinux.org/alpine/v3.21/releases/cloud/generic_alpine-3.21.0-x86_64-bios-cloudinit-r0.qcow2
|
|
|
|
# 3. Build cloud-init cidata ISO
|
|
uv run python tools/build_cidata.py vm/images/cidata.iso
|
|
|
|
# 4. Take baseline snapshot (used as metadata label in episodes)
|
|
qemu-img snapshot -c baseline-v1 vm/images/alpine-baseline.qcow2
|
|
```
|
|
|
|
**Note on snapshot mechanics:** `launch_demo.sh` uses `-drive snapshot=on`,
|
|
which creates a temporary write overlay per boot. Cloud-init runs fresh each
|
|
boot (overlay discarded between episodes), so the 35-second settle wait in
|
|
`run_real_vm_demo.py` is required and sufficient. No first-boot without
|
|
`snapshot=on` is needed.
|
|
|
|
**Recommendation:** Add these steps to `docs/lab-setup.md` and automate in
|
|
`install-lab-host.sh`.
|
|
|
|
---
|
|
|
|
### ISS-005 — `scripts/install-lab-host.sh` does not exist (open)
|
|
|
|
**Symptom:** Both `deploy.md` and `lab-setup.md` reference
|
|
`scripts/install-lab-host.sh` as the canonical install path. The `scripts/`
|
|
directory does not exist.
|
|
|
|
**Impact:** Every new lab host requires the manual steps from ISS-001 through
|
|
ISS-004 above.
|
|
|
|
**Status:** Open. Requires writing the install script. Suggested content:
|
|
- `pacman -Sy`
|
|
- Install: `uv`, `perf`, `tcpdump`, `zstd`, `qemu-system-x86_64`,
|
|
`bridge-utils`
|
|
- Download Alpine cloud image if `vm/images/alpine-baseline.qcow2` absent
|
|
- Run `tools/build_cidata.py`
|
|
- Take `baseline-v1` snapshot
|
|
- Drop systemd units from `etc/` into `/etc/systemd/system/`
|
|
- Write `/etc/cis490/lab-host.toml` template
|
|
|
|
---
|
|
|
|
### ISS-006 — Campaign files not promoted to Dev_REL1
|
|
|
|
**Symptom:** `run_campaign.py` and `shipper.py` (committed to
|
|
`Dev_REL1_042926`) were absent from `Dev_REL1_043026` (branched off
|
|
`Dev_REL1` before the merge).
|
|
|
|
**Fix applied:** Cherry-picked commit `ce2fd66` from `Dev_REL1_042926`:
|
|
```
|
|
git cherry-pick ce2fd66
|
|
```
|
|
|
|
**Recommendation:** Merge `Dev_REL1_042926` → `Dev_REL1` before cutting
|
|
future day branches so the base is always current.
|
|
|
|
---
|
|
|
|
### ISS-007 — Campaign output files not in `.gitignore`
|
|
|
|
**Symptom:** `data/campaign.json`, `data/campaign_done.marker`,
|
|
`data/outbox/`, and `data/shipped/` would appear as untracked files after
|
|
a campaign run.
|
|
|
|
**Fix applied:** Added to `.gitignore`:
|
|
```
|
|
data/campaign.json
|
|
data/campaign_done.marker
|
|
data/outbox/
|
|
data/shipped/
|
|
```
|
|
|
|
---
|
|
|
|
## Validation Results
|
|
|
|
All three checks passed on this host.
|
|
|
|
### Single episode (`run_real_vm_demo.py`)
|
|
```
|
|
episode_id = 01KQG2K4WJBE52PCD2QDCQAVKB
|
|
rows_proc = 853
|
|
phases = [clean, armed, infecting, infected_running,
|
|
dormant, infected_running, dormant, clean]
|
|
duration = 85.26s
|
|
```
|
|
|
|
### 2-episode campaign (`run_campaign.py --target 2`)
|
|
```
|
|
episode 1: id=01KQG2QRVRRTVM3TMEKCEW1E6Z rows=853 duration=85.26s ✓
|
|
episode 2: id=01KQG2VMWM6WKEQPCZ3BKYT513 rows=853 duration=85.27s ✓
|
|
campaign_done.marker written ✓
|
|
campaign.json: {"completed": 2, "done": true} ✓
|
|
```
|
|
|
|
### Episode directory structure (each episode)
|
|
```
|
|
data/episodes/<ulid>/
|
|
meta.json ✓
|
|
events.jsonl ✓
|
|
labels.jsonl ✓ (8 phase transitions)
|
|
telemetry-proc.jsonl ✓ (853 rows @ 10 Hz)
|
|
done.marker ✓
|
|
```
|
|
|
|
---
|
|
|
|
## Known Gaps (pre-existing, not introduced this session)
|
|
|
|
| Gap | File | Status |
|
|
|---|---|---|
|
|
| `perf stat` collector | `collectors/perf_qemu.py` | Not yet implemented |
|
|
| QMP collector | `collectors/qmp.py` | Not yet implemented |
|
|
| `tcpdump` / pcap collector | `collectors/pcap.py` | Not yet implemented |
|
|
| In-guest agent collector | `collectors/guest_agent.py` | Not yet implemented |
|
|
| Metasploit exploit driver (Tier 3) | `exploits/` | Planned |
|
|
| Real malware sample (Tier 4) | `samples/` | Planned |
|
|
| `scripts/install-lab-host.sh` | `scripts/` | See ISS-005 |
|