Dev_REL1_043026: lab-host bring-up, fixes, and issue report
Full bring-up of this host from a clean clone: installed uv/perf/tcpdump, downloaded Alpine 3.21 cloud image, built cidata ISO, took baseline-v1 snapshot. Validated single-episode demo (853 rows, 8 phases) and 2-episode campaign loop (campaign_done.marker written). Cherry-picked campaign runner from Dev_REL1_042926. Fixed .gitignore to cover campaign output files. Issue report at reports/Dev_REL1_043026.md covers ISS-001 through ISS-007, with ISS-005 (missing install-lab-host.sh) remaining open. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
parent
842918556b
commit
86fdd03de4
2 changed files with 216 additions and 0 deletions
4
.gitignore
vendored
4
.gitignore
vendored
|
|
@ -11,6 +11,10 @@ vm/snapshots/
|
|||
|
||||
# Telemetry output
|
||||
data/episodes/
|
||||
data/campaign.json
|
||||
data/campaign_done.marker
|
||||
data/outbox/
|
||||
data/shipped/
|
||||
*.pcap
|
||||
*.pcapng
|
||||
|
||||
|
|
|
|||
212
reports/Dev_REL1_043026.md
Normal file
212
reports/Dev_REL1_043026.md
Normal file
|
|
@ -0,0 +1,212 @@
|
|||
# Setup & Integration Report — Dev_REL1_043026
|
||||
|
||||
**Date:** 2026-04-30
|
||||
**Branch:** Dev_REL1_043026 (day branch off Dev_REL1)
|
||||
**Host:** elliott-ThinkPad (Arch Linux, kernel 6.19.14-arch1-1)
|
||||
**Scope:** Full lab-host bring-up from a clean `git clone` — install prereqs,
|
||||
build VM image, run single episode, validate campaign loop.
|
||||
|
||||
---
|
||||
|
||||
## Environment Baseline
|
||||
|
||||
| Item | Expected | Found | Status |
|
||||
|---|---|---|---|
|
||||
| `/dev/kvm` | exists | `crw-rw-rw-` (world-writable) | OK |
|
||||
| kvm group membership | required | user NOT in group | OK — world perm bypasses group |
|
||||
| QEMU | ≥ 8.0 | 11.0.0 | OK |
|
||||
| Python | ≥ 3.11 | 3.14.4 | OK |
|
||||
| `uv` | installed | missing | Fixed |
|
||||
| `perf` | installed | missing | Fixed |
|
||||
| `tcpdump` | installed | missing | Fixed |
|
||||
| `zstd` | installed | 1.5.7 | OK |
|
||||
| `vm/images/` | populated | empty directory missing | Fixed |
|
||||
| `scripts/install-lab-host.sh` | exists | missing | **Open — ISS-005** |
|
||||
|
||||
---
|
||||
|
||||
## Issues Found and Resolved
|
||||
|
||||
### ISS-001 — `uv` not installed
|
||||
|
||||
**Symptom:** `uv: command not found` — unable to install Python deps or run
|
||||
any `uv run` command.
|
||||
|
||||
**Root cause:** `deploy.md` and `lab-setup.md` both reference
|
||||
`scripts/install-lab-host.sh` as the mechanism to install `uv`, but that
|
||||
script does not exist in the repository.
|
||||
|
||||
**Fix applied:** Installed system package via pacman (after mirror sync):
|
||||
```
|
||||
sudo pacman -Sy && sudo pacman -S uv
|
||||
```
|
||||
Fallback (no sudo): `curl -LsSf https://astral.sh/uv/install.sh | sh`
|
||||
installs to `~/.local/bin/uv`.
|
||||
|
||||
**Residual:** See ISS-005 for the missing installer script.
|
||||
|
||||
---
|
||||
|
||||
### ISS-002 — `perf` and `tcpdump` not installed
|
||||
|
||||
**Symptom:** `perf: command not found`, `tcpdump: command not found`.
|
||||
Both are listed as host prerequisites in `docs/lab-setup.md`.
|
||||
|
||||
**Fix applied:**
|
||||
```
|
||||
sudo pacman -S perf tcpdump
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### ISS-003 — Pacman mirror 404 errors on linux-tools group
|
||||
|
||||
**Symptom:** `sudo pacman -S linux-tools` failed with 404 errors across all
|
||||
configured mirrors. Mirrors held linux-tools 7.0.1 while the database
|
||||
referenced 7.0.2.
|
||||
|
||||
**Root cause:** Local pacman databases were stale.
|
||||
|
||||
**Fix applied:** `sudo pacman -Sy` refreshed the databases; subsequent
|
||||
install of individual packages (`perf`, `tcpdump`, `uv`) succeeded.
|
||||
|
||||
**Recommendation:** Add `pacman -Sy` as the first step in
|
||||
`install-lab-host.sh`.
|
||||
|
||||
---
|
||||
|
||||
### ISS-004 — VM images missing (`alpine-baseline.qcow2`, `cidata.iso`)
|
||||
|
||||
**Symptom:** `launch_demo.sh` exits immediately:
|
||||
```
|
||||
no image at .../vm/images/alpine-baseline.qcow2
|
||||
```
|
||||
|
||||
**Root cause:** `vm/images/` is correctly gitignored, but the bring-up
|
||||
procedure for first-time hosts is undocumented outside of
|
||||
`docs/lab-setup.md` (which references the non-existent install script).
|
||||
|
||||
**Fix applied (manual steps):**
|
||||
```bash
|
||||
# 1. Create images directory
|
||||
mkdir -p vm/images
|
||||
|
||||
# 2. Download Alpine 3.21 cloud image
|
||||
curl -L -o vm/images/alpine-baseline.qcow2 \
|
||||
https://dl-cdn.alpinelinux.org/alpine/v3.21/releases/cloud/generic_alpine-3.21.0-x86_64-bios-cloudinit-r0.qcow2
|
||||
|
||||
# 3. Build cloud-init cidata ISO
|
||||
uv run python tools/build_cidata.py vm/images/cidata.iso
|
||||
|
||||
# 4. Take baseline snapshot (used as metadata label in episodes)
|
||||
qemu-img snapshot -c baseline-v1 vm/images/alpine-baseline.qcow2
|
||||
```
|
||||
|
||||
**Note on snapshot mechanics:** `launch_demo.sh` uses `-drive snapshot=on`,
|
||||
which creates a temporary write overlay per boot. Cloud-init runs fresh each
|
||||
boot (overlay discarded between episodes), so the 35-second settle wait in
|
||||
`run_real_vm_demo.py` is required and sufficient. No first-boot without
|
||||
`snapshot=on` is needed.
|
||||
|
||||
**Recommendation:** Add these steps to `docs/lab-setup.md` and automate in
|
||||
`install-lab-host.sh`.
|
||||
|
||||
---
|
||||
|
||||
### ISS-005 — `scripts/install-lab-host.sh` does not exist (open)
|
||||
|
||||
**Symptom:** Both `deploy.md` and `lab-setup.md` reference
|
||||
`scripts/install-lab-host.sh` as the canonical install path. The `scripts/`
|
||||
directory does not exist.
|
||||
|
||||
**Impact:** Every new lab host requires the manual steps from ISS-001 through
|
||||
ISS-004 above.
|
||||
|
||||
**Status:** Open. Requires writing the install script. Suggested content:
|
||||
- `pacman -Sy`
|
||||
- Install: `uv`, `perf`, `tcpdump`, `zstd`, `qemu-system-x86_64`,
|
||||
`bridge-utils`
|
||||
- Download Alpine cloud image if `vm/images/alpine-baseline.qcow2` absent
|
||||
- Run `tools/build_cidata.py`
|
||||
- Take `baseline-v1` snapshot
|
||||
- Drop systemd units from `etc/` into `/etc/systemd/system/`
|
||||
- Write `/etc/cis490/lab-host.toml` template
|
||||
|
||||
---
|
||||
|
||||
### ISS-006 — Campaign files not promoted to Dev_REL1
|
||||
|
||||
**Symptom:** `run_campaign.py` and `shipper.py` (committed to
|
||||
`Dev_REL1_042926`) were absent from `Dev_REL1_043026` (branched off
|
||||
`Dev_REL1` before the merge).
|
||||
|
||||
**Fix applied:** Cherry-picked commit `ce2fd66` from `Dev_REL1_042926`:
|
||||
```
|
||||
git cherry-pick ce2fd66
|
||||
```
|
||||
|
||||
**Recommendation:** Merge `Dev_REL1_042926` → `Dev_REL1` before cutting
|
||||
future day branches so the base is always current.
|
||||
|
||||
---
|
||||
|
||||
### ISS-007 — Campaign output files not in `.gitignore`
|
||||
|
||||
**Symptom:** `data/campaign.json`, `data/campaign_done.marker`,
|
||||
`data/outbox/`, and `data/shipped/` would appear as untracked files after
|
||||
a campaign run.
|
||||
|
||||
**Fix applied:** Added to `.gitignore`:
|
||||
```
|
||||
data/campaign.json
|
||||
data/campaign_done.marker
|
||||
data/outbox/
|
||||
data/shipped/
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Validation Results
|
||||
|
||||
All three checks passed on this host.
|
||||
|
||||
### Single episode (`run_real_vm_demo.py`)
|
||||
```
|
||||
episode_id = 01KQG2K4WJBE52PCD2QDCQAVKB
|
||||
rows_proc = 853
|
||||
phases = [clean, armed, infecting, infected_running,
|
||||
dormant, infected_running, dormant, clean]
|
||||
duration = 85.26s
|
||||
```
|
||||
|
||||
### 2-episode campaign (`run_campaign.py --target 2`)
|
||||
```
|
||||
episode 1: id=01KQG2QRVRRTVM3TMEKCEW1E6Z rows=853 duration=85.26s ✓
|
||||
episode 2: id=01KQG2VMWM6WKEQPCZ3BKYT513 rows=853 duration=85.27s ✓
|
||||
campaign_done.marker written ✓
|
||||
campaign.json: {"completed": 2, "done": true} ✓
|
||||
```
|
||||
|
||||
### Episode directory structure (each episode)
|
||||
```
|
||||
data/episodes/<ulid>/
|
||||
meta.json ✓
|
||||
events.jsonl ✓
|
||||
labels.jsonl ✓ (8 phase transitions)
|
||||
telemetry-proc.jsonl ✓ (853 rows @ 10 Hz)
|
||||
done.marker ✓
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Known Gaps (pre-existing, not introduced this session)
|
||||
|
||||
| Gap | File | Status |
|
||||
|---|---|---|
|
||||
| `perf stat` collector | `collectors/perf_qemu.py` | Not yet implemented |
|
||||
| QMP collector | `collectors/qmp.py` | Not yet implemented |
|
||||
| `tcpdump` / pcap collector | `collectors/pcap.py` | Not yet implemented |
|
||||
| In-guest agent collector | `collectors/guest_agent.py` | Not yet implemented |
|
||||
| Metasploit exploit driver (Tier 3) | `exploits/` | Planned |
|
||||
| Real malware sample (Tier 4) | `samples/` | Planned |
|
||||
| `scripts/install-lab-host.sh` | `scripts/` | See ISS-005 |
|
||||
Loading…
Add table
Reference in a new issue