Lays down the design surface for the CIS490 behavioral-malware-detection dataset and model. No code yet — schema and topology are decided first so collection can start without rework. Docs: - README: project goal, navigation - architecture: lab topology, KVM choice, episode state machine, deployment-mirror reasoning - threat-model: train/serve parity rule, oracle-vs-deployable feature split, two-model evaluation strategy - data-model: per-episode JSONL layout, row schemas, phase enum - transport: WG-native shipper/receiver design, idempotent uploads - deploy: one-command install for lab-host and receiver roles - lab-setup: KVM prereqs, VM build, snapshot, virtio-serial wiring Skeleton: orchestrator/, collectors/, vm/, exploits/, samples/, training/ (each with a short README explaining purpose). Extended .gitignore to exclude qcow2 images, pcaps, sample binaries, secrets. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
3.4 KiB
3.4 KiB
Deploy
Two roles. One install command each.
Roles
| Role | Where it runs | What it does |
|---|---|---|
lab-host |
any KVM-capable Linux box on WG | runs episodes, ships completed episodes to the receiver |
receiver |
Pi5 (or any always-on WG node) | accepts ship uploads, stores tarballs + index.jsonl |
Lab host install
git clone https://maxgit.wg/spectral/CIS490.git
cd CIS490
./scripts/install-lab-host.sh
The installer:
- Verifies KVM (
/dev/kvmexists, user inkvmgroup). - Installs system deps via the host package manager (qemu, tcpdump, linux-tools/perf, zstd, python ≥ 3.11).
- Bootstraps a
uv-managed venv at.venv/and installs the pinned Python deps fromuv.lock. - Drops two systemd units into
/etc/systemd/system/:cis490-orchestrator.service— runs the episode loop on a queuecis490-shipper.service— watchesdata/episodes/and ships completed episodes
- Writes a config template to
/etc/cis490/lab-host.toml(idempotent — only on first install).
You finish by editing /etc/cis490/lab-host.toml to point at your receiver
and to enroll your lab host's WG-issued client cert, then:
sudo systemctl enable --now cis490-orchestrator cis490-shipper
lab-host.toml
host_id = "lab-host-1"
[paths]
data_root = "/var/lib/cis490/data"
samples_store = "/var/lib/cis490/samples/store"
qcow_image = "/var/lib/cis490/vm/images/metasploitable2.qcow2"
[receiver]
url = "https://collector.wg"
client_cert = "/etc/cis490/certs/lab-host-1.pem"
client_key = "/etc/cis490/certs/lab-host-1.key"
ca_bundle = "/etc/cis490/certs/wg-ca.pem"
[episode]
baseline_seconds = 30
infected_seconds = 90
dormant_seconds = 60
[retention]
keep_local_for_days = 7
prune_at_disk_pct = 80
Receiver install
On the Pi5 (or designated central node):
git clone https://maxgit.wg/spectral/CIS490.git
cd CIS490
./scripts/install-receiver.sh
The installer:
- Installs Python ≥ 3.11 + zstd + a tiny WSGI runner (uvicorn).
- Bootstraps the same
uv-managed venv. - Drops
cis490-receiver.servicelistening on127.0.0.1:8443(TLS terminated by the existing Caddy inspectral/caddy, which already binds*.wg). - Writes a config template to
/etc/cis490/receiver.toml.
Caddy block (added to your spectral/caddy config) for the receiver:
collector.wg {
tls internal
reverse_proxy 127.0.0.1:8443 {
transport http {
tls
tls_client_auth /etc/cis490/certs/wg-ca.pem
}
}
}
(mTLS terminates at the receiver, not Caddy — so the receiver sees the client cert and can enforce per-host policies later.)
receiver.toml
listen_addr = "127.0.0.1:8443"
store_root = "/var/lib/cis490/episodes"
incoming_root = "/var/lib/cis490/incoming"
index_path = "/var/lib/cis490/index.jsonl"
ca_bundle = "/etc/cis490/certs/wg-ca.pem"
[limits]
max_episode_bytes = 268_435_456 # 256 MiB
Day-2 operations
# How many episodes have been shipped?
ssh collector.wg 'wc -l /var/lib/cis490/index.jsonl'
# What's in the outbox on a lab host? (failed/pending shipments)
ls /var/lib/cis490/data/outbox/
# Tail the orchestrator log
journalctl -u cis490-orchestrator -f
# Tail the shipper log
journalctl -u cis490-shipper -f
Updating
git pull
./scripts/install-lab-host.sh # idempotent; re-syncs deps and units
sudo systemctl restart cis490-orchestrator cis490-shipper