CIS490/docs/deploy.md
Maximus Gorog fa1574a0a6 Scaffold project: docs, repo skeleton, transport + deploy design
Lays down the design surface for the CIS490 behavioral-malware-detection
dataset and model. No code yet — schema and topology are decided first so
collection can start without rework.

Docs:
- README: project goal, navigation
- architecture: lab topology, KVM choice, episode state machine,
  deployment-mirror reasoning
- threat-model: train/serve parity rule, oracle-vs-deployable feature
  split, two-model evaluation strategy
- data-model: per-episode JSONL layout, row schemas, phase enum
- transport: WG-native shipper/receiver design, idempotent uploads
- deploy: one-command install for lab-host and receiver roles
- lab-setup: KVM prereqs, VM build, snapshot, virtio-serial wiring

Skeleton: orchestrator/, collectors/, vm/, exploits/, samples/,
training/ (each with a short README explaining purpose).
Extended .gitignore to exclude qcow2 images, pcaps, sample binaries,
secrets.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-28 23:21:00 -06:00

3.4 KiB

Deploy

Two roles. One install command each.

Roles

Role Where it runs What it does
lab-host any KVM-capable Linux box on WG runs episodes, ships completed episodes to the receiver
receiver Pi5 (or any always-on WG node) accepts ship uploads, stores tarballs + index.jsonl

Lab host install

git clone https://maxgit.wg/spectral/CIS490.git
cd CIS490
./scripts/install-lab-host.sh

The installer:

  1. Verifies KVM (/dev/kvm exists, user in kvm group).
  2. Installs system deps via the host package manager (qemu, tcpdump, linux-tools/perf, zstd, python ≥ 3.11).
  3. Bootstraps a uv-managed venv at .venv/ and installs the pinned Python deps from uv.lock.
  4. Drops two systemd units into /etc/systemd/system/:
    • cis490-orchestrator.service — runs the episode loop on a queue
    • cis490-shipper.service — watches data/episodes/ and ships completed episodes
  5. Writes a config template to /etc/cis490/lab-host.toml (idempotent — only on first install).

You finish by editing /etc/cis490/lab-host.toml to point at your receiver and to enroll your lab host's WG-issued client cert, then:

sudo systemctl enable --now cis490-orchestrator cis490-shipper

lab-host.toml

host_id = "lab-host-1"

[paths]
data_root = "/var/lib/cis490/data"
samples_store = "/var/lib/cis490/samples/store"
qcow_image = "/var/lib/cis490/vm/images/metasploitable2.qcow2"

[receiver]
url = "https://collector.wg"
client_cert = "/etc/cis490/certs/lab-host-1.pem"
client_key = "/etc/cis490/certs/lab-host-1.key"
ca_bundle = "/etc/cis490/certs/wg-ca.pem"

[episode]
baseline_seconds = 30
infected_seconds = 90
dormant_seconds = 60

[retention]
keep_local_for_days = 7
prune_at_disk_pct = 80

Receiver install

On the Pi5 (or designated central node):

git clone https://maxgit.wg/spectral/CIS490.git
cd CIS490
./scripts/install-receiver.sh

The installer:

  1. Installs Python ≥ 3.11 + zstd + a tiny WSGI runner (uvicorn).
  2. Bootstraps the same uv-managed venv.
  3. Drops cis490-receiver.service listening on 127.0.0.1:8443 (TLS terminated by the existing Caddy in spectral/caddy, which already binds *.wg).
  4. Writes a config template to /etc/cis490/receiver.toml.

Caddy block (added to your spectral/caddy config) for the receiver:

collector.wg {
    tls internal
    reverse_proxy 127.0.0.1:8443 {
        transport http {
            tls
            tls_client_auth /etc/cis490/certs/wg-ca.pem
        }
    }
}

(mTLS terminates at the receiver, not Caddy — so the receiver sees the client cert and can enforce per-host policies later.)

receiver.toml

listen_addr = "127.0.0.1:8443"
store_root = "/var/lib/cis490/episodes"
incoming_root = "/var/lib/cis490/incoming"
index_path = "/var/lib/cis490/index.jsonl"
ca_bundle = "/etc/cis490/certs/wg-ca.pem"

[limits]
max_episode_bytes = 268_435_456    # 256 MiB

Day-2 operations

# How many episodes have been shipped?
ssh collector.wg 'wc -l /var/lib/cis490/index.jsonl'

# What's in the outbox on a lab host? (failed/pending shipments)
ls /var/lib/cis490/data/outbox/

# Tail the orchestrator log
journalctl -u cis490-orchestrator -f

# Tail the shipper log
journalctl -u cis490-shipper -f

Updating

git pull
./scripts/install-lab-host.sh   # idempotent; re-syncs deps and units
sudo systemctl restart cis490-orchestrator cis490-shipper