CIS490 coursework
064387b7a0
End-to-end: ``python -m orchestrator --target-pid <pid> --duration N`` now
writes a complete episode directory matching docs/data-model.md, with phase
labels, events, and a 10 Hz host /proc telemetry stream. No VM yet — pid is
arbitrary so we can validate the loop against e.g. ``sleep 5`` while the lab
side comes up.
collectors/proc_qemu.py — parses /proc/<pid>/{stat,io,status} (handles parens
in comm), single-shot collect_once(), and a stop-event-driven run_loop()
that ticks at a fixed cadence and exits when the pid disappears. Tagged
``available_in_deployment: false`` per the threat-model doc.
orchestrator/episode.py — EpisodeRunner: creates data/episodes/<ulid>/,
atomic meta.json, events.jsonl + labels.jsonl writers, drives the collector
in a thread for duration_s, writes done.marker last so the shipper never
sees a half-finished episode.
orchestrator/ulid.py — tiny 26-char Crockford-base32 ULID generator.
Time-sortable, no third-party dep.
orchestrator/__main__.py — CLI entry point.
Tests (15 new, 28 total green):
- proc_qemu: real-ish stat with parens-in-comm, missing /proc/<pid>/io,
missing pid, run_loop cadence, run_loop terminates when pid disappears.
- episode: full directory shape against os.getpid(), id override,
done.marker written after meta.json finalize.
- ulid: length+alphabet, 2000-burst uniqueness, time-sortability.
Smoke-tested against ``sleep 10``: 16 rows over 1.5s at 100ms cadence,
monotonic clock, RSS stable at ~3.5 MiB as expected for an idle sleep.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|---|---|---|
| collectors | ||
| docs | ||
| etc | ||
| exploits | ||
| orchestrator | ||
| receiver | ||
| samples | ||
| tests | ||
| training | ||
| vm | ||
| .gitignore | ||
| pyproject.toml | ||
| README.md | ||
| uv.lock | ||
CIS490 — Behavioral Malware Detection Dataset & Model
Course project for CIS490 (Cybersecurity). The end-goal is an ML model that watches performance metrics on a real device, decides whether the device has been breached, and triggers a hardware-level reset when confidence is high enough.
This repository covers the dataset side of that pipeline: we run real, public malware samples against intentionally vulnerable Linux VMs and capture labeled time-series telemetry that mirrors what the same model would see in deployment on a Raspberry Pi or similarly-constrained target.
The work is grounded in the trust-over-time scoring model from IEEE 9881803 and a related proprietary follow-on that pairs detection with blockchain-anchored hardware reset.
What lives where
| Path | What it holds |
|---|---|
docs/architecture.md |
Lab topology, KVM choice, snapshot loop, deployment-mirror reasoning |
docs/threat-model.md |
Train/serve parity rule and the oracle-vs-deployable feature split |
docs/data-model.md |
On-disk JSONL schema, per-episode layout, phase enum |
docs/transport.md |
Sender/receiver design — how episodes get to the central collector over WG |
docs/deploy.md |
One-command install for the lab-host and receiver roles |
docs/lab-setup.md |
KVM prereqs, VM build, snapshot, virtio-serial wiring |
orchestrator/ |
State machine that drives the boot → arm → detonate → observe → revert loop |
collectors/ |
One module per telemetry source (host /proc, QMP, perf, pcap, guest agent) |
vm/ |
qcow2 images and snapshot scripts (binaries gitignored) |
exploits/ |
Metasploit resource scripts for repeatable exploitation |
samples/ |
Sample manifest (sha256-pinned). Binaries never committed. |
training/ |
Model training code (deferred — schema first) |
Quick orientation
- Why VMs? We need a clean snapshot/revert loop and we need to run real malware without burning hardware. KVM gives us both at near-native speed.
- Why is the network isolated? A host-only bridge keeps malware off the internet and off the WG overlay. The Pi5 gateway is the lab-side observer, playing the same role it would play in a deployed setting.
- Why JSONL and not a database (yet)? Schema-last: collect first, decide storage shape after we see what's actually useful. JSONL is crash-safe, append-only, and reshapes trivially into Postgres/Timescale/Parquet later.
- Why two models? One trained on features that exist on a real Pi (deployable), one trained on host-side QEMU-only features (oracle). The accuracy gap measures how much detection power a privileged rootkit can take from the deployed model. See docs/threat-model.md.
Status
Project bootstrap. Skeleton, documentation, and design decisions in place; collection and orchestration code in progress.