CIS490/tests
Max Gorog 4d29b7236d PIPELINE §5 step 3: target VM build infrastructure + containment posture
§4.2 calls for target VMs we BUILD, not VMs we fetch. §4.13 demands
every target ship the same isolation posture (no upstream egress, no
host-shared FS, unprivileged QEMU, fresh snapshot per episode). This
commit lands the infrastructure for both.

New surface:
  * orchestrator/target_spec.py
      Loads + validates `vm/targets/<name>/spec.toml`. Containment
      fields are not knobs — each has exactly ONE safe value, and a
      spec asserting the unsafe value is rejected at load time. There's
      no `--containment-override`; weakening §4.13 requires amending
      PIPELINE.md and operator sign-off.

  * tools/build_target.py
      Orchestrates build → verify → publish for a single target. Spec
      invalid → exit 78 (sysadmin error). build.sh failure → image not
      published. verify.sh failure → image discarded; that's the §4.2
      acceptance gate. Publishes sha256 + the manifest.toml stanza the
      operator copies in to admit the image (§16 substantive amendment
      with sign-off per §15).

  * vm/targets/<name>/{spec.toml,build.sh,verify.sh}
      Template structure. spec.toml is the contract; build.sh produces
      $OUT_PATH; verify.sh boots the produced image under the §4.13
      containment posture and asserts every promise.

  * vm/targets/shellshock/
      First real working target. CVE-2014-6271 (Apache mod_cgi + bash
      4.2 mis-parsing function-export environment values). Replaces
      the SourceForge Metasploitable2 path that §3 evidence proved
      unverifiable. Bash 4.2 is built from sha256-pinned GNU source
      inside an Alpine 3.21 cloudinit guest; the build script asserts
      the produced bash actually triggers shellshock; the verifier
      re-asserts it under restrict=on with a real CVE-2014-6271 probe.

  * vm/targets/README.md
      How operators add a target. Walks the spec → build → verify →
      manifest amendment loop.

Containment regression tests (tests/test_containment.py) — 20 new
assertions, parameterized over every target with a build/verify trio:

  * verify.sh MUST contain `restrict=on` on its netdev (§4.13)
  * verify.sh MUST contain `snapshot=on` on the boot drive (§4.13)
  * verify.sh + build.sh MUST NOT contain -virtfs / -fsdev / 9pfs
  * verify.sh + build.sh MUST NOT wrap qemu-system in `sudo`
  * Every target must ship the complete spec.toml + build.sh + verify.sh
    trio — no half-built targets (§1 default-to-removal)

Spec validation tests (tests/test_target_spec.py): 13 new tests over
spec parse, name/dir mismatch, missing fields, out-of-range port, and
the §4.13 containment field validators (each unsafe value rejected
with a clear error).

The shellshock target's image is NOT yet published to manifest.toml's
[[targets.images]] — that's the §15 sign-off amendment that lands
after a successful operator-driven build_target.py run on a lab host
with KVM. Building takes ~10 min on x86_64; cannot run on the Pi
under TCG. Operator drives the first build, verifies the sha256, then
amends manifest.toml in a follow-up commit.

261 tests passing.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-04 01:31:40 -05:00
..
__init__.py Add receiver: PUT /v1/episodes ingest with sha256 verify and idempotency 2026-04-28 23:34:04 -06:00
test_auto_fetch_samples.py auto_fetch_samples: pick Linux i386 ELF; manifest matches theZoo 2026-05-01 03:28:26 -05:00
test_collectors_emit.py PIPELINE §5 step 1: fix four root-cause defects 2026-05-03 17:05:25 -05:00
test_containment.py PIPELINE §5 step 3: target VM build infrastructure + containment posture 2026-05-04 01:31:40 -05:00
test_doctor_shipping.py shipper: systemd watchdog, quarantine cleanup; doctor surfaces ship errors 2026-05-01 12:02:59 -05:00
test_episode.py meta.json: stamp code_version (commit, branch, dirty) per episode 2026-05-01 01:29:01 -05:00
test_exploits.py catalog: remove samba_usermap_script — never landed sessions in prod 2026-05-03 22:48:03 -05:00
test_fleet.py PIPELINE §5 step 2: canonical manifest at <repo>/manifest.toml 2026-05-04 01:25:01 -05:00
test_fleet_health.py fleet-health: exit 0 when alerts found (don't mark unit failed) 2026-05-02 13:51:20 -05:00
test_guest_agent.py Collectors 2/4/5 + fleet runner + sample manifest + Tier-3 setup scripts 2026-04-30 00:02:27 -05:00
test_host_health.py fleet-health: proactive alerts on the Pi + per-host doctor reports 2026-05-02 13:48:31 -05:00
test_manifest.py PIPELINE §5 step 2: canonical manifest at <repo>/manifest.toml 2026-05-04 01:25:01 -05:00
test_pcap.py Collectors 2/4/5 + fleet runner + sample manifest + Tier-3 setup scripts 2026-04-30 00:02:27 -05:00
test_perf_qemu.py Close out the open issues: bridge pcap wiring, perf collector, Tier-4 2026-04-30 00:17:49 -05:00
test_proc_qemu.py Add v0 orchestrator + first oracle collector (host /proc) 2026-04-28 23:40:25 -06:00
test_prune.py Multi-signal prune classifier: rescue valid episodes /proc misses 2026-04-30 19:10:01 -05:00
test_qmp.py Close out the deployment-readiness gaps 2026-04-30 00:31:55 -05:00
test_quarantine_unstamped.py fix: lab-host install loop after commit-gate cutover 2026-05-01 11:36:21 -05:00
test_receiver.py Add receiver: PUT /v1/episodes ingest with sha256 verify and idempotency 2026-04-28 23:34:04 -06:00
test_shipper.py shipper: systemd watchdog, quarantine cleanup; doctor surfaces ship errors 2026-05-01 12:02:59 -05:00
test_target_spec.py PIPELINE §5 step 3: target VM build infrastructure + containment posture 2026-05-04 01:31:40 -05:00
test_tier3_local_verify.py tools/verify_tier3_local.py: Pi-runnable Tier-3 verifier 2026-05-01 03:41:21 -05:00
test_tier4.py Close out the deployment-readiness gaps 2026-04-30 00:31:55 -05:00
test_ulid.py Add v0 orchestrator + first oracle collector (host /proc) 2026-04-28 23:40:25 -06:00
test_version_gate.py robustness: gate falls back to local git, queue sweeps stale tarballs 2026-05-01 11:49:38 -05:00
test_vm_load_controller.py workload audit trail: meta.sample + per-phase events + pre-kill probe 2026-04-30 02:12:34 -05:00