CIS490/PIPELINE.md

PIPELINE.md — the CIS490 generative pipeline honesty plan

This document is canonical. It supersedes any guidance in AGENTS.md, FIXYOURSELF.md, README.md, or other repo docs that contradicts it. If another doc says something different, this doc wins and the other doc is wrong (file an issue or fix it).

This is not an architecture overview. This is a fix list. Read it, implement it, do not split it into phases.

Before proposing any change to the pipeline, re-read §1, §7, and §8 and run your proposal against §8's checklist. Then proceed.

---

1. Principle

Every episode that reaches the dataset must be ground-truth. Every host runs the same experiment with the same configured catalog. Every exploit module and every collector in the catalog has been proven to work end-to-end before it is eligible to run. There are no compensating layers — no auto-update timers that drag stale peers forward, no "fix-yourself" decision trees, no per-host divergence absorbed by trainer-side filters, no labels written by clock when the event they describe didn't happen.

If a host can't meet the bar, it produces zero episodes and says so loudly. A small honest dataset beats a large dishonest one.

Default to removal, not addition. If a problem can be fixed by deleting code or removing a layer, prefer that. Adding a layer is the suspect default and should be justified against §7 and §8 before proceeding.

---

2. What the experiments are for

CIS490 trains a behavioral malware-detection model. The dataset is the ground-truth labelled record of what the host looked like during known-clean, known-armed, known-infecting, and known-infected phases of a real exploit chain against a real target service. The model learns to distinguish those phases from in-deployment behavior. Every dishonest label is a poisoned training example.

This is why the producer's job is not "ship lots of episodes." It is "ship episodes whose labels are true."

---

3. What is currently broken (evidence)

Numbers from the 200-episode quality probe on 2026-05-03:

1. Labels lie. 0 of 67 Tier-3 exploit fires resulted in a session_open event. All 67 logged session_open_timeout. Yet every one of those 67 episodes is labelled phase=infected_running because the schedule-driven labeller transitions on a clock, not on observed events. The infected_running label in the dataset means "the schedule said so," not "an attacker session was actually open on this host."
2. Collectors are silent.
   • perf produces 0 rows on 100% of episodes on both hosts.
   • guest-agent produces 0 rows on 100% of episodes on both hosts.
   • qmp, netflow, and pcap produce 0 rows on 100% of k-gamingcom episodes (different config from elliott).
   • The host tcpdump is missing on k-gamingcom; pcap_unavailable is logged then ignored.
3. The catalog is unverified. Modules are added to the rotation without a per-module verification that the module actually lands a session against its declared target. samba_usermap_script has a 100% failure rate against the configured Metasploitable2 target and was still in the rotation.
4. Hosts run divergent experiments. elliott and k-gamingcom have different per-host manifests, different collector coverage, different qemu invocations. The dataset is a union of two different experiments, not 200 samples from one.
5. Working trees are dirty. 200/200 episodes report dirty=true, so code_version.commit is unverifiable provenance.

Each of these is a failure of the producer. Receiver-side filtering and trainer-side prune scripts are bandaids that hide them.

---

4. The fix — line items

Every item below must land. They are not phases. They are parts of one cohesive correctness story; any of them missing leaves the pipeline half-honest. Each item names its acceptance test.

4.1 Canonical manifest

There is exactly one manifest, version-pinned in the repo at manifest.toml. Every lab host loads the same manifest. There is no per-host manifest override, no per-host collector enable/disable flag, no per-host qemu argument list. Hosts that cannot run the canonical manifest exit 78 at orchestrator startup.

Acceptance: find . -name manifest.toml -not -path './.git/*' returns exactly one path. There is no --manifest CLI flag on the orchestrator that takes a different path; the path is hard-coded. Removing this line item would re-create the host divergence we just exited.

4.2 Target VMs we build, not VMs we fetch

Every target VM image is built from a declarative spec checked into the repo (Packer, mkosi, debootstrap, whatever — declarative). The image build produces a sha256-pinned artifact. The build script verifies, before producing the artifact, that:

• The vulnerable service is up after first boot.
• The service is on the port the module catalog declares.
• The service version matches the version the module catalog declares.

Metasploitable2 from a SourceForge mirror is removed. We don't ship episodes targeting black-box images.

Acceptance: scripts/build-target-<name>.sh exists for every target referenced by an exploit module. Running it produces an image whose post-boot state passes the spec's verification step. The verification step's exit code gates the build's exit code.

4.3 Module catalog admission criteria

A module is in the catalog only if it passes a recorded end-to-end verification run against its declared target. The verification is:

1. Boot the target snapshot.
2. Fire the module via msfrpcd.
3. Observe a session_open event (not session_open_timeout).
4. Observe at least one shell command round-trip on the session.
5. Confirm guest-side artifact (file written, process spawned — per-module).

If any step fails, the module does not enter the catalog. There is no "tentatively included" tier. Modules already in the catalog are re-verified by scripts/verify-catalog.sh (new) on every release; failures remove the module from the catalog.

Acceptance: every entry in exploits/modules/*.toml has a companion verified_against = "<target_name>" and last_verified = "<commit_sha>" field. scripts/verify-catalog.sh re-runs every entry and exits 0 only if every one passes.

4.4 Collector admission criteria

A collector is in the active set only if it passes a recorded end-to-end verification run that confirms it emits non-zero rows against a known-busy probe workload.

For each of the six collectors (proc, qmp, netflow, perf, guest, pcap):

1. Diagnose the current zero-row failure (read the code, run standalone, find the actual cause). Fix the cause.
2. Add a unit-or-integration test that runs the collector for N seconds against a synthesized workload (a busy-loop process for proc/perf, a packet generator for netflow/pcap, a QMP blockstats query for qmp, a guest heartbeat for guest) and asserts ≥1 row.
3. The test must run in CI and on every install via the install script.

A collector that cannot pass admission is removed from the active set with a recorded reason — not silently included with zero rows.

Acceptance: pytest tests/test_collectors_emit.py -k <name> passes for each name. The CI run gates merges.

4.5 Event-driven labelling

Phase labels are written from observed events, never from the schedule clock. The schedule becomes a time budget — maximum time the orchestrator will wait in each phase — not a label source.

Specifically:

• clean is written at episode start.
• armed is written when the orchestrator instructs the driver to fire (this is observable in code).
• infecting is written when the exploit_fire event is observed.
• infected_running is written only when the session_open event is observed.
• If session_open_timeout is observed instead, the episode terminates with a failed label and is rejected (see §4.6).
• dormant and subsequent infected_running transitions are written from observed in-session idle / activity, not from clock.

Per-module timeouts replace the global 30s timeout. Default 120s, configurable per module in exploits/modules/*.toml.

Acceptance: for every shipped episode, every entry in labels.jsonl has a corresponding event in events.jsonl with a matching t_mono_ns within ±100ms. An invariant test asserts this.

4.6 Episode acceptance gate at finalization

Before sealing meta and writing done.marker, the orchestrator verifies:

• Every collector in the active set produced ≥1 row.
• Every label has a matching event (§4.5 invariant).
• For Tier-3 episodes: a session_open event exists.
• dirty=true is absent OR dirty_override=true is present (see §4.9).

If any check fails, the episode goes to data/rejected/<id>/ with a rejected_reason.json describing which check failed. done.marker is not written. The shipper never sees it.

Acceptance: tests/test_acceptance_gate.py covers each rejection condition. A passing test asserts a clean episode is accepted; for each failure mode, the test asserts the episode is moved to rejected/ with the expected reason.

4.7 Producer preflight

orchestrator/preflight.py runs at orchestrator startup. One bar (no light/deep split). Checks:

• Every binary required by the active collector set + active module catalog is on PATH.
• /dev/kvm accessible by the service user.
• kernel.perf_event_paranoid <= 2.
• cfg.bridge_iface exists; tcpdump can capture on it.
• msfrpcd reachable; auth.login returns a token.
• For every module in catalog: module.info is fetchable.
• For every sample in catalog: file present on disk; sha256 matches.
• Probe-boot baseline-v1 snapshot; observe guest-agent heartbeat within N seconds.
• git status --porcelain empty (or CIS490_ALLOW_DIRTY=1).
• HEAD is on a commit currently in origin/main.

Failures are collected (every failed check logged with diagnosis + remediation), then sys.exit(78).

Acceptance: tests/test_preflight.py covers each check individually with mocked subprocess/filesystem. python -m orchestrator.preflight runs the checks and prints a structured report. Exit codes: 0 ok, 78 sysadmin error.

4.8 Receiver-side rejection (defense in depth)

The receiver is defense-in-depth, NOT the primary correctness mechanism. The producer is. Receiver rejection exists to catch peers running stale or broken code; it is never a substitute for fixing the producer. A change that strengthens receiver rejection without strengthening the producer is the defensive-instead-of- corrective pattern (§7.9).

The receiver enforces the same correctness invariants the orchestrator does. A peer running stale code that produces dishonest episodes still gets rejected at ingest:

• Reject any meta with dirty=true and no dirty_override=true.
• Reject any meta where phases_observed contains infected_running but events.jsonl (extracted from the tarball) lacks session_open.
• Reject any meta where any configured-collector row count is zero.
• Existing commit-allow-list gate continues.

Rejections return 422 with a JSON body naming the failed check. Rejected tarballs are not written to the index.

Acceptance: tests/test_receiver_rejects.py covers each new rejection condition.

4.9 Override discipline

The only escape hatch from the dirty-tree gate is the CIS490_ALLOW_DIRTY=1 environment variable. When set:

• Orchestrator logs WARN: dirty tree override active.
• meta.json gains dirty_override: true.
• Receiver accepts the episode only if dirty_override is also true.
• Every override use is auditable from the dataset.

There are no other override knobs. No verify_tls=false, no "skip preflight," no "include this collector even if it emits zero rows."

4.10 Regression-test discipline

Every fix in this plan lands with a test that would have caught the regression at PR time. Tests are not a follow-up. A PR that fixes the perf collector without a perf-emit test is incomplete and gets sent back.

CI runs:

• All unit tests.
• scripts/verify-catalog.sh against a smoke target subset (catalog verification full run is gated to release commits — too expensive for every PR).
• The collector-emit integration tests (§4.4) on real binaries.

4.11 systemd integration

• cis490-orchestrator.service adds RestartPreventExitStatus=78. A preflight failure stays loud and stuck instead of cycling restarts.
• On preflight failure, orchestrator writes /var/lib/cis490/preflight.failed.json with the failed checks + timestamps. Doctor surfaces this in its next report. The fleet-health alert distinguishes "preflight failed" from "host silent."

4.12 Cleanup of compensating layers

The following are deleted as part of this change. Their existence was load-bearing for the dishonest pipeline; the honest one doesn't need them.

• FIXYOURSELF.md — entire file deleted. Stuck states no longer exist as a class because the gates make them impossible.
• cis490-autoupdate.timer + scripts/auto-update.sh — deleted. Hosts run pinned commits. New code is rolled out by the operator, not auto-pulled.
• cis490-cert-fetch.timer — replaced by a one-shot first-boot fetch in install-lab-host.sh. No periodic re-fetch.
• tools/quarantine_unstamped.py — deleted. Pre-stamp episodes cannot exist because no episode is written without a valid stamp.
• tools/check_fleet_health.py — keep, but delete the "fatal-only" alert branch (that branch existed because we were shipping fatals; with the gate, we don't).
• tools/prune_episodes.py's "kept episode despite flat /proc because qmp showed write" cross-check logic — deleted. Episodes that don't pass the producer-side gate don't reach the trainer.
• AGENTS.md "symptom→fix table" — deleted (the symptoms it covers are now impossible).
• AGENTS.md "Hosts self-update" section — deleted.

4.13 Containment bar

Real malware execution requires explicit containment. Target VMs exist in an isolation context that is part of the canonical experiment, not a deployment detail. A future change that weakens any of the items below is a containment regression and is rejected regardless of what experimental realism it claims to add.

For every target VM in the catalog (§4.2):

• Network: target attaches to a bridge with NO upstream egress. No NAT to the host network, no internet route, no DNS resolution beyond what the experiment provides. Outbound C2 callbacks resolve to a sinkhole inside the experiment, never to the internet.
• Filesystem: no shared mount with the host. No 9p, no virtio-fs with host paths. The target's disk is the snapshot it was booted from, period.
• Privilege: QEMU runs as the unprivileged service user. KVM access is via group membership only; no setuid wrappers, no privileged TUN ownership transfer, no passthrough of host devices not explicitly required by the catalog.
• Lifetime: every target boots from a fresh snapshot. State from one episode never crosses into the next. The snapshot is reverted at episode end, not "cleaned."
• Escape monitoring: any QEMU exit that is not a clean shutdown is logged with full QMP state and the episode is marked failed. Two unclean exits on the same target image within a release window trigger admission-criteria re-verification (§4.3) for every module targeting that image.

Acceptance: tests/test_containment.py asserts each target build (a) has no upstream egress route from inside the guest, (b) has no host-shared filesystem mount, (c) runs QEMU as the unprivileged service user, (d) reverts to snapshot at episode end. The test runs in CI and on every install.

---

5. Build order

There is no half-honest intermediate state. The order below sequences the work; it does not phase the deployment. Everything lands to main in one merge.

1. Fix the four root-cause defects:
   • Diagnose + fix the perf collector (read code, run standalone, find why it's silent, fix).
   • Diagnose + fix the guest-agent collector (mount baseline image, verify agent installed, fix build).
   • Diagnose + fix k-gamingcom's missing qmp/netflow/pcap (compare configs, eliminate divergence — §4.1).
   • Diagnose + fix samba_usermap_script against its target (manual msfconsole drive, find why the bind shell never connects, fix or remove from catalog — §4.3).
2. Land the canonical manifest (§4.1).
3. Land the target-VM build pipeline (§4.2) and containment tests (§4.13) together — target VMs are not in the catalog without containment.
4. Land the catalog admission criteria + verifier (§4.3).
5. Land the collector admission criteria + tests (§4.4).
6. Land the event-driven labeller (§4.5).
7. Land the acceptance gate (§4.6).
8. Land the preflight (§4.7).
9. Land the receiver-side rejection (§4.8).
10. Land the override discipline + cleanup (§4.9, §4.12).
11. Land systemd integration + alert distinguishing (§4.11).

After merge: lab hosts pull the new manifest, run preflight, fail loudly if they don't meet the bar, produce zero episodes until they do. The operator brings each host to bar — fixing one root cause at a time, loudly. The dataset goes quiet, then comes back honest.

---

6. Out of scope (and why)

• Schedule jitter for label-leakage resistance. Real concern, but doesn't affect honesty — only generalization. Address after honest data is flowing.
• New collectors (audit, ftrace, etc.). Adding collectors before the existing six are honest is putting more weight on a broken floor.
• Trainer changes. This plan stops at the dataset boundary. The trainer no longer needs to filter dishonest episodes because they don't exist.
• Multi-architecture targets. All target VMs are x86_64 for now.

Each of these is fine to defer because they don't paper over a correctness defect. They add value on top of an honest pipeline; the pipeline isn't honest yet.

---

7. Anti-patterns (named — match every proposal against this list)

Each pattern below is a shape a proposal can take that has been rejected as architectural sleight-of-hand. Match every proposal against this list before submitting it. A proposal that matches a named pattern is rejected; abandon it and propose a corrective fix instead.

The patterns are named so future sessions can recognize them in their own work. A bandaid with a nice name (preflight, acceptance gate, retry layer, fleet-health) is still a bandaid.

§7 is non-exhaustive. New sleight-of-hand patterns will exist that aren't named here. The §8 decision tests are the actual filter; a proposal that fails §8 is rejected even if it matches no named pattern. Do not read §7 as a closed taxonomy and conclude "my proposal isn't on the list, so it's fine." If §8 says no, the answer is no, regardless of whether a named match exists.

7.1 Compensating-layer pattern

Definition. Adding a layer (timer, watcher, retry, alert, recovery doc) that absorbs a failure mode upstream of itself instead of fixing the upstream cause.

Example from session 2026-05-02..03. cis490-autoupdate.timer to drag stale peers forward. The actual fix was the operator's deploy process; the timer existed because deployment was unreliable and we patched around the unreliability instead of fixing it.

Test. If I removed this layer right now, would the original problem reappear immediately? If yes, the layer is a compensating bandaid for an unfixed root cause.

What to do instead. Fix the upstream cause. If you cannot in this change, fail loudly (§9) and stop.

7.2 Phasing-as-deferral pattern

Definition. Splitting a correctness fix into "phase 1, phase 2," "light vs deep," or "land this now, the harder part later." Any sequencing that ships a half-honest intermediate state.

Example from session 2026-05-02..03. "Land preflight first, labeller refactor later." The intermediate state ships dishonest data because the labeller is still clock-driven.

Test. Does each intermediate merge ship dishonest data, or rely on a layer that won't exist yet? If yes, no phasing.

What to do instead. Reduce scope (drop a feature, narrow the active set) until the change is small enough to land in one merge. Do not defer the hard part.

7.3 Single-instance-fix pattern

Definition. Fixing one item from a class while leaving the other items as future work.

Example from session 2026-05-02..03. "I'll diagnose perf and samba in parallel" while guest-agent, qmp, netflow, and the rest of the module catalog stay broken.

Test. Is this a class of N items, of which I'm fixing < N? If yes, fix all or remove the unfixed from the active set.

What to do instead. Either fix every member of the class, or shrink the active catalog to just the verified members. Unverified members do not ship.

7.4 Per-host-divergence pattern

Definition. Accepting that two hosts behave differently as a working assumption.

Example from session 2026-05-02..03. "Which host should I investigate samba on, elliott or k-gamingcom?" — implying the answer matters because hosts are different.

Test. Given identical workloads on identical canonical-manifest hosts, would the produced episodes be identical? If no, the divergence is the bug.

What to do instead. Eliminate the divergence (one canonical manifest, one canonical target VM build, one canonical collector set — §4.1). If a host can't run the canonical experiment, it produces zero episodes.

7.5 Black-box-trust pattern

Definition. Treating an externally-built artifact as if it behaves correctly under our experiments without a verifiable spec for what it should do.

Example from session 2026-05-02..03. Metasploitable2 from a SourceForge mirror — we don't know what version of Samba is running, whether the service is up, or whether the image has been altered. We were shipping modules targeting it anyway.

Test. Do we have a verifiable spec for this artifact's behavior? If no, we don't trust it.

What to do instead. Build the artifact from a declarative spec we control (§4.2). If we can't, remove modules targeting it from the catalog.

7.6 Investigation-as-deferral pattern

Definition. Proposing investigation when a verifiable gate would suffice. The investigation itself becomes the deferred work.

Example from session 2026-05-02..03. "I need to diagnose why perf is silent before I can write the gate." A gate of the form "perf must produce ≥1 row" works without knowing the cause; it forces the diagnosis to happen as part of the fix.

Test. Can the gate be expressed as an assertion ("X must produce > 0 rows" / "X must observe Y event") without knowing the root cause? If yes, write the gate first.

What to do instead. Write the strictest possible gate first. The investigation is the work of making the gate pass.

7.7 Speculation-as-evidence pattern

Definition. Asserting a claim as fact without measurement.

Example from session 2026-05-02..03. "30s vs 120s won't change this — if the exploit were almost working, we'd see occasional opens." No data was gathered; the claim was projected.

Test. Do I have a measurement that supports this claim? If no, I am speculating.

What to do instead. Say "I don't know yet." Either gather data or design the fix to be correct under both possibilities.

7.8 Out-of-scope-for-correctness pattern

Definition. Naming a correctness-affecting item as "out of scope" to avoid the harder problem.

Example from session 2026-05-02..03. "Manifest canonicalization is out of scope, flagged as known issue." Per-host config divergence is the source of half the data quality problems; excluding it from scope was a deferral.

Test. Does excluding this item leave the system half-honest? If yes, it is in scope.

What to do instead. Reduce other scope (drop a feature, narrow the active set) to fit. Correctness items cannot be deferred.

7.9 Defensive-instead-of-corrective pattern

Definition. Building rejection logic at the consumer instead of fixing the producer that produces the rejected output.

Example from session 2026-05-02..03. Receiver-side rejection of dishonest episodes without fixing why the producer produces them. Defense-in-depth (both ends gated) is good; defense-without- corrective (only consumer gated) is a bandaid.

Test. Does this fix make the dishonest behavior IMPOSSIBLE upstream, or only unobservable downstream? If only unobservable, the producer is still broken.

What to do instead. Fix the producer first. The consumer-side gate is defense-in-depth on top of a corrected producer, never a substitute.

7.10 Recovery-layer pattern

Definition. Building documentation, scripts, timers, or runbooks for "what to do when X is stuck." Applies anywhere in the pipeline — producer, receiver, trainer, dashboard, install scripts, on-device agents, anywhere a "recovery from a state that shouldn't exist" layer is contemplated. Producer-side is just the most common location.

Example from session 2026-05-02..03. FIXYOURSELF.md — a 250-line decision tree for recovering hosts whose auto-update timer couldn't fix them. The states it covered shouldn't have been possible if the producer were correct.

Test. Can the stuck state happen at all if the relevant component is correct? If no, delete the recovery layer and fix the component.

What to do instead. Make the stuck state impossible. If you can't, fail loudly (§9) and stop.

---

8. Decision tests before proposing a change

Before adding any code, doc, layer, or feature, answer all of the following. Any uncomfortable answer means stop and re-evaluate.

1. Does this change make the dishonest behavior IMPOSSIBLE, or only less likely / less observable?
2. Does this change scale to every instance of the problem class, or only one?
3. If I removed this change, would the underlying problem return immediately?
4. Am I adding a layer? If yes, can I instead remove the layer that allowed the failure?
5. Does this proposal match any pattern in §7? If yes, abandon it and propose a corrective fix.
6. Is the change complete in one merge? If not, why is the intermediate state honest?
7. Am I doing this because it's correct, or because it's the easiest thing that looks like progress?

If you cannot answer all seven cleanly, stop. Ask the operator. Do not proceed.

---

9. What to do when blocked

When you cannot fix something cleanly in scope:

• Fail loudly. Exit with a distinguishable code (e.g., 78). Write a structured failure record. Do not retry silently.
• Stop. Do not continue producing output as if the failure didn't happen.
• Ask the operator. Tell the user what's blocked, what you tried, and what you need to proceed.
• Do not build a recovery layer. That is the recovery-layer pattern (§7.10).
• Do not propose phased fixes. That is the phasing-as-deferral pattern (§7.2).
• Do not narrow scope silently. If the active set must shrink to make the change tractable, name it explicitly and get sign-off.

The operator prefers a small honest system that fails loudly over a large half-broken one that limps. A loud failure is more useful than a silent bandaid.

---

10. Definitions of ground truth

For each collector, "real row" means the row was actually emitted by the underlying mechanism for this episode, not synthesized, defaulted, or carried over from a previous run.

Collector	Ground truth means
proc	Row read from /proc/<qemu_pid>/{stat,io,status} for the live qemu PID of this episode's target VM, while that PID is alive.
qmp	Row obtained from a successful QMP query-status / query-blockstats round-trip on cfg.qmp_socket for this episode's qemu PID.
netflow	Row computed from packet capture on cfg.bridge_iface for traffic involving this episode's target VM during the episode wall-clock window.
perf	Row produced by perf (or equivalent) sampling this episode's qemu PID. Not from a previous run, not from a different PID.
guest	Row received from the in-guest agent over the virtio-serial channel during the episode wall-clock window. The agent must be running in this episode's guest, not a stale one.
pcap	Bytes captured from cfg.bridge_iface during the episode wall-clock window, written to network.pcap.

For each phase, "label justified" means the corresponding event was observed:

Phase	Justified by
clean	Episode start (orchestrator-emitted).
armed	Orchestrator instructs the driver to fire (orchestrator-emitted).
infecting	exploit_fire event observed in events.jsonl.
infected_running	session_open event observed in events.jsonl. Not session_open_timeout, not schedule-clock.
dormant	Observed in-session idle (no traffic / no command activity for N seconds).
failed	session_open_timeout or other terminal driver failure. Episode is rejected (§4.6).

A row that doesn't meet the ground-truth bar is not a row. A label that isn't justified is not a label. The acceptance gate (§4.6) enforces both.

---

11. Honest reporting

When you (a future session) report status to the operator:

• Distinguish merged from verified. "Code merged" is not "behavior verified in production." A passing test on a CI host is not the same as a working system on a lab host.
• Distinguish proposed from implemented. "I proposed X" is not "X is in the repo."
• Audit your cumulative pattern. At the end of a session, re-read your own changes against §7. It is possible to add three reasonable-looking layers in sequence that cumulatively form a compensating-layer pattern, even if no individual one looks like a bandaid.
• Name compensating layers you've built. If §7 audit finds matches, name them and propose their removal.
• Don't summarize cumulative changes as "fixes" without auditing. "I shipped 12 commits this session" is not the same as "the pipeline is honest now."
• Verify before agreeing or refuting. When the operator says something is done that you can verify, verify it before agreeing. When they say something is broken that you can verify, verify it before refuting.

---

12. Glossary

Terms used throughout this document, pinned to one definition.

Term	Definition
Canonical manifest	The single, version-pinned manifest.toml at the repo root. Every host loads this exact file. There is no per-host override (§4.1).
Active set	The collectors enabled in the canonical manifest for a given run. A collector is in the active set only if it has passed admission criteria (§4.4).
Catalog	The set of exploit modules in exploits/modules/*.toml that have passed admission (§4.3). Modules not in the catalog do not run.
Ground truth	A row or label is ground truth when it was emitted by the underlying mechanism for this episode, with the justifying event observed. See §10.
Episode boundary	An episode begins when the orchestrator emits the first clean label and ends when done.marker is written or the episode is moved to rejected/. All collector rows must fall inside this wall-clock window.
Configured collector	A collector listed as enabled in the canonical manifest. Distinct from "running collector" (the process actually started) and "active set" (the manifest-listed plus admission-passing intersection). For acceptance purposes, only the configured set matters.
Admission criteria	The bar a module / collector / target / override knob must pass to be in the active pipeline. See §4.3, §4.4, §13.
Honest	Of an episode: every label justified by an observed event, every configured collector emitted ≥1 ground-truth row, working tree was clean (or override-stamped), HEAD on origin/main. Of the pipeline: every accepted episode is honest.
Bandaid / compensating layer	A layer that absorbs a failure mode upstream of itself instead of fixing the upstream cause. See §7.1.
Override	A knob that loosens an admission criterion or gate. There is exactly one — CIS490_ALLOW_DIRTY (§14).
Operator	The human maintainer with sign-off authority. Distinct from agents that propose changes. See §15.
Containment regression	A change that weakens any of the §4.13 isolation requirements. Rejected regardless of claimed experimental value.

---

13. Admission scope (what triggers the bar)

Any change to the following is in admission scope and must pass §4 admission criteria + §15 operator sign-off:

• Any module in exploits/modules/*.toml.
• Any collector in the active set.
• Any field of manifest.toml.
• Any phase rule or label-emission code in the labeller.
• Any gate in the producer or receiver.
• Any schedule entry (phase budget, per-module timeout).
• Any target VM build spec or its containment posture (§4.13).
• Any override knob (the closed list in §14).

The following are NOT admission scope and can be changed without admission ceremony, but must still pass §8 decision tests:

• Internal refactors that do not change observable behavior of any of the above.
• Test code, fixtures, CI configuration.
• Documentation that does not contradict §1.
• Build/install scripts, insofar as they don't change what gets shipped or how it's labelled.

A future session that argues "this is just infrastructure" or "this is just tooling" to dodge admission scope: re-read this section. Anything that touches what gets shipped, how it's labelled, what runs on the host, the containment posture, or how the gate decides — is in scope. The "infrastructure / tooling" framing is a recurring sleight-of-hand vector and triggers automatic rejection.

---

14. Override knobs (closed list)

The complete list of override knobs in CIS490, version-pinned to this document:

Knob	Effect	Where audited
CIS490_ALLOW_DIRTY=1 (env var, orchestrator)	Allows the orchestrator to start with a dirty git tree. Stamps dirty_override: true in every meta.json produced. Receiver accepts only with matching stamp.	per-episode in meta.json

That is the entire list. Adding a knob to this list is itself an admission event (§13) requiring operator sign-off (§15) and an §8 review.

Knobs that have been considered and rejected (do not propose again without re-reading the rationale):

• verify_tls=false — TLS verification is a correctness boundary; bypassing it is the defensive-instead-of-corrective pattern (§7.9).
• skip_preflight=1 — preflight is the gate; bypassing it makes the gate non-functional.
• experimental_collector=true — bypassing collector admission is the single-instance-fix pattern (§7.3) wearing a flag.
• diagnostic_mode=true — generic bypass; in practice would be applied to hide failures, not investigate them.
• dry_run for the producer — episodes that aren't shipped go to rejected/; no dry-run flag needed.

If a future session proposes a new override knob, the burden is on the proposal: pass §8, get operator sign-off, amend §14 in the same merge. "Add the knob now and amend §14 later" is the phasing-as-deferral pattern (§7.2) applied to documentation.

---

15. Sign-off discipline

Admission decisions are made by the operator, not by agents acting alone. Specifically:

• Adding a module to the catalog requires operator sign-off. An agent runs scripts/verify-catalog.sh, presents the verification result, and the operator decides whether the module enters the catalog.
• Adding a collector to the active set requires operator sign-off. Agent runs the emit-test, operator decides.
• Promoting a target VM build requires operator sign-off after §4.2 verification and §4.13 containment tests pass.
• Adding an override knob (§14) requires operator sign-off.
• Amending PIPELINE.md requires operator sign-off (§16).

Removing anything from the catalog or active set does NOT require operator sign-off — the bar is asymmetric. Tightening is always permitted; loosening requires sign-off.

The operator is the human with maintainer credentials on the repository. Agents propose, run verification, and present results; the operator decides admission.

If an agent is acting in a non-interactive context (CI run, scheduled job) where no operator is available to sign off, the agent does not admit anything. It produces verification output and stops.

---

16. Amending PIPELINE.md

This document is not immutable, but it is the canonical statement of the bar. Amendments are governed by the same discipline as admission decisions:

1. Any change to §1 (principle), §4 (fix items), §7 (anti-patterns), §8 (decision tests), §10 (ground truth), §13 (admission scope), §14 (override list), or §15 (sign-off) is a substantive amendment.
2. Substantive amendments require operator sign-off (§15) and must pass §8 decision tests applied to the amendment itself.
3. The amendment lands in the same merge as the code change it justifies. "Amend the doc later" is the phasing pattern (§7.2).
4. Editorial changes (typos, formatting, link fixes, glossary wording) do not require sign-off but should be flagged in the commit message.

A future session that wants to add a feature or layer the document forbids: the path is to amend the document, not to work around it. "This isn't covered by PIPELINE.md, so I'll just do it" is the out-of-scope-for-correctness pattern (§7.8) applied to the meta-document. Anything that touches admission scope (§13) is covered even if not named explicitly.

If you find the document is wrong — internally inconsistent, contradicts observed reality, prescribes something impossible — file a Forgejo issue against the repo with the contradiction documented. Do not silently work around the doc.

---

17. What this plan supersedes

The following docs are deleted or rewritten as part of landing this plan:

Doc	Action
FIXYOURSELF.md	Deleted. Compensating-layer doc; the states it covers don't exist after §4.6.
AGENTS.md "symptom→fix table"	Deleted. Bandaid-driven.
AGENTS.md "Hosts self-update" section	Deleted. Hosts run pinned commits.
AGENTS.md "Tier 3+4 deploy zero-touch" claim	Rewritten. Targets are built locally now, not auto-fetched.
AGENTS.md "trust the in-guest probe alone, cross-check host CPU"	Deleted. The producer-side gate makes this fictional cross-check unnecessary.
TIER3-BRINGUP.md	Kept as historical record — labelled bug report, not current guidance.
README.md Tier-3+4 narrative	Reviewed and aligned.

If you are a future session reading this and find another doc that contradicts §1–§6 of this file: this file is right and the other doc is wrong. Fix the other doc.