diff --git a/etc/README.md b/etc/README.md index 6675d8d..da592bf 100644 --- a/etc/README.md +++ b/etc/README.md @@ -3,6 +3,7 @@ Templates for system-level files installed by `scripts/install-*.sh`: - `cis490-receiver.service` — systemd unit for the receiver +- `cis490-dashboard.service` — systemd unit for the dashboard.wg live display - `receiver.toml.example` — config template for the receiver - `cis490-orchestrator.service` (TODO) — systemd unit for the orchestrator - `cis490-shipper.service` (TODO) — systemd unit for the shipper diff --git a/etc/cis490-dashboard.service b/etc/cis490-dashboard.service new file mode 100644 index 0000000..9951c22 --- /dev/null +++ b/etc/cis490-dashboard.service @@ -0,0 +1,30 @@ +[Unit] +Description=CIS490 live dashboard (dashboard.wg) +Documentation=https://maxgit.wg/spectral/CIS490 +After=network-online.target +Wants=network-online.target + +[Service] +Type=simple +User=cis490 +Group=cis490 +WorkingDirectory=/opt/cis490 +ExecStart=/opt/cis490/.venv/bin/python -m training.dashboard --host 127.0.0.1 --port 8447 +Restart=on-failure +RestartSec=5 + +# Hardening +NoNewPrivileges=true +PrivateTmp=true +ProtectSystem=strict +ProtectHome=true +ProtectKernelTunables=true +ProtectKernelModules=true +ProtectControlGroups=true +LockPersonality=true +RestrictNamespaces=true +RestrictRealtime=true +SystemCallArchitectures=native + +[Install] +WantedBy=multi-user.target diff --git a/training/dashboard/README.md b/training/dashboard/README.md new file mode 100644 index 0000000..d439059 --- /dev/null +++ b/training/dashboard/README.md @@ -0,0 +1,57 @@ +# training/dashboard/ + +Live web display served at `https://dashboard.wg`. A Starlette app +on `127.0.0.1:8447` behind Caddy; messages from Python are pushed to +connected browsers over a WebSocket. + +This is intentionally a **blank slate** — the default page just +appends every received JSON message to a scrolling log. Build the +real widgets on top of `window.dashboard.onMessage`. + +## Run locally + +```sh +uv run python -m training.dashboard +# → open http://127.0.0.1:8447 +``` + +## Push live data from Python + +**Same process** (e.g. a notebook driving the page): + +```py +from training.dashboard.app import broadcaster +await broadcaster.publish({"type": "metric", "name": "loss", "value": 0.42}) +``` + +**Different process** (orchestrator, receiver, ad-hoc shell): + +```sh +curl -s http://127.0.0.1:8447/publish \ + -H 'content-type: application/json' \ + -d '{"type":"hello","msg":"from cron"}' +``` + +The `/publish` endpoint is loopback-only (403 otherwise) and is **not** +reverse-proxied by Caddy, so it cannot be hit from the WG mesh. + +## Customizing the page + +The default `static/index.html` exposes `window.dashboard`: + +```js +window.dashboard.onMessage = (msg) => { + if (msg.type === 'metric') updateChart(msg.name, msg.value); +}; +window.dashboard.send({type: 'request-snapshot'}); // browser → server +``` + +Override `onMessage` to dispatch to your own widgets. The blank-slate +log renderer is just there so a fresh deploy is observably alive. + +## Deploying to the Pi + +1. `sudo cp etc/cis490-dashboard.service /etc/systemd/system/` +2. `sudo cp training/dashboard/dashboard.caddy /etc/caddy/Caddyfile.d/` +3. `sudo systemctl daemon-reload && sudo systemctl enable --now cis490-dashboard.service` +4. `sudo systemctl reload caddy` diff --git a/training/dashboard/__init__.py b/training/dashboard/__init__.py new file mode 100644 index 0000000..7ff2af0 --- /dev/null +++ b/training/dashboard/__init__.py @@ -0,0 +1,12 @@ +"""Live dashboard served at dashboard.wg. + +Blank-slate web display whose live data is fed from Python over a +WebSocket. Other code in the repo can push events to connected +browsers two ways: + + - In-process: ``from training.dashboard.app import broadcaster`` + then ``await broadcaster.publish({...})``. + - Cross-process (e.g. orchestrator/receiver): POST a JSON body to + ``http://127.0.0.1:8447/publish``. The endpoint is bound to + loopback and is not reverse-proxied by Caddy. +""" diff --git a/training/dashboard/__main__.py b/training/dashboard/__main__.py new file mode 100644 index 0000000..23adee6 --- /dev/null +++ b/training/dashboard/__main__.py @@ -0,0 +1,46 @@ +from __future__ import annotations + +import argparse +import logging +import os +from pathlib import Path + +import uvicorn + +from .app import make_app + + +def main() -> None: + parser = argparse.ArgumentParser(prog="cis490-dashboard") + parser.add_argument( + "--host", + default=os.environ.get("CIS490_DASHBOARD_HOST", "127.0.0.1"), + help="bind address (default 127.0.0.1; Caddy reverse-proxies dashboard.wg here)", + ) + parser.add_argument( + "--port", type=int, + default=int(os.environ.get("CIS490_DASHBOARD_PORT", "8447")), + help="bind port (default 8447)", + ) + parser.add_argument( + "--data-root", type=Path, + default=Path(os.environ.get("CIS490_DATA_ROOT", "/var/lib/cis490")), + help="receiver data root that the feeders read from", + ) + parser.add_argument( + "--no-feeders", action="store_true", + help="don't start the on-disk feeders (useful for local dev)", + ) + args = parser.parse_args() + + logging.basicConfig( + level=logging.INFO, + format="%(asctime)s %(levelname)s %(name)s %(message)s", + ) + + app = make_app(enable_feeders=not args.no_feeders, data_root=args.data_root) + uvicorn.run(app, host=args.host, port=args.port, log_config=None) + + +if __name__ == "__main__": + main() diff --git a/training/dashboard/app.py b/training/dashboard/app.py new file mode 100644 index 0000000..6091b05 --- /dev/null +++ b/training/dashboard/app.py @@ -0,0 +1,143 @@ +from __future__ import annotations + +import asyncio +import json +import logging +from contextlib import asynccontextmanager +from pathlib import Path +from typing import Any + +from starlette.applications import Starlette +from starlette.requests import Request +from starlette.responses import FileResponse, JSONResponse, Response +from starlette.routing import Mount, Route, WebSocketRoute +from starlette.staticfiles import StaticFiles +from starlette.websockets import WebSocket, WebSocketDisconnect + + +log = logging.getLogger("cis490.dashboard") + +STATIC_DIR = Path(__file__).parent / "static" + + +class Broadcaster: + """Tiny fan-out hub. Per-client async queues, oldest-message-drop + on backpressure. Also holds a ``state`` dict that is sent to every + new client on connect so reconnects don't see a cold dashboard.""" + + QUEUE_MAX = 64 + + def __init__(self) -> None: + self._clients: set[asyncio.Queue[Any]] = set() + self._lock = asyncio.Lock() + self.state: dict[str, Any] = {} + + async def register(self) -> asyncio.Queue[Any]: + q: asyncio.Queue[Any] = asyncio.Queue(maxsize=self.QUEUE_MAX) + async with self._lock: + self._clients.add(q) + return q + + async def unregister(self, q: asyncio.Queue[Any]) -> None: + async with self._lock: + self._clients.discard(q) + + async def publish(self, msg: Any) -> int: + delivered = 0 + async with self._lock: + clients = list(self._clients) + for q in clients: + try: + q.put_nowait(msg) + delivered += 1 + except asyncio.QueueFull: + try: q.get_nowait() + except asyncio.QueueEmpty: pass + try: + q.put_nowait(msg) + delivered += 1 + except asyncio.QueueFull: + pass + return delivered + + @property + def client_count(self) -> int: + return len(self._clients) + + +broadcaster = Broadcaster() + + +def make_app( + *, + allow_publish_from: set[str] | None = None, + enable_feeders: bool = True, + data_root: Path = Path("/var/lib/cis490"), +) -> Starlette: + allowed = allow_publish_from if allow_publish_from is not None else {"127.0.0.1", "::1"} + + @asynccontextmanager + async def lifespan(app): + tasks: list[asyncio.Task] = [] + if enable_feeders: + from .feeder import start_feeders + try: + tasks = start_feeders(broadcaster, data_root=data_root) + log.info("started %d feeders rooted at %s", len(tasks), data_root) + except Exception: + log.exception("failed to start feeders; dashboard will run without them") + try: + yield + finally: + for t in tasks: t.cancel() + if tasks: + await asyncio.gather(*tasks, return_exceptions=True) + log.info("dashboard lifespan exiting; %d feeders cancelled", len(tasks)) + + async def index(request: Request) -> Response: + return FileResponse(STATIC_DIR / "index.html") + + async def healthz(request: Request) -> JSONResponse: + return JSONResponse({ + "status": "ok", + "clients": broadcaster.client_count, + "state_keys": sorted(broadcaster.state.keys()), + }) + + async def publish(request: Request) -> JSONResponse: + client_host = request.client.host if request.client else "" + if client_host not in allowed: + return JSONResponse({"error": "forbidden"}, status_code=403) + try: + body = await request.json() + except (ValueError, json.JSONDecodeError): + return JSONResponse({"error": "body must be JSON"}, status_code=400) + delivered = await broadcaster.publish(body) + return JSONResponse({"delivered": delivered}) + + async def ws_endpoint(ws: WebSocket) -> None: + await ws.accept() + q = await broadcaster.register() + try: + await ws.send_json({"type": "hello", "clients": broadcaster.client_count}) + # Send the current snapshot so reconnects start warm. + if broadcaster.state: + await ws.send_json({"type": "snapshot", **broadcaster.state}) + while True: + msg = await q.get() + await ws.send_json(msg) + except WebSocketDisconnect: + pass + except Exception: + log.exception("ws send failed") + finally: + await broadcaster.unregister(q) + + routes = [ + Route("/", index, methods=["GET"]), + Route("/healthz", healthz, methods=["GET"]), + Route("/publish", publish, methods=["POST"]), + WebSocketRoute("/ws", ws_endpoint), + Mount("/static", app=StaticFiles(directory=str(STATIC_DIR)), name="static"), + ] + return Starlette(routes=routes, lifespan=lifespan) diff --git a/training/dashboard/dashboard.caddy b/training/dashboard/dashboard.caddy new file mode 100644 index 0000000..9b6b3f6 --- /dev/null +++ b/training/dashboard/dashboard.caddy @@ -0,0 +1,23 @@ +# CIS490 live dashboard — blank-slate web display fed by Python over +# a WebSocket. Reachable as https://dashboard.wg from any peer on the +# WG mesh. +# +# Trust boundary: "reached :443 over WG", same as bootstrap.wg — +# iptmonads has already gated WG peers, so no client_auth here. +# +# /publish is the producer ingest endpoint. The dashboard service +# binds 127.0.0.1:8447 and enforces a loopback allow-list at the app +# layer, but everything Caddy proxies arrives at the upstream FROM +# 127.0.0.1 (Caddy itself), so the app-layer check would be bypassed +# for WG callers. We explicitly 404 it here so producers MUST POST +# directly to 127.0.0.1:8447 from the Pi. +# +# Upstream: cis490-dashboard.service on 127.0.0.1:8447. +dashboard.wg { + tls internal + + @publish path /publish /publish/* + respond @publish "Not Found" 404 + + reverse_proxy 127.0.0.1:8447 +} diff --git a/training/dashboard/feeder.py b/training/dashboard/feeder.py new file mode 100644 index 0000000..fd26e0d --- /dev/null +++ b/training/dashboard/feeder.py @@ -0,0 +1,292 @@ +"""Real producers that wire the receiver's on-disk state to the +dashboard message bus. + +Three feeders, all started by ``app.lifespan``: + + - ``watch_index_jsonl`` — tails ``/var/lib/cis490/index.jsonl`` and + publishes one ``episode`` event per new line. Survives file + rotation by tracking inode. + - ``snapshot_loop`` — periodically derives ground-truth from disk + (per-host episode counts, total counts, alert tail) and updates + the broadcaster's persistent ``state`` so reconnecting clients + see warm numbers, not zero. + - ``watch_alerts_jsonl`` — same as the index tailer but for the + receiver's alerts log. + +If a path doesn't exist (e.g. ``health/`` on a fresh deploy) the +feeder logs once and keeps polling — it'll start producing events the +moment the path appears. +""" +from __future__ import annotations + +import asyncio +import json +import logging +import os +from pathlib import Path +from typing import Any, Awaitable, Callable + + +log = logging.getLogger("cis490.dashboard.feeder") + +DEFAULT_DATA_ROOT = Path("/var/lib/cis490") + +PublishFn = Callable[[dict[str, Any]], Awaitable[int]] + + +# ───────────────────────────────────────────────────────────────────── +# Tail helpers +# ───────────────────────────────────────────────────────────────────── + +async def _tail_jsonl( + path: Path, + publish: PublishFn, + parse: Callable[[dict], dict | None], + *, + poll_interval: float = 1.0, + label: str = "tail", +) -> None: + """Generic append-only JSONL tailer. ``parse`` shapes each record + into the dict we publish (return None to skip).""" + fd = None + inode: int | None = None + missing_logged = False + while True: + try: + if not path.exists(): + if not missing_logged: + log.info("[%s] %s does not exist; will retry", label, path) + missing_logged = True + await asyncio.sleep(poll_interval * 5) + continue + missing_logged = False + + st = path.stat() + if fd is None or inode != st.st_ino: + if fd is not None: + try: fd.close() + except Exception: pass + fd = path.open("r", encoding="utf-8", errors="replace") + inode = st.st_ino + fd.seek(0, os.SEEK_END) + log.info("[%s] watching %s (inode=%d, starting at %d bytes)", + label, path, inode, fd.tell()) + + chunk = await asyncio.to_thread(fd.read) + if chunk: + for line in chunk.splitlines(): + line = line.strip() + if not line: + continue + try: + rec = json.loads(line) + except json.JSONDecodeError: + log.warning("[%s] skipping malformed line: %r", label, line[:120]) + continue + out = parse(rec) + if out is not None: + await publish(out) + await asyncio.sleep(poll_interval) + except asyncio.CancelledError: + raise + except Exception: + log.exception("[%s] error; reopening in 5s", label) + if fd is not None: + try: fd.close() + except Exception: pass + fd = None + inode = None + await asyncio.sleep(5) + + +# ───────────────────────────────────────────────────────────────────── +# Specific feeders +# ───────────────────────────────────────────────────────────────────── + +async def watch_index_jsonl(broadcaster, path: Path) -> None: + """Episode ingest log. One ``episode`` event per new line, plus + we maintain ``broadcaster.state["recent_episodes"]`` in lockstep + so reconnecting clients see warm history without re-reading + index.jsonl from disk.""" + def parse(rec: dict) -> dict | None: + if "episode_id" not in rec or "host_id" not in rec: + return None + ep = { + "episode_id": rec.get("episode_id"), + "host_id": rec.get("host_id"), + "sha256": rec.get("sha256"), + "size_bytes": rec.get("size_bytes"), + "received_at": rec.get("received_at_wall"), + } + # Keep the live ring buffer + running totals in sync. + recent = broadcaster.state.setdefault("recent_episodes", []) + recent.insert(0, ep) + if len(recent) > RECENT_EPISODES_LIMIT: + del recent[RECENT_EPISODES_LIMIT:] + sb = ep["size_bytes"] + if isinstance(sb, (int, float)): + broadcaster.state["total_bytes"] = ( + int(broadcaster.state.get("total_bytes", 0)) + int(sb) + ) + broadcaster.state["total_episodes"] = ( + int(broadcaster.state.get("total_episodes", 0)) + 1 + ) + return {"type": "episode", **ep} + await _tail_jsonl(path, broadcaster.publish, parse, label="index.jsonl") + + +async def watch_alerts_jsonl(publish: PublishFn, path: Path) -> None: + """Operator-facing alerts (sick hosts, stuck shippers, etc.).""" + def parse(rec: dict) -> dict | None: + return { + "type": "alert", + "host_id": rec.get("host"), + "symptom": rec.get("symptom"), + "detail": rec.get("detail"), + "suggested_fix": rec.get("suggested_fix"), + "detected_at": rec.get("detected_at_wall"), + "dedup_key": rec.get("dedup_key"), + } + await _tail_jsonl(path, publish, parse, label="alerts.jsonl") + + +def _count_lines(p: Path) -> int: + """Cheap line count — used for bootstrap totals. We accept 'one + extra line at the moment we read mid-write' as acceptable noise.""" + try: + with p.open("rb") as f: + return sum(1 for _ in f) + except OSError: + return 0 + + +RECENT_EPISODES_LIMIT = 200 + + +def _snapshot_state( + data_root: Path, + index_path: Path, + alerts_path: Path, + *, + recent_limit: int = RECENT_EPISODES_LIMIT, +) -> dict: + """Derive the canonical view from disk in one pass. + + Reads index.jsonl front-to-back to collect ``total_episodes``, + ``total_bytes`` and the trailing ``recent_episodes`` window for + the database-explorer widget. ~76k lines / 23 MiB takes ~1 s on + the Pi; this runs in ``to_thread`` so the event loop is unaffected. + Counts per host come from the filesystem listing — cheaper and + immune to JSON parse hiccups.""" + host_counts: dict[str, int] = {} + episodes_root = data_root / "episodes" + if episodes_root.exists(): + for host_dir in episodes_root.iterdir(): + if not host_dir.is_dir(): + continue + try: + host_counts[host_dir.name] = sum( + 1 for entry in host_dir.iterdir() if entry.is_file() + ) + except OSError: + continue + + total_episodes = 0 + total_bytes = 0 + recent: list[dict] = [] + if index_path.exists(): + try: + with index_path.open("r", encoding="utf-8", errors="replace") as f: + for line in f: + line = line.strip() + if not line: + continue + try: + rec = json.loads(line) + except json.JSONDecodeError: + continue + total_episodes += 1 + sb = rec.get("size_bytes") + if isinstance(sb, (int, float)): + total_bytes += int(sb) + recent.append({ + "episode_id": rec.get("episode_id"), + "host_id": rec.get("host_id"), + "received_at": rec.get("received_at_wall"), + "size_bytes": sb, + "sha256": rec.get("sha256"), + }) + # Periodic truncation keeps the buffer from growing + # unboundedly while we read all 76k lines. + if len(recent) > recent_limit * 4: + recent = recent[-recent_limit:] + except OSError: + pass + recent = recent[-recent_limit:] + recent.reverse() # newest-first for the UI + + return { + "total_episodes": total_episodes, + "total_alerts": _count_lines(alerts_path), + "host_counts": host_counts, + "total_bytes": total_bytes, + "recent_episodes": recent, + } + + +async def snapshot_loop( + broadcaster, + *, + data_root: Path, + index_path: Path, + alerts_path: Path, + poll_interval: float = 30.0, +) -> None: + """Refresh the broadcaster's persistent ``state`` periodically so + reconnecting clients see disk-truth, not just the in-session + delta. Also publishes a ``snapshot`` event so already-connected + widgets that want a hard reset can re-key on it.""" + first = True + while True: + try: + snap = await asyncio.to_thread( + _snapshot_state, data_root, index_path, alerts_path + ) + broadcaster.state = snap + if first: + log.info( + "snapshot: total_episodes=%d total_alerts=%d hosts=%d", + snap["total_episodes"], snap["total_alerts"], len(snap["host_counts"]), + ) + first = False + await broadcaster.publish({"type": "snapshot", **snap}) + except asyncio.CancelledError: + raise + except Exception: + log.exception("snapshot_loop error") + await asyncio.sleep(poll_interval) + + +# ───────────────────────────────────────────────────────────────────── +# Lifecycle +# ───────────────────────────────────────────────────────────────────── + +def start_feeders(broadcaster, *, data_root: Path = DEFAULT_DATA_ROOT) -> list[asyncio.Task]: + """Kick off all feeder tasks. Caller is responsible for cancelling + them on shutdown (lifespan context handles that).""" + index_path = data_root / "index.jsonl" + alerts_path = data_root / "alerts.jsonl" + publish = broadcaster.publish + tasks = [ + asyncio.create_task( + snapshot_loop(broadcaster, data_root=data_root, + index_path=index_path, alerts_path=alerts_path), + name="cis490.feeder.snapshot"), + asyncio.create_task( + watch_index_jsonl(broadcaster, index_path), + name="cis490.feeder.index"), + asyncio.create_task( + watch_alerts_jsonl(publish, alerts_path), + name="cis490.feeder.alerts"), + ] + return tasks diff --git a/training/dashboard/static/dashboard.css b/training/dashboard/static/dashboard.css new file mode 100644 index 0000000..6687f90 --- /dev/null +++ b/training/dashboard/static/dashboard.css @@ -0,0 +1,417 @@ +:root { + color-scheme: dark; + --bg: #07090d; + --bg-rgb: 7, 9, 13; + --bg-elev: #0d1117; + --bg-elev2: #161b22; + --fg: #e6edf3; + --fg-dim: #8b949e; + --fg-mute: #484f58; + --line: #1c2128; + --line-soft: #21262d; + --accent: #58a6ff; + --accent-soft: rgba(88, 166, 255, 0.15); + --warn: #f85149; + --ok: #3fb950; + --phase-clean: #3fb950; + --phase-armed: #d29922; + --phase-infecting: #db61a2; + --phase-running: #f85149; + --phase-dormant: #6e7681; + --topbar-h: 44px; + --prose-w: 36em; + --scene-fade-ms: 600ms; +} + +* { box-sizing: border-box; } +html, body { + margin: 0; padding: 0; background: var(--bg); color: var(--fg); + font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", system-ui, sans-serif; + -webkit-font-smoothing: antialiased; overflow-x: hidden; +} + +/* ─── Topbar ───────────────────────────────────────────────────────── */ +.topbar { + position: fixed; top: 0; left: 0; right: 0; height: var(--topbar-h); z-index: 50; + display: flex; align-items: center; gap: 10px; padding: 0 16px; + background: rgba(7, 9, 13, 0.85); backdrop-filter: blur(8px); + border-bottom: 1px solid var(--line); font-size: 12px; +} +.topbar .brand { font-weight: 700; letter-spacing: 0.04em; } +.topbar .spacer { flex: 1; } +.topbar .status { color: var(--fg-dim); } +.topbar .status.ok { color: var(--ok); } +.topbar .status.bad { color: var(--warn); } +.topbar .counter { + font-family: ui-monospace, SFMono-Regular, Menlo, monospace; + color: var(--fg-dim); font-variant-numeric: tabular-nums; padding: 0 6px; +} +.topbar button.ghost { + background: transparent; color: var(--fg-dim); border: 1px solid var(--line); + font: inherit; padding: 4px 10px; border-radius: 4px; cursor: pointer; + transition: color 120ms, border-color 120ms; +} +.topbar button.ghost:hover { color: var(--fg); border-color: var(--fg-mute); } +.topbar button.ghost:disabled { opacity: 0.35; cursor: not-allowed; } +.topbar button.ghost.icon { padding: 4px 8px; min-width: 28px; } +.topbar button.ghost.active { color: var(--accent); border-color: var(--accent); } + +/* ─── Layout ───────────────────────────────────────────────────────── */ +.layout { position: relative; } +.canvas-wrapper { + position: fixed; + top: var(--topbar-h); left: 0; right: 0; + height: calc(100vh - var(--topbar-h)); + z-index: 1; overflow: hidden; +} +.article { + position: relative; z-index: 2; + padding-top: var(--topbar-h); pointer-events: none; +} +.article .prose { pointer-events: auto; } + +/* ─── Stage views ──────────────────────────────────────────────────── */ +.stage { position: absolute; inset: 0; cursor: pointer; } +.stage-view { + position: absolute; inset: 0; + display: flex; align-items: center; justify-content: flex-start; + /* Reserve right-side space for prose. The clamp keeps it sane on + narrow viewports and unbounded ones. */ + padding-right: clamp(0px, calc(var(--prose-w) - 4em), 38em); + opacity: 0; transition: opacity var(--scene-fade-ms) ease; + pointer-events: none; +} +.stage-view[data-active] { opacity: 1; pointer-events: auto; } + +/* Intro stage */ +.bg-grid { + position: absolute; inset: 0; + background-image: + linear-gradient(rgba(88,166,255,0.07) 1px, transparent 1px), + linear-gradient(90deg, rgba(88,166,255,0.07) 1px, transparent 1px); + background-size: 48px 48px; + mask-image: radial-gradient(ellipse at center, #000 0%, transparent 75%); + animation: drift 18s linear infinite; +} +@keyframes drift { + from { background-position: 0 0, 0 0; } + to { background-position: 48px 48px, 48px 48px; } +} +.intro-block { + position: relative; z-index: 1; text-align: left; + padding: 0 clamp(32px, 5vw, 80px); width: 100%; +} +.intro-eyebrow { + font-size: 12px; letter-spacing: 0.18em; text-transform: uppercase; + color: var(--fg-dim); margin-bottom: 18px; +} +.intro-title { + font-size: clamp(56px, 9vw, 168px); line-height: 0.95; font-weight: 700; + letter-spacing: -0.04em; + background: linear-gradient(180deg, #fff 0%, #8b949e 100%); + -webkit-background-clip: text; background-clip: text; color: transparent; +} + +/* ─── Metric stack — calculated, viewport-relative sizing ──────────── */ +/* Wide-by-default. Inside a stage-view the right padding already + reserves room for prose, so width:100% means "use everything left + of the prose column." */ +.metric-stack { + text-align: left; + padding: 0 clamp(40px, 5vw, 88px); + width: 100%; max-width: none; + display: flex; flex-direction: column; + gap: clamp(10px, 1.4vh, 22px); +} +.metric-stack-wide { + /* Slightly less right padding than other stages so the table can + stretch into the gradient zone of the prose column. */ + padding-right: clamp(24px, 3vw, 56px); +} +.metric-eyebrow { + font-size: clamp(11px, 1vw, 14px); + letter-spacing: 0.18em; text-transform: uppercase; color: var(--fg-dim); +} +.metric-big { + font-size: clamp(72px, 13vw, 240px); + line-height: 0.95; font-weight: 700; + letter-spacing: -0.04em; font-variant-numeric: tabular-nums; + font-family: ui-monospace, SFMono-Regular, Menlo, monospace; +} +.metric-sub { color: var(--fg-dim); font-size: clamp(13px, 1vw, 16px); + line-height: 1.55; font-variant-numeric: tabular-nums; } +.metric-sub code { + font-family: ui-monospace, SFMono-Regular, Menlo, monospace; + font-size: 0.9em; color: var(--accent); + background: var(--accent-soft); padding: 1px 5px; border-radius: 3px; +} + +.awaiting { + color: var(--fg-mute); font-size: clamp(12px, 0.95vw, 14px); + font-family: ui-monospace, SFMono-Regular, Menlo, monospace; + padding: 16px 0; font-style: italic; +} + +/* Sparkline */ +.sparkline { + width: 100%; height: clamp(140px, 28vh, 360px); margin-top: 6px; +} +.sparkline path { fill: none; stroke: var(--accent); stroke-width: 1.5; + vector-effect: non-scaling-stroke; } +.sparkline #ingest-spark-fill { fill: var(--accent-soft); stroke: none; } + +/* Per-host bars */ +.bars { display: flex; flex-direction: column; gap: clamp(8px, 1.1vh, 14px); } +.bar-row { + display: grid; + grid-template-columns: minmax(140px, 18ch) 1fr minmax(72px, 10ch); + gap: clamp(10px, 1vw, 18px); align-items: center; + font-variant-numeric: tabular-nums; +} +.bar-host { color: var(--fg); font-size: clamp(13px, 1vw, 15px); + font-family: ui-monospace, SFMono-Regular, Menlo, monospace; } +.bar-track { height: clamp(24px, 3vh, 40px); background: var(--bg-elev); + border: 1px solid var(--line); border-radius: 3px; overflow: hidden; } +.bar-fill { height: 100%; background: var(--accent); + transition: width 600ms cubic-bezier(0.2, 0.8, 0.2, 1); } +.bar-count { color: var(--fg-dim); font-size: clamp(13px, 1vw, 15px); text-align: right; + font-family: ui-monospace, SFMono-Regular, Menlo, monospace; } + +/* Phase mix */ +.phase-stack { + display: flex; height: clamp(48px, 7vh, 96px); + border-radius: 4px; overflow: hidden; + background: var(--bg-elev); border: 1px solid var(--line); +} +.phase-seg { transition: flex-grow 600ms ease; flex-grow: 0; min-width: 0; } +.phase-seg.clean { background: var(--phase-clean); } +.phase-seg.armed { background: var(--phase-armed); } +.phase-seg.infecting { background: var(--phase-infecting); } +.phase-seg.infected_running { background: var(--phase-running); } +.phase-seg.dormant { background: var(--phase-dormant); } +.phase-legend { display: flex; flex-wrap: wrap; gap: 14px; + font-size: 12px; color: var(--fg-dim); + font-family: ui-monospace, SFMono-Regular, Menlo, monospace; } +.phase-legend > span { display: inline-flex; align-items: center; } +.phase-legend .swatch { display: inline-block; width: 10px; height: 10px; + border-radius: 2px; margin-right: 6px; } + +/* ─── Database explorer ────────────────────────────────────────────── */ +.db-header { + display: flex; align-items: baseline; gap: 16px; flex-wrap: wrap; +} +.db-count { color: var(--fg-mute); font-size: 12px; + font-family: ui-monospace, SFMono-Regular, Menlo, monospace; + font-variant-numeric: tabular-nums; } +.db-controls { + display: flex; gap: 10px; align-items: center; flex-wrap: wrap; +} +.db-tabs { display: flex; gap: 6px; flex-wrap: wrap; } +.db-tab { + background: transparent; color: var(--fg-dim); + border: 1px solid var(--line); border-radius: 16px; + padding: 4px 12px; font: inherit; font-size: 12px; cursor: pointer; + font-family: ui-monospace, SFMono-Regular, Menlo, monospace; + transition: color 120ms, border-color 120ms, background 120ms; +} +.db-tab:hover { color: var(--fg); border-color: var(--fg-mute); } +.db-tab.active { color: var(--accent); border-color: var(--accent); + background: var(--accent-soft); } +.db-search { + flex: 1; min-width: 200px; + background: var(--bg-elev); color: var(--fg); + border: 1px solid var(--line); border-radius: 4px; + padding: 6px 10px; font: inherit; font-size: 13px; + font-family: ui-monospace, SFMono-Regular, Menlo, monospace; +} +.db-search:focus { outline: none; border-color: var(--accent); } +.db-table-wrap { + flex: 1 1 auto; min-height: 0; + max-height: clamp(280px, 56vh, 720px); + overflow: auto; + border: 1px solid var(--line); border-radius: 4px; + background: var(--bg-elev); +} +.db-table { + width: 100%; border-collapse: collapse; + font-size: clamp(12px, 0.92vw, 14px); + font-variant-numeric: tabular-nums; +} +.db-table thead th { + position: sticky; top: 0; z-index: 1; + background: var(--bg-elev2); color: var(--fg-dim); + text-align: left; padding: 8px 12px; + font-size: 11px; letter-spacing: 0.06em; text-transform: uppercase; + font-weight: 500; border-bottom: 1px solid var(--line); +} +.db-table tbody tr { + border-bottom: 1px solid var(--line-soft); cursor: pointer; + transition: background 80ms; +} +.db-table tbody tr:hover { background: rgba(88, 166, 255, 0.06); } +.db-table tbody tr.selected { background: var(--accent-soft); } +.db-table td { + padding: 7px 12px; color: var(--fg); + font-family: ui-monospace, SFMono-Regular, Menlo, monospace; +} +.db-table td.db-size { color: var(--fg-dim); text-align: right; } +.db-host { color: var(--fg); } +.db-id { color: var(--fg-dim); } +.db-detail { + max-height: clamp(180px, 30vh, 360px); overflow: auto; + border: 1px solid var(--line); border-radius: 4px; + background: var(--bg-elev); +} +.db-detail pre { + margin: 0; padding: 14px 18px; + font-family: ui-monospace, SFMono-Regular, Menlo, monospace; + font-size: 12px; line-height: 1.5; color: var(--fg); + white-space: pre-wrap; word-break: break-all; +} + +/* ─── Attack envelope thumbnails ───────────────────────────────────── */ +.profile-grid { + display: grid; grid-template-columns: repeat(2, minmax(0, 1fr)); + gap: clamp(12px, 1.4vw, 22px); +} +.profile-card { + border: 1px solid var(--line); border-radius: 4px; + padding: clamp(12px, 1.2vw, 18px); + background: rgba(13, 17, 23, 0.6); +} +.profile-name { + font-family: ui-monospace, SFMono-Regular, Menlo, monospace; + font-size: clamp(13px, 1vw, 15px); color: var(--fg); margin-bottom: 4px; +} +.profile-shape { + font-size: clamp(11px, 0.85vw, 13px); color: var(--fg-dim); + margin-bottom: 8px; line-height: 1.4; +} +.profile-card svg { + display: block; width: 100%; + height: clamp(56px, 9vh, 120px); +} +.profile-card svg path { fill: none; stroke: var(--accent); stroke-width: 1.4; + vector-effect: non-scaling-stroke; } + +/* ─── Chunking timeline ────────────────────────────────────────────── */ +.chunk-rule, .chunk-row, .chunk-axis { display: flex; gap: 4px; } +.chunk-row { height: clamp(56px, 9vh, 120px); } +.chunk-cell { + flex: 1; border-radius: 3px; + display: flex; align-items: center; justify-content: center; + font-family: ui-monospace, SFMono-Regular, Menlo, monospace; + font-size: clamp(11px, 0.95vw, 14px); color: rgba(255,255,255,0.85); +} +.chunk-cell.clean { background: var(--phase-clean); } +.chunk-cell.armed { background: var(--phase-armed); } +.chunk-cell.infecting { background: var(--phase-infecting); } +.chunk-cell.infected_running { background: var(--phase-running); } +.chunk-cell.dormant { background: var(--phase-dormant); } +.chunk-rule { height: 8px; background: var(--bg-elev); + border: 1px solid var(--line); border-radius: 2px; padding: 1px; } +.chunk-rule .tick { flex: 1; border-right: 1px solid var(--line); } +.chunk-rule .tick:last-child { border-right: none; } +.chunk-axis { height: 16px; font-size: 10px; color: var(--fg-mute); + font-family: ui-monospace, SFMono-Regular, Menlo, monospace; } +.chunk-axis span { flex: 1; text-align: center; } + +/* ─── Model bars ───────────────────────────────────────────────────── */ +.model-bars { display: flex; flex-direction: column; gap: clamp(10px, 1.5vh, 18px); } +.model-row { + display: grid; grid-template-columns: minmax(80px, 12ch) 1fr minmax(64px, 9ch); + gap: clamp(10px, 1vw, 18px); align-items: center; font-variant-numeric: tabular-nums; +} +.model-name { font-family: ui-monospace, SFMono-Regular, Menlo, monospace; + font-size: clamp(13px, 1vw, 15px); color: var(--fg); } +.model-track { height: clamp(28px, 4vh, 48px); background: var(--bg-elev); + border: 1px solid var(--line); border-radius: 3px; overflow: hidden; } +.model-fill { height: 100%; transition: width 600ms cubic-bezier(0.2, 0.8, 0.2, 1); } +.model-fill.lstm { background: linear-gradient(90deg, #58a6ff, #1f6feb); } +.model-fill.gru { background: linear-gradient(90deg, #db61a2, #a8327f); } +.model-fill.rnn { background: linear-gradient(90deg, #d29922, #8a6a17); } +.model-fill.bert { background: linear-gradient(90deg, #f85149, #b22e2a); } +.model-acc { font-family: ui-monospace, SFMono-Regular, Menlo, monospace; + font-size: clamp(13px, 1vw, 15px); color: var(--fg-dim); text-align: right; } + +/* ─── Scatter plots (KNN, perf) ────────────────────────────────────── */ +.scatter { + width: 100%; + height: clamp(320px, 60vh, 640px); +} +.scatter .axis { stroke: var(--line); stroke-width: 1; } +.scatter .axis-label { fill: var(--fg-mute); font-size: 12px; + font-family: ui-monospace, SFMono-Regular, Menlo, monospace; } +.scatter .point { transition: r 200ms; } +.scatter .point.clean { fill: var(--phase-clean); } +.scatter .point.armed { fill: var(--phase-armed); } +.scatter .point.infecting { fill: var(--phase-infecting); } +.scatter .point.infected_running { fill: var(--phase-running); } +.scatter .point.dormant { fill: var(--phase-dormant); } +.scatter .perf-point { fill: var(--accent); stroke: #1f6feb; stroke-width: 1.5; } +.scatter .perf-label { fill: var(--fg); font-size: 13px; + font-family: ui-monospace, SFMono-Regular, Menlo, monospace; } + +/* ─── Floating advance button ──────────────────────────────────────── */ +.fab { + position: absolute; right: 24px; bottom: 24px; z-index: 5; + width: 44px; height: 44px; border-radius: 50%; + background: rgba(13, 17, 23, 0.85); color: var(--fg-dim); + border: 1px solid var(--line); font-size: 16px; cursor: pointer; + backdrop-filter: blur(8px); + display: flex; align-items: center; justify-content: center; + transition: color 150ms, border-color 150ms, transform 150ms; +} +.fab:hover { color: var(--accent); border-color: var(--accent); transform: translateY(-1px); } +.fab:disabled { opacity: 0.25; cursor: not-allowed; transform: none; } + +/* ─── Article (prose, overlaid right) ──────────────────────────────── */ +.scene { + display: flex; align-items: center; justify-content: flex-end; + min-height: 100vh; padding: 4rem 2rem; +} +.scene .prose { + width: var(--prose-w); max-width: calc(100vw - 4rem); + padding: 2.25rem 2.5rem 2.25rem 5rem; + font-size: 17px; line-height: 1.65; color: var(--fg); + background: linear-gradient( + to left, + rgba(var(--bg-rgb), 0.96) 0%, + rgba(var(--bg-rgb), 0.92) 55%, + rgba(var(--bg-rgb), 0.0) 100% + ); + opacity: 0; transform: translateY(20px); + transition: opacity var(--scene-fade-ms) ease, + transform var(--scene-fade-ms) ease; +} +.scene[data-active] .prose { opacity: 1; transform: translateY(0); } +.scene .prose h2 { margin: 0 0 14px; font-size: 24px; font-weight: 600; + letter-spacing: -0.01em; } +.scene .prose .lede { font-size: 24px; line-height: 1.4; font-weight: 500; + margin: 0 0 20px; } +.scene .prose p { margin: 0 0 14px; } +.scene .prose strong { color: #fff; font-weight: 600; } +.scene .prose code { + font-family: ui-monospace, SFMono-Regular, Menlo, monospace; + font-size: 0.9em; color: var(--accent); + background: var(--accent-soft); padding: 1px 5px; border-radius: 3px; +} +.scene .prose .hint { + color: var(--fg-mute); font-size: 12px; + letter-spacing: 0.16em; text-transform: uppercase; margin-top: 28px; +} + +.scene-end-spacer { height: 30vh; } + +@media (max-width: 880px) { + :root { --prose-w: 92vw; } + .stage-view { padding-right: 0; padding-bottom: 50vh; } + .intro-block, .metric-stack { padding: 0 24px; } + .bar-row { grid-template-columns: 110px 1fr 60px; } + .model-row { grid-template-columns: 80px 1fr 56px; } + .profile-grid { grid-template-columns: 1fr; } + .scene { padding: 2rem 1rem; min-height: 80vh; justify-content: center; } + .scene .prose { padding: 1.5rem; } + .topbar .counter { display: none; } + .db-table-wrap { max-height: 40vh; } +} diff --git a/training/dashboard/static/dashboard.js b/training/dashboard/static/dashboard.js new file mode 100644 index 0000000..98265b4 --- /dev/null +++ b/training/dashboard/static/dashboard.js @@ -0,0 +1,821 @@ +// CIS490 dashboard front-end. +// +// Layers: +// 1. Transport + typed message bus +// 2. Scrollytelling controller (hotkeys, click, prev/next, FAB) +// 3. Public API + demo machinery +// 4. Widgets — one per scene +// +// Events on the bus: +// +// Real (from server feeders): +// hello — {type, clients} one-shot on connect +// snapshot — {type, total_episodes, total_alerts, +// total_bytes, host_counts, +// recent_episodes: [...]} every 30 s + on connect +// episode — {type, episode_id, host_id, sha256, +// size_bytes, received_at} one per index.jsonl line +// alert — {type, host_id, symptom, detail, +// suggested_fix, detected_at} one per alerts.jsonl line +// +// Real (from future producers — overwrite synthetic if demo is on): +// phase — {type, phase} +// prediction — {type, episode_id, window_idx, predicted, actual} +// model_metric — {type, model, accuracy} +// embedding — {type, x, y, phase} +// model_perf — {type, model, latency_us, accuracy} +// attack_profile — {type, name, shape, curve: [...]} +// +// Local (browser-only, never hit the wire): +// demo_start — {type} emitted when demo toggles on +// demo_stop — {type} emitted when demo toggles off +// +// Demo-only widgets render NOTHING by default (they show an "awaiting" +// row). They populate synthetic data on demo_start and clear on +// demo_stop. Real producer events overwrite either way. + +(function () { + 'use strict'; + + // ──────────────────────────────────────────────────────────────── + // 1. Transport + bus + // ──────────────────────────────────────────────────────────────── + const statusEl = document.getElementById('status'); + const subscribers = new Map(); + const wildcardSubs = new Set(); + + function on(type, fn) { + if (type === '*') { wildcardSubs.add(fn); return () => wildcardSubs.delete(fn); } + if (!subscribers.has(type)) subscribers.set(type, new Set()); + subscribers.get(type).add(fn); + return () => subscribers.get(type).delete(fn); + } + function dispatch(msg) { + const t = (msg && msg.type) || '__no_type__'; + const subs = subscribers.get(t); + if (subs) subs.forEach(fn => { try { fn(msg); } catch (e) { console.error(e); } }); + wildcardSubs.forEach(fn => { try { fn(msg); } catch (e) { console.error(e); } }); + } + + let ws = null; + function connect() { + const proto = location.protocol === 'https:' ? 'wss:' : 'ws:'; + ws = new WebSocket(`${proto}//${location.host}/ws`); + ws.onopen = () => { statusEl.textContent = 'live'; statusEl.className = 'status ok'; }; + ws.onclose = () => { + statusEl.textContent = 'reconnecting…'; statusEl.className = 'status bad'; + setTimeout(connect, 1500); + }; + ws.onerror = () => {}; + ws.onmessage = ev => { + let msg; try { msg = JSON.parse(ev.data); } catch { msg = { type: 'raw', raw: ev.data }; } + dispatch(msg); + }; + } + connect(); + function send(msg) { if (ws && ws.readyState === WebSocket.OPEN) ws.send(JSON.stringify(msg)); } + + // ──────────────────────────────────────────────────────────────── + // 2. Scrollytelling controller + // ──────────────────────────────────────────────────────────────── + const scenes = Array.from(document.querySelectorAll('.scene[data-stage]')); + const stageViews = new Map(); + document.querySelectorAll('.stage-view[data-view]').forEach(el => { + stageViews.set(el.dataset.view, el); + }); + + const sceneEnterHandlers = new Map(); + const sceneExitHandlers = new Map(); + function onScene(name, { onEnter, onExit } = {}) { + if (onEnter) sceneEnterHandlers.set(name, onEnter); + if (onExit) sceneExitHandlers.set(name, onExit); + } + + const sceneIdxEl = document.getElementById('scene-idx'); + const sceneTotalEl = document.getElementById('scene-total'); + const prevBtn = document.getElementById('prev-btn'); + const nextBtn = document.getElementById('next-btn'); + const fab = document.getElementById('next-fab'); + sceneTotalEl.textContent = String(scenes.length); + + let activeIdx = -1; + function setActiveIdx(idx) { + if (idx === activeIdx) return; + if (activeIdx >= 0) { + const prevName = scenes[activeIdx].dataset.stage; + const view = stageViews.get(prevName); + if (view) view.removeAttribute('data-active'); + const fn = sceneExitHandlers.get(prevName); + if (fn) try { fn(); } catch (e) { console.error(e); } + scenes[activeIdx].removeAttribute('data-active'); + } + activeIdx = idx; + sceneIdxEl.textContent = String(idx + 1); + const name = scenes[idx].dataset.stage; + const view = stageViews.get(name); + if (view) view.setAttribute('data-active', ''); + scenes[idx].setAttribute('data-active', ''); + const fn = sceneEnterHandlers.get(name); + if (fn) try { fn(); } catch (e) { console.error(e); } + prevBtn.disabled = idx === 0; + nextBtn.disabled = idx === scenes.length - 1; + if (fab) fab.disabled = idx === scenes.length - 1; + } + + const sceneRatios = new Map(); + const io = new IntersectionObserver(entries => { + entries.forEach(e => sceneRatios.set(e.target, e.intersectionRatio)); + let bestIdx = activeIdx >= 0 ? activeIdx : 0; + let bestRatio = -1; + scenes.forEach((s, i) => { + const r = sceneRatios.get(s) || 0; + if (r > bestRatio) { bestRatio = r; bestIdx = i; } + }); + setActiveIdx(bestIdx); + }, { + threshold: [0, 0.25, 0.5, 0.75, 1], + rootMargin: '-30% 0px -30% 0px', + }); + scenes.forEach(s => io.observe(s)); + setActiveIdx(0); + + function scrollToScene(idx) { + idx = Math.max(0, Math.min(scenes.length - 1, idx)); + if (idx === activeIdx) return; + scenes[idx].scrollIntoView({ behavior: 'smooth', block: 'start' }); + } + function next() { scrollToScene(activeIdx + 1); } + function prev() { scrollToScene(activeIdx - 1); } + + window.addEventListener('keydown', e => { + if (e.metaKey || e.ctrlKey || e.altKey) return; + const tag = (e.target && e.target.tagName) || ''; + if (tag === 'INPUT' || tag === 'TEXTAREA' || (e.target && e.target.isContentEditable)) return; + switch (e.key) { + case 'ArrowDown': case 'ArrowRight': case 'PageDown': case 'j': case ' ': + e.preventDefault(); next(); break; + case 'ArrowUp': case 'ArrowLeft': case 'PageUp': case 'k': + e.preventDefault(); prev(); break; + case 'Home': e.preventDefault(); scrollToScene(0); break; + case 'End': e.preventDefault(); scrollToScene(scenes.length - 1); break; + case 'c': case 'C': e.preventDefault(); setClickNav(!clickNavOn); break; + } + }); + prevBtn.addEventListener('click', prev); + nextBtn.addEventListener('click', next); + if (fab) fab.addEventListener('click', next); + + // Click-on-stage to advance — gated by the click-nav toggle so + // interactive widgets (db table rows, search boxes) don't compete + // with the next-slide gesture by default. Topbar arrows / FAB / + // hotkeys always work regardless. + const clickNavBtn = document.getElementById('click-nav-btn'); + let clickNavOn = false; + function setClickNav(on) { + clickNavOn = on; + clickNavBtn.textContent = `click-nav: ${on ? 'on' : 'off'}`; + clickNavBtn.classList.toggle('active', on); + } + clickNavBtn.addEventListener('click', e => { e.stopPropagation(); setClickNav(!clickNavOn); }); + setClickNav(false); + + const stageCol = document.getElementById('stage-col'); + stageCol.addEventListener('click', e => { + if (!clickNavOn) return; + if (e.target.closest('[data-no-advance]')) return; + if (e.target.closest('button, a, input, select, textarea')) return; + next(); + }); + + // ──────────────────────────────────────────────────────────────── + // 3. Public API + demo machinery + // ──────────────────────────────────────────────────────────────── + window.dashboard = { on, send, scene: onScene, dispatch, next, prev, scrollToScene }; + + const demoBtn = document.getElementById('demo-btn'); + let demoTimer = null; + let demoActive = false; + const HOSTS = ['elliott-lab', 'elliott-thinkpad', 'k-gamingcom']; + const PHASES = ['clean', 'armed', 'infecting', 'infected_running', 'dormant']; + function demoTick() { + if (Math.random() < 0.7) { + const host = HOSTS[Math.floor(Math.random() * HOSTS.length)]; + dispatch({ type: 'episode', + episode_id: 'demo-' + Math.random().toString(36).slice(2, 12), + host_id: host, size_bytes: 30_000 + Math.random() * 20_000 }); + } + if (Math.random() < 0.5) { + dispatch({ type: 'phase', phase: PHASES[Math.floor(Math.random() * PHASES.length)] }); + } + // Occasionally tweak a model metric so the bars aren't static. + if (Math.random() < 0.05) { + const m = ['rnn', 'gru', 'lstm', 'bert'][Math.floor(Math.random() * 4)]; + const base = { rnn: 0.872, gru: 0.911, lstm: 0.928, bert: 0.954 }[m]; + dispatch({ type: 'model_metric', model: m, accuracy: base + (Math.random() - 0.5) * 0.012 }); + } + } + function setDemo(active) { + if (demoActive === active) return; + demoActive = active; + if (active) { + dispatch({ type: 'demo_start' }); + demoTimer = setInterval(demoTick, 350); + demoBtn.textContent = 'demo: on'; demoBtn.classList.add('active'); + } else { + if (demoTimer) clearInterval(demoTimer); demoTimer = null; + dispatch({ type: 'demo_stop' }); + demoBtn.textContent = 'demo: off'; demoBtn.classList.remove('active'); + } + } + demoBtn.addEventListener('click', e => { e.stopPropagation(); setDemo(!demoActive); }); + + // Helper for empty-state rows. + function awaitingNote(text) { + const d = document.createElement('div'); + d.className = 'awaiting'; d.textContent = text; + return d; + } + + // ──────────────────────────────────────────────────────────────── + // 4. Widgets + // ──────────────────────────────────────────────────────────────── + + // ── Ingest counter + 60-second sparkline ────────────────────── + // Real-data widget: populated by snapshot + episode events. No + // demo gating — when demo is off, it just shows real activity. + (function () { + const totalEl = document.getElementById('ingest-total'); + const rateEl = document.getElementById('ingest-rate'); + const bytesEl = document.getElementById('ingest-bytes'); + const pathEl = document.getElementById('ingest-spark-path'); + const fillEl = document.getElementById('ingest-spark-fill'); + const W = 600, H = 120, BUCKETS = 60; + const buckets = new Array(BUCKETS).fill(0); + let total = 0, totalBytes = 0; + + function bucketIndex() { return Math.floor(Date.now() / 1000) % BUCKETS; } + let lastBucket = bucketIndex(); + + function rotateIfNeeded() { + const cur = bucketIndex(); + while (lastBucket !== cur) { + lastBucket = (lastBucket + 1) % BUCKETS; + buckets[lastBucket] = 0; + } + } + + function fmtBytes(n) { + if (!n) return '0 B'; + const u = ['B', 'KB', 'MB', 'GB', 'TB']; let i = 0; + while (n >= 1024 && i < u.length - 1) { n /= 1024; i++; } + return n.toFixed(n >= 100 || i === 0 ? 0 : 1) + ' ' + u[i]; + } + + function render() { + rotateIfNeeded(); + totalEl.textContent = total.toLocaleString(); + const sum = buckets.reduce((a, b) => a + b, 0); + rateEl.textContent = (sum / BUCKETS).toFixed(1); + if (bytesEl) bytesEl.textContent = fmtBytes(totalBytes); + const max = Math.max(1, ...buckets); + const pts = []; + for (let i = 0; i < BUCKETS; i++) { + const idx = (lastBucket + 1 + i) % BUCKETS; + const x = (i / (BUCKETS - 1)) * W; + const y = H - (buckets[idx] / max) * (H - 8) - 4; + pts.push([x, y]); + } + const d = pts.map(([x, y], i) => `${i === 0 ? 'M' : 'L'}${x.toFixed(1)},${y.toFixed(1)}`).join(' '); + pathEl.setAttribute('d', d); + fillEl.setAttribute('d', `${d} L${W},${H} L0,${H} Z`); + } + + on('snapshot', m => { + if (typeof m.total_episodes === 'number') total = m.total_episodes; + if (typeof m.total_bytes === 'number') totalBytes = m.total_bytes; + render(); + }); + on('episode', m => { + rotateIfNeeded(); + buckets[lastBucket] += 1; + total += 1; + if (typeof m.size_bytes === 'number') totalBytes += m.size_bytes; + render(); + }); + setInterval(render, 1000); + render(); + })(); + + // ── Per-host bars ───────────────────────────────────────────── + // Real-data widget. Snapshot seeds absolute counts; episode events + // increment. + (function () { + const root = document.getElementById('host-bars'); + const counts = new Map(); + const rows = new Map(); + + function clearEmpty() { + const e = root.querySelector('.bars-empty, .awaiting'); + if (e) e.remove(); + } + function ensureRow(host) { + if (rows.has(host)) return rows.get(host); + clearEmpty(); + const row = document.createElement('div'); row.className = 'bar-row'; + const name = document.createElement('div'); name.className = 'bar-host'; name.textContent = host; + const track = document.createElement('div'); track.className = 'bar-track'; + const fill = document.createElement('div'); fill.className = 'bar-fill'; fill.style.width = '0%'; + track.appendChild(fill); + const label = document.createElement('div'); label.className = 'bar-count'; label.textContent = '0'; + row.append(name, track, label); + root.appendChild(row); + const entry = { row, fill, label }; + rows.set(host, entry); return entry; + } + function render() { + const max = Math.max(1, ...counts.values()); + Array.from(counts.keys()).forEach(h => { + const r = ensureRow(h); + const c = counts.get(h); + r.fill.style.width = ((c / max) * 100).toFixed(1) + '%'; + r.label.textContent = c.toLocaleString(); + }); + Array.from(counts.keys()).sort((a, b) => counts.get(b) - counts.get(a)) + .forEach(h => root.appendChild(rows.get(h).row)); + } + + on('snapshot', m => { + if (m.host_counts && typeof m.host_counts === 'object') { + Object.entries(m.host_counts).forEach(([h, c]) => counts.set(h, c)); + render(); + } + }); + on('episode', m => { + if (!m.host_id) return; + counts.set(m.host_id, (counts.get(m.host_id) || 0) + 1); render(); + }); + })(); + + // ── Phase mix (rolling 5 min) ───────────────────────────────── + // Real-data widget. Will be empty until phase events flow. + (function () { + const stack = document.getElementById('phase-stack'); + const legend = document.getElementById('phase-legend'); + const PHASES = ['clean', 'armed', 'infecting', 'infected_running', 'dormant']; + const WINDOW_MS = 5 * 60 * 1000; + const samples = []; + const segs = new Map(); + + PHASES.forEach(p => { + const seg = document.createElement('div'); + seg.className = `phase-seg ${p}`; stack.appendChild(seg); segs.set(p, seg); + const li = document.createElement('span'); + const swatchVar = p === 'infected_running' ? 'running' : p; + li.innerHTML = `${p}`; + legend.appendChild(li); + }); + + function render() { + const now = Date.now(); + while (samples.length && now - samples[0].t > WINDOW_MS) samples.shift(); + const counts = Object.fromEntries(PHASES.map(p => [p, 0])); + samples.forEach(s => { if (counts[s.phase] !== undefined) counts[s.phase]++; }); + const total = Math.max(1, samples.length); + PHASES.forEach(p => { segs.get(p).style.flexGrow = (counts[p] / total).toFixed(4); }); + } + + on('phase', m => { + if (!m.phase) return; + samples.push({ phase: m.phase, t: Date.now() }); render(); + }); + on('demo_stop', () => { samples.length = 0; render(); }); + setInterval(render, 1000); + })(); + + // ── Database explorer ───────────────────────────────────────── + // Real-data widget. Initial population from snapshot.recent_episodes + // (last 200 lines of index.jsonl). New episodes prepend live. + (function () { + const tabsEl = document.getElementById('db-tabs'); + const searchEl = document.getElementById('db-search'); + const tbodyEl = document.getElementById('db-tbody'); + const detailEl = document.getElementById('db-detail'); + const detailPre = document.getElementById('db-detail-pre'); + const countEl = document.getElementById('db-count'); + + let records = []; // newest first + let activeHost = null; // null = all + let query = ''; + + function fmtBytes(n) { + if (!n) return '—'; + const u = ['B', 'KB', 'MB', 'GB']; let i = 0; + while (n >= 1024 && i < u.length - 1) { n /= 1024; i++; } + return n.toFixed(n >= 100 || i === 0 ? 0 : 1) + ' ' + u[i]; + } + function fmtTime(iso) { + if (!iso) return '—'; + try { return new Date(iso).toLocaleTimeString('en-US', { hour12: false }); } + catch { return iso; } + } + function shortId(id) { + if (!id) return '—'; + return id.length > 24 ? id.slice(0, 16) + '…' + id.slice(-6) : id; + } + + function rebuildTabs() { + const hosts = Array.from(new Set(records.map(r => r.host_id).filter(Boolean))).sort(); + tabsEl.innerHTML = ''; + const all = document.createElement('button'); + all.className = 'db-tab' + (activeHost === null ? ' active' : ''); + all.textContent = `all · ${records.length}`; + all.addEventListener('click', e => { e.stopPropagation(); activeHost = null; rebuildTabs(); rebuildTable(); }); + tabsEl.appendChild(all); + hosts.forEach(h => { + const b = document.createElement('button'); + b.className = 'db-tab' + (activeHost === h ? ' active' : ''); + const c = records.filter(r => r.host_id === h).length; + b.textContent = `${h} · ${c}`; + b.addEventListener('click', e => { e.stopPropagation(); activeHost = h; rebuildTabs(); rebuildTable(); }); + tabsEl.appendChild(b); + }); + } + + function matches(rec) { + if (activeHost && rec.host_id !== activeHost) return false; + if (!query) return true; + const q = query.toLowerCase(); + return (rec.host_id || '').toLowerCase().includes(q) + || (rec.episode_id || '').toLowerCase().includes(q) + || (rec.sha256 || '').toLowerCase().includes(q); + } + + function rebuildTable() { + const filtered = records.filter(matches); + countEl.textContent = `${filtered.length} of ${records.length}`; + tbodyEl.innerHTML = ''; + if (!filtered.length) { + const tr = document.createElement('tr'); + const td = document.createElement('td'); + td.colSpan = 4; td.className = 'awaiting'; + td.textContent = records.length === 0 + ? 'awaiting snapshot…' + : 'no rows match the current filter'; + tr.appendChild(td); tbodyEl.appendChild(tr); + return; + } + const frag = document.createDocumentFragment(); + filtered.slice(0, 200).forEach(rec => { + const tr = document.createElement('tr'); + tr.className = 'db-row'; tr.dataset.id = rec.episode_id || ''; + tr.innerHTML = ` + ${rec.host_id || '—'} + ${shortId(rec.episode_id)} + ${fmtTime(rec.received_at)} + ${fmtBytes(rec.size_bytes)}`; + tr.addEventListener('click', e => { + e.stopPropagation(); + detailEl.hidden = false; + detailPre.textContent = JSON.stringify(rec, null, 2); + tbodyEl.querySelectorAll('.db-row').forEach(r => r.classList.remove('selected')); + tr.classList.add('selected'); + }); + frag.appendChild(tr); + }); + tbodyEl.appendChild(frag); + } + + on('snapshot', m => { + if (Array.isArray(m.recent_episodes)) { + records = m.recent_episodes.slice(); + rebuildTabs(); rebuildTable(); + } + }); + on('episode', m => { + // Prepend; cap. + records.unshift({ + episode_id: m.episode_id, host_id: m.host_id, + sha256: m.sha256, size_bytes: m.size_bytes, received_at: m.received_at, + }); + if (records.length > 200) records.length = 200; + // Cheap update: only rebuild if scene visible. + rebuildTabs(); rebuildTable(); + }); + on('demo_stop', () => { + // Drop demo-injected records (their ids start with "demo-"). + const before = records.length; + records = records.filter(r => !(r.episode_id && r.episode_id.startsWith('demo-'))); + if (records.length !== before) { rebuildTabs(); rebuildTable(); } + }); + searchEl.addEventListener('input', e => { query = e.target.value; rebuildTable(); }); + searchEl.addEventListener('click', e => e.stopPropagation()); + rebuildTabs(); rebuildTable(); + })(); + + // ── Attack envelope thumbnails — DEMO ONLY ─────────────────── + (function () { + const root = document.getElementById('profile-grid'); + const W = 200, H = 56; + function emptyState() { + root.innerHTML = ''; + root.appendChild(awaitingNote('awaiting attack_profile events · turn demo on for examples')); + } + function curveToPath(values) { + const max = Math.max(1, ...values); + return values.map((v, i) => { + const x = (i / (values.length - 1)) * W; + const y = H - (v / max) * (H - 6) - 3; + return `${i === 0 ? 'M' : 'L'}${x.toFixed(1)},${y.toFixed(1)}`; + }).join(' '); + } + function gen(seed, fn, n = 80) { + let s = seed; + const rand = () => { s = (s * 1664525 + 1013904223) >>> 0; return (s & 0xffff) / 0xffff; }; + return Array.from({ length: n }, (_, i) => fn(i / n, rand)); + } + const cards = new Map(); + function render(profile) { + let card = cards.get(profile.name); + if (!card) { + if (root.querySelector('.awaiting')) root.innerHTML = ''; + card = document.createElement('div'); card.className = 'profile-card'; + card.innerHTML = ` +
+
+ `; + root.appendChild(card); cards.set(profile.name, card); + } + card.querySelector('.profile-name').textContent = profile.name; + card.querySelector('.profile-shape').textContent = profile.shape || ''; + card.querySelector('path').setAttribute('d', curveToPath(profile.curve)); + } + function clearAll() { cards.clear(); emptyState(); } + function syntheticProfiles() { + return [ + { name: 'cpu-saturate', shape: 'sustained 1-vCPU peg (XMRig)', + curve: gen(1, (t, r) => 0.1 + (t > 0.15 ? 0.85 : 0) + 0.05 * r()) }, + { name: 'scan-and-dial', shape: 'SYN-style probes + dial-home', + curve: gen(2, (t, r) => t < 0.2 ? 0.05 : 0.15 + 0.7 * Math.exp(-Math.pow((t-0.5)*4, 2)) + 0.05 * r()) }, + { name: 'io-walk', shape: 'fs traversal + 4 KiB urandom writes', + curve: gen(3, (t, r) => 0.2 + 0.3 * Math.sin(t * 14) + 0.4 * (t > 0.3 && t < 0.85 ? 1 : 0) + 0.05 * r()) }, + { name: 'bursty-c2', shape: 'long idle + 3-packet egress bursts', + curve: gen(4, (t, r) => 0.05 + (Math.sin(t * 30) > 0.95 ? 0.9 : 0) + 0.02 * r()) }, + { name: 'low-and-slow', shape: 'minimal CPU + periodic memory churn', + curve: gen(5, (t, r) => 0.12 + 0.08 * Math.sin(t * 6) + 0.05 * r()) }, + { name: 'shell-resident', shape: 'one long TCP socket + command ticks', + curve: gen(6, (t, r) => 0.08 + (Math.sin(t * 22) > 0.7 ? 0.5 : 0) + 0.03 * r()) }, + ]; + } + on('demo_start', () => syntheticProfiles().forEach(render)); + on('demo_stop', () => clearAll()); + on('attack_profile', m => { + if (!m.name || !Array.isArray(m.curve)) return; + render({ name: m.name, shape: m.shape || '', curve: m.curve }); + }); + emptyState(); + })(); + + // ── Chunking timeline — DEMO ONLY ──────────────────────────── + (function () { + const ruleEl = document.getElementById('chunk-rule'); + const rowEl = document.getElementById('chunk-row'); + const axisEl = document.getElementById('chunk-axis'); + const N = 6; + function clearAll() { + ruleEl.innerHTML = ''; rowEl.innerHTML = ''; axisEl.innerHTML = ''; + rowEl.appendChild(awaitingNote('awaiting prediction events · turn demo on for examples')); + } + function buildExample() { + const labels = ['clean', 'clean', 'armed', 'infecting', 'infected_running', 'dormant']; + ruleEl.innerHTML = ''; rowEl.innerHTML = ''; axisEl.innerHTML = ''; + for (let i = 0; i < N; i++) ruleEl.appendChild(Object.assign(document.createElement('div'), { className: 'tick' })); + for (let i = 0; i < N; i++) { + const c = document.createElement('div'); + c.className = `chunk-cell ${labels[i]}`; + c.textContent = labels[i].replace('_', ' '); + rowEl.appendChild(c); + } + for (let i = 0; i < N; i++) { + const t = document.createElement('span'); + t.textContent = `${i * 10}s`; + axisEl.appendChild(t); + } + } + on('demo_start', buildExample); + on('demo_stop', clearAll); + on('prediction', m => { + // Real predictions can update individual cells. + if (typeof m.window_idx !== 'number') return; + const cells = rowEl.querySelectorAll('.chunk-cell'); + const cell = cells[m.window_idx]; + if (!cell) return; + const phase = m.predicted || m.actual; + if (!phase) return; + cell.className = `chunk-cell ${phase}`; + cell.textContent = phase.replace('_', ' '); + }); + clearAll(); + })(); + + // ── Model comparison bars — DEMO ONLY (until model_metric arrives) ─ + (function () { + const root = document.getElementById('model-bars'); + const rows = new Map(); + function emptyState() { + root.innerHTML = ''; + root.appendChild(awaitingNote('awaiting model_metric events · turn demo on for examples')); + } + function ensureRow(model) { + if (rows.has(model)) return rows.get(model); + if (root.querySelector('.awaiting')) root.innerHTML = ''; + const row = document.createElement('div'); row.className = 'model-row'; + row.innerHTML = ` +
${model}
+
+
0.000
`; + root.appendChild(row); + const entry = { row, fill: row.querySelector('.model-fill'), acc: row.querySelector('.model-acc') }; + rows.set(model, entry); return entry; + } + function render(model, accuracy) { + const r = ensureRow(model); + const visible = Math.max(0, Math.min(1, (accuracy - 0.5) / 0.5)); + r.fill.style.width = (visible * 100).toFixed(1) + '%'; + r.acc.textContent = accuracy.toFixed(3); + } + on('demo_start', () => { + [ ['rnn', 0.872], ['gru', 0.911], ['lstm', 0.928], ['bert', 0.954] ] + .forEach(([m, a]) => render(m, a)); + }); + on('demo_stop', () => { rows.clear(); emptyState(); }); + on('model_metric', m => { + if (!m.model || typeof m.accuracy !== 'number') return; + render(m.model, m.accuracy); + }); + emptyState(); + })(); + + // ── KNN scatter — DEMO ONLY ─────────────────────────────────── + (function () { + const svg = document.getElementById('knn-scatter'); + const legend = document.getElementById('knn-legend'); + const PHASES = ['clean', 'armed', 'infecting', 'infected_running', 'dormant']; + const phaseCenters = { + clean: [0.18, 0.75], + armed: [0.42, 0.58], + infecting: [0.72, 0.40], + infected_running: [0.85, 0.18], + dormant: [0.30, 0.30], + }; + const ns = 'http://www.w3.org/2000/svg'; + let W = 600, H = 360; + + legend.innerHTML = ''; + PHASES.forEach(p => { + const li = document.createElement('span'); + const swatchVar = p === 'infected_running' ? 'running' : p; + li.innerHTML = `${p}`; + legend.appendChild(li); + }); + + function setupAxes() { + svg.innerHTML = ''; + const ax = document.createElementNS(ns, 'line'); + ax.setAttribute('x1', 40); ax.setAttribute('y1', H - 30); + ax.setAttribute('x2', W - 10); ax.setAttribute('y2', H - 30); + ax.setAttribute('class', 'axis'); svg.appendChild(ax); + const ay = document.createElementNS(ns, 'line'); + ay.setAttribute('x1', 40); ay.setAttribute('y1', 10); + ay.setAttribute('x2', 40); ay.setAttribute('y2', H - 30); + ay.setAttribute('class', 'axis'); svg.appendChild(ay); + const xl = document.createElementNS(ns, 'text'); + xl.setAttribute('x', W / 2); xl.setAttribute('y', H - 8); + xl.setAttribute('class', 'axis-label'); xl.setAttribute('text-anchor', 'middle'); + xl.textContent = 'feature 1'; svg.appendChild(xl); + const yl = document.createElementNS(ns, 'text'); + yl.setAttribute('transform', `translate(12,${H/2}) rotate(-90)`); + yl.setAttribute('class', 'axis-label'); yl.setAttribute('text-anchor', 'middle'); + yl.textContent = 'feature 2'; svg.appendChild(yl); + } + function emptyState() { + svg.innerHTML = ''; + const t = document.createElementNS(ns, 'text'); + t.setAttribute('x', W / 2); t.setAttribute('y', H / 2); + t.setAttribute('text-anchor', 'middle'); + t.setAttribute('class', 'axis-label'); + t.textContent = 'awaiting embedding events · turn demo on for examples'; + svg.appendChild(t); + } + function project(x, y) { + const px = 40 + x * (W - 50); + const py = (H - 30) - y * (H - 40); + return [px, py]; + } + function addPoint(x, y, phase) { + const c = document.createElementNS(ns, 'circle'); + const [px, py] = project(x, y); + c.setAttribute('cx', px.toFixed(1)); c.setAttribute('cy', py.toFixed(1)); + c.setAttribute('r', 4); + c.setAttribute('class', `point ${phase}`); + svg.appendChild(c); + } + function syntheticPoints() { + let seed = 7; + const rand = () => { seed = (seed * 1664525 + 1013904223) >>> 0; return ((seed & 0xffff) / 0xffff) - 0.5; }; + setupAxes(); + PHASES.forEach(p => { + const [cx, cy] = phaseCenters[p]; + for (let i = 0; i < 14; i++) addPoint(cx + rand() * 0.16, cy + rand() * 0.16, p); + }); + } + on('demo_start', syntheticPoints); + on('demo_stop', emptyState); + on('embedding', m => { + if (typeof m.x !== 'number' || typeof m.y !== 'number' || !m.phase) return; + // First real embedding wipes the awaiting note and sets up axes. + if (svg.querySelector('.axis-label') && !svg.querySelector('.point')) setupAxes(); + addPoint(m.x, m.y, m.phase); + }); + emptyState(); + })(); + + // ── Performance scatter — DEMO ONLY ─────────────────────────── + (function () { + const svg = document.getElementById('perf-scatter'); + const ns = 'http://www.w3.org/2000/svg'; + const W = 600, H = 360; + const xmin = 0, xmax = 4000, ymin = 0.7, ymax = 1.0; + const points = new Map(); + + function setupAxes() { + svg.innerHTML = ''; + const ax = document.createElementNS(ns, 'line'); + ax.setAttribute('x1', 50); ax.setAttribute('y1', H - 30); + ax.setAttribute('x2', W - 10); ax.setAttribute('y2', H - 30); + ax.setAttribute('class', 'axis'); svg.appendChild(ax); + const ay = document.createElementNS(ns, 'line'); + ay.setAttribute('x1', 50); ay.setAttribute('y1', 10); + ay.setAttribute('x2', 50); ay.setAttribute('y2', H - 30); + ay.setAttribute('class', 'axis'); svg.appendChild(ay); + const xl = document.createElementNS(ns, 'text'); + xl.setAttribute('x', W / 2); xl.setAttribute('y', H - 8); + xl.setAttribute('class', 'axis-label'); xl.setAttribute('text-anchor', 'middle'); + xl.textContent = 'inference latency (μs / window) →'; svg.appendChild(xl); + const yl = document.createElementNS(ns, 'text'); + yl.setAttribute('transform', `translate(14,${H/2}) rotate(-90)`); + yl.setAttribute('class', 'axis-label'); yl.setAttribute('text-anchor', 'middle'); + yl.textContent = '↑ held-out accuracy'; svg.appendChild(yl); + } + function emptyState() { + points.clear(); svg.innerHTML = ''; + const t = document.createElementNS(ns, 'text'); + t.setAttribute('x', W / 2); t.setAttribute('y', H / 2); + t.setAttribute('text-anchor', 'middle'); t.setAttribute('class', 'axis-label'); + t.textContent = 'awaiting model_perf events · turn demo on for examples'; + svg.appendChild(t); + } + function project(latency, accuracy) { + const x = 50 + Math.min(1, (latency - xmin) / (xmax - xmin)) * (W - 60); + const y = (H - 30) - Math.max(0, Math.min(1, (accuracy - ymin) / (ymax - ymin))) * (H - 40); + return [x, y]; + } + function render(model, latency, accuracy) { + let g = points.get(model); + if (!g) { + if (!points.size) setupAxes(); + g = document.createElementNS(ns, 'g'); + const c = document.createElementNS(ns, 'circle'); + c.setAttribute('r', 7); c.setAttribute('class', 'perf-point'); + g.appendChild(c); + const t = document.createElementNS(ns, 'text'); + t.setAttribute('class', 'perf-label'); + g.appendChild(t); + svg.appendChild(g); + points.set(model, g); + } + const [px, py] = project(latency, accuracy); + g.querySelector('circle').setAttribute('cx', px.toFixed(1)); + g.querySelector('circle').setAttribute('cy', py.toFixed(1)); + const t = g.querySelector('text'); + t.setAttribute('x', (px + 12).toFixed(1)); + t.setAttribute('y', (py + 4).toFixed(1)); + t.textContent = model; + } + on('demo_start', () => { + [ + { model: 'knn', latency_us: 90, accuracy: 0.84 }, + { model: 'rnn', latency_us: 380, accuracy: 0.87 }, + { model: 'gru', latency_us: 520, accuracy: 0.91 }, + { model: 'lstm', latency_us: 700, accuracy: 0.93 }, + { model: 'bert', latency_us: 3200, accuracy: 0.95 }, + ].forEach(p => render(p.model, p.latency_us, p.accuracy)); + }); + on('demo_stop', emptyState); + on('model_perf', m => { + if (!m.model || typeof m.latency_us !== 'number' || typeof m.accuracy !== 'number') return; + render(m.model, m.latency_us, m.accuracy); + }); + emptyState(); + })(); + +})(); diff --git a/training/dashboard/static/index.html b/training/dashboard/static/index.html new file mode 100644 index 0000000..be61302 --- /dev/null +++ b/training/dashboard/static/index.html @@ -0,0 +1,301 @@ + + + + + + CIS490 — live + + + +
+ CIS490 + connecting… + + 1 / 1 + + + + +
+ +
+
+
+ + +
+
+
+
cis490 · live fleet telemetry
+
behavioral
malware
detection
+
+
+ + +
+
+
episodes ingested
+
0
+
+ 0.0 / sec · last 60 s · + total bytes on disk: 0 B +
+ + + + +
+
+ + +
+
+
per-host shipping
+
+
awaiting snapshot…
+
+
+
+ + +
+
+
+
episode database · last 200 records
+
0 of 0
+
+
+
+ +
+
+ + + + + + + + + + +
hostepisode_idreceivedsize
+
+ +
+
+ + +
+
+
phase mix · last 5 min
+
+
+
awaiting phase events from + the orchestrator. A clean fleet sits mostly in + clean; skew toward infecting means + the workload is firing.
+
+
+ + +
+
+
attack envelopes · /proc signature per profile
+
+
+
+ + +
+
+
10-second windows · model input shape
+
+
+
+
each window: 100 samples (10 Hz × 10 s), + labeled by the phase that occupies its center.
+
+
+ + +
+
+
sequence models · accuracy on held-out samples
+
+
+
+ + +
+
+
window features · 2-D projection
+ +
+
+
+ + +
+
+
accuracy vs inference cost
+ +
x: μs / window (lower is better) · + y: held-out accuracy (higher is better).
+
+
+ +
+ +
+ +
+ +
+
+

Most malware doesn't look like malware in a database + — it looks like a process behaving badly.

+

An intrusion detection system spots the bad + behavior; an intrusion prevention system stops it. + Both depend on knowing what bad behavior looks like at the + level of telemetry the device can actually see.

+

This deck is the live face of the dataset we're building to teach + a model that distinction — every panel on the left is a slice of + real data shipping in right now.

+

scroll, click, or → to advance

+
+
+ +
+
+

Collecting the dataset

+

Each lab host on the WireGuard mesh boots a real Alpine VM, runs + a profile-driven workload inside it, and samples + /proc/<qemu_pid> at 10 Hz. Every ~30 seconds + the labeled tarball is shipped to this Pi over mTLS.

+

The counter on the left is the running total, sourced from the + receiver's index.jsonl on disk. The sparkline is the + arrival rate over the last sixty seconds.

+
+
+ +
+
+

A multi-host fleet

+

Running the same orchestrator on multiple hosts gives novel, + non-overlapping data per host — no central coordinator. Each host + pulls a different slice of the manifest, so the dataset grows in + parallel.

+

The numbers below are absolute episode counts on disk, refreshed + from /var/lib/cis490/episodes/<host>/ every + thirty seconds.

+
+
+ +
+
+

The dataset, browsable

+

Every row is one labeled episode tarball stored at + /var/lib/cis490/episodes/<host>/<id>.tar.zst + after the receiver verifies its SHA-256 and writes it through.

+

Filter by host with the tabs, or grep by host / episode id / + sha with the search box. Click a row for the full + index.jsonl record. The view holds the most recent + two hundred records — older history is on disk, indexable + from the receiver.

+
+
+ +
+
+

A baseline of normal

+

Before we can detect a deviation, we have to know what the fleet + looks like when it's healthy. The stacked bar shows the fraction + of the last five minutes of fleet activity that sat in each phase + — a healthy mix has plenty of clean.

+

If the model only ever sees clean, it overfits to + "everything is fine." The phase schedule fixes that by forcing the + workload to walk through every phase on every run.

+
+
+ +
+
+

Linking attack to telemetry

+

The same six profiles run across every host, and each one + produces a different envelope in /proc. A + cryptominer pegs one core for minutes. A bursty C2 channel sits + idle, then exhales three packets. Ransomware walks the + filesystem and saturates I/O.

+

The thumbnails on the left are the canonical envelopes the + model has to learn to recognize — same axes, different shapes. + That shape difference is what makes detection tractable.

+
+
+ +
+
+

Ten-second windows

+

Models eat fixed-size inputs. We chop each episode into + 10-second windows — 100 samples per window at 10 Hz — and + label each window with the phase that occupies its center.

+

Window size is a knob. Too short and the model can't see slow + envelopes (low-and-slow malware, idle C2). Too long and you can't + react fast enough to be a useful prevention signal. Ten seconds + is the starting point we tune around.

+
+
+ +
+
+

Sequence models

+

RNN, GRU, LSTM — recurrent models that read the + window one timestep at a time and carry state forward. Cheap, + mature, easy to interpret.

+

BERT-style transformer — the window becomes a + sequence of "tokens"; attention captures cross-position context + instead of accumulating it through a hidden state. More + parameters, more compute, more room to overfit a small dataset.

+

Same input, same labels, four different inductive biases. The + comparison on the left is the punchline of the whole project.

+
+
+ +
+
+

Nearest-neighbor as a sanity check

+

Before anything fancy: engineer summary features per window + (mean, std, p95, slope, zero-bucket counts per channel) and run + KNN in that feature space.

+

If the phase clusters separate visibly in two dimensions, KNN + already does most of the work and a deep model is only buying + marginal improvement. If they don't separate, you've learned + something about the feature engineering before training a single + epoch.

+
+
+ +
+
+

Accuracy vs complexity

+

Bigger models earn better numbers in the validation set — but + they also need more parameters, more inference time, and more + memory at the edge. The deployed model has to fit on the device + it's protecting.

+

The scatter on the left is the usable trade-off curve: every + point above and to the left of where you currently sit is a + reachable upgrade. The point in the bottom-right is a model + you'd never ship.

+
+
+ +
+
+
+ + + +