training/dashboard: live deck at dashboard.wg, fed by receiver
Starlette + WebSocket dashboard run on the Pi as cis490-dashboard.service
(127.0.0.1:8447, Caddy-fronted at dashboard.wg). Tails
/var/lib/cis490/index.jsonl for episode events, snapshots host counts
every 30s, broadcasts to every connected browser. New connections get a
warm snapshot (recent_episodes, total_bytes, host_counts) so reloads
don't see a cold dashboard.
Frontend is a 10-scene scrollytelling deck following the project
outline: intro, collect, hosts, db explorer, baseline, attacks,
chunking, models, knn, perf. Sticky full-bleed canvas with a
right-aligned prose column (matrix-explorable layout). Hotkeys (arrows,
space, j/k, c, Home/End), prev/next chevrons, FAB, and an opt-in
click-to-advance toggle. Demo toggle drives synthetic data for the
five scenes that have no real producer yet (attack envelopes,
chunking, model bars, knn scatter, perf scatter); when off, those
scenes show "awaiting <event_type> events" rather than fake data.
Producers wire in by POSTing typed JSON to 127.0.0.1:8447/publish
(loopback only; Caddy 404s it externally). Event types the widgets
subscribe to: model_metric {model, accuracy}, embedding {x, y, phase},
model_perf {model, latency_us, accuracy}, prediction {episode_id,
window_idx, predicted, actual}, attack_profile {name, shape, curve},
phase {phase}.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Max Gorog
parent
b29d30a1b2
commit
a8157ed177
11 changed files with 2143 additions and 0 deletions
|
|
@ -3,6 +3,7 @@
|
|||
Templates for system-level files installed by `scripts/install-*.sh`:
|
||||
|
||||
- `cis490-receiver.service` — systemd unit for the receiver
|
||||
- `cis490-dashboard.service` — systemd unit for the dashboard.wg live display
|
||||
- `receiver.toml.example` — config template for the receiver
|
||||
- `cis490-orchestrator.service` (TODO) — systemd unit for the orchestrator
|
||||
- `cis490-shipper.service` (TODO) — systemd unit for the shipper
|
||||
|
|
|
|||
30
etc/cis490-dashboard.service
Normal file
30
etc/cis490-dashboard.service
Normal file
|
|
@ -0,0 +1,30 @@
|
|||
[Unit]
|
||||
Description=CIS490 live dashboard (dashboard.wg)
|
||||
Documentation=https://maxgit.wg/spectral/CIS490
|
||||
After=network-online.target
|
||||
Wants=network-online.target
|
||||
|
||||
[Service]
|
||||
Type=simple
|
||||
User=cis490
|
||||
Group=cis490
|
||||
WorkingDirectory=/opt/cis490
|
||||
ExecStart=/opt/cis490/.venv/bin/python -m training.dashboard --host 127.0.0.1 --port 8447
|
||||
Restart=on-failure
|
||||
RestartSec=5
|
||||
|
||||
# Hardening
|
||||
NoNewPrivileges=true
|
||||
PrivateTmp=true
|
||||
ProtectSystem=strict
|
||||
ProtectHome=true
|
||||
ProtectKernelTunables=true
|
||||
ProtectKernelModules=true
|
||||
ProtectControlGroups=true
|
||||
LockPersonality=true
|
||||
RestrictNamespaces=true
|
||||
RestrictRealtime=true
|
||||
SystemCallArchitectures=native
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
57
training/dashboard/README.md
Normal file
57
training/dashboard/README.md
Normal file
|
|
@ -0,0 +1,57 @@
|
|||
# training/dashboard/
|
||||
|
||||
Live web display served at `https://dashboard.wg`. A Starlette app
|
||||
on `127.0.0.1:8447` behind Caddy; messages from Python are pushed to
|
||||
connected browsers over a WebSocket.
|
||||
|
||||
This is intentionally a **blank slate** — the default page just
|
||||
appends every received JSON message to a scrolling log. Build the
|
||||
real widgets on top of `window.dashboard.onMessage`.
|
||||
|
||||
## Run locally
|
||||
|
||||
```sh
|
||||
uv run python -m training.dashboard
|
||||
# → open http://127.0.0.1:8447
|
||||
```
|
||||
|
||||
## Push live data from Python
|
||||
|
||||
**Same process** (e.g. a notebook driving the page):
|
||||
|
||||
```py
|
||||
from training.dashboard.app import broadcaster
|
||||
await broadcaster.publish({"type": "metric", "name": "loss", "value": 0.42})
|
||||
```
|
||||
|
||||
**Different process** (orchestrator, receiver, ad-hoc shell):
|
||||
|
||||
```sh
|
||||
curl -s http://127.0.0.1:8447/publish \
|
||||
-H 'content-type: application/json' \
|
||||
-d '{"type":"hello","msg":"from cron"}'
|
||||
```
|
||||
|
||||
The `/publish` endpoint is loopback-only (403 otherwise) and is **not**
|
||||
reverse-proxied by Caddy, so it cannot be hit from the WG mesh.
|
||||
|
||||
## Customizing the page
|
||||
|
||||
The default `static/index.html` exposes `window.dashboard`:
|
||||
|
||||
```js
|
||||
window.dashboard.onMessage = (msg) => {
|
||||
if (msg.type === 'metric') updateChart(msg.name, msg.value);
|
||||
};
|
||||
window.dashboard.send({type: 'request-snapshot'}); // browser → server
|
||||
```
|
||||
|
||||
Override `onMessage` to dispatch to your own widgets. The blank-slate
|
||||
log renderer is just there so a fresh deploy is observably alive.
|
||||
|
||||
## Deploying to the Pi
|
||||
|
||||
1. `sudo cp etc/cis490-dashboard.service /etc/systemd/system/`
|
||||
2. `sudo cp training/dashboard/dashboard.caddy /etc/caddy/Caddyfile.d/`
|
||||
3. `sudo systemctl daemon-reload && sudo systemctl enable --now cis490-dashboard.service`
|
||||
4. `sudo systemctl reload caddy`
|
||||
12
training/dashboard/__init__.py
Normal file
12
training/dashboard/__init__.py
Normal file
|
|
@ -0,0 +1,12 @@
|
|||
"""Live dashboard served at dashboard.wg.
|
||||
|
||||
Blank-slate web display whose live data is fed from Python over a
|
||||
WebSocket. Other code in the repo can push events to connected
|
||||
browsers two ways:
|
||||
|
||||
- In-process: ``from training.dashboard.app import broadcaster``
|
||||
then ``await broadcaster.publish({...})``.
|
||||
- Cross-process (e.g. orchestrator/receiver): POST a JSON body to
|
||||
``http://127.0.0.1:8447/publish``. The endpoint is bound to
|
||||
loopback and is not reverse-proxied by Caddy.
|
||||
"""
|
||||
46
training/dashboard/__main__.py
Normal file
46
training/dashboard/__main__.py
Normal file
|
|
@ -0,0 +1,46 @@
|
|||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import logging
|
||||
import os
|
||||
from pathlib import Path
|
||||
|
||||
import uvicorn
|
||||
|
||||
from .app import make_app
|
||||
|
||||
|
||||
def main() -> None:
|
||||
parser = argparse.ArgumentParser(prog="cis490-dashboard")
|
||||
parser.add_argument(
|
||||
"--host",
|
||||
default=os.environ.get("CIS490_DASHBOARD_HOST", "127.0.0.1"),
|
||||
help="bind address (default 127.0.0.1; Caddy reverse-proxies dashboard.wg here)",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--port", type=int,
|
||||
default=int(os.environ.get("CIS490_DASHBOARD_PORT", "8447")),
|
||||
help="bind port (default 8447)",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--data-root", type=Path,
|
||||
default=Path(os.environ.get("CIS490_DATA_ROOT", "/var/lib/cis490")),
|
||||
help="receiver data root that the feeders read from",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--no-feeders", action="store_true",
|
||||
help="don't start the on-disk feeders (useful for local dev)",
|
||||
)
|
||||
args = parser.parse_args()
|
||||
|
||||
logging.basicConfig(
|
||||
level=logging.INFO,
|
||||
format="%(asctime)s %(levelname)s %(name)s %(message)s",
|
||||
)
|
||||
|
||||
app = make_app(enable_feeders=not args.no_feeders, data_root=args.data_root)
|
||||
uvicorn.run(app, host=args.host, port=args.port, log_config=None)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
143
training/dashboard/app.py
Normal file
143
training/dashboard/app.py
Normal file
|
|
@ -0,0 +1,143 @@
|
|||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
import logging
|
||||
from contextlib import asynccontextmanager
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
from starlette.applications import Starlette
|
||||
from starlette.requests import Request
|
||||
from starlette.responses import FileResponse, JSONResponse, Response
|
||||
from starlette.routing import Mount, Route, WebSocketRoute
|
||||
from starlette.staticfiles import StaticFiles
|
||||
from starlette.websockets import WebSocket, WebSocketDisconnect
|
||||
|
||||
|
||||
log = logging.getLogger("cis490.dashboard")
|
||||
|
||||
STATIC_DIR = Path(__file__).parent / "static"
|
||||
|
||||
|
||||
class Broadcaster:
|
||||
"""Tiny fan-out hub. Per-client async queues, oldest-message-drop
|
||||
on backpressure. Also holds a ``state`` dict that is sent to every
|
||||
new client on connect so reconnects don't see a cold dashboard."""
|
||||
|
||||
QUEUE_MAX = 64
|
||||
|
||||
def __init__(self) -> None:
|
||||
self._clients: set[asyncio.Queue[Any]] = set()
|
||||
self._lock = asyncio.Lock()
|
||||
self.state: dict[str, Any] = {}
|
||||
|
||||
async def register(self) -> asyncio.Queue[Any]:
|
||||
q: asyncio.Queue[Any] = asyncio.Queue(maxsize=self.QUEUE_MAX)
|
||||
async with self._lock:
|
||||
self._clients.add(q)
|
||||
return q
|
||||
|
||||
async def unregister(self, q: asyncio.Queue[Any]) -> None:
|
||||
async with self._lock:
|
||||
self._clients.discard(q)
|
||||
|
||||
async def publish(self, msg: Any) -> int:
|
||||
delivered = 0
|
||||
async with self._lock:
|
||||
clients = list(self._clients)
|
||||
for q in clients:
|
||||
try:
|
||||
q.put_nowait(msg)
|
||||
delivered += 1
|
||||
except asyncio.QueueFull:
|
||||
try: q.get_nowait()
|
||||
except asyncio.QueueEmpty: pass
|
||||
try:
|
||||
q.put_nowait(msg)
|
||||
delivered += 1
|
||||
except asyncio.QueueFull:
|
||||
pass
|
||||
return delivered
|
||||
|
||||
@property
|
||||
def client_count(self) -> int:
|
||||
return len(self._clients)
|
||||
|
||||
|
||||
broadcaster = Broadcaster()
|
||||
|
||||
|
||||
def make_app(
|
||||
*,
|
||||
allow_publish_from: set[str] | None = None,
|
||||
enable_feeders: bool = True,
|
||||
data_root: Path = Path("/var/lib/cis490"),
|
||||
) -> Starlette:
|
||||
allowed = allow_publish_from if allow_publish_from is not None else {"127.0.0.1", "::1"}
|
||||
|
||||
@asynccontextmanager
|
||||
async def lifespan(app):
|
||||
tasks: list[asyncio.Task] = []
|
||||
if enable_feeders:
|
||||
from .feeder import start_feeders
|
||||
try:
|
||||
tasks = start_feeders(broadcaster, data_root=data_root)
|
||||
log.info("started %d feeders rooted at %s", len(tasks), data_root)
|
||||
except Exception:
|
||||
log.exception("failed to start feeders; dashboard will run without them")
|
||||
try:
|
||||
yield
|
||||
finally:
|
||||
for t in tasks: t.cancel()
|
||||
if tasks:
|
||||
await asyncio.gather(*tasks, return_exceptions=True)
|
||||
log.info("dashboard lifespan exiting; %d feeders cancelled", len(tasks))
|
||||
|
||||
async def index(request: Request) -> Response:
|
||||
return FileResponse(STATIC_DIR / "index.html")
|
||||
|
||||
async def healthz(request: Request) -> JSONResponse:
|
||||
return JSONResponse({
|
||||
"status": "ok",
|
||||
"clients": broadcaster.client_count,
|
||||
"state_keys": sorted(broadcaster.state.keys()),
|
||||
})
|
||||
|
||||
async def publish(request: Request) -> JSONResponse:
|
||||
client_host = request.client.host if request.client else ""
|
||||
if client_host not in allowed:
|
||||
return JSONResponse({"error": "forbidden"}, status_code=403)
|
||||
try:
|
||||
body = await request.json()
|
||||
except (ValueError, json.JSONDecodeError):
|
||||
return JSONResponse({"error": "body must be JSON"}, status_code=400)
|
||||
delivered = await broadcaster.publish(body)
|
||||
return JSONResponse({"delivered": delivered})
|
||||
|
||||
async def ws_endpoint(ws: WebSocket) -> None:
|
||||
await ws.accept()
|
||||
q = await broadcaster.register()
|
||||
try:
|
||||
await ws.send_json({"type": "hello", "clients": broadcaster.client_count})
|
||||
# Send the current snapshot so reconnects start warm.
|
||||
if broadcaster.state:
|
||||
await ws.send_json({"type": "snapshot", **broadcaster.state})
|
||||
while True:
|
||||
msg = await q.get()
|
||||
await ws.send_json(msg)
|
||||
except WebSocketDisconnect:
|
||||
pass
|
||||
except Exception:
|
||||
log.exception("ws send failed")
|
||||
finally:
|
||||
await broadcaster.unregister(q)
|
||||
|
||||
routes = [
|
||||
Route("/", index, methods=["GET"]),
|
||||
Route("/healthz", healthz, methods=["GET"]),
|
||||
Route("/publish", publish, methods=["POST"]),
|
||||
WebSocketRoute("/ws", ws_endpoint),
|
||||
Mount("/static", app=StaticFiles(directory=str(STATIC_DIR)), name="static"),
|
||||
]
|
||||
return Starlette(routes=routes, lifespan=lifespan)
|
||||
23
training/dashboard/dashboard.caddy
Normal file
23
training/dashboard/dashboard.caddy
Normal file
|
|
@ -0,0 +1,23 @@
|
|||
# CIS490 live dashboard — blank-slate web display fed by Python over
|
||||
# a WebSocket. Reachable as https://dashboard.wg from any peer on the
|
||||
# WG mesh.
|
||||
#
|
||||
# Trust boundary: "reached :443 over WG", same as bootstrap.wg —
|
||||
# iptmonads has already gated WG peers, so no client_auth here.
|
||||
#
|
||||
# /publish is the producer ingest endpoint. The dashboard service
|
||||
# binds 127.0.0.1:8447 and enforces a loopback allow-list at the app
|
||||
# layer, but everything Caddy proxies arrives at the upstream FROM
|
||||
# 127.0.0.1 (Caddy itself), so the app-layer check would be bypassed
|
||||
# for WG callers. We explicitly 404 it here so producers MUST POST
|
||||
# directly to 127.0.0.1:8447 from the Pi.
|
||||
#
|
||||
# Upstream: cis490-dashboard.service on 127.0.0.1:8447.
|
||||
dashboard.wg {
|
||||
tls internal
|
||||
|
||||
@publish path /publish /publish/*
|
||||
respond @publish "Not Found" 404
|
||||
|
||||
reverse_proxy 127.0.0.1:8447
|
||||
}
|
||||
292
training/dashboard/feeder.py
Normal file
292
training/dashboard/feeder.py
Normal file
|
|
@ -0,0 +1,292 @@
|
|||
"""Real producers that wire the receiver's on-disk state to the
|
||||
dashboard message bus.
|
||||
|
||||
Three feeders, all started by ``app.lifespan``:
|
||||
|
||||
- ``watch_index_jsonl`` — tails ``/var/lib/cis490/index.jsonl`` and
|
||||
publishes one ``episode`` event per new line. Survives file
|
||||
rotation by tracking inode.
|
||||
- ``snapshot_loop`` — periodically derives ground-truth from disk
|
||||
(per-host episode counts, total counts, alert tail) and updates
|
||||
the broadcaster's persistent ``state`` so reconnecting clients
|
||||
see warm numbers, not zero.
|
||||
- ``watch_alerts_jsonl`` — same as the index tailer but for the
|
||||
receiver's alerts log.
|
||||
|
||||
If a path doesn't exist (e.g. ``health/`` on a fresh deploy) the
|
||||
feeder logs once and keeps polling — it'll start producing events the
|
||||
moment the path appears.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
from pathlib import Path
|
||||
from typing import Any, Awaitable, Callable
|
||||
|
||||
|
||||
log = logging.getLogger("cis490.dashboard.feeder")
|
||||
|
||||
DEFAULT_DATA_ROOT = Path("/var/lib/cis490")
|
||||
|
||||
PublishFn = Callable[[dict[str, Any]], Awaitable[int]]
|
||||
|
||||
|
||||
# ─────────────────────────────────────────────────────────────────────
|
||||
# Tail helpers
|
||||
# ─────────────────────────────────────────────────────────────────────
|
||||
|
||||
async def _tail_jsonl(
|
||||
path: Path,
|
||||
publish: PublishFn,
|
||||
parse: Callable[[dict], dict | None],
|
||||
*,
|
||||
poll_interval: float = 1.0,
|
||||
label: str = "tail",
|
||||
) -> None:
|
||||
"""Generic append-only JSONL tailer. ``parse`` shapes each record
|
||||
into the dict we publish (return None to skip)."""
|
||||
fd = None
|
||||
inode: int | None = None
|
||||
missing_logged = False
|
||||
while True:
|
||||
try:
|
||||
if not path.exists():
|
||||
if not missing_logged:
|
||||
log.info("[%s] %s does not exist; will retry", label, path)
|
||||
missing_logged = True
|
||||
await asyncio.sleep(poll_interval * 5)
|
||||
continue
|
||||
missing_logged = False
|
||||
|
||||
st = path.stat()
|
||||
if fd is None or inode != st.st_ino:
|
||||
if fd is not None:
|
||||
try: fd.close()
|
||||
except Exception: pass
|
||||
fd = path.open("r", encoding="utf-8", errors="replace")
|
||||
inode = st.st_ino
|
||||
fd.seek(0, os.SEEK_END)
|
||||
log.info("[%s] watching %s (inode=%d, starting at %d bytes)",
|
||||
label, path, inode, fd.tell())
|
||||
|
||||
chunk = await asyncio.to_thread(fd.read)
|
||||
if chunk:
|
||||
for line in chunk.splitlines():
|
||||
line = line.strip()
|
||||
if not line:
|
||||
continue
|
||||
try:
|
||||
rec = json.loads(line)
|
||||
except json.JSONDecodeError:
|
||||
log.warning("[%s] skipping malformed line: %r", label, line[:120])
|
||||
continue
|
||||
out = parse(rec)
|
||||
if out is not None:
|
||||
await publish(out)
|
||||
await asyncio.sleep(poll_interval)
|
||||
except asyncio.CancelledError:
|
||||
raise
|
||||
except Exception:
|
||||
log.exception("[%s] error; reopening in 5s", label)
|
||||
if fd is not None:
|
||||
try: fd.close()
|
||||
except Exception: pass
|
||||
fd = None
|
||||
inode = None
|
||||
await asyncio.sleep(5)
|
||||
|
||||
|
||||
# ─────────────────────────────────────────────────────────────────────
|
||||
# Specific feeders
|
||||
# ─────────────────────────────────────────────────────────────────────
|
||||
|
||||
async def watch_index_jsonl(broadcaster, path: Path) -> None:
|
||||
"""Episode ingest log. One ``episode`` event per new line, plus
|
||||
we maintain ``broadcaster.state["recent_episodes"]`` in lockstep
|
||||
so reconnecting clients see warm history without re-reading
|
||||
index.jsonl from disk."""
|
||||
def parse(rec: dict) -> dict | None:
|
||||
if "episode_id" not in rec or "host_id" not in rec:
|
||||
return None
|
||||
ep = {
|
||||
"episode_id": rec.get("episode_id"),
|
||||
"host_id": rec.get("host_id"),
|
||||
"sha256": rec.get("sha256"),
|
||||
"size_bytes": rec.get("size_bytes"),
|
||||
"received_at": rec.get("received_at_wall"),
|
||||
}
|
||||
# Keep the live ring buffer + running totals in sync.
|
||||
recent = broadcaster.state.setdefault("recent_episodes", [])
|
||||
recent.insert(0, ep)
|
||||
if len(recent) > RECENT_EPISODES_LIMIT:
|
||||
del recent[RECENT_EPISODES_LIMIT:]
|
||||
sb = ep["size_bytes"]
|
||||
if isinstance(sb, (int, float)):
|
||||
broadcaster.state["total_bytes"] = (
|
||||
int(broadcaster.state.get("total_bytes", 0)) + int(sb)
|
||||
)
|
||||
broadcaster.state["total_episodes"] = (
|
||||
int(broadcaster.state.get("total_episodes", 0)) + 1
|
||||
)
|
||||
return {"type": "episode", **ep}
|
||||
await _tail_jsonl(path, broadcaster.publish, parse, label="index.jsonl")
|
||||
|
||||
|
||||
async def watch_alerts_jsonl(publish: PublishFn, path: Path) -> None:
|
||||
"""Operator-facing alerts (sick hosts, stuck shippers, etc.)."""
|
||||
def parse(rec: dict) -> dict | None:
|
||||
return {
|
||||
"type": "alert",
|
||||
"host_id": rec.get("host"),
|
||||
"symptom": rec.get("symptom"),
|
||||
"detail": rec.get("detail"),
|
||||
"suggested_fix": rec.get("suggested_fix"),
|
||||
"detected_at": rec.get("detected_at_wall"),
|
||||
"dedup_key": rec.get("dedup_key"),
|
||||
}
|
||||
await _tail_jsonl(path, publish, parse, label="alerts.jsonl")
|
||||
|
||||
|
||||
def _count_lines(p: Path) -> int:
|
||||
"""Cheap line count — used for bootstrap totals. We accept 'one
|
||||
extra line at the moment we read mid-write' as acceptable noise."""
|
||||
try:
|
||||
with p.open("rb") as f:
|
||||
return sum(1 for _ in f)
|
||||
except OSError:
|
||||
return 0
|
||||
|
||||
|
||||
RECENT_EPISODES_LIMIT = 200
|
||||
|
||||
|
||||
def _snapshot_state(
|
||||
data_root: Path,
|
||||
index_path: Path,
|
||||
alerts_path: Path,
|
||||
*,
|
||||
recent_limit: int = RECENT_EPISODES_LIMIT,
|
||||
) -> dict:
|
||||
"""Derive the canonical view from disk in one pass.
|
||||
|
||||
Reads index.jsonl front-to-back to collect ``total_episodes``,
|
||||
``total_bytes`` and the trailing ``recent_episodes`` window for
|
||||
the database-explorer widget. ~76k lines / 23 MiB takes ~1 s on
|
||||
the Pi; this runs in ``to_thread`` so the event loop is unaffected.
|
||||
Counts per host come from the filesystem listing — cheaper and
|
||||
immune to JSON parse hiccups."""
|
||||
host_counts: dict[str, int] = {}
|
||||
episodes_root = data_root / "episodes"
|
||||
if episodes_root.exists():
|
||||
for host_dir in episodes_root.iterdir():
|
||||
if not host_dir.is_dir():
|
||||
continue
|
||||
try:
|
||||
host_counts[host_dir.name] = sum(
|
||||
1 for entry in host_dir.iterdir() if entry.is_file()
|
||||
)
|
||||
except OSError:
|
||||
continue
|
||||
|
||||
total_episodes = 0
|
||||
total_bytes = 0
|
||||
recent: list[dict] = []
|
||||
if index_path.exists():
|
||||
try:
|
||||
with index_path.open("r", encoding="utf-8", errors="replace") as f:
|
||||
for line in f:
|
||||
line = line.strip()
|
||||
if not line:
|
||||
continue
|
||||
try:
|
||||
rec = json.loads(line)
|
||||
except json.JSONDecodeError:
|
||||
continue
|
||||
total_episodes += 1
|
||||
sb = rec.get("size_bytes")
|
||||
if isinstance(sb, (int, float)):
|
||||
total_bytes += int(sb)
|
||||
recent.append({
|
||||
"episode_id": rec.get("episode_id"),
|
||||
"host_id": rec.get("host_id"),
|
||||
"received_at": rec.get("received_at_wall"),
|
||||
"size_bytes": sb,
|
||||
"sha256": rec.get("sha256"),
|
||||
})
|
||||
# Periodic truncation keeps the buffer from growing
|
||||
# unboundedly while we read all 76k lines.
|
||||
if len(recent) > recent_limit * 4:
|
||||
recent = recent[-recent_limit:]
|
||||
except OSError:
|
||||
pass
|
||||
recent = recent[-recent_limit:]
|
||||
recent.reverse() # newest-first for the UI
|
||||
|
||||
return {
|
||||
"total_episodes": total_episodes,
|
||||
"total_alerts": _count_lines(alerts_path),
|
||||
"host_counts": host_counts,
|
||||
"total_bytes": total_bytes,
|
||||
"recent_episodes": recent,
|
||||
}
|
||||
|
||||
|
||||
async def snapshot_loop(
|
||||
broadcaster,
|
||||
*,
|
||||
data_root: Path,
|
||||
index_path: Path,
|
||||
alerts_path: Path,
|
||||
poll_interval: float = 30.0,
|
||||
) -> None:
|
||||
"""Refresh the broadcaster's persistent ``state`` periodically so
|
||||
reconnecting clients see disk-truth, not just the in-session
|
||||
delta. Also publishes a ``snapshot`` event so already-connected
|
||||
widgets that want a hard reset can re-key on it."""
|
||||
first = True
|
||||
while True:
|
||||
try:
|
||||
snap = await asyncio.to_thread(
|
||||
_snapshot_state, data_root, index_path, alerts_path
|
||||
)
|
||||
broadcaster.state = snap
|
||||
if first:
|
||||
log.info(
|
||||
"snapshot: total_episodes=%d total_alerts=%d hosts=%d",
|
||||
snap["total_episodes"], snap["total_alerts"], len(snap["host_counts"]),
|
||||
)
|
||||
first = False
|
||||
await broadcaster.publish({"type": "snapshot", **snap})
|
||||
except asyncio.CancelledError:
|
||||
raise
|
||||
except Exception:
|
||||
log.exception("snapshot_loop error")
|
||||
await asyncio.sleep(poll_interval)
|
||||
|
||||
|
||||
# ─────────────────────────────────────────────────────────────────────
|
||||
# Lifecycle
|
||||
# ─────────────────────────────────────────────────────────────────────
|
||||
|
||||
def start_feeders(broadcaster, *, data_root: Path = DEFAULT_DATA_ROOT) -> list[asyncio.Task]:
|
||||
"""Kick off all feeder tasks. Caller is responsible for cancelling
|
||||
them on shutdown (lifespan context handles that)."""
|
||||
index_path = data_root / "index.jsonl"
|
||||
alerts_path = data_root / "alerts.jsonl"
|
||||
publish = broadcaster.publish
|
||||
tasks = [
|
||||
asyncio.create_task(
|
||||
snapshot_loop(broadcaster, data_root=data_root,
|
||||
index_path=index_path, alerts_path=alerts_path),
|
||||
name="cis490.feeder.snapshot"),
|
||||
asyncio.create_task(
|
||||
watch_index_jsonl(broadcaster, index_path),
|
||||
name="cis490.feeder.index"),
|
||||
asyncio.create_task(
|
||||
watch_alerts_jsonl(publish, alerts_path),
|
||||
name="cis490.feeder.alerts"),
|
||||
]
|
||||
return tasks
|
||||
417
training/dashboard/static/dashboard.css
Normal file
417
training/dashboard/static/dashboard.css
Normal file
|
|
@ -0,0 +1,417 @@
|
|||
:root {
|
||||
color-scheme: dark;
|
||||
--bg: #07090d;
|
||||
--bg-rgb: 7, 9, 13;
|
||||
--bg-elev: #0d1117;
|
||||
--bg-elev2: #161b22;
|
||||
--fg: #e6edf3;
|
||||
--fg-dim: #8b949e;
|
||||
--fg-mute: #484f58;
|
||||
--line: #1c2128;
|
||||
--line-soft: #21262d;
|
||||
--accent: #58a6ff;
|
||||
--accent-soft: rgba(88, 166, 255, 0.15);
|
||||
--warn: #f85149;
|
||||
--ok: #3fb950;
|
||||
--phase-clean: #3fb950;
|
||||
--phase-armed: #d29922;
|
||||
--phase-infecting: #db61a2;
|
||||
--phase-running: #f85149;
|
||||
--phase-dormant: #6e7681;
|
||||
--topbar-h: 44px;
|
||||
--prose-w: 36em;
|
||||
--scene-fade-ms: 600ms;
|
||||
}
|
||||
|
||||
* { box-sizing: border-box; }
|
||||
html, body {
|
||||
margin: 0; padding: 0; background: var(--bg); color: var(--fg);
|
||||
font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", system-ui, sans-serif;
|
||||
-webkit-font-smoothing: antialiased; overflow-x: hidden;
|
||||
}
|
||||
|
||||
/* ─── Topbar ───────────────────────────────────────────────────────── */
|
||||
.topbar {
|
||||
position: fixed; top: 0; left: 0; right: 0; height: var(--topbar-h); z-index: 50;
|
||||
display: flex; align-items: center; gap: 10px; padding: 0 16px;
|
||||
background: rgba(7, 9, 13, 0.85); backdrop-filter: blur(8px);
|
||||
border-bottom: 1px solid var(--line); font-size: 12px;
|
||||
}
|
||||
.topbar .brand { font-weight: 700; letter-spacing: 0.04em; }
|
||||
.topbar .spacer { flex: 1; }
|
||||
.topbar .status { color: var(--fg-dim); }
|
||||
.topbar .status.ok { color: var(--ok); }
|
||||
.topbar .status.bad { color: var(--warn); }
|
||||
.topbar .counter {
|
||||
font-family: ui-monospace, SFMono-Regular, Menlo, monospace;
|
||||
color: var(--fg-dim); font-variant-numeric: tabular-nums; padding: 0 6px;
|
||||
}
|
||||
.topbar button.ghost {
|
||||
background: transparent; color: var(--fg-dim); border: 1px solid var(--line);
|
||||
font: inherit; padding: 4px 10px; border-radius: 4px; cursor: pointer;
|
||||
transition: color 120ms, border-color 120ms;
|
||||
}
|
||||
.topbar button.ghost:hover { color: var(--fg); border-color: var(--fg-mute); }
|
||||
.topbar button.ghost:disabled { opacity: 0.35; cursor: not-allowed; }
|
||||
.topbar button.ghost.icon { padding: 4px 8px; min-width: 28px; }
|
||||
.topbar button.ghost.active { color: var(--accent); border-color: var(--accent); }
|
||||
|
||||
/* ─── Layout ───────────────────────────────────────────────────────── */
|
||||
.layout { position: relative; }
|
||||
.canvas-wrapper {
|
||||
position: fixed;
|
||||
top: var(--topbar-h); left: 0; right: 0;
|
||||
height: calc(100vh - var(--topbar-h));
|
||||
z-index: 1; overflow: hidden;
|
||||
}
|
||||
.article {
|
||||
position: relative; z-index: 2;
|
||||
padding-top: var(--topbar-h); pointer-events: none;
|
||||
}
|
||||
.article .prose { pointer-events: auto; }
|
||||
|
||||
/* ─── Stage views ──────────────────────────────────────────────────── */
|
||||
.stage { position: absolute; inset: 0; cursor: pointer; }
|
||||
.stage-view {
|
||||
position: absolute; inset: 0;
|
||||
display: flex; align-items: center; justify-content: flex-start;
|
||||
/* Reserve right-side space for prose. The clamp keeps it sane on
|
||||
narrow viewports and unbounded ones. */
|
||||
padding-right: clamp(0px, calc(var(--prose-w) - 4em), 38em);
|
||||
opacity: 0; transition: opacity var(--scene-fade-ms) ease;
|
||||
pointer-events: none;
|
||||
}
|
||||
.stage-view[data-active] { opacity: 1; pointer-events: auto; }
|
||||
|
||||
/* Intro stage */
|
||||
.bg-grid {
|
||||
position: absolute; inset: 0;
|
||||
background-image:
|
||||
linear-gradient(rgba(88,166,255,0.07) 1px, transparent 1px),
|
||||
linear-gradient(90deg, rgba(88,166,255,0.07) 1px, transparent 1px);
|
||||
background-size: 48px 48px;
|
||||
mask-image: radial-gradient(ellipse at center, #000 0%, transparent 75%);
|
||||
animation: drift 18s linear infinite;
|
||||
}
|
||||
@keyframes drift {
|
||||
from { background-position: 0 0, 0 0; }
|
||||
to { background-position: 48px 48px, 48px 48px; }
|
||||
}
|
||||
.intro-block {
|
||||
position: relative; z-index: 1; text-align: left;
|
||||
padding: 0 clamp(32px, 5vw, 80px); width: 100%;
|
||||
}
|
||||
.intro-eyebrow {
|
||||
font-size: 12px; letter-spacing: 0.18em; text-transform: uppercase;
|
||||
color: var(--fg-dim); margin-bottom: 18px;
|
||||
}
|
||||
.intro-title {
|
||||
font-size: clamp(56px, 9vw, 168px); line-height: 0.95; font-weight: 700;
|
||||
letter-spacing: -0.04em;
|
||||
background: linear-gradient(180deg, #fff 0%, #8b949e 100%);
|
||||
-webkit-background-clip: text; background-clip: text; color: transparent;
|
||||
}
|
||||
|
||||
/* ─── Metric stack — calculated, viewport-relative sizing ──────────── */
|
||||
/* Wide-by-default. Inside a stage-view the right padding already
|
||||
reserves room for prose, so width:100% means "use everything left
|
||||
of the prose column." */
|
||||
.metric-stack {
|
||||
text-align: left;
|
||||
padding: 0 clamp(40px, 5vw, 88px);
|
||||
width: 100%; max-width: none;
|
||||
display: flex; flex-direction: column;
|
||||
gap: clamp(10px, 1.4vh, 22px);
|
||||
}
|
||||
.metric-stack-wide {
|
||||
/* Slightly less right padding than other stages so the table can
|
||||
stretch into the gradient zone of the prose column. */
|
||||
padding-right: clamp(24px, 3vw, 56px);
|
||||
}
|
||||
.metric-eyebrow {
|
||||
font-size: clamp(11px, 1vw, 14px);
|
||||
letter-spacing: 0.18em; text-transform: uppercase; color: var(--fg-dim);
|
||||
}
|
||||
.metric-big {
|
||||
font-size: clamp(72px, 13vw, 240px);
|
||||
line-height: 0.95; font-weight: 700;
|
||||
letter-spacing: -0.04em; font-variant-numeric: tabular-nums;
|
||||
font-family: ui-monospace, SFMono-Regular, Menlo, monospace;
|
||||
}
|
||||
.metric-sub { color: var(--fg-dim); font-size: clamp(13px, 1vw, 16px);
|
||||
line-height: 1.55; font-variant-numeric: tabular-nums; }
|
||||
.metric-sub code {
|
||||
font-family: ui-monospace, SFMono-Regular, Menlo, monospace;
|
||||
font-size: 0.9em; color: var(--accent);
|
||||
background: var(--accent-soft); padding: 1px 5px; border-radius: 3px;
|
||||
}
|
||||
|
||||
.awaiting {
|
||||
color: var(--fg-mute); font-size: clamp(12px, 0.95vw, 14px);
|
||||
font-family: ui-monospace, SFMono-Regular, Menlo, monospace;
|
||||
padding: 16px 0; font-style: italic;
|
||||
}
|
||||
|
||||
/* Sparkline */
|
||||
.sparkline {
|
||||
width: 100%; height: clamp(140px, 28vh, 360px); margin-top: 6px;
|
||||
}
|
||||
.sparkline path { fill: none; stroke: var(--accent); stroke-width: 1.5;
|
||||
vector-effect: non-scaling-stroke; }
|
||||
.sparkline #ingest-spark-fill { fill: var(--accent-soft); stroke: none; }
|
||||
|
||||
/* Per-host bars */
|
||||
.bars { display: flex; flex-direction: column; gap: clamp(8px, 1.1vh, 14px); }
|
||||
.bar-row {
|
||||
display: grid;
|
||||
grid-template-columns: minmax(140px, 18ch) 1fr minmax(72px, 10ch);
|
||||
gap: clamp(10px, 1vw, 18px); align-items: center;
|
||||
font-variant-numeric: tabular-nums;
|
||||
}
|
||||
.bar-host { color: var(--fg); font-size: clamp(13px, 1vw, 15px);
|
||||
font-family: ui-monospace, SFMono-Regular, Menlo, monospace; }
|
||||
.bar-track { height: clamp(24px, 3vh, 40px); background: var(--bg-elev);
|
||||
border: 1px solid var(--line); border-radius: 3px; overflow: hidden; }
|
||||
.bar-fill { height: 100%; background: var(--accent);
|
||||
transition: width 600ms cubic-bezier(0.2, 0.8, 0.2, 1); }
|
||||
.bar-count { color: var(--fg-dim); font-size: clamp(13px, 1vw, 15px); text-align: right;
|
||||
font-family: ui-monospace, SFMono-Regular, Menlo, monospace; }
|
||||
|
||||
/* Phase mix */
|
||||
.phase-stack {
|
||||
display: flex; height: clamp(48px, 7vh, 96px);
|
||||
border-radius: 4px; overflow: hidden;
|
||||
background: var(--bg-elev); border: 1px solid var(--line);
|
||||
}
|
||||
.phase-seg { transition: flex-grow 600ms ease; flex-grow: 0; min-width: 0; }
|
||||
.phase-seg.clean { background: var(--phase-clean); }
|
||||
.phase-seg.armed { background: var(--phase-armed); }
|
||||
.phase-seg.infecting { background: var(--phase-infecting); }
|
||||
.phase-seg.infected_running { background: var(--phase-running); }
|
||||
.phase-seg.dormant { background: var(--phase-dormant); }
|
||||
.phase-legend { display: flex; flex-wrap: wrap; gap: 14px;
|
||||
font-size: 12px; color: var(--fg-dim);
|
||||
font-family: ui-monospace, SFMono-Regular, Menlo, monospace; }
|
||||
.phase-legend > span { display: inline-flex; align-items: center; }
|
||||
.phase-legend .swatch { display: inline-block; width: 10px; height: 10px;
|
||||
border-radius: 2px; margin-right: 6px; }
|
||||
|
||||
/* ─── Database explorer ────────────────────────────────────────────── */
|
||||
.db-header {
|
||||
display: flex; align-items: baseline; gap: 16px; flex-wrap: wrap;
|
||||
}
|
||||
.db-count { color: var(--fg-mute); font-size: 12px;
|
||||
font-family: ui-monospace, SFMono-Regular, Menlo, monospace;
|
||||
font-variant-numeric: tabular-nums; }
|
||||
.db-controls {
|
||||
display: flex; gap: 10px; align-items: center; flex-wrap: wrap;
|
||||
}
|
||||
.db-tabs { display: flex; gap: 6px; flex-wrap: wrap; }
|
||||
.db-tab {
|
||||
background: transparent; color: var(--fg-dim);
|
||||
border: 1px solid var(--line); border-radius: 16px;
|
||||
padding: 4px 12px; font: inherit; font-size: 12px; cursor: pointer;
|
||||
font-family: ui-monospace, SFMono-Regular, Menlo, monospace;
|
||||
transition: color 120ms, border-color 120ms, background 120ms;
|
||||
}
|
||||
.db-tab:hover { color: var(--fg); border-color: var(--fg-mute); }
|
||||
.db-tab.active { color: var(--accent); border-color: var(--accent);
|
||||
background: var(--accent-soft); }
|
||||
.db-search {
|
||||
flex: 1; min-width: 200px;
|
||||
background: var(--bg-elev); color: var(--fg);
|
||||
border: 1px solid var(--line); border-radius: 4px;
|
||||
padding: 6px 10px; font: inherit; font-size: 13px;
|
||||
font-family: ui-monospace, SFMono-Regular, Menlo, monospace;
|
||||
}
|
||||
.db-search:focus { outline: none; border-color: var(--accent); }
|
||||
.db-table-wrap {
|
||||
flex: 1 1 auto; min-height: 0;
|
||||
max-height: clamp(280px, 56vh, 720px);
|
||||
overflow: auto;
|
||||
border: 1px solid var(--line); border-radius: 4px;
|
||||
background: var(--bg-elev);
|
||||
}
|
||||
.db-table {
|
||||
width: 100%; border-collapse: collapse;
|
||||
font-size: clamp(12px, 0.92vw, 14px);
|
||||
font-variant-numeric: tabular-nums;
|
||||
}
|
||||
.db-table thead th {
|
||||
position: sticky; top: 0; z-index: 1;
|
||||
background: var(--bg-elev2); color: var(--fg-dim);
|
||||
text-align: left; padding: 8px 12px;
|
||||
font-size: 11px; letter-spacing: 0.06em; text-transform: uppercase;
|
||||
font-weight: 500; border-bottom: 1px solid var(--line);
|
||||
}
|
||||
.db-table tbody tr {
|
||||
border-bottom: 1px solid var(--line-soft); cursor: pointer;
|
||||
transition: background 80ms;
|
||||
}
|
||||
.db-table tbody tr:hover { background: rgba(88, 166, 255, 0.06); }
|
||||
.db-table tbody tr.selected { background: var(--accent-soft); }
|
||||
.db-table td {
|
||||
padding: 7px 12px; color: var(--fg);
|
||||
font-family: ui-monospace, SFMono-Regular, Menlo, monospace;
|
||||
}
|
||||
.db-table td.db-size { color: var(--fg-dim); text-align: right; }
|
||||
.db-host { color: var(--fg); }
|
||||
.db-id { color: var(--fg-dim); }
|
||||
.db-detail {
|
||||
max-height: clamp(180px, 30vh, 360px); overflow: auto;
|
||||
border: 1px solid var(--line); border-radius: 4px;
|
||||
background: var(--bg-elev);
|
||||
}
|
||||
.db-detail pre {
|
||||
margin: 0; padding: 14px 18px;
|
||||
font-family: ui-monospace, SFMono-Regular, Menlo, monospace;
|
||||
font-size: 12px; line-height: 1.5; color: var(--fg);
|
||||
white-space: pre-wrap; word-break: break-all;
|
||||
}
|
||||
|
||||
/* ─── Attack envelope thumbnails ───────────────────────────────────── */
|
||||
.profile-grid {
|
||||
display: grid; grid-template-columns: repeat(2, minmax(0, 1fr));
|
||||
gap: clamp(12px, 1.4vw, 22px);
|
||||
}
|
||||
.profile-card {
|
||||
border: 1px solid var(--line); border-radius: 4px;
|
||||
padding: clamp(12px, 1.2vw, 18px);
|
||||
background: rgba(13, 17, 23, 0.6);
|
||||
}
|
||||
.profile-name {
|
||||
font-family: ui-monospace, SFMono-Regular, Menlo, monospace;
|
||||
font-size: clamp(13px, 1vw, 15px); color: var(--fg); margin-bottom: 4px;
|
||||
}
|
||||
.profile-shape {
|
||||
font-size: clamp(11px, 0.85vw, 13px); color: var(--fg-dim);
|
||||
margin-bottom: 8px; line-height: 1.4;
|
||||
}
|
||||
.profile-card svg {
|
||||
display: block; width: 100%;
|
||||
height: clamp(56px, 9vh, 120px);
|
||||
}
|
||||
.profile-card svg path { fill: none; stroke: var(--accent); stroke-width: 1.4;
|
||||
vector-effect: non-scaling-stroke; }
|
||||
|
||||
/* ─── Chunking timeline ────────────────────────────────────────────── */
|
||||
.chunk-rule, .chunk-row, .chunk-axis { display: flex; gap: 4px; }
|
||||
.chunk-row { height: clamp(56px, 9vh, 120px); }
|
||||
.chunk-cell {
|
||||
flex: 1; border-radius: 3px;
|
||||
display: flex; align-items: center; justify-content: center;
|
||||
font-family: ui-monospace, SFMono-Regular, Menlo, monospace;
|
||||
font-size: clamp(11px, 0.95vw, 14px); color: rgba(255,255,255,0.85);
|
||||
}
|
||||
.chunk-cell.clean { background: var(--phase-clean); }
|
||||
.chunk-cell.armed { background: var(--phase-armed); }
|
||||
.chunk-cell.infecting { background: var(--phase-infecting); }
|
||||
.chunk-cell.infected_running { background: var(--phase-running); }
|
||||
.chunk-cell.dormant { background: var(--phase-dormant); }
|
||||
.chunk-rule { height: 8px; background: var(--bg-elev);
|
||||
border: 1px solid var(--line); border-radius: 2px; padding: 1px; }
|
||||
.chunk-rule .tick { flex: 1; border-right: 1px solid var(--line); }
|
||||
.chunk-rule .tick:last-child { border-right: none; }
|
||||
.chunk-axis { height: 16px; font-size: 10px; color: var(--fg-mute);
|
||||
font-family: ui-monospace, SFMono-Regular, Menlo, monospace; }
|
||||
.chunk-axis span { flex: 1; text-align: center; }
|
||||
|
||||
/* ─── Model bars ───────────────────────────────────────────────────── */
|
||||
.model-bars { display: flex; flex-direction: column; gap: clamp(10px, 1.5vh, 18px); }
|
||||
.model-row {
|
||||
display: grid; grid-template-columns: minmax(80px, 12ch) 1fr minmax(64px, 9ch);
|
||||
gap: clamp(10px, 1vw, 18px); align-items: center; font-variant-numeric: tabular-nums;
|
||||
}
|
||||
.model-name { font-family: ui-monospace, SFMono-Regular, Menlo, monospace;
|
||||
font-size: clamp(13px, 1vw, 15px); color: var(--fg); }
|
||||
.model-track { height: clamp(28px, 4vh, 48px); background: var(--bg-elev);
|
||||
border: 1px solid var(--line); border-radius: 3px; overflow: hidden; }
|
||||
.model-fill { height: 100%; transition: width 600ms cubic-bezier(0.2, 0.8, 0.2, 1); }
|
||||
.model-fill.lstm { background: linear-gradient(90deg, #58a6ff, #1f6feb); }
|
||||
.model-fill.gru { background: linear-gradient(90deg, #db61a2, #a8327f); }
|
||||
.model-fill.rnn { background: linear-gradient(90deg, #d29922, #8a6a17); }
|
||||
.model-fill.bert { background: linear-gradient(90deg, #f85149, #b22e2a); }
|
||||
.model-acc { font-family: ui-monospace, SFMono-Regular, Menlo, monospace;
|
||||
font-size: clamp(13px, 1vw, 15px); color: var(--fg-dim); text-align: right; }
|
||||
|
||||
/* ─── Scatter plots (KNN, perf) ────────────────────────────────────── */
|
||||
.scatter {
|
||||
width: 100%;
|
||||
height: clamp(320px, 60vh, 640px);
|
||||
}
|
||||
.scatter .axis { stroke: var(--line); stroke-width: 1; }
|
||||
.scatter .axis-label { fill: var(--fg-mute); font-size: 12px;
|
||||
font-family: ui-monospace, SFMono-Regular, Menlo, monospace; }
|
||||
.scatter .point { transition: r 200ms; }
|
||||
.scatter .point.clean { fill: var(--phase-clean); }
|
||||
.scatter .point.armed { fill: var(--phase-armed); }
|
||||
.scatter .point.infecting { fill: var(--phase-infecting); }
|
||||
.scatter .point.infected_running { fill: var(--phase-running); }
|
||||
.scatter .point.dormant { fill: var(--phase-dormant); }
|
||||
.scatter .perf-point { fill: var(--accent); stroke: #1f6feb; stroke-width: 1.5; }
|
||||
.scatter .perf-label { fill: var(--fg); font-size: 13px;
|
||||
font-family: ui-monospace, SFMono-Regular, Menlo, monospace; }
|
||||
|
||||
/* ─── Floating advance button ──────────────────────────────────────── */
|
||||
.fab {
|
||||
position: absolute; right: 24px; bottom: 24px; z-index: 5;
|
||||
width: 44px; height: 44px; border-radius: 50%;
|
||||
background: rgba(13, 17, 23, 0.85); color: var(--fg-dim);
|
||||
border: 1px solid var(--line); font-size: 16px; cursor: pointer;
|
||||
backdrop-filter: blur(8px);
|
||||
display: flex; align-items: center; justify-content: center;
|
||||
transition: color 150ms, border-color 150ms, transform 150ms;
|
||||
}
|
||||
.fab:hover { color: var(--accent); border-color: var(--accent); transform: translateY(-1px); }
|
||||
.fab:disabled { opacity: 0.25; cursor: not-allowed; transform: none; }
|
||||
|
||||
/* ─── Article (prose, overlaid right) ──────────────────────────────── */
|
||||
.scene {
|
||||
display: flex; align-items: center; justify-content: flex-end;
|
||||
min-height: 100vh; padding: 4rem 2rem;
|
||||
}
|
||||
.scene .prose {
|
||||
width: var(--prose-w); max-width: calc(100vw - 4rem);
|
||||
padding: 2.25rem 2.5rem 2.25rem 5rem;
|
||||
font-size: 17px; line-height: 1.65; color: var(--fg);
|
||||
background: linear-gradient(
|
||||
to left,
|
||||
rgba(var(--bg-rgb), 0.96) 0%,
|
||||
rgba(var(--bg-rgb), 0.92) 55%,
|
||||
rgba(var(--bg-rgb), 0.0) 100%
|
||||
);
|
||||
opacity: 0; transform: translateY(20px);
|
||||
transition: opacity var(--scene-fade-ms) ease,
|
||||
transform var(--scene-fade-ms) ease;
|
||||
}
|
||||
.scene[data-active] .prose { opacity: 1; transform: translateY(0); }
|
||||
.scene .prose h2 { margin: 0 0 14px; font-size: 24px; font-weight: 600;
|
||||
letter-spacing: -0.01em; }
|
||||
.scene .prose .lede { font-size: 24px; line-height: 1.4; font-weight: 500;
|
||||
margin: 0 0 20px; }
|
||||
.scene .prose p { margin: 0 0 14px; }
|
||||
.scene .prose strong { color: #fff; font-weight: 600; }
|
||||
.scene .prose code {
|
||||
font-family: ui-monospace, SFMono-Regular, Menlo, monospace;
|
||||
font-size: 0.9em; color: var(--accent);
|
||||
background: var(--accent-soft); padding: 1px 5px; border-radius: 3px;
|
||||
}
|
||||
.scene .prose .hint {
|
||||
color: var(--fg-mute); font-size: 12px;
|
||||
letter-spacing: 0.16em; text-transform: uppercase; margin-top: 28px;
|
||||
}
|
||||
|
||||
.scene-end-spacer { height: 30vh; }
|
||||
|
||||
@media (max-width: 880px) {
|
||||
:root { --prose-w: 92vw; }
|
||||
.stage-view { padding-right: 0; padding-bottom: 50vh; }
|
||||
.intro-block, .metric-stack { padding: 0 24px; }
|
||||
.bar-row { grid-template-columns: 110px 1fr 60px; }
|
||||
.model-row { grid-template-columns: 80px 1fr 56px; }
|
||||
.profile-grid { grid-template-columns: 1fr; }
|
||||
.scene { padding: 2rem 1rem; min-height: 80vh; justify-content: center; }
|
||||
.scene .prose { padding: 1.5rem; }
|
||||
.topbar .counter { display: none; }
|
||||
.db-table-wrap { max-height: 40vh; }
|
||||
}
|
||||
821
training/dashboard/static/dashboard.js
Normal file
821
training/dashboard/static/dashboard.js
Normal file
|
|
@ -0,0 +1,821 @@
|
|||
// CIS490 dashboard front-end.
|
||||
//
|
||||
// Layers:
|
||||
// 1. Transport + typed message bus
|
||||
// 2. Scrollytelling controller (hotkeys, click, prev/next, FAB)
|
||||
// 3. Public API + demo machinery
|
||||
// 4. Widgets — one per scene
|
||||
//
|
||||
// Events on the bus:
|
||||
//
|
||||
// Real (from server feeders):
|
||||
// hello — {type, clients} one-shot on connect
|
||||
// snapshot — {type, total_episodes, total_alerts,
|
||||
// total_bytes, host_counts,
|
||||
// recent_episodes: [...]} every 30 s + on connect
|
||||
// episode — {type, episode_id, host_id, sha256,
|
||||
// size_bytes, received_at} one per index.jsonl line
|
||||
// alert — {type, host_id, symptom, detail,
|
||||
// suggested_fix, detected_at} one per alerts.jsonl line
|
||||
//
|
||||
// Real (from future producers — overwrite synthetic if demo is on):
|
||||
// phase — {type, phase}
|
||||
// prediction — {type, episode_id, window_idx, predicted, actual}
|
||||
// model_metric — {type, model, accuracy}
|
||||
// embedding — {type, x, y, phase}
|
||||
// model_perf — {type, model, latency_us, accuracy}
|
||||
// attack_profile — {type, name, shape, curve: [...]}
|
||||
//
|
||||
// Local (browser-only, never hit the wire):
|
||||
// demo_start — {type} emitted when demo toggles on
|
||||
// demo_stop — {type} emitted when demo toggles off
|
||||
//
|
||||
// Demo-only widgets render NOTHING by default (they show an "awaiting"
|
||||
// row). They populate synthetic data on demo_start and clear on
|
||||
// demo_stop. Real producer events overwrite either way.
|
||||
|
||||
(function () {
|
||||
'use strict';
|
||||
|
||||
// ────────────────────────────────────────────────────────────────
|
||||
// 1. Transport + bus
|
||||
// ────────────────────────────────────────────────────────────────
|
||||
const statusEl = document.getElementById('status');
|
||||
const subscribers = new Map();
|
||||
const wildcardSubs = new Set();
|
||||
|
||||
function on(type, fn) {
|
||||
if (type === '*') { wildcardSubs.add(fn); return () => wildcardSubs.delete(fn); }
|
||||
if (!subscribers.has(type)) subscribers.set(type, new Set());
|
||||
subscribers.get(type).add(fn);
|
||||
return () => subscribers.get(type).delete(fn);
|
||||
}
|
||||
function dispatch(msg) {
|
||||
const t = (msg && msg.type) || '__no_type__';
|
||||
const subs = subscribers.get(t);
|
||||
if (subs) subs.forEach(fn => { try { fn(msg); } catch (e) { console.error(e); } });
|
||||
wildcardSubs.forEach(fn => { try { fn(msg); } catch (e) { console.error(e); } });
|
||||
}
|
||||
|
||||
let ws = null;
|
||||
function connect() {
|
||||
const proto = location.protocol === 'https:' ? 'wss:' : 'ws:';
|
||||
ws = new WebSocket(`${proto}//${location.host}/ws`);
|
||||
ws.onopen = () => { statusEl.textContent = 'live'; statusEl.className = 'status ok'; };
|
||||
ws.onclose = () => {
|
||||
statusEl.textContent = 'reconnecting…'; statusEl.className = 'status bad';
|
||||
setTimeout(connect, 1500);
|
||||
};
|
||||
ws.onerror = () => {};
|
||||
ws.onmessage = ev => {
|
||||
let msg; try { msg = JSON.parse(ev.data); } catch { msg = { type: 'raw', raw: ev.data }; }
|
||||
dispatch(msg);
|
||||
};
|
||||
}
|
||||
connect();
|
||||
function send(msg) { if (ws && ws.readyState === WebSocket.OPEN) ws.send(JSON.stringify(msg)); }
|
||||
|
||||
// ────────────────────────────────────────────────────────────────
|
||||
// 2. Scrollytelling controller
|
||||
// ────────────────────────────────────────────────────────────────
|
||||
const scenes = Array.from(document.querySelectorAll('.scene[data-stage]'));
|
||||
const stageViews = new Map();
|
||||
document.querySelectorAll('.stage-view[data-view]').forEach(el => {
|
||||
stageViews.set(el.dataset.view, el);
|
||||
});
|
||||
|
||||
const sceneEnterHandlers = new Map();
|
||||
const sceneExitHandlers = new Map();
|
||||
function onScene(name, { onEnter, onExit } = {}) {
|
||||
if (onEnter) sceneEnterHandlers.set(name, onEnter);
|
||||
if (onExit) sceneExitHandlers.set(name, onExit);
|
||||
}
|
||||
|
||||
const sceneIdxEl = document.getElementById('scene-idx');
|
||||
const sceneTotalEl = document.getElementById('scene-total');
|
||||
const prevBtn = document.getElementById('prev-btn');
|
||||
const nextBtn = document.getElementById('next-btn');
|
||||
const fab = document.getElementById('next-fab');
|
||||
sceneTotalEl.textContent = String(scenes.length);
|
||||
|
||||
let activeIdx = -1;
|
||||
function setActiveIdx(idx) {
|
||||
if (idx === activeIdx) return;
|
||||
if (activeIdx >= 0) {
|
||||
const prevName = scenes[activeIdx].dataset.stage;
|
||||
const view = stageViews.get(prevName);
|
||||
if (view) view.removeAttribute('data-active');
|
||||
const fn = sceneExitHandlers.get(prevName);
|
||||
if (fn) try { fn(); } catch (e) { console.error(e); }
|
||||
scenes[activeIdx].removeAttribute('data-active');
|
||||
}
|
||||
activeIdx = idx;
|
||||
sceneIdxEl.textContent = String(idx + 1);
|
||||
const name = scenes[idx].dataset.stage;
|
||||
const view = stageViews.get(name);
|
||||
if (view) view.setAttribute('data-active', '');
|
||||
scenes[idx].setAttribute('data-active', '');
|
||||
const fn = sceneEnterHandlers.get(name);
|
||||
if (fn) try { fn(); } catch (e) { console.error(e); }
|
||||
prevBtn.disabled = idx === 0;
|
||||
nextBtn.disabled = idx === scenes.length - 1;
|
||||
if (fab) fab.disabled = idx === scenes.length - 1;
|
||||
}
|
||||
|
||||
const sceneRatios = new Map();
|
||||
const io = new IntersectionObserver(entries => {
|
||||
entries.forEach(e => sceneRatios.set(e.target, e.intersectionRatio));
|
||||
let bestIdx = activeIdx >= 0 ? activeIdx : 0;
|
||||
let bestRatio = -1;
|
||||
scenes.forEach((s, i) => {
|
||||
const r = sceneRatios.get(s) || 0;
|
||||
if (r > bestRatio) { bestRatio = r; bestIdx = i; }
|
||||
});
|
||||
setActiveIdx(bestIdx);
|
||||
}, {
|
||||
threshold: [0, 0.25, 0.5, 0.75, 1],
|
||||
rootMargin: '-30% 0px -30% 0px',
|
||||
});
|
||||
scenes.forEach(s => io.observe(s));
|
||||
setActiveIdx(0);
|
||||
|
||||
function scrollToScene(idx) {
|
||||
idx = Math.max(0, Math.min(scenes.length - 1, idx));
|
||||
if (idx === activeIdx) return;
|
||||
scenes[idx].scrollIntoView({ behavior: 'smooth', block: 'start' });
|
||||
}
|
||||
function next() { scrollToScene(activeIdx + 1); }
|
||||
function prev() { scrollToScene(activeIdx - 1); }
|
||||
|
||||
window.addEventListener('keydown', e => {
|
||||
if (e.metaKey || e.ctrlKey || e.altKey) return;
|
||||
const tag = (e.target && e.target.tagName) || '';
|
||||
if (tag === 'INPUT' || tag === 'TEXTAREA' || (e.target && e.target.isContentEditable)) return;
|
||||
switch (e.key) {
|
||||
case 'ArrowDown': case 'ArrowRight': case 'PageDown': case 'j': case ' ':
|
||||
e.preventDefault(); next(); break;
|
||||
case 'ArrowUp': case 'ArrowLeft': case 'PageUp': case 'k':
|
||||
e.preventDefault(); prev(); break;
|
||||
case 'Home': e.preventDefault(); scrollToScene(0); break;
|
||||
case 'End': e.preventDefault(); scrollToScene(scenes.length - 1); break;
|
||||
case 'c': case 'C': e.preventDefault(); setClickNav(!clickNavOn); break;
|
||||
}
|
||||
});
|
||||
prevBtn.addEventListener('click', prev);
|
||||
nextBtn.addEventListener('click', next);
|
||||
if (fab) fab.addEventListener('click', next);
|
||||
|
||||
// Click-on-stage to advance — gated by the click-nav toggle so
|
||||
// interactive widgets (db table rows, search boxes) don't compete
|
||||
// with the next-slide gesture by default. Topbar arrows / FAB /
|
||||
// hotkeys always work regardless.
|
||||
const clickNavBtn = document.getElementById('click-nav-btn');
|
||||
let clickNavOn = false;
|
||||
function setClickNav(on) {
|
||||
clickNavOn = on;
|
||||
clickNavBtn.textContent = `click-nav: ${on ? 'on' : 'off'}`;
|
||||
clickNavBtn.classList.toggle('active', on);
|
||||
}
|
||||
clickNavBtn.addEventListener('click', e => { e.stopPropagation(); setClickNav(!clickNavOn); });
|
||||
setClickNav(false);
|
||||
|
||||
const stageCol = document.getElementById('stage-col');
|
||||
stageCol.addEventListener('click', e => {
|
||||
if (!clickNavOn) return;
|
||||
if (e.target.closest('[data-no-advance]')) return;
|
||||
if (e.target.closest('button, a, input, select, textarea')) return;
|
||||
next();
|
||||
});
|
||||
|
||||
// ────────────────────────────────────────────────────────────────
|
||||
// 3. Public API + demo machinery
|
||||
// ────────────────────────────────────────────────────────────────
|
||||
window.dashboard = { on, send, scene: onScene, dispatch, next, prev, scrollToScene };
|
||||
|
||||
const demoBtn = document.getElementById('demo-btn');
|
||||
let demoTimer = null;
|
||||
let demoActive = false;
|
||||
const HOSTS = ['elliott-lab', 'elliott-thinkpad', 'k-gamingcom'];
|
||||
const PHASES = ['clean', 'armed', 'infecting', 'infected_running', 'dormant'];
|
||||
function demoTick() {
|
||||
if (Math.random() < 0.7) {
|
||||
const host = HOSTS[Math.floor(Math.random() * HOSTS.length)];
|
||||
dispatch({ type: 'episode',
|
||||
episode_id: 'demo-' + Math.random().toString(36).slice(2, 12),
|
||||
host_id: host, size_bytes: 30_000 + Math.random() * 20_000 });
|
||||
}
|
||||
if (Math.random() < 0.5) {
|
||||
dispatch({ type: 'phase', phase: PHASES[Math.floor(Math.random() * PHASES.length)] });
|
||||
}
|
||||
// Occasionally tweak a model metric so the bars aren't static.
|
||||
if (Math.random() < 0.05) {
|
||||
const m = ['rnn', 'gru', 'lstm', 'bert'][Math.floor(Math.random() * 4)];
|
||||
const base = { rnn: 0.872, gru: 0.911, lstm: 0.928, bert: 0.954 }[m];
|
||||
dispatch({ type: 'model_metric', model: m, accuracy: base + (Math.random() - 0.5) * 0.012 });
|
||||
}
|
||||
}
|
||||
function setDemo(active) {
|
||||
if (demoActive === active) return;
|
||||
demoActive = active;
|
||||
if (active) {
|
||||
dispatch({ type: 'demo_start' });
|
||||
demoTimer = setInterval(demoTick, 350);
|
||||
demoBtn.textContent = 'demo: on'; demoBtn.classList.add('active');
|
||||
} else {
|
||||
if (demoTimer) clearInterval(demoTimer); demoTimer = null;
|
||||
dispatch({ type: 'demo_stop' });
|
||||
demoBtn.textContent = 'demo: off'; demoBtn.classList.remove('active');
|
||||
}
|
||||
}
|
||||
demoBtn.addEventListener('click', e => { e.stopPropagation(); setDemo(!demoActive); });
|
||||
|
||||
// Helper for empty-state rows.
|
||||
function awaitingNote(text) {
|
||||
const d = document.createElement('div');
|
||||
d.className = 'awaiting'; d.textContent = text;
|
||||
return d;
|
||||
}
|
||||
|
||||
// ────────────────────────────────────────────────────────────────
|
||||
// 4. Widgets
|
||||
// ────────────────────────────────────────────────────────────────
|
||||
|
||||
// ── Ingest counter + 60-second sparkline ──────────────────────
|
||||
// Real-data widget: populated by snapshot + episode events. No
|
||||
// demo gating — when demo is off, it just shows real activity.
|
||||
(function () {
|
||||
const totalEl = document.getElementById('ingest-total');
|
||||
const rateEl = document.getElementById('ingest-rate');
|
||||
const bytesEl = document.getElementById('ingest-bytes');
|
||||
const pathEl = document.getElementById('ingest-spark-path');
|
||||
const fillEl = document.getElementById('ingest-spark-fill');
|
||||
const W = 600, H = 120, BUCKETS = 60;
|
||||
const buckets = new Array(BUCKETS).fill(0);
|
||||
let total = 0, totalBytes = 0;
|
||||
|
||||
function bucketIndex() { return Math.floor(Date.now() / 1000) % BUCKETS; }
|
||||
let lastBucket = bucketIndex();
|
||||
|
||||
function rotateIfNeeded() {
|
||||
const cur = bucketIndex();
|
||||
while (lastBucket !== cur) {
|
||||
lastBucket = (lastBucket + 1) % BUCKETS;
|
||||
buckets[lastBucket] = 0;
|
||||
}
|
||||
}
|
||||
|
||||
function fmtBytes(n) {
|
||||
if (!n) return '0 B';
|
||||
const u = ['B', 'KB', 'MB', 'GB', 'TB']; let i = 0;
|
||||
while (n >= 1024 && i < u.length - 1) { n /= 1024; i++; }
|
||||
return n.toFixed(n >= 100 || i === 0 ? 0 : 1) + ' ' + u[i];
|
||||
}
|
||||
|
||||
function render() {
|
||||
rotateIfNeeded();
|
||||
totalEl.textContent = total.toLocaleString();
|
||||
const sum = buckets.reduce((a, b) => a + b, 0);
|
||||
rateEl.textContent = (sum / BUCKETS).toFixed(1);
|
||||
if (bytesEl) bytesEl.textContent = fmtBytes(totalBytes);
|
||||
const max = Math.max(1, ...buckets);
|
||||
const pts = [];
|
||||
for (let i = 0; i < BUCKETS; i++) {
|
||||
const idx = (lastBucket + 1 + i) % BUCKETS;
|
||||
const x = (i / (BUCKETS - 1)) * W;
|
||||
const y = H - (buckets[idx] / max) * (H - 8) - 4;
|
||||
pts.push([x, y]);
|
||||
}
|
||||
const d = pts.map(([x, y], i) => `${i === 0 ? 'M' : 'L'}${x.toFixed(1)},${y.toFixed(1)}`).join(' ');
|
||||
pathEl.setAttribute('d', d);
|
||||
fillEl.setAttribute('d', `${d} L${W},${H} L0,${H} Z`);
|
||||
}
|
||||
|
||||
on('snapshot', m => {
|
||||
if (typeof m.total_episodes === 'number') total = m.total_episodes;
|
||||
if (typeof m.total_bytes === 'number') totalBytes = m.total_bytes;
|
||||
render();
|
||||
});
|
||||
on('episode', m => {
|
||||
rotateIfNeeded();
|
||||
buckets[lastBucket] += 1;
|
||||
total += 1;
|
||||
if (typeof m.size_bytes === 'number') totalBytes += m.size_bytes;
|
||||
render();
|
||||
});
|
||||
setInterval(render, 1000);
|
||||
render();
|
||||
})();
|
||||
|
||||
// ── Per-host bars ─────────────────────────────────────────────
|
||||
// Real-data widget. Snapshot seeds absolute counts; episode events
|
||||
// increment.
|
||||
(function () {
|
||||
const root = document.getElementById('host-bars');
|
||||
const counts = new Map();
|
||||
const rows = new Map();
|
||||
|
||||
function clearEmpty() {
|
||||
const e = root.querySelector('.bars-empty, .awaiting');
|
||||
if (e) e.remove();
|
||||
}
|
||||
function ensureRow(host) {
|
||||
if (rows.has(host)) return rows.get(host);
|
||||
clearEmpty();
|
||||
const row = document.createElement('div'); row.className = 'bar-row';
|
||||
const name = document.createElement('div'); name.className = 'bar-host'; name.textContent = host;
|
||||
const track = document.createElement('div'); track.className = 'bar-track';
|
||||
const fill = document.createElement('div'); fill.className = 'bar-fill'; fill.style.width = '0%';
|
||||
track.appendChild(fill);
|
||||
const label = document.createElement('div'); label.className = 'bar-count'; label.textContent = '0';
|
||||
row.append(name, track, label);
|
||||
root.appendChild(row);
|
||||
const entry = { row, fill, label };
|
||||
rows.set(host, entry); return entry;
|
||||
}
|
||||
function render() {
|
||||
const max = Math.max(1, ...counts.values());
|
||||
Array.from(counts.keys()).forEach(h => {
|
||||
const r = ensureRow(h);
|
||||
const c = counts.get(h);
|
||||
r.fill.style.width = ((c / max) * 100).toFixed(1) + '%';
|
||||
r.label.textContent = c.toLocaleString();
|
||||
});
|
||||
Array.from(counts.keys()).sort((a, b) => counts.get(b) - counts.get(a))
|
||||
.forEach(h => root.appendChild(rows.get(h).row));
|
||||
}
|
||||
|
||||
on('snapshot', m => {
|
||||
if (m.host_counts && typeof m.host_counts === 'object') {
|
||||
Object.entries(m.host_counts).forEach(([h, c]) => counts.set(h, c));
|
||||
render();
|
||||
}
|
||||
});
|
||||
on('episode', m => {
|
||||
if (!m.host_id) return;
|
||||
counts.set(m.host_id, (counts.get(m.host_id) || 0) + 1); render();
|
||||
});
|
||||
})();
|
||||
|
||||
// ── Phase mix (rolling 5 min) ─────────────────────────────────
|
||||
// Real-data widget. Will be empty until phase events flow.
|
||||
(function () {
|
||||
const stack = document.getElementById('phase-stack');
|
||||
const legend = document.getElementById('phase-legend');
|
||||
const PHASES = ['clean', 'armed', 'infecting', 'infected_running', 'dormant'];
|
||||
const WINDOW_MS = 5 * 60 * 1000;
|
||||
const samples = [];
|
||||
const segs = new Map();
|
||||
|
||||
PHASES.forEach(p => {
|
||||
const seg = document.createElement('div');
|
||||
seg.className = `phase-seg ${p}`; stack.appendChild(seg); segs.set(p, seg);
|
||||
const li = document.createElement('span');
|
||||
const swatchVar = p === 'infected_running' ? 'running' : p;
|
||||
li.innerHTML = `<span class="swatch" style="background:var(--phase-${swatchVar})"></span>${p}`;
|
||||
legend.appendChild(li);
|
||||
});
|
||||
|
||||
function render() {
|
||||
const now = Date.now();
|
||||
while (samples.length && now - samples[0].t > WINDOW_MS) samples.shift();
|
||||
const counts = Object.fromEntries(PHASES.map(p => [p, 0]));
|
||||
samples.forEach(s => { if (counts[s.phase] !== undefined) counts[s.phase]++; });
|
||||
const total = Math.max(1, samples.length);
|
||||
PHASES.forEach(p => { segs.get(p).style.flexGrow = (counts[p] / total).toFixed(4); });
|
||||
}
|
||||
|
||||
on('phase', m => {
|
||||
if (!m.phase) return;
|
||||
samples.push({ phase: m.phase, t: Date.now() }); render();
|
||||
});
|
||||
on('demo_stop', () => { samples.length = 0; render(); });
|
||||
setInterval(render, 1000);
|
||||
})();
|
||||
|
||||
// ── Database explorer ─────────────────────────────────────────
|
||||
// Real-data widget. Initial population from snapshot.recent_episodes
|
||||
// (last 200 lines of index.jsonl). New episodes prepend live.
|
||||
(function () {
|
||||
const tabsEl = document.getElementById('db-tabs');
|
||||
const searchEl = document.getElementById('db-search');
|
||||
const tbodyEl = document.getElementById('db-tbody');
|
||||
const detailEl = document.getElementById('db-detail');
|
||||
const detailPre = document.getElementById('db-detail-pre');
|
||||
const countEl = document.getElementById('db-count');
|
||||
|
||||
let records = []; // newest first
|
||||
let activeHost = null; // null = all
|
||||
let query = '';
|
||||
|
||||
function fmtBytes(n) {
|
||||
if (!n) return '—';
|
||||
const u = ['B', 'KB', 'MB', 'GB']; let i = 0;
|
||||
while (n >= 1024 && i < u.length - 1) { n /= 1024; i++; }
|
||||
return n.toFixed(n >= 100 || i === 0 ? 0 : 1) + ' ' + u[i];
|
||||
}
|
||||
function fmtTime(iso) {
|
||||
if (!iso) return '—';
|
||||
try { return new Date(iso).toLocaleTimeString('en-US', { hour12: false }); }
|
||||
catch { return iso; }
|
||||
}
|
||||
function shortId(id) {
|
||||
if (!id) return '—';
|
||||
return id.length > 24 ? id.slice(0, 16) + '…' + id.slice(-6) : id;
|
||||
}
|
||||
|
||||
function rebuildTabs() {
|
||||
const hosts = Array.from(new Set(records.map(r => r.host_id).filter(Boolean))).sort();
|
||||
tabsEl.innerHTML = '';
|
||||
const all = document.createElement('button');
|
||||
all.className = 'db-tab' + (activeHost === null ? ' active' : '');
|
||||
all.textContent = `all · ${records.length}`;
|
||||
all.addEventListener('click', e => { e.stopPropagation(); activeHost = null; rebuildTabs(); rebuildTable(); });
|
||||
tabsEl.appendChild(all);
|
||||
hosts.forEach(h => {
|
||||
const b = document.createElement('button');
|
||||
b.className = 'db-tab' + (activeHost === h ? ' active' : '');
|
||||
const c = records.filter(r => r.host_id === h).length;
|
||||
b.textContent = `${h} · ${c}`;
|
||||
b.addEventListener('click', e => { e.stopPropagation(); activeHost = h; rebuildTabs(); rebuildTable(); });
|
||||
tabsEl.appendChild(b);
|
||||
});
|
||||
}
|
||||
|
||||
function matches(rec) {
|
||||
if (activeHost && rec.host_id !== activeHost) return false;
|
||||
if (!query) return true;
|
||||
const q = query.toLowerCase();
|
||||
return (rec.host_id || '').toLowerCase().includes(q)
|
||||
|| (rec.episode_id || '').toLowerCase().includes(q)
|
||||
|| (rec.sha256 || '').toLowerCase().includes(q);
|
||||
}
|
||||
|
||||
function rebuildTable() {
|
||||
const filtered = records.filter(matches);
|
||||
countEl.textContent = `${filtered.length} of ${records.length}`;
|
||||
tbodyEl.innerHTML = '';
|
||||
if (!filtered.length) {
|
||||
const tr = document.createElement('tr');
|
||||
const td = document.createElement('td');
|
||||
td.colSpan = 4; td.className = 'awaiting';
|
||||
td.textContent = records.length === 0
|
||||
? 'awaiting snapshot…'
|
||||
: 'no rows match the current filter';
|
||||
tr.appendChild(td); tbodyEl.appendChild(tr);
|
||||
return;
|
||||
}
|
||||
const frag = document.createDocumentFragment();
|
||||
filtered.slice(0, 200).forEach(rec => {
|
||||
const tr = document.createElement('tr');
|
||||
tr.className = 'db-row'; tr.dataset.id = rec.episode_id || '';
|
||||
tr.innerHTML = `
|
||||
<td><span class="db-host">${rec.host_id || '—'}</span></td>
|
||||
<td><span class="db-id">${shortId(rec.episode_id)}</span></td>
|
||||
<td>${fmtTime(rec.received_at)}</td>
|
||||
<td class="db-size">${fmtBytes(rec.size_bytes)}</td>`;
|
||||
tr.addEventListener('click', e => {
|
||||
e.stopPropagation();
|
||||
detailEl.hidden = false;
|
||||
detailPre.textContent = JSON.stringify(rec, null, 2);
|
||||
tbodyEl.querySelectorAll('.db-row').forEach(r => r.classList.remove('selected'));
|
||||
tr.classList.add('selected');
|
||||
});
|
||||
frag.appendChild(tr);
|
||||
});
|
||||
tbodyEl.appendChild(frag);
|
||||
}
|
||||
|
||||
on('snapshot', m => {
|
||||
if (Array.isArray(m.recent_episodes)) {
|
||||
records = m.recent_episodes.slice();
|
||||
rebuildTabs(); rebuildTable();
|
||||
}
|
||||
});
|
||||
on('episode', m => {
|
||||
// Prepend; cap.
|
||||
records.unshift({
|
||||
episode_id: m.episode_id, host_id: m.host_id,
|
||||
sha256: m.sha256, size_bytes: m.size_bytes, received_at: m.received_at,
|
||||
});
|
||||
if (records.length > 200) records.length = 200;
|
||||
// Cheap update: only rebuild if scene visible.
|
||||
rebuildTabs(); rebuildTable();
|
||||
});
|
||||
on('demo_stop', () => {
|
||||
// Drop demo-injected records (their ids start with "demo-").
|
||||
const before = records.length;
|
||||
records = records.filter(r => !(r.episode_id && r.episode_id.startsWith('demo-')));
|
||||
if (records.length !== before) { rebuildTabs(); rebuildTable(); }
|
||||
});
|
||||
searchEl.addEventListener('input', e => { query = e.target.value; rebuildTable(); });
|
||||
searchEl.addEventListener('click', e => e.stopPropagation());
|
||||
rebuildTabs(); rebuildTable();
|
||||
})();
|
||||
|
||||
// ── Attack envelope thumbnails — DEMO ONLY ───────────────────
|
||||
(function () {
|
||||
const root = document.getElementById('profile-grid');
|
||||
const W = 200, H = 56;
|
||||
function emptyState() {
|
||||
root.innerHTML = '';
|
||||
root.appendChild(awaitingNote('awaiting attack_profile events · turn demo on for examples'));
|
||||
}
|
||||
function curveToPath(values) {
|
||||
const max = Math.max(1, ...values);
|
||||
return values.map((v, i) => {
|
||||
const x = (i / (values.length - 1)) * W;
|
||||
const y = H - (v / max) * (H - 6) - 3;
|
||||
return `${i === 0 ? 'M' : 'L'}${x.toFixed(1)},${y.toFixed(1)}`;
|
||||
}).join(' ');
|
||||
}
|
||||
function gen(seed, fn, n = 80) {
|
||||
let s = seed;
|
||||
const rand = () => { s = (s * 1664525 + 1013904223) >>> 0; return (s & 0xffff) / 0xffff; };
|
||||
return Array.from({ length: n }, (_, i) => fn(i / n, rand));
|
||||
}
|
||||
const cards = new Map();
|
||||
function render(profile) {
|
||||
let card = cards.get(profile.name);
|
||||
if (!card) {
|
||||
if (root.querySelector('.awaiting')) root.innerHTML = '';
|
||||
card = document.createElement('div'); card.className = 'profile-card';
|
||||
card.innerHTML = `
|
||||
<div class="profile-name"></div>
|
||||
<div class="profile-shape"></div>
|
||||
<svg viewBox="0 0 ${W} ${H}" preserveAspectRatio="none"><path d=""></path></svg>`;
|
||||
root.appendChild(card); cards.set(profile.name, card);
|
||||
}
|
||||
card.querySelector('.profile-name').textContent = profile.name;
|
||||
card.querySelector('.profile-shape').textContent = profile.shape || '';
|
||||
card.querySelector('path').setAttribute('d', curveToPath(profile.curve));
|
||||
}
|
||||
function clearAll() { cards.clear(); emptyState(); }
|
||||
function syntheticProfiles() {
|
||||
return [
|
||||
{ name: 'cpu-saturate', shape: 'sustained 1-vCPU peg (XMRig)',
|
||||
curve: gen(1, (t, r) => 0.1 + (t > 0.15 ? 0.85 : 0) + 0.05 * r()) },
|
||||
{ name: 'scan-and-dial', shape: 'SYN-style probes + dial-home',
|
||||
curve: gen(2, (t, r) => t < 0.2 ? 0.05 : 0.15 + 0.7 * Math.exp(-Math.pow((t-0.5)*4, 2)) + 0.05 * r()) },
|
||||
{ name: 'io-walk', shape: 'fs traversal + 4 KiB urandom writes',
|
||||
curve: gen(3, (t, r) => 0.2 + 0.3 * Math.sin(t * 14) + 0.4 * (t > 0.3 && t < 0.85 ? 1 : 0) + 0.05 * r()) },
|
||||
{ name: 'bursty-c2', shape: 'long idle + 3-packet egress bursts',
|
||||
curve: gen(4, (t, r) => 0.05 + (Math.sin(t * 30) > 0.95 ? 0.9 : 0) + 0.02 * r()) },
|
||||
{ name: 'low-and-slow', shape: 'minimal CPU + periodic memory churn',
|
||||
curve: gen(5, (t, r) => 0.12 + 0.08 * Math.sin(t * 6) + 0.05 * r()) },
|
||||
{ name: 'shell-resident', shape: 'one long TCP socket + command ticks',
|
||||
curve: gen(6, (t, r) => 0.08 + (Math.sin(t * 22) > 0.7 ? 0.5 : 0) + 0.03 * r()) },
|
||||
];
|
||||
}
|
||||
on('demo_start', () => syntheticProfiles().forEach(render));
|
||||
on('demo_stop', () => clearAll());
|
||||
on('attack_profile', m => {
|
||||
if (!m.name || !Array.isArray(m.curve)) return;
|
||||
render({ name: m.name, shape: m.shape || '', curve: m.curve });
|
||||
});
|
||||
emptyState();
|
||||
})();
|
||||
|
||||
// ── Chunking timeline — DEMO ONLY ────────────────────────────
|
||||
(function () {
|
||||
const ruleEl = document.getElementById('chunk-rule');
|
||||
const rowEl = document.getElementById('chunk-row');
|
||||
const axisEl = document.getElementById('chunk-axis');
|
||||
const N = 6;
|
||||
function clearAll() {
|
||||
ruleEl.innerHTML = ''; rowEl.innerHTML = ''; axisEl.innerHTML = '';
|
||||
rowEl.appendChild(awaitingNote('awaiting prediction events · turn demo on for examples'));
|
||||
}
|
||||
function buildExample() {
|
||||
const labels = ['clean', 'clean', 'armed', 'infecting', 'infected_running', 'dormant'];
|
||||
ruleEl.innerHTML = ''; rowEl.innerHTML = ''; axisEl.innerHTML = '';
|
||||
for (let i = 0; i < N; i++) ruleEl.appendChild(Object.assign(document.createElement('div'), { className: 'tick' }));
|
||||
for (let i = 0; i < N; i++) {
|
||||
const c = document.createElement('div');
|
||||
c.className = `chunk-cell ${labels[i]}`;
|
||||
c.textContent = labels[i].replace('_', ' ');
|
||||
rowEl.appendChild(c);
|
||||
}
|
||||
for (let i = 0; i < N; i++) {
|
||||
const t = document.createElement('span');
|
||||
t.textContent = `${i * 10}s`;
|
||||
axisEl.appendChild(t);
|
||||
}
|
||||
}
|
||||
on('demo_start', buildExample);
|
||||
on('demo_stop', clearAll);
|
||||
on('prediction', m => {
|
||||
// Real predictions can update individual cells.
|
||||
if (typeof m.window_idx !== 'number') return;
|
||||
const cells = rowEl.querySelectorAll('.chunk-cell');
|
||||
const cell = cells[m.window_idx];
|
||||
if (!cell) return;
|
||||
const phase = m.predicted || m.actual;
|
||||
if (!phase) return;
|
||||
cell.className = `chunk-cell ${phase}`;
|
||||
cell.textContent = phase.replace('_', ' ');
|
||||
});
|
||||
clearAll();
|
||||
})();
|
||||
|
||||
// ── Model comparison bars — DEMO ONLY (until model_metric arrives) ─
|
||||
(function () {
|
||||
const root = document.getElementById('model-bars');
|
||||
const rows = new Map();
|
||||
function emptyState() {
|
||||
root.innerHTML = '';
|
||||
root.appendChild(awaitingNote('awaiting model_metric events · turn demo on for examples'));
|
||||
}
|
||||
function ensureRow(model) {
|
||||
if (rows.has(model)) return rows.get(model);
|
||||
if (root.querySelector('.awaiting')) root.innerHTML = '';
|
||||
const row = document.createElement('div'); row.className = 'model-row';
|
||||
row.innerHTML = `
|
||||
<div class="model-name">${model}</div>
|
||||
<div class="model-track"><div class="model-fill ${model}" style="width:0%"></div></div>
|
||||
<div class="model-acc">0.000</div>`;
|
||||
root.appendChild(row);
|
||||
const entry = { row, fill: row.querySelector('.model-fill'), acc: row.querySelector('.model-acc') };
|
||||
rows.set(model, entry); return entry;
|
||||
}
|
||||
function render(model, accuracy) {
|
||||
const r = ensureRow(model);
|
||||
const visible = Math.max(0, Math.min(1, (accuracy - 0.5) / 0.5));
|
||||
r.fill.style.width = (visible * 100).toFixed(1) + '%';
|
||||
r.acc.textContent = accuracy.toFixed(3);
|
||||
}
|
||||
on('demo_start', () => {
|
||||
[ ['rnn', 0.872], ['gru', 0.911], ['lstm', 0.928], ['bert', 0.954] ]
|
||||
.forEach(([m, a]) => render(m, a));
|
||||
});
|
||||
on('demo_stop', () => { rows.clear(); emptyState(); });
|
||||
on('model_metric', m => {
|
||||
if (!m.model || typeof m.accuracy !== 'number') return;
|
||||
render(m.model, m.accuracy);
|
||||
});
|
||||
emptyState();
|
||||
})();
|
||||
|
||||
// ── KNN scatter — DEMO ONLY ───────────────────────────────────
|
||||
(function () {
|
||||
const svg = document.getElementById('knn-scatter');
|
||||
const legend = document.getElementById('knn-legend');
|
||||
const PHASES = ['clean', 'armed', 'infecting', 'infected_running', 'dormant'];
|
||||
const phaseCenters = {
|
||||
clean: [0.18, 0.75],
|
||||
armed: [0.42, 0.58],
|
||||
infecting: [0.72, 0.40],
|
||||
infected_running: [0.85, 0.18],
|
||||
dormant: [0.30, 0.30],
|
||||
};
|
||||
const ns = 'http://www.w3.org/2000/svg';
|
||||
let W = 600, H = 360;
|
||||
|
||||
legend.innerHTML = '';
|
||||
PHASES.forEach(p => {
|
||||
const li = document.createElement('span');
|
||||
const swatchVar = p === 'infected_running' ? 'running' : p;
|
||||
li.innerHTML = `<span class="swatch" style="background:var(--phase-${swatchVar})"></span>${p}`;
|
||||
legend.appendChild(li);
|
||||
});
|
||||
|
||||
function setupAxes() {
|
||||
svg.innerHTML = '';
|
||||
const ax = document.createElementNS(ns, 'line');
|
||||
ax.setAttribute('x1', 40); ax.setAttribute('y1', H - 30);
|
||||
ax.setAttribute('x2', W - 10); ax.setAttribute('y2', H - 30);
|
||||
ax.setAttribute('class', 'axis'); svg.appendChild(ax);
|
||||
const ay = document.createElementNS(ns, 'line');
|
||||
ay.setAttribute('x1', 40); ay.setAttribute('y1', 10);
|
||||
ay.setAttribute('x2', 40); ay.setAttribute('y2', H - 30);
|
||||
ay.setAttribute('class', 'axis'); svg.appendChild(ay);
|
||||
const xl = document.createElementNS(ns, 'text');
|
||||
xl.setAttribute('x', W / 2); xl.setAttribute('y', H - 8);
|
||||
xl.setAttribute('class', 'axis-label'); xl.setAttribute('text-anchor', 'middle');
|
||||
xl.textContent = 'feature 1'; svg.appendChild(xl);
|
||||
const yl = document.createElementNS(ns, 'text');
|
||||
yl.setAttribute('transform', `translate(12,${H/2}) rotate(-90)`);
|
||||
yl.setAttribute('class', 'axis-label'); yl.setAttribute('text-anchor', 'middle');
|
||||
yl.textContent = 'feature 2'; svg.appendChild(yl);
|
||||
}
|
||||
function emptyState() {
|
||||
svg.innerHTML = '';
|
||||
const t = document.createElementNS(ns, 'text');
|
||||
t.setAttribute('x', W / 2); t.setAttribute('y', H / 2);
|
||||
t.setAttribute('text-anchor', 'middle');
|
||||
t.setAttribute('class', 'axis-label');
|
||||
t.textContent = 'awaiting embedding events · turn demo on for examples';
|
||||
svg.appendChild(t);
|
||||
}
|
||||
function project(x, y) {
|
||||
const px = 40 + x * (W - 50);
|
||||
const py = (H - 30) - y * (H - 40);
|
||||
return [px, py];
|
||||
}
|
||||
function addPoint(x, y, phase) {
|
||||
const c = document.createElementNS(ns, 'circle');
|
||||
const [px, py] = project(x, y);
|
||||
c.setAttribute('cx', px.toFixed(1)); c.setAttribute('cy', py.toFixed(1));
|
||||
c.setAttribute('r', 4);
|
||||
c.setAttribute('class', `point ${phase}`);
|
||||
svg.appendChild(c);
|
||||
}
|
||||
function syntheticPoints() {
|
||||
let seed = 7;
|
||||
const rand = () => { seed = (seed * 1664525 + 1013904223) >>> 0; return ((seed & 0xffff) / 0xffff) - 0.5; };
|
||||
setupAxes();
|
||||
PHASES.forEach(p => {
|
||||
const [cx, cy] = phaseCenters[p];
|
||||
for (let i = 0; i < 14; i++) addPoint(cx + rand() * 0.16, cy + rand() * 0.16, p);
|
||||
});
|
||||
}
|
||||
on('demo_start', syntheticPoints);
|
||||
on('demo_stop', emptyState);
|
||||
on('embedding', m => {
|
||||
if (typeof m.x !== 'number' || typeof m.y !== 'number' || !m.phase) return;
|
||||
// First real embedding wipes the awaiting note and sets up axes.
|
||||
if (svg.querySelector('.axis-label') && !svg.querySelector('.point')) setupAxes();
|
||||
addPoint(m.x, m.y, m.phase);
|
||||
});
|
||||
emptyState();
|
||||
})();
|
||||
|
||||
// ── Performance scatter — DEMO ONLY ───────────────────────────
|
||||
(function () {
|
||||
const svg = document.getElementById('perf-scatter');
|
||||
const ns = 'http://www.w3.org/2000/svg';
|
||||
const W = 600, H = 360;
|
||||
const xmin = 0, xmax = 4000, ymin = 0.7, ymax = 1.0;
|
||||
const points = new Map();
|
||||
|
||||
function setupAxes() {
|
||||
svg.innerHTML = '';
|
||||
const ax = document.createElementNS(ns, 'line');
|
||||
ax.setAttribute('x1', 50); ax.setAttribute('y1', H - 30);
|
||||
ax.setAttribute('x2', W - 10); ax.setAttribute('y2', H - 30);
|
||||
ax.setAttribute('class', 'axis'); svg.appendChild(ax);
|
||||
const ay = document.createElementNS(ns, 'line');
|
||||
ay.setAttribute('x1', 50); ay.setAttribute('y1', 10);
|
||||
ay.setAttribute('x2', 50); ay.setAttribute('y2', H - 30);
|
||||
ay.setAttribute('class', 'axis'); svg.appendChild(ay);
|
||||
const xl = document.createElementNS(ns, 'text');
|
||||
xl.setAttribute('x', W / 2); xl.setAttribute('y', H - 8);
|
||||
xl.setAttribute('class', 'axis-label'); xl.setAttribute('text-anchor', 'middle');
|
||||
xl.textContent = 'inference latency (μs / window) →'; svg.appendChild(xl);
|
||||
const yl = document.createElementNS(ns, 'text');
|
||||
yl.setAttribute('transform', `translate(14,${H/2}) rotate(-90)`);
|
||||
yl.setAttribute('class', 'axis-label'); yl.setAttribute('text-anchor', 'middle');
|
||||
yl.textContent = '↑ held-out accuracy'; svg.appendChild(yl);
|
||||
}
|
||||
function emptyState() {
|
||||
points.clear(); svg.innerHTML = '';
|
||||
const t = document.createElementNS(ns, 'text');
|
||||
t.setAttribute('x', W / 2); t.setAttribute('y', H / 2);
|
||||
t.setAttribute('text-anchor', 'middle'); t.setAttribute('class', 'axis-label');
|
||||
t.textContent = 'awaiting model_perf events · turn demo on for examples';
|
||||
svg.appendChild(t);
|
||||
}
|
||||
function project(latency, accuracy) {
|
||||
const x = 50 + Math.min(1, (latency - xmin) / (xmax - xmin)) * (W - 60);
|
||||
const y = (H - 30) - Math.max(0, Math.min(1, (accuracy - ymin) / (ymax - ymin))) * (H - 40);
|
||||
return [x, y];
|
||||
}
|
||||
function render(model, latency, accuracy) {
|
||||
let g = points.get(model);
|
||||
if (!g) {
|
||||
if (!points.size) setupAxes();
|
||||
g = document.createElementNS(ns, 'g');
|
||||
const c = document.createElementNS(ns, 'circle');
|
||||
c.setAttribute('r', 7); c.setAttribute('class', 'perf-point');
|
||||
g.appendChild(c);
|
||||
const t = document.createElementNS(ns, 'text');
|
||||
t.setAttribute('class', 'perf-label');
|
||||
g.appendChild(t);
|
||||
svg.appendChild(g);
|
||||
points.set(model, g);
|
||||
}
|
||||
const [px, py] = project(latency, accuracy);
|
||||
g.querySelector('circle').setAttribute('cx', px.toFixed(1));
|
||||
g.querySelector('circle').setAttribute('cy', py.toFixed(1));
|
||||
const t = g.querySelector('text');
|
||||
t.setAttribute('x', (px + 12).toFixed(1));
|
||||
t.setAttribute('y', (py + 4).toFixed(1));
|
||||
t.textContent = model;
|
||||
}
|
||||
on('demo_start', () => {
|
||||
[
|
||||
{ model: 'knn', latency_us: 90, accuracy: 0.84 },
|
||||
{ model: 'rnn', latency_us: 380, accuracy: 0.87 },
|
||||
{ model: 'gru', latency_us: 520, accuracy: 0.91 },
|
||||
{ model: 'lstm', latency_us: 700, accuracy: 0.93 },
|
||||
{ model: 'bert', latency_us: 3200, accuracy: 0.95 },
|
||||
].forEach(p => render(p.model, p.latency_us, p.accuracy));
|
||||
});
|
||||
on('demo_stop', emptyState);
|
||||
on('model_perf', m => {
|
||||
if (!m.model || typeof m.latency_us !== 'number' || typeof m.accuracy !== 'number') return;
|
||||
render(m.model, m.latency_us, m.accuracy);
|
||||
});
|
||||
emptyState();
|
||||
})();
|
||||
|
||||
})();
|
||||
301
training/dashboard/static/index.html
Normal file
301
training/dashboard/static/index.html
Normal file
|
|
@ -0,0 +1,301 @@
|
|||
<!DOCTYPE html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<meta charset="utf-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1">
|
||||
<title>CIS490 — live</title>
|
||||
<link rel="stylesheet" href="/static/dashboard.css?v=8176c951">
|
||||
</head>
|
||||
<body>
|
||||
<header class="topbar">
|
||||
<span class="brand">CIS490</span>
|
||||
<span id="status" class="status">connecting…</span>
|
||||
<span class="spacer"></span>
|
||||
<span class="counter"><span id="scene-idx">1</span> / <span id="scene-total">1</span></span>
|
||||
<button id="prev-btn" class="ghost icon" title="Previous (← / k)">◀</button>
|
||||
<button id="next-btn" class="ghost icon" title="Next (→ / space / j)">▶</button>
|
||||
<button id="click-nav-btn" class="ghost" title="Click on the stage to advance to the next slide (c)">click-nav: off</button>
|
||||
<button id="demo-btn" class="ghost" title="Toggle local synthetic data">demo: off</button>
|
||||
</header>
|
||||
|
||||
<div class="layout">
|
||||
<div class="canvas-wrapper" id="stage-col">
|
||||
<div class="stage">
|
||||
|
||||
<!-- 1. intro -->
|
||||
<div class="stage-view" data-view="intro">
|
||||
<div class="bg-grid"></div>
|
||||
<div class="intro-block">
|
||||
<div class="intro-eyebrow">cis490 · live fleet telemetry</div>
|
||||
<div class="intro-title">behavioral<br>malware<br>detection</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- 2. collect -->
|
||||
<div class="stage-view" data-view="collect">
|
||||
<div class="metric-stack">
|
||||
<div class="metric-eyebrow">episodes ingested</div>
|
||||
<div class="metric-big" id="ingest-total">0</div>
|
||||
<div class="metric-sub">
|
||||
<span id="ingest-rate">0.0</span> / sec · last 60 s ·
|
||||
total bytes on disk: <span id="ingest-bytes">0 B</span>
|
||||
</div>
|
||||
<svg class="sparkline" id="ingest-spark" viewBox="0 0 600 120" preserveAspectRatio="none">
|
||||
<path id="ingest-spark-fill" d=""></path>
|
||||
<path id="ingest-spark-path" d=""></path>
|
||||
</svg>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- 3. hosts -->
|
||||
<div class="stage-view" data-view="hosts">
|
||||
<div class="metric-stack">
|
||||
<div class="metric-eyebrow">per-host shipping</div>
|
||||
<div class="bars" id="host-bars">
|
||||
<div class="awaiting">awaiting snapshot…</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- 4. db — episode database explorer -->
|
||||
<div class="stage-view" data-view="db">
|
||||
<div class="metric-stack metric-stack-wide">
|
||||
<div class="db-header">
|
||||
<div class="metric-eyebrow">episode database · last 200 records</div>
|
||||
<div class="db-count" id="db-count">0 of 0</div>
|
||||
</div>
|
||||
<div class="db-controls">
|
||||
<div class="db-tabs" id="db-tabs"></div>
|
||||
<input class="db-search" id="db-search" type="text"
|
||||
placeholder="filter by host / id / sha…" />
|
||||
</div>
|
||||
<div class="db-table-wrap">
|
||||
<table class="db-table">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>host</th>
|
||||
<th>episode_id</th>
|
||||
<th>received</th>
|
||||
<th>size</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody id="db-tbody"></tbody>
|
||||
</table>
|
||||
</div>
|
||||
<div class="db-detail" id="db-detail" hidden>
|
||||
<pre id="db-detail-pre"></pre>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- 5. baseline -->
|
||||
<div class="stage-view" data-view="baseline">
|
||||
<div class="metric-stack">
|
||||
<div class="metric-eyebrow">phase mix · last 5 min</div>
|
||||
<div class="phase-stack" id="phase-stack"></div>
|
||||
<div class="phase-legend" id="phase-legend"></div>
|
||||
<div class="metric-sub">awaiting <code>phase</code> events from
|
||||
the orchestrator. A clean fleet sits mostly in
|
||||
<code>clean</code>; skew toward <code>infecting</code> means
|
||||
the workload is firing.</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- 6. attacks -->
|
||||
<div class="stage-view" data-view="attacks">
|
||||
<div class="metric-stack">
|
||||
<div class="metric-eyebrow">attack envelopes · /proc signature per profile</div>
|
||||
<div class="profile-grid" id="profile-grid"></div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- 7. chunking -->
|
||||
<div class="stage-view" data-view="chunking">
|
||||
<div class="metric-stack">
|
||||
<div class="metric-eyebrow">10-second windows · model input shape</div>
|
||||
<div class="chunk-rule" id="chunk-rule"></div>
|
||||
<div class="chunk-row" id="chunk-row"></div>
|
||||
<div class="chunk-axis" id="chunk-axis"></div>
|
||||
<div class="metric-sub">each window: 100 samples (10 Hz × 10 s),
|
||||
labeled by the phase that occupies its center.</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- 8. models -->
|
||||
<div class="stage-view" data-view="models">
|
||||
<div class="metric-stack">
|
||||
<div class="metric-eyebrow">sequence models · accuracy on held-out samples</div>
|
||||
<div class="model-bars" id="model-bars"></div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- 9. knn -->
|
||||
<div class="stage-view" data-view="knn">
|
||||
<div class="metric-stack">
|
||||
<div class="metric-eyebrow">window features · 2-D projection</div>
|
||||
<svg class="scatter" id="knn-scatter" viewBox="0 0 600 360" preserveAspectRatio="xMidYMid meet"></svg>
|
||||
<div class="phase-legend" id="knn-legend"></div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- 10. perf -->
|
||||
<div class="stage-view" data-view="perf">
|
||||
<div class="metric-stack">
|
||||
<div class="metric-eyebrow">accuracy vs inference cost</div>
|
||||
<svg class="scatter" id="perf-scatter" viewBox="0 0 600 360" preserveAspectRatio="xMidYMid meet"></svg>
|
||||
<div class="metric-sub">x: μs / window (lower is better) ·
|
||||
y: held-out accuracy (higher is better).</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
</div>
|
||||
<button id="next-fab" class="fab" data-no-advance title="Next (→)">▼</button>
|
||||
</div>
|
||||
|
||||
<article class="article">
|
||||
|
||||
<section class="scene" data-stage="intro">
|
||||
<div class="prose">
|
||||
<p class="lede">Most malware doesn't look like malware in a database
|
||||
— it looks like a process behaving badly.</p>
|
||||
<p>An <strong>intrusion detection system</strong> spots the bad
|
||||
behavior; an <strong>intrusion prevention system</strong> stops it.
|
||||
Both depend on knowing what bad behavior <em>looks like</em> at the
|
||||
level of telemetry the device can actually see.</p>
|
||||
<p>This deck is the live face of the dataset we're building to teach
|
||||
a model that distinction — every panel on the left is a slice of
|
||||
real data shipping in right now.</p>
|
||||
<p class="hint">scroll, click, or → to advance</p>
|
||||
</div>
|
||||
</section>
|
||||
|
||||
<section class="scene" data-stage="collect">
|
||||
<div class="prose">
|
||||
<h2>Collecting the dataset</h2>
|
||||
<p>Each lab host on the WireGuard mesh boots a real Alpine VM, runs
|
||||
a profile-driven workload inside it, and samples
|
||||
<code>/proc/<qemu_pid></code> at 10 Hz. Every ~30 seconds
|
||||
the labeled tarball is shipped to this Pi over mTLS.</p>
|
||||
<p>The counter on the left is the running total, sourced from the
|
||||
receiver's <code>index.jsonl</code> on disk. The sparkline is the
|
||||
arrival rate over the last sixty seconds.</p>
|
||||
</div>
|
||||
</section>
|
||||
|
||||
<section class="scene" data-stage="hosts">
|
||||
<div class="prose">
|
||||
<h2>A multi-host fleet</h2>
|
||||
<p>Running the same orchestrator on multiple hosts gives novel,
|
||||
non-overlapping data per host — no central coordinator. Each host
|
||||
pulls a different slice of the manifest, so the dataset grows in
|
||||
parallel.</p>
|
||||
<p>The numbers below are absolute episode counts on disk, refreshed
|
||||
from <code>/var/lib/cis490/episodes/<host>/</code> every
|
||||
thirty seconds.</p>
|
||||
</div>
|
||||
</section>
|
||||
|
||||
<section class="scene" data-stage="db">
|
||||
<div class="prose">
|
||||
<h2>The dataset, browsable</h2>
|
||||
<p>Every row is one labeled episode tarball stored at
|
||||
<code>/var/lib/cis490/episodes/<host>/<id>.tar.zst</code>
|
||||
after the receiver verifies its SHA-256 and writes it through.</p>
|
||||
<p>Filter by host with the tabs, or grep by host / episode id /
|
||||
sha with the search box. Click a row for the full
|
||||
<code>index.jsonl</code> record. The view holds the most recent
|
||||
two hundred records — older history is on disk, indexable
|
||||
from the receiver.</p>
|
||||
</div>
|
||||
</section>
|
||||
|
||||
<section class="scene" data-stage="baseline">
|
||||
<div class="prose">
|
||||
<h2>A baseline of normal</h2>
|
||||
<p>Before we can detect a deviation, we have to know what the fleet
|
||||
looks like when it's healthy. The stacked bar shows the fraction
|
||||
of the last five minutes of fleet activity that sat in each phase
|
||||
— a healthy mix has plenty of <code>clean</code>.</p>
|
||||
<p>If the model only ever sees <code>clean</code>, it overfits to
|
||||
"everything is fine." The phase schedule fixes that by forcing the
|
||||
workload to walk through every phase on every run.</p>
|
||||
</div>
|
||||
</section>
|
||||
|
||||
<section class="scene" data-stage="attacks">
|
||||
<div class="prose">
|
||||
<h2>Linking attack to telemetry</h2>
|
||||
<p>The same six profiles run across every host, and each one
|
||||
produces a different envelope in <code>/proc</code>. A
|
||||
cryptominer pegs one core for minutes. A bursty C2 channel sits
|
||||
idle, then exhales three packets. Ransomware walks the
|
||||
filesystem and saturates I/O.</p>
|
||||
<p>The thumbnails on the left are the canonical envelopes the
|
||||
model has to learn to recognize — same axes, different shapes.
|
||||
That shape difference is what makes detection tractable.</p>
|
||||
</div>
|
||||
</section>
|
||||
|
||||
<section class="scene" data-stage="chunking">
|
||||
<div class="prose">
|
||||
<h2>Ten-second windows</h2>
|
||||
<p>Models eat fixed-size inputs. We chop each episode into
|
||||
10-second windows — 100 samples per window at 10 Hz — and
|
||||
label each window with the phase that occupies its center.</p>
|
||||
<p>Window size is a knob. Too short and the model can't see slow
|
||||
envelopes (low-and-slow malware, idle C2). Too long and you can't
|
||||
react fast enough to be a useful prevention signal. Ten seconds
|
||||
is the starting point we tune around.</p>
|
||||
</div>
|
||||
</section>
|
||||
|
||||
<section class="scene" data-stage="models">
|
||||
<div class="prose">
|
||||
<h2>Sequence models</h2>
|
||||
<p><strong>RNN, GRU, LSTM</strong> — recurrent models that read the
|
||||
window one timestep at a time and carry state forward. Cheap,
|
||||
mature, easy to interpret.</p>
|
||||
<p><strong>BERT-style transformer</strong> — the window becomes a
|
||||
sequence of "tokens"; attention captures cross-position context
|
||||
instead of accumulating it through a hidden state. More
|
||||
parameters, more compute, more room to overfit a small dataset.</p>
|
||||
<p>Same input, same labels, four different inductive biases. The
|
||||
comparison on the left is the punchline of the whole project.</p>
|
||||
</div>
|
||||
</section>
|
||||
|
||||
<section class="scene" data-stage="knn">
|
||||
<div class="prose">
|
||||
<h2>Nearest-neighbor as a sanity check</h2>
|
||||
<p>Before anything fancy: engineer summary features per window
|
||||
(mean, std, p95, slope, zero-bucket counts per channel) and run
|
||||
<strong>KNN</strong> in that feature space.</p>
|
||||
<p>If the phase clusters separate visibly in two dimensions, KNN
|
||||
already does most of the work and a deep model is only buying
|
||||
marginal improvement. If they don't separate, you've learned
|
||||
something about the feature engineering before training a single
|
||||
epoch.</p>
|
||||
</div>
|
||||
</section>
|
||||
|
||||
<section class="scene" data-stage="perf">
|
||||
<div class="prose">
|
||||
<h2>Accuracy vs complexity</h2>
|
||||
<p>Bigger models earn better numbers in the validation set — but
|
||||
they also need more parameters, more inference time, and more
|
||||
memory at the edge. The deployed model has to fit on the device
|
||||
it's protecting.</p>
|
||||
<p>The scatter on the left is the usable trade-off curve: every
|
||||
point above and to the left of where you currently sit is a
|
||||
reachable upgrade. The point in the bottom-right is a model
|
||||
you'd never ship.</p>
|
||||
</div>
|
||||
</section>
|
||||
|
||||
<div class="scene-end-spacer"></div>
|
||||
</article>
|
||||
</div>
|
||||
|
||||
<script src="/static/dashboard.js?v=9d42eb5f"></script>
|
||||
</body>
|
||||
</html>
|
||||
Loading…
Add table
Reference in a new issue