bolyai/CIS490
3
0
Fork 0
Code Issues 5 Pull requests Projects 1 Releases Packages Wiki Activity Actions

live demo: slow A100 inference cadence to ~0.4 events/sec

Browse source 
Was 280ms (~3.5 events/sec) — way too fast for real fleet
inference. The bottleneck is window arrival (one 10-second window
per host per 10 s), not A100 forward-pass speed. With ~3 hosts × 5
models that's ~1.5 events/sec real ceiling, so demo at 2500ms
(~0.4/sec) reads honest without claiming impossible throughput.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Max Gorog 2026-05-08 16:52:04 -05:00
parent 3b96537b3e
commit ab21217261
2 changed files with 7 additions and 2 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

7
training/dashboard/static/dashboard.js
View file

@ -2272,6 +2272,11 @@ def train_nn(*, model, X_train, y_train, X_val, y_val,
const PROFILES = ['cpu-saturate', 'ransomware-lite', 'bursty-c2', 'fork-bomb', 'crypto-miner'];
const PHASES = ['clean', 'armed', 'infecting', 'infected_running', 'dormant'];
let counter = 0;
// Inference rate is bounded by how often new 10-second windows
// close on the fleet, not by A100 forward-pass speed. With ~3
// hosts × one window every 10 s × 5 models = ~1.5 events/sec
// total — so the demo ticks every 2.5 s to stay honest. Going
// any faster is misleading about real deployment cadence.
demoTimer = setInterval(() => {
const m = MODELS[counter % MODELS.length];
counter++;
@ -2292,7 +2297,7 @@ def train_nn(*, model, X_train, y_train, X_val, y_val,
window_idx: counter,
t_wall: Date.now() / 1000,
});
}, 280);
}, 2500);
}
function demoStop() {
demoActive = false;

2
training/dashboard/static/index.html
View file

@ -1313,6 +1313,6 @@
</article>
</div>
<script src="/static/dashboard.js?v=48576a1a"></script>
<script src="/static/dashboard.js?v=aa585e5d"></script>
</body>
</html>