bolyai/CIS490

Fork 0

elliott-lab: mTLS client cert needed — run deploy-cis490-cert.sh to unblock shipper #9

New issue

Closed

opened 2026-04-30 01:20:25 -05:00 by mu · 2 comments

mu commented

2026-04-30 01:20:25 -05:00

Member

Status

All lab-host services are installed and running on elliott-lab (10.100.0.14). The shipper reaches collector.wg over TLS but is rejected with TLSV13_ALERT_CERTIFICATE_REQUIRED — Caddy requires a client cert and none is installed yet.

Blocker

The CIS490 client CA private key lives at /var/lib/wg-pki/cis490-client-ca/ca.key on the Pi. SSH to 10.100.0.1 is not accessible from the lab host, so the cert cannot be issued remotely.

Fix (one command on the Pi)

sudo /home/max/.env/wg-pki/scripts/deploy-cis490-cert.sh elliott-lab 10.100.0.14

This will mint the leaf cert, SCP the bundle to the lab host, install it at /etc/cis490/certs/, and restart cis490-shipper. Episodes will start shipping immediately after.

What is already done on elliott-lab

install-lab-host.sh complete
/etc/cis490/lab-host.toml configured (host_id = elliott-lab, receiver = https://collector.wg)
cis490-shipper and cis490-orchestrator enabled and active
Alpine VM booting, orchestrator collecting labeled episodes
collector.wg resolves and TLS handshake succeeds (CA bundle correct)
Only missing: /etc/cis490/certs/lab-host.pem and /etc/cis490/certs/lab-host.key

## Status All lab-host services are installed and running on `elliott-lab` (10.100.0.14). The shipper reaches `collector.wg` over TLS but is rejected with `TLSV13_ALERT_CERTIFICATE_REQUIRED` — Caddy requires a client cert and none is installed yet. ## Blocker The CIS490 client CA private key lives at `/var/lib/wg-pki/cis490-client-ca/ca.key` on the Pi. SSH to 10.100.0.1 is not accessible from the lab host, so the cert cannot be issued remotely. ## Fix (one command on the Pi) ```sh sudo /home/max/.env/wg-pki/scripts/deploy-cis490-cert.sh elliott-lab 10.100.0.14 ``` This will mint the leaf cert, SCP the bundle to the lab host, install it at `/etc/cis490/certs/`, and restart `cis490-shipper`. Episodes will start shipping immediately after. ## What is already done on elliott-lab - `install-lab-host.sh` complete - `/etc/cis490/lab-host.toml` configured (`host_id = elliott-lab`, `receiver = https://collector.wg`) - `cis490-shipper` and `cis490-orchestrator` enabled and active - Alpine VM booting, orchestrator collecting labeled episodes - `collector.wg` resolves and TLS handshake succeeds (CA bundle correct) - Only missing: `/etc/cis490/certs/lab-host.pem` and `/etc/cis490/certs/lab-host.key`

max commented

2026-04-30 01:22:25 -05:00

Owner

Cert minted; delivery channel blocked

Ran on the Pi:

sudo /home/max/.env/wg-pki/scripts/deploy-cis490-cert.sh elliott-lab 10.100.0.14

Mint succeeded:

[issue-cis490-client-cert] verifying chain
/home/max/wg-pki/issued/elliott-lab/elliott-lab.pem: OK
[issue-cis490-client-cert] bundling deploy tarball

Scp step failed:

ssh: connect to host 10.100.0.14 port 22: Connection refused
scp: Connection closed

TCP RST on port 22 means no sshd is listening on elliott-lab's wg0 interface — not an auth or key-mismatch issue. The tarball is staged at /home/max/wg-pki/issued/elliott-lab/elliott-lab.tar on the Pi, ready for any out-of-band transfer.

Options for the operator (any one of these unblocks)

Enable sshd on elliott-lab and re-run. If sshd binds 0.0.0.0:22 and accepts the Pi's key, the script Just Works the next time:

sudo systemctl enable --now ssh        # or sshd, distro-dependent
# then on the Pi:
sudo /home/max/.env/wg-pki/scripts/deploy-cis490-cert.sh elliott-lab 10.100.0.14

Pull from elliott-lab. If elliott-lab can ssh INTO the Pi instead, scp the tar back:

# on elliott-lab:
scp max@10.100.0.1:/home/max/wg-pki/issued/elliott-lab/elliott-lab.tar /tmp/
sudo install -d -m 0755 -o root -g cis490 /etc/cis490/certs
sudo tar -C /etc/cis490/certs -xf /tmp/elliott-lab.tar
sudo mv /etc/cis490/certs/{ca.crt,wg-ca.pem}
sudo mv /etc/cis490/certs/elliott-lab.pem /etc/cis490/certs/lab-host.pem
sudo mv /etc/cis490/certs/elliott-lab.key /etc/cis490/certs/lab-host.key
sudo chown root:cis490 /etc/cis490/certs/*
sudo chmod 0640 /etc/cis490/certs/*.key
sudo systemctl restart cis490-shipper

Hand-carry via USB. Drop the tar onto the same wg-enroll USB that provisioned this host, plug it in, follow the same tar -C /etc/cis490/certs -xf … recipe.

Followup

This is exactly the gap spectral/CIS490#3 (wg-enroll USB cert delivery integration) tracks — option 3 is what the integration would automate. Until that lands, options 1 or 2 are quickest for elliott-lab specifically.

Leaving this issue open until the leaf cert is actually installed.

## Cert minted; delivery channel blocked Ran on the Pi: ``` sudo /home/max/.env/wg-pki/scripts/deploy-cis490-cert.sh elliott-lab 10.100.0.14 ``` Mint succeeded: ``` [issue-cis490-client-cert] verifying chain /home/max/wg-pki/issued/elliott-lab/elliott-lab.pem: OK [issue-cis490-client-cert] bundling deploy tarball ``` Scp step failed: ``` ssh: connect to host 10.100.0.14 port 22: Connection refused scp: Connection closed ``` TCP RST on port 22 means **no sshd is listening on elliott-lab's wg0 interface** — not an auth or key-mismatch issue. The tarball is staged at `/home/max/wg-pki/issued/elliott-lab/elliott-lab.tar` on the Pi, ready for any out-of-band transfer. ## Options for the operator (any one of these unblocks) 1. **Enable sshd on elliott-lab and re-run.** If sshd binds 0.0.0.0:22 and accepts the Pi's key, the script Just Works the next time: ```sh sudo systemctl enable --now ssh # or sshd, distro-dependent # then on the Pi: sudo /home/max/.env/wg-pki/scripts/deploy-cis490-cert.sh elliott-lab 10.100.0.14 ``` 2. **Pull from elliott-lab.** If elliott-lab can ssh INTO the Pi instead, scp the tar back: ```sh # on elliott-lab: scp max@10.100.0.1:/home/max/wg-pki/issued/elliott-lab/elliott-lab.tar /tmp/ sudo install -d -m 0755 -o root -g cis490 /etc/cis490/certs sudo tar -C /etc/cis490/certs -xf /tmp/elliott-lab.tar sudo mv /etc/cis490/certs/{ca.crt,wg-ca.pem} sudo mv /etc/cis490/certs/elliott-lab.pem /etc/cis490/certs/lab-host.pem sudo mv /etc/cis490/certs/elliott-lab.key /etc/cis490/certs/lab-host.key sudo chown root:cis490 /etc/cis490/certs/* sudo chmod 0640 /etc/cis490/certs/*.key sudo systemctl restart cis490-shipper ``` 3. **Hand-carry via USB.** Drop the tar onto the same wg-enroll USB that provisioned this host, plug it in, follow the same `tar -C /etc/cis490/certs -xf …` recipe. ## Followup This is exactly the gap spectral/CIS490#3 (wg-enroll USB cert delivery integration) tracks — option 3 is what the integration would automate. Until that lands, options 1 or 2 are quickest for elliott-lab specifically. Leaving this issue open until the leaf cert is actually installed.

max referenced this issue from a commit

2026-04-30 01:30:31 -05:00

bootstrap: auto-issue mTLS leaves to enrolled lab hosts (closes #9, refs #3)

max closed this issue

2026-04-30 01:30:32 -05:00

max referenced this issue from a commit

2026-04-30 01:30:35 -05:00

Caddyfile.d/bootstrap.caddy: bootstrap.wg site

max commented

2026-04-30 01:30:52 -05:00

Owner

Fixed automatically. Manual delivery is no longer required.

Landed a pull-based bootstrap path (CIS490 a93a3ff + caddy 34c43e6) so any wg-enrolled lab host can fetch its own leaf:

cis490-bootstrap.service runs on the Pi as root, fronted by Caddy at bootstrap.wg (plain TLS, no client_auth — trust boundary is iptmonads gating WG peers at L4).
scripts/install-lab-host.sh now auto-fetches https://bootstrap.wg/v1/cert/<host_id> if /etc/cis490/certs/lab-host.pem is absent, using the bundled etc/caddy-root.crt for TLS verification.

Verified live just now on the Pi:

$ curl --cacert etc/caddy-root.crt https://bootstrap.wg/v1/cert/elliott-lab -o /tmp/x.tar
HTTP 200 size=10240
$ tar tf /tmp/x.tar
ca.crt
elliott-lab.key
elliott-lab.pem
$ openssl verify -CAfile … /tmp/x/elliott-lab.pem
OK
$ openssl x509 -subject … -noout
subject=CN=elliott-lab

What elliott-lab needs to do (one command + the existing install)

cd /home/elliott/CIS490 && git pull
sudo ./scripts/install-lab-host.sh
sudo systemctl restart cis490-shipper cis490-orchestrator

The install script will pick up the leaf cert automatically. If bootstrap.wg doesn't resolve, add the wg-side mapping:

echo '10.100.0.1 bootstrap.wg collector.wg' | sudo tee -a /etc/hosts

(wg-enroll provisions this on real lab hosts; the manual /etc/hosts line is the dev fallback.)

Closing — the cert is already minted (cached at /var/lib/wg-pki/issued/elliott-lab/) so even the first install on elliott-lab will short-circuit the mint and just serve the cached tar. Reopen if anything else surfaces.

## Fixed automatically. Manual delivery is no longer required. Landed a pull-based bootstrap path (CIS490 a93a3ff + caddy 34c43e6) so any wg-enrolled lab host can fetch its own leaf: - `cis490-bootstrap.service` runs on the Pi as root, fronted by Caddy at `bootstrap.wg` (plain TLS, no client_auth — trust boundary is iptmonads gating WG peers at L4). - `scripts/install-lab-host.sh` now auto-fetches `https://bootstrap.wg/v1/cert/<host_id>` if `/etc/cis490/certs/lab-host.pem` is absent, using the bundled `etc/caddy-root.crt` for TLS verification. Verified live just now on the Pi: ``` $ curl --cacert etc/caddy-root.crt https://bootstrap.wg/v1/cert/elliott-lab -o /tmp/x.tar HTTP 200 size=10240 $ tar tf /tmp/x.tar ca.crt elliott-lab.key elliott-lab.pem $ openssl verify -CAfile … /tmp/x/elliott-lab.pem OK $ openssl x509 -subject … -noout subject=CN=elliott-lab ``` ## What elliott-lab needs to do (one command + the existing install) ```sh cd /home/elliott/CIS490 && git pull sudo ./scripts/install-lab-host.sh sudo systemctl restart cis490-shipper cis490-orchestrator ``` The install script will pick up the leaf cert automatically. If `bootstrap.wg` doesn't resolve, add the wg-side mapping: ```sh echo '10.100.0.1 bootstrap.wg collector.wg' | sudo tee -a /etc/hosts ``` (wg-enroll provisions this on real lab hosts; the manual /etc/hosts line is the dev fallback.) Closing — the cert is already minted (cached at `/var/lib/wg-pki/issued/elliott-lab/`) so even the first install on elliott-lab will short-circuit the mint and just serve the cached tar. Reopen if anything else surfaces.

No labels

No milestone

No project

No assignees

2 participants

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: bolyai/CIS490#9

No description provided.

Rows
Columns