As preparation for the module system, and in hopes it will be faster
than and replace the Nix CI. Secondary build jobs do not block merging.
Also makes macOS aarch64 a secondary build job on the PR level, where it
is the current bottleneck.
---------
Co-authored-by: Mac Malone <tydeu@hatpress.net>