lab/machine-configs
1
0
Fork 0
Code Issues 2 Pull requests Projects Releases Packages Wiki Activity Actions

003 — R6/R8: first-boot sops identity + build-time key pre-generation #3

New issue
Open
opened 2026-08-14 01:30:11 +00:00 by ginjiruu · 1 comment
ginjiruu commented 2026-08-14 01:30:11 +00:00
Owner
Copy link

003 — R6/R8: first-boot sops identity + build-time key pre-generation

Status: OPEN (unticketed — awaiting this repo's first commit + Forgejo remote)

Part of the hub architecture plan (flux-clusters / clusters#1532). Implements R6 (first-boot sops identity) and R8 (pre-generate shared CA + host keys at build time) in docs/architecture-requirements.md.

What to do

R6 — First-boot sops identity

sops needs a host identity at switch time to decrypt secrets. Plan:

  • Use the SSH host key as the age identity: sops.age.sshKeyPaths = ["/etc/ssh/ssh_host_ed25519_key"]
  • Bake pre-generated host keys into the installer ISO (the installer-<host> configurations already embed the system closure; extend to embed host keys) so decryption works on first boot, offline.
  • Lab trade-off to accept: whoever holds the ISO holds the host key.

R8 — Build-time key pre-generation

The shared apiserver CA (R7, iac#40) and host identity material must be generated before the installer ISO is built, not on the target node. Extend the build-time pre-generation set:

  • Vault-issued shared apiserver CA (key + cert) written into each cluster's sops file (secrets/<cluster>.sops.yaml, keys apiserver-ca / apiserver-ca-crt) — feeds R7's /var/lib/rancher/rke2/server/tls/ seeding before rke2-server starts.
  • Per-host ssh host keys + age keys (feeds R6).
  • WG keys (feeds R4, in 002).

This is a one-time human/Vault-IaC step at CA-rotation boundaries, not per-host.

Acceptance criteria

  • First boot on a fresh machine decrypts sops without any manual key injection
  • Installer ISO embeds pre-generated host identity (ssh host key used as age)
  • Shared apiserver CA material lands in the per-cluster sops file before ISO build
  • rke2 module lays the shared CA into /var/lib/rancher/rke2/server/tls/ before rke2-server starts (R7)

Linked

  • clusters#1532 (plan D3), iac#40 (Vault as CA authority), docs/architecture-requirements.md R6/R7/R8
# 003 — R6/R8: first-boot sops identity + build-time key pre-generation Status: OPEN (unticketed — awaiting this repo's first commit + Forgejo remote) Part of the hub architecture plan (flux-clusters / clusters#1532). Implements `R6` (first-boot sops identity) and `R8` (pre-generate shared CA + host keys at build time) in `docs/architecture-requirements.md`. ## What to do ### R6 — First-boot sops identity sops needs a host identity at switch time to decrypt secrets. Plan: - Use the SSH host key as the age identity: `sops.age.sshKeyPaths = ["/etc/ssh/ssh_host_ed25519_key"]` - **Bake pre-generated host keys into the installer ISO** (the `installer-<host>` configurations already embed the system closure; extend to embed host keys) so decryption works on first boot, offline. - Lab trade-off to accept: whoever holds the ISO holds the host key. ### R8 — Build-time key pre-generation The shared apiserver CA (R7, iac#40) and host identity material must be generated **before** the installer ISO is built, not on the target node. Extend the build-time pre-generation set: - Vault-issued shared apiserver CA (key + cert) written into each cluster's sops file (`secrets/<cluster>.sops.yaml`, keys `apiserver-ca` / `apiserver-ca-crt`) — feeds R7's `/var/lib/rancher/rke2/server/tls/` seeding before `rke2-server` starts. - Per-host ssh host keys + age keys (feeds R6). - WG keys (feeds R4, in 002). This is a one-time human/Vault-IaC step at CA-rotation boundaries, not per-host. ## Acceptance criteria - [ ] First boot on a fresh machine decrypts sops without any manual key injection - [ ] Installer ISO embeds pre-generated host identity (ssh host key used as age) - [ ] Shared apiserver CA material lands in the per-cluster sops file before ISO build - [ ] rke2 module lays the shared CA into `/var/lib/rancher/rke2/server/tls/` before `rke2-server` starts (R7) ## Linked - clusters#1532 (plan D3), iac#40 (Vault as CA authority), `docs/architecture-requirements.md` R6/R7/R8
ginjiruu commented 2026-08-14 01:31:23 +00:00
Author
Owner
Copy link

LOE estimate: 3–5 days (split into two independent halves)

R6 half — ~1 day, independent, start now

  • sops.age.sshKeyPaths = ["/etc/ssh/ssh_host_ed25519_key"] in modules/base.nix is a one-liner.
  • The real work is embedding pre-generated host keys into the installer ISO. The installer-<host> configs already embed the closure (flake.nix mkInstaller); extending modules/installer.nix to also drop the pre-generated /etc/ssh/ssh_host_ed25519_key into the ISO is the delta. Accept the documented trade-off: whoever holds the ISO holds the host key.
  • Verify with the existing libvirt test flow (task vm-run → boot installed disk): first boot should decrypt sops offline with zero manual key injection — that's the R6 acceptance box, and the VM harness we already have makes it testable.

R7/R8 half — ~2–4 days, depends on iac#40 (Vault CA)

  • Build-time generation of the shared apiserver CA (key + cert) into secrets/<cluster>.sops.yaml as apiserver-ca / apiserver-ca-crt — a human/Vault-IaC step, not per-host code.
  • The fiddly bit is R7: modules/rke2.nix must lay the CA material into /var/lib/rancher/rke2/server/tls/ (server-ca.key, server-ca.crt + support files) before rke2-server starts. This means either a preStart/ordering unit in the rke2-${role} service (rke2.nix:154) or extending rke2-install.service (rke2.nix:175) — needs a careful look at rke2's actual TLS layout so the first server bootstraps with the shared CA rather than generating its own (the exact thing R7 warns about). Realistically this needs one boot-test iteration to confirm.
  • One-time cost at CA-rotation boundaries only, not per host.

R9 note (deploy key) — small, but flagging a mismatch

  • ~0.25 day for the sops-secret wiring, plus a human step (create the read-capable deploy key on Forgejo for pull.nix).
  • Heads-up: modules/base.nix sets deploy.flakeUrl = git+ssh://git@git.animeteamspeak.moe/ginjiruu/machine-configs (base.nix:82), but the actual repo/remote is lab/machine-configs. The flake URL must be updated or nixos-pull will fail on the timer. Worth fixing in the same change.

Recommendation: start the R6 half now; hold R7/R8 until iac#40.

## LOE estimate: 3–5 days (split into two independent halves) ### R6 half — ~1 day, independent, start now - `sops.age.sshKeyPaths = ["/etc/ssh/ssh_host_ed25519_key"]` in `modules/base.nix` is a one-liner. - The real work is **embedding pre-generated host keys into the installer ISO**. The `installer-<host>` configs already embed the closure (flake.nix `mkInstaller`); extending `modules/installer.nix` to also drop the pre-generated `/etc/ssh/ssh_host_ed25519_key` into the ISO is the delta. Accept the documented trade-off: whoever holds the ISO holds the host key. - Verify with the existing libvirt test flow (`task vm-run` → boot installed disk): first boot should decrypt sops offline with zero manual key injection — that's the R6 acceptance box, and the VM harness we already have makes it testable. ### R7/R8 half — ~2–4 days, depends on iac#40 (Vault CA) - Build-time generation of the shared apiserver CA (key + cert) into `secrets/<cluster>.sops.yaml` as `apiserver-ca` / `apiserver-ca-crt` — a human/Vault-IaC step, not per-host code. - The fiddly bit is R7: `modules/rke2.nix` must lay the CA material into `/var/lib/rancher/rke2/server/tls/` (`server-ca.key`, `server-ca.crt` + support files) **before** `rke2-server` starts. This means either a `preStart`/ordering unit in the `rke2-${role}` service (rke2.nix:154) or extending `rke2-install.service` (rke2.nix:175) — needs a careful look at rke2's actual TLS layout so the first server bootstraps *with* the shared CA rather than generating its own (the exact thing R7 warns about). Realistically this needs one boot-test iteration to confirm. - One-time cost at CA-rotation boundaries only, not per host. ### R9 note (deploy key) — small, but flagging a mismatch - ~0.25 day for the sops-secret wiring, plus a human step (create the read-capable deploy key on Forgejo for `pull.nix`). - **Heads-up:** `modules/base.nix` sets `deploy.flakeUrl = git+ssh://git@git.animeteamspeak.moe/ginjiruu/machine-configs` (base.nix:82), but the actual repo/remote is `lab/machine-configs`. The flake URL must be updated or `nixos-pull` will fail on the timer. Worth fixing in the same change. **Recommendation:** start the R6 half now; hold R7/R8 until iac#40.
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
No results found.
Labels
Clear labels   
needs-triage
New issue, awaiting triage

No labels
needs-triage
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
lab/machine-configs#3
No description provided.