lab/machine-configs
1
0
Fork 0
Code Issues 2 Pull requests Projects Releases Packages Wiki Activity Actions

002 — R4/R5: WireGuard peer + tunnel client for firewalled/dev sites #2

New issue
Open
opened 2026-08-14 01:30:08 +00:00 by ginjiruu · 1 comment
ginjiruu commented 2026-08-14 01:30:08 +00:00
Owner
Copy link

002 — R4/R5: WireGuard peer + tunnel client for firewalled/dev sites

Status: OPEN (unticketed — awaiting this repo's first commit + Forgejo remote)

Part of the hub architecture plan (flux-clusters / clusters#1532, decisions D2/D10). Implements R4 (WireGuard peer) and R5 (tunnel client) in docs/architecture-requirements.md. Depends on the network plan (clusters#1533) for the address plan.

What to do

R4 — WireGuard peer per host

Each host runs a WG peer:

  • Hub (portable) is the coordination point with a public endpoint. This host config (portable's nixos) carries the WG server side.
  • Spokes dial out with PersistentKeepalive; the firewalled site dials out by requirement.
  • Private keys per host → sops (age), matching the existing rke2-token-<cluster> pattern.
  • Hub-side public keys + the address plan come from clusters#1533 (overlay subnet vs routed site LANs).
  • Once #1533 lands, define the wireguard interface config here: addresses, endpoint, allowed IPs, routes for pod CIDRs (ClusterMesh fabric).

R5 — Tunnel client for firewalled/dev sites

For TCP-egress-only or dev/L4 access, a frp/rathole client at OS level:

  • systemd unit + config, token via sops (same pattern as R4 keys).
  • Must come up before the cluster is managed (agentgateway reaches the cluster through it) — OS config, not a workload.
  • Whether a given site needs it depends on #1533 / #1536 (per-site WG/UDP validation).

Acceptance criteria

  • WG interface up on hub + spokes per #1533 address plan
  • Spoke dials out, persistent tunnel, survives reboot (systemd)
  • Pod CIDRs routable across sites (ClusterMesh fabric ready)
  • frp/rathole client present on sites that need it, token via sops

Linked

  • clusters#1533 (network plan — address/fallback decisions), clusters#1536 (WG/UDP per-site validation), clusters#1532 (plan D2/D10)
# 002 — R4/R5: WireGuard peer + tunnel client for firewalled/dev sites Status: OPEN (unticketed — awaiting this repo's first commit + Forgejo remote) Part of the hub architecture plan (flux-clusters / clusters#1532, decisions D2/D10). Implements `R4` (WireGuard peer) and `R5` (tunnel client) in `docs/architecture-requirements.md`. Depends on the network plan (clusters#1533) for the address plan. ## What to do ### R4 — WireGuard peer per host Each host runs a WG peer: - **Hub (portable)** is the coordination point with a public endpoint. This host config (portable's nixos) carries the WG server side. - **Spokes dial out** with `PersistentKeepalive`; the firewalled site dials out by requirement. - Private keys per host → sops (age), matching the existing `rke2-token-<cluster>` pattern. - Hub-side public keys + the address plan come from clusters#1533 (overlay subnet vs routed site LANs). - Once #1533 lands, define the `wireguard` interface config here: addresses, endpoint, allowed IPs, routes for pod CIDRs (ClusterMesh fabric). ### R5 — Tunnel client for firewalled/dev sites For TCP-egress-only or dev/L4 access, a frp/rathole client at OS level: - systemd unit + config, token via sops (same pattern as R4 keys). - Must come up **before** the cluster is managed (agentgateway reaches the cluster through it) — OS config, not a workload. - Whether a given site needs it depends on #1533 / #1536 (per-site WG/UDP validation). ## Acceptance criteria - [ ] WG interface up on hub + spokes per #1533 address plan - [ ] Spoke dials out, persistent tunnel, survives reboot (systemd) - [ ] Pod CIDRs routable across sites (ClusterMesh fabric ready) - [ ] frp/rathole client present on sites that need it, token via sops ## Linked - clusters#1533 (network plan — address/fallback decisions), clusters#1536 (WG/UDP per-site validation), clusters#1532 (plan D2/D10)
ginjiruu commented 2026-08-14 01:31:22 +00:00
Author
Owner
Copy link

LOE estimate: 3–6 days

The heaviest ticket, and blocked on upstream decisions rather than on this repo's code.

Blocking dependencies

  • clusters#1533 (network plan) — the address plan decides the actual shape: overlay WG subnet vs routed site LANs, hub endpoint, allowed IPs, pod-CIDR routes for the ClusterMesh fabric. The WG interface config literally cannot be written until this lands.
  • clusters#1536 (per-site WG/UDP validation) — decides whether each site needs the R5 tunnel client at all.

Because of these, this is spike-plus-implement, not plumbing — budget toward the 6-day end.

Implementation sketch (once unblocked)

  • modules/wireguard.nix: declare the wireguard interface (addresses, endpoint, PersistentKeepalive on spokes), per-host private key via sops following the existing rke2-token-<cluster> pattern in modules/rke2.nix (rke2.nix:136), hub side on the portable host.
  • modules/tunnel.nix: frp/rathole client — systemd unit + config, token via sops, wantedBy = multi-user.target so it comes up before the cluster is managed (it's OS config, not a workload — matches the ticket's requirement).
  • Boot-order note: WG/tunnel must be up before rke2-server for firewalled sites, so the rke2-${role} unit's after= chain (rke2.nix:158) may need the WG/tunnel units prepended.

Risks

  • The pod-CIDR routing across sites is the fiddliest part — worth a standalone spike/test (that's effectively #1536).
  • sops key lifecycle: WG keys are per-host, so they multiply the sops secret count — expect the secrets/<cluster>.sops.yaml files to grow.

Recommendation: don't start until #1533/#1536 are resolved; 001 and 003-R6 are the parallel work that can proceed.

## LOE estimate: 3–6 days The heaviest ticket, and **blocked** on upstream decisions rather than on this repo's code. ### Blocking dependencies - **clusters#1533 (network plan)** — the address plan decides the actual shape: overlay WG subnet vs routed site LANs, hub endpoint, allowed IPs, pod-CIDR routes for the ClusterMesh fabric. The WG interface config literally cannot be written until this lands. - **clusters#1536 (per-site WG/UDP validation)** — decides *whether* each site needs the R5 tunnel client at all. Because of these, this is **spike-plus-implement, not plumbing** — budget toward the 6-day end. ### Implementation sketch (once unblocked) - `modules/wireguard.nix`: declare the `wireguard` interface (addresses, endpoint, `PersistentKeepalive` on spokes), per-host private key via sops following the existing `rke2-token-<cluster>` pattern in `modules/rke2.nix` (rke2.nix:136), hub side on the portable host. - `modules/tunnel.nix`: frp/rathole client — systemd unit + config, token via sops, `wantedBy = multi-user.target` so it comes up **before** the cluster is managed (it's OS config, not a workload — matches the ticket's requirement). - Boot-order note: WG/tunnel must be up before `rke2-server` for firewalled sites, so the `rke2-${role}` unit's `after=` chain (rke2.nix:158) may need the WG/tunnel units prepended. ### Risks - The pod-CIDR routing across sites is the fiddliest part — worth a standalone spike/test (that's effectively #1536). - sops key lifecycle: WG keys are per-host, so they multiply the sops secret count — expect the `secrets/<cluster>.sops.yaml` files to grow. **Recommendation:** don't start until #1533/#1536 are resolved; 001 and 003-R6 are the parallel work that can proceed.
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
No results found.
Labels
Clear labels   
needs-triage
New issue, awaiting triage

No labels
needs-triage
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
lab/machine-configs#2
No description provided.