lab/iac
1
0
Fork 0
Code Issues 11 Pull requests 3 Projects Releases Packages Wiki Activity Actions

T10: tofu-controller sync for provider roots (flux-clusters) #25

New issue
Open
opened 2026-08-11 20:32:58 +00:00 by ginjiruu · 1 comment
ginjiruu commented 2026-08-11 20:32:58 +00:00
Owner
Copy link

Part of #2

What to build

The tofu-controller on mini (flux-clusters, infra.contrib.fluxcd.io/v1alpha2 Terraform CRs in flux-system) currently syncs only two iac roots:

kubectl get terraforms.infra.contrib.fluxcd.io -n flux-system
NAME            READY   STATUS
kanidm-config   True    Plan no changes: master@...
vault-config    False   error running Apply: ... exit status 1

forgejo, k8s, and argocd need the same wiring so each root reconciles against its target:

  • Terraform CR per root in clusters/mini/iac/ (forgejo-config, k8s-config, argocd-config), sourceRef the existing iac GitRepository (branch master), path ./forgejo|./k8s|./argocd
  • per-root ServiceAccount credentials for the tofu-controller runner to reach the target:
    • forgejo: API token (Kanidm SA in the forgejo scope or forgejo token)
    • k8s: render-only, no fleet apply — the hashicorp/kubernetes provider does not apply against fleet clusters. The k8s root renders per-app/per-cluster manifest bundles from catalog values (yamlencode/templatefile, zero cluster connectivity — per #31), then Sveltos + FluxCD apply them to the fleet (ClusterProfiles selecting by cluster label, #31/#34/#36). The runner only writes the rendered bundles into ConfigMaps/Secrets on mini. Credential: tofu-k8s Kanidm SA API token (provisioned in #35) through the agentgateway apiserver route (https://kube-mini.<domain>, T13 #28), bearer token exchanged per request (aud=kubernetes) — same uniform path as Vault/Argo/Sveltos consumers (#29/#37/#36), no in-cluster runner SA. Note: first apply depends on the gateway + apiserver OIDC flags being live; seed out-of-band like the other roots' vars.
    • argocd: ARGOCD_AUTH_TOKEN — Kanidm SA in argocd_admins (claim admin), RFC 8693 via kanidm/scripts/kanidm-login.py
  • varsFrom Secrets per root (the existing kanidm-config-vars/vault-config-vars are created out-of-band, not in flux-clusters)

Acceptance criteria

  • Terraform CRs for forgejo, k8s, argocd roots exist in clusters/mini/iac/ and reconcile Ready
  • each root applies from the iac GitRepository on mini (plan+apply in-cluster, no manual tofu runs); k8s root renders bundles + writes ConfigMaps on mini, Sveltos applies them to the fleet
  • per-root runner credentials scoped: forgejo API token, k8s tofu-k8s Kanidm SA API token through the gateway route (ConfigMap write only), argocd admin-claim SA token
  • kanidm/vault configs untouched unless fixing the vault-config failure

Blocked by

T7: Root wiring + pilot apps (integration) (#9) — the roots must validate/plan green in iac first.

The k8s root additionally depends on the Sveltos path landing first: Sveltos installed on mini (flux-clusters PR #1526 — HelmRelease + CRDs + staged ClusterProfiles), the Sveltos research (#31), and the Sveltos bootstrap (#34) — the ClusterProfiles that consume the rendered ConfigMaps must exist before k8s-config is wired.

Part of #2 ## What to build The tofu-controller on mini (flux-clusters, `infra.contrib.fluxcd.io/v1alpha2` `Terraform` CRs in `flux-system`) currently syncs only two iac roots: ``` kubectl get terraforms.infra.contrib.fluxcd.io -n flux-system NAME READY STATUS kanidm-config True Plan no changes: master@... vault-config False error running Apply: ... exit status 1 ``` forgejo, k8s, and argocd need the same wiring so each root reconciles against its target: - `Terraform` CR per root in `clusters/mini/iac/` (`forgejo-config`, `k8s-config`, `argocd-config`), `sourceRef` the existing `iac` GitRepository (branch master), path `./forgejo|./k8s|./argocd` - per-root ServiceAccount credentials for the tofu-controller runner to reach the target: - forgejo: API token (Kanidm SA in the forgejo scope or forgejo token) - k8s: **render-only, no fleet apply** — the `hashicorp/kubernetes` provider does **not** apply against fleet clusters. The k8s root **renders** per-app/per-cluster manifest bundles from catalog values (`yamlencode`/`templatefile`, zero cluster connectivity — per #31), then **Sveltos + FluxCD apply them** to the fleet (ClusterProfiles selecting by cluster label, #31/#34/#36). The runner only writes the rendered bundles into ConfigMaps/Secrets **on mini**. Credential: `tofu-k8s` Kanidm SA API token (provisioned in #35) through the agentgateway apiserver route (`https://kube-mini.<domain>`, T13 #28), bearer token exchanged per request (`aud=kubernetes`) — same uniform path as Vault/Argo/Sveltos consumers (#29/#37/#36), **no in-cluster runner SA**. Note: first apply depends on the gateway + apiserver OIDC flags being live; seed out-of-band like the other roots' vars. - argocd: `ARGOCD_AUTH_TOKEN` — Kanidm SA in `argocd_admins` (claim `admin`), RFC 8693 via `kanidm/scripts/kanidm-login.py` - `varsFrom` Secrets per root (the existing `kanidm-config-vars`/`vault-config-vars` are created out-of-band, not in flux-clusters) ## Acceptance criteria - [ ] `Terraform` CRs for forgejo, k8s, argocd roots exist in `clusters/mini/iac/` and reconcile Ready - [ ] each root applies from the iac GitRepository on mini (plan+apply in-cluster, no manual tofu runs); k8s root renders bundles + writes ConfigMaps on mini, Sveltos applies them to the fleet - [ ] per-root runner credentials scoped: forgejo API token, k8s `tofu-k8s` Kanidm SA API token through the gateway route (ConfigMap write only), argocd admin-claim SA token - [ ] kanidm/vault configs untouched unless fixing the vault-config failure ## Blocked by T7: Root wiring + pilot apps (integration) (#9) — the roots must validate/plan green in iac first. The k8s root additionally depends on the Sveltos path landing first: Sveltos installed on mini (flux-clusters PR #1526 — HelmRelease + CRDs + staged ClusterProfiles), the Sveltos research (#31), and the Sveltos bootstrap (#34) — the ClusterProfiles that consume the rendered ConfigMaps must exist before `k8s-config` is wired.
ginjiruu commented 2026-08-13 01:55:23 +00:00
Author
Owner
Copy link

Option — argocd root runner credential via agentgateway (2026-08-12): with #39 resolved and the Golden Path (#38) live, the ARGOCD_AUTH_TOKEN / kanidm-login.py approach is no longer the only option. The runner can instead present its Kanidm SA API token to an agentgateway route for Argo CD (audience: argocd, scope: openid groups); agentgateway performs the RFC 8693 exchange per request and forwards a fresh token to the Argo CD API. Same single per-actor credential as the Sveltos/Argo cluster-auth cutovers (#36/#37), no login script in the runner. Adopt or reject when implementing.

**Option — argocd root runner credential via agentgateway (2026-08-12):** with #39 resolved and the Golden Path (#38) live, the `ARGOCD_AUTH_TOKEN` / `kanidm-login.py` approach is no longer the only option. The runner can instead present its **Kanidm SA API token** to an agentgateway route for Argo CD (`audience: argocd`, `scope: openid groups`); agentgateway performs the RFC 8693 exchange per request and forwards a fresh token to the Argo CD API. Same single per-actor credential as the Sveltos/Argo cluster-auth cutovers (#36/#37), no login script in the runner. Adopt or reject when implementing.
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
master
renovate/configure
t15-multicluster-catalog
t14-vault-k8s-auth
No results found.
Labels
Clear labels   
needs-info
Waiting on reporter for more information

  
needs-triage
Maintainer needs to evaluate this issue

  
ready-for-agent
Fully specified, ready for an AFK agent

  
ready-for-human
Requires human implementation

  
wontfix
Will not be actioned

No labels
needs-info
needs-triage
ready-for-agent
ready-for-human
wontfix
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
lab/iac#25
No description provided.