lab/iac
1
0
Fork 0
Code Issues 11 Pull requests 3 Projects Releases Packages Wiki Activity Actions

Kanidm: provision per-cluster OAuth2 clients + machine SAs for cluster tooling #35

New issue
Open
opened 2026-08-12 16:50:02 +00:00 by ginjiruu · 0 comments
ginjiruu commented 2026-08-12 16:50:02 +00:00
Owner
Copy link

Kanidm: provision per-cluster OAuth2 clients + machine SAs for cluster tooling

This is the credential source for the Golden Path (#38) — per-request RFC 8693 token exchange at the k8s-hub authz-bridge (formerly agentgateway). Sveltos and Argo CD both authenticate to remote clusters with a Kanidm SA API token (presented to the k8s-hub, exchanged per request), instead of per-tool static SA credentials. #39 resolved (2026-08-12) — Kanidm 1.11.0-itt-fix emits the RFC 8693 URN and the exchange is verified end-to-end through the gateway. (2026-09: the gateway migrated from agentgateway to the Cilium Gateway API k8s-hub.)

Sveltos and Argo CD both need to authenticate to remote clusters. The k8s apiserver OIDC flags (flux-clusters #11/#1522 / iac #11) will validate Kanidm tokens, so the tooling should use the same RFC 8693 flow Argo CD already uses (Kanidm SA API token → token via kanidm-login.py), instead of per-tool static SA credentials.

This is the shared prerequisite for all three cluster-auth consumers: #36 (Sveltos), #37 (Argo CD), and #29 (Vault kubernetes-auth TokenReview).

What to build

  1. Per-cluster Kanidm OAuth2 clients (one per apiserver) — each apiserver OIDC issuer has its own client (https://auth.animeteamspeak.moe/oauth2/openid/minicluster, clientID minicluster; …/portablecluster, clientID portablecluster) — provisioned in kanidm/oauth2.tf. The groups_name scope must be granted to the cluster's groups so the exchanged token's groups_name claim feeds the apiserver's --oidc-groups-claim=groups_name (no custom claim maps). Each clientID is also the exchange audience the authz-bridge requests — the exchanged id_token lands in the apiserver's own audience. (2026-09: replaces the original single shared kubernetes client design.)
  2. Machine service accounts — one per actor, not per tool — vault (Vault's TokenReview caller, #29), tofu-k8s (tofu-controller k8s root runner, #25), sveltos-<cluster>, argocd-<cluster>, and any future autonomous system (agents, homebox, sonarr) get their own Kanidm SA holding the relevant capability groups (ADR-0006: machines are just principals holding the same groups). Each SA's API token is the single long-lived credential for that actor — the only secret, held by the actor (in its kubeconfig/cluster secret), exchanged per request by the k8s-hub authz-bridge.
  3. Idempotent provisioning script — mirror the ADR-0005 newOAuth.sh / kanidm-login.py pattern: create the clients, scope maps, and SAs declaratively, no hand-run kanidm commands that drift. Must include the per-group groups_name scope-map entry for each cluster client (without it, exchanges with scope=openid groups_name fail access_denied — verified 2026-08-12).

Design questions (resolved)

  • Client type: a public (secret-less) client works for machine token exchange — verified live (under agentgateway, since replaced): clientAuth: { clientId: kubernetes, method: clientSecretPost } with no secret is accepted (same as the vault-cli public-client precedent, ADR-0005). The apiserver only needs JWKS + discovery; no client secret anywhere.
  • Per-cluster vs shared SA: per-actor SAs (sveltos-<cluster>, argocd-<cluster>). Per-actor revocation and per-actor audit; matches the capability-group model (ADR-0006). A machine is just a principal holding the same groups.
  • Where provisioning lives: this repo's kanidm/ root (the OAuth2 clients per ADR-0005 pattern); the ~90 user-facing UI groups in kanidm/groups.tf are untouched.

Resolved — Kanidm conformance (#39)

agentgateway previously rejected Kanidm's RFC 8693 response (issued_token_type: "AccessToken" was not the standard URN). Fixed and verified (2026-08-12): Kanidm 1.11.0-itt-fix emits the URN and the exchange returns 200 through agentgateway. Tracked internally, never upstreamed: #39.

Acceptance criteria

  • Per-cluster OAuth2 clients (minicluster, portablecluster) exist; discovery URLs resolve; groups_name scope granted to the clusters' groups
  • kanidm-login.py --client minicluster --api-token <SA> yields a token carrying the groups_name claim
  • Provisioning is scripted/idempotent
  • (Golden-Path) the k8s-hub authz-bridge (audience = the cluster's apiserver client) exchanges the SA token and forwards a token the apiserver accepts (no longer gated — #39 resolved)

Depends on / linked

  • flux-clusters #11 / #1522 (apiserver OIDC flags — must reference the per-cluster client-ids and the groups_name claim)
  • #38 (Golden Path decision) / #39 (Kanidm conformance blocker — resolved)
  • #34 (Sveltos bootstrap) → OIDC switch #36
  • #37 (Argo CD in-place cluster-auth migration)
## Kanidm: provision per-cluster OAuth2 clients + machine SAs for cluster tooling > This is the credential source for the **Golden Path (#38)** — per-request RFC 8693 token exchange at the k8s-hub authz-bridge (formerly agentgateway). Sveltos and Argo CD both authenticate to remote clusters with a Kanidm SA API token (presented to the k8s-hub, exchanged per request), instead of per-tool static SA credentials. **#39 resolved (2026-08-12)** — Kanidm `1.11.0-itt-fix` emits the RFC 8693 URN and the exchange is verified end-to-end through the gateway. *(2026-09: the gateway migrated from agentgateway to the Cilium Gateway API k8s-hub.)* Sveltos and Argo CD both need to authenticate to remote clusters. The k8s apiserver OIDC flags (flux-clusters #11/#1522 / iac #11) will validate Kanidm tokens, so the tooling should use the same RFC 8693 flow Argo CD already uses (Kanidm SA API token → token via `kanidm-login.py`), instead of per-tool static SA credentials. This is the shared prerequisite for all three cluster-auth consumers: #36 (Sveltos), #37 (Argo CD), and #29 (Vault kubernetes-auth TokenReview). ## What to build 1. **Per-cluster Kanidm OAuth2 clients** (one per apiserver) — each apiserver OIDC issuer has its own client (`https://auth.animeteamspeak.moe/oauth2/openid/minicluster`, clientID `minicluster`; `…/portablecluster`, clientID `portablecluster`) — provisioned in `kanidm/oauth2.tf`. The `groups_name` scope must be granted to the cluster's groups so the exchanged token's `groups_name` claim feeds the apiserver's `--oidc-groups-claim=groups_name` (no custom claim maps). Each clientID is also the exchange **audience** the authz-bridge requests — the exchanged id_token lands in the apiserver's own audience. *(2026-09: replaces the original single shared `kubernetes` client design.)* 2. **Machine service accounts — one per actor, not per tool** — `vault` (Vault's TokenReview caller, #29), `tofu-k8s` (tofu-controller k8s root runner, #25), `sveltos-<cluster>`, `argocd-<cluster>`, and any future autonomous system (agents, homebox, sonarr) get their **own** Kanidm SA holding the relevant capability groups (ADR-0006: machines are just principals holding the same groups). Each SA's **API token is the single long-lived credential** for that actor — the only secret, held by the actor (in its kubeconfig/cluster secret), exchanged per request by the k8s-hub authz-bridge. 3. **Idempotent provisioning script** — mirror the ADR-0005 `newOAuth.sh` / `kanidm-login.py` pattern: create the clients, scope maps, and SAs declaratively, no hand-run kanidm commands that drift. Must include the per-group `groups_name` scope-map entry for each cluster client (without it, exchanges with `scope=openid groups_name` fail `access_denied` — verified 2026-08-12). ## Design questions (resolved) - **Client type:** a **public (secret-less) client** works for machine token exchange — verified live (under agentgateway, since replaced): `clientAuth: { clientId: kubernetes, method: clientSecretPost }` with no secret is accepted (same as the `vault-cli` public-client precedent, ADR-0005). The apiserver only needs JWKS + discovery; no client secret anywhere. - **Per-cluster vs shared SA:** **per-actor SAs** (`sveltos-<cluster>`, `argocd-<cluster>`). Per-actor revocation and per-actor audit; matches the capability-group model (ADR-0006). A machine is just a principal holding the same groups. - **Where provisioning lives:** this repo's `kanidm/` root (the OAuth2 clients per ADR-0005 pattern); the ~90 user-facing UI groups in `kanidm/groups.tf` are untouched. ## Resolved — Kanidm conformance (#39) agentgateway previously rejected Kanidm's RFC 8693 response (`issued_token_type: "AccessToken"` was not the standard URN). Fixed and verified (2026-08-12): Kanidm `1.11.0-itt-fix` emits the URN and the exchange returns 200 through agentgateway. Tracked internally, never upstreamed: **#39**. ## Acceptance criteria - [ ] Per-cluster OAuth2 clients (`minicluster`, `portablecluster`) exist; discovery URLs resolve; `groups_name` scope granted to the clusters' groups - [ ] `kanidm-login.py --client minicluster --api-token <SA>` yields a token carrying the `groups_name` claim - [ ] Provisioning is scripted/idempotent - [ ] (Golden-Path) the k8s-hub authz-bridge (audience = the cluster's apiserver client) exchanges the SA token and forwards a token the apiserver accepts (no longer gated — #39 resolved) ## Depends on / linked - flux-clusters #11 / #1522 (apiserver OIDC flags — must reference the per-cluster client-ids and the `groups_name` claim) - **#38** (Golden Path decision) / **#39** (Kanidm conformance blocker — **resolved**) - #34 (Sveltos bootstrap) → OIDC switch #36 - #37 (Argo CD in-place cluster-auth migration)
ginjiruu changed title from Kanidm: provision a kubernetes OAuth2 client + machine SAs for cluster tooling to Kanidm: provision per-cluster OAuth2 clients + machine SAs for cluster tooling 2026-09-17 23:15:28 +00:00
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
master
renovate/configure
t15-multicluster-catalog
t14-vault-k8s-auth
No results found.
Labels
Clear labels   
needs-info
Waiting on reporter for more information

  
needs-triage
Maintainer needs to evaluate this issue

  
ready-for-agent
Fully specified, ready for an AFK agent

  
ready-for-human
Requires human implementation

  
wontfix
Will not be actioned

No labels
needs-info
needs-triage
ready-for-agent
ready-for-human
wontfix
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
lab/iac#35
No description provided.