lab/authz-bridge
1
0
Fork 0
Code Issues 9 Pull requests 6 Projects Releases Packages Wiki Activity Actions 12

Spec: Shared in-memory state model (Registry + Snapshot) for the ext_authz bridge #1

New issue
Closed
opened 2026-09-13 20:32:04 +00:00 by ginjiruu · 2 comments
ginjiruu commented 2026-09-13 20:32:04 +00:00
Owner
Copy link

Problem Statement

An App team wants to protect its backends (API servers, web servers, dashboards) behind any Gateway API implementation that supports the GEP-1494 ExternalAuth filter, using per-App authentication and no gateway-specific config. The gateway's ExternalAuth filter calls out to the Bridge over ext_authz; for the Bridge to answer those callouts it must know, per request: which App the request belongs to, that App's resolved IdP configuration (token/authorization endpoints, signing keys), its client-secret reference, and its per-Flow policy behaviour.

Today there is no in-cluster mechanism to turn the App team's CRDs (OauthClient, OidcPolicy, TokenExchangePolicy) into that resolved, validated, ready-to-use, per-App state, and to make it consistently available to every replica that serves traffic — without each replica redundantly re-resolving IdP config, re-reading Secrets, or the operator hand-wiring gateway config.

Solution

The shared in-memory state model, built and distributed by the operator's controller:

  • The App team declares, per App: an OauthClient (clientID, Kanidm issuer URL, client-secret reference, callbackURLs, hostnames) and one policy per Flow it runs (OidcPolicy, TokenExchangePolicy). A Flow is active iff its policy object exists.
  • The leader watches those CRDs, structurally validates them, resolves the IdP config (discovery document → token/authorization endpoints, JWKS signing keys), and writes the resolved, validated Apps into a single managed Secret in the Bridge's namespace — the Registry.
  • Every replica mirrors the Registry into a local in-memory Snapshot (rebuilt atomically on any change) and watches the referenced client Secrets, so each replica serves from a consistent, resolved, ready-to-use view. App resolution is by hostname (one App per hostname, never overlapping).
  • Fail-closed: a structurally-invalid App is excluded from the Registry and marked Degraded; a missing referenced Secret keeps the App present but degraded (served as 5xx), never mis-authorizing.
  • The operator gets a single in-cluster object (kubectl get secret) that shows exactly what is being served, 1× IdP load, secrets staying in the per-App Secrets, and last-good state surviving failover.

User Stories

  1. As an App team, I want to declare a per-App OauthClient with its clientID, Kanidm issuer URL, client-secret reference, callback URLs, and hostnames, so that I protect my App's backends without any gateway-specific config.
  2. As an App team, I want each of my Apps to be isolated (its own Client in its own namespace, credentials never shared), so that Kanidm's per-client issuer, keys, and tokens only ever work for the intended App.
  3. As an App team, I want to declare one OidcPolicy to enable the human/browser OIDC Flow for my App, so that users can sign in and I can inject their identity into my backend.
  4. As an App team, I want to declare one TokenExchangePolicy to enable the machine Exchange Flow (RFC 8693) for my App, so that service accounts can exchange their API token for tokens without user interaction.
  5. As an App team, I want a Flow to be active for my App exactly when its policy object exists, so that I enable/disable Flows by creating/deleting the policy, with no separate "enabled" flag.
  6. As an App team, I want my App's hostnames to be the single key that maps an incoming request to my App, so that the bridge resolves "which App is this request for" from the request's Host header.
  7. As an operator, I want an App whose hostnames overlap another App's to be excluded from the Registry, so that hostname resolution is unambiguous and one App per hostname is enforced.
  8. As an operator, I want a policy whose clientRef points to a non-existent Client to be excluded (structurally invalid), so that broken config never silently serves a half-configured App.
  9. As an operator, I want the leader to resolve each Client's discovery document from its issuer URL into the token and authorization endpoints, so that the bridge has ready-to-use endpoint URLs without a per-request fetch.
  10. As an operator, I want the leader to fetch and store the IdP's signing keys (JWKS) in the resolved state, so that the bridge can validate tokens without a per-request key fetch.
  11. As an operator, I want the client secret to be referenced (namespace/name/key) in the Registry rather than copied into it, so that secrets stay in the per-App Secrets and a rotation does not require re-materializing the Registry.
  12. As an operator, I want every replica to watch the referenced client Secrets and hold the bytes locally, so that a replica can complete an OIDC token exchange without reading a Secret per request.
  13. As an operator, I want the leader to be the only process that talks to the IdP (discovery + JWKS) and reads the CRDs, so that IdP load is 1× regardless of replica count.
  14. As an operator, I want every replica to mirror the single Registry into its own in-memory Snapshot, so that all replicas serve from one consistent source of truth.
  15. As an operator, I want a change to the Registry or a referenced Secret to rebuild the Snapshot atomically (immutable swap), so that a serving replica can never read a half-updated state.
  16. As an operator, I want the read path (a Check against the Snapshot) to be lock-free (a single pointer load), so that serving stays fast.
  17. As an operator, I want the hostname index and the App set to be built together, so that they can never disagree.
  18. As an operator, I want a structurally-invalid App to be excluded from the Registry and its CRD marked Degraded, so that I can see why an App is not being served.
  19. As an operator, I want a deleted referenced Secret to keep the App in the Registry with secretReady=false (served as 5xx) rather than removing it, so that a broken config surfaces as "unavailable," not a false "forbidden."
  20. As an operator, I want the 5xx (missing Secret) to take effect immediately on every replica (each loses the bytes on its own watch), so that the failure is consistent and not gated on the leader.
  21. As an operator, I want a recreated Secret to flip secretReady=true and resume normal operation, so that a transient Secret outage self-heals.
  22. As an operator, I want the leader to refresh the JWKS on a TTL and on any CRD change, so that rotated signing keys are picked up.
  23. As an operator, I want the leader to keep the last-good JWKS if a refresh fetch fails, so that a transient IdP outage does not break token validation.
  24. As an operator, I want only the leader to write the Registry and only side-effects to be leader-gated, so that state updates pause during failover but last-good serving continues.
  25. As an operator, I want the Registry to persist (managed Secret in etcd), so that the last-good state survives replica restarts and leader failover.
  26. As an operator, I want to run kubectl get secret on the managed Registry to see exactly which resolved Apps every replica is serving, so that I can debug what the bridge is doing without reading process memory.
  27. As an operator, I want each App's CRD status to reflect its Available / Degraded state and observed generation, so that per-App health is visible in the standard place.
  28. As an App team, I want the resolved per-App entry to carry my OIDC session, injection (claim→header), and logout config, so that the bridge applies my per-App behaviour.
  29. As an App team, I want the resolved per-App entry to carry my Exchange policy's scopes and subject-token source (header/prefix/token type), so that the bridge knows where to read the presented service-account token from and which scopes to request.
  30. As an App team, I want the Exchange Flow's subject token to be forwarded (opaque, access_token type) to the token endpoint rather than validated in-bridge, so that the bridge does not hold the HMAC secret and Kanidm remains the validator.
  31. As an operator, I want the OIDC policy scopes to default to openid, so that a minimal policy still requests an identity token.
  32. As an operator, I want a replica to derive its Snapshot's per-App "bytes present" from its own Secret watch (not from the leader's flag), so that the 5xx behaviour is correct even during leader latency.
  33. As an operator, I want the managed Registry to be a single object (one watch, one atomic write), so that config updates are atomic and there is one thing to inspect.
  34. As an operator, I want the state to be per-replica in memory with the Registry as the persistent source, so that a replica can serve immediately from its mirror without a leader round-trip.
  35. As an App team, I want a previously-working App whose Secret is deleted to be served as 5xx and marked Degraded (not removed), so that my users see "unavailable," not "forbidden," and I get a status signal.
  36. As an operator, I want the Exchange Flow target (the RFC 8693 audience) to be implicitly the Client's own clientID, so that machine tokens land in the App's own audience with no extra config.
  37. As an operator, I want a new replica to build its Snapshot by watching the Registry and setting up watches on the referenced Secrets (serving 5xx for any App whose Secret is absent), so that a fresh replica is correct without a leader round-trip.
  38. As an operator, I want rapid successive changes to be coalesced into a single Snapshot rebuild, so that a burst of events does not cause a rebuild storm.
  39. As an operator, I want an App that is OIDC-only, Exchange-only, or both to be represented by the presence/absence of its per-Flow sections, so that the schema reflects the App's actual Flows.

Implementation Decisions

Modules built/modified

  • A state loader (the leader role) that watches the three CRDs, structurally validates, resolves IdP config, and writes the managed Registry Secret. The three existing per-kind reconcilers (currently stubs) are repurposed as thin triggers: on any event they invoke a single idempotent, cluster-wide Registry builder.
  • Because the Registry is cluster-wide (hostnames are global) while the CRDs are namespaced, the builder lists all three kinds cluster-wide on every trigger and rebuilds the whole Registry (any change to any kind can affect the global index / validation).
  • A replica mirror component that watches the managed Registry Secret and the referenced per-App Secrets and maintains the local in-memory Snapshot.
  • The manager entrypoint is extended so the single binary runs the controller (reconcilers + leader-gated Registry writer + JWKS refresh) and, once built, the bridge; they share the per-replica Snapshot.

Leader/replica split

  • Leader election gates only the Registry-write side-effects (writing the managed Secret, CRD status updates). The mirror side runs on every replica (no leader gating), so every replica serves from its own Snapshot.

Registry (managed Secret) schema

  • One managed Secret in the Bridge's namespace, JSON-encoded, one nested entry per App:
    • identity: namespace, clientID, hostnames (also the hostname-index keys)
    • shared: tokenEndpoint (one resolved endpoint used by both Flows), jwks (raw JWKS JSON), secretRef {namespace, name, key}, secretReady
    • OIDC section (present iff the App has an OidcPolicy): authorizationEndpoint, callbackURLs, and the policy config (scopes, session, injection, logout)
    • Exchange section (present iff the App has a TokenExchangePolicy): clientRef, scopes, and the subject-token source config (header, prefix, token type)
  • Written as a whole-object atomic update. Known 1 MiB ceiling.

Structural validation gate (Registry membership)

  • hostnames non-overlapping (cluster-wide, one App per hostname); a policy's clientRef resolves to an existing OauthClient in the same namespace; clientSecretRef is well-formed (has a name).
  • A failing App is excluded from the Registry and its CRD marked Degraded.
  • Not a structural criterion: whether the referenced Secret exists — that is the runtime secretReady property.

Failure model (runtime degradation)

  • A referenced Secret missing → the App stays in the Registry with secretReady=false, CRD Degraded, served as 5xx (fail-closed, "unavailable"). On recreation → secretReady=true, normal operation. Never removed.

Secret handling (reference-and-watch)

  • The Registry stores secretRef; every replica runs an informer over the referenced per-App Secrets. The per-replica "bytes present" (→ the 5xx flag) is derived from the local watch, not the leader's flag.
  • RBAC: all replicas need get/list/watch on the managed Secret + the referenced per-App Secrets (in App namespaces); only the leader needs IdP egress, cluster-wide list of the three kinds, and read/write of the managed Secret.

JWKS refresh

  • Leader-owned; a TTL timer plus on any CRD change; on fetch failure, keep the last-good JWKS (a transient IdP outage does not blank the keys).

Snapshot (visibility mechanic — Way A)

  • An immutable Snapshot per replica; on any Registry or referenced-Secret change, rebuild the whole block and atomically swap the pointer. Lock-free read (one pointer load); the hostname index and App set are built together and can never diverge. Rapid changes are coalesced into a single rebuild.

IdP resolve (discovery + JWKS)

  • The leader fetches the discovery document from the per-client issuerURL → token_endpoint, authorization_endpoint, and the JWKS. The token endpoint is shared across Flows; the authorization endpoint is OIDC-only.

Exchange subject token

  • Forwarded (opaque, access_token type) to the token endpoint; the bridge does not validate it (no HMAC secret held); read from the subjectToken config already in the CRD; no client secret sent. (The state model carries the config + subject-token source; the actual RFC 8693 grant POST is the gRPC-server follow-on.)

CRD status

  • Each App's CRDs get metav1.Condition status — Available (in the Registry, secretReady) or Degraded (structurally invalid, or secretReady=false) — with observedGeneration (the existing CRD status already carries a Conditions field).

Testing Decisions

What makes a good test

  • Test external behavior only: given cluster state, assert the observable outputs — the managed Registry Secret's JSON content, the CRD status conditions, and (for the pure snapshot builder) the returned Snapshot's public shape. Do not assert on private struct fields, internal call order, or loop mechanics.

Modules tested

  • State loader (Seam 1 — envtest, the one integration seam): the whole leader path — structural validation, IdP resolve, Registry write, CRD status, JWKS refresh, and the structural-vs-runtime failure model — driven against real CRDs + real referenced Secrets in the existing envtest suite (real API server + etcd, config/crd/bases), with the IdP stood in by an httptest server.
  • Snapshot builder (Seam 2 — pure unit, no K8s): given a Registry object + the current watched-Secret bytes, returns an immutable Snapshot; assert the hostname index, App set, and per-App "bytes present" (→ the 5xx flag) by calling it directly.
  • IdP resolve (Seam 3 — httptest test double): the discovery-doc / JWKS fetch and the keep-last-good-on-failure refresh, against a stub IdP that can be made to fail. The refresh timer is exercised through Seam 1; the keep-last-good decision has a focused unit test.
  • Schema / structural-validation rules (pure unit + via Seam 1): the nested per-App entry shape and the structural-exclusion rules.

Prior art

  • The existing per-package suite_test.go (envtest + Ginkgo/Gomega) is the Seam 1 substrate; the stub *_controller_test.go are the placeholders to fill.

Seam count: one integration seam (Seam 1), with the snapshot builder and schema as pure unit tests and the IdP as a test double.

Out of Scope

  • The gRPC ext_authz server and its Check handling: OIDC 302/redirect + code-for-token + id_token validation + header injection; the Exchange RFC 8693 token call (the actual grant POST, the audience/resource/requested_token_type handling, the response). These consume the Snapshot.
  • The plain-HTTP callback/logout listener (:8082).
  • The Services and ReferenceGrant manifests.
  • The gRPC ext_authz proto dependency (not yet in go.mod).
  • Actual OIDC/Exchange protocol execution (token validation against the JWKS at request time, cookie handling, redirect flow) — the state model provides the config (JWKS, endpoints, policy, subject-token source); the server performs the protocol.
  • The secretReady → 5xx surfacing by the gRPC server (the state model provides the flag + semantics; the server acts on it).
  • The leader-election mechanism (standard controller-runtime Lease; not re-designed).

Further Notes

  • The managed Registry Secret has a 1 MiB ceiling — fine at lab scale; a hard cap to watch if the App count or JWKS grows.
  • Last-good state persists across failover (managed Secret in etcd); the leader is on the update path, not the serving path.
  • Watch lag → brief divergence after a leader write (a replica can be seconds behind); benign for auth (re-validate / refresh on unknown key).
  • ADRs 0005 (leader-authored Registry, reference-and-watch, replicas mirror) and 0006 (missing Secret = runtime degradation, not removal) capture the decisions this spec implements.
  • The Exchange grant semantics are documented in docs/references/kanidm-service-account-token-exchange.md; read it before working on the Exchange Flow.
  • The three existing per-kind reconcilers are stubs; this spec repurposes them as thin triggers into one idempotent, cluster-wide Registry builder.
## Problem Statement An App team wants to protect its backends (API servers, web servers, dashboards) behind any Gateway API implementation that supports the GEP-1494 `ExternalAuth` filter, using per-App authentication and no gateway-specific config. The gateway's `ExternalAuth` filter calls out to the Bridge over `ext_authz`; for the Bridge to answer those callouts it must know, per request: which App the request belongs to, that App's resolved IdP configuration (token/authorization endpoints, signing keys), its client-secret reference, and its per-Flow policy behaviour. Today there is no in-cluster mechanism to turn the App team's CRDs (`OauthClient`, `OidcPolicy`, `TokenExchangePolicy`) into that resolved, validated, ready-to-use, per-App state, and to make it consistently available to every replica that serves traffic — without each replica redundantly re-resolving IdP config, re-reading Secrets, or the operator hand-wiring gateway config. ## Solution The shared in-memory state model, built and distributed by the operator's controller: - **The App team** declares, per App: an `OauthClient` (clientID, Kanidm issuer URL, client-secret reference, `callbackURLs`, `hostnames`) and one policy per Flow it runs (`OidcPolicy`, `TokenExchangePolicy`). A Flow is active iff its policy object exists. - **The leader** watches those CRDs, structurally validates them, resolves the IdP config (discovery document → token/authorization endpoints, JWKS signing keys), and writes the resolved, validated Apps into a single managed Secret in the Bridge's namespace — the **Registry**. - **Every replica** mirrors the Registry into a local in-memory **Snapshot** (rebuilt atomically on any change) and watches the referenced client Secrets, so each replica serves from a consistent, resolved, ready-to-use view. App resolution is by hostname (one App per hostname, never overlapping). - **Fail-closed:** a structurally-invalid App is excluded from the Registry and marked Degraded; a missing referenced Secret keeps the App present but degraded (served as 5xx), never mis-authorizing. - **The operator** gets a single in-cluster object (`kubectl get secret`) that shows exactly what is being served, 1× IdP load, secrets staying in the per-App Secrets, and last-good state surviving failover. ## User Stories 1. As an App team, I want to declare a per-App `OauthClient` with its clientID, Kanidm issuer URL, client-secret reference, callback URLs, and hostnames, so that I protect my App's backends without any gateway-specific config. 2. As an App team, I want each of my Apps to be isolated (its own Client in its own namespace, credentials never shared), so that Kanidm's per-client issuer, keys, and tokens only ever work for the intended App. 3. As an App team, I want to declare one `OidcPolicy` to enable the human/browser OIDC Flow for my App, so that users can sign in and I can inject their identity into my backend. 4. As an App team, I want to declare one `TokenExchangePolicy` to enable the machine Exchange Flow (RFC 8693) for my App, so that service accounts can exchange their API token for tokens without user interaction. 5. As an App team, I want a Flow to be active for my App exactly when its policy object exists, so that I enable/disable Flows by creating/deleting the policy, with no separate "enabled" flag. 6. As an App team, I want my App's `hostnames` to be the single key that maps an incoming request to my App, so that the bridge resolves "which App is this request for" from the request's `Host` header. 7. As an operator, I want an App whose hostnames overlap another App's to be excluded from the Registry, so that hostname resolution is unambiguous and one App per hostname is enforced. 8. As an operator, I want a policy whose `clientRef` points to a non-existent Client to be excluded (structurally invalid), so that broken config never silently serves a half-configured App. 9. As an operator, I want the leader to resolve each Client's discovery document from its issuer URL into the token and authorization endpoints, so that the bridge has ready-to-use endpoint URLs without a per-request fetch. 10. As an operator, I want the leader to fetch and store the IdP's signing keys (JWKS) in the resolved state, so that the bridge can validate tokens without a per-request key fetch. 11. As an operator, I want the client secret to be referenced (namespace/name/key) in the Registry rather than copied into it, so that secrets stay in the per-App Secrets and a rotation does not require re-materializing the Registry. 12. As an operator, I want every replica to watch the referenced client Secrets and hold the bytes locally, so that a replica can complete an OIDC token exchange without reading a Secret per request. 13. As an operator, I want the leader to be the only process that talks to the IdP (discovery + JWKS) and reads the CRDs, so that IdP load is 1× regardless of replica count. 14. As an operator, I want every replica to mirror the single Registry into its own in-memory Snapshot, so that all replicas serve from one consistent source of truth. 15. As an operator, I want a change to the Registry or a referenced Secret to rebuild the Snapshot atomically (immutable swap), so that a serving replica can never read a half-updated state. 16. As an operator, I want the read path (a Check against the Snapshot) to be lock-free (a single pointer load), so that serving stays fast. 17. As an operator, I want the hostname index and the App set to be built together, so that they can never disagree. 18. As an operator, I want a structurally-invalid App to be excluded from the Registry and its CRD marked Degraded, so that I can see why an App is not being served. 19. As an operator, I want a deleted referenced Secret to keep the App in the Registry with `secretReady=false` (served as 5xx) rather than removing it, so that a broken config surfaces as "unavailable," not a false "forbidden." 20. As an operator, I want the 5xx (missing Secret) to take effect immediately on every replica (each loses the bytes on its own watch), so that the failure is consistent and not gated on the leader. 21. As an operator, I want a recreated Secret to flip `secretReady=true` and resume normal operation, so that a transient Secret outage self-heals. 22. As an operator, I want the leader to refresh the JWKS on a TTL and on any CRD change, so that rotated signing keys are picked up. 23. As an operator, I want the leader to keep the last-good JWKS if a refresh fetch fails, so that a transient IdP outage does not break token validation. 24. As an operator, I want only the leader to write the Registry and only side-effects to be leader-gated, so that state updates pause during failover but last-good serving continues. 25. As an operator, I want the Registry to persist (managed Secret in etcd), so that the last-good state survives replica restarts and leader failover. 26. As an operator, I want to run `kubectl get secret` on the managed Registry to see exactly which resolved Apps every replica is serving, so that I can debug what the bridge is doing without reading process memory. 27. As an operator, I want each App's CRD status to reflect its Available / Degraded state and observed generation, so that per-App health is visible in the standard place. 28. As an App team, I want the resolved per-App entry to carry my OIDC session, injection (claim→header), and logout config, so that the bridge applies my per-App behaviour. 29. As an App team, I want the resolved per-App entry to carry my Exchange policy's scopes and subject-token source (header/prefix/token type), so that the bridge knows where to read the presented service-account token from and which scopes to request. 30. As an App team, I want the Exchange Flow's subject token to be forwarded (opaque, `access_token` type) to the token endpoint rather than validated in-bridge, so that the bridge does not hold the HMAC secret and Kanidm remains the validator. 31. As an operator, I want the OIDC policy `scopes` to default to `openid`, so that a minimal policy still requests an identity token. 32. As an operator, I want a replica to derive its Snapshot's per-App "bytes present" from its own Secret watch (not from the leader's flag), so that the 5xx behaviour is correct even during leader latency. 33. As an operator, I want the managed Registry to be a single object (one watch, one atomic write), so that config updates are atomic and there is one thing to inspect. 34. As an operator, I want the state to be per-replica in memory with the Registry as the persistent source, so that a replica can serve immediately from its mirror without a leader round-trip. 35. As an App team, I want a previously-working App whose Secret is deleted to be served as 5xx and marked Degraded (not removed), so that my users see "unavailable," not "forbidden," and I get a status signal. 36. As an operator, I want the Exchange Flow target (the RFC 8693 `audience`) to be implicitly the Client's own clientID, so that machine tokens land in the App's own audience with no extra config. 37. As an operator, I want a new replica to build its Snapshot by watching the Registry and setting up watches on the referenced Secrets (serving 5xx for any App whose Secret is absent), so that a fresh replica is correct without a leader round-trip. 38. As an operator, I want rapid successive changes to be coalesced into a single Snapshot rebuild, so that a burst of events does not cause a rebuild storm. 39. As an operator, I want an App that is OIDC-only, Exchange-only, or both to be represented by the presence/absence of its per-Flow sections, so that the schema reflects the App's actual Flows. ## Implementation Decisions **Modules built/modified** - A **state loader** (the leader role) that watches the three CRDs, structurally validates, resolves IdP config, and writes the managed Registry Secret. The three existing per-kind reconcilers (currently stubs) are repurposed as **thin triggers**: on any event they invoke a single **idempotent, cluster-wide Registry builder**. - Because the Registry is cluster-wide (hostnames are global) while the CRDs are namespaced, the builder **lists all three kinds cluster-wide** on every trigger and rebuilds the whole Registry (any change to any kind can affect the global index / validation). - A **replica mirror** component that watches the managed Registry Secret and the referenced per-App Secrets and maintains the local in-memory Snapshot. - The **manager entrypoint** is extended so the single binary runs the controller (reconcilers + leader-gated Registry writer + JWKS refresh) and, once built, the bridge; they share the per-replica Snapshot. **Leader/replica split** - Leader election gates **only** the Registry-write side-effects (writing the managed Secret, CRD status updates). The **mirror side runs on every replica** (no leader gating), so every replica serves from its own Snapshot. **Registry (managed Secret) schema** - One managed Secret in the Bridge's namespace, JSON-encoded, one nested entry per App: - **identity:** `namespace`, `clientID`, `hostnames` (also the hostname-index keys) - **shared:** `tokenEndpoint` (one resolved endpoint used by both Flows), `jwks` (raw JWKS JSON), `secretRef {namespace, name, key}`, `secretReady` - **OIDC section (present iff the App has an `OidcPolicy`):** `authorizationEndpoint`, `callbackURLs`, and the policy config (scopes, session, injection, logout) - **Exchange section (present iff the App has a `TokenExchangePolicy`):** `clientRef`, `scopes`, and the subject-token source config (header, prefix, token type) - Written as a **whole-object atomic update**. Known 1 MiB ceiling. **Structural validation gate (Registry membership)** - hostnames non-overlapping (cluster-wide, one App per hostname); a policy's `clientRef` resolves to an existing `OauthClient` in the same namespace; `clientSecretRef` is well-formed (has a name). - A failing App is **excluded** from the Registry and its CRD marked Degraded. - **Not** a structural criterion: whether the referenced Secret *exists* — that is the runtime `secretReady` property. **Failure model (runtime degradation)** - A referenced Secret missing → the App stays in the Registry with `secretReady=false`, CRD Degraded, served as 5xx (fail-closed, "unavailable"). On recreation → `secretReady=true`, normal operation. Never removed. **Secret handling (reference-and-watch)** - The Registry stores `secretRef`; every replica runs an informer over the referenced per-App Secrets. The per-replica "bytes present" (→ the 5xx flag) is **derived from the local watch**, not the leader's flag. - RBAC: **all replicas** need get/list/watch on the managed Secret + the referenced per-App Secrets (in App namespaces); **only the leader** needs IdP egress, cluster-wide list of the three kinds, and read/write of the managed Secret. **JWKS refresh** - Leader-owned; a TTL timer plus on any CRD change; on fetch failure, keep the last-good JWKS (a transient IdP outage does not blank the keys). **Snapshot (visibility mechanic — Way A)** - An immutable Snapshot per replica; on any Registry or referenced-Secret change, rebuild the whole block and atomically swap the pointer. Lock-free read (one pointer load); the hostname index and App set are built together and can never diverge. Rapid changes are coalesced into a single rebuild. **IdP resolve (discovery + JWKS)** - The leader fetches the discovery document from the per-client `issuerURL` → `token_endpoint`, `authorization_endpoint`, and the JWKS. The token endpoint is shared across Flows; the authorization endpoint is OIDC-only. **Exchange subject token** - Forwarded (opaque, `access_token` type) to the token endpoint; the bridge does not validate it (no HMAC secret held); read from the `subjectToken` config already in the CRD; no client secret sent. (The state model carries the config + subject-token source; the actual RFC 8693 grant POST is the gRPC-server follow-on.) **CRD status** - Each App's CRDs get `metav1.Condition` status — `Available` (in the Registry, `secretReady`) or `Degraded` (structurally invalid, or `secretReady=false`) — with `observedGeneration` (the existing CRD status already carries a `Conditions` field). ## Testing Decisions **What makes a good test** - Test **external behavior only**: given cluster state, assert the observable outputs — the managed Registry Secret's JSON content, the CRD status conditions, and (for the pure snapshot builder) the returned Snapshot's public shape. Do not assert on private struct fields, internal call order, or loop mechanics. **Modules tested** - **State loader (Seam 1 — envtest, the one integration seam):** the whole leader path — structural validation, IdP resolve, Registry write, CRD status, JWKS refresh, and the structural-vs-runtime failure model — driven against real CRDs + real referenced Secrets in the existing envtest suite (real API server + etcd, `config/crd/bases`), with the IdP stood in by an `httptest` server. - **Snapshot builder (Seam 2 — pure unit, no K8s):** given a Registry object + the current watched-Secret bytes, returns an immutable Snapshot; assert the hostname index, App set, and per-App "bytes present" (→ the 5xx flag) by calling it directly. - **IdP resolve (Seam 3 — `httptest` test double):** the discovery-doc / JWKS fetch and the keep-last-good-on-failure refresh, against a stub IdP that can be made to fail. The refresh timer is exercised through Seam 1; the keep-last-good decision has a focused unit test. - **Schema / structural-validation rules (pure unit + via Seam 1):** the nested per-App entry shape and the structural-exclusion rules. **Prior art** - The existing per-package `suite_test.go` (envtest + Ginkgo/Gomega) is the Seam 1 substrate; the stub `*_controller_test.go` are the placeholders to fill. **Seam count:** one integration seam (Seam 1), with the snapshot builder and schema as pure unit tests and the IdP as a test double. ## Out of Scope - The gRPC `ext_authz` server and its `Check` handling: OIDC 302/redirect + code-for-token + `id_token` validation + header injection; the Exchange RFC 8693 token call (the actual grant POST, the `audience`/`resource`/`requested_token_type` handling, the response). These consume the Snapshot. - The plain-HTTP callback/logout listener (`:8082`). - The Services and `ReferenceGrant` manifests. - The gRPC `ext_authz` proto dependency (not yet in `go.mod`). - Actual OIDC/Exchange protocol execution (token validation against the JWKS at request time, cookie handling, redirect flow) — the state model *provides* the config (JWKS, endpoints, policy, subject-token source); the server *performs* the protocol. - The `secretReady` → 5xx surfacing by the gRPC server (the state model provides the flag + semantics; the server acts on it). - The leader-election mechanism (standard controller-runtime Lease; not re-designed). ## Further Notes - The managed Registry Secret has a 1 MiB ceiling — fine at lab scale; a hard cap to watch if the App count or JWKS grows. - Last-good state persists across failover (managed Secret in etcd); the leader is on the *update* path, not the serving path. - Watch lag → brief divergence after a leader write (a replica can be seconds behind); benign for auth (re-validate / refresh on unknown key). - ADRs 0005 (leader-authored Registry, reference-and-watch, replicas mirror) and 0006 (missing Secret = runtime degradation, not removal) capture the decisions this spec implements. - The Exchange grant semantics are documented in `docs/references/kanidm-service-account-token-exchange.md`; read it before working on the Exchange Flow. - The three existing per-kind reconcilers are stubs; this spec repurposes them as thin triggers into one idempotent, cluster-wide Registry builder.
ginjiruu commented 2026-09-15 17:40:26 +00:00
Author
Owner
Copy link

Closing: the spec's build scope is complete. All four tickets are delivered and merged to master:

  • #2 IdP resolve (discovery + JWKS, keep-last-good) — PR #7
  • #3 Registry builder (the leader's core) — PR #8
  • #4 Replica mirror (the local Snapshot) — PR #10
  • #5 Leader freshness (runtime-secret tracking + JWKS refresh) — PR #9

The spec #11 follow-on amendment (resolve + carry end_session_endpoint, ticket #13) is also merged — PR #22.

Closing: the spec's build scope is complete. All four tickets are delivered and merged to master: - #2 IdP resolve (discovery + JWKS, keep-last-good) — PR #7 - #3 Registry builder (the leader's core) — PR #8 - #4 Replica mirror (the local Snapshot) — PR #10 - #5 Leader freshness (runtime-secret tracking + JWKS refresh) — PR #9 The spec #11 follow-on amendment (resolve + carry `end_session_endpoint`, ticket #13) is also merged — PR #22.
ginjiruu commented 2026-09-15 17:40:27 +00:00
Author
Owner
Copy link

Build scope complete (PRs #7-#10 merged, plus the #11 amendment PR #22); spec delivered.

Build scope complete (PRs #7-#10 merged, plus the #11 amendment PR #22); spec delivered.
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
master
ci/repo-declared-scaffold
fix/45-exchange-forward-id-token
fix/44-oidc-bearer-tokens
t36-pkce-nonce
t34-oidc-trio
t33-securecookie
t32-public-client
t37-e2e
t35-exchange-hstern
feat/oidc-logout
feat/oidc-callback
feat/oidc-lazy-refresh
feat/services-referencedgrant
feat/exchange-token-cache
feat/exchange-check
feat/oidc-steady-state-check
feat/end-session-endpoint
feat/build-surface
feat/replica-mirror
feat/leader-freshness
feat/registry-builder
renovate/configure
feat/idp-resolve
v0.1.2-ci-test
v0.1.1
v0.1.0
Labels
Clear labels   
needs-info

   
needs-triage

   
ready-for-agent

   
ready-for-human

   
wontfix
No labels
needs-info
needs-triage
ready-for-agent
ready-for-human
wontfix
Milestone
Clear milestone
No items
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
lab/authz-bridge#1
No description provided.