Skip to content

feat: $secret:// support - manage APISIX secrets (secret managers) through the controller #2831

Description

@jens-skribble

Description

Summary

Bring APISIX's native secrets secret-manager resource (Vault / AWS / GCP) into
the controller's resource model, so a secrets manager can be declared through the
controller
and becomes part of the full-config snapshot ADC pushes. That makes
$secret://<manager>/<id>/... references usable on controller-managed resources
(routes, plugin configs, consumers, SSL, GatewayProxy), instead of the value having
to exist as a plaintext Kubernetes Secret.

Follow-up to #2821 (the standalone sync deadlock). As discussed there, the supported
answer isn't out-of-band registration — it's making secrets a first-class resource
the controller owns and syncs. This issue is the design discussion for that.

Motivation / use case

We want to reference secrets from plugin config via APISIX's native $secret://
indirection rather than materializing the value into the config. Concrete case:
central, HA-safe rotation of the openid-connect session/cookie secret via Vault —

# today (works, but the value lives in the data-plane env / a k8s Secret):
session:
  secret: "$env://APISIX_SESSION_SECRET"

# what we'd like — resolved from a Vault secret manager, rotated centrally:
session:
  secret: "$secret://vault/oidc/oidc/session_secret"

APISIX itself already resolves $secret://vault/... in API-driven standalone mode
(verified in #2821). The only missing piece is a controller-side way to declare the
backing secrets manager.

Why not secretRef

secretRef reads a Kubernetes Secret and injects the plaintext value into the
pushed config — a different mechanism. It doesn't give APISIX's native $secret://
indirection, where the data plane fetches from Vault/AWS/GCP at request time and the
secret material never has to live in the k8s config or the ADC snapshot. Both are
useful; this request is specifically about the native secrets managers.

Proposed direction (for discussion)

Add secrets to the control-plane resource model so it's part of the snapshot. Open
questions we'd love maintainer input on:

  1. Where to declare it. Two shapes we can see:

    • a secrets list on GatewayProxy (alongside pluginMetadata) — simple, one
      place, naturally scoped to the gateway the controller drives; or
    • a dedicated CRD (e.g. ApisixSecret / SecretManager) translating to APISIX's
      secrets resource — more flexible, referenceable across namespaces, its own
      lifecycle/status.
  2. Backends. APISIX supports vault, aws, and gcp managers — mirror those
    fields (uri, prefix, auth, etc.).

  3. Manager credentials should not be inline plaintext. The Vault token / AWS / GCP
    credentials the manager needs should be sourceable from a Kubernetes Secret via a
    valueFrom: { secretKeyRef: ... } pattern — the same shape the chart already uses
    for GatewayProxy adminKey.valueFrom. (This is the one place secretRef-style
    k8s-Secret reading is still needed — to bootstrap the manager's own auth.)

  4. Referencing. Confirm the reference syntax works from the plugin configs we care
    about (ApisixPluginConfig / ApisixRoute plugins, openid-connect session.secret,
    and ideally ApisixConsumer / SSL).

  5. Ownership semantics. Once the controller owns secrets, a manager registered out
    of band is (correctly) removed on the next full sync — that's expected and fine; it's
    the flip side of bug: standalone/ADC - an out-of-band secrets resource deadlocks the sync loop (secrets_conf_version 400), blocking all reconciliation #2821 and we're not asking for out-of-band to be supported.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions