RFC: Authenticate External Secrets Operator to OpenBao with Kubernetes auth

[znerol2]

Status: Draft, for the project's RFC discussion category
Related issues: #396, #11
Related decisions: ADR-0008, ADR-0021, ADR-0022, ADR-0028
Related plan: docs/plans/openbao-approle-credential-lifecycle-plan.md

Each citation reads dmfdeploy/<repo>@<rev>:<path>:<lines>. The revisions
are umbrella origin/main at 5fb2bb4, dmf-infra origin/main at c13358d
and, where marked, dmf-infra branch feat/openbao-setup-issuer at 0b1450e
(dmfdeploy/dmf-infra#118). The appendices hold the details.

Summary

External Secrets Operator (ESO) logs in to OpenBao with an AppRole SecretID
that has a 30-day TTL. This RFC proposes that the main ESO store log in with
the OpenBao Kubernetes auth method instead. ESO would request a short-lived
token for its own ServiceAccount at login, so no stored ESO SecretID could
expire while a cluster is off.

The benefit is narrow. Other things can still fail after a long power-off:
the ServiceAccount token and the OpenBao token both expire, and trust roots,
the API server and clocks all have to work. The proposal removes one stored
credential from ESO's path. It does not make the path failure-proof.

A prerequisite experiment comes first. The existing Kubernetes auth config
may depend on a copied, pod-bound token, which would break every Kubernetes
login after the OpenBao pod is replaced. Do not move ESO in production until
the experiment passes on a disposable environment.

Problem

ESO's SecretID has a finite TTL
(dmfdeploy/dmf-infra@c13358d:k3s-lab-bootstrap/roles/stack/operator/openbao/defaults/main.yml:85). When
it expired, the main ClusterSecretStore stopped being Ready and every ExternalSecret that uses it
went stale together. #396 records that this happened on
2026-08-14 on a sandbox environment. OpenBao's login lockout then delayed recovery, because
minting a new SecretID did not restore service until the lockout ended
(dmfdeploy/dmfdeploy@5fb2bb4:docs/plans/openbao-approle-credential-lifecycle-plan.md:35-38, 396-400).

The agreed plan adds a daily in-cluster job that rotates each SecretID at
two-thirds of its TTL (dmfdeploy/dmfdeploy@5fb2bb4:docs/plans/openbao-approle-credential-lifecycle-plan.md:416-427, 471-473; schedule at 425-427). For a 30-day TTL
that leaves about ten days of retries, less scheduling and processing delay.

Rotation helps only while the cluster runs. A cluster that stays off longer
than the TTL restarts with an expired SecretID, and no job could have run.
The flypack offline lane is the clearest case: its OpenBao is sealed on every
cold boot, power loss is an expected cause, and its factory seed includes an
ESO AppRole
(dmfdeploy/dmf-infra@c13358d:k3s-lab-bootstrap/docs/flypack-offline-lane.md:102-103, 120-121, 316-318). A lab environment shut down for a month has the same problem.

ADR-0008 chose AppRole for ESO (dmfdeploy/dmfdeploy@5fb2bb4:docs/decisions/0008-openbao-secrets-architecture.md:6, 25-27). I found no recorded reason for AppRole over Kubernetes auth. The
only stated cost is the bootstrap chicken-and-egg for the first AppRole write
(dmfdeploy/dmfdeploy@5fb2bb4:docs/decisions/0008-openbao-secrets-architecture.md:57-59). Role defaults and the README describe Kubernetes
auth as "TODO" and "a future extension"
(dmfdeploy/dmf-infra@c13358d:k3s-lab-bootstrap/roles/base/external-secrets/defaults/main.yml:26,
dmfdeploy/dmf-infra@c13358d:k3s-lab-bootstrap/roles/base/external-secrets/README.md:13). Those comments do not show why the choice was made. See
Appendix E for the evidence.

Proposal

Move the main openbao ClusterSecretStore to the Kubernetes auth method:

1. Add an OpenBao Kubernetes auth role for ESO, bound to ESO's controller
   ServiceAccount in its namespace, with the same eso-reader policy the
   AppRole has and a short token TTL. Set an audience on the role and ask for
   the same audience from ESO.
2. Replace the store's appRole block with a kubernetes block that uses
   serviceAccountRef. Appendix A has the fields I confirmed in the pinned
   chart's CRD.
3. Keep the AppRole for a stated rollback window, then remove it.
4. Make this a profile-scoped decision. It covers the main ESO store. It
   leaves ADR-0022's hub and edge AppRole contract unchanged, and it leaves
   the two object-storage stores for a separate choice (Decision 2).

This is more than a two-role change. The OpenBao role creation runs only on a
fresh init, the ESO role asserts that AppRole inputs exist, the break-glass
file records the AppRole fields, and the existing verifier does not cover the
main store. Appendix A lists what has to be built: an existing-install
reconciler, verifier coverage and a rollback path that works on an installed
environment.

The following rules apply however ESO logs in. They also cover the AppRole
consumers that remain.

• Use a separate rotator identity for any residual static secret. An expired
  credential cannot renew itself. The lifecycle plan already defines a
  least-privilege Kubernetes-authenticated scheduler.
• After a cold boot, use one order: start k3s and the API server; check that
  the clock is plausible; unseal; check again before any expiry decision;
  run any catch-up rotation; let AppRole consumers resume. On more than one
  node, also check clock skew between nodes.
• Keep break-glass and unseal material unique per unit. EBU R 143 v3 M-AA-05
  (P1) bans one shared password or key across all units. Per-unit shares and
  SecretIDs go beyond that text and are a design choice.
• Keep ESO's retry interval and alerts under a tested policy so a stale
  SecretID cannot reach the lockout threshold. I propose no number, because
  one interval is not shown to work across several stores and controllers
  (Appendix B).

Prerequisite experiment: the token reviewer

The existing config writes the OpenBao pod's own ServiceAccount token as the
reviewer credential, once, on a fresh init, and the task sets
failed_when: false (dmfdeploy/dmf-infra@c13358d:k3s-lab-bootstrap/roles/stack/operator/openbao/tasks/main.yml:807-828). The risk depends
on what the deployed mount holds. If the stored token is a pod-bound
projected token, Kubernetes rejects it after that Pod is deleted or the token
expires. A replaced Pod, not a restarted container, is the trigger, and a
power cycle does not always give a new Pod UID.

OpenBao documents a local-file fallback with periodic re-read. It does not
document that omitting the field on an update clears a value already stored.
So the fix must be an explicit, version-verified clearing step, and the test
must prove the result.

Run the experiment on a disposable, isolated environment. Record token
metadata only (expiry and binding claims). Never log a token. A passing
experiment establishes these properties:

• The reviewer mode is the local-file fallback. A version-verified config read
  shows disable_local_ca_jwt=false and no stored reviewer JWT. OpenBao's
  ServiceAccount holds TokenReview permission and nothing broader.
• The throwaway test client has no TokenReview permission, so a client-JWT
  review cannot mask the result.
• A baseline fresh login succeeds. After the OpenBao Pod is replaced, a fresh
  login under the old config is recorded. It is expected to fail.
• After the migration, a config read shows the fallback mode above. A fresh
  login then succeeds after a Pod replacement, before any reapply or repair
  of the reviewer config. Metadata then shows that the local reviewer token
  used before has rotated or expired, and a later fresh login succeeds.
• The other Kubernetes-auth clients, including the snapshot job, still log in.
• The throwaway roles, accounts and tokens are removed.

Three outcomes. Pass: every property is established. Inconclusive, which
takes precedence: the old-config login unexpectedly succeeds, or something
unrelated to the reviewer path fails. Investigate, and draw no conclusion.
Fail: with the preconditions holding, a fresh login fails on the migrated
reviewer path.

Recovery before the run. On the disposable environment, define and verify
the least-disruptive recovery path first: an authorized privileged identity
restores a working reviewer config, confirmed by a fresh login. A full storage
restore is a separate contingency with wider effects, and a snapshot can
contain the same pod-bound reviewer token.

The step-by-step protocol belongs in the implementing issue's work order, not
in this RFC. On existing installs, the stored reviewer config has to be
migrated deliberately, because the fresh-init task does not rerun there. That
needs the idempotent reconciler and an authorized identity (Appendix A).

Alternatives

Catch-up rotation on start. Keep AppRole. The plan's least-privilege
Kubernetes-authenticated rotator could act as a recovery identity after
unseal, under a recovery contract. Today the plan rejects an expired target
that has no recovery state (dmfdeploy/dmfdeploy@5fb2bb4:docs/plans/openbao-approle-credential-lifecycle-plan.md:281-283),
so this recovery is not built. A lockout does not rule recovery out: the
system can stop retrying, or wait, or unlock with authorized control. The
costs are a recovery state machine and the same reviewer and API dependency
as this proposal. It leaves ADR-0008 unchanged.

Longer TTLs for offline profiles. Set a TTL for each offline profile from
a defined operational bound: the longest expected off-time, the operator
visit interval and a margin, with documented compensating controls. It is a
small change and a legitimate documented choice. The cliff stays at the bound,
a leaked SecretID stays useful longer, and lockout still follows an expiry.

Status quo with daily rotation. Ship the plan as written. It is designed
and partly built (dmfdeploy/dmf-infra#118). It leaves a long power-off as an
operator recovery task with break-glass access.

These options are not exclusive. The plan stays for any AppRole consumer
that remains.

Decisions requested

1. Reviewer remediation. Approve running the prerequisite experiment,
   including the migration step, on a disposable environment now. Production
   rollout needs a separate approval after a pass. Name the identity that may
   write the Kubernetes auth config on an installed environment, checked
   against ADR-0021's separation of duties. This RFC does not assume that any
   current identity, including the break-glass path, qualifies.
2. Which stores move. Move only the main store, or also the two
   object-storage stores? If they move, each needs its own OpenBao role and
   policy, with tests for exact backup-path isolation. One ServiceAccount
   with several roles gives no workload isolation.
3. ADR scope. Record a profile-scoped decision through the RFC to ADR
   path (dmfdeploy/dmfdeploy@5fb2bb4:docs/WORKING-MODEL.md:142-150) with reciprocal supersession links.
   Confirm that hub and edge AppRole (ADR-0022) stays out of scope. Remote
   TokenReview trust is possible but is a separate network and trust design.
4. Rollback window. How long to keep the ESO AppRole, and what happens
   to its SecretID, expiry and lockout during that time.

---

Appendix A: Implementation steps and what has to be built

CRD fields (confirmed in chart external-secrets version 2.3.0, file
templates/crds/clustersecretstore.yaml in that chart archive, lines
5034-5114). The kubernetes auth block requires mountPath (default
kubernetes) and role. It also has secretRef and
serviceAccountRef{name, namespace, audiences[]}. The CRD describes
namespace as ignored unless the referent is cluster-scoped. For a
cluster-scoped store, set it explicitly. The chart's RBAC grants create on
serviceaccounts/token (chart file templates/rbac.yaml:183-188). That suggests ESO
uses TokenRequest, but the bound object, audience handling and lifetime are
not confirmed. Do not assume the token is bound to the controller Pod.

What has to be built.

• Existing-install reconciler. The ESO AppRole is created only on a fresh
  init (dmfdeploy/dmf-infra@c13358d:k3s-lab-bootstrap/roles/stack/operator/openbao/tasks/main.yml:1138-1157), and the Kubernetes auth
  config task is the same. An idempotent reconciler has to create the role
  and rewrite the config on installed environments. It needs an identity
  authorized for auth config and role writes. Check that against ADR-0021's
  separation of duties before assuming any current identity qualifies. It
  also needs ordering rules for first run and reruns.
• Callers. dmfdeploy/dmf-infra@c13358d:k3s-lab-bootstrap/roles/base/external-secrets/tasks/main.yml:4-10
  requires the role ID and SecretID, and
  dmfdeploy/dmf-infra@c13358d:k3s-lab-bootstrap/roles/base/external-secrets/tasks/main.yml:69-82
  recreates the old Secret. The break-glass file
  records eso_role_id and eso_secret_id
  (dmfdeploy/dmf-infra@c13358d:k3s-lab-bootstrap/roles/stack/operator/openbao/tasks/main.yml:1207-1220, 1578-1579). Bootstrap, restore and
  provisioning callers all depend on these. Change the input contract and
  list every caller.
• Store apply semantics. The role applies the store with
  kubernetes.core.k8s and state: present. Test whether a changed auth
  block replaces or merges with the old appRole key on an existing store.
• Verifier. dmfdeploy/dmf-infra@c13358d:k3s-lab-bootstrap/playbooks/verify-openbao-identity-model.yml:203-265 checks
  only openbao-openbao-snapshots and openbao-app-backups, not the main
  store, and Ready does not prove a fresh login. Add: a main-store check; a
  newly requested token with the right audience; a real new authentication;
  a main-store ExternalSecret refreshed with fresh observed data or version;
  and negative tests for wrong ServiceAccount, namespace and audience. Test
  fresh and existing installs, including reapply and restart. Check the
  object stores independently.

Rollback. Restoring the old appRole block is not enough. A rollback
also has to restore any removed AppRole, role ID, Kubernetes Secret, input
contract and store block, by an authorized and tested path. The rotation
playbook asserts the Secret already exists
(dmfdeploy/dmf-infra@c13358d:k3s-lab-bootstrap/playbooks/vertical-orchestration/110-eso-secret-rotation.yml:90-104)
and uses operator break-glass credentials, so check that those credentials
are authorized under current policy. Do not assume the playbook succeeds.
The retained SecretID keeps its TTL during the window. Decide whether to keep
rotating it, and how to handle an expiry and lockout in that time. Test
rollback before and after the AppRole is removed.

Docs. Update ADR-0008 through the RFC to ADR path, the role README and
default comment, and the flypack offline lane spec (factory seed at
dmfdeploy/dmf-infra@c13358d:k3s-lab-bootstrap/docs/flypack-offline-lane.md:316-318
and auth methods at
dmfdeploy/dmf-infra@c13358d:k3s-lab-bootstrap/docs/flypack-offline-lane.md:104-106).

Appendix B: Failure modes and blast radius

Compared with an AppRole SecretID for ESO.

1. Reviewer credential. See the prerequisite experiment. All Kubernetes
   logins share it: ESO, and the snapshot job and the planned identities
   when they are deployed. The snapshot role write also sets
   failed_when: false (dmfdeploy/dmf-infra@c13358d:k3s-lab-bootstrap/roles/base/openbao-raft-snapshot/tasks/main.yml:89-107),
   so a deployed job is not proof the path works.
2. Long power-off. No stored ESO SecretID ages. These can still fail:
   the OpenBao token and the ServiceAccount token, which both expire; the
   reviewer credential; the stored Kubernetes CA; and the cluster's own
   certificates. Upstream k3s documents certificates that expire and renew at
   start, and a CA that does not renew automatically (see
   https://docs.k3s.io/cli/certificate). I have not checked this on the pinned
   version, so treat certificate and CA expiry, and recovery from it, as
   conditional risks.
3. Clocks. Tokens and "expired" decisions depend on wall-clock time. A
   single node and a multi-node cluster differ: skew between nodes is a
   separate risk, and a clock with no working source has no known bound.
   Check plausibility before unseal and before any expiry decision.
4. Sealed OpenBao. Login fails while sealed, for AppRole and Kubernetes
   alike. After unseal, ESO's next retry logs in. Whether it needs further
   action depends on the retry policy.
5. API server unavailable. Kubernetes auth adds a login-time dependency:
   ESO needs the API server for its token, and OpenBao needs it for review.
   An OpenBao token already issued is a separate bearer credential. How long
   it lasts depends on its TTL, max TTL, renewal and revocation settings, not
   on the JWT or the Pod.
6. Lockout. OpenBao documents lockout for userpass, LDAP and AppRole, not
   Kubernetes. That is from upstream documentation. Verify behavior on the
   pinned version. A retry back-off policy across several stores and
   controllers, and concurrent failures, is unresolved and needs testing.
7. Blast radius. The two paths differ and I do not claim one is smaller.
   A stolen SecretID works until it expires or is revoked. A stolen
   ServiceAccount token is limited by its lifetime and audience. An audience
   is not a network-origin restriction. Any principal that can create a Pod
   using the ESO ServiceAccount, or request a token for it, can act as that
   role, depending on RBAC and admission. Compare actual privileges in the
   namespace. The eso-reader policy also grants PKI issuance on the
   internal CA (dmfdeploy/dmf-infra@c13358d:k3s-lab-bootstrap/roles/stack/operator/openbao/tasks/main.yml:1108-1110), so it is not only
   a read policy.

Appendix C: Effect on #396 and #11

Registry on dmfdeploy/dmf-infra#118
(dmfdeploy/dmf-infra@0b1450e:k3s-lab-bootstrap/vars/openbao-approle-lifecycle-registry.json:4-6, 27-30, 64-67, 110-113,
branch only): runtime targets external-secrets (720h),
object-storage-openbao-snapshots (8760h) and object-storage-app-backups
(8760h); born-inventory-netbox-reader (720h) is setup-only. The two
object-storage stores use their own AppRoles
(dmfdeploy/dmf-infra@c13358d:k3s-lab-bootstrap/roles/base/object-storage-credentials/tasks/main.yml:421-432).

If only the main store moves, the daily job keeps the two object-storage
targets, due at about eight months. A cluster off for more than a year has
the same cliff there. If those stores also move, the runtime registry is
empty. That triggers a re-scope decision, not an automatic end to the plan.
The plan also requires root-token convergence, scoped identities, a safe
rotation transaction with recovery, telemetry and alerting
(dmfdeploy/dmfdeploy@5fb2bb4:docs/plans/openbao-approle-credential-lifecycle-plan.md:24-33, 75-79, 245-283, 385-400, 404-490). If the two stores stay on AppRole, address their
long-offline recovery and rotate the SecretID separately from the underlying
object-storage credential.

Sequencing: the plan's rotation job ships first with the external-secrets
registry entry, so ESO keeps rotation coverage. The ESO migration is a
separate issue whose completing PR removes that entry and updates the plan
frontmatter. No coverage gap opens.

Appendix D: Relevant controls and how each option relates

This is not an audit and not a compliance claim. Each row notes a control
that bears on an option, with its applicability. Primary sources: NIST SP
800-53 Rev. 5
(https://nvlpubs.nist.gov/nistpubs/SpecialPublications/NIST.SP.800-53r5.pdf);
OWASP ASVS 5.0.0
(https://github.com/OWASP/ASVS/blob/v5.0.0_release/5.0/en/0x22-V13-Configuration.md);
IETF draft-ietf-wimse-arch-08, a draft and not a standard
(https://datatracker.ietf.org/doc/draft-ietf-wimse-arch/08/); EBU R 143 v3
(https://tech.ebu.ch/files/live/sites/tech/files/shared/r/r143.pdf).

Control	What it says and where it applies	How the options relate
NIST IA-5(7)	Ensure that unencrypted static authenticators are not embedded in applications or other forms of static storage.	A SecretID in a Kubernetes Secret is a static authenticator in storage, and whether it counts as unencrypted depends on how it is held. A requested token is not persistently stored by the proposed design, subject to verification. The mechanism alone does not establish compliance.
NIST IA-5(13)	Prohibit use of cached authenticators after a defined period. Its scope is cached authenticators used locally when the network is unavailable (as the Rev. 5 text reads; I did not re-read it).	At most an analogy for offline credentials. Catch-up rotation does not by itself meet it. A zero TTL does not automatically violate it.
NIST IA-9	Uniquely identify and authenticate defined system services and applications.	A ServiceAccount identity relates to this. The token mechanism alone does not establish it.
NIST SP 800-207 (tenets 3, 6), SP 800-190 (4.1.4)	Per-session dynamic authentication; secrets provided at runtime and kept out of images.	Describe broader properties. The proposal fits their direction and does not prove them.
OWASP ASVS 5.0.0 13.3.4 (level 3)	Verify that secrets are configured to expire and rotate per the application's documentation.	A verification item. A schedule or a proposed mechanism is not proof of compliance. The interval is left to the application's documentation.
WIMSE draft 3.1.3, 4.4	Bounded credential lifetimes, automatic renewal, no long-lived static secrets.	Work in progress. Kubernetes tokens relate to it. The draft sets no number.
EBU R 143 v3 M-AA-05 (P1)	No global hidden accounts with one password or key for all units.	Relates to the break-glass account. Per-unit unseal shares and SecretIDs are an additional design choice.

NIST AC-2(3) is left out of the table on purpose. It covers disabling expired
and other specified accounts. A replaced SecretID is not an expired account,
so it does not bear on catch-up rotation.

Gaps. No source I read sets a credential lifetime for a system that is often
disconnected, and NIST SP 800-82 Rev. 3 has no text on it. The TTL for any
residual static secret, and its compensating controls, are the owner's
documented choice. Possible controls to document (my inference): a minimal
policy per credential; limits by source address and use count; encrypted
storage; a store sealed on cold boot; alerts on unexpected use; per-unit
uniqueness; a rotation event at the operator's visit.

I could not read ISO/IEC 27001:2022 or IEC 62443, so I make no claim about
what they require, including whether they set a TTL.

Appendix E: Evidence for the current design

• ADR-0008 rule and decision name AppRole for ESO
  (dmfdeploy/dmfdeploy@5fb2bb4:docs/decisions/0008-openbao-secrets-architecture.md:6, 25-27). Its
  archive lists three alternatives, none of them Kubernetes auth
  (dmfdeploy/dmfdeploy@5fb2bb4:docs/archive/decisions/0008-openbao-secrets-architecture.md:7-14). The
  ADR calls the credential "self-perpetuating". The code gives it a finite
  TTL, with a comment that the TTL bounds how long a stale SecretID stays
  valid if a run is missed (dmfdeploy/dmf-infra@c13358d:k3s-lab-bootstrap/roles/stack/operator/openbao/defaults/main.yml:82-85). I did not
  verify that no other renewal path exists.
• The init task enables the Kubernetes method and writes its config
  (dmfdeploy/dmf-infra@c13358d:k3s-lab-bootstrap/roles/stack/operator/openbao/tasks/main.yml:737-757, 807-828). The config write sets
  failed_when: false, so a failed write would be silent. Whether a given
  environment has a working config needs evidence.
• Kubernetes auth identities: the snapshot job (on origin/main; ttl 10m,
  projected token with an openbao audience, role without an audience;
  dmfdeploy/dmf-infra@c13358d:k3s-lab-bootstrap/roles/base/openbao-raft-snapshot/tasks/main.yml:89-107 and
  dmfdeploy/dmf-infra@c13358d:k3s-lab-bootstrap/roles/base/openbao-raft-snapshot/templates/openbao-raft-snapshot-cronjob.yml.j2:29-35); the setup
  issuer (15m) and the dmf-promsd observer (audience
  dmf-openbao-observer, 5m), which exist only on
  feat/openbao-setup-issuer (dmfdeploy/dmf-infra@0b1450e:k3s-lab-bootstrap/roles/stack/operator/openbao/tasks/main.yml:916-921, 1043-1049). The scheduler role is planned only
  (dmfdeploy/dmfdeploy@5fb2bb4:docs/plans/openbao-approle-credential-lifecycle-plan.md:245-265). These
  are branch or unproven patterns, not evidence that the method works in a
  deployed environment.
• ADR-0022 relies on per-edge AppRole against a hub OpenBao
  (dmfdeploy/dmfdeploy@5fb2bb4:docs/decisions/0022-flypack-online-thin-edge-agent.md:19, 37, 41).
  ADR-0021 and ADR-0028 do not mention ESO's login method.

Appendix F: Unconfirmed items

• Whether OpenBao clears a stored reviewer JWT when the field is omitted on
  an update. The prerequisite experiment verifies the explicit clearing step
  and the behavior after Pod replacement, not omission, unless its work order
  tests omission separately.
• Lockout scope for the Kubernetes method on the pinned OpenBao version.
• The bound object, audience handling and lifetime of the token ESO requests
  with serviceAccountRef, and whether the store needs the namespace set.
• Whether state: present replaces or merges an auth block change.
• k3s certificate and CA behavior on the pinned version.
• Which identity may write the Kubernetes auth config on an installed
  environment under current policy, and whether the rotation playbook's
  break-glass path is authorized.
• Framework mapping: my reading. ISO/IEC 27001:2022 and IEC 62443 text were
  not read. I read the IA-5(7) text in Rev. 5, not IA-5 or IA-5(13).
• Deployed state: whether a working Kubernetes auth config exists in any
  environment, and incident details beyond AppRole SecretID rotation has no pre-expiry schedule or alert — every env silently fails at ~30 days #396.

[znerol2]

Evidence for Decision 1, from a read-only check on 2026-11-07 against a disposable sandbox environment running the pinned versions (OpenBao 2.5.2, ESO chart 2.3.0):

• None of the three non-root identities that an installed environment keeps (the ops-admin user and the two reconciler users from ADR-0021) can read auth/kubernetes/config. Each read returned 403.
• The root token is disposed of after init, as designed.

So on an installed environment today, no identity can read the Kubernetes auth config, let alone migrate it. The existing-install reconciler in Appendix A needs a new, scoped identity, or a deliberate and time-boxed root-token generation. Decision 1 should choose between them.

The same environment also showed the problem this RFC describes. Its ESO SecretID expired 30 days after install, and the main store reported invalid role or secret ID, then permission denied. Running the sanctioned rotation playbook restored it within seconds, with no lockout wait.

[znerol2]

Related: #864 plans ADR-0029 Tier 2 operator unlock for cold boots. After a long power-off, its unlock flow has to renew ESO's expired AppRole SecretID before ESO retries, or the retries trip the login lockout (#396). The Kubernetes auth proposal in this RFC would remove that catch-up step for ESO.

[znerol2]

Decisions (operator, 2026-11-07)

1. Reviewer remediation and the auth-config writer.
   • The prerequisite experiment, including the migration step, is approved on a disposable environment now. A production rollout needs a separate approval after it passes.
   • No standing identity may write the Kubernetes auth config. On a new install, the installer writes it while it still holds the initial root token, before that token is disposed of. On an installed environment, any later write uses a temporary root token from OpenBao's generate-root procedure (unseal-share quorum), used for that change only, then revoked, with the use audited. This keeps ADR-0021's separation of duties: no routine reconciler or admin identity holds this power.
2. Which stores move. Only the main openbao ClusterSecretStore. The two object-storage stores stay on AppRole for now. Moving them is separate later work with its own roles and isolation tests.
3. ADR scope. Recorded as a profile-scoped ADR through the RFC-to-ADR path, with reciprocal supersession links. Hub and edge AppRole (ADR-0022) and remote TokenReview trust are out of scope.
4. Rollback window. The ESO AppRole stays for 14 days after the switch, renewed by the planned rotation job so it can't expire or lock out during the window. After that, the role and its SecretID are deleted.

[znerol2]

Accepted: ADR-0051

The operator accepted ADR-0051 on 2026-11-08, and #877 has merged it. ESO's main openbao ClusterSecretStore will authenticate with Kubernetes auth. ADR-0008 is now partially superseded, for the ESO main-store login only.

• Rollout: none yet. Each environment still needs its own approval after the installer work lands (the plan's WP3 and WP4).
• Enforcement: WP3's verifier adds a fresh login, negative tests for a wrong ServiceAccount, namespace and audience, and a negative write probe on auth/kubernetes/config.
• Follow-up: narrowing the eso-reader policy's certificate grant is Split internal-CA certificate issuance out of the ESO policy #881.