Skip to content

fix(local): enable persistent openbao and resolve helm dependency lock bug - #540

Open
Rana-Ashar wants to merge 1 commit into
mainfrom
fix-openbao-persistence
Open

fix(local): enable persistent openbao and resolve helm dependency lock bug#540
Rana-Ashar wants to merge 1 commit into
mainfrom
fix-openbao-persistence

Conversation

@Rana-Ashar

Copy link
Copy Markdown
Contributor

Overview

This PR resolves the ephemeral OpenBao issue for local environments, ensuring that OpenBao state and secrets survive pod crashes and Docker reboots. Along the way, it also resolves two underlying bugs affecting the local bootstrap process.

Resolves Ticket: Improve local setup with persistent OpenBao

What Changed

  1. Persistent OpenBao & Auto-Unseal Sidecar

    • Disabled memory-only dev mode and enabled standalone mode backed by a 2Gi file PVC.
    • Injected a robust auto-unsealer sidecar container. The sidecar monitors the Vault API and automatically handles bao operator init (first boot) and bao operator unseal (subsequent boots/restarts) by storing and retrieving the keys from the persistent volume.
    • Refactored kubectlExec to dynamically extract the newly generated root token from the persistent volume so the CLI can securely configure auth and policies.
  2. Fixed 403 Permission Denied for External Secrets

    • Bug: The CLI was passing a static token_reviewer_jwt file path during OpenBao Kubernetes auth configuration. Modern Kubernetes dynamically rotates these tokens, causing ExternalSecrets to get locked out with a 403 error.
    • Fix: Removed the static token injection. OpenBao now natively uses the Kubernetes TokenReview API, allowing auth to survive pod restarts and token rotations perfectly.
  3. Fixed helm dependency build Lockfile Crash

    • Bug: When rendering local charts, dynamically modifying the chart caused a mismatch between Chart.yaml and Chart.lock timestamps, crashing the Helm dependency step.
    • Fix: Changed the command in dependency.go from "build" to "update" to force Helm to regenerate the lockfile dynamically.

Testing Performed

  • Ran kubara bootstrap test-cluster --local successfully.
  • Verified OpenBao initializes and unseals automatically on first boot.
  • Verified ExternalSecrets successfully binds to OpenBao (ClusterSecretStore is Ready).
  • Verified Grafana passwords dynamically sync into Kubernetes secrets.
  • Disaster Recovery Test: Deleted the openbao-0 pod manually. Verified the sidecar automatically successfully unsealed the new pod and no secrets were lost.

@Rana-Ashar
Rana-Ashar requested a review from a team August 4, 2026 06:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant