fix(auth): require out-of-band bootstrap secret for first root key

[frdomovic]

Security finding

[H] Trust-on-first-use root key (security review item 2) — crates/auth/src/providers/impls/user_password.rs.

On a fresh node with no root keys, authenticate_core treats the first login as a "bootstrap" and mints a ROOT admin key for whatever username/password the caller supplies — over the unauthenticated POST /auth/token route. On a freshly provisioned node, whoever reaches it first (potentially an attacker on the network, not the operator) becomes admin.

Before

if existing_keys.is_empty() {
    // Bootstrap case - create the first root key
    let (key_id, root_key) = self.create_root_key(username, password).await?;
    Ok((key_id, root_key.permissions))
} else {
    Err(eyre::eyre!("Invalid username or password"))
}

Any first caller → root admin. No secret, no gate.

After

The bootstrap path now requires the caller to prove possession of an out-of-band secret:

• The expected secret comes from MERO_AUTH_BOOTSTRAP_SECRET (preferred, out-of-band) or the new user_password.bootstrap_secret config field.
• The caller presents it as bootstrap_secret in the provider data (or the x-bootstrap-secret header on the header-based path).
• Comparison is constant-time over SHA-256 digests (subtle::ConstantTimeEq), so it leaks neither timing nor length.
• Fail closed: when no secret is configured, first-login bootstrap is disabled — a root key must be provisioned out of band instead.
• All rejection paths return the same generic "Invalid username or password" so a probe can't distinguish disabled / wrong secret / already bootstrapped.

Existing users are unaffected: they match on the fast verify_credentials path before the bootstrap branch and never need the secret.

⚠️ Operational impact (breaking by design)

First-login bootstrap is now off unless a secret is configured. Deployments that relied on implicit TOFU bootstrap must set MERO_AUTH_BOOTSTRAP_SECRET (or user_password.bootstrap_secret) before the first login, or provision the root key out of band. This is the intended hardening; flagging it for release notes.

How to test

Automated (added in this PR, cargo test -p mero-auth user_password):

• bootstrap_disabled_by_default_rejects_first_login — no secret configured → first login errors, zero root keys created.
• bootstrap_requires_matching_secret — missing/wrong secret → rejected, no key; correct secret → exactly one admin root key.
• existing_user_authenticates_without_bootstrap_secret — after bootstrap, the existing user logs in with no secret; a different identity cannot bootstrap a second root key even with the correct secret.
• bootstrap_secret_matches_is_exact — the constant-time comparator is exact (length/case sensitive).

Manual:

1. Fresh node, MERO_AUTH_BOOTSTRAP_SECRET unset → POST /auth/token must not create a key.
2. Set the env var, retry with provider_data.bootstrap_secret matching → root key minted.
3. Retry a third login from another identity → rejected.

Notes

• Touches user_password.rs, so it will conflict with fix(auth): enforce password bounds, key expiry, and exact node-host binding #3081 (salted-KDF key id / password bounds) at merge — both are independent security fixes off master; whichever lands first, the other rebases.
• Full suite green: 91 lib tests (+4 new) + 6 integration tests, clippy clean.

Update 2026-07-14 — review fix + e2e strategy

Empty-string bypass (meroreviewer 🔴) — fixed twice over (c7f9c269 + 90110a0d): the config source now filters empty strings exactly like the env source (bootstrap_secret = "" behaves as unset → fail closed), AND an empty presented secret is rejected before any comparison — so SHA256("") == SHA256("") can never re-enable TOFU even if one guard regresses. New tests: empty_config_bootstrap_secret_keeps_bootstrap_disabled, empty_presented_secret_never_bootstraps.

e2e strategy with the bootstrap gate on (ffcba187): CI provisions the out-of-band secret instead of weakening the gate. The auth-seam job exports MERO_AUTH_BOOTSTRAP_SECRET once — binary-mode merobox starts merod with the inherited environment and local script steps inherit it too, so the same export configures the node's expected secret and lets scripts/e2e-auth-seam.sh present it via provider_data.bootstrap_secret on the bootstrap login. The SDK-E2E job exports the secret into merod's environment and pre-mints the dev root key with a single authenticated bootstrap call before the mero-js suite runs, so the SDK's plain username/password logins authenticate on the existing-user fast path (its wrong-credentials negative test stays valid). Verified locally with merobox 0.6.41: auth-seam passes 7/7 with the secret exported and fails closed without it.

Known-remaining red: the docker-mode scenario matrix (Test (group-*)) drives logins through merobox's built-in login step, which today can neither inject env into node containers nor present a bootstrap secret. The matrix jobs already export the CI test secret (inert until merobox supports it); one small merobox PR — forward MERO_AUTH_BOOTSTRAP_SECRET into node containers + present it in the login step, both defaulting from merobox's own environment — heals all 58 scenario workflows with no further core changes, after which MIN_MEROBOX gets bumped in .github/actions/setup-merobox.

[meroreviewer]

🤖 MeroReviewer

Reviewed by 2 agents | Quality score: 47% | Review time: 367.0s

🔴 1 critical. See inline comments.

---

_{🤖 Generated by MeroReviewer | Review ID: review-8e082ce7}

[meroreviewer]

🔴 Empty-string bootstrap_secret config value silently re-enables unauthenticated TOFU bootstrap

effective_bootstrap_secret() filters the MERO_AUTH_BOOTSTRAP_SECRET env var for emptiness (.filter(|s| !s.is_empty())) but does NOT apply the same filter to self.config.bootstrap_secret. If the config value deserializes to Some("") (e.g. a TOML/env-interpolation footgun such as bootstrap_secret = "", or AUTH__USER_PASSWORD__BOOTSTRAP_SECRET= left blank in a container env file), effective_bootstrap_secret() returns Some("") instead of None. Meanwhile, at line 238, bootstrap_secret.unwrap_or_default() yields "" whenever the caller omits the field entirely. bootstrap_secret_matches("", "") then returns true (SHA-256("") == SHA-256("")), so the very first unauthenticated caller is minted a ROOT admin key with zero secret presented — exactly the trust-on-first-use vulnerability this PR sets out to close. This directly contradicts the documented fail-closed guarantee ("when this is None... bootstrap is disabled") because an empty string is a distinct, unguarded value from None.

Suggested fix:

Apply the same non-empty filter to the config fallback, e.g. `.or_else(|| self.config.bootstrap_secret.clone().filter(|s| !s.is_empty()))`, and/or reject empty strings at config-load time (e.g. via a custom `Deserialize`/`validator` check on `UserPasswordConfig::bootstrap_secret`) so an accidentally-blank value can never be treated as "configured".

[frdomovic]

Addressed in c7f9c26 — the config fallback now gets the same .filter(|s| !s.is_empty()) as the env var, the bootstrap_secret config doc states empty ≡ unset, and empty_config_secret_is_treated_as_unset pins the exact failure mode (omitted field defaulting to "" matching an empty configured secret).

[frdomovic]

Fixed — belt and braces: (1) effective_bootstrap_secret() now applies the same .filter(|s| !s.is_empty()) to the config source as to the env var, so bootstrap_secret = "" behaves as unset (bootstrap disabled, fail closed) — c7f9c26; (2) independently, authenticate_core rejects an empty presented secret with the generic error before any comparison, so an all-empty pairing can never authenticate even if the config-side filter regresses — 90110a0. Tests: empty_config_bootstrap_secret_keeps_bootstrap_disabled, empty_presented_secret_never_bootstraps (both assert zero root keys minted on a fresh KeyManager).

[meroreviewer]

Documentation Review

The following documentation may need updates based on the changes in this PR:

• 🟡 architecture/crates/auth.html: Files matching crates/auth/** were changed but architecture/crates/auth.html was not updated (per source_to_docs_mapping).

[meroreviewer]

🤖 AI Code Reviewer

Reviewed by 2 agents | Quality score: 90% | Review time: 1000.7s

---

⚠️ Review Incomplete

1 of 2 agent(s) did not finish (security-reviewer) and no findings were produced. Treat this PR as not yet reviewed, not as approved.

---

_{🤖 Generated by AI Code Reviewer | Review ID: review-fa89eff5}

[github-actions]

E2E Rust Apps Failed

One or more E2E workflows (scaffolding-e2e, xcall-example) failed.

Please check the workflow logs for more details.