Reconcile Accounts when referenced NatsCluster configuration changes

[choufraise]

Summary

Each Account in NAuth resolves to exactly one NatsCluster, either:

• explicitly via spec.natsClusterRef
• implicitly via default cluster resolution

When the effective cluster configuration changes, affected Account resources should be reconciled automatically.

Today that does not happen reliably.

Problem

Account reconciliation depends on the effective cluster identity and signing identity of the resolved NatsCluster.

The important parts are:

• the NATS URL, because it points to the actual cluster instance
• the operator signing key, where the relevant identity is the public key
• the SYS account identity used for management access

The raw system credentials themselves are not the important trigger. They may rotate frequently without requiring a new account JWT. What matters is whether the credentials still represent the same SYS account identity.

Example:

• operator signing key is rotated
• affected Account JWTs remain signed by the old key
• accounts are not re-signed until some unrelated reconcile happens

That leaves stale account state in NATS and in Account.status.

Expected behavior

Changes to the effective NatsCluster inputs should enqueue reconciliation for all affected Account resources.

This includes at least:

• changes to the resolved operator signing key identity
• changes to the resolved SYS account identity
• changes to the resolved NATS URL

Important nuance: system credentials

Rotating the system credentials alone should not force account re-signing.

The credentials are only relevant insofar as they grant access to the same SYS account. If the credentials change but still resolve to the same SYS account identity, no account JWT content needs to change.

The trigger should therefore be based on the effective SYS account identity, not on raw credential material.

Important nuance: URL changes

A URL change is special.

Today there is no reliable way, via the NATS API, to determine whether a new URL points to:

• the same cluster under a new address, or
• a different cluster entirely

The main risk is not just accidental drift. The bigger problem is that NAuth loses track of the account that was previously created in the old cluster, leaving it orphaned there.

With decentralized auth this gets worse:

• if the account signing keys remain unchanged, existing User JWTs for the old cluster will also work against the new cluster for the same account
• newly created User JWTs for the new cluster will also continue to work against the old cluster/account
• NAuth then effectively manages one logical account identity across two different clusters without realizing it

That makes a URL change potentially dangerous even when signing keys are unchanged.

We should therefore require some form of manual two-step verification for URL changes before blindly reconciling against the new target.

Scope

Implement reconciliation triggers so that:

1. updates to a NatsCluster enqueue all affected Account resources
2. changes to referenced signing-key material enqueue affected Account resources when the effective operator signing public key changes
3. changes to referenced system credentials enqueue affected Account resources when the effective SYS account identity changes
4. both explicit and implicit/default-cluster users are handled
5. URL changes are treated as a special case with an explicit safety mechanism / manual verification flow

Acceptance criteria

• Updating the effective operator signing key causes all affected accounts to reconcile
• Affected account JWTs are re-signed when the operator signing key identity changes
• Rotating system credentials without changing SYS account identity does not trigger unnecessary account JWT regeneration
• Changing SYS account identity triggers reconciliation of affected accounts
• Both explicit and implicit NatsCluster resolution paths are covered
• URL changes are guarded by an explicit safety mechanism because moving to a different cluster can orphan the old account and unintentionally preserve access across both old and new clusters
• Behavior is covered by tests

[choufraise]

Let's also look into the classic if statement on Generation vs ObservedGeneration and OperatorVersion to algin this controller with the other ones.

NatsCluster shouldn't be able to delete if any Account is still referring to it. Note that we might need to add a reference in the account.status since we allow NatsClusterRef to be absent and rely on the implicit operator NATS cluster reference from operator config.