Automate the reclaim flow for the original owning namespace

[bmarinov]

Is your feature request related to a problem? Please describe.
The namespace which creatd the bucket should be able to reclaim with zero manual interaction.

Describe the solution you'd like
Metadata in the Garage cluster is doable with a workaround - until tags are eventually supported.

Describe alternatives you've considered
State in Kubernetes, new cluster-wide CRD etc are fallback options.

Additional context
Builds on top of #79 but does not require the additional owner key in the namespace.

[bmarinov]

Owner meta can be stored in an admin bucket, there doesn't seem to be a viable alternative at the time. Pros: same-cluster recovery can be fully automated, recovery+gitops into a new cluster will require a controllable instance ID designation.

E.g. a stable clusterID (helm value?), getting it right and managing properly is required. No guardrails and its not very pretty, but it would work incl with flux/helmrelease. The main point is to not rely on nameOverride (WIP) in any shape. Suffixed names remain default & the recommendation.

Edit: guardrails for messing up and running two controller instances with the same cluster ID value?

[bmarinov]

For the record: this is going back to storing this data in a secondary (owner) alias on the bucket itself. There were too many problems with the admin bucket idea on the drawing board. Scrapping it in favor of something which gives me full consistency out of the box.

High level

Every bucket will carry two controller-assigned aliases:

• the "primary" alias based on spec.name + the originating resource UID (same as today).
• a new "owner" alias based on the location: domain prefix + meta.name + meta.namespace + cluster_id.

The controller will be able to reconstruct a stable bucket identifier out of coordinates and intent. Given namespace team1, bucket object backups, and cluster_id eu-0-prod, there is exactly one bucket carrying the alias matching this logical location.

Details

Current implementation (existing primary alias):

primary = <spec.name>-<suffix>
suffix = hex(sha256(uid)[:8])

New alias:

owner = gsc.o.<digest>.<cluster_id>
digest = hex(sha256("gsc..TBD"+ namespace + name))
cluster_id = chart / environment config (max ~ 16-24 chars)

Enabling this feature will require a CLUSTER_ID to be set. No default will be provided. This is to avoid a very likely scenario where oblivious users running N controllers on a shared Garage cluster cause collisions through a default cluster ID.

After the MVP I may add some heartbeat and kind-of-mutex to make it less likely that misconfiguration wont cause problems.

Whiteboard / quick diagram: