Skip to content

Fix content-cache-k8s hanging in maintenance when nginx-proxy is consumed via CMR - #194

Open
florentianayuwono with Copilot wants to merge 3 commits into
mainfrom
copilot/fix-content-cache-k8s-hang
Open

Fix content-cache-k8s hanging in maintenance when nginx-proxy is consumed via CMR#194
florentianayuwono with Copilot wants to merge 3 commits into
mainfrom
copilot/fix-content-cache-k8s-hang

Conversation

Copilot AI commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

In a Cross-Model Relation (CMR), Juju represents remote units with proxy names like remote-<uuid>/0 instead of real application unit names. The charm was using these tokens to build headless pod DNS entries (remote-<uuid>-0.<svc>-endpoints.<local-model>.svc.cluster.local) that don't exist, causing nginx to fail on startup and leaving the unit stuck in maintenance: Updating pebble layer config.

What this PR does

  • Detects CMR proxy units in _make_env_config by checking for the remote- prefix on unit names
  • When CMR is detected, constructs a service-level K8s DNS entry instead of a pod-level headless one:
    http://{service-name}.{service-namespace}.svc.cluster.local:{service-port}
    
    service-namespace comes from the relation app data (automatically populated by NginxRouteRequirer with the requirer's model name); falls back to the local model name if absent
  • Preserves existing pod-level headless DNS behaviour for same-model relations

Why we need it

Cross-model relations are a standard Juju capability. CMR deployments were completely broken — the charm would never reach active and the hook would fail on every retry.

Checklist

  • I followed the contributing guide
  • I added or updated the documentation (if applicable)
  • I updated docs/changelog.md with user-relevant changes
  • I added a change artifact for user-relevant changes in docs/release-notes/artifacts. If no change artifact is necessary, I tagged the PR with the label no-release-note.
  • I used AI to assist with preparing this PR
  • I added or updated tests as needed (unit and integration)
  • If integration test modules are used: I updated the workflow configuration
    (e.g., in .github/workflows/integration_tests.yaml, ensure the modules list is correct)
  • If this PR involves a Grafana dashboard: I added a screenshot of the dashboard
  • If this PR involves Rockcraft: I updated the version

Test plan

Two new unit tests added:

  • test_make_env_config_with_cmr_proxy_relation — verifies service-level DNS with service-namespace present
  • test_make_env_config_with_cmr_proxy_relation_no_namespace — verifies fallback to local model name when service-namespace is absent

Review focus

The CMR detection heuristic (unit.name.startswith("remote-")) matches Juju's documented naming convention for CMR proxy units. No Juju API exists to query this directly, so the prefix check is the standard approach.

…lations

In a Cross-Model Relation (CMR), remote unit names follow the pattern
remote-<uuid>/<n> (Juju-internal proxy tokens), not real Kubernetes pod
names. The previous code used these tokens to construct headless pod DNS
entries that don't exist in the remote namespace.

Detect CMR proxy units by checking for the remote- prefix on unit names,
then use Kubernetes service-level DNS with the service-namespace from the
relation app data (set by NginxRouteRequirer to the requirer's model name):

  http://{svc_name}.{svc_namespace}.svc.cluster.local:{svc_port}

This lets nginx correctly resolve and proxy to the backend service in the
remote Juju model on the same Kubernetes cluster.

Closes #193

Co-authored-by: florentianayuwono <76247368+florentianayuwono@users.noreply.github.com>
Copilot AI changed the title [WIP] Fix content-cache-k8s hanging in maintenance during config update Fix content-cache-k8s hanging in maintenance when nginx-proxy is consumed via CMR Jul 7, 2026
Copilot AI requested a review from florentianayuwono July 7, 2026 03:34
Co-authored-by: florentianayuwono <76247368+florentianayuwono@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Fixes content-cache-k8s getting stuck in maintenance: Updating pebble layer config when the nginx-proxy relation is established via Juju Cross-Model Relations (CMR), where remote units are represented as remote-<uuid>/<n> and cannot be used to construct pod-level headless DNS names.

Changes:

  • Update _make_env_config to detect CMR proxy unit names (remote-...) and, when detected, build a service-level Kubernetes DNS backend URL using service-name, service-namespace (fallback to local model name), and service-port.
  • Preserve existing same-model behavior that builds pod-level headless DNS entries from unit names.
  • Add unit tests validating the service-level DNS behavior for CMR (with and without service-namespace present).

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File Description
src/charm.py Adds CMR detection and switches backend URL construction to service-level DNS for CMR relations.
tests/unit/test_charm.py Adds unit tests covering CMR proxy relations and the service-namespace fallback behavior.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@florentianayuwono
florentianayuwono marked this pull request as ready for review July 8, 2026 03:17
@florentianayuwono
florentianayuwono requested a review from a team as a code owner July 8, 2026 03:17
@florentianayuwono
florentianayuwono requested review from minulo and removed request for a team July 8, 2026 03:17

@minulo minulo left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

content-cache-k8s hangs in maintenance: Updating pebble layer config when nginx-proxy endpoint is consumed via CMR

4 participants