Life Coach pod fails health check on prod (c-6f83b1c9)

[kzumpoll]

Summary

Kemp's Life Coach deployment on production (charming-squid-23) is stuck in error status. The pod was provisioned and assigned an IP but failed the 5-minute health check.

Deployment Details

Field	Value
Pod ID / Subdomain	c-6f83b1c9
Convex Deployment	charming-squid-23 (EU West 1)
User	zumpollekemp@gmail.com (nd770ba6z4pqqrt9r9zqb1xvt183sza6)
Image	europe-west3-docker.pkg.dev/lifeos-prod-491301/lifeos-prod/openclaw:latest
Config Hash	053ea4804ef1
Pod IP	10.46.4.51 (was assigned, so pod was scheduled)
Error	"Instance failed to become healthy within 5 minutes. Please try restarting."
Plan	BYOK, selectedModel: claude-sonnet, Anthropic setup token configured
Created	2026-04-14 ~02:05 UTC

What We Know

• The pod was scheduled and got an IP (10.46.4.51), so the image pulled and at least one container started.
• The health check polls https://c-6f83b1c9.lifeos.zone/ every 5s for 60 attempts. It never got a 200-399 response.
• "Try again" from the dashboard (deactivate + redeploy) produces the same result.
• The dev deployment (calculating-hornet-639) has a similar pod (c-50d7e4f4) that IS running and healthy on the dev K8s cluster — so the OpenClaw image/config works in general.

What We Could NOT Check

We don't have kubectl access to the prod GKE cluster (lifeos-prod-491301). The following commands need to be run to diagnose:

# Connect to prod cluster
gcloud container clusters get-credentials <cluster-name> --region europe-west3 --project lifeos-prod-491301

# Check pod status and events
kubectl describe pod claw-c-6f83b1c9-0 -n lifeos-users

# Check all container logs
kubectl logs claw-c-6f83b1c9-0 -n lifeos-users --all-containers

# Check init containers individually
kubectl logs claw-c-6f83b1c9-0 -n lifeos-users -c persist-setup
kubectl logs claw-c-6f83b1c9-0 -n lifeos-users -c register
kubectl logs claw-c-6f83b1c9-0 -n lifeos-users -c lifeos-setup
kubectl logs claw-c-6f83b1c9-0 -n lifeos-users -c openclaw

Possible Causes

1. Init container failure — register container calls the AI Gateway at http://gw-default.lifeos.zone/register. If the gateway is unreachable from the prod cluster, registration fails and OpenClaw won't start.
2. DNS issue — *.lifeos.zone resolves to 34.179.209.86 but the prod nginx ingress controller LB may be at a different IP (we confirmed a mismatch on the dev cluster where DNS points to 34.179.209.86 but the LB is 34.185.198.70).
3. Image pull issue — The prod image is from lifeos-prod-491301 registry. If the GKE node service account doesn't have Artifact Registry access, the pull would fail.
4. OpenClaw startup crash — The pod may start but OpenClaw gateway may crash before becoming ready (config issue, missing env var, etc).

Action Needed

@MichielMAnalytics — please check the pod logs on the prod K8s cluster using the commands above and either fix the root cause or share the logs here.

🤖 Generated with Claude Code