In EOPF Explorer, I enabled the bundled stac-auth-proxy on our production eoAPI deployment (chart 0.13.2, Traefik ingress) and ran into two separate chart bugs. Each one briefly took our API down before I found the issue with Claude helps. Both still reproduce on 0.14.0. With the two workarounds below the proxy works great.
Bug 1: the ingress points at a middleware that no longer exists, and Traefik turns off the whole host
When the proxy is enabled, the chart (correctly) decides the strip-prefix middleware isn't needed anymore: /stac must reach the proxy unstripped (it gets ROOT_PATH=/stac), the browser route never strips, and if raster/vector/multidim are disabled there's nothing left to strip. So templates/networking/traefik-middleware.yaml renders no Middleware at all.
The problem: templates/networking/ingress.yaml still adds the annotation pointing at it, unconditionally:
traefik.ingress.kubernetes.io/router.middlewares: <ns>-<release>-strip-prefix-middleware@kubernetescrd
You can see it without a cluster (helm repo add eoapi https://devseed.com/eoapi-k8s/ if needed):
helm template eoapi eoapi/eoapi --version 0.14.0 \
--set ingress.className=traefik \
--set stac-auth-proxy.enabled=true \
--set raster.enabled=false --set vector.enabled=false --set multidim.enabled=false \
| grep -E 'router.middlewares|^kind: Middleware' -A2
which gives:
traefik.ingress.kubernetes.io/router.middlewares: default-eoapi-strip-prefix-middleware@kubernetescrd,default-eoapi-browser-redirect-middleware@kubernetescrd
spec:
ingressClassName: traefik
--
kind: Middleware
metadata:
name: eoapi-browser-redirect-middleware
The annotation references two middlewares; the chart renders only one. eoapi-strip-prefix-middleware is a dangling reference, and Traefik disables every router that carries it 😬
In EOPF, we use a kustomize patch to replace the annotation's value. In eoapi-k8s, it seems that the root problem is that two templates are used for the deicsion and annotation (traefik-middleware.yaml and ingress.yaml), the fix would be to make this decision in one place. It seems that it is already address in the refactor #546 (eoapi.traefikStripPrefixes centralizes the decision and the annotation is gated on it being non-empty)
Bug 2: the probes don't know about ROOT_PATH, so the pod never becomes Ready
The chart gives the proxy ROOT_PATH=/stac, and the app then serves everything under that prefix, including its health endpoint, which moves to /stac/healthz (its RemoveRootPathMiddleware 404s any unprefixed path). But the subchart's three probes still point at /healthz and I got these types of error:
ERROR: Root path '/stac' not found in path '/healthz'
INFO: "GET /healthz HTTP/1.1" 404 Not Found
Warning Unhealthy Startup probe failed: HTTP probe failed with statuscode: 404
The app is actually perfectly healthy (it boots cleanly, upstream and OIDC checks pass), but the pod never goes Ready and we get 503 errors for /stac.
I am happy to open a PR to default the probe paths from the same value that sets ROOT_PATH ( e.g. {{ .Values.env.ROOT_PATH }}/healthz )
In EOPF Explorer, I enabled the bundled
stac-auth-proxyon our production eoAPI deployment (chart 0.13.2, Traefik ingress) and ran into two separate chart bugs. Each one briefly took our API down before I found the issue with Claude helps. Both still reproduce on 0.14.0. With the two workarounds below the proxy works great.Bug 1: the ingress points at a middleware that no longer exists, and Traefik turns off the whole host
When the proxy is enabled, the chart (correctly) decides the strip-prefix middleware isn't needed anymore:
/stacmust reach the proxy unstripped (it getsROOT_PATH=/stac), the browser route never strips, and if raster/vector/multidim are disabled there's nothing left to strip. Sotemplates/networking/traefik-middleware.yamlrenders no Middleware at all.The problem:
templates/networking/ingress.yamlstill adds the annotation pointing at it, unconditionally:You can see it without a cluster (
helm repo add eoapi https://devseed.com/eoapi-k8s/if needed):which gives:
The annotation references two middlewares; the chart renders only one.
eoapi-strip-prefix-middlewareis a dangling reference, and Traefik disables every router that carries it 😬In EOPF, we use a kustomize patch to replace the annotation's value. In eoapi-k8s, it seems that the root problem is that two templates are used for the deicsion and annotation (traefik-middleware.yaml and ingress.yaml), the fix would be to make this decision in one place. It seems that it is already address in the refactor #546 (
eoapi.traefikStripPrefixescentralizes the decision and the annotation is gated on it being non-empty)Bug 2: the probes don't know about ROOT_PATH, so the pod never becomes Ready
The chart gives the proxy
ROOT_PATH=/stac, and the app then serves everything under that prefix, including its health endpoint, which moves to/stac/healthz(itsRemoveRootPathMiddleware404s any unprefixed path). But the subchart's three probes still point at/healthzand I got these types of error:The app is actually perfectly healthy (it boots cleanly, upstream and OIDC checks pass), but the pod never goes Ready and we get 503 errors for
/stac.I am happy to open a PR to default the probe paths from the same value that sets
ROOT_PATH( e.g.{{ .Values.env.ROOT_PATH }}/healthz)