Skip to content

qq-tei stuck "unhealthy": healthcheck uses curl, which isn't in the embeddings image #25

Description

@OriNachum

Summary

The tei service (qq-tei, embeddings) reports Docker health unhealthy
even though the service is fully functional. Root cause: the healthcheck invokes
curl, which is not installed in the embeddings image, so the probe errors on
every run.

Evidence

  • docker inspect qq-teiState.Health.Status = unhealthy, but the app is up:
    • logs: Model loaded successfullyApplication startup completeUvicorn running on http://0.0.0.0:80
    • /healthHTTP 200 {"status":"ok","model_loaded":true}
  • Healthcheck log: exit=-1 | OCI runtime exec failed: ... exec: "curl": executable file not found in $PATH
  • Image tooling: python present at /usr/local/bin/python; no curl/wget.

Current healthcheck (docker-compose.yml, tei service)

healthcheck:
  test: ["CMD", "curl", "-f", "http://localhost:80/health"]
  interval: 10s
  timeout: 5s
  retries: 5

Impact

The container is perpetually unhealthy, so any depends_on: condition: service_healthy and any external monitoring treat the embeddings
service as down — a persistent false positive (it had been "unhealthy" ~7 days
while serving normally).

Fix

Use the python already in the image instead of curl (no new dependency):

healthcheck:
  test: ["CMD", "python", "-c", "import urllib.request,sys; sys.exit(0 if urllib.request.urlopen('http://localhost:80/health').status==200 else 1)"]
  interval: 10s
  timeout: 5s
  retries: 5

Alternatives: install curl in services/embeddings, or use wget — but the
python probe avoids adding anything to the image.

Status

Applied this fix locally and recreated the container (docker compose up -d tei); it reports (healthy) within ~30s. The compose edit is not yet
committed
— filing this to track committing it.

  • Claude

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions