Skip to content

Implement Health checks #94

Description

@mgonzal1

Is your feature request related to a problem? Please describe.

It is recommended that the API expose dedicated HTTP health endpoints to provide a standard mechanism for determining whether an instance is healthy and ready to receive requests.

Describe the solution you'd like
Expose dedicated HTTP health endpoints:
Suggested endpoints:
GET /healthz – Returns 200 OK when the application process is healthy.

  • Indicates whether the application process is running and able to serve HTTP requests.
  • Should be lightweight and should not depend on external services.
  • Returns HTTP 200 when healthy.

GET /readyz – Returns 200 OK when the application is fully initialized and ready to serve requests, or 503 Service Unavailable otherwise.

  • Indicates whether the application is fully initialized and ready to serve requests.
  • May verify connectivity to required dependencies (e.g., databases, DPM, Kafka, or backend systems)
  • Returns HTTP 200 when the application is ready and HTTP 503 when it is temporarily unable to serve requests.

The primary use case is Kubernetes liveness and readiness probes. These endpoints would allow Kubernetes to automatically determine whether an instance is healthy, ready to receive traffic during a rollout, or needs to be restarted. They would also provide a standard health interface that can be used by other monitoring and automation tools.

Describe alternatives you've considered
One alternative is to execute a GraphQL query that verifies the application can retrieve accelerator data, for example:

  -H "Content-Type: application/json" \
  -d '{"query":"query { acceleratorData(deviceList:[\"G:AMANDA\"]) { refId } }"}' \
  https://ad-api.fnal.gov/acsys \
| jq -e '.data.acceleratorData | length > 0'

However, this approach is not ideal. The container images is intentionally based on a slim image and do not include utilities such as curl or jq. Adding these tools solely to support health checks would unnecessarily increase the image size and expand its attack surface.

Additional context
The primary use case is Kubernetes liveness and readiness probes. These endpoints would allow Kubernetes to automatically determine whether an instance is healthy, ready to receive traffic during a rollout, or needs to be restarted. They would also provide a standard health interface that can be used by other monitoring and automation tools.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions