Skip to content

[Bug]: jaeger-remote-storage status always exits 1: health endpoint returns 204, status command expects 200 #9295

Description

@Atul-Koundal

What happened?

jaeger-remote-storage status always exits non-zero, even when the service is fully healthy and running with default configuration. No flags or config file are needed to reproduce.

The admin server's health endpoint returns 204 No Content when ready, but the status command treats anything other than 200 OK as a failure.

Steps to reproduce

Build and run with no arguments (defaults to in-memory storage):

$ go build -o /tmp/remote-storage ./cmd/remote-storage
$ /tmp/remote-storage
...
{"level":"info","caller":"flags/admin.go:110","msg":"Mounting health check on admin server","route":"/"}
{"level":"info","caller":"flags/admin.go:138","msg":"Admin server started","http.host-port":"[::]:17270"}
{"level":"info","caller":"remote-storage/main.go:40","msg":"No configuration file provided, using default configuration (memory storage on :17271)"}
{"level":"info","caller":"app/server.go:129","msg":"Starting GRPC server","addr":"[::]:17271"}

The service is healthy. In a second shell:

$ curl -i -s http://localhost:17270/
HTTP/1.1 204 No Content
Date: Tue, 11 Aug 2026 17:45:12 GMT
Connection: close

$ /tmp/remote-storage status; echo "exit=$?"
Error: abnormal value of http status code: 204
Usage:
  jaeger-remote-storage status [flags]
Flags:
  -h, --help                           help for status
      --status.http.host-port string   The host:port (e.g. 127.0.0.1:17270 or :17270) for the health check (default ":17270")
abnormal value of http status code: 204
exit=1

Expected behavior

status exits 0 against a healthy service. Per its own help text "Print Jaeger component status information, exit non-zero on any error" a non-zero exit should indicate an actual error.

It should also print the component status rather than an empty line.

Relevant log output

Screenshot

No response

Additional context

Where the mismatch is

Both halves are wired into the same binary:

  • cmd/internal/flags/healthhost.go:60 , writes http.StatusNoContent when ready
  • cmd/internal/status/command.go:41 , if resp.StatusCode != http.StatusOK
  • cmd/remote-storage/main.go:48 , flags.NewService(...), which mounts HealthHost at /
  • cmd/remote-storage/main.go:140 , status.Command(v, ports.RemoteStorageAdminHTTP)

The default for --status.http.host-port is the same admin port the health handler is mounted on, so the two always meet on default settings.

Because RunE returns an error for a non-error condition, cobra prints the full usage block, presenting a healthy service as though the user had invoked the command wrong. status also prints the response body before checking the code, which is empty for a 204, so it emits a blank line where it previously printed status information.

Why CI does not catch this

Each half is tested in isolation against a different expectation:

  • cmd/internal/status/command_test.go:19 stands up an httptest server returning http.StatusOK — the client is never exercised against the real health handler.
  • cmd/internal/flags/healthhost_test.go:29,47,59 and cmd/internal/flags/admin_test.go:56 assert http.StatusNoContent.

Both suites pass. Nothing covers the two together, which is how the contracts drifted apart.

Scope

Only cmd/remote-storage is affected. It is the sole binary that builds a flags.Service and registers status.Command , verified with:

$ grep -rn 'flags.NewService' --include='*.go' cmd/ | grep -v _test
$ grep -rn 'status.Command' --include='*.go' cmd/ | grep -v _test

The v2 cmd/jaeger binary is not affected: it uses the upstream healthcheckv2 extension on :13133 rather than flags.AdminServer.

Regression

Introduced in #7890 (c74afcb3, 2026-01-16), "Converge status reporting to Collector framework". That change deleted internal/healthcheck/handler.go and replaced it with cmd/internal/flags/healthhost.go, which returns 204 on the ready path instead of the previous 200. cmd/internal/status/command.go was not part of that changeset and still checks for 200. Released in v2.15.0 (2026-02-06) and present in all 7 releases since, including current main.

Suggested fix

Accept any 2xx in cmd/internal/status/command.go, since 204 is a valid success response and the health handler deliberately returns it (pinned by three tests):

if resp.StatusCode < 200 || resp.StatusCode >= 300 {
    return fmt.Errorf("abnormal value of http status code: %v", resp.StatusCode)
}

Alongside it, a regression test that runs the status check against flags.HealthHost.Handler() instead of a stub, so the two sides cannot drift again.

Happy to submit a PR for this if it's not already being worked on.

Jaeger backend version

main @ 9012e21 (built from source); bug present since v2.15.0

SDK

No response

Pipeline

No response

Stogage backend

No response

Operating system

No response

Deployment model

No response

Deployment configs

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions