Skip to content

Gateway blocks sandbox-routed envd /metrics requests #151

Description

@feimingxliu

Affected component

Gateway / scheduler

AgentENV version

  • Version or commit: v0.1.2 / db1492b7915a408b37f863c9e3a34b2ccb2fb1b0
  • Local modifications: none
  • The relevant unconditional route guard is also present in the initial v0.1.0 release.

Host environment

Not host-specific. The request is rejected by the gateway HTTP handler before scheduler lookup or node proxying, so no Firecracker, kernel, architecture, ublk, or storage condition is required to reproduce it.

Operation and configuration

curl -i "${GATEWAY_URL}/metrics" \
  -H "x-agentenv-sandbox-id: ${SANDBOX_ID}" \
  -H "x-agentenv-target-port: 49983"

The same problem applies to host-based sandbox routing when the requested path is exactly /metrics.

Steps to reproduce

  1. Start a multi-node AgentENV deployment with gateway, scheduler, and at least one runtime node.
  2. Create a running sandbox and record its sandbox ID.
  3. Verify another envd endpoint such as /health can be reached through the gateway with the sandbox routing headers and target port 49983.
  4. Send the request above to the exact /metrics path.
  5. Observe that the gateway returns 404 without issuing LookupNode or forwarding the request to the runtime node.

Expected behavior

When /metrics carries valid sandbox routing headers, or is addressed through a configured sandbox proxy host, the gateway should treat it as sandbox data-plane traffic:

gateway /metrics
  -> runtime node /proxy/metrics
  -> sandbox <interaction-ip>:49983/metrics

The envd response should be returned to the client. An unrouted request to the gateway HTTP listener at /metrics should continue to return 404. Gateway Prometheus metrics should remain available only from the separate metrics listener, which defaults to :9102.

Actual behavior

The gateway handler checks the exact path before any routing decision:

if r.URL.Path == "/metrics" {
    http.NotFound(w, r)
    return
}

This unconditional guard runs before host-route parsing, sandbox-header parsing, scheduler lookup, and upstream proxying. Consequently, a valid sandbox-routed envd metrics request always receives:

HTTP/1.1 404 Not Found
404 page not found

The later data-plane path rewriting already maps header-based and host-based requests to the runtime node /proxy subtree, and the node proxy strips that prefix before forwarding to the requested sandbox port. The early gateway guard is therefore the blocking point.

Logs and diagnostics

No runtime-node or envd request log is produced because the gateway terminates the request locally. The behavior follows deterministically from services/gateway/internal/server.go::Server.Handler.

Reproduction frequency

Always

Regression information

  • Last known good: none known
  • First known bad: v0.1.0 initial open-source release

Suggested acceptance criteria

  • Header-routed GET /metrics reaches envd on the requested sandbox port.
  • Host-routed GET /metrics reaches envd on the host-derived sandbox port.
  • Unrouted gateway GET /metrics remains unavailable on the public HTTP listener.
  • Missing or malformed sandbox routing information returns the same stable client errors used by other data-plane requests.
  • Sandbox-routed /metrics is included in gateway proxy instrumentation; only a truly local/unrouted /metrics request bypasses instrumentation.
  • Tests cover header routing, host routing, unrouted access, and malformed routing headers.

Additional context

Related to #5, which tracks the broader E2B API compatibility matrix and marks GET /sandboxes/{sandboxID}/metrics and GET /sandboxes/metrics as unsupported. This report is narrower and does not propose implementing those E2B endpoints; it concerns routing the existing raw envd GET /metrics endpoint through the generic sandbox data-plane proxy.

Pre-submission checklist

  • I searched existing open and closed issues and did not find a duplicate.
  • I provided a minimal reproduction.
  • I removed credentials, tokens, private image references, and other sensitive information.
  • This is not a security vulnerability.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions