Skip to content

export: configurable concurrent admission and bounded backpressure #652

Description

@ragnorc

Problem

The graph/root export cut limit is fixed at one (stream_export_slots). Capturing a second overlapping export cut produces a resource-limit error until the first is released. The server also has a separate process-wide export byte budget; that does not make concurrent cut admission configurable.

The existing transport regression covers stream_export_slots limit 1 / actual 2. The resource-limit class currently maps to HTTP 413, which does not clearly communicate temporary concurrency pressure.

Proposed change

Provide deployment-configurable export admission with explicit per-graph and process limits. Define bounded queueing or prompt rejection, a machine-readable reason/retry contract, and observable active/queued exports. Preserve memory, snapshot pin and reader-lifetime accounting.

Do not turn this into unbounded buffering or allow configurable slots to bypass the aggregate byte budget. Define how disconnect, cancellation and timeout release the cut and any retained state.

Acceptance criteria

  • The default remains safe and a supported configuration can admit more than one export within resource limits.
  • Saturation is bounded and distinguishable from an oversized request/result.
  • Slow consumers, disconnects and failures release resources reliably.
  • Concurrent exports each preserve their captured snapshot; metrics expose slots, bytes, queue wait and rejections.

Code: export cuts, HTTP transport and tests. Related: #565 for general operation lifetime/cancellation.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    featureFeature proposalneeds-triageAwaiting maintainer triage

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions