feat: implement per-request AI usage event export - #2426
Draft
shiv4289 wants to merge 6 commits into
Draft
Conversation
shiv4289
changed the title
TDD'd HTTP server backed by an embedded NATS JetStream KV store, serving as the reference UsageEventSink receiver described in the per-request AI usage event export proposal. Runs as a sidecar with no NATS network port exposed; durably persists and deduplicates events on event_id, verified via unit tests and a curl-based smoke test including restart durability. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GPQqdXqBVsbkwrW46cCx2Q Signed-off-by: Shivji Kumar Jha <shiv4289@gmail.com>
Adds the UsageEvent schema, the UsageEventSink interface, and an HTTP sink implementation (internal/usageevent), then wires a synchronous publisher into the ExtProc response path so one normalized UsageEvent is emitted per completed request. Publication failures are dropped and surfaced via constructed/exported/dropped/latency metrics; the gateway performs no retries or local persistence, per the proposal's stateless design. Adds --usageEventsHTTPURL, --usageEventsTimeout, and --usageEventsAttributes flags to the extproc binary; the feature stays disabled until a URL is configured. Provider name derivation is now exposed as metrics.ProviderName so both the metrics and usage-event paths share the same backend-schema mapping. Verified end-to-end against the usage-events-receiver sidecar from the previous commit: internal/usageevent/receiver_integration_test.go builds and runs the real receiver binary and publishes through the same HTTPSink/Publisher used in production, asserting durable storage and event_id deduplication via the receiver's own HTTP API. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GPQqdXqBVsbkwrW46cCx2Q Signed-off-by: Shivji Kumar Jha <shiv4289@gmail.com>
…egration test Removes examples/usage-events-receiver (moving to a standalone repo, shiv4289/http-proxy-nats-adapter, so the upstream PR carries no embedded NATS dependency). Replaces the subprocess/NATS-backed integration test with a zero-external-dependency test in internal/usageevent that mocks the acknowledged HTTP sink via httptest.NewServer, exercising the real production HTTPSink/Publisher end-to-end: payload serialization, the constructed/exported/dropped metrics pathway, and drop-on-failure and drop-on-timeout rules. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GPQqdXqBVsbkwrW46cCx2Q Signed-off-by: Shivji Kumar Jha <shiv4289@gmail.com>
Use internal/json instead of encoding/json (depguard) in the usageevent package, and pass UsageEvent/usageEventParams by pointer instead of by value in Sink.Publish, Publisher.Publish, and buildUsageEvent to avoid copying the ~192-byte struct on every call (gocritic hugeParam). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GPQqdXqBVsbkwrW46cCx2Q Signed-off-by: Shivji Kumar Jha <shiv4289@gmail.com>
The extproc binary already accepts --usageEventsHTTPURL, --usageEventsTimeout and --usageEventsAttributes, but mustStartExtProc built its argument list by hand and forwarded only the OTEL_AIGW_* variables. That left per-request usage event export unreachable from `aigw run`: the flags existed but nothing could set them outside a hand-rolled extproc invocation. Forwards AIGW_USAGE_EVENTS_HTTP_URL, AIGW_USAGE_EVENTS_TIMEOUT and AIGW_USAGE_EVENTS_ATTRIBUTES to the corresponding flags, following the existing env-var pattern. The feature stays disabled unless a URL is set, so an unset environment produces the same argument list as before — covered by the "disabled by default" subtest. This is transport-agnostic: it configures the acknowledged HTTP sink the proposal already defines, and knows nothing about any particular receiver. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017yutC9xh9TrKxsoKhJnurR Signed-off-by: Shivji Kumar Jha <shiv4289@gmail.com>
Signed-off-by: Shivji Kumar Jha <shiv4289@gmail.com> # Conflicts: # cmd/extproc/mainlib/main.go
shiv4289
force-pushed
the
usage-event-export
branch
from
July 26, 2026 20:41
c50b5f1 to
b6bec85
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This adds per-request usage event export as described in proposal 012. Rightnow the only way to know the gateway handled a request is best-effort metrics and access logs — there's no way to tell whether a specificrequest's usage data actually made it anywhere. This adds a UsageEvent that's built for every completed request (success or failure) and published
synchronously to a configurable HTTP sink, so operators get an acknowledged record instead of a "probably logged it somewhere" one.
The gateway stays stateless: no retries, no local buffering. If the sink doesn't ack in time the event is dropped and counted as such in metrics so it can be used for reconciliation or alerting.
What's here:
Tests cover the sink/publisher pipeline against a plain httptest.NewServer mock, no external dependencies, including the drop-on-failure and drop-on-timeout paths.
Related Issues/PRs (if applicable)
Special notes for reviewers (if applicable)
Given this is my first work on this repo, looking for help on gaps. I have considered a lot of design choices and favoured a simple and reliable (trackable) delivery. Happy to discuss the alternate suggestions.