Skip to content

[AI-stack] usage/cost rollups + message feedback (product analytics) — track only #266

Description

@rickylabs

Summary

Track (do not implement yet) a usage/cost rollups + message-feedback capability for the NetScript AI stack — the product-analytics counterpart to the tracing usage-capture in E9 (#248).

This came in as a complementary candidate on the epic (eis-chat POC #45, flagged [PRODUCT SIGN-OFF]). Fable's rev.5 scope ruling: reject as a framework slice, defer to product. The user has now made the product call:

No implementation for now — keep an open issue for tracking.

Scope decision (v1)

  • No billing/analytics sink and no fork-seam is built into E9 for v1. E9's usage-capture stays tracing-only (the OTel GenAI usage chunk from the TanStack finish event — real token counts, not chars/4). It does not grow a second capture path or a rollup sink.
  • This issue is the durable tracking home for the rollups/feedback feature so it isn't lost. Revisit as a product initiative, not a framework-scope call.

What the feature would be (for when it's revisited)

  • Usage/cost rollups — per-user / per-session / per-model aggregation of token + cost, built on the same usage events E9 captures for tracing (design the capture once so an app-owned rollup sink could consume it — but that seam is deferred with this issue, not shipped in v1).
  • Message feedback — thumbs/rating + free-text on assistant messages, persisted app-side, joinable to the usage/trace record.
  • Billing-grade accounting stays app-owned; the framework's only possible future contribution is making E9's usage event forkable — explicitly out of v1 per this decision.

References

Part of #238

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions