Skip to content

Week 5 Day 1: add provider telemetry contract#22

Merged
utkarshp845 merged 1 commit into
mainfrom
codex/week-5-provider-telemetry
Jul 21, 2026
Merged

Week 5 Day 1: add provider telemetry contract#22
utkarshp845 merged 1 commit into
mainfrom
codex/week-5-provider-telemetry

Conversation

@utkarshp845

Copy link
Copy Markdown
Owner

Summary

  • add a structured provider telemetry contract for optional LLM calls
  • expose bounded provider/model identity, attempt state, outcome, latency, fallback, and normalized token usage
  • keep prompts, evidence, credentials, provider URLs, generated output, and customer identifiers out of telemetry
  • add success, failure, unconfigured fallback, API integration, latency, usage, and privacy tests
  • document the Day 1 contract and restore the Weeks 5-8 technical roadmap alongside commercialization gates

Validation

  • assistant tests: 25 passed
  • demo-service tests: 6 passed
  • Ruff: passed
  • assistant evaluations: 7/7 cases and 35/35 checks passed
  • git diff --check: passed

Scope boundary

This PR establishes per-request telemetry. Aggregate metrics, pricing inputs, cost estimation, cost per successful evaluated analysis, and provider comparison reporting remain later Week 5 work.

@utkarshp845
utkarshp845 merged commit e803209 into main Jul 21, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant