Skip to content

Log token usage in LiteLLM internal mode - #386

Open
br32710 wants to merge 1 commit into
ossf:mainfrom
br32710:token-counting
Open

br32710 wants to merge 1 commit into
ossf:mainfrom
br32710:token-counting

Conversation

@br32710

@br32710 br32710 commented Sep 10, 2026

Copy link
Copy Markdown
Contributor

Summary

Describe what changed and why.

Added the fields completion_tokens (output tokens) and prompt_tokens (input tokens) in the log files litellm-spend-report.json and meta.json to support cost estimation even when the model costs are unknown (e.g. for local models). The token counts are determined by polling LiteLLM and only work in internal mode.

User Impact

  • User-facing change
  • Internal-only change

If user-facing, describe CLI/API/config/docs impact and migration steps.

No migration steps needed, if token counts are not present in LiteLLM they will be set to 0 by default.

Release Note / Changelog

  • I updated CHANGELOG.md ([Unreleased]) for user-facing changes
  • No changelog entry needed (internal-only refactor/test/chore)

If this PR includes deprecation or breaking behavior, include:

  • replacement path for users
  • planned removal version/release window

Validation

List tests/checks run and their results.

All tests were run and passed successfully.

Checklist

  • I followed Conventional Commits
  • I updated docs for behavior/config/CLI changes
  • I added/updated tests for behavior changes
  • I considered backward compatibility and migration impact

Signed-off-by: Brandon Luo <br32710@ll.mit.edu>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant