Skip to content

feat(bridge): record prompt-cache breakdown on llm_inference events (bridge-v0.3.3)#3

Merged
New1Direction merged 2 commits into
mainfrom
feat/bridge-cache-breakdown
Jun 4, 2026
Merged

feat(bridge): record prompt-cache breakdown on llm_inference events (bridge-v0.3.3)#3
New1Direction merged 2 commits into
mainfrom
feat/bridge-cache-breakdown

Conversation

@New1Direction

Copy link
Copy Markdown
Owner

Why

korgex now records a provable prompt-cache breakdown on every llm_inference event (korgex 0.25.0). The local-journal and HTTP transports carry it, but the PyO3 bridge dropped it: korgex's KorgBridgeClient already forwarded cache_read_tokens / cache_creation_tokens / uncached_input_tokens (with a TypeError fallback), but the Rust record_llm_call didn't accept them — so a cache hit was unprovable from a bridge-written journal. This closes that gap.

What

  • record_llm_call takes three new optional params and folds them onto the event args only when caching is active (a read or a write happened), mirroring korgex's src/korg_ledger.py::_llm_call_args byte-for-byte.
  • A cold turn keeps the legacy two-field args, so older readers and the hash-chain over historical events are undisturbed. Field order is irrelevant — args canonicalize with sorted keys at hash time, the same spec all three independent verifiers use.
  • Bump bridge-v0.3.2 → v0.3.3 (Cargo.toml + pyproject + __version__).
  • No korgex-side change needed — the breakdown flows through once a 0.3.3 wheel is installed.

Test plan

  • cargo check -p korg-bridge clean off a fresh main base
  • rebuilt via maturin develop; 19 bridge tests pass, including:
    • new test_record_llm_call_cache_breakdown (warm breakdown lands on args)
    • new test_record_llm_call_cache_creation_only (cache-write-only still records)
    • the existing test_bridge_chain.py suite — so the new field verifies in-chain
  • docs: bridge-v0.3.3 CHANGELOG entry + korg-bridge README note

Notes

This branch was cut clean off main and contains only the bridge crate + its docs — no unrelated workspace WIP.

The PyO3 bridge transport dropped the prompt-cache breakdown — korgex's
KorgBridgeClient already forwarded cache_read_tokens / cache_creation_tokens /
uncached_input_tokens (with a TypeError fallback), but record_llm_call didn't
accept them, so a cache hit was unprovable from a bridge-written journal while
the local-journal and HTTP transports carried it.

record_llm_call now takes the three optional cache params and folds them onto
the event args ONLY when caching is active (a read or a write happened),
mirroring src/korg_ledger.py::_llm_call_args. A cold turn keeps the legacy
two-field args, so older readers and the hash-chain over historical events are
undisturbed; field order is irrelevant since args are canonicalised with sorted
keys at hash time. Bump 0.3.2 -> 0.3.3.

Tests: +warm-breakdown / +creation-only cases; 19 bridge tests pass (incl. the
chain-integrity suite, so the new field verifies in-chain).
bridge-v0.3.3 entry (matching the existing per-bridge-version convention) and a
note in the korg-bridge README that record_llm_call carries the optional
prompt-cache breakdown, so a cache hit is provable from a bridge-written journal
at parity with korgex's local-journal and HTTP transports.
@New1Direction New1Direction force-pushed the feat/bridge-cache-breakdown branch from c4191f3 to 91380d8 Compare June 4, 2026 22:52
@New1Direction New1Direction merged commit 66e7063 into main Jun 4, 2026
2 checks passed
@New1Direction New1Direction deleted the feat/bridge-cache-breakdown branch June 4, 2026 22:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant