fix(cache): fold generation params (reasoning.effort/max_tokens) into semantic cache signature - #15156
fix(cache): fold generation params (reasoning.effort/max_tokens) into semantic cache signature#15156Laksopan23 wants to merge 3 commits into
Conversation
…gosouzapw#15149) The signature hashed model/messages/temperature/top_p plus the output contract (tools/tool_choice/response_format - diegosouzapw#12307/diegosouzapw#12734) but not generation params, so two temperature=0 requests differing only in reasoning.effort or max_tokens collided on one cache entry and the second was served the first's response verbatim under x-omniroute-cache: HIT. outputContractOf() now also extracts reasoning/reasoning_effort, max_tokens/max_completion_tokens, top_k, seed, stop, penalties and logit_bias when present, so both the legacy signature and the Layer-1 direct hash fold them - every read/store call site already passes the contract through. Bodies carrying none keep the byte-identical legacy key, so existing plain-chat cache entries stay valid. The combo-live test seeds its cached entry with the contract the production read path now folds in (max_tokens is in its request body); a bare legacy signature would miss the seeded entry and assert against leftover state from a previous test.
CI run 1 — evidencePassing (8): Change Classification, Merge integrity (changelog + generated skills), API Route Typecheck, Vitest (fast-path), No new ESLint warnings, semgrep, semgrep-cloud-platform/scan, (Build skipped advisory). Docs Gates (fast-path) — inherited base-red, not this PR: Verified inheritance:
Tracked by the base-red issue for this branch: #15100. (Additional local base-red reproduced identically on pristine base code, for context: |
CI run 1 — full failure attribution (all inherited base-red)Final check results: 8 pass / 4 fail (plus Failing checks1. Docs Gates (fast-path) — version drift on
2. Fast Quality Gates — All 24 are pre-existing test files ( 3. Unit Tests fast-path (1/4, 3/4, 4/4) — 9 failing tests, none cache-related:
Local base-vs-branch proof (same 9 failures both ways)Ran the failing files locally in two passes:
( Passing (8)Change Classification · Merge integrity (changelog + generated skills) · API Route Typecheck · Vitest (fast-path) · No new ESLint warnings · semgrep · semgrep-cloud-platform/scan · Mergify Merge Protections (skip) No failing check involves |
|
Retriggering CI: base has moved to \dbe703a000\ since the last run — the inherited reds (Docs Gates via #15113, unit fast-path failures incl. glm pin / geo rotation / 400-recovery / connection-test via the base-red drain rounds) reproduce green on the current base under the canonical harness. FQG mutation-gate drift stays tracked as base-red (#15185 / #15227). |
…(refresh merge-base so the new-code gates measure only this PR's changes)
…(sync to base tip 5282351 so PR checks scope only this PR's files)
|
CI evidence after merging base tip After Remaining reds are all base-side / infra — none touch this PR's diff:
|
What
Fixes #15149 — folds generation params into the semantic-cache signature so two
temperature: 0requests that differ only inreasoning.effort/max_tokens(ortop_k,seed,stop, penalties,logit_bias) no longer collide on the same cache entry.Root cause
generateSignature()hashed{model, messages, temperature, top_p}+ the output contract (tools/tool_choice/response_format — #12307/#12734), andgenerateDirectHash()folded the same contract, but neither carried generation params:outputContractOf()never extracted them. A request withreasoning: {"effort": "none"}was therefore stored and looked up under the exact signature a laterreasoning: {"effort": "max"}request computes — the second request gotx-omniroute-cache: HITwith the first request's body.Fix
outputContractOf()(src/lib/semanticCache.ts) now also extracts, when present:reasoning,reasoning_effort,max_tokens,max_completion_tokens,top_k,seed,stop,presence_penalty,frequency_penalty,logit_biasEvery read/store call site already passes that contract through — legacy signature (read fallback, non-streaming store, streaming store) and the Layer-1 direct hash (manager lookup/store) — so no call-site changes were needed. Requests carrying none of these fields still get
nullfromoutputContractOf(), so plain-chat signatures — and every cache entry already written for them — stay byte-identical.Also adjusts
tests/unit/chat-combo-live-test.test.ts: it hand-seeded its cached entry with a bare legacy signature while the production read path folds the request'smax_tokens; the seed now uses the same contract the production paths compute (the test previously only passed becauseoutputContractOfreturnednullfor generation params).Tests (TDD)
tests/unit/15149-semantic-cache-signature-generation-params.test.ts— written first, red on base (all collision assertions fail: identical digests), 7/7 green with the fix:nonevsmax→ different signatures; effort vs absent → differentmax_tokens1500 vs 800 vs absent → different signatureschat-combo-live-test5/5, pluscache-signature-roundtrip,chatcore-*,chat-route-edge-cases,cache-config-route,db-logs-cache— all green.npm run test:vitest: 51/52 files pass;mcp-server/__tests__/audit.test.tsfailed only under parallel run contention and passes 8/8 in isolation.tsc -p tsconfig.typecheck-core.json: clean; Prettier applied; pre-commit hooks (lint-staged, docs-sync, t11 any-budget, tracked-artifacts, ai-attribution) all passed.Base-red failures (inherited, not this PR)
Local runs on pristine
origin/release/v3.8.52code reproduce exactly the same failures as this branch:tests/unit/provider-request-failure-pipeline.test.ts— 6 failures (pipelinePayloadsnull) — identical count on base codetests/unit/chatcore-translation-paths.test.ts— 3 failures (same family) — identical count on base codeTracked by #15100.