Skip to content

fix(locomo): bounded retry on invalid reader completions + protocol-pinned reasoning effort - #178

Merged
fazpu merged 1 commit into
mainfrom
feat/locomo-reader-retry
Jul 29, 2026
Merged

fix(locomo): bounded retry on invalid reader completions + protocol-pinned reasoning effort#178
fazpu merged 1 commit into
mainfrom
feat/locomo-reader-retry

Conversation

@fazpu

@fazpu fazpu commented Jul 29, 2026

Copy link
Copy Markdown
Member

What

Two answer-stage robustness fixes, both protocol-identity-relevant, driven by measured v5-strong smoke evidence (2026-07-29):

  1. Bounded reader retry. The reader step intermittently receives non-JSON completions from openai/gpt-5.6-luna (3/8 questions in one pass, 1/8 in another), permanently zeroing those questions. Now: up to 2 retries (3 attempts total), only for invalid-completion errors (ProviderInvalidResponseError) after at least one tool call. Every attempt counts against the per-question call budget, the global call budget, and the cost ceiling. Provider outages (transport/HTTP errors) are not retried — the exception hierarchy separates them. Per-item reader_attempts and summary total_reader_retries record the mechanism's activity.

  2. Reasoning effort pinned in the protocol. Pinning luna to effort none via ambient REMEMBERSTACK_OPENROUTER_REASONING_EFFORT_MAP raised the smoke 1/8 → 3/8 — but ambient env is invisible to the run pin, violating reproducibility. Now answer_agent_reasoning_effort lives on the protocol dataclass (none for full-v5-strong, None/absent for full-v5) and is applied per-request via a new ModelRequest.reasoning_effort field with explicit-set-wins semantics. Engine seats never set the field, so worker behavior under the env map is byte-identical.

Both fields join the protocol fingerprint (prepare + validate + registry pin check). Fingerprints roll for both entries; all prior runs were smoke-tier diagnostics, nothing published is invalidated. Names and CLI keys unchanged; --protocol remains prepare-only.

Evidence

Smoke v5s-glm52 (pre-pin): 1/8, answer_reader failures 3. Smoke v5s-glm52-b (env-pinned): 3/8, failures 1. This PR makes the winning configuration reproducible from the run pin alone and stops the residual ~12% reader-failure bleed before the full benchmark.

Reviews

  • Grok-4.5 adversarial review: APPROVE WITH NOTES — verified retry boundedness, single-count budget accounting, exception MRO (outages not retried), model_fields_set sentinel semantics, engine call sites unchanged, fingerprint coverage on both paths. Notes are test-hardening suggestions (fingerprint identity baseline assert, explicit outage-non-retry test, cost-ceiling mid-retry test), non-blocking.

  • Claude (final review): verified budget interactions, MRO, fingerprint derivation in prepare and validate, ran the affected suites (93 + 7 = 100 passed).

  • I have read and agree to the RememberStack Contributor License Agreement v1.0.

@fazpu
fazpu enabled auto-merge (squash) July 29, 2026 17:07
@github-actions

Copy link
Copy Markdown

Coverage report

Click to see where and how coverage changed

FileStatementsMissingCoverageCoverage
(new stmts)
Lines missing
  src/rememberstack/adapters
  openrouter.py
  src/rememberstack/model
  __init__.py
  model_provider.py
Project Total  

This report was generated by python-coverage-comment-action

@fazpu
fazpu merged commit 6b6efa2 into main Jul 29, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant